Byte-for-byte YAML editing, explained
Byte-for-byte, explained
Most YAML tools work by parsing your file into data, doing something to the data, and printing the data back out. That last step is where formatting goes to die: the printer has opinions about quoting and indentation, and your file comes back wearing them.
yqr does something different. Nodes you did not touch are emitted as the original bytes from your file, sliced straight out of the source. There is no printer involved, so there are no opinions to apply.
The identity test
The clearest way to see it is to ask for everything and change nothing:
$ yqr '.' config.yaml | diff config.yaml -
$ echo $?
0
No output, exit 0. The file went through yqr and came back identical, down to the byte. That holds for the awkward cases too – CRLF line endings, a byte-order mark, trailing whitespace, tabs inside strings, a flow collection wrapped over several lines, multiple documents in one file.
It is a good thing to try on your own gnarliest config file. If it comes back clean, everything below is safe.
What survives
Take this file, which has a few things that normally do not survive a round trip:
defaults: &defaults
mode: 0640 # octal, on purpose
retries: 3
# Services below inherit the defaults.
web:
<<: *defaults
name: 'web'
ver: 1.10
Read it back with yqr and you get exactly that, including:
- Comments, and the whitespace that aligns them.
- Blank lines, which are doing real organisational work here.
- Anchors, aliases, and merge keys, in the spelling you wrote them.
- Quote style –
'web'stays single-quoted rather than becoming"web"or bareweb. - Scalar spelling –
0640stays0640. - Line breaks inside a flow collection – a
ports:orargs:list wrapped for width keeps its wrapping, closing bracket included.
That last one is worth dwelling on.
Why 0640 is not 640
Kubernetes spells file permissions in octal. If a tool re-types that scalar
as a number and prints it back, you get 640, which is a different
permission. Same story for a version pinned at 1.10, which becomes 1.1
the moment something treats it as a float.
yqr never re-types the value, so the question never arises:
$ yqr -r '.defaults.mode' config.yaml
0640
$ yqr -r '.ver' config.yaml
1.10
When you want the opposite
Sometimes you genuinely want canonical output – comparing two files that
are semantically equal but formatted differently, or feeding something
downstream that wants predictable shapes. That is what --normalize (-N)
is for.
$ yqr --normalize '.' config.yaml
defaults:
mode: 640
retries: 3
web:
name: web
mode: 640
retries: 3
ver: 1.1
Look at what that did, because it is instructive. Comments are gone. The
blank line is gone. The anchor and merge key have been resolved, so web
now carries its own copies of mode and retries. And 0640 has become
640.
Every one of those is correct as data. None of them is what you want
landing in a pull request. That is the trade, and it is why the byte-exact
path is the default and --normalize is the flag you have to ask for.
$ yqr -rN '.defaults.mode' config.yaml
640
$ yqr -rN '.ver' config.yaml
1.1
Editing works the same way
The guarantee extends to edits. Change one value and everything else is untouched:
$ yqr '.spec.replicas = 5' deploy.yaml
Only the bytes of that one scalar are replaced. The comment two spaces to its right, the indentation, the key order, the quoting elsewhere in the file – all still the original bytes.
A key someone left blank is an ordinary target. A key with nothing after it
is an implicit null: it reads as null, and writing to it fills the value in
on the line that is already there. Given this image.yaml:
image: registry.example.com/web:1.4.2
replicas: 2
digest: # filled by the release job
$ yqr '.digest = "sha256:9f0a"' image.yaml
image: registry.example.com/web:1.4.2
replicas: 2
digest: sha256:9f0a # filled by the release job
The value goes before the comment, and the gutter the author wrote is
still there. The same holds for an empty - item in a sequence.
A new key stops where its neighbour ends
Adding a key means finding where the entry above it ends, and that is not
always the line you would guess. Take this deploy.yaml:
spec:
replicas: 2
template:
image: web:1.4.2
# set by the release pipeline, do not edit
revision: 42
The last entry under spec is a nested block, and the comment below it
documents revision, not template. Add a key to spec and the comment
stays where the author put it:
$ yqr '.spec.strategy = "Recreate"' deploy.yaml
spec:
replicas: 2
template:
image: web:1.4.2
strategy: Recreate
# set by the release pipeline, do not edit
revision: 42
head_comment(.revision) reads the same before and after, which is the
check worth making: a comment that changes hands is not something a diff
of the values can show you.
The same rule covers a block scalar written |+ or >+, which keeps the
blank lines at its end as part of its value. A new key goes below them, so
the scalar reads back byte for byte after the write.
Line endings follow the file, not the platform. Write a multi-line value
into a file whose lines end \r\n and every line yqr emits ends \r\n
too:
$ yqr '.logging.level = "warn\nverbose"' config.yaml | file -
/dev/stdin: ASCII text, with CRLF line terminators
If an edit cannot be made without restructuring the document, yqr refuses
it and exits 5 rather than emitting something surprising. With -i the
file is left untouched on refusal, so a failed edit never leaves you with a
half-written file.
A key that reaches a mapping through a << merge gives you two
different edits, and the path you write picks between them. .web.mode
names web’s key, so the edit creates an explicit entry that overrides
the merge for web alone – the definition and every other inheritor
keep theirs:
$ yqr '.web.mode = 416' config.yaml
defaults: &defaults
mode: 0640 # octal, on purpose
retries: 3
# Services below inherit the defaults.
web:
<<: *defaults
name: 'web'
mode: 416
ver: 1.10
.defaults.mode = 416 is the other edit: it changes the value where it
is defined, for everything that inherits the anchor. del(.defaults.mode)
follows the same rule – the entry is removed at the definition, and
everything that inherits the anchor loses it.
One shape stays a refusal: when the mapping’s value is an alias
(web: *defaults), there is nowhere to put an explicit entry without
rewriting the alias into a block, so yqr declines and points at the
definition:
$ yqr '.web.mode = 416' aliased.yaml
yqr: runtime error: cannot assign at "web.mode": the mapping is reached through a `<<` merge or an alias, so it has no "mode" entry of its own to write. Assign where the key is defined instead
The reference itself, though, is the entry’s own byte sequence, so it
edits like any other value: assigning replaces the *defaults token,
del removes the entry, and the anchor’s definition is never touched.
Writing the value the alias already resolves to is a no-op that keeps
your *defaults spelling – a save that changes nothing re-spells
nothing:
$ yqr '.web = "standalone"' aliased.yaml
defaults: &defaults
mode: 0640
web: standalone
$ yqr 'del(.web)' aliased.yaml
defaults: &defaults
mode: 0640
A write that changes nothing changes nothing
Assigning a value that is already there does not rewrite it. yqr writes
values, and a value does not carry its own spelling – so re-emitting
0640 would print 640, exactly the way --normalize does. Instead the
write is skipped:
$ yqr '.defaults.mode = .defaults.mode' config.yaml | sed -n 2p
mode: 0640 # octal, on purpose
The same holds for comments. #tight and # tight say the same thing, so
writing a comment’s own text back leaves the line alone – which means you
can read a comment and feed it straight back:
$ yqr 'line_comment(.defaults.mode) = "octal, on purpose"' config.yaml | sed -n 2p
mode: 0640 # octal, on purpose
Anything yqr can tell apart still writes normally. This is a guard against rewriting bytes you did not ask to change, not a limit on what you can edit.
Next
- Editing Kubernetes manifests – this applied to a real workflow.
- Validating YAML – confirming a file is still correct afterwards.