Byte-for-byte YAML editing, explained

8 min read

Byte-for-byte, explained

Most YAML tools work by parsing your file into data, doing something to the data, and printing the data back out. That last step is where formatting goes to die: the printer has opinions about quoting and indentation, and your file comes back wearing them.

yqr does something different. Nodes you did not touch are emitted as the original bytes from your file, sliced straight out of the source. There is no printer involved, so there are no opinions to apply.

The identity test

The clearest way to see it is to ask for everything and change nothing:

$ yqr '.' config.yaml | diff config.yaml -
$ echo $?
0

No output, exit 0. The file went through yqr and came back identical, down to the byte. That holds for the awkward cases too – CRLF line endings, a byte-order mark, trailing whitespace, tabs inside strings, a flow collection wrapped over several lines, multiple documents in one file.

It is a good thing to try on your own gnarliest config file. If it comes back clean, everything below is safe.

What survives

Take this file, which has a few things that normally do not survive a round trip:

defaults: &defaults
  mode: 0640      # octal, on purpose
  retries: 3

# Services below inherit the defaults.
web:
  <<: *defaults
  name: 'web'
ver: 1.10

Read it back with yqr and you get exactly that, including:

  • Comments, and the whitespace that aligns them.
  • Blank lines, which are doing real organisational work here.
  • Anchors, aliases, and merge keys, in the spelling you wrote them.
  • Quote style – 'web' stays single-quoted rather than becoming "web" or bare web.
  • Scalar spelling – 0640 stays 0640.
  • Line breaks inside a flow collection – a ports: or args: list wrapped for width keeps its wrapping, closing bracket included.

That last one is worth dwelling on.

Why 0640 is not 640

Kubernetes spells file permissions in octal. If a tool re-types that scalar as a number and prints it back, you get 640, which is a different permission. Same story for a version pinned at 1.10, which becomes 1.1 the moment something treats it as a float.

yqr never re-types the value, so the question never arises:

$ yqr -r '.defaults.mode' config.yaml
0640
$ yqr -r '.ver' config.yaml
1.10

When you want the opposite

Sometimes you genuinely want canonical output – comparing two files that are semantically equal but formatted differently, or feeding something downstream that wants predictable shapes. That is what --normalize (-N) is for.

$ yqr --normalize '.' config.yaml
defaults:
  mode: 640
  retries: 3
web:
  name: web
  mode: 640
  retries: 3
ver: 1.1

Look at what that did, because it is instructive. Comments are gone. The blank line is gone. The anchor and merge key have been resolved, so web now carries its own copies of mode and retries. And 0640 has become 640.

Every one of those is correct as data. None of them is what you want landing in a pull request. That is the trade, and it is why the byte-exact path is the default and --normalize is the flag you have to ask for.

$ yqr -rN '.defaults.mode' config.yaml
640
$ yqr -rN '.ver' config.yaml
1.1

Editing works the same way

The guarantee extends to edits. Change one value and everything else is untouched:

$ yqr '.spec.replicas = 5' deploy.yaml

Only the bytes of that one scalar are replaced. The comment two spaces to its right, the indentation, the key order, the quoting elsewhere in the file – all still the original bytes.

A key someone left blank is an ordinary target. A key with nothing after it is an implicit null: it reads as null, and writing to it fills the value in on the line that is already there. Given this image.yaml:

image: registry.example.com/web:1.4.2
replicas: 2
digest:          # filled by the release job
$ yqr '.digest = "sha256:9f0a"' image.yaml
image: registry.example.com/web:1.4.2
replicas: 2
digest: sha256:9f0a          # filled by the release job

The value goes before the comment, and the gutter the author wrote is still there. The same holds for an empty - item in a sequence.

A new key stops where its neighbour ends

Adding a key means finding where the entry above it ends, and that is not always the line you would guess. Take this deploy.yaml:

spec:
  replicas: 2
  template:
    image: web:1.4.2
# set by the release pipeline, do not edit
revision: 42

The last entry under spec is a nested block, and the comment below it documents revision, not template. Add a key to spec and the comment stays where the author put it:

$ yqr '.spec.strategy = "Recreate"' deploy.yaml
spec:
  replicas: 2
  template:
    image: web:1.4.2
  strategy: Recreate
# set by the release pipeline, do not edit
revision: 42

head_comment(.revision) reads the same before and after, which is the check worth making: a comment that changes hands is not something a diff of the values can show you.

The same rule covers a block scalar written |+ or >+, which keeps the blank lines at its end as part of its value. A new key goes below them, so the scalar reads back byte for byte after the write.

Line endings follow the file, not the platform. Write a multi-line value into a file whose lines end \r\n and every line yqr emits ends \r\n too:

$ yqr '.logging.level = "warn\nverbose"' config.yaml | file -
/dev/stdin: ASCII text, with CRLF line terminators

If an edit cannot be made without restructuring the document, yqr refuses it and exits 5 rather than emitting something surprising. With -i the file is left untouched on refusal, so a failed edit never leaves you with a half-written file.

A key that reaches a mapping through a << merge gives you two different edits, and the path you write picks between them. .web.mode names web’s key, so the edit creates an explicit entry that overrides the merge for web alone – the definition and every other inheritor keep theirs:

$ yqr '.web.mode = 416' config.yaml
defaults: &defaults
  mode: 0640      # octal, on purpose
  retries: 3

# Services below inherit the defaults.
web:
  <<: *defaults
  name: 'web'
  mode: 416
ver: 1.10

.defaults.mode = 416 is the other edit: it changes the value where it is defined, for everything that inherits the anchor. del(.defaults.mode) follows the same rule – the entry is removed at the definition, and everything that inherits the anchor loses it.

One shape stays a refusal: when the mapping’s value is an alias (web: *defaults), there is nowhere to put an explicit entry without rewriting the alias into a block, so yqr declines and points at the definition:

$ yqr '.web.mode = 416' aliased.yaml
yqr: runtime error: cannot assign at "web.mode": the mapping is reached through a `<<` merge or an alias, so it has no "mode" entry of its own to write. Assign where the key is defined instead

The reference itself, though, is the entry’s own byte sequence, so it edits like any other value: assigning replaces the *defaults token, del removes the entry, and the anchor’s definition is never touched. Writing the value the alias already resolves to is a no-op that keeps your *defaults spelling – a save that changes nothing re-spells nothing:

$ yqr '.web = "standalone"' aliased.yaml
defaults: &defaults
  mode: 0640
web: standalone
$ yqr 'del(.web)' aliased.yaml
defaults: &defaults
  mode: 0640

A write that changes nothing changes nothing

Assigning a value that is already there does not rewrite it. yqr writes values, and a value does not carry its own spelling – so re-emitting 0640 would print 640, exactly the way --normalize does. Instead the write is skipped:

$ yqr '.defaults.mode = .defaults.mode' config.yaml | sed -n 2p
  mode: 0640      # octal, on purpose

The same holds for comments. #tight and # tight say the same thing, so writing a comment’s own text back leaves the line alone – which means you can read a comment and feed it straight back:

$ yqr 'line_comment(.defaults.mode) = "octal, on purpose"' config.yaml | sed -n 2p
  mode: 0640      # octal, on purpose

Anything yqr can tell apart still writes normally. This is a guard against rewriting bytes you did not ask to change, not a limit on what you can edit.

Next