Editing Kubernetes manifests without reformatting them

13 min read

Editing Kubernetes manifests

Manifests are checked in, reviewed, and argued over. So the useful property in a tool that edits them is not how much it can do – it is how little it changes.

Here is a manifest with the things real ones have: an ownership comment, a comment explaining a number, and a quoted image reference.

# Web tier. Owned by the platform team.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  labels:
    app: web
    app.kubernetes.io/name: web
    app.kubernetes.io/component: frontend
spec:
  replicas: 3          # bumped for the Black Friday load test
  template:
    spec:
      containers:
        - name: web
          image: "registry.example.com/web:1.4.2"
          ports:
            - containerPort: 8080
        - name: log-shipper
          image: "registry.example.com/shipper:2.0"

Reading a field

Paths look like jq, because that is the idea:

$ yqr '.spec.template.spec.containers[0].image' deploy.yaml
"registry.example.com/web:1.4.2"

The quotes are there because that is how the value is written in the file. When you want the bare string – to pass to docker pull, say – use -r:

$ yqr -r '.spec.template.spec.containers[0].image' deploy.yaml
registry.example.com/web:1.4.2

Bumping an image tag

$ yqr '.spec.template.spec.containers[0].image = "registry.example.com/web:1.5.0"' deploy.yaml

That prints the whole file to stdout with one value changed. Add -i to write it back in place instead:

$ yqr -i '.spec.template.spec.containers[0].image = "registry.example.com/web:1.5.0"' deploy.yaml

The write is atomic – a temporary file and a rename – so an interrupted run cannot leave you with half a manifest.

The part that matters

$ yqr -i '.spec.replicas = 5' deploy.yaml
$ git diff deploy.yaml

-  replicas: 3          # bumped for the Black Friday load test
+  replicas: 5          # bumped for the Black Friday load test

One line. The comment is still aligned where it was, the quoting elsewhere is untouched, the blank lines are where you left them. A reviewer reads one line and moves on.

Run that across forty manifests in a release script and you get forty one-line diffs, which is a review someone can actually do.

Other edits you can make today

$ yqr -i '.spec.ports += 9090' service.yaml          # append to a sequence
$ yqr -i '.metadata.labels.tier = "frontend"' deploy.yaml   # add a key
$ yqr -i 'del(.spec.template.spec.containers[1])' deploy.yaml    # remove an entry
$ yqr -i 'key(.metadata.labels.app) = "application"' deploy.yaml  # rename a key
$ yqr -i 'swap(.spec.template.spec.containers; 0; 1)' deploy.yaml # reorder a list

del handles nested blocks and multi-line values, not just single lines, and closes the gap cleanly afterwards. It also handles the two cases that used to be refused:

  • The last entry of a block. The collection is written out explicitly, because deleting the bytes would leave a dangling spec: – and a key with nothing under it reads back as null, which is a type change rather than a removal:

    $ yqr 'del(.spec.replicas)' one.yaml
spec:
  {}

    A comment documenting the removed entry goes with it, rather than being left behind describing an empty collection.

  • An item of a flow collection like ports: [80, 443]. Exactly one separator goes with the item, so you never get [, 443] or [80, ].

Keys with dots in them

Kubernetes labels and annotations are dotted: app.kubernetes.io/name. A bare path reads that as four steps, so quote the key. Both of jq’s spellings work, at the head of a path and after any later dot:

$ yqr -r '.metadata.labels."app.kubernetes.io/name"' deploy.yaml
web
$ yqr -r '.metadata.labels["app.kubernetes.io/name"]' deploy.yaml
web

Every edit works on a quoted key the way it works on a bare one:

$ yqr -i '.metadata.labels."app.kubernetes.io/name" = "api"' deploy.yaml
$ yqr -i '.metadata.labels."app.kubernetes.io/version" = "1.4.2"' deploy.yaml  # add one
$ yqr -i 'del(.metadata.labels."app.kubernetes.io/component")' deploy.yaml
$ yqr -i 'key(.metadata.labels.app) = "app.kubernetes.io/part-of"' deploy.yaml
$ yqr -i 'line_comment(.metadata.labels."app.kubernetes.io/name") = "selector"' deploy.yaml

The quotes belong to the filter, not the file. A new key is written the way its neighbours are, so app.kubernetes.io/version: 1.4.2 lands unquoted beside unquoted labels, and a read gives you the bytes as the file has them.

Copying a whole block

The right-hand side of = can name a mapping or a sequence, not just a scalar. That is how you copy one block over another:

$ yqr -i '.spec.template.spec.containers[0].resources.requests = .spec.template.spec.containers[0].resources.limits' deploy.yaml

-              cpu: 250m
-              memory: 256Mi
+              cpu: "1"
+              memory: 512Mi

The same right-hand side creates a key that is not there yet, which is the usual way to seed a block:

$ yqr -i '.metadata.annotations = .metadata.labels' deploy.yaml

     app: web
+  annotations:
+    app: web

+= takes one too, appending a whole mapping as a sequence item – a container, a step, an env entry.

What is copied is the value, not the bytes. The block is written at its new home’s indentation and quoting, and comments inside the block you copied from do not come with it. Everything outside the edit is untouched, as always.

One shape is refused rather than guessed at, because the entry would have to change shape in place:

$ yqr '.spec.replicas = .metadata.labels' deploy.yaml
yqr: runtime error: cannot assign at "spec.replicas": the value there is a
number, and yqr writes a collection only where one already is. Remove the
entry and write it again, as in `del(.spec.replicas)` then
`.spec.replicas = <path>`, which places it at the end of its mapping

The message names the remedy: a delete followed by an assignment, which moves the key to the end of its mapping.

The reverse works in place. A scalar written over a block replaces the whole block and goes on the key’s own line, where you would have typed it:

$ yqr '.metadata.labels = null' deploy.yaml

Here labels: and the two lines under it become labels: null, and nothing else moves. The block’s lines go with the old value, including a comment above one of its entries. A comment on the key’s own line stays. A block that carries an anchor or a tag (labels: &common) is refused, because the scalar would drop it.

Computing a new value from the old one

= writes what you tell it. |= runs a filter on the value that is already there and writes the result, so you can say “one more than this” without knowing what “this” is:

$ yqr -i '.spec.replicas |= (. + 1)' deploy.yaml

Inside the filter, . is the value at the path – not the document. Arithmetic is + - * / %, with the usual precedence and parentheses.

Numbers keep their type. replicas: 3 doubled is 6, never 6.0, and a division only becomes a fraction when it genuinely is one:

$ yqr '.n |= (. / 2)' <<< 'n: 4'      # n: 2
$ yqr '.n |= (. / 2)' <<< 'n: 3'      # n: 1.5

That is the same rule that keeps 0640 from becoming 640 on a read. An integer result too large for 64 bits is an error rather than a silent widening to a float, because widening is exactly the precision loss the rule prevents.

|= writes wherever = writes – a scalar, in place. A filter returning a list or a mapping is refused, the same way assigning one is.

Editing a comment

Two selectors, wrapping a path the same way key(...) does:

$ yqr 'line_comment(.spec.replicas) = "tuned for peak"' deploy.yaml
$ yqr 'head_comment(.spec.replicas) = "why this exists"' deploy.yaml
$ yqr -i 'del(line_comment(.spec.replicas))' deploy.yaml

line_comment is the # ... after the value on the entry’s own line; head_comment is the block of comment lines directly above it. Reading either gives the body without the #:

$ yqr -r 'line_comment(.spec.replicas)' deploy.yaml
tuned for peak

What you write is what you read back, including leading spaces, so a comment survives being set and read again unchanged. The reverse is not a byte-level identity: a comment authored #note, with no space, reads as note, and writing that back renders # note.

An empty body writes a bare # rather than removing anything. Removal has its own spelling, del(...), so both are reachable.

Three cases are refused rather than guessed at, each because the obvious thing to do would be wrong:

  • An entry whose value starts on the next line has no line of its own, so there is nowhere to put an inline comment. Writing one would land it on the first child instead, where it would look like it documents that.
  • A comment block separated from the entry by a blank line documents whatever came before it, not the entry below. yqr will not rewrite or delete it.
  • A comment block above a sequence item can be read but not edited – the YAML engine attaches leading comments to mapping keys only.

A head_comment on an entry whose value is a block, like spec: with the manifest under it, is the key’s: it reads, writes and deletes like any other. A comment above the block’s first child stays with that child.

$ yqr -i 'head_comment(.spec) = "scaled by the HPA, do not edit replicas"' deploy.yaml

And foot_comment(...) is refused with an explanation rather than a syntax error: a comment below an entry belongs to whatever follows it about as often as to the entry itself, so there is no unambiguous block to address.

Renaming a key

A path names a value, so there is no path that means “the key of this entry”. key(...) wraps one and names the key instead:

$ yqr 'key(.metadata.name)' deploy.yaml
name
$ yqr -i 'key(.metadata.name) = "title"' deploy.yaml

The rename rewrites the key token and nothing else. The value keeps its spelling, the entry keeps its position in the mapping – a rename is not a delete followed by an insert – and the comments stay where they were:

$ cat deploy.yaml
metadata:
  # names the app
  name: web  # required
$ yqr -i 'key(.metadata.name) = "title"' deploy.yaml
$ cat deploy.yaml
metadata:
  # names the app
  title: web  # required

Reading a key gives you the token as the file spells it, quotes included, because the read slices the document rather than echoing back the path you typed. -r unquotes it, the same way it unquotes a string value:

$ yqr 'key(.["retry count"])' config.yaml
"retry count"
$ yqr -r 'key(.["retry count"])' config.yaml
retry count

key(...) reads are total: a sequence item has no key, and neither does a key that arrived through a << merge, so both read null rather than failing a batch. Writing to those is refused with the reason, as is a rename that would collide with an existing sibling, or one to a name the path syntax could not address afterwards.

A key containing . – the app.kubernetes.io/name style – needs quotes in the filter, and then key(...) reads and renames it like any other. See Keys with dots in them.

Reordering a list

An ordering is the one thing here with no node to name – there is no path that means “third”. So it is a verb with arguments rather than a selector wrapping a path, and the arguments are separated by ;:

$ yqr -i 'swap(.jobs.build.steps; 0; 2)' ci.yaml   # exchange two items
$ yqr -i 'move(.jobs.build.steps; 0; -1)' ci.yaml  # move one, shifting the rest

swap exchanges two items and leaves everything between them alone. move takes one item out and puts it back at the destination, shifting the items in between by one. Indices count from zero, and a negative index counts from the end – -1 is the last item, the same as .[-1] in a path.

The reason this is worth having as its own verb is what travels with an item. A step in a workflow is usually two or three lines with a comment above it and another beside it, and all of that belongs to the step rather than to the position:

$ cat ci.yaml
jobs:
  build:
    steps:
      # check out first
      - uses: actions/checkout@v4  # pinned
      - name: test
        run:  cargo test
      - name: package
        run: cargo build --release
$ yqr -i 'swap(.jobs.build.steps; 0; 1)' ci.yaml
$ cat ci.yaml
jobs:
  build:
    steps:
      - name: test
        run:  cargo test
      # check out first
      - uses: actions/checkout@v4  # pinned
      - name: package
        run: cargo build --release

Both comments moved with the item they document, and the odd spacing in run: cargo test came through untouched, because nothing was re-emitted.

An inline list (ports: [80, 443, 8080]) reorders too. Its items have no lines of their own, so there is no comment to carry – just the values, in their new order.

Two refusals, both exit 5 with the file left alone under -i: an index outside the sequence, and a path that names something other than a sequence. There is no partial reorder – either the whole edit lands or none of it does.

Piping from kubectl

There is no file argument needed – yqr reads stdin:

$ kubectl get deploy web -o yaml | yqr -r '.spec.template.spec.containers[0].image'

Worth knowing: -i needs a real file, so it is an error to combine it with stdin. That is deliberate; there is nothing to write back to.

What is not here yet

Being straight about the edges, because finding them yourself is annoying:

  • There is no way to write a value the file does not already hold. A collection has to be copied from somewhere, because the filter grammar has no {} or [] literal.

  • No builtins beyond to_entries. There is no select, no map, and no string interpolation, so a filter cannot yet pick entries by a condition or reshape them.

  • A new key can land below a comment that belongs to the key after it. This happens when the block you are adding to ends with a nested block of its own and a comment follows. Adding .spec.strategy here writes it under the comment, which then reads as documenting strategy rather than revision:

    spec:
  replicas: 3
  template:
    spec:
      containers:
        - name: web
# Managed by the release pipeline. Do not edit by hand.
revision: 42

    The value is written correctly and no other byte moves, so nothing warns you. It is fixed in the YAML engine and arrives with the next engine release. Until then, check the result when the file has a comment in that position.

Next