yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor
Go to file
Abel Sen fa6fac1a76
Minor typos (#1595)
* Remove extra backtick

* Reword explanation of update

* Reword explanation of relative update

* Change "remaple" to "remain"

* Change "clovver" to "clobber"

* Reword explanation of update for comment operators

* Reword explanation of relative update for comment operators

* Change "array" to "expression"

* Change "the golangs" to "Golang's"

* Change "golangs" to "Golang's"

* Change "can durations" to "can add durations"

* Change "array scalars" to "arrays"

* Change "beit" to "be it"

* Fix typo in `eval` tip

* Fix typo in header for `has` operation

* Add space before pipe in `line` operator example

* Fix typos in explanation of deep array merges

* Change "is now used" to "is now used."

* Change "object," to "object."

* Changes "indexes" to "indices"

* Remove extraneous copied text from `..` article

* Reword explanation of `...` operator

* Change "your are" to "you are"

* Add link to `string` operator docs in `select` article

* Change "is a" to "parameter specifies" in `string` operators article

* Change "new line" to "newline"

* Change "golang regex" to "Golang's regex"

* Change "golang" to "Golang"

* Add period

* Remove comma in `subtract` article

* Remove duplicate number subtraction example

* Remove comma in `traverse` operator article

* Clarify use of brackets when `read`ing with special characters
2023-03-16 13:39:36 +11:00
.github Fix github action attempt 2023-02-11 23:50:47 +11:00
acceptance_tests Auto output format (#1599) 2023-03-15 13:22:58 +11:00
cmd Auto output format (#1599) 2023-03-15 13:22:58 +11:00
debian Minor typos (#1595) 2023-03-16 13:39:36 +11:00
examples Auto output format (#1599) 2023-03-15 13:22:58 +11:00
github-action Fix broken multiline strings from dee0c94 (#1390) 2022-10-19 10:16:07 +11:00
pkg/yqlib Minor typos (#1595) 2023-03-16 13:39:36 +11:00
scripts Minor typos (#1595) 2023-03-16 13:39:36 +11:00
snap Bumping version 2023-03-02 10:58:57 +11:00
test Fixed load operator bug 2022-10-25 13:30:38 +11:00
.dockerignore Added gosec 2021-03-03 19:44:34 +11:00
.gitignore Bumped go compiler; bumped quality tools 2022-08-29 14:13:15 +10:00
.golangci.yml refactor: move from io/ioutil to io and os packages (#1068) 2022-01-21 19:50:53 +11:00
action.yml action uses built docker image 2022-10-19 11:06:41 +11:00
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md 2020-02-21 21:13:14 +11:00
CONTRIBUTING.md Minor typos (#1595) 2023-03-16 13:39:36 +11:00
Dockerfile Stripped debugging info from binary 2023-03-16 13:37:30 +11:00
Dockerfile.dev Bump golang from 1.20.1 to 1.20.2 (#1589) 2023-03-10 10:13:22 +11:00
go.mod Bump golang.org/x/net from 0.7.0 to 0.8.0 (#1590) 2023-03-10 10:13:11 +11:00
go.sum Bump golang.org/x/net from 0.7.0 to 0.8.0 (#1590) 2023-03-10 10:13:11 +11:00
how-it-works.md chore: remove duplicate word in doc (#1322) 2022-08-31 18:43:03 +10:00
LICENSE Added XML decoder (#1044) 2021-12-21 15:02:07 +11:00
Makefile Add support for Podman as well as Docker (#1026) 2021-12-02 10:01:44 +11:00
Makefile.variables Stripped debugging info from binary 2023-03-16 13:37:30 +11:00
mkdocs.yml Removed user docs 2020-01-30 10:32:34 +11:00
README.md Update README.md 2023-03-15 21:21:15 +11:00
release_instructions.txt updating release notes 2022-11-28 08:39:48 +11:00
release_notes.txt Minor typos (#1595) 2023-03-16 13:39:36 +11:00
utf8.csv Fixed handling of UTF8 encoded CSVs #1373 2022-10-08 13:12:50 +11:00
yq.go Fix zsh completion (#1108) 2022-02-14 08:03:24 +11:00

yq

Build Docker Pulls Github Releases (by Release) Go Report CodeQL

a lightweight and portable command-line YAML, JSON and XML processor. yq uses jq like syntax but works with yaml files as well as json, xml, properties, csv and tsv. It doesn't yet support everything jq does - but it does support the most common operations and functions, and more is being added continuously.

yq is written in go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as Docker and Podman, all listed below.

Quick Usage Guide

Read a value:

yq '.a.b[0].c' file.yaml

Pipe from STDIN:

yq '.a.b[0].c' < file.yaml

Update a yaml file, inplace

yq -i '.a.b[0].c = "cool"' file.yaml

Update using environment variables

NAME=mike yq -i '.a.b[0].c = strenv(NAME)' file.yaml

Merge multiple files

# note the use of `ea` to evaluate all the files at once
# instead of in sequence
yq ea '. as $item ireduce ({}; . * $item )' path/to/*.yml

Multiple updates to a yaml file

yq -i '
  .a.b[0].c = "cool" |
  .x.y.z = "foobar" |
  .person.name = strenv(NAME)
' file.yaml

Convert JSON to YAML

yq -P sample.json

See the documentation for more examples.

Take a look at the discussions for common questions, and cool ideas

Install

Download the latest binary

wget

Use wget to download, gzipped pre-compiled binaries:

For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64

Compressed via tar.gz

wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
  tar xz && mv ${BINARY} /usr/bin/yq

Plain binary

wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
    chmod +x /usr/bin/yq

Latest version

wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq &&\
    chmod +x /usr/bin/yq

MacOS / Linux via Homebrew:

Using Homebrew

brew install yq

Linux via snap:

snap install yq

Snap notes

yq installs with strict confinement in snap, this means it doesn't have direct access to root files. To read root files you can:

sudo cat /etc/myfile | yq '.a.path'

And to write to a root file you can either use sponge:

sudo cat /etc/myfile | yq '.a.path = "value"' | sudo sponge /etc/myfile

or write to a temporary file:

sudo cat /etc/myfile | yq '.a.path = "value"' | sudo tee /etc/myfile.tmp
sudo mv /etc/myfile.tmp /etc/myfile
rm /etc/myfile.tmp

Run with Docker or Podman

Oneshot use:

docker run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...

Note that you can run yq in docker without network access and other privileges if you desire, namely --security-opt=no-new-privileges --cap-drop all --network none.

podman run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...

Pipe in via STDIN:

You'll need to pass the -i\--interactive flag to docker:

docker run -i --rm mikefarah/yq '.this.thing' < myfile.yml
podman run -i --rm mikefarah/yq '.this.thing' < myfile.yml

Run commands interactively:

docker run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
podman run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq

It can be useful to have a bash function to avoid typing the whole docker command:

yq() {
  docker run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
yq() {
  podman run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}

Running as root:

yq's container image no longer runs under root (https://github.com/mikefarah/yq/pull/860). If you'd like to install more things in the container image, or you're having permissions issues when attempting to read/write files you'll need to either:

docker run --user="root" -it --entrypoint sh mikefarah/yq
podman run --user="root" -it --entrypoint sh mikefarah/yq

Or, in your Dockerfile:

FROM mikefarah/yq

USER root
RUN apk add --no-cache bash
USER yq

Missing timezone data

By default, the alpine image yq uses does not include timezone data. If you'd like to use the tz operator, you'll need to include this data:

FROM mikefarah/yq

USER root
RUN apk add --no-cache tzdata
USER yq

Podman with SELinux

If you are using podman with SELinux, you will need to set the shared volume flag :z on the volume mount:

-v "${PWD}":/workdir:z

GitHub Action

  - name: Set foobar to cool
    uses: mikefarah/yq@master
    with:
      cmd: yq -i '.foo.bar = "cool"' 'config.yml'
  - name: Get an entry with a variable that might contain dots or spaces
    id: get_username
    uses: mikefarah/yq@master
    with:
      cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
  - name: Reuse a variable obtained in another step
    run: echo ${{ steps.get_username.outputs.result }}

See https://mikefarah.gitbook.io/yq/usage/github-action for more.

Go Install:

go install github.com/mikefarah/yq/v4@latest

Community Supported Installation methods

As these are supported by the community ❤️ - however, they may be out of date with the officially supported releases.

Please note that the Debian package (previously supported by @rmescandon) is no longer maintained. Please use an alternative installation method.

Nix

nix profile install nixpkgs#yq-go

See here

Webi

webi yq

See webi Supported by @adithyasunil26 (https://github.com/webinstall/webi-installers/tree/master/yq)

Arch Linux

pacman -S go-yq

Windows:

Chocolatey Chocolatey

choco install yq

Supported by @chillum (https://chocolatey.org/packages/yq)

and

Winget

winget install yq

https://winget.run/pkg/MikeFarah/yq

Mac:

Using MacPorts

sudo port selfupdate
sudo port install yq

Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot)

Alpine Linux

  • Enable edge/community repo by adding $MIRROR/alpine/edge/community to /etc/apk/repositories
  • Update database index with apk update
  • Install yq with apk add yq

Supported by Tuan Hoang https://pkgs.alpinelinux.org/package/edge/community/x86/yq

Features

Usage

Check out the documentation for more detailed and advanced usage.

Usage:
  yq [flags]
  yq [command]

Examples:

# yq defaults to 'eval' command if no command is specified. See "yq eval --help" for more examples.
yq '.stuff' < myfile.yml # outputs the data at the "stuff" node from "myfile.yml"

yq -i '.stuff = "foo"' myfile.yml # update myfile.yml inplace


Available Commands:
  completion       Generate the autocompletion script for the specified shell
  eval             (default) Apply the expression to each document in each yaml file in sequence
  eval-all         Loads _all_ yaml documents of _all_ yaml files and runs expression once
  help             Help about any command
  shell-completion Generate completion script

Flags:
  -C, --colors                        force print with colors
  -e, --exit-status                   set exit status if there are no matches or null or false is returned
  -f, --front-matter string           (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
      --header-preprocess             Slurp any header comments and separators before processing expression. (default true)
  -h, --help                          help for yq
  -I, --indent int                    sets indent level for output (default 2)
  -i, --inplace                       update the file inplace of first file given.
  -p, --input-format string           [yaml|y|xml|x] parse format for input. Note that json is a subset of yaml. (default "yaml")
  -M, --no-colors                     force print with no colors
  -N, --no-doc                        Don't print document separators (---)
  -n, --null-input                    Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
  -o, --output-format string          [yaml|y|json|j|props|p|xml|x] output format type. (default "yaml")
  -P, --prettyPrint                   pretty print, shorthand for '... style = ""'
  -s, --split-exp string              print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
      --unwrapScalar                  unwrap scalar, print the value with no quotes, colors or comments (default true)
  -v, --verbose                       verbose mode
  -V, --version                       Print version information and quit
      --xml-attribute-prefix string   prefix for xml attributes (default "+")
      --xml-content-name string       name for xml content (if no attribute name is present). (default "+content")

Use "yq [command] --help" for more information about a command.

Known Issues / Missing Features

See tips and tricks for more common problems and solutions.