From c8efd595fc0b8a8c4138229cb922327b4cce1063 Mon Sep 17 00:00:00 2001 From: Tommy Malmqvist Date: Tue, 16 Sep 2025 16:12:45 +0200 Subject: [PATCH] Update README.md --- README.md | 209 ++++++++++++------------------------------------------ 1 file changed, 47 insertions(+), 162 deletions(-) diff --git a/README.md b/README.md index 543983f8..f134d1d7 100644 --- a/README.md +++ b/README.md @@ -1,49 +1,35 @@ # yq -![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) -![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) -![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) -![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq) -![CodeQL](https://github.com/mikefarah/yq/workflows/CodeQL/badge.svg) +![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq) ![CodeQL](https://github.com/mikefarah/yq/workflows/CodeQL/badge.svg) -a lightweight and portable command-line YAML, JSON, INI and XML processor. `yq` -uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files -as well as json, xml, ini, properties, csv and tsv. It doesn't yet support -everything `jq` does - but it does support the most common operations and -functions, and more is being added continuously. -yq is written in go - so you can download a dependency free binary for your -platform and you are good to go! If you prefer there are a variety of package -managers that can be used as well as Docker and Podman, all listed below. +a lightweight and portable command-line YAML, JSON, INI and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files as well as json, xml, ini, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously. + +yq is written in go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as Docker and Podman, all listed below. ## Quick Usage Guide Read a value: - ```bash yq '.a.b[0].c' file.yaml ``` Pipe from STDIN: - ```bash yq '.a.b[0].c' < file.yaml ``` Update a yaml file, in place - ```bash yq -i '.a.b[0].c = "cool"' file.yaml ``` Update using environment variables - ```bash NAME=mike yq -i '.a.b[0].c = strenv(NAME)' file.yaml ``` Merge multiple files - ```bash # merge two files yq -n 'load("file1.yaml") * load("file2.yaml")' @@ -55,7 +41,6 @@ yq ea '. as $item ireduce ({}; . * $item )' path/to/*.yml ``` Multiple updates to a yaml file - ```bash yq -i ' .a.b[0].c = "cool" | @@ -65,37 +50,30 @@ yq -i ' ``` Find and update an item in an array: - ```bash yq '(.[] | select(.name == "foo") | .address) = "12 cat st"' ``` Convert JSON to YAML - ```bash yq -Poy sample.json ``` -See [recipes](https://mikefarah.gitbook.io/yq/recipes) for more examples and the -[documentation](https://mikefarah.gitbook.io/yq/) for more information. +See [recipes](https://mikefarah.gitbook.io/yq/recipes) for more examples and the [documentation](https://mikefarah.gitbook.io/yq/) for more information. -Take a look at the discussions for -[common questions](https://github.com/mikefarah/yq/discussions/categories/q-a), -and -[cool ideas](https://github.com/mikefarah/yq/discussions/categories/show-and-tell) +Take a look at the discussions for [common questions](https://github.com/mikefarah/yq/discussions/categories/q-a), and [cool ideas](https://github.com/mikefarah/yq/discussions/categories/show-and-tell) ## Install ### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest) ### wget - Use wget to download, gzipped pre-compiled binaries: + For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64 #### Compressed via tar.gz - ```bash wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\ tar xz && mv ${BINARY} /usr/local/bin/yq @@ -116,39 +94,28 @@ wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O ``` ### MacOS / Linux via Homebrew: - Using [Homebrew](https://brew.sh/) - ``` brew install yq ``` ### Linux via snap: - ``` snap install yq ``` #### Snap notes - -`yq` installs with -[_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, -this means it doesn't have direct access to root files. To read root files you -can: +`yq` installs with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can: ``` sudo cat /etc/myfile | yq '.a.path' ``` -And to write to a root file you can either use -[sponge](https://linux.die.net/man/1/sponge): - +And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge): ``` sudo cat /etc/myfile | yq '.a.path = "value"' | sudo sponge /etc/myfile ``` - or write to a temporary file: - ``` sudo cat /etc/myfile | yq '.a.path = "value"' | sudo tee /etc/myfile.tmp sudo mv /etc/myfile.tmp /etc/myfile @@ -156,16 +123,14 @@ rm /etc/myfile.tmp ``` ### Run with Docker or Podman - #### Oneshot use: ```bash docker run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE... ``` -Note that you can run `yq` in docker without network access and other privileges -if you desire, namely -`--security-opt=no-new-privileges --cap-drop all --network none`. +Note that you can run `yq` in docker without network access and other privileges if you desire, +namely `--security-opt=no-new-privileges --cap-drop all --network none`. ```bash podman run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE... @@ -193,8 +158,7 @@ docker run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq podman run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq ``` -It can be useful to have a bash function to avoid typing the whole docker -command: +It can be useful to have a bash function to avoid typing the whole docker command: ```bash yq() { @@ -207,13 +171,10 @@ yq() { podman run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@" } ``` - #### Running as root: -`yq`'s container image no longer runs under root -(https://github.com/mikefarah/yq/pull/860). If you'd like to install more things -in the container image, or you're having permissions issues when attempting to -read/write files you'll need to either: +`yq`'s container image no longer runs under root (https://github.com/mikefarah/yq/pull/860). If you'd like to install more things in the container image, or you're having permissions issues when attempting to read/write files you'll need to either: + ``` docker run --user="root" -it --entrypoint sh mikefarah/yq @@ -234,9 +195,7 @@ USER yq ``` #### Missing timezone data - -By default, the alpine image yq uses does not include timezone data. If you'd -like to use the `tz` operator, you'll need to include this data: +By default, the alpine image yq uses does not include timezone data. If you'd like to use the `tz` operator, you'll need to include this data: ``` FROM mikefarah/yq @@ -248,90 +207,45 @@ USER yq #### Podman with SELinux -If you are using podman with SELinux, you will need to set the shared volume -flag `:z` on the volume mount: +If you are using podman with SELinux, you will need to set the shared volume flag `:z` on the volume mount: ``` -v "${PWD}":/workdir:z ``` ### GitHub Action - ``` -- name: Set foobar to cool - uses: mikefarah/yq@master - with: - cmd: yq -i '.foo.bar = "cool"' 'config.yml' -- name: Get an entry with a variable that might contain dots or spaces - id: get_username - uses: mikefarah/yq@master - with: - cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml -- name: Reuse a variable obtained in another step - run: echo ${{ steps.get_username.outputs.result }} + - name: Set foobar to cool + uses: mikefarah/yq@master + with: + cmd: yq -i '.foo.bar = "cool"' 'config.yml' + - name: Get an entry with a variable that might contain dots or spaces + id: get_username + uses: mikefarah/yq@master + with: + cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml + - name: Reuse a variable obtained in another step + run: echo ${{ steps.get_username.outputs.result }} ``` See https://mikefarah.gitbook.io/yq/usage/github-action for more. -### Action: custom registry support - -The action now accepts optional inputs to control which container image is used -and (optionally) authenticate to a registry: - -- `image`: the image name to run (default `mikefarah/yq:4-githubaction`). -- `registry`: optional artifact repository hostname/prefix to be prepended to - `image` (for example `artifacts.example.com/repository/docker`). -- `registry_username` / `registry_password`: optional credentials to login - before pulling the image. - -Important caveat: GitHub-hosted runners pull action container images (when using -`runs.using: docker`) before any workflow steps run. To make -registry-authenticated pulls work on GitHub-hosted runners, either: - -- Use the composite action (this repository) which performs a `docker login` - inside the action before running the container (works when credentials are - provided via action inputs). Note that some registries block pulls even with - login if the runner cannot reach them. -- Use self-hosted runners configured to use your artifact repository as a - pull-through cache or pre-pull the image on the runner. -- Alternatively, avoid using the action container image and instead - `docker login` and `docker run` from a regular `run:` step in your workflow - (see example below). - -Example using the composite action (local action reference): - -``` -- name: Run yq from artifact repository - uses: . - with: - registry: artifacts.example.com/repository/docker - image: mikefarah/yq:4-githubaction - registry_username: ${{ secrets.ARTIFACT_REPO_USER }} - registry_password: ${{ secrets.ARTIFACT_REPO_PASS }} - cmd: yq --version -``` - ### Go Install: - ``` go install github.com/mikefarah/yq/v4@latest ``` ## Community Supported Installation methods +As these are supported by the community :heart: - however, they may be out of date with the officially supported releases. -As these are supported by the community :heart: - however, they may be out of -date with the officially supported releases. +_Please note that the Debian package (previously supported by @rmescandon) is no longer maintained. Please use an alternative installation method._ -_Please note that the Debian package (previously supported by @rmescandon) is no -longer maintained. Please use an alternative installation method._ ### X-CMD - Checkout `yq` on x-cmd: https://x-cmd.com/mod/yq - Instant Results: See the output of your yq filter in real-time. -- Error Handling: Encounter a syntax error? It will display the error message - and the results of the closest valid filter +- Error Handling: Encounter a syntax error? It will display the error message and the results of the closest valid filter Thanks @edwinjhlee! @@ -341,8 +255,8 @@ Thanks @edwinjhlee! nix profile install nixpkgs#yq-go ``` -See -[here](https://search.nixos.org/packages?channel=unstable&show=yq-go&from=0&size=50&sort=relevance&type=packages&query=yq-go) +See [here](https://search.nixos.org/packages?channel=unstable&show=yq-go&from=0&size=50&sort=relevance&type=packages&query=yq-go) + ### Webi @@ -350,8 +264,8 @@ See webi yq ``` -See [webi](https://webinstall.dev/) Supported by @adithyasunil26 -(https://github.com/webinstall/webi-installers/tree/master/yq) +See [webi](https://webinstall.dev/) +Supported by @adithyasunil26 (https://github.com/webinstall/webi-installers/tree/master/yq) ### Arch Linux @@ -365,47 +279,37 @@ Using [Chocolatey](https://chocolatey.org) [![Chocolatey](https://img.shields.io/chocolatey/v/yq.svg)](https://chocolatey.org/packages/yq) [![Chocolatey](https://img.shields.io/chocolatey/dt/yq.svg)](https://chocolatey.org/packages/yq) - ``` choco install yq ``` - Supported by @chillum (https://chocolatey.org/packages/yq) Using [scoop](https://scoop.sh/) - ``` scoop install main/yq ``` Using [winget](https://learn.microsoft.com/en-us/windows/package-manager/) - ``` winget install --id MikeFarah.yq ``` ### MacPorts: - Using [MacPorts](https://www.macports.org/) - ``` sudo port selfupdate sudo port install yq ``` - -Supported by @herbygillot -(https://ports.macports.org/maintainer/github/herbygillot) +Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot) ### Alpine Linux Alpine Linux v3.20+ (and Edge): - ``` apk add yq-go ``` Alpine Linux up to v3.19: - ``` apk add yq ``` @@ -420,8 +324,8 @@ Flox can be used to install yq on Linux, MacOS, and Windows through WSL. flox install yq ``` -### MacOS / Linux via gah: +### MacOS / Linux via gah: Using [gah](https://github.com/marverix/gah) ``` @@ -429,29 +333,19 @@ gah install yq ``` ## Features - - [Detailed documentation with many examples](https://mikefarah.gitbook.io/yq/) - Written in portable go, so you can download a lovely dependency free binary -- Uses similar syntax as `jq` but works with YAML, INI, - [JSON](https://mikefarah.gitbook.io/yq/usage/convert) and - [XML](https://mikefarah.gitbook.io/yq/usage/xml) files +- Uses similar syntax as `jq` but works with YAML, INI, [JSON](https://mikefarah.gitbook.io/yq/usage/convert) and [XML](https://mikefarah.gitbook.io/yq/usage/xml) files - Fully supports multi document yaml files -- Supports yaml - [front matter](https://mikefarah.gitbook.io/yq/usage/front-matter) blocks - (e.g. jekyll/assemble) +- Supports yaml [front matter](https://mikefarah.gitbook.io/yq/usage/front-matter) blocks (e.g. jekyll/assemble) - Colorized yaml output - [Date/Time manipulation and formatting with TZ](https://mikefarah.gitbook.io/yq/operators/datetime) - [Deeply data structures](https://mikefarah.gitbook.io/yq/operators/traverse-read) - [Sort keys](https://mikefarah.gitbook.io/yq/operators/sort-keys) -- Manipulate yaml - [comments](https://mikefarah.gitbook.io/yq/operators/comment-operators), - [styling](https://mikefarah.gitbook.io/yq/operators/style), - [tags](https://mikefarah.gitbook.io/yq/operators/tag) and - [anchors and aliases](https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators). +- Manipulate yaml [comments](https://mikefarah.gitbook.io/yq/operators/comment-operators), [styling](https://mikefarah.gitbook.io/yq/operators/style), [tags](https://mikefarah.gitbook.io/yq/operators/tag) and [anchors and aliases](https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators). - [Update in place](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate#flags) - [Complex expressions to select and update](https://mikefarah.gitbook.io/yq/operators/select#select-and-update-matching-values-in-map) -- Keeps yaml formatting and comments when updating (though there are issues with - whitespace) +- Keeps yaml formatting and comments when updating (though there are issues with whitespace) - [Decode/Encode base64 data](https://mikefarah.gitbook.io/yq/operators/encode-decode) - [Load content from other files](https://mikefarah.gitbook.io/yq/operators/load) - [Convert to/from json/ndjson](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert) @@ -459,15 +353,12 @@ gah install yq - [Convert to/from properties](https://mikefarah.gitbook.io/yq/v/v4.x/usage/properties) - [Convert to/from csv/tsv](https://mikefarah.gitbook.io/yq/usage/csv-tsv) - [General shell completion scripts (bash/zsh/fish/powershell)](https://mikefarah.gitbook.io/yq/v/v4.x/commands/shell-completion) -- [Reduce](https://mikefarah.gitbook.io/yq/operators/reduce) to merge multiple - files or sum an array or other fancy things. -- [Github Action](https://mikefarah.gitbook.io/yq/usage/github-action) to use in - your automated pipeline (thanks @devorbitus) +- [Reduce](https://mikefarah.gitbook.io/yq/operators/reduce) to merge multiple files or sum an array or other fancy things. +- [Github Action](https://mikefarah.gitbook.io/yq/usage/github-action) to use in your automated pipeline (thanks @devorbitus) ## [Usage](https://mikefarah.gitbook.io/yq/) -Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more -detailed and advanced usage. +Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more detailed and advanced usage. ``` Usage: @@ -532,16 +423,10 @@ Flags: Use "yq [command] --help" for more information about a command. ``` - ## Known Issues / Missing Features +- `yq` attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details) +- Powershell has its own...[opinions on quoting yq](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell) +- "yes", "no" were dropped as boolean values in the yaml 1.2 standard - which is the standard yq assumes. -- `yq` attempts to preserve comment positions and whitespace as much as - possible, but it does not handle all scenarios (see - https://github.com/go-yaml/yaml/tree/v3 for details) -- Powershell has its - own...[opinions on quoting yq](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell) -- "yes", "no" were dropped as boolean values in the yaml 1.2 standard - which is - the standard yq assumes. +See [tips and tricks](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks) for more common problems and solutions. -See [tips and tricks](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks) for -more common problems and solutions.