Enhance GitHub Action with custom registry support and input parameters. Updated action.yml to include optional inputs for image, registry, and authentication credentials. Refactored to use a composite action for improved container management. Updated README.md to reflect new features and usage examples.

This commit is contained in:
Tommy Malmqvist 2025-09-16 15:56:56 +02:00
parent 4532346e13
commit fc8a3fc3ce
2 changed files with 227 additions and 56 deletions

212
README.md
View File

@ -1,35 +1,49 @@
# yq
![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq) ![CodeQL](https://github.com/mikefarah/yq/workflows/CodeQL/badge.svg)
![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg)
![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg)
![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg)
![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
![CodeQL](https://github.com/mikefarah/yq/workflows/CodeQL/badge.svg)
a lightweight and portable command-line YAML, JSON, INI and XML processor. `yq`
uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files
as well as json, xml, ini, properties, csv and tsv. It doesn't yet support
everything `jq` does - but it does support the most common operations and
functions, and more is being added continuously.
a lightweight and portable command-line YAML, JSON, INI and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files as well as json, xml, ini, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
yq is written in go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as Docker and Podman, all listed below.
yq is written in go - so you can download a dependency free binary for your
platform and you are good to go! If you prefer there are a variety of package
managers that can be used as well as Docker and Podman, all listed below.
## Quick Usage Guide
Read a value:
```bash
yq '.a.b[0].c' file.yaml
```
Pipe from STDIN:
```bash
yq '.a.b[0].c' < file.yaml
```
Update a yaml file, in place
```bash
yq -i '.a.b[0].c = "cool"' file.yaml
```
Update using environment variables
```bash
NAME=mike yq -i '.a.b[0].c = strenv(NAME)' file.yaml
```
Merge multiple files
```bash
# merge two files
yq -n 'load("file1.yaml") * load("file2.yaml")'
@ -41,6 +55,7 @@ yq ea '. as $item ireduce ({}; . * $item )' path/to/*.yml
```
Multiple updates to a yaml file
```bash
yq -i '
.a.b[0].c = "cool" |
@ -50,30 +65,37 @@ yq -i '
```
Find and update an item in an array:
```bash
yq '(.[] | select(.name == "foo") | .address) = "12 cat st"'
```
Convert JSON to YAML
```bash
yq -Poy sample.json
```
See [recipes](https://mikefarah.gitbook.io/yq/recipes) for more examples and the [documentation](https://mikefarah.gitbook.io/yq/) for more information.
See [recipes](https://mikefarah.gitbook.io/yq/recipes) for more examples and the
[documentation](https://mikefarah.gitbook.io/yq/) for more information.
Take a look at the discussions for [common questions](https://github.com/mikefarah/yq/discussions/categories/q-a), and [cool ideas](https://github.com/mikefarah/yq/discussions/categories/show-and-tell)
Take a look at the discussions for
[common questions](https://github.com/mikefarah/yq/discussions/categories/q-a),
and
[cool ideas](https://github.com/mikefarah/yq/discussions/categories/show-and-tell)
## Install
### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
### wget
Use wget to download, gzipped pre-compiled binaries:
Use wget to download, gzipped pre-compiled binaries:
For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64
#### Compressed via tar.gz
```bash
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
tar xz && mv ${BINARY} /usr/local/bin/yq
@ -94,28 +116,39 @@ wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O
```
### MacOS / Linux via Homebrew:
Using [Homebrew](https://brew.sh/)
```
brew install yq
```
### Linux via snap:
```
snap install yq
```
#### Snap notes
`yq` installs with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
`yq` installs with
[_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap,
this means it doesn't have direct access to root files. To read root files you
can:
```
sudo cat /etc/myfile | yq '.a.path'
```
And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):
And to write to a root file you can either use
[sponge](https://linux.die.net/man/1/sponge):
```
sudo cat /etc/myfile | yq '.a.path = "value"' | sudo sponge /etc/myfile
```
or write to a temporary file:
```
sudo cat /etc/myfile | yq '.a.path = "value"' | sudo tee /etc/myfile.tmp
sudo mv /etc/myfile.tmp /etc/myfile
@ -123,14 +156,16 @@ rm /etc/myfile.tmp
```
### Run with Docker or Podman
#### Oneshot use:
```bash
docker run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...
```
Note that you can run `yq` in docker without network access and other privileges if you desire,
namely `--security-opt=no-new-privileges --cap-drop all --network none`.
Note that you can run `yq` in docker without network access and other privileges
if you desire, namely
`--security-opt=no-new-privileges --cap-drop all --network none`.
```bash
podman run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...
@ -158,7 +193,8 @@ docker run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
podman run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
```
It can be useful to have a bash function to avoid typing the whole docker command:
It can be useful to have a bash function to avoid typing the whole docker
command:
```bash
yq() {
@ -171,10 +207,13 @@ yq() {
podman run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
```
#### Running as root:
`yq`'s container image no longer runs under root (https://github.com/mikefarah/yq/pull/860). If you'd like to install more things in the container image, or you're having permissions issues when attempting to read/write files you'll need to either:
`yq`'s container image no longer runs under root
(https://github.com/mikefarah/yq/pull/860). If you'd like to install more things
in the container image, or you're having permissions issues when attempting to
read/write files you'll need to either:
```
docker run --user="root" -it --entrypoint sh mikefarah/yq
@ -195,7 +234,9 @@ USER yq
```
#### Missing timezone data
By default, the alpine image yq uses does not include timezone data. If you'd like to use the `tz` operator, you'll need to include this data:
By default, the alpine image yq uses does not include timezone data. If you'd
like to use the `tz` operator, you'll need to include this data:
```
FROM mikefarah/yq
@ -207,45 +248,90 @@ USER yq
#### Podman with SELinux
If you are using podman with SELinux, you will need to set the shared volume flag `:z` on the volume mount:
If you are using podman with SELinux, you will need to set the shared volume
flag `:z` on the volume mount:
```
-v "${PWD}":/workdir:z
```
### GitHub Action
```
- name: Set foobar to cool
uses: mikefarah/yq@master
with:
cmd: yq -i '.foo.bar = "cool"' 'config.yml'
- name: Get an entry with a variable that might contain dots or spaces
id: get_username
uses: mikefarah/yq@master
with:
cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
- name: Reuse a variable obtained in another step
run: echo ${{ steps.get_username.outputs.result }}
- name: Set foobar to cool
uses: mikefarah/yq@master
with:
cmd: yq -i '.foo.bar = "cool"' 'config.yml'
- name: Get an entry with a variable that might contain dots or spaces
id: get_username
uses: mikefarah/yq@master
with:
cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
- name: Reuse a variable obtained in another step
run: echo ${{ steps.get_username.outputs.result }}
```
See https://mikefarah.gitbook.io/yq/usage/github-action for more.
### Action: custom registry support
The action now accepts optional inputs to control which container image is used
and (optionally) authenticate to a registry:
- `image`: the image name to run (default `mikefarah/yq:4-githubaction`).
- `registry`: optional artifact repository hostname/prefix to be prepended to
`image` (for example `artifacts.example.com/repository/docker`).
- `registry_username` / `registry_password`: optional credentials to login
before pulling the image.
Important caveat: GitHub-hosted runners pull action container images (when using
`runs.using: docker`) before any workflow steps run. To make
registry-authenticated pulls work on GitHub-hosted runners, either:
- Use the composite action (this repository) which performs a `docker login`
inside the action before running the container (works when credentials are
provided via action inputs). Note that some registries block pulls even with
login if the runner cannot reach them.
- Use self-hosted runners configured to use your artifact repository as a
pull-through cache or pre-pull the image on the runner.
- Alternatively, avoid using the action container image and instead
`docker login` and `docker run` from a regular `run:` step in your workflow
(see example below).
Example using the composite action (local action reference):
```
- name: Run yq from artifact repository
uses: .
with:
registry: artifacts.example.com/repository/docker
image: mikefarah/yq:4-githubaction
registry_username: ${{ secrets.ARTIFACT_REPO_USER }}
registry_password: ${{ secrets.ARTIFACT_REPO_PASS }}
cmd: yq --version
```
### Go Install:
```
go install github.com/mikefarah/yq/v4@latest
```
## Community Supported Installation methods
As these are supported by the community :heart: - however, they may be out of date with the officially supported releases.
_Please note that the Debian package (previously supported by @rmescandon) is no longer maintained. Please use an alternative installation method._
As these are supported by the community :heart: - however, they may be out of
date with the officially supported releases.
_Please note that the Debian package (previously supported by @rmescandon) is no
longer maintained. Please use an alternative installation method._
### X-CMD
Checkout `yq` on x-cmd: https://x-cmd.com/mod/yq
- Instant Results: See the output of your yq filter in real-time.
- Error Handling: Encounter a syntax error? It will display the error message and the results of the closest valid filter
- Error Handling: Encounter a syntax error? It will display the error message
and the results of the closest valid filter
Thanks @edwinjhlee!
@ -255,8 +341,8 @@ Thanks @edwinjhlee!
nix profile install nixpkgs#yq-go
```
See [here](https://search.nixos.org/packages?channel=unstable&show=yq-go&from=0&size=50&sort=relevance&type=packages&query=yq-go)
See
[here](https://search.nixos.org/packages?channel=unstable&show=yq-go&from=0&size=50&sort=relevance&type=packages&query=yq-go)
### Webi
@ -264,8 +350,8 @@ See [here](https://search.nixos.org/packages?channel=unstable&show=yq-go&from=0&
webi yq
```
See [webi](https://webinstall.dev/)
Supported by @adithyasunil26 (https://github.com/webinstall/webi-installers/tree/master/yq)
See [webi](https://webinstall.dev/) Supported by @adithyasunil26
(https://github.com/webinstall/webi-installers/tree/master/yq)
### Arch Linux
@ -279,37 +365,47 @@ Using [Chocolatey](https://chocolatey.org)
[![Chocolatey](https://img.shields.io/chocolatey/v/yq.svg)](https://chocolatey.org/packages/yq)
[![Chocolatey](https://img.shields.io/chocolatey/dt/yq.svg)](https://chocolatey.org/packages/yq)
```
choco install yq
```
Supported by @chillum (https://chocolatey.org/packages/yq)
Using [scoop](https://scoop.sh/)
```
scoop install main/yq
```
Using [winget](https://learn.microsoft.com/en-us/windows/package-manager/)
```
winget install --id MikeFarah.yq
```
### MacPorts:
Using [MacPorts](https://www.macports.org/)
```
sudo port selfupdate
sudo port install yq
```
Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot)
Supported by @herbygillot
(https://ports.macports.org/maintainer/github/herbygillot)
### Alpine Linux
Alpine Linux v3.20+ (and Edge):
```
apk add yq-go
```
Alpine Linux up to v3.19:
```
apk add yq
```
@ -324,8 +420,8 @@ Flox can be used to install yq on Linux, MacOS, and Windows through WSL.
flox install yq
```
### MacOS / Linux via gah:
Using [gah](https://github.com/marverix/gah)
```
@ -333,19 +429,29 @@ gah install yq
```
## Features
- [Detailed documentation with many examples](https://mikefarah.gitbook.io/yq/)
- Written in portable go, so you can download a lovely dependency free binary
- Uses similar syntax as `jq` but works with YAML, INI, [JSON](https://mikefarah.gitbook.io/yq/usage/convert) and [XML](https://mikefarah.gitbook.io/yq/usage/xml) files
- Uses similar syntax as `jq` but works with YAML, INI,
[JSON](https://mikefarah.gitbook.io/yq/usage/convert) and
[XML](https://mikefarah.gitbook.io/yq/usage/xml) files
- Fully supports multi document yaml files
- Supports yaml [front matter](https://mikefarah.gitbook.io/yq/usage/front-matter) blocks (e.g. jekyll/assemble)
- Supports yaml
[front matter](https://mikefarah.gitbook.io/yq/usage/front-matter) blocks
(e.g. jekyll/assemble)
- Colorized yaml output
- [Date/Time manipulation and formatting with TZ](https://mikefarah.gitbook.io/yq/operators/datetime)
- [Deeply data structures](https://mikefarah.gitbook.io/yq/operators/traverse-read)
- [Sort keys](https://mikefarah.gitbook.io/yq/operators/sort-keys)
- Manipulate yaml [comments](https://mikefarah.gitbook.io/yq/operators/comment-operators), [styling](https://mikefarah.gitbook.io/yq/operators/style), [tags](https://mikefarah.gitbook.io/yq/operators/tag) and [anchors and aliases](https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators).
- Manipulate yaml
[comments](https://mikefarah.gitbook.io/yq/operators/comment-operators),
[styling](https://mikefarah.gitbook.io/yq/operators/style),
[tags](https://mikefarah.gitbook.io/yq/operators/tag) and
[anchors and aliases](https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators).
- [Update in place](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate#flags)
- [Complex expressions to select and update](https://mikefarah.gitbook.io/yq/operators/select#select-and-update-matching-values-in-map)
- Keeps yaml formatting and comments when updating (though there are issues with whitespace)
- Keeps yaml formatting and comments when updating (though there are issues with
whitespace)
- [Decode/Encode base64 data](https://mikefarah.gitbook.io/yq/operators/encode-decode)
- [Load content from other files](https://mikefarah.gitbook.io/yq/operators/load)
- [Convert to/from json/ndjson](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert)
@ -353,12 +459,15 @@ gah install yq
- [Convert to/from properties](https://mikefarah.gitbook.io/yq/v/v4.x/usage/properties)
- [Convert to/from csv/tsv](https://mikefarah.gitbook.io/yq/usage/csv-tsv)
- [General shell completion scripts (bash/zsh/fish/powershell)](https://mikefarah.gitbook.io/yq/v/v4.x/commands/shell-completion)
- [Reduce](https://mikefarah.gitbook.io/yq/operators/reduce) to merge multiple files or sum an array or other fancy things.
- [Github Action](https://mikefarah.gitbook.io/yq/usage/github-action) to use in your automated pipeline (thanks @devorbitus)
- [Reduce](https://mikefarah.gitbook.io/yq/operators/reduce) to merge multiple
files or sum an array or other fancy things.
- [Github Action](https://mikefarah.gitbook.io/yq/usage/github-action) to use in
your automated pipeline (thanks @devorbitus)
## [Usage](https://mikefarah.gitbook.io/yq/)
Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more detailed and advanced usage.
Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more
detailed and advanced usage.
```
Usage:
@ -423,9 +532,16 @@ Flags:
Use "yq [command] --help" for more information about a command.
```
## Known Issues / Missing Features
- `yq` attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)
- Powershell has its own...[opinions on quoting yq](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell)
- "yes", "no" were dropped as boolean values in the yaml 1.2 standard - which is the standard yq assumes.
See [tips and tricks](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks) for more common problems and solutions.
## Known Issues / Missing Features
- `yq` attempts to preserve comment positions and whitespace as much as
possible, but it does not handle all scenarios (see
https://github.com/go-yaml/yaml/tree/v3 for details)
- Powershell has its
own...[opinions on quoting yq](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell)
- "yes", "no" were dropped as boolean values in the yaml 1.2 standard - which is
the standard yq assumes.
See [tips and tricks](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks) for
more common problems and solutions.

View File

@ -1,17 +1,72 @@
name: 'yq - portable yaml processor'
description: 'create, read, update, delete, merge, validate and do more with yaml'
name: "yq - portable yaml processor"
description: "create, read, update, delete, merge, validate and do more with yaml"
branding:
icon: command
color: gray-dark
inputs:
image:
description: 'Container image to run. Example: "mikefarah/yq:4-githubaction" or fully qualified "artifacts.example.com/repo/mikefarah/yq:4-githubaction".'
required: false
default: "mikefarah/yq:4-githubaction"
registry:
description: "Optional artifact repository hostname to prefix the `image`. Leave empty if your `image` already includes a registry."
required: false
default: ""
registry_username:
description: "Optional registry username for `docker login` (use with `registry_password`)."
required: false
default: ""
registry_password:
description: "Optional registry password for `docker login` (use with `registry_username`). Pass secrets via workflow `with:` from secrets."
required: false
default: ""
cmd:
description: 'The Command which should be run'
description: "The Command which should be run"
required: true
runs:
using: "composite"
steps:
- id: login
name: Login to registry (if credentials provided)
if: ${{ inputs.registry_username && inputs.registry_password && inputs.registry }}
shell: bash
env:
REGISTRY: ${{ inputs.registry }}
REG_USER: ${{ inputs.registry_username }}
REG_PASS: ${{ inputs.registry_password }}
run: |
set -euo pipefail
if [ -z "$REGISTRY" ] || [ -z "$REG_USER" ] || [ -z "$REG_PASS" ]; then
echo "Missing registry or credentials; skipping login"
exit 0
fi
echo "Logging into registry: $REGISTRY"
echo "$REG_PASS" | docker login "$REGISTRY" --username "$REG_USER" --password-stdin
- id: run
name: Run yq container
shell: bash
env:
IMAGE_INPUT: ${{ inputs.image }}
REGISTRY: ${{ inputs.registry }}
CMD_INPUT: ${{ inputs.cmd }}
run: |
set -euo pipefail
IMAGE="$IMAGE_INPUT"
if [ -n "$REGISTRY" ]; then
REG="${REGISTRY%/}"
IMAGE="$REG/$IMAGE"
fi
echo "Using image: $IMAGE"
RC=0
OUTPUT=$(docker run --rm -v "$GITHUB_WORKSPACE":/work -w /work "$IMAGE" sh -lc "$CMD_INPUT" 2>&1) || RC=$?
echo "result<<EOF" >> $GITHUB_OUTPUT
echo "$OUTPUT" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
if [ "$RC" -ne 0 ]; then
exit "$RC"
fi
outputs:
result:
description: "The complete result from the yq command being run"
runs:
using: 'docker'
image: 'docker://mikefarah/yq:4-githubaction'
args:
- ${{ inputs.cmd }}
value: ${{ steps.run.outputs.result }}