* Initial plan
* Fix TOML encoder to prefer readable table sections over inline tables
When converting from YAML/JSON to TOML, the encoder now always uses
readable TOML table section syntax ([section]) instead of compact inline
hash table syntax (key = { ... }), which better matches TOML's goal as
a human-focused configuration format.
Changes:
- decoder_toml.go: Mark inline TOML tables with FlowStyle so round-trips
can be distinguished from YAML flow mappings
- encoder_toml.go:
- encodeTopLevelEntry: use FlowStyle check instead of EncodeSeparate to
decide inline vs table section (all block mappings now become tables)
- encodeSeparateMapping: count FlowStyle children as attributes; use
recursive encodeSeparateMapping for nested non-flow mappings
- encodeMappingBodyWithPath: emit non-flow child mappings as sub-table
sections instead of inline tables
- toml_test.go: add encode (YAML→TOML) test scenarios, update roundtrip
expectations for inline tables (now expanded to table sections)
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/4824a219-6d5e-42e7-bca1-a8a277bf8c6a
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Fix TOML roundtrip: use TomlInline flag instead of FlowStyle to preserve inline tables
FlowStyle affected YAML decode output (causing inline tables to appear as
YAML flow mappings). Replace it with a new TOML-specific TomlInline bool
on CandidateNode that:
- Is set by the TOML decoder for inline tables (not FlowStyle)
- Is copied by UpdateAttributesFrom so it survives DeeplyAssign merges
- Is checked by the TOML encoder alongside FlowStyle (for YAML flow maps)
- Has no effect on the YAML encoder, preserving existing TOML→YAML output
TOML roundtrip tests are restored to their original expected values (inline
tables stay inline, table sections stay as sections).
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/f59bdf62-6d16-4664-991b-38eb87c9d81c
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Refactor EncodeSeparate+TomlInline into a single EncodeHint enum
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/24db9a8f-601d-4ccf-ada7-129ed3226bb6
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Fix stale comment in hasStructuralChildren
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/24db9a8f-601d-4ccf-ada7-129ed3226bb6
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Remove unused hasStructuralChildren method from tomlEncoder
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/2c234b77-28e9-4995-ba6f-9d213ec551a0
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
recurseNodeObjectEqual and containsObject both used findInArray to
locate keys in a MappingNode's Content array. findInArray steps by 1,
so it matches against both keys (even indices) and values (odd indices).
In recurseNodeObjectEqual, when a null key in the LHS matched a null
value in the RHS at the last position, rhs.Content[indexInRHS+1]
accessed an out-of-bounds index, causing a panic.
In containsObject, a %2 guard prevented the panic but introduced false
negatives: when a null value appeared before the actual null key,
findInArray returned the value's odd index, the guard rejected it, and
the function reported the key as missing.
Both functions now use findKeyInMap, which steps by 2 and compares only
key positions. The %2 guard in containsObject is removed.
Reproducer for the panic (recurseNodeObjectEqual):
echo '? [{~: ~}]
: v1
? [{2: ~}]
: v2' | yq '. += .'
Reproducer for the false negative (containsObject):
printf '? 1\n: ~\n? ~\n: x\n' | yq 'contains({~: "x"})'
Found by OSS-Fuzz via the lima project's FuzzEvaluateExpression target.
https://issues.oss-fuzz.com/issues/383860504
Signed-off-by: Jan Dubois <jan@jandubois.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sliceArrayOperator adjusts negative indices by adding Content length,
but does not clamp the result. When the absolute value of a negative
index exceeds Content length (e.g. .[-99999:3] on a 3-element array),
the adjusted index remains negative and causes an out-of-bounds access
in the Content slice loop.
Extract the adjust-and-clamp logic into clampSliceIndex and use it for
both index positions.
Reproducer (panics before this fix, returns full array after):
echo '[a, b, c]' | yq '.[-99999:3]'
Found by OSS-Fuzz via the lima project's FuzzEvaluateExpression target.
https://issues.oss-fuzz.com/issues/438776028
Signed-off-by: Jan Dubois <jan@jandubois.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
go-yaml accepts cross-document alias references, which the YAML spec
forbids (anchors are scoped to a single document). When a nested
assignment targets such an alias, UpdateFrom copies the Alias field
between nodes, creating a self-referencing AliasNode. Both traverse()
and traverseArrayIndices() then follow this cycle indefinitely.
Extract resolveAliasChain(), which follows aliases iteratively with a
visited set and returns an error on cycles. Both traverse() and
traverseArrayIndices() now call it, eliminating the recursive alias
handling in both code paths.
Note: traverseMergeAnchor() also dereferences aliases (lines 358 and
371) but with single-step assignment, not recursion. A self-referencing
alias there falls through the kind switch silently rather than
crashing. Using resolveAliasChain() in that function would produce a
clear error instead of silently dropping the node.
Reproducer (stack overflow before this fix, returns error after):
echo '&-- a
---
*--' | yq eval-all '. = (.x = 1)'
Found by OSS-Fuzz via the lima project's FuzzEvaluateExpression target.
https://issues.oss-fuzz.com/issues/390467412
Signed-off-by: Jan Dubois <jan@jandubois.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The existing check (count > 10 million) does not account for string
length. A 68-byte string repeated 35 trillion times passes the count
check but panics in strings.Repeat with "makeslice: len out of range".
Smaller counts (e.g. 10 million * 6-byte string = 60 MB) cause OOM on
memory-constrained environments like OSS-Fuzz (2560 MB limit).
Replace the count-only check with a result size check: the product of
string length and repeat count must not exceed 10 MiB. Use division
(len > limit/count) instead of multiplication (len*count > limit) to
avoid integer overflow — a large count can wrap the product to a
negative value, bypassing the guard entirely.
Fixes at least four OSS-Fuzz bugs found via Lima's FuzzEvaluateExpression:
https://issues.oss-fuzz.com/issues/418818862 (makeslice overflow)
https://issues.oss-fuzz.com/issues/422001683 (timeout from huge alloc)
https://issues.oss-fuzz.com/issues/383195001 (OOM, 3 GB allocation)
https://issues.oss-fuzz.com/issues/385180606 (OOM, 97 TB allocation)
Signed-off-by: Jan Dubois <jan@jandubois.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When using --front-matter, yq creates a temporary file for the
extracted YAML content but replaces the original filename in args
with the temp file path. This caused the 'filename' operator to
return the temp file path instead of the original filename.
Added a filename alias mechanism: when front matter processing
replaces the file path, it registers the original filename as an
alias. The readDocuments and stream evaluator functions resolve
aliases before setting candidateNode.filename.
Fixes#2538
Co-authored-by: cobyfrombrooklyn-bot <cobyfrombrooklyn@gmail.com>
Standalone TOML comments immediately inside a table/array-table no longer end the table scope, preventing subsequent keys from being flattened to the document root.
Adds a new CLI flag that enables compact sequence indentation where '- ' is
considered part of the indentation. This leverages the CompactSeqIndent()
method from the underlying go.yaml.in/yaml/v4 library.
Example output with --yaml-compact-seq-indent:
parent:
items:
- one
- two
Instead of the default:
parent:
items:
- one
- two
Closes#1841
The colorizeToml function intended to force colors by setting
color.NoColor = false, but SprintFunc() still respects the NO_COLOR
environment variable. This caused TestTomlColourization to fail in
CI environments where NO_COLOR=1 is set.
Fixed by calling EnableColor() on each color object, which explicitly
forces colors regardless of environment settings.
Vibe-coded with Cursor (Claude Opus 4)
* feat: Add UnwrapScalar to ShellVariablesPreferences
- Add UnwrapScalar boolean field to ShellVariablesPreferences struct.
- Initialize UnwrapScalar to false in NewDefaultShellVariablesPreferences.
- This preference will control whether shell output should be quoted or raw.
* feat: Propagate unwrapScalar to ShellVariablesPreferences
- In configureEncoder function, set UnwrapScalar in ConfiguredShellVariablesPreferences.
- This ensures the -r flag's state is passed to the shell encoder for raw output control.
* feat: Implement conditional quoting in shellVariablesEncoder
- Modify doEncode method to check pe.prefs.UnwrapScalar.
- If UnwrapScalar is true, output raw node.Value.
- Otherwise, use quoteValue for shell-safe quoting.
- This enables quote-free output for Kubernetes workflows when -r is used.
* test: Add tests for UnwrapScalar in shell encoder
- Introduce assertEncodesToUnwrapped helper function.
- Add TestShellVariablesEncoderUnwrapScalar to verify quote-free output with -r.
- Add TestShellVariablesEncoderDefaultQuoting to confirm default quoting behavior without -r.
- Ensure comprehensive testing of conditional quoting logic for shell output.
* remove redundant test
* feat: K8S KYAML output format support
Reference: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/5295-kyaml/README.md
Co-authored-by: Codex <codex@openai.com>
Generated-with: OpenAI Codex CLI (partial)
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
* build: gomodcache/gocache should not be committed
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
* chore: fix spelling of behaviour
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
* build: pass GOFLAGS to docker to support buildvcs=false
In trying to develop the KYAML support, various tests gave false
positive results because they made assumptions about Git functionality
Make it possible to avoid that by passing GOFLAGS='-buildvcs=false' to
to Makefile.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
* doc: cover documentScenarios for tests
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
* build: exclude go caches from gosec
Without tuning, gosec scans all of the vendor/gocache/gomodcache, taking
several minutes (3m35 here), whereas the core of the yq takes only 15
seconds to scan.
If we intend to remediate upstream issues in future; add a seperate
target to scan those.
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
---------
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Co-authored-by: Codex <codex@openai.com>
- Add ShellVariablesPreferences struct with KeySeparator field (default: '_')
- Update shellVariablesEncoder to use configurable separator
- Add --shell-key-separator CLI flag
- Add comprehensive tests for custom separator functionality
- Update documentation with example usage for custom separator
This feature allows users to specify a custom separator (e.g. '__') when
outputting shell variables, which helps disambiguate nested keys from
keys that contain underscores in their names.
Example:
yq -o=shell --shell-key-separator='__' file.yaml
Fixes ambiguity when original YAML keys contain underscores.
- Allow inline maps instead of just aliases
- Allow aliased sequences
- Disallow other types
- Use tag `!!merge` instead of key `<<`
- Fix insertion index for sequence merge
Closes#2386
Updated the list of supported platforms in the GitHub Actions
Docker release workflow to include linux/s390x.
Signed-off-by: Ashok Pariya <ashok.pariya@ibm.com>
With docker-hub starting to introduce a much stricter limit on anonymous
pulls, it makes sense to also host the image in ghcr.io as well,
allowing users flexibility in where they pull the docker images from.
Also with the github action it makes more sense hosting the docker image
on github infrastructure.
I've introduced a github action for logging into registries as well.
Problem: When --split-exp is used and produces filenames with slashes in
them, the target directories must already exist otherwise yq fails.
Fix/feature: Create the necessary directories with os.MkdirAll().
The permissions 0750 were chosen to satisfy the vulnerability checker.
Solves
```
❯ make test
Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM Error: unable to connect to Podman socket: failed to read identity "/Users/tiger/.local/share/containers/podman/machine/machine": open /Users/tiger/.local/share/containers/podman/machine/machine: no such file or directory /opt/homebrew/bin/docker run --rm -e LDFLAGS=" -X main.GitCommit=bbdd974+CHANGES -X main.GitDescribe=v4 -w -s" -e GITHUB_TOKEN="" -v /Users/tiger/git/yq-k8s/vendor:/go/src -v /Users/tiger/git/yq-k8s:/yq/src/github.com/mikefarah/yq -w /yq/src/github.com/mikefarah/yq yq_dev go mod vendor
Error: podman-machine-default: VM already exists
```
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
* build: use goreleaser to cross-compile binaries for all platforms and archs
The main motivation behind it is because "gox" is unmaintained and archived
and it does not support "linux/riscv64" as a target.
Right now, goreleaser is only building the binaries, and I've tried to replicate
the exact same way the old script does.
In the future, if so desired, goreleaser could be used to further automate the
build and release pipeline by creating the release in GitHub.
* build: create binaries and OCI image for linux/riscv64
* convert file ext to lowercase for format detection
To ensure proper file format detection with case-insensitive file
systems.
* use filepath.Ext for more reliable file ext detection
especially for paths like index.js/foo
* add a test for file ext based format detection
This commit adds the --properties-separator option, which lets users
specify the separator used between keys and values in the properties
output format. This is done by adjusting the value of
github.com/magiconair/properties#Properties.WriteSeparator at encode
time.
Some refactoring of the properties encoder unit tests was done to make
it easier to write unit tests that include different separator values.
Fixes: #1864
Signed-off-by: Ryan Drew <ryan.drew@isovalent.com>
When the shell executing yq has no open stdin, os.Stdin.Stat() return nil and yq fails to continue. This commit fixes a missing verification on the result of os.Stdin.Stat() in the utils.processStdInArgs function and adds an acceptance test to cover this scenario in the future. This bug affects yq since version 4.26.1.
* encoder_lua: Handle explicitly positive infinity
* encoder_lua: Fix inclusion of pre-/suffix when prettyPrinted
It seems certain operations like --prettyPrint or subset selections does
not produce a DocumentNode, which is where the lua pre- and suffix was
printed, causing those to be omitted.
* encoder_lua: Improve Tag handling robustness
Using the method call seems more reliable in case the input parser
forgets to set the tag.
- Add missing package manager (scoop)
- Improve winget installationn example & add link to reference documentation
- Add link to missing reference documentation for chocolatey
* Implement basic Lua output support
Ref #1700
Basic but working serialization to Lua tables.
* Escape larger set of characters in Lua output
Started with a minimum of replacements, this should be more complete,
tho not all substitutions are strictly required in Lua.
* Print simple keys unquoted in Lua output
String keys that satisfy the requirements for variable names can be used
as keys without quotes in tables.
* Quote Lua keywords in table keys
Keywords are not valid as unquoted keys, thus must be quoted
* Make output of unquoted Lua table keys optional
Generally safer and simpler to not do it.
* Hook up settings for Lua output
* Allow special characters in Lua prefix and suffix
--lua-suffix='});^M' didn't work, so taking this approach instead
* Panic on unhandled YAML Kind in Lua encoder
* Handle YAML case varied booleans in Lua encoder
* Handle special-case numbers in Lua encoder
* Reject unhandled scalar Tags in Lua encoder
* Add note about how Lua nil is unsuitable as table key
Could add some context tracking in the future to allow rejecting nil in
a table key context.
* Return error instead of panic in Lua encoder
* Add initial test for Lua encoder
Boilerplate mostly copied from toml_test.go
* Additional Lua output tests
* Generate Lua encoder documentation
Mostly just for the boilerplate
* Convert octal for Lua output
Lua doesn't have the 0oNNN syntax for octal integers, only decimal and
hexadecimal, hence those can be passed trough as is while octal needs
special treatment.
* Implement indentation in in Lua output
* Respect string Style in Lua encoder
Lua has 'single', "double" and [[ long ]] strings.
* Expand Lua examples
* Output line comments in Lua output
* Implement Lua globals output mode
* fix typo in a comment
* implement shell output format
* fix a typo
* add two test cases, have source uses ascii only
* add integration tests and documentation
* add fixes after code revieew
This is to ensure solid parsing of complex data (with any binary
content except NUL chars) by separating the `yq` root collection
member's output with NUL char. As a safe-guard, an error will be cast
if trying to use NUL character with content that contains itself NUL
characters inside.
Allow snap users to process YAML documents under /media or /mnt paths after connecting to the `removable-media` snapd slot:
```
sudo snap connect yq :removable-media
```
Fixes#1618.
* Remove extra backtick
* Reword explanation of update
* Reword explanation of relative update
* Change "remaple" to "remain"
* Change "clovver" to "clobber"
* Reword explanation of update for comment operators
* Reword explanation of relative update for comment operators
* Change "array" to "expression"
* Change "the golangs" to "Golang's"
* Change "golangs" to "Golang's"
* Change "can durations" to "can add durations"
* Change "array scalars" to "arrays"
* Change "beit" to "be it"
* Fix typo in `eval` tip
* Fix typo in header for `has` operation
* Add space before pipe in `line` operator example
* Fix typos in explanation of deep array merges
* Change "is now used" to "is now used."
* Change "object," to "object."
* Changes "indexes" to "indices"
* Remove extraneous copied text from `..` article
* Reword explanation of `...` operator
* Change "your are" to "you are"
* Add link to `string` operator docs in `select` article
* Change "is a" to "parameter specifies" in `string` operators article
* Change "new line" to "newline"
* Change "golang regex" to "Golang's regex"
* Change "golang" to "Golang"
* Add period
* Remove comma in `subtract` article
* Remove duplicate number subtraction example
* Remove comma in `traverse` operator article
* Clarify use of brackets when `read`ing with special characters
* detect inputFormat from filename
* refactor and extract func InputFormatFromFilename
* detect inputFormat only when file is provided
* add test for automatic input format detection
* Refactor ordered_map into separate files
Separate json and xml, from the regular yaml.
Makes it possible to compile, without those...
* Refactor encoder and decoder creation
Use more consistent parameters vs globals
Return errors instead of calling panic()
* Allow build without json and xml support
* Build constraint not working for non linux
* Go Build Constraint: Fix Non-Linux Filename (#1494)
Correct the filename pkg/yqlib/chown_not_linux.go to escape the
default OS detection in filename format `*_GOOS[_test].go`.
Add an extra word after `linux` to resolve the issue.
pkg/yqlib/chown_not_linux_os.go
Signed-off-by: Bhargav Ravuri <vaguecoder0to.n@gmail.com>
Signed-off-by: Bhargav Ravuri <vaguecoder0to.n@gmail.com>
Signed-off-by: Bhargav Ravuri <vaguecoder0to.n@gmail.com>
Co-authored-by: Bhargav Ravuri <saibhargavravuri@gmail.com>
1. Escaping behavior changed as a result of set-output deprecation.
- Otherwise you'll encounter breaking changes for example "appVersion: "1.0.0"%0Aapplica..."
where %0A is escaped newline \n but shouldn't be escaped.
2. Delimeter is required for multiline strings.
- See here: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#multiline-strings
- Delimeter must be random. UUID is a solid choice.
I verified that /proc/sys/kernel/random/uuid does indeed exist on the yq image.
* adding support for --wrapScalar=false in properties encoder
* altering encoder and decoder tests somewhat
* adding .idea
* Revert "altering encoder and decoder tests somewhat"
This reverts commit e3655130e2.
* adding test scenario for encoding with wrapped scalars
4ec533b introduced a bug which causes the 'hidden' cli parameter
`__complete` to be ignored.
Add a check for this parameter, so that it can pass to `cobra`
Signed-off-by: Tom Whitwell <tom@whi.tw>
* Decoder Properties
* Added properties round trip test
* Fixed property decode for github actions
* Refactored XML test to use common functions
* Switched formatScenario parameter order for more readablity
* refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* build: update snapcraft `go-channel` to 1.17
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* chore: script to build the deb sources
now it includes the man files. This script makes only sense executing
on the machine having the gpg key needed to sign and push to ppa
* fix: dput dependency and proper configuration
This put everything concerning golangci following [this](https://golangci-lint.run/usage/configuration/).
This way, anyone running golangci with a command line or with it's favorite Ide will use the same configuration for the current project.
The current implementation of the deepMatch() has the exponential runtime.
Given the long enough input and the pattern with multiple wildcards
it takes a while if ever to complete which can potentially be used
maliciously to cause a denial of service (cpu and memory consumption).
E.g. running this in the root of this repository
time yq eval '.jobs.publishDocker.steps.[] | select (.run == "****outputs")' .github/workflows/release.yml
gives on my laptop
25.11s user 0.06s system 99% cpu 25.182 total
Whereas the updated implementation gives
0.01s user 0.01s system 36% cpu 0.049 total
There are numerous similar CVEs reported for glob evaluation in
different shells/ftp-servers/libraries.
The replacement implementation with the linear runtime is shamelessly taken
verbatim from the briliant article by Russ Cox https://research.swtch.com/glob
Installation by go will result in an error:
module declares its path as: xxx,
but was required as: xxx.
Go install with master branch will fix it.
fix: #676
json.Encoder and json.Marshal implicitly use HTMLEscape to convert
>, <, &, with \u003c, \u003e, \u0026. This behavior carries over
to yq, where chars will be escaped when outputting json but not when
outputting yaml.
This changeset disables this behavior via encoder.SetEscapeHTML(false).
Unfortunately there is no equivalent option for json.Marshal, so its
single usage has been replaced with an encoder (with escaping disabled).
The program generates a path for every leaf node in the
file-to-be-merged. It does not escape them if they contain a dot, as
the path-expressions document mentions is necessary.
Add in a test for this condition. Verified it fails without the fix.
- Move data_navigator, json_converter, merge, and path_parser to pkg/yqlib
- Extract yamlToString from yq to pkg/yqlib/yaml_converter
- Move utils_test to test/utils
When there is no match at a given path, numeric keys are assumed to be strings.
To create an array '+' must be used.
e.g: yq n thing[+].cat fred
will create an array under thing, whereas
yq n thing[0].cat fred
will create a map under thing, with a key '0'
It's recommended to check with the owner first (e.g. raise an issue) to discuss a new feature before developing, to ensure your hard efforts don't go to waste.
PRs to fix bugs and issues are almost always welcome :pray: please ensure you write tests as well.
The following types of PRs will _not_ be accepted:
- **Significant refactors** take a lot of time to understand and can have all sorts of unintended side effects. If you think there's a better way to do things (that requires significant changes) raise an issue for discussion first :)
- **Release pipeline PRs** are a security risk - it's too easy for a serious vulnerability to sneak in (either intended or not). If there is a new cool way of releasing things, raise an issue for discussion first - it will need to be gone over with a fine tooth comb.
- **Version bumps** are handled by dependabot, the bot will auto-raise PRs and they will be regularly merged in.
- **New release platforms** At this stage, yq is not going to maintain any other release platforms other than GitHub and Docker - that said, I'm more than happy to put in other community maintained methods in the README for visibility :heart:
# Development
## Initial Setup
1. Install [Golang](https://golang.org/) (version 1.24.0 or later)
2. Run `scripts/devtools.sh` to install required development tools:
- golangci-lint for code linting
- gosec for security analysis
3. Run `make [local] vendor` to install vendor dependencies
4. Run `make [local] test` to ensure you can run the existing tests
## Development Workflow
1.**Write unit tests first** - Changes will not be accepted without corresponding unit tests (see Testing section below)
2.**Make your code changes**
3.**Run tests and linting**: `make [local] test` (this runs formatting, linting, security checks, and tests)
4.**Create your PR** and get kudos! :)
## Make Commands
- Use `make [local] <command>` for local development (runs in Docker container)
- Use `make <command>` for CI/CD environments
- Common commands:
-`make [local] vendor` - Install dependencies
-`make [local] test` - Run all checks and tests
-`make [local] build` - Build the yq binary
-`make [local] format` - Format code
-`make [local] check` - Run linting and security checks
# Code Quality
## Linting and Formatting
The project uses strict linting rules defined in `.golangci.yml`. All code must pass:
- **Code formatting**: gofmt, goimports, gci
- **Linting**: revive, errorlint, gosec, misspell, and others
- **Security checks**: gosec security analysis
- **Spelling checks**: misspell detection
Run `make [local] check` to verify your code meets all quality standards.
## Code Style Guidelines
- Follow standard Go conventions
- Use meaningful variable names
- Add comments for public functions and complex logic
- Keep functions focused and reasonably sized
- Use the project's existing patterns and conventions
# Testing
## Test Structure
Tests in yq use the `expressionScenario` pattern. Each test scenario includes:
-`expression`: The yq expression to test
-`document`: Input YAML/JSON (optional)
-`expected`: Expected output
-`skipDoc`: Whether to skip documentation generation
## Writing Tests
1.**Find the appropriate test file** (e.g., `operator_add_test.go` for addition operations)
2.**Add your test scenario** to the `*OperatorScenarios` slice
3.**Run the specific test**: `go test -run TestAddOperatorScenarios` (replace with appropriate test name)
4.**Verify documentation generation** (see Documentation section)
## Test Examples
```go
varaddOperatorScenarios=[]expressionScenario{
{
skipDoc:true,
expression:`"foo" + "bar"`,
expected:[]string{
"D0, P[], (!!str)::foobar\n",
},
},
{
document:"apples: 3",
expression:`.apples + 3`,
expected:[]string{
"D0, P[apples], (!!int)::6\n",
},
},
}
```
## Running Tests
- **All tests**: `make [local] test`
- **Specific test**: `go test -run TestName`
- **With coverage**: `make [local] cover`
# Documentation
## Documentation Generation
The project uses a documentation system that combines static headers with dynamically generated content from tests.
### How It Works
1.**Static headers** are defined in `pkg/yqlib/doc/operators/headers/*.md`
2.**Dynamic content** is generated from test scenarios in `*_test.go` files
3.**Generated docs** are created in `pkg/yqlib/doc/*.md` by concatenating headers with test-generated content
4.**Documentation is synced** to the gitbook branch for the website
### Updating Operator Documentation
#### For Test-Generated Documentation
Most operator documentation is generated from tests. To update:
1.**Find the test file** (e.g., `operator_add_test.go`)
2.**Update test scenarios** - each `expressionScenario` with `skipDoc: false` becomes documentation
3.**Run the test** to regenerate docs:
```bash
cd pkg/yqlib
go test -run TestAddOperatorScenarios
```
4. **Verify the generated documentation** in `pkg/yqlib/doc/add.md`
5. **Create a PR** with your changes
#### For Header-Only Documentation
If documentation exists only in `headers/*.md` files:
1. **Update the header file directly** (e.g., `pkg/yqlib/doc/operators/headers/add.md`)
2. **Create a PR** with your changes
### Updating Static Documentation
For documentation not in the master branch:
1. **Check the gitbook branch** for additional pages
2. **Update the `*.md` files** directly
3. **Create a PR** to the gitbook branch
### Documentation Best Practices
- **Write clear, concise examples** in test scenarios
- **Use meaningful variable names** in examples
- **Include edge cases** and error conditions
- **Test your documentation changes** by running the specific test
A lightweight and portable command-line YAML, JSON, INI and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) (a popular JSON processor) like syntax but works with yaml files as well as json, kyaml, xml, ini, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
yq is written in Go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as Docker and Podman, all listed below.
See [recipes](https://mikefarah.gitbook.io/yq/recipes) for more examples and the [documentation](https://mikefarah.gitbook.io/yq/) for more information.
Take a look at the discussions for [common questions](https://github.com/mikefarah/yq/discussions/categories/q-a), and [cool ideas](https://github.com/mikefarah/yq/discussions/categories/show-and-tell)
## Install
On MacOS:
### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
### wget
Use wget to download pre-compiled binaries. Choose your platform and architecture:
**Available platforms:**`linux_amd64`, `linux_arm64`, `linux_arm`, `linux_386`, `darwin_amd64`, `darwin_arm64`, `windows_amd64`, `windows_386`, etc.
### MacOS / Linux via Homebrew:
Using [Homebrew](https://brew.sh/)
```
brew install yq
```
On Ubuntu and other Linux distros supporting `snap` packages:
### Linux via snap:
```
snap install yq
```
or, [Download latest binary](https://github.com/mikefarah/yq/releases/latest) or alternatively:
#### Snap notes
`yq` installs with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
```
go get github.com/mikefarah/yq
sudo cat /etc/myfile | yq '.a.path'
```
## Run with Docker
And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):
You'll need to pass the `-i --interactive` flag to Docker/Podman:
```bash
docker run -v ${PWD}:/workdir mikefarah/yq yq [flags] <command> FILE...
# Process piped data
docker run -i --rm mikefarah/yq '.this.thing' < myfile.yml
# Same with Podman
podman run -i --rm mikefarah/yq '.this.thing' < myfile.yml
```
Run commands interactively:
#### Run commands interactively:
```bash
docker run -it -v ${PWD}:/workdir mikefarah/yq sh
docker run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
```
```bash
podman run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
```
It can be useful to have a bash function to avoid typing the whole docker command:
```bash
yq(){
docker run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
```
```bash
yq(){
podman run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
```
#### Running as root:
`yq`'s container image no longer runs under root (https://github.com/mikefarah/yq/pull/860). If you'd like to install more things in the container image, or you're having permissions issues when attempting to read/write files you'll need to either:
```
docker run --user="root" -it --entrypoint sh mikefarah/yq
```
```
podman run --user="root" -it --entrypoint sh mikefarah/yq
```
Or, in your Dockerfile:
```
FROM mikefarah/yq
USER root
RUN apk add --no-cache bash
USER yq
```
#### Missing timezone data
By default, the alpine image yq uses does not include timezone data. If you'd like to use the `tz` operator, you'll need to include this data:
```
FROM mikefarah/yq
USER root
RUN apk add --no-cache tzdata
USER yq
```
#### Podman with SELinux
If you are using podman with SELinux, you will need to set the shared volume flag `:z` on the volume mount:
```
-v "${PWD}":/workdir:z
```
### GitHub Action
```
- name: Set foobar to cool
uses: mikefarah/yq@master
with:
cmd: yq -i '.foo.bar = "cool"' 'config.yml'
- name: Get an entry with a variable that might contain dots or spaces
Supported by @chillum (https://chocolatey.org/packages/yq)
Using [scoop](https://scoop.sh/)
```
scoop install main/yq
```
Using [winget](https://learn.microsoft.com/en-us/windows/package-manager/)
```
winget install --id MikeFarah.yq
```
### MacPorts:
Using [MacPorts](https://www.macports.org/)
```
sudo port selfupdate
sudo port install yq
```
Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot)
### Alpine Linux
Alpine Linux v3.20+ (and Edge):
```
apk add yq-go
```
Alpine Linux up to v3.19:
```
apk add yq
```
Supported by Tuan Hoang (https://pkgs.alpinelinux.org/packages?name=yq-go)
### Flox:
Flox can be used to install yq on Linux, MacOS, and Windows through WSL.
```
flox install yq
```
### MacOS / Linux via gah:
Using [gah](https://github.com/marverix/gah)
```
gah install yq
```
## Features
- [Detailed documentation with many examples](https://mikefarah.gitbook.io/yq/)
- Written in portable go, so you can download a lovely dependency free binary
-Deep read a yaml file with a given path
-Update a yaml file given a path
-Update a yaml file given a script file
-Update creates any missing entries in the path on the fly
-Create a yaml file given a deep path and value
-Create a yaml file given a script file
-Convert from json to yaml
-Convert from yaml to json
-Pipe data in by using '-'
-Merge multiple yaml files where each additional file sets values for missing or null value keys.
-Merge multiple yaml files with overwrite to support overriding previous values.
-Uses similar syntax as `jq` but works with YAML, INI, [JSON](https://mikefarah.gitbook.io/yq/usage/convert) and [XML](https://mikefarah.gitbook.io/yq/usage/xml) files
-Manipulate yaml [comments](https://mikefarah.gitbook.io/yq/operators/comment-operators), [styling](https://mikefarah.gitbook.io/yq/operators/style), [tags](https://mikefarah.gitbook.io/yq/operators/tag) and [anchors and aliases](https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators).
-[Update in place](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate#flags)
-[Complex expressions to select and update](https://mikefarah.gitbook.io/yq/operators/select#select-and-update-matching-values-in-map)
-Keeps yaml formatting and comments when updating (though there are issues with whitespace)
--csv-separator char CSV Separator character (default ,)
--debug-node-info debug node info
-e, --exit-status set exit status if there are no matches or null or false is returned
--expression string forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.
--from-file string Load expression from specified file.
-f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
--header-preprocess Slurp any header comments and separators before processing expression. (default true)
-h, --help help for yq
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the file in place of first file given.
-p, --input-format string [auto|a|yaml|y|json|j|kyaml|ky|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|hcl|h|lua|l|ini|i] parse format for input. (default "auto")
--lua-globals output keys as top-level global variables
-0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
-o, --output-format string [auto|a|yaml|y|json|j|kyaml|ky|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|hcl|h|shell|s|lua|l|ini|i] output format type. (default "auto")
-P, --prettyPrint pretty print, shorthand for '... style = ""'
--properties-array-brackets use [x] in array paths (e.g. for SpringBoot)
--properties-separator string separator to use between keys and values (default " = ")
--security-disable-env-ops Disable env related operations.
--security-disable-file-ops Disable file related operations (e.g. load)
--shell-key-separator string separator for shell variable key paths (default "_")
-s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter. The necessary directories will be created.
--split-exp-file string Use a file to specify the split-exp expression.
--string-interpolation Toggles strings interpolation of \(exp) (default true)
- **Examples**: [Recipes and examples](https://mikefarah.gitbook.io/yq/recipes)
## Known Issues / Missing Features
-`yq` attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)
- Powershell has its own...[opinions on quoting yq](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell)
- "yes", "no" were dropped as boolean values in the yaml 1.2 standard - which is the standard yq assumes.
See [tips and tricks](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks) for more common problems and solutions.
Please **do not** report security vulnerabilities through public GitHub issues.
Instead, use GitHub's private vulnerability reporting feature:
👉 https://github.com/mikefarah/yq/security
This allows vulnerabilities to be triaged and addressed confidentially before any public disclosure.
## Scope
### HTTP / TLS / Network vulnerabilities
yq is a command-line YAML/JSON/TOML processor that reads from files or standard input and writes to standard output. **yq does not include any HTTP or network libraries** and makes no network connections at runtime. CVEs related to HTTP, TLS, or networking are therefore **not applicable** to yq.
### Dependency version bumps
yq uses [Dependabot](https://docs.github.com/en/code-security/dependabot) to automatically raise pull requests for:
- Go module dependencies
- Go toolchain version
- Docker base images
Please **do not** raise pull requests or issues solely to bump dependency or Go versions — Dependabot handles this automatically and the maintainers merge those PRs regularly.
If your format has preferences/configuration options:
1. Create a preferences struct with your configuration fields
2. Update the encoder to accept preferences in its factory function
3. Update `format.go` to pass the configured preferences
4. Update `operator_encoder_decoder.go` if special indent handling is needed (see existing formats like JSON and YAML for the pattern)
This pattern is optional and only needed if your format has user-configurable options.
## Build Tags
Use build tags to allow optional compilation of formats:
- Add `//go:build !yq_no<format>` at the top of your encoder and decoder files
- Create a no-build version in `pkg/yqlib/no_<format>.go` that returns nil for encoder/decoder factories
This allows users to compile yq without certain formats using: `go build -tags yq_no<format>`
## Working with CandidateNode
The `CandidateNode` struct represents a YAML node with:
-`Kind`: The node type (ScalarNode, SequenceNode, MappingNode)
-`Tag`: The YAML tag (e.g., "!!str", "!!int", "!!map")
-`Value`: The scalar value (for ScalarNode only)
-`Content`: Child nodes (for SequenceNode and MappingNode)
Key methods:
-`node.guessTagFromCustomType()` - Infer the tag from Go type
-`node.AsList()` - Convert to a list for processing
-`node.CreateReplacement()` - Create a new replacement node
-`NewCandidate()` - Create a new CandidateNode
## Key Points
✅ **DO:**
- Implement only the `Encoder` and `Decoder` interfaces
- Register your format in `format.go` only
- Keep format-specific logic in your encoder/decoder files
- Use the candidate_node style attribute to store style information for round-trip. Ask if this needs to be updated with new styles.
- Use build tags for optional compilation
- Add comprehensive tests
- Run the specific encoder/decoder test (e.g. <format>_test.go) whenever you make ay changes to the encoder_<format> or decoder_<format>
- Handle errors gracefully
- Add the no build directive, like the xml encoder and decoder, that enables a minimal yq builds. e.g. `//go:build !yq_<format>`. Be sure to also update the build_small-yq.sh and build-tinygo-yq.sh to not include the new format.
❌ **DON'T:**
- Modify `candidate_node.go` to add format-specific logic
- Add format-specific fields to `CandidateNode`
- Create special cases in core navigation or evaluation logic
- Bypass the encoder/decoder interfaces
- Use candidate_node tag attribute for anything other than indicate the data type
## Examples
Refer to existing format implementations for patterns:
This guide explains how to add a new operator to yq. Operators are the core of yq's expression language and process `CandidateNode` objects without requiring modifications to `candidate_node.go` itself.
## Overview
Operators transform data by implementing a handler function that processes a `Context` containing `CandidateNode` objects. Each operator is:
1. Defined as an `operationType` in `operation.go`
2. Registered in the lexer in `lexer_participle.go`
3. Implemented in its own `operator_<type>.go` file
4. Tested in `operator_<type>_test.go`
5. Documented in `pkg/yqlib/doc/operators/headers/<type>.md`
## Architecture
### Key Files
-`pkg/yqlib/operation.go` - Defines `operationType` and operator registry
-`pkg/yqlib/lexer_participle.go` - Registers operators with their syntax patterns
rootCmd.PersistentFlags().BoolVarP(&outputToJSON,"tojson","j",false,"(deprecated) output as json. Set indent to 0 to print json in one line.")
err:=rootCmd.PersistentFlags().MarkDeprecated("tojson","please use -o=json instead")
iferr!=nil{
panic(err)
}
rootCmd.PersistentFlags().StringVarP(&outputFormat,"output-format","o","auto",fmt.Sprintf("[auto|a|%v] output format type.",yqlib.GetAvailableOutputFormatString()))
rootCmd.PersistentFlags().StringVarP(&inputFormat,"input-format","p","auto",fmt.Sprintf("[auto|a|%v] parse format for input.",yqlib.GetAvailableInputFormatString()))
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.AttributePrefix,"xml-attribute-prefix",yqlib.ConfiguredXMLPreferences.AttributePrefix,"prefix for xml attributes")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.ContentName,"xml-content-name",yqlib.ConfiguredXMLPreferences.ContentName,"name for xml content (if no attribute name is present).")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.StrictMode,"xml-strict-mode",yqlib.ConfiguredXMLPreferences.StrictMode,"enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.KeepNamespace,"xml-keep-namespace",yqlib.ConfiguredXMLPreferences.KeepNamespace,"enables keeping namespace after parsing attributes")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.UseRawToken,"xml-raw-token",yqlib.ConfiguredXMLPreferences.UseRawToken,"enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details.")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.ProcInstPrefix,"xml-proc-inst-prefix",yqlib.ConfiguredXMLPreferences.ProcInstPrefix,"prefix for xml processing instructions (e.g. <?xml version=\"1\"?>)")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.DirectiveName,"xml-directive-name",yqlib.ConfiguredXMLPreferences.DirectiveName,"name for xml directives (e.g. <!DOCTYPE thing cat>)")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.SkipProcInst,"xml-skip-proc-inst",yqlib.ConfiguredXMLPreferences.SkipProcInst,"skip over process instructions (e.g. <?xml version=\"1\"?>)")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.SkipDirectives,"xml-skip-directives",yqlib.ConfiguredXMLPreferences.SkipDirectives,"skip over directives (e.g. <!DOCTYPE thing cat>)")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredLuaPreferences.Globals,"lua-globals",yqlib.ConfiguredLuaPreferences.Globals,"output keys as top-level global variables")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator,"properties-separator",yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator,"separator to use between keys and values")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets,"properties-array-brackets",yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets,"use [x] in array paths (e.g. for SpringBoot)")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredShellVariablesPreferences.KeySeparator,"shell-key-separator",yqlib.ConfiguredShellVariablesPreferences.KeySeparator,"separator for shell variable key paths")
rootCmd.PersistentFlags().BoolVar(&yqlib.StringInterpolationEnabled,"string-interpolation",yqlib.StringInterpolationEnabled,"Toggles strings interpolation of \\(exp)")
rootCmd.PersistentFlags().BoolVarP(&nullInput,"null-input","n",false,"Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.")
rootCmd.Flags().BoolVarP(&version,"version","V",false,"Print version information and quit")
rootCmd.PersistentFlags().BoolVarP(&writeInplace,"inplace","i",false,"update the file in place of first file given.")
rootCmd.PersistentFlags().VarP(unwrapScalarFlag,"unwrapScalar","r","unwrap scalar, print the value with no quotes, colours or comments. Defaults to true for yaml")
rootCmd.PersistentFlags().BoolVarP(&nulSepOutput,"nul-output","0",false,"Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.")
rootCmd.PersistentFlags().BoolVarP(&prettyPrint,"prettyPrint","P",false,"pretty print, shorthand for '... style = \"\"'")
rootCmd.PersistentFlags().BoolVarP(&exitStatus,"exit-status","e",false,"set exit status if there are no matches or null or false is returned")
rootCmd.PersistentFlags().BoolVarP(&forceColor,"colors","C",false,"force print with colors")
rootCmd.PersistentFlags().BoolVarP(&forceNoColor,"no-colors","M",forceNoColor,"force print with no colors")
rootCmd.PersistentFlags().StringVarP(&frontMatter,"front-matter","f","","(extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact")
rootCmd.PersistentFlags().StringVarP(&forceExpression,"expression","","","forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.")
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredYamlPreferences.LeadingContentPreProcessing,"header-preprocess","",true,"Slurp any header comments and separators before processing expression.")
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredYamlPreferences.FixMergeAnchorToSpec,"yaml-fix-merge-anchor-to-spec","",false,"Fix merge anchor to match YAML spec. Will default to true in late 2025")
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredYamlPreferences.CompactSequenceIndent,"yaml-compact-seq-indent","c",false,"Use compact sequence indentation where '- ' is considered part of the indentation.")
rootCmd.PersistentFlags().StringVarP(&splitFileExp,"split-exp","s","","print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter. The necessary directories will be created.")
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredSecurityPreferences.DisableEnvOps,"security-disable-env-ops","",false,"Disable env related operations.")
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredSecurityPreferences.DisableFileOps,"security-disable-file-ops","",false,"Disable file related operations (e.g. load)")
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredSecurityPreferences.EnableSystemOps,"security-enable-system-operator","",false,"Enable system operator to allow execution of external commands.")
yqlib.GetLogger().Warningf("yq default output is now 'auto' (based on the filename extension). Normally yq would output '%v', but for backwards compatibility 'yaml' has been set. Please use -oy to specify yaml, or drop the -p flag.",outputFormat)
!function(e,r){"function"==typeofdefine&&define.amd?define(r):"object"==typeofexports?module.exports=r():r()(e.lunr)}(this,function(){returnfunction(e){if(void0===e)thrownewError("Lunr is not present. Please include / require Lunr before this script.");if(void0===e.stemmerSupport)thrownewError("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=function(){varr=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=newfunction(){functione(){vare,r=l.limit-l.cursor;l.cursor>=t&&(e=l.limit_backward,l.limit_backward=t,l.ket=l.cursor,l.find_among_b(a,4)?(l.bra=l.cursor,l.limit_backward=e,l.cursor=l.limit-r,l.cursor>l.limit_backward&&(l.cursor--,l.bra=l.cursor,l.slice_del())):l.limit_backward=e)}varn,t,s,o=[newr("hed",-1,1),newr("ethed",0,1),newr("ered",-1,1),newr("e",-1,1),newr("erede",3,1),newr("ende",3,1),newr("erende",5,1),newr("ene",3,1),newr("erne",3,1),newr("ere",3,1),newr("en",-1,1),newr("heden",10,1),newr("eren",10,1),newr("er",-1,1),newr("heder",13,1),newr("erer",13,1),newr("s",-1,2),newr("heds",16,1),newr("es",16,1),newr("endes",18,1),newr("erendes",19,1),newr("enes",18,1),newr("ernes",18,1),newr("eres",18,1),newr("ens",16,1),newr("hedens",24,1),newr("erens",24,1),newr("ers",16,1),newr("ets",16,1),newr("erets",28,1),newr("et",-1,1),newr("eret",30,1)],a=[newr("gd",-1,-1),newr("dt",-1,-1),newr("gt",-1,-1),newr("kt",-1,-1)],d=[newr("ig",-1,1),newr("lig",0,1),newr("elig",1,1),newr("els",-1,1),newr("løst",-1,2)],u=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],c=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],l=newi;this.setCurrent=function(e){l.setCurrent(e)},this.getCurrent=function(){returnl.getCurrent()},this.stem=function(){varr=l.cursor;returnfunction(){vare,r=l.cursor+3;if(t=l.limit,0<=r&&r<=l.limit){for(n=r;;){if(e=l.cursor,l.in_grouping(u,97,248)){l.cursor=e;break}if(l.cursor=e,e>=l.limit)return;l.cursor++}for(;!l.out_grouping(u,97,248);){if(l.cursor>=l.limit)return;l.cursor++}(t=l.cursor)<n&&(t=n)}}(),l.limit_backward=r,l.cursor=l.limit,function(){vare,r;if(l.cursor>=t&&(r=l.limit_backward,l.limit_backward=t,l.ket=l.cursor,e=l.find_among_b(o,32),l.limit_backward=r,e))switch(l.bra=l.cursor,e){case1:l.slice_del();break;case2:l.in_grouping_b(c,97,229)&&l.slice_del()}}(),l.cursor=l.limit,e(),l.cursor=l.limit,function(){varr,i,n,s=l.limit-l.cursor;if(l.ket=l.cursor,l.eq_s_b(2,"st")&&(l.bra=l.cursor,l.eq_s_b(2,"ig")&&l.slice_del()),l.cursor=l.limit-s,l.cursor>=t&&(i=l.limit_backward,l.limit_backward=t,l.ket=l.cursor,r=l.find_among_b(d,5),l.limit_backward=i,r))switch(l.bra=l.cursor,r){case1:l.slice_del(),n=l.limit-l.cursor,e(),l.cursor=l.limit-n;break;case2:l.slice_from("løs")}}(),l.cursor=l.limit,function(){vare;l.cursor>=t&&(e=l.limit_backward,l.limit_backward=t,l.ket=l.cursor,l.out_grouping_b(u,97,248)?(l.bra=l.cursor,s=l.slice_to(s),l.limit_backward=e,l.eq_v_b(s)&&l.slice_del()):l.limit_backward=e)}(),!0}};returnfunction(e){return"function"==typeofe.update?e.update(function(e){returnn.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}}(),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}});
!function(e,r){"function"==typeofdefine&&define.amd?define(r):"object"==typeofexports?module.exports=r():r()(e.lunr)}(this,function(){returnfunction(e){if(void0===e)thrownewError("Lunr is not present. Please include / require Lunr before this script.");if(void0===e.stemmerSupport)thrownewError("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.no=function(){this.pipeline.reset(),this.pipeline.add(e.no.trimmer,e.no.stopWordFilter,e.no.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.no.stemmer))},e.no.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.no.trimmer=e.trimmerSupport.generateTrimmer(e.no.wordCharacters),e.Pipeline.registerFunction(e.no.trimmer,"trimmer-no"),e.no.stemmer=function(){varr=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=newfunction(){vare,i,t=[newr("a",-1,1),newr("e",-1,1),newr("ede",1,1),newr("ande",1,1),newr("ende",1,1),newr("ane",1,1),newr("ene",1,1),newr("hetene",6,1),newr("erte",1,3),newr("en",-1,1),newr("heten",9,1),newr("ar",-1,1),newr("er",-1,1),newr("heter",12,1),newr("s",-1,2),newr("as",14,1),newr("es",14,1),newr("edes",16,1),newr("endes",16,1),newr("enes",16,1),newr("hetenes",19,1),newr("ens",14,1),newr("hetens",21,1),newr("ers",14,1),newr("ets",14,1),newr("et",-1,1),newr("het",25,1),newr("ert",-1,3),newr("ast",-1,1)],o=[newr("dt",-1,-1),newr("vt",-1,-1)],s=[newr("leg",-1,1),newr("eleg",0,1),newr("ig",-1,1),newr("eig",2,1),newr("lig",2,1),newr("elig",4,1),newr("els",-1,1),newr("lov",-1,1),newr("elov",7,1),newr("slov",7,1),newr("hetslov",9,1)],a=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],m=[119,125,149,1],u=newn;this.setCurrent=function(e){u.setCurrent(e)},this.getCurrent=function(){returnu.getCurrent()},this.stem=function(){varr=u.cursor;returnfunction(){varr,n=u.cursor+3;if(i=u.limit,0<=n||n<=u.limit){for(e=n;;){if(r=u.cursor,u.in_grouping(a,97,248)){u.cursor=r;break}if(r>=u.limit)return;u.cursor=r+1}for(;!u.out_grouping(a,97,248);){if(u.cursor>=u.limit)return;u.cursor++}(i=u.cursor)<e&&(i=e)}}(),u.limit_backward=r,u.cursor=u.limit,function(){vare,r,n;if(u.cursor>=i&&(r=u.limit_backward,u.limit_backward=i,u.ket=u.cursor,e=u.find_among_b(t,29),u.limit_backward=r,e))switch(u.bra=u.cursor,e){case1:u.slice_del();break;case2:n=u.limit-u.cursor,u.in_grouping_b(m,98,122)?u.slice_del():(u.cursor=u.limit-n,u.eq_s_b(1,"k")&&u.out_grouping_b(a,97,248)&&u.slice_del());break;case3:u.slice_from("er")}}(),u.cursor=u.limit,function(){vare,r=u.limit-u.cursor;u.cursor>=i&&(e=u.limit_backward,u.limit_backward=i,u.ket=u.cursor,u.find_among_b(o,2)?(u.bra=u.cursor,u.limit_backward=e,u.cursor=u.limit-r,u.cursor>u.limit_backward&&(u.cursor--,u.bra=u.cursor,u.slice_del())):u.limit_backward=e)}(),u.cursor=u.limit,function(){vare,r;u.cursor>=i&&(r=u.limit_backward,u.limit_backward=i,u.ket=u.cursor,(e=u.find_among_b(s,11))?(u.bra=u.cursor,u.limit_backward=r,1==e&&u.slice_del()):u.limit_backward=r)}(),!0}};returnfunction(e){return"function"==typeofe.update?e.update(function(e){returni.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}}(),e.Pipeline.registerFunction(e.no.stemmer,"stemmer-no"),e.no.stopWordFilter=e.generateStopWordFilter("alle at av bare begge ble blei bli blir blitt både båe da de deg dei deim deira deires dem den denne der dere deres det dette di din disse ditt du dykk dykkar då eg ein eit eitt eller elles en enn er et ett etter for fordi fra før ha hadde han hans har hennar henne hennes her hjå ho hoe honom hoss hossen hun hva hvem hver hvilke hvilken hvis hvor hvordan hvorfor i ikke ikkje ikkje ingen ingi inkje inn inni ja jeg kan kom korleis korso kun kunne kva kvar kvarhelst kven kvi kvifor man mange me med medan meg meget mellom men mi min mine mitt mot mykje ned no noe noen noka noko nokon nokor nokre nå når og også om opp oss over på samme seg selv si si sia sidan siden sin sine sitt sjøl skal skulle slik so som som somme somt så sånn til um upp ut uten var vart varte ved vere verte vi vil ville vore vors vort vår være være vært å".split(" ")),e.Pipeline.registerFunction(e.no.stopWordFilter,"stopWordFilter-no")}});
!function(e,r){"function"==typeofdefine&&define.amd?define(r):"object"==typeofexports?module.exports=r():r()(e.lunr)}(this,function(){returnfunction(e){if(void0===e)thrownewError("Lunr is not present. Please include / require Lunr before this script.");if(void0===e.stemmerSupport)thrownewError("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sv=function(){this.pipeline.reset(),this.pipeline.add(e.sv.trimmer,e.sv.stopWordFilter,e.sv.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sv.stemmer))},e.sv.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.sv.trimmer=e.trimmerSupport.generateTrimmer(e.sv.wordCharacters),e.Pipeline.registerFunction(e.sv.trimmer,"trimmer-sv"),e.sv.stemmer=function(){varr=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,t=newfunction(){vare,t,i=[newr("a",-1,1),newr("arna",0,1),newr("erna",0,1),newr("heterna",2,1),newr("orna",0,1),newr("ad",-1,1),newr("e",-1,1),newr("ade",6,1),newr("ande",6,1),newr("arne",6,1),newr("are",6,1),newr("aste",6,1),newr("en",-1,1),newr("anden",12,1),newr("aren",12,1),newr("heten",12,1),newr("ern",-1,1),newr("ar",-1,1),newr("er",-1,1),newr("heter",18,1),newr("or",-1,1),newr("s",-1,2),newr("as",21,1),newr("arnas",22,1),newr("ernas",22,1),newr("ornas",22,1),newr("es",21,1),newr("ades",26,1),newr("andes",26,1),newr("ens",21,1),newr("arens",29,1),newr("hetens",29,1),newr("erns",21,1),newr("at",-1,1),newr("andet",-1,1),newr("het",-1,1),newr("ast",-1,1)],s=[newr("dd",-1,-1),newr("gd",-1,-1),newr("nn",-1,-1),newr("dt",-1,-1),newr("gt",-1,-1),newr("kt",-1,-1),newr("tt",-1,-1)],a=[newr("ig",-1,1),newr("lig",0,1),newr("els",-1,1),newr("fullt",-1,3),newr("löst",-1,2)],o=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,24,0,32],u=[119,127,149],c=newn;this.setCurrent=function(e){c.setCurrent(e)},this.getCurrent=function(){returnc.getCurrent()},this.stem=function(){varr=c.cursor;returnfunction(){varr,n=c.cursor+3;if(t=c.limit,0<=n||n<=c.limit){for(e=n;;){if(r=c.cursor,c.in_grouping(o,97,246)){c.cursor=r;break}if(c.cursor=r,c.cursor>=c.limit)return;c.cursor++}for(;!c.out_grouping(o,97,246);){if(c.cursor>=c.limit)return;c.cursor++}(t=c.cursor)<e&&(t=e)}}(),c.limit_backward=r,c.cursor=c.limit,function(){vare,r=c.limit_backward;if(c.cursor>=t&&(c.limit_backward=t,c.cursor=c.limit,c.ket=c.cursor,e=c.find_among_b(i,37),c.limit_backward=r,e))switch(c.bra=c.cursor,e){case1:c.slice_del();break;case2:c.in_grouping_b(u,98,121)&&c.slice_del()}}(),c.cursor=c.limit,function(){vare=c.limit_backward;c.cursor>=t&&(c.limit_backward=t,c.cursor=c.limit,c.find_among_b(s,7)&&(c.cursor=c.limit,c.ket=c.cursor,c.cursor>c.limit_backward&&(c.bra=--c.cursor,c.slice_del())),c.limit_backward=e)}(),c.cursor=c.limit,function(){vare,r;if(c.cursor>=t){if(r=c.limit_backward,c.limit_backward=t,c.cursor=c.limit,c.ket=c.cursor,e=c.find_among_b(a,5))switch(c.bra=c.cursor,e){case1:c.slice_del();break;case2:c.slice_from("lös");break;case3:c.slice_from("full")}c.limit_backward=r}}(),!0}};returnfunction(e){return"function"==typeofe.update?e.update(function(e){returnt.setCurrent(e),t.stem(),t.getCurrent()}):(t.setCurrent(e),t.stem(),t.getCurrent())}}(),e.Pipeline.registerFunction(e.sv.stemmer,"stemmer-sv"),e.sv.stopWordFilter=e.generateStopWordFilter("alla allt att av blev bli blir blivit de dem den denna deras dess dessa det detta dig din dina ditt du där då efter ej eller en er era ert ett från för ha hade han hans har henne hennes hon honom hur här i icke ingen inom inte jag ju kan kunde man med mellan men mig min mina mitt mot mycket ni nu när någon något några och om oss på samma sedan sig sin sina sitta själv skulle som så sådan sådana sådant till under upp ut utan vad var vara varför varit varje vars vart vem vi vid vilka vilkas vilken vilket vår våra vårt än är åt över".split(" ")),e.Pipeline.registerFunction(e.sv.stopWordFilter,"stopWordFilter-sv")}});
<labelclass="md-nav__title"for="toc">Table of contents</label>
<ulclass="md-nav__list"data-md-scrollfix>
<liclass="md-nav__item">
<ahref="#creating-a-simple-yaml-file"title="Creating a simple yaml file"class="md-nav__link">
Creating a simple yaml file
</a>
</li>
<liclass="md-nav__item">
<ahref="#creating-using-a-create-script"title="Creating using a create script"class="md-nav__link">
Creating using a create script
</a>
</li>
<liclass="md-nav__item">
<ahref="#keys-with-dots"title="Keys with dots"class="md-nav__link">
Keys with dots
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<divclass="md-content">
<articleclass="md-content__inner md-typeset">
<ahref="https://github.com/mikefarah/yq/edit/master/docs/create.md"title="Edit this page"class="md-icon md-content__icon"></a>
<h1>Create</h1>
<p>Yaml files can be created using the 'new' command. This works in the same way as the write command, but you don't pass in an existing Yaml file.</p>
<pre><code>yq n <path><new value>
</code></pre>
<h3id="creating-a-simple-yaml-file">Creating a simple yaml file<aclass="headerlink"href="#creating-a-simple-yaml-file"title="Permanent link">¶</a></h3>
<pre><codeclass="bash">yq n b.c cat
</code></pre>
<p>will output:</p>
<pre><codeclass="yaml">b:
c: cat
</code></pre>
<h3id="creating-using-a-create-script">Creating using a create script<aclass="headerlink"href="#creating-using-a-create-script"title="Permanent link">¶</a></h3>
<p>Create scripts follow the same format as the update scripts.</p>
<p>Given a script create_instructions.yaml of:</p>
<pre><codeclass="yaml">b.c: 3
b.e[0].name: Howdy Partner
</code></pre>
<p>then</p>
<pre><codeclass="bash">yq n -s create_instructions.yaml
</code></pre>
<p>will output:</p>
<pre><codeclass="yaml">b:
c: 3
e:
- name: Howdy Partner
</code></pre>
<p>You can also pipe the instructions in:</p>
<pre><codeclass="bash">cat create_instructions.yaml | yq n -s -
</code></pre>
<h3id="keys-with-dots">Keys with dots<aclass="headerlink"href="#keys-with-dots"title="Permanent link">¶</a></h3>
<p>When specifying a key that has a dot use key lookup indicator.</p>
<pre><codeclass="yaml">b:
foo.bar: 7
</code></pre>
<pre><codeclass="bash">yaml r sample.yaml 'b[foo.bar]'
</code></pre>
<pre><codeclass="bash">yaml w sample.yaml 'b[foo.bar]' 9
</code></pre>
<p>Any valid yaml key can be specified as part of a key lookup.</p>
<p>Note that the path is in quotes to avoid the square brackets being interpreted by your shell.</p>
<pre><codeclass="bash">yq m -x data1.yaml data2.yaml
</code></pre>
<p>will output:</p>
<pre><codeclass="yaml">a: other
b: [1, 2]
c:
test: 1
</code></pre>
<h3id="overwrite-values-with-arrays">Overwrite values with arrays<aclass="headerlink"href="#overwrite-values-with-arrays"title="Permanent link">¶</a></h3>
<p>Given a data1.yaml file of:</p>
<pre><codeclass="yaml">a: simple
b: [1, 2]
</code></pre>
<p>and data3.yaml file of:</p>
<pre><codeclass="yaml">b: [2, 3, 4]
c:
test: 2
other: true
d: false
</code></pre>
<p>then</p>
<pre><codeclass="bash">yq m -x data1.yaml data3.yaml
</code></pre>
<p>will output:</p>
<pre><codeclass="yaml">a: simple
b: [2, 3, 4]
c:
test: 2
other: true
d: false
</code></pre>
<p>Notice that 'b' does not result in the merging of the values within an array. The underlying library does not
currently handle merging values within an array.</p>
"text":"yq\n\u00b6\n\n\nyq is a lightweight and portable command-line YAML processor\n\n\nThe aim of the project is to be the \njq\n or sed of yaml files.\n\n\nInstall\n\u00b6\n\n\nOn MacOS:\n\n\nbrew install yq\n\n\n\n\nOn Ubuntu and other Linux distros supporting \nsnap\n packages:\n\n\nsnap install yq\n\n\n\n\nor, \nDownload latest binary\n or alternatively:\n\n\ngo get github.com/mikefarah/yq\n\n\n\n\nView on GitHub",
"title":"Install"
},
{
"location":"/#yq",
"text":"yq is a lightweight and portable command-line YAML processor The aim of the project is to be the jq or sed of yaml files.",
"title":"yq"
},
{
"location":"/#install",
"text":"On MacOS: brew install yq On Ubuntu and other Linux distros supporting snap packages: snap install yq or, Download latest binary or alternatively: go get github.com/mikefarah/yq View on GitHub",
"title":"Install"
},
{
"location":"/read/",
"text":"yq r <yaml_file|json_file> <path>\n\n\n\n\nThis command can take a json file as input too, and will output yaml unless specified to export as json (-j)\n\n\nBasic\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n\n\n\n\nthen\n\n\nyq r sample.yaml b.c\n\n\n\n\nwill output the value of '2'.\n\n\nFrom Stdin\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\ncat sample.yaml | yq r - b.c\n\n\n\n\nwill output the value of '2'.\n\n\nSplat\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\n---\nbob:\n item1:\n cats: bananas\n item2:\n cats: apples\n\n\n\n\nthen\n\n\nyq r sample.yaml bob.*.cats\n\n\n\n\nwill output\n\n\n- bananas\n- apples\n\n\n\n\nArrays\n\u00b6\n\n\nYou can give an index to access a specific element:\ne.g.: given a sample file of\n\n\nb:\n e:\n - name: fred\n value: 3\n - name: sam\n value: 4\n\n\n\n\nthen\n\n\nyq r sample.yaml 'b.e[1].name'\n\n\n\n\nwill output 'sam'\n\n\nNote that the path is in quotes to avoid the square brackets being interpreted by your shell.\n\n\nArray Splat\n\u00b6\n\n\ne.g.: given a sample file of\n\n\nb:\n e:\n - name: fred\n value: 3\n - name: sam\n value: 4\n\n\n\n\nthen\n\n\nyq r sample.yaml 'b.e[*].name'\n\n\n\n\nwill output:\n\n\n- fred\n- sam\n\n\n\n\nNote that the path is in quotes to avoid the square brackets being interpreted by your shell.\n\n\nKeys with dots\n\u00b6\n\n\nWhen specifying a key that has a dot use key lookup indicator.\n\n\nb:\n foo.bar: 7\n\n\n\n\nyaml r sample.yaml 'b[foo.bar]'\n\n\n\n\nyaml w sample.yaml 'b[foo.bar]' 9\n\n\n\n\nAny valid yaml key can be specified as part of a key lookup.\n\n\nNote that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Read"
},
{
"location":"/read/#basic",
"text":"Given a sample.yaml file of: b:\n c: 2 then yq r sample.yaml b.c will output the value of '2'.",
"title":"Basic"
},
{
"location":"/read/#from-stdin",
"text":"Given a sample.yaml file of: cat sample.yaml | yq r - b.c will output the value of '2'.",
"title":"From Stdin"
},
{
"location":"/read/#splat",
"text":"Given a sample.yaml file of: ---\nbob:\n item1:\n cats: bananas\n item2:\n cats: apples then yq r sample.yaml bob.*.cats will output - bananas\n- apples",
"title":"Splat"
},
{
"location":"/read/#arrays",
"text":"You can give an index to access a specific element:\ne.g.: given a sample file of b:\n e:\n - name: fred\n value: 3\n - name: sam\n value: 4 then yq r sample.yaml 'b.e[1].name' will output 'sam' Note that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Arrays"
},
{
"location":"/read/#array-splat",
"text":"e.g.: given a sample file of b:\n e:\n - name: fred\n value: 3\n - name: sam\n value: 4 then yq r sample.yaml 'b.e[*].name' will output: - fred\n- sam Note that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Array Splat"
},
{
"location":"/read/#keys-with-dots",
"text":"When specifying a key that has a dot use key lookup indicator. b:\n foo.bar: 7 yaml r sample.yaml 'b[foo.bar]' yaml w sample.yaml 'b[foo.bar]' 9 Any valid yaml key can be specified as part of a key lookup. Note that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Keys with dots"
},
{
"location":"/write/",
"text":"yq w <yaml_file|json_file> <path> <new value>\n\n\n\n\nThis command can take a json file as input too, and will output yaml unless specified to export as json (-j)\n\n\nTo Stdout\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n\n\n\n\nthen\n\n\nyq w sample.yaml b.c cat\n\n\n\n\nwill output:\n\n\nb:\n c: cat\n\n\n\n\nFrom STDIN\n\u00b6\n\n\ncat sample.yaml | yq w - b.c blah\n\n\n\n\nAdding new fields\n\u00b6\n\n\nAny missing fields in the path will be created on the fly.\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n\n\n\n\nthen\n\n\nyq w sample.yaml b.d[0] \"new thing\"\n\n\n\n\nwill output:\n\n\nb:\n c: cat\n d:\n - new thing\n\n\n\n\nAppending value to an array field\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n d:\n - new thing\n - foo thing\n\n\n\n\nthen\n\n\nyq w sample.yaml \"b.d[+]\" \"bar thing\"\n\n\n\n\nwill output:\n\n\nb:\n c: cat\n d:\n - new thing\n - foo thing\n - bar thing\n\n\n\n\nNote that the path is in quotes to avoid the square brackets being interpreted by your shell.\n\n\nUpdating files in-place\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n\n\n\n\nthen\n\n\nyq w -i sample.yaml b.c cat\n\n\n\n\nwill update the sample.yaml file so that the value of 'c' is cat.\n\n\nUpdating multiple values with a script\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n e:\n - name: Billy Bob\n\n\n\n\nand a script update_instructions.yaml of:\n\n\nb.c: 3\nb.e[0].name: Howdy Partner\n\n\n\n\nthen\n\n\nyq w -s update_instructions.yaml sample.yaml\n\n\n\n\nwill output:\n\n\nb:\n c: 3\n e:\n - name: Howdy Partner\n\n\n\n\nAnd, of course, you can pipe the instructions in using '-':\n\n\ncat update_instructions.yaml | yq w -s - sample.yaml\n\n\n\n\nValues starting with a hyphen (or dash)\n\u00b6\n\n\nThe flag terminator needs to be used to stop the app from attempting to parse the subsequent arguments as flags:\n\n\nyq w -- my.path -3\n\n\n\n\nwill output\n\n\nmy:\n path: -3\n\n\n\n\nKeys with dots\n\u00b6\n\n\nWhen specifying a key that has a dot use key lookup indicator.\n\n\nb:\n foo.bar: 7\n\n\n\n\nyaml r sample.yaml 'b[foo.bar]'\n\n\n\n\nyaml w sample.yaml 'b[foo.bar]' 9\n\n\n\n\nAny valid yaml key can be specified as part of a key lookup.\n\n\nNote that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Write/Update"
},
{
"location":"/write/#to-stdout",
"text":"Given a sample.yaml file of: b:\n c: 2 then yq w sample.yaml b.c cat will output: b:\n c: cat",
"title":"To Stdout"
},
{
"location":"/write/#from-stdin",
"text":"cat sample.yaml | yq w - b.c blah",
"title":"From STDIN"
},
{
"location":"/write/#adding-new-fields",
"text":"Any missing fields in the path will be created on the fly. Given a sample.yaml file of: b:\n c: 2 then yq w sample.yaml b.d[0] \"new thing\" will output: b:\n c: cat\n d:\n - new thing",
"text":"Given a sample.yaml file of: b:\n c: 2\n d:\n - new thing\n - foo thing then yq w sample.yaml \"b.d[+]\" \"bar thing\" will output: b:\n c: cat\n d:\n - new thing\n - foo thing\n - bar thing Note that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Appending value to an array field"
},
{
"location":"/write/#updating-files-in-place",
"text":"Given a sample.yaml file of: b:\n c: 2 then yq w -i sample.yaml b.c cat will update the sample.yaml file so that the value of 'c' is cat.",
"text":"Given a sample.yaml file of: b:\n c: 2\n e:\n - name: Billy Bob and a script update_instructions.yaml of: b.c: 3\nb.e[0].name: Howdy Partner then yq w -s update_instructions.yaml sample.yaml will output: b:\n c: 3\n e:\n - name: Howdy Partner And, of course, you can pipe the instructions in using '-': cat update_instructions.yaml | yq w -s - sample.yaml",
"text":"The flag terminator needs to be used to stop the app from attempting to parse the subsequent arguments as flags: yq w -- my.path -3 will output my:\n path: -3",
"title":"Values starting with a hyphen (or dash)"
},
{
"location":"/write/#keys-with-dots",
"text":"When specifying a key that has a dot use key lookup indicator. b:\n foo.bar: 7 yaml r sample.yaml 'b[foo.bar]' yaml w sample.yaml 'b[foo.bar]' 9 Any valid yaml key can be specified as part of a key lookup. Note that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Keys with dots"
},
{
"location":"/delete/",
"text":"yq d <yaml_file|json_file> <path_to_delete>\n\n\n\n\nThis command can take a json file as input too, and will output yaml unless specified to export as json (-j)\n\n\nTo Stdout\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n apples: green\n\n\n\n\nthen\n\n\nyq d sample.yaml b.c\n\n\n\n\nwill output:\n\n\nb:\n apples: green\n\n\n\n\nFrom STDIN\n\u00b6\n\n\ncat sample.yaml | yq d - b.c\n\n\n\n\nDeleting array elements\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: \n - 1\n - 2\n - 3\n\n\n\n\nthen\n\n\nyq d sample.yaml 'b.c[1]'\n\n\n\n\nwill output:\n\n\nb:\n c:\n - 1\n - 3\n\n\n\n\nDeleting nodes in-place\n\u00b6\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n apples: green\n\n\n\n\nthen\n\n\nyq d -i sample.yaml b.c\n\n\n\n\nwill update the sample.yaml file so that the 'c' node is deleted\n\n\nKeys with dots\n\u00b6\n\n\nWhen specifying a key that has a dot use key lookup indicator.\n\n\nb:\n foo.bar: 7\n\n\n\n\nyaml r sample.yaml 'b[foo.bar]'\n\n\n\n\nyaml w sample.yaml 'b[foo.bar]' 9\n\n\n\n\nAny valid yaml key can be specified as part of a key lookup.\n\n\nNote that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Delete"
},
{
"location":"/delete/#to-stdout",
"text":"Given a sample.yaml file of: b:\n c: 2\n apples: green then yq d sample.yaml b.c will output: b:\n apples: green",
"title":"To Stdout"
},
{
"location":"/delete/#from-stdin",
"text":"cat sample.yaml | yq d - b.c",
"title":"From STDIN"
},
{
"location":"/delete/#deleting-array-elements",
"text":"Given a sample.yaml file of: b:\n c: \n - 1\n - 2\n - 3 then yq d sample.yaml 'b.c[1]' will output: b:\n c:\n - 1\n - 3",
"title":"Deleting array elements"
},
{
"location":"/delete/#deleting-nodes-in-place",
"text":"Given a sample.yaml file of: b:\n c: 2\n apples: green then yq d -i sample.yaml b.c will update the sample.yaml file so that the 'c' node is deleted",
"title":"Deleting nodes in-place"
},
{
"location":"/delete/#keys-with-dots",
"text":"When specifying a key that has a dot use key lookup indicator. b:\n foo.bar: 7 yaml r sample.yaml 'b[foo.bar]' yaml w sample.yaml 'b[foo.bar]' 9 Any valid yaml key can be specified as part of a key lookup. Note that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Keys with dots"
},
{
"location":"/create/",
"text":"Yaml files can be created using the 'new' command. This works in the same way as the write command, but you don't pass in an existing Yaml file.\n\n\nyq n <path> <new value>\n\n\n\n\nCreating a simple yaml file\n\u00b6\n\n\nyq n b.c cat\n\n\n\n\nwill output:\n\n\nb:\n c: cat\n\n\n\n\nCreating using a create script\n\u00b6\n\n\nCreate scripts follow the same format as the update scripts.\n\n\nGiven a script create_instructions.yaml of:\n\n\nb.c: 3\nb.e[0].name: Howdy Partner\n\n\n\n\nthen\n\n\nyq n -s create_instructions.yaml\n\n\n\n\nwill output:\n\n\nb:\n c: 3\n e:\n - name: Howdy Partner\n\n\n\n\nYou can also pipe the instructions in:\n\n\ncat create_instructions.yaml | yq n -s -\n\n\n\n\nKeys with dots\n\u00b6\n\n\nWhen specifying a key that has a dot use key lookup indicator.\n\n\nb:\n foo.bar: 7\n\n\n\n\nyaml r sample.yaml 'b[foo.bar]'\n\n\n\n\nyaml w sample.yaml 'b[foo.bar]' 9\n\n\n\n\nAny valid yaml key can be specified as part of a key lookup.\n\n\nNote that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"text":"Create scripts follow the same format as the update scripts. Given a script create_instructions.yaml of: b.c: 3\nb.e[0].name: Howdy Partner then yq n -s create_instructions.yaml will output: b:\n c: 3\n e:\n - name: Howdy Partner You can also pipe the instructions in: cat create_instructions.yaml | yq n -s -",
"title":"Creating using a create script"
},
{
"location":"/create/#keys-with-dots",
"text":"When specifying a key that has a dot use key lookup indicator. b:\n foo.bar: 7 yaml r sample.yaml 'b[foo.bar]' yaml w sample.yaml 'b[foo.bar]' 9 Any valid yaml key can be specified as part of a key lookup. Note that the path is in quotes to avoid the square brackets being interpreted by your shell.",
"title":"Keys with dots"
},
{
"location":"/convert/",
"text":"Yaml to Json\n\u00b6\n\n\nTo convert output to json, use the --tojson (or -j) flag. This can be used with any command.\n\n\nGiven a sample.yaml file of:\n\n\nb:\n c: 2\n\n\n\n\nthen\n\n\nyq r -j sample.yaml b.c\n\n\n\n\nwill output\n\n\n{\"b\":{\"c\":2}}\n\n\n\n\nJson to Yaml\n\u00b6\n\n\nTo read in json, just pass in a json file instead of yaml, it will just work :)\n\n\ne.g given a json file\n\n\n{\"a\":\"Easy! as one two three\",\"b\":{\"c\":2,\"d\":[3,4]}}\n\n\n\n\nthen\n\n\nyq r sample.json\n\n\n\n\nwill output\n\n\na: Easy! as one two three\nb:\n c: 2\n d:\n - 3\n - 4",
"title":"Convert"
},
{
"location":"/convert/#yaml-to-json",
"text":"To convert output to json, use the --tojson (or -j) flag. This can be used with any command. Given a sample.yaml file of: b:\n c: 2 then yq r -j sample.yaml b.c will output {\"b\":{\"c\":2}}",
"title":"Yaml to Json"
},
{
"location":"/convert/#json-to-yaml",
"text":"To read in json, just pass in a json file instead of yaml, it will just work :) e.g given a json file {\"a\":\"Easy! as one two three\",\"b\":{\"c\":2,\"d\":[3,4]}} then yq r sample.json will output a: Easy! as one two three\nb:\n c: 2\n d:\n - 3\n - 4",
"title":"Json to Yaml"
},
{
"location":"/merge/",
"text":"Yaml files can be merged using the 'merge' command. Each additional file merged with the first file will\nset values for any key not existing already or where the key has no value.\n\n\nyq m <yaml_file|json_file> <path>...\n\n\n\n\nThis command can take a json file as input too, and will output yaml unless specified to export as json (-j)\n\n\nTo Stdout\n\u00b6\n\n\nGiven a data1.yaml file of:\n\n\na: simple\nb: [1, 2]\n\n\n\n\nand data2.yaml file of:\n\n\na: other\nc:\n test: 1\n\n\n\n\nthen\n\n\nyq m data1.yaml data2.yaml\n\n\n\n\nwill output:\n\n\na: simple\nb: [1, 2]\nc:\n test: 1\n\n\n\n\nUpdating files in-place\n\u00b6\n\n\nGiven a data1.yaml file of:\n\n\na: simple\nb: [1, 2]\n\n\n\n\nand data2.yaml file of:\n\n\na: other\nc:\n test: 1\n\n\n\n\nthen\n\n\nyq m -i data1.yaml data2.yaml\n\n\n\n\nwill update the data1.yaml file so that the value of 'c' is 'test: 1'.\n\n\nOverwrite values\n\u00b6\n\n\nGiven a data1.yaml file of:\n\n\na: simple\nb: [1, 2]\n\n\n\n\nand data2.yaml file of:\n\n\na: other\nc:\n test: 1\n\n\n\n\nthen\n\n\nyq m -x data1.yaml data2.yaml\n\n\n\n\nwill output:\n\n\na: other\nb: [1, 2]\nc:\n test: 1\n\n\n\n\nOverwrite values with arrays\n\u00b6\n\n\nGiven a data1.yaml file of:\n\n\na: simple\nb: [1, 2]\n\n\n\n\nand data3.yaml file of:\n\n\nb: [2, 3, 4]\nc:\n test: 2\n other: true\nd: false\n\n\n\n\nthen\n\n\nyq m -x data1.yaml data3.yaml\n\n\n\n\nwill output:\n\n\na: simple\nb: [2, 3, 4]\nc:\n test: 2\n other: true\nd: false\n\n\n\n\nNotice that 'b' does not result in the merging of the values within an array. The underlying library does not\ncurrently handle merging values within an array.",
"title":"Merge"
},
{
"location":"/merge/#to-stdout",
"text":"Given a data1.yaml file of: a: simple\nb: [1, 2] and data2.yaml file of: a: other\nc:\n test: 1 then yq m data1.yaml data2.yaml will output: a: simple\nb: [1, 2]\nc:\n test: 1",
"title":"To Stdout"
},
{
"location":"/merge/#updating-files-in-place",
"text":"Given a data1.yaml file of: a: simple\nb: [1, 2] and data2.yaml file of: a: other\nc:\n test: 1 then yq m -i data1.yaml data2.yaml will update the data1.yaml file so that the value of 'c' is 'test: 1'.",
"title":"Updating files in-place"
},
{
"location":"/merge/#overwrite-values",
"text":"Given a data1.yaml file of: a: simple\nb: [1, 2] and data2.yaml file of: a: other\nc:\n test: 1 then yq m -x data1.yaml data2.yaml will output: a: other\nb: [1, 2]\nc:\n test: 1",
"text":"Given a data1.yaml file of: a: simple\nb: [1, 2] and data3.yaml file of: b: [2, 3, 4]\nc:\n test: 2\n other: true\nd: false then yq m -x data1.yaml data3.yaml will output: a: simple\nb: [2, 3, 4]\nc:\n test: 2\n other: true\nd: false Notice that 'b' does not result in the merging of the values within an array. The underlying library does not\ncurrently handle merging values within an array.",
<pre><codeclass="bash">cat sample.yaml | yq w - b.c blah
</code></pre>
<h3id="adding-new-fields">Adding new fields<aclass="headerlink"href="#adding-new-fields"title="Permanent link">¶</a></h3>
<p>Any missing fields in the path will be created on the fly.</p>
<p>Given a sample.yaml file of:</p>
<pre><codeclass="yaml">b:
c: 2
</code></pre>
<p>then</p>
<pre><codeclass="bash">yq w sample.yaml b.d[0] "new thing"
</code></pre>
<p>will output:</p>
<pre><codeclass="yaml">b:
c: cat
d:
- new thing
</code></pre>
<h3id="appending-value-to-an-array-field">Appending value to an array field<aclass="headerlink"href="#appending-value-to-an-array-field"title="Permanent link">¶</a></h3>
<p>Given a sample.yaml file of:</p>
<pre><codeclass="yaml">b:
c: 2
d:
- new thing
- foo thing
</code></pre>
<p>then</p>
<pre><codeclass="bash">yq w sample.yaml "b.d[+]""bar thing"
</code></pre>
<p>will output:</p>
<pre><codeclass="yaml">b:
c: cat
d:
- new thing
- foo thing
- bar thing
</code></pre>
<p>Note that the path is in quotes to avoid the square brackets being interpreted by your shell.</p>
<pre><codeclass="bash">yq w -i sample.yaml b.c cat
</code></pre>
<p>will update the sample.yaml file so that the value of 'c' is cat.</p>
<h3id="updating-multiple-values-with-a-script">Updating multiple values with a script<aclass="headerlink"href="#updating-multiple-values-with-a-script"title="Permanent link">¶</a></h3>
<p>Given a sample.yaml file of:</p>
<pre><codeclass="yaml">b:
c: 2
e:
- name: Billy Bob
</code></pre>
<p>and a script update_instructions.yaml of:</p>
<pre><codeclass="yaml">b.c: 3
b.e[0].name: Howdy Partner
</code></pre>
<p>then</p>
<pre><codeclass="bash">yq w -s update_instructions.yaml sample.yaml
</code></pre>
<p>will output:</p>
<pre><codeclass="yaml">b:
c: 3
e:
- name: Howdy Partner
</code></pre>
<p>And, of course, you can pipe the instructions in using '-':</p>
<pre><codeclass="bash">cat update_instructions.yaml | yq w -s - sample.yaml
</code></pre>
<h3id="values-starting-with-a-hyphen-or-dash">Values starting with a hyphen (or dash)<aclass="headerlink"href="#values-starting-with-a-hyphen-or-dash"title="Permanent link">¶</a></h3>
<p>The flag terminator needs to be used to stop the app from attempting to parse the subsequent arguments as flags:</p>
<pre><code>yq w -- my.path -3
</code></pre>
<p>will output</p>
<pre><codeclass="yaml">my:
path: -3
</code></pre>
<h3id="keys-with-dots">Keys with dots<aclass="headerlink"href="#keys-with-dots"title="Permanent link">¶</a></h3>
<p>When specifying a key that has a dot use key lookup indicator.</p>
<pre><codeclass="yaml">b:
foo.bar: 7
</code></pre>
<pre><codeclass="bash">yaml r sample.yaml 'b[foo.bar]'
</code></pre>
<pre><codeclass="bash">yaml w sample.yaml 'b[foo.bar]' 9
</code></pre>
<p>Any valid yaml key can be specified as part of a key lookup.</p>
<p>Note that the path is in quotes to avoid the square brackets being interpreted by your shell.</p>
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Mike Farah
