mirror of
https://github.com/mikefarah/yq.git
synced 2026-08-24 16:39:58 +08:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Mike Farah
|
0c24ba23f0 |
@@ -60,23 +60,3 @@ jobs:
|
||||
-t "${IMAGE_NAME}:4" \
|
||||
-t "${IMAGE_NAME}:latest" \
|
||||
.
|
||||
|
||||
cd github-action
|
||||
docker buildx build \
|
||||
--label "org.opencontainers.image.authors=https://github.com/mikefarah/yq/graphs/contributors" \
|
||||
--label "org.opencontainers.image.created=$(date --rfc-3339=seconds)" \
|
||||
--label "org.opencontainers.image.description=yq is a portable command-line YAML processor" \
|
||||
--label "org.opencontainers.image.documentation=https://mikefarah.gitbook.io/yq/" \
|
||||
--label "org.opencontainers.image.licenses=MIT" \
|
||||
--label "org.opencontainers.image.revision=$(git rev-parse HEAD)" \
|
||||
--label "org.opencontainers.image.source=https://github.com/mikefarah/yq" \
|
||||
--label "org.opencontainers.image.title=yq" \
|
||||
--label "org.opencontainers.image.url=https://mikefarah.gitbook.io/yq/" \
|
||||
--label "org.opencontainers.image.version=${IMAGE_VERSION}" \
|
||||
--platform "${PLATFORMS}" \
|
||||
--pull \
|
||||
--push \
|
||||
-t "${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
|
||||
-t "${IMAGE_NAME}:4-githubaction" \
|
||||
-t "${IMAGE_NAME}:latest-githubaction" \
|
||||
.
|
||||
|
||||
@@ -10,10 +10,10 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
|
||||
- name: Set up Go 1.19
|
||||
- name: Set up Go 1.18
|
||||
uses: actions/setup-go@v3
|
||||
with:
|
||||
go-version: 1.19
|
||||
go-version: 1.18
|
||||
id: go
|
||||
|
||||
- name: Check out code into the Go module directory
|
||||
|
||||
@@ -12,8 +12,7 @@ jobs:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-go@v3
|
||||
with:
|
||||
go-version: '^1.19'
|
||||
check-latest: true
|
||||
go-version: '^1.18'
|
||||
- name: Compile man page markup
|
||||
id: gen-man-page-md
|
||||
run: |
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
name: Test Yq Action
|
||||
on:
|
||||
push:
|
||||
workflow_dispatch:
|
||||
|
||||
on: [push]
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -15,14 +12,11 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Get test
|
||||
id: get_value
|
||||
uses: mikefarah/yq@master
|
||||
with:
|
||||
cmd: yq '.test' examples/multiline-text.yaml
|
||||
- name: Multiline test
|
||||
run: echo "### It was [${{ steps.get_value.outputs.result }}]" >> $GITHUB_STEP_SUMMARY
|
||||
cmd: yq eval '.a' examples/sample.yaml
|
||||
- name: Write inplace test
|
||||
id: lookupSdkVersion
|
||||
uses: mikefarah/yq@master
|
||||
with:
|
||||
cmd: yq -i '.b.c = 5' examples/sample.yaml
|
||||
cmd: yq eval -i '.b.c = 5' examples/sample.yaml
|
||||
+1
-6
@@ -56,9 +56,4 @@ _build
|
||||
debian/files
|
||||
|
||||
# intellij
|
||||
/.idea
|
||||
|
||||
# vscode
|
||||
.vscode
|
||||
|
||||
yq3
|
||||
/.idea
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
FROM golang:1.19.2 as builder
|
||||
FROM golang:1.18.3 as builder
|
||||
|
||||
WORKDIR /go/src/mikefarah/yq
|
||||
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
FROM golang:1.19.2
|
||||
FROM golang:1.18.3
|
||||
|
||||
COPY scripts/devtools.sh /opt/devtools.sh
|
||||
|
||||
|
||||
@@ -3,10 +3,31 @@
|
||||
    
|
||||
|
||||
|
||||
a lightweight and portable command-line YAML, JSON and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files as well as json, xml, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
|
||||
a lightweight and portable command-line YAML, JSON and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files as well as json and xml. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
|
||||
|
||||
yq is written in go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as Docker and Podman, all listed below.
|
||||
|
||||
## Notice for v4.x versions prior to 4.18.1
|
||||
Since 4.18.1, yq's 'eval/e' command is the _default_ command and no longer needs to be specified.
|
||||
|
||||
Older versions will still need to specify 'eval/e'.
|
||||
|
||||
Similarly, '-' is no longer required as a filename to read from STDIN (unless reading from one or more files).
|
||||
|
||||
TLDR:
|
||||
|
||||
Prior to 4.18.1
|
||||
```bash
|
||||
yq e '.cool' - < file.yaml
|
||||
```
|
||||
|
||||
4.18+
|
||||
```bash
|
||||
yq '.cool' < file.yaml
|
||||
```
|
||||
|
||||
When merging multiple files together, `eval-all/ea` is still required to tell `yq` to run the expression against all the document at once.
|
||||
|
||||
## Quick Usage Guide
|
||||
|
||||
Read a value:
|
||||
@@ -52,14 +73,12 @@ yq -P sample.json
|
||||
|
||||
See the [documentation](https://mikefarah.gitbook.io/yq/) for more examples.
|
||||
|
||||
Take a look at the discussions for [common questions](https://github.com/mikefarah/yq/discussions/categories/q-a), and [cool ideas](https://github.com/mikefarah/yq/discussions/categories/show-and-tell)
|
||||
|
||||
## Install
|
||||
|
||||
### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
|
||||
|
||||
### wget
|
||||
Use wget to download, gzipped pre-compiled binaries:
|
||||
Use wget to download the pre-compiled binaries:
|
||||
|
||||
#### Compressed via tar.gz
|
||||
```bash
|
||||
@@ -76,13 +95,6 @@ wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /
|
||||
|
||||
For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64
|
||||
|
||||
Use VERSION=latest to always get the latest version:
|
||||
|
||||
```bash
|
||||
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq &&\
|
||||
chmod +x /usr/bin/yq
|
||||
```
|
||||
|
||||
### MacOS / Linux via Homebrew:
|
||||
Using [Homebrew](https://brew.sh/)
|
||||
```
|
||||
@@ -195,27 +207,12 @@ RUN apk add --no-cache tzdata
|
||||
USER yq
|
||||
```
|
||||
|
||||
#### Podman with SELinux
|
||||
|
||||
If you are using podman with SELinux, you will need to set the shared volume flag `:z` on the volume mount:
|
||||
|
||||
```
|
||||
-v "${PWD}":/workdir:z
|
||||
```
|
||||
|
||||
### GitHub Action
|
||||
```
|
||||
- name: Set foobar to cool
|
||||
uses: mikefarah/yq@master
|
||||
with:
|
||||
cmd: yq -i '.foo.bar = "cool"' 'config.yml'
|
||||
- name: Get an entry with a variable that might contain dots or spaces
|
||||
id: get_username
|
||||
uses: mikefarah/yq@master
|
||||
with:
|
||||
cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
|
||||
- name: Reuse a variable obtained in another step
|
||||
run: echo ${{ steps.get_username.outputs.result }}
|
||||
```
|
||||
|
||||
See https://mikefarah.gitbook.io/yq/usage/github-action for more.
|
||||
@@ -228,17 +225,7 @@ go install github.com/mikefarah/yq/v4@latest
|
||||
## Community Supported Installation methods
|
||||
As these are supported by the community :heart: - however, they may be out of date with the officially supported releases.
|
||||
|
||||
|
||||
### Nix
|
||||
|
||||
```
|
||||
nix profile install nixpkgs#yq-go
|
||||
```
|
||||
|
||||
See [here](https://search.nixos.org/packages?channel=unstable&show=yq-go&from=0&size=50&sort=relevance&type=packages&query=yq-go)
|
||||
|
||||
|
||||
### Webi
|
||||
# Webi
|
||||
|
||||
```
|
||||
webi yq
|
||||
@@ -261,13 +248,6 @@ choco install yq
|
||||
```
|
||||
Supported by @chillum (https://chocolatey.org/packages/yq)
|
||||
|
||||
and
|
||||
|
||||
### Winget
|
||||
winget install yq
|
||||
|
||||
https://winget.run/pkg/MikeFarah/yq
|
||||
|
||||
### Mac:
|
||||
Using [MacPorts](https://www.macports.org/)
|
||||
```
|
||||
@@ -310,10 +290,10 @@ Supported by @rmescandon (https://launchpad.net/~rmescandon/+archive/ubuntu/yq)
|
||||
- Keeps yaml formatting and comments when updating (though there are issues with whitespace)
|
||||
- [Decode/Encode base64 data](https://mikefarah.gitbook.io/yq/operators/encode-decode)
|
||||
- [Load content from other files](https://mikefarah.gitbook.io/yq/operators/load)
|
||||
- [Convert to/from json/ndjson](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert)
|
||||
- [Convert to/from json](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert)
|
||||
- [Convert to/from xml](https://mikefarah.gitbook.io/yq/v/v4.x/usage/xml)
|
||||
- [Convert to/from properties](https://mikefarah.gitbook.io/yq/v/v4.x/usage/properties)
|
||||
- [Convert to/from csv/tsv](https://mikefarah.gitbook.io/yq/usage/csv-tsv)
|
||||
- [Convert to csv/tsv](https://mikefarah.gitbook.io/yq/usage/csv-tsv)
|
||||
- [General shell completion scripts (bash/zsh/fish/powershell)](https://mikefarah.gitbook.io/yq/v/v4.x/commands/shell-completion)
|
||||
- [Reduce](https://mikefarah.gitbook.io/yq/operators/reduce) to merge multiple files or sum an array or other fancy things.
|
||||
- [Github Action](https://mikefarah.gitbook.io/yq/usage/github-action) to use in your automated pipeline (thanks @devorbitus)
|
||||
|
||||
@@ -9,7 +9,7 @@ EOL
|
||||
|
||||
testEmptyEval() {
|
||||
X=$(./yq e test.yml)
|
||||
expected="# comment"
|
||||
expected=$(cat test.yml)
|
||||
assertEquals 0 $?
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
setUp() {
|
||||
rm test*.yml || true
|
||||
|
||||
}
|
||||
|
||||
testLineCountFirstLineComment() {
|
||||
cat >test.yml <<EOL
|
||||
#test123
|
||||
abc: 123
|
||||
test123: 123123
|
||||
#comment
|
||||
lalilu: lalilu
|
||||
EOL
|
||||
|
||||
X=$(./yq '.lalilu | line' --header-preprocess=false < test.yml)
|
||||
assertEquals "5" "$X"
|
||||
}
|
||||
|
||||
testArrayOfDocs() {
|
||||
cat >test.yml <<EOL
|
||||
---
|
||||
# leading comment doc 1
|
||||
a: 1
|
||||
---
|
||||
# leading comment doc 2
|
||||
a: 2
|
||||
EOL
|
||||
|
||||
read -r -d '' expected << EOM
|
||||
- # leading comment doc 1
|
||||
a: 1
|
||||
- # leading comment doc 2
|
||||
a: 2
|
||||
EOM
|
||||
|
||||
X=$(./yq ea '[.]' --header-preprocess=false < test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
}
|
||||
|
||||
source ./scripts/shunit2
|
||||
@@ -3,8 +3,6 @@
|
||||
setUp() {
|
||||
rm test*.yml 2>/dev/null || true
|
||||
rm test*.properties 2>/dev/null || true
|
||||
rm test*.csv 2>/dev/null || true
|
||||
rm test*.tsv 2>/dev/null || true
|
||||
rm test*.xml 2>/dev/null || true
|
||||
}
|
||||
|
||||
@@ -42,65 +40,6 @@ EOM
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testInputCSV() {
|
||||
cat >test.csv <<EOL
|
||||
fruit,yumLevel
|
||||
apple,5
|
||||
banana,4
|
||||
EOL
|
||||
|
||||
read -r -d '' expected << EOM
|
||||
- fruit: apple
|
||||
yumLevel: 5
|
||||
- fruit: banana
|
||||
yumLevel: 4
|
||||
EOM
|
||||
|
||||
X=$(./yq e -p=csv test.csv)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea -p=csv test.csv)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testInputCSVUTF8() {
|
||||
read -r -d '' expected << EOM
|
||||
- id: 1
|
||||
first: john
|
||||
last: smith
|
||||
- id: 1
|
||||
first: jane
|
||||
last: smith
|
||||
EOM
|
||||
|
||||
X=$(./yq -p=csv utf8.csv)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testInputTSV() {
|
||||
cat >test.tsv <<EOL
|
||||
fruit yumLevel
|
||||
apple 5
|
||||
banana 4
|
||||
EOL
|
||||
|
||||
read -r -d '' expected << EOM
|
||||
- fruit: apple
|
||||
yumLevel: 5
|
||||
- fruit: banana
|
||||
yumLevel: 4
|
||||
EOM
|
||||
|
||||
X=$(./yq e -p=t test.tsv)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea -p=t test.tsv)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
testInputXml() {
|
||||
cat >test.yml <<EOL
|
||||
<cat legs="4">BiBi</cat>
|
||||
@@ -127,7 +66,6 @@ testInputXmlNamespaces() {
|
||||
EOL
|
||||
|
||||
read -r -d '' expected << EOM
|
||||
+p_xml: version="1.0"
|
||||
map:
|
||||
+xmlns: some-namespace
|
||||
+xmlns:xsi: some-instance
|
||||
@@ -141,26 +79,6 @@ EOM
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testInputXmlRoundtrip() {
|
||||
cat >test.yml <<EOL
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
|
||||
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">Meow</map>
|
||||
EOL
|
||||
|
||||
read -r -d '' expected << EOM
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
|
||||
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">Meow</map>
|
||||
EOM
|
||||
|
||||
X=$(./yq -p=xml -o=xml test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea -p=xml -o=xml test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
|
||||
testInputXmlStrict() {
|
||||
cat >test.yml <<EOL
|
||||
@@ -174,11 +92,11 @@ testInputXmlStrict() {
|
||||
</root>
|
||||
EOL
|
||||
|
||||
X=$(./yq -p=xml --xml-strict-mode test.yml -o=xml 2>&1)
|
||||
X=$(./yq -p=xml --xml-strict-mode test.yml 2>&1)
|
||||
assertEquals 1 $?
|
||||
assertEquals "Error: bad file 'test.yml': XML syntax error on line 7: invalid character entity &writer;" "$X"
|
||||
|
||||
X=$(./yq ea -p=xml --xml-strict-mode test.yml -o=xml 2>&1)
|
||||
X=$(./yq ea -p=xml --xml-strict-mode test.yml 2>&1)
|
||||
assertEquals "Error: bad file 'test.yml': XML syntax error on line 7: invalid character entity &writer;" "$X"
|
||||
}
|
||||
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
#!/bin/bash
|
||||
|
||||
|
||||
# examples where header-preprocess is required
|
||||
|
||||
setUp() {
|
||||
rm test*.yml || true
|
||||
cat >test.yml <<EOL
|
||||
|
||||
@@ -102,48 +102,6 @@ EOM
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testOutputCSV() {
|
||||
cat >test.yml <<EOL
|
||||
- fruit: apple
|
||||
yumLevel: 5
|
||||
- fruit: banana
|
||||
yumLevel: 4
|
||||
EOL
|
||||
|
||||
read -r -d '' expected << EOM
|
||||
fruit,yumLevel
|
||||
apple,5
|
||||
banana,4
|
||||
EOM
|
||||
|
||||
X=$(./yq -o=c test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea -o=csv test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testOutputTSV() {
|
||||
cat >test.yml <<EOL
|
||||
- fruit: apple
|
||||
yumLevel: 5
|
||||
- fruit: banana
|
||||
yumLevel: 4
|
||||
EOL
|
||||
|
||||
read -r -d '' expected << EOM
|
||||
fruit yumLevel
|
||||
apple 5
|
||||
banana 4
|
||||
EOM
|
||||
|
||||
X=$(./yq -o=t test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea -o=tsv test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testOutputXml() {
|
||||
cat >test.yml <<EOL
|
||||
a: {b: {c: ["cat"]}}
|
||||
|
||||
+1
-1
@@ -12,6 +12,6 @@ outputs:
|
||||
description: "The complete result from the yq command being run"
|
||||
runs:
|
||||
using: 'docker'
|
||||
image: 'docker://mikefarah/yq:4-githubaction'
|
||||
image: 'github-action/Dockerfile'
|
||||
args:
|
||||
- ${{ inputs.cmd }}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package cmd
|
||||
|
||||
var leadingContentPreProcessing = true
|
||||
var unwrapScalar = true
|
||||
|
||||
var writeInplace = false
|
||||
@@ -7,6 +8,12 @@ var outputToJSON = false
|
||||
var outputFormat = "yaml"
|
||||
var inputFormat = "yaml"
|
||||
|
||||
var xmlAttributePrefix = "+"
|
||||
var xmlContentName = "+content"
|
||||
var xmlStrictMode = false
|
||||
var xmlKeepNamespace = true
|
||||
var xmlUseRawToken = true
|
||||
|
||||
var exitStatus = false
|
||||
var forceColor = false
|
||||
var forceNoColor = false
|
||||
|
||||
@@ -75,7 +75,7 @@ func evaluateAll(cmd *cobra.Command, args []string) (cmdError error) {
|
||||
return err
|
||||
}
|
||||
|
||||
decoder, err := configureDecoder(true)
|
||||
decoder, err := configureDecoder()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -109,13 +109,13 @@ func evaluateAll(cmd *cobra.Command, args []string) (cmdError error) {
|
||||
switch len(args) {
|
||||
case 0:
|
||||
if nullInput {
|
||||
err = yqlib.NewStreamEvaluator().EvaluateNew(processExpression(expression), printer)
|
||||
err = yqlib.NewStreamEvaluator().EvaluateNew(processExpression(expression), printer, "")
|
||||
} else {
|
||||
cmd.Println(cmd.UsageString())
|
||||
return nil
|
||||
}
|
||||
default:
|
||||
err = allAtOnceEvaluator.EvaluateFiles(processExpression(expression), args, printer, decoder)
|
||||
err = allAtOnceEvaluator.EvaluateFiles(processExpression(expression), args, printer, leadingContentPreProcessing, decoder)
|
||||
}
|
||||
|
||||
completedSuccessfully = err == nil
|
||||
|
||||
@@ -97,7 +97,7 @@ func evaluateSequence(cmd *cobra.Command, args []string) (cmdError error) {
|
||||
|
||||
printer := yqlib.NewPrinter(encoder, printerWriter)
|
||||
|
||||
decoder, err := configureDecoder(false)
|
||||
decoder, err := configureDecoder()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -123,13 +123,13 @@ func evaluateSequence(cmd *cobra.Command, args []string) (cmdError error) {
|
||||
switch len(args) {
|
||||
case 0:
|
||||
if nullInput {
|
||||
err = streamEvaluator.EvaluateNew(processExpression(expression), printer)
|
||||
err = streamEvaluator.EvaluateNew(processExpression(expression), printer, "")
|
||||
} else {
|
||||
cmd.Println(cmd.UsageString())
|
||||
return nil
|
||||
}
|
||||
default:
|
||||
err = streamEvaluator.EvaluateFiles(processExpression(expression), args, printer, decoder)
|
||||
err = streamEvaluator.EvaluateFiles(processExpression(expression), args, printer, leadingContentPreProcessing, decoder)
|
||||
}
|
||||
completedSuccessfully = err == nil
|
||||
|
||||
|
||||
+10
-23
@@ -47,23 +47,14 @@ yq -P sample.json
|
||||
if verbose {
|
||||
backend.SetLevel(logging.DEBUG, "")
|
||||
} else {
|
||||
backend.SetLevel(logging.WARNING, "")
|
||||
backend.SetLevel(logging.ERROR, "")
|
||||
}
|
||||
|
||||
logging.SetBackend(backend)
|
||||
yqlib.InitExpressionParser()
|
||||
if (inputFormat == "x" || inputFormat == "xml") &&
|
||||
outputFormat != "x" && outputFormat != "xml" &&
|
||||
yqlib.ConfiguredXMLPreferences.AttributePrefix == "+" {
|
||||
yqlib.GetLogger().Warning("The default xml-attribute-prefix will change in the v4.30 to `+@` to avoid " +
|
||||
"naming conflicts with the default content name, directive name and proc inst prefix. If you need to keep " +
|
||||
"`+` please set that value explicityly with --xml-attribute-prefix.")
|
||||
}
|
||||
|
||||
//copy preference form global setting
|
||||
yqlib.ConfiguredYamlPreferences.UnwrapScalar = unwrapScalar
|
||||
|
||||
yqlib.ConfiguredYamlPreferences.PrintDocSeparators = !noDocSeparators
|
||||
yqlib.XMLPreferences.AttributePrefix = xmlAttributePrefix
|
||||
yqlib.XMLPreferences.ContentName = xmlContentName
|
||||
yqlib.XMLPreferences.StrictMode = xmlStrictMode
|
||||
},
|
||||
}
|
||||
|
||||
@@ -78,15 +69,11 @@ yq -P sample.json
|
||||
rootCmd.PersistentFlags().StringVarP(&outputFormat, "output-format", "o", "yaml", "[yaml|y|json|j|props|p|xml|x] output format type.")
|
||||
rootCmd.PersistentFlags().StringVarP(&inputFormat, "input-format", "p", "yaml", "[yaml|y|props|p|xml|x] parse format for input. Note that json is a subset of yaml.")
|
||||
|
||||
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.AttributePrefix, "xml-attribute-prefix", "+", "prefix for xml attributes")
|
||||
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.ContentName, "xml-content-name", "+content", "name for xml content (if no attribute name is present).")
|
||||
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.StrictMode, "xml-strict-mode", false, "enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.")
|
||||
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.KeepNamespace, "xml-keep-namespace", true, "enables keeping namespace after parsing attributes")
|
||||
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.UseRawToken, "xml-raw-token", true, "enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details.")
|
||||
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.ProcInstPrefix, "xml-proc-inst-prefix", "+p_", "prefix for xml processing instructions (e.g. <?xml version=\"1\"?>)")
|
||||
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.DirectiveName, "xml-directive-name", "+directive", "name for xml directives (e.g. <!DOCTYPE thing cat>)")
|
||||
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.SkipProcInst, "xml-skip-proc-inst", false, "skip over process instructions (e.g. <?xml version=\"1\"?>)")
|
||||
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.SkipDirectives, "xml-skip-directives", false, "skip over directives (e.g. <!DOCTYPE thing cat>)")
|
||||
rootCmd.PersistentFlags().StringVar(&xmlAttributePrefix, "xml-attribute-prefix", "+", "prefix for xml attributes")
|
||||
rootCmd.PersistentFlags().StringVar(&xmlContentName, "xml-content-name", "+content", "name for xml content (if no attribute name is present).")
|
||||
rootCmd.PersistentFlags().BoolVar(&xmlStrictMode, "xml-strict-mode", false, "enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.")
|
||||
rootCmd.PersistentFlags().BoolVar(&xmlKeepNamespace, "xml-keep-namespace", true, "enables keeping namespace after parsing attributes")
|
||||
rootCmd.PersistentFlags().BoolVar(&xmlUseRawToken, "xml-raw-token", true, "enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details.")
|
||||
|
||||
rootCmd.PersistentFlags().BoolVarP(&nullInput, "null-input", "n", false, "Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.")
|
||||
rootCmd.PersistentFlags().BoolVarP(&noDocSeparators, "no-doc", "N", false, "Don't print document separators (---)")
|
||||
@@ -102,7 +89,7 @@ yq -P sample.json
|
||||
rootCmd.PersistentFlags().BoolVarP(&forceNoColor, "no-colors", "M", false, "force print with no colors")
|
||||
rootCmd.PersistentFlags().StringVarP(&frontMatter, "front-matter", "f", "", "(extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact")
|
||||
rootCmd.PersistentFlags().StringVarP(&forceExpression, "expression", "", "", "forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.")
|
||||
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredYamlPreferences.LeadingContentPreProcessing, "header-preprocess", "", true, "Slurp any header comments and separators before processing expression.")
|
||||
rootCmd.PersistentFlags().BoolVarP(&leadingContentPreProcessing, "header-preprocess", "", true, "Slurp any header comments and separators before processing expression.")
|
||||
|
||||
rootCmd.PersistentFlags().StringVarP(&splitFileExp, "split-exp", "s", "", "print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.")
|
||||
rootCmd.PersistentFlags().StringVarP(&splitFileExpFile, "split-exp-file", "", "", "Use a file to specify the split-exp expression.")
|
||||
|
||||
@@ -42,7 +42,7 @@ $ yq shell-completion fish > ~/.config/fish/completions/yq.fish
|
||||
`,
|
||||
DisableFlagsInUseLine: true,
|
||||
ValidArgs: []string{"bash", "zsh", "fish", "powershell"},
|
||||
Args: cobra.MatchAll(cobra.ExactArgs(1), cobra.OnlyValidArgs),
|
||||
Args: cobra.ExactValidArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
var err error = nil
|
||||
switch args[0] {
|
||||
|
||||
+7
-14
@@ -56,26 +56,19 @@ func initCommand(cmd *cobra.Command, args []string) (string, []string, error) {
|
||||
return expression, args, nil
|
||||
}
|
||||
|
||||
func configureDecoder(evaluateTogether bool) (yqlib.Decoder, error) {
|
||||
func configureDecoder() (yqlib.Decoder, error) {
|
||||
yqlibInputFormat, err := yqlib.InputFormatFromString(inputFormat)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch yqlibInputFormat {
|
||||
case yqlib.XMLInputFormat:
|
||||
return yqlib.NewXMLDecoder(yqlib.ConfiguredXMLPreferences), nil
|
||||
return yqlib.NewXMLDecoder(xmlAttributePrefix, xmlContentName, xmlStrictMode, xmlKeepNamespace, xmlUseRawToken), nil
|
||||
case yqlib.PropertiesInputFormat:
|
||||
return yqlib.NewPropertiesDecoder(), nil
|
||||
case yqlib.JsonInputFormat:
|
||||
return yqlib.NewJSONDecoder(), nil
|
||||
case yqlib.CSVObjectInputFormat:
|
||||
return yqlib.NewCSVObjectDecoder(','), nil
|
||||
case yqlib.TSVObjectInputFormat:
|
||||
return yqlib.NewCSVObjectDecoder('\t'), nil
|
||||
}
|
||||
prefs := yqlib.ConfiguredYamlPreferences
|
||||
prefs.EvaluateTogether = evaluateTogether
|
||||
return yqlib.NewYamlDecoder(prefs), nil
|
||||
|
||||
return yqlib.NewYamlDecoder(), nil
|
||||
}
|
||||
|
||||
func configurePrinterWriter(format yqlib.PrinterOutputFormat, out io.Writer) (yqlib.PrinterWriter, error) {
|
||||
@@ -98,7 +91,7 @@ func configurePrinterWriter(format yqlib.PrinterOutputFormat, out io.Writer) (yq
|
||||
func configureEncoder(format yqlib.PrinterOutputFormat) yqlib.Encoder {
|
||||
switch format {
|
||||
case yqlib.JSONOutputFormat:
|
||||
return yqlib.NewJSONEncoder(indent, colorsEnabled)
|
||||
return yqlib.NewJONEncoder(indent, colorsEnabled)
|
||||
case yqlib.PropsOutputFormat:
|
||||
return yqlib.NewPropertiesEncoder(unwrapScalar)
|
||||
case yqlib.CSVOutputFormat:
|
||||
@@ -106,9 +99,9 @@ func configureEncoder(format yqlib.PrinterOutputFormat) yqlib.Encoder {
|
||||
case yqlib.TSVOutputFormat:
|
||||
return yqlib.NewCsvEncoder('\t')
|
||||
case yqlib.YamlOutputFormat:
|
||||
return yqlib.NewYamlEncoder(indent, colorsEnabled, yqlib.ConfiguredYamlPreferences)
|
||||
return yqlib.NewYamlEncoder(indent, colorsEnabled, !noDocSeparators, unwrapScalar)
|
||||
case yqlib.XMLOutputFormat:
|
||||
return yqlib.NewXMLEncoder(indent, yqlib.ConfiguredXMLPreferences)
|
||||
return yqlib.NewXMLEncoder(indent, xmlAttributePrefix, xmlContentName)
|
||||
}
|
||||
panic("invalid encoder")
|
||||
}
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@ var (
|
||||
GitDescribe string
|
||||
|
||||
// Version is main version number that is being run at the moment.
|
||||
Version = "4.28.2"
|
||||
Version = "4.25.3"
|
||||
|
||||
// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
|
||||
// then it means that it is a final release. Otherwise, this is a pre-release
|
||||
|
||||
+12
-5
@@ -1,5 +1,12 @@
|
||||
volumes:
|
||||
beehive-conf:
|
||||
driver: local
|
||||
services:
|
||||
- beehive
|
||||
# --------------------------------------------------
|
||||
# It's a test with comment
|
||||
# --------------------------------------------------
|
||||
test:
|
||||
- name: a
|
||||
- name: b
|
||||
- name: c
|
||||
groups:
|
||||
- name: a
|
||||
- name: b
|
||||
- name: c
|
||||
- name: c
|
||||
+14
-7
@@ -1,7 +1,14 @@
|
||||
<?xml version="1.0"?>
|
||||
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
|
||||
<apple>
|
||||
<?coolioo version="1.0"?>
|
||||
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
|
||||
<b>things</b>
|
||||
</apple>
|
||||
<!-- before cat -->
|
||||
<cat>
|
||||
<!-- in cat before -->
|
||||
<x>3<!-- multi
|
||||
line comment
|
||||
for x --></x>
|
||||
<y>
|
||||
<!-- in y before -->
|
||||
<d><!-- in d before -->4<!-- in d after --></d>
|
||||
<!-- in y after -->
|
||||
</y>
|
||||
<!-- in_cat_after -->
|
||||
</cat>
|
||||
<!-- after cat -->
|
||||
@@ -1,3 +0,0 @@
|
||||
name,numberOfCats,likesApples,height
|
||||
Gary,1,true,168.8
|
||||
Samantha's Rabbit,2,false,-188.8
|
||||
|
@@ -2,8 +2,7 @@ FROM mikefarah/yq:4
|
||||
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
|
||||
# github action recommendation is to run as root.
|
||||
# https://docs.github.com/en/actions/creating-actions/dockerfile-support-for-github-actions#user
|
||||
USER root
|
||||
# this seems to be the default user in github actions
|
||||
USER 1001
|
||||
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
|
||||
@@ -2,11 +2,8 @@
|
||||
set -e
|
||||
echo "::debug::\$cmd: $1"
|
||||
RESULT=$(eval "$1")
|
||||
RESULT="${RESULT//'%'/'%25'}"
|
||||
RESULT="${RESULT//$'\n'/'%0A'}"
|
||||
RESULT="${RESULT//$'\r'/'%0D'}"
|
||||
echo "::debug::\$RESULT: $RESULT"
|
||||
# updating from
|
||||
# https://github.com/orgs/community/discussions/26288#discussioncomment-3876281
|
||||
# https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter
|
||||
delimiter=$(cat /proc/sys/kernel/random/uuid)
|
||||
echo "result<<${delimiter}" >> "${GITHUB_OUTPUT}"
|
||||
echo "${RESULT}" >> "${GITHUB_OUTPUT}"
|
||||
echo "${delimiter}" >> "${GITHUB_OUTPUT}"
|
||||
echo ::set-output name=result::"$RESULT"
|
||||
|
||||
@@ -2,24 +2,22 @@ module github.com/mikefarah/yq/v4
|
||||
|
||||
require (
|
||||
github.com/a8m/envsubst v1.3.0
|
||||
github.com/alecthomas/participle/v2 v2.0.0-beta.5
|
||||
github.com/alecthomas/repr v0.1.1
|
||||
github.com/dimchansky/utfbom v1.1.1
|
||||
github.com/elliotchance/orderedmap v1.5.0
|
||||
github.com/alecthomas/participle/v2 v2.0.0-beta.4
|
||||
github.com/alecthomas/repr v0.1.0
|
||||
github.com/elliotchance/orderedmap v1.4.0
|
||||
github.com/fatih/color v1.13.0
|
||||
github.com/goccy/go-json v0.9.11
|
||||
github.com/goccy/go-yaml v1.9.5
|
||||
github.com/jinzhu/copier v0.3.5
|
||||
github.com/magiconair/properties v1.8.6
|
||||
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e
|
||||
github.com/spf13/cobra v1.6.0
|
||||
github.com/spf13/cobra v1.5.0
|
||||
golang.org/x/net v0.0.0-20220708220712-1185a9018129
|
||||
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/inconshreveable/mousetrap v1.0.1 // indirect
|
||||
github.com/inconshreveable/mousetrap v1.0.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.12 // indirect
|
||||
github.com/mattn/go-isatty v0.0.14 // indirect
|
||||
github.com/spf13/pflag v1.0.5 // indirect
|
||||
@@ -28,4 +26,4 @@ require (
|
||||
golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f // indirect
|
||||
)
|
||||
|
||||
go 1.19
|
||||
go 1.18
|
||||
|
||||
@@ -1,17 +1,15 @@
|
||||
github.com/a8m/envsubst v1.3.0 h1:GmXKmVssap0YtlU3E230W98RWtWCyIZzjtf1apWWyAg=
|
||||
github.com/a8m/envsubst v1.3.0/go.mod h1:MVUTQNGQ3tsjOOtKCNd+fl8RzhsXcDvvAEzkhGtlsbY=
|
||||
github.com/alecthomas/assert/v2 v2.0.3 h1:WKqJODfOiQG0nEJKFKzDIG3E29CN2/4zR9XGJzKIkbg=
|
||||
github.com/alecthomas/participle/v2 v2.0.0-beta.5 h1:y6dsSYVb1G5eK6mgmy+BgI3Mw35a3WghArZ/Hbebrjo=
|
||||
github.com/alecthomas/participle/v2 v2.0.0-beta.5/go.mod h1:RC764t6n4L8D8ITAJv0qdokritYSNR3wV5cVwmIEaMM=
|
||||
github.com/alecthomas/repr v0.1.1 h1:87P60cSmareLAxMc4Hro0r2RBY4ROm0dYwkJNpS4pPs=
|
||||
github.com/alecthomas/repr v0.1.1/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
|
||||
github.com/alecthomas/participle/v2 v2.0.0-beta.4 h1:ublfGBm+x+p2j7KotHhrUMbKtejT7M0Gv1Mt1u3absw=
|
||||
github.com/alecthomas/participle/v2 v2.0.0-beta.4/go.mod h1:RC764t6n4L8D8ITAJv0qdokritYSNR3wV5cVwmIEaMM=
|
||||
github.com/alecthomas/repr v0.1.0 h1:ENn2e1+J3k09gyj2shc0dHr/yjaWSHRlrJ4DPMevDqE=
|
||||
github.com/alecthomas/repr v0.1.0/go.mod h1:2kn6fqh/zIyPLmm3ugklbEi5hg5wS435eygvNfaDQL8=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
|
||||
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dimchansky/utfbom v1.1.1 h1:vV6w1AhK4VMnhBno/TPVCoK9U/LP0PkLCS9tbxHdi/U=
|
||||
github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE=
|
||||
github.com/elliotchance/orderedmap v1.5.0 h1:1IsExUsjv5XNBD3ZdC7jkAAqLWOOKdbPTmkHx63OsBg=
|
||||
github.com/elliotchance/orderedmap v1.5.0/go.mod h1:wsDwEaX5jEoyhbs7x93zk2H/qv0zwuhg4inXhDkYqys=
|
||||
github.com/elliotchance/orderedmap v1.4.0 h1:wZtfeEONCbx6in1CZyE6bELEt/vFayMvsxqI5SgsR+A=
|
||||
github.com/elliotchance/orderedmap v1.4.0/go.mod h1:wsDwEaX5jEoyhbs7x93zk2H/qv0zwuhg4inXhDkYqys=
|
||||
github.com/fatih/color v1.10.0/go.mod h1:ELkj/draVOlAH/xkhN6mQ50Qd0MPOk5AAr3maGEBuJM=
|
||||
github.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w=
|
||||
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
|
||||
@@ -19,13 +17,11 @@ github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvSc
|
||||
github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
|
||||
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
|
||||
github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
|
||||
github.com/goccy/go-json v0.9.11 h1:/pAaQDLHEoCq/5FFmSKBswWmK6H0e8g4159Kc/X/nqk=
|
||||
github.com/goccy/go-json v0.9.11/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
|
||||
github.com/goccy/go-yaml v1.9.5 h1:Eh/+3uk9kLxG4koCX6lRMAPS1OaMSAi+FJcya0INdB0=
|
||||
github.com/goccy/go-yaml v1.9.5/go.mod h1:U/jl18uSupI5rdI2jmuCswEA2htH9eXfferR3KfscvA=
|
||||
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
|
||||
github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
|
||||
github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
|
||||
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
|
||||
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
|
||||
github.com/jinzhu/copier v0.3.5 h1:GlvfUwHk62RokgqVNvYsku0TATCF7bAHVwEXoBh3iJg=
|
||||
github.com/jinzhu/copier v0.3.5/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg=
|
||||
github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
|
||||
@@ -43,8 +39,8 @@ github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsK
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
|
||||
github.com/spf13/cobra v1.6.0 h1:42a0n6jwCot1pUmomAp4T7DeMD+20LFv4Q54pxLf2LI=
|
||||
github.com/spf13/cobra v1.6.0/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY=
|
||||
github.com/spf13/cobra v1.5.0 h1:X+jTBEBqF0bHN+9cSMgmfuvv2VHJ9ezmFNf9Y/XstYU=
|
||||
github.com/spf13/cobra v1.5.0/go.mod h1:dWXEIy2H428czQCjInthrTRUg7yKbok+2Qi/yBIJoUM=
|
||||
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
|
||||
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
@@ -77,6 +73,7 @@ gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8
|
||||
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473 h1:6D+BvnJ/j6e222UW8s2qTSe3wGBtvo0MbVQG/c5k8RE=
|
||||
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473/go.mod h1:N1eN2tsCx0Ydtgjl4cqmbRCsY4/+z4cYDeqwZTk6zog=
|
||||
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
||||
+1
-1
@@ -37,7 +37,7 @@ The `=` operator then pipes the 'root' context through the `rhs` expression of `
|
||||
dog
|
||||
```
|
||||
|
||||
Both sides have now been evaluated, so now the operator copies across the value from the RHS (`.b`) to the LHS (`.a`), and it returns the now updated context:
|
||||
Both sides have now been evaluated, so now the operator copies across the value from the RHS (`.b`) to the the LHS (`.a`), and it returns the now updated context:
|
||||
|
||||
```yaml
|
||||
a: dog
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
|
||||
// A yaml expression evaluator that runs the expression once against all files/nodes in memory.
|
||||
type Evaluator interface {
|
||||
EvaluateFiles(expression string, filenames []string, printer Printer, decoder Decoder) error
|
||||
EvaluateFiles(expression string, filenames []string, printer Printer, leadingContentPreProcessing bool, decoder Decoder) error
|
||||
|
||||
// EvaluateNodes takes an expression and one or more yaml nodes, returning a list of matching candidate nodes
|
||||
EvaluateNodes(expression string, nodes ...*yaml.Node) (*list.List, error)
|
||||
@@ -46,16 +46,21 @@ func (e *allAtOnceEvaluator) EvaluateCandidateNodes(expression string, inputCand
|
||||
return context.MatchingNodes, nil
|
||||
}
|
||||
|
||||
func (e *allAtOnceEvaluator) EvaluateFiles(expression string, filenames []string, printer Printer, decoder Decoder) error {
|
||||
func (e *allAtOnceEvaluator) EvaluateFiles(expression string, filenames []string, printer Printer, leadingContentPreProcessing bool, decoder Decoder) error {
|
||||
fileIndex := 0
|
||||
firstFileLeadingContent := ""
|
||||
|
||||
var allDocuments = list.New()
|
||||
for _, filename := range filenames {
|
||||
reader, err := readStream(filename)
|
||||
reader, leadingContent, err := readStream(filename, fileIndex == 0 && leadingContentPreProcessing)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if fileIndex == 0 {
|
||||
firstFileLeadingContent = leadingContent
|
||||
}
|
||||
|
||||
fileDocuments, err := readDocuments(reader, filename, fileIndex, decoder)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -70,9 +75,11 @@ func (e *allAtOnceEvaluator) EvaluateFiles(expression string, filenames []string
|
||||
Filename: "",
|
||||
Node: &yaml.Node{Kind: yaml.DocumentNode, Content: []*yaml.Node{{Tag: "!!null", Kind: yaml.ScalarNode}}},
|
||||
FileIndex: 0,
|
||||
LeadingContent: "",
|
||||
LeadingContent: firstFileLeadingContent,
|
||||
}
|
||||
allDocuments.PushBack(candidateNode)
|
||||
} else {
|
||||
allDocuments.Front().Value.(*CandidateNode).LeadingContent = firstFileLeadingContent
|
||||
}
|
||||
|
||||
matches, err := e.EvaluateCandidateNodes(expression, allDocuments)
|
||||
|
||||
@@ -157,7 +157,7 @@ func (n *CandidateNode) UpdateAttributesFrom(other *CandidateNode, prefs assignP
|
||||
n.Node.Kind = other.Node.Kind
|
||||
|
||||
// don't clobber custom tags...
|
||||
if prefs.ClobberCustomTags || strings.HasPrefix(n.Node.Tag, "!!") || n.Node.Tag == "" {
|
||||
if strings.HasPrefix(n.Node.Tag, "!!") || n.Node.Tag == "" {
|
||||
n.Node.Tag = other.Node.Tag
|
||||
}
|
||||
|
||||
|
||||
@@ -1,273 +0,0 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/mikefarah/yq/v4/test"
|
||||
)
|
||||
|
||||
const csvSimple = `name,numberOfCats,likesApples,height
|
||||
Gary,1,true,168.8
|
||||
Samantha's Rabbit,2,false,-188.8
|
||||
`
|
||||
|
||||
const expectedUpdatedSimpleCsv = `name,numberOfCats,likesApples,height
|
||||
Gary,3,true,168.8
|
||||
Samantha's Rabbit,2,false,-188.8
|
||||
`
|
||||
|
||||
const csvSimpleShort = `Name,Number of Cats
|
||||
Gary,1
|
||||
Samantha's Rabbit,2
|
||||
`
|
||||
|
||||
const tsvSimple = `name numberOfCats likesApples height
|
||||
Gary 1 true 168.8
|
||||
Samantha's Rabbit 2 false -188.8
|
||||
`
|
||||
|
||||
const expectedYamlFromCSV = `- name: Gary
|
||||
numberOfCats: 1
|
||||
likesApples: true
|
||||
height: 168.8
|
||||
- name: Samantha's Rabbit
|
||||
numberOfCats: 2
|
||||
likesApples: false
|
||||
height: -188.8
|
||||
`
|
||||
|
||||
const expectedYamlFromCSVMissingData = `- name: Gary
|
||||
numberOfCats: 1
|
||||
height: 168.8
|
||||
- name: Samantha's Rabbit
|
||||
height: -188.8
|
||||
likesApples: false
|
||||
`
|
||||
|
||||
const csvSimpleMissingData = `name,numberOfCats,height
|
||||
Gary,1,168.8
|
||||
Samantha's Rabbit,,-188.8
|
||||
`
|
||||
|
||||
const csvTestSimpleYaml = `- [i, like, csv]
|
||||
- [because, excel, is, cool]`
|
||||
|
||||
const expectedSimpleCsv = `i,like,csv
|
||||
because,excel,is,cool
|
||||
`
|
||||
|
||||
const tsvTestExpectedSimpleCsv = `i like csv
|
||||
because excel is cool
|
||||
`
|
||||
|
||||
var csvScenarios = []formatScenario{
|
||||
{
|
||||
description: "Encode CSV simple",
|
||||
input: csvTestSimpleYaml,
|
||||
expected: expectedSimpleCsv,
|
||||
scenarioType: "encode-csv",
|
||||
},
|
||||
{
|
||||
description: "Encode TSV simple",
|
||||
input: csvTestSimpleYaml,
|
||||
expected: tsvTestExpectedSimpleCsv,
|
||||
scenarioType: "encode-tsv",
|
||||
},
|
||||
{
|
||||
description: "Encode Empty",
|
||||
skipDoc: true,
|
||||
input: `[]`,
|
||||
expected: "",
|
||||
scenarioType: "encode-csv",
|
||||
},
|
||||
{
|
||||
description: "Comma in value",
|
||||
skipDoc: true,
|
||||
input: `["comma, in, value", things]`,
|
||||
expected: "\"comma, in, value\",things\n",
|
||||
scenarioType: "encode-csv",
|
||||
},
|
||||
{
|
||||
description: "Encode array of objects to csv",
|
||||
input: expectedYamlFromCSV,
|
||||
expected: csvSimple,
|
||||
scenarioType: "encode-csv",
|
||||
},
|
||||
{
|
||||
description: "Encode array of objects to custom csv format",
|
||||
subdescription: "Add the header row manually, then the we convert each object into an array of values - resulting in an array of arrays. Pick the columns and call the header whatever you like.",
|
||||
input: expectedYamlFromCSV,
|
||||
expected: csvSimpleShort,
|
||||
expression: `[["Name", "Number of Cats"]] + [.[] | [.name, .numberOfCats ]]`,
|
||||
scenarioType: "encode-csv",
|
||||
},
|
||||
{
|
||||
description: "Encode array of objects to csv - missing fields behaviour",
|
||||
subdescription: "First entry is used to determine the headers, and it is missing 'likesApples', so it is not included in the csv. Second entry does not have 'numberOfCats' so that is blank",
|
||||
input: expectedYamlFromCSVMissingData,
|
||||
expected: csvSimpleMissingData,
|
||||
scenarioType: "encode-csv",
|
||||
},
|
||||
{
|
||||
description: "Parse CSV into an array of objects",
|
||||
subdescription: "First row is assumed to be the header row.",
|
||||
input: csvSimple,
|
||||
expected: expectedYamlFromCSV,
|
||||
scenarioType: "decode-csv-object",
|
||||
},
|
||||
{
|
||||
description: "Parse TSV into an array of objects",
|
||||
subdescription: "First row is assumed to be the header row.",
|
||||
input: tsvSimple,
|
||||
expected: expectedYamlFromCSV,
|
||||
scenarioType: "decode-tsv-object",
|
||||
},
|
||||
{
|
||||
description: "Round trip",
|
||||
input: csvSimple,
|
||||
expected: expectedUpdatedSimpleCsv,
|
||||
expression: `(.[] | select(.name == "Gary") | .numberOfCats) = 3`,
|
||||
scenarioType: "roundtrip-csv",
|
||||
},
|
||||
}
|
||||
|
||||
func testCSVScenario(t *testing.T, s formatScenario) {
|
||||
switch s.scenarioType {
|
||||
case "encode-csv":
|
||||
test.AssertResultWithContext(t, s.expected, processFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewCsvEncoder(',')), s.description)
|
||||
case "encode-tsv":
|
||||
test.AssertResultWithContext(t, s.expected, processFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewCsvEncoder('\t')), s.description)
|
||||
case "decode-csv-object":
|
||||
test.AssertResultWithContext(t, s.expected, processFormatScenario(s, NewCSVObjectDecoder(','), NewYamlEncoder(2, false, ConfiguredYamlPreferences)), s.description)
|
||||
case "decode-tsv-object":
|
||||
test.AssertResultWithContext(t, s.expected, processFormatScenario(s, NewCSVObjectDecoder('\t'), NewYamlEncoder(2, false, ConfiguredYamlPreferences)), s.description)
|
||||
case "roundtrip-csv":
|
||||
test.AssertResultWithContext(t, s.expected, processFormatScenario(s, NewCSVObjectDecoder(','), NewCsvEncoder(',')), s.description)
|
||||
default:
|
||||
panic(fmt.Sprintf("unhandled scenario type %q", s.scenarioType))
|
||||
}
|
||||
}
|
||||
|
||||
func documentCSVDecodeObjectScenario(w *bufio.Writer, s formatScenario, formatType string) {
|
||||
writeOrPanic(w, fmt.Sprintf("## %v\n", s.description))
|
||||
|
||||
if s.subdescription != "" {
|
||||
writeOrPanic(w, s.subdescription)
|
||||
writeOrPanic(w, "\n\n")
|
||||
}
|
||||
|
||||
writeOrPanic(w, fmt.Sprintf("Given a sample.%v file of:\n", formatType))
|
||||
writeOrPanic(w, fmt.Sprintf("```%v\n%v\n```\n", formatType, s.input))
|
||||
|
||||
writeOrPanic(w, "then\n")
|
||||
writeOrPanic(w, fmt.Sprintf("```bash\nyq -p=%v sample.%v\n```\n", formatType, formatType))
|
||||
writeOrPanic(w, "will output\n")
|
||||
|
||||
separator := ','
|
||||
if formatType == "tsv" {
|
||||
separator = '\t'
|
||||
}
|
||||
|
||||
writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n\n",
|
||||
processFormatScenario(s, NewCSVObjectDecoder(separator), NewYamlEncoder(s.indent, false, ConfiguredYamlPreferences))),
|
||||
)
|
||||
}
|
||||
|
||||
func documentCSVEncodeScenario(w *bufio.Writer, s formatScenario, formatType string) {
|
||||
writeOrPanic(w, fmt.Sprintf("## %v\n", s.description))
|
||||
|
||||
if s.subdescription != "" {
|
||||
writeOrPanic(w, s.subdescription)
|
||||
writeOrPanic(w, "\n\n")
|
||||
}
|
||||
|
||||
writeOrPanic(w, "Given a sample.yml file of:\n")
|
||||
writeOrPanic(w, fmt.Sprintf("```yaml\n%v\n```\n", s.input))
|
||||
|
||||
writeOrPanic(w, "then\n")
|
||||
|
||||
expression := s.expression
|
||||
|
||||
if expression != "" {
|
||||
writeOrPanic(w, fmt.Sprintf("```bash\nyq -o=%v '%v' sample.yml\n```\n", formatType, expression))
|
||||
} else {
|
||||
writeOrPanic(w, fmt.Sprintf("```bash\nyq -o=%v sample.yml\n```\n", formatType))
|
||||
}
|
||||
writeOrPanic(w, "will output\n")
|
||||
|
||||
separator := ','
|
||||
if formatType == "tsv" {
|
||||
separator = '\t'
|
||||
}
|
||||
|
||||
writeOrPanic(w, fmt.Sprintf("```%v\n%v```\n\n", formatType,
|
||||
processFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewCsvEncoder(separator))),
|
||||
)
|
||||
}
|
||||
|
||||
func documentCSVRoundTripScenario(w *bufio.Writer, s formatScenario, formatType string) {
|
||||
writeOrPanic(w, fmt.Sprintf("## %v\n", s.description))
|
||||
|
||||
if s.subdescription != "" {
|
||||
writeOrPanic(w, s.subdescription)
|
||||
writeOrPanic(w, "\n\n")
|
||||
}
|
||||
|
||||
writeOrPanic(w, fmt.Sprintf("Given a sample.%v file of:\n", formatType))
|
||||
writeOrPanic(w, fmt.Sprintf("```%v\n%v\n```\n", formatType, s.input))
|
||||
|
||||
writeOrPanic(w, "then\n")
|
||||
|
||||
expression := s.expression
|
||||
|
||||
if expression != "" {
|
||||
writeOrPanic(w, fmt.Sprintf("```bash\nyq -p=%v -o=%v '%v' sample.%v\n```\n", formatType, formatType, expression, formatType))
|
||||
} else {
|
||||
writeOrPanic(w, fmt.Sprintf("```bash\nyq -p=%v -o=%v sample.%v\n```\n", formatType, formatType, formatType))
|
||||
}
|
||||
writeOrPanic(w, "will output\n")
|
||||
|
||||
separator := ','
|
||||
if formatType == "tsv" {
|
||||
separator = '\t'
|
||||
}
|
||||
|
||||
writeOrPanic(w, fmt.Sprintf("```%v\n%v```\n\n", formatType,
|
||||
processFormatScenario(s, NewCSVObjectDecoder(separator), NewCsvEncoder(separator))),
|
||||
)
|
||||
}
|
||||
|
||||
func documentCSVScenario(t *testing.T, w *bufio.Writer, i interface{}) {
|
||||
s := i.(formatScenario)
|
||||
if s.skipDoc {
|
||||
return
|
||||
}
|
||||
switch s.scenarioType {
|
||||
case "encode-csv":
|
||||
documentCSVEncodeScenario(w, s, "csv")
|
||||
case "encode-tsv":
|
||||
documentCSVEncodeScenario(w, s, "tsv")
|
||||
case "decode-csv-object":
|
||||
documentCSVDecodeObjectScenario(w, s, "csv")
|
||||
case "decode-tsv-object":
|
||||
documentCSVDecodeObjectScenario(w, s, "tsv")
|
||||
case "roundtrip-csv":
|
||||
documentCSVRoundTripScenario(w, s, "csv")
|
||||
|
||||
default:
|
||||
panic(fmt.Sprintf("unhandled scenario type %q", s.scenarioType))
|
||||
}
|
||||
}
|
||||
|
||||
func TestCSVScenarios(t *testing.T) {
|
||||
for _, tt := range csvScenarios {
|
||||
testCSVScenario(t, tt)
|
||||
}
|
||||
genericScenarios := make([]interface{}, len(csvScenarios))
|
||||
for i, s := range csvScenarios {
|
||||
genericScenarios[i] = s
|
||||
}
|
||||
documentScenarios(t, "usage", "csv-tsv", genericScenarios, documentCSVScenario)
|
||||
}
|
||||
+4
-11
@@ -3,6 +3,8 @@ package yqlib
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
type InputFormat uint
|
||||
@@ -12,14 +14,11 @@ const (
|
||||
XMLInputFormat
|
||||
PropertiesInputFormat
|
||||
Base64InputFormat
|
||||
JsonInputFormat
|
||||
CSVObjectInputFormat
|
||||
TSVObjectInputFormat
|
||||
)
|
||||
|
||||
type Decoder interface {
|
||||
Init(reader io.Reader) error
|
||||
Decode() (*CandidateNode, error)
|
||||
Init(reader io.Reader)
|
||||
Decode(node *yaml.Node) error
|
||||
}
|
||||
|
||||
func InputFormatFromString(format string) (InputFormat, error) {
|
||||
@@ -30,12 +29,6 @@ func InputFormatFromString(format string) (InputFormat, error) {
|
||||
return XMLInputFormat, nil
|
||||
case "props", "p":
|
||||
return PropertiesInputFormat, nil
|
||||
case "json", "ndjson", "j":
|
||||
return JsonInputFormat, nil
|
||||
case "csv", "c":
|
||||
return CSVObjectInputFormat, nil
|
||||
case "tsv", "t":
|
||||
return TSVObjectInputFormat, nil
|
||||
default:
|
||||
return 0, fmt.Errorf("unknown format '%v' please use [yaml|xml|props]", format)
|
||||
}
|
||||
|
||||
@@ -19,22 +19,21 @@ func NewBase64Decoder() Decoder {
|
||||
return &base64Decoder{finished: false, encoding: *base64.StdEncoding}
|
||||
}
|
||||
|
||||
func (dec *base64Decoder) Init(reader io.Reader) error {
|
||||
func (dec *base64Decoder) Init(reader io.Reader) {
|
||||
dec.reader = reader
|
||||
dec.readAnything = false
|
||||
dec.finished = false
|
||||
return nil
|
||||
}
|
||||
|
||||
func (dec *base64Decoder) Decode() (*CandidateNode, error) {
|
||||
func (dec *base64Decoder) Decode(rootYamlNode *yaml.Node) error {
|
||||
if dec.finished {
|
||||
return nil, io.EOF
|
||||
return io.EOF
|
||||
}
|
||||
base64Reader := base64.NewDecoder(&dec.encoding, dec.reader)
|
||||
buf := new(bytes.Buffer)
|
||||
|
||||
if _, err := buf.ReadFrom(base64Reader); err != nil {
|
||||
return nil, err
|
||||
return err
|
||||
}
|
||||
if buf.Len() == 0 {
|
||||
dec.finished = true
|
||||
@@ -43,15 +42,12 @@ func (dec *base64Decoder) Decode() (*CandidateNode, error) {
|
||||
// otherwise if we've already read some bytes, and now we get
|
||||
// an empty string, then we are done.
|
||||
if dec.readAnything {
|
||||
return nil, io.EOF
|
||||
return io.EOF
|
||||
}
|
||||
}
|
||||
dec.readAnything = true
|
||||
return &CandidateNode{
|
||||
Node: &yaml.Node{
|
||||
Kind: yaml.ScalarNode,
|
||||
Tag: "!!str",
|
||||
Value: buf.String(),
|
||||
},
|
||||
}, nil
|
||||
rootYamlNode.Kind = yaml.ScalarNode
|
||||
rootYamlNode.Tag = "!!str"
|
||||
rootYamlNode.Value = buf.String()
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -1,81 +0,0 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"encoding/csv"
|
||||
"errors"
|
||||
"io"
|
||||
|
||||
"github.com/dimchansky/utfbom"
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
type csvObjectDecoder struct {
|
||||
separator rune
|
||||
reader csv.Reader
|
||||
finished bool
|
||||
}
|
||||
|
||||
func NewCSVObjectDecoder(separator rune) Decoder {
|
||||
return &csvObjectDecoder{separator: separator}
|
||||
}
|
||||
|
||||
func (dec *csvObjectDecoder) Init(reader io.Reader) error {
|
||||
cleanReader, enc := utfbom.Skip(reader)
|
||||
log.Debugf("Detected encoding: %s\n", enc)
|
||||
dec.reader = *csv.NewReader(cleanReader)
|
||||
dec.reader.Comma = dec.separator
|
||||
dec.finished = false
|
||||
return nil
|
||||
}
|
||||
|
||||
func (dec *csvObjectDecoder) convertToYamlNode(content string) *yaml.Node {
|
||||
node, err := parseSnippet(content)
|
||||
if err != nil {
|
||||
return createScalarNode(content, content)
|
||||
}
|
||||
return node
|
||||
}
|
||||
|
||||
func (dec *csvObjectDecoder) createObject(headerRow []string, contentRow []string) *yaml.Node {
|
||||
objectNode := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"}
|
||||
|
||||
for i, header := range headerRow {
|
||||
objectNode.Content = append(
|
||||
objectNode.Content,
|
||||
createScalarNode(header, header),
|
||||
dec.convertToYamlNode(contentRow[i]))
|
||||
}
|
||||
return objectNode
|
||||
}
|
||||
|
||||
func (dec *csvObjectDecoder) Decode() (*CandidateNode, error) {
|
||||
if dec.finished {
|
||||
return nil, io.EOF
|
||||
}
|
||||
headerRow, err := dec.reader.Read()
|
||||
log.Debugf(": headerRow%v", headerRow)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rootArray := &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
||||
|
||||
contentRow, err := dec.reader.Read()
|
||||
|
||||
for err == nil && len(contentRow) > 0 {
|
||||
log.Debugf("Adding contentRow: %v", contentRow)
|
||||
rootArray.Content = append(rootArray.Content, dec.createObject(headerRow, contentRow))
|
||||
contentRow, err = dec.reader.Read()
|
||||
log.Debugf("Read next contentRow: %v, %v", contentRow, err)
|
||||
}
|
||||
if !errors.Is(err, io.EOF) {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CandidateNode{
|
||||
Node: &yaml.Node{
|
||||
Kind: yaml.DocumentNode,
|
||||
Content: []*yaml.Node{rootArray},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
@@ -1,87 +0,0 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
|
||||
"github.com/goccy/go-json"
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
type jsonDecoder struct {
|
||||
decoder json.Decoder
|
||||
}
|
||||
|
||||
func NewJSONDecoder() Decoder {
|
||||
return &jsonDecoder{}
|
||||
}
|
||||
|
||||
func (dec *jsonDecoder) Init(reader io.Reader) error {
|
||||
dec.decoder = *json.NewDecoder(reader)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (dec *jsonDecoder) Decode() (*CandidateNode, error) {
|
||||
|
||||
var dataBucket orderedMap
|
||||
log.Debug("going to decode")
|
||||
err := dec.decoder.Decode(&dataBucket)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
node, err := dec.convertToYamlNode(&dataBucket)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &CandidateNode{
|
||||
Node: &yaml.Node{
|
||||
Kind: yaml.DocumentNode,
|
||||
Content: []*yaml.Node{node},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (dec *jsonDecoder) convertToYamlNode(data *orderedMap) (*yaml.Node, error) {
|
||||
if data.kv == nil {
|
||||
switch rawData := data.altVal.(type) {
|
||||
case nil:
|
||||
return createScalarNode(nil, "null"), nil
|
||||
case float64, float32:
|
||||
// json decoder returns ints as float.
|
||||
return parseSnippet(fmt.Sprintf("%v", rawData))
|
||||
case int, int64, int32, string, bool:
|
||||
return createScalarNode(rawData, fmt.Sprintf("%v", rawData)), nil
|
||||
case []*orderedMap:
|
||||
return dec.parseArray(rawData)
|
||||
default:
|
||||
return nil, fmt.Errorf("unrecognised type :( %v", rawData)
|
||||
}
|
||||
}
|
||||
|
||||
var yamlMap = &yaml.Node{Kind: yaml.MappingNode}
|
||||
for _, keyValuePair := range data.kv {
|
||||
yamlValue, err := dec.convertToYamlNode(&keyValuePair.V)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
yamlMap.Content = append(yamlMap.Content, createScalarNode(keyValuePair.K, keyValuePair.K), yamlValue)
|
||||
}
|
||||
return yamlMap, nil
|
||||
|
||||
}
|
||||
|
||||
func (dec *jsonDecoder) parseArray(dataArray []*orderedMap) (*yaml.Node, error) {
|
||||
|
||||
var yamlMap = &yaml.Node{Kind: yaml.SequenceNode}
|
||||
|
||||
for _, value := range dataArray {
|
||||
yamlValue, err := dec.convertToYamlNode(value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
yamlMap.Content = append(yamlMap.Content, yamlValue)
|
||||
}
|
||||
return yamlMap, nil
|
||||
}
|
||||
@@ -2,7 +2,6 @@ package yqlib
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -21,10 +20,9 @@ func NewPropertiesDecoder() Decoder {
|
||||
return &propertiesDecoder{d: NewDataTreeNavigator(), finished: false}
|
||||
}
|
||||
|
||||
func (dec *propertiesDecoder) Init(reader io.Reader) error {
|
||||
func (dec *propertiesDecoder) Init(reader io.Reader) {
|
||||
dec.reader = reader
|
||||
dec.finished = false
|
||||
return nil
|
||||
}
|
||||
|
||||
func parsePropKey(key string) []interface{} {
|
||||
@@ -48,49 +46,15 @@ func (dec *propertiesDecoder) processComment(c string) string {
|
||||
return "# " + c
|
||||
}
|
||||
|
||||
func (dec *propertiesDecoder) applyPropertyComments(context Context, path []interface{}, comments []string) error {
|
||||
assignmentOp := &Operation{OperationType: assignOpType, Preferences: assignPreferences{}}
|
||||
|
||||
rhsCandidateNode := &CandidateNode{
|
||||
Path: path,
|
||||
Node: &yaml.Node{
|
||||
Tag: "!!str",
|
||||
Value: fmt.Sprintf("%v", path[len(path)-1]),
|
||||
HeadComment: dec.processComment(strings.Join(comments, "\n")),
|
||||
Kind: yaml.ScalarNode,
|
||||
},
|
||||
}
|
||||
|
||||
rhsCandidateNode.Node.Tag = guessTagFromCustomType(rhsCandidateNode.Node)
|
||||
|
||||
rhsOp := &Operation{OperationType: valueOpType, CandidateNode: rhsCandidateNode}
|
||||
|
||||
assignmentOpNode := &ExpressionNode{
|
||||
Operation: assignmentOp,
|
||||
LHS: createTraversalTree(path, traversePreferences{}, true),
|
||||
RHS: &ExpressionNode{Operation: rhsOp},
|
||||
}
|
||||
|
||||
_, err := dec.d.GetMatchingNodes(context, assignmentOpNode)
|
||||
return err
|
||||
}
|
||||
|
||||
func (dec *propertiesDecoder) applyProperty(context Context, properties *properties.Properties, key string) error {
|
||||
func (dec *propertiesDecoder) applyProperty(properties *properties.Properties, context Context, key string) error {
|
||||
value, _ := properties.Get(key)
|
||||
path := parsePropKey(key)
|
||||
|
||||
propertyComments := properties.GetComments(key)
|
||||
if len(propertyComments) > 0 {
|
||||
err := dec.applyPropertyComments(context, path, propertyComments)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
rhsNode := &yaml.Node{
|
||||
Value: value,
|
||||
Tag: "!!str",
|
||||
Kind: yaml.ScalarNode,
|
||||
Value: value,
|
||||
Tag: "!!str",
|
||||
Kind: yaml.ScalarNode,
|
||||
LineComment: dec.processComment(properties.GetComment(key)),
|
||||
}
|
||||
|
||||
rhsNode.Tag = guessTagFromCustomType(rhsNode)
|
||||
@@ -114,22 +78,22 @@ func (dec *propertiesDecoder) applyProperty(context Context, properties *propert
|
||||
return err
|
||||
}
|
||||
|
||||
func (dec *propertiesDecoder) Decode() (*CandidateNode, error) {
|
||||
func (dec *propertiesDecoder) Decode(rootYamlNode *yaml.Node) error {
|
||||
if dec.finished {
|
||||
return nil, io.EOF
|
||||
return io.EOF
|
||||
}
|
||||
buf := new(bytes.Buffer)
|
||||
|
||||
if _, err := buf.ReadFrom(dec.reader); err != nil {
|
||||
return nil, err
|
||||
return err
|
||||
}
|
||||
if buf.Len() == 0 {
|
||||
dec.finished = true
|
||||
return nil, io.EOF
|
||||
return io.EOF
|
||||
}
|
||||
properties, err := properties.LoadString(buf.String())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return err
|
||||
}
|
||||
properties.DisableExpansion = true
|
||||
|
||||
@@ -144,18 +108,15 @@ func (dec *propertiesDecoder) Decode() (*CandidateNode, error) {
|
||||
context = context.SingleChildContext(rootMap)
|
||||
|
||||
for _, key := range properties.Keys() {
|
||||
if err := dec.applyProperty(context, properties, key); err != nil {
|
||||
return nil, err
|
||||
if err := dec.applyProperty(properties, context, key); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
}
|
||||
dec.finished = true
|
||||
|
||||
return &CandidateNode{
|
||||
Node: &yaml.Node{
|
||||
Kind: yaml.DocumentNode,
|
||||
Content: []*yaml.Node{rootMap.Node},
|
||||
},
|
||||
}, nil
|
||||
rootYamlNode.Kind = yaml.DocumentNode
|
||||
rootYamlNode.Content = []*yaml.Node{rootMap.Node}
|
||||
dec.finished = true
|
||||
return nil
|
||||
|
||||
}
|
||||
|
||||
@@ -23,7 +23,7 @@ func processFormatScenario(s formatScenario, decoder Decoder, encoder Encoder) s
|
||||
writer := bufio.NewWriter(&output)
|
||||
|
||||
if decoder == nil {
|
||||
decoder = NewYamlDecoder(ConfiguredYamlPreferences)
|
||||
decoder = NewYamlDecoder()
|
||||
}
|
||||
|
||||
inputs, err := readDocuments(strings.NewReader(s.input), "sample.yml", 0, decoder)
|
||||
|
||||
+36
-36
@@ -12,28 +12,38 @@ import (
|
||||
)
|
||||
|
||||
type xmlDecoder struct {
|
||||
reader io.Reader
|
||||
readAnything bool
|
||||
finished bool
|
||||
prefs XmlPreferences
|
||||
reader io.Reader
|
||||
readAnything bool
|
||||
attributePrefix string
|
||||
contentName string
|
||||
strictMode bool
|
||||
keepNamespace bool
|
||||
useRawToken bool
|
||||
finished bool
|
||||
}
|
||||
|
||||
func NewXMLDecoder(prefs XmlPreferences) Decoder {
|
||||
func NewXMLDecoder(attributePrefix string, contentName string, strictMode bool, keepNamespace bool, useRawToken bool) Decoder {
|
||||
if contentName == "" {
|
||||
contentName = "content"
|
||||
}
|
||||
return &xmlDecoder{
|
||||
finished: false,
|
||||
prefs: prefs,
|
||||
attributePrefix: attributePrefix,
|
||||
contentName: contentName,
|
||||
finished: false,
|
||||
strictMode: strictMode,
|
||||
keepNamespace: keepNamespace,
|
||||
useRawToken: useRawToken,
|
||||
}
|
||||
}
|
||||
|
||||
func (dec *xmlDecoder) Init(reader io.Reader) error {
|
||||
func (dec *xmlDecoder) Init(reader io.Reader) {
|
||||
dec.reader = reader
|
||||
dec.readAnything = false
|
||||
dec.finished = false
|
||||
return nil
|
||||
}
|
||||
|
||||
func (dec *xmlDecoder) createSequence(nodes []*xmlNode) (*yaml.Node, error) {
|
||||
yamlNode := &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
||||
yamlNode := &yaml.Node{Kind: yaml.SequenceNode}
|
||||
for _, child := range nodes {
|
||||
yamlChild, err := dec.convertToYamlNode(child)
|
||||
if err != nil {
|
||||
@@ -54,10 +64,10 @@ func (dec *xmlDecoder) processComment(c string) string {
|
||||
|
||||
func (dec *xmlDecoder) createMap(n *xmlNode) (*yaml.Node, error) {
|
||||
log.Debug("createMap: headC: %v, footC: %v", n.HeadComment, n.FootComment)
|
||||
yamlNode := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"}
|
||||
yamlNode := &yaml.Node{Kind: yaml.MappingNode}
|
||||
|
||||
if len(n.Data) > 0 {
|
||||
label := dec.prefs.ContentName
|
||||
label := dec.contentName
|
||||
labelNode := createScalarNode(label, label)
|
||||
labelNode.HeadComment = dec.processComment(n.HeadComment)
|
||||
labelNode.FootComment = dec.processComment(n.FootComment)
|
||||
@@ -76,7 +86,9 @@ func (dec *xmlDecoder) createMap(n *xmlNode) (*yaml.Node, error) {
|
||||
|
||||
}
|
||||
|
||||
// if i == len(n.Children)-1 {
|
||||
labelNode.FootComment = dec.processComment(keyValuePair.FootComment)
|
||||
// }
|
||||
|
||||
log.Debug("len of children in %v is %v", label, len(children))
|
||||
if len(children) > 1 {
|
||||
@@ -119,36 +131,32 @@ func (dec *xmlDecoder) convertToYamlNode(n *xmlNode) (*yaml.Node, error) {
|
||||
return scalar, nil
|
||||
}
|
||||
|
||||
func (dec *xmlDecoder) Decode() (*CandidateNode, error) {
|
||||
func (dec *xmlDecoder) Decode(rootYamlNode *yaml.Node) error {
|
||||
if dec.finished {
|
||||
return nil, io.EOF
|
||||
return io.EOF
|
||||
}
|
||||
root := &xmlNode{}
|
||||
// cant use xj - it doesn't keep map order.
|
||||
err := dec.decodeXML(root)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return err
|
||||
}
|
||||
firstNode, err := dec.convertToYamlNode(root)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return err
|
||||
} else if firstNode.Tag == "!!null" {
|
||||
dec.finished = true
|
||||
if dec.readAnything {
|
||||
return nil, io.EOF
|
||||
return io.EOF
|
||||
}
|
||||
}
|
||||
dec.readAnything = true
|
||||
rootYamlNode.Kind = yaml.DocumentNode
|
||||
rootYamlNode.Content = []*yaml.Node{firstNode}
|
||||
dec.finished = true
|
||||
|
||||
return &CandidateNode{
|
||||
Node: &yaml.Node{
|
||||
Kind: yaml.DocumentNode,
|
||||
Content: []*yaml.Node{firstNode},
|
||||
},
|
||||
}, nil
|
||||
return nil
|
||||
}
|
||||
|
||||
type xmlNode struct {
|
||||
@@ -197,7 +205,7 @@ type element struct {
|
||||
// of the map keys.
|
||||
func (dec *xmlDecoder) decodeXML(root *xmlNode) error {
|
||||
xmlDec := xml.NewDecoder(dec.reader)
|
||||
xmlDec.Strict = dec.prefs.StrictMode
|
||||
xmlDec.Strict = dec.strictMode
|
||||
// That will convert the charset if the provided XML is non-UTF-8
|
||||
xmlDec.CharsetReader = charset.NewReaderLabel
|
||||
|
||||
@@ -208,7 +216,7 @@ func (dec *xmlDecoder) decodeXML(root *xmlNode) error {
|
||||
}
|
||||
|
||||
getToken := func() (xml.Token, error) {
|
||||
if dec.prefs.UseRawToken {
|
||||
if dec.useRawToken {
|
||||
return xmlDec.RawToken()
|
||||
}
|
||||
return xmlDec.Token()
|
||||
@@ -236,12 +244,12 @@ func (dec *xmlDecoder) decodeXML(root *xmlNode) error {
|
||||
|
||||
// Extract attributes as children
|
||||
for _, a := range se.Attr {
|
||||
if dec.prefs.KeepNamespace {
|
||||
if dec.keepNamespace {
|
||||
if a.Name.Space != "" {
|
||||
a.Name.Local = a.Name.Space + ":" + a.Name.Local
|
||||
}
|
||||
}
|
||||
elem.n.AddChild(dec.prefs.AttributePrefix+a.Name.Local, &xmlNode{Data: a.Value})
|
||||
elem.n.AddChild(dec.attributePrefix+a.Name.Local, &xmlNode{Data: a.Value})
|
||||
}
|
||||
case xml.CharData:
|
||||
// Extract XML data (if any)
|
||||
@@ -274,14 +282,6 @@ func (dec *xmlDecoder) decodeXML(root *xmlNode) error {
|
||||
elem.n.HeadComment = joinFilter([]string{elem.n.HeadComment, commentStr})
|
||||
}
|
||||
|
||||
case xml.ProcInst:
|
||||
if !dec.prefs.SkipProcInst {
|
||||
elem.n.AddChild(dec.prefs.ProcInstPrefix+se.Target, &xmlNode{Data: string(se.Inst)})
|
||||
}
|
||||
case xml.Directive:
|
||||
if !dec.prefs.SkipDirectives {
|
||||
elem.n.AddChild(dec.prefs.DirectiveName, &xmlNode{Data: string(se)})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,114 +1,23 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"io"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
type yamlDecoder struct {
|
||||
decoder yaml.Decoder
|
||||
// work around of various parsing issues by yaml.v3 with document headers
|
||||
prefs YamlPreferences
|
||||
leadingContent string
|
||||
readAnything bool
|
||||
firstFile bool
|
||||
}
|
||||
|
||||
func NewYamlDecoder(prefs YamlPreferences) Decoder {
|
||||
return &yamlDecoder{prefs: prefs, firstFile: true}
|
||||
func NewYamlDecoder() Decoder {
|
||||
return &yamlDecoder{}
|
||||
}
|
||||
|
||||
func (dec *yamlDecoder) processReadStream(reader *bufio.Reader) (io.Reader, string, error) {
|
||||
var commentLineRegEx = regexp.MustCompile(`^\s*#`)
|
||||
var sb strings.Builder
|
||||
for {
|
||||
peekBytes, err := reader.Peek(3)
|
||||
if errors.Is(err, io.EOF) {
|
||||
// EOF are handled else where..
|
||||
return reader, sb.String(), nil
|
||||
} else if err != nil {
|
||||
return reader, sb.String(), err
|
||||
} else if string(peekBytes) == "---" {
|
||||
_, err := reader.ReadString('\n')
|
||||
sb.WriteString("$yqDocSeperator$\n")
|
||||
if errors.Is(err, io.EOF) {
|
||||
return reader, sb.String(), nil
|
||||
} else if err != nil {
|
||||
return reader, sb.String(), err
|
||||
}
|
||||
} else if commentLineRegEx.MatchString(string(peekBytes)) {
|
||||
line, err := reader.ReadString('\n')
|
||||
sb.WriteString(line)
|
||||
if errors.Is(err, io.EOF) {
|
||||
return reader, sb.String(), nil
|
||||
} else if err != nil {
|
||||
return reader, sb.String(), err
|
||||
}
|
||||
} else {
|
||||
return reader, sb.String(), nil
|
||||
}
|
||||
}
|
||||
func (dec *yamlDecoder) Init(reader io.Reader) {
|
||||
dec.decoder = *yaml.NewDecoder(reader)
|
||||
}
|
||||
|
||||
func (dec *yamlDecoder) Init(reader io.Reader) error {
|
||||
readerToUse := reader
|
||||
leadingContent := ""
|
||||
var err error
|
||||
// if we 'evaluating together' - we only process the leading content
|
||||
// of the first file - this ensures comments from subsequent files are
|
||||
// merged together correctly.
|
||||
if dec.prefs.LeadingContentPreProcessing && (!dec.prefs.EvaluateTogether || dec.firstFile) {
|
||||
readerToUse, leadingContent, err = dec.processReadStream(bufio.NewReader(reader))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
dec.leadingContent = leadingContent
|
||||
dec.readAnything = false
|
||||
dec.decoder = *yaml.NewDecoder(readerToUse)
|
||||
dec.firstFile = false
|
||||
return nil
|
||||
}
|
||||
|
||||
func (dec *yamlDecoder) Decode() (*CandidateNode, error) {
|
||||
var dataBucket yaml.Node
|
||||
|
||||
err := dec.decoder.Decode(&dataBucket)
|
||||
if errors.Is(err, io.EOF) && dec.leadingContent != "" && !dec.readAnything {
|
||||
// force returning an empty node with a comment.
|
||||
dec.readAnything = true
|
||||
return dec.blankNodeWithComment(), nil
|
||||
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
candidateNode := &CandidateNode{
|
||||
Node: &dataBucket,
|
||||
}
|
||||
|
||||
if dec.leadingContent != "" {
|
||||
candidateNode.LeadingContent = dec.leadingContent
|
||||
dec.leadingContent = ""
|
||||
}
|
||||
// move document comments into candidate node
|
||||
// otherwise unwrap drops them.
|
||||
candidateNode.TrailingContent = dataBucket.FootComment
|
||||
dataBucket.FootComment = ""
|
||||
return candidateNode, nil
|
||||
}
|
||||
|
||||
func (dec *yamlDecoder) blankNodeWithComment() *CandidateNode {
|
||||
return &CandidateNode{
|
||||
Document: 0,
|
||||
Filename: "",
|
||||
Node: &yaml.Node{Kind: yaml.DocumentNode, Content: []*yaml.Node{{Tag: "!!null", Kind: yaml.ScalarNode}}},
|
||||
FileIndex: 0,
|
||||
LeadingContent: dec.leadingContent,
|
||||
}
|
||||
func (dec *yamlDecoder) Decode(rootYamlNode *yaml.Node) error {
|
||||
return dec.decoder.Decode(rootYamlNode)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
@@ -9,6 +9,12 @@ Add behaves differently according to the type of the LHS:
|
||||
Use `+=` as a relative append assign for things like increment. Note that `.a += .x` is equivalent to running `.a = .a + .x`.
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Concatenate arrays
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This operator is used to provide alternative (or default) values when a particular expression is either null or false.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## LHS is defined
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -73,36 +79,3 @@ will output
|
||||
cat
|
||||
```
|
||||
|
||||
## Update or create - entity exists
|
||||
This initialises `a` if it's not present
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 1
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '(.a // (.a = 0)) += 1' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: 2
|
||||
```
|
||||
|
||||
## Update or create - entity does not exist
|
||||
This initialises `a` if it's not present
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
b: camel
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '(.a // (.a = 0)) += 1' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
b: camel
|
||||
a: 1
|
||||
```
|
||||
|
||||
|
||||
@@ -5,6 +5,12 @@ Use the `alias` and `anchor` operators to read and write yaml aliases and anchor
|
||||
`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Merge one map
|
||||
see https://yaml.org/type/merge.html
|
||||
|
||||
|
||||
@@ -7,10 +7,11 @@ Which will assign the LHS node values to the RHS node values. The RHS expression
|
||||
|
||||
### relative form: `|=`
|
||||
This will do a similar thing to the plain form, however, the RHS expression is run against _the LHS nodes_. This is useful for updating values based on old values, e.g. increment.
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
|
||||
### Flags
|
||||
- `c` clobber custom tags
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Create yaml file
|
||||
Running
|
||||
@@ -234,37 +235,3 @@ a:
|
||||
- bogs
|
||||
```
|
||||
|
||||
## Custom types are maintained by default
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: !cat meow
|
||||
b: !dog woof
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.a = .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: !cat woof
|
||||
b: !dog woof
|
||||
```
|
||||
|
||||
## Custom types: clovver
|
||||
Use the `c` option to clobber custom tags
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: !cat meow
|
||||
b: !dog woof
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.a =c .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: !dog woof
|
||||
b: !dog woof
|
||||
```
|
||||
|
||||
|
||||
@@ -16,6 +16,12 @@ These are most commonly used with the `select` operator to filter particular nod
|
||||
- comparison (`>=`, `<` etc) operators [here](https://mikefarah.gitbook.io/yq/operators/compare)
|
||||
- select operator [here](https://mikefarah.gitbook.io/yq/operators/select)
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## `or` example
|
||||
Running
|
||||
```bash
|
||||
|
||||
@@ -3,6 +3,12 @@
|
||||
This creates an array using the expression between the square brackets.
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Collect empty
|
||||
Running
|
||||
```bash
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Returns the column of the matching node. Starts from 1, 0 indicates there was no column data.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Returns column of _value_ node
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Comment Operators
|
||||
|
||||
Use these comment operators to set or retrieve comments. Note that line comments on maps/arrays are actually set on the _key_ node as opposed to the _value_ (map/array). See below for examples.
|
||||
Use these comment operators to set or retrieve comments.
|
||||
|
||||
Like the `=` and `|=` assign operators, the same syntax applies when updating comments:
|
||||
|
||||
@@ -10,9 +10,13 @@ This will assign the LHS nodes comments to the expression on the RHS. The RHS is
|
||||
### relative form: `|=`
|
||||
Similar to the plain form, however the RHS evaluates against each matching LHS node! This is useful if you want to set the comments as a relative expression of the node, for instance its value or path.
|
||||
|
||||
## Set line comment
|
||||
Set the comment on the key node for more reliability (see below).
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Set line comment
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: cat
|
||||
@@ -26,24 +30,6 @@ will output
|
||||
a: cat # single
|
||||
```
|
||||
|
||||
## Set line comment of a maps/arrays
|
||||
For maps and arrays, you need to set the line comment on the _key_ node. This will also work for scalars.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
b: things
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '(.a | key) line_comment="single"' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: # single
|
||||
b: things
|
||||
```
|
||||
|
||||
## Use update assign to perform relative updates
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -60,121 +46,6 @@ a: cat # cat
|
||||
b: dog # dog
|
||||
```
|
||||
|
||||
## Where is the comment - map key example
|
||||
The underlying yaml parser can assign comments in a document to surprising nodes. Use an expression like this to find where you comment is. 'p' indicates the path, 'isKey' is if the node is a map key (as opposed to a map value).
|
||||
From this, you can see the 'hello-world-comment' is actually on the 'hello' key
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
hello: # hello-world-comment
|
||||
message: world
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '[... | {"p": path | join("."), "isKey": is_key, "hc": headComment, "lc": lineComment, "fc": footComment}]' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- p: ""
|
||||
isKey: false
|
||||
hc: ""
|
||||
lc: ""
|
||||
fc: ""
|
||||
- p: hello
|
||||
isKey: true
|
||||
hc: ""
|
||||
lc: hello-world-comment
|
||||
fc: ""
|
||||
- p: hello
|
||||
isKey: false
|
||||
hc: ""
|
||||
lc: ""
|
||||
fc: ""
|
||||
- p: hello.message
|
||||
isKey: true
|
||||
hc: ""
|
||||
lc: ""
|
||||
fc: ""
|
||||
- p: hello.message
|
||||
isKey: false
|
||||
hc: ""
|
||||
lc: ""
|
||||
fc: ""
|
||||
```
|
||||
|
||||
## Retrieve comment - map key example
|
||||
From the previous example, we know that the comment is on the 'hello' _key_ as a lineComment
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
hello: # hello-world-comment
|
||||
message: world
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.hello | key | line_comment' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
hello-world-comment
|
||||
```
|
||||
|
||||
## Where is the comment - array example
|
||||
The underlying yaml parser can assign comments in a document to surprising nodes. Use an expression like this to find where you comment is. 'p' indicates the path, 'isKey' is if the node is a map key (as opposed to a map value).
|
||||
From this, you can see the 'under-name-comment' is actually on the first child
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
name:
|
||||
# under-name-comment
|
||||
- first-array-child
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '[... | {"p": path | join("."), "isKey": is_key, "hc": headComment, "lc": lineComment, "fc": footComment}]' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- p: ""
|
||||
isKey: false
|
||||
hc: ""
|
||||
lc: ""
|
||||
fc: ""
|
||||
- p: name
|
||||
isKey: true
|
||||
hc: ""
|
||||
lc: ""
|
||||
fc: ""
|
||||
- p: name
|
||||
isKey: false
|
||||
hc: ""
|
||||
lc: ""
|
||||
fc: ""
|
||||
- p: name.0
|
||||
isKey: false
|
||||
hc: under-name-comment
|
||||
lc: ""
|
||||
fc: ""
|
||||
```
|
||||
|
||||
## Retrieve comment - array example
|
||||
From the previous example, we know that the comment is on the first child as a headComment
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
name:
|
||||
# under-name-comment
|
||||
- first-array-child
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.name[0] | headComment' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
under-name-comment
|
||||
```
|
||||
|
||||
## Set head comment
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -191,25 +62,6 @@ will output
|
||||
a: cat
|
||||
```
|
||||
|
||||
## Set head comment of a map entry
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
f: foo
|
||||
a:
|
||||
b: cat
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '(.a | key) head_comment="single"' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
f: foo
|
||||
# single
|
||||
a:
|
||||
b: cat
|
||||
```
|
||||
|
||||
## Set foot comment, using an expression
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -246,7 +98,6 @@ Note the use of `...` to ensure key nodes are included.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# hi
|
||||
a: cat # comment
|
||||
# great
|
||||
b: # key comment
|
||||
@@ -264,7 +115,6 @@ b:
|
||||
## Get line comment
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# welcome!
|
||||
a: cat # meow
|
||||
# have a great day
|
||||
```
|
||||
|
||||
@@ -14,6 +14,12 @@ The following types are currently supported:
|
||||
- boolean operators (`and`, `or`, `any` etc) [here](https://mikefarah.gitbook.io/yq/operators/boolean-operators)
|
||||
- select operator [here](https://mikefarah.gitbook.io/yq/operators/select)
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Compare numbers (>)
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This returns `true` if the context contains the passed in parameter, and false otherwise.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Array contains array
|
||||
Array is equal or subset of
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This is used to construct objects (or maps). This can be used against existing yaml, or to create fresh yaml documents.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Collect empty object
|
||||
Running
|
||||
```bash
|
||||
|
||||
@@ -13,11 +13,11 @@ To specify a custom parsing format, use the `with_dtf` operator. The first param
|
||||
yq 'with_dtf("myformat"; .a + "3h" | tz("Australia/Melbourne"))'
|
||||
```
|
||||
|
||||
See the [library docs](https://pkg.go.dev/time#pkg-constants) for examples of formatting options.
|
||||
See https://pkg.go.dev/time#pkg-constants for examples of formatting options.
|
||||
|
||||
|
||||
## Timezones
|
||||
This uses golangs built in LoadLocation function to parse timezones strings. See the [library docs](https://pkg.go.dev/time#LoadLocation) for more details.
|
||||
This uses golangs built in LoadLocation function to parse timezones strings. See https://pkg.go.dev/time#LoadLocation for more details.
|
||||
|
||||
|
||||
## Durations
|
||||
@@ -25,6 +25,12 @@ Durations are parsed using golangs built in [ParseDuration](https://pkg.go.dev/t
|
||||
|
||||
You can durations to time using the `+` operator.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Format: from standard RFC3339 format
|
||||
Providing a single parameter assumes a standard RFC3339 datetime format. If the target format is not a valid yaml datetime format, the result will be a string tagged node.
|
||||
|
||||
@@ -150,7 +156,7 @@ a: 2021-01-01T03:10:00Z
|
||||
```
|
||||
|
||||
## Date subtraction
|
||||
You can subtract durations from dates. Assumes RFC3339 date time format, see [date-time operators](https://mikefarah.gitbook.io/yq/operators/datetime#date-time-formattings) for more information.
|
||||
You can subtract durations from dates. Assumes RFC3339 date time format, see [date-time operators](https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Deletes matching entries in maps or arrays.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Delete entry in map
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Use the `documentIndex` operator (or the `di` shorthand) to select nodes of a particular document.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Retrieve a document index
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -9,22 +9,28 @@ These operators are useful to process yaml documents that have stringified embed
|
||||
|
||||
| Format | Decode (from string) | Encode (to string) |
|
||||
| --- | -- | --|
|
||||
| Yaml | from_yaml/@yamld | to_yaml(i)/@yaml |
|
||||
| JSON | from_json/@jsond | to_json(i)/@json |
|
||||
| Properties | from_props/@propsd | to_props/@props |
|
||||
| CSV | from_csv/@csvd | to_csv/@csv |
|
||||
| TSV | from_tsv/@tsvd | to_tsv/@tsv |
|
||||
| XML | from_xml/@xmld | to_xml(i)/@xml |
|
||||
| Yaml | from_yaml | to_yaml(i)/@yaml |
|
||||
| JSON | from_json | to_json(i)/@json |
|
||||
| Properties | from_props | to_props/@props |
|
||||
| CSV | | to_csv/@csv |
|
||||
| TSV | | to_tsv/@tsv |
|
||||
| XML | from_xml | to_xml(i)/@xml |
|
||||
| Base64 | @base64d | @base64 |
|
||||
|
||||
|
||||
See CSV and TSV [documentation](https://mikefarah.gitbook.io/yq/usage/csv-tsv) for accepted formats.
|
||||
CSV and TSV format both accept either a single array or scalars (representing a single row), or an array of array of scalars (representing multiple rows).
|
||||
|
||||
XML uses the `--xml-attribute-prefix` and `xml-content-name` flags to identify attributes and content fields.
|
||||
|
||||
|
||||
Base64 assumes [rfc4648](https://rfc-editor.org/rfc/rfc4648.html) encoding. Encoding and decoding both assume that the content is a string.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Encode value as json string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -126,7 +132,7 @@ a: |-
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.a |= @propsd' sample.yml
|
||||
yq '.a |= from_props' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
@@ -135,42 +141,6 @@ a:
|
||||
dogs: cool as well
|
||||
```
|
||||
|
||||
## Decode csv encoded string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: |-
|
||||
cats,dogs
|
||||
great,cool as well
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.a |= @csvd' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
- cats: great
|
||||
dogs: cool as well
|
||||
```
|
||||
|
||||
## Decode tsv encoded string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: |-
|
||||
cats dogs
|
||||
great cool as well
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.a |= @tsvd' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
- cats: great
|
||||
dogs: cool as well
|
||||
```
|
||||
|
||||
## Encode value as yaml string
|
||||
Indent defaults to 2
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Similar to the same named functions in `jq` these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## to_entries Map
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -101,26 +107,6 @@ KEY_a: 1
|
||||
KEY_b: 2
|
||||
```
|
||||
|
||||
## Custom sort map keys
|
||||
Use to_entries to convert to an array of key/value pairs, sort the array using sort/sort_by/etc, and convert it back.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 1
|
||||
c: 3
|
||||
b: 2
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'to_entries | sort_by(.key) | reverse | from_entries' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
c: 3
|
||||
b: 2
|
||||
a: 1
|
||||
```
|
||||
|
||||
## Use with_entries to filter the map
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -30,6 +30,12 @@ yq '(.. | select(tag == "!!str")) |= envsubst' file.yaml
|
||||
```
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Read string environment variable
|
||||
Running
|
||||
```bash
|
||||
|
||||
@@ -21,6 +21,12 @@ The not equals `!=` operator returns `false` if the LHS is equal to the RHS.
|
||||
- select operator [here](https://mikefarah.gitbook.io/yq/operators/select)
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Match string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Use this operation to short-circuit expressions. Useful for validation.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Validate a particular value
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -6,6 +6,12 @@ Use `eval` to dynamically process an expression - for instance from an environme
|
||||
|
||||
Tip: This can be useful way parameterise complex scripts.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Dynamically evaluate a path
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -10,6 +10,12 @@ Note the use of eval-all to ensure all documents are loaded into memory.
|
||||
yq eval-all 'select(fi == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml
|
||||
```
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Get filename
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
# Flatten
|
||||
This recursively flattens arrays.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Flatten
|
||||
Recursively flattens all arrays
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This is used to group items in an array by an expression.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Group by field
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This is operation that returns true if the key exists in a map (or index in an array), false otherwise.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Has map key
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -6,8 +6,4 @@ This operator is used to update node values. It can be used in either the:
|
||||
Which will assign the LHS node values to the RHS node values. The RHS expression is run against the matching nodes in the pipeline.
|
||||
|
||||
### relative form: `|=`
|
||||
This will do a similar thing to the plain form, however, the RHS expression is run against _the LHS nodes_. This is useful for updating values based on old values, e.g. increment.
|
||||
|
||||
|
||||
### Flags
|
||||
- `c` clobber custom tags
|
||||
This will do a similar thing to the plain form, however, the RHS expression is run against _the LHS nodes_. This is useful for updating values based on old values, e.g. increment.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Comment Operators
|
||||
|
||||
Use these comment operators to set or retrieve comments. Note that line comments on maps/arrays are actually set on the _key_ node as opposed to the _value_ (map/array). See below for examples.
|
||||
Use these comment operators to set or retrieve comments.
|
||||
|
||||
Like the `=` and `|=` assign operators, the same syntax applies when updating comments:
|
||||
|
||||
|
||||
@@ -13,11 +13,11 @@ To specify a custom parsing format, use the `with_dtf` operator. The first param
|
||||
yq 'with_dtf("myformat"; .a + "3h" | tz("Australia/Melbourne"))'
|
||||
```
|
||||
|
||||
See the [library docs](https://pkg.go.dev/time#pkg-constants) for examples of formatting options.
|
||||
See https://pkg.go.dev/time#pkg-constants for examples of formatting options.
|
||||
|
||||
|
||||
## Timezones
|
||||
This uses golangs built in LoadLocation function to parse timezones strings. See the [library docs](https://pkg.go.dev/time#LoadLocation) for more details.
|
||||
This uses golangs built in LoadLocation function to parse timezones strings. See https://pkg.go.dev/time#LoadLocation for more details.
|
||||
|
||||
|
||||
## Durations
|
||||
|
||||
@@ -9,16 +9,16 @@ These operators are useful to process yaml documents that have stringified embed
|
||||
|
||||
| Format | Decode (from string) | Encode (to string) |
|
||||
| --- | -- | --|
|
||||
| Yaml | from_yaml/@yamld | to_yaml(i)/@yaml |
|
||||
| JSON | from_json/@jsond | to_json(i)/@json |
|
||||
| Properties | from_props/@propsd | to_props/@props |
|
||||
| CSV | from_csv/@csvd | to_csv/@csv |
|
||||
| TSV | from_tsv/@tsvd | to_tsv/@tsv |
|
||||
| XML | from_xml/@xmld | to_xml(i)/@xml |
|
||||
| Yaml | from_yaml | to_yaml(i)/@yaml |
|
||||
| JSON | from_json | to_json(i)/@json |
|
||||
| Properties | from_props | to_props/@props |
|
||||
| CSV | | to_csv/@csv |
|
||||
| TSV | | to_tsv/@tsv |
|
||||
| XML | from_xml | to_xml(i)/@xml |
|
||||
| Base64 | @base64d | @base64 |
|
||||
|
||||
|
||||
See CSV and TSV [documentation](https://mikefarah.gitbook.io/yq/usage/csv-tsv) for accepted formats.
|
||||
CSV and TSV format both accept either a single array or scalars (representing a single row), or an array of array of scalars (representing multiple rows).
|
||||
|
||||
XML uses the `--xml-attribute-prefix` and `xml-content-name` flags to identify attributes and content fields.
|
||||
|
||||
|
||||
@@ -14,7 +14,6 @@ You can control how objects are merged by using one or more of the following fla
|
||||
- `d` deeply merge arrays
|
||||
- `?` only merge _existing_ fields
|
||||
- `n` only merge _new_ fields
|
||||
- `c` clobber custom tags
|
||||
|
||||
|
||||
### Merge two files together
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
# Path
|
||||
|
||||
The `path` operator can be used to get the traversal paths of matching nodes in an expression. The path is returned as an array, which if traversed in order will lead to the matching node.
|
||||
The path operator can be used to get the traversal paths of matching nodes in an expression. The path is returned as an array, which if traversed in order will lead to the matching node.
|
||||
|
||||
You can get the key/index of matching nodes by using the `path` operator to return the path array then piping that through `.[-1]` to get the last element of that array, the key.
|
||||
|
||||
Use `setpath` to set a value to the path array returned by `path`, and similarly `delpaths` for an array of path arrays.
|
||||
|
||||
|
||||
@@ -11,6 +11,3 @@ diff file1.yml file2.yml
|
||||
```
|
||||
|
||||
Note that `yq` does not yet consider anchors when sorting by keys - this may result in invalid yaml documents if your are using merge anchors.
|
||||
|
||||
For more advanced sorting, using `to_entries` to convert the map to an array, then sort/process the array as you like (e.g. using `sort_by`) and convert back to a map using `from_entries`.
|
||||
See [here](https://mikefarah.gitbook.io/yq/operators/entries#custom-sort-map-keys) for an example.
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Use the `keys` operator to return map keys or array indices.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Map keys
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -112,43 +118,3 @@ will output
|
||||
comment on key
|
||||
```
|
||||
|
||||
## Check node is a key
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
b:
|
||||
- cat
|
||||
c: frog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '[... | { "p": path | join("."), "isKey": is_key, "tag": tag }]' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- p: ""
|
||||
isKey: false
|
||||
tag: '!!map'
|
||||
- p: a
|
||||
isKey: true
|
||||
tag: '!!str'
|
||||
- p: a
|
||||
isKey: false
|
||||
tag: '!!map'
|
||||
- p: a.b
|
||||
isKey: true
|
||||
tag: '!!str'
|
||||
- p: a.b
|
||||
isKey: false
|
||||
tag: '!!seq'
|
||||
- p: a.b.0
|
||||
isKey: false
|
||||
tag: '!!str'
|
||||
- p: a.c
|
||||
isKey: true
|
||||
tag: '!!str'
|
||||
- p: a.c
|
||||
isKey: false
|
||||
tag: '!!str'
|
||||
```
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Returns the lengths of the nodes. Length is defined according to the type of the node.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## String length
|
||||
returns length of string
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Returns the line of the matching node. Starts from 1, 0 indicates there was no line data.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Returns line of _value_ node
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -45,6 +45,12 @@ this.is = a properties file
|
||||
bXkgc2VjcmV0IGNoaWxsaSByZWNpcGUgaXMuLi4u
|
||||
```
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Simple example
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Maps values of an array. Use `map_values` to map values of an object.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Map array
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -14,7 +14,6 @@ You can control how objects are merged by using one or more of the following fla
|
||||
- `d` deeply merge arrays
|
||||
- `?` only merge _existing_ fields
|
||||
- `n` only merge _new_ fields
|
||||
- `c` clobber custom tags
|
||||
|
||||
|
||||
### Merge two files together
|
||||
@@ -36,6 +35,12 @@ By default - `yq` merge is naive. It merges maps when they match the key name, a
|
||||
For more complex array merging (e.g. merging items that match on a certain key) please see the example [here](https://mikefarah.gitbook.io/yq/operators/multiply-merge#merge-arrays-of-objects-together-matching-on-a-key)
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Multiply integers
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@@ -476,26 +481,3 @@ b: !goat
|
||||
dog: woof
|
||||
```
|
||||
|
||||
## Custom types: clobber tags
|
||||
Use the `c` option to clobber custom tags. Note that the second tag is now used
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: !horse
|
||||
cat: meow
|
||||
b: !goat
|
||||
dog: woof
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.a *=c .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: !goat
|
||||
cat: meow
|
||||
dog: woof
|
||||
b: !goat
|
||||
dog: woof
|
||||
```
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Parent simply returns the parent nodes of the matching nodes.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Simple example
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
# Path
|
||||
|
||||
The `path` operator can be used to get the traversal paths of matching nodes in an expression. The path is returned as an array, which if traversed in order will lead to the matching node.
|
||||
The path operator can be used to get the traversal paths of matching nodes in an expression. The path is returned as an array, which if traversed in order will lead to the matching node.
|
||||
|
||||
You can get the key/index of matching nodes by using the `path` operator to return the path array then piping that through `.[-1]` to get the last element of that array, the key.
|
||||
|
||||
Use `setpath` to set a value to the path array returned by `path`, and similarly `delpaths` for an array of path arrays.
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Map path
|
||||
Given a sample.yml file of:
|
||||
@@ -95,114 +98,3 @@ will output
|
||||
value: frog
|
||||
```
|
||||
|
||||
## Set path
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
b: cat
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'setpath(["a", "b"]; "things")' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
b: things
|
||||
```
|
||||
|
||||
## Set on empty document
|
||||
Running
|
||||
```bash
|
||||
yq --null-input 'setpath(["a", "b"]; "things")'
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
b: things
|
||||
```
|
||||
|
||||
## Set array path
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
- cat
|
||||
- frog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'setpath(["a", 0]; "things")' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
- things
|
||||
- frog
|
||||
```
|
||||
|
||||
## Set array path empty
|
||||
Running
|
||||
```bash
|
||||
yq --null-input 'setpath(["a", 0]; "things")'
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
- things
|
||||
```
|
||||
|
||||
## Delete path
|
||||
Notice delpaths takes an _array_ of paths.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
b: cat
|
||||
c: dog
|
||||
d: frog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'delpaths([["a", "c"], ["a", "d"]])' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
b: cat
|
||||
```
|
||||
|
||||
## Delete array path
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
- cat
|
||||
- frog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'delpaths([["a", 0]])' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
- frog
|
||||
```
|
||||
|
||||
## Delete - wrong parameter
|
||||
delpaths does not work with a single path array
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
- cat
|
||||
- frog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'delpaths(["a", 0])' sample.yml
|
||||
```
|
||||
will output
|
||||
```bash
|
||||
Error: DELPATHS: expected entry [0] to be a sequence, but its a !!str. Note that delpaths takes an array of path arrays, e.g. [["a", "b"]]
|
||||
```
|
||||
|
||||
|
||||
@@ -4,6 +4,12 @@ Filter a map by the specified list of keys. Map is returned with the key in the
|
||||
|
||||
Similarly, filter an array by the specified list of indices.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Pick keys from map
|
||||
Note that the order of the keys matches the pick order and non existent keys are skipped.
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Pipe the results of an expression into another. Like the bash operator.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Simple Pipe
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -19,6 +19,12 @@ For instance to set the `style` of all nodes in a yaml doc, including the map ke
|
||||
```bash
|
||||
yq '... style= "flow"' file.yaml
|
||||
```
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Recurse map (values only)
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -21,6 +21,12 @@ Reduce syntax in `yq` is a little different from `jq` - as `yq` (currently) isn'
|
||||
|
||||
To that end, the reduce operator is called `ireduce` for backwards compatibility if a `jq` like prefix version of `reduce` is ever added.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Sum numbers
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Reverses the order of the items in an array
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Reverse
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -8,6 +8,12 @@ Select is used to filter arrays and maps by a boolean expression.
|
||||
- comparison (`>=`, `<` etc) operators [here](https://mikefarah.gitbook.io/yq/operators/compare)
|
||||
- boolean operators (`and`, `or`, `any` etc) [here](https://mikefarah.gitbook.io/yq/operators/boolean-operators)
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Select elements from array using wildcard prefix
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -12,8 +12,11 @@ diff file1.yml file2.yml
|
||||
|
||||
Note that `yq` does not yet consider anchors when sorting by keys - this may result in invalid yaml documents if your are using merge anchors.
|
||||
|
||||
For more advanced sorting, using `to_entries` to convert the map to an array, then sort/process the array as you like (e.g. using `sort_by`) and convert back to a map using `from_entries`.
|
||||
See [here](https://mikefarah.gitbook.io/yq/operators/entries#custom-sort-map-keys) for an example.
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Sort keys of map
|
||||
Given a sample.yml file of:
|
||||
|
||||
@@ -7,6 +7,12 @@ To sort by descending order, pipe the results through the `reverse` operator aft
|
||||
Note that at this stage, `yq` only sorts scalar fields.
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Sort by string field
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This operator splits all matches into separate documents
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Split empty
|
||||
Running
|
||||
```bash
|
||||
|
||||
@@ -56,6 +56,12 @@ IFS= read -rd '' output < <(cat my_file)
|
||||
output=$output ./yq '.data.values = strenv(output)' first.yml
|
||||
```
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## To up (upper) case
|
||||
Works with unicode characters
|
||||
|
||||
@@ -106,26 +112,6 @@ will output
|
||||
cat; meow; 1; ; true
|
||||
```
|
||||
|
||||
## Trim strings
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- ' cat'
|
||||
- 'dog '
|
||||
- ' cow cow '
|
||||
- horse
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.[] | trim' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
cat
|
||||
dog
|
||||
cow cow
|
||||
horse
|
||||
```
|
||||
|
||||
## Match string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
The style operator can be used to get or set the style of nodes (e.g. string style, yaml style)
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Update and set style of a particular node (simple)
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
You can use subtract to subtract numbers, as well as removing elements from an array.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Array subtraction
|
||||
Running
|
||||
```bash
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
The tag operator can be used to get or set the tag of nodes (e.g. `!!str`, `!!int`, `!!bool`).
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Get tag
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This is the simplest (and perhaps most used) operator, it is used to navigate deeply into yaml structures.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Simple map navigation
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
This operator is used to combine different results together.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Combine scalars
|
||||
Running
|
||||
```bash
|
||||
|
||||
@@ -3,6 +3,12 @@
|
||||
This is used to filter out duplicated items in an array. Note that the original order of the array is maintained.
|
||||
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Unique array of scalars (string/numbers)
|
||||
Note that unique maintains the original order of the array.
|
||||
|
||||
|
||||
@@ -4,6 +4,12 @@ Like the `jq` equivalents, variables are sometimes required for the more complex
|
||||
|
||||
Note that there is also an additional `ref` operator that holds a reference (instead of a copy) of the path, allowing you to make multiple changes to the same path.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Single value variable
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
Use the `with` operator to conveniently make multiple updates to a deeply nested path, or to update array elements relatively to each other. The first argument expression sets the root context, and the second expression runs against that root context.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Update and style
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
# JSON
|
||||
|
||||
Encode and decode to and from JSON. Supports multiple JSON documents in a single file (e.g. NDJSON).
|
||||
Encode and decode to and from JSON. Note that YAML is a _superset_ of JSON - so `yq` can read any json file without doing anything special.
|
||||
|
||||
Note that YAML is a superset of (single document) JSON - so you don't have to use the JSON parser to read JSON when there is only one JSON document in the input. You will probably want to pretty print the result in this case, to get idiomatic YAML styling.
|
||||
This means you don't need to 'convert' a JSON file to YAML - however if you want idiomatic YAML styling, then you can use the `-P/--prettyPrint` flag, see examples below.
|
||||
|
||||
{% hint style="warning" %}
|
||||
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
|
||||
|
||||
`yq e <exp> <file>`
|
||||
{% endhint %}
|
||||
|
||||
## Parse json: simple
|
||||
JSON is a subset of yaml, so all you need to do is prettify the output
|
||||
@@ -125,129 +130,3 @@ will output
|
||||
{"whatever":"cat"}
|
||||
```
|
||||
|
||||
## Roundtrip NDJSON
|
||||
Unfortunately the json encoder strips leading spaces of values.
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=0 sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{"this":"is a multidoc json file"}
|
||||
{"each":["line is a valid json document"]}
|
||||
{"a number":4}
|
||||
```
|
||||
|
||||
## Roundtrip multi-document JSON
|
||||
The NDJSON parser can also handle multiple multi-line json documents in a single file!
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{
|
||||
"this": "is a multidoc json file"
|
||||
}
|
||||
{
|
||||
"it": [
|
||||
"has",
|
||||
"consecutive",
|
||||
"json documents"
|
||||
]
|
||||
}
|
||||
{
|
||||
"a number": 4
|
||||
}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=2 sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
"this": "is a multidoc json file"
|
||||
}
|
||||
{
|
||||
"it": [
|
||||
"has",
|
||||
"consecutive",
|
||||
"json documents"
|
||||
]
|
||||
}
|
||||
{
|
||||
"a number": 4
|
||||
}
|
||||
```
|
||||
|
||||
## Update a specific document in a multi-document json
|
||||
Documents are indexed by the `documentIndex` or `di` operator.
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=0 '(select(di == 1) | .each ) += "cool"' sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{"this":"is a multidoc json file"}
|
||||
{"each":["line is a valid json document","cool"]}
|
||||
{"a number":4}
|
||||
```
|
||||
|
||||
## Find and update a specific document in a multi-document json
|
||||
Use expressions as you normally would.
|
||||
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json -o=json -I=0 '(select(has("each")) | .each ) += "cool"' sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{"this":"is a multidoc json file"}
|
||||
{"each":["line is a valid json document","cool"]}
|
||||
{"a number":4}
|
||||
```
|
||||
|
||||
## Decode NDJSON
|
||||
Given a sample.json file of:
|
||||
```json
|
||||
{"this": "is a multidoc json file"}
|
||||
{"each": ["line is a valid json document"]}
|
||||
{"a number": 4}
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=json sample.json
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
this: is a multidoc json file
|
||||
---
|
||||
each:
|
||||
- line is a valid json document
|
||||
---
|
||||
a number: 4
|
||||
```
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user