diff --git a/SUMMARY.md b/SUMMARY.md index 92bde42c..d248500f 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -35,6 +35,7 @@ * [Keys](operators/keys.md) * [Length](operators/length.md) * [Load](operators/load.md) + * [Map](operators/map.md) * [Multiply (Merge)](operators/multiply-merge.md) * [Parent](operators/parent.md) * [Path](operators/path.md) @@ -42,6 +43,7 @@ * [Recursive Descent (Glob)](operators/recursive-descent-glob.md) * [Reduce](operators/reduce.md) * [Select](operators/select.md) + * [Sort](operators/sort.md) * [Sort Keys](operators/sort-keys.md) * [Split into Documents](operators/split-into-documents.md) * [String Operators](operators/string-operators.md) @@ -58,6 +60,7 @@ * [Output format](usage/output-format.md) * [Working with Properties](usage/properties.md) +* [Working with CSV, TSV](usage/csv-tsv.md) * [Working with JSON](usage/convert.md) * [Front Matter](usage/front-matter.md) * [Split into multiple files](usage/split-into-multiple-files.md) diff --git a/operators/assign-update.md b/operators/assign-update.md index ad71c81b..ad175b53 100644 --- a/operators/assign-update.md +++ b/operators/assign-update.md @@ -196,6 +196,22 @@ will output {a: {b: bogs}} ``` +## Update node value that has an anchor +Anchor will remaple + +Given a sample.yml file of: +```yaml +a: &cool cat +``` +then +```bash +yq eval '.a = "dog"' sample.yml +``` +will output +```yaml +a: &cool dog +``` + ## Update empty object and array Given a sample.yml file of: ```yaml diff --git a/operators/encode-decode.md b/operators/encode-decode.md index 384b5559..d4416608 100644 --- a/operators/encode-decode.md +++ b/operators/encode-decode.md @@ -6,6 +6,111 @@ Note that you can optionally pass an indent value to the encode functions (see b These operators are useful to process yaml documents that have stringified embeded yaml/json/props in them. + +| Format | Decode (from string) | Encode (to string) | +| --- | -- | --| +| Yaml | from_yaml | to_yaml(i)/@yaml | +| JSON | from_json | to_json(i)/@json | +| Properties | | to_props/@props | +| CSV | | to_csv/@csv | +| TSV | | to_tsv/@tsv | + + +CSV and TSV format both accept either a single array or scalars (representing a single row), or an array of array of scalars (representing multiple rows). + + +## Encode value as json string +Given a sample.yml file of: +```yaml +a: + cool: thing +``` +then +```bash +yq eval '.b = (.a | to_json)' sample.yml +``` +will output +```yaml +a: + cool: thing +b: | + { + "cool": "thing" + } +``` + +## Encode value as json string, on one line +Pass in a 0 indent to print json on a single line. + +Given a sample.yml file of: +```yaml +a: + cool: thing +``` +then +```bash +yq eval '.b = (.a | to_json(0))' sample.yml +``` +will output +```yaml +a: + cool: thing +b: '{"cool":"thing"}' +``` + +## Encode value as json string, on one line shorthand +Pass in a 0 indent to print json on a single line. + +Given a sample.yml file of: +```yaml +a: + cool: thing +``` +then +```bash +yq eval '.b = (.a | @json)' sample.yml +``` +will output +```yaml +a: + cool: thing +b: '{"cool":"thing"}' +``` + +## Decode a json encoded string +Keep in mind JSON is a subset of YAML. If you want idiomatic yaml, pipe through the style operator to clear out the JSON styling. + +Given a sample.yml file of: +```yaml +a: '{"cool":"thing"}' +``` +then +```bash +yq eval '.a | from_json | ... style=""' sample.yml +``` +will output +```yaml +cool: thing +``` + +## Encode value as props string +Given a sample.yml file of: +```yaml +a: + cool: thing +``` +then +```bash +yq eval '.b = (.a | @props)' sample.yml +``` +will output +```yaml +a: + cool: thing +b: | + cool = thing +``` + ## Encode value as yaml string Indent defaults to 2 @@ -52,83 +157,6 @@ b: | bob: dylan ``` -## Encode value as yaml string, using toyaml -Does the same thing as to_yaml, matching jq naming convention. - -Given a sample.yml file of: -```yaml -a: - cool: thing -``` -then -```bash -yq eval '.b = (.a | to_yaml)' sample.yml -``` -will output -```yaml -a: - cool: thing -b: | - cool: thing -``` - -## Encode value as json string -Given a sample.yml file of: -```yaml -a: - cool: thing -``` -then -```bash -yq eval '.b = (.a | to_json)' sample.yml -``` -will output -```yaml -a: - cool: thing -b: | - { - "cool": "thing" - } -``` - -## Encode value as json string, on one line -Pass in a 0 indent to print json on a single line. - -Given a sample.yml file of: -```yaml -a: - cool: thing -``` -then -```bash -yq eval '.b = (.a | to_json(0))' sample.yml -``` -will output -```yaml -a: - cool: thing -b: '{"cool":"thing"}' -``` - -## Encode value as props string -Given a sample.yml file of: -```yaml -a: - cool: thing -``` -then -```bash -yq eval '.b = (.a | to_props)' sample.yml -``` -will output -```yaml -a: - cool: thing -b: | - cool = thing -``` - ## Decode a yaml encoded string Given a sample.yml file of: ```yaml @@ -178,3 +206,68 @@ will output a: 'foo: cat' ``` +## Encode array of scalars as csv string +Scalars are strings, numbers and booleans. + +Given a sample.yml file of: +```yaml +- cat +- thing1,thing2 +- true +- 3.40 +``` +then +```bash +yq eval '@csv' sample.yml +``` +will output +```yaml +cat,"thing1,thing2",true,3.40 +``` + +## Encode array of arrays as csv string +Given a sample.yml file of: +```yaml +- - cat + - thing1,thing2 + - true + - 3.40 +- - dog + - thing3 + - false + - 12 +``` +then +```bash +yq eval '@csv' sample.yml +``` +will output +```yaml +cat,"thing1,thing2",true,3.40 +dog,thing3,false,12 +``` + +## Encode array of array scalars as tsv string +Scalars are strings, numbers and booleans. + +Given a sample.yml file of: +```yaml +- - cat + - thing1,thing2 + - true + - 3.40 +- - dog + - thing3 + - false + - 12 +``` +then +```bash +yq eval '@tsv' sample.yml +``` +will output +```yaml +cat thing1,thing2 true 3.40 +dog thing3 false 12 +``` + diff --git a/operators/map.md b/operators/map.md new file mode 100644 index 00000000..1d64292c --- /dev/null +++ b/operators/map.md @@ -0,0 +1,40 @@ +# Map + +Maps values of an array. Use `map_values` to map values of an object. + +## Map array +Given a sample.yml file of: +```yaml +- 1 +- 2 +- 3 +``` +then +```bash +yq eval 'map(. + 1)' sample.yml +``` +will output +```yaml +- 2 +- 3 +- 4 +``` + +## Map object values +Given a sample.yml file of: +```yaml +a: 1 +b: 2 +c: 3 +``` +then +```bash +yq eval 'map_values(. + 1)' sample.yml +``` +will output +```yaml +a: 2 +b: 3 +c: 4 +``` + diff --git a/operators/path.md b/operators/path.md index 3f8313bf..7abc714e 100644 --- a/operators/path.md +++ b/operators/path.md @@ -78,7 +78,7 @@ a: ``` then ```bash -yq eval '.a.[] | select(. == "*og") | [{"path":path, "value":.}]' sample.yml +yq eval '.a[] | select(. == "*og") | [{"path":path, "value":.}]' sample.yml ``` will output ```yaml diff --git a/operators/sort.md b/operators/sort.md new file mode 100644 index 00000000..fbecdefc --- /dev/null +++ b/operators/sort.md @@ -0,0 +1,98 @@ +# Sort + +Sorts an array. Use `sort` to sort an array as is, or `sort_by` to sort by a particular subfield. + +Note that at this stage, `yq` only sorts scalar fields. + +## Sort by string field +Given a sample.yml file of: +```yaml +- a: banana +- a: cat +- a: apple +``` +then +```bash +yq eval 'sort_by(.a)' sample.yml +``` +will output +```yaml +- a: apple +- a: banana +- a: cat +``` + +## Sort is stable +Note the order of the elements in unchanged when equal in sorting. + +Given a sample.yml file of: +```yaml +- a: banana + b: 1 +- a: banana + b: 2 +- a: banana + b: 3 +- a: banana + b: 4 +``` +then +```bash +yq eval 'sort_by(.a)' sample.yml +``` +will output +```yaml +- a: banana + b: 1 +- a: banana + b: 2 +- a: banana + b: 3 +- a: banana + b: 4 +``` + +## Sort by numeric field +Given a sample.yml file of: +```yaml +- a: 10 +- a: 100 +- a: 1 +``` +then +```bash +yq eval 'sort_by(.a)' sample.yml +``` +will output +```yaml +- a: 1 +- a: 10 +- a: 100 +``` + +## Sort, nulls come first +Given a sample.yml file of: +```yaml +- 8 +- 3 +- null +- 6 +- true +- false +- cat +``` +then +```bash +yq eval 'sort' sample.yml +``` +will output +```yaml +- null +- false +- true +- 3 +- 6 +- 8 +- cat +``` + diff --git a/usage/csv-tsv.md b/usage/csv-tsv.md new file mode 100644 index 00000000..5a50158c --- /dev/null +++ b/usage/csv-tsv.md @@ -0,0 +1,41 @@ +# Working with CSV and TSV + +## Yaml to CSV/TSV + +You can convert compatible yaml structures to CSV or TSV by using: +- `--outputformat=csv` or `-o=c` for csv (comma separated values) +- `--outputformat=tsv` or `-o=t` for tsv (tab separated values) + +Compatible structures is either an array of scalars (strings/numbers/booleans), which is a single row; or an array of arrays of scalars (multiple rows). + +```yaml +- [i, like, csv] +- [because, excel, is, cool] +``` + +then + +```bash +yq e '.' -o=csv sample.yaml +``` + +will output: + +```csv +i,like,csv +because,excel,is,cool +``` + +Similarly, for tsv: + +```bash +yq e '.' -o=tsv sample.yaml +``` + +will output: + +```tsv +i like csv +because excel is cool +``` +