From a41131816a970d3ffd27b07705239ae16a6c6e87 Mon Sep 17 00:00:00 2001 From: Mike Farah Date: Wed, 3 Nov 2021 15:00:28 +1100 Subject: [PATCH] Update document generation script --- how-it-works.md | 2 +- operators/anchor-and-alias-operators.md | 100 +--------------- operators/assign-update.md | 82 ++----------- operators/boolean-operators.md | 83 +------------ operators/collect-into-array.md | 16 +-- operators/comment-operators.md | 69 ++--------- operators/contains.md | 33 +----- operators/create-collect-into-object.md | 27 +---- operators/delete.md | 43 +------ operators/document-index.md | 31 +---- operators/entries.md | 59 ++-------- operators/env-variable-operators.md | 31 +---- operators/equals.md | 41 +------ operators/file-operators.md | 29 +---- operators/has.md | 20 +--- operators/keys.md | 15 +-- operators/length.md | 25 +--- operators/multiply-merge.md | 102 ++-------------- operators/path.md | 31 +---- operators/pipe.md | 13 +-- operators/recursive-descent-glob.md | 42 +------ operators/reduce.md | 24 +--- operators/select.md | 13 +-- operators/sort-keys.md | 13 +-- operators/split-into-documents.md | 12 +- operators/string-operators.md | 120 ++++--------------- operators/style.md | 73 +----------- operators/subtract.md | 41 +------ operators/tag.md | 19 +-- operators/traverse-read.md | 147 +----------------------- operators/union.md | 11 +- operators/unique.md | 25 +--- operators/variable-operators.md | 31 +---- operators/with.md | 25 +--- 34 files changed, 111 insertions(+), 1337 deletions(-) diff --git a/how-it-works.md b/how-it-works.md index a463883d..01fc048f 100644 --- a/how-it-works.md +++ b/how-it-works.md @@ -118,4 +118,4 @@ The assignment operator then copies across the value from the RHS to the value o ```yaml a: 2 b: thing -``` \ No newline at end of file +``` diff --git a/operators/anchor-and-alias-operators.md b/operators/anchor-and-alias-operators.md index 276ac5f8..484a99bd 100644 --- a/operators/anchor-and-alias-operators.md +++ b/operators/anchor-and-alias-operators.md @@ -4,12 +4,11 @@ Use the `alias` and `anchor` operators to read and write yaml aliases and anchor `yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag. -## Merge one map -see [https://yaml.org/type/merge.html](https://yaml.org/type/merge.html) +## Merge one map +see https://yaml.org/type/merge.html Given a sample.yml file of: - ```yaml - &CENTER x: 1 @@ -24,15 +23,11 @@ Given a sample.yml file of: - !!merge <<: *CENTER r: 10 ``` - then - ```bash yq eval '.[4] | explode(.)' sample.yml ``` - will output - ```yaml x: 1 y: 2 @@ -40,11 +35,9 @@ r: 10 ``` ## Merge multiple maps - -see [https://yaml.org/type/merge.html](https://yaml.org/type/merge.html) +see https://yaml.org/type/merge.html Given a sample.yml file of: - ```yaml - &CENTER x: 1 @@ -60,15 +53,11 @@ Given a sample.yml file of: - *CENTER - *BIG ``` - then - ```bash yq eval '.[4] | explode(.)' sample.yml ``` - will output - ```yaml r: 10 x: 1 @@ -76,11 +65,9 @@ y: 2 ``` ## Override - -see [https://yaml.org/type/merge.html](https://yaml.org/type/merge.html) +see https://yaml.org/type/merge.html Given a sample.yml file of: - ```yaml - &CENTER x: 1 @@ -98,15 +85,11 @@ Given a sample.yml file of: - *SMALL x: 1 ``` - then - ```bash yq eval '.[4] | explode(.)' sample.yml ``` - will output - ```yaml r: 10 x: 1 @@ -114,173 +97,125 @@ y: 2 ``` ## Get anchor - Given a sample.yml file of: - ```yaml a: &billyBob cat ``` - then - ```bash yq eval '.a | anchor' sample.yml ``` - will output - ```yaml billyBob ``` ## Set anchor - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval '.a anchor = "foobar"' sample.yml ``` - will output - ```yaml a: &foobar cat ``` ## Set anchor relatively using assign-update - Given a sample.yml file of: - ```yaml a: b: cat ``` - then - ```bash yq eval '.a anchor |= .b' sample.yml ``` - will output - ```yaml a: &cat b: cat ``` ## Get alias - Given a sample.yml file of: - ```yaml b: &billyBob meow a: *billyBob ``` - then - ```bash yq eval '.a | alias' sample.yml ``` - will output - ```yaml billyBob ``` ## Set alias - Given a sample.yml file of: - ```yaml b: &meow purr a: cat ``` - then - ```bash yq eval '.a alias = "meow"' sample.yml ``` - will output - ```yaml b: &meow purr a: *meow ``` ## Set alias to blank does nothing - Given a sample.yml file of: - ```yaml b: &meow purr a: cat ``` - then - ```bash yq eval '.a alias = ""' sample.yml ``` - will output - ```yaml b: &meow purr a: cat ``` ## Set alias relatively using assign-update - Given a sample.yml file of: - ```yaml b: &meow purr a: f: meow ``` - then - ```bash yq eval '.a alias |= .f' sample.yml ``` - will output - ```yaml b: &meow purr a: *meow ``` ## Explode alias and anchor - Given a sample.yml file of: - ```yaml f: a: &a cat b: *a ``` - then - ```bash yq eval 'explode(.f)' sample.yml ``` - will output - ```yaml f: a: cat @@ -288,43 +223,31 @@ f: ``` ## Explode with no aliases or anchors - Given a sample.yml file of: - ```yaml a: mike ``` - then - ```bash yq eval 'explode(.a)' sample.yml ``` - will output - ```yaml a: mike ``` ## Explode with alias keys - Given a sample.yml file of: - ```yaml f: a: &a cat *a: b ``` - then - ```bash yq eval 'explode(.f)' sample.yml ``` - will output - ```yaml f: a: cat @@ -332,9 +255,7 @@ f: ``` ## Explode with merge anchors - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -355,15 +276,11 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval 'explode(.)' sample.yml ``` - will output - ```yaml foo: a: foo_a @@ -385,11 +302,9 @@ foobar: ``` ## Dereference and update a field - -\`Use explode with multiply to dereference an object +`Use explode with multiply to dereference an object Given a sample.yml file of: - ```yaml item_value: &item_value value: true @@ -400,15 +315,11 @@ thingTwo: name: item_2 !!merge <<: *item_value ``` - then - ```bash yq eval '.thingOne |= explode(.) * {"value": false}' sample.yml ``` - will output - ```yaml item_value: &item_value value: true @@ -419,3 +330,4 @@ thingTwo: name: item_2 !!merge <<: *item_value ``` + diff --git a/operators/assign-update.md b/operators/assign-update.md index 23f2846c..ad71c81b 100644 --- a/operators/assign-update.md +++ b/operators/assign-update.md @@ -1,15 +1,18 @@ # Assign (Update) +This operator is used to update node values. It can be used in either the: + +### plain form: `=` +Which will assign the LHS node values to the RHS node values. The RHS expression is run against the matching nodes in the pipeline. + +### relative form: `|=` +This will do a similar thing to the plain form, however, the RHS expression is run against _the LHS nodes_. This is useful for updating values based on old values, e.g. increment. ## Create yaml file - Running - ```bash yq eval --null-input '.a.b = "cat" | .x = "frog"' ``` - will output - ```yaml a: b: cat @@ -17,46 +20,34 @@ x: frog ``` ## Update node to be the child value - Given a sample.yml file of: - ```yaml a: b: g: foof ``` - then - ```bash yq eval '.a |= .b' sample.yml ``` - will output - ```yaml a: g: foof ``` ## Double elements in an array - Given a sample.yml file of: - ```yaml - 1 - 2 - 3 ``` - then - ```bash yq eval '.[] |= . * 2' sample.yml ``` - will output - ```yaml - 2 - 4 @@ -64,75 +55,55 @@ will output ``` ## Update node from another file - Note this will also work when the second file is a scalar (string/number) Given a sample.yml file of: - ```yaml a: apples ``` - And another sample another.yml file of: - ```yaml b: bob ``` - then - ```bash yq eval-all 'select(fileIndex==0).a = select(fileIndex==1) | select(fileIndex==0)' sample.yml another.yml ``` - will output - ```yaml a: b: bob ``` ## Update node to be the sibling value - Given a sample.yml file of: - ```yaml a: b: child b: sibling ``` - then - ```bash yq eval '.a = .b' sample.yml ``` - will output - ```yaml a: sibling b: sibling ``` ## Updated multiple paths - Given a sample.yml file of: - ```yaml a: fieldA b: fieldB c: fieldC ``` - then - ```bash yq eval '(.a, .c) = "potatoe"' sample.yml ``` - will output - ```yaml a: potatoe b: fieldB @@ -140,69 +111,53 @@ c: potatoe ``` ## Update string value - Given a sample.yml file of: - ```yaml a: b: apple ``` - then - ```bash yq eval '.a.b = "frog"' sample.yml ``` - will output - ```yaml a: b: frog ``` ## Update string value via |= - Note there is no difference between `=` and `|=` when the RHS is a scalar Given a sample.yml file of: - ```yaml a: b: apple ``` - then - ```bash yq eval '.a.b |= "frog"' sample.yml ``` - will output - ```yaml a: b: frog ``` -## Update selected results +## Update deeply selected results +Note that the LHS is wrapped in brackets! This is to ensure we dont first filter out the yaml and then update the snippet. Given a sample.yml file of: - ```yaml a: b: apple c: cactus ``` - then - ```bash yq eval '(.a[] | select(. == "apple")) = "frog"' sample.yml ``` - will output - ```yaml a: b: frog @@ -210,23 +165,17 @@ a: ``` ## Update array values - Given a sample.yml file of: - ```yaml - candy - apple - sandy ``` - then - ```bash yq eval '(.[] | select(. == "*andy")) = "bogs"' sample.yml ``` - will output - ```yaml - bogs - apple @@ -234,41 +183,30 @@ will output ``` ## Update empty object - Given a sample.yml file of: - ```yaml {} ``` - then - ```bash yq eval '.a.b |= "bogs"' sample.yml ``` - will output - ```yaml {a: {b: bogs}} ``` ## Update empty object and array - Given a sample.yml file of: - ```yaml {} ``` - then - ```bash yq eval '.a.b.[0] |= "bogs"' sample.yml ``` - will output - ```yaml {a: {b: [bogs]}} ``` + diff --git a/operators/boolean-operators.md b/operators/boolean-operators.md index 14af1c84..c60f26a8 100644 --- a/operators/boolean-operators.md +++ b/operators/boolean-operators.md @@ -1,8 +1,8 @@ # Boolean Operators -The `or` and `and` operators take two parameters and return a boolean result. +The `or` and `and` operators take two parameters and return a boolean result. -`not` flips a boolean from true to false, or vice versa. +`not` flips a boolean from true to false, or vice versa. `any` will return `true` if there are any `true` values in a array sequence, and `all` will return true if _all_ elements in an array are true. @@ -11,37 +11,27 @@ The `or` and `and` operators take two parameters and return a boolean result. These are most commonly used with the `select` operator to filter particular nodes. ## `or` example - Running - ```bash yq eval --null-input 'true or false' ``` - will output - ```yaml true ``` ## `and` example - Running - ```bash yq eval --null-input 'true and false' ``` - will output - ```yaml false ``` ## Matching nodes with select, equals and or - Given a sample.yml file of: - ```yaml - a: bird b: dog @@ -50,15 +40,11 @@ Given a sample.yml file of: - a: cat b: fly ``` - then - ```bash yq eval '[.[] | select(.a == "cat" or .b == "dog")]' sample.yml ``` - will output - ```yaml - a: bird b: dog @@ -67,50 +53,36 @@ will output ``` ## `any` returns true if any boolean in a given array is true - Given a sample.yml file of: - ```yaml - false - true ``` - then - ```bash yq eval 'any' sample.yml ``` - will output - ```yaml true ``` ## `any` returns false for an empty array - Given a sample.yml file of: - ```yaml [] ``` - then - ```bash yq eval 'any' sample.yml ``` - will output - ```yaml false ``` ## `any_c` returns true if any element in the array is true for the given condition. - Given a sample.yml file of: - ```yaml a: - rad @@ -119,65 +91,47 @@ b: - meh - whatever ``` - then - ```bash yq eval '.[] |= any_c(. == "awesome")' sample.yml ``` - will output - ```yaml a: true b: false ``` ## `all` returns true if all booleans in a given array are true - Given a sample.yml file of: - ```yaml - true - true ``` - then - ```bash yq eval 'all' sample.yml ``` - will output - ```yaml true ``` ## `all` returns true for an empty array - Given a sample.yml file of: - ```yaml [] ``` - then - ```bash yq eval 'all' sample.yml ``` - will output - ```yaml true ``` ## `all_c` returns true if all elements in the array are true for the given condition. - Given a sample.yml file of: - ```yaml a: - rad @@ -186,114 +140,83 @@ b: - meh - 12 ``` - then - ```bash yq eval '.[] |= all_c(tag == "!!str")' sample.yml ``` - will output - ```yaml a: true b: false ``` ## Not true is false - Running - ```bash yq eval --null-input 'true | not' ``` - will output - ```yaml false ``` ## Not false is true - Running - ```bash yq eval --null-input 'false | not' ``` - will output - ```yaml true ``` ## String values considered to be true - Running - ```bash yq eval --null-input '"cat" | not' ``` - will output - ```yaml false ``` ## Empty string value considered to be true - Running - ```bash yq eval --null-input '"" | not' ``` - will output - ```yaml false ``` ## Numbers are considered to be true - Running - ```bash yq eval --null-input '1 | not' ``` - will output - ```yaml false ``` ## Zero is considered to be true - Running - ```bash yq eval --null-input '0 | not' ``` - will output - ```yaml false ``` ## Null is considered to be false - Running - ```bash yq eval --null-input '~ | not' ``` - will output - ```yaml true ``` + diff --git a/operators/collect-into-array.md b/operators/collect-into-array.md index 49000d61..8195ee9a 100644 --- a/operators/collect-into-array.md +++ b/operators/collect-into-array.md @@ -2,52 +2,40 @@ This creates an array using the expression between the square brackets. + ## Collect empty - Running - ```bash yq eval --null-input '[]' ``` - will output - ```yaml [] ``` ## Collect single - Running - ```bash yq eval --null-input '["cat"]' ``` - will output - ```yaml - cat ``` ## Collect many - Given a sample.yml file of: - ```yaml a: cat b: dog ``` - then - ```bash yq eval '[.a, .b]' sample.yml ``` - will output - ```yaml - cat - dog ``` + diff --git a/operators/comment-operators.md b/operators/comment-operators.md index cfaa5bb3..507a6a7f 100644 --- a/operators/comment-operators.md +++ b/operators/comment-operators.md @@ -5,71 +5,51 @@ Use these comment operators to set or retrieve comments. Like the `=` and `|=` assign operators, the same syntax applies when updating comments: ### plain form: `=` - This will assign the LHS nodes comments to the expression on the RHS. The RHS is run against the matching nodes in the pipeline -### relative form: `|=` - +### relative form: `|=` Similar to the plain form, however the RHS evaluates against each matching LHS node! This is useful if you want to set the comments as a relative expression of the node, for instance its value or path. ## Set line comment - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval '.a lineComment="single"' sample.yml ``` - will output - ```yaml a: cat # single ``` ## Use update assign to perform relative updates - Given a sample.yml file of: - ```yaml a: cat b: dog ``` - then - ```bash yq eval '.. lineComment |= .' sample.yml ``` - will output - ```yaml a: cat # cat b: dog # dog ``` ## Set head comment - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval '. headComment="single"' sample.yml ``` - will output - ```yaml # single @@ -77,21 +57,15 @@ a: cat ``` ## Set foot comment, using an expression - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval '. footComment=.a' sample.yml ``` - will output - ```yaml a: cat @@ -99,106 +73,87 @@ a: cat ``` ## Remove comment - Given a sample.yml file of: - ```yaml a: cat # comment b: dog # leave this ``` - then - ```bash yq eval '.a lineComment=""' sample.yml ``` - will output - ```yaml a: cat b: dog # leave this ``` ## Remove (strip) all comments - Note the use of `...` to ensure key nodes are included. Given a sample.yml file of: - ```yaml a: cat # comment # great b: # key comment ``` - then - ```bash yq eval '... comments=""' sample.yml ``` - will output - ```yaml a: cat b: ``` ## Get line comment - Given a sample.yml file of: - ```yaml a: cat # meow ``` - then - ```bash yq eval '.a | lineComment' sample.yml ``` - will output - ```yaml meow ``` ## Get head comment - Given a sample.yml file of: - ```yaml +# welcome! + a: cat # meow + +# have a great day ``` - then - ```bash yq eval '. | headComment' sample.yml ``` - will output - ```yaml +welcome! ``` ## Get foot comment - Given a sample.yml file of: - ```yaml +# welcome! + a: cat # meow + +# have a great day ``` - then - ```bash yq eval '. | footComment' sample.yml ``` - will output - ```yaml +have a great day ``` + diff --git a/operators/contains.md b/operators/contains.md index 686b3de4..5f3c7f3b 100644 --- a/operators/contains.md +++ b/operators/contains.md @@ -1,33 +1,27 @@ # Contains -## Array contains array +This returns `true` if the context contains the passed in parameter, and false otherwise. +## Array contains array Array is equal or subset of Given a sample.yml file of: - ```yaml - foobar - foobaz - blarp ``` - then - ```bash yq eval 'contains(["baz", "bar"])' sample.yml ``` - will output - ```yaml true ``` ## Object included in array - Given a sample.yml file of: - ```yaml "foo": 12 "bar": @@ -36,23 +30,17 @@ Given a sample.yml file of: - "barp": 12 "blip": 13 ``` - then - ```bash yq eval 'contains({"bar": [{"barp": 12}]})' sample.yml ``` - will output - ```yaml true ``` ## Object not included in array - Given a sample.yml file of: - ```yaml "foo": 12 "bar": @@ -61,55 +49,40 @@ Given a sample.yml file of: - "barp": 12 "blip": 13 ``` - then - ```bash yq eval 'contains({"foo": 12, "bar": [{"barp": 15}]})' sample.yml ``` - will output - ```yaml false ``` ## String contains substring - Given a sample.yml file of: - ```yaml foobar ``` - then - ```bash yq eval 'contains("bar")' sample.yml ``` - will output - ```yaml true ``` ## String equals string - Given a sample.yml file of: - ```yaml meow ``` - then - ```bash yq eval 'contains("meow")' sample.yml ``` - will output - ```yaml true ``` + diff --git a/operators/create-collect-into-object.md b/operators/create-collect-into-object.md index f8133793..66c446af 100644 --- a/operators/create-collect-into-object.md +++ b/operators/create-collect-into-object.md @@ -3,68 +3,50 @@ This is used to construct objects (or maps). This can be used against existing yaml, or to create fresh yaml documents. ## Collect empty object - Running - ```bash yq eval --null-input '{}' ``` - will output - ```yaml {} ``` ## Wrap (prefix) existing object - Given a sample.yml file of: - ```yaml name: Mike ``` - then - ```bash yq eval '{"wrap": .}' sample.yml ``` - will output - ```yaml wrap: name: Mike ``` ## Using splat to create multiple objects - Given a sample.yml file of: - ```yaml name: Mike pets: - cat - dog ``` - then - ```bash yq eval '{.name: .pets.[]}' sample.yml ``` - will output - ```yaml Mike: cat Mike: dog ``` ## Working with multiple documents - Given a sample.yml file of: - ```yaml name: Mike pets: @@ -76,15 +58,11 @@ pets: - monkey - sheep ``` - then - ```bash yq eval '{.name: .pets.[]}' sample.yml ``` - will output - ```yaml Mike: cat Mike: dog @@ -93,15 +71,12 @@ Rosey: sheep ``` ## Creating yaml from scratch - Running - ```bash yq eval --null-input '{"wrap": "frog"}' ``` - will output - ```yaml wrap: frog ``` + diff --git a/operators/delete.md b/operators/delete.md index eeb4ef49..a7073327 100644 --- a/operators/delete.md +++ b/operators/delete.md @@ -3,141 +3,103 @@ Deletes matching entries in maps or arrays. ## Delete entry in map - Given a sample.yml file of: - ```yaml a: cat b: dog ``` - then - ```bash yq eval 'del(.b)' sample.yml ``` - will output - ```yaml a: cat ``` ## Delete nested entry in map - Given a sample.yml file of: - ```yaml a: a1: fred a2: frood ``` - then - ```bash yq eval 'del(.a.a1)' sample.yml ``` - will output - ```yaml a: a2: frood ``` ## Delete entry in array - Given a sample.yml file of: - ```yaml - 1 - 2 - 3 ``` - then - ```bash yq eval 'del(.[1])' sample.yml ``` - will output - ```yaml - 1 - 3 ``` ## Delete nested entry in array - Given a sample.yml file of: - ```yaml - a: cat b: dog ``` - then - ```bash yq eval 'del(.[0].a)' sample.yml ``` - will output - ```yaml - b: dog ``` ## Delete no matches - Given a sample.yml file of: - ```yaml a: cat b: dog ``` - then - ```bash yq eval 'del(.c)' sample.yml ``` - will output - ```yaml a: cat b: dog ``` ## Delete matching entries - Given a sample.yml file of: - ```yaml a: cat b: dog c: bat ``` - then - ```bash yq eval 'del( .[] | select(. == "*at") )' sample.yml ``` - will output - ```yaml b: dog ``` ## Recursively delete matching keys - Given a sample.yml file of: - ```yaml a: name: frog @@ -145,17 +107,14 @@ a: name: blog age: 12 ``` - then - ```bash yq eval 'del(.. | select(has("name")).name)' sample.yml ``` - will output - ```yaml a: b: age: 12 ``` + diff --git a/operators/document-index.md b/operators/document-index.md index f7e2eeb9..acd479b5 100644 --- a/operators/document-index.md +++ b/operators/document-index.md @@ -3,23 +3,17 @@ Use the `documentIndex` operator (or the `di` shorthand) to select nodes of a particular document. ## Retrieve a document index - Given a sample.yml file of: - ```yaml a: cat --- a: frog ``` - then - ```bash yq eval '.a | documentIndex' sample.yml ``` - will output - ```yaml 0 --- @@ -27,23 +21,17 @@ will output ``` ## Retrieve a document index, shorthand - Given a sample.yml file of: - ```yaml a: cat --- a: frog ``` - then - ```bash yq eval '.a | di' sample.yml ``` - will output - ```yaml 0 --- @@ -51,70 +39,53 @@ will output ``` ## Filter by document index - Given a sample.yml file of: - ```yaml a: cat --- a: frog ``` - then - ```bash yq eval 'select(documentIndex == 1)' sample.yml ``` - will output - ```yaml a: frog ``` ## Filter by document index shorthand - Given a sample.yml file of: - ```yaml a: cat --- a: frog ``` - then - ```bash yq eval 'select(di == 1)' sample.yml ``` - will output - ```yaml a: frog ``` ## Print Document Index with matches - Given a sample.yml file of: - ```yaml a: cat --- a: frog ``` - then - ```bash yq eval '.a | ({"match": ., "doc": documentIndex})' sample.yml ``` - will output - ```yaml match: cat doc: 0 match: frog doc: 1 ``` + diff --git a/operators/entries.md b/operators/entries.md index f04ae609..95fae132 100644 --- a/operators/entries.md +++ b/operators/entries.md @@ -2,23 +2,17 @@ Similar to the same named functions in `jq` these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps. -## to\_entries Map - +## to_entries Map Given a sample.yml file of: - ```yaml a: 1 b: 2 ``` - then - ```bash yq eval 'to_entries' sample.yml ``` - will output - ```yaml - key: a value: 1 @@ -26,23 +20,17 @@ will output value: 2 ``` -## to\_entries Array - +## to_entries Array Given a sample.yml file of: - ```yaml - a - b ``` - then - ```bash yq eval 'to_entries' sample.yml ``` - will output - ```yaml - key: 0 value: a @@ -50,113 +38,84 @@ will output value: b ``` -## to\_entries null - +## to_entries null Given a sample.yml file of: - ```yaml null ``` - then - ```bash yq eval 'to_entries' sample.yml ``` - will output - ```yaml ``` -## from\_entries map - +## from_entries map Given a sample.yml file of: - ```yaml a: 1 b: 2 ``` - then - ```bash yq eval 'to_entries | from_entries' sample.yml ``` - will output - ```yaml a: 1 b: 2 ``` -## from\_entries with numeric key indexes - -from\_entries always creates a map, even for numeric keys +## from_entries with numeric key indexes +from_entries always creates a map, even for numeric keys Given a sample.yml file of: - ```yaml - a - b ``` - then - ```bash yq eval 'to_entries | from_entries' sample.yml ``` - will output - ```yaml 0: a 1: b ``` -## Use with\_entries to update keys - +## Use with_entries to update keys Given a sample.yml file of: - ```yaml a: 1 b: 2 ``` - then - ```bash yq eval 'with_entries(.key |= "KEY_" + .)' sample.yml ``` - will output - ```yaml KEY_a: 1 KEY_b: 2 ``` -## Use with\_entries to filter the map - +## Use with_entries to filter the map Given a sample.yml file of: - ```yaml a: b: bird c: d: dog ``` - then - ```bash yq eval 'with_entries(select(.value | has("b")))' sample.yml ``` - will output - ```yaml a: b: bird ``` + diff --git a/operators/env-variable-operators.md b/operators/env-variable-operators.md index 92f16b24..e4a1817f 100644 --- a/operators/env-variable-operators.md +++ b/operators/env-variable-operators.md @@ -3,106 +3,77 @@ This operator is used to handle environment variables usage in path expressions. While environment variables can, of course, be passed in via your CLI with string interpolation, this often comes with complex quote escaping and can be tricky to write and read. Note that there are two forms, `env` which will parse the environment variable as a yaml (be it a map, array, string, number of boolean) and `strenv` which will always parse the argument as a string. ## Read string environment variable - Running - ```bash myenv="cat meow" yq eval --null-input '.a = env(myenv)' ``` - will output - ```yaml a: cat meow ``` ## Read boolean environment variable - Running - ```bash myenv="true" yq eval --null-input '.a = env(myenv)' ``` - will output - ```yaml a: true ``` ## Read numeric environment variable - Running - ```bash myenv="12" yq eval --null-input '.a = env(myenv)' ``` - will output - ```yaml a: 12 ``` ## Read yaml environment variable - Running - ```bash myenv="{b: fish}" yq eval --null-input '.a = env(myenv)' ``` - will output - ```yaml a: {b: fish} ``` ## Read boolean environment variable as a string - Running - ```bash myenv="true" yq eval --null-input '.a = strenv(myenv)' ``` - will output - ```yaml a: "true" ``` ## Read numeric environment variable as a string - Running - ```bash myenv="12" yq eval --null-input '.a = strenv(myenv)' ``` - will output - ```yaml a: "12" ``` ## Dynamic key lookup with environment variable - Given a sample.yml file of: - ```yaml cat: meow dog: woof ``` - then - ```bash myenv="cat" yq eval '.[env(myenv)]' sample.yml ``` - will output - ```yaml meow ``` + diff --git a/operators/equals.md b/operators/equals.md index 59029159..d7129a79 100644 --- a/operators/equals.md +++ b/operators/equals.md @@ -13,23 +13,17 @@ select(.a == .b) ``` ## Match string - Given a sample.yml file of: - ```yaml - cat - goat - dog ``` - then - ```bash yq eval '.[] | (. == "*at")' sample.yml ``` - will output - ```yaml true true @@ -37,23 +31,17 @@ false ``` ## Don't match string - Given a sample.yml file of: - ```yaml - cat - goat - dog ``` - then - ```bash yq eval '.[] | (. != "*at")' sample.yml ``` - will output - ```yaml false false @@ -61,23 +49,17 @@ true ``` ## Match number - Given a sample.yml file of: - ```yaml - 3 - 4 - 5 ``` - then - ```bash yq eval '.[] | (. == 4)' sample.yml ``` - will output - ```yaml false true @@ -85,23 +67,17 @@ false ``` ## Dont match number - Given a sample.yml file of: - ```yaml - 3 - 4 - 5 ``` - then - ```bash yq eval '.[] | (. != 4)' sample.yml ``` - will output - ```yaml true false @@ -109,55 +85,40 @@ true ``` ## Match nulls - Running - ```bash yq eval --null-input 'null == ~' ``` - will output - ```yaml true ``` ## Non exisitant key doesn't equal a value - Given a sample.yml file of: - ```yaml a: frog ``` - then - ```bash yq eval 'select(.b != "thing")' sample.yml ``` - will output - ```yaml a: frog ``` ## Two non existant keys are equal - Given a sample.yml file of: - ```yaml a: frog ``` - then - ```bash yq eval 'select(.b == .c)' sample.yml ``` - will output - ```yaml a: frog ``` + diff --git a/operators/file-operators.md b/operators/file-operators.md index bcca08db..a5e2bef3 100644 --- a/operators/file-operators.md +++ b/operators/file-operators.md @@ -5,75 +5,53 @@ File operators are most often used with merge when needing to merge specific fil Note that the `fileIndex` operator has a short alias of `fi`. ## Merging files - Note the use of eval-all to ensure all documents are loaded into memory. - ```bash yq eval-all 'select(fi == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml ``` ## Get filename - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval 'filename' sample.yml ``` - will output - ```yaml sample.yml ``` ## Get file index - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval 'fileIndex' sample.yml ``` - will output - ```yaml 0 ``` ## Get file indices of multiple documents - Given a sample.yml file of: - ```yaml a: cat ``` - And another sample another.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval-all 'fileIndex' sample.yml another.yml ``` - will output - ```yaml 0 --- @@ -81,21 +59,16 @@ will output ``` ## Get file index alias - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval 'fi' sample.yml ``` - will output - ```yaml 0 ``` + diff --git a/operators/has.md b/operators/has.md index afde5bc5..00cd5feb 100644 --- a/operators/has.md +++ b/operators/has.md @@ -3,24 +3,18 @@ This is operation that returns true if the key exists in a map (or index in an array), false otherwise. ## Has map key - Given a sample.yml file of: - ```yaml - a: yes - a: ~ - a: - b: nope ``` - then - ```bash yq eval '.[] | has("a")' sample.yml ``` - will output - ```yaml true true @@ -29,11 +23,9 @@ false ``` ## Select, checking for existence of deep paths - Simply pipe in parent expressions into `has` Given a sample.yml file of: - ```yaml - a: b: @@ -42,15 +34,11 @@ Given a sample.yml file of: b: d: dog ``` - then - ```bash yq eval '.[] | select(.a.b | has("c"))' sample.yml ``` - will output - ```yaml a: b: @@ -58,25 +46,20 @@ a: ``` ## Has array index - Given a sample.yml file of: - ```yaml - [] - [1] - [1, 2] - [1, null] - [1, 2, 3] + ``` - then - ```bash yq eval '.[] | has(1)' sample.yml ``` - will output - ```yaml false false @@ -84,3 +67,4 @@ true true true ``` + diff --git a/operators/keys.md b/operators/keys.md index 119f5972..9d1cea1b 100644 --- a/operators/keys.md +++ b/operators/keys.md @@ -1,47 +1,36 @@ # Keys -Use the `keys` operator to return map keys or array indices. +Use the `keys` operator to return map keys or array indices. ## Map keys - Given a sample.yml file of: - ```yaml dog: woof cat: meow ``` - then - ```bash yq eval 'keys' sample.yml ``` - will output - ```yaml - dog - cat ``` ## Array keys - Given a sample.yml file of: - ```yaml - apple - banana ``` - then - ```bash yq eval 'keys' sample.yml ``` - will output - ```yaml - 0 - 1 ``` + diff --git a/operators/length.md b/operators/length.md index 035c001d..b4e78276 100644 --- a/operators/length.md +++ b/operators/length.md @@ -3,91 +3,68 @@ Returns the lengths of the nodes. Length is defined according to the type of the node. ## String length - returns length of string Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval '.a | length' sample.yml ``` - will output - ```yaml 3 ``` ## null length - Given a sample.yml file of: - ```yaml a: null ``` - then - ```bash yq eval '.a | length' sample.yml ``` - will output - ```yaml 0 ``` ## Map length - returns number of entries Given a sample.yml file of: - ```yaml a: cat c: dog ``` - then - ```bash yq eval 'length' sample.yml ``` - will output - ```yaml 2 ``` ## Array length - returns number of elements Given a sample.yml file of: - ```yaml - 2 - 4 - 6 - 8 ``` - then - ```bash yq eval 'length' sample.yml ``` - will output - ```yaml 4 ``` + diff --git a/operators/multiply-merge.md b/operators/multiply-merge.md index 92aac7ad..ffc36b11 100644 --- a/operators/multiply-merge.md +++ b/operators/multiply-merge.md @@ -3,21 +3,18 @@ Like the multiple operator in jq, depending on the operands, this multiply operator will do different things. Currently numbers, arrays and objects are supported. ## Objects and arrays - merging - Objects are merged deeply matching on matching keys. By default, array values override and are not deeply merged. Note that when merging objects, this operator returns the merged object (not the parent). This will be clearer in the examples below. ### Merge Flags +You can control how objects are merged by using one or more of the following flags. Multiple flags can be used together, e.g. `.a *+? .b`. See examples below -You can control how objects are merged by using one or more of the following flags. Multiple flags can be used together, e.g. `.a *+? .b`. See examples below - -* `+` to append arrays -* `?` to only merge existing fields -* `d` to deeply merge arrays +- `+` to append arrays +- `?` to only merge existing fields +- `d` to deeply merge arrays ### Merging files - Note the use of `eval-all` to ensure all documents are loaded into memory. ```bash @@ -25,23 +22,17 @@ yq eval-all 'select(fileIndex == 0) * select(fileIndex == 1)' file1.yaml file2.y ``` ## Multiply integers - Running - ```bash yq eval --null-input '3 * 4' ``` - will output - ```yaml 12 ``` ## Merge objects together, returning merged result only - Given a sample.yml file of: - ```yaml a: field: me @@ -51,15 +42,11 @@ b: g: wizz fieldB: dog ``` - then - ```bash yq eval '.a * .b' sample.yml ``` - will output - ```yaml field: g: wizz @@ -68,9 +55,7 @@ fieldB: dog ``` ## Merge objects together, returning parent object - Given a sample.yml file of: - ```yaml a: field: me @@ -80,15 +65,11 @@ b: g: wizz fieldB: dog ``` - then - ```bash yq eval '. * {"a":.b}' sample.yml ``` - will output - ```yaml a: field: @@ -102,23 +83,17 @@ b: ``` ## Merge keeps style of LHS - Given a sample.yml file of: - ```yaml a: {things: great} b: also: "me" ``` - then - ```bash yq eval '. * {"a":.b}' sample.yml ``` - will output - ```yaml a: {things: great, also: "me"} b: @@ -126,9 +101,7 @@ b: ``` ## Merge arrays - Given a sample.yml file of: - ```yaml a: - 1 @@ -139,15 +112,11 @@ b: - 4 - 5 ``` - then - ```bash yq eval '. * {"a":.b}' sample.yml ``` - will output - ```yaml a: - 3 @@ -160,9 +129,7 @@ b: ``` ## Merge, only existing fields - Given a sample.yml file of: - ```yaml a: thing: one @@ -171,24 +138,18 @@ b: missing: two thing: two ``` - then - ```bash yq eval '.a *? .b' sample.yml ``` - will output - ```yaml thing: two cat: frog ``` ## Merge, appending arrays - Given a sample.yml file of: - ```yaml a: array: @@ -203,15 +164,11 @@ b: - animal: cat value: banana ``` - then - ```bash yq eval '.a *+ .b' sample.yml ``` - will output - ```yaml array: - 1 @@ -224,9 +181,7 @@ value: banana ``` ## Merge, only existing fields, appending arrays - Given a sample.yml file of: - ```yaml a: thing: @@ -239,15 +194,11 @@ b: another: - 1 ``` - then - ```bash yq eval '.a *?+ .b' sample.yml ``` - will output - ```yaml thing: - 1 @@ -257,11 +208,9 @@ thing: ``` ## Merge, deeply merging arrays - Merging arrays deeply means arrays are merge like objects, with indexes as their key. In this case, we merge the first item in the array, and do nothing with the second. Given a sample.yml file of: - ```yaml a: - name: fred @@ -272,15 +221,11 @@ b: - name: fred age: 34 ``` - then - ```bash yq eval '.a *d .b' sample.yml ``` - will output - ```yaml - name: fred age: 34 @@ -289,11 +234,13 @@ will output ``` ## Merge arrays of objects together, matching on a key +It's a complex command, the trickyness comes from needing to have the right context in the expressions. +First we save the second array into a variable '$two' which lets us reference it later. +We then need to update the first array. We will use the relative update (|=) because we need to update relative to the current element of the array in the LHS in the RHS expression. +We set the current element of the first array as $cur. Now we multiply (merge) $cur with the matching entry in $two, by passing $two through a select filter. -It's a complex command, the trickyness comes from needing to have the right context in the expressions. First we save the second array into a variable '$two' which lets us reference it later. We then need to update the first array. We will use the relative update (|=) because we need to update relative to the current element of the array in the LHS in the RHS expression. We set the current element of the first array as $cur. Now we multiply (merge) $cur with the matching entry in $two, by passing $two through a select filter. Given a sample.yml file of: - ```yaml - a: apple b: appleB @@ -302,9 +249,7 @@ Given a sample.yml file of: - a: banana b: bananaB ``` - And another sample another.yml file of: - ```yaml - a: banana c: bananaC @@ -313,15 +258,11 @@ And another sample another.yml file of: - a: dingo c: dingoC ``` - then - ```bash yq eval-all '(select(fi==1) | .[]) as $two | select(fi==0) | .[] |= (. as $cur | $cur * ($two | select(.a == $cur.a)))' sample.yml another.yml ``` - will output - ```yaml - a: apple b: appleB2 @@ -333,22 +274,16 @@ will output ``` ## Merge to prefix an element - Given a sample.yml file of: - ```yaml a: cat b: dog ``` - then - ```bash yq eval '. * {"a": {"c": .a}}' sample.yml ``` - will output - ```yaml a: c: cat @@ -356,9 +291,7 @@ b: dog ``` ## Merge with simple aliases - Given a sample.yml file of: - ```yaml a: &cat c: frog @@ -367,24 +300,18 @@ b: c: g: thongs ``` - then - ```bash yq eval '.c * .b' sample.yml ``` - will output - ```yaml g: thongs f: *cat ``` ## Merge copies anchor names - Given a sample.yml file of: - ```yaml a: c: &cat frog @@ -393,24 +320,18 @@ b: c: g: thongs ``` - then - ```bash yq eval '.c * .a' sample.yml ``` - will output - ```yaml g: thongs c: &cat frog ``` ## Merge with merge anchors - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -431,20 +352,17 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobar * .foobarList' sample.yml ``` - will output - ```yaml c: foobarList_c -<<: +!!merge <<: - *foo - *bar thing: foobar_thing b: foobarList_b ``` + diff --git a/operators/path.md b/operators/path.md index 4af4a095..3f8313bf 100644 --- a/operators/path.md +++ b/operators/path.md @@ -5,112 +5,82 @@ The path operator can be used to get the traversal paths of matching nodes in an You can get the key/index of matching nodes by using the `path` operator to return the path array then piping that through `.[-1]` to get the last element of that array, the key. ## Map path - Given a sample.yml file of: - ```yaml a: b: cat ``` - then - ```bash yq eval '.a.b | path' sample.yml ``` - will output - ```yaml - a - b ``` ## Get map key - Given a sample.yml file of: - ```yaml a: b: cat ``` - then - ```bash yq eval '.a.b | path | .[-1]' sample.yml ``` - will output - ```yaml b ``` ## Array path - Given a sample.yml file of: - ```yaml a: - cat - dog ``` - then - ```bash yq eval '.a.[] | select(. == "dog") | path' sample.yml ``` - will output - ```yaml - a - 1 ``` ## Get array index - Given a sample.yml file of: - ```yaml a: - cat - dog ``` - then - ```bash yq eval '.a.[] | select(. == "dog") | path | .[-1]' sample.yml ``` - will output - ```yaml 1 ``` ## Print path and value - Given a sample.yml file of: - ```yaml a: - cat - dog - frog ``` - then - ```bash yq eval '.a.[] | select(. == "*og") | [{"path":path, "value":.}]' sample.yml ``` - will output - ```yaml - path: - a @@ -121,3 +91,4 @@ will output - 2 value: frog ``` + diff --git a/operators/pipe.md b/operators/pipe.md index 7b9b97eb..b6da327d 100644 --- a/operators/pipe.md +++ b/operators/pipe.md @@ -3,46 +3,35 @@ Pipe the results of an expression into another. Like the bash operator. ## Simple Pipe - Given a sample.yml file of: - ```yaml a: b: cat ``` - then - ```bash yq eval '.a | .b' sample.yml ``` - will output - ```yaml cat ``` ## Multiple updates - Given a sample.yml file of: - ```yaml a: cow b: sheep c: same ``` - then - ```bash yq eval '.a = "cat" | .b = "dog"' sample.yml ``` - will output - ```yaml a: cat b: dog c: same ``` + diff --git a/operators/recursive-descent-glob.md b/operators/recursive-descent-glob.md index d2f55302..4edede1e 100644 --- a/operators/recursive-descent-glob.md +++ b/operators/recursive-descent-glob.md @@ -1,9 +1,8 @@ # Recursive Descent (Glob) -This operator recursively matches (or globs) all children nodes given of a particular element, including that node itself. This is most often used to apply a filter recursively against all matches. It can be used in either the +This operator recursively matches (or globs) all children nodes given of a particular element, including that node itself. This is most often used to apply a filter recursively against all matches. It can be used in either the ## match values form `..` - This will, like the `jq` equivalent, recursively match all _value_ nodes. Use it to find/manipulate particular values. For instance to set the `style` of all _value_ nodes in a yaml doc, excluding map keys: @@ -13,7 +12,6 @@ yq eval '.. style= "flow"' file.yaml ``` ## match values and map keys form `...` - The also includes map keys in the results set. This is particularly useful in YAML as unlike JSON, map keys can have their own styling, tags and use anchors and aliases. For instance to set the `style` of all nodes in a yaml doc, including the map keys: @@ -21,34 +19,25 @@ For instance to set the `style` of all nodes in a yaml doc, including the map ke ```bash yq eval '... style= "flow"' file.yaml ``` - ## Recurse map (values only) - Given a sample.yml file of: - ```yaml a: frog ``` - then - ```bash yq eval '..' sample.yml ``` - will output - ```yaml a: frog frog ``` ## Recursively find nodes with keys - Note that this example has wrapped the expression in `[]` to show that there are two matches returned. You do not have to wrap in `[]` in your path expression. Given a sample.yml file of: - ```yaml a: name: frog @@ -56,15 +45,11 @@ a: name: blog age: 12 ``` - then - ```bash yq eval '[.. | select(has("name"))]' sample.yml ``` - will output - ```yaml - name: frog b: @@ -75,9 +60,7 @@ will output ``` ## Recursively find nodes with values - Given a sample.yml file of: - ```yaml a: nameA: frog @@ -85,38 +68,28 @@ a: nameB: frog age: 12 ``` - then - ```bash yq eval '.. | select(. == "frog")' sample.yml ``` - will output - ```yaml frog frog ``` ## Recurse map (values and keys) - Note that the map key appears in the results Given a sample.yml file of: - ```yaml a: frog ``` - then - ```bash yq eval '...' sample.yml ``` - will output - ```yaml a: frog a @@ -124,23 +97,17 @@ frog ``` ## Aliases are not traversed - Given a sample.yml file of: - ```yaml a: &cat c: frog b: *cat ``` - then - ```bash yq eval '[..]' sample.yml ``` - will output - ```yaml - a: &cat c: frog @@ -152,9 +119,7 @@ will output ``` ## Merge docs are not traversed - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -175,15 +140,11 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobar | [..]' sample.yml ``` - will output - ```yaml - c: foobar_c !!merge <<: *foo @@ -192,3 +153,4 @@ will output - *foo - foobar_thing ``` + diff --git a/operators/reduce.md b/operators/reduce.md index 48b7f2f0..1cb9f592 100644 --- a/operators/reduce.md +++ b/operators/reduce.md @@ -14,84 +14,64 @@ e.g. On the LHS we are configuring the collection of items that will be reduced `` as well as what each element will be called `$`. Note that the array has been splatted into its individual elements. -On the RHS there is ``, the starting value of the accumulator and ``, the expression that will update the accumulator for each element in the collection. Note that within the block expression, `.` will evaluate to the current value of the accumulator. +On the RHS there is ``, the starting value of the accumulator and ``, the expression that will update the accumulator for each element in the collection. Note that within the block expression, `.` will evaluate to the current value of the accumulator. ## yq vs jq syntax - Reduce syntax in `yq` is a little different from `jq` - as `yq` (currently) isn't as sophisticated as `jq` and its only supports infix notation (e.g. a + b, where the operator is in the middle of the two parameters) - where as `jq` uses a mix of infix notation with _prefix_ notation (e.g. `reduce a b` is like writing `+ a b`). To that end, the reduce operator is called `ireduce` for backwards compatability if a `jq` like prefix version of `reduce` is ever added. ## Sum numbers - Given a sample.yml file of: - ```yaml - 10 - 2 - 5 - 3 ``` - then - ```bash yq eval '.[] as $item ireduce (0; . + $item)' sample.yml ``` - will output - ```yaml 20 ``` ## Merge all yaml files together - Given a sample.yml file of: - ```yaml a: cat ``` - And another sample another.yml file of: - ```yaml b: dog ``` - then - ```bash yq eval-all '. as $item ireduce ({}; . * $item )' sample.yml another.yml ``` - will output - ```yaml a: cat b: dog ``` ## Convert an array to an object - Given a sample.yml file of: - ```yaml - name: Cathy has: apples - name: Bob has: bananas ``` - then - ```bash yq eval '.[] as $item ireduce ({}; .[$item | .name] = ($item | .has) )' sample.yml ``` - will output - ```yaml Cathy: apples Bob: bananas ``` + diff --git a/operators/select.md b/operators/select.md index 1ba51c52..aabed120 100644 --- a/operators/select.md +++ b/operators/select.md @@ -3,50 +3,39 @@ Select is used to filter arrays and maps by a boolean expression. ## Select elements from array - Given a sample.yml file of: - ```yaml - cat - goat - dog ``` - then - ```bash yq eval '.[] | select(. == "*at")' sample.yml ``` - will output - ```yaml cat goat ``` ## Select and update matching values in map - Given a sample.yml file of: - ```yaml a: things: cat bob: goat horse: dog ``` - then - ```bash yq eval '(.a.[] | select(. == "*at")) |= "rabbit"' sample.yml ``` - will output - ```yaml a: things: rabbit bob: rabbit horse: dog ``` + diff --git a/operators/sort-keys.md b/operators/sort-keys.md index cb19ac90..e8f4d6dc 100644 --- a/operators/sort-keys.md +++ b/operators/sort-keys.md @@ -11,23 +11,17 @@ diff file1.yml file2.yml ``` ## Sort keys of map - Given a sample.yml file of: - ```yaml c: frog a: blah b: bing ``` - then - ```bash yq eval 'sortKeys(.)' sample.yml ``` - will output - ```yaml a: blah b: bing @@ -35,11 +29,9 @@ c: frog ``` ## Sort keys recursively - Note the array elements are left unsorted, but maps inside arrays are sorted Given a sample.yml file of: - ```yaml bParent: c: dog @@ -55,15 +47,11 @@ aParent: - b: ew a: apple ``` - then - ```bash yq eval 'sortKeys(..)' sample.yml ``` - will output - ```yaml aParent: x: @@ -79,3 +67,4 @@ bParent: - 2 c: dog ``` + diff --git a/operators/split-into-documents.md b/operators/split-into-documents.md index b9fee9c3..a902b43c 100644 --- a/operators/split-into-documents.md +++ b/operators/split-into-documents.md @@ -3,37 +3,29 @@ This operator splits all matches into separate documents ## Split empty - Running - ```bash yq eval --null-input 'splitDoc' ``` - will output - ```yaml + ``` ## Split array - Given a sample.yml file of: - ```yaml - a: cat - b: dog ``` - then - ```bash yq eval '.[] | splitDoc' sample.yml ``` - will output - ```yaml a: cat --- b: dog ``` + diff --git a/operators/string-operators.md b/operators/string-operators.md index 1ccb92e1..dafe6f30 100644 --- a/operators/string-operators.md +++ b/operators/string-operators.md @@ -1,13 +1,10 @@ # String Operators -## String Operators +## RegEx +This uses golangs native regex functions under the hood - See https://github.com/google/re2/wiki/Syntax for the supported syntax. -### RegEx - -This uses golangs native regex functions under the hood - See [https://github.com/google/re2/wiki/Syntax](https://github.com/google/re2/wiki/Syntax) for the supported syntax. ## String blocks, bash and newlines - Bash is notorious for chomping on precious trailing newline characters, making it tricky to set strings with newlines properly. In particular, the `$( exp )` _will trim trailing newlines_. For instance to get this yaml: @@ -25,7 +22,6 @@ a: cat ``` However, using printf works: - ``` printf -v m "cat\n" ; m="$m" yq e -n '.a = strenv(m)' a: | @@ -33,7 +29,6 @@ a: | ``` As well as having multiline expressions: - ``` m="cat " yq e -n '.a = strenv(m)' @@ -48,10 +43,8 @@ IFS= read -rd '' output < <(cat my_file) output=$output ./yq e '.data.values = strenv(output)' first.yml ``` -### Join strings - +## Join strings Given a sample.yml file of: - ```yaml - cat - meow @@ -59,35 +52,25 @@ Given a sample.yml file of: - null - true ``` - then - ```bash yq eval 'join("; ")' sample.yml ``` - will output - ```yaml cat; meow; 1; ; true ``` -### Match string - +## Match string Given a sample.yml file of: - ```yaml foo bar foo ``` - then - ```bash yq eval 'match("foo")' sample.yml ``` - will output - ```yaml string: foo offset: 0 @@ -95,22 +78,16 @@ length: 3 captures: [] ``` -### Match string, case insensitive - +## Match string, case insensitive Given a sample.yml file of: - ```yaml foo bar FOO ``` - then - ```bash yq eval '[match("(?i)foo"; "g")]' sample.yml ``` - will output - ```yaml - string: foo offset: 0 @@ -122,22 +99,16 @@ will output captures: [] ``` -### Match with capture groups - +## Match with capture groups Given a sample.yml file of: - ```yaml abc abc ``` - then - ```bash yq eval '[match("(abc)+"; "g")]' sample.yml ``` - will output - ```yaml - string: abc offset: 0 @@ -155,22 +126,16 @@ will output length: 3 ``` -### Match with named capture groups - +## Match with named capture groups Given a sample.yml file of: - ```yaml foo bar foo foo foo ``` - then - ```bash yq eval '[match("foo (?Pbar)? foo"; "g")]' sample.yml ``` - will output - ```yaml - string: foo bar foo offset: 0 @@ -190,43 +155,31 @@ will output name: bar123 ``` -### Capture named groups into a map - +## Capture named groups into a map Given a sample.yml file of: - ```yaml xyzzy-14 ``` - then - ```bash yq eval 'capture("(?P[a-z]+)-(?P[0-9]+)")' sample.yml ``` - will output - ```yaml a: xyzzy n: "14" ``` -### Match without global flag - +## Match without global flag Given a sample.yml file of: - ```yaml cat cat ``` - then - ```bash yq eval 'match("cat")' sample.yml ``` - will output - ```yaml string: cat offset: 0 @@ -234,22 +187,16 @@ length: 3 captures: [] ``` -### Match with global flag - +## Match with global flag Given a sample.yml file of: - ```yaml cat cat ``` - then - ```bash yq eval '[match("cat"; "g")]' sample.yml ``` - will output - ```yaml - string: cat offset: 0 @@ -261,92 +208,70 @@ will output captures: [] ``` -### Test using regex - +## Test using regex Like jq'q equivalant, this works like match but only returns true/false instead of full match details Given a sample.yml file of: - ```yaml - cat - dog ``` - then - ```bash yq eval '.[] | test("at")' sample.yml ``` - will output - ```yaml true false ``` -### Substitute / Replace string - -This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax) Note the use of `|=` to run in context of the current string value. +## Substitute / Replace string +This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax) +Note the use of `|=` to run in context of the current string value. Given a sample.yml file of: - ```yaml a: dogs are great ``` - then - ```bash yq eval '.a |= sub("dogs", "cats")' sample.yml ``` - will output - ```yaml a: cats are great ``` -### Substitute / Replace string with regex - -This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax) Note the use of `|=` to run in context of the current string value. +## Substitute / Replace string with regex +This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax) +Note the use of `|=` to run in context of the current string value. Given a sample.yml file of: - ```yaml a: cat b: heat ``` - then - ```bash yq eval '.[] |= sub("(a)", "${1}r")' sample.yml ``` - will output - ```yaml a: cart b: heart ``` -### Split strings - +## Split strings Given a sample.yml file of: - ```yaml cat; meow; 1; ; true ``` - then - ```bash yq eval 'split("; ")' sample.yml ``` - will output - ```yaml - cat - meow @@ -355,22 +280,17 @@ will output - "true" ``` -### Split strings one match - +## Split strings one match Given a sample.yml file of: - ```yaml word ``` - then - ```bash yq eval 'split("; ")' sample.yml ``` - will output - ```yaml - word ``` + diff --git a/operators/style.md b/operators/style.md index fdf4fe01..3bc3e4c1 100644 --- a/operators/style.md +++ b/operators/style.md @@ -3,23 +3,17 @@ The style operator can be used to get or set the style of nodes (e.g. string style, yaml style) ## Update and set style of a particular node (simple) - Given a sample.yml file of: - ```yaml a: b: thing c: something ``` - then - ```bash yq eval '.a.b = "new" | .a.b style="double"' sample.yml ``` - will output - ```yaml a: b: "new" @@ -27,23 +21,17 @@ a: ``` ## Update and set style of a particular node using path variables - Given a sample.yml file of: - ```yaml a: b: thing c: something ``` - then - ```bash yq eval 'with(.a.b ; . = "new" | . style="double")' sample.yml ``` - will output - ```yaml a: b: "new" @@ -51,24 +39,18 @@ a: ``` ## Set tagged style - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '.. style="tagged"' sample.yml ``` - will output - ```yaml !!map a: !!str cat @@ -78,24 +60,18 @@ e: !!bool true ``` ## Set double quote style - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '.. style="double"' sample.yml ``` - will output - ```yaml a: "cat" b: "5" @@ -104,24 +80,18 @@ e: "true" ``` ## Set double quote style on map keys too - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '... style="double"' sample.yml ``` - will output - ```yaml "a": "cat" "b": "5" @@ -130,24 +100,18 @@ will output ``` ## Set single quote style - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '.. style="single"' sample.yml ``` - will output - ```yaml a: 'cat' b: '5' @@ -156,24 +120,18 @@ e: 'true' ``` ## Set literal quote style - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '.. style="literal"' sample.yml ``` - will output - ```yaml a: |- cat @@ -186,24 +144,18 @@ e: |- ``` ## Set folded quote style - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '.. style="folded"' sample.yml ``` - will output - ```yaml a: >- cat @@ -216,49 +168,37 @@ e: >- ``` ## Set flow quote style - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '.. style="flow"' sample.yml ``` - will output - ```yaml {a: cat, b: 5, c: 3.2, e: true} ``` ## Reset style - or pretty print - Set empty (default) quote style, note the usage of `...` to match keys too. Note that there is a `--prettyPrint/-P` short flag for this. Given a sample.yml file of: - ```yaml a: cat "b": 5 'c': 3.2 "e": true ``` - then - ```bash yq eval '... style=""' sample.yml ``` - will output - ```yaml a: cat b: 5 @@ -267,45 +207,34 @@ e: true ``` ## Set style relatively with assign-update - Given a sample.yml file of: - ```yaml a: single b: double ``` - then - ```bash yq eval '.[] style |= .' sample.yml ``` - will output - ```yaml a: 'single' b: "double" ``` ## Read style - Given a sample.yml file of: - ```yaml {a: "cat", b: 'thing'} ``` - then - ```bash yq eval '.. | style' sample.yml ``` - will output - ```yaml flow double single ``` + diff --git a/operators/subtract.md b/operators/subtract.md index fbb0984d..bcc93615 100644 --- a/operators/subtract.md +++ b/operators/subtract.md @@ -1,148 +1,113 @@ # Subtract +You can use subtract to subtract numbers, as well as removing elements from an array. + ## Array subtraction - Running - ```bash yq eval --null-input '[1,2] - [2,3]' ``` - will output - ```yaml - 1 ``` ## Array subtraction with nested array - Running - ```bash yq eval --null-input '[[1], 1, 2] - [[1], 3]' ``` - will output - ```yaml - 1 - 2 ``` ## Array subtraction with nested object - Note that order of the keys does not matter Given a sample.yml file of: - ```yaml - a: b c: d - a: b ``` - then - ```bash yq eval '. - [{"c": "d", "a": "b"}]' sample.yml ``` - will output - ```yaml - a: b ``` ## Number subtraction - float - If the lhs or rhs are floats then the expression will be calculated with floats. Given a sample.yml file of: - ```yaml a: 3 b: 4.5 ``` - then - ```bash yq eval '.a = .a - .b' sample.yml ``` - will output - ```yaml a: -1.5 b: 4.5 ``` ## Number subtraction - float - If the lhs or rhs are floats then the expression will be calculated with floats. Given a sample.yml file of: - ```yaml a: 3 b: 4.5 ``` - then - ```bash yq eval '.a = .a - .b' sample.yml ``` - will output - ```yaml a: -1.5 b: 4.5 ``` ## Number subtraction - int - If both the lhs and rhs are ints then the expression will be calculated with ints. Given a sample.yml file of: - ```yaml a: 3 b: 4 ``` - then - ```bash yq eval '.a = .a - .b' sample.yml ``` - will output - ```yaml a: -1 b: 4 ``` ## Decrement numbers - Given a sample.yml file of: - ```yaml a: 3 b: 5 ``` - then - ```bash yq eval '.[] -= 1' sample.yml ``` - will output - ```yaml a: 2 b: 4 ``` + diff --git a/operators/tag.md b/operators/tag.md index ab5cdb9a..96895a67 100644 --- a/operators/tag.md +++ b/operators/tag.md @@ -3,9 +3,7 @@ The tag operator can be used to get or set the tag of nodes (e.g. `!!str`, `!!int`, `!!bool`). ## Get tag - Given a sample.yml file of: - ```yaml a: cat b: 5 @@ -13,15 +11,11 @@ c: 3.2 e: true f: [] ``` - then - ```bash yq eval '.. | tag' sample.yml ``` - will output - ```yaml !!map !!str @@ -32,47 +26,36 @@ will output ``` ## Set custom tag - Given a sample.yml file of: - ```yaml a: str ``` - then - ```bash yq eval '.a tag = "!!mikefarah"' sample.yml ``` - will output - ```yaml a: !!mikefarah str ``` ## Find numbers and convert them to strings - Given a sample.yml file of: - ```yaml a: cat b: 5 c: 3.2 e: true ``` - then - ```bash yq eval '(.. | select(tag == "!!int")) tag= "!!str"' sample.yml ``` - will output - ```yaml a: cat b: "5" c: 3.2 e: true ``` + diff --git a/operators/traverse-read.md b/operators/traverse-read.md index 497ba72f..4cf25c3b 100644 --- a/operators/traverse-read.md +++ b/operators/traverse-read.md @@ -3,381 +3,277 @@ This is the simplest (and perhaps most used) operator, it is used to navigate deeply into yaml structures. ## Simple map navigation - Given a sample.yml file of: - ```yaml a: b: apple ``` - then - ```bash yq eval '.a' sample.yml ``` - will output - ```yaml b: apple ``` ## Splat - Often used to pipe children into other operators Given a sample.yml file of: - ```yaml - b: apple - c: banana ``` - then - ```bash yq eval '.[]' sample.yml ``` - will output - ```yaml b: apple c: banana ``` ## Optional Splat - Just like splat, but won't error if you run it against scalars Given a sample.yml file of: - ```yaml cat ``` - then - ```bash yq eval '.[]' sample.yml ``` - will output - ```yaml ``` ## Special characters - Use quotes with brackets around path elements with special characters Given a sample.yml file of: - ```yaml "{}": frog ``` - then - ```bash yq eval '.["{}"]' sample.yml ``` - will output - ```yaml frog ``` ## Nested special characters - Given a sample.yml file of: - ```yaml a: "key.withdots": "another.key": apple ``` - then - ```bash yq eval '.a["key.withdots"]["another.key"]' sample.yml ``` - will output - ```yaml apple ``` ## Keys with spaces - Use quotes with brackets around path elements with special characters Given a sample.yml file of: - ```yaml "red rabbit": frog ``` - then - ```bash yq eval '.["red rabbit"]' sample.yml ``` - will output - ```yaml frog ``` ## Dynamic keys - -Expressions within \[] can be used to dynamically lookup / calculate keys +Expressions within [] can be used to dynamically lookup / calculate keys Given a sample.yml file of: - ```yaml b: apple apple: crispy yum banana: soft yum ``` - then - ```bash yq eval '.[.b]' sample.yml ``` - will output - ```yaml crispy yum ``` ## Children don't exist - Nodes are added dynamically while traversing Given a sample.yml file of: - ```yaml c: banana ``` - then - ```bash yq eval '.a.b' sample.yml ``` - will output - ```yaml null ``` ## Optional identifier - Like jq, does not output an error when the yaml is not an array or object as expected Given a sample.yml file of: - ```yaml - 1 - 2 - 3 ``` - then - ```bash yq eval '.a?' sample.yml ``` - will output - ```yaml ``` ## Wildcard matching - Given a sample.yml file of: - ```yaml a: cat: apple mad: things ``` - then - ```bash yq eval '.a."*a*"' sample.yml ``` - will output - ```yaml apple things ``` ## Aliases - Given a sample.yml file of: - ```yaml a: &cat c: frog b: *cat ``` - then - ```bash yq eval '.b' sample.yml ``` - will output - ```yaml *cat ``` ## Traversing aliases with splat - Given a sample.yml file of: - ```yaml a: &cat c: frog b: *cat ``` - then - ```bash yq eval '.b[]' sample.yml ``` - will output - ```yaml frog ``` ## Traversing aliases explicitly - Given a sample.yml file of: - ```yaml a: &cat c: frog b: *cat ``` - then - ```bash yq eval '.b.c' sample.yml ``` - will output - ```yaml frog ``` ## Traversing arrays by index - Given a sample.yml file of: - ```yaml - 1 - 2 - 3 ``` - then - ```bash yq eval '.[0]' sample.yml ``` - will output - ```yaml 1 ``` ## Traversing nested arrays by index - Given a sample.yml file of: - ```yaml [[], [cat]] ``` - then - ```bash yq eval '.[1][0]' sample.yml ``` - will output - ```yaml cat ``` ## Maps with numeric keys - Given a sample.yml file of: - ```yaml 2: cat ``` - then - ```bash yq eval '.[2]' sample.yml ``` - will output - ```yaml cat ``` ## Maps with non existing numeric keys - Given a sample.yml file of: - ```yaml a: b ``` - then - ```bash yq eval '.[0]' sample.yml ``` - will output - ```yaml null ``` ## Traversing merge anchors - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -398,23 +294,17 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobar.a' sample.yml ``` - will output - ```yaml foo_a ``` ## Traversing merge anchors with override - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -435,23 +325,17 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobar.c' sample.yml ``` - will output - ```yaml foo_c ``` ## Traversing merge anchors with local override - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -472,23 +356,17 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobar.thing' sample.yml ``` - will output - ```yaml foobar_thing ``` ## Splatting merge anchors - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -509,15 +387,11 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobar[]' sample.yml ``` - will output - ```yaml foo_c foo_a @@ -525,11 +399,9 @@ foobar_thing ``` ## Traversing merge anchor lists - Note that the later merge anchors override previous Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -550,23 +422,17 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobarList.thing' sample.yml ``` - will output - ```yaml bar_thing ``` ## Splatting merge anchor lists - Given a sample.yml file of: - ```yaml foo: &foo a: foo_a @@ -587,15 +453,11 @@ foobar: !!merge <<: *foo thing: foobar_thing ``` - then - ```bash yq eval '.foobarList[]' sample.yml ``` - will output - ```yaml bar_b foo_a @@ -604,25 +466,20 @@ foobarList_c ``` ## Select multiple indices - Given a sample.yml file of: - ```yaml a: - a - b - c ``` - then - ```bash yq eval '.a[0, 2]' sample.yml ``` - will output - ```yaml a c ``` + diff --git a/operators/union.md b/operators/union.md index e3e05760..0c0abd38 100644 --- a/operators/union.md +++ b/operators/union.md @@ -3,15 +3,11 @@ This operator is used to combine different results together. ## Combine scalars - Running - ```bash yq eval --null-input '1, true, "cat"' ``` - will output - ```yaml 1 true @@ -19,24 +15,19 @@ cat ``` ## Combine selected paths - Given a sample.yml file of: - ```yaml a: fieldA b: fieldB c: fieldC ``` - then - ```bash yq eval '.a, .c' sample.yml ``` - will output - ```yaml fieldA fieldC ``` + diff --git a/operators/unique.md b/operators/unique.md index 65a53b98..6be33dce 100644 --- a/operators/unique.md +++ b/operators/unique.md @@ -3,24 +3,18 @@ This is used to filter out duplicated items in an array. ## Unique array of scalars (string/numbers) - Given a sample.yml file of: - ```yaml - 1 - 2 - 3 - 2 ``` - then - ```bash yq eval 'unique' sample.yml ``` - will output - ```yaml - 1 - 2 @@ -28,60 +22,46 @@ will output ``` ## Unique nulls - Unique works on the node value, so it considers different representations of nulls to be different Given a sample.yml file of: - ```yaml - ~ - null - ~ - null ``` - then - ```bash yq eval 'unique' sample.yml ``` - will output - ```yaml - ~ - null ``` ## Unique all nulls - Run against the node tag to unique all the nulls Given a sample.yml file of: - ```yaml - ~ - null - ~ - null ``` - then - ```bash yq eval 'unique_by(tag)' sample.yml ``` - will output - ```yaml - ~ ``` ## Unique array object fields - Given a sample.yml file of: - ```yaml - name: harry pet: cat @@ -90,18 +70,15 @@ Given a sample.yml file of: - name: harry pet: dog ``` - then - ```bash yq eval 'unique_by(.name)' sample.yml ``` - will output - ```yaml - name: harry pet: cat - name: billy pet: dog ``` + diff --git a/operators/variable-operators.md b/operators/variable-operators.md index a9d20836..c2a0c224 100644 --- a/operators/variable-operators.md +++ b/operators/variable-operators.md @@ -5,53 +5,39 @@ Like the `jq` equivalents, variables are sometimes required for the more complex Note that there is also an additional `ref` operator that holds a reference (instead of a copy) of the path, allowing you to make multiple changes to the same path. ## Single value variable - Given a sample.yml file of: - ```yaml a: cat ``` - then - ```bash yq eval '.a as $foo | $foo' sample.yml ``` - will output - ```yaml cat ``` ## Multi value variable - Given a sample.yml file of: - ```yaml - cat - dog ``` - then - ```bash yq eval '.[] as $foo | $foo' sample.yml ``` - will output - ```yaml cat dog ``` ## Using variables as a lookup - Example taken from [jq](https://stedolan.github.io/jq/manual/#Variable/SymbolicBindingOperator:...as$identifier|...) Given a sample.yml file of: - ```yaml "posts": - "title": Frist psot @@ -62,15 +48,11 @@ Given a sample.yml file of: "anon": Anonymous Coward "person1": Person McPherson ``` - then - ```bash yq eval '.realnames as $names | .posts[] | {"title":.title, "author": $names[.author]}' sample.yml ``` - will output - ```yaml title: Frist psot author: Anonymous Coward @@ -79,49 +61,38 @@ author: Person McPherson ``` ## Using variables to swap values - Given a sample.yml file of: - ```yaml a: a_value b: b_value ``` - then - ```bash yq eval '.a as $x | .b as $y | .b = $x | .a = $y' sample.yml ``` - will output - ```yaml a: b_value b: a_value ``` ## Use ref to reference a path repeatedly - Note: You may find the `with` operator more useful. Given a sample.yml file of: - ```yaml a: b: thing c: something ``` - then - ```bash yq eval '.a.b ref $x | $x = "new" | $x style="double"' sample.yml ``` - will output - ```yaml a: b: "new" c: something ``` + diff --git a/operators/with.md b/operators/with.md index 2cd0735a..a8224e60 100644 --- a/operators/with.md +++ b/operators/with.md @@ -3,23 +3,17 @@ Use the `with` operator to conveniently make multiple updates to a deeply nested path, or to update array elements relatively to each other. The first argument expression sets the root context, and the second expression runs against that root context. ## Update and style - Given a sample.yml file of: - ```yaml a: deeply: nested: value ``` - then - ```bash -yq eval 'with(.a.deeply.nested ; . = "newValue" | . style="single")' sample.yml +yq eval 'with(.a.deeply.nested; . = "newValue" | . style="single")' sample.yml ``` - will output - ```yaml a: deeply: @@ -27,24 +21,18 @@ a: ``` ## Update multiple deeply nested properties - Given a sample.yml file of: - ```yaml a: deeply: nested: value other: thing ``` - then - ```bash -yq eval 'with(.a.deeply ; .nested = "newValue" | .other= "newThing")' sample.yml +yq eval 'with(.a.deeply; .nested = "newValue" | .other= "newThing")' sample.yml ``` - will output - ```yaml a: deeply: @@ -53,25 +41,19 @@ a: ``` ## Update array elements relatively - The second expression runs with each element of the array as it's contextual root. This allows you to make updates relative to the element. Given a sample.yml file of: - ```yaml myArray: - a: apple - a: banana ``` - then - ```bash -yq eval 'with(.myArray[] ; .b = .a + " yum")' sample.yml +yq eval 'with(.myArray[]; .b = .a + " yum")' sample.yml ``` - will output - ```yaml myArray: - a: apple @@ -79,3 +61,4 @@ myArray: - a: banana b: banana yum ``` +