From 1b494f46bf445399ad494b4c589348bbfd32cf3f Mon Sep 17 00:00:00 2001 From: Mike Farah Date: Sun, 6 Feb 2022 14:41:27 +1100 Subject: [PATCH] Updating docs for v4.19.1 release --- operators/add.md | 137 ++++++++---------------- operators/alternative-default-value.md | 6 ++ operators/anchor-and-alias-operators.md | 6 ++ operators/assign-update.md | 6 ++ operators/boolean-operators.md | 6 ++ operators/collect-into-array.md | 6 ++ operators/comment-operators.md | 6 ++ operators/contains.md | 6 ++ operators/create-collect-into-object.md | 6 ++ operators/delete.md | 6 ++ operators/document-index.md | 6 ++ operators/encode-decode.md | 6 ++ operators/entries.md | 6 ++ operators/env-variable-operators.md | 28 +++++ operators/equals.md | 6 ++ operators/eval.md | 54 ++++++++++ operators/file-operators.md | 6 ++ operators/flatten.md | 6 ++ operators/group-by.md | 6 ++ operators/has.md | 6 ++ operators/keys.md | 6 ++ operators/length.md | 6 ++ operators/load.md | 6 ++ operators/map.md | 6 ++ operators/multiply-merge.md | 25 +++-- operators/parent.md | 6 ++ operators/path.md | 6 ++ operators/pipe.md | 6 ++ operators/recursive-descent-glob.md | 6 ++ operators/reduce.md | 6 ++ operators/select.md | 6 ++ operators/sort-keys.md | 6 ++ operators/sort.md | 6 ++ operators/split-into-documents.md | 6 ++ operators/string-operators.md | 6 ++ operators/style.md | 6 ++ operators/subtract.md | 6 ++ operators/tag.md | 6 ++ operators/traverse-read.md | 6 ++ operators/union.md | 6 ++ operators/unique.md | 6 ++ operators/variable-operators.md | 6 ++ operators/with.md | 6 ++ usage/convert.md | 6 ++ usage/properties.md | 84 +++++++++++++-- usage/xml.md | 6 ++ 46 files changed, 463 insertions(+), 111 deletions(-) create mode 100644 operators/eval.md diff --git a/operators/add.md b/operators/add.md index 7872d8d1..8a61c0eb 100644 --- a/operators/add.md +++ b/operators/add.md @@ -9,28 +9,11 @@ Add behaves differently according to the type of the LHS: Use `+=` as a relative append assign for things like increment. Note that `.a += .x` is equivalent to running `.a = .a + .x`. -## Concatenate and assign arrays -Given a sample.yml file of: -```yaml -a: - val: thing - b: - - cat - - dog -``` -then -```bash -yq '.a.b += ["cow"]' sample.yml -``` -will output -```yaml -a: - val: thing - b: - - cat - - dog - - cow -``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} ## Concatenate arrays Given a sample.yml file of: @@ -54,6 +37,28 @@ will output - 4 ``` +## Concatenate to existing array +Note that the styling of `a` is kept. + +Given a sample.yml file of: +```yaml +a: [1,2] +b: + - 3 + - 4 +``` +then +```bash +yq '.a += .b' sample.yml +``` +will output +```yaml +a: [1, 2, 3, 4] +b: + - 3 + - 4 +``` + ## Concatenate null to array Given a sample.yml file of: ```yaml @@ -71,6 +76,22 @@ will output - 2 ``` +## Append to existing array +Note that the styling is copied from existing array elements + +Given a sample.yml file of: +```yaml +a: ['dog'] +``` +then +```bash +yq '.a += "cat"' sample.yml +``` +will output +```yaml +a: ['dog', 'cat'] +``` + ## Add new object to array Given a sample.yml file of: ```yaml @@ -87,76 +108,6 @@ will output - cat: meow ``` -## Add string to array -Given a sample.yml file of: -```yaml -a: - - 1 - - 2 -``` -then -```bash -yq '.a + "hello"' sample.yml -``` -will output -```yaml -- 1 -- 2 -- hello -``` - -## Append to array -Given a sample.yml file of: -```yaml -a: - - 1 - - 2 -b: - - 3 - - 4 -``` -then -```bash -yq '.a = .a + .b' sample.yml -``` -will output -```yaml -a: - - 1 - - 2 - - 3 - - 4 -b: - - 3 - - 4 -``` - -## Append another array using += -Given a sample.yml file of: -```yaml -a: - - 1 - - 2 -b: - - 3 - - 4 -``` -then -```bash -yq '.a += .b' sample.yml -``` -will output -```yaml -a: - - 1 - - 2 - - 3 - - 4 -b: - - 3 - - 4 -``` - ## Relative append Given a sample.yml file of: ```yaml @@ -195,7 +146,7 @@ b: meow ``` then ```bash -yq '.a = .a + .b' sample.yml +yq '.a += .b' sample.yml ``` will output ```yaml diff --git a/operators/alternative-default-value.md b/operators/alternative-default-value.md index 84c5ef01..a1a3208a 100644 --- a/operators/alternative-default-value.md +++ b/operators/alternative-default-value.md @@ -2,6 +2,12 @@ This operator is used to provide alternative (or default) values when a particular expression is either null or false. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## LHS is defined Given a sample.yml file of: ```yaml diff --git a/operators/anchor-and-alias-operators.md b/operators/anchor-and-alias-operators.md index 8dbfd996..08204a51 100644 --- a/operators/anchor-and-alias-operators.md +++ b/operators/anchor-and-alias-operators.md @@ -5,6 +5,12 @@ Use the `alias` and `anchor` operators to read and write yaml aliases and anchor `yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Merge one map see https://yaml.org/type/merge.html diff --git a/operators/assign-update.md b/operators/assign-update.md index 14d69263..d3a9a6b8 100644 --- a/operators/assign-update.md +++ b/operators/assign-update.md @@ -7,6 +7,12 @@ Which will assign the LHS node values to the RHS node values. The RHS expression ### relative form: `|=` This will do a similar thing to the plain form, however, the RHS expression is run against _the LHS nodes_. This is useful for updating values based on old values, e.g. increment. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Create yaml file Running ```bash diff --git a/operators/boolean-operators.md b/operators/boolean-operators.md index 98e11e29..70c19bd2 100644 --- a/operators/boolean-operators.md +++ b/operators/boolean-operators.md @@ -10,6 +10,12 @@ The `or` and `and` operators take two parameters and return a boolean result. These are most commonly used with the `select` operator to filter particular nodes. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## `or` example Running ```bash diff --git a/operators/collect-into-array.md b/operators/collect-into-array.md index 73152e51..14c2bda0 100644 --- a/operators/collect-into-array.md +++ b/operators/collect-into-array.md @@ -3,6 +3,12 @@ This creates an array using the expression between the square brackets. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Collect empty Running ```bash diff --git a/operators/comment-operators.md b/operators/comment-operators.md index 8a63dc3e..9c5f4aef 100644 --- a/operators/comment-operators.md +++ b/operators/comment-operators.md @@ -10,6 +10,12 @@ This will assign the LHS nodes comments to the expression on the RHS. The RHS is ### relative form: `|=` Similar to the plain form, however the RHS evaluates against each matching LHS node! This is useful if you want to set the comments as a relative expression of the node, for instance its value or path. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Set line comment Given a sample.yml file of: ```yaml diff --git a/operators/contains.md b/operators/contains.md index 046b4ca2..4295a0b5 100644 --- a/operators/contains.md +++ b/operators/contains.md @@ -2,6 +2,12 @@ This returns `true` if the context contains the passed in parameter, and false otherwise. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Array contains array Array is equal or subset of diff --git a/operators/create-collect-into-object.md b/operators/create-collect-into-object.md index eaf5ad3b..b90866c9 100644 --- a/operators/create-collect-into-object.md +++ b/operators/create-collect-into-object.md @@ -2,6 +2,12 @@ This is used to construct objects (or maps). This can be used against existing yaml, or to create fresh yaml documents. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Collect empty object Running ```bash diff --git a/operators/delete.md b/operators/delete.md index 6d0a99a5..9eecca59 100644 --- a/operators/delete.md +++ b/operators/delete.md @@ -2,6 +2,12 @@ Deletes matching entries in maps or arrays. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Delete entry in map Given a sample.yml file of: ```yaml diff --git a/operators/document-index.md b/operators/document-index.md index 1b35e111..140d1d4d 100644 --- a/operators/document-index.md +++ b/operators/document-index.md @@ -2,6 +2,12 @@ Use the `documentIndex` operator (or the `di` shorthand) to select nodes of a particular document. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Retrieve a document index Given a sample.yml file of: ```yaml diff --git a/operators/encode-decode.md b/operators/encode-decode.md index 86ea520e..191a1c56 100644 --- a/operators/encode-decode.md +++ b/operators/encode-decode.md @@ -22,6 +22,12 @@ CSV and TSV format both accept either a single array or scalars (representing a XML uses the `--xml-attribute-prefix` and `xml-content-name` flags to identify attributes and content fields. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Encode value as json string Given a sample.yml file of: ```yaml diff --git a/operators/entries.md b/operators/entries.md index a10ad0ef..b9d1ae9d 100644 --- a/operators/entries.md +++ b/operators/entries.md @@ -2,6 +2,12 @@ Similar to the same named functions in `jq` these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## to_entries Map Given a sample.yml file of: ```yaml diff --git a/operators/env-variable-operators.md b/operators/env-variable-operators.md index 34cc5ef3..b8adf077 100644 --- a/operators/env-variable-operators.md +++ b/operators/env-variable-operators.md @@ -17,6 +17,12 @@ yq '(.. | select(tag == "!!str")) |= envsubst' file.yaml ``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Read string environment variable Running ```bash @@ -77,6 +83,28 @@ will output a: "12" ``` +## Dynamically update a path from an environment variable +The env variable can be any valid yq expression. + +Given a sample.yml file of: +```yaml +a: + b: + - name: dog + - name: cat +``` +then +```bash +pathEnv=".a.b[0].name" valueEnv="moo" yq 'eval(strenv(pathEnv)) = strenv(valueEnv)' sample.yml +``` +will output +```yaml +a: + b: + - name: moo + - name: cat +``` + ## Dynamic key lookup with environment variable Given a sample.yml file of: ```yaml diff --git a/operators/equals.md b/operators/equals.md index e9d5a7bf..e15b6a27 100644 --- a/operators/equals.md +++ b/operators/equals.md @@ -12,6 +12,12 @@ It is most often used with the select operator to find particular nodes: select(.a == .b) ``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Match string Given a sample.yml file of: ```yaml diff --git a/operators/eval.md b/operators/eval.md new file mode 100644 index 00000000..eae33bd1 --- /dev/null +++ b/operators/eval.md @@ -0,0 +1,54 @@ +# Eval + +Use `eval` to dynamically process an expression - for instance from an environment variable. + +`eval` takes a single argument, and evaluates that as a `yq` expression. Any valid expression can be used, beit a path `.a.b.c | select(. == "cat")`, or an update `.a.b.c = "gogo"`. + +Tip: This can be useful way parameterise complex scripts. + +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + +## Dynamically evaluate a path +Given a sample.yml file of: +```yaml +pathExp: .a.b[] | select(.name == "cat") +a: + b: + - name: dog + - name: cat +``` +then +```bash +yq 'eval(.pathExp)' sample.yml +``` +will output +```yaml +name: cat +``` + +## Dynamically update a path from an environment variable +The env variable can be any valid yq expression. + +Given a sample.yml file of: +```yaml +a: + b: + - name: dog + - name: cat +``` +then +```bash +pathEnv=".a.b[0].name" valueEnv="moo" yq 'eval(strenv(pathEnv)) = strenv(valueEnv)' sample.yml +``` +will output +```yaml +a: + b: + - name: moo + - name: cat +``` + diff --git a/operators/file-operators.md b/operators/file-operators.md index b7977ea1..d7988509 100644 --- a/operators/file-operators.md +++ b/operators/file-operators.md @@ -10,6 +10,12 @@ Note the use of eval-all to ensure all documents are loaded into memory. yq eval-all 'select(fi == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml ``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Get filename Given a sample.yml file of: ```yaml diff --git a/operators/flatten.md b/operators/flatten.md index cb3bc268..f4533646 100644 --- a/operators/flatten.md +++ b/operators/flatten.md @@ -1,6 +1,12 @@ # Flatten This recursively flattens arrays. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Flatten Recursively flattens all arrays diff --git a/operators/group-by.md b/operators/group-by.md index f5169b72..fc5bb254 100644 --- a/operators/group-by.md +++ b/operators/group-by.md @@ -2,6 +2,12 @@ This is used to group items in an array by an expression. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Group by field Given a sample.yml file of: ```yaml diff --git a/operators/has.md b/operators/has.md index 4a5e294a..007ce289 100644 --- a/operators/has.md +++ b/operators/has.md @@ -2,6 +2,12 @@ This is operation that returns true if the key exists in a map (or index in an array), false otherwise. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Has map key Given a sample.yml file of: ```yaml diff --git a/operators/keys.md b/operators/keys.md index 42485d37..9be768e5 100644 --- a/operators/keys.md +++ b/operators/keys.md @@ -2,6 +2,12 @@ Use the `keys` operator to return map keys or array indices. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Map keys Given a sample.yml file of: ```yaml diff --git a/operators/length.md b/operators/length.md index 6315e5b2..947a1b6e 100644 --- a/operators/length.md +++ b/operators/length.md @@ -2,6 +2,12 @@ Returns the lengths of the nodes. Length is defined according to the type of the node. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## String length returns length of string diff --git a/operators/load.md b/operators/load.md index 1c5a5e99..8f37566a 100644 --- a/operators/load.md +++ b/operators/load.md @@ -13,6 +13,12 @@ a: apple is included b: cool ``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Simple example Given a sample.yml file of: ```yaml diff --git a/operators/map.md b/operators/map.md index 991167e2..5555b1a8 100644 --- a/operators/map.md +++ b/operators/map.md @@ -2,6 +2,12 @@ Maps values of an array. Use `map_values` to map values of an object. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Map array Given a sample.yml file of: ```yaml diff --git a/operators/multiply-merge.md b/operators/multiply-merge.md index 3e21bcf0..20bbd968 100644 --- a/operators/multiply-merge.md +++ b/operators/multiply-merge.md @@ -22,6 +22,12 @@ Note the use of `eval-all` to ensure all documents are loaded into memory. yq eval-all 'select(fileIndex == 0) * select(fileIndex == 1)' file1.yaml file2.yaml ``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Multiply integers Given a sample.yml file of: ```yaml @@ -262,18 +268,23 @@ will output ``` ## Merge arrays of objects together, matching on a key + +This is a fairly complex expression - you can use it as is by providing the environment variables as seen in the example below. + +It merges in the array provided in the second file into the first - matching on equal keys. + +Explanation: + The approach, at a high level, is to reduce into a merged map (keyed by the unique key) and then convert that back into an array. -First the expression will create a map from the arrays keyed by '.a', the unique field we want to merge by. +First the expression will create a map from the arrays keyed by the idPath, the unique field we want to merge by. The reduce operator is merging '({}; . * $item )', so array elements with the matching key will be merged together. Next, we convert the map back to an array, using reduce again, concatenating all the map values together. Finally, we set the result of the merged array back into the first doc. -To use this, you will need to update '.myArray' in the expression to your array (e.g. .my.array), and '.a' to be the key field of your array (e.g. '.name') - Thanks Kev from [stackoverflow](https://stackoverflow.com/a/70109529/1168223) @@ -290,7 +301,7 @@ something: else ``` And another sample another.yml file of: ```yaml -myArray: +newArray: - a: banana c: bananaC - a: apple @@ -300,12 +311,12 @@ myArray: ``` then ```bash -yq eval-all ' +idPath=".a" originalPath=".myArray" otherPath=".newArray" yq eval-all ' ( - ((.myArray[] | {.a: .}) as $item ireduce ({}; . * $item )) as $uniqueMap + (( (eval(strenv(originalPath)) + eval(strenv(otherPath))) | .[] | {(eval(strenv(idPath))): .}) as $item ireduce ({}; . * $item )) as $uniqueMap | ( $uniqueMap | to_entries | .[]) as $item ireduce([]; . + $item.value) ) as $mergedArray -| select(fi == 0) | .myArray = $mergedArray +| select(fi == 0) | (eval(strenv(originalPath))) = $mergedArray ' sample.yml another.yml ``` will output diff --git a/operators/parent.md b/operators/parent.md index 0ef18d55..60cc1529 100644 --- a/operators/parent.md +++ b/operators/parent.md @@ -2,6 +2,12 @@ Parent simply returns the parent nodes of the matching nodes. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Simple example Given a sample.yml file of: ```yaml diff --git a/operators/path.md b/operators/path.md index 96c3244b..27ccf016 100644 --- a/operators/path.md +++ b/operators/path.md @@ -4,6 +4,12 @@ The path operator can be used to get the traversal paths of matching nodes in an You can get the key/index of matching nodes by using the `path` operator to return the path array then piping that through `.[-1]` to get the last element of that array, the key. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Map path Given a sample.yml file of: ```yaml diff --git a/operators/pipe.md b/operators/pipe.md index 0821554a..21aa3807 100644 --- a/operators/pipe.md +++ b/operators/pipe.md @@ -2,6 +2,12 @@ Pipe the results of an expression into another. Like the bash operator. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Simple Pipe Given a sample.yml file of: ```yaml diff --git a/operators/recursive-descent-glob.md b/operators/recursive-descent-glob.md index 1bb1656c..45c6052b 100644 --- a/operators/recursive-descent-glob.md +++ b/operators/recursive-descent-glob.md @@ -19,6 +19,12 @@ For instance to set the `style` of all nodes in a yaml doc, including the map ke ```bash yq '... style= "flow"' file.yaml ``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Recurse map (values only) Given a sample.yml file of: ```yaml diff --git a/operators/reduce.md b/operators/reduce.md index 952e68ed..37b21645 100644 --- a/operators/reduce.md +++ b/operators/reduce.md @@ -21,6 +21,12 @@ Reduce syntax in `yq` is a little different from `jq` - as `yq` (currently) isn' To that end, the reduce operator is called `ireduce` for backwards compatability if a `jq` like prefix version of `reduce` is ever added. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Sum numbers Given a sample.yml file of: ```yaml diff --git a/operators/select.md b/operators/select.md index 8b4a6daa..43a35f12 100644 --- a/operators/select.md +++ b/operators/select.md @@ -2,6 +2,12 @@ Select is used to filter arrays and maps by a boolean expression. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Select elements from array using wildcard prefix Given a sample.yml file of: ```yaml diff --git a/operators/sort-keys.md b/operators/sort-keys.md index 1be4c998..b5101dc1 100644 --- a/operators/sort-keys.md +++ b/operators/sort-keys.md @@ -12,6 +12,12 @@ diff file1.yml file2.yml Note that `yq` does not yet consider anchors when sorting by keys - this may result in invalid yaml documents if your are using merge anchors. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Sort keys of map Given a sample.yml file of: ```yaml diff --git a/operators/sort.md b/operators/sort.md index f7090c88..9885d092 100644 --- a/operators/sort.md +++ b/operators/sort.md @@ -4,6 +4,12 @@ Sorts an array. Use `sort` to sort an array as is, or `sort_by(exp)` to sort by Note that at this stage, `yq` only sorts scalar fields. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Sort by string field Given a sample.yml file of: ```yaml diff --git a/operators/split-into-documents.md b/operators/split-into-documents.md index 9924c40a..a9f70d30 100644 --- a/operators/split-into-documents.md +++ b/operators/split-into-documents.md @@ -2,6 +2,12 @@ This operator splits all matches into separate documents +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Split empty Running ```bash diff --git a/operators/string-operators.md b/operators/string-operators.md index 3be4bc7e..a13288fa 100644 --- a/operators/string-operators.md +++ b/operators/string-operators.md @@ -43,6 +43,12 @@ IFS= read -rd '' output < <(cat my_file) output=$output ./yq '.data.values = strenv(output)' first.yml ``` +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Join strings Given a sample.yml file of: ```yaml diff --git a/operators/style.md b/operators/style.md index b21a5b26..e4f4a924 100644 --- a/operators/style.md +++ b/operators/style.md @@ -2,6 +2,12 @@ The style operator can be used to get or set the style of nodes (e.g. string style, yaml style) +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Update and set style of a particular node (simple) Given a sample.yml file of: ```yaml diff --git a/operators/subtract.md b/operators/subtract.md index 95493181..da5bfe48 100644 --- a/operators/subtract.md +++ b/operators/subtract.md @@ -2,6 +2,12 @@ You can use subtract to subtract numbers, as well as removing elements from an array. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Array subtraction Running ```bash diff --git a/operators/tag.md b/operators/tag.md index ad3f3aca..31bc4643 100644 --- a/operators/tag.md +++ b/operators/tag.md @@ -2,6 +2,12 @@ The tag operator can be used to get or set the tag of nodes (e.g. `!!str`, `!!int`, `!!bool`). +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Get tag Given a sample.yml file of: ```yaml diff --git a/operators/traverse-read.md b/operators/traverse-read.md index 3a4452e3..f8c07b8d 100644 --- a/operators/traverse-read.md +++ b/operators/traverse-read.md @@ -2,6 +2,12 @@ This is the simplest (and perhaps most used) operator, it is used to navigate deeply into yaml structures. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Simple map navigation Given a sample.yml file of: ```yaml diff --git a/operators/union.md b/operators/union.md index abf0befe..6a477698 100644 --- a/operators/union.md +++ b/operators/union.md @@ -2,6 +2,12 @@ This operator is used to combine different results together. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Combine scalars Running ```bash diff --git a/operators/unique.md b/operators/unique.md index d1052024..af33321c 100644 --- a/operators/unique.md +++ b/operators/unique.md @@ -2,6 +2,12 @@ This is used to filter out duplicated items in an array. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Unique array of scalars (string/numbers) Given a sample.yml file of: ```yaml diff --git a/operators/variable-operators.md b/operators/variable-operators.md index 7a5bf83a..b06458f5 100644 --- a/operators/variable-operators.md +++ b/operators/variable-operators.md @@ -4,6 +4,12 @@ Like the `jq` equivalents, variables are sometimes required for the more complex Note that there is also an additional `ref` operator that holds a reference (instead of a copy) of the path, allowing you to make multiple changes to the same path. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Single value variable Given a sample.yml file of: ```yaml diff --git a/operators/with.md b/operators/with.md index 6b1a913f..9804cfe9 100644 --- a/operators/with.md +++ b/operators/with.md @@ -2,6 +2,12 @@ Use the `with` operator to conveniently make multiple updates to a deeply nested path, or to update array elements relatively to each other. The first argument expression sets the root context, and the second expression runs against that root context. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Update and style Given a sample.yml file of: ```yaml diff --git a/usage/convert.md b/usage/convert.md index b91feb4c..138ee542 100644 --- a/usage/convert.md +++ b/usage/convert.md @@ -4,6 +4,12 @@ Encode and decode to and from JSON. Note that YAML is a _superset_ of JSON - so This means you don't need to 'convert' a JSON file to YAML - however if you want idiomatic YAML styling, then you can use the `-P/--prettyPrint` flag, see examples below. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Parse json: simple JSON is a subset of yaml, so all you need to do is prettify the output diff --git a/usage/properties.md b/usage/properties.md index 84611655..8f7e7a08 100644 --- a/usage/properties.md +++ b/usage/properties.md @@ -1,11 +1,19 @@ -# Working with Properties +# Properties -## Yaml to Properties +Encode to a property file (decode not yet supported). Line comments on value nodes will be copied across. -To convert to property file format, use `--outputformat=props` or `-o=p`. Porting the comments from the `yaml` file to the property file is still in progress, currently line comments on the node values will be copied across. +By default, empty maps and arrays are not encoded - see below for an example on how to encode a value for these. -Given a sample file of: +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. +`yq e ` +{% endhint %} + +## Encode properties +Note that empty arrays and maps are not encoded by default. + +Given a sample.yml file of: ```yaml # block comments don't come through person: # neither do comments on maps @@ -13,17 +21,16 @@ person: # neither do comments on maps pets: - cat # comments on array values appear food: [pizza] # comments on arrays do not -``` +emptyArray: [] +emptyMap: [] +``` then - ```bash -yq -o=p sample.yaml -``` - -will output: - +yq -o=props -I=0 '.' sample.yml ``` +will output +```properties # comments on values appear person.name = Mike @@ -32,3 +39,58 @@ person.pets.0 = cat person.food.0 = pizza ``` +## Encode properties: no comments +Given a sample.yml file of: +```yaml +# block comments don't come through +person: # neither do comments on maps + name: Mike # comments on values appear + pets: + - cat # comments on array values appear + food: [pizza] # comments on arrays do not +emptyArray: [] +emptyMap: [] + +``` +then +```bash +yq -o=props -I=0 '... comments = ""' sample.yml +``` +will output +```properties +person.name = Mike +person.pets.0 = cat +person.food.0 = pizza +``` + +## Encode properties: include empty maps and arrays +Use a yq expression to set the empty maps and sequences to your desired value. + +Given a sample.yml file of: +```yaml +# block comments don't come through +person: # neither do comments on maps + name: Mike # comments on values appear + pets: + - cat # comments on array values appear + food: [pizza] # comments on arrays do not +emptyArray: [] +emptyMap: [] + +``` +then +```bash +yq -o=props -I=0 '(.. | select( (tag == "!!map" or tag =="!!seq") and length == 0)) = ""' sample.yml +``` +will output +```properties +# comments on values appear +person.name = Mike + +# comments on array values appear +person.pets.0 = cat +person.food.0 = pizza +emptyArray = +emptyMap = +``` + diff --git a/usage/xml.md b/usage/xml.md index fec4ca66..17ca941f 100644 --- a/usage/xml.md +++ b/usage/xml.md @@ -6,6 +6,12 @@ Consecutive xml nodes with the same name are assumed to be arrays. XML content data and attributes are created as fields. This can be controlled by the `'--xml-attribute-prefix` and `--xml-content-name` flags - see below for examples. +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + ## Parse xml: simple Notice how all the values are strings, see the next example on how you can fix that.