v4.21.1

SUMMARY.md

@@ -19,6 +19,7 @@
   * [Assign (Update)](operators/assign-update.md)
   * [Boolean Operators](operators/boolean-operators.md)
   * [Collect into Array](operators/collect-into-array.md)
+  * [Column](operators/column.md)
   * [Comment Operators](operators/comment-operators.md)
   * [Contains](operators/contains.md)
   * [Create, Collect into Object](operators/create-collect-into-object.md)
@@ -36,6 +37,7 @@
   * [Has](operators/has.md)
   * [Keys](operators/keys.md)
   * [Length](operators/length.md)
+  * [Line](operators/line.md)
   * [Load](operators/load.md)
   * [Map](operators/map.md)
   * [Multiply (Merge)](operators/multiply-merge.md)
@@ -44,6 +46,7 @@
   * [Pipe](operators/pipe.md)
   * [Recursive Descent (Glob)](operators/recursive-descent-glob.md)
   * [Reduce](operators/reduce.md)
+  * [Reverse](operators/reverse.md)
   * [Select](operators/select.md)
   * [Sort](operators/sort.md)
   * [Sort Keys](operators/sort-keys.md)
@@ -61,9 +64,9 @@
 ## Usage
 
 * [Output format](usage/output-format.md)
-* [Working with Properties](usage/properties.md)
 * [Working with CSV, TSV](usage/csv-tsv.md)
 * [Working with JSON](usage/convert.md)
+* [Working with Properties](usage/properties.md)
 * [Working with XML](usage/xml.md)
 * [Front Matter](usage/front-matter.md)
 * [Split into multiple files](usage/split-into-multiple-files.md)

operators/column.md

@@ -0,0 +1,66 @@
+# Column
+
+Returns the column of the matching node. Starts from 1, 0 indicates there was no column data.
+
+{% hint style="warning" %}
+Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
+
+`yq e <exp> <file>`
+{% endhint %}
+
+## Returns column of _value_ node
+Given a sample.yml file of:
+```yaml
+a: cat
+b: bob
+```
+then
+```bash
+yq '.b | column' sample.yml
+```
+will output
+```yaml
+4
+```
+
+## Returns column of _key_ node
+Pipe through the key operator to get the column of the key
+
+Given a sample.yml file of:
+```yaml
+a: cat
+b: bob
+```
+then
+```bash
+yq '.b | key | column' sample.yml
+```
+will output
+```yaml
+1
+```
+
+## First column is 1
+Given a sample.yml file of:
+```yaml
+a: cat
+```
+then
+```bash
+yq '.a | key | column' sample.yml
+```
+will output
+```yaml
+1
+```
+
+## No column data is 0
+Running
+```bash
+yq --null-input '{"a": "new entry"} | column'
+```
+will output
+```yaml
+0
+```
+

operators/encode-decode.md

@@ -15,6 +15,7 @@ These operators are useful to process yaml documents that have stringified embed
 | CSV |  | to_csv/@csv |
 | TSV |  | to_tsv/@tsv |
 | XML | from_xml | to_xml(i)/@xml |
+| Base64 | @base64d | @base64 |
 
 
 CSV and TSV format both accept either a single array or scalars (representing a single row), or an array of array of scalars (representing multiple rows). 
@@ -22,6 +23,8 @@ CSV and TSV format both accept either a single array or scalars (representing a
 XML uses the `--xml-attribute-prefix` and `xml-content-name` flags to identify attributes and content fields.
 
 
+Base64 assumes [rfc4648](https://rfc-editor.org/rfc/rfc4648.html) encoding. Encoding and decoding both assume that the content is a string.
+
 {% hint style="warning" %}
 Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
 
@@ -354,3 +357,66 @@ b:
   foo: bar
 ```
 
+## Encode a string to base64
+Given a sample.yml file of:
+```yaml
+coolData: a special string
+```
+then
+```bash
+yq '.coolData | @base64' sample.yml
+```
+will output
+```yaml
+YSBzcGVjaWFsIHN0cmluZw==
+```
+
+## Encode a yaml document to base64
+Pipe through @yaml first to convert to a string, then use @base64 to encode it.
+
+Given a sample.yml file of:
+```yaml
+a: apple
+```
+then
+```bash
+yq '@yaml | @base64' sample.yml
+```
+will output
+```yaml
+YTogYXBwbGUK
+```
+
+## Decode a base64 encoded string
+Decoded data is assumed to be a string.
+
+Given a sample.yml file of:
+```yaml
+coolData: V29ya3Mgd2l0aCBVVEYtMTYg8J+Yig==
+```
+then
+```bash
+yq '.coolData | @base64d' sample.yml
+```
+will output
+```yaml
+Works with UTF-16 😊
+```
+
+## Decode a base64 encoded yaml document
+Pipe through `from_yaml` to parse the decoded base64 string as a yaml document.
+
+Given a sample.yml file of:
+```yaml
+coolData: YTogYXBwbGUK
+```
+then
+```bash
+yq '.coolData |= (@base64d | from_yaml)' sample.yml
+```
+will output
+```yaml
+coolData:
+  a: apple
+```
+

operators/line.md

@@ -0,0 +1,68 @@
+# Line
+
+Returns the line of the matching node. Starts from 1, 0 indicates there was no line data.
+
+{% hint style="warning" %}
+Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
+
+`yq e <exp> <file>`
+{% endhint %}
+
+## Returns line of _value_ node
+Given a sample.yml file of:
+```yaml
+a: cat
+b:
+  c: cat
+```
+then
+```bash
+yq '.b | line' sample.yml
+```
+will output
+```yaml
+3
+```
+
+## Returns line of _key_ node
+Pipe through the key operator to get the line of the key
+
+Given a sample.yml file of:
+```yaml
+a: cat
+b:
+  c: cat
+```
+then
+```bash
+yq '.b | key| line' sample.yml
+```
+will output
+```yaml
+2
+```
+
+## First line is 1
+Given a sample.yml file of:
+```yaml
+a: cat
+```
+then
+```bash
+yq '.a | line' sample.yml
+```
+will output
+```yaml
+1
+```
+
+## No line data is 0
+Running
+```bash
+yq --null-input '{"a": "new entry"} | line'
+```
+will output
+```yaml
+0
+```
+

operators/load.md

@@ -1,18 +1,50 @@
 # Load
 
-The `load`/`strload` operator allows you to load in content from another file referenced in your yaml document.
+The load operators allows you to load in content from another file.
 
 Note that you can use string operators like `+` and `sub` to modify the value in the yaml file to a path that exists in your system.
 
-Use `strload` to load text based content as a string block, and `load` to interpret the file as yaml.
+You can load files of the following supported types:
 
-Lets say there is a file `../../examples/thing.yml`:
+|Format | Load Operator |
+| --- | --- |
+| Yaml | load |
+| XML | load_xml |
+| Properties | load_props |
+| Plain String | load_str |
+| Base64 | load_base64 |
+
+## Samples files for tests:
+
+### yaml
+
+`../../examples/thing.yml`:
 
 ```yaml
 a: apple is included
 b: cool
 ```
 
+### xml
+`small.xml`:
+
+```xml
+<this>is some xml</this>
+```
+
+### properties
+`small.properties`:
+
+```properties
+this.is = a properties file
+```
+
+### base64
+`base64.txt`:
+```
+bXkgc2VjcmV0IGNoaWxsaSByZWNpcGUgaXMuLi4u
+```
+
 {% hint style="warning" %}
 Note that versions prior to 4.18 require the 'eval/e' command to be specified.&#x20;
 
@@ -89,7 +121,7 @@ something:
 ```
 then
 ```bash
-yq '.something |= strload("../../examples/" + .file)' sample.yml
+yq '.something |= load_str("../../examples/" + .file)' sample.yml
 ```
 will output
 ```yaml
@@ -98,3 +130,71 @@ something: |-
   b: cool.
 ```
 
+## Load from XML
+Given a sample.yml file of:
+```yaml
+cool: things
+```
+then
+```bash
+yq '.more_stuff = load_xml("../../examples/small.xml")' sample.yml
+```
+will output
+```yaml
+cool: things
+more_stuff:
+  this: is some xml
+```
+
+## Load from Properties
+Given a sample.yml file of:
+```yaml
+cool: things
+```
+then
+```bash
+yq '.more_stuff = load_props("../../examples/small.properties")' sample.yml
+```
+will output
+```yaml
+cool: things
+more_stuff:
+  this:
+    is: a properties file
+```
+
+## Merge from properties
+This can be used as a convenient way to update a yaml document
+
+Given a sample.yml file of:
+```yaml
+this:
+  is: from yaml
+  cool: ay
+```
+then
+```bash
+yq '. *= load_props("../../examples/small.properties")' sample.yml
+```
+will output
+```yaml
+this:
+  is: a properties file
+  cool: ay
+```
+
+## Load from base64 encoded file
+Given a sample.yml file of:
+```yaml
+cool: things
+```
+then
+```bash
+yq '.more_stuff = load_base64("../../examples/base64.txt")' sample.yml
+```
+will output
+```yaml
+cool: things
+more_stuff: my secret chilli recipe is....
+```
+

operators/reverse.md

@@ -0,0 +1,48 @@
+# Reverse
+
+Reverses the order of the items in an array 
+
+{% hint style="warning" %}
+Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
+
+`yq e <exp> <file>`
+{% endhint %}
+
+## Reverse
+Given a sample.yml file of:
+```yaml
+- 1
+- 2
+- 3
+```
+then
+```bash
+yq 'reverse' sample.yml
+```
+will output
+```yaml
+- 3
+- 2
+- 1
+```
+
+## Sort descending by string field
+Use sort with reverse to sort in descending order.
+
+Given a sample.yml file of:
+```yaml
+- a: banana
+- a: cat
+- a: apple
+```
+then
+```bash
+yq 'sort_by(.a) | reverse' sample.yml
+```
+will output
+```yaml
+- a: cat
+- a: banana
+- a: apple
+```
+

operators/sort.md

@@ -2,8 +2,11 @@
 
 Sorts an array. Use `sort` to sort an array as is, or `sort_by(exp)` to sort by a particular expression (e.g. subfield).
 
+To sort by descending order, pipe the results through the `reverse` operator after sorting.
+
 Note that at this stage, `yq` only sorts scalar fields.
 
+
 {% hint style="warning" %}
 Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
 
@@ -28,6 +31,26 @@ will output
 - a: cat
 ```
 
+## Sort descending by string field
+Use sort with reverse to sort in descending order.
+
+Given a sample.yml file of:
+```yaml
+- a: banana
+- a: cat
+- a: apple
+```
+then
+```bash
+yq 'sort_by(.a) | reverse' sample.yml
+```
+will output
+```yaml
+- a: cat
+- a: banana
+- a: apple
+```
+
 ## Sort array in place
 Given a sample.yml file of:
 ```yaml

operators/string-operators.md

@@ -49,6 +49,38 @@ Note that versions prior to 4.18 require the 'eval/e' command to be specified.&#
 `yq e <exp> <file>`
 {% endhint %}
 
+## To up (upper) case
+Works with unicode characters
+
+Given a sample.yml file of:
+```yaml
+água
+```
+then
+```bash
+yq 'upcase' sample.yml
+```
+will output
+```yaml
+ÁGUA
+```
+
+## To down (lower) case
+Works with unicode characters
+
+Given a sample.yml file of:
+```yaml
+ÁgUA
+```
+then
+```bash
+yq 'downcase' sample.yml
+```
+will output
+```yaml
+água
+```
+
 ## Join strings
 Given a sample.yml file of:
 ```yaml
@@ -105,14 +137,14 @@ will output
   captures: []
 ```
 
-## Match with capture groups
+## Match with global capture group
 Given a sample.yml file of:
 ```yaml
 abc abc
 ```
 then
 ```bash
-yq '[match("(abc)+"; "g")]' sample.yml
+yq '[match("(ab)(c)"; "g")]' sample.yml
 ```
 will output
 ```yaml
@@ -120,16 +152,22 @@ will output
   offset: 0
   length: 3
   captures:
-    - string: abc
+    - string: ab
       offset: 0
-      length: 3
+      length: 2
+    - string: c
+      offset: 2
+      length: 1
 - string: abc
   offset: 4
   length: 3
   captures:
-    - string: abc
+    - string: ab
       offset: 4
-      length: 3
+      length: 2
+    - string: c
+      offset: 6
+      length: 1
 ```
 
 ## Match with named capture groups
@@ -268,6 +306,24 @@ a: cart
 b: heart
 ```
 
+## Custom types: that are really strings
+When custom tags are encountered, yq will try to decode the underlying type.
+
+Given a sample.yml file of:
+```yaml
+a: !horse cat
+b: !goat heat
+```
+then
+```bash
+yq '.[] |= sub("(a)", "${1}r")' sample.yml
+```
+will output
+```yaml
+a: !horse cart
+b: !goat heart
+```
+
 ## Split strings
 Given a sample.yml file of:
 ```yaml