From 7abf77b32386247b670da983acfe6aea4bbd099a Mon Sep 17 00:00:00 2001
From: Mike Farah <mikefarah@gmail.com>
Date: Sat, 22 Jan 2022 11:49:12 +1100
Subject: [PATCH] Updating docs for v4.17.1 release

---
 operators/encode-decode.md  |  77 ++++++++++
 operators/multiply-merge.md |  28 +++-
 usage/convert.md            | 173 +++++++++++++---------
 usage/xml.md                | 284 ++++++++++++++++++++++++++++++++++++
 4 files changed, 487 insertions(+), 75 deletions(-)
 create mode 100644 usage/xml.md

diff --git a/operators/encode-decode.md b/operators/encode-decode.md
index d4416608..473ff995 100644
--- a/operators/encode-decode.md
+++ b/operators/encode-decode.md
@@ -14,10 +14,13 @@ These operators are useful to process yaml documents that have stringified embed
 | Properties |  | to_props/@props |
 | CSV |  | to_csv/@csv |
 | TSV |  | to_tsv/@tsv |
+| XML | from_xml | to_xml(i)/@xml |
 
 
 CSV and TSV format both accept either a single array or scalars (representing a single row), or an array of array of scalars (representing multiple rows). 
 
+XML uses the `--xml-attribute-prefix` and `xml-content-name` flags to identify attributes and content fields.
+
 
 ## Encode value as json string
 Given a sample.yml file of:
@@ -271,3 +274,77 @@ cat	thing1,thing2	true	3.40
 dog	thing3	false	12
 ```
 
+## Encode value as xml string
+Given a sample.yml file of:
+```yaml
+a:
+  cool:
+    foo: bar
+    +id: hi
+```
+then
+```bash
+yq eval '.a | to_xml' sample.yml
+```
+will output
+```yaml
+<cool id="hi">
+  <foo>bar</foo>
+</cool>
+
+```
+
+## Encode value as xml string on a single line
+Given a sample.yml file of:
+```yaml
+a:
+  cool:
+    foo: bar
+    +id: hi
+```
+then
+```bash
+yq eval '.a | @xml' sample.yml
+```
+will output
+```yaml
+<cool id="hi"><foo>bar</foo></cool>
+
+```
+
+## Encode value as xml string with custom indentation
+Given a sample.yml file of:
+```yaml
+a:
+  cool:
+    foo: bar
+    +id: hi
+```
+then
+```bash
+yq eval '{"cat": .a | to_xml(1)}' sample.yml
+```
+will output
+```yaml
+cat: |
+  <cool id="hi">
+   <foo>bar</foo>
+  </cool>
+```
+
+## Decode a xml encoded string
+Given a sample.yml file of:
+```yaml
+a: <foo>bar</foo>
+```
+then
+```bash
+yq eval '.b = (.a | from_xml)' sample.yml
+```
+will output
+```yaml
+a: <foo>bar</foo>
+b:
+  foo: bar
+```
+
diff --git a/operators/multiply-merge.md b/operators/multiply-merge.md
index b06a6834..1fea5319 100644
--- a/operators/multiply-merge.md
+++ b/operators/multiply-merge.md
@@ -10,9 +10,10 @@ Note that when merging objects, this operator returns the merged object (not the
 ### Merge Flags
 You can control how objects are merged by using one or more of the following flags. Multiple flags can be used together, e.g. `.a *+? .b`.  See examples below
 
-- `+` to append arrays
-- `?` to only merge existing fields
-- `d` to deeply merge arrays
+- `+` append arrays
+- `d` deeply merge arrays
+- `?` only merge _existing_ fields
+- `n` only merge _new_ fields
 
 ### Merging files
 Note the use of `eval-all` to ensure all documents are loaded into memory.
@@ -148,6 +149,27 @@ thing: two
 cat: frog
 ```
 
+## Merge, only new fields
+Given a sample.yml file of:
+```yaml
+a:
+  thing: one
+  cat: frog
+b:
+  missing: two
+  thing: two
+```
+then
+```bash
+yq eval '.a *n .b' sample.yml
+```
+will output
+```yaml
+thing: one
+cat: frog
+missing: two
+```
+
 ## Merge, appending arrays
 Given a sample.yml file of:
 ```yaml
diff --git a/usage/convert.md b/usage/convert.md
index dbdb6c2e..5e9dcf80 100644
--- a/usage/convert.md
+++ b/usage/convert.md
@@ -1,97 +1,126 @@
-# Working with JSON
+# JSON
 
-## Yaml to Json
+Encode and decode to and from JSON. Note that YAML is a _superset_ of JSON - so `yq` can read any json file without doing anything special.
 
-To convert output to json, use the `--output-format=json` (or `-o=j`) flag. You can change the json output format by using the [indent](output-format.md#indent) flag.&#x20;
+This means you don't need to 'convert' a JSON file to YAML - however if you want idiomatic YAML styling, then you can use the `-P/--prettyPrint` flag, see examples below.
 
-Given a sample.yaml file of:
+## Parse json: simple
+JSON is a subset of yaml, so all you need to do is prettify the output
 
-```yaml
-b:
-  c: 2
+Given a sample.json file of:
+```json
+{"cat": "meow"}
 ```
-
 then
-
 ```bash
-yq eval -o=j sample.yaml
+yq e -P '.' sample.json
 ```
-
 will output
-
-```javascript
-{
-  "b": {
-    "c": 2
-  }
-}
-```
-
-To format the json:
-
 ```yaml
-yq eval -o=j -I=0 sample.yaml
+cat: meow
 ```
 
-will yield
+## Parse json: complex
+JSON is a subset of yaml, so all you need to do is prettify the output
 
-```yaml
-{"b":{"c":2}}
-```
-
-### Multiple matches
-
-Each matching yaml node will be converted to json and printed out as a separate json doc. You may want to set the [indent](output-format.md#indent) flags to 0 if you want a json doc per line.
-
-Given a sample.yaml file of:
-
-```yaml
-bob:
-  c: 2
-bab:
-  c: 5
-```
-
-then
-
-```bash
-yq eval -o=j '.b*' sample.yaml
-```
-
-will output
-
-```javascript
-{
-  "c": 2
-}
-{
-  "c": 5
-}
-```
-
-## Json to Yaml
-
-To read in json, just pass in a json file instead of yaml, it will just work - as json is a subset of yaml. However, you will probably want to use the [Style Operator](broken-reference) or `--prettyPrint/-P` flag to make look more like an idiomatic yaml document. This can be done by resetting the style of all elements.
-
-e.g given a json file
-
-```javascript
+Given a sample.json file of:
+```json
 {"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
 ```
-
 then
-
 ```bash
-yq eval -P sample.json
+yq e -P '.' sample.json
 ```
-
 will output
-
 ```yaml
 a: Easy! as one two three
 b:
   c: 2
   d:
-  - 3
-  - 4
+    - 3
+    - 4
 ```
+
+## Encode json: simple
+Given a sample.yml file of:
+```yaml
+cat: meow
+```
+then
+```bash
+yq e -o=json '.' sample.yml
+```
+will output
+```json
+{
+  "cat": "meow"
+}
+```
+
+## Encode json: simple - in one line
+Given a sample.yml file of:
+```yaml
+cat: meow # this is a comment, and it will be dropped.
+```
+then
+```bash
+yq e -o=json -I=0 '.' sample.yml
+```
+will output
+```json
+{"cat":"meow"}
+```
+
+## Encode json: comments
+Given a sample.yml file of:
+```yaml
+cat: meow # this is a comment, and it will be dropped.
+```
+then
+```bash
+yq e -o=json '.' sample.yml
+```
+will output
+```json
+{
+  "cat": "meow"
+}
+```
+
+## Encode json: anchors
+Anchors are dereferenced
+
+Given a sample.yml file of:
+```yaml
+cat: &ref meow
+anotherCat: *ref
+```
+then
+```bash
+yq e -o=json '.' sample.yml
+```
+will output
+```json
+{
+  "cat": "meow",
+  "anotherCat": "meow"
+}
+```
+
+## Encode json: multiple results
+Each matching node is converted into a json doc. This is best used with 0 indent (json document per line)
+
+Given a sample.yml file of:
+```yaml
+things: [{stuff: cool}, {whatever: cat}]
+```
+then
+```bash
+yq e -o=json -I=0 '.things[]' sample.yml
+```
+will output
+```json
+{"stuff":"cool"}
+{"whatever":"cat"}
+```
+
diff --git a/usage/xml.md b/usage/xml.md
new file mode 100644
index 00000000..c18a039e
--- /dev/null
+++ b/usage/xml.md
@@ -0,0 +1,284 @@
+# XML
+
+Encode and decode to and from XML. Whitespace is not conserved for round trips - but the order of the fields are.
+
+Consecutive xml nodes with the same name are assumed to be arrays.
+
+All values in XML are assumed to be strings - but you can use `from_yaml` to parse them into their correct types:
+
+
+```
+yq e -p=xml '.myNumberField |= from_yaml' my.xml
+```
+
+
+```xml
+<cat name="tiger">meow</cat>
+```
+
+The content of the node will be set as a field in the map with the key "+content". Use the `--xml-content-name` flag to change this.
+
+## Parse xml: simple
+Given a sample.xml file of:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<cat>meow</cat>
+```
+then
+```bash
+yq e -p=xml '.' sample.xml
+```
+will output
+```yaml
+cat: meow
+```
+
+## Parse xml: array
+Consecutive nodes with identical xml names are assumed to be arrays.
+
+Given a sample.xml file of:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<animal>1</animal>
+<animal>2</animal>
+```
+then
+```bash
+yq e -p=xml '.' sample.xml
+```
+will output
+```yaml
+animal:
+  - "1"
+  - "2"
+```
+
+## Parse xml: attributes
+Attributes are converted to fields, with the default attribute prefix '+'. Use '--xml-attribute-prefix` to set your own.
+
+Given a sample.xml file of:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<cat legs="4">
+  <legs>7</legs>
+</cat>
+```
+then
+```bash
+yq e -p=xml '.' sample.xml
+```
+will output
+```yaml
+cat:
+  +legs: "4"
+  legs: "7"
+```
+
+## Parse xml: attributes with content
+Content is added as a field, using the default content name of '+content'. Use `--xml-content-name` to set your own.
+
+Given a sample.xml file of:
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<cat legs="4">meow</cat>
+```
+then
+```bash
+yq e -p=xml '.' sample.xml
+```
+will output
+```yaml
+cat:
+  +content: meow
+  +legs: "4"
+```
+
+## Parse xml: with comments
+A best attempt is made to preserve comments.
+
+Given a sample.xml file of:
+```xml
+
+<!-- before cat -->
+<cat>
+	<!-- in cat before -->
+	<x>3<!-- multi
+line comment 
+for x --></x>
+	<!-- before y -->
+	<y>
+		<!-- in y before -->
+		<d><!-- in d before -->z<!-- in d after --></d>
+		
+		<!-- in y after -->
+	</y>
+	<!-- in_cat_after -->
+</cat>
+<!-- after cat -->
+
+```
+then
+```bash
+yq e -p=xml '.' sample.xml
+```
+will output
+```yaml
+# before cat
+cat:
+  # in cat before
+  x: "3" # multi
+  # line comment 
+  # for x
+  # before y
+
+  y:
+    # in y before
+    # in d before
+    d: z # in d after
+    # in y after
+  # in_cat_after
+# after cat
+```
+
+## Encode xml: simple
+Given a sample.yml file of:
+```yaml
+cat: purrs
+```
+then
+```bash
+yq e -o=xml '.' sample.yml
+```
+will output
+```xml
+<cat>purrs</cat>
+```
+
+## Encode xml: array
+Given a sample.yml file of:
+```yaml
+pets:
+  cat:
+    - purrs
+    - meows
+```
+then
+```bash
+yq e -o=xml '.' sample.yml
+```
+will output
+```xml
+<pets>
+  <cat>purrs</cat>
+  <cat>meows</cat>
+</pets>
+```
+
+## Encode xml: attributes
+Fields with the matching xml-attribute-prefix are assumed to be attributes.
+
+Given a sample.yml file of:
+```yaml
+cat:
+  +name: tiger
+  meows: true
+
+```
+then
+```bash
+yq e -o=xml '.' sample.yml
+```
+will output
+```xml
+<cat name="tiger">
+  <meows>true</meows>
+</cat>
+```
+
+## Encode xml: attributes with content
+Fields with the matching xml-content-name is assumed to be content.
+
+Given a sample.yml file of:
+```yaml
+cat:
+  +name: tiger
+  +content: cool
+
+```
+then
+```bash
+yq e -o=xml '.' sample.yml
+```
+will output
+```xml
+<cat name="tiger">cool</cat>
+```
+
+## Encode xml: comments
+A best attempt is made to copy comments to xml.
+
+Given a sample.yml file of:
+```yaml
+# above_cat
+cat: # inline_cat
+  # above_array
+  array: # inline_array
+    - val1 # inline_val1
+    # above_val2
+    - val2 # inline_val2
+# below_cat
+
+```
+then
+```bash
+yq e -o=xml '.' sample.yml
+```
+will output
+```xml
+<!-- above_cat inline_cat --><cat><!-- above_array inline_array -->
+  <array>val1<!-- inline_val1 --></array>
+  <array><!-- above_val2 -->val2<!-- inline_val2 --></array>
+</cat><!-- below_cat -->
+```
+
+## Round trip: with comments
+A best effort is made, but comment positions and white space are not preserved perfectly.
+
+Given a sample.xml file of:
+```xml
+
+<!-- before cat -->
+<cat>
+	<!-- in cat before -->
+	<x>3<!-- multi
+line comment 
+for x --></x>
+	<!-- before y -->
+	<y>
+		<!-- in y before -->
+		<d><!-- in d before -->z<!-- in d after --></d>
+		
+		<!-- in y after -->
+	</y>
+	<!-- in_cat_after -->
+</cat>
+<!-- after cat -->
+
+```
+then
+```bash
+yq e -p=xml -o=xml '.' sample.xml
+```
+will output
+```xml
+<!-- before cat --><cat><!-- in cat before -->
+  <x>3<!-- multi
+line comment 
+for x --></x><!-- before y -->
+  <y><!-- in y before
+in d before -->
+    <d>z<!-- in d after --></d><!-- in y after -->
+  </y><!-- in_cat_after -->
+</cat><!-- after cat -->
+```
+