suyiiyii/yq
1
0
Fork 0
mirror of https://github.com/mikefarah/yq.git synced 2026-09-05 17:36:59 +08:00
Code Issues Packages Projects Releases Wiki Activity

v4.29.1

Browse Source
This commit is contained in:
Mike Farah
2022-10-29 18:24:11 +11:00
parent 28a82e9ae8
commit a7e397757b
56 changed files with 238 additions and 335 deletions
Download Patch File Download Diff File Expand all files Collapse all files
usage/convert.md
-6
View File
@@ -5,12 +5,6 @@ Encode and decode to and from JSON. Supports multiple JSON documents in a single
Note that YAML is a superset of (single document) JSON - so you don't have to use the JSON parser to read JSON when there is only one JSON document in the input. You will probably want to pretty print the result in this case, to get idiomatic YAML styling.
{% hint style="warning" %}
Note that versions prior to 4.18 require the 'eval/e' command to be specified. 
`yq e <exp> <file>`
{% endhint %}
## Parse json: simple
JSON is a subset of yaml, so all you need to do is prettify the output
usage/csv-tsv.md
-6
View File
@@ -29,12 +29,6 @@ Fifi,cat
```
{% hint style="warning" %}
Note that versions prior to 4.18 require the 'eval/e' command to be specified.&#x20;
`yq e <exp> <file>`
{% endhint %}
## Encode CSV simple
Given a sample.yml file of:
```yaml
usage/properties.md
+15 -12
View File
@@ -4,18 +4,12 @@ Encode/Decode/Roundtrip to/from a property file. Line comments on value nodes wi
By default, empty maps and arrays are not encoded - see below for an example on how to encode a value for these.
{% hint style="warning" %}
Note that versions prior to 4.18 require the 'eval/e' command to be specified.&#x20;
`yq e <exp> <file>`
{% endhint %}
## Encode properties
Note that empty arrays and maps are not encoded by default.
Given a sample.yml file of:
```yaml
# block comments don't come through
# block comments come through
person: # neither do comments on maps
name: Mike Wazowski # comments on values appear
pets:
@@ -31,6 +25,7 @@ yq -o=props sample.yml
```
will output
```properties
# block comments come through
# comments on values appear
person.name = Mike Wazowski
@@ -44,7 +39,7 @@ Note that string values with blank characters in them are encapsulated with doub
Given a sample.yml file of:
```yaml
# block comments don't come through
# block comments come through
person: # neither do comments on maps
name: Mike Wazowski # comments on values appear
pets:
@@ -60,6 +55,7 @@ yq -o=props --unwrapScalar=false sample.yml
```
will output
```properties
# block comments come through
# comments on values appear
person.name = "Mike Wazowski"
@@ -71,7 +67,7 @@ person.food.0 = pizza
## Encode properties: no comments
Given a sample.yml file of:
```yaml
# block comments don't come through
# block comments come through
person: # neither do comments on maps
name: Mike Wazowski # comments on values appear
pets:
@@ -97,7 +93,7 @@ Use a yq expression to set the empty maps and sequences to your desired value.
Given a sample.yml file of:
```yaml
# block comments don't come through
# block comments come through
person: # neither do comments on maps
name: Mike Wazowski # comments on values appear
pets:
@@ -113,6 +109,7 @@ yq -o=props '(.. | select( (tag == "!!map" or tag =="!!seq") and length == 0)) =
```
will output
```properties
# block comments come through
# comments on values appear
person.name = Mike Wazowski
@@ -126,6 +123,7 @@ emptyMap =
## Decode properties
Given a sample.properties file of:
```properties
# block comments come through
# comments on values appear
person.name = Mike Wazowski
@@ -141,9 +139,12 @@ yq -p=props sample.properties
will output
```yaml
person:
name: Mike Wazowski # comments on values appear
# block comments come through
# comments on values appear
name: Mike Wazowski
pets:
- cat # comments on array values appear
# comments on array values appear
- cat
food:
- pizza
```
@@ -151,6 +152,7 @@ person:
## Roundtrip
Given a sample.properties file of:
```properties
# block comments come through
# comments on values appear
person.name = Mike Wazowski
@@ -165,6 +167,7 @@ yq -p=props -o=props '.person.pets.0 = "dog"' sample.properties
```
will output
```properties
# block comments come through
# comments on values appear
person.name = Mike Wazowski
usage/xml.md
+138 -11
View File
@@ -4,14 +4,41 @@ Encode and decode to and from XML. Whitespace is not conserved for round trips -
Consecutive xml nodes with the same name are assumed to be arrays.
XML content data and attributes are created as fields. This can be controlled by the `'--xml-attribute-prefix` and `--xml-content-name` flags - see below for examples.
XML content data, attributes processing instructions and directives are all created as plain fields.
This can be controlled by:
| Flag | Default |Sample XML |
| -- | -- | -- |
| `--xml-attribute-prefix` | `+` (changing to `+@` soon) | Legs in ```<cat legs="4"/>``` |
| `--xml-content-name` | `+content` | Meow in ```<cat>Meow <fur>true</true></cat>``` |
| `--xml-directive-name` | `+directive` | ```<!DOCTYPE config system "blah">``` |
| `--xml-proc-inst-prefix` | `+p_` | ```<?xml version="1"?>``` |
{% hint style="warning" %}
Note that versions prior to 4.18 require the 'eval/e' command to be specified.&#x20;
Default Attribute Prefix will be changing in v4.30!
In order to avoid name conflicts (e.g. having an attribute named "content" will create a field that clashes with the default content name of "+content") the attribute prefix will be changing to "+@".
This will affect users that have not set their own prefix and are not roundtripping XML changes.
`yq e <exp> <file>`
{% endhint %}
## Encoder / Decoder flag options
In addition to the above flags, there are the following xml encoder/decoder options controlled by flags:
| Flag | Default | Description |
| -- | -- | -- |
| `--xml-strict-mode` | false | Strict mode enforces the requirements of the XML specification. When switched off the parser allows input containing common mistakes. See [the Golang xml decoder ](https://pkg.go.dev/encoding/xml#Decoder) for more details.|
| `--xml-keep-namespace` | true | Keeps the namespace of attributes |
| `--xml-raw-token` | true | Does not verify that start and end elements match and does not translate name space prefixes to their corresponding URLs. |
| `--xml-skip-proc-inst` | false | Skips over processing instructions, e.g. `<?xml version="1"?>` |
| `--xml-skip-directives` | false | Skips over directives, e.g. ```<!DOCTYPE config system "blah">``` |
See below for examples
## Parse xml: simple
Notice how all the values are strings, see the next example on how you can fix that.
@@ -30,6 +57,7 @@ yq -p=xml '.' sample.xml
```
will output
```yaml
+p_xml: version="1.0" encoding="UTF-8"
cat:
says: meow
legs: "4"
@@ -54,6 +82,7 @@ yq -p=xml ' (.. | select(tag == "!!str")) |= from_yaml' sample.xml
```
will output
```yaml
+p_xml: version="1.0" encoding="UTF-8"
cat:
says: meow
legs: 4
@@ -75,6 +104,7 @@ yq -p=xml '.' sample.xml
```
will output
```yaml
+p_xml: version="1.0" encoding="UTF-8"
animal:
- cat
- goat
@@ -96,6 +126,7 @@ yq -p=xml '.' sample.xml
```
will output
```yaml
+p_xml: version="1.0" encoding="UTF-8"
cat:
+legs: "4"
legs: "7"
@@ -115,13 +146,14 @@ yq -p=xml '.' sample.xml
```
will output
```yaml
+p_xml: version="1.0" encoding="UTF-8"
cat:
+content: meow
+legs: "4"
```
## Parse xml: custom dtd
DTD entities are ignored.
DTD entities are processed as directives.
Given a sample.xml file of:
```xml
@@ -137,12 +169,45 @@ Given a sample.xml file of:
```
then
```bash
yq -p=xml '.' sample.xml
yq -p=xml -o=xml '.' sample.xml
```
will output
```yaml
root:
item: '&writer;&copyright;'
```xml
<?xml version="1.0"?>
<!DOCTYPE root [
<!ENTITY writer "Blah.">
<!ENTITY copyright "Blah">
]>
<root>
<item>&amp;writer;&amp;copyright;</item>
</root>
```
## Parse xml: skip custom dtd
DTDs are directives, skip over directives to skip DTDs.
Given a sample.xml file of:
```xml
<?xml version="1.0"?>
<!DOCTYPE root [
<!ENTITY writer "Blah.">
<!ENTITY copyright "Blah">
]>
<root>
<item>&writer;&copyright;</item>
</root>
```
then
```bash
yq -p=xml -o=xml --xml-skip-directives '.' sample.xml
```
will output
```xml
<?xml version="1.0"?>
<root>
<item>&amp;writer;&amp;copyright;</item>
</root>
```
## Parse xml: with comments
@@ -207,12 +272,14 @@ yq -p=xml -o=xml --xml-keep-namespace '.' sample.xml
```
will output
```xml
<?xml version="1.0"?>
<map xmlns="some-namespace" xmlns:xsi="some-instance" some-instance:schemaLocation="some-url"></map>
```
instead of
```xml
<map xmlns="some-namespace" xsi="some-instance" schemaLocation="some-url"></map>
<?xml version="1.0"?>
<map xmlns="some-namespace" xmlns:xsi="some-instance" some-instance:schemaLocation="some-url"></map>
```
## Parse xml: keep raw attribute namespace
@@ -230,11 +297,13 @@ yq -p=xml -o=xml --xml-keep-namespace --xml-raw-token '.' sample.xml
```
will output
```xml
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>
<?xml version="1.0"?>
<map xmlns="some-namespace" xmlns:xsi="some-instance" some-instance:schemaLocation="some-url"></map>
```
instead of
```xml
<?xml version="1.0"?>
<map xmlns="some-namespace" xsi="some-instance" schemaLocation="some-url"></map>
```
@@ -317,6 +386,7 @@ A best attempt is made to copy comments to xml.
Given a sample.yml file of:
```yaml
# header comment
# above_cat
cat: # inline_cat
# above_array
@@ -333,12 +403,41 @@ yq -o=xml '.' sample.yml
```
will output
```xml
<!-- above_cat inline_cat --><cat><!-- above_array inline_array -->
<!--
header comment
above_cat
--><!-- inline_cat --><cat><!-- above_array inline_array -->
<array>val1<!-- inline_val1 --></array>
<array><!-- above_val2 -->val2<!-- inline_val2 --></array>
</cat><!-- below_cat -->
```
## Encode: doctype and xml declaration
Use the special xml names to add/modify proc instructions and directives.
Given a sample.yml file of:
```yaml
+p_xml: version="1.0"
+directive: 'DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" '
apple:
+p_coolioo: version="1.0"
+directive: 'CATYPE meow purr puss '
b: things
```
then
```bash
yq -o=xml '.' sample.yml
```
will output
```xml
<?xml version="1.0"?>
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
<apple><?coolioo version="1.0"?><!CATYPE meow purr puss >
<b>things</b>
</apple>
```
## Round trip: with comments
A best effort is made, but comment positions and white space are not preserved perfectly.
@@ -380,3 +479,31 @@ in d before -->
</cat><!-- after cat -->
```
## Roundtrip: with doctype and declaration
yq parses XML proc instructions and directives into nodes.
Unfortunately the underlying XML parser loses whitespace information.
Given a sample.xml file of:
```xml
<?xml version="1.0"?>
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
<apple>
<?coolioo version="1.0"?>
<!CATYPE meow purr puss >
<b>things</b>
</apple>
```
then
```bash
yq -p=xml -o=xml '.' sample.xml
```
will output
```xml
<?xml version="1.0"?>
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
<apple><?coolioo version="1.0"?><!CATYPE meow purr puss >
<b>things</b>
</apple>
```