suyiiyii/yq
1
0
Fork 0
mirror of https://github.com/mikefarah/yq.git synced 2024-12-19 20:19:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

nicer reduce example

Browse Source
This commit is contained in:
Mike Farah 2021-02-15 18:23:50 +11:00
parent ee1f55630f
commit a6cd250987
4 changed files with 34 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

73
pkg/yqlib/doc/Reduce.md
View File

@ -12,16 +12,14 @@ e.g.
On the LHS we are configuring the collection of items that will be reduced `<exp>` as well as what each element will be called `$<name>`. Note that the array has been splatted into its individual elements.
On the RHS there is `<init>`, the starting value of the accumulator and `<block>`, the expression that will update the accumulator for each element in the collection.
Note that within the block expression, `.` will evaluate to the current value of the accumulator. This effectively means that within the `reduce` block you can no longer access data other than elements of the array set as `$<name>`. For simple things, this is probably fine, but often you will need to refer to other data elements.
This can be done by setting a variable using `as` and piping that into the `reduce` operation, or you can simply refer to `$context` which is exactly that, automatically set for you for convenience. See examples below.
On the RHS there is `<init>`, the starting value of the accumulator and `<block>`, the expression that will update the accumulator for each element in the collection. Note that within the block expression, `.` will evaluate to the current value of the accumulator.
## yq vs jq syntax
Reduce syntax in `yq` is a little different from `jq` - as `yq` (currently) isn't as sophisticated as `jq` and its only supports infix notation (e.g. a + b, where the operator is in the middle of the two parameters) - where as `jq` uses a mix of infix notation with _prefix_ notation (e.g. `reduce a b` is like writing `+ a b`).
To that end, the reduce operator is called `ireduce` for backwards compatability if a prefix version of `reduce` is ever added.
To that end, the reduce operator is called `ireduce` for backwards compatability if a `jq` like prefix version of `reduce` is ever added.
## Sum numbers
Given a sample.yml file of:
```yaml
@ -39,6 +37,25 @@ will output
20
```
## Merge all yaml files together
Given a sample.yml file of:
```yaml
a: cat
```
And another sample another.yml file of:
```yaml
b: dog
```
then
```bash
yq eval-all '. as $item ireduce ({}; . * $item )' sample.yml another.yml
```
will output
```yaml
a: cat
b: dog
```
## Convert an array to an object
Given a sample.yml file of:
```yaml
@ -57,47 +74,3 @@ Cathy: apples
Bob: bananas
```
## Merge all documents together - using context
The _$context_ variable set by reduce lets you access the data outside the reduce block.
Given a sample.yml file of:
```yaml
a: cat
```
And another sample another.yml file of:
```yaml
b: dog
```
then
```bash
yq eval-all 'fileIndex as $item ireduce ({}; . * ($context | select(fileIndex==$item)) )' sample.yml another.yml
```
will output
```yaml
a: cat
b: dog
```
## Merge all documents together - without using context
`$context` is just a convenient variable that `reduce` sets, you can use your own for more control
Given a sample.yml file of:
```yaml
c:
a: cat
```
And another sample another.yml file of:
```yaml
c:
b: dog
```
then
```bash
yq eval-all '.c as $root | fileIndex as $item ireduce ({}; . * ($root | select(fileIndex==$item)) )' sample.yml another.yml
```
will output
```yaml
a: cat
b: dog
```

9
pkg/yqlib/doc/headers/Reduce.md
View File

@ -12,13 +12,10 @@ e.g.
On the LHS we are configuring the collection of items that will be reduced `<exp>` as well as what each element will be called `$<name>`. Note that the array has been splatted into its individual elements.
On the RHS there is `<init>`, the starting value of the accumulator and `<block>`, the expression that will update the accumulator for each element in the collection.
Note that within the block expression, `.` will evaluate to the current value of the accumulator. This effectively means that within the `reduce` block you can no longer access data other than elements of the array set as `$<name>`. For simple things, this is probably fine, but often you will need to refer to other data elements.
This can be done by setting a variable using `as` and piping that into the `reduce` operation, or you can simply refer to `$context` which is exactly that, automatically set for you for convenience. See examples below.
On the RHS there is `<init>`, the starting value of the accumulator and `<block>`, the expression that will update the accumulator for each element in the collection. Note that within the block expression, `.` will evaluate to the current value of the accumulator.
## yq vs jq syntax
Reduce syntax in `yq` is a little different from `jq` - as `yq` (currently) isn't as sophisticated as `jq` and its only supports infix notation (e.g. a + b, where the operator is in the middle of the two parameters) - where as `jq` uses a mix of infix notation with _prefix_ notation (e.g. `reduce a b` is like writing `+ a b`).
To that end, the reduce operator is called `ireduce` for backwards compatability if a prefix version of `reduce` is ever added.
To that end, the reduce operator is called `ireduce` for backwards compatability if a `jq` like prefix version of `reduce` is ever added.

2
pkg/yqlib/operator_reduce.go
View File

@ -39,8 +39,6 @@ func reduceOperator(d *dataTreeNavigator, context Context, expressionNode *Expre
return Context{}, err
}
accum.SetVariable("context", context.MatchingNodes)
log.Debugf("with variable %v", variableName)
blockExp := expressionNode.Rhs.Rhs

27
pkg/yqlib/operator_reduce_test.go
View File

@ -14,31 +14,20 @@ var reduceOperatorScenarios = []expressionScenario{
},
},
{
description: "Convert an array to an object",
document: `[{name: Cathy, has: apples},{name: Bob, has: bananas}]`,
expression: `.[] as $item ireduce ({}; .[$item | .name] = ($item | .has) )`,
expected: []string{
"D0, P[], (!!map)::Cathy: apples\nBob: bananas\n",
},
},
{
description: "Merge all documents together - using context",
subdescription: "The _$context_ variable set by reduce lets you access the data outside the reduce block.",
document: `a: cat`,
document2: `b: dog`,
expression: `fi as $item ireduce ({}; . * ($context | select(fileIndex==$item)) )`,
description: "Merge all yaml files together",
document: `a: cat`,
document2: `b: dog`,
expression: `. as $item ireduce ({}; . * $item )`,
expected: []string{
"D0, P[], (!!map)::a: cat\nb: dog\n",
},
},
{
description: "Merge all documents together - without using context",
subdescription: "`$context` is just a convenient variable that `reduce` sets, you can use your own for more control",
document: `c: {a: cat}`,
document2: `c: {b: dog}`,
expression: `.c as $root | fileIndex as $item ireduce ({}; . * ($root | select(fileIndex==$item)) )`,
description: "Convert an array to an object",
document: `[{name: Cathy, has: apples},{name: Bob, has: bananas}]`,
expression: `.[] as $item ireduce ({}; .[$item | .name] = ($item | .has) )`,
expected: []string{
"D0, P[], (!!map)::{a: cat, b: dog}\n",
"D0, P[], (!!map)::Cathy: apples\nBob: bananas\n",
},
},
}