suyiiyii/yq
1
0
Fork 0
mirror of https://github.com/mikefarah/yq.git synced 2026-07-05 20:15:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

Making examples of the merge fix flag clearer

Browse Source
This commit is contained in:
Mike Farah 2025-07-22 10:18:39 +10:00
parent f35e57d901
commit 84b095bbc4
8 changed files with 69 additions and 36 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
pkg/yqlib/doc/operators/anchor-and-alias-operators.md
View File

@ -5,6 +5,14 @@ Use the `alias` and `anchor` operators to read and write yaml aliases and anchor
`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag. `yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
## NOTE --yaml-fix-merge-anchor-to-spec flag
`yq` doesn't merge anchors `<<:` to spec, in some circumstances it incorrectly overrides existing keys when the spec documents not to do that.
To minimise disruption while still fixing the issue, a flag has been added to toggle this behaviour. This will first default to false; and log warnings to users. Then it will default to true (and still allow users to specify false if needed)
See examples of the flag difference below.
## Merge one map ## Merge one map
see https://yaml.org/type/merge.html see https://yaml.org/type/merge.html
@ -285,7 +293,7 @@ thingTwo:
``` ```
## LEGACY: Explode with merge anchors ## LEGACY: Explode with merge anchors
Caution: this is for when --yaml-fix-merge-anchor-to-spec=false; it's not to YAML spec because the merge anchors incorrectly override the object values. Flag will default to true in late 2025 Caution: this is for when --yaml-fix-merge-anchor-to-spec=false; it's not to YAML spec because the merge anchors incorrectly override the object values (foobarList.b is set to bar_b when it should still be foobarList_b). Flag will default to true in late 2025
Given a sample.yml file of: Given a sample.yml file of:
```yaml ```yaml
@ -334,7 +342,7 @@ foobar:
``` ```
## FIXED: Explode with merge anchors ## FIXED: Explode with merge anchors
Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Flag will default to true in late 2025 See the foobarList.b property is still foobarList_b. Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Flag will default to true in late 2025
Given a sample.yml file of: Given a sample.yml file of:
```yaml ```yaml
@ -383,7 +391,7 @@ foobar:
``` ```
## FIXED: Explode with merge anchors ## FIXED: Explode with merge anchors
Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Flag will default to true in late 2025 See the foobarList.b property is still foobarList_b. Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Flag will default to true in late 2025
Given a sample.yml file of: Given a sample.yml file of:
```yaml ```yaml

8
pkg/yqlib/doc/operators/headers/anchor-and-alias-operators.md
View File

@ -4,3 +4,11 @@ Use the `alias` and `anchor` operators to read and write yaml aliases and anchor
`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag. `yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
## NOTE --yaml-fix-merge-anchor-to-spec flag
`yq` doesn't merge anchors `<<:` to spec, in some circumstances it incorrectly overrides existing keys when the spec documents not to do that.
To minimise disruption while still fixing the issue, a flag has been added to toggle this behaviour. This will first default to false; and log warnings to users. Then it will default to true (and still allow users to specify false if needed)
See examples of the flag difference below.

9
pkg/yqlib/doc/operators/headers/traverse-read.md
View File

@ -1,3 +1,12 @@
# Traverse (Read) # Traverse (Read)
This is the simplest (and perhaps most used) operator. It is used to navigate deeply into yaml structures. This is the simplest (and perhaps most used) operator. It is used to navigate deeply into yaml structures.
## NOTE --yaml-fix-merge-anchor-to-spec flag
`yq` doesn't merge anchors `<<:` to spec, in some circumstances it incorrectly overrides existing keys when the spec documents not to do that.
To minimise disruption while still fixing the issue, a flag has been added to toggle this behaviour. This will first default to false; and log warnings to users. Then it will default to true (and still allow users to specify false if needed)
See examples of the flag differences below.

17
pkg/yqlib/doc/operators/traverse-read.md
View File

@ -2,6 +2,15 @@
This is the simplest (and perhaps most used) operator. It is used to navigate deeply into yaml structures. This is the simplest (and perhaps most used) operator. It is used to navigate deeply into yaml structures.
## NOTE --yaml-fix-merge-anchor-to-spec flag
`yq` doesn't merge anchors `<<:` to spec, in some circumstances it incorrectly overrides existing keys when the spec documents not to do that.
To minimise disruption while still fixing the issue, a flag has been added to toggle this behaviour. This will first default to false; and log warnings to users. Then it will default to true (and still allow users to specify false if needed)
See examples of the flag differences below.
## Simple map navigation ## Simple map navigation
Given a sample.yml file of: Given a sample.yml file of:
```yaml ```yaml
@ -352,7 +361,7 @@ a
c c
``` ```
## Traversing merge anchors with override ## LEGACY: Traversing merge anchors with override
This is legacy behaviour, see --yaml-fix-merge-anchor-to-spec This is legacy behaviour, see --yaml-fix-merge-anchor-to-spec
Given a sample.yml file of: Given a sample.yml file of:
@ -385,7 +394,7 @@ will output
foo_c foo_c
``` ```
## Traversing merge anchor lists ## LEGACY: Traversing merge anchor lists
Note that the later merge anchors override previous, but this is legacy behaviour, see --yaml-fix-merge-anchor-to-spec Note that the later merge anchors override previous, but this is legacy behaviour, see --yaml-fix-merge-anchor-to-spec
Given a sample.yml file of: Given a sample.yml file of:
@ -418,7 +427,7 @@ will output
bar_thing bar_thing
``` ```
## Splatting merge anchors ## LEGACY: Splatting merge anchors
With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec
Given a sample.yml file of: Given a sample.yml file of:
@ -453,7 +462,7 @@ foo_a
foobar_thing foobar_thing
``` ```
## Splatting merge anchor lists ## LEGACY: Splatting merge anchor lists
With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec
Given a sample.yml file of: Given a sample.yml file of:

2
pkg/yqlib/operator_anchors_aliases.go
View File

@ -162,7 +162,7 @@ func reconstructAliasedMap(node *CandidateNode, context Context) error {
} }
} }
} else if showMergeAnchorToSpecWarning { } else if showMergeAnchorToSpecWarning {
log.Warning("--yaml-fix-merge-anchor-to-spec is false; causing merge anchors to override the existing values which isn't to the yaml spec. This flag will default to true in late 2025.") log.Warning("--yaml-fix-merge-anchor-to-spec is false; causing merge anchors to override the existing values which isn't to the yaml spec. This flag will default to true in late 2025. See https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators for more details.")
showMergeAnchorToSpecWarning = false showMergeAnchorToSpecWarning = false
} }

6
pkg/yqlib/operator_anchors_aliases_test.go
View File

@ -104,14 +104,14 @@ var fixedAnchorOperatorScenarios = []expressionScenario{
{ {
skipDoc: true, skipDoc: true,
description: "merge anchor after existing keys", description: "merge anchor after existing keys",
subdescription: "Does not override existing keys", subdescription: "Does not override existing keys - note the name field in the second element is still ellipse.",
document: explodeWhenKeysExistDocument, document: explodeWhenKeysExistDocument,
expression: "explode(.)", expression: "explode(.)",
expected: []string{explodeWhenKeysExistExpected}, expected: []string{explodeWhenKeysExistExpected},
}, },
{ {
description: "FIXED: Explode with merge anchors", description: "FIXED: Explode with merge anchors",
subdescription: "Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Flag will default to true in late 2025 ", subdescription: "See the foobarList.b property is still foobarList_b. Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Flag will default to true in late 2025 ",
document: mergeDocSample, document: mergeDocSample,
expression: `explode(.)`, expression: `explode(.)`,
expected: []string{explodeMergeAnchorsFixedExpected}, expected: []string{explodeMergeAnchorsFixedExpected},
@ -148,7 +148,7 @@ var badAnchorOperatorScenarios = []expressionScenario{
}, },
{ {
description: "LEGACY: Explode with merge anchors", // incorrect overrides description: "LEGACY: Explode with merge anchors", // incorrect overrides
subdescription: "Caution: this is for when --yaml-fix-merge-anchor-to-spec=false; it's not to YAML spec because the merge anchors incorrectly override the object values. Flag will default to true in late 2025", subdescription: "Caution: this is for when --yaml-fix-merge-anchor-to-spec=false; it's not to YAML spec because the merge anchors incorrectly override the object values (foobarList.b is set to bar_b when it should still be foobarList_b). Flag will default to true in late 2025",
document: mergeDocSample, document: mergeDocSample,
expression: `explode(.)`, expression: `explode(.)`,
expected: []string{explodeMergeAnchorsExpected}, expected: []string{explodeMergeAnchorsExpected},

2
pkg/yqlib/operator_traverse_path.go
View File

@ -295,7 +295,7 @@ func doTraverseMap(newMatches *orderedmap.OrderedMap, node *CandidateNode, wante
if !ConfiguredYamlPreferences.FixMergeAnchorToSpec { if !ConfiguredYamlPreferences.FixMergeAnchorToSpec {
log.Debug("Merge anchor") log.Debug("Merge anchor")
if showMergeAnchorToSpecWarning { if showMergeAnchorToSpecWarning {
log.Warning("--yaml-fix-merge-anchor-to-spec is false; causing merge anchors to override the existing values which isn't to the yaml spec. This flag will default to true in late 2025.") log.Warning("--yaml-fix-merge-anchor-to-spec is false; causing merge anchors to override the existing values which isn't to the yaml spec. This flag will default to true in late 2025. See https://mikefarah.gitbook.io/yq/operators/traverse-read for more details.")
showMergeAnchorToSpecWarning = false showMergeAnchorToSpecWarning = false
} }
err := traverseMergeAnchor(newMatches, value, wantedKey, prefs, splat) err := traverseMergeAnchor(newMatches, value, wantedKey, prefs, splat)

47
pkg/yqlib/operator_traverse_path_test.go
View File

@ -27,9 +27,17 @@ foobar:
var fixedTraversePathOperatorScenarios = []expressionScenario{ var fixedTraversePathOperatorScenarios = []expressionScenario{
{ {
skipDoc: true, description: "FIXED: Traversing merge anchors with override",
description: "Traversing merge anchor lists", subdescription: "Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour.",
subdescription: "Note that the keys earlier in the merge anchors sequence override later ones", document: mergeDocSample,
expression: `.foobar.c`,
expected: []string{
"D0, P[foobar c], (!!str)::foobar_c\n",
},
},
{
description: "FIXED: Traversing merge anchor lists",
subdescription: "Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Note that the keys earlier in the merge anchors sequence override later ones",
document: mergeDocSample, document: mergeDocSample,
expression: `.foobarList.thing`, expression: `.foobarList.thing`,
expected: []string{ expected: []string{
@ -37,19 +45,10 @@ var fixedTraversePathOperatorScenarios = []expressionScenario{
}, },
}, },
{ {
skipDoc: true, description: "FIXED: Splatting merge anchors",
description: "Traversing merge anchors with override", subdescription: "Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Note that the keys earlier in the merge anchors sequence override later ones",
document: mergeDocSample, document: mergeDocSample,
expression: `.foobar.c`, expression: `.foobar[]`,
expected: []string{
"D0, P[foobar c], (!!str)::foobar_c\n",
},
},
{
skipDoc: true,
description: "Splatting merge anchors",
document: mergeDocSample,
expression: `.foobar[]`,
expected: []string{ expected: []string{
"D0, P[foo a], (!!str)::foo_a\n", "D0, P[foo a], (!!str)::foo_a\n",
"D0, P[foobar thing], (!!str)::foobar_thing\n", "D0, P[foobar thing], (!!str)::foobar_thing\n",
@ -57,10 +56,10 @@ var fixedTraversePathOperatorScenarios = []expressionScenario{
}, },
}, },
{ {
skipDoc: true, description: "FIXED: Splatting merge anchor lists",
description: "Splatting merge anchor lists", subdescription: "Set `--yaml-fix-merge-anchor-to-spec=true` to get this correct merge behaviour. Note that the keys earlier in the merge anchors sequence override later ones",
document: mergeDocSample, document: mergeDocSample,
expression: `.foobarList[]`, expression: `.foobarList[]`,
expected: []string{ expected: []string{
"D0, P[foobarList b], (!!str)::foobarList_b\n", "D0, P[foobarList b], (!!str)::foobarList_b\n",
"D0, P[foo thing], (!!str)::foo_thing\n", "D0, P[foo thing], (!!str)::foo_thing\n",
@ -80,7 +79,7 @@ var fixedTraversePathOperatorScenarios = []expressionScenario{
var badTraversePathOperatorScenarios = []expressionScenario{ var badTraversePathOperatorScenarios = []expressionScenario{
{ {
description: "Traversing merge anchors with override", description: "LEGACY: Traversing merge anchors with override",
subdescription: "This is legacy behaviour, see --yaml-fix-merge-anchor-to-spec", subdescription: "This is legacy behaviour, see --yaml-fix-merge-anchor-to-spec",
document: mergeDocSample, document: mergeDocSample,
expression: `.foobar.c`, expression: `.foobar.c`,
@ -89,7 +88,7 @@ var badTraversePathOperatorScenarios = []expressionScenario{
}, },
}, },
{ {
description: "Traversing merge anchor lists", description: "LEGACY: Traversing merge anchor lists",
subdescription: "Note that the later merge anchors override previous, " + subdescription: "Note that the later merge anchors override previous, " +
"but this is legacy behaviour, see --yaml-fix-merge-anchor-to-spec", "but this is legacy behaviour, see --yaml-fix-merge-anchor-to-spec",
document: mergeDocSample, document: mergeDocSample,
@ -99,7 +98,7 @@ var badTraversePathOperatorScenarios = []expressionScenario{
}, },
}, },
{ {
description: "Splatting merge anchors", description: "LEGACY: Splatting merge anchors",
subdescription: "With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec", subdescription: "With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec",
document: mergeDocSample, document: mergeDocSample,
expression: `.foobar[]`, expression: `.foobar[]`,
@ -110,7 +109,7 @@ var badTraversePathOperatorScenarios = []expressionScenario{
}, },
}, },
{ {
description: "Splatting merge anchor lists", description: "LEGACY: Splatting merge anchor lists",
subdescription: "With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec", subdescription: "With legacy override behaviour, see --yaml-fix-merge-anchor-to-spec",
document: mergeDocSample, document: mergeDocSample,
expression: `.foobarList[]`, expression: `.foobarList[]`,