Compare commits

...

77 Commits

Author SHA1 Message Date
dependabot[bot]
e2f1d5ccf7
Bump go.yaml.in/yaml/v4 from 4.0.0-rc.5 to 4.0.0-rc.6 (#2759)
Bumps [go.yaml.in/yaml/v4](https://github.com/yaml/go-yaml) from 4.0.0-rc.5 to 4.0.0-rc.6.
- [Commits](https://github.com/yaml/go-yaml/compare/v4.0.0-rc.5...v4.0.0-rc.6)

---
updated-dependencies:
- dependency-name: go.yaml.in/yaml/v4
  dependency-version: 4.0.0-rc.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 16:54:05 +10:00
dependabot[bot]
16f149b351
Bump github.com/pelletier/go-toml/v2 from 2.4.0 to 2.4.2 (#2762)
Bumps [github.com/pelletier/go-toml/v2](https://github.com/pelletier/go-toml) from 2.4.0 to 2.4.2.
- [Release notes](https://github.com/pelletier/go-toml/releases)
- [Commits](https://github.com/pelletier/go-toml/compare/v2.4.0...v2.4.2)

---
updated-dependencies:
- dependency-name: github.com/pelletier/go-toml/v2
  dependency-version: 2.4.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 16:53:39 +10:00
dependabot[bot]
5da9215306
Bump actions/setup-go from 6.4.0 to 6.5.0 (#2763)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 6.4.0 to 6.5.0.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](4a3601121d...924ae3a1cd)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: 6.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-29 16:53:18 +10:00
dependabot[bot]
e95bb7e472
Bump golang.org/x/net from 0.55.0 to 0.56.0 (#2740)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.55.0 to 0.56.0.
- [Commits](https://github.com/golang/net/compare/v0.55.0...v0.56.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.56.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-23 11:11:00 +10:00
dependabot[bot]
2074319595
Bump golang.org/x/mod from 0.36.0 to 0.37.0 (#2741)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.36.0 to 0.37.0.
- [Commits](https://github.com/golang/mod/compare/v0.36.0...v0.37.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-version: 0.37.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-23 10:04:36 +10:00
dependabot[bot]
be992d8add
Bump alpine from a2d49ea to 28bd5fe (#2752)
Bumps alpine from `a2d49ea` to `28bd5fe`.

---
updated-dependencies:
- dependency-name: alpine
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-23 10:04:28 +10:00
dependabot[bot]
637bb1fecd
Bump golang from 11fd8f7 to 792443b (#2753)
Bumps golang from `11fd8f7` to `792443b`.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.26.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-23 10:04:05 +10:00
dependabot[bot]
bc23b42789
Bump github.com/pelletier/go-toml/v2 from 2.3.1 to 2.4.0 (#2754)
Bumps [github.com/pelletier/go-toml/v2](https://github.com/pelletier/go-toml) from 2.3.1 to 2.4.0.
- [Release notes](https://github.com/pelletier/go-toml/releases)
- [Commits](https://github.com/pelletier/go-toml/compare/v2.3.1...v2.4.0)

---
updated-dependencies:
- dependency-name: github.com/pelletier/go-toml/v2
  dependency-version: 2.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-23 10:03:55 +10:00
dependabot[bot]
8e2c9b612d
Bump golang from 68cb6d6 to 11fd8f7 (#2738)
Bumps golang from `68cb6d6` to `11fd8f7`.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.26.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-16 16:46:10 +10:00
dependabot[bot]
0970cd4b05
Bump alpine from 5b10f43 to a2d49ea (#2739)
Bumps alpine from `5b10f43` to `a2d49ea`.

---
updated-dependencies:
- dependency-name: alpine
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-16 16:46:03 +10:00
dependabot[bot]
bf3591a234
Bump golang.org/x/text from 0.37.0 to 0.38.0 (#2742)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.37.0 to 0.38.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.37.0...v0.38.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-version: 0.38.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-16 16:45:37 +10:00
dependabot[bot]
09f1565d51
Bump github/codeql-action from 4.35.2 to 4.36.2 (#2743)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.35.2 to 4.36.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](95e58e9a2c...8aad20d150)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 4.36.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-16 16:29:01 +10:00
dependabot[bot]
13d340ff51
Bump sigstore/cosign-installer from 3.10.1 to 4.1.2 (#2744)
Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 3.10.1 to 4.1.2.
- [Release notes](https://github.com/sigstore/cosign-installer/releases)
- [Commits](7e8b541eb2...6f9f177880)

---
updated-dependencies:
- dependency-name: sigstore/cosign-installer
  dependency-version: 4.1.2
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-16 16:28:52 +10:00
Mike Farah
5cf0adcc5b Adding some tests 2026-06-09 14:23:11 +10:00
William Floyd
30e16a33c3
Fix for #2677 (#2705)
* Update docs given https://github.com/yaml/go-yaml/pull/348

* Fix for https://github.com/mikefarah/yq/issues/2677

Depends on https://github.com/yaml/go-yaml/pull/348

* Test for https://github.com/mikefarah/yq/issues/2677

* Remove redundant check and add test case for explicit `!!merge` on `*+` traversal

* Bump go.yaml.in/yaml/v4 from 4.0.0-rc.4 to 4.0.0-rc.5
2026-06-09 14:08:47 +10:00
Mike Farah
25dfcf280f Switch to typos from cspell - no more npm 2026-06-07 20:09:44 +10:00
Mike Farah
91a166e8d8 Pinning using hashes 2026-06-07 19:57:15 +10:00
Mike Farah
f9b0d7e45d Pinning using hashes 2026-06-07 19:49:39 +10:00
Mike Farah
48a851bf57 Fixing toml docs 2026-06-07 19:47:11 +10:00
Mike Farah
131aa0b7cc pinning deps 2026-06-07 19:46:58 +10:00
Mike Farah
ef3c14f806 project words 2026-06-07 18:21:59 +10:00
Mike Farah
26434e221e Process for SHA-pin github action 2026-06-07 18:19:13 +10:00
Mike Farah
0eebc242fb New workflow for github action 2026-06-07 08:56:21 +10:00
Mike Farah
87a62da881 New workflow for github action 2026-06-07 08:54:17 +10:00
Mike Farah
ef507264e1 New workflow for github action 2026-06-07 08:50:24 +10:00
Mike Farah
2a40eb3d04 Bumping github action docker file 2026-06-07 08:45:27 +10:00
Mike Farah
e3cb1dc7c6 Fixing build 2026-06-07 07:03:26 +10:00
Mike Farah
1b9b4ac518 Bumping version 2026-06-06 20:59:47 +10:00
Mike Farah
9b67d655f1 Portable bump version 2026-06-06 20:59:43 +10:00
Mike Farah
f8850c043c Preparting release 2026-06-06 20:57:28 +10:00
Mike Farah
c5a342359d Refining 2026-06-06 19:14:59 +10:00
Mike Farah
196a99e912 Agent disclosure 2026-06-06 19:13:33 +10:00
Mike Farah
fa99bf12c3 Merging agents 2026-06-06 18:38:30 +10:00
Mike Farah
71117613d6
docs: add AGENTS.md with Cursor Cloud development instructions (#2735)
* docs: add AGENTS.md with Cursor Cloud development instructions

Co-authored-by: Mike Farah <mikefarah@users.noreply.github.com>

* docs: fix AGENTS.md spelling and note spellcheck before PR

Co-authored-by: Mike Farah <mikefarah@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mike Farah <mikefarah@users.noreply.github.com>
2026-06-06 18:25:36 +10:00
Mike Farah
0cb1bbe698 Fixing test 2026-06-06 18:25:12 +10:00
Mike Farah
8fc8eedd3b Minor doc update 2026-06-06 18:23:24 +10:00
Xie Benyi
7620a727b5
fix: reset INI decoder state on init (#2719) 2026-06-06 16:21:13 +10:00
Tony
6f9201a0ca
feat: add --ini-preserve-quotes flag for INI round-trip quote preservation (#2728)
Add INIPreferences.PreserveSurroundedQuote option that wires through to
go-ini/ini's LoadOptions.PreserveSurroundedQuote. When enabled, existing
surrounding quotes on INI values are preserved during decode/encode
round-trips.

Fixes #2456

Co-authored-by: toller892 <toller892@gmail.com>
2026-06-06 16:20:29 +10:00
dependabot[bot]
5b19ddbcd0
Bump docker/setup-buildx-action from 4.0.0 to 4.1.0 (#2724)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](4d04d5d948...d7f5e7f509)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-06 16:02:50 +10:00
dependabot[bot]
1ac2a3d96a
Bump docker/setup-qemu-action from 4.0.0 to 4.1.0 (#2726)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](ce360397dd...06116385d9)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-06 16:00:51 +10:00
dependabot[bot]
4a12c3f908
Bump golang.org/x/net from 0.54.0 to 0.55.0 (#2725)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.54.0 to 0.55.0.
- [Commits](https://github.com/golang/net/compare/v0.54.0...v0.55.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.55.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-06 16:00:38 +10:00
dependabot[bot]
41377138ef
Bump docker/login-action from 4.1.0 to 4.2.0 (#2727)
Bumps [docker/login-action](https://github.com/docker/login-action) from 4.1.0 to 4.2.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](4907a6ddec...650006c6eb)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-06 15:53:24 +10:00
dependabot[bot]
ae5cf9ff04
Bump golang from 1.26.3 to 1.26.4 (#2731)
Bumps golang from 1.26.3 to 1.26.4.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.26.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-06 15:53:06 +10:00
dependabot[bot]
fe449b956a
Bump actions/checkout from 6.0.2 to 6.0.3 (#2732)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](de0fac2e45...df4cb1c069)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-06 15:52:53 +10:00
Rayan Salhab
8f3291d316
fix: decode properties array bracket paths (#2693)
* fix: decode properties array bracket paths

* test: add nested array bracket properties decode case

---------

Co-authored-by: cyphercodes <cyphercodes@users.noreply.github.com>
2026-05-15 22:22:06 +10:00
ChrisJr404
2861815f71
fix(json): preserve floats with trailing zero when encoding YAML to JSON (#2701)
YAML scalars tagged `!!float` were round-tripped through `float64` and
re-serialized by Go's JSON encoder, which strips the decimal part of
whole-number floats. As a result, `50.0` came out as `50` and a
sequence like `[50.0, 95.0, 99.0, 99.9]` became `[50,95,99,99.9]`,
turning a uniform array of floats into a mixed int/float array that
downstream consumers (Horreum, JSON Schema validators, jq, etc.)
reject.

The JSON spec does not distinguish ints from floats, but every common
JSON library (Go's `encoding/json`, Python's `json`, jq) preserves the
fractional form of values that came in as floats. yq's YAML decoder
already parses these as `!!float` with the original text intact, so we
can emit them verbatim instead of round-tripping.

`MarshalJSON` for `ScalarNode` now special-cases `!!float`:
- if `Value` is already a JSON-shaped number literal containing a `.`
  or exponent, emit it verbatim (e.g. `50.0`, `99.9`, `1.5e-3`, `-7.0`);
- if `Value` is an integer-shaped string tagged `!!float` (e.g.
  `!!float 5`), format the parsed float and append `.0` so it stays a
  JSON number with a fractional part;
- otherwise (empty value, parse error, or non-finite result), fall back
  to the existing encoding path so behaviour for `.inf` / `.nan` and
  anything unusual is unchanged.

`!!int` nodes still encode as JSON integers.

Closes #2683

Signed-off-by: ChrisJr404 <chris@hacknow.com>
2026-05-14 20:00:34 +10:00
梦曦·花已落
fcb79822dd
feat(toml): fix JSON to TOML root scope and null handling (#2689)
Ensure root-level TOML attributes are emitted before table sections so fields like sort remain root-scoped. Skip null-valued object fields during TOML encoding
    instead of converting them to empty strings.
2026-05-14 19:57:42 +10:00
dependabot[bot]
e9acb9b734
Bump golang.org/x/mod from 0.35.0 to 0.36.0 (#2709)
Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.35.0 to 0.36.0.
- [Commits](https://github.com/golang/mod/compare/v0.35.0...v0.36.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-14 19:56:57 +10:00
dependabot[bot]
83b282c413
Bump golang.org/x/net from 0.53.0 to 0.54.0 (#2707)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.53.0 to 0.54.0.
- [Commits](https://github.com/golang/net/compare/v0.53.0...v0.54.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.54.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-14 19:38:12 +10:00
dependabot[bot]
54fa4324ea
Bump golang from 1.26.2 to 1.26.3 (#2706)
Bumps golang from 1.26.2 to 1.26.3.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.26.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-14 19:38:00 +10:00
Terminal Chai
ee6c30dac2
fix: reset TOML decoder finished flag on Init to fix multi-doc evaluation (#2704)
* fix: reset TOML decoder between files

* test: fix TOML regression fixture spelling
2026-05-14 19:37:43 +10:00
Rayan Salhab
722c9aa16c
Fix nested inline YAML merge explode (#2699)
Co-authored-by: cyphercodes <cyphercodes@users.noreply.github.com>
2026-05-14 19:33:50 +10:00
dependabot[bot]
702dd16048
Bump github.com/pelletier/go-toml/v2 from 2.3.0 to 2.3.1 (#2695)
Bumps [github.com/pelletier/go-toml/v2](https://github.com/pelletier/go-toml) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/pelletier/go-toml/releases)
- [Commits](https://github.com/pelletier/go-toml/compare/v2.3.0...v2.3.1)

---
updated-dependencies:
- dependency-name: github.com/pelletier/go-toml/v2
  dependency-version: 2.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-14 19:31:08 +10:00
Rayan Salhab
d1dff4661b
fix: preserve TOML inline table array scope (#2694)
Co-authored-by: cyphercodes <cyphercodes@users.noreply.github.com>
2026-05-14 19:30:52 +10:00
Copilot
cb97935554
fix: TOML encoder uses inline tables for YAML FlowStyle mappings, inconsistent with explicit JSON parsing (#2687)
* Initial plan

* fix: TOML encoder no longer treats YAML FlowStyle as inline tables

Remove FlowStyle checks from the TOML encoder. YAML flow-style mappings
are a YAML-specific rendering hint and should not influence TOML output.
Only nodes explicitly marked with EncodeHintInline (set by the TOML
decoder for actual TOML inline tables) will produce TOML inline table
syntax.

This fixes the bug where JSON auto-detected via the YAML parser (which
parses {} as flow-style mappings) would produce inline TOML tables
instead of readable table sections, while explicitly parsing with
-p json produced correct table sections.

Updated tests: YAML flow mappings now produce table sections (same as
block mappings), consistent with the fix. Added new test cases for the
JSON → TOML conversion via both YAML decoder (auto-detection) and JSON
decoder.

Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/3e504870-b585-4998-af9c-a451e2f6a6a3

Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
2026-04-28 19:32:07 +10:00
Rayan Salhab
cfe2eee7e6
Preserve empty TOML arrays in tables (#2686)
Co-authored-by: cyphercodes <cyphercodes@users.noreply.github.com>
2026-04-27 19:12:30 +10:00
dependabot[bot]
1a433d1035
Bump actions/upload-artifact from 4.6.1 to 7.0.1 (#2663)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.6.1 to 7.0.1.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](4cec3d8aa0...043fb46d1a)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-26 09:40:39 +10:00
dependabot[bot]
1c0d8b9da9
Bump actions/checkout from 4.2.2 to 6.0.2 (#2668)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 6.0.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4.2.2...de0fac2e4500dabe0009e67214ff5f5447ce83dd)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 6.0.2
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-26 09:40:25 +10:00
dependabot[bot]
0110a3cea8
Bump golang.org/x/net from 0.52.0 to 0.53.0 (#2669)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.52.0 to 0.53.0.
- [Commits](https://github.com/golang/net/compare/v0.52.0...v0.53.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.53.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-26 09:39:45 +10:00
dependabot[bot]
54482d44b3
Bump golang from 2a2b4b5 to 5f3787b (#2664)
Bumps golang from `2a2b4b5` to `5f3787b`.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.26.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-26 09:19:59 +10:00
dependabot[bot]
33f3351c01
Bump ossf/scorecard-action from 2.4.1 to 2.4.3 (#2665)
Bumps [ossf/scorecard-action](https://github.com/ossf/scorecard-action) from 2.4.1 to 2.4.3.
- [Release notes](https://github.com/ossf/scorecard-action/releases)
- [Changelog](https://github.com/ossf/scorecard-action/blob/main/RELEASE.md)
- [Commits](f49aabe0b5...4eaacf0543)

---
updated-dependencies:
- dependency-name: ossf/scorecard-action
  dependency-version: 2.4.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-26 09:19:38 +10:00
dependabot[bot]
6cb656ced0
Bump alpine from 2510918 to 5b10f43 (#2667)
Bumps alpine from `2510918` to `5b10f43`.

---
updated-dependencies:
- dependency-name: alpine
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-26 09:19:22 +10:00
Terminal Chai
ecc43d7c9e
fix: reset TOML decoder between files when evaluating all at once (#2685)
* fix: reset TOML decoder between files

* test: fix TOML regression fixture spelling
2026-04-26 09:18:45 +10:00
Jan Dubois
1deec5e450
Fix repeatString overflow test on 32-bit platforms (#2680)
The test literal "ab" * 4611686018427387904 (2^62) exceeds MaxInt32,
so parseInt rejects it before the size guard runs. Compute the count
with 1 << (bits.UintSize - 2) to yield 2^30 on 32-bit and 2^62 on
64-bit. Both values, when doubled by len("ab"), wrap past MaxInt and
bypass a naive len*count guard, exercising the division-safe check
added in #2644.

Signed-off-by: Jan Dubois <jan@jandubois.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-26 09:18:34 +10:00
dependabot[bot]
ff45fad14c
Bump github.com/zclconf/go-cty from 1.18.0 to 1.18.1 (#2682)
Bumps [github.com/zclconf/go-cty](https://github.com/zclconf/go-cty) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/zclconf/go-cty/releases)
- [Changelog](https://github.com/zclconf/go-cty/blob/main/CHANGELOG.md)
- [Commits](https://github.com/zclconf/go-cty/compare/v1.18.0...v1.18.1)

---
updated-dependencies:
- dependency-name: github.com/zclconf/go-cty
  dependency-version: 1.18.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-26 09:17:34 +10:00
dependabot[bot]
6679d3c02b
Bump github/codeql-action from 3 to 4 (#2671)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3 to 4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Commits](https://github.com/github/codeql-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-22 21:04:18 +10:00
dependabot[bot]
54a7fc8f0c
Bump softprops/action-gh-release from 2.6.2 to 3.0.0 (#2672)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2.6.2 to 3.0.0.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](3bb12739c2...b430933298)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: 3.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-22 21:03:55 +10:00
dependabot[bot]
0d3ab07928
Bump golang.org/x/text from 0.35.0 to 0.36.0 (#2670)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.35.0 to 0.36.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.35.0...v0.36.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-version: 0.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-22 21:02:08 +10:00
Mike Farah
d93987a93a
release notes 2026-04-17 16:04:18 +10:00
Mike Farah
751d8ad57b
Bumping version 2026-04-17 16:03:30 +10:00
Mike Farah
6dd681a7c0
Fixing release signing 2026-04-17 16:03:18 +10:00
Mike Farah
fc7c337d8f
Updating bump version script 2026-04-17 15:36:35 +10:00
Mike Farah
e969dd789f
Bumping version 2026-04-17 15:36:22 +10:00
Mike Farah
dc4b4ea1df
Preparing release notes 2026-04-17 15:31:40 +10:00
Mike Farah
602586d8fd
Create scorecard.yml
Signed-off-by: Mike Farah <mikefarah@gmail.com>
2026-04-14 18:43:11 +10:00
Copilot
9a0335abb2
fix: restrict GitHub Actions workflow token permissions (OSSF least-privilege) (#2662)
* Initial plan

* fix: add least-privilege token permissions to GitHub workflows (OSSF)

Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/1b5db5e2-af78-4289-a6e0-2e972fc68ef1

Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
2026-04-13 19:11:10 +10:00
Mike Farah
838c51691c
Trying to test release 2026-04-12 19:54:34 +10:00
69 changed files with 1237 additions and 588 deletions

View File

@ -20,6 +20,8 @@ on:
schedule: schedule:
- cron: '24 3 * * 1' - cron: '24 3 * * 1'
permissions: {}
jobs: jobs:
analyze: analyze:
name: Analyze name: Analyze
@ -38,11 +40,11 @@ jobs:
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
# Initializes the CodeQL tools for scanning. # Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@c10b8064de6f491fea524254123dbe5e09572f13 # v4.35.1 uses: github/codeql-action/init@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2
with: with:
languages: ${{ matrix.language }} languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file. # If you wish to specify custom queries, you can do so here or in a config file.
@ -53,7 +55,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java). # Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below) # If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild - name: Autobuild
uses: github/codeql-action/autobuild@c10b8064de6f491fea524254123dbe5e09572f13 # v4.35.1 uses: github/codeql-action/autobuild@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2
# Command-line programs to run using the OS shell. # Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@ -67,4 +69,4 @@ jobs:
# make release # make release
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@c10b8064de6f491fea524254123dbe5e09572f13 # v4.35.1 uses: github/codeql-action/analyze@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2

View File

@ -0,0 +1,107 @@
name: Release Docker GitHub Action
on:
workflow_dispatch:
permissions: {}
jobs:
publishGithubActionDocker:
environment: dockerhub
env:
IMAGE_NAME: mikefarah/yq
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Set up QEMU
uses: docker/setup-qemu-action@06116385d9baf250c9f4dcb4858b16962ea869c3 # v4.1.0
with:
platforms: all
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
with:
version: latest
- name: Verify Dockerfile base image digest matches yq:4
run: |
PINNED_DIGEST=$(grep -oE 'sha256:[a-f0-9]{64}' github-action/Dockerfile | head -1)
if [ -z "${PINNED_DIGEST}" ]; then
echo "::error::Could not find a sha256 digest in github-action/Dockerfile"
exit 1
fi
LATEST_DIGEST=$(docker buildx imagetools inspect "${IMAGE_NAME}:4" --format '{{printf "%s" .Manifest.Digest}}')
echo "Dockerfile pins: ${PINNED_DIGEST}"
echo "mikefarah/yq:4 is: ${LATEST_DIGEST}"
if [ "${PINNED_DIGEST}" != "${LATEST_DIGEST}" ]; then
echo "::error::github-action/Dockerfile digest does not match the current mikefarah/yq:4 image"
echo "Update the FROM line in github-action/Dockerfile to:"
echo " FROM mikefarah/yq:4@${LATEST_DIGEST}"
exit 1
fi
- name: Resolve version from yq:4
run: |
IMAGE_VERSION=$(docker run --rm "${IMAGE_NAME}:4" --version | awk '{print $NF}' | sed 's/^v//')
if [ -z "${IMAGE_VERSION}" ]; then
echo "::error::Could not determine yq version from ${IMAGE_NAME}:4"
exit 1
fi
echo "Resolved yq version: ${IMAGE_VERSION}"
echo "IMAGE_VERSION=${IMAGE_VERSION}" >> "${GITHUB_ENV}"
- name: Login to Docker Hub
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GitHub Container Registry
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push github-action image
working-directory: github-action
run: |
PLATFORMS="linux/amd64,linux/ppc64le,linux/arm64,linux/arm/v7,linux/s390x"
echo "Building and pushing github-action image for version ${IMAGE_VERSION}"
docker buildx build \
--label "org.opencontainers.image.authors=https://github.com/mikefarah/yq/graphs/contributors" \
--label "org.opencontainers.image.created=$(date --rfc-3339=seconds)" \
--label "org.opencontainers.image.description=yq is a portable command-line data file processor" \
--label "org.opencontainers.image.documentation=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.licenses=MIT" \
--label "org.opencontainers.image.revision=$(git rev-parse HEAD)" \
--label "org.opencontainers.image.source=https://github.com/mikefarah/yq" \
--label "org.opencontainers.image.title=yq" \
--label "org.opencontainers.image.url=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.version=${IMAGE_VERSION}" \
--platform "${PLATFORMS}" \
--pull \
--push \
-t "${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "${IMAGE_NAME}:4-githubaction" \
-t "${IMAGE_NAME}:latest-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:4-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:latest-githubaction" \
.
- name: Report action.yml digest to pin
run: |
GITHUBACTION_DIGEST=$(docker buildx imagetools inspect "${IMAGE_NAME}:4-githubaction" --format '{{printf "%s" .Manifest.Digest}}')
echo "Published ${IMAGE_NAME}:4-githubaction at ${GITHUBACTION_DIGEST}"
echo "Update action.yml image to:"
echo " docker://${IMAGE_NAME}:4-githubaction@${GITHUBACTION_DIGEST}"

View File

@ -7,23 +7,28 @@ on:
# Allows you to run this workflow manually from the Actions tab # Allows you to run this workflow manually from the Actions tab
workflow_dispatch: workflow_dispatch:
permissions: {}
jobs: jobs:
publishDocker: publishDocker:
environment: dockerhub environment: dockerhub
env: env:
IMAGE_NAME: mikefarah/yq IMAGE_NAME: mikefarah/yq
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0 uses: docker/setup-qemu-action@06116385d9baf250c9f4dcb4858b16962ea869c3 # v4.1.0
with: with:
platforms: all platforms: all
- name: Set up Docker Buildx - name: Set up Docker Buildx
id: buildx id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0 uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
with: with:
version: latest version: latest
@ -31,13 +36,13 @@ jobs:
run: echo ${{ steps.buildx.outputs.platforms }} && docker version run: echo ${{ steps.buildx.outputs.platforms }} && docker version
- name: Login to Docker Hub - name: Login to Docker Hub
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0 uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with: with:
username: ${{ secrets.DOCKER_USERNAME }} username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }} password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0 uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ github.actor }} username: ${{ github.actor }}
@ -75,26 +80,3 @@ jobs:
-t "ghcr.io/${IMAGE_NAME}:4" \ -t "ghcr.io/${IMAGE_NAME}:4" \
-t "ghcr.io/${IMAGE_NAME}:latest" \ -t "ghcr.io/${IMAGE_NAME}:latest" \
. .
cd github-action
docker buildx build \
--label "org.opencontainers.image.authors=https://github.com/mikefarah/yq/graphs/contributors" \
--label "org.opencontainers.image.created=$(date --rfc-3339=seconds)" \
--label "org.opencontainers.image.description=yq is a portable command-line data file processor" \
--label "org.opencontainers.image.documentation=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.licenses=MIT" \
--label "org.opencontainers.image.revision=$(git rev-parse HEAD)" \
--label "org.opencontainers.image.source=https://github.com/mikefarah/yq" \
--label "org.opencontainers.image.title=yq" \
--label "org.opencontainers.image.url=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.version=${IMAGE_VERSION}" \
--platform "${PLATFORMS}" \
--pull \
--push \
-t "${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "${IMAGE_NAME}:4-githubaction" \
-t "${IMAGE_NAME}:latest-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:4-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:latest-githubaction" \
.

View File

@ -5,25 +5,51 @@ permissions:
jobs: jobs:
verify-action-digest:
name: Verify action.yml image digest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Verify action.yml digest matches published image
run: |
PINNED_DIGEST=$(grep -oE 'sha256:[a-f0-9]{64}' action.yml | head -1)
if [ -z "${PINNED_DIGEST}" ]; then
echo "::error::action.yml does not pin the runtime image by digest"
exit 1
fi
LATEST_DIGEST=$(docker buildx imagetools inspect docker.io/mikefarah/yq:4-githubaction --format '{{printf "%s" .Manifest.Digest}}')
echo "action.yml pins: ${PINNED_DIGEST}"
echo "mikefarah/yq:4-githubaction: ${LATEST_DIGEST}"
if [ "${PINNED_DIGEST}" != "${LATEST_DIGEST}" ]; then
echo "::error::action.yml digest does not match the current mikefarah/yq:4-githubaction image"
echo "Update the image line in action.yml to:"
echo " docker://mikefarah/yq:4-githubaction@${LATEST_DIGEST}"
exit 1
fi
build: build:
name: Build name: Build
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
with: with:
go-version: '^1.20' go-version: '^1.20'
id: go id: go
- name: Check out code into the Go module directory - name: Check out code into the Go module directory
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Get dependencies - name: Get dependencies
run: | run: |
go get -v -t -d ./... go get -v -t -d ./...
if [ -f Gopkg.toml ]; then if [ -f Gopkg.toml ]; then
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh curl -sSfL https://raw.githubusercontent.com/golang/dep/1f7c19e5f52f49ffb9f956f64c010be14683468b/install.sh | env DEP_RELEASE_TAG=v0.5.4 sh
dep ensure dep ensure
fi fi

View File

@ -5,6 +5,8 @@ on:
- 'v4.*' - 'v4.*'
- 'draft-*' - 'draft-*'
permissions: {}
jobs: jobs:
publishGitRelease: publishGitRelease:
runs-on: ubuntu-latest runs-on: ubuntu-latest
@ -12,8 +14,8 @@ jobs:
contents: write contents: write
id-token: write id-token: write
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0 - uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
with: with:
go-version: '^1.20' go-version: '^1.20'
check-latest: true check-latest: true
@ -27,7 +29,7 @@ jobs:
run: echo "VERSION=${GITHUB_REF##*/}" >> "${GITHUB_OUTPUT}" run: echo "VERSION=${GITHUB_REF##*/}" >> "${GITHUB_OUTPUT}"
- name: Generate man page - name: Generate man page
uses: docker://pandoc/core:2.14.2 uses: docker://pandoc/core:2.14.2@sha256:04e127c6642a2b9d447c26fe0ac6a5932efa8f508eda9f07da51b6e621dd7c19
id: gen-man-page id: gen-man-page
with: with:
args: >- args: >-
@ -41,21 +43,21 @@ jobs:
man.md man.md
- name: Install cosign - name: Install cosign
uses: sigstore/cosign-installer@v3 uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
- name: Cross compile - name: Cross compile
run: | run: |
sudo apt-get install rhash -y sudo apt-get install rhash -y
go install github.com/goreleaser/goreleaser/v2@latest go install github.com/goreleaser/goreleaser/v2@v2.16.0
./scripts/xcompile.sh ./scripts/xcompile.sh
- name: Sign checksums - name: Sign checksums
run: | run: |
cosign sign-blob --yes --output-bundle build/checksums.bundle build/checksums cosign sign-blob --yes --bundle build/checksums.bundle build/checksums
cosign sign-blob --yes --output-bundle build/checksums-bsd.bundle build/checksums-bsd cosign sign-blob --yes --bundle build/checksums-bsd.bundle build/checksums-bsd
- name: Release - name: Release
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2.6.2 uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with: with:
files: build/* files: build/*
draft: true draft: true

78
.github/workflows/scorecard.yml vendored Normal file
View File

@ -0,0 +1,78 @@
# This workflow uses actions that are not certified by GitHub. They are provided
# by a third-party and are governed by separate terms of service, privacy
# policy, and support documentation.
name: Scorecard supply-chain security
on:
# For Branch-Protection check. Only the default branch is supported. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection
branch_protection_rule:
# To guarantee Maintained check is occasionally updated. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained
schedule:
- cron: '39 7 * * 2'
push:
branches: [ "master" ]
# Declare default permissions as read only.
permissions: read-all
jobs:
analysis:
name: Scorecard analysis
runs-on: ubuntu-latest
# `publish_results: true` only works when run from the default branch. conditional can be removed if disabled.
if: github.event.repository.default_branch == github.ref_name || github.event_name == 'pull_request'
permissions:
# Needed to upload the results to code-scanning dashboard.
security-events: write
# Needed to publish results and get a badge (see publish_results below).
id-token: write
# Uncomment the permissions below if installing in a private repository.
# contents: read
# actions: read
steps:
- name: "Checkout code"
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: "Run analysis"
uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a # v2.4.3
with:
results_file: results.sarif
results_format: sarif
# (Optional) "write" PAT token. Uncomment the `repo_token` line below if:
# - you want to enable the Branch-Protection check on a *public* repository, or
# - you are installing Scorecard on a *private* repository
# To create the PAT, follow the steps in https://github.com/ossf/scorecard-action?tab=readme-ov-file#authentication-with-fine-grained-pat-optional.
# repo_token: ${{ secrets.SCORECARD_TOKEN }}
# Public repositories:
# - Publish results to OpenSSF REST API for easy access by consumers
# - Allows the repository to include the Scorecard badge.
# - See https://github.com/ossf/scorecard-action#publishing-results.
# For private repositories:
# - `publish_results` will always be set to `false`, regardless
# of the value entered here.
publish_results: true
# (Optional) Uncomment file_mode if you have a .gitattributes with files marked export-ignore
# file_mode: git
# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# format to the repository Actions tab.
- name: "Upload artifact"
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: SARIF file
path: results.sarif
retention-days: 5
# Upload the results to GitHub's code scanning dashboard (optional).
# Commenting out will disable upload of results to your repo's Code Scanning dashboard
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@8aad20d150bbac5944a9f9d289da16a4b0d87c1e # v4.36.2
with:
sarif_file: results.sarif

View File

@ -7,12 +7,16 @@ on:
# Allows you to run this workflow manually from the Actions tab # Allows you to run this workflow manually from the Actions tab
workflow_dispatch: workflow_dispatch:
permissions: {}
jobs: jobs:
buildSnap: buildSnap:
environment: snap environment: snap
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions:
contents: read
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: snapcore/action-build@3bdaa03e1ba6bf59a65f84a751d943d549a54e79 # v1.3.0 - uses: snapcore/action-build@3bdaa03e1ba6bf59a65f84a751d943d549a54e79 # v1.3.0
id: build id: build
env: env:

View File

@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Get test - name: Get test
id: get_value id: get_value
uses: mikefarah/yq@master uses: mikefarah/yq@master

View File

@ -1,3 +1,79 @@
# yq — agent instructions
## ⚠️ MANDATORY: GitHub agent disclosure
**Always required. No exceptions.**
Whenever you perform **any** GitHub action on behalf of the user, you **must** disclose that an AI agent (Cursor) wrote the content and is acting on the user's behalf — **not the user personally**. Do this **before** submitting; never post first and add the disclosure later.
Applies to **all** GitHub interactions, including:
- Pull requests (titles, descriptions, and reviews)
- PR comments and inline review comments
- Issues (new issues, comments, and updates)
- Any other post or reply on GitHub
**How to disclose:** Put it prominently at the **top** of every PR description, review body, comment, or issue. Use wording like:
Inline review comments must include a short disclosure too (e.g. `> Generated by Cursor acting on the user's behalf, not the user personally.`).
**Never** submit a GitHub action without this disclosure.
---
Always run the spellcheck before raising a PR:
```bash
bash scripts/spelling.sh
```
This is also included in the full CI pipeline via `make local test`.
## Cursor Cloud specific instructions
### Overview
**yq** is a Go CLI for querying and transforming YAML, JSON, XML, INI, and other structured formats. There are no long-running services — development is build-and-test against a local `./yq` binary.
### Prerequisites
- **Go ≥ 1.25** (see `go.mod`)
- **Bash** (acceptance tests)
- **Docker/Podman** is optional; use `make local <target>` to run natively when containers are unavailable
### PATH
After `scripts/devtools.sh`, add Go tool binaries to PATH:
```bash
export PATH="$HOME/go/bin:$PATH"
```
`golangci-lint` and `typos` install to `$HOME/go/bin`; `gosec` installs to `./bin/gosec` in the repo root.
### Common commands (local, no Docker)
| Task | Command |
|------|---------|
| Install dev tools | `bash scripts/devtools.sh` |
| Vendor dependencies | `make local vendor` |
| Build binary | `go build -o yq .` or `make local build` |
| Format | `make local format` |
| Lint | `make local check` |
| Unit tests | `make local test` or `bash scripts/test.sh` |
| Acceptance (E2E) | `bash scripts/acceptance.sh` (requires `./yq` built first) |
`make local build` runs the full CI chain (format → spelling → gosec → lint → unit tests → build → acceptance). For a faster loop, build with `go build -o yq .` and run `bash scripts/acceptance.sh`.
### Caveats
- **`make` without `local`** tries Docker/Podman (`Dockerfile.dev`). In Cloud Agent VMs without Docker, always prefix with `make local`.
- **Spelling step** uses `typos` (installed by `scripts/devtools.sh`).
- **`make local test` / `scripts/check.sh`** require `golangci-lint` on PATH (`devtools.sh`).
---
# General rules # General rules
✅ **DO:** ✅ **DO:**
- You can use ./yq with the `--debug-node-info` flag to get a deeper understanding of the ast. - You can use ./yq with the `--debug-node-info` flag to get a deeper understanding of the ast.
@ -5,10 +81,12 @@
- Add comprehensive tests to cover the changes - Add comprehensive tests to cover the changes
- Run test suite to ensure there is no regression - Run test suite to ensure there is no regression
- Use UK english spelling - Use UK english spelling
- **Follow the mandatory GitHub agent disclosure rule above** on every GitHub action — no exceptions
❌ **DON'T:** ❌ **DON'T:**
- Git add or commit - Git add or commit
- Add comments to functions that are self-explanatory - Add comments to functions that are self-explanatory
- **Post to GitHub without the mandatory agent disclosure** (PRs, reviews, comments, issues, or any other GitHub interaction)

View File

@ -1,4 +1,4 @@
FROM golang:1.26.2@sha256:2a2b4b5791cea8ae09caecba7bad0bd9631def96e5fe362e4a5e67009fe4ae61 AS builder FROM golang:1.26.4@sha256:792443b89f65105abba56b9bd5e97f680a80074ac62fc844a584212f8c8102c3 AS builder
WORKDIR /go/src/mikefarah/yq WORKDIR /go/src/mikefarah/yq
@ -10,7 +10,7 @@ RUN ./scripts/acceptance.sh
# Choose alpine as a base image to make this useful for CI, as many # Choose alpine as a base image to make this useful for CI, as many
# CI tools expect an interactive shell inside the container # CI tools expect an interactive shell inside the container
FROM alpine:3@sha256:25109184c71bdad752c8312a8623239686a9a2071e8825f20acb8f2198c3f659 AS production FROM alpine:3@sha256:28bd5fe8b56d1bd048e5babf5b10710ebe0bae67db86916198a6eec434943f8b AS production
LABEL maintainer="Mike Farah <mikefarah@users.noreply.github.com>" LABEL maintainer="Mike Farah <mikefarah@users.noreply.github.com>"
COPY --from=builder /go/src/mikefarah/yq/yq /usr/bin/yq COPY --from=builder /go/src/mikefarah/yq/yq /usr/bin/yq

View File

@ -1,8 +1,4 @@
FROM golang:1.26.2@sha256:2a2b4b5791cea8ae09caecba7bad0bd9631def96e5fe362e4a5e67009fe4ae61 FROM golang:1.26.4@sha256:792443b89f65105abba56b9bd5e97f680a80074ac62fc844a584212f8c8102c3
RUN apt-get update && \
apt-get install -y npm && \
npm install -g npx cspell@latest
COPY scripts/devtools.sh /opt/devtools.sh COPY scripts/devtools.sh /opt/devtools.sh

View File

@ -42,7 +42,7 @@ quiet: # this is silly but shuts up 'Nothing to be done for `local`'
@: @:
prepare: tmp/dev_image_id prepare: tmp/dev_image_id
tmp/dev_image_id: Dockerfile.dev scripts/devtools.sh tmp/dev_image_id: Dockerfile.dev scripts/devtools.sh _typos.toml
@mkdir -p tmp @mkdir -p tmp
@${ENGINE} rmi -f ${DEV_IMAGE} > /dev/null 2>&1 || true @${ENGINE} rmi -f ${DEV_IMAGE} > /dev/null 2>&1 || true
@${ENGINE} build -t ${DEV_IMAGE} -f Dockerfile.dev . @${ENGINE} build -t ${DEV_IMAGE} -f Dockerfile.dev .

20
_typos.toml Normal file
View File

@ -0,0 +1,20 @@
[files]
extend-exclude = ["vendor", "bin"]
[default]
locale = "en"
extend-ignore-identifiers-re = [
"NdJson",
]
[default.extend-identifiers]
AttributeIDSupressMenu = "AttributeIDSupressMenu"
[default.extend-words]
Teh = "Teh"
teh = "teh"
Supress = "Supress"
HashiCorp = "HashiCorp"
Hashi = "Hashi"
fot = "fot"
nd = "nd"

View File

@ -12,6 +12,6 @@ outputs:
description: "The complete result from the yq command being run" description: "The complete result from the yq command being run"
runs: runs:
using: 'docker' using: 'docker'
image: 'docker://mikefarah/yq:4-githubaction' image: 'docker://mikefarah/yq:4-githubaction@sha256:e1b8c865f299ea6b02910a7ddf147d5d431244d4cc116f89c2148c9f53822906'
args: args:
- ${{ inputs.cmd }} - ${{ inputs.cmd }}

View File

@ -156,6 +156,8 @@ yq -P -oy sample.json
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredLuaPreferences.UnquotedKeys, "lua-unquoted", yqlib.ConfiguredLuaPreferences.UnquotedKeys, "output unquoted string keys (e.g. {foo=\"bar\"})") rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredLuaPreferences.UnquotedKeys, "lua-unquoted", yqlib.ConfiguredLuaPreferences.UnquotedKeys, "output unquoted string keys (e.g. {foo=\"bar\"})")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredLuaPreferences.Globals, "lua-globals", yqlib.ConfiguredLuaPreferences.Globals, "output keys as top-level global variables") rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredLuaPreferences.Globals, "lua-globals", yqlib.ConfiguredLuaPreferences.Globals, "output keys as top-level global variables")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredINIPreferences.PreserveSurroundedQuote, "ini-preserve-quotes", yqlib.ConfiguredINIPreferences.PreserveSurroundedQuote, "preserve surrounding quotes on INI values during round-trip")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator, "properties-separator", yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator, "separator to use between keys and values") rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator, "properties-separator", yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator, "separator to use between keys and values")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets, "properties-array-brackets", yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets, "use [x] in array paths (e.g. for SpringBoot)") rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets, "properties-array-brackets", yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets, "use [x] in array paths (e.g. for SpringBoot)")

View File

@ -11,7 +11,7 @@ var (
GitDescribe string GitDescribe string
// Version is main version number that is being run at the moment. // Version is main version number that is being run at the moment.
Version = "v4.52.5" Version = "v4.53.3"
// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string) // VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
// then it means that it is a final release. Otherwise, this is a pre-release // then it means that it is a final release. Otherwise, this is a pre-release

View File

@ -1,14 +0,0 @@
---
$schema: https://raw.githubusercontent.com/streetsidesoftware/cspell/main/cspell.schema.json
version: '0.2'
language: en-GB
dictionaryDefinitions:
- name: project-words
path: './project-words.txt'
addWords: true
dictionaries:
- project-words
ignorePaths:
- 'vendor'
- 'bin'
- '/project-words.txt'

View File

@ -1,4 +1,4 @@
FROM mikefarah/yq:4@sha256:603ebff15eb308a05f1c5b8b7613179cad859aed3ec9fdd04f2ef5d32345950e FROM mikefarah/yq:4@sha256:11a1f0b604b13dbbdc662260d8db6f644b22d8553122a25c1b5b2e8713ca6977
COPY entrypoint.sh /entrypoint.sh COPY entrypoint.sh /entrypoint.sh

18
go.mod
View File

@ -13,16 +13,16 @@ require (
github.com/hashicorp/hcl/v2 v2.24.0 github.com/hashicorp/hcl/v2 v2.24.0
github.com/jinzhu/copier v0.4.0 github.com/jinzhu/copier v0.4.0
github.com/magiconair/properties v1.8.10 github.com/magiconair/properties v1.8.10
github.com/pelletier/go-toml/v2 v2.3.0 github.com/pelletier/go-toml/v2 v2.4.2
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e
github.com/spf13/cobra v1.10.2 github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10 github.com/spf13/pflag v1.0.10
github.com/yuin/gopher-lua v1.1.2 github.com/yuin/gopher-lua v1.1.2
github.com/zclconf/go-cty v1.18.0 github.com/zclconf/go-cty v1.18.1
go.yaml.in/yaml/v4 v4.0.0-rc.4 go.yaml.in/yaml/v4 v4.0.0-rc.6
golang.org/x/mod v0.34.0 golang.org/x/mod v0.37.0
golang.org/x/net v0.52.0 golang.org/x/net v0.56.0
golang.org/x/text v0.35.0 golang.org/x/text v0.38.0
) )
require ( require (
@ -33,9 +33,9 @@ require (
github.com/mattn/go-colorable v0.1.14 // indirect github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect github.com/mitchellh/go-wordwrap v1.0.1 // indirect
golang.org/x/sync v0.20.0 // indirect golang.org/x/sync v0.21.0 // indirect
golang.org/x/sys v0.42.0 // indirect golang.org/x/sys v0.46.0 // indirect
golang.org/x/tools v0.42.0 // indirect golang.org/x/tools v0.45.0 // indirect
) )
go 1.25.0 go 1.25.0

36
go.sum
View File

@ -46,8 +46,8 @@ github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWE
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0= github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0=
github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0= github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
github.com/pelletier/go-toml/v2 v2.3.0 h1:k59bC/lIZREW0/iVaQR8nDHxVq8OVlIzYCOJf421CaM= github.com/pelletier/go-toml/v2 v2.4.2 h1:M2fKKbmyvI+hGId/D0W64qDBMVhJnNR10O5gIbMc//Q=
github.com/pelletier/go-toml/v2 v2.3.0/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= github.com/pelletier/go-toml/v2 v2.4.2/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e h1:aoZm08cpOy4WuID//EZDgcC4zIxODThtZNPirFr42+A= github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e h1:aoZm08cpOy4WuID//EZDgcC4zIxODThtZNPirFr42+A=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@ -63,26 +63,26 @@ github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5Cc
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/yuin/gopher-lua v1.1.2 h1:yF/FjE3hD65tBbt0VXLE13HWS9h34fdzJmrWRXwobGA= github.com/yuin/gopher-lua v1.1.2 h1:yF/FjE3hD65tBbt0VXLE13HWS9h34fdzJmrWRXwobGA=
github.com/yuin/gopher-lua v1.1.2/go.mod h1:7aRmXIWl37SqRf0koeyylBEzJ+aPt8A+mmkQ4f1ntR8= github.com/yuin/gopher-lua v1.1.2/go.mod h1:7aRmXIWl37SqRf0koeyylBEzJ+aPt8A+mmkQ4f1ntR8=
github.com/zclconf/go-cty v1.18.0 h1:pJ8+HNI4gFoyRNqVE37wWbJWVw43BZczFo7KUoRczaA= github.com/zclconf/go-cty v1.18.1 h1:yEGE8M4iIZlyKQURZNb2SnEyZlZHUcBCnx6KF81KuwM=
github.com/zclconf/go-cty v1.18.0/go.mod h1:qpnV6EDNgC1sns/AleL1fvatHw72j+S+nS+MJ+T2CSg= github.com/zclconf/go-cty v1.18.1/go.mod h1:qpnV6EDNgC1sns/AleL1fvatHw72j+S+nS+MJ+T2CSg=
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940 h1:4r45xpDWB6ZMSMNJFMOjqrGHynW3DIBuR2H9j0ug+Mo= github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940 h1:4r45xpDWB6ZMSMNJFMOjqrGHynW3DIBuR2H9j0ug+Mo=
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940/go.mod h1:CmBdvvj3nqzfzJ6nTCIwDTPZ56aVGvDrmztiO5g3qrM= github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940/go.mod h1:CmBdvvj3nqzfzJ6nTCIwDTPZ56aVGvDrmztiO5g3qrM=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v4 v4.0.0-rc.4 h1:UP4+v6fFrBIb1l934bDl//mmnoIZEDK0idg1+AIvX5U= go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
go.yaml.in/yaml/v4 v4.0.0-rc.4/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0= go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI= golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY= golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0= golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw= golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo= golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8= golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA= golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k= golang.org/x/tools v0.45.0 h1:18qN3FAooORvApf5XjCXgsuayZOEtXf6JK18I3+ONa8=
golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0= golang.org/x/tools v0.45.0/go.mod h1:LuUGqqaXcXMEFEruIVJVm5mgDD8vww/z/SR1gQ4uE/0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@ -1,3 +1,5 @@
//go:build goinstall
package main package main
import ( import (
@ -11,6 +13,10 @@ import (
// TestGoInstallCompatibility ensures the module can be zipped for go install. // TestGoInstallCompatibility ensures the module can be zipped for go install.
// This is an integration test that uses the same zip.CreateFromDir function // This is an integration test that uses the same zip.CreateFromDir function
// that go install uses internally. If this test fails, go install will fail. // that go install uses internally. If this test fails, go install will fail.
//
// Built with the goinstall tag and run after the main test suite (see scripts/test.sh)
// so it does not race with pkg/yqlib tests that rewrite doc/*.md during execution.
//
// See: https://github.com/mikefarah/yq/issues/2587 // See: https://github.com/mikefarah/yq/issues/2587
func TestGoInstallCompatibility(t *testing.T) { func TestGoInstallCompatibility(t *testing.T) {
mod := module.Version{ mod := module.Version{

View File

@ -46,7 +46,11 @@ const (
func createStringScalarNode(stringValue string) *CandidateNode { func createStringScalarNode(stringValue string) *CandidateNode {
var node = &CandidateNode{Kind: ScalarNode} var node = &CandidateNode{Kind: ScalarNode}
node.Value = stringValue node.Value = stringValue
node.Tag = "!!str" if stringValue == "<<" {
node.Tag = "!!merge"
} else {
node.Tag = "!!str"
}
return node return node
} }

View File

@ -7,6 +7,9 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"math"
"strconv"
"strings"
"github.com/goccy/go-json" "github.com/goccy/go-json"
) )
@ -140,6 +143,12 @@ func (o *CandidateNode) MarshalJSON() ([]byte, error) {
return buf.Bytes(), err return buf.Bytes(), err
case ScalarNode: case ScalarNode:
log.Debugf("MarshalJSON ScalarNode") log.Debugf("MarshalJSON ScalarNode")
if o.guessTagFromCustomType() == "!!float" {
if raw, ok := jsonFloatLiteral(o.Value); ok {
buf.WriteString(raw)
return buf.Bytes(), nil
}
}
value, err := o.GetValueRep() value, err := o.GetValueRep()
if err != nil { if err != nil {
return buf.Bytes(), err return buf.Bytes(), err
@ -177,3 +186,85 @@ func (o *CandidateNode) MarshalJSON() ([]byte, error) {
return buf.Bytes(), err return buf.Bytes(), err
} }
} }
// jsonFloatLiteral returns a JSON-shaped representation of a YAML !!float scalar
// value, preserving the original textual form (e.g. "50.0" stays "50.0") whenever
// possible. The second return value is false when the value cannot be safely
// rendered as a JSON number (e.g. ".inf", ".nan", or anything that parses to a
// non-finite float); callers should fall back to the normal encoding path in
// that case, which preserves the existing behaviour for those inputs.
func jsonFloatLiteral(raw string) (string, bool) {
if raw == "" {
return "", false
}
f, err := strconv.ParseFloat(raw, 64)
if err != nil {
return "", false
}
if math.IsInf(f, 0) || math.IsNaN(f) {
return "", false
}
if isJSONNumberLiteral(raw) {
return raw, true
}
formatted := strconv.FormatFloat(f, 'f', -1, 64)
if !strings.ContainsAny(formatted, ".eE") {
formatted += ".0"
}
return formatted, true
}
// isJSONNumberLiteral reports whether s is already a valid JSON number literal
// representing a fractional value (i.e. contains a "." or an exponent), so it
// can be emitted verbatim without round-tripping through a float64.
func isJSONNumberLiteral(s string) bool {
if s == "" {
return false
}
i := 0
if s[i] == '-' {
i++
if i == len(s) {
return false
}
}
// integer part: 0 or [1-9][0-9]*
if s[i] == '0' {
i++
} else if s[i] >= '1' && s[i] <= '9' {
for i < len(s) && s[i] >= '0' && s[i] <= '9' {
i++
}
} else {
return false
}
hasFraction := false
if i < len(s) && s[i] == '.' {
hasFraction = true
i++
if i == len(s) || s[i] < '0' || s[i] > '9' {
return false
}
for i < len(s) && s[i] >= '0' && s[i] <= '9' {
i++
}
}
hasExponent := false
if i < len(s) && (s[i] == 'e' || s[i] == 'E') {
hasExponent = true
i++
if i < len(s) && (s[i] == '+' || s[i] == '-') {
i++
}
if i == len(s) || s[i] < '0' || s[i] > '9' {
return false
}
for i < len(s) && s[i] >= '0' && s[i] <= '9' {
i++
}
}
if i != len(s) {
return false
}
return hasFraction || hasExponent
}

View File

@ -106,6 +106,31 @@ func TestCreateScalarNodeScenarios(t *testing.T) {
} }
} }
type createStringScalarNodeScenario struct {
stringValue string
expectedTag string
}
var createStringScalarNodeScenarios = []createStringScalarNodeScenario{
{
stringValue: "yourKey",
expectedTag: "!!str",
},
{
stringValue: "<<",
expectedTag: "!!merge",
},
}
func TestCreateStringScalarNodeScenarios(t *testing.T) {
for _, tt := range createStringScalarNodeScenarios {
actual := createStringScalarNode(tt.stringValue)
test.AssertResultWithContext(t, tt.stringValue, actual.Value, fmt.Sprintf("Value for: %v", tt.stringValue))
test.AssertResultWithContext(t, tt.expectedTag, actual.Tag, fmt.Sprintf("Tag for: %v", tt.stringValue))
test.AssertResultWithContext(t, ScalarNode, actual.Kind, fmt.Sprintf("Kind for: %v", tt.stringValue))
}
}
func TestGetKeyForMapValue(t *testing.T) { func TestGetKeyForMapValue(t *testing.T) {
key := createStringScalarNode("yourKey") key := createStringScalarNode("yourKey")
n := CandidateNode{Key: key, Value: "meow", document: 3} n := CandidateNode{Key: key, Value: "meow", document: 3}

View File

@ -12,17 +12,20 @@ import (
type iniDecoder struct { type iniDecoder struct {
reader io.Reader reader io.Reader
finished bool // Flag to signal completion of processing finished bool // Flag to signal completion of processing
prefs INIPreferences
} }
func NewINIDecoder() Decoder { func NewINIDecoder(prefs INIPreferences) Decoder {
return &iniDecoder{ return &iniDecoder{
finished: false, // Initialise the flag as false finished: false, // Initialise the flag as false
prefs: prefs,
} }
} }
func (dec *iniDecoder) Init(reader io.Reader) error { func (dec *iniDecoder) Init(reader io.Reader) error {
// Store the reader for use in Decode // Store the reader for use in Decode
dec.reader = reader dec.reader = reader
dec.finished = false
return nil return nil
} }
@ -39,7 +42,10 @@ func (dec *iniDecoder) Decode() (*CandidateNode, error) {
} }
// Parse the INI content // Parse the INI content
cfg, err := ini.Load(content) loadOpts := ini.LoadOptions{
PreserveSurroundedQuote: dec.prefs.PreserveSurroundedQuote,
}
cfg, err := ini.LoadSources(loadOpts, content)
if err != nil { if err != nil {
return nil, fmt.Errorf("failed to parse INI content: %w", err) return nil, fmt.Errorf("failed to parse INI content: %w", err)
} }

View File

@ -16,10 +16,11 @@ type propertiesDecoder struct {
reader io.Reader reader io.Reader
finished bool finished bool
d DataTreeNavigator d DataTreeNavigator
prefs PropertiesPreferences
} }
func NewPropertiesDecoder() Decoder { func NewPropertiesDecoder() Decoder {
return &propertiesDecoder{d: NewDataTreeNavigator(), finished: false} return &propertiesDecoder{d: NewDataTreeNavigator(), finished: false, prefs: ConfiguredPropertiesPreferences.Copy()}
} }
func (dec *propertiesDecoder) Init(reader io.Reader) error { func (dec *propertiesDecoder) Init(reader io.Reader) error {
@ -28,20 +29,56 @@ func (dec *propertiesDecoder) Init(reader io.Reader) error {
return nil return nil
} }
func parsePropKey(key string) []interface{} { func parsePropKey(key string, prefs PropertiesPreferences) []interface{} {
pathStrArray := strings.Split(key, ".") pathStrArray := strings.Split(key, ".")
path := make([]interface{}, len(pathStrArray)) path := make([]interface{}, 0, len(pathStrArray))
for i, pathStr := range pathStrArray { for _, pathStr := range pathStrArray {
num, err := strconv.ParseInt(pathStr, 10, 32) path = appendPropKeySegment(path, pathStr, prefs.UseArrayBrackets)
if err == nil {
path[i] = num
} else {
path[i] = pathStr
}
} }
return path return path
} }
func appendPropKeySegment(path []interface{}, segment string, useArrayBrackets bool) []interface{} {
if useArrayBrackets && strings.Contains(segment, "[") {
bracketPath, ok := parsePropKeyArrayBracketSegment(segment)
if ok {
return append(path, bracketPath...)
}
}
num, err := strconv.ParseInt(segment, 10, 32)
if err == nil {
return append(path, num)
}
return append(path, segment)
}
func parsePropKeyArrayBracketSegment(segment string) ([]interface{}, bool) {
path := []interface{}{}
bracketIndex := strings.Index(segment, "[")
if bracketIndex > 0 {
path = append(path, segment[:bracketIndex])
}
remaining := segment[bracketIndex:]
for remaining != "" {
if !strings.HasPrefix(remaining, "[") {
return nil, false
}
closingBracket := strings.Index(remaining, "]")
if closingBracket < 0 {
return nil, false
}
arrayIndex, err := strconv.ParseInt(remaining[1:closingBracket], 10, 32)
if err != nil {
return nil, false
}
path = append(path, arrayIndex)
remaining = remaining[closingBracket+1:]
}
return path, true
}
func (dec *propertiesDecoder) processComment(c string) string { func (dec *propertiesDecoder) processComment(c string) string {
if c == "" { if c == "" {
return "" return ""
@ -75,7 +112,7 @@ func (dec *propertiesDecoder) applyPropertyComments(context Context, path []inte
func (dec *propertiesDecoder) applyProperty(context Context, properties *properties.Properties, key string) error { func (dec *propertiesDecoder) applyProperty(context Context, properties *properties.Properties, key string) error {
value, _ := properties.Get(key) value, _ := properties.Get(key)
path := parsePropKey(key) path := parsePropKey(key, dec.prefs)
propertyComments := properties.GetComments(key) propertyComments := properties.GetComments(key)
if len(propertyComments) > 0 { if len(propertyComments) > 0 {

View File

@ -2,7 +2,7 @@
Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference (or expands) aliases and remove anchor names). Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference (or expands) aliases and remove anchor names).
`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag. `yq` supports merge keys (like `<<: *blah`) from YAML 1.1. These are no longer part of the YAML 1.2 standard, but remain common in practice. Plain `<<:` keys are recognised as merge keys and round-trip as `<<:` without an explicit `!!merge` tag. When the source uses an explicit `!!merge` tag, that is preserved on output. Internally, when `yq` synthesises a `<<` map key (for example during merge operations), it tags the key as `!!merge` rather than `!!str`.
## NOTE --yaml-fix-merge-anchor-to-spec flag ## NOTE --yaml-fix-merge-anchor-to-spec flag
@ -32,7 +32,7 @@ Given a sample.yml file of:
r: 10 r: 10
- &SMALL - &SMALL
r: 1 r: 1
- !!merge <<: *CENTRE - <<: *CENTRE
r: 10 r: 10
``` ```
then then
@ -213,10 +213,10 @@ item_value: &item_value
value: true value: true
thingOne: thingOne:
name: item_1 name: item_1
!!merge <<: *item_value <<: *item_value
thingTwo: thingTwo:
name: item_2 name: item_2
!!merge <<: *item_value <<: *item_value
``` ```
then then
```bash ```bash
@ -231,7 +231,7 @@ thingOne:
value: false value: false
thingTwo: thingTwo:
name: item_2 name: item_2
!!merge <<: *item_value <<: *item_value
``` ```
## LEGACY: Explode with merge anchors ## LEGACY: Explode with merge anchors
@ -249,13 +249,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -298,7 +298,7 @@ Given a sample.yml file of:
r: 10 r: 10
- &SMALL - &SMALL
r: 1 r: 1
- !!merge <<: - <<:
- *CENTRE - *CENTRE
- *BIG - *BIG
``` ```
@ -328,7 +328,7 @@ Given a sample.yml file of:
r: 10 r: 10
- &SMALL - &SMALL
r: 1 r: 1
- !!merge <<: - <<:
- *BIG - *BIG
- *LEFT - *LEFT
- *SMALL - *SMALL
@ -361,13 +361,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -411,7 +411,7 @@ Given a sample.yml file of:
r: 10 r: 10
- &SMALL - &SMALL
r: 1 r: 1
- !!merge <<: - <<:
- *CENTRE - *CENTRE
- *BIG - *BIG
``` ```
@ -442,7 +442,7 @@ Given a sample.yml file of:
r: 10 r: 10
- &SMALL - &SMALL
r: 1 r: 1
- !!merge <<: - <<:
- *BIG - *BIG
- *LEFT - *LEFT
- *SMALL - *SMALL
@ -467,7 +467,7 @@ Given a sample.yml file of:
```yaml ```yaml
a: a:
b: &b 42 b: &b 42
!!merge <<: <<:
c: *b c: *b
``` ```
then then

View File

@ -55,7 +55,7 @@ yq '.a = .a / 0 | .b = .b / 0' sample.yml
``` ```
will output will output
```yaml ```yaml
a: !!float +Inf a: +Inf
b: !!float -Inf b: -Inf
``` ```

View File

@ -2,7 +2,7 @@
Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference (or expands) aliases and remove anchor names). Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference (or expands) aliases and remove anchor names).
`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag. `yq` supports merge keys (like `<<: *blah`) from YAML 1.1. These are no longer part of the YAML 1.2 standard, but remain common in practice. Plain `<<:` keys are recognised as merge keys and round-trip as `<<:` without an explicit `!!merge` tag. When the source uses an explicit `!!merge` tag, that is preserved on output. Internally, when `yq` synthesises a `<<` map key (for example during merge operations), it tags the key as `!!merge` rather than `!!str`.
## NOTE --yaml-fix-merge-anchor-to-spec flag ## NOTE --yaml-fix-merge-anchor-to-spec flag

View File

@ -34,7 +34,7 @@ yq '.a = .a % .b' sample.yml
``` ```
will output will output
```yaml ```yaml
a: !!float 2 a: 2
b: 2.5 b: 2.5
``` ```
@ -69,7 +69,7 @@ yq '.a = .a % .b' sample.yml
``` ```
will output will output
```yaml ```yaml
a: !!float NaN a: NaN
b: 0 b: 0
``` ```

View File

@ -471,13 +471,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -487,7 +487,7 @@ yq '.foobar * .foobarList' sample.yml
will output will output
```yaml ```yaml
c: foobarList_c c: foobarList_c
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
thing: foobar_thing thing: foobar_thing

View File

@ -131,13 +131,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -147,7 +147,7 @@ yq '.foobar | [..]' sample.yml
will output will output
```yaml ```yaml
- c: foobar_c - c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
- foobar_c - foobar_c
- *foo - *foo

View File

@ -51,7 +51,7 @@ country: Australia
``` ```
then then
```bash ```bash
yq --security-enable-system-operator '.country = system("/usr/bin/echo"; "test")' sample.yml yq --security-enable-system-operator '.country = system("/bin/echo"; "test")' sample.yml
``` ```
will output will output
```yaml ```yaml
@ -67,7 +67,7 @@ a: hello
``` ```
then then
```bash ```bash
yq --security-enable-system-operator '.a = system("/usr/bin/echo")' sample.yml yq --security-enable-system-operator '.a = system("/bin/echo")' sample.yml
``` ```
will output will output
```yaml ```yaml

View File

@ -294,13 +294,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -325,13 +325,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -376,13 +376,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -409,13 +409,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -442,13 +442,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -477,13 +477,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -513,13 +513,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -546,13 +546,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -579,13 +579,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then
@ -614,13 +614,13 @@ bar: &bar
c: bar_c c: bar_c
foobarList: foobarList:
b: foobarList_b b: foobarList_b
!!merge <<: <<:
- *foo - *foo
- *bar - *bar
c: foobarList_c c: foobarList_c
foobar: foobar:
c: foobar_c c: foobar_c
!!merge <<: *foo <<: *foo
thing: foobar_thing thing: foobar_thing
``` ```
then then

View File

@ -125,6 +125,22 @@ will output
{"whatever":"cat"} {"whatever":"cat"}
``` ```
## Encode json: preserve floats with trailing zero
Whole-number floats keep their decimal point so downstream consumers see a JSON number with a fractional part (matches Go's encoding/json, Python's json, and jq).
Given a sample.yml file of:
```yaml
percentiles: [50.0, 95.0, 99.0, 99.9]
```
then
```bash
yq -o=json -I=0 '.' sample.yml
```
will output
```json
{"percentiles":[50.0,95.0,99.0,99.9]}
```
## Roundtrip JSON Lines / NDJSON ## Roundtrip JSON Lines / NDJSON
Given a sample.json file of: Given a sample.json file of:
```json ```json

View File

@ -1,4 +1,4 @@
# TOML # TOML
Decode from TOML. Note that `yq` does not yet support outputting in TOML format (and therefore it cannot roundtrip) Encode and decode to and from TOML.

View File

@ -1,6 +1,6 @@
# TOML # TOML
Decode from TOML. Note that `yq` does not yet support outputting in TOML format (and therefore it cannot roundtrip) Encode and decode to and from TOML.
## Parse: Simple ## Parse: Simple

View File

@ -132,12 +132,23 @@ func (te *tomlEncoder) encodeRootMapping(w io.Writer, node *CandidateNode) error
} }
} }
// Preserve existing order by iterating Content
for i := 0; i < len(node.Content); i += 2 { for i := 0; i < len(node.Content); i += 2 {
keyNode := node.Content[i] keyNode := node.Content[i]
valNode := node.Content[i+1] valNode := node.Content[i+1]
if err := te.encodeTopLevelEntry(w, []string{keyNode.Value}, valNode); err != nil { if isTomlAttribute(valNode) {
return err if err := te.encodeTopLevelEntry(w, []string{keyNode.Value}, valNode); err != nil {
return err
}
}
}
for i := 0; i < len(node.Content); i += 2 {
keyNode := node.Content[i]
valNode := node.Content[i+1]
if !isTomlAttribute(valNode) {
if err := te.encodeTopLevelEntry(w, []string{keyNode.Value}, valNode); err != nil {
return err
}
} }
} }
return nil return nil
@ -170,8 +181,13 @@ func (te *tomlEncoder) encodeTopLevelEntry(w io.Writer, path []string, node *Can
if allMaps { if allMaps {
key := path[len(path)-1] key := path[len(path)-1]
quotedKey := tomlKey(key) quotedKey := tomlKey(key)
if te.wroteRootAttr {
if _, err := w.Write([]byte("\n")); err != nil {
return err
}
te.wroteRootAttr = false
}
for _, it := range node.Content { for _, it := range node.Content {
// [[key]] then body
if _, err := w.Write([]byte("[[" + quotedKey + "]]\n")); err != nil { if _, err := w.Write([]byte("[[" + quotedKey + "]]\n")); err != nil {
return err return err
} }
@ -184,9 +200,12 @@ func (te *tomlEncoder) encodeTopLevelEntry(w io.Writer, path []string, node *Can
// Regular array attribute // Regular array attribute
return te.writeArrayAttribute(w, path[len(path)-1], node) return te.writeArrayAttribute(w, path[len(path)-1], node)
case MappingNode: case MappingNode:
// Use inline table syntax for nodes explicitly marked as TOML inline tables // Use inline table syntax only for nodes explicitly marked as TOML inline tables.
// or YAML flow mappings. All other mappings become readable TOML table sections. // YAML flow-style mappings are not treated as inline tables; the FlowStyle attribute
if node.EncodeHint == EncodeHintInline || node.Style&FlowStyle != 0 { // is a YAML-specific rendering hint and should not affect TOML output. This ensures
// that auto-detected JSON input (parsed as YAML flow style) produces readable table
// sections, consistent with explicitly parsed JSON input.
if node.EncodeHint == EncodeHintInline {
return te.writeInlineTableAttribute(w, path[len(path)-1], node) return te.writeInlineTableAttribute(w, path[len(path)-1], node)
} }
return te.encodeSeparateMapping(w, path, node) return te.encodeSeparateMapping(w, path, node)
@ -195,7 +214,30 @@ func (te *tomlEncoder) encodeTopLevelEntry(w io.Writer, path []string, node *Can
} }
} }
func isTomlArrayOfTables(seq *CandidateNode) bool {
if len(seq.Content) == 0 {
return false
}
for _, it := range seq.Content {
if it.Kind != MappingNode || it.EncodeHint == EncodeHintInline {
return false
}
}
return true
}
func isTomlAttribute(node *CandidateNode) bool {
if node.Kind == ScalarNode {
return true
}
return node.Kind == SequenceNode && !isTomlArrayOfTables(node)
}
func (te *tomlEncoder) writeAttribute(w io.Writer, key string, value *CandidateNode) error { func (te *tomlEncoder) writeAttribute(w io.Writer, key string, value *CandidateNode) error {
if value.Tag == "!!null" {
return nil
}
te.wroteRootAttr = true // Mark that we wrote a root attribute te.wroteRootAttr = true // Mark that we wrote a root attribute
// Write head comment before the attribute // Write head comment before the attribute
@ -391,6 +433,9 @@ func (te *tomlEncoder) mappingToInlineTable(m *CandidateNode) (string, error) {
v := m.Content[i+1] v := m.Content[i+1]
switch v.Kind { switch v.Kind {
case ScalarNode: case ScalarNode:
if v.Tag == "!!null" {
continue
}
parts = append(parts, fmt.Sprintf("%s = %s", tomlKey(k), te.formatScalar(v))) parts = append(parts, fmt.Sprintf("%s = %s", tomlKey(k), te.formatScalar(v)))
case SequenceNode: case SequenceNode:
// inline array in inline table // inline array in inline table
@ -453,24 +498,16 @@ func (te *tomlEncoder) encodeSeparateMapping(w io.Writer, path []string, m *Cand
hasAttrs := false hasAttrs := false
for i := 0; i < len(m.Content); i += 2 { for i := 0; i < len(m.Content); i += 2 {
v := m.Content[i+1] v := m.Content[i+1]
if v.Kind == ScalarNode { if v.Kind == ScalarNode && v.Tag != "!!null" {
hasAttrs = true hasAttrs = true
break break
} }
if v.Kind == MappingNode && (v.EncodeHint == EncodeHintInline || v.Style&FlowStyle != 0) { if v.Kind == MappingNode && v.EncodeHint == EncodeHintInline {
hasAttrs = true hasAttrs = true
break break
} }
if v.Kind == SequenceNode { if v.Kind == SequenceNode {
// Check if it's NOT an array of tables if !isTomlArrayOfTables(v) {
allMaps := true
for _, it := range v.Content {
if it.Kind != MappingNode {
allMaps = false
break
}
}
if !allMaps {
hasAttrs = true hasAttrs = true
break break
} }
@ -500,15 +537,14 @@ func (te *tomlEncoder) encodeSeparateMapping(w io.Writer, path []string, m *Cand
} }
case SequenceNode: case SequenceNode:
// If sequence of maps, emit [[path.k]] per element // If sequence of maps, emit [[path.k]] per element
allMaps := true if isTomlArrayOfTables(v) {
for _, it := range v.Content {
if it.Kind != MappingNode {
allMaps = false
break
}
}
if allMaps {
key := tomlDottedKey(append(append([]string{}, path...), k)) key := tomlDottedKey(append(append([]string{}, path...), k))
if te.wroteRootAttr {
if _, err := w.Write([]byte("\n")); err != nil {
return err
}
te.wroteRootAttr = false
}
for _, it := range v.Content { for _, it := range v.Content {
if _, err := w.Write([]byte("[[" + key + "]]\n")); err != nil { if _, err := w.Write([]byte("[[" + key + "]]\n")); err != nil {
return err return err
@ -535,7 +571,7 @@ func (te *tomlEncoder) encodeSeparateMapping(w io.Writer, path []string, m *Cand
// encodeMappingBodyWithPath encodes attributes and nested arrays of tables using full dotted path context // encodeMappingBodyWithPath encodes attributes and nested arrays of tables using full dotted path context
func (te *tomlEncoder) encodeMappingBodyWithPath(w io.Writer, path []string, m *CandidateNode) error { func (te *tomlEncoder) encodeMappingBodyWithPath(w io.Writer, path []string, m *CandidateNode) error {
// First, attributes (scalars and non-map arrays) // First, attributes (scalars, inline mappings, and non-map arrays)
for i := 0; i < len(m.Content); i += 2 { for i := 0; i < len(m.Content); i += 2 {
k := m.Content[i].Value k := m.Content[i].Value
v := m.Content[i+1] v := m.Content[i+1]
@ -544,15 +580,14 @@ func (te *tomlEncoder) encodeMappingBodyWithPath(w io.Writer, path []string, m *
if err := te.writeAttribute(w, k, v); err != nil { if err := te.writeAttribute(w, k, v); err != nil {
return err return err
} }
case SequenceNode: case MappingNode:
allMaps := true if v.EncodeHint == EncodeHintInline {
for _, it := range v.Content { if err := te.writeInlineTableAttribute(w, k, v); err != nil {
if it.Kind != MappingNode { return err
allMaps = false
break
} }
} }
if !allMaps { case SequenceNode:
if !isTomlArrayOfTables(v) {
if err := te.writeArrayAttribute(w, k, v); err != nil { if err := te.writeArrayAttribute(w, k, v); err != nil {
return err return err
} }
@ -565,14 +600,7 @@ func (te *tomlEncoder) encodeMappingBodyWithPath(w io.Writer, path []string, m *
k := m.Content[i].Value k := m.Content[i].Value
v := m.Content[i+1] v := m.Content[i+1]
if v.Kind == SequenceNode { if v.Kind == SequenceNode {
allMaps := true if isTomlArrayOfTables(v) {
for _, it := range v.Content {
if it.Kind != MappingNode {
allMaps = false
break
}
}
if allMaps {
dotted := tomlDottedKey(append(append([]string{}, path...), k)) dotted := tomlDottedKey(append(append([]string{}, path...), k))
for _, it := range v.Content { for _, it := range v.Content {
if _, err := w.Write([]byte("[[" + dotted + "]]\n")); err != nil { if _, err := w.Write([]byte("[[" + dotted + "]]\n")); err != nil {
@ -586,21 +614,15 @@ func (te *tomlEncoder) encodeMappingBodyWithPath(w io.Writer, path []string, m *
} }
} }
// Finally, child mappings: inline-hint or flow-style ones become inline table attributes, // Finally, child mappings: inline-hint ones were emitted above as attributes,
// while all others are emitted as separate sub-table sections. // while all others are emitted as separate sub-table sections.
for i := 0; i < len(m.Content); i += 2 { for i := 0; i < len(m.Content); i += 2 {
k := m.Content[i].Value k := m.Content[i].Value
v := m.Content[i+1] v := m.Content[i+1]
if v.Kind == MappingNode { if v.Kind == MappingNode && v.EncodeHint != EncodeHintInline {
if v.EncodeHint == EncodeHintInline || v.Style&FlowStyle != 0 { subPath := append(append([]string{}, path...), k)
if err := te.writeInlineTableAttribute(w, k, v); err != nil { if err := te.encodeSeparateMapping(w, subPath, v); err != nil {
return err return err
}
} else {
subPath := append(append([]string{}, path...), k)
if err := te.encodeSeparateMapping(w, subPath, v); err != nil {
return err
}
} }
} }
} }
@ -655,7 +677,7 @@ func (te *tomlEncoder) colorizeToml(input []byte) []byte {
// Table sections - [section] or [[array]] // Table sections - [section] or [[array]]
// Only treat '[' as a table section if it appears at the start of the line // Only treat '[' as a table section if it appears at the start of the line
// (possibly after whitespace). This avoids mis-colouring inline arrays like // (possibly after whitespace). This avoids incorrectly colouring inline arrays like
// "ports = [8000, 8001]" as table sections. // "ports = [8000, 8001]" as table sections.
if ch == '[' { if ch == '[' {
isSectionHeader := true isSectionHeader := true

View File

@ -40,7 +40,7 @@ func tryRemoveTempFile(filename string) {
// thanks https://stackoverflow.com/questions/21060945/simple-way-to-copy-a-file-in-golang // thanks https://stackoverflow.com/questions/21060945/simple-way-to-copy-a-file-in-golang
func copyFileContents(src, dst string) (err error) { func copyFileContents(src, dst string) (err error) {
// ignore CWE-22 gosec issue - that's more targeted for http based apps that run in a public directory, // ignore CWE-22 gosec issue - that's more targeted for http based apps that run in a public directory,
// and ensuring that it's not possible to give a path to a file outside thar directory. // and ensuring that it's not possible to give a path to a file outside that directory.
in, err := os.Open(src) // #nosec in, err := os.Open(src) // #nosec
if err != nil { if err != nil {

View File

@ -90,7 +90,7 @@ var LuaFormat = &Format{"lua", []string{"l"},
var INIFormat = &Format{"ini", []string{"i"}, var INIFormat = &Format{"ini", []string{"i"},
func() Encoder { return NewINIEncoder() }, func() Encoder { return NewINIEncoder() },
func() Decoder { return NewINIDecoder() }, func() Decoder { return NewINIDecoder(ConfiguredINIPreferences) },
} }
var Formats = []*Format{ var Formats = []*Format{

View File

@ -230,8 +230,7 @@ var goccyYamlFormatScenarios = []formatScenario{
description: "merge anchor", description: "merge anchor",
skipDoc: true, skipDoc: true,
input: "a: &remember\n c: mike\nb:\n <<: *remember", input: "a: &remember\n c: mike\nb:\n <<: *remember",
// fine to have !!merge as that's what the current impl does expected: "a: &remember\n c: mike\nb:\n <<: *remember\n",
expected: "a: &remember\n c: mike\nb:\n !!merge <<: *remember\n",
}, },
{ {
description: "custom tag", description: "custom tag",

View File

@ -1,18 +1,21 @@
package yqlib package yqlib
type INIPreferences struct { type INIPreferences struct {
ColorsEnabled bool ColorsEnabled bool
PreserveSurroundedQuote bool
} }
func NewDefaultINIPreferences() INIPreferences { func NewDefaultINIPreferences() INIPreferences {
return INIPreferences{ return INIPreferences{
ColorsEnabled: false, ColorsEnabled: false,
PreserveSurroundedQuote: false,
} }
} }
func (p *INIPreferences) Copy() INIPreferences { func (p *INIPreferences) Copy() INIPreferences {
return INIPreferences{ return INIPreferences{
ColorsEnabled: p.ColorsEnabled, ColorsEnabled: p.ColorsEnabled,
PreserveSurroundedQuote: p.PreserveSurroundedQuote,
} }
} }

View File

@ -5,6 +5,7 @@ package yqlib
import ( import (
"bufio" "bufio"
"fmt" "fmt"
"strings"
"testing" "testing"
"github.com/mikefarah/yq/v4/test" "github.com/mikefarah/yq/v4/test"
@ -22,6 +23,16 @@ const expectedSimpleINIYaml = `section:
key: value key: value
` `
const quotedINIInput = `[section]
color_theme = "Default"
theme_background = "False"
`
const expectedQuotedINIOutput = `[section]
color_theme = "Default"
theme_background = "False"
`
var iniScenarios = []formatScenario{ var iniScenarios = []formatScenario{
{ {
description: "Parse INI: simple", description: "Parse INI: simple",
@ -49,6 +60,22 @@ var iniScenarios = []formatScenario{
}, },
} }
// iniPreserveQuotesPrefs returns INIPreferences with PreserveSurroundedQuote enabled.
func iniPreserveQuotesPrefs() INIPreferences {
prefs := NewDefaultINIPreferences()
prefs.PreserveSurroundedQuote = true
return prefs
}
var iniPreserveQuotesScenarios = []formatScenario{
{
description: "Roundtrip INI: preserve quotes",
input: quotedINIInput,
expected: expectedQuotedINIOutput,
scenarioType: "roundtrip",
},
}
func documentRoundtripINIScenario(w *bufio.Writer, s formatScenario) { func documentRoundtripINIScenario(w *bufio.Writer, s formatScenario) {
writeOrPanic(w, fmt.Sprintf("## %v\n", s.description)) writeOrPanic(w, fmt.Sprintf("## %v\n", s.description))
@ -70,7 +97,7 @@ func documentRoundtripINIScenario(w *bufio.Writer, s formatScenario) {
} }
writeOrPanic(w, "will output\n") writeOrPanic(w, "will output\n")
writeOrPanic(w, fmt.Sprintf("```ini\n%v```\n\n", mustProcessFormatScenario(s, NewINIDecoder(), NewINIEncoder()))) writeOrPanic(w, fmt.Sprintf("```ini\n%v```\n\n", mustProcessFormatScenario(s, NewINIDecoder(NewDefaultINIPreferences()), NewINIEncoder())))
} }
func documentDecodeINIScenario(w *bufio.Writer, s formatScenario) { func documentDecodeINIScenario(w *bufio.Writer, s formatScenario) {
@ -94,7 +121,7 @@ func documentDecodeINIScenario(w *bufio.Writer, s formatScenario) {
} }
writeOrPanic(w, "will output\n") writeOrPanic(w, "will output\n")
writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n\n", mustProcessFormatScenario(s, NewINIDecoder(), NewYamlEncoder(ConfiguredYamlPreferences)))) writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n\n", mustProcessFormatScenario(s, NewINIDecoder(NewDefaultINIPreferences()), NewYamlEncoder(ConfiguredYamlPreferences))))
} }
func testINIScenario(t *testing.T, s formatScenario) { func testINIScenario(t *testing.T, s formatScenario) {
@ -102,11 +129,11 @@ func testINIScenario(t *testing.T, s formatScenario) {
case "encode": case "encode":
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewINIEncoder()), s.description) test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewINIEncoder()), s.description)
case "decode": case "decode":
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewINIDecoder(), NewYamlEncoder(ConfiguredYamlPreferences)), s.description) test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewINIDecoder(NewDefaultINIPreferences()), NewYamlEncoder(ConfiguredYamlPreferences)), s.description)
case "roundtrip": case "roundtrip":
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewINIDecoder(), NewINIEncoder()), s.description) test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewINIDecoder(NewDefaultINIPreferences()), NewINIEncoder()), s.description)
case "decode-error": case "decode-error":
result, err := processFormatScenario(s, NewINIDecoder(), NewINIEncoder()) result, err := processFormatScenario(s, NewINIDecoder(NewDefaultINIPreferences()), NewINIEncoder())
if err == nil { if err == nil {
t.Errorf("Expected error '%v' but it worked: %v", s.expectedError, result) t.Errorf("Expected error '%v' but it worked: %v", s.expectedError, result)
} else { } else {
@ -175,6 +202,21 @@ func documentDecodeErrorINIScenario(w *bufio.Writer, s formatScenario) {
writeOrPanic(w, fmt.Sprintf("```\n%v\n```\n\n", s.expectedError)) writeOrPanic(w, fmt.Sprintf("```\n%v\n```\n\n", s.expectedError))
} }
func TestINIDecoderInitResetsFinished(t *testing.T) {
decoder := NewINIDecoder(NewDefaultINIPreferences())
firstDocuments, err := readDocuments(strings.NewReader("[first]\nkey = value\n"), "first.ini", 0, decoder)
if err != nil {
t.Fatal(err)
}
test.AssertResult(t, 1, firstDocuments.Len())
secondDocuments, err := readDocuments(strings.NewReader("[second]\nkey = value\n"), "second.ini", 1, decoder)
if err != nil {
t.Fatal(err)
}
test.AssertResult(t, 1, secondDocuments.Len())
}
func TestINIScenarios(t *testing.T) { func TestINIScenarios(t *testing.T) {
for _, tt := range iniScenarios { for _, tt := range iniScenarios {
testINIScenario(t, tt) testINIScenario(t, tt)
@ -185,3 +227,19 @@ func TestINIScenarios(t *testing.T) {
} }
documentScenarios(t, "usage", "convert", genericScenarios, documentINIScenario) documentScenarios(t, "usage", "convert", genericScenarios, documentINIScenario)
} }
func testINIPreserveQuotesScenario(t *testing.T, s formatScenario) {
prefs := iniPreserveQuotesPrefs()
switch s.scenarioType {
case "roundtrip":
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewINIDecoder(prefs), NewINIEncoder()), s.description)
default:
panic(fmt.Sprintf("unhandled scenario type %q", s.scenarioType))
}
}
func TestINIPreserveQuotesScenarios(t *testing.T) {
for _, tt := range iniPreserveQuotesScenarios {
testINIPreserveQuotesScenario(t, tt)
}
}

View File

@ -220,6 +220,54 @@ var jsonScenarios = []formatScenario{
expected: "{\"stuff\":\"cool\"}\n{\"whatever\":\"cat\"}\n", expected: "{\"stuff\":\"cool\"}\n{\"whatever\":\"cat\"}\n",
scenarioType: "encode", scenarioType: "encode",
}, },
{
description: "Encode json: preserve floats with trailing zero",
subdescription: "Whole-number floats keep their decimal point so downstream consumers see a JSON number with a fractional part (matches Go's encoding/json, Python's json, and jq).",
input: `percentiles: [50.0, 95.0, 99.0, 99.9]`,
indent: 0,
expected: "{\"percentiles\":[50.0,95.0,99.0,99.9]}\n",
scenarioType: "encode",
},
{
description: "Encode json: ints stay ints",
skipDoc: true,
input: `a: 50`,
indent: 0,
expected: "{\"a\":50}\n",
scenarioType: "encode",
},
{
description: "Encode json: !!float tagged whole number gets .0",
skipDoc: true,
input: `a: !!float 5`,
indent: 0,
expected: "{\"a\":5.0}\n",
scenarioType: "encode",
},
{
description: "Encode json: scientific notation float preserved",
skipDoc: true,
input: `a: 1.5e-3`,
indent: 0,
expected: "{\"a\":1.5e-3}\n",
scenarioType: "encode",
},
{
description: "Encode json: negative float preserved",
skipDoc: true,
input: `a: -7.0`,
indent: 0,
expected: "{\"a\":-7.0}\n",
scenarioType: "encode",
},
{
description: "Encode json: mixed int and float array",
skipDoc: true,
input: `a: [1, 2.0, 3, 4.5]`,
indent: 0,
expected: "{\"a\":[1,2.0,3,4.5]}\n",
scenarioType: "encode",
},
{ {
description: "Roundtrip JSON Lines / NDJSON", description: "Roundtrip JSON Lines / NDJSON",
input: sampleNdJson, input: sampleNdJson,

View File

@ -24,7 +24,7 @@ type parseSnippetScenario struct {
var parseSnippetScenarios = []parseSnippetScenario{ var parseSnippetScenarios = []parseSnippetScenario{
{ {
snippet: ":", snippet: ":",
expectedError: "yaml: while parsing a block mapping at <unknown position>: did not find expected key", expectedError: "go-yaml load error in parser (while parsing a block mapping) at L1.C1: did not find expected key",
}, },
{ {
snippet: "", snippet: "",

View File

@ -2,7 +2,7 @@
package yqlib package yqlib
func NewINIDecoder() Decoder { func NewINIDecoder(prefs INIPreferences) Decoder {
return nil return nil
} }

View File

@ -170,6 +170,10 @@ func fixedReconstructAliasedMap(node *CandidateNode) error {
if mergeNodeSeq.Kind == AliasNode { if mergeNodeSeq.Kind == AliasNode {
mergeNodeSeq = mergeNodeSeq.Alias mergeNodeSeq = mergeNodeSeq.Alias
} }
mergeNodeSeq = mergeNodeSeq.Copy()
if err := explodeNode(mergeNodeSeq, Context{}); err != nil {
return err
}
if mergeNodeSeq.Kind != MappingNode { if mergeNodeSeq.Kind != MappingNode {
return fmt.Errorf("can only use merge anchors with maps (!!map) or sequences (!!seq) of maps, but got sequence containing %v", mergeNodeSeq.Tag) return fmt.Errorf("can only use merge anchors with maps (!!map) or sequences (!!seq) of maps, but got sequence containing %v", mergeNodeSeq.Tag)
} }
@ -179,12 +183,7 @@ func fixedReconstructAliasedMap(node *CandidateNode) error {
}) })
for _, item := range itemsToAdd { for _, item := range itemsToAdd {
// copy to ensure exploding doesn't modify the original node newContent = append(newContent, item.Copy())
itemCopy := item.Copy()
if err := explodeNode(itemCopy, Context{}); err != nil {
return err
}
newContent = append(newContent, itemCopy)
} }
} }
} }

View File

@ -31,7 +31,7 @@ thingOne:
value: false value: false
thingTwo: thingTwo:
name: item_2 name: item_2
!!merge <<: *item_value <<: *item_value
` `
var explodeMergeAnchorsFixedExpected = `D0, P[], (!!map)::foo: var explodeMergeAnchorsFixedExpected = `D0, P[], (!!map)::foo:
@ -198,6 +198,15 @@ var fixedAnchorOperatorScenarios = []expressionScenario{
"D0, P[], (!!map)::{a: 42}\n", "D0, P[], (!!map)::{a: 42}\n",
}, },
}, },
{
skipDoc: true,
description: "Nested merge anchor with inline map",
document: `{<<: {<<: {a: 42}}}`,
expression: `explode(.)`,
expected: []string{
"D0, P[], (!!map)::{a: 42}\n",
},
},
{ {
skipDoc: true, skipDoc: true,
description: "Merge anchor with sequence with inline map", description: "Merge anchor with sequence with inline map",
@ -279,7 +288,63 @@ var badAnchorOperatorScenarios = []expressionScenario{
}, },
} }
var mixedMergeTagStyleDocument = `
constants:
errorResponse: &errorResponse
status: 200
endpoints:
- condition: true
!!merge <<: *errorResponse
- condition: false
<<: *errorResponse
other:
!!merge <<: *errorResponse
somethingElse:
<<: *errorResponse
`
var mixedMergeTagStyleExplodedDocument = `
constants:
errorResponse:
status: 200
endpoints:
- condition: true
status: 200
- condition: false
status: 200
other:
status: 200
somethingElse:
status: 200
`
var anchorOperatorScenarios = []expressionScenario{ var anchorOperatorScenarios = []expressionScenario{
{
// mergeObjects previously skipped all !!merge-tagged nodes. Since !!merge only appears on
// << map keys, this meant applyAssignment was never called for the << key. It was later
// autocreated by createStringScalarNode("<<") with tag !!str, silently dropping !!merge.
// DontFollowAlias:true already prevents aliases being followed, so the skip was redundant.
// Old (buggy) output: "D0, P[], (!!map)::base: &base\n x: 1\ndest:\n <<: *base\n"
skipDoc: true,
description: "direct *+ preserves explicit !!merge tag on << key (regression for issue 2677)",
document: "base: &base\n x: 1\ndest:\n !!merge <<: *base\n",
expression: `. as $d | {} *+ $d`,
expected: []string{"D0, P[], (!!map)::base: &base\n x: 1\ndest:\n !!merge <<: *base\n"},
},
{
skipDoc: true,
description: "explicit !!merge tag on << key is preserved through ireduce merge",
document: mixedMergeTagStyleDocument,
expression: `. as $item ireduce ({}; . *+ $item)`,
expected: []string{"D0, P[], (!!map)::" + mixedMergeTagStyleDocument},
},
{
skipDoc: true,
description: "explode expands << merge keys regardless of explicit tag style (!!merge or plain)",
document: mixedMergeTagStyleDocument,
expression: `explode(.)`,
expected: []string{"D0, P[], (!!map)::" + mixedMergeTagStyleExplodedDocument},
},
{ {
skipDoc: true, skipDoc: true,
description: "merge anchor to alias alias", description: "merge anchor to alias alias",

View File

@ -45,7 +45,7 @@ var divideOperatorScenarios = []expressionScenario{
document: `{a: 1, b: -1}`, document: `{a: 1, b: -1}`,
expression: `.a = .a / 0 | .b = .b / 0`, expression: `.a = .a / 0 | .b = .b / 0`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::{a: !!float +Inf, b: !!float -Inf}\n", "D0, P[], (!!map)::{a: +Inf, b: -Inf}\n",
}, },
}, },
{ {

View File

@ -77,7 +77,7 @@ var loadScenarios = []expressionScenario{
document: `{something: {file: "thing.yml"}, over: {here: [{file: "thing.yml"}]}}`, document: `{something: {file: "thing.yml"}, over: {here: [{file: "thing.yml"}]}}`,
expression: `(.. | select(has("file"))) |= load("../../examples/" + .file)`, expression: `(.. | select(has("file"))) |= load("../../examples/" + .file)`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::{something: {a: apple is included, b: cool.}, over: {here: [{a: apple is included, b: cool.}]}}\n", "D0, P[], (!!map)::{something: {a: apple is included, b: cool.}, over: {here: [{a: apple is included,\n b: cool.}]}}\n",
}, },
}, },
{ {

View File

@ -37,7 +37,7 @@ var moduloOperatorScenarios = []expressionScenario{
document: `{a: 12, b: 2.5}`, document: `{a: 12, b: 2.5}`,
expression: `.a = .a % .b`, expression: `.a = .a % .b`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::{a: !!float 2, b: 2.5}\n", "D0, P[], (!!map)::{a: 2, b: 2.5}\n",
}, },
}, },
{ {
@ -53,7 +53,7 @@ var moduloOperatorScenarios = []expressionScenario{
document: `{a: 1.1, b: 0}`, document: `{a: 1.1, b: 0}`,
expression: `.a = .a % .b`, expression: `.a = .a % .b`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::{a: !!float NaN, b: 0}\n", "D0, P[], (!!map)::{a: NaN, b: 0}\n",
}, },
}, },
{ {
@ -70,7 +70,7 @@ var moduloOperatorScenarios = []expressionScenario{
document: "a: 2\nb: !goat 2.3", document: "a: 2\nb: !goat 2.3",
expression: `.a = .a % .b`, expression: `.a = .a % .b`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::a: !!float 2\nb: !goat 2.3\n", "D0, P[], (!!map)::a: 2\nb: !goat 2.3\n",
}, },
}, },
{ {

View File

@ -189,10 +189,6 @@ func mergeObjects(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs
log.Debugf("going to applied assignment to LHS: %v with RHS: %v", NodeToString(lhs), NodeToString(candidate)) log.Debugf("going to applied assignment to LHS: %v with RHS: %v", NodeToString(lhs), NodeToString(candidate))
if candidate.Tag == "!!merge" {
continue
}
err := applyAssignment(d, context, pathIndexToStartFrom, lhs, candidate, preferences) err := applyAssignment(d, context, pathIndexToStartFrom, lhs, candidate, preferences)
if err != nil { if err != nil {
return nil, err return nil, err

View File

@ -2,6 +2,7 @@ package yqlib
import ( import (
"fmt" "fmt"
"math/bits"
"strings" "strings"
"testing" "testing"
) )
@ -120,7 +121,7 @@ var multiplyOperatorScenarios = []expressionScenario{
document: mergeArrayWithAnchors, document: mergeArrayWithAnchors,
expression: `. * .`, expression: `. * .`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::sample:\n - &a\n - !!merge <<: *a\n", "D0, P[], (!!map)::sample:\n - &a\n - <<: *a\n",
}, },
}, },
{ {
@ -513,7 +514,7 @@ var multiplyOperatorScenarios = []expressionScenario{
environmentVariables: map[string]string{"originalPath": ".myArray", "otherPath": ".newArray", "idPath": ".a"}, environmentVariables: map[string]string{"originalPath": ".myArray", "otherPath": ".newArray", "idPath": ".a"},
expression: mergeExpression, expression: mergeExpression,
expected: []string{ expected: []string{
"D0, P[], (!!map)::{myArray: [{a: apple, b: appleB2}, {a: kiwi, b: kiwiB}, {a: banana, b: bananaB, c: bananaC}, {a: dingo, c: dingoC}], something: else}\n", "D0, P[], (!!map)::{myArray: [{a: apple, b: appleB2}, {a: kiwi, b: kiwiB}, {a: banana, b: bananaB, c: bananaC},\n {a: dingo, c: dingoC}], something: else}\n",
}, },
}, },
{ {
@ -545,7 +546,7 @@ var multiplyOperatorScenarios = []expressionScenario{
document: mergeDocSample, document: mergeDocSample,
expression: `.foobar * .foobarList`, expression: `.foobar * .foobarList`,
expected: []string{ expected: []string{
"D0, P[foobar], (!!map)::c: foobarList_c\n!!merge <<: [*foo, *bar]\nthing: foobar_thing\nb: foobarList_b\n", "D0, P[foobar], (!!map)::c: foobarList_c\n<<: [*foo, *bar]\nthing: foobar_thing\nb: foobarList_b\n",
}, },
}, },
{ {
@ -553,7 +554,7 @@ var multiplyOperatorScenarios = []expressionScenario{
document: document, document: document,
expression: `.b * .c`, expression: `.b * .c`,
expected: []string{ expected: []string{
"D0, P[b], (!!map)::{name: dog, \"<<\": *cat}\n", "D0, P[b], (!!map)::{name: dog, <<: *cat}\n",
}, },
}, },
{ {
@ -580,7 +581,7 @@ var multiplyOperatorScenarios = []expressionScenario{
document: "a: 2\nb: !goat 3.5", document: "a: 2\nb: !goat 3.5",
expression: ".a = .a * .b", expression: ".a = .a * .b",
expected: []string{ expected: []string{
"D0, P[], (!!map)::a: !!float 7\nb: !goat 3.5\n", "D0, P[], (!!map)::a: 7\nb: !goat 3.5\n",
}, },
}, },
{ {
@ -707,11 +708,13 @@ var multiplyOperatorScenarios = []expressionScenario{
expectedError: "result of repeating string (1 bytes) by 99999999 would exceed 10485760 bytes", expectedError: "result of repeating string (1 bytes) by 99999999 would exceed 10485760 bytes",
}, },
{ {
// The size guard must not overflow: len * count can wrap to // Pick a count whose product with len("ab") overflows int on
// a negative or small value on 64-bit, bypassing the check. // any architecture: 2^30 on 32-bit, 2^62 on 64-bit. Doubling
// either yields MaxInt+1, which wraps to MinInt and bypasses
// a naive len*count guard.
skipDoc: true, skipDoc: true,
expression: `"ab" * 4611686018427387904`, expression: fmt.Sprintf(`"ab" * %d`, 1<<(bits.UintSize-2)),
expectedError: "result of repeating string (2 bytes) by 4611686018427387904 would exceed 10485760 bytes", expectedError: fmt.Sprintf("result of repeating string (2 bytes) by %d would exceed 10485760 bytes", 1<<(bits.UintSize-2)),
}, },
} }

View File

@ -187,7 +187,7 @@ var recursiveDescentOperatorScenarios = []expressionScenario{
document: mergeDocSample, document: mergeDocSample,
expression: `.foobar | [..]`, expression: `.foobar | [..]`,
expected: []string{ expected: []string{
"D0, P[foobar], (!!seq)::- c: foobar_c\n !!merge <<: *foo\n thing: foobar_thing\n- foobar_c\n- *foo\n- foobar_thing\n", "D0, P[foobar], (!!seq)::- c: foobar_c\n <<: *foo\n thing: foobar_thing\n- foobar_c\n- *foo\n- foobar_thing\n",
}, },
}, },
{ {
@ -195,7 +195,7 @@ var recursiveDescentOperatorScenarios = []expressionScenario{
document: mergeDocSample, document: mergeDocSample,
expression: `.foobar | [...]`, expression: `.foobar | [...]`,
expected: []string{ expected: []string{
"D0, P[foobar], (!!seq)::- c: foobar_c\n !!merge <<: *foo\n thing: foobar_thing\n- c\n- foobar_c\n- !!merge <<\n- *foo\n- thing\n- foobar_thing\n", "D0, P[foobar], (!!seq)::- c: foobar_c\n <<: *foo\n thing: foobar_thing\n- c\n- foobar_c\n- <<\n- *foo\n- thing\n- foobar_thing\n",
}, },
}, },
{ {
@ -203,7 +203,7 @@ var recursiveDescentOperatorScenarios = []expressionScenario{
document: mergeDocSample, document: mergeDocSample,
expression: `.foobarList | ..`, expression: `.foobarList | ..`,
expected: []string{ expected: []string{
"D0, P[foobarList], (!!map)::b: foobarList_b\n!!merge <<: [*foo, *bar]\nc: foobarList_c\n", "D0, P[foobarList], (!!map)::b: foobarList_b\n<<: [*foo, *bar]\nc: foobarList_c\n",
"D0, P[foobarList b], (!!str)::foobarList_b\n", "D0, P[foobarList b], (!!str)::foobarList_b\n",
"D0, P[foobarList <<], (!!seq)::[*foo, *bar]\n", "D0, P[foobarList <<], (!!seq)::[*foo, *bar]\n",
"D0, P[foobarList << 0], (alias)::*foo\n", "D0, P[foobarList << 0], (alias)::*foo\n",
@ -216,7 +216,7 @@ var recursiveDescentOperatorScenarios = []expressionScenario{
document: mergeDocSample, document: mergeDocSample,
expression: `.foobarList | ...`, expression: `.foobarList | ...`,
expected: []string{ expected: []string{
"D0, P[foobarList], (!!map)::b: foobarList_b\n!!merge <<: [*foo, *bar]\nc: foobarList_c\n", "D0, P[foobarList], (!!map)::b: foobarList_b\n<<: [*foo, *bar]\nc: foobarList_c\n",
"D0, P[foobarList b], (!!str)::b\n", "D0, P[foobarList b], (!!str)::b\n",
"D0, P[foobarList b], (!!str)::foobarList_b\n", "D0, P[foobarList b], (!!str)::foobarList_b\n",
"D0, P[foobarList <<], (!!merge)::<<\n", "D0, P[foobarList <<], (!!merge)::<<\n",

View File

@ -98,7 +98,7 @@ var selectOperatorScenarios = []expressionScenario{
document: `[{animal: cat, legs: {cool: true}}, {animal: fish}]`, document: `[{animal: cat, legs: {cool: true}}, {animal: fish}]`,
expression: `(.[] | select(.legs.cool == true).canWalk) = true | (.[] | .alive.things) = "yes"`, expression: `(.[] | select(.legs.cool == true).canWalk) = true | (.[] | .alive.things) = "yes"`,
expected: []string{ expected: []string{
"D0, P[], (!!seq)::[{animal: cat, legs: {cool: true}, canWalk: true, alive: {things: yes}}, {animal: fish, alive: {things: yes}}]\n", "D0, P[], (!!seq)::[{animal: cat, legs: {cool: true}, canWalk: true, alive: {things: yes}}, {animal: fish,\n alive: {things: yes}}]\n",
}, },
}, },
{ {

View File

@ -37,7 +37,7 @@ var sortKeysOperatorScenarios = []expressionScenario{
document: `{bParent: {c: dog, array: [3,1,2]}, aParent: {z: donkey, x: [{c: yum, b: delish}, {b: ew, a: apple}]}}`, document: `{bParent: {c: dog, array: [3,1,2]}, aParent: {z: donkey, x: [{c: yum, b: delish}, {b: ew, a: apple}]}}`,
expression: `sort_keys(..)`, expression: `sort_keys(..)`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::{aParent: {x: [{b: delish, c: yum}, {a: apple, b: ew}], z: donkey}, bParent: {array: [3, 1, 2], c: dog}}\n", "D0, P[], (!!map)::{aParent: {x: [{b: delish, c: yum}, {a: apple, b: ew}], z: donkey}, bParent: {array: [\n 3, 1, 2], c: dog}}\n",
}, },
}, },
} }

View File

@ -50,7 +50,7 @@ var styleOperatorScenarios = []expressionScenario{
document: "bing: &foo {x: z}\na:\n c: cat\n <<: [*foo]", document: "bing: &foo {x: z}\na:\n c: cat\n <<: [*foo]",
expression: `(... | select(tag=="!!str")) style="single"`, expression: `(... | select(tag=="!!str")) style="single"`,
expected: []string{ expected: []string{
"D0, P[], (!!map)::'bing': &foo {'x': 'z'}\n'a':\n 'c': 'cat'\n !!merge <<: [*foo]\n", "D0, P[], (!!map)::'bing': &foo {'x': 'z'}\n'a':\n 'c': 'cat'\n <<: [*foo]\n",
}, },
}, },
{ {

View File

@ -494,7 +494,7 @@ var traversePathOperatorScenarios = []expressionScenario{
document: mergeDocSample, document: mergeDocSample,
expression: `.foobar`, expression: `.foobar`,
expected: []string{ expected: []string{
"D0, P[foobar], (!!map)::c: foobar_c\n!!merge <<: *foo\nthing: foobar_thing\n", "D0, P[foobar], (!!map)::c: foobar_c\n<<: *foo\nthing: foobar_thing\n",
}, },
}, },
{ {
@ -518,7 +518,7 @@ var traversePathOperatorScenarios = []expressionScenario{
document: mergeDocSample, document: mergeDocSample,
expression: `.foobarList`, expression: `.foobarList`,
expected: []string{ expected: []string{
"D0, P[foobarList], (!!map)::b: foobarList_b\n!!merge <<: [*foo, *bar]\nc: foobarList_c\n", "D0, P[foobarList], (!!map)::b: foobarList_b\n<<: [*foo, *bar]\nc: foobarList_c\n",
}, },
}, },
{ {

View File

@ -202,6 +202,37 @@ var propertyScenarios = []formatScenario{
expected: expectedDecodedYaml, expected: expectedDecodedYaml,
scenarioType: "decode", scenarioType: "decode",
}, },
{
skipDoc: true,
description: "Decode properties with array brackets",
input: `user.credentials[0].username=user1
user.credentials[0].password=$2b$08$...
user.credentials[1].username=user2
user.credentials[1].password=$2b$08$...
user.credentials[2].username=user3
user.credentials[2].password=$2b$10$...`,
expected: `user:
credentials:
- username: user1
password: $2b$08$...
- username: user2
password: $2b$08$...
- username: user3
password: $2b$10$...
`,
scenarioType: "decode-array-brackets",
},
{
skipDoc: true,
description: "Decode properties with nested array brackets",
input: `user.clowns[0][1] = "cool"`,
expected: `user:
clowns:
- - null
- '"cool"'
`,
scenarioType: "decode-array-brackets",
},
{ {
skipDoc: true, skipDoc: true,
@ -442,6 +473,12 @@ func TestPropertyScenarios(t *testing.T) {
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewPropertiesEncoder(ConfiguredPropertiesPreferences)), s.description) test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewPropertiesEncoder(ConfiguredPropertiesPreferences)), s.description)
case "decode": case "decode":
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewPropertiesDecoder(), NewYamlEncoder(ConfiguredYamlPreferences)), s.description) test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewPropertiesDecoder(), NewYamlEncoder(ConfiguredYamlPreferences)), s.description)
case "decode-array-brackets":
previousPreferences := ConfiguredPropertiesPreferences.Copy()
ConfiguredPropertiesPreferences.UseArrayBrackets = true
actual := mustProcessFormatScenario(s, NewPropertiesDecoder(), NewYamlEncoder(ConfiguredYamlPreferences))
ConfiguredPropertiesPreferences = previousPreferences
test.AssertResultWithContext(t, s.expected, actual, s.description)
case "encode-wrapped": case "encode-wrapped":
prefs := ConfiguredPropertiesPreferences.Copy() prefs := ConfiguredPropertiesPreferences.Copy()
prefs.UnwrapScalar = false prefs.UnwrapScalar = false

View File

@ -209,6 +209,50 @@ address = "12 cat st"
var rtEmptyArray = `A = [] var rtEmptyArray = `A = []
` `
var rtEmptyArrayInTable = `[features]
my-feature = []
`
var rtMixedEmptyArraysInTable = `[features]
my-other-feature = []
my-feature = ["my-other-feature"]
`
var yamlEmptyArrayInTable = `features:
my-feature: []
`
var expectedTomlEmptyArrayInTable = `[features]
my-feature = []
`
var yamlMixedEmptyArraysInTable = `features:
my-other-feature: []
my-feature:
- my-other-feature
`
var expectedTomlMixedEmptyArraysInTable = `[features]
my-other-feature = []
my-feature = ["my-other-feature"]
`
var issue2688SampleToml = `[project]
name = "some-project"
version = "0.5.1"
authors = [{name = "Author", email = "author@example.com"}]
license = { file = "LICENSE" }
readme = "README.md"
`
var issue2688SampleExpected = `[project]
name = "some-project"
version = "0.5.2"
authors = [{ name = "Author", email = "author@example.com" }]
license = { file = "LICENSE" }
readme = "README.md"
`
var rtSampleTable = `var = "x" var rtSampleTable = `var = "x"
[owner.contact] [owner.contact]
@ -563,6 +607,36 @@ var tomlScenarios = []formatScenario{
expected: rtEmptyArray, expected: rtEmptyArray,
scenarioType: "roundtrip", scenarioType: "roundtrip",
}, },
{
skipDoc: true,
description: "Issue #2674: roundtrip empty array in table",
input: rtEmptyArrayInTable,
expression: ".",
expected: rtEmptyArrayInTable,
scenarioType: "roundtrip",
},
{
skipDoc: true,
description: "Issue #2674: roundtrip mixed empty and non-empty arrays in table",
input: rtMixedEmptyArraysInTable,
expression: ".",
expected: rtMixedEmptyArraysInTable,
scenarioType: "roundtrip",
},
{
skipDoc: true,
description: "Issue #2674: encode empty array in table",
input: yamlEmptyArrayInTable,
expected: expectedTomlEmptyArrayInTable,
scenarioType: "encode",
},
{
skipDoc: true,
description: "Issue #2674: encode mixed empty and non-empty arrays in table",
input: yamlMixedEmptyArraysInTable,
expected: expectedTomlMixedEmptyArraysInTable,
scenarioType: "encode",
},
{ {
description: "Roundtrip: sample table", description: "Roundtrip: sample table",
input: rtSampleTable, input: rtSampleTable,
@ -649,11 +723,33 @@ var tomlScenarios = []formatScenario{
}, },
{ {
skipDoc: true, skipDoc: true,
description: "Encode: YAML flow mapping stays inline", description: "Encode: YAML flow mapping produces table section (same as block mapping)",
input: "arg: {hello: foo}\n", input: "arg: {hello: foo}\n",
expected: "arg = { hello = \"foo\" }\n", expected: "[arg]\nhello = \"foo\"\n",
scenarioType: "encode", scenarioType: "encode",
}, },
{
skipDoc: true,
description: "Issue: JSON auto-detected via YAML decoder produces table sections",
input: `{"arg":{"hello": "foo"}}`,
expected: "[arg]\nhello = \"foo\"\n",
scenarioType: "encode",
},
{
skipDoc: true,
description: "Issue: JSON via JSON decoder produces table sections",
input: `{"arg":{"hello": "foo"}}`,
expected: "[arg]\nhello = \"foo\"\n",
scenarioType: "encode-json",
},
{
skipDoc: true,
description: "Issue 2688: inline table arrays do not change following table scope",
input: issue2688SampleToml,
expression: `.project.version = "0.5.2"`,
expected: issue2688SampleExpected,
scenarioType: "roundtrip",
},
{ {
skipDoc: true, skipDoc: true,
description: "Roundtrip: key with special characters in inline table", description: "Roundtrip: key with special characters in inline table",
@ -695,6 +791,8 @@ func testTomlScenario(t *testing.T, s formatScenario) {
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewTomlDecoder(), NewTomlEncoder()), s.description) test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewTomlDecoder(), NewTomlEncoder()), s.description)
case "encode": case "encode":
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewTomlEncoder()), s.description) test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewTomlEncoder()), s.description)
case "encode-json":
test.AssertResultWithContext(t, s.expected, mustProcessFormatScenario(s, NewJSONDecoder(), NewTomlEncoder()), s.description)
} }
} }
@ -775,7 +873,7 @@ func documentTomlScenario(_ *testing.T, w *bufio.Writer, i interface{}) {
documentTomlDecodeScenario(w, s) documentTomlDecodeScenario(w, s)
case "roundtrip": case "roundtrip":
documentTomlRoundtripScenario(w, s) documentTomlRoundtripScenario(w, s)
case "encode": case "encode", "encode-json":
documentTomlEncodeScenario(w, s) documentTomlEncodeScenario(w, s)
default: default:
@ -794,6 +892,60 @@ func TestTomlScenarios(t *testing.T) {
documentScenarios(t, "usage", "toml", genericScenarios, documentTomlScenario) documentScenarios(t, "usage", "toml", genericScenarios, documentTomlScenario)
} }
func TestTomlEncodeJsonKeepsRootArrayBeforeTables(t *testing.T) {
scenario := formatScenario{
description: "Encode: JSON root array stays outside later tables",
input: `{
"_source": {
"cookie": [
{
"Domain": "",
"Expires": "0001-01-01T00:00:00Z",
"HttpOnly": false,
"MaxAge": 0,
"Name": "name",
"Path": "",
"Raw": "",
"RawExpires": "",
"SameSite": 0,
"Secure": false,
"Unparsed": null,
"Value": "value"
}
]
},
"highlight": {
"did": [
"did"
]
},
"sort": [
1
]
}`,
expected: `sort = [1]
[[_source.cookie]]
Domain = ""
Expires = "0001-01-01T00:00:00Z"
HttpOnly = false
MaxAge = 0
Name = "name"
Path = ""
Raw = ""
RawExpires = ""
SameSite = 0
Secure = false
Value = "value"
[highlight]
did = ["did"]
`,
}
test.AssertResultWithContext(t, scenario.expected, mustProcessFormatScenario(scenario, NewJSONDecoder(), NewTomlEncoder()), scenario.description)
}
// TestTomlColourization tests that colourization correctly distinguishes // TestTomlColourization tests that colourization correctly distinguishes
// between table section headers and inline arrays // between table section headers and inline arrays
func TestTomlColourization(t *testing.T) { func TestTomlColourization(t *testing.T) {

View File

@ -1,306 +0,0 @@
abxbbxdbxebxczzx
abxbbxdbxebxczzy
accum
Accum
adithyasunil
AEDT
água
ÁGUA
alecthomas
appleapple
Astuff
autocreating
autoparse
AWST
axbxcxdxe
axbxcxdxexxx
bananabanana
barp
nbaz
bitnami
blarp
blddir
Bobo
BODMAS
bonapite
Brien
Bstuff
BUILDKIT
buildpackage
catmeow
CATYPE
CBVVE
chardata
chillum
choco
chomper
cleanup
cmlu
colorise
colors
Colors
colourize
compinit
coolioo
coverprofile
createmap
csvd
CSVUTF
currentlabel
cygpath
czvf
datestring
datetime
Datetime
datetimes
DEBEMAIL
debhelper
Debugf
debuild
delish
delpaths
DELPATHS
devorbitus
devscripts
dimchansky
Dont
dput
elliotchance
endhint
endofname
Entriesfrom
envsubst
errorlevel
Escandón
Evalall
fakefilename
fakeroot
Farah
fatih
Fifi
filebytes
Fileish
foobar
foobaz
foof
frood
fullpath
gitbook
githubactions
gnupg
goccy
gofmt
gogo
golangci
goreleaser
GORELEASER
GOMODCACHE
GOPATH
gosec
gota
goversion
GOVERSION
haha
hellno
herbygillot
hexdump
Hoang
hostpath
hotdog
howdy
incase
Infof
inlinetables
inplace
ints
ireduce
iwatch
jinzhu
jq's
jsond
keygrip
Keygrip
KEYGRIP
KEYID
keyvalue
kwak
lalilu
ldflags
LDFLAGS
lexer
Lexer
libdistro
lindex
linecomment
LVAs
magiconair
mapvalues
Mier
mikefarah
minideb
minishift
mipsle
mitchellh
mktemp
Mult
multidoc
multimaint
myenv
myenvnonexisting
myfile
myformat
ndjson
NDJSON
NFKD
nixpkgs
nojson
nonascii
nonempty
noninteractive
Nonquoting
nosec
notoml
noxml
nolua
nullinput
onea
Oneshot
opencollect
opstack
orderedmap
osarch
overridign
pacman
Padder
pandoc
parsechangelog
pcsv
pelletier
pflag
prechecking
Prerelease
proc
propsd
qylib
readline
realnames
realpath
repr
rhash
rindex
risentveber
rmescandon
Rosey
roundtrip
roundtrips
Roundtrip
roundtripping
Interp
interp
runningvms
sadface
selfupdate
setpath
sharedfolder
Sharedfolder
shellvariables
shellvars
shortfunc
shortpipe
shunit
snapcraft
somevalue
splt
srcdir
stackoverflow
stiched
Strc
strenv
strload
stylig
subarray
subchild
subdescription
submatch
submatches
SUBSTR
tempfile
tfstate
Tfstate
thar
timezone
Timezone
timezones
Timezones
tojson
Tokenvalue
tsvd
Tuan
tzdata
Uhoh
updateassign
urid
utfbom
Warningf
Wazowski
webi
Webi
wherever
winget
withdots
wizz
woop
workdir
Writable
xmld
xyzzy
yamld
yqlib
yuin
zabbix
tonumber
noyaml
nolint
shortfile
Unmarshalling
noini
nocsv
nobase64
nouri
noprops
nosh
noshell
tinygo
nonexistent
hclsyntax
hclwrite
nohcl
zclconf
cty
go-cty
Colorisation
goimports
errorlint
RDBMS
expeñded
bananabananabananabanana
edwinjhlee
flox
unlabelled
kyaml
KYAML
nokyaml
buildvcs
behaviour
GOFLAGS
gocache
subsubarray
Ffile
Fquery
coverpkg
gsub
ralia
Austr
ustrali
héllo
alia

View File

@ -8,15 +8,22 @@
- git push --tags - git push --tags
- use github actions to publish docker and make github release - use github actions to publish docker and make github release
- check github updated yq action in marketplace - check github updated yq action in marketplace
- update github-action/Dockerfile to pin the newly published docker image digest: - update github-action/Dockerfile to pin the newly published docker image digest (must match the mikefarah/yq:4 manifest digest):
skopeo inspect docker://docker.io/mikefarah/yq:4 | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['Digest'])" docker buildx imagetools inspect docker.io/mikefarah/yq:4 --format '{{printf "%s" .Manifest.Digest}}'
then update the FROM line in github-action/Dockerfile with the new digest: then update the FROM line in github-action/Dockerfile with the new digest:
FROM mikefarah/yq:4@sha256:<new-digest> FROM mikefarah/yq:4@<digest-from-above>
- commit the Dockerfile change, then manually run the "Release Docker GitHub Action" workflow
(Actions -> Release Docker GitHub Action -> Run workflow)
- update action.yml to pin the newly published github-action image digest (must match the mikefarah/yq:4-githubaction manifest digest):
docker buildx imagetools inspect docker.io/mikefarah/yq:4-githubaction --format '{{printf "%s" .Manifest.Digest}}'
then update the image line in action.yml with the new digest:
image: 'docker://mikefarah/yq:4-githubaction@<digest-from-above>'
- commit the action.yml change and push
// release artifacts are signed with cosign keyless signing (Sigstore) // release artifacts are signed with cosign keyless signing (Sigstore)
// users can verify with: // users can verify with:
// cosign verify-blob --bundle checksums.bundle checksums // cosign verify-blob --bundle checksums.bundle checksums
// install cosign: brew install cosign OR go install github.com/sigstore/cosign/v2/cmd/cosign@latest // install cosign: brew install cosign OR go install github.com/sigstore/cosign/v2/cmd/cosign@v2.6.1
- snapcraft - snapcraft

View File

@ -1,3 +1,34 @@
4.53.3:
- Add `--ini-preserve-quotes` flag for INI round-trip quote preservation (#2728) Thanks @toller892!
- Fix: reset INI decoder state on init (#2719) Thanks @xieby1!
- Fix: decode properties array bracket paths (#2693) Thanks @cyphercodes!
- Fix: preserve floats with trailing zero when encoding YAML to JSON (#2701) Thanks @ChrisJr404!
- Fix: JSON to TOML root scope and null handling (#2689) Thanks @LovesAsuna!
- Fix: reset TOML decoder finished flag on Init for multi-doc evaluation (#2704) Thanks @terminalchai!
- Fix: reset TOML decoder between files when evaluating all at once (#2685) Thanks @terminalchai!
- Fix: preserve TOML inline table array scope (#2694) Thanks @cyphercodes!
- Fix: preserve empty TOML arrays in tables (#2686) Thanks @cyphercodes!
- Fix: TOML encoder uses inline tables for YAML FlowStyle mappings (#2687)
- Fix nested inline YAML merge explode (#2699) Thanks @cyphercodes!
- Fix repeatString overflow test on 32-bit platforms (#2680) Thanks @jandubois!
- Bumped dependencies
4.53.2:
- Fixing release process
4.53.1:
- Releases and tags now signed and immutable!
- Add system(command; args) operator (disabled by default) (#2640)
- TOML encoder: prefer readable table sections over inline tables (#2649)
- Fix TOML encoder to quote keys containing special characters (#2648)
- Add string slicing support (#2639)
- Fix findInArray misuse on MappingNodes in equality and contains (#2645) Thanks @jandubois!
- Fix panic on negative slice indices that underflow after adjustment (#2646) Thanks @jandubois!
- Fix stack overflow from circular alias in traverse (#2647) Thanks @jandubois!
- Fix panic and OOM in repeatString for large repeat counts (#2644) Thanks @jandubois!
- Bumped dependencies
4.52.5: 4.52.5:
- Fix: reset TOML decoder state between files (#2634) thanks @terminalchai - Fix: reset TOML decoder state between files (#2634) thanks @terminalchai
- Fix: preserve original filename when using --front-matter (#2613) thanks @cobyfrombrooklyn-bot - Fix: preserve original filename when using --front-matter (#2613) thanks @cobyfrombrooklyn-bot

View File

@ -8,14 +8,17 @@ fi
version=$1 version=$1
# validate version is in the right format # validate version is in the right format (bash regex — portable; GNU sed's q1 is not on macOS)
echo $version | sed -r '/v4\.[0-9][0-9]\.[0-9][0-9]?$/!{q1}' if [[ ! $version =~ ^v4\.[0-9][0-9]\.[0-9][0-9]?$ ]]; then
echo "Please specify a valid version (e.g. v4.53.3)"
exit 1
fi
previousVersion=$(cat cmd/version.go| sed -n 's/.*Version = "\([^"]*\)"/\1/p') previousVersion=$(cat cmd/version.go| sed -n 's/.*Version = "\([^"]*\)"/\1/p')
echo "Updating from $previousVersion to $version" echo "Updating from $previousVersion to $version"
sed -i "s/\(.*Version =\).*/\1 \"$version\"/" cmd/version.go sed "s/\(.*Version =\).*/\1 \"$version\"/" cmd/version.go > cmd/version.go.tmp && mv cmd/version.go.tmp cmd/version.go
go build . go build .
actualVersion=$(./yq --version) actualVersion=$(./yq --version)
@ -49,5 +52,5 @@ fi
git add cmd/version.go snap/snapcraft.yaml git add cmd/version.go snap/snapcraft.yaml
git commit -m 'Bumping version' git commit -m 'Bumping version'
git tag $version git tag $version -m "releasing"
git tag -f v4 git tag -f v4 -m "releasing $version"

View File

@ -1,5 +1,50 @@
#!/bin/sh #!/bin/sh
set -ex set -ex
go mod download golang.org/x/tools@latest go mod download golang.org/x/tools@v0.44.0
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.11.3 curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/6008b81b81c690c046ffc3fd5bce896da715d5fd/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.11.3
curl -sSfL https://raw.githubusercontent.com/securego/gosec/master/install.sh | sh -s v2.22.11 curl -sSfL https://raw.githubusercontent.com/securego/gosec/424fc4cd9c82ea0fd6bee9cd49c2db2c3cc0c93f/install.sh | sh -s v2.22.11
TYPOS_VERSION=v1.47.2
TYPOS_OS=$(uname -s | tr '[:upper:]' '[:lower:]')
TYPOS_ARCH=$(uname -m)
case "${TYPOS_ARCH}" in
x86_64) TYPOS_ARCH=x86_64 ;;
aarch64|arm64) TYPOS_ARCH=aarch64 ;;
*)
echo "unsupported architecture for typos: ${TYPOS_ARCH}"
exit 1
;;
esac
case "${TYPOS_OS}" in
linux) TYPOS_TARGET="${TYPOS_ARCH}-unknown-linux-musl" ;;
darwin) TYPOS_TARGET="${TYPOS_ARCH}-apple-darwin" ;;
*)
echo "unsupported OS for typos: ${TYPOS_OS}"
exit 1
;;
esac
TYPOS_ARCHIVE="typos-${TYPOS_VERSION}-${TYPOS_TARGET}.tar.gz"
TYPOS_URL="https://github.com/crate-ci/typos/releases/download/${TYPOS_VERSION}/${TYPOS_ARCHIVE}"
case "${TYPOS_TARGET}" in
aarch64-apple-darwin) TYPOS_SHA256=23ca24a9186b5cb395b5f6c8eea8cdb02911c8980833e016454b56e90c3bd474 ;;
aarch64-unknown-linux-musl) TYPOS_SHA256=596d5c6b9ecf34307f68bea649178c5b45a4398fe3a1fcef9598e85aa2ccb742 ;;
x86_64-apple-darwin) TYPOS_SHA256=469a2d9fc894b0cdcec6e4fa3719b4c4638e195feee6517d4845450f8e8985c6 ;;
x86_64-unknown-linux-musl) TYPOS_SHA256=7aef58932fc123b4cf4b40d86468e89a3297d80169051d7cfd13a235e05fc426 ;;
*)
echo "unsupported typos target: ${TYPOS_TARGET}"
exit 1
;;
esac
TYPOS_TMPDIR=$(mktemp -d)
curl -sSfL "${TYPOS_URL}" -o "${TYPOS_TMPDIR}/${TYPOS_ARCHIVE}"
TYPOS_ACTUAL_SHA256=$(sha256sum "${TYPOS_TMPDIR}/${TYPOS_ARCHIVE}" 2>/dev/null | cut -d' ' -f1)
if [ -z "${TYPOS_ACTUAL_SHA256}" ]; then
TYPOS_ACTUAL_SHA256=$(shasum -a 256 "${TYPOS_TMPDIR}/${TYPOS_ARCHIVE}" | cut -d' ' -f1)
fi
if [ "${TYPOS_ACTUAL_SHA256}" != "${TYPOS_SHA256}" ]; then
echo "typos archive checksum mismatch: expected ${TYPOS_SHA256}, got ${TYPOS_ACTUAL_SHA256}"
exit 1
fi
tar xzf "${TYPOS_TMPDIR}/${TYPOS_ARCHIVE}" -C "${TYPOS_TMPDIR}"
install -m 755 "${TYPOS_TMPDIR}/typos" "$(go env GOPATH)/bin/typos"
rm -rf "${TYPOS_TMPDIR}"

View File

@ -1,3 +1,18 @@
#!/bin/bash #!/bin/bash
npx cspell --no-progress "**/*.{sh,go,md}" set -euo pipefail
GOPATH_TYPOS="$(go env GOPATH)/bin/typos"
TYPOS_CMD=""
if [ -f "${GOPATH_TYPOS}" ]; then
TYPOS_CMD="${GOPATH_TYPOS}"
elif command -v typos >/dev/null 2>&1; then
TYPOS_CMD="typos"
else
echo "Error: typos not found in $(go env GOPATH)/bin or PATH."
echo "Please run scripts/devtools.sh or ensure typos is installed correctly."
exit 1
fi
git ls-files '*.go' '*.sh' '*.md' | "${TYPOS_CMD}" --file-list -

View File

@ -1,3 +1,7 @@
#!/bin/bash #!/bin/bash
go test $(go list ./... | grep -v -E 'examples' | grep -v -E 'test') go test $(go list ./... | grep -v -E 'examples' | grep -v -E 'test')
# Run after the main test suite: TestGoInstallCompatibility zips the module tree and
# must not run in parallel with pkg/yqlib tests that rewrite doc/*.md files.
go test -tags goinstall -run TestGoInstallCompatibility .

View File

@ -2,9 +2,9 @@
set -eo pipefail set -eo pipefail
# You may need to go install github.com/goreleaser/goreleaser/v2@latest first # You may need to go install github.com/goreleaser/goreleaser/v2@v2.16.0 first
GORELEASER="goreleaser build --clean" GORELEASER="goreleaser build --clean"
if [ -z "$CI" ]; then if [ -z "$CI" ] || [[ "${GITHUB_REF_NAME:-}" == draft-* ]]; then
GORELEASER+=" --snapshot" GORELEASER+=" --snapshot"
fi fi

View File

@ -1,5 +1,5 @@
name: yq name: yq
version: 'v4.52.5' version: 'v4.53.3'
summary: A lightweight and portable command-line data file processor summary: A lightweight and portable command-line data file processor
description: | description: |
`yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml, json, xml, csv, properties and TOML files. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml, json, xml, csv, properties and TOML files.
@ -32,6 +32,6 @@ parts:
build-environment: build-environment:
- CGO_ENABLED: 0 - CGO_ENABLED: 0
source: https://github.com/mikefarah/yq.git source: https://github.com/mikefarah/yq.git
source-tag: v4.52.5 source-tag: v4.53.3
build-snaps: build-snaps:
- go/latest/stable - go/latest/stable