Compare commits

..

29 Commits

Author SHA1 Message Date
dependabot[bot]
3980b6cd82
Bump github.com/a8m/envsubst from 1.4.2 to 1.4.3
Bumps [github.com/a8m/envsubst](https://github.com/a8m/envsubst) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/a8m/envsubst/releases)
- [Commits](https://github.com/a8m/envsubst/compare/v1.4.2...v1.4.3)

---
updated-dependencies:
- dependency-name: github.com/a8m/envsubst
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-04 00:53:57 +00:00
Mike Farah
2bd08ea4e8 Dont panic on tests 2025-05-04 10:33:09 +10:00
Mike Farah
72fa3cca98 goccy decoder supports merge maps 2025-05-03 20:20:50 +10:00
Mike Farah
f89605e3c0 goccy yaml decoder supports anchor/aliases 2025-05-03 20:01:16 +10:00
Mike Farah
c59fa8de59 Bumping version 2025-05-03 16:40:24 +10:00
Mike Farah
ef8b520f89 Updating release notes 2025-05-03 16:39:36 +10:00
Mike Farah
20b5129120 Updating golanglint 2025-05-03 16:34:21 +10:00
Mike Farah
89518a09b8 Bumping golint 2025-05-03 15:58:14 +10:00
Mike Farah
d01ac7801d Regen doc 2025-05-03 11:37:52 +10:00
Mike Farah
67520b5e7b Adding windows arm build #2344 2025-05-03 11:37:52 +10:00
Romain Lebrun Thauront
7b6e61ba2c Update README.md 2025-05-03 11:36:12 +10:00
Romain Lebrun Thauront
cf8621d9e3 Install to /usr/local in README install instruction 2025-05-03 11:36:12 +10:00
dependabot[bot]
340b897252 Bump github.com/spf13/cobra from 1.8.1 to 1.9.1
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.8.1 to 1.9.1.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.8.1...v1.9.1)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-03 11:28:22 +10:00
n4zukker
4155c99a12 Typo in cats example
The quotation mark and closing parenthesis were swapped.

Before:
```
$ echo "Hello kitty" | yq eval 'test("(?i)cats)"'
Error: bad expression - probably missing close bracket on TEST
```

After:
```
$ echo "Hello kitty" | yq eval 'test("(?i)cats")'
false
```
2025-05-03 11:27:51 +10:00
dependabot[bot]
b26de94758 Bump github.com/magiconair/properties from 1.8.9 to 1.8.10
Bumps [github.com/magiconair/properties](https://github.com/magiconair/properties) from 1.8.9 to 1.8.10.
- [Release notes](https://github.com/magiconair/properties/releases)
- [Commits](https://github.com/magiconair/properties/compare/v1.8.9...v1.8.10)

---
updated-dependencies:
- dependency-name: github.com/magiconair/properties
  dependency-version: 1.8.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-03 11:24:22 +10:00
dependabot[bot]
7dc5efee62 Bump golang.org/x/net from 0.34.0 to 0.39.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.34.0 to 0.39.0.
- [Commits](https://github.com/golang/net/compare/v0.34.0...v0.39.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-03 11:24:12 +10:00
Alberto Cavalcante
2cb7498444 chore(release): add windows arm64 binary target for release 2025-05-03 11:23:55 +10:00
Alberto Cavalcante
649c9a083a chore(check): use golangci-lint from GOPATH/bin as first option 2025-05-03 11:23:55 +10:00
Alberto Cavalcante
074d785396 chore(devtools): use curl instead of wget for better compatibility with MacOS 2025-05-03 11:23:55 +10:00
dependabot[bot]
de2f77b49c Bump github.com/alecthomas/participle/v2 from 2.1.1 to 2.1.4
Bumps [github.com/alecthomas/participle/v2](https://github.com/alecthomas/participle) from 2.1.1 to 2.1.4.
- [Release notes](https://github.com/alecthomas/participle/releases)
- [Changelog](https://github.com/alecthomas/participle/blob/master/CHANGES.md)
- [Commits](https://github.com/alecthomas/participle/compare/v2.1.1...v2.1.4)

---
updated-dependencies:
- dependency-name: github.com/alecthomas/participle/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-05 21:36:59 +11:00
Ashok Pariya
fe06096514 Add s390x platform support to Docker release workflow
Updated the list of supported platforms in the GitHub Actions
Docker release workflow to include linux/s390x.

Signed-off-by: Ashok Pariya <ashok.pariya@ibm.com>
2025-04-05 20:18:39 +11:00
dependabot[bot]
40f2c2381b Bump golang from 1.24.1 to 1.24.2
Bumps golang from 1.24.1 to 1.24.2.

---
updated-dependencies:
- dependency-name: golang
  dependency-version: 1.24.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-05 20:17:56 +11:00
dependabot[bot]
64bee0fbe3 Bump golang from 1.24.0 to 1.24.1
Bumps golang from 1.24.0 to 1.24.1.

---
updated-dependencies:
- dependency-name: golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-25 15:45:53 +11:00
dependabot[bot]
6e6575d5b0 Bump golang.org/x/text from 0.22.0 to 0.23.0
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.22.0 to 0.23.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-25 15:45:42 +11:00
Oleksandr Redko
ce2ce51236 Remove mistakenly committed cover.out 2025-03-25 15:45:27 +11:00
Zoltán Reegn
c9766c1cab Also push docker images to ghcr.io
With docker-hub starting to introduce a much stricter limit on anonymous
pulls, it makes sense to also host the image in ghcr.io as well,
allowing users flexibility in where they pull the docker images from.

Also with the github action it makes more sense hosting the docker image
on github infrastructure.

I've introduced a github action for logging into registries as well.
2025-03-25 15:45:15 +11:00
Mike Farah
337960a6d1 Fixing add when there is no node match #2325 2025-03-25 15:35:16 +11:00
damuzhi0810
1395d6e230 refactor: using slices.Contains to simplify the code
Signed-off-by: damuzhi0810 <rust@before.tech>
2025-03-25 15:10:25 +11:00
Mike Farah
ba1b9737b6 Added recursive update map keys example 2025-03-03 15:10:42 +11:00
56 changed files with 587 additions and 3414 deletions

View File

@ -30,6 +30,19 @@ jobs:
- name: Available platforms
run: echo ${{ steps.buildx.outputs.platforms }} && docker version
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push image
run: |
echo "GithubRef: ${GITHUB_REF}"
@ -38,10 +51,9 @@ jobs:
IMAGE_VERSION=${VERSION:1}
echo "IMAGE_VERSION: ${IMAGE_VERSION}"
PLATFORMS="linux/amd64,linux/ppc64le,linux/arm64,linux/arm/v7"
PLATFORMS="linux/amd64,linux/ppc64le,linux/arm64,linux/arm/v7,linux/s390x"
echo "Building and pushing version ${IMAGE_VERSION} of image ${IMAGE_NAME}"
echo '${{ secrets.DOCKER_PASSWORD }}' | docker login -u '${{ secrets.DOCKER_USERNAME }}' --password-stdin
docker buildx build \
--label "org.opencontainers.image.authors=https://github.com/mikefarah/yq/graphs/contributors" \
--label "org.opencontainers.image.created=$(date --rfc-3339=seconds)" \
@ -59,6 +71,9 @@ jobs:
-t "${IMAGE_NAME}:${IMAGE_VERSION}" \
-t "${IMAGE_NAME}:4" \
-t "${IMAGE_NAME}:latest" \
-t "ghcr.io/${IMAGE_NAME}:${IMAGE_VERSION}" \
-t "ghcr.io/${IMAGE_NAME}:4" \
-t "ghcr.io/${IMAGE_NAME}:latest" \
.
cd github-action
@ -79,4 +94,7 @@ jobs:
-t "${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "${IMAGE_NAME}:4-githubaction" \
-t "${IMAGE_NAME}:latest-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "ghrc.io/${IMAGE_NAME}:4-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:latest-githubaction" \
.

1
.gitignore vendored
View File

@ -22,6 +22,7 @@ _cgo_gotypes.go
_cgo_export.*
_testmain.go
cover.out
coverage.out
coverage.html
*.exe

38
.golangci.bck.yml Normal file
View File

@ -0,0 +1,38 @@
run:
timeout: 5m
linters:
enable:
- asciicheck
- depguard
- errorlint
- gci
- gochecknoinits
- gofmt
- goimports
- gosec
- gosimple
- staticcheck
- unused
- misspell
- nakedret
- nolintlint
- predeclared
- revive
- unconvert
- unparam
linters-settings:
depguard:
rules:
prevent_unmaintained_packages:
list-mode: lax
files:
- $all
- "!$test"
deny:
- pkg: io/ioutil
desc: "replaced by io and os packages since Go 1.16: https://tip.golang.org/doc/go1.16#ioutil"
issues:
exclude-rules:
- linters:
- revive
text: "var-naming"

View File

@ -1,18 +1,11 @@
run:
timeout: 5m
version: "2"
linters:
enable:
- asciicheck
- depguard
- errorlint
- gci
- gochecknoinits
- gofmt
- goimports
- gosec
- gosimple
- staticcheck
- unused
- misspell
- nakedret
- nolintlint
@ -20,19 +13,40 @@ linters:
- revive
- unconvert
- unparam
linters-settings:
depguard:
settings:
depguard:
rules:
prevent_unmaintained_packages:
list-mode: lax
files:
- $all
- '!$test'
deny:
- pkg: io/ioutil
desc: 'replaced by io and os packages since Go 1.16: https://tip.golang.org/doc/go1.16#ioutil'
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
rules:
prevent_unmaintained_packages:
list-mode: lax
files:
- $all
- "!$test"
deny:
- pkg: io/ioutil
desc: "replaced by io and os packages since Go 1.16: https://tip.golang.org/doc/go1.16#ioutil"
issues:
exclude-rules:
- linters:
- revive
text: "var-naming"
- linters:
- revive
text: var-naming
paths:
- third_party$
- builtin$
- examples$
formatters:
enable:
- gci
- gofmt
- goimports
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$

View File

@ -38,6 +38,8 @@ builds:
- openbsd_amd64
- windows_386
- windows_amd64
- windows_arm
- windows_arm64
no_unique_dist_dir: true

View File

@ -1,4 +1,4 @@
FROM golang:1.24.0 AS builder
FROM golang:1.24.2 AS builder
WORKDIR /go/src/mikefarah/yq

View File

@ -1,4 +1,4 @@
FROM golang:1.24.0
FROM golang:1.24.2
RUN apt-get update && \
apt-get install -y npm && \

View File

@ -76,21 +76,21 @@ For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64
#### Compressed via tar.gz
```bash
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
tar xz && mv ${BINARY} /usr/bin/yq
tar xz && mv ${BINARY} /usr/local/bin/yq
```
#### Plain binary
```bash
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
chmod +x /usr/bin/yq
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/local/bin/yq &&\
chmod +x /usr/local/bin/yq
```
#### Latest version
```bash
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq &&\
chmod +x /usr/bin/yq
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq &&\
chmod +x /usr/local/bin/yq
```
### MacOS / Linux via Homebrew:

View File

@ -3,7 +3,7 @@
testLoadFileNotExist() {
result=$(./yq e -n 'load("cat.yml")' 2>&1)
assertEquals 1 $?
assertEquals "Error: Failed to load cat.yml: open cat.yml: no such file or directory" "$result"
assertEquals "Error: failed to load cat.yml: open cat.yml: no such file or directory" "$result"
}
testLoadFileExpNotExist() {
@ -15,7 +15,7 @@ testLoadFileExpNotExist() {
testStrLoadFileNotExist() {
result=$(./yq e -n 'strload("cat.yml")' 2>&1)
assertEquals 1 $?
assertEquals "Error: Failed to load cat.yml: open cat.yml: no such file or directory" "$result"
assertEquals "Error: failed to load cat.yml: open cat.yml: no such file or directory" "$result"
}
testStrLoadFileExpNotExist() {

View File

@ -11,7 +11,7 @@ var (
GitDescribe string
// Version is main version number that is being run at the moment.
Version = "v4.45.1"
Version = "v4.45.2"
// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
// then it means that it is a final release. Otherwise, this is a pre-release
@ -45,5 +45,5 @@ func getHumanVersion() string {
}
// Strip off any single quotes added by the git information.
return strings.Replace(version, "'", "", -1)
return strings.ReplaceAll(version, "'", "")
}

18
go.mod
View File

@ -2,32 +2,32 @@ module github.com/mikefarah/yq/v4
require (
github.com/a8m/envsubst v1.4.3
github.com/alecthomas/participle/v2 v2.1.1
github.com/alecthomas/participle/v2 v2.1.4
github.com/alecthomas/repr v0.4.0
github.com/dimchansky/utfbom v1.1.1
github.com/elliotchance/orderedmap v1.8.0
github.com/fatih/color v1.18.0
github.com/goccy/go-json v0.10.5
github.com/goccy/go-yaml v1.13.3
github.com/goccy/go-yaml v1.17.1
github.com/jinzhu/copier v0.4.0
github.com/magiconair/properties v1.8.9
github.com/magiconair/properties v1.8.10
github.com/pelletier/go-toml/v2 v2.2.3
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e
github.com/spf13/cobra v1.8.1
github.com/spf13/cobra v1.9.1
github.com/spf13/pflag v1.0.6
github.com/yuin/gopher-lua v1.1.1
golang.org/x/net v0.34.0
golang.org/x/text v0.22.0
golang.org/x/net v0.39.0
golang.org/x/text v0.24.0
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
golang.org/x/sys v0.29.0 // indirect
golang.org/x/sys v0.32.0 // indirect
)
go 1.23.0
toolchain go1.24.0
toolchain go1.24.1

55
go.sum
View File

@ -1,12 +1,12 @@
github.com/a8m/envsubst v1.4.3 h1:kDF7paGK8QACWYaQo6KtyYBozY2jhQrTuNNuUxQkhJY=
github.com/a8m/envsubst v1.4.3/go.mod h1:4jjHWQlZoaXPoLQUb7H2qT4iLkZDdmEQiOUogdUmqVU=
github.com/alecthomas/assert/v2 v2.3.0 h1:mAsH2wmvjsuvyBvAmCtm7zFsBlb8mIHx5ySLVdDZXL0=
github.com/alecthomas/assert/v2 v2.3.0/go.mod h1:pXcQ2Asjp247dahGEmsZ6ru0UVwnkhktn7S0bBDLxvQ=
github.com/alecthomas/participle/v2 v2.1.1 h1:hrjKESvSqGHzRb4yW1ciisFJ4p3MGYih6icjJvbsmV8=
github.com/alecthomas/participle/v2 v2.1.1/go.mod h1:Y1+hAs8DHPmc3YUFzqllV+eSQ9ljPTk0ZkPMtEdAx2c=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/participle/v2 v2.1.4 h1:W/H79S8Sat/krZ3el6sQMvMaahJ+XcM9WSI2naI7w2U=
github.com/alecthomas/participle/v2 v2.1.4/go.mod h1:8tqVbpTX20Ru4NfYQgZf4mP18eXPTBViyMWiArNEgGI=
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@ -16,33 +16,20 @@ github.com/elliotchance/orderedmap v1.8.0 h1:TrOREecvh3JbS+NCgwposXG5ZTFHtEsQiCG
github.com/elliotchance/orderedmap v1.8.0/go.mod h1:wsDwEaX5jEoyhbs7x93zk2H/qv0zwuhg4inXhDkYqys=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.22.1 h1:40JcKH+bBNGFczGuoBYgX4I6m/i27HYW8P9FDk5PbgA=
github.com/go-playground/validator/v10 v10.22.1/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.13.3 h1:IXRULR8mAa0MXQobzzp0VOfMUJ8EnaQ4x3jhf7S0/nI=
github.com/goccy/go-yaml v1.13.3/go.mod h1:IjYwxUiJDoqpx2RmbdjMUceGHZwYLon3sfOGl5Hi9lc=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/goccy/go-yaml v1.17.1 h1:LI34wktB2xEE3ONG/2Ar54+/HJVBriAGJ55PHls4YuY=
github.com/goccy/go-yaml v1.17.1/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jinzhu/copier v0.4.0 h1:w3ciUoD19shMCRargcpm0cm91ytaBhDvuRpz1ODO/U8=
github.com/jinzhu/copier v0.4.0/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/magiconair/properties v1.8.9 h1:nWcCbLq1N2v/cpNsy5WvQ37Fb+YElfq20WJ/a8RkpQM=
github.com/magiconair/properties v1.8.9/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
@ -52,9 +39,8 @@ github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsK
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@ -63,16 +49,13 @@ github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsT
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
golang.org/x/crypto v0.32.0 h1:euUpcYgM8WcP71gNpTqQCn6rC2t6ULUPiOzfWaXVVfc=
golang.org/x/crypto v0.32.0/go.mod h1:ZnnJkOaASj8g0AjIduWNlq2NRxL0PlBrbKVyZ6V/Ugc=
golang.org/x/net v0.34.0 h1:Mb7Mrk043xzHgnRM88suvJFwzVrRfHEHJEl5/71CKw0=
golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.22.0 h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=
golang.org/x/text v0.22.0/go.mod h1:YRoo4H8PVmsu+E3Ou7cqLVH8oXWIHVoX0jqUWALQhfY=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473 h1:6D+BvnJ/j6e222UW8s2qTSe3wGBtvo0MbVQG/c5k8RE=

View File

@ -201,13 +201,14 @@ func (n *CandidateNode) SetParent(parent *CandidateNode) {
type ValueVisitor func(*CandidateNode) error
func (n *CandidateNode) VisitValues(visitor ValueVisitor) error {
if n.Kind == MappingNode {
switch n.Kind {
case MappingNode:
for i := 1; i < len(n.Content); i = i + 2 {
if err := visitor(n.Content[i]); err != nil {
return err
}
}
} else if n.Kind == SequenceNode {
case SequenceNode:
for i := 0; i < len(n.Content); i = i + 1 {
if err := visitor(n.Content[i]); err != nil {
return err

View File

@ -9,14 +9,14 @@ import (
goccyToken "github.com/goccy/go-yaml/token"
)
func (o *CandidateNode) goccyDecodeIntoChild(childNode ast.Node, cm yaml.CommentMap) (*CandidateNode, error) {
func (o *CandidateNode) goccyDecodeIntoChild(childNode ast.Node, cm yaml.CommentMap, anchorMap map[string]*CandidateNode) (*CandidateNode, error) {
newChild := o.CreateChild()
err := newChild.UnmarshalGoccyYAML(childNode, cm)
err := newChild.UnmarshalGoccyYAML(childNode, cm, anchorMap)
return newChild, err
}
func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap) error {
func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap, anchorMap map[string]*CandidateNode) error {
log.Debugf("UnmarshalYAML %v", node)
log.Debugf("UnmarshalYAML %v", node.Type().String())
log.Debugf("UnmarshalYAML Node Value: %v", node.String())
@ -62,6 +62,7 @@ func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap) er
o.Kind = ScalarNode
o.Tag = "!!bool"
case ast.NullType:
log.Debugf("its a null type with value %v", node.GetToken().Value)
o.Kind = ScalarNode
o.Tag = "!!null"
o.Value = node.GetToken().Value
@ -92,7 +93,7 @@ func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap) er
// to solve the multiline > problem
o.Value = astLiteral.Value.Value
case ast.TagType:
if err := o.UnmarshalGoccyYAML(node.(*ast.TagNode).Value, cm); err != nil {
if err := o.UnmarshalGoccyYAML(node.(*ast.TagNode).Value, cm, anchorMap); err != nil {
return err
}
o.Tag = node.(*ast.TagNode).Start.Value
@ -106,7 +107,7 @@ func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap) er
o.Style = FlowStyle
}
for _, mappingValueNode := range mappingNode.Values {
err := o.goccyProcessMappingValueNode(mappingValueNode, cm)
err := o.goccyProcessMappingValueNode(mappingValueNode, cm, anchorMap)
if err != nil {
return ast.ErrInvalidAnchorName
}
@ -120,7 +121,7 @@ func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap) er
o.Kind = MappingNode
o.Tag = "!!map"
mappingValueNode := node.(*ast.MappingValueNode)
err := o.goccyProcessMappingValueNode(mappingValueNode, cm)
err := o.goccyProcessMappingValueNode(mappingValueNode, cm, anchorMap)
if err != nil {
return ast.ErrInvalidAnchorName
}
@ -141,7 +142,7 @@ func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap) er
keyNode.Kind = ScalarNode
keyNode.Value = fmt.Sprintf("%v", i)
valueNode, err := o.goccyDecodeIntoChild(astSeq[i], cm)
valueNode, err := o.goccyDecodeIntoChild(astSeq[i], cm, anchorMap)
if err != nil {
return err
}
@ -149,24 +150,45 @@ func (o *CandidateNode) UnmarshalGoccyYAML(node ast.Node, cm yaml.CommentMap) er
valueNode.Key = keyNode
o.Content[i] = valueNode
}
case ast.AnchorType:
log.Debugf("UnmarshalYAML - an anchor node")
anchorNode := node.(*ast.AnchorNode)
err := o.UnmarshalGoccyYAML(anchorNode.Value, cm, anchorMap)
if err != nil {
return err
}
o.Anchor = anchorNode.Name.String()
anchorMap[o.Anchor] = o
case ast.AliasType:
log.Debugf("UnmarshalYAML - an alias node")
aliasNode := node.(*ast.AliasNode)
o.Kind = AliasNode
o.Value = aliasNode.Value.String()
o.Alias = anchorMap[o.Value]
case ast.MergeKeyType:
log.Debugf("UnmarshalYAML - a merge key")
o.Kind = ScalarNode
o.Value = "<<"
default:
log.Debugf("UnmarshalYAML - node idea of the type!!")
log.Debugf("UnmarshalYAML - no idea of the type!!\n%v: %v", node.Type(), node.String())
}
log.Debugf("KIND: %v", o.Kind)
return nil
}
func (o *CandidateNode) goccyProcessMappingValueNode(mappingEntry *ast.MappingValueNode, cm yaml.CommentMap) error {
func (o *CandidateNode) goccyProcessMappingValueNode(mappingEntry *ast.MappingValueNode, cm yaml.CommentMap, anchorMap map[string]*CandidateNode) error {
log.Debug("UnmarshalYAML MAP KEY entry %v", mappingEntry.Key)
keyNode, err := o.goccyDecodeIntoChild(mappingEntry.Key, cm)
keyNode, err := o.goccyDecodeIntoChild(mappingEntry.Key, cm, anchorMap)
if err != nil {
return err
}
keyNode.IsMapKey = true
log.Debug("UnmarshalYAML MAP VALUE entry %v", mappingEntry.Value)
valueNode, err := o.goccyDecodeIntoChild(mappingEntry.Value, cm)
valueNode, err := o.goccyDecodeIntoChild(mappingEntry.Value, cm, anchorMap)
if err != nil {
return err
}

View File

@ -78,9 +78,6 @@ func (o *CandidateNode) copyToYamlNode(node *yaml.Node) {
node.Value = o.Value
node.Anchor = o.Anchor
// node.Alias = TODO - find Alias in our own structure
// might need to be a post process thing
node.HeadComment = o.HeadComment
node.LineComment = o.LineComment

File diff suppressed because it is too large Load Diff

View File

@ -64,6 +64,6 @@ func (d *dataTreeNavigator) GetMatchingNodes(context Context, expressionNode *Ex
if handler != nil {
return handler(d, context, expressionNode)
}
return Context{}, fmt.Errorf("Unknown operator %v", expressionNode.Operation.OperationType)
return Context{}, fmt.Errorf("unknown operator %v", expressionNode.Operation.OperationType)
}

View File

@ -16,6 +16,8 @@ import (
type goccyYamlDecoder struct {
decoder yaml.Decoder
cm yaml.CommentMap
// anchor map persists over multiple documents for convenience.
anchorMap map[string]*CandidateNode
}
func NewGoccyYAMLDecoder() Decoder {
@ -25,6 +27,7 @@ func NewGoccyYAMLDecoder() Decoder {
func (dec *goccyYamlDecoder) Init(reader io.Reader) error {
dec.cm = yaml.CommentMap{}
dec.decoder = *yaml.NewDecoder(reader, yaml.CommentToMap(dec.cm), yaml.UseOrderedMap())
dec.anchorMap = make(map[string]*CandidateNode)
return nil
}
@ -38,7 +41,7 @@ func (dec *goccyYamlDecoder) Decode() (*CandidateNode, error) {
}
candidateNode := &CandidateNode{}
if err := candidateNode.UnmarshalGoccyYAML(ast, dec.cm); err != nil {
if err := candidateNode.UnmarshalGoccyYAML(ast, dec.cm, dec.anchorMap); err != nil {
return nil, err
}

View File

@ -68,7 +68,8 @@ func mustProcessFormatScenario(s formatScenario, decoder Decoder, encoder Encode
result, err := processFormatScenario(s, decoder, encoder)
if err != nil {
panic(fmt.Errorf("Bad scenario %v: %w", s.description, err))
log.Error("Bad scenario %v: %w", s.description, err)
return fmt.Sprintf("Bad scenario %v: %v", s.description, err.Error())
}
return result

View File

@ -249,11 +249,12 @@ func (dec *tomlDecoder) processTopLevelNode(currentNode *toml.Node) (bool, error
var runAgainstCurrentExp bool
var err error
log.Debug("processTopLevelNode: Going to process %v state is current %v", currentNode.Kind, NodeToString(dec.rootMap))
if currentNode.Kind == toml.Table {
switch currentNode.Kind {
case toml.Table:
runAgainstCurrentExp, err = dec.processTable(currentNode)
} else if currentNode.Kind == toml.ArrayTable {
case toml.ArrayTable:
runAgainstCurrentExp, err = dec.processArrayTable(currentNode)
} else {
default:
runAgainstCurrentExp, err = dec.decodeKeyValuesIntoMap(dec.rootMap, currentNode)
}

View File

@ -315,13 +315,14 @@ func (dec *xmlDecoder) decodeXML(root *xmlNode) error {
case xml.Comment:
commentStr := string(xml.CharData(se))
if elem.state == "started" {
switch elem.state {
case "started":
applyFootComment(elem, commentStr)
} else if elem.state == "chardata" {
case "chardata":
log.Debug("got a line comment for (%v) %v: [%v]", elem.state, elem.label, commentStr)
elem.n.LineComment = joinComments([]string{elem.n.LineComment, commentStr}, " ")
} else {
default:
log.Debug("got a head comment for (%v) %v: [%v]", elem.state, elem.label, commentStr)
elem.n.HeadComment = joinComments([]string{elem.n.HeadComment, commentStr}, " ")
}

View File

@ -2,6 +2,8 @@
Similar to the same named functions in `jq` these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps.
Use `with_entries(op)` as a syntatic sugar for doing `to_entries | op | from_entries`.
## to_entries Map
Given a sample.yml file of:
```yaml
@ -101,6 +103,28 @@ KEY_a: 1
KEY_b: 2
```
## Use with_entries to update keys recursively
We use (.. | select(tag="map")) to find all the maps in the doc, then |= to update each one of those maps. In the update, with_entries is used.
Given a sample.yml file of:
```yaml
a: 1
b:
b_a: nested
b_b: thing
```
then
```bash
yq '(.. | select(tag=="!!map")) |= with_entries(.key |= "KEY_" + .)' sample.yml
```
will output
```yaml
KEY_a: 1
KEY_b:
KEY_b_a: nested
KEY_b_b: thing
```
## Custom sort map keys
Use to_entries to convert to an array of key/value pairs, sort the array using sort/sort_by/etc, and convert it back.

View File

@ -1,3 +1,5 @@
# Entries
Similar to the same named functions in `jq` these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps.
Use `with_entries(op)` as a syntatic sugar for doing `to_entries | op | from_entries`.

View File

@ -3,7 +3,7 @@
## RegEx
This uses Golang's native regex functions under the hood - See their [docs](https://github.com/google/re2/wiki/Syntax) for the supported syntax.
Case insensitive tip: prefix the regex with `(?i)` - e.g. `test("(?i)cats)"`.
Case insensitive tip: prefix the regex with `(?i)` - e.g. `test("(?i)cats")`.
### match(regEx)
This operator returns the substring match details of the given regEx.

View File

@ -110,7 +110,7 @@ cool:
```
## Sort a map
Sorting a map, by default, will sort by the values
Sorting a map, by default this will sort by the values
Given a sample.yml file of:
```yaml

View File

@ -3,7 +3,7 @@
## RegEx
This uses Golang's native regex functions under the hood - See their [docs](https://github.com/google/re2/wiki/Syntax) for the supported syntax.
Case insensitive tip: prefix the regex with `(?i)` - e.g. `test("(?i)cats)"`.
Case insensitive tip: prefix the regex with `(?i)` - e.g. `test("(?i)cats")`.
### match(regEx)
This operator returns the substring match details of the given regEx.

View File

@ -167,10 +167,14 @@ func needsQuoting(s string) bool {
// [%a_][%w_]*
for i, c := range s {
if i == 0 {
// keeping for legacy reasons, upgraded linter
//nolint:staticcheck
if !((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_') {
return true
}
} else {
// keeping for legacy reasons, upgraded linter
//nolint:staticcheck
if !((c >= '0' && c <= '9') || (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_') {
return true
}
@ -299,10 +303,10 @@ func (le *luaEncoder) encodeAny(writer io.Writer, node *CandidateNode) error {
return writeString(writer, node.Value)
}
default:
return fmt.Errorf("Lua encoder NYI -- %s", node.Tag)
return fmt.Errorf("lua encoder NYI -- %s", node.Tag)
}
default:
return fmt.Errorf("Lua encoder NYI -- %s", node.Tag)
return fmt.Errorf("lua encoder NYI -- %s", node.Tag)
}
}

View File

@ -107,7 +107,7 @@ func (pe *propertiesEncoder) doEncode(p *properties.Properties, node *CandidateN
case AliasNode:
return pe.doEncode(p, node.Alias, path, nil)
default:
return fmt.Errorf("Unsupported node %v", node.Tag)
return fmt.Errorf("unsupported node %v", node.Tag)
}
}

View File

@ -75,7 +75,7 @@ func (pe *shellVariablesEncoder) doEncode(w *io.Writer, node *CandidateNode, pat
case AliasNode:
return pe.doEncode(w, node.Alias, path)
default:
return fmt.Errorf("Unsupported node %v", node.Tag)
return fmt.Errorf("unsupported node %v", node.Tag)
}
}

View File

@ -50,14 +50,15 @@ func (p *expressionParserImpl) createExpressionTree(postFixPath []*Operation) (*
log.Debugf("pathTree %v ", Operation.toString())
if Operation.OperationType.NumArgs > 0 {
numArgs := Operation.OperationType.NumArgs
if numArgs == 1 {
switch numArgs {
case 1:
if len(stack) < 1 {
return nil, fmt.Errorf("'%v' expects 1 arg but received none", strings.TrimSpace(Operation.StringValue))
}
remaining, rhs := stack[:len(stack)-1], stack[len(stack)-1]
newNode.RHS = rhs
stack = remaining
} else if numArgs == 2 {
case 2:
if len(stack) < 2 {
return nil, fmt.Errorf("'%v' expects 2 args but there is %v", strings.TrimSpace(Operation.StringValue), len(stack))
}

View File

@ -26,11 +26,12 @@ func popOpToResult(opStack []*token, result []*Operation) ([]*token, []*Operatio
}
func validateNoOpenTokens(token *token) error {
if token.TokenType == openCollect {
switch token.TokenType {
case openCollect:
return fmt.Errorf(("bad expression, could not find matching `]`"))
} else if token.TokenType == openCollectObject {
case openCollectObject:
return fmt.Errorf(("bad expression, could not find matching `}`"))
} else if token.TokenType == openBracket {
case openBracket:
return fmt.Errorf(("bad expression, could not find matching `)`"))
}
return nil
@ -64,7 +65,7 @@ func (p *expressionPostFixerImpl) ConvertToPostfix(infixTokens []*token) ([]*Ope
opStack, result = popOpToResult(opStack, result)
}
if len(opStack) == 0 {
return nil, errors.New("Bad path expression, got close collect brackets without matching opening bracket")
return nil, errors.New("bad path expression, got close collect brackets without matching opening bracket")
}
// now we should have [ as the last element on the opStack, get rid of it
opStack = opStack[0 : len(opStack)-1]

View File

@ -3,6 +3,7 @@ package yqlib
import (
"fmt"
"path/filepath"
"slices"
"strings"
)
@ -95,12 +96,7 @@ func (f *Format) MatchesName(name string) bool {
if f.FormalName == name {
return true
}
for _, n := range f.Names {
if n == name {
return true
}
}
return false
return slices.Contains(f.Names, name)
}
func (f *Format) GetConfiguredEncoder() Encoder {

View File

@ -7,132 +7,138 @@ import (
)
var goccyYamlFormatScenarios = []formatScenario{
{
description: "basic - 3",
skipDoc: true,
input: "3",
expected: "3\n",
},
{
description: "basic - 3.1",
skipDoc: true,
input: "3.1",
expected: "3.1\n",
},
{
description: "basic - mike",
skipDoc: true,
input: "mike: 3",
expected: "mike: 3\n",
},
{
description: "basic - map multiple entries",
skipDoc: true,
input: "mike: 3\nfred: 12\n",
expected: "mike: 3\nfred: 12\n",
},
{
description: "basic - 3.1",
skipDoc: true,
input: "{\n mike: 3\n}",
expected: "{mike: 3}\n",
},
{
description: "basic - tag with number",
skipDoc: true,
input: "mike: !!cat 3",
expected: "mike: !!cat 3\n",
},
{
description: "basic - array of numbers",
skipDoc: true,
input: "- 3",
expected: "- 3\n",
},
{
description: "basic - single line array",
skipDoc: true,
input: "[3]",
expected: "[3]\n",
},
{
description: "basic - plain string",
skipDoc: true,
input: `a: meow`,
expected: "a: meow\n",
},
{
description: "basic - double quoted string",
skipDoc: true,
input: `a: "meow"`,
expected: "a: \"meow\"\n",
},
{
description: "basic - single quoted string",
skipDoc: true,
input: `a: 'meow'`,
expected: "a: 'meow'\n",
},
{
description: "basic - string block",
skipDoc: true,
input: "a: |\n meow\n",
expected: "a: |\n meow\n",
},
{
description: "basic - long string",
skipDoc: true,
input: "a: the cute cat wrote a long sentence that wasn't wrapped at all.\n",
expected: "a: the cute cat wrote a long sentence that wasn't wrapped at all.\n",
},
{
description: "basic - string block",
skipDoc: true,
input: "a: |-\n meow\n",
expected: "a: |-\n meow\n",
},
{
description: "basic - line comment",
skipDoc: true,
input: "a: meow # line comment\n",
expected: "a: meow # line comment\n",
},
{
description: "basic - line comment",
skipDoc: true,
input: "# head comment\na: #line comment\n meow\n",
expected: "# head comment\na: meow #line comment\n", // go-yaml does this
},
{
description: "basic - foot comment",
skipDoc: true,
input: "a: meow\n# foot comment\n",
expected: "a: meow\n# foot comment\n",
},
{
description: "basic - foot comment",
skipDoc: true,
input: "a: meow\nb: woof\n# foot comment\n",
expected: "a: meow\nb: woof\n# foot comment\n",
},
{
description: "basic - boolean",
skipDoc: true,
input: "true\n",
expected: "true\n",
},
{
description: "basic - null",
skipDoc: true,
input: "a: null\n",
expected: "a: null\n",
},
{
description: "basic - ~",
skipDoc: true,
input: "a: ~\n",
expected: "a: ~\n",
},
// {
// description: "basic - 3",
// skipDoc: true,
// input: "3",
// expected: "3\n",
// },
// {
// description: "basic - 3.1",
// skipDoc: true,
// input: "3.1",
// expected: "3.1\n",
// },
// {
// description: "basic - mike",
// skipDoc: true,
// input: "mike: 3",
// expected: "mike: 3\n",
// },
// {
// description: "basic - map multiple entries",
// skipDoc: true,
// input: "mike: 3\nfred: 12\n",
// expected: "mike: 3\nfred: 12\n",
// },
// {
// description: "basic - 3.1",
// skipDoc: true,
// input: "{\n mike: 3\n}",
// expected: "{mike: 3}\n",
// },
// {
// description: "basic - tag with number",
// skipDoc: true,
// input: "mike: !!cat 3",
// expected: "mike: !!cat 3\n",
// },
// {
// description: "basic - array of numbers",
// skipDoc: true,
// input: "- 3",
// expected: "- 3\n",
// },
// {
// description: "basic - single line array",
// skipDoc: true,
// input: "[3]",
// expected: "[3]\n",
// },
// {
// description: "basic - plain string",
// skipDoc: true,
// input: `a: meow`,
// expected: "a: meow\n",
// },
// {
// description: "basic - double quoted string",
// skipDoc: true,
// input: `a: "meow"`,
// expected: "a: \"meow\"\n",
// },
// {
// description: "basic - single quoted string",
// skipDoc: true,
// input: `a: 'meow'`,
// expected: "a: 'meow'\n",
// },
// {
// description: "basic - string block",
// skipDoc: true,
// input: "a: |\n meow\n",
// expected: "a: |\n meow\n",
// },
// {
// description: "basic - long string",
// skipDoc: true,
// input: "a: the cute cat wrote a long sentence that wasn't wrapped at all.\n",
// expected: "a: the cute cat wrote a long sentence that wasn't wrapped at all.\n",
// },
// {
// description: "basic - string block",
// skipDoc: true,
// input: "a: |-\n meow\n",
// expected: "a: |-\n meow\n",
// },
// {
// description: "basic - line comment",
// skipDoc: true,
// input: "a: meow # line comment\n",
// expected: "a: meow # line comment\n",
// },
// {
// description: "basic - head comment",
// skipDoc: true,
// input: "# head comment\na: meow\n",
// expected: "# head comment\na: meow\n", // go-yaml does this
// },
// {
// description: "basic - head and line comment",
// skipDoc: true,
// input: "# head comment\na: #line comment\n meow\n",
// expected: "# head comment\na: meow #line comment\n", // go-yaml does this
// },
// {
// description: "basic - foot comment",
// skipDoc: true,
// input: "a: meow\n# foot comment\n",
// expected: "a: meow\n# foot comment\n",
// },
// {
// description: "basic - foot comment",
// skipDoc: true,
// input: "a: meow\nb: woof\n# foot comment\n",
// expected: "a: meow\nb: woof\n# foot comment\n",
// },
// {
// description: "basic - boolean",
// skipDoc: true,
// input: "true\n",
// expected: "true\n",
// },
// {
// description: "basic - null",
// skipDoc: true,
// input: "a: null\n",
// expected: "a: null\n",
// },
// {
// description: "basic - ~",
// skipDoc: true,
// input: "a: ~\n",
// expected: "a: ~\n",
// },
// {
// description: "basic - ~",
// skipDoc: true,
@ -148,9 +154,121 @@ var goccyYamlFormatScenarios = []formatScenario{
// {
// skipDoc: true,
// description: "trailing comment",
// input: "test:\n# this comment will be removed",
// expected: "test:\n# this comment will be removed",
// input: "test: null\n# this comment will be removed",
// expected: "test: null\n# this comment will be removed\n",
// },
// {
// description: "doc separator",
// skipDoc: true,
// input: "# hi\n---\na: cat\n---",
// expected: "---\na: cat\n",
// },
// {
// description: "scalar with doc separator",
// skipDoc: true,
// input: "--- cat",
// expected: "---\ncat\n",
// },
// {
// description: "scalar with doc separator",
// skipDoc: true,
// input: "---cat",
// expected: "---cat\n",
// },
// {
// description: "basic - null",
// skipDoc: true,
// input: "null",
// expected: "null\n",
// },
// {
// description: "basic - ~",
// skipDoc: true,
// input: "~",
// expected: "~\n",
// },
// {
// description: "octal",
// skipDoc: true,
// input: "0o30",
// expression: "tag",
// expected: "!!int\n",
// },
// {
// description: "basic - [null]",
// skipDoc: true,
// input: "[null]",
// expected: "[null]\n",
// },
// {
// description: "multi document",
// skipDoc: true,
// input: "a: mike\n---\nb: remember",
// expected: "a: mike\n---\nb: remember\n",
// },
// {
// description: "single doc anchor map",
// skipDoc: true,
// input: "a: &remember mike\nb: *remember",
// expected: "a: mike\nb: *remember\n",
// },
// {
// description: "explode doc anchor map",
// skipDoc: true,
// input: "a: &remember mike\nb: *remember",
// expression: "explode(.)",
// expected: "a: mike\nb: mike\n",
// },
// {
// description: "multi document anchor map",
// skipDoc: true,
// input: "a: &remember mike\n---\nb: *remember",
// expression: "explode(.)",
// expected: "a: mike\n---\nb: mike\n",
// },
// {
// description: "merge anchor",
// skipDoc: true,
// input: "a: &remember\n c: mike\nb:\n <<: *remember",
// expected: "a: &remember\n c: mike\nb:\n <<: *remember\n",
// },
{
description: "custom tag",
skipDoc: true,
input: "a: !cat mike",
expected: "a: !cat mike\n",
},
// {
// description: "basic - [~]",
// skipDoc: true,
// input: "[~]",
// expected: "[~]\n",
// },
// {
// description: "basic - null map value",
// skipDoc: true,
// input: "a: null",
// expected: "a: null\n",
// },
// {
// description: "basic - number",
// skipDoc: true,
// input: "3",
// expected: "3\n",
// },
// {
// description: "basic - float",
// skipDoc: true,
// input: "3.1",
// expected: "3.1\n",
// },
// {
// description: "basic - float",
// skipDoc: true,
// input: "[1, 2]",
// expected: "[1, 2]\n",
// },
}
func testGoccyYamlScenario(t *testing.T, s formatScenario) {

View File

@ -31,24 +31,25 @@ type token struct {
}
func (t *token) toString(detail bool) string {
if t.TokenType == operationToken {
switch t.TokenType {
case operationToken:
if detail {
return fmt.Sprintf("%v (%v)", t.Operation.toString(), t.Operation.OperationType.Precedence)
}
return t.Operation.toString()
} else if t.TokenType == openBracket {
case openBracket:
return "("
} else if t.TokenType == closeBracket {
case closeBracket:
return ")"
} else if t.TokenType == openCollect {
case openCollect:
return "["
} else if t.TokenType == closeCollect {
case closeCollect:
return "]"
} else if t.TokenType == openCollectObject {
case openCollectObject:
return "{"
} else if t.TokenType == closeCollectObject {
case closeCollectObject:
return "}"
} else if t.TokenType == traverseArrayCollect {
case traverseArrayCollect:
return ".["
}

View File

@ -39,13 +39,19 @@ func toNodes(candidate *CandidateNode, lhs *CandidateNode) []*CandidateNode {
func addOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
log.Debugf("Add operator")
return crossFunction(d, context.ReadOnlyClone(), expressionNode, add, false)
return crossFunction(d, context.ReadOnlyClone(), expressionNode, add, true)
}
func add(_ *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
lhsNode := lhs
if lhsNode.Tag == "!!null" {
if lhs == nil && rhs == nil {
return nil, nil
} else if lhs == nil {
return rhs.Copy(), nil
} else if rhs == nil {
return lhs.Copy(), nil
} else if lhsNode.Tag == "!!null" {
return lhs.CopyAsReplacement(rhs), nil
}

View File

@ -310,6 +310,15 @@ var addOperatorScenarios = []expressionScenario{
"D0, P[], (!!map)::a: !cat Saturday, 15-Dec-01 at 6:00AM GMT\n",
},
},
{
skipDoc: true,
description: "Add to empty",
subdescription: "should behave like null",
expression: `.nada + "cat"`,
expected: []string{
"D0, P[], (!!str)::cat\n",
},
},
{
description: "Add to null",
subdescription: "Adding to null simply returns the rhs",

View File

@ -26,11 +26,12 @@ func deleteChildOperator(d *dataTreeNavigator, context Context, expressionNode *
candidatePath := candidate.GetPath()
childPath := candidatePath[len(candidatePath)-1]
if parentNode.Kind == MappingNode {
switch parentNode.Kind {
case MappingNode:
deleteFromMap(candidate.Parent, childPath)
} else if parentNode.Kind == SequenceNode {
case SequenceNode:
deleteFromArray(candidate.Parent, childPath)
} else {
default:
return Context{}, fmt.Errorf("cannot delete nodes from parent of tag %v", parentNode.Tag)
}
}

View File

@ -72,6 +72,15 @@ var entriesOperatorScenarios = []expressionScenario{
"D0, P[], (!!map)::KEY_a: 1\nKEY_b: 2\n",
},
},
{
description: "Use with_entries to update keys recursively",
document: `{a: 1, b: {b_a: nested, b_b: thing}}`,
expression: `(.. | select(tag=="!!map")) |= with_entries(.key |= "KEY_" + .)`,
subdescription: "We use (.. | select(tag=\"map\")) to find all the maps in the doc, then |= to update each one of those maps. In the update, with_entries is used.",
expected: []string{
"D0, P[], (!!map)::{KEY_a: 1, KEY_b: {KEY_b_a: nested, KEY_b_b: thing}}\n",
},
},
{
skipDoc: true,
description: "Use with_entries to update keys comment",

View File

@ -45,12 +45,13 @@ func keysOperator(_ *dataTreeNavigator, context Context, _ *ExpressionNode) (Con
candidate := el.Value.(*CandidateNode)
var targetNode *CandidateNode
if candidate.Kind == MappingNode {
switch candidate.Kind {
case MappingNode:
targetNode = getMapKeys(candidate)
} else if candidate.Kind == SequenceNode {
case SequenceNode:
targetNode = getIndices(candidate)
} else {
return Context{}, fmt.Errorf("Cannot get keys of %v, keys only works for maps and arrays", candidate.Tag)
default:
return Context{}, fmt.Errorf("cannot get keys of %v, keys only works for maps and arrays", candidate.Tag)
}
results.PushBack(targetNode)

View File

@ -82,7 +82,7 @@ func loadStringOperator(d *dataTreeNavigator, context Context, expressionNode *E
contentsCandidate, err := loadString(filename)
if err != nil {
return Context{}, fmt.Errorf("Failed to load %v: %w", filename, err)
return Context{}, fmt.Errorf("failed to load %v: %w", filename, err)
}
results.PushBack(contentsCandidate)
@ -118,7 +118,7 @@ func loadOperator(d *dataTreeNavigator, context Context, expressionNode *Express
contentsCandidate, err := loadWithDecoder(filename, loadPrefs.decoder)
if err != nil {
return Context{}, fmt.Errorf("Failed to load %v: %w", filename, err)
return Context{}, fmt.Errorf("failed to load %v: %w", filename, err)
}
results.PushBack(contentsCandidate)

View File

@ -154,9 +154,9 @@ func repeatString(lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error
if err != nil {
return nil, err
} else if count < 0 {
return nil, fmt.Errorf("Cannot repeat string by a negative number (%v)", count)
return nil, fmt.Errorf("cannot repeat string by a negative number (%v)", count)
} else if count > 10000000 {
return nil, fmt.Errorf("Cannot repeat string by more than 100 million (%v)", count)
return nil, fmt.Errorf("cannot repeat string by more than 100 million (%v)", count)
}
target.Value = strings.Repeat(stringNode.Value, count)

View File

@ -206,7 +206,7 @@ var multiplyOperatorScenarios = []expressionScenario{
skipDoc: true,
document: `n: -4`,
expression: `"banana" * .n`,
expectedError: "Cannot repeat string by a negative number (-4)",
expectedError: "cannot repeat string by a negative number (-4)",
},
{
description: "Multiply string X by more than 100 million",
@ -214,7 +214,7 @@ var multiplyOperatorScenarios = []expressionScenario{
skipDoc: true,
document: `n: 100000001`,
expression: `"banana" * .n`,
expectedError: "Cannot repeat string by more than 100 million (100000001)",
expectedError: "cannot repeat string by more than 100 million (100000001)",
},
{
description: "Multiply int node X string",

View File

@ -58,11 +58,12 @@ func omitOperator(d *dataTreeNavigator, context Context, expressionNode *Express
var replacement *CandidateNode
if node.Kind == MappingNode {
switch node.Kind {
case MappingNode:
replacement = omitMap(node, indicesToOmit)
} else if node.Kind == SequenceNode {
case SequenceNode:
replacement = omitSequence(node, indicesToOmit)
} else {
default:
log.Debugf("Omit from type %v (%v) is noop", node.Tag, node.GetNicePath())
return context, nil
}

View File

@ -22,15 +22,16 @@ func getPathArrayFromNode(funcName string, node *CandidateNode) ([]interface{},
path := make([]interface{}, len(node.Content))
for i, childNode := range node.Content {
if childNode.Tag == "!!str" {
switch childNode.Tag {
case "!!str":
path[i] = childNode.Value
} else if childNode.Tag == "!!int" {
case "!!int":
number, err := parseInt(childNode.Value)
if err != nil {
return nil, fmt.Errorf("%v: could not parse %v as an int: %w", funcName, childNode.Value, err)
}
path[i] = number
} else {
default:
return nil, fmt.Errorf("%v: expected either a !!str or !!int in the path, found %v instead", funcName, childNode.Tag)
}

View File

@ -64,15 +64,16 @@ func pickOperator(d *dataTreeNavigator, context Context, expressionNode *Express
node := el.Value.(*CandidateNode)
var replacement *CandidateNode
if node.Kind == MappingNode {
switch node.Kind {
case MappingNode:
replacement = pickMap(node, indicesToPick)
} else if node.Kind == SequenceNode {
case SequenceNode:
replacement, err = pickSequence(node, indicesToPick)
if err != nil {
return Context{}, err
}
} else {
default:
return Context{}, fmt.Errorf("cannot pick indices from type %v (%v)", node.Tag, node.GetNicePath())
}

View File

@ -47,11 +47,12 @@ func sortByOperator(d *dataTreeNavigator, context Context, expressionNode *Expre
sort.Stable(sortableArray)
sortedList := candidate.CopyWithoutContent()
if candidate.Kind == MappingNode {
switch candidate.Kind {
case MappingNode:
for _, sortedNode := range sortableArray {
sortedList.AddKeyValueChild(sortedNode.Node.Key, sortedNode.Node)
}
} else if candidate.Kind == SequenceNode {
case SequenceNode:
for _, sortedNode := range sortableArray {
sortedList.AddChild(sortedNode.Node)
}

View File

@ -410,7 +410,7 @@ func extractMatchArguments(d *dataTreeNavigator, context Context, expressionNode
return nil, matchPrefs, fmt.Errorf(`'i' is not a valid option for match. To ignore case, use an expression like match("(?i)cat")`)
}
if len(paramText) > 0 {
return nil, matchPrefs, fmt.Errorf(`Unrecognised match params '%v', please see docs at https://mikefarah.gitbook.io/yq/operators/string-operators`, paramText)
return nil, matchPrefs, fmt.Errorf(`unrecognised match params '%v', please see docs at https://mikefarah.gitbook.io/yq/operators/string-operators`, paramText)
}
}

View File

@ -19,7 +19,7 @@ func parseStyle(customStyle string) (Style, error) {
} else if customStyle == "flow" {
return FlowStyle, nil
} else if customStyle != "" {
return 0, fmt.Errorf("Unknown style %v", customStyle)
return 0, fmt.Errorf("unknown style %v", customStyle)
}
return 0, nil
}

View File

@ -135,12 +135,13 @@ func traverseArrayIndices(context Context, matchingNode *CandidateNode, indicesT
}
}
if matchingNode.Kind == AliasNode {
switch matchingNode.Kind {
case AliasNode:
matchingNode = matchingNode.Alias
return traverseArrayIndices(context, matchingNode, indicesToTraverse, prefs)
} else if matchingNode.Kind == SequenceNode {
case SequenceNode:
return traverseArrayWithIndices(matchingNode, indicesToTraverse, prefs)
} else if matchingNode.Kind == MappingNode {
case MappingNode:
return traverseMapWithIndices(context, matchingNode, indicesToTraverse, prefs)
}
log.Debugf("OperatorArrayTraverse skipping %v as its a %v", matchingNode, matchingNode.Tag)

View File

@ -132,7 +132,7 @@ func (p *resultsPrinter) PrintResults(matchingNodes *list.List) error {
tempBufferBytes := tempBuffer.Bytes()
if bytes.IndexByte(tempBufferBytes, 0) != -1 {
return fmt.Errorf(
"Can't serialize value because it contains NUL char and you are using NUL separated output",
"can't serialize value because it contains NUL char and you are using NUL separated output",
)
}
if _, err := writer.Write(tempBufferBytes); err != nil {

View File

@ -319,10 +319,11 @@ func documentUnwrappedEncodePropertyScenario(w *bufio.Writer, s formatScenario)
prefs := NewDefaultPropertiesPreferences()
useArrayBracketsFlag := ""
useCustomSeparatorFlag := ""
if s.scenarioType == "encode-array-brackets" {
switch s.scenarioType {
case "encode-array-brackets":
useArrayBracketsFlag = " --properties-array-brackets"
prefs.UseArrayBrackets = true
} else if s.scenarioType == "encode-custom-separator" {
case "encode-custom-separator":
prefs.KeyValueSeparator = " :@ "
useCustomSeparatorFlag = ` --properties-separator=" :@ "`
}

View File

@ -1,3 +1,11 @@
4.45.2:
- Added windows arm builds (Thanks @albertocavalcante, @ShukantPal)
- Added s390x platform support (Thanks @ashokpariya0)
- Additionally push docker images to ghcr.io (Thanks @reegnz)
- Fixing add when there is no node match #2325
- sort_by works on maps
- Bumped dependencies
4.45.1:
- Create parent directories when --split-exp is used, Thanks @rudo-thomas
- Bumped dependencies

View File

@ -3,10 +3,23 @@
set -o errexit
set -o pipefail
if command -v golangci-lint &> /dev/null
then
golangci-lint run --verbose
# TODO: Check if the found golangci-lint version matches the expected version (e.g., v1.62.0), especially if falling back to PATH version.
GOPATH_LINT="$(go env GOPATH)/bin/golangci-lint"
BIN_LINT="./bin/golangci-lint"
LINT_CMD=""
if [ -f "$GOPATH_LINT" ]; then
LINT_CMD="$GOPATH_LINT"
elif [ -f "$BIN_LINT" ]; then
LINT_CMD="$BIN_LINT"
elif command -v golangci-lint &> /dev/null; then
# Using PATH version, ensure compatibility (see TODO)
LINT_CMD="golangci-lint"
else
./bin/golangci-lint run --verbose
echo "Error: golangci-lint not found in $GOPATH/bin, ./bin, or PATH."
echo "Please run scripts/devtools.sh or ensure golangci-lint is installed correctly."
exit 1
fi
"$LINT_CMD" run --verbose

View File

@ -1,5 +1,5 @@
#!/bin/sh
set -ex
go mod download golang.org/x/tools@latest
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.62.0
wget -O- -nv https://raw.githubusercontent.com/securego/gosec/master/install.sh | sh -s
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.1.5
curl -sSfL https://raw.githubusercontent.com/securego/gosec/master/install.sh | sh -s

View File

@ -24,9 +24,11 @@ tar czvf yq_man_page_only.tar.gz yq.1 -C ../scripts install-man-page.sh
rm yq_windows_386.exe.tar.gz
rm yq_windows_amd64.exe.tar.gz
rm yq_windows_arm64.exe.tar.gz
zip yq_windows_386.zip yq_windows_386.exe
zip yq_windows_amd64.zip yq_windows_amd64.exe
zip yq_windows_arm64.zip yq_windows_arm64.exe
rm yq.1

View File

@ -1,5 +1,5 @@
name: yq
version: 'v4.45.1'
version: 'v4.45.2'
summary: A lightweight and portable command-line data file processor
description: |
`yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml, json, xml, csv, properties and TOML files.
@ -24,6 +24,6 @@ parts:
build-environment:
- CGO_ENABLED: 0
source: https://github.com/mikefarah/yq.git
source-tag: v4.45.1
source-tag: v4.45.2
build-snaps:
- go/latest/stable