mirror of
https://github.com/mikefarah/yq.git
synced 2026-08-24 16:39:58 +08:00
Compare commits
54
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bbdd97482f | ||
|
|
9940668729 | ||
|
|
95aaccee46 | ||
|
|
b9c3ff6f0a | ||
|
|
b80e1cb35e | ||
|
|
0ff8415244 | ||
|
|
e8dcf3f1a4 | ||
|
|
208302cf6b | ||
|
|
ef6fb92e7f | ||
|
|
28646c7b50 | ||
|
|
0b7d4b799c | ||
|
|
7e1722510a | ||
|
|
f15500b20a | ||
|
|
b8aec71e66 | ||
|
|
68aafb6a53 | ||
|
|
d58870b056 | ||
|
|
f1964dece0 | ||
|
|
45eda8dbae | ||
|
|
d9af2dd976 | ||
|
|
7bad2f4f06 | ||
|
|
3ea19ad2ed | ||
|
|
42120e1341 | ||
|
|
1147113fd3 | ||
|
|
2edfa834d5 | ||
|
|
8c7cfb720c | ||
|
|
356c60f40c | ||
|
|
4cc646b16b | ||
|
|
2e44ab4512 | ||
|
|
557dcb87b8 | ||
|
|
2daa39e306 | ||
|
|
0c15cf35a7 | ||
|
|
d9adcdb668 | ||
|
|
9010809d89 | ||
|
|
8250f0c2bd | ||
|
|
8ac9f605f6 | ||
|
|
1748171463 | ||
|
|
21ba506edb | ||
|
|
b54d7eb21c | ||
|
|
db6f2dc6a2 | ||
|
|
1411f45055 | ||
|
|
c7baa8bc34 | ||
|
|
a4b9c3049b | ||
|
|
fe84f62288 | ||
|
|
28c406706a | ||
|
|
101cf14b8c | ||
|
|
3283c65dc4 | ||
|
|
2ee7508b76 | ||
|
|
2ecd8b6092 | ||
|
|
021d5f05f9 | ||
|
|
19a0e6dd8b | ||
|
|
fbd4c8428a | ||
|
|
d9ef03a2d1 | ||
|
|
f5bfe5a248 | ||
|
|
3b85cef340 |
+3
-1
@@ -1,4 +1,4 @@
|
||||
# Development
|
||||
.# Development
|
||||
|
||||
1. Install (Golang)[https://golang.org/]
|
||||
1. Run `scripts/devtools.sh` to install the required devtools
|
||||
@@ -25,6 +25,8 @@ The pipeline will run the tests and automatically concatenate the files together
|
||||
|
||||
The first step is to find if what you want is automatically generated or not - start by looking in the master branch.
|
||||
|
||||
Note that PRs with small changes (e.g. minor typos) may not be merged (see https://joel.net/how-one-guy-ruined-hacktoberfest2020-drama).
|
||||
|
||||
### Updating dynamic documentation from master
|
||||
- Search for the documentation you want to update. If you find matches in a `*_test.go` file - update that, as that will automatically update the matching `*.md` file
|
||||
- Assuming you are updating a `*_test.go` file, once updated, run the test to regenerated the docs. E.g. for the 'Add' test generated docs, from the pkg/yqlib folder run:
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
FROM golang:1.22.1 as builder
|
||||
FROM golang:1.22.5 as builder
|
||||
|
||||
WORKDIR /go/src/mikefarah/yq
|
||||
|
||||
|
||||
+8
-5
@@ -1,12 +1,15 @@
|
||||
FROM golang:1.22.1
|
||||
FROM golang:1.22.5
|
||||
|
||||
RUN apt-get update && \
|
||||
apt-get install -y npm && \
|
||||
npm install -g npx cspell@latest
|
||||
|
||||
COPY scripts/devtools.sh /opt/devtools.sh
|
||||
|
||||
RUN set -e -x \
|
||||
&& /opt/devtools.sh
|
||||
ENV PATH=/go/bin:$PATH
|
||||
RUN set -e -x && \
|
||||
/opt/devtools.sh
|
||||
|
||||
RUN apt-get update && apt-get install -y npm && npm install -g npx cspell@latest
|
||||
ENV PATH=/go/bin:$PATH
|
||||
|
||||
ENV CGO_ENABLED 0
|
||||
ENV GOPATH /go:/yq
|
||||
|
||||
+3
-2
@@ -26,6 +26,7 @@ ifeq ($(CYG_CHECK),1)
|
||||
else
|
||||
# all non-windows environments
|
||||
ROOT := $(shell pwd)
|
||||
SELINUX := $(shell which getenforce 2>&1 >/dev/null && echo :z)
|
||||
endif
|
||||
|
||||
DEV_IMAGE := ${PROJECT}_dev
|
||||
@@ -33,7 +34,7 @@ DEV_IMAGE := ${PROJECT}_dev
|
||||
ENGINERUN := ${ENGINE} run --rm \
|
||||
-e LDFLAGS="${LDFLAGS}" \
|
||||
-e GITHUB_TOKEN="${GITHUB_TOKEN}" \
|
||||
-v ${ROOT}/vendor:/go/src \
|
||||
-v ${ROOT}:/${PROJECT}/src/${IMPORT_PATH} \
|
||||
-v ${ROOT}/vendor:/go/src${SELINUX} \
|
||||
-v ${ROOT}:/${PROJECT}/src/${IMPORT_PATH}${SELINUX} \
|
||||
-w /${PROJECT}/src/${IMPORT_PATH} \
|
||||
${DEV_IMAGE}
|
||||
|
||||
@@ -295,13 +295,26 @@ sudo port install yq
|
||||
Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot)
|
||||
|
||||
### Alpine Linux
|
||||
- Enable edge/community repo by adding ```$MIRROR/alpine/edge/community``` to ```/etc/apk/repositories```
|
||||
- Update database index with ```apk update```
|
||||
- Install yq with ```apk add yq```
|
||||
|
||||
Supported by Tuan Hoang
|
||||
https://pkgs.alpinelinux.org/package/edge/community/x86/yq
|
||||
Alpine Linux v3.20+ (and Edge):
|
||||
```
|
||||
apk add yq-go
|
||||
```
|
||||
|
||||
Alpine Linux up to v3.19:
|
||||
```
|
||||
apk add yq
|
||||
```
|
||||
|
||||
Supported by Tuan Hoang (https://pkgs.alpinelinux.org/packages?name=yq-go)
|
||||
|
||||
### Flox:
|
||||
|
||||
Flox can be used to install yq on Linux, MacOS, and Windows through WSL.
|
||||
|
||||
```
|
||||
flox install yq
|
||||
```
|
||||
|
||||
## Features
|
||||
- [Detailed documentation with many examples](https://mikefarah.gitbook.io/yq/)
|
||||
@@ -349,7 +362,6 @@ Available Commands:
|
||||
eval (default) Apply the expression to each document in each yaml file in sequence
|
||||
eval-all Loads _all_ yaml documents of _all_ yaml files and runs expression once
|
||||
help Help about any command
|
||||
shell-completion Generate completion script
|
||||
|
||||
Flags:
|
||||
-C, --colors force print with colors
|
||||
|
||||
@@ -7,19 +7,20 @@ import (
|
||||
)
|
||||
|
||||
var completionCmd = &cobra.Command{
|
||||
Use: "shell-completion [bash|zsh|fish|powershell]",
|
||||
Short: "Generate completion script",
|
||||
Use: "completion [bash|zsh|fish|powershell]",
|
||||
Aliases: []string{"shell-completion"},
|
||||
Short: "Generate the autocompletion script for the specified shell",
|
||||
Long: `To load completions:
|
||||
|
||||
Bash:
|
||||
|
||||
$ source <(yq shell-completion bash)
|
||||
$ source <(yq completion bash)
|
||||
|
||||
# To load completions for each session, execute once:
|
||||
Linux:
|
||||
$ yq shell-completion bash > /etc/bash_completion.d/yq
|
||||
$ yq completion bash > /etc/bash_completion.d/yq
|
||||
MacOS:
|
||||
$ yq shell-completion bash > /usr/local/etc/bash_completion.d/yq
|
||||
$ yq completion bash > /usr/local/etc/bash_completion.d/yq
|
||||
|
||||
Zsh:
|
||||
|
||||
@@ -29,16 +30,16 @@ Zsh:
|
||||
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
|
||||
|
||||
# To load completions for each session, execute once:
|
||||
$ yq shell-completion zsh > "${fpath[1]}/_yq"
|
||||
$ yq completion zsh > "${fpath[1]}/_yq"
|
||||
|
||||
# You will need to start a new shell for this setup to take effect.
|
||||
|
||||
Fish:
|
||||
|
||||
$ yq shell-completion fish | source
|
||||
$ yq completion fish | source
|
||||
|
||||
# To load completions for each session, execute once:
|
||||
$ yq shell-completion fish > ~/.config/fish/completions/yq.fish
|
||||
$ yq completion fish > ~/.config/fish/completions/yq.fish
|
||||
`,
|
||||
DisableFlagsInUseLine: true,
|
||||
ValidArgs: []string{"bash", "zsh", "fish", "powershell"},
|
||||
+6
-1
@@ -71,9 +71,14 @@ yq -P -oy sample.json
|
||||
level := logging.WARNING
|
||||
stringFormat := `[%{level}] %{color}%{time:15:04:05}%{color:reset} %{message}`
|
||||
|
||||
if verbose {
|
||||
if verbose && forceNoColor {
|
||||
level = logging.DEBUG
|
||||
stringFormat = `[%{level:5.5s}] %{time:15:04:05} %{shortfile:-33s} %{shortfunc:-25s} %{message}`
|
||||
} else if verbose {
|
||||
level = logging.DEBUG
|
||||
stringFormat = `[%{level:5.5s}] %{color}%{time:15:04:05}%{color:bold} %{shortfile:-33s} %{shortfunc:-25s}%{color:reset} %{message}`
|
||||
} else if forceNoColor {
|
||||
stringFormat = `[%{level}] %{time:15:04:05} %{message}`
|
||||
}
|
||||
|
||||
var format = logging.MustStringFormatter(stringFormat)
|
||||
|
||||
@@ -75,10 +75,6 @@ func initCommand(cmd *cobra.Command, args []string) (string, []string, error) {
|
||||
outputFormat = "yaml"
|
||||
}
|
||||
} else if isAutomaticOutputFormat() {
|
||||
// automatic input worked, we can do it for output too unless specified
|
||||
if inputFormat == "json" {
|
||||
yqlib.GetLogger().Warning("JSON file output is now JSON by default (instead of yaml). Use '-oy' or '--output-format=yaml' for yaml output")
|
||||
}
|
||||
outputFormat = inputFormat
|
||||
}
|
||||
} else if isAutomaticOutputFormat() {
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@ var (
|
||||
GitDescribe string
|
||||
|
||||
// Version is main version number that is being run at the moment.
|
||||
Version = "v4.42.1"
|
||||
Version = "v4.44.3"
|
||||
|
||||
// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
|
||||
// then it means that it is a final release. Otherwise, this is a pre-release
|
||||
|
||||
Vendored
-210
@@ -1,210 +0,0 @@
|
||||
yq (4.16.2) focal; urgency=medium
|
||||
|
||||
* Fixed with semicolon space issue
|
||||
* Updating with documentation
|
||||
* Added STDIN example to the top
|
||||
* minor readme cleanup
|
||||
* Help text tweak
|
||||
* Fixed docker timeout - simplify docker builds
|
||||
* New release with docker build fixes
|
||||
* Updating to go 1.17 to fix CVE #944
|
||||
* Fix a typo in root.go
|
||||
* Skip the tests if the nocheck Debian build option is specified
|
||||
* Fixed select bug (#958)
|
||||
* Sped up explode operator
|
||||
* Slight performance improvement to context.ChildContext
|
||||
* Speed up multiply
|
||||
* Update README with recently added / changed options
|
||||
* Make deepMatch report in linear time
|
||||
* Removed leadingContentPreProcessing flag - header preprocessing is stable
|
||||
* Revert "Removed leadingContentPreProcessing flag - header preprocessing is stable"
|
||||
* Keep flag, it is needed in corner cases
|
||||
* Updated Readme
|
||||
* Man page
|
||||
* Fixed expression parsing bug #970
|
||||
* Bumping go-lang, docker versions
|
||||
* Added test release flow
|
||||
* Updated github action release to generate man page
|
||||
* Bumping version
|
||||
* Removing no longer needed github action
|
||||
* Added decoder op
|
||||
* Fixed newline handling when decoding/encoding
|
||||
* Fixed newline handling in encoder/decoder
|
||||
* Can specify indent in encode ops
|
||||
* Added group_by operator
|
||||
* Added flatten operator
|
||||
* Fixed flatten error message
|
||||
* Improving docs
|
||||
* Split printer
|
||||
* Refactored command logic
|
||||
* Fix JSON encoding removing null #985
|
||||
* Fixed acceptance tests
|
||||
* gitbook
|
||||
* Update document generation script
|
||||
* Updating README
|
||||
* Updating release instructions
|
||||
* github action no longer uses data1.yml
|
||||
* Create dependabot.yml
|
||||
* Bump actions/create-release from 1.0.0 to 1.1.4
|
||||
* Bump actions/setup-go from 1 to 2.1.4
|
||||
* Bump github.com/goccy/go-yaml from 1.8.9 to 1.9.4
|
||||
* Bump github.com/jinzhu/copier from 0.2.8 to 0.3.2
|
||||
* Bump github.com/fatih/color from 1.10.0 to 1.13.0
|
||||
* Bump github.com/spf13/cobra from 1.1.3 to 1.2.1
|
||||
* Update dependabot.yml
|
||||
* Update go.yml
|
||||
* add build check to PRs
|
||||
* Include secure as part of build process
|
||||
* Fixing bad label in github action
|
||||
* fixed printer test
|
||||
* remove leading content indicator
|
||||
* Fixed header preprocessing!
|
||||
* lint : define golangci configuration file
|
||||
* Update check.sh
|
||||
* Load file acceptance test
|
||||
* Minor improvement on handling front matter
|
||||
* Improved load doc
|
||||
* feature: detect MANPATh and install there
|
||||
* Update install-man-page.sh
|
||||
* simplify prod stage, move version label to action
|
||||
* add labels, quote some values
|
||||
* enable errorlint linter
|
||||
* Added errorlint to devtools
|
||||
* Added key operator
|
||||
* Added more tests
|
||||
* Fixing comments
|
||||
* Attempt to fix golint problem
|
||||
* Include version query for tools
|
||||
* Clean up errored file?
|
||||
* enable misspell linter
|
||||
* updated readme
|
||||
* update Golangci version to v1.43.0
|
||||
* gci linter
|
||||
* Better merge array by key example
|
||||
* Added credit for merge by array example
|
||||
* Better formatting of merge arrays example
|
||||
* Better merge example
|
||||
* Add accessor for the yq logger instance (#1013)
|
||||
* Fixed collect op when working with multiple nodes
|
||||
* Added map, map_values
|
||||
* Add support for Podman as well as Docker (#1026)
|
||||
* Bump github.com/jinzhu/copier from 0.3.2 to 0.3.4 (#1027)
|
||||
* Added csv, tsv output formats
|
||||
* Added encoder tests
|
||||
* Cleanup test
|
||||
* Fixed docker permission issue #1014
|
||||
* Recording release notes for next release
|
||||
* Assignment op no longer clobbers anchor (#1029)
|
||||
* Added sort_by operator
|
||||
* Improved error message
|
||||
* Improved tips and tricks
|
||||
* Report while filename failed to parse #1030
|
||||
* Added script for extracting checksums
|
||||
* Improved extract-checksum.sh
|
||||
* Bump github.com/spf13/cobra from 1.2.1 to 1.3.0 (#1039)
|
||||
* enable more linters (#1043)
|
||||
* Bump Golang compiler #1037
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Tue, 21 Dec 2021 09:41:44 +0000
|
||||
|
||||
yq (4.13.0) focal; urgency=medium
|
||||
|
||||
* New `with` operator for making multiple changes to a given path
|
||||
* New `contains` operator, works like the `jq` equivalent
|
||||
* Subtract operator now supports subtracting elements from arrays!
|
||||
* Fixed Swapping values using variables #934
|
||||
* Github Action now properly supports multiline output #936, thanks @pjxiao
|
||||
* Fixed missing closing bracket validation #932
|
||||
* Fix processing of hex numbers #929
|
||||
* Fixed alternative and union operator issues #930
|
||||
* Can now convert yaml to properties properties format (`-o=props`), See [docs](https://mikefarah.gitbook.io/yq/v/v4.x/usage/properties) for more info.
|
||||
* Fixed document header/footer comment handling when merging (https://github.com/mikefarah/yq/issues/919)
|
||||
* pretty print yaml 1.1 compatibility (https://github.com/mikefarah/yq/issues/914)
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Thu, 16 Sep 2021 20:58:30 +0200
|
||||
|
||||
yq (4.9.6) focal; urgency=medium
|
||||
|
||||
* Added darwin/arm64 build, thanks @alecthomas
|
||||
* Incremented docker alpine base version, thanks @da6d6i7-bronga
|
||||
* Bug fix: multiline expression
|
||||
* Bug fix: special character
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Tue, 29 Jun 2021 21:32:14 +0200
|
||||
|
||||
yq (3.3.2) focal; urgency=medium
|
||||
|
||||
* Bug fix: existStatus bug (#459)
|
||||
* Automatically makes an os temp directory if it does not exist (#461)
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Fri, 07 Aug 2020 18:53:01 +0200
|
||||
|
||||
yq (3.3-0) focal; urgency=medium
|
||||
|
||||
* You can control string styles (quotes) using the new --style flag
|
||||
* String values now always have quotes when outputting to json
|
||||
* Negative array indices now traverse the array backwards
|
||||
* Added a --stripComments flag to print yaml without any comments
|
||||
* Bumped go to version 1.14
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Thu, 30 Apr 2020 20:45:44 +0200
|
||||
|
||||
yq (3.1-2) eoan; urgency=medium
|
||||
|
||||
* Bug fix: yq 3 was removing empty inline-style objects and arrays (#355)
|
||||
* Bug fix: Merge option returned different output when switching order of
|
||||
merging files(#347)
|
||||
* Bug fix: Add new object to existing array object was failing in 3.1.1 (#361)
|
||||
* Bug fix: yq 3 empty keys did not allow merging of values (#356)
|
||||
* Bug fix: keys quoted during merge (#363)
|
||||
* Bug fix: Correct length with wc -l (#362)
|
||||
* Bug fix: Write to empty document removed path (#359)
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Mon, 24 Feb 2020 20:31:58 +0100
|
||||
|
||||
yq (3.1-1) eoan; urgency=medium
|
||||
|
||||
* Keeps yaml comments and formatting, can specify yaml tags when updating.
|
||||
* Handles anchors
|
||||
* Can print out matching paths and values when splatting
|
||||
* JSON output works for all commands
|
||||
* Yaml files with multiple documents are printed out as one JSON
|
||||
document per line.
|
||||
* Deep splat (**) to match arbitrary paths
|
||||
* Update scripts file format has changed to be more powerful
|
||||
* Reading and splatting, matching results are printed once per line
|
||||
* Bugfixing
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Tue, 11 Feb 2020 22:18:24 +0100
|
||||
|
||||
yq (2.2-1) bionic; urgency=medium
|
||||
|
||||
* Added Windows support for the "--inplace" command flag
|
||||
* Prefix now supports arrays
|
||||
* Add prefix command
|
||||
* Bump Alpine version to 3.8
|
||||
* Improved docker build process
|
||||
* Lint fixes
|
||||
* Build support for all linux architectures supported by gox
|
||||
|
||||
-- Roberto Mier Escandon <rmescandon@gmail.com> Sat, 19 Jan 2019 15:50:47 +0100
|
||||
|
||||
yq (2.1-0) bionic; urgency=medium
|
||||
|
||||
* Ability to read multiple documents in a single file
|
||||
* Ability to append list items instead of overwriting
|
||||
|
||||
-- Roberto Mier Escandón <rmescandon@gmail.com> Tue, 10 Jul 2018 14:02:42 +0200
|
||||
|
||||
yq (2.0-0) bionic; urgency=medium
|
||||
|
||||
* Release 2.0.0
|
||||
|
||||
-- Roberto Mier Escandón <rmescandon@gmail.com> Wed, 20 Jun 2018 10:29:53 +0200
|
||||
|
||||
yq (1.15-0) bionic; urgency=medium
|
||||
|
||||
* Release 1.15
|
||||
|
||||
-- Roberto Mier Escandón <rmescandon@gmail.com> Wed, 06 Jun 2018 11:32:03 +0200
|
||||
|
||||
Vendored
-1
@@ -1 +0,0 @@
|
||||
10
|
||||
Vendored
-22
@@ -1,22 +0,0 @@
|
||||
Source: yq
|
||||
Section: devel
|
||||
Priority: optional
|
||||
Maintainer: Roberto Mier Escandón <rmescandon@gmail.com>
|
||||
Build-Depends: debhelper (>=10),
|
||||
golang-1.17-go,
|
||||
pandoc,
|
||||
rsync
|
||||
Standards-Version: 4.1.4
|
||||
Homepage: https://github.com/mikefarah/yq.git
|
||||
Vcs-Browser: https://github.com/mikefarah/yq.git
|
||||
Vcs-Git: https://github.com/mikefarah/yq.git
|
||||
XS-Go-Import-Path: github.com/mikefarah/yq
|
||||
XSBC-Original-Maintainer: Roberto Mier Escandón <rmescandon@gmail.com>
|
||||
|
||||
Package: yq
|
||||
Architecture: any
|
||||
Depends: ${shlibs:Depends}, ${misc:Depends}
|
||||
Description: lightweight and portable command-line data file processor
|
||||
.
|
||||
The aim of the project is to be the
|
||||
[jq](https://github.com/stedolan/jq) or sed of yaml files.
|
||||
Vendored
-24
@@ -1,24 +0,0 @@
|
||||
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
|
||||
Upstream-Name: yq
|
||||
Source: https://github.com/mikefarah/yq.git
|
||||
|
||||
Files: *
|
||||
Copyright: 2017 Mike Farah
|
||||
License: Expat
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
.
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
.
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
Vendored
-2
@@ -1,2 +0,0 @@
|
||||
[DEFAULT]
|
||||
pristine-tar = True
|
||||
Vendored
-74
@@ -1,74 +0,0 @@
|
||||
#!/usr/bin/make -f
|
||||
#
|
||||
# Copyright (C) 2018-2021 Roberto Mier Escandón <rmescandon@gmail.com>
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License version 3 as
|
||||
# published by the Free Software Foundation.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
PROJECT := yq
|
||||
OWNER := mikefarah
|
||||
REPO := github.com
|
||||
|
||||
export DH_OPTIONS
|
||||
export DH_GOPKG := ${REPO}/${OWNER}/${PROJECT}
|
||||
export GOROOT := /usr/local/go
|
||||
export GOPATH := ${CURDIR}/_build
|
||||
export GOBIN := ${GOPATH}/bin
|
||||
export PATH := ${GOROOT}/bin:${GOBIN}:${PATH}
|
||||
export GOCACHE := /tmp/gocache
|
||||
export GOFLAGS := -mod=vendor
|
||||
export GO111MODULE := on
|
||||
|
||||
SRCDIR := ${GOPATH}/src/${DH_GOPKG}
|
||||
DESTDIR := ${CURDIR}/debian/${PROJECT}
|
||||
BINDIR := /usr/bin
|
||||
MANDIR := /usr/share/man/man1/
|
||||
ASSETSDIR := /usr/share/${PROJECT}
|
||||
|
||||
%:
|
||||
dh $@
|
||||
|
||||
override_dh_auto_build:
|
||||
mkdir -p ${SRCDIR}
|
||||
mkdir -p ${GOBIN}
|
||||
# copy project to local srcdir to build from there
|
||||
rsync -avz --progress --exclude=_build --exclude=debian --exclude=tmp. --exclude=go.mod --exclude=docs . $(SRCDIR)
|
||||
# build go code
|
||||
( \
|
||||
cd ${SRCDIR} && \
|
||||
go install -buildmode=pie ./... \
|
||||
)
|
||||
|
||||
# build man page
|
||||
( \
|
||||
cd ${SRCDIR} && \
|
||||
./scripts/generate-man-page-md.sh && \
|
||||
./scripts/generate-man-page.sh \
|
||||
)
|
||||
|
||||
override_dh_auto_test:
|
||||
ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
|
||||
(cd ${SRCDIR} && go test -v ./...)
|
||||
endif
|
||||
|
||||
override_dh_auto_install:
|
||||
cp ${GOBIN}/yq ${DESTDIR}/${BINDIR}
|
||||
cp -f ${SRCDIR}/LICENSE ${DESTDIR}/${ASSETSDIR}
|
||||
chmod a+x ${DESTDIR}/${BINDIR}/yq
|
||||
|
||||
# man
|
||||
mkdir -p "${DESTDIR}"/"${MANDIR}"
|
||||
cp "${SRCDIR}"/yq.1 "${DESTDIR}"/"${MANDIR}" \
|
||||
|
||||
override_dh_auto_clean:
|
||||
dh_clean
|
||||
rm -rf ${CURDIR}/_build
|
||||
Vendored
-1
@@ -1 +0,0 @@
|
||||
3.0 (native)
|
||||
Vendored
-3
@@ -1,3 +0,0 @@
|
||||
usr/bin
|
||||
usr/share/yq
|
||||
usr/share/man/man1
|
||||
@@ -5,19 +5,19 @@ require (
|
||||
github.com/alecthomas/participle/v2 v2.1.1
|
||||
github.com/alecthomas/repr v0.4.0
|
||||
github.com/dimchansky/utfbom v1.1.1
|
||||
github.com/elliotchance/orderedmap v1.5.1
|
||||
github.com/fatih/color v1.16.0
|
||||
github.com/goccy/go-json v0.10.2
|
||||
github.com/goccy/go-yaml v1.11.3
|
||||
github.com/elliotchance/orderedmap v1.6.0
|
||||
github.com/fatih/color v1.17.0
|
||||
github.com/goccy/go-json v0.10.3
|
||||
github.com/goccy/go-yaml v1.12.0
|
||||
github.com/jinzhu/copier v0.4.0
|
||||
github.com/magiconair/properties v1.8.7
|
||||
github.com/pelletier/go-toml/v2 v2.1.1
|
||||
github.com/pelletier/go-toml/v2 v2.2.2
|
||||
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e
|
||||
github.com/spf13/cobra v1.8.0
|
||||
github.com/spf13/cobra v1.8.1
|
||||
github.com/spf13/pflag v1.0.5
|
||||
github.com/yuin/gopher-lua v1.1.1
|
||||
golang.org/x/net v0.22.0
|
||||
golang.org/x/text v0.14.0
|
||||
golang.org/x/net v0.27.0
|
||||
golang.org/x/text v0.16.0
|
||||
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
@@ -26,7 +26,7 @@ require (
|
||||
github.com/inconshreveable/mousetrap v1.1.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.13 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
golang.org/x/sys v0.18.0 // indirect
|
||||
golang.org/x/sys v0.22.0 // indirect
|
||||
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
|
||||
)
|
||||
|
||||
|
||||
@@ -6,26 +6,26 @@ github.com/alecthomas/participle/v2 v2.1.1 h1:hrjKESvSqGHzRb4yW1ciisFJ4p3MGYih6i
|
||||
github.com/alecthomas/participle/v2 v2.1.1/go.mod h1:Y1+hAs8DHPmc3YUFzqllV+eSQ9ljPTk0ZkPMtEdAx2c=
|
||||
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
|
||||
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dimchansky/utfbom v1.1.1 h1:vV6w1AhK4VMnhBno/TPVCoK9U/LP0PkLCS9tbxHdi/U=
|
||||
github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE=
|
||||
github.com/elliotchance/orderedmap v1.5.1 h1:G1X4PYlljzimbdQ3RXmtIZiQ9d6aRQ3sH1nzjq5mECE=
|
||||
github.com/elliotchance/orderedmap v1.5.1/go.mod h1:wsDwEaX5jEoyhbs7x93zk2H/qv0zwuhg4inXhDkYqys=
|
||||
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
|
||||
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
|
||||
github.com/elliotchance/orderedmap v1.6.0 h1:xjn+kbbKXeDq6v9RVE+WYwRbYfAZKvlWfcJNxM8pvEw=
|
||||
github.com/elliotchance/orderedmap v1.6.0/go.mod h1:wsDwEaX5jEoyhbs7x93zk2H/qv0zwuhg4inXhDkYqys=
|
||||
github.com/fatih/color v1.17.0 h1:GlRw1BRJxkpqUCBKzKOw098ed57fEsKeNjpTe3cSjK4=
|
||||
github.com/fatih/color v1.17.0/go.mod h1:YZ7TlrGPkiz6ku9fK3TLD/pl3CpsiFyu8N92HLgmosI=
|
||||
github.com/go-playground/locales v0.13.0 h1:HyWk6mgj5qFqCT5fjGBuRArbVDfE4hi8+e8ceBS/t7Q=
|
||||
github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
|
||||
github.com/go-playground/universal-translator v0.17.0 h1:icxd5fm+REJzpZx7ZfpaD876Lmtgy7VtROAbHHXk8no=
|
||||
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
|
||||
github.com/go-playground/validator/v10 v10.4.1 h1:pH2c5ADXtd66mxoE0Zm9SUhxE20r7aM3F26W0hOn+GE=
|
||||
github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
|
||||
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
|
||||
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
|
||||
github.com/goccy/go-yaml v1.11.3 h1:B3W9IdWbvrUu2OYQGwvU1nZtvMQJPBKgBUuweJjLj6I=
|
||||
github.com/goccy/go-yaml v1.11.3/go.mod h1:wKnAMd44+9JAAnGQpWVEgBzGt3YuTaQ4uXoHvE4m7WU=
|
||||
github.com/goccy/go-json v0.10.3 h1:KZ5WoDbxAIgm2HNbYckL0se1fHD6rz5j4ywS6ebzDqA=
|
||||
github.com/goccy/go-json v0.10.3/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||
github.com/goccy/go-yaml v1.12.0 h1:/1WHjnMsI1dlIBQutrvSMGZRQufVO3asrHfTwfACoPM=
|
||||
github.com/goccy/go-yaml v1.12.0/go.mod h1:wKnAMd44+9JAAnGQpWVEgBzGt3YuTaQ4uXoHvE4m7WU=
|
||||
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
|
||||
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
|
||||
@@ -43,37 +43,39 @@ github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovk
|
||||
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/pelletier/go-toml/v2 v2.1.1 h1:LWAJwfNvjQZCFIDKWYQaM62NcYeYViCmWIwmOStowAI=
|
||||
github.com/pelletier/go-toml/v2 v2.1.1/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc=
|
||||
github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
|
||||
github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
|
||||
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e h1:aoZm08cpOy4WuID//EZDgcC4zIxODThtZNPirFr42+A=
|
||||
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
|
||||
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
|
||||
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
|
||||
github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
|
||||
github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
|
||||
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
|
||||
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
||||
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
|
||||
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
|
||||
golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
|
||||
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
|
||||
golang.org/x/net v0.22.0 h1:9sGLhx7iRIHEiX0oAJ3MRZMUCElJgy7Br1nO+AMN3Tc=
|
||||
golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
|
||||
golang.org/x/crypto v0.25.0 h1:ypSNr+bnYL2YhwoMt2zPxHFmbAN1KZs/njMG3hxUp30=
|
||||
golang.org/x/crypto v0.25.0/go.mod h1:T+wALwcMOSE0kXgUAnPAHqTLW+XHgcELELW8VaDgm/M=
|
||||
golang.org/x/net v0.27.0 h1:5K3Njcw06/l2y9vpGCSdcxWOYHOUk3dVNGDXN+FvAys=
|
||||
golang.org/x/net v0.27.0/go.mod h1:dDi0PyhWNoiUOrAS8uXv/vnScO4wnHQO4mj9fn/RytE=
|
||||
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
|
||||
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
|
||||
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
|
||||
golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
|
||||
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
|
||||
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 h1:H2TDz8ibqkAF6YGhCdN3jS9O0/s90v0rJh3X/OLHEUk=
|
||||
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
|
||||
@@ -7,6 +7,6 @@ import (
|
||||
"os"
|
||||
)
|
||||
|
||||
func changeOwner(info fs.FileInfo, file *os.File) error {
|
||||
func changeOwner(_ fs.FileInfo, _ *os.File) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -186,6 +186,20 @@ var csvScenarios = []formatScenario{
|
||||
expected: expectedYamlFromCSVNoParsing,
|
||||
scenarioType: "decode-csv-no-auto",
|
||||
},
|
||||
{
|
||||
description: "values starting with #, no auto parse",
|
||||
skipDoc: true,
|
||||
input: "value\n#ffff",
|
||||
expected: "- value: '#ffff'\n",
|
||||
scenarioType: "decode-csv-no-auto",
|
||||
},
|
||||
{
|
||||
description: "values starting with #",
|
||||
skipDoc: true,
|
||||
input: "value\n#ffff",
|
||||
expected: "- value: #ffff\n",
|
||||
scenarioType: "decode-csv",
|
||||
},
|
||||
{
|
||||
description: "Scalar roundtrip",
|
||||
skipDoc: true,
|
||||
|
||||
@@ -31,7 +31,7 @@ func (dec *csvObjectDecoder) convertToNode(content string) *CandidateNode {
|
||||
node, err := parseSnippet(content)
|
||||
// if we're not auto-parsing, then we wont put in parsed objects or arrays
|
||||
// but we still parse scalars
|
||||
if err != nil || (!dec.prefs.AutoParse && node.Kind != ScalarNode) {
|
||||
if err != nil || (!dec.prefs.AutoParse && (node.Kind != ScalarNode || node.Value != content)) {
|
||||
return createScalarNode(content, content)
|
||||
}
|
||||
return node
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Max
|
||||
|
||||
Computes the maximum among an incoming sequence of scalar values.
|
||||
@@ -0,0 +1,3 @@
|
||||
# Min
|
||||
|
||||
Computes the minimum among an incoming sequence of scalar values.
|
||||
@@ -0,0 +1,3 @@
|
||||
# Omit
|
||||
|
||||
Works like `pick`, but instead you specify the keys/indices that you _don't_ want included.
|
||||
@@ -0,0 +1,3 @@
|
||||
# Pivot
|
||||
|
||||
Emulates the `PIVOT` function supported by several popular RDBMS systems.
|
||||
@@ -2,4 +2,4 @@
|
||||
|
||||
The slice array operator takes an array as input and returns a subarray. Like the `jq` equivalent, `.[10:15]` will return an array of length 5, starting from index 10 inclusive, up to index 15 exclusive. Negative numbers count backwards from the end of the array.
|
||||
|
||||
You may leave out the first or second number, which will will refer to the start or end of the array respectively.
|
||||
You may leave out the first or second number, which will refer to the start or end of the array respectively.
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Max
|
||||
|
||||
Computes the maximum among an incoming sequence of scalar values.
|
||||
|
||||
## Maximum int
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- 99
|
||||
- 16
|
||||
- 12
|
||||
- 6
|
||||
- 66
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'max' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
99
|
||||
```
|
||||
|
||||
## Maximum string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- foo
|
||||
- bar
|
||||
- baz
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'max' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
foo
|
||||
```
|
||||
|
||||
## Maximum of empty
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
[]
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'max' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
```
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Min
|
||||
|
||||
Computes the minimum among an incoming sequence of scalar values.
|
||||
|
||||
## Minimum int
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- 99
|
||||
- 16
|
||||
- 12
|
||||
- 6
|
||||
- 66
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'min' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
6
|
||||
```
|
||||
|
||||
## Minimum string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- foo
|
||||
- bar
|
||||
- baz
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'min' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
bar
|
||||
```
|
||||
|
||||
## Minimum of empty
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
[]
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'min' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
```
|
||||
|
||||
@@ -55,6 +55,62 @@ a: 12
|
||||
b: 4
|
||||
```
|
||||
|
||||
## Multiply string node X int
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
b: banana
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.b * 4' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
bananabananabananabanana
|
||||
```
|
||||
|
||||
## Multiply int X string node
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
b: banana
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '4 * .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
bananabananabananabanana
|
||||
```
|
||||
|
||||
## Multiply string X int node
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
n: 4
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '"banana" * .n' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
bananabananabananabanana
|
||||
```
|
||||
|
||||
## Multiply int node X string
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
n: 4
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.n * "banana"' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
bananabananabananabanana
|
||||
```
|
||||
|
||||
## Merge objects together, returning merged result only
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# Omit
|
||||
|
||||
Works like `pick`, but instead you specify the keys/indices that you _don't_ want included.
|
||||
|
||||
## Omit keys from map
|
||||
Note that non existent keys are skipped.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
myMap:
|
||||
cat: meow
|
||||
dog: bark
|
||||
thing: hamster
|
||||
hamster: squeak
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq '.myMap |= omit(["hamster", "cat", "goat"])' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
myMap:
|
||||
dog: bark
|
||||
thing: hamster
|
||||
```
|
||||
|
||||
## Omit indices from array
|
||||
Note that non existent indices are skipped.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- cat
|
||||
- leopard
|
||||
- lion
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'omit([2, 0, 734, -5])' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- leopard
|
||||
```
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
# Pivot
|
||||
|
||||
Emulates the `PIVOT` function supported by several popular RDBMS systems.
|
||||
|
||||
## Pivot a sequence of sequences
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- - foo
|
||||
- bar
|
||||
- baz
|
||||
- - sis
|
||||
- boom
|
||||
- bah
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'pivot' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- - foo
|
||||
- sis
|
||||
- - bar
|
||||
- boom
|
||||
- - baz
|
||||
- bah
|
||||
```
|
||||
|
||||
## Pivot sequence of heterogeneous sequences
|
||||
Missing values are "padded" to null.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- - foo
|
||||
- bar
|
||||
- baz
|
||||
- - sis
|
||||
- boom
|
||||
- bah
|
||||
- blah
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'pivot' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- - foo
|
||||
- sis
|
||||
- - bar
|
||||
- boom
|
||||
- - baz
|
||||
- bah
|
||||
- -
|
||||
- blah
|
||||
```
|
||||
|
||||
## Pivot sequence of maps
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- foo: a
|
||||
bar: b
|
||||
baz: c
|
||||
- foo: x
|
||||
bar: y
|
||||
baz: z
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'pivot' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
foo:
|
||||
- a
|
||||
- x
|
||||
bar:
|
||||
- b
|
||||
- y
|
||||
baz:
|
||||
- c
|
||||
- z
|
||||
```
|
||||
|
||||
## Pivot sequence of heterogeneous maps
|
||||
Missing values are "padded" to null.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- foo: a
|
||||
bar: b
|
||||
baz: c
|
||||
- foo: x
|
||||
bar: y
|
||||
baz: z
|
||||
what: ever
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'pivot' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
foo:
|
||||
- a
|
||||
- x
|
||||
bar:
|
||||
- b
|
||||
- y
|
||||
baz:
|
||||
- c
|
||||
- z
|
||||
what:
|
||||
-
|
||||
- ever
|
||||
```
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
The slice array operator takes an array as input and returns a subarray. Like the `jq` equivalent, `.[10:15]` will return an array of length 5, starting from index 10 inclusive, up to index 15 exclusive. Negative numbers count backwards from the end of the array.
|
||||
|
||||
You may leave out the first or second number, which will will refer to the start or end of the array respectively.
|
||||
You may leave out the first or second number, which will refer to the start or end of the array respectively.
|
||||
|
||||
## Slicing arrays
|
||||
Given a sample.yml file of:
|
||||
|
||||
@@ -63,7 +63,29 @@ will output
|
||||
- ~
|
||||
```
|
||||
|
||||
## Unique array object fields
|
||||
## Unique array objects
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- name: harry
|
||||
pet: cat
|
||||
- name: billy
|
||||
pet: dog
|
||||
- name: harry
|
||||
pet: cat
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'unique' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- name: harry
|
||||
pet: cat
|
||||
- name: billy
|
||||
pet: dog
|
||||
```
|
||||
|
||||
## Unique array of objects by a field
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- name: harry
|
||||
@@ -85,3 +107,25 @@ will output
|
||||
pet: dog
|
||||
```
|
||||
|
||||
## Unique array of arrays
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- - cat
|
||||
- dog
|
||||
- - cat
|
||||
- sheep
|
||||
- - cat
|
||||
- dog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq 'unique' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- - cat
|
||||
- dog
|
||||
- - cat
|
||||
- sheep
|
||||
```
|
||||
|
||||
|
||||
@@ -230,6 +230,23 @@ person:
|
||||
- pizza
|
||||
```
|
||||
|
||||
## Decode properties: numbers
|
||||
All values are assumed to be strings when parsing properties, but you can use the `from_yaml` operator on all the strings values to autoparse into the correct type.
|
||||
|
||||
Given a sample.properties file of:
|
||||
```properties
|
||||
a.b = 10
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -p=props ' (.. | select(tag == "!!str")) |= from_yaml' sample.properties
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
b: 10
|
||||
```
|
||||
|
||||
## Decode properties - array should be a map
|
||||
If you have a numeric map key in your property files, use array_to_map to convert them to maps.
|
||||
|
||||
|
||||
+10
-8
@@ -2,6 +2,7 @@ package yqlib
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
@@ -107,12 +108,11 @@ func (f *Format) GetConfiguredEncoder() Encoder {
|
||||
}
|
||||
|
||||
func FormatStringFromFilename(filename string) string {
|
||||
|
||||
if filename != "" {
|
||||
GetLogger().Debugf("checking file extension '%s' for auto format detection", filename)
|
||||
nPos := strings.LastIndex(filename, ".")
|
||||
if nPos > -1 {
|
||||
format := filename[nPos+1:]
|
||||
GetLogger().Debugf("checking filename '%s' for auto format detection", filename)
|
||||
ext := filepath.Ext(filename)
|
||||
if ext != "" && ext[0] == '.' {
|
||||
format := strings.ToLower(ext[1:])
|
||||
GetLogger().Debugf("detected format '%s'", format)
|
||||
return format
|
||||
}
|
||||
@@ -123,9 +123,11 @@ func FormatStringFromFilename(filename string) string {
|
||||
}
|
||||
|
||||
func FormatFromString(format string) (*Format, error) {
|
||||
for _, printerFormat := range Formats {
|
||||
if printerFormat.MatchesName(format) {
|
||||
return printerFormat, nil
|
||||
if format != "" {
|
||||
for _, printerFormat := range Formats {
|
||||
if printerFormat.MatchesName(format) {
|
||||
return printerFormat, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil, fmt.Errorf("unknown format '%v' please use [%v]", format, GetAvailableOutputFormatString())
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/mikefarah/yq/v4/test"
|
||||
)
|
||||
|
||||
type formatStringScenario struct {
|
||||
description string
|
||||
input string
|
||||
expectedFormat *Format
|
||||
expectedError string
|
||||
}
|
||||
|
||||
var formatStringScenarios = []formatStringScenario{
|
||||
{
|
||||
description: "yaml",
|
||||
input: "yaml",
|
||||
expectedFormat: YamlFormat,
|
||||
},
|
||||
{
|
||||
description: "Unknown format type",
|
||||
input: "doc",
|
||||
expectedError: "unknown format 'doc' please use",
|
||||
},
|
||||
{
|
||||
description: "blank should error",
|
||||
input: "",
|
||||
expectedError: "unknown format '' please use",
|
||||
},
|
||||
}
|
||||
|
||||
func TestFormatFromString(t *testing.T) {
|
||||
for _, tt := range formatStringScenarios {
|
||||
actualFormat, actualError := FormatFromString(tt.input)
|
||||
|
||||
if tt.expectedError != "" {
|
||||
if actualError == nil {
|
||||
t.Errorf("Expected [%v] error but found none", tt.expectedError)
|
||||
} else {
|
||||
test.AssertResultWithContext(t, true, strings.Contains(actualError.Error(), tt.expectedError),
|
||||
fmt.Sprintf("Expected [%v] to contain [%v]", actualError.Error(), tt.expectedError),
|
||||
)
|
||||
}
|
||||
} else {
|
||||
test.AssertResult(t, tt.expectedFormat, actualFormat)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatStringFromFilename(t *testing.T) {
|
||||
test.AssertResult(t, "yaml", FormatStringFromFilename("test.Yaml"))
|
||||
test.AssertResult(t, "yaml", FormatStringFromFilename("test.index.Yaml"))
|
||||
test.AssertResult(t, "yaml", FormatStringFromFilename("test"))
|
||||
test.AssertResult(t, "json", FormatStringFromFilename("test.json"))
|
||||
test.AssertResult(t, "json", FormatStringFromFilename("TEST.JSON"))
|
||||
test.AssertResult(t, "yaml", FormatStringFromFilename("test.json/foo"))
|
||||
test.AssertResult(t, "yaml", FormatStringFromFilename(""))
|
||||
}
|
||||
@@ -40,6 +40,7 @@ var participleYqRules = []*participleYqRule{
|
||||
simpleOp("map", mapOpType),
|
||||
simpleOp("filter", filterOpType),
|
||||
simpleOp("pick", pickOpType),
|
||||
simpleOp("omit", omitOpType),
|
||||
|
||||
{"FlattenWithDepth", `flatten\([0-9]+\)`, flattenWithDepth(), 0},
|
||||
{"Flatten", `flatten`, opTokenWithPrefs(flattenOpType, nil, flattenPreferences{depth: -1}), 0},
|
||||
@@ -198,6 +199,9 @@ var participleYqRules = []*participleYqRule{
|
||||
{"GreaterThan", `\s*>\s*`, opTokenWithPrefs(compareOpType, nil, compareTypePref{OrEqual: false, Greater: true}), 0},
|
||||
{"LessThan", `\s*<\s*`, opTokenWithPrefs(compareOpType, nil, compareTypePref{OrEqual: false, Greater: false}), 0},
|
||||
|
||||
simpleOp("min", minOpType),
|
||||
simpleOp("max", maxOpType),
|
||||
|
||||
{"AssignRelative", `\|=[c]*`, assignOpToken(true), 0},
|
||||
{"Assign", `=[c]*`, assignOpToken(false), 0},
|
||||
|
||||
@@ -223,6 +227,8 @@ var participleYqRules = []*participleYqRule{
|
||||
{"SubtractAssign", `\-=`, opToken(subtractAssignOpType), 0},
|
||||
{"Subtract", `\-`, opToken(subtractOpType), 0},
|
||||
{"Comment", `#.*`, nil, 0},
|
||||
|
||||
simpleOp("pivot", pivotOpType),
|
||||
}
|
||||
|
||||
type yqAction func(lexer.Token) (*token, error)
|
||||
@@ -322,7 +328,7 @@ func flattenWithDepth() yqAction {
|
||||
|
||||
prefs := flattenPreferences{depth: depth}
|
||||
op := &Operation{OperationType: flattenOpType, Value: flattenOpType.Type, StringValue: value, Preferences: prefs}
|
||||
return &token{TokenType: operationToken, Operation: op}, nil
|
||||
return &token{TokenType: operationToken, Operation: op, CheckForPostTraverse: flattenOpType.CheckForPostTraverse}, nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -400,7 +406,7 @@ func envOp(strenv bool) yqAction {
|
||||
envOperation.OperationType = envOpType
|
||||
envOperation.Preferences = preferences
|
||||
|
||||
return &token{TokenType: operationToken, Operation: envOperation}, nil
|
||||
return &token{TokenType: operationToken, Operation: envOperation, CheckForPostTraverse: envOpType.CheckForPostTraverse}, nil
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+655
-661
File diff suppressed because it is too large
Load Diff
+16
-2
@@ -95,10 +95,20 @@ func parseSnippet(value string) (*CandidateNode, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if result.Kind == ScalarNode {
|
||||
result.LineComment = result.LeadingContent
|
||||
} else {
|
||||
result.HeadComment = result.LeadingContent
|
||||
}
|
||||
result.LeadingContent = ""
|
||||
|
||||
if result.Tag == "!!str" {
|
||||
// use the original string value, as
|
||||
// decoding drops new lines
|
||||
return createScalarNode(value, value), nil
|
||||
newNode := createScalarNode(value, value)
|
||||
newNode.LineComment = result.LineComment
|
||||
return newNode, nil
|
||||
}
|
||||
result.Line = 0
|
||||
result.Column = 0
|
||||
@@ -135,8 +145,12 @@ func recursiveNodeEqual(lhs *CandidateNode, rhs *CandidateNode) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// yaml numbers can be hex and octal encoded...
|
||||
// yaml numbers can have underscores, be hex and octal encoded...
|
||||
func parseInt64(numberString string) (string, int64, error) {
|
||||
if strings.Contains(numberString, "_") {
|
||||
numberString = strings.ReplaceAll(numberString, "_", "")
|
||||
}
|
||||
|
||||
if strings.HasPrefix(numberString, "0x") ||
|
||||
strings.HasPrefix(numberString, "0X") {
|
||||
num, err := strconv.ParseInt(numberString[2:], 16, 64)
|
||||
|
||||
+25
-1
@@ -62,6 +62,16 @@ var parseSnippetScenarios = []parseSnippetScenario{
|
||||
Column: 0,
|
||||
},
|
||||
},
|
||||
{
|
||||
snippet: "# things",
|
||||
expected: &CandidateNode{
|
||||
Kind: ScalarNode,
|
||||
Tag: "!!null",
|
||||
LineComment: "# things",
|
||||
Line: 0,
|
||||
Column: 0,
|
||||
},
|
||||
},
|
||||
{
|
||||
snippet: "3.1",
|
||||
expected: &CandidateNode{
|
||||
@@ -106,6 +116,7 @@ func TestParseSnippet(t *testing.T) {
|
||||
type parseInt64Scenario struct {
|
||||
numberString string
|
||||
expectedParsedNumber int64
|
||||
expectedFormatString string
|
||||
}
|
||||
|
||||
var parseInt64Scenarios = []parseInt64Scenario{
|
||||
@@ -113,10 +124,20 @@ var parseInt64Scenarios = []parseInt64Scenario{
|
||||
numberString: "34",
|
||||
expectedParsedNumber: 34,
|
||||
},
|
||||
{
|
||||
numberString: "10_000",
|
||||
expectedParsedNumber: 10000,
|
||||
expectedFormatString: "10000",
|
||||
},
|
||||
{
|
||||
numberString: "0x10",
|
||||
expectedParsedNumber: 16,
|
||||
},
|
||||
{
|
||||
numberString: "0x10_000",
|
||||
expectedParsedNumber: 65536,
|
||||
expectedFormatString: "0x10000",
|
||||
},
|
||||
{
|
||||
numberString: "0o10",
|
||||
expectedParsedNumber: 8,
|
||||
@@ -132,7 +153,10 @@ func TestParseInt64(t *testing.T) {
|
||||
t.Error(err)
|
||||
}
|
||||
test.AssertResultComplexWithContext(t, tt.expectedParsedNumber, actualNumber, tt.numberString)
|
||||
if tt.expectedFormatString == "" {
|
||||
tt.expectedFormatString = tt.numberString
|
||||
}
|
||||
|
||||
test.AssertResultComplexWithContext(t, tt.numberString, fmt.Sprintf(format, actualNumber), fmt.Sprintf("Formatting of: %v", tt.numberString))
|
||||
test.AssertResultComplexWithContext(t, tt.expectedFormatString, fmt.Sprintf(format, actualNumber), fmt.Sprintf("Formatting of: %v", tt.numberString))
|
||||
}
|
||||
}
|
||||
|
||||
+30
-24
@@ -73,6 +73,8 @@ var equalsOpType = &operationType{Type: "EQUALS", NumArgs: 2, Precedence: 40, Ha
|
||||
var notEqualsOpType = &operationType{Type: "NOT_EQUALS", NumArgs: 2, Precedence: 40, Handler: notEqualsOperator}
|
||||
|
||||
var compareOpType = &operationType{Type: "COMPARE", NumArgs: 2, Precedence: 40, Handler: compareOperator}
|
||||
var minOpType = &operationType{Type: "MIN", NumArgs: 0, Precedence: 40, Handler: minOperator}
|
||||
var maxOpType = &operationType{Type: "MAX", NumArgs: 0, Precedence: 40, Handler: maxOperator}
|
||||
|
||||
// createmap needs to be above union, as we use union to build the components of the objects
|
||||
var createMapOpType = &operationType{Type: "CREATE_MAP", NumArgs: 2, Precedence: 15, Handler: createMapOperator}
|
||||
@@ -83,15 +85,17 @@ var lengthOpType = &operationType{Type: "LENGTH", NumArgs: 0, Precedence: 50, Ha
|
||||
var lineOpType = &operationType{Type: "LINE", NumArgs: 0, Precedence: 50, Handler: lineOperator}
|
||||
var columnOpType = &operationType{Type: "LINE", NumArgs: 0, Precedence: 50, Handler: columnOperator}
|
||||
|
||||
// Use this expression to create alias/syntactic sugar expressions (in lexer_participle).
|
||||
var expressionOpType = &operationType{Type: "EXP", NumArgs: 0, Precedence: 50, Handler: expressionOperator}
|
||||
|
||||
var collectOpType = &operationType{Type: "COLLECT", NumArgs: 1, Precedence: 50, Handler: collectOperator}
|
||||
var mapOpType = &operationType{Type: "MAP", NumArgs: 1, Precedence: 50, Handler: mapOperator}
|
||||
var filterOpType = &operationType{Type: "FILTER", NumArgs: 1, Precedence: 50, Handler: filterOperator}
|
||||
var mapOpType = &operationType{Type: "MAP", NumArgs: 1, Precedence: 52, Handler: mapOperator, CheckForPostTraverse: true}
|
||||
var filterOpType = &operationType{Type: "FILTER", NumArgs: 1, Precedence: 52, Handler: filterOperator, CheckForPostTraverse: true}
|
||||
var errorOpType = &operationType{Type: "ERROR", NumArgs: 1, Precedence: 50, Handler: errorOperator}
|
||||
var pickOpType = &operationType{Type: "PICK", NumArgs: 1, Precedence: 50, Handler: pickOperator}
|
||||
var evalOpType = &operationType{Type: "EVAL", NumArgs: 1, Precedence: 50, Handler: evalOperator}
|
||||
var mapValuesOpType = &operationType{Type: "MAP_VALUES", NumArgs: 1, Precedence: 50, Handler: mapValuesOperator}
|
||||
var pickOpType = &operationType{Type: "PICK", NumArgs: 1, Precedence: 52, Handler: pickOperator, CheckForPostTraverse: true}
|
||||
var omitOpType = &operationType{Type: "OMIT", NumArgs: 1, Precedence: 52, Handler: omitOperator, CheckForPostTraverse: true}
|
||||
var evalOpType = &operationType{Type: "EVAL", NumArgs: 1, Precedence: 52, Handler: evalOperator, CheckForPostTraverse: true}
|
||||
var mapValuesOpType = &operationType{Type: "MAP_VALUES", NumArgs: 1, Precedence: 52, Handler: mapValuesOperator, CheckForPostTraverse: true}
|
||||
|
||||
var formatDateTimeOpType = &operationType{Type: "FORMAT_DATE_TIME", NumArgs: 1, Precedence: 50, Handler: formatDateTime}
|
||||
var withDtFormatOpType = &operationType{Type: "WITH_DATE_TIME_FORMAT", NumArgs: 1, Precedence: 50, Handler: withDateTimeFormat}
|
||||
@@ -113,9 +117,9 @@ var toEntriesOpType = &operationType{Type: "TO_ENTRIES", NumArgs: 0, Precedence:
|
||||
var fromEntriesOpType = &operationType{Type: "FROM_ENTRIES", NumArgs: 0, Precedence: 50, Handler: fromEntriesOperator}
|
||||
var withEntriesOpType = &operationType{Type: "WITH_ENTRIES", NumArgs: 1, Precedence: 50, Handler: withEntriesOperator}
|
||||
|
||||
var withOpType = &operationType{Type: "WITH", NumArgs: 1, Precedence: 50, Handler: withOperator}
|
||||
var withOpType = &operationType{Type: "WITH", NumArgs: 1, Precedence: 52, Handler: withOperator, CheckForPostTraverse: true}
|
||||
|
||||
var splitDocumentOpType = &operationType{Type: "SPLIT_DOC", NumArgs: 0, Precedence: 50, Handler: splitDocumentOperator}
|
||||
var splitDocumentOpType = &operationType{Type: "SPLIT_DOC", NumArgs: 0, Precedence: 52, Handler: splitDocumentOperator, CheckForPostTraverse: true}
|
||||
var getVariableOpType = &operationType{Type: "GET_VARIABLE", NumArgs: 0, Precedence: 55, Handler: getVariableOperator}
|
||||
var getStyleOpType = &operationType{Type: "GET_STYLE", NumArgs: 0, Precedence: 50, Handler: getStyleOperator}
|
||||
var getTagOpType = &operationType{Type: "GET_TAG", NumArgs: 0, Precedence: 50, Handler: getTagOperator}
|
||||
@@ -132,33 +136,33 @@ var getDocumentIndexOpType = &operationType{Type: "GET_DOCUMENT_INDEX", NumArgs:
|
||||
var getFilenameOpType = &operationType{Type: "GET_FILENAME", NumArgs: 0, Precedence: 50, Handler: getFilenameOperator}
|
||||
var getFileIndexOpType = &operationType{Type: "GET_FILE_INDEX", NumArgs: 0, Precedence: 50, Handler: getFileIndexOperator}
|
||||
|
||||
var getPathOpType = &operationType{Type: "GET_PATH", NumArgs: 0, Precedence: 50, Handler: getPathOperator}
|
||||
var getPathOpType = &operationType{Type: "GET_PATH", NumArgs: 0, Precedence: 52, Handler: getPathOperator, CheckForPostTraverse: true}
|
||||
var setPathOpType = &operationType{Type: "SET_PATH", NumArgs: 1, Precedence: 50, Handler: setPathOperator}
|
||||
var delPathsOpType = &operationType{Type: "DEL_PATHS", NumArgs: 1, Precedence: 50, Handler: delPathsOperator}
|
||||
var delPathsOpType = &operationType{Type: "DEL_PATHS", NumArgs: 1, Precedence: 52, Handler: delPathsOperator, CheckForPostTraverse: true}
|
||||
|
||||
var explodeOpType = &operationType{Type: "EXPLODE", NumArgs: 1, Precedence: 50, Handler: explodeOperator}
|
||||
var sortByOpType = &operationType{Type: "SORT_BY", NumArgs: 1, Precedence: 50, Handler: sortByOperator}
|
||||
var reverseOpType = &operationType{Type: "REVERSE", NumArgs: 0, Precedence: 50, Handler: reverseOperator}
|
||||
var sortOpType = &operationType{Type: "SORT", NumArgs: 0, Precedence: 50, Handler: sortOperator}
|
||||
var shuffleOpType = &operationType{Type: "SHUFFLE", NumArgs: 0, Precedence: 50, Handler: shuffleOperator}
|
||||
var explodeOpType = &operationType{Type: "EXPLODE", NumArgs: 1, Precedence: 52, Handler: explodeOperator, CheckForPostTraverse: true}
|
||||
var sortByOpType = &operationType{Type: "SORT_BY", NumArgs: 1, Precedence: 52, Handler: sortByOperator, CheckForPostTraverse: true}
|
||||
var reverseOpType = &operationType{Type: "REVERSE", NumArgs: 0, Precedence: 52, Handler: reverseOperator, CheckForPostTraverse: true}
|
||||
var sortOpType = &operationType{Type: "SORT", NumArgs: 0, Precedence: 52, Handler: sortOperator, CheckForPostTraverse: true}
|
||||
var shuffleOpType = &operationType{Type: "SHUFFLE", NumArgs: 0, Precedence: 52, Handler: shuffleOperator, CheckForPostTraverse: true}
|
||||
|
||||
var sortKeysOpType = &operationType{Type: "SORT_KEYS", NumArgs: 1, Precedence: 50, Handler: sortKeysOperator}
|
||||
var sortKeysOpType = &operationType{Type: "SORT_KEYS", NumArgs: 1, Precedence: 52, Handler: sortKeysOperator, CheckForPostTraverse: true}
|
||||
|
||||
var joinStringOpType = &operationType{Type: "JOIN", NumArgs: 1, Precedence: 50, Handler: joinStringOperator}
|
||||
var subStringOpType = &operationType{Type: "SUBSTR", NumArgs: 1, Precedence: 50, Handler: substituteStringOperator}
|
||||
var matchOpType = &operationType{Type: "MATCH", NumArgs: 1, Precedence: 50, Handler: matchOperator}
|
||||
var captureOpType = &operationType{Type: "CAPTURE", NumArgs: 1, Precedence: 50, Handler: captureOperator}
|
||||
var testOpType = &operationType{Type: "TEST", NumArgs: 1, Precedence: 50, Handler: testOperator}
|
||||
var splitStringOpType = &operationType{Type: "SPLIT", NumArgs: 1, Precedence: 50, Handler: splitStringOperator}
|
||||
var splitStringOpType = &operationType{Type: "SPLIT", NumArgs: 1, Precedence: 52, Handler: splitStringOperator, CheckForPostTraverse: true}
|
||||
var changeCaseOpType = &operationType{Type: "CHANGE_CASE", NumArgs: 0, Precedence: 50, Handler: changeCaseOperator}
|
||||
var trimOpType = &operationType{Type: "TRIM", NumArgs: 0, Precedence: 50, Handler: trimSpaceOperator}
|
||||
var toStringOpType = &operationType{Type: "TO_STRING", NumArgs: 0, Precedence: 50, Handler: toStringOperator}
|
||||
var stringInterpolationOpType = &operationType{Type: "STRING_INT", NumArgs: 0, Precedence: 50, Handler: stringInterpolationOperator, ToString: valueToStringFunc}
|
||||
|
||||
var loadOpType = &operationType{Type: "LOAD", NumArgs: 1, Precedence: 52, Handler: loadOperator}
|
||||
var loadOpType = &operationType{Type: "LOAD", NumArgs: 1, Precedence: 52, Handler: loadOperator, CheckForPostTraverse: true}
|
||||
var loadStringOpType = &operationType{Type: "LOAD_STRING", NumArgs: 1, Precedence: 52, Handler: loadStringOperator}
|
||||
|
||||
var keysOpType = &operationType{Type: "KEYS", NumArgs: 0, Precedence: 50, Handler: keysOperator}
|
||||
var keysOpType = &operationType{Type: "KEYS", NumArgs: 0, Precedence: 52, Handler: keysOperator, CheckForPostTraverse: true}
|
||||
|
||||
var collectObjectOpType = &operationType{Type: "COLLECT_OBJECT", NumArgs: 0, Precedence: 50, Handler: collectObjectOperator}
|
||||
|
||||
@@ -172,7 +176,7 @@ var traverseArrayOpType = &operationType{Type: "TRAVERSE_ARRAY", NumArgs: 2, Pre
|
||||
var selfReferenceOpType = &operationType{Type: "SELF", NumArgs: 0, Precedence: 55, Handler: selfOperator}
|
||||
var valueOpType = &operationType{Type: "VALUE", NumArgs: 0, Precedence: 50, Handler: valueOperator, ToString: valueToStringFunc}
|
||||
var referenceOpType = &operationType{Type: "REF", NumArgs: 0, Precedence: 50, Handler: referenceOperator}
|
||||
var envOpType = &operationType{Type: "ENV", NumArgs: 0, Precedence: 50, Handler: envOperator}
|
||||
var envOpType = &operationType{Type: "ENV", NumArgs: 0, Precedence: 52, Handler: envOperator, CheckForPostTraverse: true}
|
||||
var notOpType = &operationType{Type: "NOT", NumArgs: 0, Precedence: 50, Handler: notOperator}
|
||||
var toNumberOpType = &operationType{Type: "TO_NUMBER", NumArgs: 0, Precedence: 50, Handler: toNumberOperator}
|
||||
var emptyOpType = &operationType{Type: "EMPTY", Precedence: 50, Handler: emptyOperator}
|
||||
@@ -181,14 +185,16 @@ var envsubstOpType = &operationType{Type: "ENVSUBST", NumArgs: 0, Precedence: 50
|
||||
|
||||
var recursiveDescentOpType = &operationType{Type: "RECURSIVE_DESCENT", NumArgs: 0, Precedence: 50, Handler: recursiveDescentOperator}
|
||||
|
||||
var selectOpType = &operationType{Type: "SELECT", NumArgs: 1, Precedence: 50, Handler: selectOperator}
|
||||
var selectOpType = &operationType{Type: "SELECT", NumArgs: 1, Precedence: 52, Handler: selectOperator, CheckForPostTraverse: true}
|
||||
var hasOpType = &operationType{Type: "HAS", NumArgs: 1, Precedence: 50, Handler: hasOperator}
|
||||
var uniqueOpType = &operationType{Type: "UNIQUE", NumArgs: 0, Precedence: 50, Handler: unique}
|
||||
var uniqueByOpType = &operationType{Type: "UNIQUE_BY", NumArgs: 1, Precedence: 50, Handler: uniqueBy}
|
||||
var groupByOpType = &operationType{Type: "GROUP_BY", NumArgs: 1, Precedence: 50, Handler: groupBy}
|
||||
var flattenOpType = &operationType{Type: "FLATTEN_BY", NumArgs: 0, Precedence: 50, Handler: flattenOp}
|
||||
var uniqueOpType = &operationType{Type: "UNIQUE", NumArgs: 0, Precedence: 52, Handler: unique, CheckForPostTraverse: true}
|
||||
var uniqueByOpType = &operationType{Type: "UNIQUE_BY", NumArgs: 1, Precedence: 52, Handler: uniqueBy, CheckForPostTraverse: true}
|
||||
var groupByOpType = &operationType{Type: "GROUP_BY", NumArgs: 1, Precedence: 52, Handler: groupBy, CheckForPostTraverse: true}
|
||||
var flattenOpType = &operationType{Type: "FLATTEN_BY", NumArgs: 0, Precedence: 52, Handler: flattenOp, CheckForPostTraverse: true}
|
||||
var deleteChildOpType = &operationType{Type: "DELETE", NumArgs: 1, Precedence: 40, Handler: deleteChildOperator}
|
||||
|
||||
var pivotOpType = &operationType{Type: "PIVOT", NumArgs: 0, Precedence: 52, Handler: pivotOperator, CheckForPostTraverse: true}
|
||||
|
||||
// debugging purposes only
|
||||
func (p *Operation) toString() string {
|
||||
if p == nil {
|
||||
|
||||
@@ -188,6 +188,16 @@ var anchorOperatorScenarios = []expressionScenario{
|
||||
"D0, P[b], (!!str)::cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Explode splat",
|
||||
skipDoc: true,
|
||||
document: `{a: &a cat, b: *a}`,
|
||||
expression: `explode(.)[]`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!str)::cat\n",
|
||||
"D0, P[b], (!!str)::cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Explode alias and anchor - check original parent",
|
||||
skipDoc: true,
|
||||
|
||||
@@ -28,6 +28,16 @@ var collectObjectOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::dog: great\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "collect splat",
|
||||
skipDoc: true,
|
||||
document: `[{name: cat}, {name: dog}]`,
|
||||
expression: `.[] | {.name: "great"}[]`,
|
||||
expected: []string{
|
||||
"D0, P[cat], (!!str)::great\n",
|
||||
"D0, P[dog], (!!str)::great\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
expression: `({} + {}) | (.b = 3)`,
|
||||
|
||||
@@ -109,6 +109,14 @@ var collectOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::- 1\n- 2\n- 3\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
expression: `[1,2][]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!int)::1\n",
|
||||
"D0, P[1], (!!int)::2\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
document: `a: {b: [1,2,3]}`,
|
||||
expression: `[.a.b.[]]`,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"fmt"
|
||||
"strconv"
|
||||
)
|
||||
@@ -129,3 +130,40 @@ func compareScalars(context Context, prefs compareTypePref, lhs *CandidateNode,
|
||||
|
||||
return false, fmt.Errorf("%v not yet supported for comparison", lhs.Tag)
|
||||
}
|
||||
|
||||
func superlativeByComparison(d *dataTreeNavigator, context Context, prefs compareTypePref) (Context, error) {
|
||||
fn := compare(prefs)
|
||||
|
||||
var results = list.New()
|
||||
|
||||
for seq := context.MatchingNodes.Front(); seq != nil; seq = seq.Next() {
|
||||
splatted, err := splat(context.SingleChildContext(seq.Value.(*CandidateNode)), traversePreferences{})
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
result := splatted.MatchingNodes.Front()
|
||||
if result != nil {
|
||||
for el := result.Next(); el != nil; el = el.Next() {
|
||||
cmp, err := fn(d, context, el.Value.(*CandidateNode), result.Value.(*CandidateNode))
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
if isTruthyNode(cmp) {
|
||||
result = el
|
||||
}
|
||||
}
|
||||
results.PushBack(result.Value)
|
||||
}
|
||||
}
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
|
||||
func minOperator(d *dataTreeNavigator, context Context, _ *ExpressionNode) (Context, error) {
|
||||
log.Debug(("Min"))
|
||||
return superlativeByComparison(d, context, compareTypePref{Greater: false})
|
||||
}
|
||||
|
||||
func maxOperator(d *dataTreeNavigator, context Context, _ *ExpressionNode) (Context, error) {
|
||||
log.Debug(("Max"))
|
||||
return superlativeByComparison(d, context, compareTypePref{Greater: true})
|
||||
}
|
||||
|
||||
@@ -5,6 +5,16 @@ import (
|
||||
)
|
||||
|
||||
var envOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Read string environment variable",
|
||||
skipDoc: true,
|
||||
environmentVariables: map[string]string{"myenv": "[cat,dog]"},
|
||||
expression: `env(myenv)[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!str)::cat\n",
|
||||
"D0, P[1], (!!str)::dog\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Read string environment variable",
|
||||
environmentVariables: map[string]string{"myenv": "cat meow"},
|
||||
|
||||
@@ -14,6 +14,15 @@ var evalOperatorScenarios = []expressionScenario{
|
||||
"D0, P[a b 1], (!!map)::{name: cat}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "eval splat",
|
||||
skipDoc: true,
|
||||
document: `{pathExp: '.a.b[] | select(.name == "cat")', a: {b: [{name: dog}, {name: cat}]}}`,
|
||||
expression: `eval(.pathExp)[]`,
|
||||
expected: []string{
|
||||
"D0, P[a b 1 name], (!!str)::cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Dynamically update a path from an environment variable",
|
||||
subdescription: "The env variable can be any valid yq expression.",
|
||||
|
||||
@@ -13,6 +13,16 @@ var filterOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[1, 2]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Filter array splat",
|
||||
skipDoc: true,
|
||||
document: `[1,2,3]`,
|
||||
expression: `filter(. < 3)[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!int)::1\n",
|
||||
"D0, P[1], (!!int)::2\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Filter map values",
|
||||
document: `{c: {things: cool, frog: yes}, d: {things: hot, frog: false}}`,
|
||||
|
||||
@@ -14,6 +14,17 @@ var flattenOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[1, 2, 3]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Flatten splat",
|
||||
skipDoc: true,
|
||||
document: `[1, [2], [[3]]]`,
|
||||
expression: `flatten[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!int)::1\n",
|
||||
"D0, P[0], (!!int)::2\n",
|
||||
"D0, P[0], (!!int)::3\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Flatten with depth of one",
|
||||
document: `[1, [2], [[3]]]`,
|
||||
@@ -22,6 +33,17 @@ var flattenOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[1, 2, [3]]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Flatten with depth and splat",
|
||||
skipDoc: true,
|
||||
document: `[1, [2], [[3]]]`,
|
||||
expression: `flatten(1)[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!int)::1\n",
|
||||
"D0, P[0], (!!int)::2\n",
|
||||
"D0, P[0], (!!seq)::[3]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Flatten empty array",
|
||||
document: `[[]]`,
|
||||
|
||||
@@ -13,6 +13,16 @@ var groupByOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::- - {foo: 1, bar: 10}\n - {foo: 1, bar: 1}\n- - {foo: 3, bar: 100}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Group splat",
|
||||
skipDoc: true,
|
||||
document: `[{foo: 1, bar: 10}, {foo: 3, bar: 100}, {foo: 1, bar: 1}]`,
|
||||
expression: `group_by(.foo)[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!seq)::- {foo: 1, bar: 10}\n- {foo: 1, bar: 1}\n",
|
||||
"D0, P[1], (!!seq)::- {foo: 3, bar: 100}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Group by field, with nulls",
|
||||
document: `[{cat: dog}, {foo: 1, bar: 10}, {foo: 3, bar: 100}, {no: foo for you}, {foo: 1, bar: 1}]`,
|
||||
|
||||
@@ -39,6 +39,16 @@ var keysOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::- dog\n- cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Map keys with splat",
|
||||
skipDoc: true,
|
||||
document: `{dog: woof, cat: meow}`,
|
||||
expression: `keys[]`,
|
||||
expected: []string{
|
||||
"D0, P[dog], (!!str)::dog\n",
|
||||
"D0, P[cat], (!!str)::cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{}`,
|
||||
|
||||
@@ -15,6 +15,15 @@ var mapOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[6, 7, 8]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[1,2]`,
|
||||
expression: `map(. + 1)[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!int)::2\n",
|
||||
"D0, P[1], (!!int)::3\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Map array",
|
||||
document: `[1,2,3]`,
|
||||
@@ -33,6 +42,16 @@ var mapOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::{x: 11, y: 21, z: 31}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "map values splat",
|
||||
skipDoc: true,
|
||||
document: `{a: 1, b: 2}`,
|
||||
expression: `map_values(. + 1)[]`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!int)::2\n",
|
||||
"D0, P[b], (!!int)::3\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Map object values",
|
||||
document: `{a: 1, b: 2, c: 3}`,
|
||||
|
||||
@@ -91,6 +91,8 @@ func multiplyScalars(lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, er
|
||||
return multiplyIntegers(lhs, rhs)
|
||||
} else if (lhsTag == "!!int" || lhsTag == "!!float") && (rhsTag == "!!int" || rhsTag == "!!float") {
|
||||
return multiplyFloats(lhs, rhs, lhsIsCustom)
|
||||
} else if (lhsTag == "!!str" && rhsTag == "!!int") || (lhsTag == "!!int" && rhsTag == "!!str") {
|
||||
return repeatString(lhs, rhs)
|
||||
}
|
||||
return nil, fmt.Errorf("cannot multiply %v with %v", lhs.Tag, rhs.Tag)
|
||||
}
|
||||
@@ -135,6 +137,28 @@ func multiplyIntegers(lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, e
|
||||
return target, nil
|
||||
}
|
||||
|
||||
func repeatString(lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
var stringNode *CandidateNode
|
||||
var intNode *CandidateNode
|
||||
if lhs.Tag == "!!str" {
|
||||
stringNode = lhs
|
||||
intNode = rhs
|
||||
} else {
|
||||
stringNode = rhs
|
||||
intNode = lhs
|
||||
}
|
||||
target := lhs.CopyWithoutContent()
|
||||
target.UpdateAttributesFrom(stringNode, assignPreferences{})
|
||||
|
||||
count, err := parseInt(intNode.Value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
target.Value = strings.Repeat(stringNode.Value, count)
|
||||
|
||||
return target, nil
|
||||
}
|
||||
|
||||
func mergeObjects(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode, preferences multiplyPreferences) (*CandidateNode, error) {
|
||||
var results = list.New()
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -174,6 +176,40 @@ var multiplyOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::a: 12\nb: 4\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Multiply string node X int",
|
||||
document: docNoComments,
|
||||
expression: ".b * 4",
|
||||
expected: []string{
|
||||
fmt.Sprintf("D0, P[b], (!!str)::%s\n", strings.Repeat("banana", 4)),
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Multiply int X string node",
|
||||
document: docNoComments,
|
||||
expression: "4 * .b",
|
||||
expected: []string{
|
||||
fmt.Sprintf("D0, P[], (!!str)::%s\n", strings.Repeat("banana", 4)),
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Multiply string X int node",
|
||||
document: `n: 4
|
||||
`,
|
||||
expression: `"banana" * .n`,
|
||||
expected: []string{
|
||||
fmt.Sprintf("D0, P[], (!!str)::%s\n", strings.Repeat("banana", 4)),
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Multiply int node X string",
|
||||
document: `n: 4
|
||||
`,
|
||||
expression: `.n * "banana"`,
|
||||
expected: []string{
|
||||
fmt.Sprintf("D0, P[n], (!!str)::%s\n", strings.Repeat("banana", 4)),
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: doc1,
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
func omitMap(original *CandidateNode, indices *CandidateNode) *CandidateNode {
|
||||
filteredContent := make([]*CandidateNode, 0, max(0, len(original.Content)-len(indices.Content)*2))
|
||||
|
||||
for index := 0; index < len(original.Content); index += 2 {
|
||||
pos := findInArray(indices, original.Content[index])
|
||||
if pos < 0 {
|
||||
clonedKey := original.Content[index].Copy()
|
||||
clonedValue := original.Content[index+1].Copy()
|
||||
filteredContent = append(filteredContent, clonedKey, clonedValue)
|
||||
}
|
||||
}
|
||||
result := original.CopyWithoutContent()
|
||||
result.AddChildren(filteredContent)
|
||||
return result
|
||||
}
|
||||
|
||||
func omitSequence(original *CandidateNode, indices *CandidateNode) *CandidateNode {
|
||||
filteredContent := make([]*CandidateNode, 0, max(0, len(original.Content)-len(indices.Content)))
|
||||
|
||||
for index := 0; index < len(original.Content); index++ {
|
||||
pos := findInArray(indices, createScalarNode(index, strconv.Itoa(index)))
|
||||
if pos < 0 {
|
||||
filteredContent = append(filteredContent, original.Content[index].Copy())
|
||||
}
|
||||
}
|
||||
result := original.CopyWithoutContent()
|
||||
result.AddChildren(filteredContent)
|
||||
return result
|
||||
}
|
||||
|
||||
func omitOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("Omit")
|
||||
|
||||
contextIndicesToOmit, err := d.GetMatchingNodes(context, expressionNode.RHS)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
indicesToOmit := &CandidateNode{}
|
||||
if contextIndicesToOmit.MatchingNodes.Len() > 0 {
|
||||
indicesToOmit = contextIndicesToOmit.MatchingNodes.Front().Value.(*CandidateNode)
|
||||
}
|
||||
if len(indicesToOmit.Content) == 0 {
|
||||
log.Debugf("No omit indices specified")
|
||||
return context, nil
|
||||
}
|
||||
var results = list.New()
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
node := el.Value.(*CandidateNode)
|
||||
|
||||
var replacement *CandidateNode
|
||||
|
||||
if node.Kind == MappingNode {
|
||||
replacement = omitMap(node, indicesToOmit)
|
||||
} else if node.Kind == SequenceNode {
|
||||
replacement = omitSequence(node, indicesToOmit)
|
||||
} else {
|
||||
log.Debugf("Omit from type %v (%v) is noop", node.Tag, node.GetNicePath())
|
||||
return context, nil
|
||||
}
|
||||
replacement.LeadingContent = node.LeadingContent
|
||||
results.PushBack(replacement)
|
||||
}
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
var omitOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Omit keys from map",
|
||||
subdescription: "Note that non existent keys are skipped.",
|
||||
document: "myMap: {cat: meow, dog: bark, thing: hamster, hamster: squeak}\n",
|
||||
expression: `.myMap |= omit(["hamster", "cat", "goat"])`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!map)::myMap: {dog: bark, thing: hamster}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Omit splat",
|
||||
skipDoc: true,
|
||||
document: "{cat: meow, dog: bark, hamster: squeak}\n",
|
||||
expression: `omit(["dog"])[]`,
|
||||
expected: []string{
|
||||
"D0, P[cat], (!!str)::meow\n",
|
||||
"D0, P[hamster], (!!str)::squeak\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Omit keys from map",
|
||||
skipDoc: true,
|
||||
document: "!things myMap: {cat: meow, dog: bark, thing: hamster, hamster: squeak}\n",
|
||||
expression: `.myMap |= omit(["hamster", "cat", "goat"])`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!map)::!things myMap: {dog: bark, thing: hamster}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Omit keys from map with comments",
|
||||
skipDoc: true,
|
||||
document: "# abc\nmyMap: {cat: meow, dog: bark, thing: hamster, hamster: squeak}\n# xyz\n",
|
||||
expression: `.myMap |= omit(["hamster", "cat", "goat"])`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!map)::# abc\nmyMap: {dog: bark, thing: hamster}\n# xyz\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Omit indices from array",
|
||||
subdescription: "Note that non existent indices are skipped.",
|
||||
document: `[cat, leopard, lion]`,
|
||||
expression: `omit([2, 0, 734, -5])`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::[leopard]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Omit indices from array with comments",
|
||||
skipDoc: true,
|
||||
document: "# abc\n[cat, leopard, lion]\n# xyz",
|
||||
expression: `omit([2, 0, 734, -5])`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::# abc\n[leopard]\n# xyz\n",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func TestOmitOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range omitOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentOperatorScenarios(t, "omit", omitOperatorScenarios)
|
||||
}
|
||||
@@ -23,6 +23,16 @@ var pathOperatorScenarios = []expressionScenario{
|
||||
"D0, P[a b], (!!seq)::- a\n- b\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Map path",
|
||||
document: `{a: {b: cat}}`,
|
||||
skipDoc: true,
|
||||
expression: `.a.b | path[]`,
|
||||
expected: []string{
|
||||
"D0, P[a b 0], (!!str)::a\n",
|
||||
"D0, P[a b 1], (!!str)::b\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a:
|
||||
@@ -129,6 +139,15 @@ var pathOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::a: [frog]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Delete splat",
|
||||
skipDoc: true,
|
||||
document: `a: [cat, frog]`,
|
||||
expression: `delpaths([["a", 0]])[]`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!seq)::[frog]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Delete - wrong parameter",
|
||||
subdescription: "delpaths does not work with a single path array",
|
||||
|
||||
@@ -14,6 +14,16 @@ var pickOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::myMap: {hamster: squeak, cat: meow}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Pick splat",
|
||||
skipDoc: true,
|
||||
document: "{cat: meow, dog: bark, thing: hamster, hamster: squeak}\n",
|
||||
expression: `pick(["hamster", "cat"])[]`,
|
||||
expected: []string{
|
||||
"D0, P[hamster], (!!str)::squeak\n",
|
||||
"D0, P[cat], (!!str)::meow\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Pick keys from map",
|
||||
skipDoc: true,
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
func getUniqueElementTag(seq *CandidateNode) (string, error) {
|
||||
switch l := len(seq.Content); l {
|
||||
case 0:
|
||||
return "", nil
|
||||
default:
|
||||
result := seq.Content[0].Tag
|
||||
for i := 1; i < l; i++ {
|
||||
t := seq.Content[i].Tag
|
||||
if t != result {
|
||||
return "", fmt.Errorf("sequence contains elements of %v and %v types", result, t)
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
}
|
||||
|
||||
var nullNodeFactory = func() *CandidateNode { return createScalarNode(nil, "") }
|
||||
|
||||
func pad[E any](array []E, length int, factory func() E) []E {
|
||||
sz := len(array)
|
||||
if sz >= length {
|
||||
return array
|
||||
}
|
||||
pad := make([]E, length-sz)
|
||||
for i := 0; i < len(pad); i++ {
|
||||
pad[i] = factory()
|
||||
}
|
||||
return append(array, pad...)
|
||||
}
|
||||
|
||||
func pivotSequences(seq *CandidateNode) *CandidateNode {
|
||||
sz := len(seq.Content)
|
||||
if sz == 0 {
|
||||
return seq
|
||||
}
|
||||
m := make(map[int][]*CandidateNode)
|
||||
|
||||
for i := 0; i < sz; i++ {
|
||||
row := seq.Content[i]
|
||||
for j := 0; j < len(row.Content); j++ {
|
||||
e := m[j]
|
||||
if e == nil {
|
||||
e = make([]*CandidateNode, 0, sz)
|
||||
}
|
||||
m[j] = append(pad(e, i, nullNodeFactory), row.Content[j])
|
||||
}
|
||||
}
|
||||
result := CandidateNode{Kind: SequenceNode}
|
||||
|
||||
for i := 0; i < len(m); i++ {
|
||||
e := CandidateNode{Kind: SequenceNode}
|
||||
e.AddChildren(pad(m[i], sz, nullNodeFactory))
|
||||
result.AddChild(&e)
|
||||
}
|
||||
return &result
|
||||
}
|
||||
|
||||
func pivotMaps(seq *CandidateNode) *CandidateNode {
|
||||
sz := len(seq.Content)
|
||||
if sz == 0 {
|
||||
return &CandidateNode{Kind: MappingNode}
|
||||
}
|
||||
m := make(map[string][]*CandidateNode)
|
||||
keys := make([]string, 0)
|
||||
|
||||
for i := 0; i < sz; i++ {
|
||||
row := seq.Content[i]
|
||||
for j := 0; j < len(row.Content); j += 2 {
|
||||
k := row.Content[j].Value
|
||||
v := row.Content[j+1]
|
||||
e := m[k]
|
||||
if e == nil {
|
||||
keys = append(keys, k)
|
||||
e = make([]*CandidateNode, 0, sz)
|
||||
}
|
||||
m[k] = append(pad(e, i, nullNodeFactory), v)
|
||||
}
|
||||
}
|
||||
result := CandidateNode{Kind: MappingNode}
|
||||
for _, k := range keys {
|
||||
pivotRow := CandidateNode{Kind: SequenceNode}
|
||||
pivotRow.AddChildren(
|
||||
pad(m[k], sz, nullNodeFactory))
|
||||
result.AddKeyValueChild(createScalarNode(k, k), &pivotRow)
|
||||
}
|
||||
return &result
|
||||
}
|
||||
|
||||
func pivotOperator(_ *dataTreeNavigator, context Context, _ *ExpressionNode) (Context, error) {
|
||||
log.Debug("Pivot")
|
||||
results := list.New()
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
if candidate.Tag != "!!seq" {
|
||||
return Context{}, fmt.Errorf("cannot pivot node of type %v", candidate.Tag)
|
||||
}
|
||||
tag, err := getUniqueElementTag(candidate)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
var pivot *CandidateNode
|
||||
switch tag {
|
||||
case "!!seq":
|
||||
pivot = pivotSequences(candidate)
|
||||
case "!!map":
|
||||
pivot = pivotMaps(candidate)
|
||||
default:
|
||||
return Context{}, fmt.Errorf("can only pivot elements of !!seq or !!map types, received %v", tag)
|
||||
}
|
||||
results.PushBack(pivot)
|
||||
}
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
package yqlib
|
||||
|
||||
import "testing"
|
||||
|
||||
var pivotOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Pivot a sequence of sequences",
|
||||
document: "[[foo, bar, baz], [sis, boom, bah]]\n",
|
||||
expression: `pivot`,
|
||||
expected: []string{
|
||||
"D0, P[], ()::- - foo\n - sis\n- - bar\n - boom\n- - baz\n - bah\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Pivot splat",
|
||||
skipDoc: true,
|
||||
document: "[[foo, bar], [sis, boom]]\n",
|
||||
expression: `pivot[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], ()::- foo\n- sis\n",
|
||||
"D0, P[1], ()::- bar\n- boom\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Pivot sequence of heterogeneous sequences",
|
||||
subdescription: `Missing values are "padded" to null.`,
|
||||
document: "[[foo, bar, baz], [sis, boom, bah, blah]]\n",
|
||||
expression: `pivot`,
|
||||
expected: []string{
|
||||
"D0, P[], ()::- - foo\n - sis\n- - bar\n - boom\n- - baz\n - bah\n- -\n - blah\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Pivot sequence of maps",
|
||||
document: "[{foo: a, bar: b, baz: c}, {foo: x, bar: y, baz: z}]\n",
|
||||
expression: `pivot`,
|
||||
expected: []string{
|
||||
"D0, P[], ()::foo:\n - a\n - x\nbar:\n - b\n - y\nbaz:\n - c\n - z\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Pivot sequence of heterogeneous maps",
|
||||
subdescription: `Missing values are "padded" to null.`,
|
||||
document: "[{foo: a, bar: b, baz: c}, {foo: x, bar: y, baz: z, what: ever}]\n",
|
||||
expression: `pivot`,
|
||||
expected: []string{
|
||||
"D0, P[], ()::foo:\n - a\n - x\nbar:\n - b\n - y\nbaz:\n - c\n - z\nwhat:\n -\n - ever\n",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func TestPivotOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range pivotOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentOperatorScenarios(t, "pivot", pivotOperatorScenarios)
|
||||
}
|
||||
@@ -11,6 +11,16 @@ var reverseOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[3, 2, 1]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Reverse",
|
||||
skipDoc: true,
|
||||
document: "[1, 2]",
|
||||
expression: `reverse[]`,
|
||||
expected: []string{
|
||||
"D0, P[1], (!!int)::2\n",
|
||||
"D0, P[0], (!!int)::1\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "[]",
|
||||
|
||||
@@ -75,6 +75,17 @@ var selectOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::b: world\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "select splat",
|
||||
skipDoc: true,
|
||||
document: "a: hello",
|
||||
document2: "b: world",
|
||||
expression: `select(.a == "hello" or .b == "world")[]`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!str)::hello\n",
|
||||
"D0, P[b], (!!str)::world\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "select does not update the map",
|
||||
skipDoc: true,
|
||||
|
||||
@@ -11,6 +11,17 @@ var shuffleOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[5, 2, 4, 1, 3]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Shuffle array",
|
||||
skipDoc: true,
|
||||
document: "[1, 2, 3]",
|
||||
expression: `shuffle[]`,
|
||||
expected: []string{
|
||||
"D0, P[2], (!!int)::3\n",
|
||||
"D0, P[0], (!!int)::1\n",
|
||||
"D0, P[1], (!!int)::2\n",
|
||||
},
|
||||
},
|
||||
|
||||
{
|
||||
description: "Shuffle array in place",
|
||||
|
||||
@@ -13,6 +13,16 @@ var sortKeysOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::{a: blah, b: bing, c: frog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Sort keys of map",
|
||||
skipDoc: true,
|
||||
document: `{c: frog, a: zoo}`,
|
||||
expression: `sort_keys(.)[]`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!str)::zoo\n",
|
||||
"D0, P[c], (!!str)::frog\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{c: frog}`,
|
||||
|
||||
@@ -11,6 +11,16 @@ var sortByOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[{a: apple}, {a: banana}, {a: cat}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Sort by string field",
|
||||
skipDoc: true,
|
||||
document: "[{a: banana},{a: apple}]",
|
||||
expression: `sort_by(.a)[]`,
|
||||
expected: []string{
|
||||
"D0, P[1], (!!map)::{a: apple}\n",
|
||||
"D0, P[0], (!!map)::{a: banana}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Sort by multiple fields",
|
||||
document: "[{a: dog},{a: cat, b: banana},{a: cat, b: apple}]",
|
||||
@@ -104,6 +114,16 @@ var sortByOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[null, false, true, 3, 6, 8, cat]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Sort, nulls come first",
|
||||
skipDoc: true,
|
||||
document: "[8,null]",
|
||||
expression: `sort[]`,
|
||||
expected: []string{
|
||||
"D0, P[1], (!!null)::null\n",
|
||||
"D0, P[0], (!!int)::8\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
description: "false before true",
|
||||
|
||||
@@ -22,6 +22,16 @@ var splitDocOperatorScenarios = []expressionScenario{
|
||||
"D1, P[1], (!!map)::{b: dog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Split splat",
|
||||
skipDoc: true,
|
||||
document: `[{a: cat}, {b: dog}]`,
|
||||
expression: `.[] | split_doc[]`,
|
||||
expected: []string{
|
||||
"D0, P[0 a], (!!str)::cat\n",
|
||||
"D1, P[1 b], (!!str)::dog\n",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func TestSplitDocOperatorScenarios(t *testing.T) {
|
||||
|
||||
@@ -54,43 +54,56 @@ func interpolate(d *dataTreeNavigator, context Context, str string) (string, err
|
||||
for i := 0; i < len(runes); i++ {
|
||||
char := runes[i]
|
||||
if !inExpression {
|
||||
if char == '\\' && i != len(runes)-1 && runes[i+1] == '(' {
|
||||
inExpression = true
|
||||
i = i + 1 // skip over the next open bracket
|
||||
continue
|
||||
if char == '\\' && i < len(runes)-1 {
|
||||
switch runes[i+1] {
|
||||
case '(':
|
||||
inExpression = true
|
||||
// skip the lparen
|
||||
i++
|
||||
continue
|
||||
case '\\':
|
||||
// skip the escaped backslash
|
||||
i++
|
||||
default:
|
||||
log.Debugf("Ignoring non-escaping backslash @ %v[%d]", str, i)
|
||||
}
|
||||
}
|
||||
sb.WriteRune(char)
|
||||
} else { // we are in an expression
|
||||
if char == ')' && nestedBracketsCounter == 0 {
|
||||
// finished the expression!
|
||||
log.Debugf("Expression is :%v", expSb.String())
|
||||
value, err := evaluate(d, context, expSb.String())
|
||||
if err != nil {
|
||||
return "", err
|
||||
if char == ')' {
|
||||
if nestedBracketsCounter == 0 {
|
||||
// finished the expression!
|
||||
log.Debugf("Expression is :%v", expSb.String())
|
||||
value, err := evaluate(d, context, expSb.String())
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
inExpression = false
|
||||
expSb = strings.Builder{} // reset this
|
||||
|
||||
sb.WriteString(value)
|
||||
continue
|
||||
}
|
||||
|
||||
inExpression = false
|
||||
expSb = strings.Builder{} // reset this
|
||||
|
||||
sb.WriteString(value)
|
||||
continue
|
||||
nestedBracketsCounter--
|
||||
} else if char == '(' {
|
||||
nestedBracketsCounter++
|
||||
} else if char == '\\' && i != len(runes)-1 && runes[i+1] == ')' {
|
||||
// close brackets is escaped, skip over it
|
||||
expSb.WriteRune(char)
|
||||
expSb.WriteRune(runes[i+1])
|
||||
i = i + 1
|
||||
continue
|
||||
} else if char == ')' {
|
||||
nestedBracketsCounter--
|
||||
} else if char == '\\' && i < len(runes)-1 {
|
||||
switch esc := runes[i+1]; esc {
|
||||
case ')', '\\':
|
||||
// write escaped character
|
||||
expSb.WriteRune(esc)
|
||||
i++
|
||||
continue
|
||||
default:
|
||||
log.Debugf("Ignoring non-escaping backslash @ %v[%d]", str, i)
|
||||
}
|
||||
}
|
||||
expSb.WriteRune(char)
|
||||
}
|
||||
|
||||
}
|
||||
if inExpression {
|
||||
return "", fmt.Errorf("unclosed interpolation string \\(")
|
||||
log.Warning("unclosed interpolation string, skipping interpolation")
|
||||
return str, nil
|
||||
}
|
||||
return sb.String(), nil
|
||||
}
|
||||
|
||||
@@ -26,7 +26,7 @@ var stringsOperatorScenarios = []expressionScenario{
|
||||
description: "Interpolation - just escape",
|
||||
expression: `"\\"`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!str)::\\\\\n",
|
||||
"D0, P[], (!!str)::\\\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -47,6 +47,15 @@ var stringsOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!str)::Hi (.value)\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
description: "Interpolation - don't!",
|
||||
document: `value: things`,
|
||||
expression: `"Hi \\(.value)"`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!str)::Hi \\(.value)\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
description: "Interpolation - random close bracket",
|
||||
@@ -57,11 +66,22 @@ var stringsOperatorScenarios = []expressionScenario{
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
description: "Interpolation - unclosed interpolation string",
|
||||
document: `value: things`,
|
||||
expression: `"Hi \("`,
|
||||
expectedError: "unclosed interpolation string \\(",
|
||||
skipDoc: true,
|
||||
description: "Interpolation - unclosed interpolation string",
|
||||
document: `value: things`,
|
||||
expression: `"Hi \("`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!str)::Hi \\(\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
description: "Interpolation - unclosed interpolation string due to escape",
|
||||
document: `value: things`,
|
||||
expression: `"Hi \(\)"`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!str)::Hi \\(\\)\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "To up (upper) case",
|
||||
@@ -304,6 +324,16 @@ var stringsOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::- word\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Split splat",
|
||||
skipDoc: true,
|
||||
document: `"word; cat"`,
|
||||
expression: `split("; ")[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!str)::word\n",
|
||||
"D0, P[1], (!!str)::cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `!horse "word"`,
|
||||
|
||||
@@ -23,7 +23,7 @@ func uniqueBy(d *dataTreeNavigator, context Context, expressionNode *ExpressionN
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
|
||||
if candidate.Kind != SequenceNode {
|
||||
return Context{}, fmt.Errorf("Only arrays are supported for unique")
|
||||
return Context{}, fmt.Errorf("only arrays are supported for unique")
|
||||
}
|
||||
|
||||
var newMatches = orderedmap.NewOrderedMap()
|
||||
@@ -34,12 +34,9 @@ func uniqueBy(d *dataTreeNavigator, context Context, expressionNode *ExpressionN
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
keyValue := "null"
|
||||
|
||||
if rhs.MatchingNodes.Len() > 0 {
|
||||
first := rhs.MatchingNodes.Front()
|
||||
keyCandidate := first.Value.(*CandidateNode)
|
||||
keyValue = keyCandidate.Value
|
||||
keyValue, err := getUniqueKeyValue(rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
_, exists := newMatches.Get(keyValue)
|
||||
@@ -59,3 +56,18 @@ func uniqueBy(d *dataTreeNavigator, context Context, expressionNode *ExpressionN
|
||||
return context.ChildContext(results), nil
|
||||
|
||||
}
|
||||
|
||||
func getUniqueKeyValue(rhs Context) (string, error) {
|
||||
keyValue := "null"
|
||||
var err error
|
||||
|
||||
if rhs.MatchingNodes.Len() > 0 {
|
||||
first := rhs.MatchingNodes.Front()
|
||||
keyCandidate := first.Value.(*CandidateNode)
|
||||
keyValue = keyCandidate.Value
|
||||
if keyCandidate.Kind != ScalarNode {
|
||||
keyValue, err = encodeToString(keyCandidate, encoderPreferences{YamlFormat, 0})
|
||||
}
|
||||
}
|
||||
return keyValue, err
|
||||
}
|
||||
|
||||
@@ -14,6 +14,16 @@ var uniqueOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[2, 1, 3]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique splat",
|
||||
skipDoc: true,
|
||||
document: `[2,1,2]`,
|
||||
expression: `unique[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!int)::2\n",
|
||||
"D0, P[1], (!!int)::1\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique nulls",
|
||||
subdescription: "Unique works on the node value, so it considers different representations of nulls to be different",
|
||||
@@ -33,13 +43,29 @@ var uniqueOperatorScenarios = []expressionScenario{
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique array object fields",
|
||||
description: "Unique array objects",
|
||||
document: `[{name: harry, pet: cat}, {name: billy, pet: dog}, {name: harry, pet: cat}]`,
|
||||
expression: `unique`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::[{name: harry, pet: cat}, {name: billy, pet: dog}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique array of objects by a field",
|
||||
document: `[{name: harry, pet: cat}, {name: billy, pet: dog}, {name: harry, pet: dog}]`,
|
||||
expression: `unique_by(.name)`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::[{name: harry, pet: cat}, {name: billy, pet: dog}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique array of arrays",
|
||||
document: `[[cat,dog], [cat, sheep], [cat,dog]]`,
|
||||
expression: `unique`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::[[cat, dog], [cat, sheep]]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{name: harry, pet: cat}, {pet: fish}, {name: harry, pet: dog}]`,
|
||||
@@ -48,6 +74,16 @@ var uniqueOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::[{name: harry, pet: cat}, {pet: fish}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "unique by splat",
|
||||
skipDoc: true,
|
||||
document: `[{name: harry, pet: cat}, {pet: fish}, {name: harry, pet: dog}]`,
|
||||
expression: `unique_by(.name)[]`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!map)::{name: harry, pet: cat}\n",
|
||||
"D0, P[1], (!!map)::{pet: fish}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{name: harry, pet: cat}, {pet: fish}, {name: harry, pet: dog}]`,
|
||||
|
||||
@@ -11,6 +11,15 @@ var withOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!map)::a: {deeply: {nested: 'newValue'}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "with splat",
|
||||
skipDoc: true,
|
||||
document: `a: {deeply: {nested: value}}`,
|
||||
expression: `with(.a.deeply.nested; . = "newValue" | . style="single")[]`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!map)::{deeply: {nested: 'newValue'}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Update multiple deeply nested properties",
|
||||
document: `a: {deeply: {nested: value, other: thing}}`,
|
||||
|
||||
@@ -383,3 +383,67 @@ func TestCompareOperatorScenarios(t *testing.T) {
|
||||
}
|
||||
documentOperatorScenarios(t, "compare", compareOperatorScenarios)
|
||||
}
|
||||
|
||||
var minOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Minimum int",
|
||||
document: "[99, 16, 12, 6, 66]\n",
|
||||
expression: `min`,
|
||||
expected: []string{
|
||||
"D0, P[3], (!!int)::6\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Minimum string",
|
||||
document: "[foo, bar, baz]\n",
|
||||
expression: `min`,
|
||||
expected: []string{
|
||||
"D0, P[1], (!!str)::bar\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Minimum of empty",
|
||||
document: "[]\n",
|
||||
expression: `min`,
|
||||
expected: []string{},
|
||||
},
|
||||
}
|
||||
|
||||
func TestMinOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range minOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentOperatorScenarios(t, "min", minOperatorScenarios)
|
||||
}
|
||||
|
||||
var maxOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Maximum int",
|
||||
document: "[99, 16, 12, 6, 66]\n",
|
||||
expression: `max`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!int)::99\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Maximum string",
|
||||
document: "[foo, bar, baz]\n",
|
||||
expression: `max`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!str)::foo\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Maximum of empty",
|
||||
document: "[]\n",
|
||||
expression: `max`,
|
||||
expected: []string{},
|
||||
},
|
||||
}
|
||||
|
||||
func TestMaxOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range maxOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentOperatorScenarios(t, "max", maxOperatorScenarios)
|
||||
}
|
||||
|
||||
@@ -227,7 +227,14 @@ var propertyScenarios = []formatScenario{
|
||||
expected: "- person\n- name\n",
|
||||
scenarioType: "decode",
|
||||
},
|
||||
|
||||
{
|
||||
description: "Decode properties: numbers",
|
||||
subdescription: "All values are assumed to be strings when parsing properties, but you can use the `from_yaml` operator on all the strings values to autoparse into the correct type.",
|
||||
input: "a.b = 10",
|
||||
expression: " (.. | select(tag == \"!!str\")) |= from_yaml",
|
||||
expected: "a:\n b: 10\n",
|
||||
scenarioType: "decode",
|
||||
},
|
||||
{
|
||||
description: "Decode properties - array should be a map",
|
||||
subdescription: "If you have a numeric map key in your property files, use array_to_map to convert them to maps.",
|
||||
|
||||
+27
-1
@@ -1,10 +1,36 @@
|
||||
4.43.1: (not released yet)
|
||||
4.44.3:
|
||||
- Fixed upper-case file extension detection, Thanks @ryenus (#2121)
|
||||
- Log printing follow no-colors flag #2082
|
||||
- Skip and warn when interpolating strings and theres a unclosed bracket #2083
|
||||
- Fixed CSV content starting with # issue #2076
|
||||
- Bumped dependencies
|
||||
|
||||
4.44.2:
|
||||
- Handle numbers with underscores #2039
|
||||
- Unique now works on maps and arrays #2068
|
||||
- Added support for short hand splat with env[] expression #2071,
|
||||
as well as many other operators (split,select,eval,pick..)
|
||||
- Bumped dependencies
|
||||
|
||||
4.44.1:
|
||||
- Added min/max operators (#1992) Thanks @mbenson
|
||||
- Added pivot oeprator (#1993) Thanks @mbenson
|
||||
- Fix: shell-completion (#2006) Thanks @codekow
|
||||
- Handle escaped backslashes (#1997) Thanks @mbenson
|
||||
- Fix npe when given filename ending with "." (#1994)
|
||||
- Fix: linux (w/ selinux) build (#2004) Thanks @codekow
|
||||
- Bumped dependencies
|
||||
|
||||
4.43.1:
|
||||
- Added omit operator #1989 thanks @mbenson!
|
||||
- Added tostring #72
|
||||
- Added string interpolation #1149
|
||||
- Can specify parent(n) levels #1970
|
||||
- Can now multiply strings by numbers #1988 thanks @mbenson!
|
||||
- Fixed CSV line break issue #1974
|
||||
- Adding a EvaluateAll function to StringEvaluator #1966
|
||||
- yqlib, default to colors off when using yaml library #1964
|
||||
- Removed JSON output warning
|
||||
- Bumped dependencies
|
||||
|
||||
|
||||
|
||||
+4
-2
@@ -1,5 +1,5 @@
|
||||
name: yq
|
||||
version: 'v4.42.1'
|
||||
version: 'v4.44.3'
|
||||
summary: A lightweight and portable command-line data file processor
|
||||
description: |
|
||||
`yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml, json, xml, csv, properties and TOML files.
|
||||
@@ -20,7 +20,9 @@ apps:
|
||||
parts:
|
||||
yq:
|
||||
plugin: go
|
||||
build-environment:
|
||||
- CGO_ENABLED: 0
|
||||
source: https://github.com/mikefarah/yq.git
|
||||
source-tag: v4.42.1
|
||||
source-tag: v4.44.3
|
||||
build-snaps:
|
||||
- go/latest/stable
|
||||
|
||||
Reference in New Issue
Block a user