mirror of
https://github.com/mikefarah/yq.git
synced 2026-07-01 09:51:40 +00:00
feat: K8S KYAML output format support (#2560)
* feat: K8S KYAML output format support Reference: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/5295-kyaml/README.md Co-authored-by: Codex <codex@openai.com> Generated-with: OpenAI Codex CLI (partial) Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com> * build: gomodcache/gocache should not be committed Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com> * chore: fix spelling of behaviour Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> * build: pass GOFLAGS to docker to support buildvcs=false In trying to develop the KYAML support, various tests gave false positive results because they made assumptions about Git functionality Make it possible to avoid that by passing GOFLAGS='-buildvcs=false' to to Makefile. Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> * doc: cover documentScenarios for tests Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com> * build: exclude go caches from gosec Without tuning, gosec scans all of the vendor/gocache/gomodcache, taking several minutes (3m35 here), whereas the core of the yq takes only 15 seconds to scan. If we intend to remediate upstream issues in future; add a seperate target to scan those. Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com> --------- Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org> Co-authored-by: Codex <codex@openai.com>
This commit is contained in:
parent
23abf50fef
commit
c6029376a5
4
.github/ISSUE_TEMPLATE/bug_report_v4.md
vendored
4
.github/ISSUE_TEMPLATE/bug_report_v4.md
vendored
@ -34,13 +34,13 @@ The command you ran:
|
||||
yq eval-all 'select(fileIndex==0) | .a.b.c' data1.yml data2.yml
|
||||
```
|
||||
|
||||
**Actual behavior**
|
||||
**Actual behaviour**
|
||||
|
||||
```yaml
|
||||
cat: meow
|
||||
```
|
||||
|
||||
**Expected behavior**
|
||||
**Expected behaviour**
|
||||
|
||||
```yaml
|
||||
this: should really work
|
||||
|
||||
4
.gitignore
vendored
4
.gitignore
vendored
@ -69,3 +69,7 @@ debian/files
|
||||
.vscode
|
||||
|
||||
yq3
|
||||
|
||||
# Golang
|
||||
.gomodcache/
|
||||
.gocache/
|
||||
|
||||
@ -197,6 +197,21 @@ Note: PRs with small changes (e.g. minor typos) may not be merged (see https://j
|
||||
make [local] test # Run in Docker container
|
||||
```
|
||||
|
||||
- **Problem**: Tests fail with a VCS error:
|
||||
```bash
|
||||
error obtaining VCS status: exit status 128
|
||||
Use -buildvcs=false to disable VCS stamping.
|
||||
```
|
||||
- **Solution**:
|
||||
Git security mechanisms prevent Golang from detecting the Git details inside
|
||||
the container; either build with the `local` option, or pass GOFLAGS to
|
||||
disable Golang buildvcs behaviour.
|
||||
```bash
|
||||
make local test
|
||||
# OR
|
||||
make test GOFLAGS='-buildvcs=true'
|
||||
```
|
||||
|
||||
### Documentation Generation Issues
|
||||
- **Problem**: Generated docs don't update after test changes
|
||||
- **Solution**:
|
||||
|
||||
1
Makefile
1
Makefile
@ -35,6 +35,7 @@ clean:
|
||||
## prefix before other make targets to run in your local dev environment
|
||||
local: | quiet
|
||||
@$(eval ENGINERUN= )
|
||||
@$(eval GOFLAGS="$(GOFLAGS)" )
|
||||
@mkdir -p tmp
|
||||
@touch tmp/dev_image_id
|
||||
quiet: # this is silly but shuts up 'Nothing to be done for `local`'
|
||||
|
||||
@ -4,6 +4,7 @@ IMPORT_PATH := github.com/mikefarah/${PROJECT}
|
||||
export GIT_COMMIT = $(shell git rev-parse --short HEAD)
|
||||
export GIT_DIRTY = $(shell test -n "$$(git status --porcelain)" && echo "+CHANGES" || true)
|
||||
export GIT_DESCRIBE = $(shell git describe --tags --always)
|
||||
GOFLAGS :=
|
||||
LDFLAGS :=
|
||||
LDFLAGS += -X main.GitCommit=${GIT_COMMIT}${GIT_DIRTY}
|
||||
LDFLAGS += -X main.GitDescribe=${GIT_DESCRIBE}
|
||||
@ -33,6 +34,7 @@ DEV_IMAGE := ${PROJECT}_dev
|
||||
|
||||
ENGINERUN := ${ENGINE} run --rm \
|
||||
-e LDFLAGS="${LDFLAGS}" \
|
||||
-e GOFLAGS="${GOFLAGS}" \
|
||||
-e GITHUB_TOKEN="${GITHUB_TOKEN}" \
|
||||
-v ${ROOT}/vendor:/go/src${SELINUX} \
|
||||
-v ${ROOT}:/${PROJECT}/src/${IMPORT_PATH}${SELINUX} \
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
    
|
||||
|
||||
|
||||
A lightweight and portable command-line YAML, JSON, INI and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) (a popular JSON processor) like syntax but works with yaml files as well as json, xml, ini, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
|
||||
A lightweight and portable command-line YAML, JSON, INI and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) (a popular JSON processor) like syntax but works with yaml files as well as json, kyaml, xml, ini, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
|
||||
|
||||
yq is written in Go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as Docker and Podman, all listed below.
|
||||
|
||||
@ -415,7 +415,7 @@ Flags:
|
||||
-h, --help help for yq
|
||||
-I, --indent int sets indent level for output (default 2)
|
||||
-i, --inplace update the file in place of first file given.
|
||||
-p, --input-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|hcl|h|lua|l|ini|i] parse format for input. (default "auto")
|
||||
-p, --input-format string [auto|a|yaml|y|json|j|kyaml|ky|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|hcl|h|lua|l|ini|i] parse format for input. (default "auto")
|
||||
--lua-globals output keys as top-level global variables
|
||||
--lua-prefix string prefix (default "return ")
|
||||
--lua-suffix string suffix (default ";\n")
|
||||
@ -424,7 +424,7 @@ Flags:
|
||||
-N, --no-doc Don't print document separators (---)
|
||||
-0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.
|
||||
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
|
||||
-o, --output-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|hcl|h|shell|s|lua|l|ini|i] output format type. (default "auto")
|
||||
-o, --output-format string [auto|a|yaml|y|json|j|kyaml|ky|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|hcl|h|shell|s|lua|l|ini|i] output format type. (default "auto")
|
||||
-P, --prettyPrint pretty print, shorthand for '... style = ""'
|
||||
--properties-array-brackets use [x] in array paths (e.g. for SpringBoot)
|
||||
--properties-separator string separator to use between keys and values (default " = ")
|
||||
|
||||
@ -154,6 +154,37 @@ EOM
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testInputKYaml() {
|
||||
cat >test.kyaml <<'EOL'
|
||||
# leading
|
||||
{
|
||||
a: 1, # a line
|
||||
# head b
|
||||
b: 2,
|
||||
c: [
|
||||
# head d
|
||||
"d", # d line
|
||||
],
|
||||
}
|
||||
EOL
|
||||
|
||||
read -r -d '' expected <<'EOM'
|
||||
# leading
|
||||
a: 1 # a line
|
||||
# head b
|
||||
b: 2
|
||||
c:
|
||||
# head d
|
||||
- d # d line
|
||||
EOM
|
||||
|
||||
X=$(./yq e -p=kyaml -P test.kyaml)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea -p=kyaml -P test.kyaml)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
@ -313,4 +344,4 @@ EOM
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
source ./scripts/shunit2
|
||||
source ./scripts/shunit2
|
||||
|
||||
@ -280,6 +280,55 @@ EOM
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testOutputKYaml() {
|
||||
cat >test.yml <<'EOL'
|
||||
# leading
|
||||
a: 1 # a line
|
||||
# head b
|
||||
b: 2
|
||||
c:
|
||||
# head d
|
||||
- d # d line
|
||||
EOL
|
||||
|
||||
read -r -d '' expected <<'EOM'
|
||||
# leading
|
||||
{
|
||||
a: 1, # a line
|
||||
# head b
|
||||
b: 2,
|
||||
c: [
|
||||
# head d
|
||||
"d", # d line
|
||||
],
|
||||
}
|
||||
EOM
|
||||
|
||||
X=$(./yq e --output-format=kyaml test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea --output-format=kyaml test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testOutputKYamlShort() {
|
||||
cat >test.yml <<EOL
|
||||
a: b
|
||||
EOL
|
||||
|
||||
read -r -d '' expected <<'EOM'
|
||||
{
|
||||
a: "b",
|
||||
}
|
||||
EOM
|
||||
|
||||
X=$(./yq e -o=ky test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
|
||||
X=$(./yq ea -o=ky test.yml)
|
||||
assertEquals "$expected" "$X"
|
||||
}
|
||||
|
||||
testOutputXmComplex() {
|
||||
cat >test.yml <<EOL
|
||||
a: {b: {c: ["cat", "dog"], +@f: meow}}
|
||||
|
||||
@ -83,6 +83,7 @@ Create a test file `pkg/yqlib/<format>_test.go` using the `formatScenario` patte
|
||||
- `scenarioType` can be `"decode"` (test decoding to YAML) or `"roundtrip"` (encode/decode preservation)
|
||||
- Create a helper function `test<Format>Scenario()` that switches on `scenarioType`
|
||||
- Create main test function `Test<Format>FormatScenarios()` that iterates over scenarios
|
||||
- The main test function should use `documentScenarios` to ensure testcase documentation is generated.
|
||||
|
||||
Test coverage must include:
|
||||
- Basic data types (scalars, arrays, objects/maps)
|
||||
@ -338,6 +339,7 @@ Create `pkg/yqlib/operator_<type>_test.go` using the `expressionScenario` patter
|
||||
- Include `subdescription` for longer test names
|
||||
- Set `expectedError` if testing error cases
|
||||
- Create main test function that iterates over scenarios
|
||||
- The main test function should use `documentScenarios` to ensure testcase documentation is generated.
|
||||
|
||||
Test coverage must include:
|
||||
- Basic data types and nested structures
|
||||
|
||||
@ -166,6 +166,9 @@ func configureDecoder(evaluateTogether bool) (yqlib.Decoder, error) {
|
||||
}
|
||||
yqlib.ConfiguredYamlPreferences.EvaluateTogether = evaluateTogether
|
||||
|
||||
if format.DecoderFactory == nil {
|
||||
return nil, fmt.Errorf("no support for %s input format", inputFormat)
|
||||
}
|
||||
yqlibDecoder := format.DecoderFactory()
|
||||
if yqlibDecoder == nil {
|
||||
return nil, fmt.Errorf("no support for %s input format", inputFormat)
|
||||
@ -197,18 +200,22 @@ func configureEncoder() (yqlib.Encoder, error) {
|
||||
}
|
||||
yqlib.ConfiguredXMLPreferences.Indent = indent
|
||||
yqlib.ConfiguredYamlPreferences.Indent = indent
|
||||
yqlib.ConfiguredKYamlPreferences.Indent = indent
|
||||
yqlib.ConfiguredJSONPreferences.Indent = indent
|
||||
|
||||
yqlib.ConfiguredYamlPreferences.UnwrapScalar = unwrapScalar
|
||||
yqlib.ConfiguredKYamlPreferences.UnwrapScalar = unwrapScalar
|
||||
yqlib.ConfiguredPropertiesPreferences.UnwrapScalar = unwrapScalar
|
||||
yqlib.ConfiguredJSONPreferences.UnwrapScalar = unwrapScalar
|
||||
|
||||
yqlib.ConfiguredYamlPreferences.ColorsEnabled = colorsEnabled
|
||||
yqlib.ConfiguredKYamlPreferences.ColorsEnabled = colorsEnabled
|
||||
yqlib.ConfiguredJSONPreferences.ColorsEnabled = colorsEnabled
|
||||
yqlib.ConfiguredHclPreferences.ColorsEnabled = colorsEnabled
|
||||
yqlib.ConfiguredTomlPreferences.ColorsEnabled = colorsEnabled
|
||||
|
||||
yqlib.ConfiguredYamlPreferences.PrintDocSeparators = !noDocSeparators
|
||||
yqlib.ConfiguredKYamlPreferences.PrintDocSeparators = !noDocSeparators
|
||||
|
||||
encoder := yqlibOutputFormat.EncoderFactory()
|
||||
|
||||
|
||||
10
examples/kyaml.kyaml
Normal file
10
examples/kyaml.kyaml
Normal file
@ -0,0 +1,10 @@
|
||||
# leading
|
||||
{
|
||||
a: 1, # a line
|
||||
# head b
|
||||
b: 2,
|
||||
c: [
|
||||
# head d
|
||||
"d", # d line
|
||||
],
|
||||
}
|
||||
7
examples/kyaml.yml
Normal file
7
examples/kyaml.yml
Normal file
@ -0,0 +1,7 @@
|
||||
# leading
|
||||
a: 1 # a line
|
||||
# head b
|
||||
b: 2
|
||||
c:
|
||||
# head d
|
||||
- d # d line
|
||||
9
pkg/yqlib/doc/usage/headers/kyaml.md
Normal file
9
pkg/yqlib/doc/usage/headers/kyaml.md
Normal file
@ -0,0 +1,9 @@
|
||||
# KYaml
|
||||
|
||||
Encode and decode to and from KYaml (a restricted subset of YAML that uses flow-style collections).
|
||||
|
||||
KYaml is useful when you want YAML data rendered in a compact, JSON-like form while still supporting YAML features like comments.
|
||||
|
||||
Notes:
|
||||
- Strings are always double-quoted in KYaml output.
|
||||
- Anchors and aliases are expanded (KYaml output does not emit them).
|
||||
253
pkg/yqlib/doc/usage/kyaml.md
Normal file
253
pkg/yqlib/doc/usage/kyaml.md
Normal file
@ -0,0 +1,253 @@
|
||||
# KYaml
|
||||
|
||||
Encode and decode to and from KYaml (a restricted subset of YAML that uses flow-style collections).
|
||||
|
||||
KYaml is useful when you want YAML data rendered in a compact, JSON-like form while still supporting YAML features like comments.
|
||||
|
||||
Notes:
|
||||
- Strings are always double-quoted in KYaml output.
|
||||
- Anchors and aliases are expanded (KYaml output does not emit them).
|
||||
|
||||
## Encode kyaml: plain string scalar
|
||||
Strings are always double-quoted in KYaml output.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
"cat"
|
||||
```
|
||||
|
||||
## encode flow mapping and sequence
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: b
|
||||
c:
|
||||
- d
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
a: "b",
|
||||
c: [
|
||||
"d",
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
## encode non-string scalars
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 12
|
||||
b: true
|
||||
c: null
|
||||
d: "true"
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
a: 12,
|
||||
b: true,
|
||||
c: null,
|
||||
d: "true",
|
||||
}
|
||||
```
|
||||
|
||||
## quote non-identifier keys
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
"1a": b
|
||||
"has space": c
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
"1a": "b",
|
||||
"has space": "c",
|
||||
}
|
||||
```
|
||||
|
||||
## escape quoted strings
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: "line1\nline2\t\"q\""
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
a: "line1\nline2\t\"q\"",
|
||||
}
|
||||
```
|
||||
|
||||
## preserve comments when encoding
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
# leading
|
||||
a: 1 # a line
|
||||
# head b
|
||||
b: 2
|
||||
c:
|
||||
# head d
|
||||
- d # d line
|
||||
- e
|
||||
# trailing
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
# leading
|
||||
{
|
||||
a: 1, # a line
|
||||
# head b
|
||||
b: 2,
|
||||
c: [
|
||||
# head d
|
||||
"d", # d line
|
||||
"e",
|
||||
],
|
||||
# trailing
|
||||
}
|
||||
```
|
||||
|
||||
## Encode kyaml: anchors and aliases
|
||||
KYaml output does not support anchors/aliases; they are expanded to concrete values.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
base: &base
|
||||
a: b
|
||||
copy: *base
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
base: {
|
||||
a: "b",
|
||||
},
|
||||
copy: {
|
||||
a: "b",
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
## Encode kyaml: yaml to kyaml shows formatting differences
|
||||
KYaml uses flow-style collections (braces/brackets) and explicit commas.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
person:
|
||||
name: John
|
||||
pets:
|
||||
- cat
|
||||
- dog
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
{
|
||||
person: {
|
||||
name: "John",
|
||||
pets: [
|
||||
"cat",
|
||||
"dog",
|
||||
],
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
## Encode kyaml: nested lists of objects
|
||||
Lists and objects can be nested arbitrarily; KYaml always uses flow-style collections.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- name: a
|
||||
items:
|
||||
- id: 1
|
||||
tags:
|
||||
- k: x
|
||||
v: y
|
||||
- k: x2
|
||||
v: y2
|
||||
- id: 2
|
||||
tags:
|
||||
- k: z
|
||||
v: w
|
||||
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq -o=kyaml '.' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
[
|
||||
{
|
||||
name: "a",
|
||||
items: [
|
||||
{
|
||||
id: 1,
|
||||
tags: [
|
||||
{
|
||||
k: "x",
|
||||
v: "y",
|
||||
},
|
||||
{
|
||||
k: "x2",
|
||||
v: "y2",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 2,
|
||||
tags: [
|
||||
{
|
||||
k: "z",
|
||||
v: "w",
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
]
|
||||
```
|
||||
|
||||
@ -1,7 +1,12 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"github.com/fatih/color"
|
||||
)
|
||||
|
||||
type Encoder interface {
|
||||
@ -25,3 +30,63 @@ func mapKeysToStrings(node *CandidateNode) {
|
||||
mapKeysToStrings(child)
|
||||
}
|
||||
}
|
||||
|
||||
// Some funcs are shared between encoder_yaml and encoder_kyaml
|
||||
func PrintYAMLDocumentSeparator(writer io.Writer, PrintDocSeparators bool) error {
|
||||
if PrintDocSeparators {
|
||||
log.Debug("writing doc sep")
|
||||
if err := writeString(writer, "---\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func PrintYAMLLeadingContent(writer io.Writer, content string, PrintDocSeparators bool, ColorsEnabled bool) error {
|
||||
reader := bufio.NewReader(strings.NewReader(content))
|
||||
|
||||
// reuse precompiled package-level regex
|
||||
// (declared in decoder_yaml.go)
|
||||
|
||||
for {
|
||||
|
||||
readline, errReading := reader.ReadString('\n')
|
||||
if errReading != nil && !errors.Is(errReading, io.EOF) {
|
||||
return errReading
|
||||
}
|
||||
if strings.Contains(readline, "$yqDocSeparator$") {
|
||||
// Preserve the original line ending (CRLF or LF)
|
||||
lineEnding := "\n"
|
||||
if strings.HasSuffix(readline, "\r\n") {
|
||||
lineEnding = "\r\n"
|
||||
}
|
||||
if PrintDocSeparators {
|
||||
if err := writeString(writer, "---"+lineEnding); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
} else {
|
||||
if len(readline) > 0 && readline != "\n" && readline[0] != '%' && !commentLineRe.MatchString(readline) {
|
||||
readline = "# " + readline
|
||||
}
|
||||
if ColorsEnabled && strings.TrimSpace(readline) != "" {
|
||||
readline = format(color.FgHiBlack) + readline + format(color.Reset)
|
||||
}
|
||||
if err := writeString(writer, readline); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if errors.Is(errReading, io.EOF) {
|
||||
if readline != "" {
|
||||
// the last comment we read didn't have a newline, put one in
|
||||
if err := writeString(writer, "\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
318
pkg/yqlib/encoder_kyaml.go
Normal file
318
pkg/yqlib/encoder_kyaml.go
Normal file
@ -0,0 +1,318 @@
|
||||
//go:build !yq_nokyaml
|
||||
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type kyamlEncoder struct {
|
||||
prefs KYamlPreferences
|
||||
}
|
||||
|
||||
func NewKYamlEncoder(prefs KYamlPreferences) Encoder {
|
||||
return &kyamlEncoder{prefs: prefs}
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) CanHandleAliases() bool {
|
||||
// KYAML is a restricted subset; avoid emitting anchors/aliases.
|
||||
return false
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) PrintDocumentSeparator(writer io.Writer) error {
|
||||
return PrintYAMLDocumentSeparator(writer, ke.prefs.PrintDocSeparators)
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) PrintLeadingContent(writer io.Writer, content string) error {
|
||||
return PrintYAMLLeadingContent(writer, content, ke.prefs.PrintDocSeparators, ke.prefs.ColorsEnabled)
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) Encode(writer io.Writer, node *CandidateNode) error {
|
||||
log.Debug("encoderKYaml - going to print %v", NodeToString(node))
|
||||
if node.Kind == ScalarNode && ke.prefs.UnwrapScalar {
|
||||
return writeString(writer, node.Value+"\n")
|
||||
}
|
||||
|
||||
destination := writer
|
||||
tempBuffer := bytes.NewBuffer(nil)
|
||||
if ke.prefs.ColorsEnabled {
|
||||
destination = tempBuffer
|
||||
}
|
||||
|
||||
// Mirror the YAML encoder behaviour: trailing comments on the document root
|
||||
// are stored in FootComment and need to be printed after the document.
|
||||
trailingContent := node.FootComment
|
||||
|
||||
if err := ke.writeCommentBlock(destination, node.HeadComment, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.writeNode(destination, node, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.writeInlineComment(destination, node.LineComment); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeString(destination, "\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.PrintLeadingContent(destination, trailingContent); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if ke.prefs.ColorsEnabled {
|
||||
return colorizeAndPrint(tempBuffer.Bytes(), writer)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) writeNode(writer io.Writer, node *CandidateNode, indent int) error {
|
||||
switch node.Kind {
|
||||
case MappingNode:
|
||||
return ke.writeMapping(writer, node, indent)
|
||||
case SequenceNode:
|
||||
return ke.writeSequence(writer, node, indent)
|
||||
case ScalarNode:
|
||||
return writeString(writer, ke.formatScalar(node))
|
||||
case AliasNode:
|
||||
// Should have been exploded by the printer, but handle defensively.
|
||||
if node.Alias == nil {
|
||||
return writeString(writer, "null")
|
||||
}
|
||||
return ke.writeNode(writer, node.Alias, indent)
|
||||
default:
|
||||
return writeString(writer, "null")
|
||||
}
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) writeMapping(writer io.Writer, node *CandidateNode, indent int) error {
|
||||
if len(node.Content) == 0 {
|
||||
return writeString(writer, "{}")
|
||||
}
|
||||
if err := writeString(writer, "{\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for i := 0; i+1 < len(node.Content); i += 2 {
|
||||
keyNode := node.Content[i]
|
||||
valueNode := node.Content[i+1]
|
||||
|
||||
entryIndent := indent + ke.prefs.Indent
|
||||
if err := ke.writeCommentBlock(writer, keyNode.HeadComment, entryIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
if valueNode.HeadComment != "" && valueNode.HeadComment != keyNode.HeadComment {
|
||||
if err := ke.writeCommentBlock(writer, valueNode.HeadComment, entryIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if err := ke.writeIndent(writer, entryIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeString(writer, ke.formatKey(keyNode)); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeString(writer, ": "); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.writeNode(writer, valueNode, entryIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Always emit a trailing comma; KYAML encourages explicit separators,
|
||||
// and this ensures all quoted strings have a trailing `",` as requested.
|
||||
if err := writeString(writer, ","); err != nil {
|
||||
return err
|
||||
}
|
||||
inline := valueNode.LineComment
|
||||
if inline == "" {
|
||||
inline = keyNode.LineComment
|
||||
}
|
||||
if err := ke.writeInlineComment(writer, inline); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeString(writer, "\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
foot := valueNode.FootComment
|
||||
if foot == "" {
|
||||
foot = keyNode.FootComment
|
||||
}
|
||||
if err := ke.writeCommentBlock(writer, foot, entryIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if err := ke.writeIndent(writer, indent); err != nil {
|
||||
return err
|
||||
}
|
||||
return writeString(writer, "}")
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) writeSequence(writer io.Writer, node *CandidateNode, indent int) error {
|
||||
if len(node.Content) == 0 {
|
||||
return writeString(writer, "[]")
|
||||
}
|
||||
if err := writeString(writer, "[\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, child := range node.Content {
|
||||
itemIndent := indent + ke.prefs.Indent
|
||||
if err := ke.writeCommentBlock(writer, child.HeadComment, itemIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.writeIndent(writer, itemIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.writeNode(writer, child, itemIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeString(writer, ","); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.writeInlineComment(writer, child.LineComment); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeString(writer, "\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := ke.writeCommentBlock(writer, child.FootComment, itemIndent); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if err := ke.writeIndent(writer, indent); err != nil {
|
||||
return err
|
||||
}
|
||||
return writeString(writer, "]")
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) writeIndent(writer io.Writer, indent int) error {
|
||||
if indent <= 0 {
|
||||
return nil
|
||||
}
|
||||
return writeString(writer, strings.Repeat(" ", indent))
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) formatKey(keyNode *CandidateNode) string {
|
||||
// KYAML examples use bare keys. Quote keys only when needed.
|
||||
key := keyNode.Value
|
||||
if isValidKYamlBareKey(key) {
|
||||
return key
|
||||
}
|
||||
return `"` + escapeDoubleQuotedString(key) + `"`
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) formatScalar(node *CandidateNode) string {
|
||||
switch node.Tag {
|
||||
case "!!null":
|
||||
return "null"
|
||||
case "!!bool":
|
||||
return strings.ToLower(node.Value)
|
||||
case "!!int", "!!float":
|
||||
return node.Value
|
||||
case "!!str":
|
||||
return `"` + escapeDoubleQuotedString(node.Value) + `"`
|
||||
default:
|
||||
// Fall back to a string representation to avoid implicit typing surprises.
|
||||
return `"` + escapeDoubleQuotedString(node.Value) + `"`
|
||||
}
|
||||
}
|
||||
|
||||
var kyamlBareKeyRe = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_-]*$`)
|
||||
|
||||
func isValidKYamlBareKey(s string) bool {
|
||||
// Conservative: require an identifier-like key; otherwise quote.
|
||||
if s == "" {
|
||||
return false
|
||||
}
|
||||
return kyamlBareKeyRe.MatchString(s)
|
||||
}
|
||||
|
||||
func escapeDoubleQuotedString(s string) string {
|
||||
var b strings.Builder
|
||||
b.Grow(len(s) + 2)
|
||||
|
||||
for _, r := range s {
|
||||
switch r {
|
||||
case '\\':
|
||||
b.WriteString(`\\`)
|
||||
case '"':
|
||||
b.WriteString(`\"`)
|
||||
case '\n':
|
||||
b.WriteString(`\n`)
|
||||
case '\r':
|
||||
b.WriteString(`\r`)
|
||||
case '\t':
|
||||
b.WriteString(`\t`)
|
||||
default:
|
||||
if r < 0x20 {
|
||||
// YAML double-quoted strings support \uXXXX escapes.
|
||||
b.WriteString(`\u`)
|
||||
hex := "0000" + strings.ToUpper(strconv.FormatInt(int64(r), 16))
|
||||
b.WriteString(hex[len(hex)-4:])
|
||||
} else {
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) writeCommentBlock(writer io.Writer, comment string, indent int) error {
|
||||
if strings.TrimSpace(comment) == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
lines := strings.Split(strings.ReplaceAll(comment, "\r\n", "\n"), "\n")
|
||||
for _, line := range lines {
|
||||
trimmed := strings.TrimSpace(line)
|
||||
if trimmed == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
if err := ke.writeIndent(writer, indent); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
toWrite := line
|
||||
if !commentLineRe.MatchString(toWrite) {
|
||||
toWrite = "# " + toWrite
|
||||
}
|
||||
if err := writeString(writer, toWrite); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeString(writer, "\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (ke *kyamlEncoder) writeInlineComment(writer io.Writer, comment string) error {
|
||||
comment = strings.TrimSpace(strings.ReplaceAll(comment, "\r\n", "\n"))
|
||||
if comment == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
lines := strings.Split(comment, "\n")
|
||||
first := strings.TrimSpace(lines[0])
|
||||
if first == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(first, "#") {
|
||||
first = "# " + first
|
||||
}
|
||||
|
||||
if err := writeString(writer, " "); err != nil {
|
||||
return err
|
||||
}
|
||||
return writeString(writer, first)
|
||||
}
|
||||
@ -1,13 +1,10 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"errors"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"github.com/fatih/color"
|
||||
"go.yaml.in/yaml/v4"
|
||||
)
|
||||
|
||||
@ -24,63 +21,11 @@ func (ye *yamlEncoder) CanHandleAliases() bool {
|
||||
}
|
||||
|
||||
func (ye *yamlEncoder) PrintDocumentSeparator(writer io.Writer) error {
|
||||
if ye.prefs.PrintDocSeparators {
|
||||
log.Debug("writing doc sep")
|
||||
if err := writeString(writer, "---\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
return PrintYAMLDocumentSeparator(writer, ye.prefs.PrintDocSeparators)
|
||||
}
|
||||
|
||||
func (ye *yamlEncoder) PrintLeadingContent(writer io.Writer, content string) error {
|
||||
reader := bufio.NewReader(strings.NewReader(content))
|
||||
|
||||
// reuse precompiled package-level regex
|
||||
// (declared in decoder_yaml.go)
|
||||
|
||||
for {
|
||||
|
||||
readline, errReading := reader.ReadString('\n')
|
||||
if errReading != nil && !errors.Is(errReading, io.EOF) {
|
||||
return errReading
|
||||
}
|
||||
if strings.Contains(readline, "$yqDocSeparator$") {
|
||||
// Preserve the original line ending (CRLF or LF)
|
||||
lineEnding := "\n"
|
||||
if strings.HasSuffix(readline, "\r\n") {
|
||||
lineEnding = "\r\n"
|
||||
}
|
||||
if ye.prefs.PrintDocSeparators {
|
||||
if err := writeString(writer, "---"+lineEnding); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
} else {
|
||||
if len(readline) > 0 && readline != "\n" && readline[0] != '%' && !commentLineRe.MatchString(readline) {
|
||||
readline = "# " + readline
|
||||
}
|
||||
if ye.prefs.ColorsEnabled && strings.TrimSpace(readline) != "" {
|
||||
readline = format(color.FgHiBlack) + readline + format(color.Reset)
|
||||
}
|
||||
if err := writeString(writer, readline); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if errors.Is(errReading, io.EOF) {
|
||||
if readline != "" {
|
||||
// the last comment we read didn't have a newline, put one in
|
||||
if err := writeString(writer, "\n"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
return PrintYAMLLeadingContent(writer, content, ye.prefs.PrintDocSeparators, ye.prefs.ColorsEnabled)
|
||||
}
|
||||
|
||||
func (ye *yamlEncoder) Encode(writer io.Writer, node *CandidateNode) error {
|
||||
|
||||
@ -22,6 +22,12 @@ var YamlFormat = &Format{"yaml", []string{"y", "yml"},
|
||||
func() Decoder { return NewYamlDecoder(ConfiguredYamlPreferences) },
|
||||
}
|
||||
|
||||
var KYamlFormat = &Format{"kyaml", []string{"ky"},
|
||||
func() Encoder { return NewKYamlEncoder(ConfiguredKYamlPreferences) },
|
||||
// KYaml is stricter YAML
|
||||
func() Decoder { return NewYamlDecoder(ConfiguredYamlPreferences) },
|
||||
}
|
||||
|
||||
var JSONFormat = &Format{"json", []string{"j"},
|
||||
func() Encoder { return NewJSONEncoder(ConfiguredJSONPreferences) },
|
||||
func() Decoder { return NewJSONDecoder() },
|
||||
@ -89,6 +95,7 @@ var INIFormat = &Format{"ini", []string{"i"},
|
||||
|
||||
var Formats = []*Format{
|
||||
YamlFormat,
|
||||
KYamlFormat,
|
||||
JSONFormat,
|
||||
PropertiesFormat,
|
||||
CSVFormat,
|
||||
|
||||
30
pkg/yqlib/kyaml.go
Normal file
30
pkg/yqlib/kyaml.go
Normal file
@ -0,0 +1,30 @@
|
||||
//go:build !yq_nokyaml
|
||||
|
||||
package yqlib
|
||||
|
||||
type KYamlPreferences struct {
|
||||
Indent int
|
||||
ColorsEnabled bool
|
||||
PrintDocSeparators bool
|
||||
UnwrapScalar bool
|
||||
}
|
||||
|
||||
func NewDefaultKYamlPreferences() KYamlPreferences {
|
||||
return KYamlPreferences{
|
||||
Indent: 2,
|
||||
ColorsEnabled: false,
|
||||
PrintDocSeparators: true,
|
||||
UnwrapScalar: true,
|
||||
}
|
||||
}
|
||||
|
||||
func (p *KYamlPreferences) Copy() KYamlPreferences {
|
||||
return KYamlPreferences{
|
||||
Indent: p.Indent,
|
||||
ColorsEnabled: p.ColorsEnabled,
|
||||
PrintDocSeparators: p.PrintDocSeparators,
|
||||
UnwrapScalar: p.UnwrapScalar,
|
||||
}
|
||||
}
|
||||
|
||||
var ConfiguredKYamlPreferences = NewDefaultKYamlPreferences()
|
||||
542
pkg/yqlib/kyaml_test.go
Normal file
542
pkg/yqlib/kyaml_test.go
Normal file
@ -0,0 +1,542 @@
|
||||
//go:build !yq_nokyaml
|
||||
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/mikefarah/yq/v4/test"
|
||||
)
|
||||
|
||||
var ansiRe = regexp.MustCompile(`\x1b\[[0-9;]*m`)
|
||||
|
||||
func stripANSI(s string) string {
|
||||
return ansiRe.ReplaceAllString(s, "")
|
||||
}
|
||||
|
||||
var kyamlFormatScenarios = []formatScenario{
|
||||
{
|
||||
description: "Encode kyaml: plain string scalar",
|
||||
subdescription: "Strings are always double-quoted in KYaml output.",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "cat\n",
|
||||
expected: "\"cat\"\n",
|
||||
},
|
||||
{
|
||||
description: "encode plain int scalar",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "12\n",
|
||||
expected: "12\n",
|
||||
skipDoc: true,
|
||||
},
|
||||
{
|
||||
description: "encode plain bool scalar",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "true\n",
|
||||
expected: "true\n",
|
||||
skipDoc: true,
|
||||
},
|
||||
{
|
||||
description: "encode plain null scalar",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "null\n",
|
||||
expected: "null\n",
|
||||
skipDoc: true,
|
||||
},
|
||||
{
|
||||
description: "encode flow mapping and sequence",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "a: b\nc:\n - d\n",
|
||||
expected: "{\n" +
|
||||
" a: \"b\",\n" +
|
||||
" c: [\n" +
|
||||
" \"d\",\n" +
|
||||
" ],\n" +
|
||||
"}\n",
|
||||
},
|
||||
{
|
||||
description: "encode non-string scalars",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "a: 12\n" +
|
||||
"b: true\n" +
|
||||
"c: null\n" +
|
||||
"d: \"true\"\n",
|
||||
expected: "{\n" +
|
||||
" a: 12,\n" +
|
||||
" b: true,\n" +
|
||||
" c: null,\n" +
|
||||
" d: \"true\",\n" +
|
||||
"}\n",
|
||||
},
|
||||
{
|
||||
description: "quote non-identifier keys",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "\"1a\": b\n\"has space\": c\n",
|
||||
expected: "{\n" +
|
||||
" \"1a\": \"b\",\n" +
|
||||
" \"has space\": \"c\",\n" +
|
||||
"}\n",
|
||||
},
|
||||
{
|
||||
description: "escape quoted strings",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "a: \"line1\\nline2\\t\\\"q\\\"\"\n",
|
||||
expected: "{\n" +
|
||||
" a: \"line1\\nline2\\t\\\"q\\\"\",\n" +
|
||||
"}\n",
|
||||
},
|
||||
{
|
||||
description: "preserve comments when encoding",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "# leading\n" +
|
||||
"a: 1 # a line\n" +
|
||||
"# head b\n" +
|
||||
"b: 2\n" +
|
||||
"c:\n" +
|
||||
" # head d\n" +
|
||||
" - d # d line\n" +
|
||||
" - e\n" +
|
||||
"# trailing\n",
|
||||
expected: "# leading\n" +
|
||||
"{\n" +
|
||||
" a: 1, # a line\n" +
|
||||
" # head b\n" +
|
||||
" b: 2,\n" +
|
||||
" c: [\n" +
|
||||
" # head d\n" +
|
||||
" \"d\", # d line\n" +
|
||||
" \"e\",\n" +
|
||||
" ],\n" +
|
||||
" # trailing\n" +
|
||||
"}\n",
|
||||
},
|
||||
{
|
||||
description: "Encode kyaml: anchors and aliases",
|
||||
subdescription: "KYaml output does not support anchors/aliases; they are expanded to concrete values.",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "base: &base\n" +
|
||||
" a: b\n" +
|
||||
"copy: *base\n",
|
||||
expected: "{\n" +
|
||||
" base: {\n" +
|
||||
" a: \"b\",\n" +
|
||||
" },\n" +
|
||||
" copy: {\n" +
|
||||
" a: \"b\",\n" +
|
||||
" },\n" +
|
||||
"}\n",
|
||||
},
|
||||
{
|
||||
description: "Encode kyaml: yaml to kyaml shows formatting differences",
|
||||
subdescription: "KYaml uses flow-style collections (braces/brackets) and explicit commas.",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "person:\n" +
|
||||
" name: John\n" +
|
||||
" pets:\n" +
|
||||
" - cat\n" +
|
||||
" - dog\n",
|
||||
expected: "{\n" +
|
||||
" person: {\n" +
|
||||
" name: \"John\",\n" +
|
||||
" pets: [\n" +
|
||||
" \"cat\",\n" +
|
||||
" \"dog\",\n" +
|
||||
" ],\n" +
|
||||
" },\n" +
|
||||
"}\n",
|
||||
},
|
||||
{
|
||||
description: "Encode kyaml: nested lists of objects",
|
||||
subdescription: "Lists and objects can be nested arbitrarily; KYaml always uses flow-style collections.",
|
||||
scenarioType: "encode",
|
||||
indent: 2,
|
||||
input: "- name: a\n" +
|
||||
" items:\n" +
|
||||
" - id: 1\n" +
|
||||
" tags:\n" +
|
||||
" - k: x\n" +
|
||||
" v: y\n" +
|
||||
" - k: x2\n" +
|
||||
" v: y2\n" +
|
||||
" - id: 2\n" +
|
||||
" tags:\n" +
|
||||
" - k: z\n" +
|
||||
" v: w\n",
|
||||
expected: "[\n" +
|
||||
" {\n" +
|
||||
" name: \"a\",\n" +
|
||||
" items: [\n" +
|
||||
" {\n" +
|
||||
" id: 1,\n" +
|
||||
" tags: [\n" +
|
||||
" {\n" +
|
||||
" k: \"x\",\n" +
|
||||
" v: \"y\",\n" +
|
||||
" },\n" +
|
||||
" {\n" +
|
||||
" k: \"x2\",\n" +
|
||||
" v: \"y2\",\n" +
|
||||
" },\n" +
|
||||
" ],\n" +
|
||||
" },\n" +
|
||||
" {\n" +
|
||||
" id: 2,\n" +
|
||||
" tags: [\n" +
|
||||
" {\n" +
|
||||
" k: \"z\",\n" +
|
||||
" v: \"w\",\n" +
|
||||
" },\n" +
|
||||
" ],\n" +
|
||||
" },\n" +
|
||||
" ],\n" +
|
||||
" },\n" +
|
||||
"]\n",
|
||||
},
|
||||
}
|
||||
|
||||
func testKYamlScenario(t *testing.T, s formatScenario) {
|
||||
prefs := ConfiguredKYamlPreferences.Copy()
|
||||
prefs.Indent = s.indent
|
||||
prefs.UnwrapScalar = false
|
||||
|
||||
switch s.scenarioType {
|
||||
case "encode":
|
||||
test.AssertResultWithContext(
|
||||
t,
|
||||
s.expected,
|
||||
mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewKYamlEncoder(prefs)),
|
||||
s.description,
|
||||
)
|
||||
default:
|
||||
panic(fmt.Sprintf("unhandled scenario type %q", s.scenarioType))
|
||||
}
|
||||
}
|
||||
|
||||
func documentKYamlScenario(_ *testing.T, w *bufio.Writer, i interface{}) {
|
||||
s := i.(formatScenario)
|
||||
if s.skipDoc {
|
||||
return
|
||||
}
|
||||
|
||||
switch s.scenarioType {
|
||||
case "encode":
|
||||
documentKYamlEncodeScenario(w, s)
|
||||
default:
|
||||
panic(fmt.Sprintf("unhandled scenario type %q", s.scenarioType))
|
||||
}
|
||||
}
|
||||
|
||||
func documentKYamlEncodeScenario(w *bufio.Writer, s formatScenario) {
|
||||
writeOrPanic(w, fmt.Sprintf("## %v\n", s.description))
|
||||
|
||||
if s.subdescription != "" {
|
||||
writeOrPanic(w, s.subdescription)
|
||||
writeOrPanic(w, "\n\n")
|
||||
}
|
||||
|
||||
writeOrPanic(w, "Given a sample.yml file of:\n")
|
||||
writeOrPanic(w, fmt.Sprintf("```yaml\n%v\n```\n", s.input))
|
||||
|
||||
writeOrPanic(w, "then\n")
|
||||
|
||||
expression := s.expression
|
||||
if expression == "" {
|
||||
expression = "."
|
||||
}
|
||||
|
||||
if s.indent == 2 {
|
||||
writeOrPanic(w, fmt.Sprintf("```bash\nyq -o=kyaml '%v' sample.yml\n```\n", expression))
|
||||
} else {
|
||||
writeOrPanic(w, fmt.Sprintf("```bash\nyq -o=kyaml -I=%v '%v' sample.yml\n```\n", s.indent, expression))
|
||||
}
|
||||
|
||||
writeOrPanic(w, "will output\n")
|
||||
|
||||
prefs := ConfiguredKYamlPreferences.Copy()
|
||||
prefs.Indent = s.indent
|
||||
prefs.UnwrapScalar = false
|
||||
|
||||
writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n\n", mustProcessFormatScenario(s, NewYamlDecoder(ConfiguredYamlPreferences), NewKYamlEncoder(prefs))))
|
||||
}
|
||||
|
||||
func TestKYamlFormatScenarios(t *testing.T) {
|
||||
for _, s := range kyamlFormatScenarios {
|
||||
testKYamlScenario(t, s)
|
||||
}
|
||||
|
||||
genericScenarios := make([]interface{}, len(kyamlFormatScenarios))
|
||||
for i, s := range kyamlFormatScenarios {
|
||||
genericScenarios[i] = s
|
||||
}
|
||||
documentScenarios(t, "usage", "kyaml", genericScenarios, documentKYamlScenario)
|
||||
}
|
||||
|
||||
func TestKYamlEncoderPrintDocumentSeparator(t *testing.T) {
|
||||
t.Run("enabled", func(t *testing.T) {
|
||||
prefs := NewDefaultKYamlPreferences()
|
||||
prefs.PrintDocSeparators = true
|
||||
|
||||
var buf bytes.Buffer
|
||||
err := NewKYamlEncoder(prefs).PrintDocumentSeparator(&buf)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "---\n" {
|
||||
t.Fatalf("expected doc separator, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("disabled", func(t *testing.T) {
|
||||
prefs := NewDefaultKYamlPreferences()
|
||||
prefs.PrintDocSeparators = false
|
||||
|
||||
var buf bytes.Buffer
|
||||
err := NewKYamlEncoder(prefs).PrintDocumentSeparator(&buf)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "" {
|
||||
t.Fatalf("expected no output, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestKYamlEncoderEncodeUnwrapScalar(t *testing.T) {
|
||||
prefs := NewDefaultKYamlPreferences()
|
||||
prefs.UnwrapScalar = true
|
||||
|
||||
var buf bytes.Buffer
|
||||
err := NewKYamlEncoder(prefs).Encode(&buf, &CandidateNode{
|
||||
Kind: ScalarNode,
|
||||
Tag: "!!str",
|
||||
Value: "cat",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "cat\n" {
|
||||
t.Fatalf("expected unwrapped scalar, got %q", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestKYamlEncoderEncodeColorsEnabled(t *testing.T) {
|
||||
prefs := NewDefaultKYamlPreferences()
|
||||
prefs.UnwrapScalar = false
|
||||
prefs.ColorsEnabled = true
|
||||
|
||||
var buf bytes.Buffer
|
||||
err := NewKYamlEncoder(prefs).Encode(&buf, &CandidateNode{
|
||||
Kind: MappingNode,
|
||||
Content: []*CandidateNode{
|
||||
{Kind: ScalarNode, Tag: "!!str", Value: "a"},
|
||||
{Kind: ScalarNode, Tag: "!!str", Value: "b"},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
out := stripANSI(buf.String())
|
||||
if !strings.Contains(out, "a:") || !strings.Contains(out, "\"b\"") {
|
||||
t.Fatalf("expected colourised output to contain rendered tokens, got %q", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKYamlEncoderWriteNodeAliasAndUnknown(t *testing.T) {
|
||||
ke := NewKYamlEncoder(NewDefaultKYamlPreferences()).(*kyamlEncoder)
|
||||
|
||||
t.Run("alias_nil", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeNode(&buf, &CandidateNode{Kind: AliasNode}, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "null" {
|
||||
t.Fatalf("expected null for nil alias, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("alias_value", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeNode(&buf, &CandidateNode{
|
||||
Kind: AliasNode,
|
||||
Alias: &CandidateNode{Kind: ScalarNode, Tag: "!!int", Value: "12"},
|
||||
}, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "12" {
|
||||
t.Fatalf("expected dereferenced alias value, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("unknown_kind", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeNode(&buf, &CandidateNode{Kind: Kind(12345)}, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "null" {
|
||||
t.Fatalf("expected null for unknown kind, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestKYamlEncoderEmptyCollections(t *testing.T) {
|
||||
ke := NewKYamlEncoder(NewDefaultKYamlPreferences()).(*kyamlEncoder)
|
||||
|
||||
t.Run("empty_mapping", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeNode(&buf, &CandidateNode{Kind: MappingNode}, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "{}" {
|
||||
t.Fatalf("expected empty mapping, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty_sequence", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeNode(&buf, &CandidateNode{Kind: SequenceNode}, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "[]" {
|
||||
t.Fatalf("expected empty sequence, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestKYamlEncoderScalarFallbackAndEscaping(t *testing.T) {
|
||||
ke := NewKYamlEncoder(NewDefaultKYamlPreferences()).(*kyamlEncoder)
|
||||
|
||||
t.Run("unknown_tag_falls_back_to_string", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeNode(&buf, &CandidateNode{Kind: ScalarNode, Tag: "!!timestamp", Value: "2020-01-01T00:00:00Z"}, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != "\"2020-01-01T00:00:00Z\"" {
|
||||
t.Fatalf("expected quoted fallback, got %q", buf.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("escape_double_quoted", func(t *testing.T) {
|
||||
got := escapeDoubleQuotedString("a\\b\"c\n\r\t" + string(rune(0x01)))
|
||||
want := "a\\\\b\\\"c\\n\\r\\t\\u0001"
|
||||
if got != want {
|
||||
t.Fatalf("expected %q, got %q", want, got)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid_bare_key", func(t *testing.T) {
|
||||
if isValidKYamlBareKey("") {
|
||||
t.Fatalf("expected empty string to be invalid")
|
||||
}
|
||||
if isValidKYamlBareKey("1a") {
|
||||
t.Fatalf("expected leading digit to be invalid")
|
||||
}
|
||||
if !isValidKYamlBareKey("a_b-2") {
|
||||
t.Fatalf("expected identifier-like key to be valid")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestKYamlEncoderCommentsInMapping(t *testing.T) {
|
||||
prefs := NewDefaultKYamlPreferences()
|
||||
prefs.UnwrapScalar = false
|
||||
ke := NewKYamlEncoder(prefs).(*kyamlEncoder)
|
||||
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeNode(&buf, &CandidateNode{
|
||||
Kind: MappingNode,
|
||||
Content: []*CandidateNode{
|
||||
{
|
||||
Kind: ScalarNode,
|
||||
Tag: "!!str",
|
||||
Value: "a",
|
||||
HeadComment: "key head",
|
||||
LineComment: "key line",
|
||||
FootComment: "key foot",
|
||||
},
|
||||
{
|
||||
Kind: ScalarNode,
|
||||
Tag: "!!str",
|
||||
Value: "b",
|
||||
HeadComment: "value head",
|
||||
},
|
||||
},
|
||||
}, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
out := buf.String()
|
||||
if !strings.Contains(out, "# key head\n") {
|
||||
t.Fatalf("expected key head comment, got %q", out)
|
||||
}
|
||||
if !strings.Contains(out, "# value head\n") {
|
||||
t.Fatalf("expected value head comment, got %q", out)
|
||||
}
|
||||
if !strings.Contains(out, ", # key line\n") {
|
||||
t.Fatalf("expected inline key comment fallback, got %q", out)
|
||||
}
|
||||
if !strings.Contains(out, "# key foot\n") {
|
||||
t.Fatalf("expected foot comment fallback, got %q", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKYamlEncoderCommentBlockAndInlineComment(t *testing.T) {
|
||||
ke := NewKYamlEncoder(NewDefaultKYamlPreferences()).(*kyamlEncoder)
|
||||
|
||||
t.Run("comment_block_prefixing_and_crlf", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeCommentBlock(&buf, "line1\r\n\r\n# already\r\nline2", 2)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := " # line1\n # already\n # line2\n"
|
||||
if buf.String() != want {
|
||||
t.Fatalf("expected %q, got %q", want, buf.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("inline_comment_prefix_and_first_line_only", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeInlineComment(&buf, "hello\r\nsecond line")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != " # hello" {
|
||||
t.Fatalf("expected %q, got %q", " # hello", buf.String())
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("inline_comment_already_prefixed", func(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
err := ke.writeInlineComment(&buf, "# hello")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if buf.String() != " # hello" {
|
||||
t.Fatalf("expected %q, got %q", " # hello", buf.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
7
pkg/yqlib/no_kyaml.go
Normal file
7
pkg/yqlib/no_kyaml.go
Normal file
@ -0,0 +1,7 @@
|
||||
//go:build yq_nokyaml
|
||||
|
||||
package yqlib
|
||||
|
||||
func NewKYamlEncoder(_ KYamlPreferences) Encoder {
|
||||
return nil
|
||||
}
|
||||
@ -284,4 +284,12 @@ RDBMS
|
||||
expeñded
|
||||
bananabananabananabanana
|
||||
edwinjhlee
|
||||
flox
|
||||
flox
|
||||
unlabelled
|
||||
kyaml
|
||||
KYAML
|
||||
nokyaml
|
||||
buildvcs
|
||||
behaviour
|
||||
GOFLAGS
|
||||
gocache
|
||||
|
||||
@ -1,2 +1,2 @@
|
||||
#!/bin/bash
|
||||
go build -tags "yq_nolua yq_noini yq_notoml yq_noxml yq_nojson yq_nohcl" -ldflags "-s -w" .
|
||||
go build -tags "yq_nolua yq_noini yq_notoml yq_noxml yq_nojson yq_nohcl yq_nokyaml" -ldflags "-s -w" .
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Currently, the `yq_nojson` feature must be enabled when using TinyGo.
|
||||
tinygo build -no-debug -tags "yq_nolua yq_noini yq_notoml yq_noxml yq_nojson yq_nocsv yq_nobase64 yq_nouri yq_noprops yq_nosh yq_noshell yq_nohcl" .
|
||||
tinygo build -no-debug -tags "yq_nolua yq_noini yq_notoml yq_noxml yq_nojson yq_nocsv yq_nobase64 yq_nouri yq_noprops yq_nosh yq_noshell yq_nohcl yq_nokyaml" .
|
||||
|
||||
@ -3,9 +3,11 @@
|
||||
set -o errexit
|
||||
set -o pipefail
|
||||
|
||||
if command -v gosec &> /dev/null
|
||||
then
|
||||
gosec "${PWD}" ./...
|
||||
else
|
||||
./bin/gosec "${PWD}" ./...
|
||||
fi
|
||||
OPTS=(
|
||||
-exclude-dir=vendor
|
||||
-exclude-dir=.gomodcache
|
||||
-exclude-dir=.gocache
|
||||
)
|
||||
|
||||
command -v gosec &> /dev/null && BIN=gosec || BIN=./bin/gosec
|
||||
"${BIN}" "${OPTS[@]}" "${PWD}" ./...
|
||||
|
||||
@ -783,7 +783,7 @@ _FAIL_NOT_SAME_='eval failNotSame --lineno "${LINENO:-}"'
|
||||
# None
|
||||
startSkipping() { __shunit_skip=${SHUNIT_TRUE}; }
|
||||
|
||||
# Resume the normal recording behavior of assert and fail calls.
|
||||
# Resume the normal recording behaviour of assert and fail calls.
|
||||
#
|
||||
# Args:
|
||||
# None
|
||||
@ -1293,7 +1293,7 @@ if command [ "${__shunit_mode}" = "${__SHUNIT_MODE_STANDALONE}" ]; then
|
||||
command . "`_shunit_prepForSourcing \"${__shunit_script}\"`"
|
||||
fi
|
||||
|
||||
# Configure default output coloring behavior.
|
||||
# Configure default output coloring behaviour.
|
||||
_shunit_configureColor "${SHUNIT_COLOR}"
|
||||
|
||||
# Execute the oneTimeSetUp function (if it exists).
|
||||
|
||||
Loading…
Reference in New Issue
Block a user