YAML scalars tagged `!!float` were round-tripped through `float64` and
re-serialized by Go's JSON encoder, which strips the decimal part of
whole-number floats. As a result, `50.0` came out as `50` and a
sequence like `[50.0, 95.0, 99.0, 99.9]` became `[50,95,99,99.9]`,
turning a uniform array of floats into a mixed int/float array that
downstream consumers (Horreum, JSON Schema validators, jq, etc.)
reject.
The JSON spec does not distinguish ints from floats, but every common
JSON library (Go's `encoding/json`, Python's `json`, jq) preserves the
fractional form of values that came in as floats. yq's YAML decoder
already parses these as `!!float` with the original text intact, so we
can emit them verbatim instead of round-tripping.
`MarshalJSON` for `ScalarNode` now special-cases `!!float`:
- if `Value` is already a JSON-shaped number literal containing a `.`
or exponent, emit it verbatim (e.g. `50.0`, `99.9`, `1.5e-3`, `-7.0`);
- if `Value` is an integer-shaped string tagged `!!float` (e.g.
`!!float 5`), format the parsed float and append `.0` so it stays a
JSON number with a fractional part;
- otherwise (empty value, parse error, or non-finite result), fall back
to the existing encoding path so behaviour for `.inf` / `.nan` and
anything unusual is unchanged.
`!!int` nodes still encode as JSON integers.
Closes#2683
Signed-off-by: ChrisJr404 <chris@hacknow.com>
* Initial plan
* Fix TOML encoder to prefer readable table sections over inline tables
When converting from YAML/JSON to TOML, the encoder now always uses
readable TOML table section syntax ([section]) instead of compact inline
hash table syntax (key = { ... }), which better matches TOML's goal as
a human-focused configuration format.
Changes:
- decoder_toml.go: Mark inline TOML tables with FlowStyle so round-trips
can be distinguished from YAML flow mappings
- encoder_toml.go:
- encodeTopLevelEntry: use FlowStyle check instead of EncodeSeparate to
decide inline vs table section (all block mappings now become tables)
- encodeSeparateMapping: count FlowStyle children as attributes; use
recursive encodeSeparateMapping for nested non-flow mappings
- encodeMappingBodyWithPath: emit non-flow child mappings as sub-table
sections instead of inline tables
- toml_test.go: add encode (YAML→TOML) test scenarios, update roundtrip
expectations for inline tables (now expanded to table sections)
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/4824a219-6d5e-42e7-bca1-a8a277bf8c6a
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Fix TOML roundtrip: use TomlInline flag instead of FlowStyle to preserve inline tables
FlowStyle affected YAML decode output (causing inline tables to appear as
YAML flow mappings). Replace it with a new TOML-specific TomlInline bool
on CandidateNode that:
- Is set by the TOML decoder for inline tables (not FlowStyle)
- Is copied by UpdateAttributesFrom so it survives DeeplyAssign merges
- Is checked by the TOML encoder alongside FlowStyle (for YAML flow maps)
- Has no effect on the YAML encoder, preserving existing TOML→YAML output
TOML roundtrip tests are restored to their original expected values (inline
tables stay inline, table sections stay as sections).
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/f59bdf62-6d16-4664-991b-38eb87c9d81c
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Refactor EncodeSeparate+TomlInline into a single EncodeHint enum
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/24db9a8f-601d-4ccf-ada7-129ed3226bb6
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Fix stale comment in hasStructuralChildren
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/24db9a8f-601d-4ccf-ada7-129ed3226bb6
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* Remove unused hasStructuralChildren method from tomlEncoder
Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/2c234b77-28e9-4995-ba6f-9d213ec551a0
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
* feat: K8S KYAML output format support
Reference: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/5295-kyaml/README.md
Co-authored-by: Codex <codex@openai.com>
Generated-with: OpenAI Codex CLI (partial)
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
* build: gomodcache/gocache should not be committed
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
* chore: fix spelling of behaviour
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
* build: pass GOFLAGS to docker to support buildvcs=false
In trying to develop the KYAML support, various tests gave false
positive results because they made assumptions about Git functionality
Make it possible to avoid that by passing GOFLAGS='-buildvcs=false' to
to Makefile.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
* doc: cover documentScenarios for tests
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
* build: exclude go caches from gosec
Without tuning, gosec scans all of the vendor/gocache/gomodcache, taking
several minutes (3m35 here), whereas the core of the yq takes only 15
seconds to scan.
If we intend to remediate upstream issues in future; add a seperate
target to scan those.
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
---------
Signed-off-by: Robin H. Johnson <rjohnson@coreweave.com>
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
Co-authored-by: Codex <codex@openai.com>
- Add ShellVariablesPreferences struct with KeySeparator field (default: '_')
- Update shellVariablesEncoder to use configurable separator
- Add --shell-key-separator CLI flag
- Add comprehensive tests for custom separator functionality
- Update documentation with example usage for custom separator
This feature allows users to specify a custom separator (e.g. '__') when
outputting shell variables, which helps disambiguate nested keys from
keys that contain underscores in their names.
Example:
yq -o=shell --shell-key-separator='__' file.yaml
Fixes ambiguity when original YAML keys contain underscores.
This commit adds the --properties-separator option, which lets users
specify the separator used between keys and values in the properties
output format. This is done by adjusting the value of
github.com/magiconair/properties#Properties.WriteSeparator at encode
time.
Some refactoring of the properties encoder unit tests was done to make
it easier to write unit tests that include different separator values.
Fixes: #1864
Signed-off-by: Ryan Drew <ryan.drew@isovalent.com>
* encoder_lua: Handle explicitly positive infinity
* encoder_lua: Fix inclusion of pre-/suffix when prettyPrinted
It seems certain operations like --prettyPrint or subset selections does
not produce a DocumentNode, which is where the lua pre- and suffix was
printed, causing those to be omitted.
* encoder_lua: Improve Tag handling robustness
Using the method call seems more reliable in case the input parser
forgets to set the tag.
* Implement basic Lua output support
Ref #1700
Basic but working serialization to Lua tables.
* Escape larger set of characters in Lua output
Started with a minimum of replacements, this should be more complete,
tho not all substitutions are strictly required in Lua.
* Print simple keys unquoted in Lua output
String keys that satisfy the requirements for variable names can be used
as keys without quotes in tables.
* Quote Lua keywords in table keys
Keywords are not valid as unquoted keys, thus must be quoted
* Make output of unquoted Lua table keys optional
Generally safer and simpler to not do it.
* Hook up settings for Lua output
* Allow special characters in Lua prefix and suffix
--lua-suffix='});^M' didn't work, so taking this approach instead
* Panic on unhandled YAML Kind in Lua encoder
* Handle YAML case varied booleans in Lua encoder
* Handle special-case numbers in Lua encoder
* Reject unhandled scalar Tags in Lua encoder
* Add note about how Lua nil is unsuitable as table key
Could add some context tracking in the future to allow rejecting nil in
a table key context.
* Return error instead of panic in Lua encoder
* Add initial test for Lua encoder
Boilerplate mostly copied from toml_test.go
* Additional Lua output tests
* Generate Lua encoder documentation
Mostly just for the boilerplate
* Convert octal for Lua output
Lua doesn't have the 0oNNN syntax for octal integers, only decimal and
hexadecimal, hence those can be passed trough as is while octal needs
special treatment.
* Implement indentation in in Lua output
* Respect string Style in Lua encoder
Lua has 'single', "double" and [[ long ]] strings.
* Expand Lua examples
* Output line comments in Lua output
* Implement Lua globals output mode
* fix typo in a comment
* implement shell output format
* fix a typo
* add two test cases, have source uses ascii only
* add integration tests and documentation
* add fixes after code revieew
