Commit Graph

796 Commits

Author SHA1 Message Date
Mike Farah
447bf28cd8 Introduced 'format' to encapsulate encoding and decoding formats together 2024-02-24 16:37:13 +11:00
Mike Farah
8f6d642012 Refactoring encoding configuration 2024-02-24 16:07:15 +11:00
Mike Farah
1d371b712f Refactoring encoding configuration 2024-02-24 15:59:12 +11:00
Mike Farah
55f6a3a49d Refactoring JSON encoder prefs 2024-02-24 15:48:59 +11:00
Mike Farah
7a01e216c4 Refactoring Yaml encoder prefs 2024-02-24 15:36:16 +11:00
Mike Farah
f44d47a204 Refactoring Props encoder prefs 2024-02-24 15:14:21 +11:00
Mike Farah
2866e90c49 Refactoring XML encoder prefs 2024-02-24 15:03:30 +11:00
Mike Farah
571caa696a Refactoring Output types 2024-02-24 14:58:11 +11:00
Mike Farah
abef92d7d9 Added prune recipe 2024-02-21 14:50:55 +11:00
Mike Farah
9f3822fbaa Improving property docs 2024-02-20 13:45:31 +11:00
Mike Farah
b24b484efc Added --properties-array-brackets flag for properties encoder #1933 2024-02-20 11:39:56 +11:00
Ryan Drew
2865022cf8
Add --properties-separator option (#1951)
This commit adds the --properties-separator option, which lets users
specify the separator used between keys and values in the properties
output format. This is done by adjusting the value of
github.com/magiconair/properties#Properties.WriteSeparator at encode
time.

Some refactoring of the properties encoder unit tests was done to make
it easier to write unit tests that include different separator values.

Fixes: #1864

Signed-off-by: Ryan Drew <ryan.drew@isovalent.com>
2024-02-20 10:57:44 +11:00
Mike Farah
9a8151d316 Added csv separator flag #1950 2024-02-17 19:11:41 +11:00
Mike Farah
42439b7d00 Adding shebang documentation 2024-02-16 11:03:35 +11:00
Mike Farah
796b4a0955 Adding shebang documentation 2024-02-16 10:56:18 +11:00
Mike Farah
3c3f1180d9 Cleaning up log message formats 2024-02-16 09:44:22 +11:00
Mike Farah
d9e1a789ea Fixed Toml decoding when table array defined before parent #1922 2024-02-15 16:15:01 +11:00
Mike Farah
c32a9ceab8 Added new CSV option to turn off auto-parsing #1947 2024-02-15 13:11:53 +11:00
Mike Farah
e81b600744 Fixing with_entries context #1925
Can now update key/value w.r.t each other
2024-02-11 10:25:38 +11:00
Mike Farah
8cde0c837c Can now retrieve the alias names of merge anchors #1942 2024-02-09 17:24:59 +11:00
Mike Farah
b55f52f377 Improve merge docs 2024-02-09 14:54:22 +11:00
Mike Farah
1d06d8b482 Another comment example 2024-02-09 13:58:59 +11:00
Mike Farah
2dda0a203d Comments in yq expressions! #1919 2024-02-09 13:54:27 +11:00
Mike Farah
24bd563680 Updating style docs 2024-02-09 10:47:09 +11:00
Mike Farah
5513ac8a7d Empty TOML table is an empty object 2024-02-08 13:31:56 +11:00
Elijah Lopez
d4e16a413e
fix: empty TOML table (#1936) 2024-02-07 19:59:50 +11:00
Mike Farah
6e21c9f77f Need to escape single quotes in bash, updated docs #1932 2024-02-02 14:58:49 +11:00
Mike Farah
010750daf6 Updated to latest golint 2024-01-11 13:17:34 +11:00
Mike Farah
dea8f95670 Fixed "all" error message #1845 2023-12-13 11:02:22 +11:00
Mike Farah
eb4278784a Added another new recipe 2023-12-13 10:55:53 +11:00
Mike Farah
cbd03f8a93 Added new recipe 2023-12-13 10:41:56 +11:00
Mike Farah
4cf123fed5 Fixed to_entries[] 2023-12-12 15:44:34 +11:00
Mike Farah
7f26d91cd0 Fixed handling of --- #1890, #1896 2023-12-12 11:30:12 +11:00
Mike Farah
d474bb8ba4 Fixing sef fault on bad XML #1888 2023-12-05 13:21:12 +11:00
Mike Farah
a988cd9005 Updating JSON doc: NDJSON spec deprecated, prefer JSON Lines 2023-12-01 11:39:02 +11:00
Mike Farah
779b26310e Updating references to yq being a YAML processor 2023-11-30 15:32:21 +11:00
Mike Farah
730f240d24 Fixed to_entries and del bug #1886 2023-11-30 14:04:54 +11:00
Ville Skyttä
998b4edb34
Grammar fixes (#1877) 2023-11-23 11:54:25 +11:00
Mike Farah
26effddb8c Fixed empty array json bug #1880 2023-11-23 11:53:18 +11:00
Mike Farah
3d64bdadc1 Fixes issue when update against self #1869 2023-11-18 12:19:27 +11:00
Mike Farah
c7ef946031 Fixed multi doc anchor bug #1861 2023-11-14 13:00:30 +11:00
Mike Farah
d6e099f7ff Formatting 2023-11-08 09:09:21 +11:00
Valentin Lab
a60d27975a
add tests for root node unwrap bug in previous versions (#1837) 2023-10-23 20:03:51 +11:00
Mike Farah
c8f4ba7f45 Removing copier lib 2023-10-18 12:36:52 +11:00
Mike Farah
13d1bbb45f
Generic ast (#1829)
Remove dependency on yaml.Node for internal AST representation. Yaml decoder is now just another decoder.
2023-10-18 12:11:53 +11:00
Kim Alvefur
7c78a15b23
Lua output fixes (#1811)
* encoder_lua: Handle explicitly positive infinity

* encoder_lua: Fix inclusion of pre-/suffix when prettyPrinted

It seems certain operations like --prettyPrint or subset selections does
not produce a DocumentNode, which is where the lua pre- and suffix was
printed, causing those to be omitted.

* encoder_lua: Improve Tag handling robustness

Using the method call seems more reliable in case the input parser
forgets to set the tag.
2023-10-12 11:25:26 +11:00
Mike Farah
d113344abf Added tonumber support #71 2023-10-05 15:15:01 +11:00
Mike Farah
6e65d44a98 Can build yq with no lua 2023-10-03 16:22:23 +11:00
Kim Alvefur
5fa41624c9
Add support for Lua input (#1810) 2023-10-03 16:00:51 +11:00
Mike Farah
ee900ec997 Added kind operator 2023-10-02 14:43:12 +11:00
Mike Farah
0cf3adf5dd Adding another xml test; spelling fixes 2023-09-26 14:43:08 +10:00
-pthread -Wall
05ad1dad90
Fix typos (#1798) 2023-09-26 14:18:18 +10:00
Mike Farah
cdb9b56066 Spellig with cspell 2023-09-19 10:43:11 +10:00
Mike Farah
e09779e004 Fixed number parsing as float bug in JSON #1756 2023-09-19 09:42:25 +10:00
Mike Farah
c66480448d Added recipes documentation 2023-09-01 14:40:33 +10:00
Mike Farah
cda69bff5e Adding recipes 2023-09-01 12:17:54 +10:00
Mike Farah
e5564c18fe Another xml example 2023-09-01 11:52:58 +10:00
Mike Farah
55c0d485ba Fixed string, null concact consistency #1712 2023-08-30 09:46:45 +10:00
Mike Farah
71f95ab87b Fixed .a!= parsing issue #1711 2023-08-30 09:40:57 +10:00
Mike Farah
d18a5e3adc Fixed new linting issue CWE-118 2023-08-28 17:39:46 +10:00
Mike Farah
44c7f3aec3 Preparing release notes 2023-08-11 13:52:50 +10:00
Kim Alvefur
d302d75c77
Implement basic Lua output support (#1745)
* Implement basic Lua output support

Ref #1700

Basic but working serialization to Lua tables.

* Escape larger set of characters in Lua output

Started with a minimum of replacements, this should be more complete,
tho not all substitutions are strictly required in Lua.

* Print simple keys unquoted in Lua output

String keys that satisfy the requirements for variable names can be used
as keys without quotes in tables.

* Quote Lua keywords in table keys

Keywords are not valid as unquoted keys, thus must be quoted

* Make output of unquoted Lua table keys optional

Generally safer and simpler to not do it.

* Hook up settings for Lua output

* Allow special characters in Lua prefix and suffix

--lua-suffix='});^M' didn't work, so taking this approach instead

* Panic on unhandled YAML Kind in Lua encoder

* Handle YAML case varied booleans in Lua encoder

* Handle special-case numbers in Lua encoder

* Reject unhandled scalar Tags in Lua encoder

* Add note about how Lua nil is unsuitable as table key

Could add some context tracking in the future to allow rejecting nil in
a table key context.

* Return error instead of panic in Lua encoder

* Add initial test for Lua encoder

Boilerplate mostly copied from toml_test.go

* Additional Lua output tests

* Generate Lua encoder documentation

Mostly just for the boilerplate

* Convert octal for Lua output

Lua doesn't have the 0oNNN syntax for octal integers, only decimal and
hexadecimal, hence those can be passed trough as is while octal needs
special treatment.

* Implement indentation in in Lua output

* Respect string Style in Lua encoder

Lua has 'single', "double" and [[ long ]] strings.

* Expand Lua examples

* Output line comments in Lua output

* Implement Lua globals output mode
2023-08-11 12:56:49 +10:00
Mike Farah
fcc04823ca Added docs on base64 only working with utf8 content 2023-05-23 15:38:08 +10:00
Mike Farah
5950329e2c Added docs on yes/no being strings in yaml spec 1.2 2023-05-23 15:35:39 +10:00
Mike Farah
4d50b001d1 Added test for null array json example 2023-05-04 14:48:57 +10:00
Arthur Fritz Santiago
d14eaa9b49
fix: runtime error: invalid memory address or nil pointer dereference (#1649) 2023-05-04 14:43:48 +10:00
Giorgio Gallo
80b42b81fd
Implement shell output format (#1645)
* fix typo in a comment

* implement shell output format

* fix a typo

* add two test cases, have source uses ascii only

* add integration tests and documentation

* add fixes after code revieew
2023-05-04 11:06:56 +10:00
Mike Farah
385921a3e2 Fixed passing of readonly context in pipe 2023-04-11 12:04:04 +10:00
Mike Farah
eba075294a Added scalar output for TOML (#1617) 2023-04-03 15:40:06 +10:00
Mike Farah
a389bb64b8 Fixed bug when splatting empty array #1613 2023-03-31 16:24:23 +11:00
Mike Farah
496035c75a toml docs 2023-03-31 10:24:12 +11:00
Valentin Lab
5fd2890d1b
Add `--nul-output|-0` flag to separate element with NUL character (#1550)
This is to ensure solid parsing of complex data (with any binary
content except NUL chars) by separating the `yq` root collection
member's output with NUL char. As a safe-guard, an error will be cast
if trying to use NUL character with content that contains itself NUL
characters inside.
2023-03-29 09:51:55 +11:00
Mike Farah
8d516ce535 Handle scalars in csv, xml files 2023-03-27 13:54:24 +11:00
Mike Farah
1b0a62d08d Handle scalars in property files 2023-03-27 13:51:08 +11:00
Mike Farah
7103b78d38
Draft: Toml (#1439)
* toml wip

* wip

* Fixed auto parsing toml

* Added build flag not to include toml

* Parse toml docs and tests

* minor updates
2023-03-26 10:59:15 +11:00
Mike Farah
95aafdd8d5 Fixed modulo docs 2023-03-18 14:53:36 +11:00
Mike Farah
fdce8fce13 Added docs 2023-03-18 14:51:31 +11:00
Mike Farah
dccba10997 added another base64 test 2023-03-18 14:33:48 +11:00
TJ Miller
69d3e3f3f8
Add support for decoding base64 strings without padding (#1604) 2023-03-18 14:30:35 +11:00
Mike Farah
ac8ffa287e doc gen 2023-03-16 13:41:10 +11:00
Abel Sen
fa6fac1a76
Minor typos (#1595)
* Remove extra backtick

* Reword explanation of update

* Reword explanation of relative update

* Change "remaple" to "remain"

* Change "clovver" to "clobber"

* Reword explanation of update for comment operators

* Reword explanation of relative update for comment operators

* Change "array" to "expression"

* Change "the golangs" to "Golang's"

* Change "golangs" to "Golang's"

* Change "can durations" to "can add durations"

* Change "array scalars" to "arrays"

* Change "beit" to "be it"

* Fix typo in `eval` tip

* Fix typo in header for `has` operation

* Add space before pipe in `line` operator example

* Fix typos in explanation of deep array merges

* Change "is now used" to "is now used."

* Change "object," to "object."

* Changes "indexes" to "indices"

* Remove extraneous copied text from `..` article

* Reword explanation of `...` operator

* Change "your are" to "you are"

* Add link to `string` operator docs in `select` article

* Change "is a" to "parameter specifies" in `string` operators article

* Change "new line" to "newline"

* Change "golang regex" to "Golang's regex"

* Change "golang" to "Golang"

* Add period

* Remove comma in `subtract` article

* Remove duplicate number subtraction example

* Remove comma in `traverse` operator article

* Clarify use of brackets when `read`ing with special characters
2023-03-16 13:39:36 +11:00
TJ Miller
a466821e8f
Added divide and modulo operators (#1593)
* Added divide operator (#49)

* Tidy up divide operator logic

* Added modulo operator

* Fix divide test typo

* Add divide by zero test

* Handle int modulo by 0 and add tests

* Tidy up divide/modulo operator node creation

* Fix linter errors
2023-03-15 20:14:23 +11:00
Mike Farah
360a47fddc Fixed npe in log #1596 2023-03-15 13:57:47 +11:00
Mike Farah
08a6cb65fe
Auto output format (#1599)
* Use file extension to auto detect output format!

* Use file extension to auto detect output format!

* formatting
2023-03-15 13:22:58 +11:00
Robert Brennan
9539877ff6
Add filter operation (#1588)
* add filter operation

* add tests

* add tests

* revert debug

* simplify filter

* fix tests

* remove logs
2023-03-09 10:30:47 +11:00
ryenus
d30941b575
Detect input format based on file name extension (#1582)
* detect inputFormat from filename

* refactor and extract func InputFormatFromFilename

* detect inputFormat only when file is provided

* add test for automatic input format detection
2023-03-09 10:17:19 +11:00
Mike Farah
2195df9e7a Fixed xml encoding of ProcInst #1563, improved XML comment handling 2023-03-02 10:57:54 +11:00
Mike Farah
3f1f66a8ee Fixed merged anchor reference problem #1482 2023-03-01 13:45:35 +11:00
Anders Björklund
cf8cfbd865
Allow build without json and xml support (#1556)
* Refactor ordered_map into separate files

Separate json and xml, from the regular yaml.

Makes it possible to compile, without those...

* Refactor encoder and decoder creation

Use more consistent parameters vs globals

Return errors instead of calling panic()

* Allow build without json and xml support
2023-03-01 13:19:06 +11:00
Mike Farah
62d167c141
Variable loop - Fixes #1566 (#1577)
* Variable loop wip

* Variable loop wip

* Variable loop wip

* Variable loop wip

* Fixed variable operator to work like jq
2023-02-28 16:40:38 +11:00
Mike Farah
d17fd9424e Added shuffle command #1503 2023-02-11 05:08:20 +11:00
Mike Farah
a1698b740a Added ability to sort by multiple fields #1541 2023-02-11 04:46:39 +11:00
Vít Zikmund
93b7c999be
Use a lazy-quoting @sh encoder (#1548)
* Use a lazy-quoting @sh encoder

* Add internal quoting style switch to @sh

* Add test for stray empty quotes in @sh
2023-02-09 18:15:07 +11:00
Mike Farah
88a6b20ba5 Fixed date comparison with string date #1537 2023-02-02 13:30:48 +11:00
Mike Farah
915ab69922 Added to_unix operator 2023-02-02 12:56:16 +11:00
Mike Farah
f9f340b6bf Github pipeline not in AU tz :D 2023-02-02 12:47:59 +11:00
Mike Farah
75920481b1 Added from_unix operator #1535 2023-02-02 12:42:36 +11:00
Mike Farah
d21bb920d6 Added shell string encoder (@sh) #1526 2023-02-02 12:23:08 +11:00
Mike Farah
3b1bcac5b3 review feed back on URI 2023-01-25 11:18:48 +11:00
Mike Farah
6d7d76a3f1 Adds @uri/@urid #1529 2023-01-23 11:37:18 +11:00