suyiiyii/yq
1
0
Fork 0
mirror of https://github.com/mikefarah/yq.git synced 2024-11-14 15:18:06 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,727 Commits 67 Branches 180 Tags 12 MiB
8f6d642012
Commit Graph

746 Commits

Author SHA1 Message Date
Mike Farah
c640888133 Fixed bug - can now delete documents #1377 2022-10-11 11:48:18 +11:00
Mike Farah
33ec66cfdd Fixed handling of UTF8 encoded CSVs #1373 2022-10-08 13:12:50 +11:00
Mike Farah
1b8d399de4 Detect and fail on missing closing brackets #1366 2022-10-08 12:44:54 +11:00
Mike Farah
b5b81abb90 Added DELPATHS operator 2022-10-05 20:09:53 +11:00
Mike Farah
da3f3b93b4 Added SETPATH operator 2022-10-05 14:12:08 +11:00
Mike Farah
7f4c8e1c78 Added comment operator examples 2022-09-30 11:30:31 +10:00
Mike Farah
60f30f8a48 Added comment operator examples 2022-09-30 11:26:36 +10:00
Mike Farah
2362451fda Added comment operator examples 2022-09-30 11:22:58 +10:00
Mike Farah
55a7fdfd8a Added is_key operator 2022-09-30 10:27:35 +10:00
Mike Farah
3dd5f0c80c Added another comment test 2022-09-30 10:15:41 +10:00
Mike Farah
be304a1102 Added another comment test 2022-09-30 09:46:07 +10:00
Mike Farah
b20477210f Added missing validation for adding sequences to maps #1341 2022-09-16 10:04:48 +10:00
Mike Farah
51b64e6cf3 Fixing relative merge bug #1333 2022-09-09 12:28:38 +10:00
Mike Farah
a64eea3b1a Fixed bug in alternative op, dont evaluate RHS if LHS is truthy 2022-09-09 11:05:12 +10:00
Mike Farah
bf6b4d3941 Fixed footer comment issue #1321 2022-08-30 11:24:26 +10:00
Mike Farah
b4ca184108 Added new "c" flag to clobber custom tags when needed 2022-08-29 15:38:21 +10:00
Mike Farah
b1a40a9fb7 Bumped go compiler; bumped quality tools 2022-08-29 14:13:15 +10:00
Mike Farah
a696dceea4 Added string trim operator 2022-08-08 13:35:57 +10:00
Mike Farah
2c9b5be408 Fixed json decode to maintain key order 2022-08-01 16:26:43 +10:00
Mike Farah
a91a8ccc66 Improving docs 2022-08-01 14:12:35 +10:00
Mike Farah
c8815f5ab9
Csv decoder (#1290) 
* WIP: adding CSV decoder

* Adding CSV decoder

* Added CSV roundtrip

* Fixing from review
 2022-08-01 10:28:34 +10:00
Jih-Wei, Liang
3c222d8707
Added StringEvaluator for evaluating string input #1266 (#1278) 2022-08-01 08:50:56 +10:00
Mike Farah
8c3a8c085e Fixes xml decode missing tags #1284 2022-07-29 10:26:50 +10:00
Mike Farah
b9a1ef89fe
Added NDJSON decoder (#1281) 2022-07-27 12:26:22 +10:00
Mike Farah
03ba38eaff
Using new lexer by alecthomas! (#1270) 2022-07-13 14:45:28 +10:00
Mike Farah
f393cf37c2 Dont clobber anchor when adding nodes - fixes #1269 2022-07-13 11:12:15 +10:00
Mike Farah
86c2b03630 Added error operator for custom validation #1259 2022-06-25 12:46:24 +10:00
Daniel Carbone
98b411f82e
adding support for --wrapScalar=false in properties encoder (#1241) 
* adding support for --wrapScalar=false in properties encoder

* altering encoder and decoder tests somewhat

* adding .idea

* Revert "altering encoder and decoder tests somewhat"

This reverts commit e3655130e2.

* adding test scenario for encoding with wrapped scalars
 2022-06-25 12:22:03 +10:00
Mike Farah
06d2aaad80 Adding acceptance test for #1257, fixed panic 2022-06-25 12:15:28 +10:00
Hayama
9b47a29864
hotfix: fix panic multiply assign (#1257) 2022-06-25 12:08:52 +10:00
Mike Farah
be05df03b5 Fixing usage of quoted numeric keys #1247 2022-06-23 19:22:11 +10:00
Mike Farah
dfd396b480 When split expression includes an extension, dont add .yml automatically 2022-06-23 14:31:09 +10:00
Mike Farah
60c037f57e Can add string to scalars in any order #1234 2022-06-15 17:09:17 -07:00
rndmit
b9309a42a4
XML decoder additions (#1239) 
* Add xml-keep-namespace and xml-raw-token features

* Add tests

* Change flags usage strings

* Append docs
 2022-06-15 09:40:31 +10:00
Mike Farah
8d3be1a23c Length (and some other operators) should not include head/foot comments #1231 2022-05-31 16:28:53 +10:00
Mike Farah
439bcaad25 Added multiple object example from PR #1226 2022-05-27 11:22:10 +10:00
Mike Farah
294da55012 Fixed base64/xml decode an empty string #1209 2022-05-27 11:18:38 +10:00
Mike Farah
8d32e6a82c Fixed base64 decode an empty string #1209 2022-05-27 10:56:43 +10:00
Mike Farah
1e27e39927 Fixing typos (prefer UK english) 2022-05-25 11:02:18 +10:00
Mike Farah
5126f3dfee Fixed doc 2022-05-25 10:54:56 +10:00
Mike Farah
6bce23d9c2 Improved handling of trailing document comments 2022-05-25 10:54:03 +10:00
Christoph Stäbler
dee592eb3b
Fix typo (#1222) 2022-05-24 18:18:27 +10:00
Mike Farah
451d8643ee Updating merge docs 2022-05-24 16:16:58 +10:00
Mike Farah
fbe49c3700 Added colors to json output #1208 2022-05-22 21:19:59 +10:00
Mike Farah
72cdde78f6 Fixing issues found by codeql 2022-05-06 13:46:14 +10:00
Mike Farah
689c535a29 Updated unique docs 2022-04-29 12:16:57 +10:00
Mike Farah
67c79b7cb1 Fixes append map bug when key matches value in existing map #1200 2022-04-28 14:45:45 +10:00
Mike Farah
26529fae94 Nicer error message when trying to use merge anchor tags other than maps #1184 2022-04-27 14:46:52 +10:00
Mike Farah
90261a2fdd Added type as an alias for tag #1195 2022-04-27 09:11:43 +10:00
Mike Farah
b11075ec98 now assumes yaml idiomatic formatting when adding to empty maps/arrays 2022-04-15 08:27:22 +10:00
Mike Farah
ea66a73f43 Fixed sort dropping head comment #1181 2022-04-14 12:19:15 +10:00
Mike Farah
6f9f80cc48 Fixes with operation bug #1174 2022-04-14 12:03:18 +10:00
Mike Farah
523b8335d8 Fixing string docs 2022-04-01 13:21:55 +11:00
Mike Farah
0956eefb1a Fixed help links 2022-03-29 11:08:50 +11:00
Mike Farah
a9ed08a5f8 Fixed compare docs 2022-03-29 11:01:02 +11:00
Benjamin Wuethrich
ae71824adc
Fix Markdown link formatting (#1157) 
* Fix Markdown link formatting

* Fix parens
 2022-03-29 10:23:32 +11:00
Mike Farah
186b319093 Added from_props 2022-03-28 19:48:30 +11:00
Mike Farah
3cd999bb87 Bumping to go1.18, bumping dev tools 2022-03-28 14:18:55 +11:00
Mike Farah
bbeae229ca Disable strict XML parsing by default #1155 2022-03-28 14:05:10 +11:00
Mike Farah
3a1e2c7518 Fixed silently ignored error in xml decoder 2022-03-27 11:42:07 +11:00
Mike Farah
fb148c305a Added comparison ops (>,>=...) 2022-03-24 11:38:05 +11:00
Mike Farah
897604142f Better string docs 2022-03-23 10:28:45 +11:00
Mike Farah
03f57b7034 Bumping version 2022-03-20 13:03:40 +11:00
Mike Farah
005b097cee
Boolean fix (#1148) 
* Fixing booleans

* Fixed "and", "or" evaluating RHS when not required
 2022-03-20 12:55:58 +11:00
Mike Farah
78e9cc7998
Envsubst params (#1147) 
* Can give envsubst optional arguments
 2022-03-20 10:56:50 +11:00
Mike Farah
0ffee92175 Fixed += with multiple matches #1145 2022-03-19 18:42:12 +11:00
Mike Farah
4bb2fa1652 Updated equals doc 2022-03-16 10:16:49 +11:00
Mike Farah
d27fb0ec78 Update multiply docs 2022-03-16 10:04:45 +11:00
Mikhail Katychev
406bbb8ca4
handled EvaluateNodes panic (#1140) 2022-03-15 13:28:52 +11:00
Mike Farah
3924ed8a35 Bumping version 2022-03-10 13:50:53 +11:00
Mike Farah
58be9829f9 Added pick operator 2022-03-10 13:39:51 +11:00
Mike Farah
edbdb16ab7 Fixes auto expansion of properties #1127 2022-03-01 11:29:11 +11:00
Mike Farah
8698433d44 Added line and column operators 2022-02-27 11:56:46 +11:00
Mike Farah
ff047d0748 Fixes #1123 2022-02-27 11:39:31 +11:00
Mike Farah
0e0183b9ea Update load docs 2022-02-23 09:40:05 +11:00
Mike Farah
61978b34df Added load_base64 2022-02-23 09:38:17 +11:00
Mike Farah
d9bca65626 Added base64 support 2022-02-23 09:26:35 +11:00
Mike Farah
d7b158f855 Added string case operators 2022-02-22 16:17:23 +11:00
Mike Farah
71706af3d4 String op can now run on custom types 2022-02-22 14:50:45 +11:00
Mike Farah
8142e94349 Added reverse op 2022-02-22 14:15:31 +11:00
Mike Farah
665f6b0267 Fixed self assigned issue #1107 2022-02-20 14:29:52 +11:00
Mike Farah
fc447b46ce Fixed bad cature groups with multiple matches #1114 2022-02-20 13:28:13 +11:00
Mike Farah
75fffc17e1 Added support for load_props, added missing load_xml tests 2022-02-15 14:51:22 +11:00
Mike Farah
b35893d783
Date Time Ops (#1110) 
* Added datetime operators

* Added date subtract support
 2022-02-14 15:37:43 +11:00
Mike Farah
a5f6a80cf6 Added snake_case version of camelCase operators (snake case now prefered) 2022-02-11 09:05:17 +11:00
Mike Farah
a9c3617b4f
Decoder Properties (#1099) 
* Decoder Properties

* Added properties round trip test

* Fixed property decode for github actions

* Refactored XML test to use common functions

* Switched formatScenario parameter order for more readablity
 2022-02-10 12:02:53 +11:00
Mike Farah
bfaafa66f9 Fixed SEGV error #1096 2022-02-09 11:47:21 +11:00
Mike Farah
26356ff4be Added github action fix for parsing xml, updated linter 2022-02-07 11:55:55 +11:00
Mike Farah
06e944dcb6 Added github action fix for parsing xml 2022-02-07 11:26:48 +11:00
Mike Farah
703418d0c4 Automatically add "-" when piping and its missing 2022-02-07 09:09:11 +11:00
Mike Farah
1f367ac200 Added notification banner to docs 2022-02-06 14:39:46 +11:00
Mike Farah
d00153de71 Adding to array copies styling of previous elements #722 2022-02-04 09:24:48 +11:00
Mike Farah
992fe066aa Improved property conversion docs 2022-02-01 15:34:17 +11:00
Mike Farah
b80080a26d Parameterised merge array by key example 2022-02-01 15:11:39 +11:00
Mike Farah
0afb59c65e Tests can have multiple env variables 2022-02-01 14:58:53 +11:00
Mike Farah
535799462f Added eval operator 2022-02-01 14:47:51 +11:00
Mike Farah
a6fc7aa881 Updating docs 2022-01-28 10:06:25 +11:00
Mike Farah
3ba728d40c Working around github strangeness 2022-01-27 13:54:29 +11:00
Mike Farah
394be5c65a Debug github build 2022-01-27 12:47:06 +11:00
Mike Farah
c3d815998a Auto style when adding to empty maps/arrays 2022-01-27 09:58:13 +11:00
Mike Farah
a9e7f669d0 Merged envsubst into env operators 2022-01-27 09:20:53 +11:00
Samuel Cormier-Iijima
8195ff8b9b
Add a new envsubst operator to replace environment variables in strings (#1082) 2022-01-25 19:33:30 +11:00
Mike Farah
9b1a7bf451 Added support for adding objects 2022-01-23 11:35:44 +11:00
Mike Farah
6e33132de9 Added support for *= 2022-01-22 16:40:17 +11:00
Mike Farah
a6c79f3410 Multiply, substract with custom types 2022-01-22 16:12:59 +11:00
Mike Farah
50df792e49 Auto cast for add 2022-01-22 13:17:16 +11:00
Mike Farah
6f24e878aa Updated README, better xml docs 2022-01-22 12:35:33 +11:00
Mike Farah
2526b03d67 Added JSON conversion tests and doc generation 2022-01-21 20:26:19 +11:00
Eng Zer Jun
e347090571
refactor: move from io/ioutil to io and os packages (#1068) 
* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* build: update snapcraft `go-channel` to 1.17

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-01-21 19:50:53 +11:00
Mike Farah
de0716e875 Tweaking select docs 2022-01-15 18:22:26 +11:00
Mike Farah
21cdbab0d7 Improving select docs (#839) 2022-01-15 18:18:52 +11:00
Mike Farah
8c94a96ee0 New merge flag (n) to only merge in new fields (#1038) 2022-01-15 15:48:34 +11:00
Mike Farah
ec8ef312ef
Added XML encoding/decoding (#1067) 
* Added XML encoding/decoding

* Minor fixes

* Improve yq doc

* Lint
 2022-01-15 11:57:59 +11:00
Mike Farah
606ef91cc6 Better fix #1062 (return error instead of panic) 2022-01-14 15:40:24 +11:00
Mike Farah
78b45a3eb0 Fixed #1062 2022-01-14 15:22:55 +11:00
Mike Farah
84ddf1862f Fixed #1048 2022-01-14 14:55:25 +11:00
Mike Farah
ca1c5dab59 Can encode in XML! 2021-12-21 16:59:09 +11:00
Mike Farah
2ee9c65bc2 Added global xml prefs for simplicity 2021-12-21 16:52:54 +11:00
Mike Farah
6bcbd873a6 wip 2021-12-21 16:19:27 +11:00
Mike Farah
ae8df5ea87 wip 2021-12-21 16:08:37 +11:00
Mike Farah
851a43b9b6 wip 2021-12-21 15:56:08 +11:00
Mike Farah
df32baedf1
Added XML decoder (#1044) 2021-12-21 15:02:07 +11:00
Matthieu MOREL
59752fb36d
enable more linters (#1043) 
* enable revive linter

* enable gochecknoinits linter

* enable unconvert linter

* enable unparam linter

* enable asciicheck linter

* enable depguard linter

* enable nakedret linter

* enable megacheck linter

* enable nolintlint linter

* enable predeclared linter

* Update go.yml

* Update go.yml
 2021-12-21 09:30:08 +11:00
Mike Farah
ffd1f022aa Report while filename failed to parse #1030 2021-12-06 17:25:32 +11:00
Mike Farah
c17cf82791 Improving docs 2021-12-05 13:03:38 +11:00
Mike Farah
b4db4496d5 Improved tips and tricks 2021-12-05 11:12:09 +11:00
Mike Farah
8b04d972f3 Improved error message 2021-12-05 10:58:26 +11:00
Mike Farah
e22df34158 Added sort_by operator 2021-12-04 13:54:12 +11:00
Mike Farah
2f05f7390f wip 2021-12-04 13:25:32 +11:00
Mike Farah
dba41ffed7 Assignment op no longer clobbers anchor (#1029) 2021-12-03 09:23:24 +11:00
Mike Farah
855b3c4b47 Cleanup test 2021-12-02 20:06:51 +11:00
Mike Farah
df5128fa36 Added encoder tests 2021-12-02 20:06:51 +11:00
Mike Farah
f62cfe5ec9 Added encoder tests 2021-12-02 20:06:51 +11:00
Mike Farah
d0419ceedf Added csv, tsv output formats 2021-12-02 20:06:51 +11:00
Mike Farah
14f8f92b76 Added map, map_values 2021-12-01 10:32:36 +11:00
Mike Farah
54b355bffb Added another test 2021-12-01 09:14:18 +11:00
Mike Farah
1c50ae4e6a Added another test 2021-11-30 13:22:18 +11:00
Mike Farah
feb97c9872 Fixed collect op when working with multiple nodes 2021-11-30 13:19:44 +11:00
Rob Ferguson
d41b44dc42
Add accessor for the yq logger instance (#1013) 
Allow consumers of yqlib to customize the logger instance.

Closes #432

Signed-off-by: Carolyn Van Slyck <me@carolynvanslyck.com>

Co-authored-by: Carolyn Van Slyck <me@carolynvanslyck.com>
Co-authored-by: Rob Ferguson <robert.j.ferguson@nasa.gov>
 2021-11-30 10:51:49 +11:00
Mike Farah
176fec639d Better merge example 2021-11-29 21:28:25 +11:00
Mike Farah
f5f148cadb Better formatting of merge arrays example 2021-11-29 21:06:09 +11:00
Mike Farah
418bd1a806 Added credit for merge by array example 2021-11-29 20:56:04 +11:00
Mike Farah
4c7c0b83aa Better merge array by key example 2021-11-29 20:53:36 +11:00
Mike Farah
51e946e6ff Better merge array by key example 2021-11-29 20:46:12 +11:00
Mike Farah
8c5d1e0252 gci linter 2021-11-26 20:24:21 +11:00
Matthieu MOREL
15d7c9e118 enable misspell linter 2021-11-25 22:01:30 +11:00
Mike Farah
7e09a66b0b Fixing comments 2021-11-24 10:59:19 +11:00
Mike Farah
733e63d1fb Added more tests 2021-11-24 10:16:48 +11:00
Mike Farah
b44fecdfa5 Added key operator 2021-11-24 09:57:35 +11:00
Mike Farah
7f629d5e36 Added errorlint to devtools 2021-11-24 09:27:17 +11:00
Matthieu MOREL
8711042c98 enable errorlint linter 2021-11-24 09:14:02 +11:00
Mike Farah
3d620100f8 Improved load doc 2021-11-18 17:34:08 +11:00
Mike Farah
a804e7370f Minor improvement on handling front matter 2021-11-16 16:34:39 +11:00
Mike Farah
356eff3b0b wip 2021-11-16 16:21:08 +11:00
Mike Farah
33871bf007 Fixed header preprocessing! 2021-11-14 10:59:50 +11:00
Mike Farah
11b6261e8b remove leading content indicator 2021-11-14 10:59:50 +11:00
Mike Farah
5df71162c9 fixed printer test 2021-11-14 10:59:50 +11:00
Mike Farah
63f54563ea wip 2021-11-14 10:59:50 +11:00
Mike Farah
d912d7d178 wip 2021-11-14 10:59:50 +11:00
Mike Farah
0557439765 Update document generation script 2021-11-03 22:37:07 +11:00
Mike Farah
de90b3e85e gitbook wip 2021-11-03 22:37:07 +11:00
Mike Farah
08fc058934 Fix JSON encoding removing null #985 2021-10-30 13:37:21 +11:00
Mike Farah
0b2688c0f1 Split printer 2021-10-30 10:04:41 +11:00
Mike Farah
65fd001575 Improving docs 2021-10-28 09:15:28 +11:00
Mike Farah
d508ac3fa4 Fixed flatten error message 2021-10-26 15:43:43 +11:00
Mike Farah
80084e89cc Added flatten operator 2021-10-26 15:42:25 +11:00
Mike Farah
2491051fd9 Added group_by operator 2021-10-26 15:13:26 +11:00
Mike Farah
d390fdc641 better docs 2021-10-26 14:33:57 +11:00
Mike Farah
91717b3c5d Can specify indent in encode ops 2021-10-24 11:35:40 +11:00
Mike Farah
587af7f722 Fixed newline handling in encoder/decoder 2021-10-22 15:21:01 +11:00
Mike Farah
a1af1b95d0 better docs 2021-10-22 14:55:58 +11:00
Mike Farah
b1e64a0d80 Fixed newline handling when decoding/encoding 2021-10-22 14:53:39 +11:00
Mike Farah
7288d34778 Added decoder op 2021-10-22 12:37:47 +11:00
Mike Farah
cdc5ef7b15 Added encoder op 2021-10-22 12:00:47 +11:00
Mike Farah
5819dd0853 Fixed expression parsing bug #970 2021-10-18 11:47:03 +11:00
Mike Farah
0064b68ed9 Revert "Removed leadingContentPreProcessing flag - header preprocessing is stable" 
This reverts commit 4ce9433468.
 2021-10-15 11:47:00 +11:00
Mike Farah
e741e6d52f Man page wip 2021-10-15 10:39:33 +11:00
Mike Farah
948399f163 Man page wip 2021-10-15 09:59:08 +11:00
Mike Farah
0436b77d91 Man page wip 2021-10-15 09:51:42 +11:00
Mike Farah
4ce9433468 Removed leadingContentPreProcessing flag - header preprocessing is stable 2021-10-15 08:53:00 +11:00
Peter Matseykanets
dd259b4957 Make deepMatch report in linear time 
The current implementation of the deepMatch() has the exponential runtime.
Given the long enough input and the pattern with multiple wildcards
it takes a while if ever to complete which can potentially be used
maliciously to cause a denial of service (cpu and memory consumption).

E.g. running this in the root of this repository
time yq eval '.jobs.publishDocker.steps.[] | select (.run == "****outputs")' .github/workflows/release.yml
gives on my laptop
25.11s user 0.06s system 99% cpu 25.182 total

Whereas the updated implementation gives
0.01s user 0.01s system 36% cpu 0.049 total

There are numerous similar CVEs reported for glob evaluation in
different shells/ftp-servers/libraries.

The replacement implementation with the linear runtime is shamelessly taken
verbatim from the briliant article by Russ Cox https://research.swtch.com/glob
 2021-10-14 18:45:25 +11:00
Mike Farah
ed4d888bfa Speed up multiply 2021-10-11 14:46:46 +11:00
Mike Farah
79bc1f95cb Slight performance improvement to context.ChildContext 2021-10-11 12:56:27 +11:00
Mike Farah
4a4aae00af Sped up explode operator 2021-10-11 10:41:15 +11:00
Mike Farah
839f795710 Fixed select bug (#958) 2021-10-07 15:04:36 +11:00
Mike Farah
43e40a6dda Updating with documentation 2021-09-16 10:55:26 +10:00
Mike Farah
2292f0ffb4 Fixed with semicolon space issue 2021-09-15 22:24:03 +10:00
Mike Farah
2db8140d7f Added contains operator 2021-09-15 15:18:10 +10:00
Mike Farah
084a0f6f1e Merge branch 'subtract' 2021-09-12 21:59:07 +10:00
Mike Farah
6002604251 Added with operator 2021-09-12 21:52:02 +10:00
Mike Farah
b2ee131a4c Updated var to work like jq #934 2021-09-12 16:55:55 +10:00
Mike Farah
1cfbbde796 Added missing closing bracket error check 2021-09-12 15:49:50 +10:00
Mike Farah
13c42db238 Adding subtraction support for arrays 2021-09-07 16:58:34 +10:00
Mike Farah
3543a2dbdc Fixed union infinite loop #930 2021-09-05 11:07:40 +10:00
Mike Farah
8941573c1a Fixing alternative op bug #930 2021-09-02 15:36:23 +10:00
Mike Farah
eea2c97cd8 Can process hex numbers 2021-09-02 15:26:44 +10:00
Mike Farah
cb95ab1494 Fixed merge comment issue #919 2021-08-26 16:31:26 +10:00
Mike Farah
d18aa3e9e0 Added shorthand output formats 2021-08-20 15:46:33 +10:00
Mike Farah
4ba96d902b Fixed document header/footer comment handling when merging 2021-08-17 09:54:12 +10:00
Mike Farah
9de0d0aae1 Adding string readme 2021-08-01 18:47:04 +10:00