From caf36e39b853bd32ee8ad376133084f180121ec2 Mon Sep 17 00:00:00 2001 From: Mike Farah Date: Tue, 15 Feb 2022 14:16:04 +1100 Subject: [PATCH] Latest release --- SUMMARY.md | 1 + operators/add.md | 32 +++++ operators/comment-operators.md | 18 +-- operators/datetime.md | 203 ++++++++++++++++++++++++++++++ operators/document-index.md | 6 +- operators/file-operators.md | 4 +- operators/split-into-documents.md | 4 +- operators/subtract.md | 32 +++++ usage/properties.md | 69 +++++++++- 9 files changed, 350 insertions(+), 19 deletions(-) create mode 100644 operators/datetime.md diff --git a/SUMMARY.md b/SUMMARY.md index 1add587f..6b5e758b 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -22,6 +22,7 @@ * [Comment Operators](operators/comment-operators.md) * [Contains](operators/contains.md) * [Create, Collect into Object](operators/create-collect-into-object.md) + * [Date Time](operators/datetime.md) * [Delete](operators/delete.md) * [Document Index](operators/document-index.md) * [Encode / Decode](operators/encode-decode.md) diff --git a/operators/add.md b/operators/add.md index 8a61c0eb..52452ef4 100644 --- a/operators/add.md +++ b/operators/add.md @@ -206,6 +206,38 @@ a: 4 b: 6 ``` +## Date addition +You can add durations to dates. Assumes RFC3339 date time format, see [date-time operators](https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information. + +Given a sample.yml file of: +```yaml +a: 2021-01-01T00:00:00Z +``` +then +```bash +yq '.a += "3h10m"' sample.yml +``` +will output +```yaml +a: 2021-01-01T03:10:00Z +``` + +## Date addition - custom format +You can add durations to dates. See [date-time operators](https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information. + +Given a sample.yml file of: +```yaml +a: Saturday, 15-Dec-01 at 2:59AM GMT +``` +then +```bash +yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST", .a += "3h1m")' sample.yml +``` +will output +```yaml +a: Saturday, 15-Dec-01 at 6:00AM GMT +``` + ## Add to null Adding to null simply returns the rhs diff --git a/operators/comment-operators.md b/operators/comment-operators.md index 9c5f4aef..9e3b61ec 100644 --- a/operators/comment-operators.md +++ b/operators/comment-operators.md @@ -23,7 +23,7 @@ a: cat ``` then ```bash -yq '.a lineComment="single"' sample.yml +yq '.a line_comment="single"' sample.yml ``` will output ```yaml @@ -38,7 +38,7 @@ b: dog ``` then ```bash -yq '.. lineComment |= .' sample.yml +yq '.. line_comment |= .' sample.yml ``` will output ```yaml @@ -53,7 +53,7 @@ a: cat ``` then ```bash -yq '. headComment="single"' sample.yml +yq '. head_comment="single"' sample.yml ``` will output ```yaml @@ -69,7 +69,7 @@ a: cat ``` then ```bash -yq '. footComment=.a' sample.yml +yq '. foot_comment=.a' sample.yml ``` will output ```yaml @@ -86,7 +86,7 @@ b: dog # leave this ``` then ```bash -yq '.a lineComment=""' sample.yml +yq '.a line_comment=""' sample.yml ``` will output ```yaml @@ -120,7 +120,7 @@ a: cat # meow ``` then ```bash -yq '.a | lineComment' sample.yml +yq '.a | line_comment' sample.yml ``` will output ```yaml @@ -138,7 +138,7 @@ a: cat # meow ``` then ```bash -yq '. | headComment' sample.yml +yq '. | head_comment' sample.yml ``` will output ```yaml @@ -157,7 +157,7 @@ a: cat # meow ``` then ```bash -yq 'headComment' sample.yml +yq 'head_comment' sample.yml ``` will output ```yaml @@ -177,7 +177,7 @@ a: cat # meow ``` then ```bash -yq '. | footComment' sample.yml +yq '. | foot_comment' sample.yml ``` will output ```yaml diff --git a/operators/datetime.md b/operators/datetime.md new file mode 100644 index 00000000..f97a37ea --- /dev/null +++ b/operators/datetime.md @@ -0,0 +1,203 @@ +# Date Time + +Various operators for parsing and manipulating dates. + +## Date time formattings +This uses the golangs built in time library for parsing and formatting date times. + +When not specified, the RFC3339 standard is assumed `2006-01-02T15:04:05Z07:00` for parsing. + +To specify a custom parsing format, use the `with_dtf` operator. The first parameter sets the datetime parsing format for the expression in the second parameter. The expression can be any valid `yq` expression tree. + +```bash +yq 'with_dtf("myformat"; .a + "3h" | tz("Australia/Melbourne"))' +``` + +See https://pkg.go.dev/time#pkg-constants for examples of formatting options. + + +## Timezones +This uses golangs built in LoadLocation function to parse timezones strings. See https://pkg.go.dev/time#LoadLocation for more details. + + +## Durations +Durations are parsed using golangs built in [ParseDuration](https://pkg.go.dev/time#ParseDuration) function. + +You can durations to time using the `+` operator. + +{% hint style="warning" %} +Note that versions prior to 4.18 require the 'eval/e' command to be specified. + +`yq e ` +{% endhint %} + +## Format: from standard RFC3339 format +Providing a single parameter assumes a standard RFC3339 datetime format. If the target format is not a valid yaml datetime format, the result will be a string tagged node. + +Given a sample.yml file of: +```yaml +a: 2001-12-15T02:59:43.1Z +``` +then +```bash +yq '.a |= format_datetime("Monday, 02-Jan-06 at 3:04PM")' sample.yml +``` +will output +```yaml +a: Saturday, 15-Dec-01 at 2:59AM +``` + +## Format: from custom date time +Use with_dtf to set a custom datetime format for parsing. + +Given a sample.yml file of: +```yaml +a: Saturday, 15-Dec-01 at 2:59AM +``` +then +```bash +yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM"; format_datetime("2006-01-02"))' sample.yml +``` +will output +```yaml +a: 2001-12-15 +``` + +## Format: get the day of the week +Given a sample.yml file of: +```yaml +a: 2001-12-15T02:59:43.1Z +``` +then +```bash +yq '.a | format_datetime("Monday")' sample.yml +``` +will output +```yaml +Saturday +``` + +## Now +Given a sample.yml file of: +```yaml +a: cool +``` +then +```bash +yq '.updated = now' sample.yml +``` +will output +```yaml +a: cool +updated: 2021-05-19T01:02:03Z +``` + +## Timezone: from standard RFC3339 format +Returns a new datetime in the specified timezone. Specify standard IANA Time Zone format or 'utc', 'local'. When given a single parameter, this assumes the datetime is in RFC3339 format. + +Given a sample.yml file of: +```yaml +a: cool +``` +then +```bash +yq '.updated = (now | tz("Australia/Sydney"))' sample.yml +``` +will output +```yaml +a: cool +updated: 2021-05-19T11:02:03+10:00 +``` + +## Timezone: with custom format +Specify standard IANA Time Zone format or 'utc', 'local' + +Given a sample.yml file of: +```yaml +a: Saturday, 15-Dec-01 at 2:59AM GMT +``` +then +```bash +yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; tz("Australia/Sydney"))' sample.yml +``` +will output +```yaml +a: Saturday, 15-Dec-01 at 1:59PM AEDT +``` + +## Add and tz custom format +Specify standard IANA Time Zone format or 'utc', 'local' + +Given a sample.yml file of: +```yaml +a: Saturday, 15-Dec-01 at 2:59AM GMT +``` +then +```bash +yq '.a |= with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; tz("Australia/Sydney"))' sample.yml +``` +will output +```yaml +a: Saturday, 15-Dec-01 at 1:59PM AEDT +``` + +## Date addition +Given a sample.yml file of: +```yaml +a: 2021-01-01T00:00:00Z +``` +then +```bash +yq '.a += "3h10m"' sample.yml +``` +will output +```yaml +a: 2021-01-01T03:10:00Z +``` + +## Date subtraction +You can subtract durations from dates. Assumes RFC3339 date time format, see [date-time operators](https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information. + +Given a sample.yml file of: +```yaml +a: 2021-01-01T03:10:00Z +``` +then +```bash +yq '.a -= "3h10m"' sample.yml +``` +will output +```yaml +a: 2021-01-01T00:00:00Z +``` + +## Date addition - custom format +Given a sample.yml file of: +```yaml +a: Saturday, 15-Dec-01 at 2:59AM GMT +``` +then +```bash +yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; .a += "3h1m")' sample.yml +``` +will output +```yaml +a: Saturday, 15-Dec-01 at 6:00AM GMT +``` + +## Date script with custom format +You can embed full expressions in with_dtf if needed. + +Given a sample.yml file of: +```yaml +a: Saturday, 15-Dec-01 at 2:59AM GMT +``` +then +```bash +yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST"; .a = (.a + "3h1m" | tz("Australia/Perth")))' sample.yml +``` +will output +```yaml +a: Saturday, 15-Dec-01 at 2:00PM AWST +``` + diff --git a/operators/document-index.md b/operators/document-index.md index 140d1d4d..ffb0936d 100644 --- a/operators/document-index.md +++ b/operators/document-index.md @@ -17,7 +17,7 @@ a: frog ``` then ```bash -yq '.a | documentIndex' sample.yml +yq '.a | document_index' sample.yml ``` will output ```yaml @@ -53,7 +53,7 @@ a: frog ``` then ```bash -yq 'select(documentIndex == 1)' sample.yml +yq 'select(document_index == 1)' sample.yml ``` will output ```yaml @@ -85,7 +85,7 @@ a: frog ``` then ```bash -yq '.a | ({"match": ., "doc": documentIndex})' sample.yml +yq '.a | ({"match": ., "doc": document_index})' sample.yml ``` will output ```yaml diff --git a/operators/file-operators.md b/operators/file-operators.md index d7988509..fdfb9e3d 100644 --- a/operators/file-operators.md +++ b/operators/file-operators.md @@ -37,7 +37,7 @@ a: cat ``` then ```bash -yq 'fileIndex' sample.yml +yq 'file_index' sample.yml ``` will output ```yaml @@ -55,7 +55,7 @@ a: cat ``` then ```bash -yq eval-all 'fileIndex' sample.yml another.yml +yq eval-all 'file_index' sample.yml another.yml ``` will output ```yaml diff --git a/operators/split-into-documents.md b/operators/split-into-documents.md index a9f70d30..a4f22f3d 100644 --- a/operators/split-into-documents.md +++ b/operators/split-into-documents.md @@ -11,7 +11,7 @@ Note that versions prior to 4.18 require the 'eval/e' command to be specified.&# ## Split empty Running ```bash -yq --null-input 'splitDoc' +yq --null-input 'split_doc' ``` will output ```yaml @@ -26,7 +26,7 @@ Given a sample.yml file of: ``` then ```bash -yq '.[] | splitDoc' sample.yml +yq '.[] | split_doc' sample.yml ``` will output ```yaml diff --git a/operators/subtract.md b/operators/subtract.md index da5bfe48..a8f8d338 100644 --- a/operators/subtract.md +++ b/operators/subtract.md @@ -117,6 +117,38 @@ a: 2 b: 4 ``` +## Date subtraction +You can subtract durations from dates. Assumes RFC3339 date time format, see [date-time operators](https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information. + +Given a sample.yml file of: +```yaml +a: 2021-01-01T03:10:00Z +``` +then +```bash +yq '.a -= "3h10m"' sample.yml +``` +will output +```yaml +a: 2021-01-01T00:00:00Z +``` + +## Date subtraction - custom format +Use with_dtf to specify your datetime format. See [date-time operators](https://mikefarah.gitbook.io/yq/operators/date-time-operators) for more information. + +Given a sample.yml file of: +```yaml +a: Saturday, 15-Dec-01 at 6:00AM GMT +``` +then +```bash +yq 'with_dtf("Monday, 02-Jan-06 at 3:04PM MST", .a -= "3h1m")' sample.yml +``` +will output +```yaml +a: Saturday, 15-Dec-01 at 2:59AM GMT +``` + ## Custom types: that are really numbers When custom tags are encountered, yq will try to decode the underlying type. diff --git a/usage/properties.md b/usage/properties.md index 8f7e7a08..d5477434 100644 --- a/usage/properties.md +++ b/usage/properties.md @@ -27,7 +27,7 @@ emptyMap: [] ``` then ```bash -yq -o=props -I=0 '.' sample.yml +yq -o=props sample.yml ``` will output ```properties @@ -54,7 +54,7 @@ emptyMap: [] ``` then ```bash -yq -o=props -I=0 '... comments = ""' sample.yml +yq -o=props '... comments = ""' sample.yml ``` will output ```properties @@ -80,7 +80,7 @@ emptyMap: [] ``` then ```bash -yq -o=props -I=0 '(.. | select( (tag == "!!map" or tag =="!!seq") and length == 0)) = ""' sample.yml +yq -o=props '(.. | select( (tag == "!!map" or tag =="!!seq") and length == 0)) = ""' sample.yml ``` will output ```properties @@ -94,3 +94,66 @@ emptyArray = emptyMap = ``` +## Decode properties +Given a sample.properties file of: +```properties +# comments on values appear +person.name = Mike + +# comments on array values appear +person.pets.0 = cat +person.food.0 = pizza + +``` +then +```bash +yq -p=props sample.properties +``` +will output +```yaml +person: + name: Mike # comments on values appear + pets: + - cat # comments on array values appear + food: + - pizza +``` + +## Roundtrip +Given a sample.properties file of: +```properties +# comments on values appear +person.name = Mike + +# comments on array values appear +person.pets.0 = cat +person.food.0 = pizza + +``` +then +```bash +yq -p=props -o=props '.person.pets.0 = "dog"' sample.properties +``` +will output +```properties +# comments on values appear +person.name = Mike + +# comments on array values appear +person.pets.0 = dog +person.food.0 = pizza +``` + +## Empty doc +Given a sample.properties file of: +```properties + +``` +then +```bash +yq -p=props sample.properties +``` +will output +```yaml +``` +