yq/operators/string-operators.md

String Operators

RegEx

This uses golangs native regex functions under the hood - See their docs for the supported syntax.

Case insensitive tip: prefix the regex with (?i) - e.g. test("(?i)cats)".

match(regEx)

This operator returns the substring match details of the given regEx.

capture(regEx)

Capture returns named RegEx capture groups in a map. Can be more convenient than match depending on what you are doing.

test(regEx)

Returns true if the string matches the RegEx, false otherwise.

sub(regEx, replacement)

Substitutes matched substrings. The first parameter is the regEx to match substrings within the original string. The second is a what to replace those matches with. This can refer to capture groups from the first RegEx.

String blocks, bash and newlines

Bash is notorious for chomping on precious trailing newline characters, making it tricky to set strings with newlines properly. In particular, the $( exp ) will trim trailing newlines.

For instance to get this yaml:

a: |
  cat

Using $( exp ) wont work, as it will trim the trailing new line.

m=$(echo "cat\n") yq -n '.a = strenv(m)'
a: cat

However, using printf works:

printf -v m "cat\n" ; m="$m" yq -n '.a = strenv(m)'
a: |
  cat

As well as having multiline expressions:

m="cat
"  yq -n '.a = strenv(m)'
a: |
  cat

Similarly, if you're trying to set the content from a file, and want a trailing new line:

IFS= read -rd '' output < <(cat my_file)
output=$output ./yq '.data.values = strenv(output)' first.yml

{% hint style="warning" %} Note that versions prior to 4.18 require the 'eval/e' command to be specified.

yq e <exp> <file> {% endhint %}

To up (upper) case

Works with unicode characters

Given a sample.yml file of:

água

then

yq 'upcase' sample.yml

will output

ÁGUA

To down (lower) case

Works with unicode characters

Given a sample.yml file of:

ÁgUA

then

yq 'downcase' sample.yml

will output

água

Join strings

Given a sample.yml file of:

- cat
- meow
- 1
- null
- true

then

yq 'join("; ")' sample.yml

will output

cat; meow; 1; ; true

Match string

Given a sample.yml file of:

foo bar foo

then

yq 'match("foo")' sample.yml

will output

string: foo
offset: 0
length: 3
captures: []

Match string, case insensitive

Given a sample.yml file of:

foo bar FOO

then

yq '[match("(?i)foo"; "g")]' sample.yml

will output

- string: foo
  offset: 0
  length: 3
  captures: []
- string: FOO
  offset: 8
  length: 3
  captures: []

Match with global capture group

Given a sample.yml file of:

abc abc

then

yq '[match("(ab)(c)"; "g")]' sample.yml

will output

- string: abc
  offset: 0
  length: 3
  captures:
    - string: ab
      offset: 0
      length: 2
    - string: c
      offset: 2
      length: 1
- string: abc
  offset: 4
  length: 3
  captures:
    - string: ab
      offset: 4
      length: 2
    - string: c
      offset: 6
      length: 1

Match with named capture groups

Given a sample.yml file of:

foo bar foo foo  foo

then

yq '[match("foo (?P<bar123>bar)? foo"; "g")]' sample.yml

will output

- string: foo bar foo
  offset: 0
  length: 11
  captures:
    - string: bar
      offset: 4
      length: 3
      name: bar123
- string: foo  foo
  offset: 12
  length: 8
  captures:
    - string: null
      offset: -1
      length: 0
      name: bar123

Capture named groups into a map

Given a sample.yml file of:

xyzzy-14

then

yq 'capture("(?P<a>[a-z]+)-(?P<n>[0-9]+)")' sample.yml

will output

a: xyzzy
n: "14"

Match without global flag

Given a sample.yml file of:

cat cat

then

yq 'match("cat")' sample.yml

will output

string: cat
offset: 0
length: 3
captures: []

Match with global flag

Given a sample.yml file of:

cat cat

then

yq '[match("cat"; "g")]' sample.yml

will output

- string: cat
  offset: 0
  length: 3
  captures: []
- string: cat
  offset: 4
  length: 3
  captures: []

Test using regex

Like jq's equivalent, this works like match but only returns true/false instead of full match details

Given a sample.yml file of:

- cat
- dog

then

yq '.[] | test("at")' sample.yml

will output

true
false

Substitute / Replace string

This uses golang regex, described here Note the use of |= to run in context of the current string value.

Given a sample.yml file of:

a: dogs are great

then

yq '.a |= sub("dogs", "cats")' sample.yml

will output

a: cats are great

Substitute / Replace string with regex

This uses golang regex, described here Note the use of |= to run in context of the current string value.

Given a sample.yml file of:

a: cat
b: heat

then

yq '.[] |= sub("(a)", "${1}r")' sample.yml

will output

a: cart
b: heart

Custom types: that are really strings

When custom tags are encountered, yq will try to decode the underlying type.

Given a sample.yml file of:

a: !horse cat
b: !goat heat

then

yq '.[] |= sub("(a)", "${1}r")' sample.yml

will output

a: !horse cart
b: !goat heart

Split strings

Given a sample.yml file of:

cat; meow; 1; ; true

then

yq 'split("; ")' sample.yml

will output

- cat
- meow
- "1"
- ""
- "true"

Split strings one match

Given a sample.yml file of:

word

then

yq 'split("; ")' sample.yml

will output

- word