yq/pkg/yqlib/doc/operators/unique.md

Unique

This is used to filter out duplicated items in an array. Note that the original order of the array is maintained.

Unique array of scalars (string/numbers)

Note that unique maintains the original order of the array.

Given a sample.yml file of:

- 2
- 1
- 3
- 2

then

yq 'unique' sample.yml

will output

- 2
- 1
- 3

Unique nulls

Unique works on the node value, so it considers different representations of nulls to be different

Given a sample.yml file of:

- ~
- null
- ~
- null

then

yq 'unique' sample.yml

will output

- ~
- null

Unique all nulls

Run against the node tag to unique all the nulls

Given a sample.yml file of:

- ~
- null
- ~
- null

then

yq 'unique_by(tag)' sample.yml

will output

- ~

Unique array objects

Given a sample.yml file of:

- name: harry
  pet: cat
- name: billy
  pet: dog
- name: harry
  pet: cat

then

yq 'unique' sample.yml

will output

- name: harry
  pet: cat
- name: billy
  pet: dog

Unique array of objects by a field

Given a sample.yml file of:

- name: harry
  pet: cat
- name: billy
  pet: dog
- name: harry
  pet: dog

then

yq 'unique_by(.name)' sample.yml

will output

- name: harry
  pet: cat
- name: billy
  pet: dog

Unique array of arrays

Given a sample.yml file of:

- - cat
  - dog
- - cat
  - sheep
- - cat
  - dog

then

yq 'unique' sample.yml

will output

- - cat
  - dog
- - cat
  - sheep