yq/usage/csv-tsv.md

241 lines
4.3 KiB
Markdown
Raw Normal View History

2022-08-01 06:46:09 +00:00
# CSV
Encode/Decode/Roundtrip CSV and TSV files.
2021-12-04 03:08:16 +00:00
2022-08-01 06:46:09 +00:00
## Encode
2023-09-26 04:50:18 +00:00
Currently supports arrays of homogeneous flat objects, that is: no nesting and it assumes the _first_ object has all the keys required:
2022-02-06 03:31:21 +00:00
2022-08-01 06:46:09 +00:00
```yaml
- name: Bobo
type: dog
- name: Fifi
type: cat
```
As well as arrays of arrays of scalars (strings/numbers/booleans):
```yaml
- [Bobo, dog]
- [Fifi, cat]
```
2022-02-06 03:29:26 +00:00
2022-08-01 06:46:09 +00:00
## Decode
Decode assumes the first CSV/TSV row is the header row, and all rows beneath are the entries.
The data will be coded into an array of objects, using the header rows as keys.
2021-12-04 03:08:16 +00:00
2022-08-01 06:46:09 +00:00
```csv
name,type
Bobo,dog
Fifi,cat
```
2022-02-06 03:31:21 +00:00
2021-12-04 03:08:16 +00:00
2022-08-01 06:46:09 +00:00
## Encode CSV simple
Given a sample.yml file of:
2021-12-04 03:08:16 +00:00
```yaml
- [i, like, csv]
- [because, excel, is, cool]
```
then
2022-08-01 06:46:09 +00:00
```bash
yq -o=csv sample.yml
```
will output
```csv
i,like,csv
because,excel,is,cool
```
2021-12-04 03:08:16 +00:00
2022-08-01 06:46:09 +00:00
## Encode TSV simple
Given a sample.yml file of:
```yaml
- [i, like, csv]
- [because, excel, is, cool]
```
then
2021-12-04 03:08:16 +00:00
```bash
2022-08-01 06:46:09 +00:00
yq -o=tsv sample.yml
```
will output
```tsv
i like csv
because excel is cool
2021-12-04 03:08:16 +00:00
```
2022-08-01 06:46:09 +00:00
## Encode array of objects to csv
Given a sample.yml file of:
```yaml
- name: Gary
numberOfCats: 1
likesApples: true
height: 168.8
- name: Samantha's Rabbit
numberOfCats: 2
likesApples: false
height: -188.8
2021-12-04 03:08:16 +00:00
2022-08-01 06:46:09 +00:00
```
then
```bash
yq -o=csv sample.yml
```
will output
2021-12-04 03:08:16 +00:00
```csv
2022-08-01 06:46:09 +00:00
name,numberOfCats,likesApples,height
Gary,1,true,168.8
Samantha's Rabbit,2,false,-188.8
2021-12-04 03:08:16 +00:00
```
2022-08-01 06:46:09 +00:00
## Encode array of objects to custom csv format
Add the header row manually, then the we convert each object into an array of values - resulting in an array of arrays. Pick the columns and call the header whatever you like.
2021-12-04 03:08:16 +00:00
2022-08-01 06:46:09 +00:00
Given a sample.yml file of:
```yaml
- name: Gary
numberOfCats: 1
likesApples: true
height: 168.8
- name: Samantha's Rabbit
numberOfCats: 2
likesApples: false
height: -188.8
```
then
2021-12-04 03:08:16 +00:00
```bash
2022-08-01 06:46:09 +00:00
yq -o=csv '[["Name", "Number of Cats"]] + [.[] | [.name, .numberOfCats ]]' sample.yml
2021-12-04 03:08:16 +00:00
```
2022-08-01 06:46:09 +00:00
will output
```csv
Name,Number of Cats
Gary,1
Samantha's Rabbit,2
```
## Encode array of objects to csv - missing fields behaviour
First entry is used to determine the headers, and it is missing 'likesApples', so it is not included in the csv. Second entry does not have 'numberOfCats' so that is blank
2021-12-04 03:08:16 +00:00
2022-08-01 06:46:09 +00:00
Given a sample.yml file of:
```yaml
- name: Gary
numberOfCats: 1
height: 168.8
- name: Samantha's Rabbit
height: -188.8
likesApples: false
2021-12-04 03:08:16 +00:00
2022-02-06 03:31:21 +00:00
```
2022-08-01 06:46:09 +00:00
then
```bash
yq -o=csv sample.yml
```
will output
```csv
name,numberOfCats,height
Gary,1,168.8
Samantha's Rabbit,,-188.8
2021-12-04 03:08:16 +00:00
```
2022-07-29 04:33:26 +00:00
2022-08-01 06:46:09 +00:00
## Parse CSV into an array of objects
2024-02-15 22:55:01 +00:00
First row is assumed to be the header row. By default, entries with YAML/JSON formatting will be parsed!
2022-07-29 04:33:26 +00:00
2022-08-01 06:46:09 +00:00
Given a sample.csv file of:
```csv
2024-02-15 22:55:01 +00:00
name,numberOfCats,likesApples,height,facts
Gary,1,true,168.8,cool: true
Samantha's Rabbit,2,false,-188.8,tall: indeed
2022-07-29 04:33:26 +00:00
2022-08-01 06:46:09 +00:00
```
then
```bash
yq -p=csv sample.csv
```
will output
2022-07-29 04:33:26 +00:00
```yaml
2022-08-01 06:46:09 +00:00
- name: Gary
numberOfCats: 1
likesApples: true
height: 168.8
2024-02-15 22:55:01 +00:00
facts:
cool: true
- name: Samantha's Rabbit
numberOfCats: 2
likesApples: false
height: -188.8
facts:
tall: indeed
```
## Parse CSV into an array of objects, no auto-parsing
First row is assumed to be the header row. Entries with YAML/JSON will be left as strings.
Given a sample.csv file of:
```csv
name,numberOfCats,likesApples,height,facts
Gary,1,true,168.8,cool: true
Samantha's Rabbit,2,false,-188.8,tall: indeed
```
then
```bash
yq -p=csv --csv-auto-parse=f sample.csv
```
will output
```yaml
- name: Gary
numberOfCats: 1
likesApples: true
height: 168.8
facts: 'cool: true'
2022-08-01 06:46:09 +00:00
- name: Samantha's Rabbit
numberOfCats: 2
likesApples: false
height: -188.8
2024-02-15 22:55:01 +00:00
facts: 'tall: indeed'
2022-07-29 04:33:26 +00:00
```
2022-08-01 06:46:09 +00:00
## Parse TSV into an array of objects
First row is assumed to be the header row.
Given a sample.tsv file of:
```tsv
name numberOfCats likesApples height
Gary 1 true 168.8
Samantha's Rabbit 2 false -188.8
2022-07-29 04:33:26 +00:00
2022-08-01 06:46:09 +00:00
```
then
2022-07-29 04:33:26 +00:00
```bash
2022-08-01 06:46:09 +00:00
yq -p=tsv sample.tsv
```
will output
```yaml
- name: Gary
numberOfCats: 1
likesApples: true
height: 168.8
- name: Samantha's Rabbit
numberOfCats: 2
likesApples: false
height: -188.8
2022-07-29 04:33:26 +00:00
```
2022-08-01 06:46:09 +00:00
## Round trip
Given a sample.csv file of:
```csv
name,numberOfCats,likesApples,height
Gary,1,true,168.8
Samantha's Rabbit,2,false,-188.8
2022-07-29 04:33:26 +00:00
2022-08-01 06:46:09 +00:00
```
then
```bash
yq -p=csv -o=csv '(.[] | select(.name == "Gary") | .numberOfCats) = 3' sample.csv
```
will output
2022-07-29 04:33:26 +00:00
```csv
2022-08-01 06:46:09 +00:00
name,numberOfCats,likesApples,height
Gary,3,true,168.8
Samantha's Rabbit,2,false,-188.8
2022-07-29 04:33:26 +00:00
```
2022-08-01 06:46:09 +00:00