Compare commits

..
140 Commits
Author SHA1 Message Date
Mike Farah 718f66dfec v4.54.3 2026-08-19 15:23:47 +10:00
Mike Farah d51f1fad31 v4.53.2 2026-04-17 16:31:31 +10:00
Mike Farah 0dcba85400 v4.50.1 - HCL 2025-12-15 11:38:27 +11:00
Mike Farah 06517a463e v4.49.1 2025-11-22 15:01:51 +11:00
Mike Farah 451724d514 Version bump 2025-11-22 09:44:39 +11:00
furkanu-workandMike Farah ccc34b5022 Add version information to fix-merge-anchor option 2025-11-07 20:03:21 +11:00
Mike Farah 2f56a8df95 Improving how it works 2025-10-12 18:30:02 +11:00
Mike Farah d38fd1f256 v4.48.1 2025-10-12 14:31:03 +11:00
Mike Farah eaf26894c1 Updating readme 2025-07-23 16:15:42 +10:00
Mike Farah f7f6320e88 v4.47.1 2025-07-23 16:12:12 +10:00
Mike Farah 7b57e31a76 Fixing typo 2025-05-03 11:38:15 +10:00
Mike Farah 500ee98a48 Removing old notice 2025-03-03 15:23:47 +11:00
Mike Farah 785ee2cabb Updating docs 2025-03-03 15:12:22 +11:00
Mike Farah 19118c5552 v4.44.5 2024-12-07 17:35:26 +11:00
Mike Farah 57e4639631 updating docs 2024-09-06 12:33:38 +10:00
Mike Farah 1f1efb08ad Fixing pivot url 2024-07-07 10:42:08 +10:00
Mike Farah aaf1782788 v4.44.1 abd v4.44.2 2024-06-16 11:16:43 +10:00
Mike Farah 450a4b4d37 Added property number example 2024-05-12 12:14:14 +10:00
Mike Farah 31f8a80b5f typos 2024-03-24 11:31:54 +11:00
Mike Farah 7200b5afa0 typo 2024-03-24 11:27:08 +11:00
Mike Farah b9f4f3bf5b Fixing docs 2024-03-24 11:09:39 +11:00
Mike Farah 30e6afaf68 v4.43.1 2024-03-24 10:59:15 +11:00
Mike Farah f81f7a216d v4.41.2 2024-03-05 10:14:15 +11:00
Mike Farah 50adfee6ff Moving tip to recipe 2024-02-21 14:52:39 +11:00
Mike Farah b2965d58a6 v4.41.1 2024-02-16 09:55:01 +11:00
Mike Farah 5edabcf934 Improve merge docs 2024-02-09 14:54:36 +11:00
Mike Farah 2346332e86 Updating style docs 2024-02-09 10:47:22 +11:00
Mike Farah 461bb9c5fe Need to escape single quotes in bash, updated docs #1932 2024-02-02 14:59:08 +11:00
Mike Farah f46e04d6f2 Added another new recipe 2023-12-13 10:56:05 +11:00
Mike Farah 8ae7ad5c10 Updating JSON doc: NDJSON spec deprecated, prefer JSON Lines 2023-12-01 11:39:30 +11:00
Mike Farah 7bed0cc86f missing links 2023-11-23 12:01:23 +11:00
Mike Farah cac23ac003 v4.40.3 2023-11-23 11:59:14 +11:00
Mike Farah b06ab987b1 v4.35.2 2023-09-26 14:50:18 +10:00
Mike Farah 4a36926788 Added recipes documentation 2023-09-01 14:41:07 +10:00
Mike Farah d25785c3f3 updating docs 2023-09-01 11:52:38 +10:00
ryohma0510andGitHub 892308bf54 Improve comparing tips document (#1727) 2023-07-18 08:53:50 +10:00
Mike Farah 8986b92d05 Added docs on base64 only working with utf8 content 2023-05-23 15:38:15 +10:00
Mike Farah d0ccd7c8c8 Added docs on yes/no being strings in yaml spec 1.2 2023-05-23 15:36:12 +10:00
Mike Farah e0558d4b9b Added base64 disclaimer re ut8 strings 2023-05-23 13:30:36 +10:00
Mike Farah 3388f9f3e2 v4.34.1 2023-05-23 13:28:47 +10:00
Mike Farah 2f98ff832c toml docs 2023-03-31 10:24:19 +11:00
Mike Farah dc45add0e9 toml docs 2023-03-31 10:22:57 +11:00
Mike Farah 93adeb2d79 v4.32.1 2023-03-18 14:54:23 +11:00
Mike Farah f6fad90a48 v4.31.2 2023-03-02 11:22:04 +11:00
Mike Farah 814a6114a8 v4.31.1 2023-02-20 16:08:51 +11:00
Mike Farah 3f13bf65a1 v4.30.7 2023-01-14 14:06:09 +11:00
Mike Farah 651c981cad bash loop example 2023-01-14 13:57:32 +11:00
Mike Farah c6537a9d66 added update multiple files tips 2023-01-10 13:53:54 +11:00
Mike Farah 663713c288 v4.30.6 2022-12-17 11:16:27 +11:00
Mike Farah cdc91e518f v4.30.5 2022-12-17 11:15:38 +11:00
muzimuzhiandGitHub 95a5e3b9e3 Fixed doc typos (#1457) 2022-12-05 16:49:31 +11:00
Mike Farah 25e91abe3a v4.30.2 2022-11-13 11:16:29 +11:00
Mike Farah 245a3dbe34 v4.30.1 2022-11-11 14:58:39 +11:00
Mike Farah db8a7617ce Improved set path example 2022-11-10 10:24:32 +11:00
Mike Farah 2926e1d9b1 Improved set path example 2022-11-10 10:22:22 +11:00
Mike Farah f64376da9b Added set path example 2022-11-10 10:19:41 +11:00
Mike Farah 2bab6385bd Added set path example 2022-11-10 10:18:08 +11:00
Mike Farah ef7f61444d Added subset example 2022-11-01 09:30:22 +11:00
Mike Farah a7e397757b v4.29.1 2022-10-29 18:24:11 +11:00
Mike Farah 28a82e9ae8 Added custom sort map example 2022-10-17 15:03:29 +11:00
Mike Farah 2a3e78cb32 Added custom sort map example 2022-10-17 15:02:33 +11:00
Mike Farah de053f6764 Fixed broken link 2022-10-14 14:24:51 +11:00
Mike Farah 2db47ec2f2 v4.28.1 2022-10-05 20:23:03 +11:00
Mike Farah 5f21996380 Added another comment test 2022-09-30 09:46:17 +10:00
Mike Farah c6394d2058 Bumping version 2022-09-09 11:23:34 +10:00
Mike Farah 05ee3403ca v4.27.3 updates 2022-08-29 18:31:50 +10:00
Mike Farah 8c2488d25f improved v3 merge docs 2022-08-29 10:58:08 +10:00
Mike Farah 1bb6132620 improved v3 merge docs 2022-08-29 10:22:32 +10:00
Mike Farah dcab9032d6 Added github action env example 2022-08-14 10:36:23 +10:00
Mike Farah 3e8636a0f8 Add link to discussions on readme 2022-08-01 16:56:21 +10:00
Mike Farah 9458fa1cf5 Updating doc for release 2022-08-01 16:46:12 +10:00
Mike Farah 97c5c563e6 Added CSV example for array of objects 2022-07-29 14:33:26 +10:00
Mike Farah 84f2ca4d32 Added github trouble shooting 2022-07-28 11:33:14 +10:00
Mike Farah fa84423a82 Added github trouble shooting 2022-07-28 11:32:35 +10:00
Mike Farah 848160c7e7 fixed link 2022-07-22 08:52:45 +10:00
Mike Farah 7f0d38fe90 adedd string blocks link in tips and tricks 2022-07-22 08:51:28 +10:00
Mike Farah e397591cb4 Updating docs for v4.26.1 release 2022-07-15 10:45:22 +10:00
Karl HorkyandGitHub 465765929a Fix flag (#1267) 2022-07-07 10:27:40 +10:00
Mike Farah 68c5982ba5 Updating docs for v4.25.3 release 2022-06-23 19:46:28 +10:00
Mike Farah b52d86e468 Added multiple object example from PR #1226 2022-05-27 11:22:29 +10:00
Mike Farah deb7aeb966 Fixing typos 2022-05-25 11:01:35 +10:00
Mike Farah 27a14ec188 v4.25.1 2022-04-29 12:16:48 +10:00
Mike Farah 3fe9e55f36 v4.24.5 2022-04-15 08:52:46 +10:00
Mike Farah 91ac643fbf Added if statement tip and trick 2022-04-01 13:41:39 +11:00
Mike Farah 5908987ef6 Fixing string docs 2022-04-01 13:22:25 +11:00
Mike Farah fe4a753ca4 Added tip 2022-03-29 13:35:30 +11:00
Mike Farah ac68be0a3b Fixed help links 2022-03-29 11:12:22 +11:00
Mike Farah 58f27a1f20 v4.24.1 2022-03-28 14:24:57 +11:00
Mike Farah 3cca5f7aff Better string docs 2022-03-23 10:28:38 +11:00
Mike Farah e402e47522 v4.23.1 2022-03-20 13:05:09 +11:00
Mike Farah 831d601238 Added pick operator 2022-03-10 15:44:35 +11:00
Mike Farah a253c51310 v4.22.1 2022-03-10 13:50:03 +11:00
Mike Farah 85df955bf1 Fixed broken link 2022-03-08 14:04:38 +11:00
Mike Farah 2dbcb51b82 v4.21.1 2022-02-27 12:10:01 +11:00
Mike Farah caf36e39b8 Latest release 2022-02-15 14:16:04 +11:00
Mike Farah 38f40fbcff Added eval to contents page 2022-02-06 14:46:52 +11:00
Mike Farah 1b494f46bf Updating docs for v4.19.1 release 2022-02-06 14:41:27 +11:00
Mike Farahandgitbook-bot dd22aada6d GitBook: [#217] No subject 2022-02-06 03:31:21 +00:00
Mike Farah deb71a277c trial hint 2022-02-06 14:29:26 +11:00
Mike Farahandgitbook-bot e30f679bea GitBook: [#216] No subject 2022-02-06 03:26:23 +00:00
Mike Farahandgitbook-bot 61feeed64f GitBook: [#215] test 2022-02-06 03:24:42 +00:00
Mike Farah 05ee6a44cf Fixed load from another file tip 2022-01-31 08:23:09 +11:00
Mike Farah ab1855e30d Added notice for older versions of yq 2022-01-28 13:06:58 +11:00
Mike Farah 2e3f5e410e Simplified yq usage 2022-01-28 12:50:13 +11:00
Mike Farah 9896245f71 Simplified yq usage 2022-01-28 12:45:43 +11:00
Mike Farah e84a50db2e Better xml docs 2022-01-22 12:36:01 +11:00
Mike Farah 078d1d0c76 Added xml link 2022-01-22 12:13:15 +11:00
Mike Farah 7abf77b323 Updating docs for v4.17.1 release 2022-01-22 11:49:12 +11:00
Mike Farah 9bebbfffb7 Improving select docs 2022-01-15 18:22:12 +11:00
Mike Farah 3793f723b8 Improving select docs (#839) 2022-01-15 18:19:12 +11:00
Mike Farah 6c9a77b336 minor change 2021-12-21 09:40:32 +11:00
Mike Farah 68d13b2220 Improving docs 2021-12-05 13:03:45 +11:00
Mike Farah 3e0da390ee better troubleshooting 2021-12-05 12:15:19 +11:00
Mike Farah 7598f73204 shorten filename in example 2021-12-05 11:14:06 +11:00
Mike Farah cd3813e285 Improved tips and tricks 2021-12-05 11:11:59 +11:00
Mike Farah ed03d21a0b Added another sort example 2021-12-05 10:59:12 +11:00
Mike Farah 1468c37e0b Docs for 4.16.1 2021-12-04 14:08:16 +11:00
Mike Farah 3e0e2c83ed Better merge example 2021-11-29 21:29:05 +11:00
Mike Farah a1db47122d Fixed header 2021-11-29 21:06:49 +11:00
Mike Farah 6c347598ad Better formatting of merge arrays example 2021-11-29 21:06:00 +11:00
Mike Farah de3110cf16 Added link to merge array example in tips and tricks 2021-11-29 21:04:55 +11:00
Mike Farah fd3ee46ccf Better merge array by key example, fixed spelling mistakes 2021-11-29 20:56:49 +11:00
Mike Farah 61f561a9e0 Updating docs 2021-11-26 09:50:45 +11:00
Mike Farahandgitbook-bot 76359d94c3 GitBook: [#212] poewrshell tips and tricks 2021-11-14 06:01:55 +00:00
Mike Farah 3cdbfebe46 Updating comment op doc 2021-11-14 11:10:43 +11:00
Mike Farah 832db92eb2 Tweaking docs 2021-11-03 21:57:50 +11:00
Mike Farah 8c4ae979a2 Updating README 2021-11-03 21:51:28 +11:00
Mike Farah a41131816a Update document generation script 2021-11-03 15:00:28 +11:00
Mike Farah 5bedd759a8 no op doc 2021-11-03 13:36:38 +11:00
Mike Farah 566972f201 How it works 2021-11-03 13:33:58 +11:00
Mike Farah b6a1f59464 Moved split nav order 2021-10-30 16:10:35 +11:00
Mike Farahandgitbook-bot 27268aae26 GitBook: [#209] split docs 2021-10-30 05:09:29 +00:00
Mike Farah 7ec745a2f7 Added flatten doc 2021-10-30 15:52:05 +11:00
Mike Farah e92dc5060c Added encode / decode docs 2021-10-30 15:49:10 +11:00
Mike Farah 0dfb73a5fc added group-by doc 2021-10-30 15:00:37 +11:00
Mike Farah a7411db345 added group-by doc 2021-10-30 14:43:45 +11:00
Mike Farah a178934061 auto update docs poc 2021-10-30 14:40:51 +11:00
Mike Farah ab732e0cc6 try updating add 2021-10-30 14:23:02 +11:00
Mike Farahandgitbook-bot fbe266a166 GitBook: [#206] deep update example 2021-10-30 03:14:39 +00:00
gitbook-bot b723783231 Initialize repository 2021-10-30 03:14:17 +00:00
499 changed files with 3681 additions and 38916 deletions
-1
View File
@@ -1 +0,0 @@
bin/*
Binary file not shown.
-1
View File
@@ -1 +0,0 @@
github: mikefarah
-51
View File
@@ -1,51 +0,0 @@
---
name: Bug report - V3
about: Create a report to help us improve
title: ''
labels: bug, v3
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
Note that any how to questions should be posted in the discussion board and not raised as an issue.
Version of yq: 3.X.X
Operating system: mac/linux/windows/....
Installed via: docker/binary release/homebrew/snap/...
**Input Yaml**
Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
data1.yml:
```yaml
this: should really work
```
data2.yml:
```yaml
but: it strangely didn't
```
**Command**
The command you ran:
```
yq merge data1.yml data2.yml
```
**Actual behavior**
```yaml
cat: meow
```
**Expected behavior**
```yaml
this: should really work
but: it strangely didn't
```
**Additional context**
Add any other context about the problem here.
-51
View File
@@ -1,51 +0,0 @@
---
name: Bug report - V4
about: Create a report to help us improve
title: ''
labels: bug, v4
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
Note that any how to questions should be posted in the discussion board and not raised as an issue.
Version of yq: 4.X.X
Operating system: mac/linux/windows/....
Installed via: docker/binary release/homebrew/snap/...
**Input Yaml**
Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
data1.yml:
```yaml
this: should really work
```
data2.yml:
```yaml
but: it strangely didn't
```
**Command**
The command you ran:
```
yq eval-all 'select(fileIndex==0) | .a.b.c' data1.yml data2.yml
```
**Actual behavior**
```yaml
cat: meow
```
**Expected behavior**
```yaml
this: should really work
but: it strangely didn't
```
**Additional context**
Add any other context about the problem here.
-42
View File
@@ -1,42 +0,0 @@
---
name: Feature request - V4
about: Suggest an idea for this project
title: ''
labels: enhancement, v4
assignees: ''
---
**Please describe your feature request.**
A clear and concise description of what the request is and what it would solve.
Eg. I wish I could use yq to [...]
Note:
- how to questions should be posted in the discussion board and not raised as an issue.
- V3 will no longer have any enhancements.
**Describe the solution you'd like**
If we have data1.yml like:
(please keep to around 10 lines )
```yaml
country: Australia
```
And we run a command:
```bash
yq 'predictWeatherOf(.country)'
```
it could output
```yaml
temp: 32
```
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
-22
View File
@@ -1,22 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: docker
directory: /
schedule:
day: thursday
interval: weekly
- package-ecosystem: github-actions
directory: /
schedule:
day: thursday
interval: weekly
- package-ecosystem: gomod
directory: /
schedule:
day: thursday
interval: weekly
-70
View File
@@ -1,70 +0,0 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"
on:
push:
branches: [ master ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
schedule:
- cron: '24 3 * * 1'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@v4
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3
# ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language
#- run: |
# make bootstrap
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
-100
View File
@@ -1,100 +0,0 @@
name: Release Docker
on:
release:
types: [released]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs:
publishDocker:
environment: dockerhub
env:
IMAGE_NAME: mikefarah/yq
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
platforms: all
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
with:
version: latest
- name: Available platforms
run: echo ${{ steps.buildx.outputs.platforms }} && docker version
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push image
run: |
echo "GithubRef: ${GITHUB_REF}"
VERSION=${GITHUB_REF##*/}
echo "VERSION: ${VERSION}"
IMAGE_VERSION=${VERSION:1}
echo "IMAGE_VERSION: ${IMAGE_VERSION}"
PLATFORMS="linux/amd64,linux/ppc64le,linux/arm64,linux/arm/v7,linux/s390x"
echo "Building and pushing version ${IMAGE_VERSION} of image ${IMAGE_NAME}"
docker buildx build \
--label "org.opencontainers.image.authors=https://github.com/mikefarah/yq/graphs/contributors" \
--label "org.opencontainers.image.created=$(date --rfc-3339=seconds)" \
--label "org.opencontainers.image.description=yq is a portable command-line data file processor" \
--label "org.opencontainers.image.documentation=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.licenses=MIT" \
--label "org.opencontainers.image.revision=$(git rev-parse HEAD)" \
--label "org.opencontainers.image.source=https://github.com/mikefarah/yq" \
--label "org.opencontainers.image.title=yq" \
--label "org.opencontainers.image.url=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.version=${IMAGE_VERSION}" \
--platform "${PLATFORMS}" \
--pull \
--push \
-t "${IMAGE_NAME}:${IMAGE_VERSION}" \
-t "${IMAGE_NAME}:4" \
-t "${IMAGE_NAME}:latest" \
-t "ghcr.io/${IMAGE_NAME}:${IMAGE_VERSION}" \
-t "ghcr.io/${IMAGE_NAME}:4" \
-t "ghcr.io/${IMAGE_NAME}:latest" \
.
cd github-action
docker buildx build \
--label "org.opencontainers.image.authors=https://github.com/mikefarah/yq/graphs/contributors" \
--label "org.opencontainers.image.created=$(date --rfc-3339=seconds)" \
--label "org.opencontainers.image.description=yq is a portable command-line data file processor" \
--label "org.opencontainers.image.documentation=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.licenses=MIT" \
--label "org.opencontainers.image.revision=$(git rev-parse HEAD)" \
--label "org.opencontainers.image.source=https://github.com/mikefarah/yq" \
--label "org.opencontainers.image.title=yq" \
--label "org.opencontainers.image.url=https://mikefarah.gitbook.io/yq/" \
--label "org.opencontainers.image.version=${IMAGE_VERSION}" \
--platform "${PLATFORMS}" \
--pull \
--push \
-t "${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "${IMAGE_NAME}:4-githubaction" \
-t "${IMAGE_NAME}:latest-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:${IMAGE_VERSION}-githubaction" \
-t "ghrc.io/${IMAGE_NAME}:4-githubaction" \
-t "ghcr.io/${IMAGE_NAME}:latest-githubaction" \
.
-35
View File
@@ -1,35 +0,0 @@
name: Build
on: [push, pull_request]
permissions:
contents: read
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '^1.20'
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v4
- name: Get dependencies
run: |
go get -v -t -d ./...
if [ -f Gopkg.toml ]; then
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
dep ensure
fi
- name: Check the build
shell: bash -l {0}
run: |
export PATH=${PATH}:`go env GOPATH`/bin
scripts/devtools.sh
make local build
-51
View File
@@ -1,51 +0,0 @@
name: Release YQ
on:
push:
tags:
- 'v4.*'
- 'draft-*'
jobs:
publishGitRelease:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '^1.20'
check-latest: true
- name: Compile man page markup
id: gen-man-page-md
run: |
./scripts/generate-man-page-md.sh
- name: Get the version
id: get_version
run: echo "VERSION=${GITHUB_REF##*/}" >> "${GITHUB_OUTPUT}"
- name: Generate man page
uses: docker://pandoc/core:2.14.2
id: gen-man-page
with:
args: >-
--standalone
--to man
--variable=title:"YQ"
--variable=section:"1"
--variable=header:"yq (https://github.com/mikefarah/yq/) version ${{ steps.get_version.outputs.VERSION }}"
--variable=author:"Mike Farah"
--output=yq.1
man.md
- name: Cross compile
run: |
sudo apt-get install rhash -y
go install github.com/goreleaser/goreleaser/v2@latest
./scripts/xcompile.sh
- name: Release
uses: softprops/action-gh-release@v1
with:
files: build/*
draft: true
fail_on_unmatched_files: true
-23
View File
@@ -1,23 +0,0 @@
name: Release Snap
on:
release:
types: [released]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
jobs:
buildSnap:
environment: snap
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: snapcore/action-build@v1
id: build
- uses: snapcore/action-publish@v1
env:
SNAPCRAFT_STORE_CREDENTIALS: ${{ secrets.STORE_LOGIN }}
with:
snap: ${{ steps.build.outputs.snap }}
release: stable
-28
View File
@@ -1,28 +0,0 @@
name: Test Yq Action
on:
push:
workflow_dispatch:
permissions:
contents: read
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Get test
id: get_value
uses: mikefarah/yq@master
with:
cmd: yq '.test' examples/multiline-text.yaml
- name: Multiline test
run: echo "### It was [${{ steps.get_value.outputs.result }}]" >> $GITHUB_STEP_SUMMARY
- name: Write inplace test
id: lookupSdkVersion
uses: mikefarah/yq@master
with:
cmd: yq -i '.b.c = 5' examples/sample.yaml
-69
View File
@@ -1,69 +0,0 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
bin
build
build-done
.DS_Store
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
cover.out
coverage.out
coverage.html
*.exe
*.test
*.prof
yaml
vendor/
tmp/
cover/
yq
# snapcraft
parts/
prime/
.snapcraft/
yq*.snap
test.yml
test*.yml
test*.xml
test*.toml
test*.yaml
test_dir1/
test_dir2/
0.yml
1.yml
2.yml
# man page
man.md
yq.1
# debian pkg
_build
debian/files
# intellij
/.idea
# vscode
.vscode
yq3
-38
View File
@@ -1,38 +0,0 @@
run:
timeout: 5m
linters:
enable:
- asciicheck
- depguard
- errorlint
- gci
- gochecknoinits
- gofmt
- goimports
- gosec
- gosimple
- staticcheck
- unused
- misspell
- nakedret
- nolintlint
- predeclared
- revive
- unconvert
- unparam
linters-settings:
depguard:
rules:
prevent_unmaintained_packages:
list-mode: lax
files:
- $all
- "!$test"
deny:
- pkg: io/ioutil
desc: "replaced by io and os packages since Go 1.16: https://tip.golang.org/doc/go1.16#ioutil"
issues:
exclude-rules:
- linters:
- revive
text: "var-naming"
-52
View File
@@ -1,52 +0,0 @@
version: "2"
linters:
enable:
- asciicheck
- depguard
- errorlint
- gochecknoinits
- gosec
- misspell
- nakedret
- nolintlint
- predeclared
- revive
- unconvert
- unparam
settings:
depguard:
rules:
prevent_unmaintained_packages:
list-mode: lax
files:
- $all
- '!$test'
deny:
- pkg: io/ioutil
desc: 'replaced by io and os packages since Go 1.16: https://tip.golang.org/doc/go1.16#ioutil'
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
rules:
- linters:
- revive
text: var-naming
paths:
- third_party$
- builtin$
- examples$
formatters:
enable:
- gci
- gofmt
- goimports
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
-48
View File
@@ -1,48 +0,0 @@
version: 2
dist: build
builds:
- id: yq
binary: yq_{{ .Os }}_{{ .Arch }}
ldflags:
- -s -w
env:
- CGO_ENABLED=0
targets:
- darwin_amd64
- darwin_arm64
- freebsd_386
- freebsd_amd64
- freebsd_arm
- linux_386
- linux_amd64
- linux_arm
- linux_arm64
- linux_mips
- linux_mips64
- linux_mips64le
- linux_mipsle
- linux_ppc64
- linux_ppc64le
- linux_riscv64
- linux_s390x
- netbsd_386
- netbsd_amd64
- netbsd_arm
- openbsd_386
- openbsd_amd64
- windows_386
- windows_amd64
- windows_arm
- windows_arm64
no_unique_dist_dir: true
release:
disable: true
skip_upload: true
-76
View File
@@ -1,76 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at mikefarah@gmail.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq
-42
View File
@@ -1,42 +0,0 @@
.# Development
1. Install (Golang)[https://golang.org/]
1. Run `scripts/devtools.sh` to install the required devtools
2. Run `make [local] vendor` to install the vendor dependencies
2. Run `make [local] test` to ensure you can run the existing tests
3. Write unit tests - (see existing examples). Changes will not be accepted without corresponding unit tests.
4. Make the code changes.
5. `make [local] test` to lint code and run tests
6. Profit! ok no profit, but raise a PR and get kudos :)
# Documentation
The documentation is a bit of a mixed bag (sorry in advance, I do plan on simplifying it...) - with some parts automatically generated and stiched together and some statically defined.
Documentation is written in markdown, and is published in the 'gitbook' branch.
The various operator documentation (e.g. 'strings') are generated from the 'master' branch, and have a statically defined header (e.g. `pkg/yqlib/doc/operators/headers/add.md`) and the bulk of the docs are generated from the unit tests e.g. `pkg/yqlib/operator_add_test.go`.
The pipeline will run the tests and automatically concatenate the files together, and put them under
`pkg/qylib/doc/add.md`. These files are checked in the master branch (and are copied to the gitbook branch as part of the release process).
## How to contribute
The first step is to find if what you want is automatically generated or not - start by looking in the master branch.
Note that PRs with small changes (e.g. minor typos) may not be merged (see https://joel.net/how-one-guy-ruined-hacktoberfest2020-drama).
### Updating dynamic documentation from master
- Search for the documentation you want to update. If you find matches in a `*_test.go` file - update that, as that will automatically update the matching `*.md` file
- Assuming you are updating a `*_test.go` file, once updated, run the test to regenerated the docs. E.g. for the 'Add' test generated docs, from the pkg/yqlib folder run:
`go test -run TestAddOperatorScenarios` which will run that test defined in the `operator_add_test.go` file.
- Ensure the tests still pass, and check the generated documentation have your update.
- Note: If the documentation is only in a `headers/*.md` file, then just update that directly
- Raise a PR to merge the changes into master!
### Updating static documentation from the gitbook branch
If you haven't found what you want to update in the master branch, then check the gitbook branch directly as there are a few pages in there that are not in master.
- Update the `*.md` files
- Raise a PR to merge the changes into gitbook.
-28
View File
@@ -1,28 +0,0 @@
FROM golang:1.24.2 AS builder
WORKDIR /go/src/mikefarah/yq
COPY . .
RUN CGO_ENABLED=0 go build -ldflags "-s -w" .
# RUN ./scripts/test.sh -- this too often times out in the github pipeline.
RUN ./scripts/acceptance.sh
# Choose alpine as a base image to make this useful for CI, as many
# CI tools expect an interactive shell inside the container
FROM alpine:3 AS production
LABEL maintainer="Mike Farah <mikefarah@users.noreply.github.com>"
COPY --from=builder /go/src/mikefarah/yq/yq /usr/bin/yq
WORKDIR /workdir
RUN set -eux; \
addgroup -g 1000 yq; \
adduser -u 1000 -G yq -s /bin/sh -h /home/yq -D yq
RUN chown -R yq:yq /workdir
USER yq
ENTRYPOINT ["/usr/bin/yq"]
-15
View File
@@ -1,15 +0,0 @@
FROM golang:1.24.2
RUN apt-get update && \
apt-get install -y npm && \
npm install -g npx cspell@latest
COPY scripts/devtools.sh /opt/devtools.sh
RUN set -e -x && \
/opt/devtools.sh
ENV PATH=/go/bin:$PATH
ENV CGO_ENABLED 0
ENV GOPATH /go:/yq
-21
View File
@@ -1,21 +0,0 @@
The MIT License (MIT)
Copyright (c) 2017 Mike Farah
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-123
View File
@@ -1,123 +0,0 @@
MAKEFLAGS += --warn-undefined-variables
SHELL := /bin/bash
.SHELLFLAGS := -o pipefail -euc
.DEFAULT_GOAL := install
ENGINE := $(shell { (podman version > /dev/null 2>&1 && command -v podman) || command -v docker; } 2>/dev/null)
include Makefile.variables
.PHONY: help
help:
@echo 'Management commands for cicdtest:'
@echo
@echo 'Usage:'
@echo ' ## Develop / Test Commands'
@echo ' make build Build yq binary.'
@echo ' make install Install yq.'
@echo ' make xcompile Build cross-compiled binaries of yq.'
@echo ' make vendor Install dependencies to vendor directory.'
@echo ' make format Run code formatter.'
@echo ' make check Run static code analysis (lint).'
@echo ' make secure Run gosec.'
@echo ' make test Run tests on project.'
@echo ' make cover Run tests and capture code coverage metrics on project.'
@echo ' make clean Clean the directory tree of produced artifacts.'
@echo
@echo ' ## Utility Commands'
@echo ' make setup Configures Minishfit/Docker directory mounts.'
@echo
.PHONY: clean
clean:
@rm -rf bin build cover *.out
## prefix before other make targets to run in your local dev environment
local: | quiet
@$(eval ENGINERUN= )
@mkdir -p tmp
@touch tmp/dev_image_id
quiet: # this is silly but shuts up 'Nothing to be done for `local`'
@:
prepare: tmp/dev_image_id
tmp/dev_image_id: Dockerfile.dev scripts/devtools.sh
@mkdir -p tmp
@${ENGINE} rmi -f ${DEV_IMAGE} > /dev/null 2>&1 || true
@${ENGINE} build -t ${DEV_IMAGE} -f Dockerfile.dev .
@${ENGINE} inspect -f "{{ .ID }}" ${DEV_IMAGE} > tmp/dev_image_id
# ----------------------------------------------
# build
.PHONY: build
build: build/dev
.PHONY: build/dev
build/dev: test *.go
@mkdir -p bin/
${ENGINERUN} go build --ldflags "$(LDFLAGS)"
${ENGINERUN} bash ./scripts/acceptance.sh
## Compile the project for multiple OS and Architectures.
xcompile: check
@rm -rf build/
@mkdir -p build
${ENGINERUN} bash ./scripts/xcompile.sh
@find build -type d -exec chmod 755 {} \; || :
@find build -type f -exec chmod 755 {} \; || :
.PHONY: install
install: build
${ENGINERUN} go install
# Each of the fetch should be an entry within vendor.json; not currently included within project
.PHONY: vendor
vendor: tmp/dev_image_id
@mkdir -p vendor
${ENGINERUN} go mod vendor
# ----------------------------------------------
# develop and test
.PHONY: format
format: vendor
${ENGINERUN} bash ./scripts/format.sh
.PHONY: spelling
spelling: format
${ENGINERUN} bash ./scripts/spelling.sh
.PHONY: secure
secure: spelling
${ENGINERUN} bash ./scripts/secure.sh
.PHONY: check
check: secure
${ENGINERUN} bash ./scripts/check.sh
.PHONY: test
test: check
${ENGINERUN} bash ./scripts/test.sh
.PHONY: cover
cover: check
@rm -rf cover/
@mkdir -p cover
${ENGINERUN} bash ./scripts/coverage.sh
@find cover -type d -exec chmod 755 {} \; || :
@find cover -type f -exec chmod 644 {} \; || :
.PHONY: release
release: xcompile
${ENGINERUN} bash ./scripts/publish.sh
# ----------------------------------------------
# utilities
.PHONY: setup
setup:
@bash ./scripts/setup.sh
-40
View File
@@ -1,40 +0,0 @@
export PROJECT = yq
IMPORT_PATH := github.com/mikefarah/${PROJECT}
export GIT_COMMIT = $(shell git rev-parse --short HEAD)
export GIT_DIRTY = $(shell test -n "$$(git status --porcelain)" && echo "+CHANGES" || true)
export GIT_DESCRIBE = $(shell git describe --tags --always)
LDFLAGS :=
LDFLAGS += -X main.GitCommit=${GIT_COMMIT}${GIT_DIRTY}
LDFLAGS += -X main.GitDescribe=${GIT_DESCRIBE}
LDFLAGS += -w -s
GITHUB_TOKEN ?=
# Windows environment?
CYG_CHECK := $(shell hash cygpath 2>/dev/null && echo 1)
ifeq ($(CYG_CHECK),1)
VBOX_CHECK := $(shell hash VBoxManage 2>/dev/null && echo 1)
# Docker Toolbox (pre-Windows 10)
ifeq ($(VBOX_CHECK),1)
ROOT := /${PROJECT}
else
# Docker Windows
ROOT := $(shell cygpath -m -a "$(shell pwd)")
endif
else
# all non-windows environments
ROOT := $(shell pwd)
SELINUX := $(shell which getenforce 2>&1 >/dev/null && echo :z)
endif
DEV_IMAGE := ${PROJECT}_dev
ENGINERUN := ${ENGINE} run --rm \
-e LDFLAGS="${LDFLAGS}" \
-e GITHUB_TOKEN="${GITHUB_TOKEN}" \
-v ${ROOT}/vendor:/go/src${SELINUX} \
-v ${ROOT}:/${PROJECT}/src/${IMPORT_PATH}${SELINUX} \
-w /${PROJECT}/src/${IMPORT_PATH} \
${DEV_IMAGE}
+3 -334
View File
@@ -1,9 +1,6 @@
# yq
![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq) ![CodeQL](https://github.com/mikefarah/yq/workflows/CodeQL/badge.svg)
a lightweight and portable command-line YAML, JSON and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files as well as json, xml, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
a lightweight and portable command-line YAML, JSON, INI and XML processor. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files as well as json, xml, ini, properties, csv and tsv. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
yq is written in go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as Docker and Podman, all listed below.
@@ -65,338 +62,10 @@ Take a look at the discussions for [common questions](https://github.com/mikefar
## Install
### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
See the [github page](https://github.com/mikefarah/yq/#install) for the various ways you can install and use `yq`
### wget
Use wget to download, gzipped pre-compiled binaries:
For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64
#### Compressed via tar.gz
```bash
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
tar xz && mv ${BINARY} /usr/local/bin/yq
```
#### Plain binary
```bash
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/local/bin/yq &&\
chmod +x /usr/local/bin/yq
```
#### Latest version
```bash
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq &&\
chmod +x /usr/local/bin/yq
```
### MacOS / Linux via Homebrew:
Using [Homebrew](https://brew.sh/)
```
brew install yq
```
### Linux via snap:
```
snap install yq
```
#### Snap notes
`yq` installs with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
```
sudo cat /etc/myfile | yq '.a.path'
```
And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):
```
sudo cat /etc/myfile | yq '.a.path = "value"' | sudo sponge /etc/myfile
```
or write to a temporary file:
```
sudo cat /etc/myfile | yq '.a.path = "value"' | sudo tee /etc/myfile.tmp
sudo mv /etc/myfile.tmp /etc/myfile
rm /etc/myfile.tmp
```
### Run with Docker or Podman
#### Oneshot use:
```bash
docker run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...
```
Note that you can run `yq` in docker without network access and other privileges if you desire,
namely `--security-opt=no-new-privileges --cap-drop all --network none`.
```bash
podman run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...
```
#### Pipe in via STDIN:
You'll need to pass the `-i\--interactive` flag to docker:
```bash
docker run -i --rm mikefarah/yq '.this.thing' < myfile.yml
```
```bash
podman run -i --rm mikefarah/yq '.this.thing' < myfile.yml
```
#### Run commands interactively:
```bash
docker run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
```
```bash
podman run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
```
It can be useful to have a bash function to avoid typing the whole docker command:
```bash
yq() {
docker run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
```
```bash
yq() {
podman run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
```
#### Running as root:
`yq`'s container image no longer runs under root (https://github.com/mikefarah/yq/pull/860). If you'd like to install more things in the container image, or you're having permissions issues when attempting to read/write files you'll need to either:
```
docker run --user="root" -it --entrypoint sh mikefarah/yq
```
```
podman run --user="root" -it --entrypoint sh mikefarah/yq
```
Or, in your Dockerfile:
```
FROM mikefarah/yq
USER root
RUN apk add --no-cache bash
USER yq
```
#### Missing timezone data
By default, the alpine image yq uses does not include timezone data. If you'd like to use the `tz` operator, you'll need to include this data:
```
FROM mikefarah/yq
USER root
RUN apk add --no-cache tzdata
USER yq
```
#### Podman with SELinux
If you are using podman with SELinux, you will need to set the shared volume flag `:z` on the volume mount:
```
-v "${PWD}":/workdir:z
```
### GitHub Action
```
- name: Set foobar to cool
uses: mikefarah/yq@master
with:
cmd: yq -i '.foo.bar = "cool"' 'config.yml'
- name: Get an entry with a variable that might contain dots or spaces
id: get_username
uses: mikefarah/yq@master
with:
cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
- name: Reuse a variable obtained in another step
run: echo ${{ steps.get_username.outputs.result }}
```
See https://mikefarah.gitbook.io/yq/usage/github-action for more.
### Go Install:
```
go install github.com/mikefarah/yq/v4@latest
```
## Community Supported Installation methods
As these are supported by the community :heart: - however, they may be out of date with the officially supported releases.
_Please note that the Debian package (previously supported by @rmescandon) is no longer maintained. Please use an alternative installation method._
### X-CMD
Checkout `yq` on x-cmd: https://x-cmd.com/mod/yq
- Instant Results: See the output of your yq filter in real-time.
- Error Handling: Encounter a syntax error? It will display the error message and the results of the closest valid filter
Thanks @edwinjhlee!
### Nix
```
nix profile install nixpkgs#yq-go
```
See [here](https://search.nixos.org/packages?channel=unstable&show=yq-go&from=0&size=50&sort=relevance&type=packages&query=yq-go)
### Webi
```
webi yq
```
See [webi](https://webinstall.dev/)
Supported by @adithyasunil26 (https://github.com/webinstall/webi-installers/tree/master/yq)
### Arch Linux
```
pacman -S go-yq
```
### Windows:
Using [Chocolatey](https://chocolatey.org)
[![Chocolatey](https://img.shields.io/chocolatey/v/yq.svg)](https://chocolatey.org/packages/yq)
[![Chocolatey](https://img.shields.io/chocolatey/dt/yq.svg)](https://chocolatey.org/packages/yq)
```
choco install yq
```
Supported by @chillum (https://chocolatey.org/packages/yq)
Using [scoop](https://scoop.sh/)
```
scoop install main/yq
```
Using [winget](https://learn.microsoft.com/en-us/windows/package-manager/)
```
winget install --id MikeFarah.yq
```
### Mac:
Using [MacPorts](https://www.macports.org/)
```
sudo port selfupdate
sudo port install yq
```
Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot)
### Alpine Linux
Alpine Linux v3.20+ (and Edge):
```
apk add yq-go
```
Alpine Linux up to v3.19:
```
apk add yq
```
Supported by Tuan Hoang (https://pkgs.alpinelinux.org/packages?name=yq-go)
### Flox:
Flox can be used to install yq on Linux, MacOS, and Windows through WSL.
```
flox install yq
```
## Features
- [Detailed documentation with many examples](https://mikefarah.gitbook.io/yq/)
- Written in portable go, so you can download a lovely dependency free binary
- Uses similar syntax as `jq` but works with YAML, [JSON](https://mikefarah.gitbook.io/yq/usage/convert) and [XML](https://mikefarah.gitbook.io/yq/usage/xml) files
- Fully supports multi document yaml files
- Supports yaml [front matter](https://mikefarah.gitbook.io/yq/usage/front-matter) blocks (e.g. jekyll/assemble)
- Colorized yaml output
- [Date/Time manipulation and formatting with TZ](https://mikefarah.gitbook.io/yq/operators/datetime)
- [Deeply data structures](https://mikefarah.gitbook.io/yq/operators/traverse-read)
- [Sort keys](https://mikefarah.gitbook.io/yq/operators/sort-keys)
- Manipulate yaml [comments](https://mikefarah.gitbook.io/yq/operators/comment-operators), [styling](https://mikefarah.gitbook.io/yq/operators/style), [tags](https://mikefarah.gitbook.io/yq/operators/tag) and [anchors and aliases](https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators).
- [Update in place](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate#flags)
- [Complex expressions to select and update](https://mikefarah.gitbook.io/yq/operators/select#select-and-update-matching-values-in-map)
- Keeps yaml formatting and comments when updating (though there are issues with whitespace)
- [Decode/Encode base64 data](https://mikefarah.gitbook.io/yq/operators/encode-decode)
- [Load content from other files](https://mikefarah.gitbook.io/yq/operators/load)
- [Convert to/from json/ndjson](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert)
- [Convert to/from xml](https://mikefarah.gitbook.io/yq/v/v4.x/usage/xml)
- [Convert to/from properties](https://mikefarah.gitbook.io/yq/v/v4.x/usage/properties)
- [Convert to/from csv/tsv](https://mikefarah.gitbook.io/yq/usage/csv-tsv)
- [General shell completion scripts (bash/zsh/fish/powershell)](https://mikefarah.gitbook.io/yq/v/v4.x/commands/shell-completion)
- [Reduce](https://mikefarah.gitbook.io/yq/operators/reduce) to merge multiple files or sum an array or other fancy things.
- [Github Action](https://mikefarah.gitbook.io/yq/usage/github-action) to use in your automated pipeline (thanks @devorbitus)
## [Usage](https://mikefarah.gitbook.io/yq/)
Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more detailed and advanced usage.
```
Usage:
yq [flags]
yq [command]
Examples:
# yq defaults to 'eval' command if no command is specified. See "yq eval --help" for more examples.
yq '.stuff' < myfile.yml # outputs the data at the "stuff" node from "myfile.yml"
yq -i '.stuff = "foo"' myfile.yml # update myfile.yml in place
Available Commands:
completion Generate the autocompletion script for the specified shell
eval (default) Apply the expression to each document in each yaml file in sequence
eval-all Loads _all_ yaml documents of _all_ yaml files and runs expression once
help Help about any command
Flags:
-C, --colors force print with colors
-e, --exit-status set exit status if there are no matches or null or false is returned
-f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
--header-preprocess Slurp any header comments and separators before processing expression. (default true)
-h, --help help for yq
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the file in place of first file given.
-p, --input-format string [yaml|y|xml|x] parse format for input. Note that json is a subset of yaml. (default "yaml")
-M, --no-colors force print with no colors
-N, --no-doc Don't print document separators (---)
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
-o, --output-format string [yaml|y|json|j|props|p|xml|x] output format type. (default "yaml")
-P, --prettyPrint pretty print, shorthand for '... style = ""'
-s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
--unwrapScalar unwrap scalar, print the value with no quotes, colors or comments (default true)
-v, --verbose verbose mode
-V, --version Print version information and quit
--xml-attribute-prefix string prefix for xml attributes (default "+")
--xml-content-name string name for xml content (if no attribute name is present). (default "+content")
Use "yq [command] --help" for more information about a command.
```
## Known Issues / Missing Features
- `yq` attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)
- Powershell has its own...[opinions on quoting yq](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks#quotes-in-windows-powershell)
- "yes", "no" were dropped as boolean values in the yaml 1.2 standard - which is the standard yq assumes.
See [tips and tricks](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks) for more common problems and solutions.
See [tips and tricks](https://mikefarah.gitbook.io/yq/usage/tips-and-tricks) for more common problems and solutions.
+101
View File
@@ -0,0 +1,101 @@
# Table of contents
* [yq](README.md)
* [How It Works](how-it-works.md)
* [Recipes](usage/recipes.md)
* [Upgrading from V3](upgrading-from-v3.md)
## Commands
* [Evaluate](commands/evaluate.md)
* [Evaluate All](commands/evaluate-all.md)
* [Shell Completion](commands/shell-completion.md)
***
* [Operators](operators/README.md)
* [Add](operators/add.md)
* [Alternative (Default value)](operators/alternative-default-value.md)
* [Anchor and Alias Operators](operators/anchor-and-alias-operators.md)
* [Array to Map](operators/array-to-map.md)
* [Assign (Update)](operators/assign-update.md)
* [Boolean Operators](operators/boolean-operators.md)
* [Collect into Array](operators/collect-into-array.md)
* [Column](operators/column.md)
* [Comment Operators](operators/comment-operators.md)
* [Compare Operators](operators/compare.md)
* [Contains](operators/contains.md)
* [Create, Collect into Object](operators/create-collect-into-object.md)
* [Date Time](operators/datetime.md)
* [Delete](operators/delete.md)
* [Divide](operators/divide.md)
* [Document Index](operators/document-index.md)
* [Encode / Decode](operators/encode-decode.md)
* [Entries](operators/entries.md)
* [Env Variable Operators](operators/env-variable-operators.md)
* [Equals](operators/equals.md)
* [Eval](operators/eval.md)
* [File Operators](operators/file-operators.md)
* [Filter Operator](operators/filter.md)
* [First Operator](operators/first.md)
* [Flatten](operators/flatten.md)
* [Group By](operators/group-by.md)
* [Has](operators/has.md)
* [Keys](operators/keys.md)
* [Kind](operators/kind.md)
* [Length](operators/length.md)
* [Line](operators/line.md)
* [Load](operators/load.md)
* [Min](operators/min.md)
* [Map](operators/map.md)
* [Max](operators/max.md)
* [Modulo](operators/modulo.md)
* [Multiply (Merge)](operators/multiply-merge.md)
* [Omit](operators/omit.md)
* [Parent](operators/parent.md)
* [Path](operators/path.md)
* [Pick](operators/pick.md)
* [Pipe](operators/pipe.md)
* [Pivot](operators/pivot.md)
* [Recursive Descent (Glob)](operators/recursive-descent-glob.md)
* [Reduce](operators/reduce.md)
* [Reverse](operators/reverse.md)
* [Select](operators/select.md)
* [Shuffle](operators/shuffle.md)
* [Slice Array](operators/slice-array.md)
* [Sort](operators/sort.md)
* [Sort Keys](operators/sort-keys.md)
* [Split into Documents](operators/split-into-documents.md)
* [String Operators](operators/string-operators.md)
* [Style](operators/style.md)
* [Subtract](operators/subtract.md)
* [System](operators/system-operators.md)
* [Tag](operators/tag.md)
* [To Number](operators/to_number.md)
* [Traverse (Read)](operators/traverse-read.md)
* [Union](operators/union.md)
* [Unique](operators/unique.md)
* [Variable Operators](operators/variable-operators.md)
* [With](operators/with.md)
## Usage
* [Output format](usage/output-format.md)
* [Working with Base64](usage/base64.md)
* [Working with CSV, TSV](usage/csv-tsv.md)
* [Working with JSON](usage/convert.md)
* [Working with Properties](usage/properties.md)
* [Working with XML](usage/xml.md)
* [Working with HCL](usage/hcl.md)
* [Working with LUA](usage/lua.md)
* [Working with TOML](usage/toml.md)
* [Working with KYAML](usage/kyaml.md)
* [Working with Shell Output](usage/shellvariables.md)
* [Front Matter](usage/front-matter.md)
* [Split into multiple files](usage/split-into-multiple-files.md)
* [GitHub Action](usage/github-action.md)
* [Tips, Tricks, Troubleshooting](usage/tips-and-tricks.md)
***
* [Github Page](https://github.com/mikefarah/yq)
-41
View File
@@ -1,41 +0,0 @@
#!/bin/bash
testWriteInPlacePipeIn() {
result=$(./yq e -i -n '.a' 2>&1)
assertEquals 1 $?
assertEquals "Error: write in place flag only applicable when giving an expression and at least one file" "$result"
}
testWriteInPlacePipeInEvalall() {
result=$(./yq ea -i -n '.a' 2>&1)
assertEquals 1 $?
assertEquals "Error: write in place flag only applicable when giving an expression and at least one file" "$result"
}
testWriteInPlaceWithSplit() {
result=$(./yq e -s "cat" -i '.a = "thing"' test.yml 2>&1)
assertEquals 1 $?
assertEquals "Error: write in place cannot be used with split file" "$result"
}
testWriteInPlaceWithSplitEvalAll() {
result=$(./yq ea -s "cat" -i '.a = "thing"' test.yml 2>&1)
assertEquals 1 $?
assertEquals "Error: write in place cannot be used with split file" "$result"
}
testNullWithFiles() {
result=$(./yq e -n '.a = "thing"' test.yml 2>&1)
assertEquals 1 $?
assertEquals "Error: cannot pass files in when using null-input flag" "$result"
}
testNullWithFilesEvalAll() {
result=$(./yq ea -n '.a = "thing"' test.yml 2>&1)
assertEquals 1 $?
assertEquals "Error: cannot pass files in when using null-input flag" "$result"
}
source ./scripts/shunit2
-373
View File
@@ -1,373 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml 2>/dev/null || true
rm .xyz 2>/dev/null || true
rm instructions.txt 2>/dev/null || true
}
testBasicEvalRoundTrip() {
./yq -n ".a = 123" > test.yml
X=$(./yq '.a' test.yml)
assertEquals 123 "$X"
}
testBasicTrailingContent() {
cat >test-trailing.yml <<EOL
test:
# this comment will be removed
EOL
read -r -d '' expected << EOM
test:
# this comment will be removed
EOM
X=$(./yq test-trailing.yml -P)
assertEquals "$expected" "$X"
}
testBasicTrailingContent() {
cat >test-trailing.yml <<EOL
test:
# this comment will be removed
EOL
read -r -d '' expected << EOM
test:
# hi
EOM
X=$(./yq '. footComment = "hi"' test-trailing.yml)
assertEquals "$expected" "$X"
}
testBasicTrailingContentEvalAll() {
cat >test-trailing.yml <<EOL
test:
# this comment will be removed
EOL
read -r -d '' expected << EOM
test:
# this comment will be removed
EOM
X=$(./yq ea test-trailing.yml -P)
assertEquals "$expected" "$X"
}
testBasicTrailingContentEvalAll() {
cat >test-trailing.yml <<EOL
test:
# this comment will be removed
EOL
read -r -d '' expected << EOM
test:
# hi
EOM
X=$(./yq ea '. footComment = "hi"' test-trailing.yml)
assertEquals "$expected" "$X"
}
testBasicPipeWithDot() {
./yq -n ".a = 123" > test.yml
X=$(cat test.yml | ./yq '.')
assertEquals "a: 123" "$X"
}
testBasicExpressionMatchesFileName() {
./yq -n ".xyz = 123" > test.yml
touch .xyz
X=$(./yq --expression '.xyz' test.yml)
assertEquals "123" "$X"
X=$(./yq ea --expression '.xyz' test.yml)
assertEquals "123" "$X"
}
testBasicExpressionFromFile() {
./yq -n ".xyz = 123" > test.yml
echo '.xyz = "meow" | .cool = "frog"' > instructions.txt
X=$(./yq --from-file instructions.txt test.yml -o=j -I=0)
assertEquals '{"xyz":"meow","cool":"frog"}' "$X"
X=$(./yq ea --from-file instructions.txt test.yml -o=j -I=0)
assertEquals '{"xyz":"meow","cool":"frog"}' "$X"
}
testBasicExpressionFromFileDos() {
./yq -n ".xyz = 123" > test.yml
echo '.xyz = "meow" | .cool = "frog"' | sed 's/$'"/`echo \\\r`/" > instructions.txt
X=$(./yq --from-file instructions.txt test.yml -o=j -I=0)
assertEquals '{"xyz":"meow","cool":"frog"}' "$X"
X=$(./yq ea --from-file instructions.txt test.yml -o=j -I=0)
assertEquals '{"xyz":"meow","cool":"frog"}' "$X"
}
testBasicGitHubAction() {
./yq -n ".a = 123" > test.yml
X=$(cat /dev/null | ./yq test.yml)
assertEquals "a: 123" "$X"
X=$(cat /dev/null | ./yq e test.yml)
assertEquals "a: 123" "$X"
X=$(cat /dev/null | ./yq ea test.yml)
assertEquals "a: 123" "$X"
}
testBasicGitHubActionWithExpression() {
./yq -n ".a = 123" > test.yml
X=$(cat /dev/null | ./yq '.a' test.yml)
assertEquals "123" "$X"
X=$(cat /dev/null | ./yq e '.a' test.yml)
assertEquals "123" "$X"
X=$(cat /dev/null | ./yq ea '.a' test.yml)
assertEquals "123" "$X"
}
testBasicEvalAllAllFiles() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(./yq ea test.yml test2.yml)
Y=$(./yq e '.' test.yml test2.yml)
assertEquals "$Y" "$X"
}
# when given a file, don't read STDIN
# otherwise strange things start happening
# in scripts
# https://github.com/mikefarah/yq/issues/1115
testBasicCatWithFilesNoDash() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq test2.yml)
Y=$(./yq e '.' test2.yml)
assertEquals "$Y" "$X"
}
# when the nullinput flag is used
# don't automatically read STDIN (this breaks github actions)
testBasicCreateFileGithubAction() {
cat /dev/null | ./yq -n ".a = 123" > test.yml
}
testBasicEvalAllCatWithFilesNoDash() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq ea test2.yml)
Y=$(./yq e '.' test2.yml)
assertEquals "$Y" "$X"
}
testBasicCatWithFilesNoDashWithExp() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq '.a' test2.yml)
Y=$(./yq e '.a' test2.yml)
assertEquals "$Y" "$X"
}
testBasicEvalAllCatWithFilesNoDashWithExp() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq ea '.a' test2.yml)
Y=$(./yq e '.a' test2.yml)
assertEquals "$Y" "$X"
}
testBasicStdInWithFiles() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq - test2.yml)
Y=$(./yq e '.' test.yml test2.yml)
assertEquals "$Y" "$X"
}
testBasicEvalAllStdInWithFiles() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq ea - test2.yml)
Y=$(./yq e '.' test.yml test2.yml)
assertEquals "$Y" "$X"
}
testBasicStdInWithFilesReverse() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq test2.yml -)
Y=$(./yq e '.' test2.yml test.yml)
assertEquals "$Y" "$X"
}
testBasicEvalAllStdInWithFilesReverse() {
./yq -n ".a = 123" > test.yml
./yq -n ".a = 124" > test2.yml
X=$(cat test.yml | ./yq ea test2.yml -)
Y=$(./yq e '.' test2.yml test.yml)
assertEquals "$Y" "$X"
}
testBasicEvalRoundTripNoEval() {
./yq -n ".a = 123" > test.yml
X=$(./yq '.a' test.yml)
assertEquals 123 "$X"
}
testBasicStdInWithOneArg() {
./yq e -n ".a = 123" > test.yml
X=$(cat test.yml | ./yq e ".a")
assertEquals 123 "$X"
X=$(cat test.yml | ./yq ea ".a")
assertEquals 123 "$X"
X=$(cat test.yml | ./yq ".a")
assertEquals 123 "$X"
}
testBasicUpdateInPlaceSequence() {
cat >test.yml <<EOL
a: 0
EOL
./yq e -i ".a = 10" test.yml
X=$(./yq e '.a' test.yml)
assertEquals "10" "$X"
}
testBasicUpdateInPlaceSequenceNoEval() {
cat >test.yml <<EOL
a: 0
EOL
./yq -i ".a = 10" test.yml
X=$(./yq '.a' test.yml)
assertEquals "10" "$X"
}
testBasicUpdateInPlaceSequenceEvalAll() {
cat >test.yml <<EOL
a: 0
EOL
./yq ea -i ".a = 10" test.yml
X=$(./yq e '.a' test.yml)
assertEquals "10" "$X"
}
testBasicUpdateInPlaceMultipleFilesNoExpressionEval() {
cat >test.yml <<EOL
a: 0
EOL
cat >test2.yml <<EOL
a: 1
EOL
read -r -d '' expected << EOM
0
---
1
EOM
./yq -i test.yml test2.yml
X=$(./yq e '.a' test.yml)
assertEquals "$expected" "$X"
}
testBasicUpdateInPlaceMultipleFilesNoExpressionEvalAll() {
cat >test.yml <<EOL
a: 0
EOL
cat >test2.yml <<EOL
a: 1
EOL
read -r -d '' expected << EOM
0
---
1
EOM
./yq -i ea test.yml test2.yml
X=$(./yq e '.a' test.yml)
assertEquals "$expected" "$X"
}
testBasicNoExitStatus() {
echo "a: cat" > test.yml
X=$(./yq e '.z' test.yml)
assertEquals "null" "$X"
}
testBasicExitStatus() {
echo "a: cat" > test.yml
X=$(./yq e -e '.z' test.yml 2&>/dev/null)
assertEquals 1 "$?"
}
testBasicExitStatusNoEval() {
echo "a: cat" > test.yml
X=$(./yq -e '.z' test.yml 2&>/dev/null)
assertEquals 1 "$?"
}
testBasicExtractFieldWithSeparator() {
cat >test.yml <<EOL
---
name: chart-name
version: 1.2.3
EOL
X=$(./yq e '.name' test.yml)
assertEquals "chart-name" "$X"
}
testBasicExtractMultipleFieldWithSeparator() {
cat >test.yml <<EOL
---
name: chart-name
version: 1.2.3
---
name: thing
version: 1.2.3
EOL
read -r -d '' expected << EOM
chart-name
---
thing
EOM
X=$(./yq e '.name' test.yml)
assertEquals "$expected" "$X"
}
testBasicMultiplyAssignMultiDoc() {
cat >test.yml <<EOL
a: 1
---
b: 2
EOL
read -r -d '' expected << EOM
a: 1
c: 3
---
b: 2
c: 3
EOM
X=$(./yq '. *= {"c":3}' test.yml)
assertEquals "$expected" "$X"
}
testBasicClosedStdIn() {
cat >test.yml <<EOL
a: 1
EOL
X=$(./yq e '.a' test.yml <&-)
assertEquals "1" "$X"
}
source ./scripts/shunit2
-9
View File
@@ -1,9 +0,0 @@
#!/bin/bash
testCompletionRuns() {
result=$(./yq __complete "" 2>&1)
assertEquals 0 $?
assertContains "$result" "Completion ended with directive:"
}
source ./scripts/shunit2
-83
View File
@@ -1,83 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
cat >test.yml <<EOL
# comment
EOL
}
testEmptyEval() {
X=$(./yq e test.yml)
expected="# comment"
assertEquals 0 $?
assertEquals "$expected" "$X"
}
testEmptyEvalNoNewLine() {
echo -n "#comment" >test.yml
X=$(./yq e test.yml)
expected=$(cat test.yml)
assertEquals 0 $?
assertEquals "$expected" "$X"
}
testEmptyEvalNoNewLineWithExpression() {
echo -n "# comment" >test.yml
X=$(./yq e '.apple = "tree"' test.yml)
read -r -d '' expected << EOM
# comment
apple: tree
EOM
assertEquals "$expected" "$X"
}
testEmptyEvalPipe() {
X=$(./yq e - < test.yml)
assertEquals 0 $?
}
testEmptyCommentsWithExpressionEval() {
read -r -d '' expected << EOM
# comment
apple: tree
EOM
X=$(./yq e '.apple="tree"' test.yml)
assertEquals "$expected" "$X"
}
testEmptyCommentsWithExpressionEvalAll() {
read -r -d '' expected << EOM
# comment
apple: tree
EOM
X=$(./yq ea '.apple="tree"' test.yml)
assertEquals "$expected" "$X"
}
testEmptyWithExpressionEval() {
rm test.yml
touch test.yml
expected="apple: tree"
X=$(./yq e '.apple="tree"' test.yml)
assertEquals "$expected" "$X"
}
testEmptyWithExpressionEvalAll() {
rm test.yml
touch test.yml
expected="apple: tree"
X=$(./yq ea '.apple="tree"' test.yml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-21
View File
@@ -1,21 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
cat >test.yml <<EOL
# comment
EOL
}
testStringInterpolation() {
X=$(./yq -n '"Mike \(3 + 4)"')
assertEquals "Mike 7" "$X"
}
testNoStringInterpolation() {
X=$(./yq --string-interpolation=f -n '"Mike \(3 + 4)"')
assertEquals "Mike \(3 + 4)" "$X"
}
source ./scripts/shunit2
-76
View File
@@ -1,76 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
cat >test.yml <<EOL
---
a: apple
b: cat
---
not yaml
c: at
EOL
}
testFrontMatterProcessEval() {
read -r -d '' expected << EOM
---
a: apple
b: dog
---
not yaml
c: at
EOM
./yq e --front-matter="process" '.b = "dog"' test.yml -i
assertEquals "$expected" "$(cat test.yml)"
}
testFrontMatterProcessEvalAll() {
read -r -d '' expected << EOM
---
a: apple
b: dog
---
not yaml
c: at
EOM
./yq ea --front-matter="process" '.b = "dog"' test.yml -i
assertEquals "$expected" "$(cat test.yml)"
}
testFrontMatterExtractEval() {
cat >test.yml <<EOL
a: apple
b: cat
---
not yaml
c: at
EOL
read -r -d '' expected << EOM
a: apple
b: dog
EOM
./yq e --front-matter="extract" '.b = "dog"' test.yml -i
assertEquals "$expected" "$(cat test.yml)"
}
testFrontMatterExtractEvalAll() {
cat >test.yml <<EOL
a: apple
b: cat
---
not yaml
c: at
EOL
read -r -d '' expected << EOM
a: apple
b: dog
EOM
./yq ea --front-matter="extract" '.b = "dog"' test.yml -i
assertEquals "$expected" "$(cat test.yml)"
}
source ./scripts/shunit2
-43
View File
@@ -1,43 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
}
testLineCountFirstLineComment() {
cat >test.yml <<EOL
#test123
abc: 123
test123: 123123
#comment
lalilu: lalilu
EOL
X=$(./yq '.lalilu | line' --header-preprocess=false < test.yml)
assertEquals "5" "$X"
}
testArrayOfDocs() {
cat >test.yml <<EOL
---
# leading comment doc 1
a: 1
---
# leading comment doc 2
a: 2
EOL
read -r -d '' expected << EOM
- # leading comment doc 1
a: 1
- # leading comment doc 2
a: 2
EOM
X=$(./yq ea '[.]' --header-preprocess=false < test.yml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-249
View File
@@ -1,249 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml 2>/dev/null || true
rm test*.toml 2>/dev/null || true
rm test*.tfstate 2>/dev/null || true
rm test*.json 2>/dev/null || true
rm test*.properties 2>/dev/null || true
rm test*.csv 2>/dev/null || true
rm test*.tsv 2>/dev/null || true
rm test*.xml 2>/dev/null || true
}
testInputJson() {
cat >test.json <<EOL
{ "mike" : { "things": "cool" } }
EOL
read -r -d '' expected << EOM
{
"mike": {
"things": "cool"
}
}
EOM
X=$(./yq test.json)
assertEquals "$expected" "$X"
X=$(./yq ea test.json)
assertEquals "$expected" "$X"
}
testInputToml() {
cat >test.toml <<EOL
[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00
EOL
read -r -d '' expected << EOM
owner:
name: Tom Preston-Werner
dob: 1979-05-27T07:32:00-08:00
EOM
X=$(./yq -oy test.toml)
assertEquals "$expected" "$X"
X=$(./yq ea -oy test.toml)
assertEquals "$expected" "$X"
}
testInputTfstate() {
cat >test.tfstate <<EOL
{ "mike" : { "things": "cool" } }
EOL
read -r -d '' expected << EOM
{"mike": {"things": "cool"}}
EOM
X=$(./yq test.tfstate)
assertEquals "$expected" "$X"
X=$(./yq ea test.tfstate)
assertEquals "$expected" "$X"
}
testInputJsonOutputYaml() {
cat >test.json <<EOL
{ "mike" : { "things": "cool" } }
EOL
read -r -d '' expected << EOM
mike:
things: cool
EOM
X=$(./yq test.json -oy)
assertEquals "$expected" "$X"
X=$(./yq ea test.json -oy)
assertEquals "$expected" "$X"
}
testInputProperties() {
cat >test.properties <<EOL
mike.things = hello
EOL
read -r -d '' expected << EOM
mike.things = hello
EOM
X=$(./yq e test.properties)
assertEquals "$expected" "$X"
X=$(./yq test.properties)
assertEquals "$expected" "$X"
X=$(./yq ea test.properties)
assertEquals "$expected" "$X"
}
testInputPropertiesGitHubAction() {
cat >test.properties <<EOL
mike.things = hello
EOL
read -r -d '' expected << EOM
mike.things = hello
EOM
X=$(cat /dev/null | ./yq e test.properties)
assertEquals "$expected" "$X"
X=$(cat /dev/null | ./yq ea test.properties)
assertEquals "$expected" "$X"
}
testInputCSV() {
cat >test.csv <<EOL
fruit,yumLevel
apple,5
banana,4
EOL
read -r -d '' expected << EOM
fruit,yumLevel
apple,5
banana,4
EOM
X=$(./yq e test.csv)
assertEquals "$expected" "$X"
X=$(./yq ea test.csv)
assertEquals "$expected" "$X"
}
testInputCSVUTF8() {
read -r -d '' expected << EOM
id,first,last
1,john,smith
1,jane,smith
EOM
X=$(./yq utf8.csv)
assertEquals "$expected" "$X"
}
testInputTSV() {
cat >test.tsv <<EOL
fruit yumLevel
apple 5
banana 4
EOL
read -r -d '' expected << EOM
fruit yumLevel
apple 5
banana 4
EOM
X=$(./yq e test.tsv)
assertEquals "$expected" "$X"
X=$(./yq ea test.tsv)
assertEquals "$expected" "$X"
}
testInputXml() {
cat >test.xml <<EOL
<cat legs="4">BiBi</cat>
EOL
read -r -d '' expected << EOM
<cat legs="4">BiBi</cat>
EOM
X=$(./yq e test.xml)
assertEquals "$expected" "$X"
X=$(./yq ea test.xml)
assertEquals "$expected" "$X"
}
testInputXmlNamespaces() {
cat >test.xml <<EOL
<?xml version="1.0"?>
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">
</map>
EOL
read -r -d '' expected << EOM
<?xml version="1.0"?>
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url"></map>
EOM
X=$(./yq e test.xml)
assertEquals "$expected" "$X"
X=$(./yq ea test.xml)
assertEquals "$expected" "$X"
}
testInputXmlStrict() {
cat >test.xml <<EOL
<?xml version="1.0"?>
<!DOCTYPE root [
<!ENTITY writer "Catherine.">
<!ENTITY copyright "(r) Great">
]>
<root>
<item>&writer;&copyright;</item>
</root>
EOL
X=$(./yq --xml-strict-mode test.xml 2>&1)
assertEquals 1 $?
assertEquals "Error: bad file 'test.xml': XML syntax error on line 7: invalid character entity &writer;" "$X"
X=$(./yq ea --xml-strict-mode test.xml 2>&1)
assertEquals "Error: bad file 'test.xml': XML syntax error on line 7: invalid character entity &writer;" "$X"
}
testInputXmlGithubAction() {
cat >test.xml <<EOL
<cat legs="4">BiBi</cat>
EOL
read -r -d '' expected << EOM
<cat legs="4">BiBi</cat>
EOM
X=$(cat /dev/null | ./yq e test.xml)
assertEquals "$expected" "$X"
X=$(cat /dev/null | ./yq ea test.xml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-258
View File
@@ -1,258 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml 2>/dev/null || true
rm test*.properties 2>/dev/null || true
rm test*.csv 2>/dev/null || true
rm test*.tsv 2>/dev/null || true
rm test*.xml 2>/dev/null || true
}
testInputProperties() {
cat >test.properties <<EOL
mike.things = hello
EOL
read -r -d '' expected << EOM
mike:
things: hello
EOM
X=$(./yq e -p=props test.properties)
assertEquals "$expected" "$X"
X=$(./yq ea -p=props test.properties)
assertEquals "$expected" "$X"
}
testInputPropertiesGitHubAction() {
cat >test.properties <<EOL
mike.things = hello
EOL
read -r -d '' expected << EOM
mike:
things: hello
EOM
X=$(cat /dev/null | ./yq e -p=props test.properties)
assertEquals "$expected" "$X"
X=$(cat /dev/null | ./yq ea -p=props test.properties)
assertEquals "$expected" "$X"
}
testInputCSV() {
cat >test.csv <<EOL
fruit,yumLevel
apple,5
banana,4
EOL
read -r -d '' expected << EOM
- fruit: apple
yumLevel: 5
- fruit: banana
yumLevel: 4
EOM
X=$(./yq e -p=csv test.csv)
assertEquals "$expected" "$X"
X=$(./yq ea -p=csv test.csv)
assertEquals "$expected" "$X"
}
testInputCSVCustomSeparator() {
cat >test.csv <<EOL
fruit;yumLevel
apple;5
banana;4
EOL
read -r -d '' expected << EOM
- fruit: apple
yumLevel: 5
- fruit: banana
yumLevel: 4
EOM
X=$(./yq -p=csv --csv-separator ";" test.csv)
assertEquals "$expected" "$X"
X=$(./yq ea -p=csv --csv-separator ";" test.csv)
assertEquals "$expected" "$X"
}
testInputCSVNoAuto() {
cat >test.csv <<EOL
thing1
name: cat
EOL
read -r -d '' expected << EOM
- thing1: 'name: cat'
EOM
X=$(./yq --csv-auto-parse=f test.csv -oy)
assertEquals "$expected" "$X"
X=$(./yq ea --csv-auto-parse=f test.csv -oy)
assertEquals "$expected" "$X"
}
testInputTSVNoAuto() {
cat >test.tsv <<EOL
thing1
name: cat
EOL
read -r -d '' expected << EOM
- thing1: 'name: cat'
EOM
X=$(./yq --tsv-auto-parse=f test.tsv -oy)
assertEquals "$expected" "$X"
X=$(./yq ea --tsv-auto-parse=f test.tsv -oy)
assertEquals "$expected" "$X"
}
testInputCSVUTF8() {
read -r -d '' expected << EOM
- id: 1
first: john
last: smith
- id: 1
first: jane
last: smith
EOM
X=$(./yq -p=csv utf8.csv)
assertEquals "$expected" "$X"
}
testInputTSV() {
cat >test.tsv <<EOL
fruit yumLevel
apple 5
banana 4
EOL
read -r -d '' expected << EOM
- fruit: apple
yumLevel: 5
- fruit: banana
yumLevel: 4
EOM
X=$(./yq e -p=t test.tsv)
assertEquals "$expected" "$X"
X=$(./yq ea -p=t test.tsv)
assertEquals "$expected" "$X"
}
testInputXml() {
cat >test.yml <<EOL
<cat legs="4">BiBi</cat>
EOL
read -r -d '' expected << EOM
cat:
+content: BiBi
+@legs: "4"
EOM
X=$(./yq e -p=xml test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -p=xml test.yml)
assertEquals "$expected" "$X"
}
testInputXmlNamespaces() {
cat >test.xml <<EOL
<?xml version="1.0"?>
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">
</map>
EOL
read -r -d '' expected << EOM
+p_xml: version="1.0"
map:
+@xmlns: some-namespace
+@xmlns:xsi: some-instance
+@xsi:schemaLocation: some-url
EOM
X=$(./yq e -p=xml test.xml)
assertEquals "$expected" "$X"
X=$(./yq ea -p=xml test.xml)
assertEquals "$expected" "$X"
}
testInputXmlRoundtrip() {
cat >test.yml <<EOL
<?xml version="1.0"?>
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">Meow</map>
EOL
read -r -d '' expected << EOM
<?xml version="1.0"?>
<!DOCTYPE config SYSTEM "/etc/iwatch/iwatch.dtd" >
<map xmlns="some-namespace" xmlns:xsi="some-instance" xsi:schemaLocation="some-url">Meow</map>
EOM
X=$(./yq -p=xml -o=xml test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -p=xml -o=xml test.yml)
assertEquals "$expected" "$X"
}
testInputXmlStrict() {
cat >test.yml <<EOL
<?xml version="1.0"?>
<!DOCTYPE root [
<!ENTITY writer "Catherine.">
<!ENTITY copyright "(r) Great">
]>
<root>
<item>&writer;&copyright;</item>
</root>
EOL
X=$(./yq -p=xml --xml-strict-mode test.yml -o=xml 2>&1)
assertEquals 1 $?
assertEquals "Error: bad file 'test.yml': XML syntax error on line 7: invalid character entity &writer;" "$X"
X=$(./yq ea -p=xml --xml-strict-mode test.yml -o=xml 2>&1)
assertEquals "Error: bad file 'test.yml': XML syntax error on line 7: invalid character entity &writer;" "$X"
}
testInputXmlGithubAction() {
cat >test.yml <<EOL
<cat legs="4">BiBi</cat>
EOL
read -r -d '' expected << EOM
cat:
+content: BiBi
+@legs: "4"
EOM
X=$(cat /dev/null | ./yq e -p=xml test.yml)
assertEquals "$expected" "$X"
X=$(cat /dev/null | ./yq ea -p=xml test.yml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-529
View File
@@ -1,529 +0,0 @@
#!/bin/bash
# examples where header-preprocess is required
setUp() {
rm test*.yml || true
cat >test.yml <<EOL
---
a: test
EOL
}
testLeadingSeparatorWithDoc() {
cat >test.yml <<EOL
# hi peeps
# cool
---
a: test
---
b: cool
EOL
read -r -d '' expected << EOM
# hi peeps
# cool
---
a: thing
---
b: cool
EOM
X=$(./yq e '(select(di == 0) | .a) = "thing"' - < test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorWithNewlinesNewDoc() {
cat >test.yml <<EOL
# hi peeps
# cool
---
a: test
---
b: cool
EOL
read -r -d '' expected << EOM
# hi peeps
# cool
---
a: thing
---
b: cool
EOM
X=$(./yq e '(select(di == 0) | .a) = "thing"' - < test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorWithNewlinesMoreComments() {
cat >test.yml <<EOL
# hi peeps
# cool
---
# great
a: test
---
b: cool
EOL
read -r -d '' expected << EOM
# hi peeps
# cool
---
# great
a: thing
---
b: cool
EOM
X=$(./yq e '(select(di == 0) | .a) = "thing"' - < test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorWithDirective() {
cat >test.yml <<EOL
%YAML 1.1
---
this: should really work
EOL
read -r -d '' expected << EOM
%YAML 1.1
---
this: should really work
EOM
X=$(./yq < test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorPipeIntoEvalSeq() {
X=$(./yq e - < test.yml)
expected=$(cat test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorExtractField() {
X=$(./yq e '.a' - < test.yml)
assertEquals "test" "$X"
}
testLeadingSeparatorExtractFieldWithCommentsAfterSep() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
EOL
X=$(./yq e '.a' test.yml)
assertEquals "test" "$X"
}
testLeadingSeparatorExtractFieldWithCommentsBeforeSep() {
cat >test.yml <<EOL
# hi peeps
# cool
---
a: test
EOL
X=$(./yq e '.a' test.yml)
assertEquals "test" "$X"
}
testLeadingSeparatorExtractFieldMultiDoc() {
cat >test.yml <<EOL
---
a: test
---
a: test2
EOL
read -r -d '' expected << EOM
test
---
test2
EOM
X=$(./yq e '.a' test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorExtractFieldMultiDocWithComments() {
cat >test.yml <<EOL
# here
---
# there
a: test
# wherever
---
# you are
a: test2
# woop
EOL
read -r -d '' expected << EOM
test
---
test2
EOM
X=$(./yq e '.a' test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorEvalSeq() {
X=$(./yq e test.yml)
expected=$(cat test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorPipeIntoEvalAll() {
X=$(./yq ea - < test.yml)
expected=$(cat test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorEvalAll() {
X=$(./yq ea test.yml)
expected=$(cat test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalSimple() {
read -r -d '' expected << EOM
---
a: test
---
version: 3
application: MyApp
EOM
X=$(./yq e '.' test.yml examples/order.yaml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocInOneFile() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
---
b: things
EOL
expected=$(cat test.yml)
X=$(./yq e '.' test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocInOneFileEvalAll() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
---
b: things
EOL
expected=$(cat test.yml)
X=$(./yq ea '.' test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalComments() {
cat >test.yml <<EOL
# hi peeps
# cool
a: test
EOL
cat >test2.yml <<EOL
# this is another doc
# great
b: sane
EOL
read -r -d '' expected << EOM
# hi peeps
# cool
a: test
---
# this is another doc
# great
b: sane
EOM
X=$(./yq e '.' test.yml test2.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalCommentsTrailingSep() {
cat >test.yml <<EOL
# hi peeps
# cool
---
a: test
EOL
cat >test2.yml <<EOL
# this is another doc
# great
---
b: sane
EOL
read -r -d '' expected << EOM
# hi peeps
# cool
---
a: test
---
# this is another doc
# great
---
b: sane
EOM
X=$(./yq e '.' test.yml test2.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiMultiDocEvalCommentsTrailingSep() {
cat >test.yml <<EOL
# hi peeps
# cool
---
a: test
---
a1: test2
EOL
cat >test2.yml <<EOL
# this is another doc
# great
---
b: sane
---
b2: cool
EOL
read -r -d '' expected << EOM
# hi peeps
# cool
---
a: test
---
a1: test2
---
# this is another doc
# great
---
b: sane
---
b2: cool
EOM
X=$(./yq e '.' test.yml test2.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalCommentsLeadingSep() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
EOL
cat >test2.yml <<EOL
---
# this is another doc
# great
b: sane
EOL
read -r -d '' expected << EOM
---
# hi peeps
# cool
a: test
---
# this is another doc
# great
b: sane
EOM
X=$(./yq e '.' test.yml test2.yml)
assertEquals "$expected" "$X"
}
# https://github.com/mikefarah/yq/issues/919
testLeadingSeparatorDoesNotBreakCommentsOnOtherFiles() {
cat >test.yml <<EOL
# a1
a: 1
# a2
EOL
cat >test2.yml <<EOL
# b1
b: 2
# b2
EOL
read -r -d '' expected << EOM
# a1
a: 1
# a2
# b1
b: 2
# b2
EOM
X=$(./yq ea 'select(fi == 0) * select(fi == 1)' test.yml test2.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalCommentsStripComments() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
---
# this is another doc
# great
b: sane
EOL
# it will be hard to remove that top level separator
read -r -d '' expected << EOM
a: test
---
b: sane
EOM
X=$(./yq e '... comments=""' test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalCommentsLeadingSepNoDocFlag() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
---
# this is another doc
# great
b: sane
EOL
read -r -d '' expected << EOM
# hi peeps
# cool
a: test
# this is another doc
# great
b: sane
EOM
X=$(./yq e '.' --no-doc test.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalJsonFlag() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
EOL
cat >test2.yml <<EOL
---
# this is another doc
# great
b: sane
EOL
read -r -d '' expected << EOM
{
"a": "test"
}
{
"b": "sane"
}
EOM
X=$(./yq e '.' -j test.yml test2.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalAllJsonFlag() {
cat >test.yml <<EOL
---
# hi peeps
# cool
a: test
EOL
cat >test2.yml <<EOL
---
# this is another doc
# great
b: sane
EOL
read -r -d '' expected << EOM
{
"a": "test"
}
{
"b": "sane"
}
EOM
X=$(./yq ea '.' -j test.yml test2.yml)
assertEquals "$expected" "$X"
}
testLeadingSeparatorMultiDocEvalAll() {
read -r -d '' expected << EOM
---
a: test
---
version: 3
application: MyApp
EOM
X=$(./yq ea '.' test.yml examples/order.yaml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-27
View File
@@ -1,27 +0,0 @@
#!/bin/bash
testLoadFileNotExist() {
result=$(./yq e -n 'load("cat.yml")' 2>&1)
assertEquals 1 $?
assertEquals "Error: failed to load cat.yml: open cat.yml: no such file or directory" "$result"
}
testLoadFileExpNotExist() {
result=$(./yq e -n 'load(.a)' 2>&1)
assertEquals 1 $?
assertEquals "Error: filename expression returned nil" "$result"
}
testStrLoadFileNotExist() {
result=$(./yq e -n 'strload("cat.yml")' 2>&1)
assertEquals 1 $?
assertEquals "Error: failed to load cat.yml: open cat.yml: no such file or directory" "$result"
}
testStrLoadFileExpNotExist() {
result=$(./yq e -n 'strload(.a)' 2>&1)
assertEquals 1 $?
assertEquals "Error: filename expression returned nil" "$result"
}
source ./scripts/shunit2
-286
View File
@@ -1,286 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
}
## Convenient bash shortcut to read records of NUL separated values
## from stdin the safe way. See example usage in the next tests.
read-0() {
local eof="" IFS=''
while [ "$1" ]; do
## - The `-r` avoids bad surprise with '\n' and other interpreted
## sequences that can be read.
## - The `-d ''` is the (strange?) way to refer to NUL delimiter.
## - The `--` is how to avoid unpleasant surprises if your
## "$1" starts with "-" (minus) sign. This protection also
## will produce a readable error if you want to try to start
## your variable names with a "-".
read -r -d '' -- "$1" || eof=1
shift
done
[ -z "$eof" ] ## fail on EOF
}
## Convenient bash shortcut to be used with the next function `p-err`
## to read NUL separated values the safe way AND catch any errors from
## the process creating the stream of NUL separated data. See example
## usage in the tests.
read-0-err() {
local ret="$1" eof="" idx=0 last=
read -r -- "${ret?}" <<<"0"
shift
while [ "$1" ]; do
last=$idx
read -r -d '' -- "$1" || {
## Put this last value in ${!ret}
eof="$1"
read -r -- "$ret" <<<"${!eof}"
break
}
((idx++))
shift
done
[ -z "$eof" ] || {
if [ "$last" != 0 ]; then
## Uhoh, we have no idea if the errorlevel of the internal
## command was properly delimited with a NUL char, and
## anyway something went really wrong at least about the
## number of fields separated by NUL char and the one
## expected.
echo "Error: read-0-err couldn't fill all value $ret = '${!ret}', '$eof', '${!eof}'" >&2
read -r -- "$ret" <<<"not-enough-values"
else
if ! [[ "${!ret}" =~ ^[0-9]+$ && "${!ret}" -ge 0 && "${!ret}" -le 127 ]]; then
## This could happen if you don't use `p-err` wrapper,
## or used stdout in unexpected ways in your inner
## command.
echo "Error: last value is not a number, did you finish with an errorlevel ?" >&2
read -r -- "$ret" <<<"last-value-not-a-number"
fi
fi
false
}
}
## Simply runs command given as argument and adds errorlevel in the
## standard output. Is expected to be used in tandem with
## `read-0-err`.
p-err() {
local exp="$1"
"$@"
printf "%s" "$?"
}
wyq-r() {
local exp="$1"
./yq e -0 -r=false "$1"
printf "%s" "$?"
}
testBasicUsageRaw() {
cat >test.yml <<EOL
a: foo
b: bar
EOL
printf "foo\0bar\0" > expected.out
## We need to compare binary content here. We have to filter the compared
## content through a representation that gets rid of NUL chars but accurately
## transcribe the content.
## Also as it would be nice to have a pretty output in case the test fails,
## we use here 'hd': a widely available shortcut to 'hexdump' that will
## pretty-print any binary to it's hexadecimal representation.
##
## Note that the standard `assertEquals` compare its arguments
## value, but they can't hold NUL characters (this comes from the
## limitation of the C API of `exec*(..)` functions that requires
## `const char *arv[]`). And these are NUL terminated strings. As a
## consequence, the NUL characters gets removed in bash arguments.
assertEquals "$(hd expected.out)" \
"$(./yq e -0 '.a, .b' test.yml | hd)"
rm expected.out
}
testBasicUsage() {
local a b
cat >test.yml <<EOL
a: foo
b: bar
EOL
## We provide 2 values, and ask to fill 2 variables.
read-0 a b < <(./yq e -0 '.a, .b' test.yml)
assertEquals "$?" "0" ## Everything is fine
assertEquals "foo" "$a" ## Values are correctly parsed
assertEquals "bar" "$b"
a=YYY ; b=XXX
## Not enough values provided to fill `a` and `b`.
read-0 a b < <(./yq e -0 '.a' test.yml)
assertEquals "$?" "1" ## An error was emitted
assertEquals "foo" "$a" ## First value was correctly parsed
assertEquals "" "$b" ## Second was still reset
## Error from inner command are not catchable !. Use
## `read-0-err`/`p-err` for that.
read-0 a < <(printf "\0"; ./yq e -0 'xxx' test.yml; )
assertEquals "$?" "0"
}
testBasicUsageJson() {
cat >test.yml <<EOL
a:
x: foo
b: bar
EOL
read-0 a b < <(./yq e -0 -o=json '.a, .b' test.yml)
assertEquals '{
"x": "foo"
}' "$a"
assertEquals '"bar"' "$b"
}
testFailWithValueContainingNUL() {
local a b c
## Note that value of field 'a' actually contains a NUL char !
cat >test.yml <<EOL
a: "foo\u0000bar"
b: 1
c: |
wiz
boom
EOL
## We are looking for trouble with asking to separated fields with NUL
## char and requested value `.a` actually contains itself a NUL char !
read-0 a b c < <(./yq e -0 '.a, .b, .c' test.yml)
assertNotEquals "0" "$?" ## read-0 failed to fill all values
## But here, we can request for one value, even if `./yq` fails
read-0 b < <(./yq e -0 '.b, .a' test.yml)
assertEquals "0" "$?" ## read-0 succeeds at feeding the first value
## Note: to catch the failure of `yq`, see in the next tests the usage
## of `read-0-err`.
## using -r=false solves any NUL containing value issues, but keeps
## all in YAML representation:
read-0 a b c < <(./yq e -0 -r=false '.a, .b, .c' test.yml)
assertEquals "0" "$?" ## All goes well despite asking for `a` value
assertEquals '"foo\0bar"' "$a" ## This is a YAML string representation
assertEquals '1' "$b"
assertEquals '|
wiz
boom' "$c"
}
testStandardLoop() {
local E a b res
## Here everything is normal: 4 values, that will be paired
## in key/values.
cat >test.yml <<EOL
- yay
- wiz
- hop
- pow
EOL
res=""
while read-0-err E a b; do
res+="$a: $b;"
done < <(p-err ./yq -0 '.[]' test.yml)
assertEquals "0" "$E" ## errorlevel of internal command
assertEquals "yay: wiz;hop: pow;" "$res" ## expected result
}
testStandardLoopWithoutEnoughValues() {
local E a b res
## Here 5 values, there will be a missing value when reading
## pairs of value.
cat >test.yml <<EOL
- yay
- wiz
- hop
- pow
- kwak
EOL
res=""
## The loop will succeed 2 times then fail
while read-0-err E a b; do
res+="$a: $b;"
done < <(p-err ./yq -0 '.[]' test.yml)
assertEquals "not-enough-values" "$E" ## Not enough value error
assertEquals "yay: wiz;hop: pow;" "$res" ## the 2 full key/value pairs
}
testStandardLoopWithInternalCmdError() {
local E a b res
## Note the third value contains a NUL char !
cat >test.yml <<EOL
- yay
- wiz
- "foo\0bar"
- hop
- pow
EOL
res=""
## It should be only upon the second pass in the loop that
## read-0-err will catch the fact that there is an error !
while read-0-err E a b; do
res+="$a: $b;"
done < <(p-err ./yq -0 '.[]' test.yml)
assertEquals "1" "$E" ## Internal command errorlevel (from `./yq`)
assertEquals "yay: wiz;" "$res" ## first 2 values were ok at least
}
testStandardLoopNotEnoughErrorEatsCmdError() {
local E a b res
## Because of possible edge cases where the internal errorlevel
## reported by `p-err` in the standard output might be mangled
## with the unfinished record, `read-0-err E ...` will NOT report
## the internal command error in the variable E and instead will
## store the value 'not-enough-values'. In real world, anyway, you
## will want to react the same if the internal command failed
## and/or you didn't get as much values as expected while
## reading. Keep in mind also that standard error is not
## swallowed, so you can read reports from the inner command AND
## from `read-0-err`.
## Here, note that the fourth value contains a NUL char !
cat >test.yml <<EOL
- yay
- wiz
- hop
- "foo\0bar"
- pow
EOL
res=""
## It should be only upon the second loop that read-0-err will catch
## the fact that there are not enough data to fill the requested variables
while read-0-err E a b; do
res+="$a: $b;"
done < <(p-err ./yq -0 '.[]' test.yml)
assertEquals "not-enough-values" "$E" ## Not enough values error eats internal error !
assertEquals "yay: wiz;" "$res" ## first 2 values were ok at least
}
source ./scripts/shunit2
-343
View File
@@ -1,343 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
}
testOutputJsonDeprecated() {
cat >test.yml <<EOL
a: {b: ["cat"]}
EOL
read -r -d '' expected << EOM
{
"a": {
"b": [
"cat"
]
}
}
EOM
X=$(./yq e -j test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -j test.yml)
assertEquals "$expected" "$X"
}
testOutputJson() {
cat >test.yml <<EOL
a: {b: ["cat"]}
EOL
read -r -d '' expected << EOM
{
"a": {
"b": [
"cat"
]
}
}
EOM
X=$(./yq e --output-format=json test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --output-format=json test.yml)
assertEquals "$expected" "$X"
}
testOutputYamlRawDefault() {
cat >test.yml <<EOL
a: "cat"
EOL
X=$(./yq e '.a' test.yml)
assertEquals "cat" "$X"
X=$(./yq ea '.a' test.yml)
assertEquals "cat" "$X"
}
testOutputYamlRawOff() {
cat >test.yml <<EOL
a: "cat"
EOL
X=$(./yq e -r=false '.a' test.yml)
assertEquals "\"cat\"" "$X"
X=$(./yq ea -r=false '.a' test.yml)
assertEquals "\"cat\"" "$X"
}
testOutputYamlRawOnRoot() {
cat >test.yml <<EOL
'a'
EOL
X=$(./yq e -r '.' test.yml)
assertEquals "a" "$X"
}
testOutputJsonRaw() {
cat >test.yml <<EOL
a: cat
EOL
X=$(./yq e -r --output-format=json '.a' test.yml)
assertEquals "cat" "$X"
X=$(./yq ea -r --output-format=json '.a' test.yml)
assertEquals "cat" "$X"
}
testOutputJsonDefault() {
cat >test.yml <<EOL
a: cat
EOL
X=$(./yq e --output-format=json '.a' test.yml)
assertEquals "\"cat\"" "$X"
X=$(./yq ea --output-format=json '.a' test.yml)
assertEquals "\"cat\"" "$X"
}
testOutputJsonShort() {
cat >test.yml <<EOL
a: {b: ["cat"]}
EOL
read -r -d '' expected << EOM
{
"a": {
"b": [
"cat"
]
}
}
EOM
X=$(./yq e -o=j test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -o=j test.yml)
assertEquals "$expected" "$X"
}
testOutputProperties() {
cat >test.yml <<EOL
a: {b: {c: ["cat cat"]}}
EOL
read -r -d '' expected << EOM
a.b.c.0 = cat cat
EOM
X=$(./yq e --output-format=props test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --output-format=props test.yml)
assertEquals "$expected" "$X"
}
testOutputPropertiesDontUnwrap() {
cat >test.yml <<EOL
a: {b: {c: ["cat cat"]}}
EOL
read -r -d '' expected << EOM
a.b.c.0 = "cat cat"
EOM
X=$(./yq e -r=false --output-format=props test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -r=false --output-format=props test.yml)
assertEquals "$expected" "$X"
}
testOutputPropertiesShort() {
cat >test.yml <<EOL
a: {b: {c: ["cat cat"]}}
EOL
read -r -d '' expected << EOM
a.b.c.0 = cat cat
EOM
X=$(./yq e -o=p test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -o=p test.yml)
assertEquals "$expected" "$X"
}
testOutputCSV() {
cat >test.yml <<EOL
- fruit: apple
yumLevel: 5
- fruit: banana
yumLevel: 4
EOL
read -r -d '' expected << EOM
fruit,yumLevel
apple,5
banana,4
EOM
X=$(./yq -o=c test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -o=csv test.yml)
assertEquals "$expected" "$X"
}
testOutputCSVCustomSeparator() {
cat >test.yml <<EOL
- fruit: apple
yumLevel: 5
- fruit: banana
yumLevel: 4
EOL
read -r -d '' expected << EOM
fruit;yumLevel
apple;5
banana;4
EOM
X=$(./yq -oc --csv-separator ";" test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -o=csv --csv-separator ";" test.yml)
assertEquals "$expected" "$X"
}
testOutputTSV() {
cat >test.yml <<EOL
- fruit: apple
yumLevel: 5
- fruit: banana
yumLevel: 4
EOL
read -r -d '' expected << EOM
fruit yumLevel
apple 5
banana 4
EOM
X=$(./yq -o=t test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea -o=tsv test.yml)
assertEquals "$expected" "$X"
}
testOutputXml() {
cat >test.yml <<EOL
a: {b: {c: ["cat"]}}
EOL
read -r -d '' expected << EOM
<a>
<b>
<c>cat</c>
</b>
</a>
EOM
X=$(./yq e --output-format=xml test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --output-format=xml test.yml)
assertEquals "$expected" "$X"
}
testOutputXmlShort() {
cat >test.yml <<EOL
a: {b: {c: ["cat"]}}
EOL
read -r -d '' expected << EOM
<a>
<b>
<c>cat</c>
</b>
</a>
EOM
X=$(./yq e --output-format=x test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --output-format=x test.yml)
assertEquals "$expected" "$X"
}
testOutputXmComplex() {
cat >test.yml <<EOL
a: {b: {c: ["cat", "dog"], +@f: meow}}
EOL
read -r -d '' expected << EOM
<a>
<b f="meow">
<c>cat</c>
<c>dog</c>
</b>
</a>
EOM
X=$(./yq e --output-format=x test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --output-format=x test.yml)
assertEquals "$expected" "$X"
}
testLuaOutputPretty() {
cat >test.yml <<EOL
animals:
cat: meow
EOL
read -r -d '' expected << EOM
return {
["animals"] = {
["cat"] = "meow";
};
};
EOM
X=$(./yq e --output-format=lua test.yml)
assertEquals "$expected" "$X"
X=$(./yq e --output-format=lua --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
testLuaOutputSubset() {
cat >test.yml <<EOL
animals:
cat: meow
EOL
read -r -d '' expected << EOM
return {
["cat"] = "meow";
};
EOM
X=$(./yq e --output-format=lua '.animals' test.yml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-70
View File
@@ -1,70 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
cat >test.yml <<EOL
a: frog
EOL
}
testPipeViaCatWithParam() {
X=$(cat test.yml | ./yq '.a')
assertEquals "frog" "$X"
}
testPipeViaCatWithParamEval() {
X=$(cat test.yml | ./yq e '.a')
assertEquals "frog" "$X"
}
testPipeViaCatWithParamEvalAll() {
X=$(cat test.yml | ./yq ea '.a')
assertEquals "frog" "$X"
}
testPipeViaCatNoParam() {
X=$(cat test.yml | ./yq)
assertEquals "a: frog" "$X"
}
testPipeViaCatNoParamEval() {
X=$(cat test.yml | ./yq e)
assertEquals "a: frog" "$X"
}
testPipeViaCatNoParamEvalAll() {
X=$(cat test.yml | ./yq ea)
assertEquals "a: frog" "$X"
}
testPipeViaFileishWithParam() {
X=$(./yq '.a' < test.yml)
assertEquals "frog" "$X"
}
testPipeViaFileishWithParamEval() {
X=$(./yq e '.a' < test.yml)
assertEquals "frog" "$X"
}
testPipeViaFileishWithParamEvalAll() {
X=$(./yq ea '.a' < test.yml)
assertEquals "frog" "$X"
}
testPipeViaFileishNoParam() {
X=$(./yq < test.yml)
assertEquals "a: frog" "$X"
}
testPipeViaFileishNoParamEval() {
X=$(./yq e < test.yml)
assertEquals "a: frog" "$X"
}
testPipeViaFileishNoParamEvalAll() {
X=$(./yq ea < test.yml)
assertEquals "a: frog" "$X"
}
source ./scripts/shunit2
-176
View File
@@ -1,176 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
}
testPrettyPrintWithBooleans() {
cat >test.yml <<EOL
leaveUnquoted: [yes, no, on, off, y, n, true, false]
leaveQuoted: ["yes", "no", "on", "off", "y", "n", "true", "false"]
EOL
read -r -d '' expected << EOM
leaveUnquoted:
- yes
- no
- on
- off
- y
- n
- true
- false
leaveQuoted:
- "yes"
- "no"
- "on"
- "off"
- "y"
- "n"
- "true"
- "false"
EOM
X=$(./yq e --prettyPrint test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
testPrettyPrintWithBooleansCapitals() {
cat >test.yml <<EOL
leaveUnquoted: [YES, NO, ON, OFF, Y, N, TRUE, FALSE]
leaveQuoted: ["YES", "NO", "ON", "OFF", "Y", "N", "TRUE", "FALSE"]
EOL
read -r -d '' expected << EOM
leaveUnquoted:
- YES
- NO
- ON
- OFF
- Y
- N
- TRUE
- FALSE
leaveQuoted:
- "YES"
- "NO"
- "ON"
- "OFF"
- "Y"
- "N"
- "TRUE"
- "FALSE"
EOM
X=$(./yq e --prettyPrint test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
testPrettyPrintOtherStringValues() {
cat >test.yml <<EOL
leaveUnquoted: [yesSir, hellno, bonapite]
makeUnquoted: ["yesSir", "hellno", "bonapite"]
EOL
read -r -d '' expected << EOM
leaveUnquoted:
- yesSir
- hellno
- bonapite
makeUnquoted:
- yesSir
- hellno
- bonapite
EOM
X=$(./yq e --prettyPrint test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
testPrettyPrintKeys() {
cat >test.yml <<EOL
"removeQuotes": "please"
EOL
read -r -d '' expected << EOM
removeQuotes: please
EOM
X=$(./yq e --prettyPrint test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
testPrettyPrintOtherStringValues() {
cat >test.yml <<EOL
leaveUnquoted: [yesSir, hellno, bonapite]
makeUnquoted: ["yesSir", "hellno", "bonapite"]
EOL
read -r -d '' expected << EOM
leaveUnquoted:
- yesSir
- hellno
- bonapite
makeUnquoted:
- yesSir
- hellno
- bonapite
EOM
X=$(./yq e --prettyPrint test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
testPrettyPrintStringBlocks() {
cat >test.yml <<EOL
"removeQuotes": |
"please"
EOL
read -r -d '' expected << EOM
removeQuotes: |
"please"
EOM
X=$(./yq e --prettyPrint test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
testPrettyPrintWithExpression() {
cat >test.yml <<EOL
a: {b: {c: ["cat"]}}
EOL
read -r -d '' expected << EOM
b:
c:
- cat
EOM
X=$(./yq e '.a' --prettyPrint test.yml)
assertEquals "$expected" "$X"
X=$(./yq ea '.a' --prettyPrint test.yml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-26
View File
@@ -1,26 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yq || true
cat >test.yq <<EOL
#!./yq
.a.b
EOL
chmod +x test.yq
rm test*.yml || true
cat >test.yml <<EOL
a: {b: apple}
EOL
}
testCanExecYqFile() {
read -r -d '' expected << EOM
apple
EOM
X=$(./test.yq test.yml)
assertEquals "$expected" "$X"
}
source ./scripts/shunit2
-227
View File
@@ -1,227 +0,0 @@
#!/bin/bash
setUp() {
rm test*.yml || true
rm -rf test_dir* || true
}
testBasicSplitWithName() {
cat >test.yml <<EOL
a: test_doc1
---
a: test_doc2
EOL
./yq e test.yml -s ".a"
doc1=$(cat test_doc1.yml)
assertEquals "a: test_doc1" "$doc1"
doc2=$(cat test_doc2.yml)
read -r -d '' expectedDoc2 << EOM
---
a: test_doc2
EOM
assertEquals "$expectedDoc2" "$doc2"
}
testBasicSplitWithNameCustomExtension() {
rm test*.yaml || true
cat >test.yml <<EOL
a: test_doc1
---
a: test_doc2
EOL
./yq e test.yml -s '.a + ".yaml"'
doc1=$(cat test_doc1.yaml)
assertEquals "a: test_doc1" "$doc1"
doc2=$(cat test_doc2.yaml)
read -r -d '' expectedDoc2 << EOM
---
a: test_doc2
EOM
assertEquals "$expectedDoc2" "$doc2"
}
testSplitFromFile() {
cat >test.yml <<EOL
a: test_doc1
---
a: test_doc2
EOL
cat >test_splitExp.yml <<EOL
.a
EOL
./yq test.yml --split-exp-file test_splitExp.yml
doc1=$(cat test_doc1.yml)
assertEquals "a: test_doc1" "$doc1"
doc2=$(cat test_doc2.yml)
read -r -d '' expectedDoc2 << EOM
---
a: test_doc2
EOM
assertEquals "$expectedDoc2" "$doc2"
}
testBasicSplitWithNameEvalAll() {
cat >test.yml <<EOL
a: test_doc1
---
a: test_doc2
EOL
./yq ea test.yml -s ".a"
doc1=$(cat test_doc1.yml)
assertEquals "a: test_doc1" "$doc1"
doc2=$(cat test_doc2.yml)
read -r -d '' expectedDoc2 << EOM
---
a: test_doc2
EOM
assertEquals "$expectedDoc2" "$doc2"
}
testBasicSplitWithIndex() {
cat >test.yml <<EOL
a: test_doc1
---
a: test_doc2
EOL
./yq e test.yml -s '"test_" + $index'
doc1=$(cat test_0.yml)
assertEquals "a: test_doc1" "$doc1"
doc2=$(cat test_1.yml)
read -r -d '' expectedDoc2 << EOM
---
a: test_doc2
EOM
assertEquals "$expectedDoc2" "$doc2"
}
testBasicSplitWithIndexEvalAll() {
cat >test.yml <<EOL
a: test_doc1
---
a: test_doc2
EOL
./yq ea test.yml -s '"test_" + $index'
doc1=$(cat test_0.yml)
assertEquals "a: test_doc1" "$doc1"
doc2=$(cat test_1.yml)
read -r -d '' expectedDoc2 << EOM
---
a: test_doc2
EOM
assertEquals "$expectedDoc2" "$doc2"
}
testArraySplitWithNameNoSeparators() {
cat >test.yml <<EOL
- name: test_fred
age: 35
- name: test_catherine
age: 37
EOL
./yq e --no-doc -s ".name" ".[]" test.yml
doc1=$(cat test_fred.yml)
read -r -d '' expectedDoc1 << EOM
name: test_fred
age: 35
EOM
assertEquals "$expectedDoc1" "$doc1"
doc2=$(cat test_catherine.yml)
read -r -d '' expectedDoc2 << EOM
name: test_catherine
age: 37
EOM
assertEquals "$expectedDoc2" "$doc2"
}
testArraySplitWithNameNoSeparatorsEvalAll() {
cat >test.yml <<EOL
- name: test_fred
age: 35
- name: test_catherine
age: 37
EOL
cat >test2.yml <<EOL
- name: test_mike
age: 564
EOL
./yq ea --no-doc -s ".name" ".[]" test.yml test2.yml
doc1=$(cat test_fred.yml)
read -r -d '' expectedDoc1 << EOM
name: test_fred
age: 35
EOM
assertEquals "$expectedDoc1" "$doc1"
doc2=$(cat test_catherine.yml)
read -r -d '' expectedDoc2 << EOM
name: test_catherine
age: 37
EOM
assertEquals "$expectedDoc2" "$doc2"
doc3=$(cat test_mike.yml)
read -r -d '' expectedDoc3 << EOM
name: test_mike
age: 564
EOM
assertEquals "$expectedDoc3" "$doc3"
}
testSplitWithDirectories() {
cat >test.yml <<EOL
f: test_dir1/test_file1
---
f: test_dir2/dir22/test_file2
---
f: test_file3
EOL
./yq e --no-doc -s ".f" test.yml
doc1=$(cat test_dir1/test_file1.yml)
assertEquals "f: test_dir1/test_file1" "$doc1"
doc2=$(cat test_dir2/dir22/test_file2.yml)
assertEquals "f: test_dir2/dir22/test_file2" "$doc2"
doc3=$(cat test_file3.yml)
assertEquals "f: test_file3" "$doc3"
}
source ./scripts/shunit2
-17
View File
@@ -1,17 +0,0 @@
name: 'yq - portable yaml processor'
description: 'create, read, update, delete, merge, validate and do more with yaml'
branding:
icon: command
color: gray-dark
inputs:
cmd:
description: 'The Command which should be run'
required: true
outputs:
result:
description: "The complete result from the yq command being run"
runs:
using: 'docker'
image: 'docker://mikefarah/yq:4-githubaction'
args:
- ${{ inputs.cmd }}
-62
View File
@@ -1,62 +0,0 @@
package cmd
import (
"os"
"github.com/spf13/cobra"
)
var completionCmd = &cobra.Command{
Use: "completion [bash|zsh|fish|powershell]",
Aliases: []string{"shell-completion"},
Short: "Generate the autocompletion script for the specified shell",
Long: `To load completions:
Bash:
$ source <(yq completion bash)
# To load completions for each session, execute once:
Linux:
$ yq completion bash > /etc/bash_completion.d/yq
MacOS:
$ yq completion bash > /usr/local/etc/bash_completion.d/yq
Zsh:
# If shell completion is not already enabled in your environment you will need
# to enable it. You can execute the following once:
$ echo "autoload -U compinit; compinit" >> ~/.zshrc
# To load completions for each session, execute once:
$ yq completion zsh > "${fpath[1]}/_yq"
# You will need to start a new shell for this setup to take effect.
Fish:
$ yq completion fish | source
# To load completions for each session, execute once:
$ yq completion fish > ~/.config/fish/completions/yq.fish
`,
DisableFlagsInUseLine: true,
ValidArgs: []string{"bash", "zsh", "fish", "powershell"},
Args: cobra.MatchAll(cobra.ExactArgs(1), cobra.OnlyValidArgs),
RunE: func(cmd *cobra.Command, args []string) error {
var err error = nil
switch args[0] {
case "bash":
err = cmd.Root().GenBashCompletionV2(os.Stdout, true)
case "zsh":
err = cmd.Root().GenZshCompletion(os.Stdout)
case "fish":
err = cmd.Root().GenFishCompletion(os.Stdout, true)
case "powershell":
err = cmd.Root().GenPowerShellCompletion(os.Stdout)
}
return err
},
}
-37
View File
@@ -1,37 +0,0 @@
package cmd
var unwrapScalarFlag = newUnwrapFlag()
var unwrapScalar = false
var writeInplace = false
var outputToJSON = false
var outputFormat = ""
var inputFormat = ""
var exitStatus = false
var indent = 2
var noDocSeparators = false
var nullInput = false
var nulSepOutput = false
var verbose = false
var version = false
var prettyPrint = false
var forceColor = false
var forceNoColor = false
var colorsEnabled = false
// can be either "" (off), "extract" or "process"
var frontMatter = ""
var splitFileExp = ""
var splitFileExpFile = ""
var completedSuccessfully = false
var forceExpression = ""
var expressionFile = ""
-140
View File
@@ -1,140 +0,0 @@
package cmd
import (
"errors"
"github.com/mikefarah/yq/v4/pkg/yqlib"
"github.com/spf13/cobra"
)
func createEvaluateAllCommand() *cobra.Command {
var cmdEvalAll = &cobra.Command{
Use: "eval-all [expression] [yaml_file1]...",
Aliases: []string{"ea"},
Short: "Loads _all_ yaml documents of _all_ yaml files and runs expression once",
ValidArgsFunction: func(_ *cobra.Command, args []string, _ string) ([]string, cobra.ShellCompDirective) {
if len(args) == 0 {
return nil, cobra.ShellCompDirectiveNoFileComp
}
return nil, cobra.ShellCompDirectiveDefault
},
Example: `
# Merge f2.yml into f1.yml (in place)
yq eval-all --inplace 'select(fileIndex == 0) * select(fileIndex == 1)' f1.yml f2.yml
## the same command and expression using shortened names:
yq ea -i 'select(fi == 0) * select(fi == 1)' f1.yml f2.yml
# Merge all given files
yq ea '. as $item ireduce ({}; . * $item )' file1.yml file2.yml ...
# Pipe from STDIN
## use '-' as a filename to pipe from STDIN
cat file2.yml | yq ea '.a.b' file1.yml - file3.yml
`,
Long: `yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.
## Evaluate All ##
This command loads _all_ yaml documents of _all_ yaml files and runs expression once
Useful when you need to run an expression across several yaml documents or files (like merge).
Note that it consumes more memory than eval.
`,
RunE: evaluateAll,
}
return cmdEvalAll
}
func evaluateAll(cmd *cobra.Command, args []string) (cmdError error) {
// 0 args, read std in
// 1 arg, null input, process expression
// 1 arg, read file in sequence
// 2+ args, [0] = expression, file the rest
var err error
expression, args, err := initCommand(cmd, args)
if err != nil {
return err
}
out := cmd.OutOrStdout()
if writeInplace {
// only use colors if its forced
colorsEnabled = forceColor
writeInPlaceHandler := yqlib.NewWriteInPlaceHandler(args[0])
out, err = writeInPlaceHandler.CreateTempFile()
if err != nil {
return err
}
// need to indirectly call the function so that completedSuccessfully is
// passed when we finish execution as opposed to now
defer func() {
if cmdError == nil {
cmdError = writeInPlaceHandler.FinishWriteInPlace(completedSuccessfully)
}
}()
}
format, err := yqlib.FormatFromString(outputFormat)
if err != nil {
return err
}
decoder, err := configureDecoder(true)
if err != nil {
return err
}
printerWriter, err := configurePrinterWriter(format, out)
if err != nil {
return err
}
encoder, err := configureEncoder()
if err != nil {
return err
}
printer := yqlib.NewPrinter(encoder, printerWriter)
if nulSepOutput {
printer.SetNulSepOutput(true)
}
if frontMatter != "" {
frontMatterHandler := yqlib.NewFrontMatterHandler(args[0])
err = frontMatterHandler.Split()
if err != nil {
return err
}
args[0] = frontMatterHandler.GetYamlFrontMatterFilename()
if frontMatter == "process" {
reader := frontMatterHandler.GetContentReader()
printer.SetAppendix(reader)
defer yqlib.SafelyCloseReader(reader)
}
defer frontMatterHandler.CleanUp()
}
allAtOnceEvaluator := yqlib.NewAllAtOnceEvaluator()
switch len(args) {
case 0:
if nullInput {
err = yqlib.NewStreamEvaluator().EvaluateNew(processExpression(expression), printer)
} else {
cmd.Println(cmd.UsageString())
return nil
}
default:
err = allAtOnceEvaluator.EvaluateFiles(processExpression(expression), args, printer, decoder)
}
completedSuccessfully = err == nil
if err == nil && exitStatus && !printer.PrintedAnything() {
return errors.New("no matches found")
}
return err
}
-153
View File
@@ -1,153 +0,0 @@
package cmd
import (
"errors"
"fmt"
"github.com/mikefarah/yq/v4/pkg/yqlib"
"github.com/spf13/cobra"
)
func createEvaluateSequenceCommand() *cobra.Command {
var cmdEvalSequence = &cobra.Command{
Use: "eval [expression] [yaml_file1]...",
Aliases: []string{"e"},
Short: "(default) Apply the expression to each document in each yaml file in sequence",
ValidArgsFunction: func(_ *cobra.Command, args []string, _ string) ([]string, cobra.ShellCompDirective) {
if len(args) == 0 {
return nil, cobra.ShellCompDirectiveNoFileComp
}
return nil, cobra.ShellCompDirectiveDefault
},
Example: `
# Reads field under the given path for each file
yq e '.a.b' f1.yml f2.yml
# Prints out the file
yq e sample.yaml
# Pipe from STDIN
## use '-' as a filename to pipe from STDIN
cat file2.yml | yq e '.a.b' file1.yml - file3.yml
# Creates a new yaml document
## Note that editing an empty file does not work.
yq e -n '.a.b.c = "cat"'
# Update a file in place
yq e '.a.b = "cool"' -i file.yaml
`,
Long: `yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.
## Evaluate Sequence ##
This command iterates over each yaml document from each given file, applies the
expression and prints the result in sequence.`,
RunE: evaluateSequence,
}
return cmdEvalSequence
}
func processExpression(expression string) string {
if prettyPrint && expression == "" {
return yqlib.PrettyPrintExp
} else if prettyPrint {
return fmt.Sprintf("%v | %v", expression, yqlib.PrettyPrintExp)
}
return expression
}
func evaluateSequence(cmd *cobra.Command, args []string) (cmdError error) {
// 0 args, read std in
// 1 arg, null input, process expression
// 1 arg, read file in sequence
// 2+ args, [0] = expression, file the rest
out := cmd.OutOrStdout()
var err error
expression, args, err := initCommand(cmd, args)
if err != nil {
return err
}
if writeInplace {
// only use colors if its forced
colorsEnabled = forceColor
writeInPlaceHandler := yqlib.NewWriteInPlaceHandler(args[0])
out, err = writeInPlaceHandler.CreateTempFile()
if err != nil {
return err
}
// need to indirectly call the function so that completedSuccessfully is
// passed when we finish execution as opposed to now
defer func() {
if cmdError == nil {
cmdError = writeInPlaceHandler.FinishWriteInPlace(completedSuccessfully)
}
}()
}
format, err := yqlib.FormatFromString(outputFormat)
if err != nil {
return err
}
printerWriter, err := configurePrinterWriter(format, out)
if err != nil {
return err
}
encoder, err := configureEncoder()
if err != nil {
return err
}
printer := yqlib.NewPrinter(encoder, printerWriter)
if nulSepOutput {
printer.SetNulSepOutput(true)
}
decoder, err := configureDecoder(false)
if err != nil {
return err
}
streamEvaluator := yqlib.NewStreamEvaluator()
if frontMatter != "" {
yqlib.GetLogger().Debug("using front matter handler")
frontMatterHandler := yqlib.NewFrontMatterHandler(args[0])
err = frontMatterHandler.Split()
if err != nil {
return err
}
args[0] = frontMatterHandler.GetYamlFrontMatterFilename()
if frontMatter == "process" {
reader := frontMatterHandler.GetContentReader()
printer.SetAppendix(reader)
defer yqlib.SafelyCloseReader(reader)
}
defer frontMatterHandler.CleanUp()
}
switch len(args) {
case 0:
if nullInput {
err = streamEvaluator.EvaluateNew(processExpression(expression), printer)
} else {
cmd.Println(cmd.UsageString())
return nil
}
default:
err = streamEvaluator.EvaluateFiles(processExpression(expression), args, printer, decoder)
}
completedSuccessfully = err == nil
if err == nil && exitStatus && !printer.PrintedAnything() {
return errors.New("no matches found")
}
return err
}
-220
View File
@@ -1,220 +0,0 @@
package cmd
import (
"fmt"
"os"
"strings"
"github.com/mikefarah/yq/v4/pkg/yqlib"
"github.com/spf13/cobra"
logging "gopkg.in/op/go-logging.v1"
)
type runeValue rune
func newRuneVar(p *rune) *runeValue {
return (*runeValue)(p)
}
func (r *runeValue) String() string {
return string(*r)
}
func (r *runeValue) Set(rawVal string) error {
val := strings.ReplaceAll(rawVal, "\\n", "\n")
val = strings.ReplaceAll(val, "\\t", "\t")
val = strings.ReplaceAll(val, "\\r", "\r")
val = strings.ReplaceAll(val, "\\f", "\f")
val = strings.ReplaceAll(val, "\\v", "\v")
if len(val) != 1 {
return fmt.Errorf("[%v] is not a valid character. Must be length 1 was %v", val, len(val))
}
*r = runeValue(rune(val[0]))
return nil
}
func (r *runeValue) Type() string {
return "char"
}
func New() *cobra.Command {
var rootCmd = &cobra.Command{
Use: "yq",
Short: "yq is a lightweight and portable command-line data file processor.",
Long: `yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.`,
Example: `
# yq tries to auto-detect the file format based off the extension, and defaults to YAML if it's unknown (or piping through STDIN)
# Use the '-p/--input-format' flag to specify a format type.
cat file.xml | yq -p xml
# read the "stuff" node from "myfile.yml"
yq '.stuff' < myfile.yml
# update myfile.yml in place
yq -i '.stuff = "foo"' myfile.yml
# print contents of sample.json as idiomatic YAML
yq -P -oy sample.json
`,
RunE: func(cmd *cobra.Command, args []string) error {
if version {
cmd.Print(GetVersionDisplay())
return nil
}
return evaluateSequence(cmd, args)
},
PersistentPreRunE: func(cmd *cobra.Command, _ []string) error {
cmd.SetOut(cmd.OutOrStdout())
level := logging.WARNING
stringFormat := `[%{level}] %{color}%{time:15:04:05}%{color:reset} %{message}`
// when NO_COLOR environment variable presents and not an empty string the coloured output should be disabled;
// refer to no-color.org
forceNoColor = forceNoColor || os.Getenv("NO_COLOR") != ""
if verbose && forceNoColor {
level = logging.DEBUG
stringFormat = `[%{level:5.5s}] %{time:15:04:05} %{shortfile:-33s} %{shortfunc:-25s} %{message}`
} else if verbose {
level = logging.DEBUG
stringFormat = `[%{level:5.5s}] %{color}%{time:15:04:05}%{color:bold} %{shortfile:-33s} %{shortfunc:-25s}%{color:reset} %{message}`
} else if forceNoColor {
stringFormat = `[%{level}] %{time:15:04:05} %{message}`
}
var format = logging.MustStringFormatter(stringFormat)
var backend = logging.AddModuleLevel(
logging.NewBackendFormatter(logging.NewLogBackend(os.Stderr, "", 0), format))
backend.SetLevel(level, "")
logging.SetBackend(backend)
yqlib.InitExpressionParser()
return nil
},
}
rootCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "verbose mode")
rootCmd.PersistentFlags().BoolVarP(&outputToJSON, "tojson", "j", false, "(deprecated) output as json. Set indent to 0 to print json in one line.")
err := rootCmd.PersistentFlags().MarkDeprecated("tojson", "please use -o=json instead")
if err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVarP(&outputFormat, "output-format", "o", "auto", fmt.Sprintf("[auto|a|%v] output format type.", yqlib.GetAvailableOutputFormatString()))
var outputCompletions = []string{"auto"}
for _, formats := range yqlib.GetAvailableOutputFormats() {
outputCompletions = append(outputCompletions, formats.FormalName)
}
if err = rootCmd.RegisterFlagCompletionFunc("output-format", cobra.FixedCompletions(outputCompletions, cobra.ShellCompDirectiveNoFileComp)); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVarP(&inputFormat, "input-format", "p", "auto", fmt.Sprintf("[auto|a|%v] parse format for input.", yqlib.GetAvailableInputFormatString()))
var inputCompletions = []string{"auto"}
for _, formats := range yqlib.GetAvailableInputFormats() {
inputCompletions = append(inputCompletions, formats.FormalName)
}
if err = rootCmd.RegisterFlagCompletionFunc("input-format", cobra.FixedCompletions(inputCompletions, cobra.ShellCompDirectiveNoFileComp)); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.AttributePrefix, "xml-attribute-prefix", yqlib.ConfiguredXMLPreferences.AttributePrefix, "prefix for xml attributes")
if err = rootCmd.RegisterFlagCompletionFunc("xml-attribute-prefix", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.ContentName, "xml-content-name", yqlib.ConfiguredXMLPreferences.ContentName, "name for xml content (if no attribute name is present).")
if err = rootCmd.RegisterFlagCompletionFunc("xml-content-name", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.StrictMode, "xml-strict-mode", yqlib.ConfiguredXMLPreferences.StrictMode, "enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.KeepNamespace, "xml-keep-namespace", yqlib.ConfiguredXMLPreferences.KeepNamespace, "enables keeping namespace after parsing attributes")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.UseRawToken, "xml-raw-token", yqlib.ConfiguredXMLPreferences.UseRawToken, "enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details.")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.ProcInstPrefix, "xml-proc-inst-prefix", yqlib.ConfiguredXMLPreferences.ProcInstPrefix, "prefix for xml processing instructions (e.g. <?xml version=\"1\"?>)")
if err = rootCmd.RegisterFlagCompletionFunc("xml-proc-inst-prefix", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredXMLPreferences.DirectiveName, "xml-directive-name", yqlib.ConfiguredXMLPreferences.DirectiveName, "name for xml directives (e.g. <!DOCTYPE thing cat>)")
if err = rootCmd.RegisterFlagCompletionFunc("xml-directive-name", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.SkipProcInst, "xml-skip-proc-inst", yqlib.ConfiguredXMLPreferences.SkipProcInst, "skip over process instructions (e.g. <?xml version=\"1\"?>)")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredXMLPreferences.SkipDirectives, "xml-skip-directives", yqlib.ConfiguredXMLPreferences.SkipDirectives, "skip over directives (e.g. <!DOCTYPE thing cat>)")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredCsvPreferences.AutoParse, "csv-auto-parse", yqlib.ConfiguredCsvPreferences.AutoParse, "parse CSV YAML/JSON values")
rootCmd.PersistentFlags().Var(newRuneVar(&yqlib.ConfiguredCsvPreferences.Separator), "csv-separator", "CSV Separator character")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredTsvPreferences.AutoParse, "tsv-auto-parse", yqlib.ConfiguredTsvPreferences.AutoParse, "parse TSV YAML/JSON values")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredLuaPreferences.DocPrefix, "lua-prefix", yqlib.ConfiguredLuaPreferences.DocPrefix, "prefix")
if err = rootCmd.RegisterFlagCompletionFunc("lua-prefix", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredLuaPreferences.DocSuffix, "lua-suffix", yqlib.ConfiguredLuaPreferences.DocSuffix, "suffix")
if err = rootCmd.RegisterFlagCompletionFunc("lua-suffix", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredLuaPreferences.UnquotedKeys, "lua-unquoted", yqlib.ConfiguredLuaPreferences.UnquotedKeys, "output unquoted string keys (e.g. {foo=\"bar\"})")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredLuaPreferences.Globals, "lua-globals", yqlib.ConfiguredLuaPreferences.Globals, "output keys as top-level global variables")
rootCmd.PersistentFlags().StringVar(&yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator, "properties-separator", yqlib.ConfiguredPropertiesPreferences.KeyValueSeparator, "separator to use between keys and values")
rootCmd.PersistentFlags().BoolVar(&yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets, "properties-array-brackets", yqlib.ConfiguredPropertiesPreferences.UseArrayBrackets, "use [x] in array paths (e.g. for SpringBoot)")
rootCmd.PersistentFlags().BoolVar(&yqlib.StringInterpolationEnabled, "string-interpolation", yqlib.StringInterpolationEnabled, "Toggles strings interpolation of \\(exp)")
rootCmd.PersistentFlags().BoolVarP(&nullInput, "null-input", "n", false, "Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.")
rootCmd.PersistentFlags().BoolVarP(&noDocSeparators, "no-doc", "N", false, "Don't print document separators (---)")
rootCmd.PersistentFlags().IntVarP(&indent, "indent", "I", 2, "sets indent level for output")
if err = rootCmd.RegisterFlagCompletionFunc("indent", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.Flags().BoolVarP(&version, "version", "V", false, "Print version information and quit")
rootCmd.PersistentFlags().BoolVarP(&writeInplace, "inplace", "i", false, "update the file in place of first file given.")
rootCmd.PersistentFlags().VarP(unwrapScalarFlag, "unwrapScalar", "r", "unwrap scalar, print the value with no quotes, colors or comments. Defaults to true for yaml")
rootCmd.PersistentFlags().Lookup("unwrapScalar").NoOptDefVal = "true"
rootCmd.PersistentFlags().BoolVarP(&nulSepOutput, "nul-output", "0", false, "Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.")
rootCmd.PersistentFlags().BoolVarP(&prettyPrint, "prettyPrint", "P", false, "pretty print, shorthand for '... style = \"\"'")
rootCmd.PersistentFlags().BoolVarP(&exitStatus, "exit-status", "e", false, "set exit status if there are no matches or null or false is returned")
rootCmd.PersistentFlags().BoolVarP(&forceColor, "colors", "C", false, "force print with colors")
rootCmd.PersistentFlags().BoolVarP(&forceNoColor, "no-colors", "M", forceNoColor, "force print with no colors")
rootCmd.PersistentFlags().StringVarP(&frontMatter, "front-matter", "f", "", "(extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact")
if err = rootCmd.RegisterFlagCompletionFunc("front-matter", cobra.FixedCompletions([]string{"extract", "process"}, cobra.ShellCompDirectiveNoFileComp)); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVarP(&forceExpression, "expression", "", "", "forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.")
if err = rootCmd.RegisterFlagCompletionFunc("expression", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().BoolVarP(&yqlib.ConfiguredYamlPreferences.LeadingContentPreProcessing, "header-preprocess", "", true, "Slurp any header comments and separators before processing expression.")
rootCmd.PersistentFlags().StringVarP(&splitFileExp, "split-exp", "s", "", "print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter. The necessary directories will be created.")
if err = rootCmd.RegisterFlagCompletionFunc("split-exp", cobra.NoFileCompletions); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVarP(&splitFileExpFile, "split-exp-file", "", "", "Use a file to specify the split-exp expression.")
if err = rootCmd.MarkPersistentFlagFilename("split-exp-file"); err != nil {
panic(err)
}
rootCmd.PersistentFlags().StringVarP(&expressionFile, "from-file", "", "", "Load expression from specified file.")
if err = rootCmd.MarkPersistentFlagFilename("from-file"); err != nil {
panic(err)
}
rootCmd.AddCommand(
createEvaluateSequenceCommand(),
createEvaluateAllCommand(),
completionCmd,
)
return rootCmd
}
-46
View File
@@ -1,46 +0,0 @@
package cmd
import (
"strconv"
"github.com/spf13/pflag"
)
type boolFlag interface {
pflag.Value
IsExplicitlySet() bool
IsSet() bool
}
type unwrapScalarFlagStrc struct {
explicitlySet bool
value bool
}
func newUnwrapFlag() boolFlag {
return &unwrapScalarFlagStrc{value: true}
}
func (f *unwrapScalarFlagStrc) IsExplicitlySet() bool {
return f.explicitlySet
}
func (f *unwrapScalarFlagStrc) IsSet() bool {
return f.value
}
func (f *unwrapScalarFlagStrc) String() string {
return strconv.FormatBool(f.value)
}
func (f *unwrapScalarFlagStrc) Set(value string) error {
v, err := strconv.ParseBool(value)
f.value = v
f.explicitlySet = true
return err
}
func (*unwrapScalarFlagStrc) Type() string {
return "bool"
}
-243
View File
@@ -1,243 +0,0 @@
package cmd
import (
"fmt"
"io"
"os"
"strings"
"github.com/mikefarah/yq/v4/pkg/yqlib"
"github.com/spf13/cobra"
"gopkg.in/op/go-logging.v1"
)
func isAutomaticOutputFormat() bool {
return outputFormat == "" || outputFormat == "auto" || outputFormat == "a"
}
func initCommand(cmd *cobra.Command, args []string) (string, []string, error) {
cmd.SilenceUsage = true
fileInfo, _ := os.Stdout.Stat()
if forceColor || (!forceNoColor && (fileInfo.Mode()&os.ModeCharDevice) != 0) {
colorsEnabled = true
}
expression, args, err := processArgs(args)
if err != nil {
return "", nil, err
}
if splitFileExpFile != "" {
splitExpressionBytes, err := os.ReadFile(splitFileExpFile)
if err != nil {
return "", nil, err
}
splitFileExp = string(splitExpressionBytes)
}
// backwards compatibility
if outputToJSON {
outputFormat = "json"
}
if writeInplace && (len(args) == 0 || args[0] == "-") {
return "", nil, fmt.Errorf("write in place flag only applicable when giving an expression and at least one file")
}
if frontMatter != "" && len(args) == 0 {
return "", nil, fmt.Errorf("front matter flag only applicable when giving an expression and at least one file")
}
if writeInplace && splitFileExp != "" {
return "", nil, fmt.Errorf("write in place cannot be used with split file")
}
if nullInput && len(args) > 0 {
return "", nil, fmt.Errorf("cannot pass files in when using null-input flag")
}
inputFilename := ""
if len(args) > 0 {
inputFilename = args[0]
}
if inputFormat == "" || inputFormat == "auto" || inputFormat == "a" {
inputFormat = yqlib.FormatStringFromFilename(inputFilename)
_, err := yqlib.FormatFromString(inputFormat)
if err != nil {
// unknown file type, default to yaml
yqlib.GetLogger().Debug("Unknown file format extension '%v', defaulting to yaml", inputFormat)
inputFormat = "yaml"
if isAutomaticOutputFormat() {
outputFormat = "yaml"
}
} else if isAutomaticOutputFormat() {
outputFormat = inputFormat
}
} else if isAutomaticOutputFormat() {
// backwards compatibility -
// before this was introduced, `yq -pcsv things.csv`
// would produce *yaml* output.
//
outputFormat = yqlib.FormatStringFromFilename(inputFilename)
if inputFilename != "-" {
yqlib.GetLogger().Warning("yq default output is now 'auto' (based on the filename extension). Normally yq would output '%v', but for backwards compatibility 'yaml' has been set. Please use -oy to specify yaml, or drop the -p flag.", outputFormat)
}
outputFormat = "yaml"
}
outputFormatType, err := yqlib.FormatFromString(outputFormat)
if err != nil {
return "", nil, err
}
yqlib.GetLogger().Debug("Using input format %v", inputFormat)
yqlib.GetLogger().Debug("Using output format %v", outputFormat)
if outputFormatType == yqlib.YamlFormat ||
outputFormatType == yqlib.PropertiesFormat {
unwrapScalar = true
}
if unwrapScalarFlag.IsExplicitlySet() {
unwrapScalar = unwrapScalarFlag.IsSet()
}
return expression, args, nil
}
func configureDecoder(evaluateTogether bool) (yqlib.Decoder, error) {
format, err := yqlib.FormatFromString(inputFormat)
if err != nil {
return nil, err
}
yqlib.ConfiguredYamlPreferences.EvaluateTogether = evaluateTogether
yqlibDecoder := format.DecoderFactory()
if yqlibDecoder == nil {
return nil, fmt.Errorf("no support for %s input format", inputFormat)
}
return yqlibDecoder, nil
}
func configurePrinterWriter(format *yqlib.Format, out io.Writer) (yqlib.PrinterWriter, error) {
var printerWriter yqlib.PrinterWriter
if splitFileExp != "" {
colorsEnabled = forceColor
splitExp, err := yqlib.ExpressionParser.ParseExpression(splitFileExp)
if err != nil {
return nil, fmt.Errorf("bad split document expression: %w", err)
}
printerWriter = yqlib.NewMultiPrinterWriter(splitExp, format)
} else {
printerWriter = yqlib.NewSinglePrinterWriter(out)
}
return printerWriter, nil
}
func configureEncoder() (yqlib.Encoder, error) {
yqlibOutputFormat, err := yqlib.FormatFromString(outputFormat)
if err != nil {
return nil, err
}
yqlib.ConfiguredXMLPreferences.Indent = indent
yqlib.ConfiguredYamlPreferences.Indent = indent
yqlib.ConfiguredJSONPreferences.Indent = indent
yqlib.ConfiguredYamlPreferences.UnwrapScalar = unwrapScalar
yqlib.ConfiguredPropertiesPreferences.UnwrapScalar = unwrapScalar
yqlib.ConfiguredJSONPreferences.UnwrapScalar = unwrapScalar
yqlib.ConfiguredYamlPreferences.ColorsEnabled = colorsEnabled
yqlib.ConfiguredJSONPreferences.ColorsEnabled = colorsEnabled
yqlib.ConfiguredYamlPreferences.PrintDocSeparators = !noDocSeparators
encoder := yqlibOutputFormat.EncoderFactory()
if encoder == nil {
return nil, fmt.Errorf("no support for %s output format", outputFormat)
}
return encoder, err
}
// this is a hack to enable backwards compatibility with githubactions (which pipe /dev/null into everything)
// and being able to call yq with the filename as a single parameter
//
// without this - yq detects there is stdin (thanks githubactions),
// then tries to parse the filename as an expression
func maybeFile(str string) bool {
yqlib.GetLogger().Debugf("checking '%v' is a file", str)
stat, err := os.Stat(str) // #nosec
result := err == nil && !stat.IsDir()
if yqlib.GetLogger().IsEnabledFor(logging.DEBUG) {
if err != nil {
yqlib.GetLogger().Debugf("error: %v", err)
} else {
yqlib.GetLogger().Debugf("error: %v, dir: %v", err, stat.IsDir())
}
yqlib.GetLogger().Debugf("result: %v", result)
}
return result
}
func processStdInArgs(args []string) []string {
stat, err := os.Stdin.Stat()
if err != nil {
yqlib.GetLogger().Debugf("error getting stdin: %v", err)
}
pipingStdin := stat != nil && (stat.Mode()&os.ModeCharDevice) == 0
// if we've been given a file, don't automatically
// read from stdin.
// this happens if there is more than one argument
// or only one argument and its a file
if nullInput || !pipingStdin || len(args) > 1 || (len(args) > 0 && maybeFile(args[0])) {
return args
}
for _, arg := range args {
if arg == "-" {
return args
}
}
yqlib.GetLogger().Debugf("missing '-', adding it to the end")
// we're piping from stdin, but there's no '-' arg
// lets add one to the end
return append(args, "-")
}
func processArgs(originalArgs []string) (string, []string, error) {
expression := forceExpression
args := processStdInArgs(originalArgs)
maybeFirstArgIsAFile := len(args) > 0 && maybeFile(args[0])
if expressionFile == "" && maybeFirstArgIsAFile && strings.HasSuffix(args[0], ".yq") {
// lets check if an expression file was given
yqlib.GetLogger().Debug("Assuming arg %v is an expression file", args[0])
expressionFile = args[0]
args = args[1:]
}
if expressionFile != "" {
expressionBytes, err := os.ReadFile(expressionFile)
if err != nil {
return "", nil, err
}
//replace \r\n (windows) with good ol' unix file endings.
expression = strings.ReplaceAll(string(expressionBytes), "\r\n", "\n")
}
yqlib.GetLogger().Debugf("processed args: %v", args)
if expression == "" && len(args) > 0 && args[0] != "-" && !maybeFile(args[0]) {
yqlib.GetLogger().Debug("assuming expression is '%v'", args[0])
expression = args[0]
args = args[1:]
}
return expression, args, nil
}
-49
View File
@@ -1,49 +0,0 @@
package cmd
import (
"fmt"
"strings"
)
// The git commit that was compiled. This will be filled in by the compiler.
var (
GitCommit string
GitDescribe string
// Version is main version number that is being run at the moment.
Version = "v4.45.2"
// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
// then it means that it is a final release. Otherwise, this is a pre-release
// such as "dev" (in development), "beta", "rc1", etc.
VersionPrerelease = ""
)
// ProductName is the name of the product
const ProductName = "yq"
// GetVersionDisplay composes the parts of the version in a way that's suitable
// for displaying to humans.
func GetVersionDisplay() string {
return fmt.Sprintf("yq (https://github.com/mikefarah/yq/) version %s\n", getHumanVersion())
}
func getHumanVersion() string {
version := Version
if GitDescribe != "" {
version = GitDescribe
}
release := VersionPrerelease
if release != "" {
if !strings.Contains(version, release) {
version += fmt.Sprintf("-%s", release)
}
if GitCommit != "" {
version += fmt.Sprintf(" (%s)", GitCommit)
}
}
// Strip off any single quotes added by the git information.
return strings.ReplaceAll(version, "'", "")
}
-51
View File
@@ -1,51 +0,0 @@
package cmd
import "testing"
func TestGetVersionDisplay(t *testing.T) {
var expectedVersion = ProductName + " (https://github.com/mikefarah/yq/) version " + Version
if VersionPrerelease != "" {
expectedVersion = expectedVersion + "-" + VersionPrerelease
}
expectedVersion = expectedVersion + "\n"
tests := []struct {
name string
want string
}{
{
name: "Display Version",
want: expectedVersion,
},
}
for _, tt := range tests {
if got := GetVersionDisplay(); got != tt.want {
t.Errorf("%q. GetVersionDisplay() = %v, want %v", tt.name, got, tt.want)
}
}
}
func Test_getHumanVersion(t *testing.T) {
GitDescribe = "e42813d"
GitCommit = "e42813d+CHANGES"
var wanted string
if VersionPrerelease == "" {
wanted = GitDescribe
} else {
wanted = "e42813d-" + VersionPrerelease + " (e42813d+CHANGES)"
}
tests := []struct {
name string
want string
}{
{
name: "Git Variables defined",
want: wanted,
},
}
for _, tt := range tests {
if got := getHumanVersion(); got != tt.want {
t.Errorf("%q. getHumanVersion() = %v, want %v", tt.name, got, tt.want)
}
}
}
+45
View File
@@ -0,0 +1,45 @@
---
description: >-
Read all documents of all given yaml files into memory, then run the given
expression once against the lot.
---
# Evaluate All
Evaluate All is most useful when needing to run expressions that depend on multiple yaml documents or files. Merge is probably the most common reason why evaluate all would be used. Note that `eval-all` consumes more memory than `evaluate`.
Like `evaluate` you can use `-` to pipe from STDIN.
## Usage
```bash
yq eval-all [expression] [yaml_file1]... [flags]
```
Aliases: `eval-all, ea`
## Examples
```bash
# merges f2.yml into f1.yml (inplace)
yq eval-all --inplace 'select(fileIndex == 0) * select(fileIndex == 1)' f1.yml f2.yml
# you can merge into a file, piping from STDIN
cat somefile.yml | yq eval-all --inplace 'select(fileIndex == 0) * select(fileIndex == 1)' f1.yml -
```
## Flags
```bash
-h, --help help for eval-all
-C, --colors force print with colors
-e, --exit-status set exit status if there are no matches or null or false is returned
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the yaml file inplace of first yaml file given.
-M, --no-colors force print with no colors
-N, --no-doc Don't print document separators (---)
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating yaml docs from scratch.
-j, --tojson output as json. Set indent to 0 to print json in one line.
-v, --verbose verbose mode
```
+56
View File
@@ -0,0 +1,56 @@
---
description: >-
Evaluates the given expression against each yaml document in each file, in
sequence
---
# Evaluate
Note that (as of 4.18.1) this is the default command when none is supplied to yq.
## Usage:
```bash
yq eval [expression] [yaml_file1]... [flags]
```
Aliases: `eval, e`
Note that you can pass in `-` as a filename to pipe from STDIN.
## Examples:
```bash
# runs the expression against each file, in series
yq '.a.b | length' f1.yml f2.yml
# '-' will pipe from STDIN
cat file.yml | yq '.a.b' f1.yml - f2.yml
# prints out the file
yq sample.yaml
cat sample.yml | yq e
# prints a new yaml document
yq -n '.a.b.c = "cat"'
# updates file.yaml directly
yq '.a.b = "cool"' -i file.yaml
```
## Flags:
```
-h, --help help for eval
-C, --colors force print with colors
-e, --exit-status set exit status if there are no matches or null or false is returned
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the yaml file inplace of first yaml file given.
-M, --no-colors force print with no colors
-N, --no-doc Don't print document separators (---)
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating yaml docs from scratch.
-j, --tojson output as json. Set indent to 0 to print json in one line.
-v, --verbose verbose mode
```
+69
View File
@@ -0,0 +1,69 @@
---
description: >-
Generate a shell completion file for supported shells
(bash/fish/zsh/powershell)
---
# Shell Completion
```bash
yq shell-completion zsh
```
Prints to StdOut a shell completion script for zsh shell.
### Bash (default)
```bash
source <(yq shell-completion bash)
```
#### To load completions for each session, execute once:
Linux:
```bash
yq shell-completion bash > /etc/bash_completion.d/yq
```
MacOS:
```bash
yq shell-completion bash > /usr/local/etc/bash_completion.d/yq
```
### zsh
If shell completion is not already enabled in your environment you will need to enable it. You can execute the following once:
```bash
echo "autoload -U compinit; compinit" >> ~/.zshrc
```
#### To load completions for each session, execute once:
```bash
yq shell-completion zsh > "${fpath[1]}/_yq"
```
You will need to start a new shell for this setup to take effect.
### fish
```bash
yq shell-completion fish | source
```
#### To load completions for each session, execute once:
```
yq shell-completion fish > ~/.config/fish/completions/yq.fish
```
### PowerShell
```bash
yq shell-completion powershell
```
Users need PowerShell version 5.0 or above, which comes with Windows 10 and can be downloaded separately for Windows 7 or 8.1. They can then write the completions to a file and source this file from their PowerShell profile, which is referenced by the $Profile environment variable.
-14
View File
@@ -1,14 +0,0 @@
---
$schema: https://raw.githubusercontent.com/streetsidesoftware/cspell/main/cspell.schema.json
version: '0.2'
language: en-GB
dictionaryDefinitions:
- name: project-words
path: './project-words.txt'
addWords: true
dictionaries:
- project-words
ignorePaths:
- 'vendor'
- 'bin'
- '/project-words.txt'
-2
View File
@@ -1,2 +0,0 @@
- [cat, dog, frog, cow]
- [apple, banana, grape, mango]
-8
View File
@@ -1,8 +0,0 @@
b:
d: be gone
c: 2
e:
- name: Billy Bob # comment over here
---
[123123
-1
View File
@@ -1 +0,0 @@
bXkgc2VjcmV0IGNoaWxsaSByZWNpcGUgaXMuLi4u
-9
View File
@@ -1,9 +0,0 @@
return {
["country"] = "Australia"; -- this place
["cities"] = {
"Sydney",
"Melbourne",
"Brisbane",
"Perth",
};
};
-4
View File
@@ -1,4 +0,0 @@
a: simple
b: [1, 2]
c:
test: 1
-3
View File
@@ -1,3 +0,0 @@
Foo: 3
apple: 1
bar: 2
-5
View File
@@ -1,5 +0,0 @@
# --------------------------------------------------
# It's a test with comment
# --------------------------------------------------
groups:
- name: d
-4
View File
@@ -1,4 +0,0 @@
a: "simple" # just the best
b: [1, 3]
c:
test: 1
View File
-1
View File
@@ -1 +0,0 @@
# comment
-5
View File
@@ -1,5 +0,0 @@
#! yq
.[] |(
( select(kind == "scalar") | key + "='" + . + "'"),
( select(kind == "seq") | key + "=(" + (map("'" + . + "'") | join(",")) + ")")
)
-6
View File
@@ -1,6 +0,0 @@
# comments on values appear
person.name = Mike
# comments on array values appear
person.pets.0 = cat
person.food.0 = pizza
-6
View File
@@ -1,6 +0,0 @@
---
a: apple
b: bannana
---
hello there
apples: great
-7
View File
@@ -1,7 +0,0 @@
- command: update
path: b.c
value:
#great
things: frog # wow!
- command: delete
path: b.d
-2
View File
@@ -1,2 +0,0 @@
---
a: test
-19
View File
@@ -1,19 +0,0 @@
foo: &foo
a: foo_a
thing: foo_thing
c: foo_c
bar: &bar
b: bar_b
thing: bar_thing
c: bar_c
foobarList:
b: foobarList_b
<<: [*foo,*bar]
c: foobarList_c
foobar:
c: foobar_c
<<: *foo
thing: foobar_thing
-1
View File
@@ -1 +0,0 @@
<zoo><thing><frog>boing</frog></thing></zoo>
-11
View File
@@ -1,11 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- osm-->
<osm version="0.6" generator="CGImap 0.0.2">
<!-- bounds-->
<bounds minlat="54.0889580" minlon="12.2487570" maxlat="54.0913900" maxlon="12.2524800">
<!-- great -->
cool
</bounds>
<foo>ba2234r</foo>
<foo>bar2234233</foo>
</osm>
-5
View File
@@ -1,5 +0,0 @@
test: |
abcdefg
hijklmno
-18
View File
@@ -1,18 +0,0 @@
commonKey: first document
a: Easy! as one two three
b:
c: 2
d: [3, 4]
e:
- name: fred
value: 3
- name: sam
value: 4
---
commonKey: second document
another:
document: here
---
commonKey: third document
wow:
- here is another
-7
View File
@@ -1,7 +0,0 @@
a: Easy! as one two three
---
another:
document: here
---
- 1
- 2
-2
View File
@@ -1,2 +0,0 @@
5:
6: camel!
-2
View File
@@ -1,2 +0,0 @@
version: 3
application: MyApp
-6
View File
@@ -1,6 +0,0 @@
version: '2'
services:
test:
image: ubuntu:14.04
stdin_open: true
tty: true
-1
View File
@@ -1 +0,0 @@
{"a":"Easy! as one two three","b":{"c":2,"d":[3,4],"e":[{"name":"fred","value":3},{"name":"sam","value":4}]},"ab":"must appear last"}
-26
View File
@@ -1,26 +0,0 @@
# This is a TOML document
title = "TOML Example"
[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00
[database]
enabled = true
ports = [ 8000, 8001, 8002 ]
data = [ ["delta", "phi"], [3.14] ]
temp_targets = { cpu = 79.5, case = 72.0 }
[servers]
[servers.alpha]
ip = "10.0.0.1"
role = "frontend"
[servers.beta]
ip = "10.0.0.2"
role = "backend"
-2
View File
@@ -1,2 +0,0 @@
# things
a: apple
-1
View File
@@ -1 +0,0 @@
[1,2,3]
-2
View File
@@ -1,2 +0,0 @@
- 4
- 5
-3
View File
@@ -1,3 +0,0 @@
name,numberOfCats,likesApples,height
,1,true,168.8
Samantha's Rabbit,2,false,-188.8
1 name numberOfCats likesApples height
2 1 true 168.8
3 Samantha's Rabbit 2 false -188.8
-1
View File
@@ -1 +0,0 @@
hi
-5
View File
@@ -1,5 +0,0 @@
foo:
a: 1
foobar:
a: 1
-5
View File
@@ -1,5 +0,0 @@
foo: &foo
a: 1
foobar:
<<: *foo
-1
View File
@@ -1 +0,0 @@
this.is = a properties file
-1
View File
@@ -1 +0,0 @@
<this>is some xml</this>
-4
View File
@@ -1,4 +0,0 @@
---
# comment
# about things
a: cat
-2
View File
@@ -1,2 +0,0 @@
a: apple is included
b: cool.
-9
View File
@@ -1,9 +0,0 @@
FROM mikefarah/yq:4
COPY entrypoint.sh /entrypoint.sh
# github action recommendation is to run as root.
# https://docs.github.com/en/actions/creating-actions/dockerfile-support-for-github-actions#user
USER root
ENTRYPOINT ["/entrypoint.sh"]
-12
View File
@@ -1,12 +0,0 @@
#!/bin/sh -l
set -e
echo "::debug::\$cmd: $1"
RESULT=$(eval "$1")
echo "::debug::\$RESULT: $RESULT"
# updating from
# https://github.com/orgs/community/discussions/26288#discussioncomment-3876281
# https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter
delimiter=$(cat /proc/sys/kernel/random/uuid)
echo "result<<${delimiter}" >> "${GITHUB_OUTPUT}"
echo "${RESULT}" >> "${GITHUB_OUTPUT}"
echo "${delimiter}" >> "${GITHUB_OUTPUT}"
-32
View File
@@ -1,32 +0,0 @@
module github.com/mikefarah/yq/v4
require (
github.com/a8m/envsubst v1.4.2
github.com/alecthomas/participle/v2 v2.1.4
github.com/alecthomas/repr v0.4.0
github.com/dimchansky/utfbom v1.1.1
github.com/elliotchance/orderedmap v1.8.0
github.com/fatih/color v1.18.0
github.com/goccy/go-json v0.10.5
github.com/goccy/go-yaml v1.13.3
github.com/jinzhu/copier v0.4.0
github.com/magiconair/properties v1.8.10
github.com/pelletier/go-toml/v2 v2.2.3
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e
github.com/spf13/cobra v1.9.1
github.com/spf13/pflag v1.0.6
github.com/yuin/gopher-lua v1.1.1
golang.org/x/net v0.39.0
golang.org/x/text v0.24.0
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
golang.org/x/sys v0.32.0 // indirect
)
go 1.23.0
-81
View File
@@ -1,81 +0,0 @@
github.com/a8m/envsubst v1.4.2 h1:4yWIHXOLEJHQEFd4UjrWDrYeYlV7ncFWJOCBRLOZHQg=
github.com/a8m/envsubst v1.4.2/go.mod h1:MVUTQNGQ3tsjOOtKCNd+fl8RzhsXcDvvAEzkhGtlsbY=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/participle/v2 v2.1.4 h1:W/H79S8Sat/krZ3el6sQMvMaahJ+XcM9WSI2naI7w2U=
github.com/alecthomas/participle/v2 v2.1.4/go.mod h1:8tqVbpTX20Ru4NfYQgZf4mP18eXPTBViyMWiArNEgGI=
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dimchansky/utfbom v1.1.1 h1:vV6w1AhK4VMnhBno/TPVCoK9U/LP0PkLCS9tbxHdi/U=
github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE=
github.com/elliotchance/orderedmap v1.8.0 h1:TrOREecvh3JbS+NCgwposXG5ZTFHtEsQiCGOhPElnMw=
github.com/elliotchance/orderedmap v1.8.0/go.mod h1:wsDwEaX5jEoyhbs7x93zk2H/qv0zwuhg4inXhDkYqys=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.22.1 h1:40JcKH+bBNGFczGuoBYgX4I6m/i27HYW8P9FDk5PbgA=
github.com/go-playground/validator/v10 v10.22.1/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.13.3 h1:IXRULR8mAa0MXQobzzp0VOfMUJ8EnaQ4x3jhf7S0/nI=
github.com/goccy/go-yaml v1.13.3/go.mod h1:IjYwxUiJDoqpx2RmbdjMUceGHZwYLon3sfOGl5Hi9lc=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jinzhu/copier v0.4.0 h1:w3ciUoD19shMCRargcpm0cm91ytaBhDvuRpz1ODO/U8=
github.com/jinzhu/copier v0.4.0/go.mod h1:DfbEm0FYsaqBcKcFuvmOZb218JkPGtvSHsKg8S8hyyg=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
github.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e h1:aoZm08cpOy4WuID//EZDgcC4zIxODThtZNPirFr42+A=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M=
github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw=
golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473 h1:6D+BvnJ/j6e222UW8s2qTSe3wGBtvo0MbVQG/c5k8RE=
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473/go.mod h1:N1eN2tsCx0Ydtgjl4cqmbRCsY4/+z4cYDeqwZTk6zog=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+60 -6
View File
@@ -1,8 +1,63 @@
# How it works
# Expression Syntax: A Visual Guide
In `yq`, expressions are made up of operators and pipes. A context of nodes is passed through the expression, and each operation takes the context as input and returns a new context as output. That output is piped in as input for the next operation in the expression.
In `yq` expressions are made up of operators and pipes. A context of nodes is passed through the expression and each operation takes the context as input and returns a new context as output. That output is piped in as input for the next operation in the expression. To begin with, the context is set to the first yaml document of the first yaml file (if processing in sequence using eval).
Let's break down the process step by step using a diagram. We'll start with a single YAML document, apply an expression, and observe how the context changes at each step.
Lets look at a couple of examples.
Given a document like:
```yaml
root:
items:
- name: apple
type: fruit
- name: carrot
type: vegetable
- name: banana
type: fruit
```
You can use dot notation to access nested structures. For example, to access the `name` of the first item, you would use the expression `.root.items[0].name`, which would return `apple`.
But lets see how we could find all the fruit under `items`
## Step 1: Initial Context
The context starts at the root of the YAML document. In this case, the entire document is the initial context.
```
root
└── items
├── name: apple
│ type: fruit
├── name: carrot
│ type: vegetable
└── name: banana
type: fruit
```
## Step 2: Splatting the Array
Using the expression `.root.items[]`, we "splat" the items array. This means each element of the array becomes its own node in the context:
```
Node 1: { name: apple, type: fruit }
Node 2: { name: carrot, type: vegetable }
Node 3: { name: banana, type: fruit }
```
## Step 3: Filtering the Nodes
Next, we apply a filter to select only the nodes where type is fruit. The expression `.root.items[] | select(.type == "fruit")` filters the nodes:
```
Filtered Node 1: { name: apple, type: fruit }
Filtered Node 2: { name: banana, type: fruit }
```
## Step 4: Extracting a Field
Finally, we extract the name field from the filtered nodes using `.root.items[] | select(.type == "fruit") | .name` This results in:
```
apple
banana
```
## Simple assignment example
@@ -44,7 +99,6 @@ a: dog
b: dog
```
## Complex assignment, operator precedence rules
Just like math expressions - `yq` expressions have an order of precedence. The pipe `|` operator has a low order of precedence, so operators with higher precedence will get evaluated first.
@@ -73,7 +127,7 @@ name: sally
fruit: mango
```
To properly update this yaml, you will need to use brackets (think BODMAS from maths) and wrap the entire LHS:
**Important**: To properly update this YAML, you must wrap the entire LHS in parentheses. Think of it like using brackets in math to ensure the correct order of operations.
`(.[] | select(.name == "sally") | .fruit) = "mango"`
@@ -126,4 +180,4 @@ The assignment operator then copies across the value from the RHS to the value o
```yaml
a: 2
b: thing
```
```
-7
View File
@@ -1,7 +0,0 @@
docs_dir: mkdocs
site_dir: docs
site_name: Yq
theme: 'material'
repo_name: 'mikefarah/yq'
repo_url: 'https://github.com/mikefarah/yq'

Some files were not shown because too many files have changed in this diff Show More