AML (YAML Ain't Markup Language) is a human-readable data serialization format that is often used for configuration files and data exchange. Working with YAML files can sometimes be a bit challenging, especially when you need to extract or manipulate specific data within them. That's where yq comes into play. yq is a versatile command-line tool that allows you to parse, transform, and manipulate YAML documents with ease.
Getting Started with yq
Usage: yq [flags] yq [command]Examples:# yq defaults to 'eval' command if no command is specified. See "yq eval --help" for more examples.# read the "stuff" node from "myfile.yml"yq '.stuff' < myfile.yml# update myfile.yml in placeyq -i '.stuff = "foo"' myfile.yml# print contents of sample.json as idiomatic YAMLyq -P sample.jsonAvailable Commands: completion Generate the autocompletion script for the specified shell eval (default) Apply the expression to each document in each yaml file in sequence eval-all Loads _all_ yaml documents of _all_ yaml files and runs expression once help Help about any command shell-completion Generate completion scriptFlags: -C, --colors force print with colors -e, --exit-status set exit status if there are no matches or null or false is returned --expression string forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file. --from-file string Load expression from specified file. -f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact --header-preprocess Slurp any header comments and separators before processing expression. (default true) -h, --help help for yq -I, --indent int sets indent level for output (default 2) -i, --inplace update the file in place of first file given. -p, --input-format string [auto|a|yaml|y|props|p|xml|x|tsv|t|csv|c|toml] parse format for input. Note that json is a subset of yaml. (default "auto") --lua-globals output keys as top-level global variables --lua-prefix string prefix (default "return ") --lua-suffix string suffix (default ";\n") --lua-unquoted output unquoted string keys (e.g. {foo="bar"}) -M, --no-colors force print with no colors -N, --no-doc Don't print document separators (---) -0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char. -n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch. -o, --output-format string [auto|a|yaml|y|json|j|props|p|xml|x|tsv|t|csv|c] output format type. (default "auto") -P, --prettyPrint pretty print, shorthand for '... style = ""' -s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter. --split-exp-file string Use a file to specify the split-exp expression. -r, --unwrapScalar unwrap scalar, print the value with no quotes, colors or comments. Defaults to true for yaml (default true) -v, --verbose verbose mode -V, --version Print version information and quit --xml-attribute-prefix string prefix for xml attributes (default "+@") --xml-content-name string name for xml content (if no attribute name is present). (default "+content") --xml-directive-name string name for xml directives (e.g. <!DOCTYPE thing cat>) (default "+directive") --xml-keep-namespace enables keeping namespace after parsing attributes (default true) --xml-proc-inst-prefix string prefix for xml processing instructions (e.g. <?xml version="1"?>) (default "+p_") --xml-raw-token enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details. (default true) --xml-skip-directives skip over directives (e.g. <!DOCTYPE thing cat>) --xml-skip-proc-inst skip over process instructions (e.g. <?xml version="1"?>) --xml-strict-mode enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.Use "yq [command] --help" for more information about a command.Code language: Bash (bash)Installation
Before you can start using yq, you need to install it. Fortunately, the installation process is straightforward.
Windows:
If you're on Windows, you can use Chocolatey, a popular package manager:
choco install yqCode language: Bash (bash)Linux / macOS:
For Linux or macOS users, Homebrew simplifies the installation process:
brew install yqCode language: Bash (bash)With yq installed, let's dive into its usage.
Basic Usage
Let's start with a simple YAML file called fruits.yml, which contains information about different fruits:
fruits: apple: red pear: green banana: yellowCode language: YAML (yaml)Printing the Entire YAML
To print the entire YAML content, you can use the eval (or e) command followed by the file name:
yq e fruits.ymlCode language: CSS (css)This will display the complete YAML content, like so:
fruits: apple: red pear: green banana: yellowCode language: YAML (yaml)Extracting Specific Values
To target a specific key and extract its value, use the eval command with a path:
yq e '.fruits.apple' fruits.ymlCode language: JavaScript (javascript)The output will be:
redUpdating YAML
You can also modify YAML files directly. To update a value, use the eval command and the -i flag:
yq e '.fruits.apple = "orange"' -i fruits.ymlCode language: JavaScript (javascript)This command will change the “apple” value from “red” to “orange” in the fruits.yml file.
Adding New Entries
You can add new entries to your YAML files using yq. For instance, let's add a “kiwi” entry:
yq e '.fruits.kiwi = "brown"' -i fruits.ymlCode language: JavaScript (javascript)This command will insert a new key-value pair, resulting in:
fruits: apple: orange pear: green banana: yellow kiwi: brownCode language: YAML (yaml)Using Variables
yq allows you to use variables for more complex operations. For example, you can set a variable and then use it in an expression:
color="green" yq e '.fruits.banana = env(color)' -i fruits.ymlCode language: JavaScript (javascript)This command updates the “banana” key with the value of the color variable, resulting in “banana: green” in the YAML.
Taking it a step further, you can save the output of a command and assign it to an environment variable:
colorArray=("purple" "blue" "cyan") color=${colorArray[1]} yq e '.fruits.pear = env(color)' -i fruits.ymlCode language: JavaScript (javascript)This command assigns “blue” to the color variable and updates the “pear” value accordingly in the YAML.
What's next…
yq is a powerful tool for working with YAML files. It simplifies tasks like extracting specific values, updating YAML documents, adding new entries, and even working with variables. Whether you're a developer or system administrator, yq is a valuable addition to your toolbox for YAML manipulation. So, give it a try and streamline your YAML file handling today!
For more details and advanced use cases, check out the yq GitHub repository and its documentation.
If you encounter any issues or have questions, don't forget to refer to the official documentation or seek help from the yq community. Happy YAML parsing with yq!
