Using Variables in JQ (Command Line JSON Parser)

jq is a lightweight command line JSON processor that is very easy to use. Sometimes being able to use variables within a jq script is very useful. Below are various examples of doing this. Here is a sample record from the JSON file we use in most examples:

{
  "id": 5679162,
  "username": "ryderw",
  "phone_number": "167-417-399",
  "first_name": "Ryder",
  "last_name": "Wong",
  "middle_name": "Connor",
  "sex": "MALE",
  "birthdate": "2010-11-05",
  "join_date": "2009-11-10T14:50:05.813-08:00",
  "previous_logins": 5554,
  "last_ip": "38.10.117.193"
}

Using Simple Variables in jq

To use a simple variable in jq use the --arg option. Using --arg var value will set the variable $var to value. For example, here we set the $dt variable to add dates to our JSON records:

$ cat sample.json | jq --arg dt "2017-01-01" '{"first_name":.first_name, "last_name":.last_name, "date":$dt}'
{
  "first_name": "Ryder",
  "last_name": "Wong",
  "date": "2017-01-01"
}

This variable can also be used to represent both a JSON field and value:

$ cat sample.json | jq --arg dt "2017-01-01" '{first_name, last_name, $dt}'
{
  "first_name": "Ryder",
  "last_name": "Wong",
  "dt": "2017-01-01"
}

Using Environment Variables in jq

Environment variables can be used in jq in a couple of different ways shown below:

export city="San Francisco"
cat sample.json | jq --arg city "$city" '{"first_name":.first_name, "last_name":.last_name, "location":$city}'
{
  "first_name": "Ryder",
  "last_name": "Wong",
  "location": "San Francisco"
}

In jq versions > 1.4 you can reference environment variables directory in your jq program using env.varname:

export city="San Francisco"
cat sample.json | jq '{"first_name":.first_name, "last_name":.last_name, "location":env.city}'
{
  "first_name": "Ryder",
  "last_name": "Wong",
  "location": "San Francisco"
}

Using JSON Variables in jq

If you want to use variables that reference JSON objects, this can be done with the --argsjson option. Using --argjson var object will set the variable $var to object. In the example below we set the $location variable to a JSON object, and nest this object into our results.

cat sample.json \
  | jq --argjson location "{\"city\":\"san francisco\",\"state\":\"california\"}" '{first_name: .first_name, location: $location}'
{
  "first_name": "Ryder",
  "location": {
    "city": "san francisco",
    "state": "california"
  }
}

Using JSON Array Variables in jq

Using the --slurp option you can store all JSON input records into an array. Using the --slurpfile option you can store all JSON records from a file into an array, and reference this array using a variable.

Here is an example of reading all JSON records from sample.json into $myArray, and using this variable to build another JSON object.

jq -n --slurpfile myArray sample.json '{"document_type":"user data", "user_records":$myArray}'
{
  "document_type": "user data",
  "user_records": [
    {
      "id": 10406592,
      "username": "sallison",
      "email_address": "[email protected]",
      "phone_number": "960-491-156",
      "first_name": "Stella",
      "last_name": "Allison",
      "middle_name": "Amelia",
      "sex": "FEMALE",
      "birthdate": "2005-01-25",
      "join_date": "2005-11-19T12:04:50.105-08:00",
      "previous_logins": 9263,
      "last_ip": "12.184.173.143"
    },
......

jq is a great library and I highly recommend reading through the official documentation found here.

Leave a Reply

Using Variables in JQ (Command Line JSON Parser)