jq
is a lightweight command line JSON processor that is very easy to use. Sometimes being able to use variables within a jq script is very useful. Below are various examples of doing this. Here is a sample record from the JSON file we use in most examples:
{
"id": 5679162,
"username": "ryderw",
"phone_number": "167-417-399",
"first_name": "Ryder",
"last_name": "Wong",
"middle_name": "Connor",
"sex": "MALE",
"birthdate": "2010-11-05",
"join_date": "2009-11-10T14:50:05.813-08:00",
"previous_logins": 5554,
"last_ip": "38.10.117.193"
}
Using Simple Variables in jq
To use a simple variable in jq use the --arg
option. Using --arg var value
will set the variable $var
to value
. For example, here we set the $dt
variable to add dates to our JSON records:
$ cat sample.json | jq --arg dt "2017-01-01" '{"first_name":.first_name, "last_name":.last_name, "date":$dt}'
{
"first_name": "Ryder",
"last_name": "Wong",
"date": "2017-01-01"
}
This variable can also be used to represent both a JSON field and value:
$ cat sample.json | jq --arg dt "2017-01-01" '{first_name, last_name, $dt}'
{
"first_name": "Ryder",
"last_name": "Wong",
"dt": "2017-01-01"
}
Using Environment Variables in jq
Environment variables can be used in jq in a couple of different ways shown below:
export city="San Francisco"
cat sample.json | jq --arg city "$city" '{"first_name":.first_name, "last_name":.last_name, "location":$city}'
{
"first_name": "Ryder",
"last_name": "Wong",
"location": "San Francisco"
}
In jq versions > 1.4 you can reference environment variables directory in your jq program using env.varname
:
export city="San Francisco"
cat sample.json | jq '{"first_name":.first_name, "last_name":.last_name, "location":env.city}'
{
"first_name": "Ryder",
"last_name": "Wong",
"location": "San Francisco"
}
Using JSON Variables in jq
If you want to use variables that reference JSON objects, this can be done with the --argsjson
option. Using --argjson var object
will set the variable $var
to object
. In the example below we set the $location
variable to a JSON object, and nest this object into our results.
cat sample.json \
| jq --argjson location "{\"city\":\"san francisco\",\"state\":\"california\"}" '{first_name: .first_name, location: $location}'
{
"first_name": "Ryder",
"location": {
"city": "san francisco",
"state": "california"
}
}
Using JSON Array Variables in jq
Using the --slurp
option you can store all JSON input records into an array. Using the --slurpfile
option you can store all JSON records from a file into an array, and reference this array using a variable.
Here is an example of reading all JSON records from sample.json
into $myArray
, and using this variable to build another JSON object.
jq -n --slurpfile myArray sample.json '{"document_type":"user data", "user_records":$myArray}'
{
"document_type": "user data",
"user_records": [
{
"id": 10406592,
"username": "sallison",
"email_address": "[email protected]",
"phone_number": "960-491-156",
"first_name": "Stella",
"last_name": "Allison",
"middle_name": "Amelia",
"sex": "FEMALE",
"birthdate": "2005-01-25",
"join_date": "2005-11-19T12:04:50.105-08:00",
"previous_logins": 9263,
"last_ip": "12.184.173.143"
},
......
jq is a great library and I highly recommend reading through the official documentation found here.