Filtering with JMESpath#
An idem describe operation is able to filter its results using a tool called JMESpath. JMESpath is a query language for JSON.
When the --filter option is used with idem describe,
the sls data gets changed into a format that is easy to use with JMESpath.
For example, a traditional SLS state in JSON format looks like this:
{
"Description of test.succeed_with_comment": {
"test.succeed_with_comment": [
{"name": "succeed_with_comment"},
{"comment": None},
]
},
}
When performing a JMESpath search on the data, it first gets transformed to look like this:
[
{
"name": "Description of test.succeed_with_comment",
"ref": "test.succeed_with_comment",
"resource": [{"name": "succeed_with_comment"}, {"comment": None}],
},
]
The data has been flattened into an list of dictionaries and the keys name, ref, and resource have been added for easy filtering.
Don’t worry, the end result is turned back into the sls form unless you supply the --output=jmespath flag.
You can always run idem describe --output=jmespath without --filter to see what the internal jmespath structure looks like.
Practicing with Static Data#
Gathering data from the cloud can take a long time. When you are learning how to write JMESpaths, try writing a small script like this one to practice on static data:
# my_filter.py
import jmespath
import pprint
import sys
# In this example, "data" is the output of "idem describe test --output=jmespath"
data = [
{
"name": "Description of test.anop",
"resource": [{"name": "anop"}],
"ref": "test.anop",
},
{
"name": "Description of test.configurable_test_state",
"resource": [
{"name": "configurable_test_state"},
{"changes": True},
{"result": True},
{"comment": ""},
],
"ref": "test.configurable_test_state",
},
{"name": "Description of test.describe", "resource": [], "ref": "test.describe"},
{
"name": "Description of test.fail_with_changes",
"resource": [{"name": "fail_with_changes"}],
"ref": "test.fail_with_changes",
},
{
"name": "Description of test.fail_without_changes",
"resource": [{"name": "fail_without_changes"}],
"ref": "test.fail_without_changes",
},
{
"name": "Description of test.mod_watch",
"resource": [{"name": "mod_watch"}],
"ref": "test.mod_watch",
},
{
"name": "Description of test.none_without_changes",
"resource": [{"name": "none_without_changes"}],
"ref": "test.none_without_changes",
},
{
"name": "Description of test.nop",
"resource": [{"name": "nop"}],
"ref": "test.nop",
},
{
"name": "Description of test.succeed_with_changes",
"resource": [{"name": "succeed_with_changes"}],
"ref": "test.succeed_with_changes",
},
{
"name": "Description of test.succeed_with_comment",
"resource": [{"name": "succeed_with_comment"}, {"comment": None}],
"ref": "test.succeed_with_comment",
},
{
"name": "Description of test.succeed_without_changes",
"resource": [{"name": "succeed_without_changes"}],
"ref": "test.succeed_without_changes",
},
{
"name": "Description of test.treq",
"resource": [{"name": "treq"}],
"ref": "test.treq",
},
{
"name": "Description of test.update_low",
"resource": [{"name": "update_low"}],
"ref": "test.update_low",
},
]
search_path = sys.argv[1]
pprint.pprint(jmespath.search(search_path, data))
Examples#
Now for some examples of filtering with JMESpath.
I will use the format of idem describe test --filter="<JMESpath>" in the following examples.
If you called the little script we wrote above my_filter.py then the following two commands are equivalent.
Keep that in mind as you move your one-off experiments to idem describe:
# Equivalent commands
# my_filter.py "<JMESpath>"
$ idem describe test --output=pretty --filter="<JMESpath>"
Return only the states that use test.update_low
$ idem describe test --filter="[?ref=='test.update_low']"
Output:
Description of test.update_low:
test.update_low:
- name: update_low
Return only the states that start with test.succeed
$ idem describe test --filter="[?starts_with(ref, 'test.succeed']"
Output:
Description of test.succeed_with_changes:
test.succeed_with_changes:
- name: succeed_with_changes
Description of test.succeed_with_comment:
test.succeed_with_comment:
- name: succeed_with_comment
- comment: null
Description of test.succeed_without_changes:
test.succeed_without_changes:
- name: succeed_without_changes
Return only tests that have changes in the state name:
$ idem describe test --filter="[?contains(name, 'changes')]"
Output:
Description of test.fail_with_changes:
test.fail_with_changes:
- name: fail_with_changes
Description of test.fail_without_changes:
test.fail_without_changes:
- name: fail_without_changes
Description of test.none_without_changes:
test.none_without_changes:
- name: none_without_changes
Description of test.succeed_with_changes:
test.succeed_with_changes:
- name: succeed_with_changes
Description of test.succeed_without_changes:
test.succeed_without_changes:
- name: succeed_without_changes
Return only states that have succeed_with_comment in the name parameter
$ idem describe test --filter="[?resource[?name=='succeed_with_comment']]"
Output:
Description of test.succeed_with_comment:
test.succeed_with_comment:
- name: succeed_with_comment
- comment: null
Learn More#
https://jmespath.org/tutorial.html
https://jmespath.org/examples.html
https://jmespath.org/specification.html
https://pypi.org/project/jmespath
https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_jmespath.html