Quickstart with Swagger#

In this tutorial, you’ll learn how to generate an Idem plugin from a sample petstore Swagger specification.

Create a virtual environment#

Before you start, ensure that you installed Python 3.8 or later. If you are running 3.7 or earlier, you might need to use python3 instead of python in the commands in the rest of this tutorial.

To verify your Python version, run the following command:

python -V

Next, create your virtual environment:

python -m venv env
source env/bin/activate

Now you should be in your new Python virtual environment.

Update pip#

Next, update to the latest version of pip inside your virtual environment:

pip install -U pip

Install dependencies#

Next, you need to install idem and pop-create-idem:

pip install idem
pip install pop-create-idem

The above command installs the commands you need to create your plugin.

Create your plugin#

In this tutorial, you’ll use the project name “idem-petstore.”

Run the following command to create your plugin:

pop-create -n idem-petstore --cloud petstore --specification https://petstore.swagger.io/v2/swagger.json --directory /path/to/new/project --author "Petstore Inc."

This command creates an empty project with the directory structure needed for your Idem plugin:

.
├── CONTRIBUTING.rst
├── LICENSE
├── README.rst
├── build.conf
├── docs
│   ├── conf.py
│   ├── index.rst
│   ├── releases
│   │   ├── 0.1.0.rst
│   │   └── index.rst
│   ├── sitevars.rst
│   ├── topics
│   │   ├── contributing.rst
│   │   └── license.rst
│   └── tutorial
│       ├── example.rst
│       └── index.rst
├── example
│   ├── absent
│   │   ├── init.sls
│   │   ├── pet.sls
│   │   ├── store.sls
│   │   └── user.sls
│   └── present
│       ├── init.sls
│       ├── pet.sls
│       ├── store.sls
│       └── user.sls
├── idem-petstore
│   ├── __init__.py
│   ├── acct
│   │   ├── contracts
│   │   └── petstore
│   │       ├── api_key_auth.py
│   │       ├── basic_auth.py
│   │       ├── client_credentials_auth.py
│   │       └── default_auth.py
│   ├── autogen
│   │   └── petstore
│   │       └── templates
│   │           ├── exec
│   │           │   ├── create.jinja2
│   │           │   ├── delete.jinja2
│   │           │   ├── get.jinja2
│   │           │   ├── list.jinja2
│   │           │   └── update.jinja2
│   │           ├── state
│   │           │   ├── absent.jinja2
│   │           │   ├── describe.jinja2
│   │           │   └── present.jinja2
│   │           ├── tests
│   │           │   ├── exec
│   │           │   │   ├── test_create.jinja2
│   │           │   │   ├── test_delete.jinja2
│   │           │   │   ├── test_get.jinja2
│   │           │   │   ├── test_list.jinja2
│   │           │   │   └── test_update.jinja2
│   │           │   ├── states
│   │           │   │   ├── test_absent.jinja2
│   │           │   │   ├── test_describe.jinja2
│   │           │   │   └── test_present.jinja2
│   │           │   └── tool
│   │           │       └── default.jinja2
│   │           └── tool
│   │               └── default.jinja2
│   ├── cloudspec
│   │   ├── contracts
│   │   └── customize
│   │       └── petstore.py
│   ├── conf.py
│   ├── exec
│   │   ├── contracts
│   │   └── petstore
│   │       ├── init.py
│   │       ├── pet.py
│   │       ├── store.py
│   │       ├── user.py
│   │       └── v1_0_6
│   │           ├── init.py
│   │           └── recursive_contracts
│   │               └── init.py
│   ├── states
│   │   ├── contracts
│   │   └── petstore
│   │       ├── init.py
│   │       ├── pet.py
│   │       ├── store.py
│   │       └── user.py
│   ├── tool
│   │   ├── contracts
│   │   └── petstore
│   │       ├── pet.py
│   │       ├── session.py
│   │       ├── store.py
│   │       ├── test_state_utils.py
│   │       └── user.py
│   └── version.py
├── noxfile.py
├── requirements
│   ├── base.txt
│   ├── docs.txt
│   ├── py3.10
│   │   └── tests.txt
│   ├── py3.11
│   │   └── tests.txt
│   ├── py3.8
│   │   └── tests.txt
│   ├── py3.9
│   │   └── tests.txt
│   └── tests.in
├── setup.cfg
├── setup.py
└── tests
    ├── integration
    │   ├── conftest.py
    │   ├── exec
    │   │   ├── __init__.py
    │   │   ├── test_pet.py
    │   │   ├── test_store.py
    │   │   └── test_user.py
    │   ├── states
    │   │   ├── __init__.py
    │   │   ├── test_pet.py
    │   │   ├── test_store.py
    │   │   └── test_user.py
    │   └── tool
    │       ├── __init__.py
    │       ├── test_pet.py
    │       ├── test_store.py
    │       └── test_user.py
    └── unit
        └── conftest.py

Acct module#

An acct plugin manages authentication with your provider. Several default authentication modules are created for you:

.
├── api_key_auth.py
├── basic_auth.py
├── client_credentials_auth.py
└── default_auth.py

You can use or rewrite the modules to satisfy your provider’s authentication requirements.

Exec module#

An exec module provides imperative functions to interact with your API. These functions take actions such as creating, reading, updating or deleting resources on the API.

pop-create identifies available resources and generates exec modules for get, list, create, update, or delete functions on those resources. The following are the identified resources and their exec modules for the petstore Swagger specification:

exec
├── petstore
    ├── init.py
    ├── pet.py
    ├── store.py
    └── user.py

State modules#

A state module provides idempotent functions to configure your provider’s resource. State modules generally use functions from their respective exec modules to check for the current status of the resources and make any necessary changes.

pop-create creates state modules for all identified resources. The following are the identified resources and their state modules for the petstore Swagger specification:

states
├── petstore
    ├── init.py
    ├── pet.py
    ├── store.py
    └── user.py

Tool module#

A tool module contains functions that you wish to keep internal to your plugin and not expose to the command line. pop-create auto-generates these modules for identified resources and functions that do not get, list, update, create, or delete resources:

tool
├── petstore
    ├── pet.py
    ├── session.py
    ├── store.py
    ├── test_state_utils.py
    └── user.py

The tool module also contains several utility methods, such as a generic HTTP request module (session.py), which is referenced in all exec modules. There is another utility helper, test_state_utils, which is used for creating state outputs when the –test option is used in Idem commands.

Autogen module#

pop-create provides default templates that are used when creating exec, states, and tool module functions under the autogen module. These templates can be customized to generate custom function definitions. The following are the default templates for each respective functions in each respective module:

autogen
└── petstore
    └── templates
        ├── exec
            ├── create.jinja2
            ├── delete.jinja2
            ├── get.jinja2
            ├── list.jinja2
            └── update.jinja2
        ├── state
            ├── absent.jinja2
            ├── describe.jinja2
            └── present.jinja2
        ├── tests
            ├── exec
                    ├── test_create.jinja2
                    ├── test_delete.jinja2
                    ├── test_get.jinja2
                    ├── test_list.jinja2
                    └── test_update.jinja2
            ├── states
                    ├── test_absent.jinja2
                    ├── test_describe.jinja2
                    └── test_present.jinja2
            └── tool
                └── default.jinja2
        └── tool
            └── default.jinja2

CloudSpec module#

pop-create provides a way to customize a generated cloudspec from Swagger or OpenAPI specifications. If customized, the specification will be used for creating exec, states, and tool module functions.

Test module#

pop-create auto-generates an integration test for each function identified for each resource in the exec, states, and tool modules. The following are the integration tests for the petstore Swagger specification:

.
├── conftest.py
├── exec
    ├── __init__.py
    ├── test_pet.py
    ├── test_store.py
    └── test_user.py
├── states
    ├── __init__.py
    ├── test_pet.py
    ├── test_store.py
    └── test_user.py
└── tool
    ├── __init__.py
    ├── test_pet.py
    ├── test_store.py
    └── test_user.py

Example module#

pop-create auto-generates sample SLS files for present and absent functions in state modules. The following are sample SLS files for the petstore Swagger specification:

example
├── absent
   │── init.sls
   ├── pet.sls
   ├── store.sls
   └── user.sls
└── present
    ├── init.sls
    ├── pet.sls
    ├── store.sls
    └── user.sls

Plugin docs#

pop-create configures automation pipelines to generate user documentation with Sphinx.

.
├── CONTRIBUTING.rst
├── LICENSE
├── README.rst
├── docs
│   ├── _includes
│   │   ├── modindex-note.rst
│   │   ├── reference-toc-template.rst
│   │   └── reference-toc.rst
│   ├── conf.py
│   ├── index.rst
│   ├── py-modindex.rst
│   ├── ref
          ...
│   ├── sitevars.rst
│   ├── topics
│   │   ├── contributing.rst
│   │   └── license.rst
│   └── tutorial
│       ├── example.rst
│       └── index.rst
├── noxfile.py
.gitlab-ci.yaml
conf.py
.

Several manual steps are required to complete documentation setup: * Add a plugin icon * Configure GitLab URL * Specify which S3 bucket to publish generated HTML pages * Address all TODOs in the generated code and user docs

Note: Look for TODO in the code to complete configurations.

Todo

(Set todo_include_todos = False in docs/conf.py to hide rst todos)

RST files will have todos if there are any manual changes required.

Next, you can install nox with pip and run the following command to generate HTML pages for your plugin user docs.

pip install nox
nox -e 'docs-html(clean=True)'

You can now view the generated HTML pages with your browser at /path/to/new/project/docs/_build/html/index.html.

Next steps#