SLS Resolver Plugins

Idem SLS resolvers are very easy to write and can often work in just a few lines of code. They are implemented in a nested pop subsystem under idem, this means that you can vertical app-merge these plugins into idem if that makes the most sense.

The vast majority of the work to gather SLS files is completely generic, so adding new sources should be very easy. For instance, the code to load from the filesystem is just a few lines of code.

First create a directory at my_project_root/my_provider/source. In your project’s conf.py, extend Idem’s namespace with your source directory.

# my_project_root/my_provider/conf.py
DYNE = {"source": ["source"]}

Now create a plugin in your source directory.

The plugin simply needs to implement the cache async function. This function receives the hub, protocol, source, and loc. These will be passed into the function.

The protocol is usually just the name of the resolver plugin. Sometimes it will contain an additional protocol like git+https

The source is the root path to pull the file from. In the case of the file resolver this will be file:///path/to/sls/root.

The location or location is the desired file location relative to the source.

# my_project_root/my_provider/source/my_provider.py

# Generic python imports
from typing import ByteString
from typing import Tuple
import tempfile

try:
    # Import plugin-specific libraries here
    import my_provider.sdk

    HAS_LIBS = (True,)
except ImportError as e:
    HAS_LIBS = False, str(e)


def __virtual__(hub):
    # Perform other dependency checks as needed here I.E check for "git" or "fuse" installed via the OS
    return HAS_LIBS


# The virtualname is how the provider will appear on the hub,
# use this to avoid clashes with python module names and python keywords
__virtualname__ = "my_provider"


def __init__(hub):
    # This tells idem that the "my_provider" key in the acct file contains profiles for this plugin
    hub.source.my_provider.ACCT = ["my_provider"]


async def cache(
    hub, ctx, protocol: str, source: str, location: str
) -> Tuple[str, ByteString]:
    """
    Read data from my_provider
    :param hub:
    :param ctx: ctx.acct will contain the appropriate credentials to connect to my sls source's sdk
    :param source: The url/path/location of this sls source
    :param location: The path/key to access my SLS data relative to the source


    Define what credentials should be used in a profile for "my_provider"

        my_provider:
          my_profile:
            my_kwarg_1: my_val_1
            other_arbitrary_kwarg: arbitrary_value
    """
    # ctx.acct will have the credentials for the profile specified in the sls-sources/params-sources string
    data: bytes = my_provider.sdk.read(**ctx.acct)

    # There are two ways we can pass the sources on to idem
    if "option 1":
        # Store the data returned from the sdk in memory (preferred method)
        return f"{location}.sls", data
    elif "option 2":
        # Cache the data returned from the sdk in a temporary location
        with tempfile.NamedTemporaryFile(suffix=".sls", delete=True) as fh:
            fh.write(data)
            fh.flush()
            # Process the cached data like a traditional sls file source
            return await hub.source.file.cache(
                ctx=None, protocol="file", source=fh.name, location=location
            )

Your sls source can be invoked from the CLI like so:

$ idem state location --sls-sources "my_provider://my_profile@<source>" --params-sources "my_provider://my_profile@<source>"