Example: Creating and maintaining resources with Idem#

In this example, Idem creates AWS resources from scratch and maintains them in their original state.

Use this workflow as a guideline for greenfield resource deployment and management with Idem.

Prerequisites#

This example assumes that the user:

Create state files#

At the heart of Idem are state (SLS) files that describe what to deploy. In this example, a state file creates a virtual private cloud (VPC) for a virtual machine based on a found Amazon machine image, and connects it to a subnet.

aws-demo.sls contents:

vpc-idem-test:
  aws.ec2.vpc.present:
    - cidr_block_association_set:
        - CidrBlock: '172.32.0.0/16'
    - instance_tenancy: 'default'
    - tags:
        Name: 'vpc-idem-demo'
        Environment: 'Development'

subnet-idem-test:
  aws.ec2.subnet.present:
    - cidr_block: '172.32.16.0/20'
    - vpc_id: ${aws.ec2.vpc:vpc-idem-test:resource_id}
    - availability_zone: us-west-2b
    - tags:
        Name: 'subnet-idem-demo'
        Description: 'Subnet created for VPC ${aws.ec2.vpc:vpc-idem-test:name}.
                  VPC CIDR ${aws.ec2.vpc:vpc-idem-test:cidr_block_association_set[0]:CidrBlock}
                  association ID is ${aws.ec2.vpc:vpc-idem-test:cidr_block_association_set[0]:CidrBlock}'

test_ami:
  exec.run:
    - path: aws.ec2.ami.get
    - kwargs:
        name: test_ami
        most_recent: true
        owners:
          - amazon
        filters:
            - name: 'description'
              values: ["{{ params['ami_desc'] }}"]
            - name: 'architecture'
              values: ["x86_64"]

instance-idem-test:
  aws.ec2.instance.present:
    - image_id: ${exec:test_ami:resource_id}
    - instance_type: 't1.micro'
    - subnet_id: ${aws.ec2.subnet:subnet-idem-test:resource_id}
    - tags:
        Name: 'instance-idem-demo'

There’s also a secondary state file that supplies a parameter of which AWS virtual machine image to search for and use. Parameter files are how you create and apply variables in Idem.

params.sls contents:

ami_desc: "Amazon Linux 2*"

Note that the main state file calls the parameter file:

filters:
    - name: 'description'
      values: ["{{ params['ami_desc'] }}"]

Idem state files are plain text and have the .sls extension.

Deploy the resources#

From the Idem command line, create the deployment with an idem state command.

idem state aws-demo.sls --params=params.sls

--------
      ID: test_ami
Function: exec.run
  Result: True
 Comment: ["idem exec aws.ec2.ami.get --acct-profile=default name=test_ami most_recent=True owners=['amazon'] filters=[{'name': 'description', 'values': ['Amazon Linux 2*']}, {'name': 'architecture', 'values': ['x86_64']}]"]
 Changes:
new:
    ----------
    resource_id:
        ami-0e422fc2f52a735de
    name:
        amzn2-ami-minimal-selinux-enforcing-hvm-2.0.20221103.3-x86_64-gp2
    creation_date:
        2022-11-14T23:23:35.000Z
    image_location:
        amazon/amzn2-ami-minimal-selinux-enforcing-hvm-2.0.20221103.3-x86_64-gp2
    architecture:
        x86_64
    block_device_mappings:
        |_
          ----------
          DeviceName:
              /dev/xvda
          Ebs:
              ----------
              DeleteOnTermination:
                  True
              SnapshotId:
                  snap-01893d46916277792
              VolumeSize:
                  2
              VolumeType:
                  gp2
              Encrypted:
                  False
    description:
        Amazon Linux 2 SELinux Enforcing AMI 2.0.20221103.3 x86_64 Minimal HVM gp2
    ena_support:
        True
    root_device_name:
        /dev/xvda
    sriov_net_support:
        simple
    virtualization_type:
        hvm
--------
      ID: subnet-idem-test
Function: aws.ec2.subnet.present
  Result: True
 Comment: ("Created aws.ec2.subnet 'subnet-idem-test'",)
 Changes:
new:
    ----------
    name:
        subnet-idem-test
    resource_id:
        subnet-0b86ef7a9e67167eb
    vpc_id:
        vpc-0afa8340b0a167d54
    cidr_block:
        172.32.16.0/20
    availability_zone:
        us-west-2b
    map_public_ip_on_launch:
        False
    assign_ipv6_address_on_creation:
        False
    map_customer_owned_ip_on_launch:
        False
    enable_dns_64:
        False
    private_dns_name_options_on_launch:
        ----------
        HostnameType:
            ip-name
        EnableResourceNameDnsARecord:
            False
        EnableResourceNameDnsAAAARecord:
            False
    tags:
        ----------
        Name:
            subnet-idem-demo
        Description:
            Subnet created for VPC vpc-idem-test. VPC CIDR 172.32.0.0/16 association ID is 172.32.0.0/16
--------
      ID: instance-idem-test
Function: aws.ec2.instance.present
  Result: True
 Comment: ["Created 'instance-idem-test'"]
 Changes:
new:
    ----------
    name:
        i-0e5716c94c7265b59
    resource_id:
        i-0e5716c94c7265b59
    image_id:
        ami-0e422fc2f52a735de
    instance_type:
        t1.micro
    volume_attachments:
        ----------
    block_device_mappings:
    ebs_optimized:
        False
    subnet_id:
        subnet-0b86ef7a9e67167eb
    network_interfaces:
        |_
          ----------
          AssociatePublicIpAddress:
              False
          DeleteOnTermination:
              True
          Description:
          DeviceIndex:
              0
          Groups:
              - sg-0d81ee3d37de684b4
          InterfaceType:
              interface
          Ipv6Addresses:
          PrivateIpAddresses:
              |_
                ----------
                Primary:
                    True
                PrivateIpAddress:
                    172.32.18.202
          SubnetId:
              subnet-0b86ef7a9e67167eb
          NetworkCardIndex:
              0
    monitoring_enabled:
        False
    root_device_name:
        /dev/xvda
    client_token:
        28ffddd8-9ee7-451f-ba60-dbf52c26ac8e
    product_codes:
    source_dest_check:
        True
    running:
        False
    private_ip_address:
        172.32.18.202
    reservation_id:
        r-0451ffb9816298068
    owner_id:
        840258433862
    availability_zone:
        us-west-2b
    group_name:
    tenancy:
        default
    disable_api_termination:
        False
    tags:
        ----------
        Name:
            instance-idem-demo
    iam_profile_arn:
        ----------
    instance_initiated_shutdown_behavior:
        stop
    auto_recovery_enabled:
        True
    sriov_net_support:
        simple
    nitro_enclave_enabled:
        False
    license_arns:
    hibernation_enabled:
        False
    valid_until:
    http_tokens:
        optional
    http_put_response_hop_limit:
        1
    http_endpoint_enabled:
        True
    http_protocol_ipv6_enabled:
        False
    metadata_tags_enabled:
        False
    hostname_type:
        ip-name
    enable_resource_name_dns_a_record:
        False
    enable_resource_name_dns_aaaa_record:
        False
    capacity_reservation_preference:
        open
    bootstrap:
--------
      ID: vpc-idem-test
Function: aws.ec2.vpc.present
  Result: True
 Comment: ("Created aws.ec2.vpc 'vpc-idem-test'",)
 Changes:
new:
    ----------
    name:
        vpc-idem-test
    resource_id:
        vpc-0afa8340b0a167d54
    instance_tenancy:
        default
    tags:
        ----------
        Name:
            vpc-idem-demo
        Environment:
            Development
    cidr_block_association_set:
        |_
          ----------
          AssociationId:
              vpc-cidr-assoc-0686a721abfb9a64c
          CidrBlock:
              172.32.0.0/16
          CidrBlockState:
              ----------
              State:
                  associated
    enable_dns_hostnames:
        False
    enable_dns_support:
        True

run: 1 successful
present: 3 created successfully

The command output reports that the search for the machine image was successful. In addition, the VPC, subnet, and virtual machine were successfully created.

Verify the deployment#

In AWS, inspect the VPC, subnet, and virtual machine.

VPC deployed in AWS:

../../_images/greenfield-aws.png

Verify the resource states#

Check that the state file still accurately describes the deployment. From the Idem command line, rerun the idem state command with the addition of the --test flag.

idem state aws-demo.sls --params=params.sls --test

--------
      ID: vpc-idem-test
Function: aws.ec2.vpc.present
  Result: True
 Comment: ()
 Changes:

--------
      ID: test_ami
Function: exec.run
  Result: True
 Comment: ["idem exec aws.ec2.ami.get --acct-profile=default name=test_ami most_recent=True owners=['amazon'] filters=[{'name': 'description', 'values': ['Amazon Linux 2*']}, {'name': 'architecture', 'values': ['x86_64']}]"]
 Changes:
new:
    ----------
    resource_id:
        ami-0e422fc2f52a735de
    name:
        amzn2-ami-minimal-selinux-enforcing-hvm-2.0.20221103.3-x86_64-gp2
    creation_date:
        2022-11-14T23:23:35.000Z
    image_location:
        amazon/amzn2-ami-minimal-selinux-enforcing-hvm-2.0.20221103.3-x86_64-gp2
    architecture:
        x86_64
    block_device_mappings:
        |_
          ----------
          DeviceName:
              /dev/xvda
          Ebs:
              ----------
              DeleteOnTermination:
                  True
              SnapshotId:
                  snap-01893d46916277792
              VolumeSize:
                  2
              VolumeType:
                  gp2
              Encrypted:
                  False
    description:
        Amazon Linux 2 SELinux Enforcing AMI 2.0.20221103.3 x86_64 Minimal HVM gp2
    ena_support:
        True
    root_device_name:
        /dev/xvda
    sriov_net_support:
        simple
    virtualization_type:
        hvm
--------
      ID: subnet-idem-test
Function: aws.ec2.subnet.present
  Result: True
 Comment: ()
 Changes:

--------
      ID: instance-idem-test
Function: aws.ec2.instance.present
  Result: True
 Comment: ()
 Changes:

present: 3 no-op
run: 1 successful

The command output reports that the VPC, subnet, and virtual machine are unchanged on AWS and that a search for the machine image is still successful.

Without the --test flag, if there were a mismatch, Idem would have modified the resources to match the state file.

Change the existing deployment#

From AWS, change the VPC tags. Use the Manage tags option to change Development to Test.

This example only changes a simple tag, but a dynamic environment might have many changes that Idem needs to address.

../../_images/greenfield-tag-change.png

Correct the deployment drift#

From the Idem command line, use the same idem state command to bring the deployment back into compliance.

Consider running the command twice, first with the --test flag so that you can review what will change, then without --test to actually make those changes.

idem state aws-demo.sls --params=params.sls

--------
      ID: test_ami
Function: exec.run
  Result: True
 Comment: ["idem exec aws.ec2.ami.get --acct-profile=default name=test_ami most_recent=True owners=['amazon'] filters=[{'name': 'description', 'values': ['Amazon Linux 2*']}, {'name': 'architecture', 'values': ['x86_64']}]"]
 Changes:
new:
    ----------
    resource_id:
        ami-0e422fc2f52a735de
    name:
        amzn2-ami-minimal-selinux-enforcing-hvm-2.0.20221103.3-x86_64-gp2
    creation_date:
        2022-11-14T23:23:35.000Z
    image_location:
        amazon/amzn2-ami-minimal-selinux-enforcing-hvm-2.0.20221103.3-x86_64-gp2
    architecture:
        x86_64
    block_device_mappings:
        |_
          ----------
          DeviceName:
              /dev/xvda
          Ebs:
              ----------
              DeleteOnTermination:
                  True
              SnapshotId:
                  snap-01893d46916277792
              VolumeSize:
                  2
              VolumeType:
                  gp2
              Encrypted:
                  False
    description:
        Amazon Linux 2 SELinux Enforcing AMI 2.0.20221103.3 x86_64 Minimal HVM gp2
    ena_support:
        True
    root_device_name:
        /dev/xvda
    sriov_net_support:
        simple
    virtualization_type:
        hvm
--------
      ID: subnet-idem-test
Function: aws.ec2.subnet.present
  Result: True
 Comment: ()
 Changes:

--------
      ID: instance-idem-test
Function: aws.ec2.instance.present
  Result: True
 Comment: ()
 Changes:

--------
      ID: vpc-idem-test
Function: aws.ec2.vpc.present
  Result: True
 Comment: ("Update tags: Add keys dict_keys(['Environment']) Remove keys dict_keys(['Environment'])",)
 Changes:
old:
    ----------
    tags:
        ----------
        Environment:
            Test
new:
    ----------
    tags:
        ----------
        Environment:
            Development


run: 1 successful
present: 1 updated successfully
present: 2 no-op

The output reports the change back to Development on the VPC. The rest of the deployment was already in compliance.

To verify the restoration, you can inspect the instance in AWS or run an Idem describe command.

../../_images/greenfield-tag-restore.png