Source code for idem_aws.states.aws.efs.file_system

"""State module for managing Amazon File System."""
import copy
from dataclasses import field
from dataclasses import make_dataclass
from typing import Any
from typing import Dict
from typing import List

__contracts__ = ["resource"]
__reconcile_wait__ = {"static": {"wait_in_seconds": 5}}


[docs]async def present( hub, ctx, name: str, creation_token: str, resource_id: str = None, performance_mode: str = None, encrypted: bool = None, kms_key_id: str = None, throughput_mode: str = None, provisioned_throughput_in_mibps: float = None, availability_zone_name: str = None, backup: bool = None, tags: Dict[str, Any] or List[ make_dataclass("Tag", [("Key", str), ("Value", str, field(default=None))]) ] = None, ) -> Dict[str, Any]: """Creates a new, empty file system. The operation requires a creation token in the request that Amazon EFS uses to ensure idempotent creation (calling the operation with same creation token has no effect). If a file system does not currently exist that is owned by the caller's Amazon Web Services account with the specified creation token, this operation does the following: * Creates a new, empty file system. The file system will have an Amazon EFS assigned ID, and an initial lifecycle state creating. * Returns with the description of the created file system. Otherwise, this operation returns a FileSystemAlreadyExists error with the ID of the existing file system. .. Note:: For basic use cases, you can use a randomly generated UUID for the creation token. The idempotent operation allows you to retry a CreateFileSystem call without risk of creating an extra file system. This can happen when an initial call fails in a way that leaves it uncertain whether or not a file system was actually created. An example might be that a transport level timeout occurred or your connection was reset. As long as you use the same creation token, if the initial call had succeeded in creating a file system, the client can learn of its existence from the FileSystemAlreadyExists error. For more information, see Creating a file system in the Amazon EFS User Guide. .. Note:: The CreateFileSystem call returns while the file system's lifecycle state is still creating. You can check the file system creation status by calling the DescribeFileSystems operation, which among other things returns the file system state. This operation accepts an optional PerformanceMode parameter that you choose for your file system. We recommend generalPurpose performance mode for most file systems. File systems using the maxIO performance mode can scale to higher levels of aggregate throughput and operations per second with a tradeoff of slightly higher latencies for most file operations. The performance mode can't be changed after the file system has been created. For more information, see Amazon EFS performance modes. You can set the throughput mode for the file system using the ThroughputMode parameter. After the file system is fully created, Amazon EFS sets its lifecycle state to available, at which point you can create one or more mount targets for the file system in your VPC. For more information, see CreateMountTarget. You mount your Amazon EFS file system on an EC2 instances in your VPC by using the mount target. For more information, see Amazon EFS: How it Works. This operation requires permissions for the elasticfilesystem:CreateFileSystem action. Args: name(str): An Idem name of the resource. creation_token(str): A string of up to 64 ASCII characters. Amazon EFS uses this to ensure idempotent creation. resource_id(str, Optional): EFS file system ID. performance_mode(str, Optional): The performance mode of the file system. We recommend generalPurpose performance mode for most file systems. File systems using the maxIO performance mode can scale to higher levels of aggregate throughput and operations per second with a tradeoff of slightly higher latencies for most file operations. The performance mode can't be changed after the file system has been created. .. Note:: The maxIO mode is not supported on file systems using One Zone storage classes. Defaults to None. encrypted(bool, Optional): A Boolean value that, if true, creates an encrypted file system. When creating an encrypted file system, you have the option of specifying an existing Key Management Service key (KMS key). If you don't specify a KMS key, then the default KMS key for Amazon EFS, /aws/elasticfilesystem, is used to protect the encrypted file system. Defaults to None. kms_key_id(str, Optional): The ID of the KMS key that you want to use to protect the encrypted file system. This parameter is only required if you want to use a non-default KMS key. If this parameter is not specified, the default KMS key for Amazon EFS is used. You can specify a KMS key ID using the following formats: * Key ID - A unique identifier of the key, for example 1234abcd-12ab-34cd-56ef-1234567890ab. * ARN - An Amazon Resource Name (ARN) for the key, for example arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab. * Key alias - A previously created display name for a key, for example alias/projectKey1. * Key alias ARN - An ARN for a key alias, for example arn:aws:kms:us-west-2:444455556666:alias/projectKey1. If you use KmsKeyId, you must set the CreateFileSystemRequest$Encrypted parameter to true. .. Warning:: EFS accepts only symmetric KMS keys. You cannot use asymmetric KMS keys with Amazon EFS file systems. Defaults to None. throughput_mode(str, Optional): Specifies the throughput mode for the file system, either bursting or provisioned. If you set ThroughputMode to provisioned, you must also set a value for ProvisionedThroughputInMibps. After you create the file system, you can decrease your file system's throughput in Provisioned Throughput mode or change between the throughput modes, as long as it’s been more than 24 hours since the last decrease or throughput mode change. For more information, see Specifying throughput with provisioned mode in the Amazon EFS User Guide. Default is bursting. Defaults to None. provisioned_throughput_in_mibps(float, Optional): The throughput, measured in MiB/s, that you want to provision for a file system that you're creating. Valid values are 1-1024. Required if ThroughputMode is set to provisioned. The upper limit for throughput is 1024 MiB/s. To increase this limit, contact Amazon Web Services Support. For more information, see Amazon EFS quotas that you can increase in the Amazon EFS User Guide. Defaults to None. availability_zone_name(str, Optional): Used to create a file system that uses One Zone storage classes. It specifies the Amazon Web Services Availability Zone in which to create the file system. Use the format us-east-1a to specify the Availability Zone. For more information about One Zone storage classes, see Using EFS storage classes in the Amazon EFS User Guide. .. Note:: One Zone storage classes are not available in all Availability Zones in Amazon Web Services Regions where Amazon EFS is available. Defaults to None. backup(bool, Optional): Specifies whether automatic backups are enabled on the file system that you are creating. Set the value to true to enable automatic backups. If you are creating a file system that uses One Zone storage classes, automatic backups are enabled by default. For more information, see Automatic backups in the Amazon EFS User Guide. Default is false. However, if you specify an AvailabilityZoneName, the default is true. .. Note:: Backup is not available in all Amazon Web Services Regions where Amazon EFS is available. Defaults to None. tags(dict or list, Optional): Dict in the format of {tag-key: tag-value} or List of tags in the format of [{"Key": tag-key, "Value": tag-value}] to associate with the file system. Each tag consists of a key name and an associated value. Defaults to None. * Key (str, Optional): The key of the tag. Constraints: Tag keys are case-sensitive and accept a maximum of 127 Unicode characters. May not begin with aws:. * Value(str, Optional): The value of the tag. Constraints: Tag values are case-sensitive and accept a maximum of 256 Unicode characters. Request Syntax: .. code-block:: sls [file-system-resource-id]: aws.efs.file_system.present: - name: "string" - resource_id: "string" - performance_mode: generalPurpose - encrypted: "boolean" - kms_key_id: "string" - creation_token: "string" - throughput_mode: bursting/provisioned - provisioned_throughput_in_mibps: "float" - availability_zone_name: "string" - backup: "boolean" - tags: - "string": "string" Returns: Dict[str, Any] Examples: .. code-block:: sls fs-xxxxxxxxx: aws.efs.file_system.present: - name: efs-name - resource_id: fs-xxxxxxxxx - performance_mode: generalPurpose - encrypted: true - kms_key_id: arn:aws:kms:us-west-1:5xxxxxx:key/597f1c02-94dc-4c88-9088-f9aca5e39cf7 - creation_token: xxx - throughput_mode: bursting - provisioned_throughput_in_mibps: 500.0 - availability_zone_name: us-west-1b - backup: true - tags: Name: efs-name """ result = dict(comment=[], old_state=None, new_state=None, name=name, result=True) # Query for filesystem file_systems = None if resource_id: file_systems = await hub.exec.boto3.client.efs.describe_file_systems( ctx, FileSystemId=resource_id ) if not file_systems["result"]: result["comment"] = file_systems["comment"] result["result"] = file_systems["result"] return result # Check for existing filesystem before = None if file_systems and file_systems["ret"]["FileSystems"]: before = file_systems["ret"]["FileSystems"][0] if isinstance(tags, List): tags = hub.tool.aws.tag_utils.convert_tag_list_to_dict(tags) backup_enabled = False if before: backup_result = await hub.exec.boto3.client.efs.describe_backup_policy( ctx, FileSystemId=resource_id, ) if not backup_result[ "result" ] and not hub.tool.aws.efs.file_system_utils.is_backup_policy_not_found( backup_result=backup_result ): hub.log.warning( f"Could not describe backup policy for file system {resource_id} : {backup_result['comment']}" ) result["comment"] = backup_result["comment"] result["result"] = False return result backup_enabled = hub.tool.aws.efs.file_system_utils.is_backup_enabled( backup_result=backup_result ) # Update current state current_state = ( hub.tool.aws.efs.file_system_utils.convert_raw_file_system_to_present( file_system=before, backup_enabled=backup_enabled ) ) result["old_state"] = current_state # Handle No change behaviour desired_state = { "name": name, "resource_id": resource_id, "performance_mode": performance_mode, "encrypted": encrypted, "kms_key_id": kms_key_id, "creation_token": creation_token, "throughput_mode": throughput_mode, "provisioned_throughput_in_mibps": provisioned_throughput_in_mibps, "availability_zone_name": availability_zone_name, "backup": backup, "tags": tags, } if current_state: is_throughput_mode_updated = ( desired_state.get("throughput_mode") is not None ) and hub.tool.aws.efs.file_system_utils.is_throughput_mode_updated( current_state=current_state, desired_state=desired_state ) is_tags_updated = (tags is not None) and (current_state.get("tags")) != tags is_backup_policy_updated = ( desired_state.get("backup") is not None ) and current_state.get("backup") != desired_state.get("backup") is_change_detected = ( before is None or is_throughput_mode_updated or is_backup_policy_updated or is_tags_updated ) if not is_change_detected: result["comment"] += [f"aws.efs.file_system '{name}' already exists"] result["new_state"] = copy.deepcopy(result["old_state"]) return result # Handle test behaviour if ctx.get("test", False): result["new_state"] = hub.tool.aws.test_state_utils.generate_test_state( enforced_state=current_state, desired_state=desired_state, ) result["comment"] = ( hub.tool.aws.comment_utils.would_update_comment( resource_type="aws.efs.file_system", name=name ) if before else hub.tool.aws.comment_utils.would_create_comment( resource_type="aws.efs.file_system", name=name ) ) return result # Handle actual resource create or update if before: # Update ThroughputMode and ProvisionedThroughputInMibps if is_throughput_mode_updated: ret = await hub.exec.boto3.client.efs.update_file_system( ctx, FileSystemId=resource_id, ThroughputMode=throughput_mode, ProvisionedThroughputInMibps=provisioned_throughput_in_mibps if throughput_mode == "provisioned" else None, ) result["result"] = ret["result"] result["comment"] += [ f"Updated ThroughputMode for aws.efs.file_system '{name}'" ] if not result["result"]: result["comment"] = ret["comment"] return result # Update backup policy if is_backup_policy_updated: ret = await hub.exec.boto3.client.efs.put_backup_policy( ctx, FileSystemId=resource_id, BackupPolicy={"Status": "ENABLED" if backup else "DISABLED"}, ) result["result"] = result["result"] and ret["result"] result["comment"] += [ f"Updated Backup policy for aws.efs.file_system '{name}'" ] if not result["result"]: result["comment"] = ret["comment"] return result # Update tags if is_tags_updated: update_tags = await hub.tool.aws.efs.tag.update_tags( ctx, resource_id=resource_id, old_tags=current_state["tags"], new_tags=desired_state["tags"], ) result["comment"] += update_tags["comment"] result["result"] = result["result"] and update_tags["result"] if not result["result"]: return result else: ret = await hub.exec.boto3.client.efs.create_file_system( ctx, **{ "CreationToken": creation_token, "PerformanceMode": performance_mode, "Encrypted": encrypted, "KmsKeyId": kms_key_id, "ThroughputMode": throughput_mode, "ProvisionedThroughputInMibps": provisioned_throughput_in_mibps if throughput_mode == "provisioned" else None, "AvailabilityZoneName": availability_zone_name, "Backup": backup, "Tags": hub.tool.aws.tag_utils.convert_tag_dict_to_list(tags) if tags else None, }, ) result["result"] = ret["result"] if not result["result"]: result["comment"] = ret["comment"] return result result["comment"] = hub.tool.aws.comment_utils.create_comment( resource_type="aws.efs.file_system", name=name ) resource_id = ret["ret"]["FileSystemId"] # This makes sure the created file is saved into esm regardless of subsequent updates fails, or it is thrown # into reconciliation loop because the file system is not available yet. result["new_state"] = dict(name=name, resource_id=resource_id) # Fetch the updated resource and update new_state file_systems = await hub.exec.boto3.client.efs.describe_file_systems( ctx, FileSystemId=resource_id ) if not file_systems["result"]: result["comment"] += file_systems["comment"] result["result"] = file_systems["result"] return result after = file_systems["ret"]["FileSystems"][0] if "LifeCycleState" in after and "available" != after.get("LifeCycleState", ""): # The create_file_system could return while the file system’s lifecycle state is still 'creating'. # There seems to be no waiter on this service resource # Throwing this back into reconciler loop for a retry until it becomes available. # If a status is not changed to 'available' for the configured retries, it could still fail. result["comment"] += [ f"aws.efs.file_system '{name}' is in '{after.get('LifeCycleState')}' state." ] result["rerun_data"] = dict(name=name, resource_id=resource_id) return result backup_result = await hub.exec.boto3.client.efs.describe_backup_policy( ctx, FileSystemId=resource_id, ) if not backup_result[ "result" ] and not hub.tool.aws.efs.file_system_utils.is_backup_policy_not_found( backup_result=backup_result ): hub.log.warning( f"Could not describe backup policy for file system {resource_id} : {backup_result['comment']}" ) result["comment"] += backup_result["comment"] result["result"] = False return result backup_enabled = hub.tool.aws.efs.file_system_utils.is_backup_enabled(backup_result) result[ "new_state" ] = hub.tool.aws.efs.file_system_utils.convert_raw_file_system_to_present( file_system=after, backup_enabled=backup_enabled ) return result
[docs]async def absent(hub, ctx, name: str, resource_id: str = None) -> Dict[str, Any]: """Deletes a file system. Deletes a file system, permanently severing access to its contents. Upon return, the file system no longer exists and you can't access any contents of the deleted file system. You can't delete a file system that is in use. That is, if the file system has any mount targets, you must first delete them. For more information, see DescribeMountTargets and DeleteMountTarget. .. Note:: The DeleteFileSystem call returns while the file system state is still deleting. You can check the file system deletion status by calling the DescribeFileSystems operation, which returns a list of file systems in your account. If you pass file system ID or creation token for the deleted file system, the DescribeFileSystems returns a 404 FileSystemNotFound error. This operation requires permissions for the elasticfilesystem:DeleteFileSystem action. Args: name(str): An Idem name of the resource. resource_id(str, Optional): EFS file system ID. Request Syntax: .. code-block:: sls [file-system-resource-id]: aws.efs.file_system.absent: - name: "string" - resource_id: "string" Returns: Dict[str, Any] Examples: .. code-block:: sls fs-xxxxxxxxx: aws.efs.file_system.absent: - name: efs-name - resource_id: fs-xxxxxxxxx """ result = dict(comment=[], old_state=None, new_state=None, name=name, result=True) file_systems = None if resource_id: file_systems = await hub.exec.boto3.client.efs.describe_file_systems( ctx, FileSystemId=resource_id ) # Check for existing filesystem before = None if file_systems and file_systems["result"] and file_systems["ret"]["FileSystems"]: before = file_systems["ret"]["FileSystems"][0] if before: backup_result = await hub.exec.boto3.client.efs.describe_backup_policy( ctx, FileSystemId=resource_id, ) if not backup_result[ "result" ] and not hub.tool.aws.efs.file_system_utils.is_backup_policy_not_found( backup_result=backup_result ): hub.log.warning( f"Could not describe backup policy for file system {resource_id} : {backup_result['comment']}" ) result["comment"] += backup_result["comment"] result["result"] = False return result backup_enabled = hub.tool.aws.efs.file_system_utils.is_backup_enabled( backup_result ) result[ "old_state" ] = hub.tool.aws.efs.file_system_utils.convert_raw_file_system_to_present( file_system=before, backup_enabled=backup_enabled ) if not before: result["comment"] = hub.tool.aws.comment_utils.already_absent_comment( resource_type="aws.efs.file_system", name=name ) elif ctx.get("test", False): result["comment"] = hub.tool.aws.comment_utils.would_delete_comment( resource_type="aws.efs.file_system", name=name ) else: ret = await hub.exec.boto3.client.efs.delete_file_system( ctx, FileSystemId=resource_id ) result["result"] = ret["result"] result["comment"] = ( hub.tool.aws.comment_utils.delete_comment( resource_type="aws.efs.file_system", name=name ) if result["result"] else (ret["comment"],) ) return result
[docs]async def describe(hub, ctx) -> Dict[str, Dict[str, Any]]: """Returns a list of Amazon EFS file systems. Describe the resource in a way that can be recreated/managed with the corresponding "present" function. Returns the description of a specific Amazon EFS file system if either the file system CreationToken or the FileSystemId is provided. Otherwise, it returns descriptions of all file systems owned by the caller's Amazon Web Services account in the Amazon Web Services Region of the endpoint that you're calling. When retrieving all file system descriptions, you can optionally specify the MaxItems parameter to limit the number of descriptions in a response. Currently, this number is automatically set to 10. If more file system descriptions remain, Amazon EFS returns a NextMarker, an opaque token, in the response. In this case, you should send a subsequent request with the Marker request parameter set to the value of NextMarker. To retrieve a list of your file system descriptions, this operation is used in an iterative process, where DescribeFileSystems is called first without the Marker and then the operation continues to call it with the Marker parameter set to the value of the NextMarker from the previous response until the response has no NextMarker. The order of file systems returned in the response of one DescribeFileSystems call and the order of file systems returned across the responses of a multi-call iteration is unspecified. This operation requires permissions for the ``elasticfilesystem:DescribeFileSystems`` action. Returns: Dict[str, Any] Examples: .. code-block:: bash $ idem describe aws.efs.file_system """ result = {} ret = await hub.exec.boto3.client.efs.describe_file_systems(ctx) if not ret["result"]: hub.log.warning(f"Could not describe file_system {ret['comment']}") return {} for file_system in ret["ret"]["FileSystems"]: resource_id = file_system["FileSystemId"] backup_result = await hub.exec.boto3.client.efs.describe_backup_policy( ctx, FileSystemId=resource_id, ) if not backup_result[ "result" ] and not hub.tool.aws.efs.file_system_utils.is_backup_policy_not_found( backup_result=backup_result ): hub.log.warning( f"Could not describe backup policy for file system {resource_id} : {backup_result['comment']}" ) continue backup_enabled = hub.tool.aws.efs.file_system_utils.is_backup_enabled( backup_result ) resource = ( hub.tool.aws.efs.file_system_utils.convert_raw_file_system_to_present( file_system=file_system, backup_enabled=backup_enabled ) ) result[resource_id] = { "aws.efs.file_system.present": [ {parameter_key: parameter_value} for parameter_key, parameter_value in resource.items() ] } return result