Automated EBS Volume Snapshot Using AWS Lambda Based On EC2 Tags

Lambda Function To Take Snapshot Of EBS Volume Based On EC2 Tags

Take Snapshot Of Secondary Volumes Of Instance Having particular Tag

Create AWS Lambda Function For EBS Snapshot Having Certain Tags On EC2 Instance

It is often required to take snapshot of EBS volume for backup as there could be important data residing on those volumes. This can be achieved by run some script on ec2 instance as a cron. But its very unreliable to run a script on some ec2 instance as the script may fail due to any reason and we won't come to know about it.

To solve this problem we will use a python script on AWS Lambda which can scheduled to run at required interval. You can also set cloudwatch alarm to trigger a mail if the lambda function return some error.

Note: We are going to take Snapshot of only secondary volumes as assuming for most of the databases, data resides on secondary disk instead of root volume.

# End Goal

• Take Snapshot of only secondary volume
• Snapshot should be taken of a volume attached to an instance having a tag "Backup:true"

# Prerequisites

• IAM role (volume-snapshot-role) having policy to read ec2 instance details and to create snapshot.
• Instance with secondary volume.

# Video Tutorial

# IAM Role

IAM role should have following policy attached to it:

1.  AWS Managed Policy: AmazonEC2ReadOnlyAccess
2. volume-snapshot-policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeVolumeStatus",
                "ec2:DescribeVolumes",
                "ec2:DescribeSnapshots",
                "ec2:CreateSnapshot",
                "ec2:CreateTags"
            ],
            "Resource": "*"
        }
    ]
}

# Lambda Function

Following is the lambda function snippet:

from __future__ import print_function
import boto3
from boto3 import resource

'''
    This script takes snapshot of volumes which are secondary volumes and are attached to instance having "Backup:true" tag
'''

SNAPSHOT_TAGS = {'createdby': 'lambda'}        # Dictionary of tags to apply to the created snapshots
TAG_FILTERS = [{'Name': 'tag:Backup','Values': ['true']}]   # Tags on which ec2 instance should get filter
REGION = "ap-south-1"                                       # AWS region in which the volumes exist


def take_snapshots(volume, tags_kwargs):
    snapshot = volume.create_snapshot(
           Description='created by lambda'
           )
    #snapshot = ec2.create_snapshot(VolumeId=volume,Description='Created by Lambda function ebs-snapshots')
    print(snapshot)
    if tags_kwargs:
        snapshot.create_tags(**tags_kwargs)

def process_tags():
    tags = []
    tags_kwargs = {}
    # AWS allows 10 tags per resource
    if SNAPSHOT_TAGS and len(SNAPSHOT_TAGS) <= 10:
        for key, value in SNAPSHOT_TAGS.items():
            tags.append({'Key': key, 'Value': value})
        tags_kwargs['Tags'] = tags
    return tags_kwargs

def print_summary(counts):
    print("\nSUMMARY:\n")
    print("Snapshots created:  {}{}".format(counts,""))
    print("-------------------------------------------\n")


def lambda_handler(event, context):

    snap_count = 0

    # List of devices that should be consider to take snapshot. Root volume is excluded here i.e. /dev/xvda or /dev/sda
    DEVICES = ['/dev/sdb', '/dev/sdc', '/dev/sde', '/dev/sdf', '/dev/sdg', '/dev/sdh', '/dev/sdi', '/dev/sdj', '/dev/sdk', '/dev/sdl', '/dev/sdm', '/dev/sdn', '/dev/sdo', '/dev/sdp', '/dev/sdq', '/dev/sdr', '/dev/sds', '/dev/sdt', '/dev/sdu', '/dev/sdv', '/dev/sdw', '/dev/sdx', '/dev/sdy', '/dev/sdz']

    ec2 = resource("ec2", region_name=REGION)
    ec2_client = boto3.client('ec2')

    # Get information for all instances with tag "Backup:true"
    instances = ec2.instances.filter(Filters=TAG_FILTERS)

    # Filter all instances that have are attached to ec2 instances in "instances"
    for instance in instances:
        ec2_instance = ec2.Instance(instance.id)
        print("for instance:", ec2_instance )
        volumes = ec2.volumes.filter(Filters=[{'Name': 'attachment.instance-id', 'Values': [ec2_instance.id]}, {'Name': 'attachment.device', 'Values': DEVICES} ])
        #print(volumes)
  
        for tags in instance.tags:
            #print("inside for loop of instance.tags")
            if tags["Key"] == 'Name':
                SNAPSHOT_TAGS['Name'] = tags["Value"]
  
        # Take Backup
        for volume in volumes:
            tags_kwargs = process_tags()

            print("Taking snapshot..")
            take_snapshots(ec2.Volume(volume.id), tags_kwargs)
            snap_count += 1
            
    print("at last summary")
    print_summary(snap_count)