HOWTO: Use CloudFormation to tag new EC2 instances with Lambda function, EventBridge, VPC and CloudTrail

/ AWS Tutorial, CloudFormation, CloudTrail, EventBridge, Lambda, Python, VPC

In this tutorial, we will automate the tagging of new EC2 instances using CloudFormation templates. In the previous tutorial, we learnt how to tag AWS EC2 instances when they are created. We created an event rule in EventBridge to trigger a Lambda function to tag new EC2 instances with the name of the user that created it. That tutorial had step by step instructions on how to accomplish this via the AWS management console. This tutorial covers all the steps using CloudFormation templates for easy build and tear down and re-use in the future.

Steps

  1. Create a python file with the Lambda function to tag EC2 instances when they are created
  2. Upload the file to a S3 bucket
  3. Create the VPC stack
  4. Create the CloudTrail stack
  5. Create the Lambda deployment and EventBridge rules stack
  6. Create EC2 stack
  7. Validate the tags on the EC2 instances 
  8. Clean up resources. Delete all the stacks above

Step 1: Create a python file with the Lambda function to tag EC2 instances when they are created

import json
import boto3

print('loading tagEC2 lambda function')
 
ec2 = boto3.client('ec2')
 
def lambda_handler(event, context):
    #print(event)
    
    #user name; please note that this may differ based on the structure of your IAM user
    userName = event['detail']['userIdentity']['sessionContext']['sessionIssuer']['userName']
    
    #instance id
    instanceId = event['detail']['responseElements']['instancesSet']['items'][0]['instanceId']
    
    try:
        ec2.create_tags(
            Resources=[
                instanceId,
            ],
            Tags=[
                {
                    'Key': 'CreatedBy',
                    'Value': userName
                },
            ]
        )
        print("completed executing tagEC2 lambda function")
    except Exception as e:
        print(e)
        raise e
    
    return

Step 2: Upload the file to a S3 bucket

To upload the newly created Lambda function to the S3 bucket, first convert it to a zip file as follows:

zip maghilda_tagResourceCreation.zip maghilda_tagResourceCreation.py

Step 3: Create the VPC stack

AWSTemplateFormatVersion: 2010-09-09
Description: Set up VPC template for test environment with one public subnet in one availabily zone. 

Parameters:
  EnvironmentName:
    Description: An environment name that is prefixed to resource names
    Type: String
    Default: test

  VpcCIDR:
    Description: Please enter the IP range (CIDR notation) for this VPC
    Type: String
    Default: 10.0.0.0/16

  PublicSubnetCIDR:
    Description: Please enter the IP range (CIDR notation) for the public subnet in the one Availability Zone
    Type: String
    Default: 10.0.10.0/24

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

  InternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

  PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      CidrBlock: !Ref PublicSubnetCIDR
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} Public Subnet (AZ)

  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} Public Routes

  DefaultPublicRoute:
    Type: AWS::EC2::Route
    DependsOn: InternetGatewayAttachment
    Properties:
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  PublicSubnetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnet

  NoIngressSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: "no-ingress-sg"
      GroupDescription: "Security group with no ingress rule"
      VpcId: !Ref VPC

Outputs:
  VPC:
    Description: A reference to the created VPC
    Value: !Ref VPC

  PublicSubnet:
    Description: A reference to the public subnet in the Availability Zone
    Value: !Ref PublicSubnet

  NoIngressSecurityGroup:
    Description: Security group with no ingress rule
    Value: !Ref NoIngressSecurityGroup

Deploy the stack via the AWS CLI

aws cloudformation create-stack --stack-name testvpc --template-body file://vpc-template.yml

Step 4: Create the CloudTrail stack

EventBridge requires CloudTrail Trail to exist as it uses the CloudTrail API. To set up CloudTrail trail with S3, please review my post here.

Step 5: Create the Lambda deployment and EventBridge rules stack

AWSTemplateFormatVersion: 2010-09-09
Description: Set up EventBridge Trigger and Lambda function to tag EC2 resources 
    
Resources:
  # BEGIN MANAGED POLICY 
  ManagedPolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      PolicyDocument: 
        Version: '2012-10-17'
        Statement:
          - Sid: CloudWatchLogs #similar to AWSLambdaBasicExecutionRole
            Effect: Allow
            Action:
              - logs:PutLogEvents
              - logs:CreateLogGroup
              - logs:CreateLogStream
            Resource: "arn:aws:logs:*:*:*"
          - Sid: CreateTags
            Effect: Allow
            Action: ec2:CreateTags
            Resource: "*" 

  # BEGIN LAMBDA IAM RESOURCES
  EventBridgeLambdaRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      ManagedPolicyArns:
          - !Ref ManagedPolicy
  
  # Deploy Lambda function
  myLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Runtime: python3.11
      FunctionName: maghilda_tagResourceCreation
      Role: !GetAtt EventBridgeLambdaRole.Arn
      Handler: maghilda_tagResourceCreation.lambda_handler #filename.handler
      Code:
        S3Bucket: m-lambdafunctions  #s3 bucket name
        S3Key: maghilda_tagResourceCreation.zip   #file uploaded to the s3 bucket
      Description: tagEC2 lambda function
      TracingConfig:
        Mode: Active

  #Provide Lambda permissions to execute Events
  LambdaPerms:
      Type: AWS::Lambda::Permission
      Properties:
        Action: 'lambda:InvokeFunction'
        FunctionName: !Ref myLambdaFunction
        Principal: events.amazonaws.com 
        SourceAccount: !Ref AWS::AccountId  
        SourceArn: !GetAtt EventBridgeTrigger.Arn

  #Set up EventBridge Rule
  EventBridgeTrigger:
      Type: AWS::Events::Rule
      Properties:
        Description: 'EventTrigger on EC2'
        State: 'ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS'
        EventPattern: 
          source:
            - 'aws.ec2'
          detail-type:
            - 'AWS API Call via CloudTrail'
          detail: #the service generating the event determines the content of this field.
            eventSource: 
              - 'ec2.amazonaws.com'
            eventName: 
              - 'RunInstances'
        Targets:
          - Arn: !GetAtt myLambdaFunction.Arn
            Id: 'LambdaFunction'

Outputs :
  LambdaFunctionName:
    Value: !Ref myLambdaFunction

  LambdaRoleArn:
    Value: !GetAtt EventBridgeLambdaRole.Arn

  EventRuleArn:
    Value: !GetAtt EventBridgeTrigger.Arn

Deploy the stack via the AWS CLI

aws cloudformation create-stack --stack-name testlambda  --capabilities CAPABILITY_NAMED_IAM --template-body file://TagEC2Lambda-template.yml

Step 6: Create EC2 stack

AWSTemplateFormatVersion: 2010-09-09
Description: Create EC2 instance. The security group allows only SSH traffic 

Parameters:
  InstanceType:
    Description: Instance Type to use
    Type: String
    Default: t2.micro
  AMI:
    Description: AMI to use; Amazon Linux 2023 AMI
    Type: String
    Default: 'ami-0e731c8a588258d0d'
  Key:
    Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
    Type: AWS::EC2::KeyPair::KeyName
    Default: 'app-key-pair' #change to use your key-pair name
  SSHLocation:
    Description: The IP address range that can be used to SSH to the EC2 instances
    Type: String
    Default: 18.206.107.24/29 #using EC2 Instance Connect IP address range to connect to an instance
  VPC:
    Description: The ID of the VPC for the security group
    Type: String
    Default: 'vpc-0ac7ad98b5f0b664b' 
  SubnetID:
    Description: The ID of the subnet to launch the instance into.
    Type: String
    Default: 'subnet-0f7be13cfe8e7f030' 


Resources:
  MyEC2Instance: 
    Type: AWS::EC2::Instance
    Properties: 
      ImageId: !Ref AMI
      KeyName: !Ref Key
      InstanceType: !Ref InstanceType
      NetworkInterfaces: 
        - AssociatePublicIpAddress: "true"
          DeviceIndex: "0"
          GroupSet: [!Ref SSHSecurityGroup]
          SubnetId: !Ref SubnetID

  SSHSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: "ssh-ingress-sg"
      GroupDescription: "Enable SSH access via port 22"
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: !Ref SSHLocation   
      VpcId: !Ref VPC

Outputs:
  InstanceID:
    Description: InstanceId of the newly created EC2 instance
    Value: !Ref MyEC2Instance
  AZ:
    Description: Availability Zone of the newly created EC2 instance
    Value: !GetAtt [MyEC2Instance, AvailabilityZone]
  PublicDNS:
    Description: Public DNSName of the newly created EC2 instance
    Value: !GetAtt [MyEC2Instance, PublicDnsName]
  PublicIP:
    Description: Public IP address of the newly created EC2 instance
    Value: !GetAtt [MyEC2Instance, PublicIp]
  SSHSecurityGroup:
    Description: Security group with SSH access only
    Value: !Ref SSHSecurityGroup

Deploy the stack via the AWS CLI

aws cloudformation create-stack --stack-name testEC2 --template-body file://EC2-template.yml

You should have the four CloudFormation Stacks on CloudFormation as follows:

Step 7: Validate the tags on the EC2 instances 

Step 8: Clean up resources. Delete all the stacks above

Delete Lambda function deployment and EventBridge Rules

aws cloudformation delete-stack --stack-name testlambda

Delete the EC2 instances 

aws cloudformation delete-stack --stack-name testEC2

Delete the VPC and associated subnet and security groups

aws cloudformation delete-stack --stack-name testvpc

Delete the CloudTrail trail

aws cloudformation delete-stack --stack-name testcreatetrail

This is the end of the tutorial. Hope it was helpful to you.

References