Creating an AMI using the AWS CLI

This post describes the process of building a custom AMI (Amazon Machine Image) using the AWS CLI. The goal is to automate the entire process, making it completely repeatable.

Why Create an AMI?

Why would you want to go to the effort of creating an AMI?

That’s a good question.

Well, if you are just creating a once-off EC2 instance then there’s probably no point in creating an AMI.

If, however, you find yourself frequently setting up instances with the same requirements, then you can definitely benefit from creating a custom AMI. It’ll essentially act as a template, giving you precisely what you need without you having to go through all of the setup.

Suppose, for example, you frequently need to launch an EC2 instance with the following installed:

• R
• Python
• Docker
• MySQL client and
• PostgreSQL server.

This would mean that each time you’d need to update package information, download the packages and then wait for them to install. Sure, you could script that. But then you’d have to upload and run the script each time. Just different hoops to jump through.

If, however, you create a custom AMI then, rather than launching from a generic AMI, you’d use your custom AMI which has everything that you need already baked into it.

Setup

We’ll start by defining a few environment variables.

# The name of the base AMI (Ubuntu 20.04).
AMI_ID_BASE="ami-00399ec92321828f5"
# The name of the custom AMI.
AMI_NAME_TARGET="cedalion"

# Region in which creating AMI.
REGION="eu-west-1"
# The type of EC2 instance used to build the AMI.
TYPE="t2.nano"

# File into instance configuration will be written.
JSON_EC2="ec2-details.json"
# File into AMI configuration will be written.
JSON_AMI="ami-details.json"

We’ll also define KEYPAIR as the name of a SSH key pair registered on EC2.

Create a Device Mapping

We’ll be specific about the parameters of the root volume for the EC2 instance, so create a JSON file with the details.

cat << EOF >device-mapping.json
[
    {
        "DeviceName": "/dev/sda1",
        "Ebs": {
            "VolumeSize": 8,
            "DeleteOnTermination": true,
            "Encrypted": false
        }
    }
]
EOF

Launch an EC2 Instance

Launch the EC2 instance using the selected base image (in this case Ubuntu 20.04). This will form the foundation on which we’ll provision the new image.

aws ec2 run-instances \
    --region $REGION \
    --image-id $AMI_ID_BASE \
    --count 1 \
    --block-device-mappings file://device-mapping.json \
    --instance-type $TYPE \
    --key-name $KEYPAIR \
    >$JSON_EC2

This command returns the initial configuration data for the instance, which we’ll stash in a JSON file.

Further interactions with the instance will require referencing it via an instance ID. We’ll extract that from the configuration data using jq.

INSTANCE_ID=$(jq -r '.Instances[0].InstanceId' $JSON_EC2)

Retrieving Public DNS Name

The initial configuration data doesn’t contain the public DNS name or IP address of the instance. We’ll wait for the instance to be running before retrieving the its final parameters.

aws ec2 wait instance-status-ok \
    --region $REGION \
    --instance-ids $INSTANCE_ID

When this command returns we can be confident that the instance is ready to accept connections.

Now request the full description.

aws ec2 describe-instances \
    --region $REGION \
    --instance-ids $INSTANCE_ID \
    >$JSON_EC2

Extract the public DNS name.

EC2_DNS=$(jq -r '.Reservations[0].Instances[0].PublicDnsName' $JSON_EC2)

Installing Software

Next we’ll provision the instance via SSH. The StrictHostKeyChecking option just prevents SSH from stopping to prompt you for confirmation.

ssh -o StrictHostKeyChecking=no ubuntu@$EC2_DNS << EOF
sudo apt-get update -y

sudo apt-get install -y r-base-core
sudo apt-get install -y docker.io docker-compose
sudo apt-get install -y default-mysql-client
sudo apt-get install -y postgresql postgresql-contrib
EOF

You can add whatever content you want at this stage. For the moment we’ll install R, Python, Docker, the MySQL client and PostgreSQL.

Create the AMI

Once the required software is installed we can build the AMI.

First we’ll need to check that there’s not an existing AMI with the same name.

aws ec2 describe-images \
    --region $REGION \
    --filters "Name=name,Values=$AMI_NAME_TARGET" \
    >$JSON_AMI

If there is, then deregister it.

AMI_ID_TARGET=$(jq -r '.Images[0].ImageId' $JSON_AMI)

if [ $AMI_ID_TARGET != "null" ]
then
   aws ec2 deregister-image --region $REGION --image-id $AMI_ID_TARGET
fi

Now we can be confident when assigning the name to a new AMI.

aws ec2 create-image \
    --region $REGION \
    --instance-id $INSTANCE_ID \
    --name "$AMI_NAME_TARGET" \
    --no-reboot \
    >$JSON_AMI

🚀 The custom AMI is ready!

Terminate the EC2 Instance

Finally we can terminate the EC2 instance we used to build the AMI.

aws ec2 terminate-instances --region $REGION --instance-ids $INSTANCE_ID

Using the Custom AMI

The sparkling new AMI can now be used to launch further EC2 instances, each of which will be provisioned with the required software. If you’re going to be creating such instances frequently then this can save you a lot of time and effort.

Let’s indulge in the fruits of our labour and launch a new image based on the custom AMI.

First get the AMI ID.

AMI_ID_TARGET=$(jq -r '.ImageId' $JSON_AMI)

Launch an EC2 instance using the new AMI.

aws ec2 run-instances \
    --region $REGION \
    --image-id $AMI_ID_TARGET \
    --count 1 \
    --instance-type $TYPE \
    --key-name $KEYPAIR