In the last few posts we’ve looked at a few ways to set up the infrastructure for a Selenium crawler using Docker to run both the crawler and Selenium. In this post we’ll launch this setup in the cloud using AWS Elastic Container Service (ECS).
There are certainly better and more efficient ways to do this, but that’s not the objective here. We’re just aiming to build something minimal: it’s simple and it works.
The first step towards deploying on ECS is to create a cluster. If you have an existing cluster then you can skip this step.
Login to AWS and go to the ECS Dashboard. Press .
Select the Networking only template and press .
Configure the cluster.
- Choose a name for the cluster.
- If you want the cluster to be on a new VPC then check the box.
- 🔎 Check the box to enable container insights. This is not really necessary but will give you some very granular performance metrics.
- Press .
If everything went smoothly then you will see a message informing you that the cluster has been successfully created.
Smash the button.