LocalStack empowers developers to harness the full power of AWS services locally, cutting costs significantly. Let’s look at its benefits and find out how to build a simple AWS integration entirely on your local machine.
LocalStack is a cloud development platform that enables an AWS emulator, which runs as a Docker image on your local developer machine or your continuous integration environment. It exposes various AWS services that let you run your cloud and serverless applications without connecting to an AWS account. With LocalStack, you can access all the necessary AWS resources for your application locally and conduct automated tests in an AWS environment without the expenses of AWS developer accounts, time-consuming redeployments, or potential errors caused by remote connections.
LocalStack started as an open source project at Atlassian as an easy-to-use test/mocking framework that exposes various AWS services, such as S3, Lambda, SQS, Elasticsearch, SES, and more. The project spun off as an individual organisation and has seen two major releases — LocalStack 1.0 and LocalStack 2.0. The project now enables 80+ AWS services, encompassing various sophisticated services like Glue, Athena, RDS, QLDB, and more, to help developers improve their experience of building and testing their AWS applications.
Developing and testing cloud apps today
In the GitLab DevSecOps 2021 Survey, many developers pointed out that the lack of testing has led to delays in releasing quality software. Developers want to close the testing gap, find more errors and bugs during the development phase itself, and further enable cloud native, cross-platform development. Modern software development encompasses various cloud native dependencies — databases, virtual machines, message queues, and more. With public cloud service providers, the development and testing loop is extremely slow and tedious. Every local change must be packaged and uploaded to the cloud for testing.
The problem does not stop at just local development and testing — the closed source nature of public providers has further aggravated the workflow. Developers cannot attach a debugger to debug their code from the IDE directly; enabling debug logging is another pain, and figuring out red builds on the staging or the CI pipeline blocks other developers from testing their applications in the same time frame.
In an interesting comparison, a Hackernews user named ‘gigawatts’ equates the cloud to time share mainframes of the past, highlighting similarities such as the lack of local workstations, closed source nature, limited debugging capabilities, and delayed log-based feedback. It becomes extremely hard for managers as dev environments in the cloud are hard to manage since there is an explosion of different environments required for testing. On top of this, there is a noticeable spike in cloud costs as the resources are not cleaned properly after testing, leading to bloated cloud waste.
Getting started with LocalStack
Today cloud development is slow, tedious, and costly. LocalStack’s fully functional local cloud stack enables teams to adopt extremely efficient dev and test loops. There are plenty of advantages to using LocalStack:
- With LocalStack, you gain greater control over your environment, ensuring consistent and reliable results across different setups.
- LocalStack enables faster deploy-test-redeploy cycles, allowing for rapid iteration and development.
- It provides easier debugging capabilities and facilitates the replication of bugs locally, leading to improved software quality.
- You no longer need to create individual AWS developer accounts for each user, streamlining administrative tasks and reducing complexity.
- You can overcome barriers in regulated environments where access to the cloud is often restricted, enabling developers to leverage its benefits regardless of limitations.
- Finally, cost considerations play a significant role. You don’t need to worry about uncleaned resources since they are created locally and can be easily cleaned.
To install LocalStack on your machine, you need Python (and pip package manager) and Docker. Let us install LocalStack using pip:
$ pip3 install localstack
The above command will install the localstack-cli which runs the Docker image that packages the LocalStack runtime. Start LocalStack in a detached mode by running the following command:
$ localstack start -d
It will initialise LocalStack in a detached mode, and now you have a full suite of local AWS services that you can use for testing and playing around.
You can take LocalStack for a run by installing AWS CLI on your machine. After installing AWS CLI, run the following commands to create an S3 bucket, add some content to your bucket, and check its content:
$ aws --endpoint-url=http://localhost:4566 s3 mb s3://test make_bucket: test $ echo “hello world” > /tmp/hello-world $ aws --endpoint-url=http://localhost:4566 s3 cp /tmp/hello-world s3://test/hello-world upload: ../../tmp/hello-world to s3://test/hello-world $ aws --endpoint-url=http://localhost:4566 s3 ls s3://test/ 2023-05-27 15:39:38 12 hello-world
If you notice closely, we have configured the –endpoint-url flag to point towards http://localhost:4566. This is the edge URL where all LocalStack services are available. You can point your AWS CLI or any third-party integration towards this URL, and it will route your AWS API requests to LocalStack instead.
Building on LocalStack — S3 notifications with SQS
We will create a simple S3 and SQS integration to set up a notification configuration using AWS CLI. We will use S3 Event Notifications to send event notification messages to an SQS queue and test the integration locally with LocalStack, without creating any real cloud resource or setting up an AWS account. We assume you are operating an OSX or a Linux distribution for this tutorial. If you are using Windows, using Windows Subsystem for Linux (WSL) is preferable.
Start LocalStack using the localstack CLI. Let us specify a few configurations for our S3 Bucket, SQS queue, and AWS region:
$ export BUCKET_NAME=s3-event-notification $ export AWS_REGION=us-east-1 $ export SQS_QUEUE=queue-notification
Next, create an SQS queue using AWS CLI:
$ aws --endpoint-url=http://localhost:4566 sqs create-queue --queue-name $SQS_QUEUE
{
“QueueUrl”: “http://localhost:4566/000000000000/queue-notification”
}
You can fetch the Queue ARN (Amazon Resource Name) to specify the value while configuring the bucket notification:
$ export SQS_QUEUE_ARN=$(aws --endpoint-url=http://localhost:4566 sqs get-queue-attributes\
--attribute-name QueueArn --queue-url=http://localhost:4566/000000000000/”$SQS_QUEUE”\
| sed ‘s/”QueueArn”/\n”QueueArn”/g’ | grep ‘”QueueArn”’ | awk -F ‘”QueueArn”:’ ‘{print $2}’ | tr -d ‘”’ | xargs)
$ echo $SQS_QUEUE_ARN
arn:aws:sqs:us-east-1:000000000000:queue-notification
Let us create an S3 bucket now and apply the notification configuration:
$ aws --endpoint-url=http://localhost:4566 s3api create-bucket --bucket $BUCKET_NAME --region $AWS_REGION
{
“Location”: “/s3-event-notification”
}
$ aws --endpoint-url=http://localhost:4566 s3api put-bucket-notification-configuration \
--bucket $BUCKET_NAME\
--notification-configuration ‘{
“QueueConfigurations”: [
{
“QueueArn”: “’”$SQS_QUEUE_ARN”’”,
“Events”: [“s3:ObjectCreated:*”]
}
]
}’
In the above command, we have configured the notification settings, as these associate our S3 bucket with the SQS queue specified by the ARN we configured before, and set the events that will trigger notifications, such as object creation.
You can now easily trigger a notification by adding some dummy content in your S3 bucket:
$ echo “Hello” > app.txt $ cat app.txt Hello $ aws --endpoint-url=http://localhost:4566 s3 cp app.txt s3://$BUCKET_NAME upload: ./app.txt to s3://s3-event-notification/app.txt
You can use the Queue URl we retrieved before to check for new notifications coming from our S3 bucket:
$ aws --endpoint-url http://localhost:4566 \ sqs receive-message \ --queue-url http://localhost:4566/000000000000/queue-notification \ | jq -r ‘.Messages[0].Body’ | jq .
In the JSON returned, you will find an object (file) named app.txt with the eventVersion and eventSource parts of the metadata associated with the SQS record.
LocalStack integrates with many AWS integrations — Cloud Development Kit (CDK), CloudFormation, Serverless Application Model (SAM), language SDKs, and third-party integrations such as Terraform, Pulumi, Architect, and more! With LocalStack, you can continue building your AWS apps and infrastructure by cutting down on your cloud costs and improving the efficiency of your development and testing loops.
In this article, we’ve discussed LocalStack and how to build a simple AWS integration entirely on your local machine. We hope this article helps you start with LocalStack and accelerates your development workflow. You can further explore LocalStack on its website and in its documentation.
