To lambda or not to lambda
Data Eng Weekly - Ep 18
The tech community is currently on fire with a debate following the announcement from Prime Video, which managed to significantly reduce its AWS costs by transitioning from a serverless architecture (Lambda and Step Functions) to a monolithic application.
What makes this news particularly intriguing is that the decision came from an internal department within AWS itself.
The logical follow-up question that arose was: Is AWS selling managed services that they aren't using internally? ….
https://twitter.com/jrhunt/status/1655112652178481157
In today's post, we will focus on the interesting question raised by this debate: when should we choose serverless services over traditional container services?
Lambda ? Monolithic ?
Let's dive deeper into the decision made by the Amazon Prime team and understand the context of their choice.
Below is a simplified overview of what the Prime Video team accomplished:
Prime Video service is responsible for monitoring the quality of thousands of live streams.
To achieve this, they initially employed a fully serverless architecture on AWS. They used Step Functions and Lambda functions that are billed based on usage: Lambda charges for each invocation, considering the number of requests and the duration of function execution.
Step Functions, on the other hand, charge for every state transition that occurs within a workflow.
Additionally, if each of these steps involves querying data from or sending data to Amazon S3, the cost can increase even further. Data transfer and storage costs in S3, as well as the number of requests, can contribute to the overall expense of the serverless architecture.
The number of small operations at a large scale that the serverless design requires and the intensive s3 queries made the cost explode.
The Amazon Prime Video team decided to transition to a monolithic architecture to address these escalating costs. In this approach, all code is centralized within a single application, and the entire application is deployed as one container.
This solution allowed them to:
Reduce calls to S3
Decrease the number of states to maintain
and therefore reduce their bill by 90% …
So lambda or container?
From my experience, Serverless and monolithic architectures have to be combined to create a hybrid approach that leverages the strengths of both. The choice between these two solutions depends on the following factors:
load profile
job profile
organization context
Load profile: spike sizes + predictability
In the article by Vlad Ionescu, the scaling speed of various AWS services is compared:
AWS Lambda is the fastest service to scale, making it a strong choice for applications that need to handle sudden bursts of traffic or accommodate variable workloads.
However, it is essential to note that there is a default limit of 3,000 concurrent executions concurrently.
So my take regarding this is:
Case 1 - Predictable load: Use a container.
You can turn on/off and scale your container based on a schedule.
Case 2 - Unpredictable spikes within the 3,000 concurrent execution limit: Use Lambda: you will get the best response time.
Case 3 - Unpredictable spikes exceeding the 3,000 concurrent execution limit: Use a container.
Job profile: long-running or memory intensive
AWS Lambda has an execution time limit of 15 minutes and a maximum memory allocation of 10 GB.
If your tasks are expected to exceed these limits or require long-running processes, it is more efficient to switch to a container-based solution.
Attempting to work around these limitations leads to increased development time and complexity, ultimately becoming more challenging to manage.
You will end up trying to split your task into multiple sub-tasks. This process can quickly become a maintenance and operational nightmare, as it adds multiple layers of complexity to the system.
Example of jobs I would do in a container:
call a long-running API
process a large CSV file
long-running DB query/extracts
Organization context
Serverless services can provide significant advantages in the early stages of product development. They offer rapid development speed, allowing you to quickly set up databases, processing units, and other components.
Additionally, serverless services provide scalability in both directions:
If your service has low usage, your costs will be minimal.
The ability to scale up quickly can accommodate sudden growth.
Therefore, leveraging serverless services makes sense for startups and early-stage products that prioritize speed and cost-effectiveness.
When a company handles a substantial amount of requests per second (e.g., +1k requests/s) and has more significant resources, the tradeoff shifts. In this context, a company can afford the fixed cost of a container to manage its load.
Nothing is definitive
As mentioned here, migrating code from Lambda to a container is often straightforward.
The Prime Video team reused all their Lambda code components directly within their container.
As your service evolves, and you gain a better understanding of the job topology and load patterns, it is indeed possible to adapt your architecture and migrate services from serverless to container-based solutions.
Thanks for reading,
-Ju
I would be grateful if you could help me to improve this newsletter. Don’t hesitate to share with me what you liked/disliked and the topic you would like to be tackled.
P.S. you can reply to this email; it will get to me.