How AWS Lambda Works Under the Hood

Why I Wanted to Understand This

Lambda is the service that made "serverless" a word. You upload a function, wire it to a trigger, and it runs — one invoke or a million, you never think about machines. But the questions underneath are wild: how does AWS run code from millions of mutually untrusting customers on shared hardware, start it in milliseconds on a machine that has never seen it before, and do this trillions of times a month?

These are my notes from going deep on Lambda internals — the isolation model, the invoke path, and the two systems that quietly killed most cold starts.

The Core Problem: Isolation vs. Density

Every design decision in Lambda flows from one tension:

Isolation. Customer A's code must never read customer B's memory, escape to the host, or starve neighbors. Process- and container-level isolation (namespaces, cgroups, seccomp) all share one kernel — and the kernel's syscall surface is a big attack target.
Density. A worker is profitable only if it's packed. Full virtual machines give great isolation, but a traditional VM takes seconds to boot and burns hundreds of MB before your code runs a single line.

Early Lambda picked isolation and paid for it: each customer got their own dedicated EC2 instances, and functions from different customers never shared a VM. Safe, but terrible economics — a customer running one 128 MB function still occupied a whole VM, and scaling up meant booting full instances.

The fix wasn't choosing a side. It was making VMs so cheap that you could afford one per sandbox.

Firecracker: A VM That Boots in ~125ms

In 2018 AWS deployed Firecracker, a virtual machine monitor written in Rust (~50K lines, versus over a million in QEMU). It leans on KVM for the hard parts and ruthlessly deletes everything else:

No BIOS, no bootloader — it loads an uncompressed Linux kernel directly.
A minimal device model — virtio network, virtio block, a serial console, and not much else. No PCI, no USB, no video. Less emulated hardware means less code a guest can attack.
Tiny footprint — under 5 MB of memory overhead per microVM, boot to guest kernel in ~125ms, and a single host can create ~150 microVMs per second.

	Containers	Traditional VMs	Firecracker microVMs
Isolation boundary	Shared kernel	Hardware virtualization	Hardware virtualization
Boot time	~ms	Seconds	~125ms
Memory overhead	~0	100s of MB	under 5 MB
Density per host	Very high	Low	Thousands

Each Firecracker process is additionally wrapped in a jailer — chroot, namespaces, cgroups, and a tight seccomp filter — so even a VMM compromise lands in a cage. Defense in depth: to reach the host, an attacker has to escape the guest kernel and the VMM and the jail.

The payoff is the headline property: every execution environment is its own virtual machine with its own guest kernel, and there are thousands of them, from different customers, packed on one box.

Anatomy of a Worker

Workers are EC2 bare-metal instances — Firecracker needs KVM, and nested virtualization would eat the performance budget.

EC2 bare-metal worker
┌────────────────────────────────────────────────────┐
│ ┌──────────┐  ┌──────────┐  ┌──────────┐           │
│ │ microVM  │  │ microVM  │  │ microVM  │   ...     │
│ │ guest    │  │ guest    │  │ guest    │  (1000s)  │
│ │ kernel   │  │ kernel   │  │ kernel   │           │
│ │ runtime  │  │ runtime  │  │ runtime  │           │
│ │ code A   │  │ code B   │  │ code C   │           │
│ └──────────┘  └──────────┘  └──────────┘           │
│  customer 1    customer 2    customer 3            │
└────────────────────────────────────────────────────┘

One microVM + language runtime + your code = a sandbox (an "execution environment"). The rules:

A sandbox is never shared across customers or functions. It belongs to exactly one function version.
A sandbox is reused across invokes of that same function — that's what a warm start is.
A sandbox handles one invoke at a time. Ten concurrent requests means ten sandboxes.

Workers themselves are deliberately short-lived — leased for hours, then drained and recycled. Nothing on a worker is precious; all durable state lives elsewhere.

The Invoke Path

Here's what happens between your Invoke call and your handler running:

Client ──▶ Frontend ──▶ Counting Service
              │          "is this account under its
              │           concurrency limit?"
              ▼
      Assignment Service
      "is there a warm sandbox
       for this function?"
        │              │
    yes │              │ no
        ▼              ▼
     Worker      Placement Service
     (run!)      "create a sandbox on
                  the best worker"
                       │
                       ▼
                 Worker boots microVM,
                 then run

The Frontend authenticates the request and looks up the function's config from a distributed cache. The Counting Service enforces concurrency limits — it's quorum-replicated across availability zones and sits in the hot path of every single invoke, so it's built to answer in a millisecond or two at region scale.

Then comes the fork in the road. The warm path — a sandbox already exists, idle, initialized — is a straight shot to the worker. The cold path detours through the Placement Service to build a sandbox first. Everything AWS has done to Lambda performance in the last five years is about making the cold path look like the warm path.

The Assignment Service: Tracking Every Sandbox in a Region

Someone has to know, for every function in the region, which sandboxes exist, where they live, and whether they're busy. That was originally the Worker Manager — which kept this state in memory on a single node per function. If that node died, the state was gone, and every affected function paid cold starts until it was rebuilt.

Its replacement, the Assignment Service, treats sandbox state like a database problem:

State changes are written to a durable journal log, replicated across multiple availability zones.
The service is partitioned; each partition runs one leader and two followers reading the same journal.
If a leader dies, a follower takes over with no lost state — it has the journal. Failover is quick and cold starts don't spike.
It was rewritten in Rust, cutting tail latency on a service that sits in the path of every invoke.

This rhymes with the S3 lesson: the bytes (sandboxes, workers) can be cattle, but the pointers to them need consensus-grade durability.

The Placement Service: Deciding Where Sandboxes Go

When there's no warm sandbox, the Placement Service picks a worker for the new one. It's the direct analog of S3's placement service, but the objective function is different: instead of spreading replicas across failure domains, it's packing for utilization without creating hot spots.

Placement uses statistical models of function behavior — invoke rates, memory sizes, lifetimes — to choose workers so that busy functions and idle functions mix well on the same hosts. Bad placement shows up as noisy neighbors and throttled cold starts; good placement means the fleet runs hot and p99 stays flat. It also manages worker leases — provisioning fresh bare-metal capacity as leases expire and demand shifts.

Cold Starts, Dissected

A cold start is everything between "no sandbox exists" and "handler running":

├─ routing ─┤├──── AWS-owned init ────┤├── your init ──┤├─ invoke ─┤
  placement    boot microVM              run top-level     handler
  picks a      fetch your code           code: imports,    runs
  worker       start the runtime         SDK clients,
                                         DB connections

The AWS-owned portion is where all the engineering below goes — booting the microVM is ~125ms of it, and fetching code used to dominate. The your-init portion is the part only you can fix: every import at module top-level, every client constructed outside the handler, runs before the first invoke completes.

One historical cold-start villain deserves a mention: VPC networking. Attaching a function to a VPC used to mean creating and attaching an ENI per sandbox — routinely adding seconds, sometimes ten. Since 2019, Lambda creates a shared ENI per function-VPC configuration at configuration time (backed by AWS Hyperplane, with cross-account attachment), and sandboxes tunnel through it. The cost moved from every cold start to a one-time setup step.

Container Images: Loading 10 GB in Under a Second

In 2020 Lambda started accepting container images up to 10 GB. The naive implementation — pull the image, unpack it, boot — would take minutes. Lambda does it with barely any cold-start penalty, and the trick (published in an award-winning ATC '23 paper) is my favorite part of the whole system.

Step 1: Flatten at publish time. When you push an image, Lambda unpacks all the layers into a single ext4 filesystem, built deterministically so identical inputs produce identical bytes. Then it splits that filesystem image into 512 KB chunks.

Step 2: Deduplicate with convergent encryption. Chunks must be encrypted, but normal encryption would make identical chunks look different, killing deduplication. So each chunk's key is derived from a hash of its own plaintext:

key = HMAC(hash(chunk_plaintext))
encrypted_chunk = AES(chunk_plaintext, key)

Identical plaintext → identical key → identical ciphertext. Two customers who both build on public.ecr.aws/lambda/python:3.13 produce byte-identical encrypted chunks for the entire base image — dedupable in a shared cache without AWS ever holding a shared secret. The numbers are striking: about 80% of newly uploaded functions produce zero unique chunks, and the rest average only ~4.3% unique.

Step 3: Load lazily through tiered caches. The microVM boots immediately against a virtual block device. Chunks are fetched only when the guest actually reads those blocks:

Tier	Where	Latency (512 KB chunk)
L1	Worker-local cache	~fastest, local disk/memory
L2	Dedicated per-AZ cache fleet	~550µs median
Origin	S3	~36ms

Because dedup concentrates most reads on a small set of wildly popular chunks (base images), the caches are extremely effective — and popular chunks are erasure-coded across the L2 fleet so no single cache node becomes a hot spot. Most functions boot having downloaded only a tiny fraction of their 10 GB image, because most of those bytes are never read.

SnapStart: Skipping Init Entirely

Lazy loading fixes code fetch, but the "your init" phase — JVM startup, framework wiring, heavyweight imports — can still take seconds. SnapStart's answer: do the init once, snapshot the microVM, and restore clones of it forever after.

When you publish a version with SnapStart enabled:

Lambda boots a sandbox and runs your full init phase.
It takes a Firecracker snapshot — the microVM's entire memory and disk state — encrypts it, chunks it (512 KB again), and caches it in the same L1 / L2 / S3 tier structure.
Snapshots are layered with separate keys for OS, runtime, and function state, so the OS and runtime chunks deduplicate across every function using them — convergent encryption again.

A cold start becomes a resume: restore the working set of pages and start executing, typically in a few dozen milliseconds instead of seconds. Lambda even records which pages get touched during real invokes and ships that access profile with the snapshot, so the restore prefetches exactly the right pages instead of guessing.

The subtle problem is uniqueness. Restoring one snapshot into a thousand clones means a thousand VMs with identical memory — identical PRNG state, identical "random" seeds. Cryptography breaks if two VMs generate the same "random" number. The fix spans the stack: Firecracker exposes a virtual machine generation ID (VMGenID) that changes on restore, the guest kernel reseeds its entropy pools when it sees it, and runtime hooks (Java's CRaC — Coordinated Restore at Checkpoint — being the mature example) let frameworks refresh anything stale, like DB connections established before the snapshot.

Async Invokes: Queues in Front of the Same Machine

Everything above described synchronous invokes. Async invokes (InvocationType: Event) — and event sources like SQS, Kinesis, and DynamoDB Streams — go through one extra stage: the request lands in an internal SQS queue, a poller fleet picks it up, and the poller performs a plain synchronous invoke through the exact same frontend → assignment → worker path.

Async Lambda is sync Lambda with a queue bolted on the front. One execution substrate, two admission paths — retries, dead-letter queues, and event-source batching all live in the poller layer and never complicate the core invoke path.

Key Takeaways

Isolation vs. density was resolved by making VMs nearly free. Firecracker deleted enough of the traditional VM (BIOS, PCI, device sprawl) that hardware-grade isolation now costs ~125ms and 5 MB — cheap enough to give every sandbox its own kernel.

The state that matters gets consensus; everything else is cattle. Sandboxes and workers are disposable, but the map of them — the Assignment Service's journal, like S3's metadata layer — is replicated, leader-elected, and survives failures without losing a beat.

Convergent encryption turns encryption and deduplication from enemies into allies. Deriving the key from the content's own hash lets AWS dedupe encrypted chunks across customers who share base images — which is almost all of them.

Cold starts died by a thousand cuts, all shaped like laziness. Don't fetch the image — fault in blocks as they're read. Don't run init — restore a snapshot. Don't guess pages — replay a recorded access profile. Do work only when something actually demands it.

Complexity lives at the edges, not the core. Async queues, event-source pollers, retries — all bolt onto the front of one simple, heavily-optimized synchronous path.

These same primitives — cheap microVMs, snapshot restore, container lazy loading — are exactly what AWS recombined to host AI agents, where the workload is an 8-hour stateful session instead of a 15-minute stateless invoke. I dig into that in How Bedrock AgentCore Runtime Works Under the Hood.