How Bedrock AgentCore Runtime Works Under the Hood

Why I Wanted to Understand This

After digging into how Lambda works under the hood, the natural next question was: what happens when the workload is an agent? An agent isn't a function. It holds a conversation for an hour, calls an LLM twenty times, runs tool code it effectively wrote itself, and spends most of its life waiting on I/O. Lambda's whole design — stateless, 15-minute cap, sandbox-per-function — is wrong-shaped for that.

AgentCore Runtime is AWS's answer, and it's built from the same parts as Lambda — microVMs, container images, versioned deployments — recombined around a different unit of isolation. These are my notes on how it works.

One honesty note up front: AgentCore went GA in late 2025, and AWS hasn't published a Firecracker-paper-level account of its internals yet. What follows is the documented behavior plus inference from known AWS primitives — I'll flag which is which.

Why Lambda Wasn't Enough

Look at what an agent session actually needs versus what Lambda's execution model offers:

	Lambda's model	What agents need
Duration	15-minute hard cap	Conversations and tasks spanning hours
State	Stateless between invokes	Context accumulated across turns
Duty cycle	Billed on wall-clock × allocated memory	30–70% of time idle, waiting on LLM calls
Isolation unit	Per function — one user's invoke can reuse a sandbox another user warmed	Per session — agent behavior is non-deterministic
Payloads	6 MB synchronous	Documents, images, datasets — 100 MB

The isolation row is the deep one. In Lambda, a sandbox belongs to a function version, and sequential invokes from different end users of your app happily reuse the same warm sandbox. That's fine when the code is deterministic — it does what you wrote, every time. An agent is different: an LLM decides at runtime what code runs, what files get written, what APIs get called. A sandbox that just finished serving user A might have A's downloaded files sitting on disk when user B's request lands in it. For deterministic code that's a non-issue; for an agent it's a data leak waiting for a sufficiently creative prompt.

You could force per-user isolation onto Lambda yourself — but then you're rebuilding session routing, state pinning, and lifecycle management on top of a platform actively designed to hide placement from you.

The Core Primitive: A microVM per Session

AgentCore Runtime's answer is to move the isolation boundary from the function to the session:

Lambda:                          AgentCore Runtime:

┌─ sandbox: function F ─┐        ┌─ microVM: session abc ─┐
│  invoke (user A)      │        │  user A's conversation │
│  invoke (user B)  ⚠   │        │  turn 1, 2, 3... state │
│  invoke (user A)      │        │  accumulates safely    │
└───────────────────────┘        └────────────────────────┘
                                 ┌─ microVM: session def ─┐
                                 │  user B's conversation │
                                 └────────────────────────┘

Every session — identified by a runtimeSessionId your application supplies (or the Runtime generates on first invoke) — gets a dedicated microVM with isolated CPU, memory, and filesystem. Invoke again with the same session ID and you land in the same microVM, with everything exactly as your agent left it: conversation history in memory, files on disk, connections open. Statefulness isn't bolted on; it's what "your session is a VM" gives you for free.

For the AgentCore Tools (Code Interpreter and Browser), AWS explicitly documents this as one session, one Firecracker microVM. The Runtime docs say only "dedicated microVM" — but it's the same isolation story, and after the Lambda deep dive you know exactly what that machinery looks like: a minimal Rust VMM, a guest kernel per sandbox, ~milliseconds-cheap enough to hand one to every conversation.

The end of a session is as important as the start: when a session terminates, the entire microVM is destroyed and its memory sanitized. Nothing — no filesystem artifact, no memory page — survives into any other session. Reusing a terminated session's ID doesn't resurrect anything; it builds a fresh environment.

The Session Lifecycle: Active, Idle, Terminated

A session moves through three states:

   first invoke                    request or
   (cold start:                    background work
   provision microVM)                  │
        │                              ▼
        └────────▶  ACTIVE  ◀────── IDLE
                      │    ────────▶  │
                      │   work done   │ 15 min inactivity,
                      │               │ 8 hr max lifetime,
                      ▼               ▼ or failed health check
                    TERMINATED (microVM destroyed,
                                memory sanitized)

Active — processing a request or running background tasks.
Idle — no request in flight, but the microVM stays provisioned, context intact. Follow-up invokes hit a warm environment: within a session, there is no such thing as a second cold start.
Terminated — reaped after 15 minutes of inactivity, at the 8-hour maximum lifetime, or on failed health checks (both limits are defaults you can tune in lifecycle settings).

The corollary: session state is ephemeral by design. The microVM's memory and disk are a scratchpad, not a database. Anything that must outlive the session — conversation history, user preferences, extracted facts — belongs in AgentCore Memory (a separate service, and a separate article).

The Service Contract: Bring a Container, Speak Two Routes

Like Lambda's container support, your agent ships as an OCI image in ECR (linux/arm64). Unlike Lambda, there's no proprietary handler interface — the contract is plain HTTP on port 8080:

POST /invocations — the entrypoint. Receives the request payload (up to 100 MB), returns a response or streams partial results as Server-Sent Events. Long tasks stream progress instead of making the user stare at a spinner.
GET /ping — the health check, which turns out to be load-bearing (next section).

Anything that can serve those two routes runs here: LangGraph, CrewAI, Strands Agents, or a hand-rolled loop — any framework, any model provider, in any language. The Runtime doesn't know or care that there's an LLM inside.

Beyond plain HTTP, the Runtime natively hosts MCP servers (so your tools speak Model Context Protocol to any client), A2A servers (agent-to-agent communication and discovery), and WebSocket endpoints for bidirectional streaming — same microVM-per-session substrate, different wire protocol on top.

Deployments borrow Lambda's versioning scheme wholesale: every configuration change creates an immutable version, and named endpoints (a DEFAULT one tracks latest; you create others for dev/test/prod) point at specific versions. Rollback is repointing an endpoint, not redeploying.

HealthyBusy: How Agents Outlive Their Requests

The 15-minute idle reaper poses a problem for genuinely agentic work. An agent asked to "analyze these 500 documents" should respond immediately — "on it, check back later" — and keep grinding in the background. But a session that isn't serving requests looks idle, and idle sessions get terminated.

The escape hatch is the /ping contract. The health check returns one of two statuses:

Healthy — idle, safe to reap on the normal timeout.
HealthyBusy — no request in flight, but background work is running; keep this session alive.

The Runtime polls /ping, and a HealthyBusy session survives past the idle timeout — up to the full 8-hour lifetime. Your agent responds to the caller, spawns its background task, flips its status, and works autonomously; the client polls back in on the same session ID whenever it likes. Asynchronous, long-running agents fall out of the health check protocol — no queues, no step functions, no task orchestration service.

It's a neat inversion of Lambda, where the sandbox is frozen the instant your handler returns and background threads simply stop running. Here, the platform asks the workload whether it's done.

The Pricing Model Is an Architecture Diagram

AgentCore Runtime bills CPU and memory separately, and the asymmetry between them tells you how the fleet works:

CPU (~$0.0895 per vCPU-hour): charged on actual consumption. While your agent blocks on an LLM response or a tool call, CPU charges are approximately zero. I/O wait is free.
Memory (~$0.00945 per GB-hour): charged on peak footprint for every second the session exists, idle time included.

Compare Lambda, which bills allocated-memory × wall-clock milliseconds — waiting ten seconds for a model response costs the same as computing for ten seconds. For a workload that's 30–70% I/O wait, that's the majority of your bill spent on await. AWS's own illustrative example: a 60-second support-agent session with 18 seconds of actual CPU comes out ~70% cheaper than duration-based billing.

Why price it this way? Because they can — and the can is the architecture (this is the inference part, but it follows directly). A blocked process consumes no cycles, so a worker packed with hundreds of session microVMs that are mostly awaiting LLM responses can oversubscribe CPU aggressively — the same bet telephone networks and Lambda's placement service make, applied to agent duty cycles. Memory is different: an idle session's conversation state stays resident, so memory is the resource the fleet actually runs out of. Hence memory bills for the session's whole lifetime, and hence the 15-minute idle reaper — it's the mechanism that bounds how long AWS holds RAM for a session that may never come back.

Read it as incentives: the platform charges you for exactly the resource your idle session pins (memory), gives away the resource it can oversubscribe (idle CPU), and reaps the sessions that would break the model.

The Fences: Quotas, Throttles, and What They Reveal

The service quotas read the same way the pricing does — each fence sits in front of a resource the fleet actually worries about. Here's the envelope you get by default (all numbers as of July 2026; most are adjustable via Service Quotas):

Quota	Default	What it's really metering
Active session workloads per account	5,000 (us-east-1, us-west-2); 2,500 elsewhere	Resident microVMs — RAM the fleet is holding for you
New sessions per endpoint	400/minute (container images)	The cold-start machinery: placement + microVM provisioning
`InvokeAgentRuntime`, per agent	200 requests/second	The warm path — routing into already-running microVMs
Synchronous request timeout	15 minutes	One turn
Streaming connection	60 minutes	One long turn
Async (`HealthyBusy`) work	8 hours	One session lifetime
Hardware per session	2 vCPU / 8 GB, 1 GB disk	Sessions scale out, not up

Two things jump out. First, the warm/cold asymmetry: you can invoke 200 times per second, but create only ~6.7 sessions per second (400/minute). Riding an existing microVM is cheap; conjuring one is the guarded operation — which is exactly what the lifecycle section would predict. Second, the numbers are policy, not physics: AWS raised these defaults roughly 5× in July 2026 (concurrent sessions 1,000 → 5,000, invokes 25 → 200 TPS, session creation 100 → 400 TPM), so treat any blog post's numbers — including this one's — as a snapshot and check the Service Quotas console.

There's also a fun echo in that table: Lambda's infamous 15-minute cap didn't disappear — it reincarnated as the length of a single synchronous turn.

The Throttle Underneath: Model Tokens per Minute

Here's the thing the Runtime quota table doesn't show: for most agent fleets, none of those limits bind first. The real ceiling is the model. Bedrock enforces per-model, per-region on-demand quotas — requests per minute (RPM) and tokens per minute (TPM) — and every session in your fleet drains the same regional bucket.

The TPM accounting has teeth:

Input and output tokens both count, and output tokens burn down at a model-specific multiplier — Bedrock's quota docs currently list 15× for Claude 4.8, 10× for Sonnet 5, 5× for older Anthropic models, and 1:1 for most others. A verbose agent is quota-expensive out of proportion to its bill.
max_tokens is reserved at admission, not on use. Set max_tokens: 8192 "just in case" on every tool-selection call and you're burning quota headroom for tokens you never generate — a classic self-inflicted throttle.
Exceed either quota and you get a ThrottlingException, which your agent loop must treat as backpressure: exponential backoff with jitter, client-side concurrency caps, and cross-region inference profiles to spread load (invocations via a profile can reach up to double the local region's default quota).

Do the arithmetic on a concrete fleet: 1,000 concurrent sessions, each making an LLM call every 30 seconds at ~10K tokens of quota cost each, is ~20M TPM of demand against a bucket that is typically single-digit millions. Your 5,000-session allowance is an empty promise if the model bucket drains at session #400. Runtime quotas are account fences you can raise with a ticket; model TPM is the shared scarcity everything competes for.

Two Fleet Shapes: Real-Time Chat vs. Overnight Batch

Those constraints push interactive and bulk workloads toward opposite designs.

Real-time chat maps naturally onto the primitive: one conversation, one session. Turns ride the warm path, where 200 TPS per agent is generous. The two things to engineer for are the creation-rate spike — Monday 9am, everyone opens a fresh conversation at once, and 400 new sessions/minute becomes your login-storm admission rate, so queue conversation starts rather than failing them — and the idle-timeout trade-off: a short timeout saves memory-hours, but a user who replies at minute 16 lands in a fresh microVM with amnesia, so pair the reaper with AgentCore Memory rehydration rather than pretending sessions are immortal.

Overnight bulk processing should not map one work item to one session, tempting as it is. Ten thousand documents as ten thousand sessions means 25 minutes of pure admission at 400/minute, ten thousand cold starts, and ten thousand memory-lifetimes billed. The better shape is a worker pool: a few dozen long-lived sessions, each pulling items off a queue, reporting HealthyBusy while it grinds, and retiring at the 8-hour maxLifetime like a shift worker. Size the pool from the model budget, not the session quota — if each document costs ~50K tokens of quota against a 2M TPM bucket, the pipeline tops out at ~40 documents/minute no matter how many sessions you run; a handful of workers saturate that, and the other 4,900 sessions you're entitled to would just take turns catching ThrottlingExceptions.

And before building the worker pool at all, ask which pipeline steps even need an agent. A step that's one model call per item — classify, summarize, extract — can leave the agent loop entirely and run as Bedrock batch inference: S3 manifest in, S3 results out, typically complete within 24 hours, at 50% of the on-demand token price (with priority/flex service tiers for select models filling the gaps between real-time and batch). Save the sessions — and the TPM — for the steps that genuinely need tools, iteration, and judgment. The cheapest agent invoke is the one you turned into a batch job.

Everything Else Rides on the Same Primitive

Runtime is one service in the AgentCore suite, and the others lean on the same session isolation:

AgentCore Tools — Code Interpreter and Browser, each session in its own Firecracker microVM: the agent executes LLM-generated code or drives a real browser inside a disposable VM, not inside your process.
AgentCore Identity — inbound auth (IAM SigV4 or OAuth bearer tokens validated before your container ever sees the request) and outbound auth (a token vault exchanging validated user tokens for scoped credentials to Slack, GitHub, AWS APIs), so secrets never live in agent code.
AgentCore Gateway — turns Lambda functions and OpenAPI specs into MCP tools your agent can call.
AgentCore Memory — the durable store for what must survive the microVM's death.

Gateway and Memory each have enough machinery underneath for their own deep dives.

Key Takeaways

The unit of isolation follows the unit of trust. Lambda isolates customers from each other and lets one function's users share warm sandboxes — fine for deterministic code. Agents execute LLM-directed behavior, so AgentCore shrinks the blast radius to a single session: one conversation, one microVM, destroyed and sanitized at the end.

Statefulness is a placement decision, not a feature. Pin a session to a dedicated microVM and "state between turns" stops being something you engineer with external stores — it's just process memory that's still there.

The health check is the coordination protocol. Healthy vs HealthyBusy is the entire async-agent API: the platform polls, the workload answers, and long-running autonomy falls out without any queueing infrastructure in your architecture.

Pricing reveals architecture. Consumption-billed CPU means the fleet oversubscribes cores against agents' I/O-heavy duty cycles; lifetime-billed memory plus a 15-minute idle reaper means resident RAM is the scarce resource. The bill is a map of what's cheap and expensive inside the fleet.

Size fleets against the scarcest pool, which is usually the model. Runtime quotas are account-level fences you can raise with a ticket; the per-region model TPM bucket is shared scarcity that every session drains. Worker pools beat session-per-item for bulk work, and any step that's a single model call per item belongs in Bedrock batch inference at half price, not in an agent loop.

It's Lambda's parts, re-composed. MicroVMs, ECR images, immutable versions, endpoint aliases — nothing here required new physics. The invention is aiming the same primitives at sessions instead of invokes, and letting everything else (state, async, isolation) fall out of that one move.