Helium: What If Your Agent Framework Had a SQL Optimizer?

Your AI agent is wasting most of its compute budget on work it already did three calls ago.

Paper: "Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective" by Noppanat Wadlom, Junyi Shen, and Yao Lu (arXiv:2603.16104, March 17, 2026)

---

That's the uncomfortable truth behind a new paper from a team that looked at how modern agentic systems — the kind that chain multiple LLM calls together, spawn sub-agents, debate with themselves, and speculatively explore solution paths — actually consume GPU resources. The answer? Badly. Very badly.

The paper introduces Helium, a workflow-aware serving framework that treats agentic workloads the way a database treats SQL queries: as structured plans that can be analyzed, optimized, and executed with maximum reuse. The result is up to 1.56x speedup over current state-of-the-art systems — not by making the model faster, but by making the plumbing smarter.

The Problem: Operator-Level Myopia

If you've built anything with LangGraph, CrewAI, or even raw multi-agent orchestration, you know the pattern: Agent A processes context, Agent B gets A's output plus shared instructions, Agent C debates with both, and a summarizer wraps it all up. Each of those steps hits an LLM endpoint independently.

Current serving systems like vLLM are exceptionally good at optimizing individual inference calls — continuous batching, PagedAttention, prefix caching with LRU eviction. But they're fundamentally blind to the workflow above them. They optimize each call in isolation, completely unaware that Agent B's prompt shares 80% of its tokens with Agent A's, or that the summarizer's system prompt hasn't changed since yesterday.

The authors call this "operator-level myopia," and it manifests in three ways:

1. No cross-call KV cache coordination. In a multi-agent debate, every turn rebuilds shared conversational history from scratch instead of extending the existing KV cache.
2. Passive prefix caching. Existing systems react to cache hits opportunistically — they can't anticipate which prefixes will be needed next based on workflow structure.
3. Optimizer blindness. When LLM calls are wrapped as black-box UDFs (user-defined functions) in frameworks like Spark or Dask, the query optimizer can't see inside them to identify redundancy.

The Insight: LLM Calls Are Just Expensive Database Operators

Here's where the paper gets clever. The authors observe that an agentic workflow is structurally equivalent to a database query plan — a DAG where nodes are operators (LLM calls, data retrievals, transformations) and edges are data flows. This isn't a loose analogy. They demonstrate that decades of query optimization research directly applies:

• Common subexpression elimination catches duplicate computation across agents
• Dead code elimination prunes speculative branches that never contribute to output
• Cost-based scheduling orders operations to maximize cache reuse
• Proactive caching pre-warms KV states for predictable prefixes instead of reacting to cache misses

Helium implements this through three key components:

1. A Workflow-Aware DSL and DAG Parser

Developers define workflows in a Python-based DSL that constructs a symbolic DAG at compile time. Rather than executing eagerly, operations are recorded as a graph — similar to how TensorFlow builds computation graphs before execution. This gives the optimizer full visibility into the workflow structure before a single GPU cycle is spent.

2. The Templated Radix Tree (TRT)

This is the paper's most novel data structure. A standard radix tree organizes strings by shared prefixes. Helium's TRT extends this to handle both static prompt components (system prompts, instructions) and dynamic placeholders (outputs from upstream operators). The TRT captures the global prefix hierarchy across all operators in a workflow, enabling the scheduler to identify exactly which KV cache states can be shared and when.

This matters because existing approaches like SGLang organize caches within individual calls but can't see across the workflow boundary. Parrot (a prior system) tracks dependencies but not prefix structure. The TRT does both simultaneously.

3. Proactive Cache Management

Instead of waiting for cache hits, Helium pre-computes and pins KV cache states for static prompt prefixes during the first batch execution. Subsequent batches skip prefill entirely for these segments. A global prompt cache also stores complete outputs of deterministic operators, allowing the system to bypass entire LLM calls when inputs haven't changed.

Key Results: How Much Does This Actually Help?

The evaluation is thorough, covering five primitive workflow patterns (map-reduce, multi-agent debate, reflection, iterative refinement, parallel chains) and a complex financial trading workflow that combines multiple patterns. Tests used Qwen3-8B and Qwen3-14B on dual NVIDIA H100 NVL GPUs.

Microbenchmark results (normalized latency, lower is better):

The 100x vs. naive vLLM reflects the baseline's sequential execution — not a realistic comparison. The more meaningful numbers are the 1.3-1.56x improvements over Parrot and KVFlow, which are already optimized for agent workloads.

End-to-end on the complex financial trading workflow: Helium achieved 1.34x speedup with proactive caching contributing the most gains on workflows with repeated static prefixes.

Ablation highlights:

• Proactive KV caching alone contributed 1.18x improvement on the trading workflow
• Cache-aware scheduling added another 1.14x
• The prompt cache (bypassing entire operator executions) provided 1.08x on top

What This Means for Practitioners

If you're running agentic workflows in production — or planning to — there are several takeaways:

Limitations Worth Noting

The paper is honest about its scope constraints:

• All agents must use the same base LLM — workflows mixing GPT-4 and Claude aren't supported
• Only on-premise deployment is evaluated — cloud API latencies would change the optimization calculus
• Deterministic caching only works with greedy sampling (temperature=0) — stochastic sampling breaks cache reuse
• No support for agents that make remote API calls — only local LLM and data operations

These are reasonable simplifications for a research paper, but they highlight how far we are from production-grade workflow optimization.

Our Take

The code is open source at github.com/mlsys-io/helium_demo. If you're deploying self-hosted agents on your own GPUs, this is worth a serious look.

---

Citation: Wadlom, N., Shen, J., & Lu, Y. (2026). Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective. arXiv:2603.16104. https://arxiv.org/abs/2603.16104