The Infra Pod brings you insightful and thought-provoking discussions on the world of infrastructure software. This podcast is started by two engineers, Ian Livingstone (tech advisor for Snyk) and Tim Chen (General Partner at Essence VC), team up with a rotating cast of guests to dive deep into the latest trends and hot topics in the software infrastructure space.
AI Is the Fourth Application Architecture! Chat with Dave McJannet (ex-CEO of Hashicorp, CEO of Dome Systems)
Dave McJannet — CEO of Dome Systems, former CEO of HashiCorp, and a veteran of Cloud Foundry, Hortonworks, and GitHub — joins Tim (Essence) and Ian Livingston (Keycard) to argue that AI isn't a feature wave, it's the fourth application architecture after mainframe, client-server, and cloud.
Dave walks the stack layer by layer to separate what endures from what gets layered over: compute shifting from CPU to GPU and onto an inferencing layer, messaging (Kafka vs. Temporal) becoming more critical, not less, APM racing to monitor a new substrate, and a data stack that probably doesn't get rebuilt at all. He makes the case that agent governance is a DevOps problem, not a security one — the platform-engineering team that can't answer "what will this thing do once deployed?" is what's actually blocking enterprise agents today, exactly like security teams stalled cloud growth in 2015 before the cloud 2.0 playbook unlocked it.
Plus: why he thinks the open-source business model is broken in a world where an agent can reconstruct your repo overnight, what replaces it as a moat, why heterogeneity of models is inevitable (and why India got there first), and a spicy take that vertical SaaS and the incumbents are far more defensible than the Valley believes.
- 0:00 — Intro & AI as the fourth application architecture
- 12:39 — Why Dome Systems: agent governance as a DevOps problem
- 22:56 — Enterprise adoption, heterogeneity, and the death of open source as a moat
- 35:49 — The Spicy Future: hot takes and closing thoughts
24 Aug 2026
1,000 tokens per second: the case against predicting one word at a time (Kumar, VP of Engineering at Inception)
What if the entire LLM industry has been solving language generation the slow way — one token at a time?
In this episode of The Infra Pod, hosts Tim Chen (GP at Essence VC) and Ian Livingstone (CEO of Keycard) sit down with Kumar, VP of Engineering at Inception, to unpack Mercury 2, the company's diffusion-based language model, and make the case for a fundamentally different way to generate text and code.
Kumar breaks down the core mechanical difference: where a GPT-style transformer predicts the next token one pass at a time, a diffusion model predicts a whole batch of upcoming tokens at once and iteratively denoises them in parallel, freezing the easy ones early and spending extra compute only on the hard ones. That approach — borrowed from image generation but re-engineered for text, where output length isn't known in advance and streaming is a hard requirement — yields roughly a 10x speedup and 3-5x cost efficiency by simply doing fewer forward passes. Mercury 2 hits 1,000 tokens/second on commodity NVIDIA hardware and now benchmarks competitively against cost-optimized models like Claude Haiku, Gemini Flash, and GPT-mini, though Kumar is candid that no diffusion model — Inception's included — has yet reached Sonnet or frontier-tier intelligence.
The conversation moves from algorithm to product: why Inception keeps its API OpenAI-compatible (Kumar's electric-car analogy — same interface, very different feel under the hood), why most agentic sub-tasks don't need frontier intelligence at all, and why voice and search are the workloads where sub-second latency stops being a nice-to-have and becomes existential. Kumar closes with a genuinely spicy take!
[00:00] Guest introduction: Kumar, VP of Engineering at Inception Labs
[01:31] Diffusion vs. autoregressive LLMs: what's actually different under the hood
[07:00] Why isn't diffusion the default already? Trade-offs and diffusion's late start on text
[09:05] From Mercury 1 to Mercury 2: the road to enterprise-readiness
[11:26] Open source diffusion models, ICML's best paper, and Inception's head start
[16:15] Under the hood: how Inception hits 1,000 tokens/sec without sacrificing latency
[20:52] Data strategy: what training a diffusion model actually requires
[22:51] Does diffusion change how you build agents and products on top of it?
[28:00] Mercury 2 benchmarked against cost-optimized frontier models
[29:23] The model routing problem — and why it may already be mostly solved
[37:47] Spicy Future: AGI
20 Jul 2026
What happens to your service mesh when the workloads running on it aren't written by humans? (Chat with William at Buoyant)
Hosts Tim Chen and Ian Livingstone interview William Morgan, co-founder and CEO of Buoyant, about how AI agents are transforming infrastructure and service meshes.
Who has been a guest on Yet Another Infra Deep Dive
Names identified in recent episode analyses. Showing up to five guests.
William Morgan
Tyler Wells
Shubo
Benny Chen
Louis Vichy
Reach and audience
Public platform figures. Ratings count people who left a rating, not total listeners.
Apple Podcasts (US)
5.0 / 5
2 ratings
Spotify
4.8 / 5
17 ratings
Podcast Authority Score: 63 / 100
A composite of feed quality, social presence, YouTube performance and engagement. Read the methodology.
Quality
73
Social presence
0
YouTube
63
Engagement
71
Contact Yet Another Infra Deep Dive
Guest appearances
Books guests
Based on episode analysis; this does not confirm that the show is currently accepting guests.
Pod Engine is an independent podcast discovery and analytics service and is not affiliated with or endorsed by this podcast. Artwork and show content belong to their owners. Full legal notice.
Explore this show Podcast research with Pod Engine
