System Prompt

About System Prompt

System Prompt is a podcast about what’s actually happening in AI. Not hype. Not surface-level takes. We break down how AI is changing software, SaaS, infrastructure, and the way systems are built focusing on real-world tradeoffs, architecture decisions, and where the value is actually shifting. If you’re building, deploying, or thinking seriously about AI, this is for you.

Publisher

Peter

Category

business · technology

Language

en

Explicit

Yes

First episode

19 Mar 2026

Latest episode

10 Nov 2026

Latest episodes

33 episodes in the feed.

1. 

   10 Nov 2026

   AI Software Factories | The Future of Software Engineering?

   Episode 33: AI Software Factories | The Future of Software Engineering? What happens when AI coding agents move beyond individual tasks and become part of a repeatable engineering process? In Episode 33 of System Prompt, Peter and Val explore AI software factories: systems designed to take engineering work from tickets through implementation, verification, review, and delivery. Drawing on AI DevCon NYC, Peter shares lessons about verification-aware programming, engineering standards, and deterministic guardrails. They discuss how preconditions, postconditions, and invariants can make specific software properties mathematically verifiable, reducing reliance on AI-generated tests and model judgment alone. But building a reliable factory introduces new challenges. How do you prevent agents from drifting away from specifications? Who makes architectural decisions? And can multiple agents coordinate without introducing more complexity than they solve? Val raises the economic question: do software factories actually reduce development costs, or do inference, orchestration, and supervision erase the savings? They close by examining greenfield development, the future of engineering teams, and how AI factories could reshape software development and SaaS businesses. IN THIS EPISODE • Coding agents vs. AI software factories • Lessons from AI DevCon NYC • Verification-aware programming and mathematical proofs • Engineering constitutions and specifications • Preventing agent drift and unreliable testing • Multi-agent orchestration and independent review • Ticket-to-PR workflows and human oversight • Open-source models, inference costs, and ROI • Greenfield development and future engineering teams TOOLS & RESOURCES AI DevCon NYC: https://tessl.io/devcon (https://tessl.io/devcon) LemmaScript — Verification-aware programming: https://docs.lemmascript.org/ (https://docs.lemmascript.org/) Tessl — Agentic software engineering: https://tessl.io/ (https://tessl.io/) KEY TAKEAWAYS THE FACTORY IS MORE THAN THE AGENT Reliable factories need repeatable workflows, engineering standards, authorization boundaries, and trustworthy verification. SPECIFICATIONS DRIVE THE OUTCOME Agents can implement misunderstood requirements. Clear specifications and human judgment remain essential. VERIFICATION GOES BEYOND AI REVIEW Formal contracts can mathematically verify specified properties. A second AI reviewer is useful but cannot replace mechanical verification. THE HARNESS MATTERS Tools, execution environments, context, and deterministic controls shape reliability as much as model capabilities. ECONOMICS DETERMINE ADOPTION Cheaper inference doesn't guarantee cheaper development. Cost per accepted task is what matters. ENGINEERING ROLES WILL EVOLVE Architecture, design, verification, and domain expertise may become increasingly important as implementation becomes automated. CHAPTERS 00:00 Introducing AI Software Factories 01:52 What Is an AI Software Factory? 04:43 Engineering Standards and Workflows 07:28 Lessons from AI DevCon NYC 09:57 Mathematical Verification and Contracts 13:47 Verification-Aware Programming 15:43 More AI DevCon Insights 20:35 Engineering Challenges and Specifications 23:42 Quality, Security, and Coordination 27:00 Engineering Constitutions 28:16 Optimizing Prompts and Skills 30:56 Factory Workflows in Practice 32:46 Starting with Ticket-to-PR 35:03 The Software Factory Assembly Line 38:22 Models and Coding Harnesses 41:37 Do Factories Make Economic Sense? 44:59 Cost per PR and Business ROI 47:36 Model Selection and Cost per Task 48:00 Greenfield Development 50:53 The Future of Engineering and SaaS 55:59 AI Infrastructure and Competition 57:14 Final Thoughts

2. 

   30 Oct 2026

   How to Pick the Right AI Model?

   Episode 32: How to Pick the Right AI Model | DeepSeek vs Qwen Evals You've seen how an AI agent is built, configured, and given a job. But how do you know if the model behind it is right for the work? In Episode 32 of System Prompt, Peter and Val close out their agent series with a live demonstration of AI evaluations, or "evals." Instead of trusting public benchmarks or model reputation, they test the tasks that matter to a real workflow. Using smevals, they compare DeepSeek V4.1 Flash running locally across two NVIDIA DGX Sparks against Qwen 3.8 27B. Their tests cover structured data extraction, instruction following, reasoning, and coding. The smaller Qwen performs surprisingly well. But the real lesson comes from unexpected failures: DeepSeek adds a timezone to an otherwise correct answer, while Qwen initially returns no output on a task it passes when rerun. Peter and Val examine the outputs, question the grading rules, and troubleshoot live rather than presenting a polished happy path. They also explore how harnesses handle model behavior, why larger models aren't always necessary, and how to connect performance, hardware costs, and business ROI. The goal isn't to choose the biggest model. It's to choose one that can reliably do your job at a sensible cost. IN THIS EPISODE • Why benchmarks don't always represent your workload • How smevals uses tasks, runners, graders, and reports • DeepSeek V4.1 Flash vs Qwen 3.8 27B • Structured data, instruction following, reasoning, and coding tests • Diagnosing grading mismatches and missing responses • Why repeatability and failure severity matter • Harness engineering and deterministic validation • Local hardware costs, model selection, and business ROI TOOLS & RESOURCES smevals — Open-source evaluation framework: https://github.com/prime-radiant-inc/smevals (https://github.com/prime-radiant-inc/smevals) KEY TAKEAWAYS BENCHMARKS AREN'T YOUR WORKLOAD Test against your real inputs, constraints, and expected outputs—not a leaderboard alone. NOT ALL FAILURES ARE EQUAL A formatting mismatch, empty response, or strict grader doesn't automatically prove a model can't do the task. Investigate the cause. SMALLER CAN BE GOOD ENOUGH For routine tasks, a 27B model may meet requirements that don't justify the cost of running a much larger model. EVALUATE THE EVALUATIONS Graders, runtimes, and model outputs can all vary. Rerun tests and examine failures before drawing conclusions. THE HARNESS MATTERS Prompts, tools, validation, and deterministic logic shape the reliability of the full system—not just the model. ROI STARTS WITH THE WORKFLOW Understand the task, its cost, and the value of automating it before deciding which model to deploy. CHAPTERS 00:00 Closing the Agent Series: Picking a Model 01:19 Why Evals Matter 04:52 Introducing smevals 07:33 DeepSeek vs Qwen 09:25 Running the Evaluation Suite 12:04 Structured Data Results 14:23 Instruction-Following Failures 15:49 Investigating a Grading Mismatch 18:29 Failure Severity and Impact 19:48 Deterministic Output Handling 21:14 Harness Engineering 23:03 Models vs Agent Harnesses 25:19 Investigating Qwen's Missing Response 26:28 Repeated Evaluations 27:21 Rerunning Tests 28:57 Human Bias vs Data 29:41 Coding Results 32:01 When Smaller Is Good Enough 34:56 Productivity and ROI 37:55 Understanding Business Problems 40:09 Wrap-Up

3. 

   23 Oct 2026

   Setting Up Hermes Agent: From Configuration to Automation

   You've seen what goes into an AI agent. Now what does it take to actually configure one and give it a job? In Episode 31 of System Prompt, Peter and Val move from the architecture of Hermes Agent to a hands-on walkthrough of its Windows desktop interface. With Hermes already installed and connected to DeepSeek V4.1 Flash running locally across two NVIDIA DGX Sparks, they explore the settings that shape an agent's behavior, access, memory, and execution. They work through model selection and fallbacks, chat personalities, workspace configuration, approvals, file checkpoints, context compression, providers, tool integrations, and plugins. Not every feature needs changing, and not every experiment works as expected. That's part of the demonstration: no prearranged happy path. Then they give Hermes a real task: create a scheduled 8 a.m. AI news briefing, add sentiment analysis, inspect sources and output, and request a more polished PDF format. The conversation expands into what makes an automation useful—from daily operational reporting and ad analytics to finding RFPs and preparing proposals. The question isn't just what an agent can do. It's which steps in your existing workflow are worth handing off. • Configuring Hermes through the Windows GUI • Primary models, reasoning settings, fallbacks, and auxiliary models • Personalities, attachments, interface settings, and workspaces • Command approvals, secret redaction, file checkpoints, and browser access • Persistent memory and context compression • Agent step limits, subagents, and local versus cloud providers • Search tools, custom integrations, MCP, skills, and plugins • Testing web access with a locally hosted model • Creating an automated AI news brief with sentiment analysis • Applying agents to real business workflows KEY TAKEAWAYS CONFIGURATION IS PART OF AGENT ENGINEERING Choosing a model is only the beginning. Workspace access, approval rules, memory, tools, and execution limits determine how an agent operates. LOCAL DOESN'T MEAN UNRESTRICTED A locally hosted model can use web tools and run workflows, but access to browsers, files, credentials, and commands still needs deliberate controls. YOU DON'T NEED TO CHANGE EVERY SETTING Some defaults are fine. Understand what each setting governs, then adjust it based on the task and the risk. A WORKING DEMO ISN'T A FINISHED EVALUATION Hermes creates a schedule and generates an initial briefing, but repeatability, source quality, and output consistency still need testing. START WITH YOUR WORKFLOW, NOT THE TOOL Look at what you retrieve, compare, update, email, and report each day. The valuable agent is the one that removes real work from that process. CHAPTERS 00:00 From AI Infrastructure to a Working Agent 02:29 Models, Reasoning, and Fallbacks 03:30 Chat Personalities and Attachments 05:39 Appearance and Desktop Preferences 07:14 Workspaces, Approvals, and File Safety 08:44 Memory, Compression, and Voice 10:51 Advanced Settings and Agent Limits 12:00 Providers, Local Models, and Gateways 14:00 Tool Integrations, Plugins, and Skills 16:55 Can Hermes Build Its Own Tools? 18:24 Testing Web Access 19:20 What Should You Actually Do With an Agent? 21:12 Building a Scheduled AI News Brief 28:18 Adding Sentiment Analysis 30:58 Agent Personalities and Human Interaction 33:45 Reviewing the Briefing and Sources 37:20 Requesting a PDF Output 39:01 RFPs and Business Automation Opportunities 41:59 Finding Workflows Worth Automating 47:05 Wrap-Up and What's Next

Reach and audience

Public platform figures. Ratings count people who left a rating, not total listeners.

Spotify

5.0 / 5

1 ratings