Talking Product

About Talking Product

John Young & Collin Lyons explore all things related to building digital products and leading digital transformations. In every episode we give you actions that you can put into practice immediately to reduce risk, create more effective and efficient product development capabilities, and build a culture of continuous learning.

Publisher

John Young & Collin Lyons

Category

business · technology

Language

en

Explicit

No

First episode

17 Jul 2023

Latest episode

9 Aug 2026

Latest episodes

15 episodes in the feed.

1. 

   9 Aug 2026

   Episode 15 – LLMs as System Components: AI Hype vs ROI

   Over the last several episodes of Talking Product, Collin and I have been exploring how we increase the probability that our AI investment will translate into business value. Recent news has shown how organisations are having to confront the economics of AI more directly: token budgets being consumed faster than expected, tighter controls on usage, the challenges of moving from prototype to production, how to shape experiments around coherent hypotheses tied to a business outcome, and whether the use of GenAI is actually translating into customer value. In Episode 15, we explore the mindset shift that I have experienced through working with the DSPy framework and becoming familiar with the work of Omar Khattab. This is not a technical discussion. Instead, Collin and I continue to explore the role non-technical senior leaders should play in shaping the AI journey. I believe anyone in an AI leadership role should familiarise themselves with Omar’s work if they haven’t done so already. I have found him to be a grounded, coherent voice in a world inundated by hyperbole and breathlessness. Omar Khattab is an Assistant Professor at MIT. His doctorate is from Stanford. His research focuses on how we program, evaluate and optimise systems built around language models. He created DSPy, an open-source framework for building and optimising LLM applications programmatically, rather than relying primarily on manually engineered prompts. In this episode, Collin and I explore: What belongs to LLM inference and what belongs to good ol’ deterministic programming. Defining tight, measurable evaluations — yes, we’ve been banging on about this since covering Chip Huyen’s book. The power of optimisation — don’t just believe us, go and look at the use cases on the DSPy website. Building on that optimisation, how lower-cost models can sometimes deliver equal and even better performance. The role non-technical senior leaders should play in shaping the AI journey. If you like the session, please subscribe through your preferred podcast service and leave us a smashing five-star review. It helps us build an audience and attract guests to the show. And, as always, feedback is more than welcome. Thanks. Links referred to in the show: Omar's home page: https://omarkhattab.com/ Two podcasts: How Foundation Models Evolved: A PhD Journey Through AI's Breakthrough Era: https://podcasts.apple.com/us/podcast/how-foundation-models-evolved-a-phd-journey-through/id1740178076?i=1000737229448 DSPy: Transforming Language Model Calls into Smart Pipelines // Omar Khattab: https://podcasts.apple.com/gb/podcast/dspy-transforming-language-model-calls-into-smart-pipelines/id1505372978?i=1000637537521 I also found this post helpful: https://x.com/lateinteraction/status/1921565300690149759

2. 

   11 May 2026

   Episode 14 – Who Is Asking the Questions About Digital Capability?

   In this episode of Talking Product, Collin and I discuss a tension we increasingly see inside organisations: digital capability is becoming an enterprise survival issue rather than a technology optimisation issue. Unlike some of our previous episodes, this is less a discussion about a single topic and more a conversation between two practitioners trying to reconcile lived experience with organisational inertia. We believe this tension is surfacing more visibly now because of the disruption caused by generative AI. Collin highlights an article I published on Substack at the end of last year. In it, I argued that the tolerance for organisational inefficiency and limited digital capability is likely to compress dramatically in the years ahead. Organisations that have invested time and energy into improving product delivery practices, architectures, infrastructure, data quality, and operating models are materially better positioned than those that have largely treated such initiatives as secondary concerns. This led me to a broader question: if executives and boards are not asking deeper questions about AI, data, software capability, and operating models, who is? My conclusion is that institutional investors need to engage far more directly with digital capability risk because many portfolios are likely carrying significantly more exposure than is currently recognised. We do not pretend to have definitive answers to many of these questions. In fact, part of the episode is an acknowledgement of just how difficult these topics are to navigate. But we once again argue for the importance of having these conversations inside organisations — because avoiding them does not reduce the risk. It simply delays the moment when those risks become visible. Lastly, we once again argue that getting Chip Huyen’s book AI Engineering and starting conversations inside your organisation about the subjects she raises will be very good value for money — and may save you a great deal of money and grief further down the road. Takeaways: An exploration of why digital capability is an enterprise-level risk rather than a delegated IT concern. A discussion about how generative AI accelerates the consequences of technical debt and organisational underinvestment. Questions around whether executives, boards, and institutional investors are asking sufficiently deep questions about AI, data, and operating models. The Digital Laggard Thesis: https://johnyoung.substack.com/p/towards-an-investment-thesis-on-digital

3. 

   27 Mar 2026

   Episode 13: More questions to improve the return on your AI investments

   In this episode, Collin and I continue building our list of questions to help increase the chances of your AI investments delivering a return. We focus in particular on the role non-technical C-level leaders should play in this effort. We use Chip Huyen’s AI Engineering as a practical framework, exploring how it gives digital leaders and non-technical senior managers a structured way to engage more deeply with AI initiatives. In particular, it provides a way into the ecosystem and lifecycle of AI application development—helping leaders ask better questions around things like data, prompt design, fine-tuning, and evaluation—so they can have more meaningful discussions with technical teams about how these applications are built, where the risks sit, and what criteria should be used to define and measure success. We walk through some of the key differences between traditional software and AI systems—particularly the shift from deterministic to probabilistic behaviour, and the central role of data in shaping outcomes. From there, we build on the questions we believe leaders should be asking: What problem are we solving? How are we evaluating outputs? How are we managing risks around data quality, safety, and factual accuracy? What trade-offs are we making between quality, cost, and latency? We spend some time looking at Chip’s section on evaluation criteria, using it as a springboard for non-technical senior leaders to delve deeper into the thinking behind—and expected outcomes of—AI applications. We also introduce the concept of “evals”—ongoing evaluation frameworks that extend beyond traditional testing—and why they require continuous iteration, collaboration, and oversight, even after deployment. This episode continues our exploration of how leaders can better understand what they are funding, engage more effectively with product and delivery teams, and create the conditions for AI investments to deliver real value. Links to Chip's book & interview on evals referred to in the episode: Chip Huyen’s AI Engineering: https://www.oreilly.com/library/view/ai-engineering/9781098166298/ Lenny’s podcast - Why AI evals are the hottest new skill for product builders | Hamel Husain & Shreya Shankar: https://www.youtube.com/watch?v=BsWxPI9UM4c

4. 

   8 Feb 2026

   Episode 12 - Questions worth asking when your AI investments aren't showing returns

   A significant number of AI initiatives are consuming budget and executive attention, while failing to reach production or deliver measurable value. Episode 12 of Talking Product helps you pressure-test your organisation's AI spend today. In this episode, Collin and I use a Financial Times article as a springboard for a conversation about the state of AI initiatives in large organisations. The article—"AI's awfully exciting until companies want to use it"—captures a familiar pattern in technology: high expectations, significant investment, but limited impact. The article is freely available with FT registration. In this episode, we explore three themes: Why pilots aren't scaling (often it's your data, not the technology) Whether organisations are bringing experimental rigour to AI adoption, or just buying impressive demos The leadership knowledge gap—understanding not just what AI can do, but what it can't What you'll get: Questions that will help reveal whether your organisation is truly learning from its AI experiments or just spending Insight into why data governance problems you've been kicking down the road are now becoming existential What Gartner found about hidden costs in AI initiatives—and questions to ask to bring them out into the open Practical guidance on what experimental rigour actually looks like in an AI context #AI #DigitalTransformation #Leadership #ProductManagement

5. 

   7 Dec 2025

   Episode 11 - Are we as product leaders really operating as a team?

   In this episode, Collin shares a case about a leadership team building a hardware and software product. We dig into this case because it highlights familiar product dynamics—great, passionate people, working to tight deadlines, but stuck in semi-siloed patterns and not really operating as one team. Each group pushes its own goals but isn’t fully aligned on the end game. And when the product leaders aren’t working effectively as a team, it has a real impact on outcomes and the ability to deliver at pace. We chose this topic to give product leaders footholds and practical ideas for starting similar conversations in your own organisation. We often call ourselves a team—but are we, as product leaders, really operating as a team? What do we really mean when we use the word “team”? In this session, Collin describes how he asked the leadership group that he was working with this simple, but powerful question: “Are you working as a team?” To their credit, they took the time to work with Collin and reflect on this question, as a collective. One area of the case that I found interesting was when Collin unpacks the complexity of “owning” requirements as new information emerges or thinking shifts about a particular problem. It is this inherent complexity of digital product development that makes siloed, and even semi-siloed, ways of working so dangerous. Collin also highlights a deeper systemic issue: senior and executive leaders often don’t grasp the importance of ways of working in digital product development. Instead, they simply push the silos to deliver faster. I personally feel that this level of naivety at the top is a serious competitive problem. I hope you find this case useful. We welcome recommendations and comments—and we’re looking forward to getting a few more episodes out, sooner rather than later. At the end of the podcast, I refer to the work of Nonaka & Takeuchi and said I would include a link: 1) The New New Product Development Game  and 2) The Knowledge Creating Company are probably their most well known work. But there is a wealth of other material that is also worth exploring.

6. 

   3 Jun 2025

   Episode 10 - Offsites that matter: Keeping the momentum alive

   Host discusses how coaches help teams maintain momentum after offsites by providing structure and accountability, navigating team dynamics, and addressing unconscious leadership beliefs.

7. 

   16 Nov 2024

   Episode 9 - What if you could measure what really matters to your customer? Alex Hidalgo on building an SLO culture

   What if you could measure what really matters to your customer? In our latest episode of Talking Product, Collin and I chat with Alex Hidalgo—author of the O’Reilly book Implementing Service Level Objectives. Alex shares how Service Level Objectives (SLOs) can help teams zero in on customer priorities and bring people across the organisation together to define what really matters. SLOs are powerful, but they can also be tricky to implement. I’ve struggled with them myself, so it was great to have the chance to explore this topic in depth with Alex and learn from his expertise. In this session, we cover: Practical steps to get started with Service Level Indicators (SLIs), the foundation of SLOs. How to start small and iterate as you refine your SLOs over time. The cultural challenges of implementing SLOs—and how to navigate them. If you’re just starting out or looking to improve your approach, I hope you find this episode full of insights to help you on your SLO journey. Alex’s book is an excellent resource for making SLOs meaningful and achievable in your organisation. I highly recommend it. I hope you enjoy listening to this session as much as we enjoyed recording it. If you liked the episode, please give us a rating—or go all out and write us a nice review. It makes a big difference! A huge thank you to Alex for making the time to share his insights with us.

8. 

   9 Oct 2024

   Episode 8 - VW's struggles with EV

   In this session Collin & I use Volkswagen’s struggle to build Electric Vehicle (EV) software as a way to explore two themes: the importance of getting a digital transformation right, and the opportunity presented to competitors when large players simply cannot deliver good quality digital products. We use two articles from the Financial Times as a springboard for our conversation. The first article, dated Jun 28, 2024 “Oliver Blume, VW and Porsche boss steering an EV strategy shift”  and the second one, published September 2, 2024 “VW considers closing factories in Germany and cutting jobs”  (both behind a paywall - sorry) The first article describes an investment of $5 billion in the US based EV startup Rivian. The purpose of the investment is to create a joint partnership that allows VW to use Rivian’s EV software. The article quotes VW’s CEO Oliver Blume as saying, “In terms of a big tech transformation, you can’t do it all on your own… It should be a win-win situation . . . The motivation from our side is to speed up software transformation at Volkswagen in all our brands. Rivian has best in class architecture . . . Volkswagen has the scale”. Collin and I explore this thinking of an acquisition / strategic partnership as a means for fixing an internal digital transformation challenge. For those of you who have listened to Collin and I, you probably won’t be surprised to hear that we are not convinced that such an approach will easily remedy digital product development challenges. As we ask in the show, what mental models will need to change for these two companies - a digital native and a non-digital native, to work effectively together? As Collin highlights, ultimately these transformations are culture changes, which require heavy-lifting. Since recording this session, the FT has come out with more articles on the challenges Volkswagen is facing. I have also started listening to some of the VW YouTube influencers who have been discussing software issues for a number of years. I am finding this to be a fascinating case study, albeit quite sad for the workers of Wolfsburg if the leadership of VW can’t get its digital act together. In this session, I also refer to a 2017 Medium post on how Revolut used the build-measure-learn cycle to develop digital products quickly and acquire customers. Hope you find the session useful. If you like the show, please give us a nice rating and recommend the show. It helps others find the podcast.

9. 

   28 Jul 2024

   Bringing reality into the room - part 3 of avoiding the pitfalls of data projects

   In this last episode of our three part series, Collin and I continue to role-play an imaginary data-heavy product development project. In our last episode, Collin was granted some authority to get more involved in the project in an effort to reduce the risk that Collin felt was inherent in the delivery team’s approach. The delivery teams is a consulting firm that has been brought in by another group in the company. In this episode, Collin elaborates on the types of risks he sees, as well as how he would address those risks through an alternative approach to product development. Collin and I are using this fictitious scenario to explore how ways of working, power dynamics and group-think play out in digital product development. In my opinion, these three forces are always at play in this type of work. Our goal in exploring this is to help listeners become better equipped to work with these forces on their own projects. In this episode, Collin and I discuss the fact that using an approach that pushes risk out into the open earlier in the product-development lifecycle brings reality into the room very quickly. It forces people to stop imagining “happy-path” outcomes through Power Point and Gantt charts. This can often be experienced as “raining on the parade.” In this episode, I ask Collin to help a senior manager identify the risks themselves by role playing the person who procured the consulting firm. In this scenario, a senior manager wants to be able to see the types of risks that Collin is seeing. This manager wants to be able to ask the questions that Collin is asking. And, they want to mitigate some of those risks without having to rely on delivery managers like Collin - “What is it that allows Collin to see digital product development in the way he sees it?” Through the role-play, we discover risks that were hidden in the consulting firm’s Power Point presentation and Gantt charts. Because of the approach the consulting firm was taking, made worse by shrouding it in Agile lingo, these risks would not have revealed themselves until very late in the consulting firm’s engagement. As Collin says, “If the consulting firm had followed that plan they would have delivered late and asking for an extension. That may be good for the bottom-line on the consulting side, but puts the company’s strategic goals at risk.” Hope you enjoy it. As always we would look forward to hearing from people. If you are liking these podcasts, please do throw us some love in the ratings. Thanks

10. 

    11 Apr 2024

    Episode 6 - Avoiding the pitfalls of data projects part 2

    In this episode, Collin and I continue to explore a digital product development project. Our project is heavily dependent on data and is to be delivered by an outside consulting firm. Our consulting firm says that they are are using Agile practices and principles to deliver the project, but as we learned in part 1, this team is in fact using a much more sequential “black-box” approach to  delivery. For Collin and I, this is a realistic scenario. It is something that we have seen multiple times in our careers. It is also a high risk project, not simply because of the heavy dependence on data, but also because of the proposed ways of working combined with the organisational and power dynamics of the situation. We feel that such dynamics are present in all real-world digital product development situations, but often not taken into consideration when talking about ways of working. Our goal in this exploration is to give listeners tangible actions that they can use when faced with a similar situation. In this episode, we explore further Collin’s idea of latent assumptions, which he introduced in episode 1. These assumptions are baked into projects, unrecognised as assumptions. Collin and I feel that this is particularly true for projects with a heavy reliance on data. Dependent upon the ways of working, teams may not learn that their assumptions were incorrect until very late in the project, often when the budget is nearly exhausted. We delve into the mechanisms that Collin is using to assess the risks within such a project and how to bring these risks into the light early to prove or disprove them? We spend a lot of time in this session exploring how do you get people with the right level of authority to recognise the risks of such a project, particularly when many of those people aren’t equipped to see such risks? We hope you enjoy it. As always please let us know what you think. And, if you liked the session, we would be very appreciative for a positive rating.

11. 

    21 Feb 2024

    Episode 5 - Avoiding the pitfalls of data projects part 1

    In this session, Collin & I explore a made-up data heavy project. It is based on projects that Collin and I have experienced previously in our careers. It is very typical of many digital projects. In this session, Collin and I are interested in exploring is the interplay between ways of working, power dynamics and the pressure to conform to the thinking of the group, within the context of a real-world delivery project. As Collin highlights in this session, these types of projects are rife with latent assumptions, which run the risk of getting baked into delivery. This problem is exacerbated by the mental model that guides the approach to delivery. Power dynamics and group think suppress the ability to bring those risks into the open in a way that allows a team to have a coherent constructive discussion. Our hope is that by unpacking this somewhat real-world case you will be in a better position to minimise the risk in a future project or a project that may be currently in flight. This is the first part of a three part exploration of this project.

12. 

    25 Nov 2023

    Episode 4 - How do you facilitate a mindset shift?

    We use the term "mindset shift" a lot in the digital transformation space. But what does that term actually mean? How do you facilitate the shifting of a mindset? In this session, Collin and I explore this idea of a mindset shift. We talk about why we feel it is such an important topic to examine if you hope to lead a succesful digital transformation. We put forward three “environmental conditions” that we believe are needed to create an environment within which a team or organization can examine their assumptions about digital product development. Only by bringing assumptions out into the open - i.e. the mindsets that are guiding people's thinking about digital product development, can a group find more efficient and more effective ways of working.

13. 

    18 Oct 2023

    Episode 3 - Questions for working with uncertainty

    In this episode, Collin and I work at unpacking this rather ambiguous idea of a “mindset shift”, which is so often used in digital transformations. We explore how the types of questions people in leadership roles ask can help create a learning environment. Collin proposes that the types of questions people in leadership roles ask can impose a veneer of certainty onto product development, where in actuality, certainty does not exist. Collin offers lines of inquiry, which can help teams more effectively work with the uncertainty that is inherent in digital product development. We also explore some questions about budgets. Too often, we see digital product development projects kick off with rigid budgets, limited scenario planning, poorly defined desired outcomes (if any) and, ultimately, little financial rigour. Needless to say, these projects usually end in tears. Again, the veneer of certainty where certainty doesn’t exist often bites people in the end. We are using Collin’s book Make Learn Change (https://ustwo.com/work/product-led-transformation/) as a springboard for our conversation. Please do send us feedback. We’d love to hear from you. And, a very big thank you to those who have been listening.

14. 

    24 Sept 2023

    Episode 2 - Breaking apart the shorthand

    In this episode, Collin and I continue to explore the relationship between making product, learning, and facilitating organisational change. We use Collin's book, Make Learn Change (https://ustwo.com/work/product-led-transformation/) as the springboard for this discussion. In this episode Collin and I unpack some of the key beliefs that underpin Waterfall and Agile. Often we see Waterfall and Agile being juxtaposed against each other as different “methodologies”. In this episode, I argue that you will be better served looking at them as differing belief systems about product development and product delivery - beliefs that have validity to those who hold them. Our feeling is that if you can develop a more refined language for articulating these differing belief systems, you will be better positioned to diagnosis where your organization is getting stuck in its own journey to becoming a more effective and more efficient digital product development company. You will also be better positioned to help your team and your organization navigate this rather ambiguous journey that is often simply termed as a “mindset” shift. As in Episode 1, we continue to argue for the benefits of trying to make a thin slice of value to get into your customers hand, so that you can learn and adapt, where needed. We also revisit how oragnizations can use the obstacles they hit when trying to deliver a thin slice of value as the core focus of a digital or Agile transformation.

15. 

    17 Jul 2023

    Episode 1 - A first look at Make Learn Change

    In this first episode, Collin and I start to explore Collin's book Make Learn Change (https://ustwo.com/work/product-led-transformation/). We discuss why a fair number of so-called “Agile transformations” simply don’t lead to meaningful improvements in a company’s ability to deliver high-value products faster to customers. We talk about the importance of creating tight feedback loops with customers and the need to bring out into the open the obstacles that are preventing teams from getting product into customers' hands fast. Collin & I talk about the importance of creating tight feedback loops with customers and bringing out into the open the obstacles that are preventing your teams from getting product faster into customers' hands.

Reach and audience

Public platform figures. Ratings count people who left a rating, not total listeners.

Spotify

5.0 / 5

1 ratings

Podcast Authority Score: 14 / 100

A composite of feed quality, social presence, YouTube performance and engagement. Read the methodology.

Quality

5

Social presence

0

YouTube

0

Engagement

60