Where can I listen to this podcast?

This podcast is available on 4 platforms including Apple Podcasts, Spotify, and more. You can also use the RSS feed directly.

Does this podcast accept guests?

No, this podcast does not typically feature guests.

Inference & Intelligence Lab

Claim This Podcast

by Lin Jia

11 episodes

Updated Daily

Accepts GuestsHas Sponsors

Podcast Overview

Inference & Intelligence Lab is a podcast on statistical inference, causal inference, machine learning, and GenAI evaluation, focused on making decisions that hold up in real-world data science. The show features two series—Causal Inference From the Ground Up and Inference in the Wild—covering both first principles and practical pitfalls.

Language

🇺🇲

Publishing Since

1/12/2026

View on Apple Podcasts RSS Feed

Reach the team behind Inference & Intelligence Lab

Verified contact details for this show aren't on file yet — sign up to get notified when they land.

Recent Episodes

June 5, 2026

Two Ways to Measure Demand, and When the Market Lens Matters | EP4: Inference in the Wild

EP4: Two Ways to Measure Demand, and When the Market Lens Matters"Demand" can mean more than one thing. In day-to-day product analytics, it’s the chart right in front of us: sessions, searches, transactions, and conversion rates. This is Funnel Demand—the activity on our own surface. But there is a second, critical lens worth holding alongside it: Market Demand. In this episode of Inference in the Wild, we explore the friction between how data scientists count activity and how economists define markets. By the time a user reaches your website, they aren't a random sample; they are the heavily filtered "website survivors" let through by search engines, recommendation systems, competitors, and targeted ad delivery. If your team is relying solely on the funnel lens to drive high-stakes choices like pricing strategy, you are making decisions on a fundamentally biased population. In this episode, we discuss:<ul><li>Funnel Demand vs. Market Demand: Defining demand as a static count of internal metrics versus an economist's true price-to-quantity curve. </li><li>The Subscription Pricing Blindspot: Why estimating price sensitivity from your own subscriber data tells you who will churn (funnel lens) but leaves you completely blind to the prospective customers you will never win (market lens). </li><li>Choosing a Lens is Choosing an Estimand: The critical importance of naming your target population up front—from the inner ring of website visitors to the outer ring of the addressable market. </li><li>GenAI and the Dark Funnel: How LLMs break traditional market measurement tools by stripping away impression logs, auctions, and visible ranking data. </li></ul><ul><li>Define the Target Population Up Front: Before running the math, explicitly state your estimand. Are you analyzing the users on your surface, or the whole market that would consider you at a plausible price point? </li><li>Look Where You Aren't Tracking: When assessing strategic risks, remember that the most price-sensitive people are often the ones who never enter your data because they were never your customers. </li><li>Prepare for the Invisible Funnel: As discovery goes dark under LLM-mediated filtering, the future data challenge won't just be calculating conversion drops, but understanding the demand we never observed at all. </li></ul>📖 Read the companion deep dive (with illustrations and takeaways): <a href="https://inferenceintel.substack.com/p/are-we-measuring-demand-or-only-the?r=7bs4uy" target="_blank" rel="noopener noreferer">https://inferenceintel.substack.com/p/are-we-measuring-demand-or-only-the?r=7bs4uy</a> About the HostLin Jia is a Senior Data Scientist and Craft Lead at Booking.com with over 9 years of experience. Operating at the intersection of statistical inference, causal machine learning, and GenAI evaluation, she specializes in building the frameworks that enable trustworthy, decision-ready insights under real-world constraints. A recognized expert in the field, Lin has authored research on sensitivity analysis presented at KDD 2024 and leads the development of organization-wide standards for experimentation and causal inference. 🤝 Connect with me on LinkedIn: <a href="https://www.linkedin.com/in/linjia/">⁠⁠https://www.linkedin.com/in/linjia/</a> If you found this episode valuable, please consider:<ul><li>Following the Podcast: Tap the "+" or "Follow" button on Spotify to stay at the cutting edge of measurement strategy.</li><li>Sharing the Episode: Know a Data Scientist or Product Manager trying to optimize strategy using only internal funnel metrics? Send this their way.</li><li>Joining the Conversation: Share your thoughts on today’s topic on LinkedIn—let’s raise the standard of the DS craft together.</li></ul>

May 22, 2026

The Causality Gap: Measuring the True Impact of Voluntary Adoption in Digital Marketplaces

Across the tech industry, many of the most valuable features rely on voluntary adoption. A traveler chooses whether to join a loyalty program, or a marketplace seller decides whether to opt into a smart-pricing tool. Because you cannot force users to adopt a feature, standard A/B tests leave teams with a diluted, flat topline result. Genuinely great features get prematurely killed simply because the bottleneck was an adoption problem, not a product quality problem.In this special episode, we break down The Causality Gap. We expose the structural math flaws that cause standard observational methods (like PSM or regression adjustment) to fail in opt-in scenarios, and reveal how combining Randomized Encouragement Design (RED) with Double Machine Learning (DoubleML) provides a diagnostic map to save your highest-potential features.In this episode, we discuss:<ul><li>The "Opt-In" Trilemma: Why voluntary adoption, extreme user heterogeneity, and finite samples break traditional product feedback loops.</li><li>The Collider Bias Trap: Why matching or conditioning on post-treatment adoption creates a spurious correlation that breaks your counterfactuals by design.</li><li>Randomized Encouragement Design (RED): Leveraging randomized nudges as Instrumental Variables to build a clean causal chain reaction.</li><li>Denoise First, Estimate Second: How DoubleML strips out immense marketplace noise while avoiding regularization bias through cross-fitting.</li><li>ATT vs. ITT: How decomposing your rollout-level impact from your adopter-level impact tells you exactly whether to iterate on the feature or optimize the funnel.</li></ul>📖 Read the deep dive on booking.ai medium blogpost (with illustrations and takeaways): <a href="https://medium.com/booking-com-data-science/the-causality-gap-measuring-the-true-impact-of-voluntary-adoption-in-digital-marketplaces-ea68b5a35120" target="_blank" rel="noopener noreferer">https://medium.com/booking-com-data-science/the-causality-gap-measuring-the-true-impact-of-voluntary-adoption-in-digital-marketplaces-ea68b5a35120</a> About the HostLin Jia is a Senior Data Scientist and Craft Lead at Booking.com with over 9 years of experience . Operating at the intersection of statistical inference, causal machine learning, and GenAI evaluation, she specializes in building the frameworks that enable trustworthy, decision-ready insights under real-world constraints . A recognized expert in the field, Lin has authored research on sensitivity analysis presented at KDD 2024 and leads the development of organization-wide standards for experimentation and causal inference .🤝 Connect with me on LinkedIn: <a href="https://www.linkedin.com/in/linjia/" target="_blank" rel="ugc noopener noreferrer">⁠⁠https://www.linkedin.com/in/linjia/</a> 🚀 Support the CraftIf you found this episode valuable, please consider:<ul><li>Following the Podcast: Tap the "+" or "Follow" button on Spotify to stay at the cutting edge of measurement strategy.</li><li>Sharing the Episode: Know a Data Scientist or Product Manager struggling to measure opt-in platform features? Send this their way.</li><li>Joining the Conversation: Share your thoughts on today’s topic on LinkedIn—let’s raise the standard of the DS craft together.</li></ul>

April 10, 2026

Build the Camera — How Measurement Design Guides Statistical Testing | EP2: Inference in the Wild

EP2: Build the Camera — Why Measurement Design Trumps Statistical TestingRunning a statistical test is simply pressing the shutter. But designing the measurement system? That is building the camera.In this episode, we challenge the industry’s obsession with "which test to run" and shift the focus to what actually matters: whether your metric captures meaningful change in user behavior. We explore why even a successful feature can "fail" a T-test (p=0.34) not because the feature failed, but because the raw metric amplified noise from outliers and suppressed the pattern that mattered.In this episode, we discuss:<ul><li>The Shutter vs. The Camera: Why statistical tests are secondary to how you define and reduce noise in your metrics.</li><li>The Estimand Trade-off: How common transformations (like Log-Transforms) don't just change the distribution—they fundamentally alter the business question you are answering.</li><li>Leverage through Design: Why the most successful Data Science teams focus on what to measure rather than just how to test.</li><li>Case Study: Rank Transformation: Using ranks as a strategic design choice to neutralize outliers while preserving the directional "truth" of your data.</li></ul>Before choosing a statistical test, every practitioner should ask:<ol><li>The Business Question: What do stakeholders actually need to know (e.g., "by how many minutes" or simply "is it better")? </li><li>Metric Topology: What does the distribution really look like, and where is the noise coming from? </li><li>The Noise Reduction Strategy: Which approach preserves the "truth" while eliminating the interference of outliers? </li><li>The Reliability Proof: Does simulation verify that this method achieves 80% power without inflating the False Positive Rate for this specific metric? </li></ol>📖 Read the companion deep dive (with illustrations and takeaways): <a href="https://inferenceintel.substack.com/p/build-the-camera-how-measurement" target="_blank" rel="ugc noopener noreferrer">https://inferenceintel.substack.com/p/build-the-camera-how-measurement</a> Lin Jia is a Senior Data Scientist and Craft Lead at Booking.com with over 9 years of experience. Operating at the intersection of statistical inference, causal machine learning, and GenAI evaluation, she specializes in building the frameworks that enable trustworthy, decision-ready insights under real-world constraints. A recognized expert in the field, Lin has authored research on sensitivity analysis presented at KDD 2024 and leads the development of organization-wide standards for experimentation and causal inference.🤝 Connect with me on LinkedIn: <a href="https://www.linkedin.com/in/linjia/" target="_blank" rel="ugc noopener noreferrer">⁠https://www.linkedin.com/in/linjia/</a> If you found this episode valuable, please consider:<ul><li>Following the Podcast: Tap the "+" or "Follow" button on Spotify to stay at the cutting edge of measurement strategy.</li><li>Sharing the Episode: Know a Data Scientist frustrated by "insignificant" results on successful features? Send this their way.</li><li>Joining the Conversation: Share your thoughts on today’s topic on LinkedIn—let’s raise the standard of the DS craft together.</li></ul>

11 total episodes available

Deep-dive analytics for Inference & Intelligence Lab

Frequently asked questions

Have a different question and can't find the answer you're looking for? Reach out to our support team by sending us an email and we'll get back to you as soon as we can.

What is Inference & Intelligence Lab?: Inference & Intelligence Lab is a podcast on statistical inference, causal inference, machine learning, and GenAI evaluation, focused on making decisions that hold up in real-world data science. The show features two series—Causal Inference From the Ground Up and Inference in the Wild—covering both first principles and practical pitfalls.
How often does this podcast release new episodes?: This podcast updates daily.
Where can I listen to this podcast?: This podcast is available on 4 platforms including Apple Podcasts, Spotify, and more. You can also use the RSS feed directly.
Does this podcast accept guests?: No, this podcast does not typically feature guests.

Legal Disclaimer

Pod Engine is not affiliated with, endorsed by, or officially connected with any of the podcasts displayed on this platform. We operate independently as a podcast discovery and analytics service.

All podcast artwork, thumbnails, and content displayed on this page are the property of their respective owners and are protected by applicable copyright laws. This includes, but is not limited to, podcast cover art, episode artwork, show descriptions, episode titles, transcripts, audio snippets, and any other content originating from the podcast creators or their licensors.

We display this content under fair use principles and/or implied license for the purpose of podcast discovery, information, and commentary. We make no claim of ownership over any podcast content, artwork, or related materials shown on this platform. All trademarks, service marks, and trade names are the property of their respective owners.

While we strive to ensure all content usage is properly authorized, if you are a rights holder and believe your content is being used inappropriately or without proper authorization, please contact us immediately at hey@podengine.ai for prompt review and appropriate action, which may include content removal or proper attribution.

By accessing and using this platform, you acknowledge and agree to respect all applicable copyright laws and intellectual property rights of content owners. Any unauthorized reproduction, distribution, or commercial use of the content displayed on this platform is strictly prohibited.

Inference &amp; Intelligence Lab