14 Oct 2026
Ep. 21 - AI Agents: The Gap Between Demo and Production
My Data Guest is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
After a two-month break, My Data Guest is back with Bilge Yücel, Senior DevRel Engineer at deepset. This one’s less about a product and more about the shifts happening underneath the AI hype: how people learn to code now, what documentation means when agents read it instead of humans, and how you measure trust in a community that can’t be reduced to a metric.
Bilge’s own path wasn’t linear. She started as a frontend engineer in Turkey, got curious about how tools like Google Translate actually worked, and left a stable job to study NLP, pre-ChatGPT. She didn’t want to be a researcher chasing a PhD. She wanted to build things people could use, which is what eventually pulled her into developer relations.
That personal arc sets up the bigger question in the episode: what happens to problem-solving skills when AI writes the code? Bilge points out that her generation learned by writing C by hand, wrestling with memory, debugging without a model to lean on. Today’s students are allowed to use generative AI for coursework. The skill isn’t gone, it’s shifted, from writing code line by line to knowing how to direct, evaluate, and trust an agent’s output.
That shift shows up again later in a different form. Stack Overflow traffic has cratered since ChatGPT. People don’t search anymore, they ask an agent. So documentation itself has had to adapt: not just written for humans, but structured so coding assistants retrieve the current version instead of hallucinating outdated syntax. It’s a small but telling example of how much invisible infrastructure work has to happen just to keep information trustworthy in an agent-first world.
The conversation also gets into the murkier side of community-facing work. How do you measure whether a DevRel role is succeeding? Pip installs and GitHub stars are noisy signals at best, mostly useful for spotting seasonality, not proving impact. Bilge’s answer is blunter: trust is the real currency, and trust doesn’t show up on a dashboard. You collect it in anecdotes, in feedback forms, in the fact that someone comes up to you at a conference and says a tool genuinely helped them. It’s unsatisfying from a KPI standpoint and probably the most honest answer available.
There’s good practical advice here too, for anyone who’s ever wanted to speak at a conference. Developers, she says, should look at whatever they just shipped, even something small, and find the story in it. DevRels operate differently: submitting talks a year out based on a roadmap, sometimes with no slides and no code yet written, just a belief they’ll build it in time.
And a broader, honest tension runs through the whole episode: what does neutrality look like when you work for a company that needs to make money? Bilge’s answer is that her job isn’t to sell, it’s to be trusted, which sometimes means telling someone a tool isn’t the right fit for them.
Worth a full listen if you care about how technical communities actually function, and what’s really being lost or gained as AI reshapes how we learn and document things.
My Data Guest is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Get full access to My Data Guest at mydataguest.substack.com/subscribe (https://mydataguest.substack.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4)
