Knowledge Distillation is the podcast that brings together a mixture of experts from across the Artificial Intelligence community.
We talk to the world’s leading researchers about their experiences developing cutting-edge models as well as the technologists taking AI tools out of the lab and turning them into commercial products and services.
Knowledge Distillation also takes a critical look at the impact of artificial intelligence on society – opting for expert analysis instead of hysterical headlines.
We are committed to featuring at least 50% female voices on the podcast – elevating the many brilliant women working in AI.
Host Helen Byrne is a VP at the British AI compute systems maker Graphcore where she leads the Solution Architects team, helping innovators build their AI solutions using Graphcore’s technology.
Helen previously led AI Field Engineering and worked in AI Research, tackling problems in distributed machine learning.
Before landing in Artificial Intelligence, Helen worked in FinTech, and as a secondary school teacher. Her background is in mathematics and she has a MSc in Artificial Intelligence.
Knowledge Distillation is produced by Iain Mackenzie.
Neuroscience and AI with Basis co-founder Emily Mackevicius
Emily Mackevicius is a co-founder and director of Basis, a nonprofit applied research organization focused on understanding and building intelligence while advancing society’s ability to solve intractable problems.
Emily is a member of the Simons Society of Fellows, and a postdoc in the Aronov lab and the Center for Theoretical Neuroscience at Columbia’s Zuckerman Institute.
Her research uncovers how complex cognitive behaviors are generated by networks of neurons through local interactions and learning mechanisms.
Links to work mentioned in this episode:
Basis, the research institute co-founded by Emily: basis.ai (http://basis.ai)
Emily's work with Fang et. al. relating brain computations to AI/ML algorithms: https://elifesciences.org/articles/80680 (https://elifesciences.org/articles/80680)
Basis blog post about this work (Fang et. al.): https://www.basis.ai/blog/sr-fang2023/ (https://www.basis.ai/blog/sr-fang2023/)
Stachenfeld et al. paper: https://www.nature.com/articles/nn.4650 (https://www.nature.com/articles/nn.4650)
Emily's work with Michale Fee relating Reinforcement Learning algorithms to brain areas that birds use when they learn to sing: https://www.sciencedirect.com/science/article/abs/pii/S0959438817302349 (https://www.sciencedirect.com/science/article/abs/pii/S0959438817302349)
Emily's work with Aronov lab colleagues on how the hippocampus forms one-shot/episodic memory 'barcodes' in food-caching birds: https://www.cell.com/cell/fulltext/S0092-8674(24)00235-6 (https://www.cell.com/cell/fulltext/S0092-8674(24)00235-6)
NPR story about this work: https://www.npr.org/2024/04/05/1198909635/chickadee-bird-brain-memory-brain-pattern-food (https://www.npr.org/2024/04/05/1198909635/chickadee-bird-brain-memory-brain-pattern-food)
Github collab-creatures repo for the Basis collaborative intelligent systems project: https://github.com/BasisResearch/collab-creatures (https://github.com/BasisResearch/collab-creatures)
Basis's core open-source code repository for causal reasoning, ChiRho: https://basisresearch.github.io/chirho/getting_started.html (https://basisresearch.github.io/chirho/getting_started.html)
Basis's city policy dashboard, polis: http://polis.basis.ai/ (http://polis.basis.ai/)
7 Apr 2024
Stable Diffusion 3 with Stability AI's Kate Hodesdon
Stability AI’s Stable Diffusion model is one of the best known and most widely used text-to-image systems.
The decision to open-source both the model weights and code has ensured its mass adoption, with the company claiming more than 330 million downloads.
Details of the latest version - Stable Diffusion 3 - were revealed in a paper, published by the company in March 2024.
In this episode, Stability AI’s Kate Hodesdon joins Helen to discuss some of SD3’s new features, including improved capabilities for generating text within images and overall image quality.
Kate also talks about developments to the underlying model structure of Stable Diffusion, as well as the challenges associated with creating models that deliver more efficient inference.
The Stable Diffusion 3 paper can be found here: https://arxiv.org/pdf/2403.03206.pdf (https://arxiv.org/pdf/2403.03206.pdf)
7 Mar 2024
Inside OpenAI's trust and safety operation - with Rosie Campbell
No organisation in the AI world is under more intense scrutiny than OpenAI. The maker of Dall-E, GPT4, ChatGPT and Sora is constantly pushing the boundaries of artificial intelligence and has supercharged the enthusiasm of the general public for AI technologies.
With that elevated position come questions about how OpenAI can ensure its models are not used for malign purposes.
In this interview we talk to Rosie Campbell from OpenAI’s policy research team about the many processes and safeguards in place to prevent abuse. Rosie also talks about the forward-looking work of the policy research team, anticipating longer-term risks that might emerge with more advanced AI systems.
Helen and Rosie discuss the challenges associated with agentic systems (AI that can interface with the wider world via APIs and other technologies), red-teaming new models, and whether advanced AIs should have ‘rights’ in the same way that humans or animals do.
You can read the paper referenced in this episode ‘Practices for Governing Agentic AI Systems’ co-written by Rosie and her colleagues: https://cdn.openai.com/papers/practices-for-governing-agentic-ai-systems.pdf (https://cdn.openai.com/papers/practices-for-governing-agentic-ai-systems.pdf)
Watch the video of the interview here: https://www.youtube.com/watch?v=81LNrlEqgcM (https://www.youtube.com/watch?v=81LNrlEqgcM)
9 Feb 2024
Deepfakes deep dive with Nina Schick
Nina Schick is a leading commentator on Artificial Intelligence and its impact on business, geopolitics and humanity.
Her book ‘Deepfakes and the Infocalypse’ charts the early use of gen AI to create deepfake pornography and the technology’s subsequent use as a tool of political manipulation.
With over two decades of geopolitical experience, Nina has long been focused on macro-trends for society. She has advised global leaders, including Joe Biden, the President of the United States, and Anders Fogh Rasmussen, the former Secretary General of NATO.
She has also worked with some of the world’s premier companies and organisations, including Microsoft, Adobe, DARPA, and the UN.
A familiar face at technology conferences such as CES, TEDx, CogX and WebSummit, Nina is also a regular contributor to discussions about AI on the BBC, CNN, Sky News, Bloomberg and more.
In her conversation with Helen, Nina outlines the continuing risks posed by deepfake technologies and the technological counter-measures that can be used to safeguard against them.
You can watch the video of her interview on YouTube: https://youtu.be/f4zTbGWYan8
2 Feb 2024
Papers of the Month with Charlie Blake, Research Engineer at Graphcore
Charlie Blake from Graphcore’s research team discusses their AI Papers of the Month for January 2024.
Graphcore research has been collating and sharing a review of the most consequential AI papers internally, every month, for a number of years.
Now – for the first time – the research team is making this valuable resource public, to help the wider AI community keep up-to-date with the most exciting breakthroughs.
Papers of the Month for January 2024 (with some work from December 2023) includes:
Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding
https://arxiv.org/abs/2312.05328
Authors: Talfan Evans, Shreya Pathak, Hamza Merzic, et al. (Google DeepMind, UCL)
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
https://arxiv.org/abs/2401.00448
Authors: Nikhil Sardana and Jonathan Frankle (MosaicML)
Analyzing and Improving the Training Dynamics of Diffusion Models
https://arxiv.org/abs/2312.02696
Authors: Tero Karras et al. (Nvidia, Aalto University)
Solving olympiad geometry without human demonstrations
https://www.nature.com/articles/s41586-023-06747-5
Authors: Trieu H. Trinh, Yuhuai Wu, Quoc V. Le, He He and Thang Luong (Google DeepMind, New York University)
To read about January’s Papers of the Month, visit the Graphcore blog.
https://www.graphcore.ai/posts/great-teachers-and-beyond-chinchilla-papers-of-the-month-jan-2024
29 Jan 2024
The rise of synthetic data with Florian Hönicke from Jina AI
Data is the fuel that is powering the AI revolution - but what do we do when there's just not enough data to satisfy the insatiable appetite of new model training?
In this episode, Florian Hönicke, Principal AI Engineer at Jina AI, discusses the use of LLMs to generate synthetic data to help solve the data bottleneck. He also addresses the potential risks associated with an over-reliance on synthetic data.
German startup Jina AI is one of the many exciting companies coming out of Europe, supporting the development and commercialisation of generative AI.
The team at Jina AI gained widespread attention in late 2023 for the release of the first open-source text embedding model with an 8192 token length. Jina-embeddings-v2 achieves state-of-the-art performance on a range of embedding-related tasks and matches the performance of OpenAI's proprietary ada-002 model.
Watch the video of our interview: https://youtu.be/AP80hZajk5w
22 Dec 2023
NeurIPS Special
NeurIPS is the world’s largest AI conference, where leading AI practitioners come together to share the latest research and debate the way forward for artificial intelligence.
In this special episode, Helen examines some of the big themes of NeurIPS 2023 and talks to a range of attendees about their work, the big issues of the day, and what they’ve seen at NeurIPS that caught their attention.
It’s fair to say that LLMs loomed large over this year’s conference, but there’s plenty more to discuss – from AI’s potential to combat climate change to new techniques for computational efficiency.
Helen’s guests are:
Sofia Liguori – Research Engineer at Google Deepmind, specialising in the application of AI to sustainability and climate change.
Priya Donti – Assistant Professor in Electrical Engineering and Computer Science at MIT and Co-founder of Climate Change AI. Priya discusses the challenges associated with introducing leading-edge AI systems into highly complex real-world power generation and delivery systems.
Irene Chen – Assistant Professor at UC Berkeley and UCSF’s Computational Precision Health program. Irene talks about her goal of delivering more equitable healthcare at a time when AI is set to disrupt the field. She also discusses the potential to make use of commercial LLMs in a way that protects sensitive user data.
James Briggs – AI engineer at Graphcore. James and colleagues were presenting their paper ‘Training and inference of large language models using 8-bit floating point’ at this year’s NeurIPS. James explains their work and the importance of using smaller numerical representations to unlock computational efficiency in AI.
Abhinav (Abhi) Venigalla – is a member of the technical staff at Databricks. The company provides a range of products to help organisations unlock the potential of enterprise-grade AI. Abhi talks about the increasing emphasis on inference tools and computational efficiency as AI moves out of the research lab and into commercial deployment.
5 Dec 2023
Miranda Mowbray - Honorary Lecturer in Computer Science, University of Bristol: AI and Ethics
Miranda Mowbray is one of Britain’s leading thinkers on the ethics of Artificial Intelligence.
After a long and distinguished career as a research scientist with HP, she is now an Honorary Lecturer in Computer Science at the University of Bristol where she specialises in ethics for AI, and data science for cybersecurity.
In our wide-ranging conversation, Miranda breaks down the definition of AI ethics into its many constituent parts – including safety, transparency, non-discrimination and fairness.
She tells us that there’s probably too much focus on the dire predictions of AI ‘doomers’ and not enough on the more immediate, but less apocalyptic outcomes.
On a lighter note, Miranda reveals her personal mission to change the world, and shows off a sculpture that she had commissioned, based on the imaginings of generative AI.
You can watch a video of our interview with Miranda here: https://youtu.be/tbnHxbM5ZR8 (https://youtu.be/tbnHxbM5ZR8)
4 Dec 2023
Danijela Horak - Head of AI Research, BBC R&D
Danijela Horak explains how the BBC is making use of AI and its plans for the future, including detecting deepfakes as well as using deepfake technology as part of its production process.
Danijela and Helen discuss the Corporation's use of open source models and its view on closed source technologies such as the GPT family of models from OpenAI.
We find out how the BBC uses AI for recommendation, while taking cautious approach to user data, and Helen and Danjela reflect on why there needs to be more rigour in AI research so that the field doesn't end up on a par with 'social sciences'!
Watch the video of our conversation with Danijela at: https://www.youtube.com/watch?v=QOwecs8KRLg
4 Dec 2023
Introducing... Knowledge Distillation
First episode coming soon...
Join Graphcore's Helen Byrne and leading figures in the world of Artificial Intelligence as they discuss advances in AI research, commercial deployment, ethics, and occasional weird and wonderful applications.
Reach and audience
Public platform figures. Ratings count people who left a rating, not total listeners.
Spotify
5.0 / 5
14 ratings
Contact Knowledge Distillation with Helen Byrne
Guest appearances
Books guests
Based on episode analysis; this does not confirm that the show is currently accepting guests.
Host of Knowledge Distillation with Helen Byrne?
Claim your podcast to manage its listing and keep your show details accurate.
Pod Engine is an independent podcast discovery and analytics service and is not affiliated with or endorsed by this podcast. Artwork and show content belong to their owners. Full legal notice.
Explore this show Podcast research with Pod Engine
