Welcome to the SAI podcast, where intellectual curiosity and technical expertise meet.
Natalie is an OpenAI Developer Ambassador and Ivan is a Senior Security Researcher at Kaspersky.
Together we ask questions and learn, each one bringing their expertise. We're covering the different cross sections: how can AI be used in cybersecurity, how can AI be more secure, what attacks can be done, how to prepare, etc.
All the topics are introduced at a high level and occasionally we dive deeper in.
In this episode, Natalie and Ivan, in a conference call with members of GenAI, discuss OpenAI's cybersecurity grant program, and go over the suggestions made to applicants by the company on its webpage.
21 Jul 2023
Episode 6: OWASP's Top 10 Vulnerability Classes For LLMs
In this episode, we go through OWASP's recently released list of vulnerability classes for LLMs (https://owasp.org/www-project-top-10-for-large-language-model-applications/descriptions/). The list includes:
Prompt injections
Data leakage
Inadequate sandboxing
Unauthorized code execution
SSRF vulnerabilities
Overreliance on LLM-generated content
Inadequate AI alignment
Insufficient access controls
Improper error handling
Training data poisoning
We note that while some of the items on this list are definitely worth mentioning, others feel a bit more disconnected from the topic of cybersecurity and likely can't be solved through technical means.
Reference: Rich Harang's twitter thread on the subject (https://twitter.com/rharang/status/1661473877623713793).
22 Jun 2023
Episode 5: AI and Social-Engineering
This episode focuses on whether social engineering techniques, traditionally used against humans, could be leveraged against language models.
Natalie and Ivan start by discussing the use of LLMs in capture the flag competitions, and mention the emergence of new types of challenges revolving around prompt hacking (https://www.aicrowd.com/challenges/hackaprompt-2023). This leads the discussion towards the various ways (https://www.jailbreakchat.com/) that can be used to circumvent a model's fine tuning. The hosts discuss the challenges and scoring systems in such competitions, as well as the potential ethical concerns in exploring the capabilities and limitations of AI language models.
They draw parallels between AI prompt crafting and social engineering, speculating on how techniques used in human social engineering could potentially be adapted for AI prompt hacking. Overall, the conversation explores the evolving relationship between security, AI, and the ethical considerations surrounding the use of language models in hacking competitions.
31 May 2023
Episode 4: Decision Making and Explainability with AI
AI can often make decisions that are difficult for humans to understand. I
n this episode, we'll discuss the importance of explainability and transparency in AI systems, how they can be achieved, what's the problem with biased training data and how does explainability help us overcome bias.
What is explainability and why is it significant in the context of AI?
How to increase reliance on AI-based systems for security applications using explainability?
How can we have AI transparency? How to use it to overcome bias?
Given bias won’t be gone, how to take into account its impact?
Examples of problematic biases
Do we have sufficient data to train AIs to recognize and classify APT (Advanced Persistent Threat) activities? What are the risks of overfitting and underfitting?
Links mentioned in the show:
Acedemic paper evaluating the security of CoPilot from 12/2021 (https://arxiv.org/pdf/2108.09293.pdf)
Can OS LLM detect bugs in code? (https://twitter.com/MrCatid/status/1647155706548695040)
31 May 2023
Episode 3: AI in Reverse Engineering
In this episode we're deep diving into the fascinating world of Reverse Engineering and how AI is transforming this field.
Introduction to Reverse Engineering: A brief overview of the concept of RE, its significance in the fields of security, software development, and technology, and the state of the art.
AI Techniques in Reverse Engineering: A discussion on the various AI techniques and algorithms used in Reverse Engineering.
AI-driven RE Applications: Insight into the practical applications of AI in Reverse Engineering, including malware analysis, vulnerability detection, and software de-obfuscation.
Ivan's AI-based Open Source Reverse Engineering tool Gepetto: It uses OpenAI's gpt-3.5-turbo and gpt-4 models to provide meaning to functions decompiled by IDA Pro. https://github.com/JusticeRage/Gepetto
Challenges and Limitations: A discussion on the potential challenges and limitations of using AI in reverse engineering, such as the need for large datasets.
31 May 2023
Episode 2: Using AI to Carry Out Attacks
In this episode we're discussing five ways you can use AI to carry out attacks:
AI-assisted Social Engineering Attacks
AI-powered Malware
AI-driven Password Cracking
AI-assisted DDoS Attacks
AI-based Vulnerability Discovery and Exploitation
Needless to say, the purpose is to get acquainted with what can be, and learn how to be better prepared.
3 May 2023
Episode 1: Attack Vectors on AI, and How to Defend Against Them
In this podcast, we'll discuss the cross section of AI and Security.
Sometimes it will be more AI focused, sometimes more security focused, sometimes discussing what’s new.
In this very first episode, we’re introducing ourselves: Ivan is a Senior Security Researcher at Kaspersky, and Natalie is an AI Consultant and an OpenAI Developer Ambassador.
We then discuss four ways of attacking AI and how to protect against those:
Attacks on the model
Attacks on the data used by the model
Crafting the prompt to cause unwanted outputs (with time this got the name Prompt Injections)
Adversarial Model-to-Model Interaction
Reach and audience
Public platform figures. Ratings count people who left a rating, not total listeners.
Apple Podcasts (US)
5.0 / 5
1 ratings
Spotify
5.0 / 5
2 ratings
Contact SAI: The Security and AI Podcast
Guest appearances
Does not typically book guests
Based on episode analysis; this does not confirm that the show is currently accepting guests.
Host of SAI: The Security and AI Podcast?
Claim your podcast to manage its listing and keep your show details accurate.
Pod Engine is an independent podcast discovery and analytics service and is not affiliated with or endorsed by this podcast. Artwork and show content belong to their owners. Full legal notice.
Explore this show Podcast research with Pod Engine
