Grumpy old Coders - S3Ep4 - Transformers everywhere
David and Thomas dive into the inner workings of the Transformer architecture behind modern Large Language Models. They break down foundational concepts including vectors, matrices, function approximations, artificial neural networks, and backpropagation. Supported by a live Python code demonstration, they walk through tokenization, word embeddings, sinusoidal positional encodings, and Query-Key-Value attention dynamics. The episode wraps up with a discussion on scaling context windows, KV caching, and the philosophical debate around AI consciousness versus human self-awareness.
Outline:
00:21 – Episode intro, podcast banter, & topic overview
02:37 – Mathematical foundations: Vectors, matrices, & functions
06:18 – Artificial Neural Networks: Layers, weights, & backpropagation
11:42 – ANN practical example: Image classification mechanics
14:51 – Tokenization & vocabulary constraints
18:40 – Word embeddings & GloVe vector spaces
21:00 – Positional encoding & sinusoidal matrices
27:00 – Self-attention components: Queries (Q), Keys (K), and Values (V)
31:37 – Live Python code walkthrough & script execution
37:25 – Attention matrix calculations & Softmax normalization
45:56 – Multi-head attention & multi-stage learning dynamics
49:44 – Large Language Model context window scaling
55:13 – KV caching & inference optimization
58:28 – AI consciousness debate
1:14:24 – Guardrails, sampling temperature, & LLM watermarking
1:20:51 – Closing thoughts and sign-off
17 Aug 2026
Grumpy old Coders - S3Ep3 - Motorcycle tour
David and Thomas recap their multi-country motorcycle journey through Germany, Austria, Slovenia, and Italy. They share key moments from the road, including navigating winding mountain serpentines. Along the way, they reflect on awkward luggage setups and unpredictable alpine weather. The episode wraps up with a discussion on the tech job market, the realities of AI "vibe coding" versus software engineering fundamentals, and preliminary planning for their next road trip.
Outline:
00:20 – Episode intro, camera/mic troubleshooting, & tour premise
01:00 – Bike comparison: Suzuki SV650 vs. BMW R 1250 GS
03:11 – Lake Chiemsee stop, rain, and duck watching
06:08 – Crossing into Austria via Kufstein & taking biker photos
08:20 – Triassic Park landmark & tour snacking habits
10:32 – Riding through Slovenia: Serpentines
15:00 – Wörthersee stopover
20:04 – Tom's leg injury
24:25 – High-altitude dining & Slovenian road marking confusion
29:13 – Luggage setup challenges
35:00 – Nolan helmet visor breakdown
40:14 – Trattoria al Norgé: Homemade pasta & Italian hospitality
45:37 – Narrow mountain pass & wet bitumen hazards
52:30 – Staying in Innervillgraten & Innsbruck hospital visit
1:01:54 – Returning to Germany
1:11:00 – Tech industry state, LLMs, & "vibe coding" vs. engineering
1:19:10 – Future trip planning
1:22:50 – Closing thoughts and sign-off
1 Jul 2026
Grumpy old Coders - S3Ep2 - Context matters
David and Thomas dive into the critical role of context management and agentic memory with large language models. They explore the core mechanics of AI agents, breaking down the differences between low-level tools, Model Context Protocol (MCP) servers, and high-level skills, before tackling the token cost challenges of stateless LLMs. The hosts analyze human-inspired cognitive strategies like context "compaction" and background "dreaming". The episode closes with thoughts on codebase visualization using Graphify and Obsidian, and a humorous nod to Inception-style agent recursion.
Outline:
00:21 – Heatwave talk & audio troubleshooting
07:16 – Chatbots vs. AI agents
12:20 – Defining tools, MCP, and skills
20:51 – Context boundaries & compaction
33:12 – Cognitive concepts and "dreaming"
40:14 – Exploring several tools
1:11:15 – Closing
Contact Grumpy old Coders
Guest appearances
Books guests
Based on episode analysis; this does not confirm that the show is currently accepting guests.
Host of Grumpy old Coders?
Claim your podcast to manage its listing and keep your show details accurate.
Pod Engine is an independent podcast discovery and analytics service and is not affiliated with or endorsed by this podcast. Artwork and show content belong to their owners. Full legal notice.
Explore this show Podcast research with Pod Engine
