Where can I listen to this podcast?

This podcast is available on 4 platforms including Apple Podcasts, Spotify, and more. You can also use the RSS feed directly.

As The Geek Learns

Claim This Podcast

by The Geek

18 episodes

Updated Daily

Accepts GuestsHas Sponsors

Podcast Overview

Tools and training for IT professionals. Join James Cruce—a systems engineer with 25+ years managing enterprise VMware infrastructure—for practical PowerCLI tutorials, automation tips, and lessons from the trenches. Much to learn, there always is. <a href="https://astgl.com?utm_medium=podcast">astgl.com</a>

Language

🇺🇲

Publishing Since

12/6/2025

Visit Website View on Apple Podcasts RSS Feed

1 verified contact email on file for As The Geek Learns

Pitch yourself as a guest, propose sponsorships, or reach out directly to the host.

See contact details

Recent Episodes

June 10, 2026

Your DNS Changed and Nobody Told You. Here's the Nightly-Diff Pattern That Catches It.

It was a Tuesday at 2:17 PM, and the marketing team's contact form was returning 502s. Not 404. Not a timeout. A clean 502, which means something was answering, just not the thing it was supposed to be.An hour in, I'd checked the app server logs, restarted the nginx process twice, confirmed the SSL cert was valid, and pinged our cloud provider's status page like it owed me money. Everything looked fine everywhere I looked. Then, almost by accident, I ran `dig +short www.company.com CNAME` and saw a hostname I didn't recognize. Something like `legacy-assets.decommissioned-vendor-name.com`.Vendor had been off the account for four months. The CNAME had quietly repointed to their infrastructure during the migration wind-down, sat there untouched, and then their old infrastructure finally went dark. Nobody changed our DNS intentionally. Nobody got notified when it happened. We found out when a sales rep tried to submit a lead form.That was the day I stopped trusting that "nothing changed in DNS" was a statement anyone could actually verify.Why DNS Is the Silent-Failure Layer of Every InfrastructureDNS is configuration. It's just not a configuration you can store in your repo, lint on a commit, or review in a pull request. It lives in a registrar panel or a DNS provider dashboard, updated by humans who may or may not be following a change-control process, and it's completely invisible until something breaks.Every other layer of your stack has some kind of drift detection built in these days. Config management tools track the desired state of your servers. Container orchestrators know what's supposed to be running. Infrastructure-as-code tools will tell you if something drifted from the Terraform state. DNS gets none of that by default. You get a text field in a web UI, a change that takes effect whenever the TTL expires, and exactly zero notifications.The operational pattern most teams rely on is "we'll know when it breaks." And they're right. They will know. They'll know at 2 PM on a Tuesday when a customer reports it, after a sales lead gets lost, after the support team has spent 45 minutes ruling out everything else. The detection mechanism is user reports, which is among the worst possible monitoring strategies.There's also a subtler problem. The change usually isn't malicious. It's not a security incident, at least not at first. It's a vendor cleanup, a platform migration, someone at a partner org tidying up their infrastructure without realizing your CNAME still pointed at them. It's the kind of change that feels harmless to whoever made it and catastrophic to whoever depends on it.The fix isn't complicated. What you need is a declared source of truth for what your DNS should look like, a way to compare that against what it actually looks like right now, and something that runs that comparison regularly enough to catch drift before users do.That's the pattern. The implementation fits in a bash script.As The Geek Learns is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.The Pattern, the Four States, and a Wrapper You Can Use TodayThe idea is straightforward: declare your expected DNS state once in a YAML file, then run a script nightly that queries your authoritative nameservers and compares what it finds against what you declared. Any gap between the two gets reported.The baseline file is the key piece. It's not generated. You write it manually, and that act of writing it is itself useful, because it forces you to actually look up what each record currently is and decide "yes, that's correct." Once it exists, it becomes your source of truth. Commit it to your repo. Update it when you make a legitimate DNS change. The baseline is always what you intend, and the script is always asking whether reality matches.When the diff runs, every record type for every domain you declared lands in one of four states:MATCH means the live record matches the baseline exactly. This is the quiet result. Nothing to do.NEW means a record exists in live DNS that isn't in your baseline. It could be a vendor auto-adding a TXT verification record. It could be someone provisioning a new subdomain. It could be something you should care about. The script surfaces it; you decide.MISSING means your baseline declared a record that doesn't exist in live DNS anymore. An A record that was decommissioned without cleaning up. An MX record that got deleted. A CNAME that was removed when a vendor migrated their platform.DRIFT means the baseline and live DNS both have records for a type, but the values don't match. This is the Tuesday-afternoon scenario: the CNAME target changed, the IP behind an A record flipped, the SPF policy was modified.NEW and MISSING and DRIFT all mean something in your environment changed without you being told. The script exits nonzero when any of those occur, which makes it trivially composable with cron, alerting pipelines, or anything else that reads exit codes.Here's a minimal working bash wrapper you can adapt right now. It keeps the dependencies to just `dig` and `bash`, uses a simple shell-array for your expected records instead of parsing YAML, and is short enough to read in under five minutes:#!/usr/bin/env bash # dns-check.sh # WHAT: Minimal DNS drift checker - declare expected records, diff against live DNS # WHY: Catches silent DNS changes before they become incidents # Usage: ./dns-check.sh # Add to cron: 0 2 * * * /path/to/dns-check.sh || echo "DNS DRIFT DETECTED" | mail -s "DNS Alert" you@example.com set -euo pipefail # ── CONFIGURATION ──────────────────────────────────────────────────────────── # Authoritative resolver to query against (use your domain's actual nameserver) # WHY: Querying authoritative NS catches changes before they propagate to resolvers RESOLVER="8.8.8.8" # Declare expected records as: "domain|TYPE|expected_value" # Get current values with: dig +short example.com A # Run once to populate, then treat this as your source of truth EXPECTED_RECORDS=( "example.com|A|93.184.216.34" "www.example.com|CNAME|example.com.cdn.cloudflare.net" "example.com|MX|10 mail.example.com" "example.com|TXT|v=spf1 include:_spf.google.com ~all" ) # ── DIFF ENGINE ────────────────────────────────────────────────────────────── DRIFT_FOUND=0 for record in "${EXPECTED_RECORDS[@]}"; do # Parse the declared record into its three parts domain="${record%%|*}" rest="${record#*|}" rtype="${rest%%|*}" expected="${rest#*|}" # Query live DNS at the authoritative resolver # WHY: +short gives us clean output; @resolver pins which nameserver answers actual=$(dig +short "@${RESOLVER}" "${domain}" "${rtype}" 2>/dev/null \ | sort \ | tr '\n' '|' \ | sed 's/\.$//g; s/|$//') # Normalize expected for comparison (sort, strip trailing dots) expected_norm=$(printf '%s\n' "${expected}" \ | sort \ | tr '\n' '|' \ | sed 's/\.$//g; s/|$//') # Compare and classify the result if [[ -z "${actual}" ]]; then # Record existed in baseline but dig returned nothing: MISSING printf "MISSING %s %s (expected: %s)\n" "${domain}" "${rtype}" "${expected}" DRIFT_FOUND=1 elif [[ "${actual}" != "${expected_norm}" ]]; then # Record exists but value changed: DRIFT printf "DRIFT %s %s\n expected: %s\n actual: %s\n" \ "${domain}" "${rtype}" "${expected}" "${actual}" DRIFT_FOUND=1 else # Values match: MATCH (silent - no output unless you add --verbose logic) : # nothing to report fi done # Exit nonzero on any drift - composable with cron, alerting, CI checks if [[ "${DRIFT_FOUND}" -eq 1 ]]; then printf "\nDrift detected. Review records above.\n" >&2 exit 1 fi printf "All %d declared records match live DNS.\n" "${#EXPECTED_RECORDS[@]}" exit 0Save that, drop your actual records into `EXPECTED_RECORDS`, and run it once to confirm it sees what you expect. Then add it to your crontab:# Run nightly at 2 AM, email on drift 0 2 * * * /path/to/dns-check.sh || echo "DNS drift detected on $(hostname)" | mail -s "[ALERT] DNS Drift" you@example.comThe "NEW record exists in live DNS" state isn't in this minimal version, since detecting it requires knowing which record types to scan for beyond what you declared. The four-state model handles that fully once you know which types to watch, which is what the complete kit covers. For a first pass, catching MISSING and DRIFT gets you most of the value.A few practical notes. Use `dig +short` rather than `dig` without `+short` or you'll spend time parsing the human-readable output format. Always query a specific nameserver with `@resolver` rather than relying on your local resolver, since caching can hide drift for hours. The MX record normalization is worth being careful about: `dig +short` returns the priority prefix as part of the value (`10 mail.example.com`), so your expected strings need to include it exactly that way. And commit the script alongside your baseline declaration. If the baseline lives in the repo, you get history, diffs, and code review for DNS changes as a side effect.What Else Lives in the Full KitThe script above covers the core pattern. The full DNS Drift Detector kit is what you reach for once you've outgrown the wrapper.The main `dns-drift-detector.sh` handles all five record types: A, AAAA, CNAME, MX, and TXT. That last group matters more than it seems. TXT records are where SPF policies live, where DKIM selectors sit, where domain verification tokens accumulate. Quiet SPF drift can break your email deliverability for days before anyone notices. DKIM drift means legitimate mail starts hitting spam folders. These aren't hypothetical edge cases.The color-coded output makes the diff results readable at a glance during incident response, and the `--quiet` flag strips all of that for cron-friendly logging where you only want the exit code to speak. There's a `--no-color` flag too, so piping to a log file doesn't fill it with ANSI escape sequences.`install-cron.sh` is a one-command idempotent installer. It checks that `dig` is available, puts the scripts where they belong, creates a log directory, and writes the cron entry with a duplicate-guard marker so running it twice doesn't add the job twice. That kind of thing is boring to write and annoying to get wrong.The `baseline.yaml` in the kit is annotated with examples for ten common services: Google Workspace MX, Cloudflare CDN CNAMEs, SendGrid SPF, common DKIM selectors, and a few others. It's the reference you use when you're populating your own baseline for the first time.The runbook covers install, baseline setup, how to read the output, what to do for each of the four states, how to update the baseline after a legitimate change, and an FAQ. That last one matters during an incident, when you don't want to be making judgment calls about whether a NEW record means "update the baseline" or "call the registrar."Try the Pattern TodayThe pattern described here is genuinely useful as-is. Declare your records, schedule the diff, react to the exits. That alone puts you ahead of the "we'll know when it breaks" approach that most infrastructure environments are actually running.If you want the full kit, it's at <a target="_blank" href="https://shop.asthegeeklearns.com/products/dns-drift-detector">shop.asthegeeklearns.com/products/dns-drift-detector</a> for $19. You get the complete `dns-drift-detector.sh` with all five record types, color output, quiet mode, and cron logging; the idempotent `install-cron.sh`; the annotated `baseline.yaml` with ten real-world service examples; and the full operator runbook.The Tuesday-afternoon incident I described at the top cost more than $19 worth of everyone's time. The detection script would have caught it the night before.As The Geek Learns is a newsletter about systems engineering, automation, and the gap between knowing something and actually applying it. If this was useful, subscribe for free to get new articles as they land. Get full access to As The Geek Learns at <a href="https://astgl.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4">astgl.com/subscribe</a>

June 8, 2026

I Secured My AI Agent With a 7-Layer Threat Model

I have an autonomous AI agent running on my Mac Studio. It has full shell access, reads my calendar, manages my tasks, and sends iMessages on my behalf. It runs 24/7 as a background service.If that sentence doesn’t make you slightly nervous, you haven’t been paying attention. In <a target="_blank" href="https://www.isec.news/2026/02/10/securityscorecard-135000-plus-internet-exposed-openclaw-instances-found/">February 2026, researchers found over 135,000 OpenClaw instances exposed to the public internet</a>. A coordinated attack called <a target="_blank" href="https://cybersecuritynews.com/clawhavoc-poisoned-openclaws-clawhub/">ClawHavoc</a> planted over a thousand malicious plugins in the community registry. Nine CVEs have been disclosed, including remote code execution.I needed to take security seriously. Not “I changed the default password” seriously. Threat-model seriously.MAESTRO: Seven Layers of Things That Can Go WrongThe <a target="_blank" href="https://cloudsecurityalliance.org/">Cloud Security Alliance </a>published a framework called <a target="_blank" href="https://github.com/CloudSecurityAlliance/MAESTRO">MAESTRO</a>—a 7-layer threat model specifically designed for agentic AI systems. Ken Huang mapped it directly to OpenClaw’s codebase, identifying 35+ specific threats across every layer of the stack.Here are the seven layers, translated from security-paper language into “things that could actually ruin your day”:Layer 1: Foundation Models: Someone sends your agent a crafted message that hijacks its behavior. Prompt injection. Jailbreaks. System prompt leakage. Your agent does what an attacker tells it to instead of what you told it to.Layer 2: Data Operations: Your credentials are stored in plaintext JSON files. Your session logs contain every conversation forever. A malicious skill injects code through your workspace.Layer 3: Agent Frameworks: The agent misuses its own tools. It runs shell commands it shouldn’t. It spawns sessions without authorization. It escalates its own privileges.Layer 4: Deployment & Infrastructure: Your gateway is exposed to the network. Someone brute-forces the WebSocket token. A reverse proxy misconfiguration bypasses authentication entirely.Layer 5: Evaluation & Observability: Nobody’s watching the agent for anomalous behavior. There’s no audit trail. Logs can be tampered with. If the agent starts acting weird, nothing catches it.Layer 6: Security & Compliance: Your DM policy is misconfigured. Anyone can message the agent. Pairing codes can be brute-forced. Identity can be spoofed across channels.Layer 7: Agent Ecosystem: A malicious plugin gets installed. A legitimate plugin’s npm dependency gets compromised. The skill registry serves poisoned packages.The critical attack chain MAESTRO identifies: compromise the gateway (Layer 4) → access the session store (Layer 2) → poison conversation history (Layer 1) → control the agent (Layer 3) → spread via messaging (Layer 7).Reading this was humbling. I’d addressed some of these by instinct during setup. Loopback binding, directory permissions, and pairing-based access control were all implemented. But “some” isn’t a security posture.SecureClaw: The Audit<a target="_blank" href="https://github.com/adversa-ai/secureclaw">SecureClaw</a> is an open-source security tool built specifically for OpenClaw by Adversa AI. It maps to MAESTRO, OWASP, MITRE ATLAS, and NIST AI 100-2. The install is a git clone and a bash script, no npm install, no network calls, and no surprises.git clone https://github.com/adversa-ai/secureclaw.git bash secureclaw/secureclaw/skill/scripts/install.shThen you run the audit:bash ~/.openclaw/skills/secureclaw/scripts/quick-audit.shMy baseline score: 57 out of 100. Zero criticals. Three HIGHs. Three MEDIUMs. Eight checks passing.Here’s what passed without any work:• Gateway bound to loopback (127.0.0.1) not exposed to network• Gateway authentication present• Directory permissions set to 700 (owner only)• No browser relay exposed• DM policy set to pairing (not open)• Skills clean of malicious patternsAnd here’s what failed:🟠 HIGH Plaintext key exposure: Keys in openclaw.json and 5 backup files🟠 HIGH Sandbox mode: commands run directly on host🟠 HIGH Exec approval mode: agent acts without human approval🟡 MED No cognitive file baselines: can’t detect tampering🟡 MED Default control tokens: vulnerable to spoofing🟡 MED No failure mode: no graceful degradationThe HardeningStep 1: Clean up credential leaks. OpenClaw creates .bak files every time you change config. Each backup contains your full config, including Slack tokens and API keys. I had five of them sitting in the OpenClaw directory. Deleted them all. Set the main config to 600 permissions.This is the kind of thing that’s easy to miss and catastrophic to ignore. A single ls -la ~/.openclaw/ would show them. But who runs ls -la on their config directory after every change?Step 2: Create integrity baselines. SecureClaw’s hardener generates SHA256 hashes of your “cognitive files” IDENTITY.md, AGENTS.md, and HEARTBEAT.md. These are the files that define who your agent is and what it does. If an attacker or a hallucinating agent modifies them, the nightly integrity check will catch it.bash ~/.openclaw/skills/secureclaw/scripts/quick-harden.shStep 3: Exec approvals. This is the big one. MAESTRO recommends human-in-the-loop approval for all shell commands. But my agent runs morning briefings and heartbeat checks on cron—unattended. Setting approvals to “always” would break all automation.The solution: an allowlist with on-miss approval. I created ~/.openclaw/exec-approvals.json with 17 safe command patterns: imsg, calctl, apple-reminders, cairn, and basic file operations. Tars can run these freely. Anything else; curl, rm, pip install, or any command not on the list, requires human approval.{ “defaults”: { “security”: “allowlist”, “ask”: “on-miss” }, “agents”: { “main”: { “allowlist”: [ { “pattern”: “imsg *”, “note”: “iMessage send/read” }, { “pattern”: “calctl *”, “note”: “Apple Calendar” }, { “pattern”: “cairn *”, “note”: “Task management” } ] } } }This is the trade-off MAESTRO doesn’t talk about: security versus automation. Maximum security means every action needs approval. Maximum automation means the agent acts freely. The allowlist is the middle ground. Routine operations are pre-approved, and novel or dangerous operations require a human.Step 4: Full plugin install. Beyond the bash scripts, SecureClaw has a full npm plugin with 56 runtime audit checks, background monitors for config drift, and real-time integrity verification. Installing it required building from source (TypeScript → JavaScript) and registering it with OpenClaw’s plugin system.openclaw plugins install -l /path/to/secureclaw openclaw config set plugins.allow ‘[”secureclaw”]’That plugins.allow line is important. By default, OpenClaw will auto-load any discovered plugin. Explicit trust means only plugins you’ve approved get loaded.Step 5: Nightly audit cron. A macOS LaunchAgent runs the full audit suite every night at 2 AM which includes quick-audit, integrity check, and supply chain scan. Results go to secureclaw-audit.log. If something changes overnight, it shows up in the morning.The Final ScoreAfter hardening: 64 out of 100. Nine checks passing. Zero criticals. The three remaining HIGHs are documented, accepted trade-offs:Findings I accepted (with reasoning)—Sandbox mode (Docker sandboxing would break imsg, calctl, and Apple Reminders); Plaintext keys in config (inherent to the platform config format, file is locked to 600); Exec approval not “always” (using allowlist + on-miss; full “always” breaks unattended cron automation).The two MEDIUMs, control token customization and failure mode configuration, aren’t supported in OpenClaw v2026.3.2’s config schema yet. SecureClaw checks for them proactively. They’ll be fixable when OpenClaw adds the config options.What I Actually LearnedSecurity isn’t a feature you enable. It’s a series of trade-offs you make with your eyes open. Sandbox mode is “more secure” but breaks the tools that make the agent useful. Approval mode “always” is “more secure” but kills the automation that makes the agent worthwhile. The right security posture isn’t maximum restriction; it’s documented, intentional decisions about what risks you accept and why.Automated scanning is essential but insufficient. SecureClaw’s audit caught things I would have missed, including the .bak files with credentials, the missing integrity baselines, and the open exec policy. But the HIGHs it flagged as failures are things I’ve consciously accepted. No scanner can evaluate your specific trade-offs.The biggest threat isn’t external. In my setup (loopback-bound, pairing-gated, allowlist-filtered), the most likely security failure isn’t a network attacker. It’s a malicious skill, a compromised npm package, or the agent itself hallucinating destructive actions. Layer 7 (ecosystem) and Layer 1 (model behavior) are the real attack surfaces for a local-first setup. The exec approval allowlist is my primary defense for both.Clean up after yourself. OpenClaw creates backup files containing credentials on every config change. There’s no auto-cleanup. If you’re running OpenClaw, go check your directory right now: ls ~/.openclaw/*.bak*. You might be surprised.Quick ReferenceHardening actions and commands: install, run audit, apply hardening, check integrity, scan skills, check for credential leaks, set exec approvals, set plugin trust. Commands target ~/.openclaw/skills/secureclaw/scripts/. Full command details in the image.Update—June 2026: What I Actually Did When I Moved to ClaudeClawI wrote this piece in March, when OpenClaw was still the thing running my Mac Studio. By the end of April, I’d shut it down. Disabled the cron jobs, quarantined the LaunchAgents, and rebuilt the whole stack on the <a target="_blank" href="https://docs.claude.com/en/api/agent-sdk/overview">Claude Agent SDK</a>. Based off of <a target="_blank" href="https://github.com/earlyaidopters/claudeclaw">ClaudeClaw</a> from the <a target="_blank" href="https://www.skool.com/earlyaidopters/about">Early AI-Dopters</a> AI learning group. The full post-mortem on why:Why? The short version is this: I couldn’t see into OpenClaw. Which, if you scroll back up, is Layer 5: Evaluation & Observability, the exact layer this audit was weakest on.You may wonder whether I just copied the 7-layer hardening over to the new stack. I didn’t, and I want to be honest about that. I did not port MAESTRO one-for-one. SecureClaw was written specifically for OpenClaw. Some of its thinking transferred; some of it didn’t. And the threat model itself moved on (more on that at the end). What the seven layers became was a checklist: for each one, how does the new architecture answer this? Here’s the scorecard.The two layers that changed the most.Layer 5 (Observability) went from my single biggest weakness to the entire reason ClaudeClaw exists. There’s now a dedicated agent, WATCHMAN, running seven probes every hour: failed tasks, stuck tasks, missed scheduler slots, daemon liveness, content-pipeline health, hidden failures (it greps the success logs for crash text), and delegation crashes. More importantly, there’s a second healthcheck running as a separate LaunchAgent with its own keychain-backed alert token. If the main daemon dies, the thing that tells me about it is still alive. The rule I wrote for myself out of this: the watcher cannot share fate with the watched. There’s also a behavioral dashboard, DefenseClaw, sitting on 127.0.0.1:3141.Layer 3 (Agent Frameworks) is where my OpenClaw work actually carried forward. The exec-approvals allowlist from Step 3 above is the direct ancestor of what ClaudeClaw does now, except the enforcement dropped down a level. The first thing I shipped was killing bypassPermissions (the main agent had been running with permission checks disabled, which means a compromised agent has unlimited tool access. The SDK was no ceiling at all), switching to the SDK’s default permission mode, and handing the main agent a 15-tool allowlist as the single source of truth. Same idea as the OpenClaw allowlist. Enforced by the SDK itself instead of a config file I had to maintain.The rest mapped like this:How each of the seven MAESTRO layers from the OpenClaw audit is answered in ClaudeClaw. Layer 1 Foundation Models: channel tagging and a trust gradient that treats retrieved text as data, not directives (evolved).Layer 2 Data Operations: Chamberlain outbound scanner, exfiltration-guard, queryable Memory v2, and ingest-time canonicalization (replaced and extended).Layer 3 Agent Frameworks: SDK permission ceiling and a 15-tool allowlist, the direct successor to the OpenClaw exec-approvals list (kept, moved into the SDK).Layer 4 Deployment and Infrastructure: an egress gateway plus kernel-level pf default-deny (replaced). Layer 5 Evaluation and Observability: WATCHMAN’s seven probes and a fate-isolated external healthcheck, the biggest upgrade.Layer 6 Security and Compliance: out-of-band Telegram confirmation for state-changing actions and a role policy kept separate from content memory (evolved).Layer 7 Agent Ecosystem: an MCP allowlist plus the tool ceiling as a second layer (kept and hardened). Plus a new row beyond MAESTRO. Memory persistence: TTLs, a hash-chained write log, and canaries.Where the 7-layer model ran out.MAESTRO is a static threat model. It’s a map of what can go wrong at each layer, frozen in time. What it doesn’t have a layer for is persistence. An attack that lands quietly in your agent’s memory or vector store and just waits. My scheduler re-enters context every 60 seconds, which means anything dormant in memory fires on a clock. That’s a different class of problem, and it has a name now: <a target="_blank" href="https://www.semanticscholar.org/paper/Logic-layer-Prompt-Control-Injection-(LPCI)%3A-A-in-Atta-Huang/7209db0a616b54335db85d6e73a0dc9505192e59?utm_source=direct_link">LPCI, Logic-layer Prompt-based Conditional Injection</a>. Hardening against it (I am planning a separate two-part write-up on <a target="_blank" href="https://astgl.substack.com">As The Geek Learns</a>) meant building things MAESTRO never asked for, including a canonicalizer that decodes payloads before they reach the vector store, channel-tagged prompts so the model knows retrieved text is data and not instructions, memory TTLs, a hash-chained write log, and canary entries that page me if memory ever leaks into output.What I gave up and what I kept. The honest cost of the move: I lost local-first. OpenClaw ran on Ollama, fully offline; ClaudeClaw talks to Anthropic’s API. I still own every byte of my data; it’s all on my SSD; I just don’t own the weights anymore. What carried over intact was the philosophy this whole series is built on: every document is a file I can grep, every config is version-controlled, and every decision has a session note. That part never changed.This is Part 5 of the Notion Replacement series. We went from “install an AI agent” to “secure it against a 7-layer threat model” in two days. Follow along at <a target="_blank" href="https://astgl.substack.com">As The Geek Learns</a>. Get full access to As The Geek Learns at <a href="https://astgl.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4">astgl.com/subscribe</a>

June 4, 2026

5 Questions to Ask Before You Build the AI Project Your CEO Just Pitched

5 Questions to Ask Before You Build the AI Project Your CEO Just PitchedYou know the email. It shows up Tuesday morning, forwarded with a few lines of enthusiasm and a ChatGPT-drafted proposal attached. "Saw this and thought of us. Can we do this?" The PDF has a logo, bullet points, and exactly zero integration requirements. It also has a six-week timeline and a budget that assumes nothing goes wrong.You have somewhere between 24 and 72 hours before your CEO follows up asking what you think.If you say yes, you're on the hook for a project you didn't scope. If you say no, you're the person who kills ideas. Neither answer is actually available to you. What you need is a third path: a structured evaluation that produces a defensible, professional response in the time it takes to drink your morning coffee.That's what the Technical Reality Check is. Five questions. One page. Every answer points directly at a commitment your organization will have to honor if this project moves forward.Here it is in full.The Technical Reality Check: 5 Questions That Surface What the Proposal Left OutQuestion 1: What specific business outcome does this solve, and how will we measure success?AI tools generate confident-sounding proposals that describe solutions, not problems. A proposal for "an AI-powered IT ticketing system" describes a technology. It doesn't describe what's broken right now, how broken it is, or what "fixed" looks like in measurable terms.Before any conversation about implementation, you need an answer to: what does success look like in six months, and how will we know we hit it? Ticket resolution time down 30%? First-contact resolution rate up 20%? Those are real answers. "Things will be more efficient" is not.Unmeasurable projects never officially fail. Which means they never stop consuming resources. This question isn't about being difficult. It's about making sure the organization is buying an outcome, not a technology.The red flag: Any proposal where the only success metric is "we deployed it."Question 2: Who owns the ongoing maintenance, security patching, and vendor relationship?Vendor proposals describe launch day. They are almost entirely silent about year two.Every new system creates a permanent maintenance obligation: patching, credential rotation, user access reviews, API deprecations, contract renewals, and a support relationship with a vendor whose incentives are not aligned with yours. If that obligation doesn't have a named owner before the project starts, IT inherits it by default. Forever. Without headcount.This question forces the conversation about operational reality before anyone has signed a contract. The answer also tells you a lot about how seriously the proposal was thought through. If nobody has asked "who maintains this?", nobody has thought past the demo.The red flag: "The vendor handles everything." Vendors handle their system. You handle the integration, the credentials, the user provisioning, the data pipeline, and the 2 AM alert when something breaks between their system and yours.Question 3: What happens to our existing systems, data, and processes?New systems don't exist in a vacuum. They touch your directory, your ticketing system, your identity provider, your backup scope, your audit logs. Each of those integration points is a potential failure mode, a migration cost, or a compliance question.AI-generated proposals routinely skip integration complexity. This isn't because the AI is being deceptive. It's because the AI generating the proposal doesn't know your stack. The proposal was written in a context-free environment. Your environment is anything but.Before committing, you need to know: what does this touch, and what has to move or change for it to work? And who does that work? Data migration alone can turn a "simple" deployment into a multi-month project. Asking this question early is how you find out.The red flag: "It integrates easily with your existing tools." That's a sales phrase, not an engineering estimate. "Easy" is undefined until your systems engineer has looked at the API docs.As The Geek Learns is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.Question 4: What's the realistic timeline and resource cost, not the optimistic one?Vendor timelines assume clean data, available staff, smooth approvals, and nothing else on the backlog. Your timeline accounts for your actual team, their current commitments, the security review cycle, the change management process, and the three things nobody predicted.The gap between those two numbers is usually where projects go sideways. Not because the technology failed, but because the plan never accounted for reality.This question also surfaces a common pattern: the timeline was set before IT was consulted. Any timeline that precedes a technical assessment is a guess dressed up as a schedule. You're the one who'll be explaining the delay when the guess turns out to be wrong.The red flag: A go-live date in the proposal. That's not a plan, it's a target somebody made up. Ask who set it and what it was based on.Question 5: What's the exit strategy if this doesn't work as expected?Every vendor says their product works. You need a plan for when it doesn't. When the pricing doubles at renewal. When the company gets acquired and support degrades. When a compliance requirement changes and the product doesn't keep up.Data portability, rollback procedures, and contractual exit terms are not pessimism. They're the difference between a manageable failure and a situation where you're paying for a system that doesn't work because migrating off it is too expensive to contemplate.This question also signals organizational maturity. IT teams that ask exit questions before they sign contracts don't get held hostage. IT teams that don't ask end up managing a five-year sunset project for a tool they stopped believing in three years ago.The red flag: "We can always just stop using it." Can you migrate your data? In what format? At what cost? How long does it take? If nobody has asked those questions, stopping isn't as simple as it sounds.The Checklist in Practice: Walking Through a Real ScenarioHere's what a Technical Reality Check pass looks like when you actually run it.Your CEO forwards a ChatGPT-drafted proposal on a Monday morning. The subject line is "AI Agent for IT Ticket Triage." The proposal is two pages. It describes an AI system that reads incoming IT tickets, categorizes them by priority and type, routes them to the right team, and drafts first-response emails automatically. There's a mockup screenshot. There's a line about "easy integration with your existing ITSM." There's a timeline: six weeks to deployment.You open the Technical Reality Check.Q1: What specific business outcome does this solve?The proposal says "reduce response times and improve IT efficiency." No baseline. No metric. You check your current ITSM data: average first response is 4.2 hours, your SLA target is 2 hours, you're meeting it 71% of the time. Now you have a problem worth solving. You write it down: "We need first-response SLA compliance above 85%. Current state: 71%." That's the outcome. If the AI system can't demonstrate a path to that specific number, the conversation is premature.Q2: Who owns maintenance and the vendor relationship?Nobody is named in the proposal. You have a team of four. One of them is already carrying the ITSM admin role. You note: this needs a named owner and a rough estimate of ongoing hours before it can go to planning. You also flag the API integration dependency: your ITSM has a rate-limited API that's caused problems before. Someone needs to read the vendor's API docs before "easy integration" gets treated as a fact.Q3: What happens to existing systems and data?Your ticketing data includes ticket histories, customer records, and some attachments. The proposal doesn't mention data handling. You note two questions: where does ticket data go once the AI processes it, and what are the data residency requirements given that you handle some HIPAA-adjacent systems? That second question alone could be a blocker. You don't know yet, but you know to ask.Q4: What's the realistic timeline and resource cost?Six weeks assumes nothing else is happening. Your team is currently in the middle of a server migration that runs through the end of the month. Realistically, this project can't start until mid-next-month, and your most experienced engineer (the one who'd need to own the integration) is at 90% utilization. You write down: "Realistic start: six weeks out. Realistic deployment: 12-16 weeks from proposal receipt. Not 6."Q5: What's the exit strategy?The proposal doesn't mention it. You note: before any contract, you need to know the data export format, the contract term length, and what happens to stored ticket data at offboarding.That's it. You've just done a Technical Reality Check. Total time: 15 minutes.Now you can write a response. Not "no." Not "yes." Something like: "I've done a preliminary review. Before we can assess feasibility, I need answers to five specific questions. Here they are. Happy to set up 30 minutes to walk through them together." You've moved the conversation from enthusiasm to decision-ready. You've protected the organization without being obstructionist. And you have a written record of the questions you asked, which matters if the project later goes sideways without those answers ever being provided.That's the whole point of the Technical Reality Check. It's not a rejection letter. It's the question set that separates proposals worth pursuing from proposals worth deferring.What the Rest of the Toolkit CoversThe Technical Reality Check is the first thing you run. It gets you to a defensible position in 15 minutes. But the full response (the one that protects your career, your team's credibility, and the organization's resources) needs more than five questions.The complete AI Request Deflection Toolkit includes three email templates that turn your Reality Check findings into professional communications: an initial deflection that buys time while signaling genuine interest, a risk escalation that documents specific technical concerns in business-impact terms, and a stakeholder alignment template that ends the email thread and gets the right people in a room with a decision mandate. Every template has a filled-in worked example so you can see exactly what "adapted" looks like.There's also a 15-question weighted scoring matrix in CSV and Sheets format. It turns "I have concerns" into "the proposal scores 41% against our evaluation criteria, which triggers a formal risk review." Objective. Defensible. Exportable. The kind of documentation that holds up in a post-project conversation.And there's an escalation playbook for situations where the initial deflection didn't land and the project is being pushed forward without proper review. That one's for the harder conversations.The Cost of Not Having a ProcessMost IT managers who get burned by an executive-forwarded AI project didn't fail because the technology was bad. They failed because they said yes before they had answers, or they said no in a way that got overridden, or they said "we have concerns" without the documentation to back it up when the concerns turned out to be right.A 15-minute structured evaluation is the cheapest investment in that problem. Run it every time. Document the answers. Keep the record.If you want the full toolkit (the email templates, the scoring matrix, the escalation playbook, and all the worked examples), it's at the store for $24.99.<a target="_blank" href="https://shop.asthegeeklearns.com/products/ai-deflection-toolkit">Get the AI Request Deflection Toolkit</a>The Technical Reality Check above is yours to use as-is. Print it. Keep it at your desk. The next email is coming. Get full access to As The Geek Learns at <a href="https://astgl.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4">astgl.com/subscribe</a>

18 total episodes available

Deep-dive analytics for As The Geek Learns

Frequently asked questions

Have a different question and can't find the answer you're looking for? Reach out to our support team by sending us an email and we'll get back to you as soon as we can.

What is As The Geek Learns?: Tools and training for IT professionals. Join James Cruce—a systems engineer with 25+ years managing enterprise VMware infrastructure—for practical PowerCLI tutorials, automation tips, and lessons from the trenches. Much to learn, there always is. <a href="https://astgl.com?utm_medium=podcast">astgl.com</a>
How often does this podcast release new episodes?: This podcast updates daily.
Where can I listen to this podcast?: This podcast is available on 4 platforms including Apple Podcasts, Spotify, and more. You can also use the RSS feed directly.
Does this podcast accept guests?: Information about guest appearances is not available.

Legal Disclaimer

Pod Engine is not affiliated with, endorsed by, or officially connected with any of the podcasts displayed on this platform. We operate independently as a podcast discovery and analytics service.

All podcast artwork, thumbnails, and content displayed on this page are the property of their respective owners and are protected by applicable copyright laws. This includes, but is not limited to, podcast cover art, episode artwork, show descriptions, episode titles, transcripts, audio snippets, and any other content originating from the podcast creators or their licensors.

We display this content under fair use principles and/or implied license for the purpose of podcast discovery, information, and commentary. We make no claim of ownership over any podcast content, artwork, or related materials shown on this platform. All trademarks, service marks, and trade names are the property of their respective owners.

While we strive to ensure all content usage is properly authorized, if you are a rights holder and believe your content is being used inappropriately or without proper authorization, please contact us immediately at hey@podengine.ai for prompt review and appropriate action, which may include content removal or proper attribution.

By accessing and using this platform, you acknowledge and agree to respect all applicable copyright laws and intellectual property rights of content owners. Any unauthorized reproduction, distribution, or commercial use of the content displayed on this platform is strictly prohibited.