Ask our SOC anything: Alert noise and what analysts do all day Session 2
Seth Thompson, Senior Manager, Security Operations
This session was Critical Start’s second Live SOC session, where Seth answered questions submitted by registrants on how the SOC validates AI, detects AI-enabled attackers, retains analyst expertise, and recovers when things go wrong. Seth has spent eight years in the Critical Start SOC, from analyst to senior manager.
Topics covered in this session:
- How SOC AI accuracy is measured through human validation, weekly sampling, and one-change-at-a-time change control, and why SOC AI never closes alerts on its own
- Why AI-enabled adversaries are detected by behavior rather than by tool, and how blast radius analysis and cross-customer correlation catch campaigns
- How the analyst, senior analyst, and principal analyst progression path supports 90% analyst retention
- What happens behind the scenes: the Trusted Behavior Registry (TBR), SOC AI, and analyst collaboration
- How the SOC responds when it gets a big call wrong, including two-person verification, gap analysis, and RCAs
- How investigations continue during API rate limits or vendor-side outages
