Has anyone found a sane way to tune GuardDuty alongside Falco in EKS without muting real threats? After a cluster rollout on Tuesday, Falco’s k8s_audit alerts jumped 3x while GuardDuty only surfaced a Recon:EC2/PortProbe until VPC Flow Logs showed repeated 169.254.169.254 hits, which makes me think we’ve got both gaps and alert fatigue. What concrete rulesets or routing (CloudWatch → Kinesis → Panther, etc.) improved detection fidelity without creating blind spots?
got both gaps and alert fatigue. What concrete rulesets or routing (CloudWatch → Kinesis → Panther, etc.) improved We cut noise by enforcing IMDSv2 and adding a Falco rule to alert only on 169.254.169.254 egress from non-allowlisted pods (aws-node/kiam/kube2iam), plus a k8s_audit exception that mutes system:serviceaccount:kube-system during rollout‑labeled jobs. Then we route Falco + GuardDuty + VPC Flow Logs via EventBridge → Kinesis → Panther and only page when Falco IMDS egress and a GD finding match the same pod/IP within 5 minutes.
My take: I’d lean toward the simplest next step and see if it changes anything this week — if not, you’ve got a clear case to escalate. What would block you from trying that?
What helped us was giving each tool a clear lane: enable GuardDuty EKS Audit Log + Runtime Monitoring and drop most Falco k8s_audit rules, keeping Falco focused on syscall/runtime and only a few high-signal k8s checks — like one bouncer checks IDs while the other watches the dance floor. We routed Falco via Sidekick into Security Hub and let Insights de-dupe with GuardDuty (Amazon GuardDuty). If you can’t turn on GD Runtime, flip it: keep Falco runtime, let GD cover VPC/CloudTrail, and forward only sev>=Medium from Falco — are you sending Falco to Security Hub yet or straight to your S.
Quick example: our biggest win was enabling Falco Sidekick dedupe/throttle so repeats of the same rule per pod collapse to one for about 10m via EventBridge, with a High-only bypass so real incidents still page; @rjensen71 does that line up with how you’re routing?
Piggybacking on @tdawson07, I’d go one step stricter: write a single Falco rule that only fires on ‘unexpected metadata access’ from pods whose service account isn’t on a tiny allowlist, and back it with a network policy that blocks that endpoint for everyone else. Then let GuardDuty own node-level stuff; IMDSv2 with hop-limit=1 keeps hosts honest. Since you rolled out Tuesday, did new controllers get their SAs added to the allowlist?