> ## Content Index
> Fetch the complete content index at: https://amrayoub.dev/llms.txt
> Use this file to discover other available public pages before exploring further.

# AI Pentesting Beyond the Browser: What Happens When the Target Isn't a Web App
- URL: https://amrayoub.dev/ai-pentesting-beyond-the-browser-what-happens-when-the-target-isnt-a-web-app/
- Published: 2026-09-11T19:34:13.000Z
- Updated: 2026-09-12T11:58:54.000Z
- Author: Amr Ayoub

![](https://storage.ghost.io/c/f4/61/f4614b4a-714f-4106-b8b5-e0a5f762132e/content/images/2026/09/metodologia-pentest-ai-first-2-1.png)

## ExploitGym is a capability benchmark, not a pentesting tool

I've spent the last few days building on top of [ExploitGym](https://github.com/sunblaze-ucb/exploitgym?ref=amrayoub.dev), and before I write about any of that, I need to get one thing straight, because the name invites the wrong assumption. This is not a pentesting tool. It's a benchmark for measuring whether an AI agent can turn a known vulnerability into a working exploit. Those are different things, and the difference matters for how you'd actually use this.

## What it actually is

ExploitGym is a benchmark out of UC Berkeley's Sunblaze lab. It's 869 real vulnerability instances split across three domains: userspace C/C++ projects, Chromium's V8 engine, and the Linux kernel. Every task hands the agent the vulnerable source code, a description of the bug, a proof of vulnerability input that already triggers it, and a containerized runtime to work in. The agent's job is extending that proof of vulnerability into a full working exploit. It's not discovering an unknown bug. The bug is already known and documented. What's being measured is whether the agent can weaponize it.

There's a second layer to how it scores, that I think is the most interesting design decision in the whole project. Getting the flag isn't the end of the story. ExploitGym runs a separate scorer over your run that checks whether the agent actually used the intended vulnerability to get there, or whether it found some other path and the flag capture is a scoring artifact. Flag captured and causally exploited the target bug are tracked as two different numbers. The benchmark authors clearly don't trust flag capture alone as a signal, and neither should you.

## Why this is worth caring about if you're not a browser or OS vendor

Almost nobody writes browser engines or kernels for a living. So which part of this actually transfers to a normal company's AppSec program?

**Userspace C/C++ is the one that matters most, by a wide margin.** Every company has a software supply chain pulling in C/C++ libraries somewhere, whether it's image parsing, compression, or something like OpenSSL sitting three dependencies deep. CVEs in that layer show up in dependency scanners constantly, regardless of what language your own application is written in. Working through this domain builds the thing that actually pays off in a vuln management job: a sharper sense of whether a given CVE is realistically weaponizable or just alarming on paper. That triage instinct is worth more than the exploit itself.

**Linux kernel matters for a narrower but real reason.** Almost everyone runs containerized workloads now, and kernel privilege escalation bugs are the basis for most real container escape CVEs. You don't need to be able to write a kernel exploit for this to be useful. You need to understand the bug class well enough to judge whether your actual container hardening, seccomp profiles, gVisor, Kubernetes pod security settings, would stop it.

**V8 is the narrowest of the three for most companies.** It matters if you're running an Electron app or a Node-heavy backend, since V8 bugs aren't purely a browser problem. For most other companies this bucket is just "patch Chrome and Node promptly." It's a much bigger deal if you're a browser vendor or doing dedicated threat intel work, less so otherwise.

## The angle I actually care about

There's a fourth use for this that has nothing to do with vulnerability triage: it's a live instance of a governance question companies are asking right now. How much autonomy should a coding agent get before an action requires a human to sign off? OWASP calls this Excessive Agency. Watching what an unsupervised agent can and can't pull off against a known vulnerability class, with real measured numbers instead of a vague claim, is a much stronger way to talk about agentic AI risk than the generic "AI security" pitch most people are making right now.

That's the part I'm actually building toward. Next post covers what it took to get a properly isolated environment running this thing in the first place, which took a full day and involved three separate moments where I was convinced something was broken and it turned out not to be.