Testing AI for Vulnerability Research: 4 Approaches & Where I Failed

TL;DR

I tested four AI-assisted vulnerability-research workflows over roughly one week. The strongest run found 14 confirmed bugs in one target in 20 minutes; another approach produced many parser bugs but nothing I could report as a vulnerability. Across all four, the models were useful for coverage, hypothesis generation, and code analysis, but consistently overstated impact and produced unreliable validation. I still had to verify each finding and build every exploitable chain manually.

Note: I don’t have a formal AI/ML background. This was my first time seriously using AI for vulnerability research. Results will vary with target selection, setup, prompts, and workflow. I deliberately chose some heavily reviewed targets, which skews the numbers. The work took about a week and a half. The bugs shown here are examples rather than the complete list; some findings are omitted because disclosure is pending or the details would identify a target.

Approach 1: Blackbox RFC Spray

I used AI to help build the testing setup, pointed it at the research direction, and had it systematically map RFC gaps, contradictions, and MAY/SHOULD ambiguities across HTTP/1.0, HTTP/1.1, proxy specifications, and HTTP/2.

The primary focus was HTTP/1.1. The AI mapped where specs disagree - RFC 7230 vs RFC 9110 on Content-Length handling (first-wins vs last-wins), where “MUST reject” becomes “MAY ignore” across spec versions, where proxies are allowed to be lenient. From those gaps, it generated payload variations and I sprayed them across a matrix of 23 backend servers and proxies in Docker labs.

Results: 55 findings across 23 backend servers, tested through H1 and H2→H1 proxy chains. After triage: 8 confirmed exploitable with end-to-end combo lab PoCs, 11 path traversal bugs, 4 logic bugs, plus a funky chunk RFC violation across 16 servers and 6 proxies. 3 turned out to be false positives, 2 were already reported by other researchers.

Confirmed Bugs

The exploitable bugs all follow the same pattern: the AI finds a parser disagreement, I build the proxy→backend chain in Docker labs to prove it’s exploitable end-to-end.

CL octal parsing via strtol(base=0). A backend’s Content-Length parser uses strtol with auto-base detection. CL:0400 is read as octal 256, not decimal 400. Proxy reads it as decimal 400 and forwards 400 body bytes. Backend only consumes 256 - the remaining 144 bytes become a smuggled request. Confirmed exploitable through two major proxies in default configuration - no lenient mode needed.

POST / HTTP/1.1
Content-Length: 0400       ← proxy: decimal 400. backend: octal 256.

[256 bytes padding]GET /smuggled HTTP/1.1
Host: target               ← backend treats these 144 leftover bytes as next request

Bare LF chunk framing. A backend accepts bare \n (without \r) as chunk line terminator. One proxy forwards bare LF chunks. Backend and proxy disagree on body boundaries - 1 request in, 2 responses out. Confirmed end-to-end.

Duplicate TE first-wins. Two TE headers: TE:identity + TE:chunked. Backend takes first (identity → falls to CL). A proxy that forwards both uses chunked. Body framing disagrees - smuggling confirmed through one proxy.

Multi-valued TE ignored. TE:gzip,chunked - backend doesn’t recognize the comma-separated value as chunked, falls to CL. Proxy parses chunked from the list. Same desync pattern, confirmed through same proxy.

Space-in-name header truncation. A backend silently drops ALL headers after encountering a space in a header name. Content-Length placed after the malformed header vanishes. Body becomes a smuggled request. Dead in pure H1 - every proxy rejects in default config. Revived through a major proxy’s documented lenient mode:

POST / HTTP/1.1
Host: target
X -Forwarded: x           ← space terminates backend's header parsing
Content-Length: 43         ← backend NEVER SEES THIS (truncated)

GET /admin HTTP/1.1        ← these 43 bytes aren't consumed as body
Host: target               ← backend treats them as the next request

Proxy checked ACLs on the POST. Backend processes the smuggled GET to /admin. ACL bypass with full PoC.

NUL byte header splitting. A backend splits headers on NUL bytes. X-Foo\x00Content-Length: 50 becomes two separate headers. Dead in pure H1 - all proxies reject NUL in headers. Revived via H2 downgrade: one H2→H1 proxy forwards NUL bytes in header values.

Path traversal bugs (11 across 3 backends). Backslash traversal (/public\..dmin → /admin), null byte truncation (/admin%00.jpg → /admin), encoded dot traversal (/%2e%2e/admin → /admin), semicolon stripping (/public;/../admin → /admin), fragment as path separator. These work through any proxy - proxies forward URI paths without normalizing these patterns. ACL bypass trivial to demo.

Underscore-to-hyphen header normalization. Two backends map _ to - in header names internally. X_Forwarded_For → X-Forwarded-For. Proxies that strip X-Forwarded-For from clients won’t strip X_Forwarded_For. IP spoofing past proxy XFF filters through any proxy.

CRLF in quoted chunk-extension. RFC doesn’t allow CRLF inside quoted strings in chunk-ext. 16 servers + 6 proxies don’t track quote state - CRLF inside a quoted chunk-ext value is treated as real line terminator, promoting attacker text as trailer headers. Reported across the board, but chunk-extensions are not implemented properly in ecosystem, no proper exploitation since most treat it in the same way.

H2→H1 Downgrade Testing

We also tested the H2→H1 conversion layer across 15 proxies with ~3,600 test cases across 26 categories. The H2 testing found mostly RFC violations - proxies accepting things they shouldn’t per spec, but not actually forwarding them in exploitable ways. The main value was reviving dead H1 bugs: the NUL byte header splitting and bare CR/LF delivery paths only exist through H2 proxies that have different validation than H1 proxies.

Some hypotheses were my ideas that the AI formalized, others the AI generated on its own from the research material we’d gathered:

Empty DATA frame → zero-size chunk injection. From the H2 spec material, the AI reasoned that an empty DATA frame (length=0, no END_STREAM) mid-body could convert to 0\r\n\r\n - the chunked terminator - smuggling everything after it.

RST_STREAM after partial body → dirty backend connection. H2 POST with CL:1000, deliver 200 bytes, RST_STREAM. Proxy pools the backend connection, next victim’s request fills the remaining body. In practice, proxies handle RST_STREAM cleanly and don’t return dirty connections - this one didn’t pan out.

What AI missed

The AI did not find any of the exploitable chains. It found backend parser bugs and mapped which proxies forwarded each input; I identified the relevant lenient modes, reduced the triggering bytes, and built the end-to-end proxy-to-backend chains.

It also inflated results badly. Out of 55 initial findings, many were false chains that looked plausible on paper but fell apart in testing. 3 were outright false positives (0 drain hits on re-test). The model would flag a payload as “working” when the proxy-backend pair would never actually desync on it. I still had to sit in labs validating every single one.

It was useful for mapping RFC gaps, generating research directions, and finding parser bugs. The chaining logic and exploitability analysis came from manual work.

Approach 2: Two Fine-Tuned Models + Opus Brain

The most involved setup. Three-tier model architecture: a 32B security expert fine-tuned on ~24,000 training pairs (CVE analysis, attack patterns, spec gaps, encoding tricks), a 14B implementation expert fine-tuned per-target on code tracing and edge case behavior, and Opus as orchestrator deciding what to ask each one.

Target: XML parsers. I defined 61 parsers across 11 language ecosystems. Combined all attack surface knowledge into the security expert model. Tested 12 parsers across ~2,800 payloads.

Results: 53 confirmed bugs, 147 suspicious behaviors across 12 parsers. Zero reportable.

Bugs Found (Not Exploitable)

The numbers look impressive: 200 total findings across 12 parsers. These are real spec violations, real wrong behaviors, real bugs. But I tested them against major downstream projects - SAML libraries, signature verifiers, document processors - and none of the bugs were exploitable in practice. Every single one needs a specific precondition that doesn’t hold in real deployments.

Some examples of why they don’t matter:

Prototype pollution via constructor.prototype. The parser sanitizes __proto__ but not the constructor chain. <constructor><prototype><isAdmin>true</isAdmin></prototype></constructor> creates a pollution gadget. But exploitation requires the downstream application to pass parsed output through a deep-merge utility like _.merge - and the major projects consuming these parsers don’t do that.

Numeric character reference passthrough. XML §4.1 says &#N; MUST be resolved. Multiple parsers leave them as literal strings. <script> passes untouched but a browser would decode it. Exploitable only if the parsed output goes directly into an HTML context without re-encoding - and the downstream SAML/XML-signing libraries I checked don’t do that.

Duplicate attribute acceptance (last-write-wins). XML §3.1 forbids duplicate attributes. Multiple parsers silently accept them. But for this to be a security issue, a SAML library would need to rely on attribute uniqueness for assertion integrity - and the ones I tested validate signatures before extracting attributes, so duplicate attributes in unsigned content don’t help.

Missing attribute newline normalization. XML §3.3.3 says LF in attribute values must be replaced with space. Multiple parsers preserve raw newlines. Breaks C14N and digital signatures - but only if the signing library and the verifying library use different parsers with different normalization behavior. The libraries I tested use the same parser for both.

Stopnode premature close. </script> inside a string literal breaks out of the stop node. Direct XSS vector - but only in an HTML-in-XML configuration that nobody uses in the downstream projects I checked.

In each example, the parser behavior and specification violation are real, but the required exploitation precondition is absent from the downstream projects I checked.

Why parser-only testing failed

Testing parsers in isolation omits the downstream use that determines impact. The models found unusual behavior reliably but did not establish whether a real application reached the affected path.

The fine-tuned setup did give the orchestrator better implementation context. It could ask a specialist model whether a parser normalizes whitespace before or after entity expansion and receive an answer grounded in that codebase. For prototype pollution, it could use the implementation model’s account of how output objects are built instead of relying on general knowledge. The architecture may be useful against an application such as a SAML library, but isolated parsers were a poor target because downstream preconditions determined whether any result mattered.

Each target run cost about $4–6, but preparing the training material and fine-tuning the models required substantially more time and compute. That setup only makes sense for systematic research across an ecosystem.

Approach 3: Hypothesis-Driven from Code and CVE History

Instead of spraying blind, this approach reasons about the target:

1. Indexes the target codebase using tree-sitter - function signatures, call graphs, entry points
2. Summarizes code structure and identifies trust boundaries
3. Pulls CVE history from OSV.dev for the target and its dependencies
4. Runs behavioral probes - 30-50 edge-case tests to observe actual library behavior before theorizing
5. Generates structured hypotheses with confidence levels based on code patterns + past vulns + observed behavior
6. Maps dependency impact chains - if library A has a flaw, which downstream libraries inherit it?

I tested this on a SAML authentication library and its dependency chain - 7 libraries covering XML parsing, signature verification, XPath evaluation, and session management.

Results: 38 hypotheses tested across 3 runs. 12 confirmed (31%), 3 partial, 23 rejected.

The hypothesis quality was the real output here, more than the bugs themselves. Each hypothesis wasn’t just “test X for vulnerability Y.” It had structure: the specific function, what assumption it violates, the attack vector, and why it thinks this based on CVE patterns in similar libraries. One hypothesis looked at a 2021 signature bypass CVE, traced how the fix changed XPath evaluation, cross-referenced that with the current logout flow, and asked “does this same trust boundary violation exist here?” It cited the specific code lines, the CVE pattern, the assumption being violated. That kind of cross-library reasoning across a dependency chain takes days to do manually.

Bugs Found (Low Exploitation Impact)

The biggest differentiator was the deep analysis step - Opus with a 100K token budget for extended thinking.

Baseline findings (9 medium/low):

• Unvalidated relay parameter - open redirect/XSS, no sanitization. Confirmed across GET and POST bindings, no auth required. But depends on application-level handling that most frameworks already mitigate.
• No size validation on auth payloads - 5MB base64 payload = ~2.5s CPU, ~30MB RAM. Zero size checks. Unauthenticated, but a slow resource drain, not a crash.
• Cross-tenant certificate exposure - Host header manipulation in multi-tenant metadata generation leaks one tenant’s certs. Requires multi-tenant config most deployments don’t use.
• Cache design forces disabling replay protection - default cache provider doesn’t work in clustered deployments, pushing users to disable replay checks entirely.
• Async logout race - logout URL generation runs async but session destruction fires unconditionally before it completes.
• Deprecated URL parser differential - url.parse creates req.url/req.query divergence for redirect binding signature verification.

Deep findings (3 the baseline missed):

• Session survives SP-initiated logout - when a logout acknowledgment arrives, the handler calls this.pass() - continuing to the next middleware - without ever calling req.logout(). The session survives indefinitely after the user “logs out.” Confirmed: 5/5 tests showed pass_called: true, req_logout_called: false, session_still_active: true. But requires a specific SLO flow most deployments don’t exercise.
• IdP-initiated SLO race condition - logout sends the redirect before session destruction completes, leaving a ~50ms race window. If the session store write fails, the session survives. Tiny window, depends on store failure.
• Query parameter priority override - query-string parameters unconditionally take priority over POST body, allowing GET parameters to override POST-binding messages.

The bugs are real. The reasoning that found them was impressive. But the exploitation impact was limited across the board. Individually, none of these were things I’d confidently report as vulnerabilities.

Where validation failed

Validation was the weakest part. Test cases often looked correct without triggering the bug reliably. The model sometimes claimed a PoC worked when it did not, or asserted the wrong condition after partially reproducing the behavior. Its hypotheses were useful, but I could not rely on its lab results.

Approach 4: Domain Research + Frontier Model (SECRA)

Unlike Approach 3, this workflow drops the hypothesis-generation step. SECRA gathers domain-level research (CVEs, attack techniques, spec violations, and code patterns) and structures it into individual investigation goals for a frontier model.

The pipeline builds a research package for a target domain:

1. CVE analysis - Pull every CVE for the domain, extract root causes, fix gaps, and variant angles. For SAML: 100 CVEs across all known implementations.
2. Technique extraction - From blogs, writeups, commits, and advisories, extract concrete attack techniques. 149 techniques for SAML.
3. Pattern mapping - 25 code-level vulnerability patterns with what to search for, why it’s a bug, how to test it, and which CVEs it maps to.
4. Spec analysis - Map MUST/SHOULD requirements, discretion zones (where the spec says “MAY” and implementations diverge), and undefined behaviors. For SAML: things like “spec doesn’t define behavior with multiple Assertion elements,” “spec doesn’t address XML comment injection in NameID values,” “URL comparison rules for Destination/Recipient not specified.”
5. Novel angles - Attack hypotheses that don’t come from existing CVEs. Things like xml:base attribute injection to redirect Reference URI resolution, or internal DTD ATTLIST injection to fabricate ID attributes on forged elements.

The package instructs the model to take one pattern or angle at a time, read the target code, trace the data flow, write and run a concrete test, and record the result.

The frontier model receives this package alongside the target codebase. It reads the source, writes and executes test harnesses, and checks each pattern or angle against the target instead of generating new hypotheses first.

One caveat: I had to review everything to make sure the model wasn’t taking shortcuts - disabling security features in the test setup, weakening configurations, or subtly making the code more vulnerable and then “finding” the bug it introduced. This happens. The model wants to show results, so it’ll disable signature verification “for testing,” use an insecure default it configured itself, or write a harness that strips validation the real library would perform. You need to verify every finding exists in the real library with its real configuration, not just in the model’s lab. Can’t skip this step.

I ran this against multiple targets. The clearest predictor of the results was how much prior security review each target had received.

Hardened Target: A Mature SAML Library

18 findings. Breakdown:

• 7 were design choices the maintainers documented
• 4 were spec violations maintainers call “working as intended”
• 3 were unreachable dead code
• 2 were theoretical with no practical exploitation
• 1 was cosmetic

1 was reportable. A scope filtering function used string suffix matching instead of domain boundary matching:

host  = "login.notevil.com"
scope = "evil.com"

# Vulnerable check: is "evil.com" a suffix of "login.notevil.com"?
# position("evil.com") == len("login.notevil.com") - len("evil.com")
# 10 == 10 → TRUE (should be FALSE - not a domain boundary)

An attacker’s IdP at notevil.com can assert scoped attributes for evil.com. That’s a 5% hit rate. The AI inflated everything else. It couldn’t tell the difference between “this code does something weird” and “this is exploitable.” Every finding came with high confidence and detailed reasoning that sounded convincing. 17 out of 18 were noise.

Fresh Target: A SAML Library

I then tested an actively used SAML library with significant downloads but little focused security review.

Bugs Found

14 confirmed vulnerabilities in 20 minutes.

• 2 authentication bypasses - full SAML assertion forgery, requires understanding the entire authentication flow and where signature verification actually binds to assertion consumption
• CBC padding panic - CBC padding validation crashes on malformed input. The error path is distinguishable (theoretically a padding oracle), but full Vaudenay decryption wasn’t exploitable in practice. The DoS via the panic is real - unauthenticated crash with a single crafted message.
• XSLT execution - attacker-controlled XSLT transforms executed during signature verification. Code execution surface.
• Multiple DoS vectors - resource exhaustion through crafted SAML messages

Plus 3 patterns from the research that were checked and correctly ruled out - the model didn’t hallucinate findings where they didn’t exist.

Deeper Ecosystem Run

On a separate run against a different SAML ecosystem (7 libraries), the frontier model worked through the research package - patterns, techniques, spec violations - against each library in the chain. 9 findings were CVE-worthy after triage, 6 validated with standalone PoCs.

The structured domain research made a real difference here. The model wasn’t guessing. It was checking whether known vulnerability patterns from 100 CVEs across the SAML ecosystem exist in this specific target.

Bugs Found

CBC padding crash via error type leak. A CBC decryption function disables the crypto library’s built-in padding check, then does manual validation. On invalid padding, it calls a callback variable that doesn’t exist in scope - throwing a ReferenceError instead of a generic Error. The caller’s try/catch distinguishes these error types. Theoretically that’s a padding oracle - the model flagged it because the research package included Vaudenay patterns from CVE history, and the code matched: manual padding check after setAutoPadding(false). In practice, the full decryption attack wasn’t exploitable - but the crash on the error path is real. Unauthenticated DoS with a single crafted ciphertext. The AI correctly identified the vulnerable code pattern; it just overestimated the exploitation impact.

XPath injection in key retrieval. An encryption library takes a URI attribute from an XML element and concatenates it directly into an XPath expression:

//*[@Id='ATTACKER_CONTROLLED_VALUE']/*[...]

No escaping. Inject a single quote and a union operator, and you select arbitrary elements as the encryption key material:

<RetrievalMethod URI="#'] | //*[local-name(.)='CipherValue'][1] | //*[@Id='x" />

Silent algorithm downgrade. A switch statement matching OAEP digest algorithm URIs uses the wrong XML namespace in its case labels. The standard URIs sent by every major identity provider don’t match any case, so they all fall through to the default: SHA-1. Every deployment unknowingly uses SHA-1 for OAEP instead of the SHA-256/512 they configured.

Unsigned message acceptance. A signature verification function for redirect-binding messages returns true when the Signature query parameter is absent entirely - not “invalid signature,” but “no signature = valid.” Unauthenticated attacker sends a logout request with no signature, and the library accepts it:

GET /slo?SAMLRequest=<deflated LogoutRequest>&RelayState=...
# No Signature, no SigAlg query parameters
# hasValidSignatureForRedirect() returns true
# User's session terminated

Replay bypass via unsigned attribute stripping. Replay protection checks an attribute on the outer (unsigned) message envelope. The signed inner assertion doesn’t include this attribute. Strip it from the envelope - the signature still validates, but the replay check sees “no attribute present” and skips validation entirely, treating the message as unsolicited. Same signed assertion accepted multiple times.

Text node truncation via XML comment injection. An XML library extracts identity values using .data on the first child text node. The attacker controls the IdP and signs an assertion containing a comment-injected NameID. The signature is valid because the attacker signed it themselves. But the SP’s identity extraction only reads the first text node before the comment boundary:

<!-- Attacker (evil.com IdP) signs assertion with: -->
<NameID>admin@victim.com<!--.evil.com--></NameID>

<!-- Signature verifies: attacker signed it, digest matches ✓ -->
<!-- DOM .data extraction: "admin@victim.com" (first text node only) -->
<!-- SP authenticates attacker as admin@victim.com -->

The dependency impact chain analysis was useful here. The pipeline traced how a bug in an encryption library flows through the signature verification layer into the SSO login flow, confirming the crash is reachable from the unauthenticated attack surface.

These are not all major SAML issues such as signature bypasses. On heavily reviewed targets, most of the confirmed bugs were situational and depended on specific configurations or flows.

Other Findings

Across other SAML libraries with the same pipeline:

• CBC padding panic - unauthenticated DoS, full PoC
• Deflate bomb - 464KB compressed → 350MB expanded, 2 requests kill a 1GB container
• InResponseTo replay - nonce deletion commented out in the source, responses replayable indefinitely
• Metadata signature bypass - unsigned metadata accepted as trusted, enabling attacker IdP registration
• KeyInfo certificate injection - untrusted certificates from the SAML response used for signature verification. Full authentication bypass.
• Expired certificate acceptance - certificate validation always ignores expiry. Revoked/expired certs still trusted.
• Double-encrypted assertion signature bypass - signature verification skipped on the second decryption loop. Attacker with the SP’s public key can inject a forged assertion.

These had clear impact and were suitable for reporting.

Why the results differed

The hardened target produced a 5% hit rate in about 20–30 minutes, including finding, validation, and impact analysis. The fresh target produced 14 confirmed bugs in 20 minutes, while the ecosystem run produced nine CVE-worthy findings across seven libraries.

The model and pipeline did not change between runs; prior security attention did. On the hardened library, the obvious bugs had already been found. The remaining questions required specification knowledge and deployment context, including SAML canonicalization, XML Signature Reference URI resolution, and ambiguous behavior described as “working as intended.”

The fresh targets were not necessarily simple, but they had not been reviewed against the same known vulnerability patterns. Applying those patterns to previously untested code produced the better results.

What I learned

Target selection mattered more than model sophistication in these tests. The fine-tuned models produced no reportable parser findings, while the standard SECRA workflow found 14 bugs in 20 minutes on a less-reviewed target.

Every approach overstated findings. On the hardened target, 17 of 18 results were design choices, unreachable code, or behavior without practical impact, even though the model described them confidently.

Extended reasoning helped with code-path and state analysis that additional payload spraying would not have reached. Supplying vulnerability patterns grounded in prior CVEs also produced better results than asking the model to theorize from the target code alone.

Manual validation remained necessary. I had to check that a harness had not disabled a security feature, weakened a configuration, or asserted a condition that the real library did not satisfy.

In these runs, AI increased research throughput but did not replace target selection, exploit construction, impact analysis, or verification.

SECRA

SECRA is the tool I built and used across all 4 approaches. It’s a context engine - it gathers domain-level security research and packages it so a frontier model can work through it systematically.

For each target domain, it builds a research package: CVE history with root causes and variant angles, attack techniques extracted from blogs/writeups/commits, code-level vulnerability patterns, spec analysis (MUST/SHOULD requirements, discretion zones, undefined behaviors), and novel attack angles that don’t come from existing CVEs. The model gets this alongside the target codebase and works through it one pattern at a time - read the code, write a test, run it, report.

Each approach in this post used SECRA differently. Approach 1 used it to map RFC gaps and generate payload variations. Approach 2 used it to build training data and feed the fine-tuned models. Approach 3 used it for CVE history and code indexing to generate hypotheses. Approach 4 used it most directly - the full research package fed to a frontier model.

Sonnet handles bulk work such as CVE summarization and technique extraction. Findings are cached by ecosystem, name, and version so a dependency analyzed once can be reused across targets. I currently have four versions of the tool and plan to revisit the hybrid version when I have the budget.

Numbers

The week ranged from unproductive parser testing to 14 confirmed bugs in a short run. Specific, research-backed goals worked much better than a broad instruction such as Go look for bugs.

The bugs listed here are representative examples rather than the complete set. Some findings are pending disclosure and will be updated when identifiers are assigned. Many depend on specific deployment conditions, which is why those limitations are included alongside the results.