AI, LLM, agent and MCP scopes
An AI finding needs an attacker-controlled input, a security boundary, a measurable unauthorized result and reproducible conditions. Offensive wording, hallucinations and a model repeating supplied text are not automatically vulnerabilities. A program may separately reward model-safety failures; check that scope explicitly. OWASP's AI Testing Guide covers broader trustworthiness properties as well as security, while its LLM Top 10 supplies a risk taxonomy rather than universal bounty eligibility. AI Testing Guide, LLM Top 10
Investigation modules#
Use these procedures with this playbook's setup and scope. Each contains the full testing sequence, command cheat sheet, interpretation, and checklist; open its canonical page for cross-target references.
Object, function, and property authorizationTurn identities, tenants, objects, roles, and lifecycle transitions into a reproducible permission matrix.
Open canonical module and reverse references
- 01Establish permission rule
- 02Seed two controlled identities
- 03Replay one changed dimension
- 04Verify durable effect
- 05Revoke and repeat
- 06Record smallest violation
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Use accounts A and B, two private tenants, and one synthetic document per tenant. Create a viewer role only if the product supports it. Record the exact build, API version, actor, tenant membership, document visibility, and expected permission rule. The invariant is: the server authorizes the actor, action, object, and each protected property together. UI visibility is supporting context; product policy establishes the boundary. Horizontal, vertical, and context-dependent authorization are separate dimensions. Access-control models.
Numbered test procedure
- Capture a working baseline. As A, create private O1 containing
BB-A-ONLY-01; as B create O2 containingBB-B-ONLY-01. Save each normal read response and a normal reversible rename. Keep separate browser profiles and credentials. Confirm anonymous access to O1 fails. - Test the object dimension. Replay A's O1 read using B's credentials, changing nothing else. Then keep B's identity and replace O1 with O2 as a positive control. Substitute only known controlled identifiers, including parent tenant IDs, child IDs, attachment IDs, and export-job IDs. A nonexistent synthetic ID is the negative control for error handling; it does not replace the cross-account test.
- Test the function dimension. Give B a documented viewer role in A's test tenant. Capture an owner-only operation as A, such as renaming a private test folder, and replay it as B. If the product permits viewer renames, select an actually restricted operation. Inspect durable state from A's browser after the request completes.
- Test the property dimension. Start with a permitted profile update. Add exactly one known server-controlled property from schema or source, such as a role field in the isolated tenant. Read back the stored object, then test one corresponding restricted action. Repeat separately for unauthorized response properties. Automatic binding of request fields can create mass-assignment issues; read-side property disclosure is a different mechanism. OWASP mass assignment.
- Test lifecycle and alternate delivery. Remove B's membership, replay the previously successful request, and check an existing subscription or queued export. Repeat the same permission rule at list/search, download, batch, and versioned endpoints where present. Use two-item batches containing O1 and O2; do not enumerate identifiers.
- Interpret and clean up. Match markers, object IDs, committed changes, and resulting permissions. A 200 with an empty list, a 202 for a rejected job, or a redirect without private content is not proof. Remove synthetic records and memberships after preserving minimal evidence.
Command card
These illustrative routes require your local replica. Create A.header and B.header containing each test account's complete Authorization: Bearer … header; protect the files and exclude them from reports. curl can load headers from a file. curl manual.
mkdir -p evidence/authz
chmod 600 A.header B.header
curl -q --noproxy '*' --max-time 10 --header @A.header \
-D evidence/authz/a.h -o evidence/authz/a.json \
http://127.0.0.1:8000/api/documents/O1
curl -q --noproxy '*' --max-time 10 --header @B.header \
-D evidence/authz/b.h -o evidence/authz/b.json \
http://127.0.0.1:8000/api/documents/O1
curl -q --noproxy '*' --max-time 10 --header @B.header \
-o evidence/authz/control.json \
http://127.0.0.1:8000/api/documents/O2
Report threshold and checklist
Report a demonstrated unauthorized read, action, or protected-property change. Attach the intended rule, both actors' roles, minimal request difference, and independent state verification. UUID disclosure, shared objects, intentionally public exports, stale UI state, and identical status codes are common false positives.
- Ownership and visibility confirmed before testing.
- Object, function, and property decisions tested separately.
- Positive and negative controls captured.
- Background output and revoked membership checked.
- No unrelated customer records accessed.
Local curl examples use --noproxy '*' to bypass inherited proxy variables; -q alone only disables the default curl configuration file. Configure intentional interception separately. curl manual.
Authentication, recovery, federation, and session lifecycleTest registration, recovery, identity changes, invitations, SSO enforcement, and deprovisioning as one account state machine.
Open canonical module and reverse references
- 01Map identity states
- 02Capture valid and invalid controls
- 03Alter one binding
- 04Verify account identity
- 05Test revocation
- 06Restore account settings
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Prepare two controlled accounts, separate inboxes, browser profiles, and a recovery route for each. Use an isolated identity provider for federation modifications. Write the accepted identity states: unverified, password accepted, MFA pending, fully authenticated, elevated, recovered, and revoked. The invariant is: each privileged transition requires the right identity proof, and credentials remain bound to their intended account and purpose. Consult documented expiry and revocation semantics before judging residual access. OWASP authentication.
Numbered test procedure
- Inventory entry points. Record registration, email verification, password login, passkey/MFA completion, mobile login, SSO, API-token issuance, email change, linking, reset, and logout. Capture one complete normal sequence with timestamps and account identifiers. An anonymous call to
/api/meand an intentionally invalid credential form negative controls. - Check incomplete authentication. Stop A's normal login immediately before MFA. Use that intermediate cookie or token for a single protected profile read. Compare it with a completed A session and an anonymous request. Also check the observed alternate API route; avoid guessing or spraying credentials. Protected access without the required second factor is the finding, not the mere existence of an intermediate token.
- Check recovery binding. Request one reset for A and one for B. In the local replica, submit A's reset proof with B's account selector, if that selector is client-controlled. After an authorized normal reset of A, replay the consumed proof once. Confirm which controlled account's credential changed by a fresh login. Restore passwords afterward. Single-use, expiry, and account binding are independent checks. OWASP recovery.
- Check session transitions. Save the pre-login identifier, completed session, and any refresh credential separately. Compare identifiers after login and elevation. Perform logout or explicit device revocation and make one old-session profile request. Record access and refresh outcomes separately; a refresh token being revoked does not necessarily revoke already-issued access tokens. Evaluate the actual advertised security behavior. OWASP sessions.
- Check identity changes and pre-existing state. Change A's verified email to a second owned inbox; compare verification, pending state, and recovery delivery before/after confirmation. In the replica, keep an unverified password account open, then complete the legitimate verified/federated signup for the same owned inbox in another profile. Inspect whether the original password, session, or linked identity can still access the now-verified account. Independently test reusing a stale verification link after another email change. A duplicate signup error or two aliases reaching the same mailbox proves no takeover. Do not assume case folding, dot removal, or plus-address handling is universal. Email identity comparison and verification.
- Follow enterprise onboarding and offboarding. Use a disposable tenant with a second recovery administrator. Record invitation recipient/role/tenant, acceptance, cancellation, SSO enforcement, group assignment, and deactivation. Compare a fresh invite with one cancelled invite; confirm whether a changed recipient is allowed by the product contract. After enabling enforced SSO, attempt the observed password, reset/magic-link, and alternate client entry paths once with controlled accounts, then request one protected tenant object. Deliberate break-glass exceptions are separate policy. For SCIM, use the tenant's provisioning credentials and verify deactivation, group removal, token refresh, and existing streams after the configured sync interval. SCIM
activehas provider-defined semantics; it is not a universal immediate-session-kill instruction. SCIM schema. Provisioning credentials must still map to the permitted tenant/resources. SCIM authorization. - Follow the specialized proof branches. Use MFA/passkey lifecycle for enrollment, factor replacement, backup-code reuse, and WebAuthn verifier checks. Use federation/token trust for OAuth PKCE, OIDC claim and account-linking decisions, JWT validation, and SAML. Do not substitute decoded claims for a protected-resource result. Document which providers, authentication methods, and enterprise features were unavailable rather than marking them passed.
Command card
Export A's local cookie jar before using the application's real logout button. These requests compare protected content before and after logout; routes and credential transport must match the replica.
mkdir -p evidence/session
chmod 600 A-before.cookies
curl -q --noproxy '*' --max-time 10 -b A-before.cookies \
-D evidence/session/before.h -o evidence/session/before.json \
http://127.0.0.1:8000/api/me
# Perform the actual logout or device-revocation workflow now.
curl -q --noproxy '*' --max-time 10 -b A-before.cookies \
-D evidence/session/after.h -o evidence/session/after.json \
http://127.0.0.1:8000/api/me
curl -q --noproxy '*' --max-time 10 -o evidence/session/anonymous.json \
http://127.0.0.1:8000/api/me
Report threshold and checklist
Report access as the wrong controlled identity, completion without required authentication, unauthorized account linking, or a material violation of revocation guarantees. Cosmetic logout, unsent browser cookies, decoded JWT contents, and reset forms displaying without completing are common false positives.
- Account and session state recorded for every request.
- Browser binding distinguished from token validity.
- Correct and incorrect proofs compared independently.
- Fresh login verifies any credential or linking change.
- Verification, linking, invitations, and enforced-SSO fallback paths evaluated.
- Provisioning changes checked against configured sync and revocation behavior.
- Recovery settings restored; live tokens removed from report.
Trace input into queries, commands, and templatesProve interpretation at a specific sink using small paired experiments instead of payload spraying.
Open canonical module and reverse references
- 01Identify input and sink
- 02Freeze a synthetic fixture
- 03Send paired controls
- 04Observe sink semantics
- 05Confirm smallest effect
- 06Document root cause
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Use a local replica seeded only with synthetic records, a normal request, and an observable sink: database query logs, a template render, or a disposable worker's process log. Identify input type and decoding layers before selecting syntax. The invariant is: untrusted input remains data, or is constrained to an explicitly authorized language. A SQL search field, MongoDB filter, template editor, and process argument have different trust contracts; the same string is not a universal test.
Numbered test procedure
- Trace the route. From the handler, follow validation, normalization, service calls, query builders, template compilation, and subprocess invocation. In a black-box target, infer this from observed feature behavior and errors, then reproduce locally when possible. Record the last point where input is still data and the first interpreter boundary. An ORM can still contain unsafe raw fragments.
- Freeze controls. Create exactly one matching synthetic record and one nonmatching record. Capture the normal response. Choose an input that must produce no match under the intended semantics. Keep actor, tenant, pagination, and cache state unchanged across each pair.
- Test SQL or NoSQL semantics. For a known SQL string context in the lab, compare logically true and false expressions while retaining the same synthetic predicate. Prefer local query logs over timing or data extraction. For a JSON filter expected to accept a string, compare
{"name":"BB-CANARY"}with{"name":{"$eq":"BB-CANARY"}}, then$eqwith a nonexistent marker. Operator acceptance becomes a vulnerability when it escapes the allowed query contract or security predicate; intentional query APIs are different. SQL injection, NoSQL injection. - Test template evaluation. For a documented Jinja/Twig-like local sink, compare literal text,
{{7*7}}, and{{7*8}}. Inspect the raw HTTP response before JavaScript executes. Results 49 and 56 support server interpretation; a single 49 might be ordinary content. If the feature intentionally supports expressions, determine whether a lower-trust actor crosses a promised sandbox or data boundary before reporting. Arithmetic alone does not establish OS execution. Template injection. - Test command interpretation locally. With an identified POSIX shell sink, use a disposable process and a marker-only operation such as
printf BB-CMD-01; compare the literal argument path with the shell-interpreted path in process logs. Avoid file writes, network callbacks, account enumeration, and long sleeps. Also distinguish argument injection into a legitimate executable from starting an additional shell command. Command injection. - Confirm causality and clean up. Repeat the baseline and one successful pair, then switch to the validated/parameterized route in the replica as a negative control. Capture the input-to-sink path, raw values after decoding, and smallest observable impact. Remove fixtures and stop at proof.
Command card
The following performs three bounded template probes against an illustrative local endpoint. Use only when that endpoint is expected to render the parameter as text; do not infer engine identity from one expression.
mkdir -p evidence/injection
curl -q --noproxy '*' --max-time 10 --get --data-urlencode 'name=BB-LITERAL' \
-o evidence/injection/baseline.txt \
http://127.0.0.1:8000/preview
curl -q --noproxy '*' --max-time 10 --get --data-urlencode 'name={{7*7}}' \
-o evidence/injection/expression-a.txt \
http://127.0.0.1:8000/preview
curl -q --noproxy '*' --max-time 10 --get --data-urlencode 'name={{7*8}}' \
-o evidence/injection/expression-b.txt \
http://127.0.0.1:8000/preview
Report threshold and checklist
Report attacker-controlled interpretation that violates the feature's contract, supported by paired results or an instrumented source trace. Syntax errors, WAF blocks, reflection, slow requests, and intentionally supported expression syntax are insufficient alone. State whether the demonstrated impact is query manipulation, expression evaluation, argument injection, or command execution.
- Sink, language, quoting context, and decode order identified.
- Synthetic positive and negative records available.
- One input changed per comparison.
- Raw server output separated from client rendering.
- Evidence stops short of secret retrieval or destructive execution.
Local curl examples bypass environment proxies with --noproxy '*'; -q alone only suppresses curl configuration files. These snippets target loopback and do not follow redirects. curl manual.
Delayed consumers and argument-only execution
A safe database insert does not make the stored value trusted forever. In the replica, save a marker through the normal create/update route, then trigger one observed export, search, scheduled job, or administration view that consumes it. Capture both requests, the stored bytes, and the later interpreter call. Repeat with a literal marker and one context-specific expression; keep the actor and selected synthetic record constant. This checks second-order injection that an immediate-response scan misses. Second-order SQL injection.
Treat process arguments as a separate boundary. A subprocess argument array with shell=False can still pass an attacker-controlled option to the intended executable. In a local wrapper, compare a normal filename-like argument with a harmless, documented mode switch and observe the exact argument vector and output. Prove that the caller was allowed to supply a value but not that mode. Do not test options that write files, run another program, or contact a network. Report shell-command interpretation under CWE-78 and argument/option injection under CWE-88; neither automatically proves the other. CWE-88.
- Stored input followed through its later privileged consumer where one exists.
- Argument vector distinguished from shell command text.
Server-side fetching and network trust boundariesSeparate an intended fetch from a request that reaches a prohibited destination, forwards authority, or exposes a protected response.
Open canonical module and reverse references
- 01Map fetch contract
- 02Start controlled canary
- 03Confirm ordinary fetch
- 04Change one destination rule
- 05Correlate server evidence
- 06Bound the impact
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Identify a feature that causes server-side requests: preview, import, webhook verification, document conversion, image fetch, repository import, or AI tool execution. Record allowed schemes, destinations, redirect rules, credentials, and whether response bodies are returned. Use a local replica with a tiny HTTP canary reachable from its worker. If the application runs in a container, localhost means that container; place the canary on an explicitly controlled test network instead of guessing host addresses.
The invariant is: every actual destination and credential transfer follows the feature's intended policy. A URL-fetching feature contacting your public receiver is often normal functionality. SSRF requires an unintended request or trust-boundary violation. SSRF mechanics.
Numbered test procedure
- Document the ordinary path. Submit a permitted static URL on your own receiver, containing a unique marker in its path. Record submission time, job ID, request method, receiver log, worker identity if available, and returned output. Confirm the browser did not fetch the URL directly by repeating through curl with the browser closed.
- Establish controls. Submit a second permitted URL that returns a different marker. Also submit a deliberately rejected scheme or destination from the feature's documented policy. Record whether rejection happens before queueing, at DNS resolution, after connection, or during response handling. A DNS lookup alone establishes resolution, not HTTP access.
- Test one policy boundary in the replica. Compare a permitted receiver with an owner-provided private canary that policy should block. If arbitrary public destinations are intended, concentrate on restricted network ranges, forbidden ports or schemes, or unintended credentials, using only designated canaries. Never use production metadata credentials as proof.
- Test redirect handling. Configure your first controlled endpoint to return one redirect to the second controlled canary. First verify both endpoints independently. Submit the first URL and correlate logs to show whether the second destination was validated. An initial allowlist check is insufficient if the actual redirected destination crosses the rule. Where authorized, study DNS validation/connection consistency in the replica with a controlled resolver; do not perform Internet-wide rebinding experiments. OWASP SSRF prevention.
- Inspect authority and output. Check whether a synthetic authorization header, test cookie, or privileged worker identity reaches the receiver unexpectedly. If the app displays a response, return only a fixed canary string. Distinguish blind connectivity from a readable internal response, and identify whether headers, method, body, or destination are attacker-controlled.
- Interpret and stop. Correlate exact marker, time, source, request, and job. Rule out your browser, webhook delivery by design, scanners, and unfurling services. Stop after the first controlled boundary violation; do not sweep adjacent hosts or ports. Shut down canaries and delete test jobs.
Command card
Run the receiver in a new empty directory in one terminal. This Python server serves only the canary directory and binds loopback. It is a local test receiver, not production hosting. Python HTTP server.
mkdir -p ssrf-canary
printf '%s\n' 'BB-SSRF-CANARY-01' > ssrf-canary/probe.txt
python3 -m http.server 8765 --bind 127.0.0.1 --directory ssrf-canary
In another terminal, send one request to your local application's observed fetch route. The example assumes same-host processes and JSON request data.
curl -q --noproxy '*' --max-time 10 --header @A.header \
-H 'Content-Type: application/json' \
--data '{"url":"http://127.0.0.1:8765/probe.txt?case=BB01"}' \
http://127.0.0.1:8000/api/preview
Report threshold and checklist
Report the prohibited destination or credential boundary actually reached. Label DNS-only evidence and environmental limitations explicitly. An open redirect or successful public image import alone does not prove internal access.
- Feature's destination contract written down.
- Receiver ownership and worker network namespace established.
- Ordinary fetch and rejected control recorded.
- Redirect destination and credential behavior inspected.
- No metadata secrets, unrelated internal addresses, or port sweeps used.
Local curl examples bypass environment proxies with --noproxy '*'; -q alone only suppresses curl configuration files. These snippets target loopback and do not follow redirects. curl manual.
Resolver, retry and connection identity
In source review, trace the URL parser, A/AAAA lookup, selected socket address, redirect handling and retry path as one decision. Verify the checked address is the one actually connected to; a second resolution, IPv6 path, alternate IP representation, redirect or worker retry must not escape the intended policy. For a DNS-change hypothesis, use a controlled resolver and designated canaries in the replica, record returned addresses and actual connections, and compare with a fixed-answer control. Merely accepting an unusual URL spelling does not prove a prohibited connection. Keep TLS verification enabled where HTTPS is used; do not change the client trust model to make the proof succeed. OWASP SSRF prevention.
Browser execution, cross-origin reads, and forged actionsUse real browser behavior to distinguish XSS, CSRF, CORS, and messaging failures from header-only leads.
Open canonical module and reverse references
- 01Locate source and browser sink
- 02Prepare two controlled origins
- 03Establish normal behavior
- 04Run browser-feasible probe
- 05Verify execution or state
- 06Remove stored content
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Use two controlled accounts and separate browser profiles. For cross-origin tests, use a researcher page and the local target on different ports; these are different origins but may still be the same site. For conclusions involving SameSite cookies, reproduce with genuinely different sites in an authorized HTTPS setup. Record browser version, cookie attributes, CSP, iframe sandboxing, request initiator, and navigation context.
The invariant is: a lower-trust origin or stored value cannot execute in the application origin, read protected responses, or cause an unauthorized credentialed action. Each of those outcomes is a separate test. XSS, CSRF, CORS.
Numbered test procedure
- Map sources and sinks. Start with a distinctive text marker. Track it through reflected HTML, attributes, JavaScript strings, DOM insertion, Markdown preview, stored comments, embedded documents, and
postMessagehandlers. Record what is escaped and whether rendering occurs server-side or in JavaScript. Do not copy one payload across unrelated contexts. - Build a normal and inert control. As A, save plain text into a private object B is allowed to view. Verify B sees it. An HTML-escaped version is the inert comparison. For a local HTML text sink, a marker-only example is
<img src=x onerror="document.documentElement.dataset.bb='BB-XSS-01'">. Inspect B's DOM for the resulting attribute; the payload never reads cookies or sends data. If CSP blocks execution, preserve that outcome instead of claiming execution. - Test CSRF delivery. Capture B's normal reversible preference change. From the controlled other origin, reproduce it using a browser-deliverable form or fetch request without a manually inserted target bearer token. Verify which cookies were actually sent, whether preflight occurred, and whether the setting changed in a fresh target tab. Use an invalid CSRF token or logged-out profile as a control. A manually edited HTTP request is not by itself a browser exploit.
- Test CORS reads. From the researcher page, call a private test endpoint with
credentials: 'include'and attempt to read the response body. Compare an allowed application origin, the researcher origin, and an anonymous request. Save the actual browser result and credential behavior. A request reaching the server is different from JavaScript reading its response; wildcard origins do not enable credentialed response access. Browser CORS behavior. - Test messages and embedding where present. For an observed
postMessagehandler, send the expected message shape from your controlled page and change only the sender origin. Verify a harmless state marker or a restricted action on your account. For iframe interaction, prove the sensitive action is practically inducible, rather than reporting absent framing headers alone. - Interpret delivery and cleanup. Record which account planted content, which viewed it, the effective origin, and the boundary crossed. Remove stored payloads and reverse preference changes. A payload that only executes after the tester pastes JavaScript into DevTools has not established an application injection path.
Command card
Create browser-probe/index.html locally with this content, then serve it using the command below. Open it while the separate controlled target session exists; confirm that your browser's cookie policy permits the intended experiment.
<!doctype html><meta charset="utf-8"><pre id="result"></pre>
<script>
fetch('http://127.0.0.1:8000/api/me', {credentials: 'include'})
.then(r => r.text())
.then(t => { document.querySelector('#result').textContent = t; })
.catch(() => { document.querySelector('#result').textContent = 'Read blocked or request failed'; });
</script>
python3 -m http.server 8766 --bind 127.0.0.1 --directory browser-probe
Report threshold and checklist
Report cross-user execution, a browser-readable private response, or a forged protected state change with realistic delivery. Reflection, arbitrary Origin acceptance in curl, missing CSRF tokens, and same-site laboratory results generalized to cross-site attacks are common false positives.
- Real browser reproduces the claimed behavior.
- Origin and site relationships recorded separately.
- Credentials, CSP, preflight, and redirects observed.
- Benign marker demonstrates only the required impact.
- Stored content and account settings cleaned up.
Uploads, downloads, paths, archives, and XMLFollow a file through acceptance, storage, processing, publication, and retrieval while preserving ownership and path boundaries.
Open canonical module and reverse references
- 01Map file lifecycle
- 02Seed harmless canaries
- 03Change one file property
- 04Check parser and destination
- 05Verify another controlled reader
- 06Remove every derivative
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Use a local replica, a tiny supported file, two controlled accounts, and a dedicated scratch directory. Identify upload handler, object store, converter, background worker, CDN/download route, and deletion path. Place a unique harmless canary just outside the permitted download root, through normal local filesystem access. The invariant is: user-controlled file metadata or contents cannot escape storage, interpretation, or ownership rules.
Separate filename, MIME declaration, actual bytes, storage key, filesystem path, and retrieval origin. Each component may normalize them differently. Acceptance of an unusual extension alone does not prove unsafe execution. Upload handling.
Numbered test procedure
- Record a supported baseline. Upload a tiny text/image fixture through the documented route. Save its object ID, resulting filename, declared and detected type, processor status, retrieval headers, and response bytes. Verify A can retrieve it and B cannot when the file is private.
- Change one property. Resubmit the same harmless content with a different filename or MIME declaration, independently. For a type-restricted image endpoint, text labeled
image/pngshould either fail or be safely handled. Inspect the stored result and actual browser rendering. Do not upload executable shells or server configuration files to demonstrate an extension mismatch. - Check path containment. In the local replica, request the planted out-of-root marker using one relevant
../path. Compare a valid in-root file and a nonexistent sibling path. Preserve the raw request target and server-resolved path; a client or proxy may normalize dot segments before they reach the application. If testing writes, use only a dedicated disposable output directory and check that an outside marker is not created. Path traversal. - Check ownership through derivatives. As B, request A's controlled original, thumbnail, conversion result, archive export, and any observed signed URL. Record whether the URL is intentionally a share capability and its documented lifetime. Deleting the primary record does not prove every derivative became inaccessible; evaluate the product's promised behavior.
- Check archive and XML processing locally. For an extractor, create a tiny archive with one ordinary member and one canary path outside its designated extraction directory; inspect resolved paths without overwriting anything important. For an XML parser, compare a plain literal field with one external entity referencing the controlled HTTP canary from the SSRF module. Correlate callback logs and returned marker. Do not use recursive entities, oversized documents, or operating-system files. XML acceptance or a DOCTYPE error alone is not external-entity resolution. XXE.
- Interpret downstream effects. Wait for the single test job to finish. Compare bytes, filesystem location, ownership, browser origin, and any parser callback. Source review should locate the exact join/extraction/parse operation and whether containment checks occur after normalization. Delete originals, derivatives, exports, and queued test jobs.
Command card
This creates a local download fixture. Configure the replica's allowed download directory as file-lab/public; illustrative endpoint and parameter names must match the application.
mkdir -p file-lab/public evidence/files
printf '%s\n' 'BB-INSIDE-01' > file-lab/public/inside.txt
printf '%s\n' 'BB-OUTSIDE-01' > file-lab/outside.txt
curl -q --noproxy '*' --max-time 10 --get --header @A.header \
--data-urlencode 'name=inside.txt' \
-o evidence/files/baseline.txt \
http://127.0.0.1:8000/api/download
curl -q --noproxy '*' --max-time 10 --get --header @A.header \
--data-urlencode 'name=../outside.txt' \
-o evidence/files/traversal.txt \
http://127.0.0.1:8000/api/download
Report threshold and checklist
Report a proven outside-root read/write, unauthorized controlled-file access, unsafe browser/server interpretation, or prohibited parser fetch. A reflected filename, harmless stored extension, public sharing by design, and asynchronous rejection are frequent false positives.
- Upload and retrieval rules documented independently.
- Only tiny, synthetic files and dedicated canaries used.
- Filename, MIME, contents, and ownership varied separately.
- Raw request and resolved destination retained.
- Processed derivatives included in evidence and cleanup.
Local curl examples bypass environment proxies with --noproxy '*'; -q alone only suppresses curl configuration files. These snippets target loopback and do not follow redirects. curl manual.
Containment beyond dot segments
For a link-capable filesystem or archive format, test a tiny symbolic-link fixture in the disposable directory as a separate case. Point it only to the already-planted harmless outside canary, then exercise the product's real reader/extractor. Record the final resolved destination and the identity that opened it. A lexical .. rejection does not establish containment when links, absolute member paths, platform separators, or case collisions change the destination. Do not generalize a local administrator-created link into a remote vulnerability: show how the stated attacker can create or influence that link through the product.
Pin the runtime and actual extraction API/filter. Python's tar extraction filters can reject outside destinations and links, but filter availability, explicit overrides, and additional resource limits matter. A failed extraction may also leave earlier members on disk; inspect and remove only the disposable output. If a TOCTOU claim depends on replacing a path between validation and opening, demonstrate that timing separately in the replica; a static link test does not prove a race. Python extraction filters and limitations.
- Final resolved file or link target stays inside the intended root.
- Actual runtime/filter and partial-extraction cleanup documented.
REST, GraphQL, gRPC, WebSocket, and SSE permission parityCarry the same permission invariant across protocols, nested fields, streams, and alternate API versions.
Open canonical module and reverse references
- 01Inventory observed operations
- 02Choose one permission invariant
- 03Compare controlled identities
- 04Test alternate protocol path
- 05Check stream lifecycle
- 06Record semantic result
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Start from normal client traffic and supplied API schemas. Create two controlled accounts and private synthetic objects. Map REST paths, GraphQL operations/fields, gRPC methods, and WebSocket message types to the same business action. The invariant is: changing transport, API version, batching, or connection lifetime does not widen an actor's permission. Reuse the authorization matrix and session modules; protocol-specific testing adds delivery details rather than duplicating those foundations.
Numbered test procedure
Build a semantic inventory. Record operation name, method/path, actor, tenant selector, object ID, request schema, response fields, and side effects. Observe one success and one forbidden controlled-object request per available protocol. Preserve response bodies, GraphQL errors, gRPC status/trailers, or socket messages; HTTP status alone does not define application success.
Check REST parity. Replay the same controlled-object rule through observed v1/v2 routes, nested resources, exports, and a two-item batch. Compare read, update, and server-controlled properties independently. An alternate route being live is not a vulnerability unless its behavior violates the intended permission or lifecycle rule.
Check GraphQL resolvers and fields. Begin with the client's existing query. As B, change only A's known object ID, then request one protected field omitted from B's normal response. Try the observed global-node route if present. Use at most two aliases as a bounded comparison of resolver behavior; do not use recursive depth or massive aliases. A schema or introspection response helps discovery but is not a confidentiality breach by itself. Where cookie authentication and alternate HTTP methods exist, check a reversible mutation through the observed transport: GraphQL GET is for queries, not mutations. If claiming CSRF, prove delivery in a real browser using the browser-trust procedure. Persisted query IDs are operation selectors, not permission proofs. GraphQL HTTP semantics. GraphQL testing.
Check gRPC context. Use the provided
.protofile or authorized reflection to identify one method. Compare channel identity, per-call credentials, tenant metadata, and the selected resource. Repeat a forbidden request with B's genuine credentials, not a token fabricated from decoded claims. For streaming methods, check whether a permitted start improperly authorizes later messages for another controlled resource. Channel credentials and call credentials serve different roles. If an HTTP/JSON transcoding gateway or gRPC-Web route is actually deployed, replay this same invariant there: a translated request can travel through different authentication middleware. Preserve the gateway path and backend method instead of inferring exposure from a.protoannotation. HTTP/JSON transcoding. gRPC authentication.Check WebSocket and SSE lifecycle. Capture a normal handshake and one subscription message. In B's profile, request A's private room using its known ID; have A send a single harmless marker. Remove B from a previously shared room and test one further message on the existing socket. For cross-site hijacking, use a real controlled origin and actual browser cookies. An accepted forged Origin header from a nonbrowser client does not prove browser delivery. OWASP WebSocket security. For server-sent events, use the observed event endpoint with A and B, then remove B and inspect one new synthetic event. A reconnect cursor or
Last-Event-IDselects delivery position; test that it cannot select another controlled tenant's history. Close the browser EventSource during cleanup because it normally reconnects automatically. SSE connection and reconnection.Interpret and consolidate. Read partial data as well as error arrays; an error on one field can coexist with leaked data elsewhere. Correlate socket subscription IDs and asynchronous messages. Group instances by shared root cause only when the evidence supports it, and name untested transports explicitly. Close sockets and remove test resources.
Command card
Prepare protected A.header and B.header files as described in the authorization module. The example assumes the local schema exposes document(id:). Replace it with the already observed query, not an invented production operation. The second request changes only identity.
mkdir -p evidence/protocols
cat > document-query.json <<'JSON'
{"query":"query($id:ID!){document(id:$id){id title}}","variables":{"id":"O1"}}
JSON
curl -q --noproxy '*' --max-time 10 --header @A.header \
-H 'Content-Type: application/json' --data @document-query.json \
-o evidence/protocols/a.json http://127.0.0.1:8000/graphql
curl -q --noproxy '*' --max-time 10 --header @B.header \
-H 'Content-Type: application/json' --data @document-query.json \
-o evidence/protocols/b.json http://127.0.0.1:8000/graphql
For a local socket, use the browser's Network → WS view to copy the observed message schema. Change one controlled room ID and send one message; protocol libraries often add framing, acknowledgements, or subscription IDs that a generic text frame would omit.
Report threshold and checklist
Report forbidden controlled data/action, a protected-field leak, or unauthorized ongoing delivery after the applicable revocation point. Reflection, introspection, websocket connectivity, and partial errors without forbidden output are discovery observations.
- Normal client operation and schema preserved.
- Same permission rule compared across available transports.
- Application-level errors and partial output inspected.
- Authentication at connection and message levels distinguished.
- Every stream closed and fixture cleaned up.
OSS lab: pin, install, prove, resetTurn an eligible release into a repeatable synthetic environment with preserved versions, realistic controls, and an explicit reset recipe.
Open canonical module and reverse references
- 01Qualify release
- 02Freeze inputs
- 03Review installation
- 04Build fixtures
- 05Compare controls
- 06Reset and repeat
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Start with a reproducibility contract#
Record repository, product edition, supported release, immutable commit, package locks, image digests, operating system, database, authentication mode, and enabled extensions. Keep tested, inferred, and unknown versions separate. A source tag does not identify the image running in a vendor deployment. Your initial output is a manifest plus a working baseline, before any vulnerability attempt.
Use a disposable VM for an unfamiliar project's build. Review its Dockerfiles, build backends, package lifecycle scripts, Makefiles, and downloaded installers first. Provide synthetic credentials and fixtures. Separate dependency preparation from validation; disable unnecessary external integration calls during validation. Preserve the installation instructions you followed and every deviation. This makes it possible to distinguish a supported insecure default from a flaw introduced by the research setup.
Freeze source without overwriting work#
These are adaptable local examples, not a claimed execution. Start with an existing reviewed clone at ./repo; create a fresh output directory. Substitute a real eligible release in the second command. A detached worktree allows a separate copy at that revision while sharing Git objects. Git worktrees.
mkdir -p ./evidence
git -C ./repo rev-parse --verify 'v1.2.3^{commit}'
git -C ./repo worktree add --detach ../lab-source v1.2.3
git -C ./lab-source rev-parse HEAD > ./evidence/source-commit.txt
git -C ./lab-source status --porcelain=v1 > ./evidence/source-status.txt
git -C ./lab-source submodule status --recursive > ./evidence/submodules.txt
Expected: a full commit identifier and an initially clean status file. A submodule entry is inventory, not confirmation its source has been obtained or reviewed. Save compiler/runtime/tool versions separately. Never silently replace a failing eligible release with current main.
Review the effective deployment#
Compose configuration can interpolate environment values, so use a synthetic lab environment and inspect output before sharing. Render first; check host mounts, image references, ports, capabilities, external volumes, privileged mode, and startup scripts. A project name alone does not isolate explicitly named shared volumes. Docker Compose config.
docker compose -p bounty-lab -f ./lab-source/compose.yaml config \
> ./evidence/compose-effective.yaml
docker compose -p bounty-lab -f ./lab-source/compose.yaml \
up -d --wait --wait-timeout 120
Run up only after review and required dependency preparation. --wait depends on declared health checks for application readiness; a running process alone is insufficient. Establish login, ordinary creation, retrieval, and a worker task through the real interface. Docker Compose up.
| Fixture | Normal operation | Boundary case |
|---|---|---|
| Tenant A and B | Each reads its own document | A requests B's synthetic document |
| Member and administrator | Administrator manages invitations | Member attempts the same operation |
| Import directory | Valid sample stays inside destination | Malformed sample attempts a harmless boundary crossing |
| Background worker | Queued operation completes once | Membership changes before job execution |
Preserve the failing and passing stories#
Example trace: member A requests an export, the route returns a job ID, and the worker reads a document owned by B. Capture the request, job record, worker resolution, synthetic output, and authorized B control. A direct call to the export helper explains the implementation but does not establish route reachability. Continue in source data flow and reuse the authorization matrix.
Diagnose setup failures before testing: a migration error invalidates database assumptions; disabled workers invalidate asynchronous conclusions; missing enterprise features limit edition claims. Record unresolved differences instead of compensating by disabling security controls.
Reset and report gate#
Prefer restoring a dedicated VM snapshot or documented fixture/database snapshot. Include queues, object storage, cache, local mail, and generated files in the reset recipe. Container restart is not state reset. Repeat the minimal reproduction after restoration; a result dependent on an earlier experiment is not yet isolated.
- Eligible revision, dependency locks, and deployment differences recorded.
- Normal user workflow and comparison account work.
- Required worker/integration fixtures are present.
- Synthetic evidence survives outside reset storage.
- Clean restoration reproduces the same boundary failure.
- Report distinguishes product behavior from lab-only configuration.
Source review: entry point to protected operationTrace attacker control and authorization through handlers, services, queues, and sensitive operations; promote only reachable violations of a stated invariant.
Open canonical module and reverse references
- 01State invariant
- 02Find entry
- 03Trace transformations
- 04Check guards
- 05Exercise interface
- 06Retest sibling paths
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Write the invariant first#
Choose a business rule precise enough to falsify: “a member can export documents only from a project where membership is currently active.” Record actor, object, operation, tenant, and when authorization must hold. A suspicious function becomes useful when you identify who can reach it and what protected state it changes. Reuse the authorization matrix for principal/object combinations and injection tracing for interpreter-specific input handling.
Start forward at a real interface: HTTP route, websocket event, CLI argument, uploaded project, plugin callback, or queue producer. Then work backward from the sensitive read, write, process invocation, parser, or credential use. Join the two traces; unresolved dynamic dispatch is an explicit gap. Check route registration and feature gating rather than assuming a named handler is exposed.
A worked asynchronous trace#
The following is a hypothetical review worksheet, not a finding against a named project.
| Hop | Value or context | Question to resolve |
|---|---|---|
POST /exports |
Authenticated member A supplies document_id |
Does the route check document membership or only login? |
ExportService.create |
A job stores document_id, requested_by |
Is the trusted tenant copied from the session or accepted from input? |
| Queue serialization | Job survives after request completes | Can another low-trust producer alter the fields? |
run_export(job) |
Worker reloads document by ID | Is current membership enforced at execution time? |
| Object storage write | Export receives a download reference | Who can fetch it, and can authorization expire? |
A useful candidate is “worker rehydrates an object globally after a tenant-scoped request.” Confirm whether the earlier guard binds the same immutable identifier, whether membership revocation should affect queued jobs, and whether the output is actually accessible. Business requirements may deliberately snapshot permissions; document that distinction rather than treating every delayed check as defective.
Local search and modeling cheat sheet#
Examples inspect a reviewed local repository. Patterns are leads, not a complete inventory. Place evidence outside the scan root; prepare local rules before offline/private-code analysis.
rg -n --hidden -g '!.git' \
'route\(|router\.|websocket|add_task|enqueue|\.delay\(' ./repo
rg -n --hidden -g '!.git' \
'pickle\.loads|yaml\.load|subprocess\.|child_process|exec\(' ./repo
semgrep scan --oss-only --config ./rules --metrics=off \
--disable-version-check --json \
--output ./evidence/source-leads.json ./repo
The Semgrep command uses reviewed local rules and the OSS engine. Inspect stderr, skipped files, and scan completion; an empty result with parser failures is not a clean audit. Rule licenses and engine capabilities are separate concerns. Semgrep CLI reference.
Model sources, propagation, context-specific sanitizers, and sinks. A path validator does not sanitize a shell argument; a role check does not validate serialized data. Custom wrappers can hide a source or sink from default rules. Create one vulnerable fixture and one safe sibling to validate your model. Static taint analysis does not automatically establish tenant policy or full runtime reachability. Semgrep taint model.
Prove the boundary through supported use#
Instrument the local application to log identifiers, decisions, and function names without credentials. Reproduce from the ordinary interface with synthetic accounts. Keep four observations together: allowed own-object request; denied or expected-to-deny foreign-object request; actual protected effect; resulting output access. Repeat without instrumentation changing security behavior. If only an internal helper is callable, qualify the finding as a library/caller-contract issue until a consumer path is established.
Inspect alternate callers after proving one flaw: bulk API, export service, websocket subscription, scheduled task, and administrative UI may reuse the same operation with different guards. Group equivalent paths by root cause and record tested variants. Do not inflate one shared defect into five independent findings.
Completion and report threshold#
- Invariant identifies actor, object, operation, and trust boundary.
- Attacker-controlled field is tracked through transformations and aliases.
- Every guard is evaluated with concrete fixture values.
- Feature, edition, role, and configuration prerequisites are explicit.
- Real interface reproduces the protected effect beside an authorized control.
- Safe sibling and patch retest preserve legitimate behavior.
Submit the smallest connected argument: entry point → attacker control → ineffective boundary → sensitive operation → demonstrated impact. A sink search, scanner trace, or direct helper call alone belongs in the hypothesis queue.
CI and releases: trace untrusted input into privileged workAudit workflow events, shell interpolation, untrusted builds, artifact consumers, and release identity as one privilege chain.
Open canonical module and reverse references
- 01Identify event trust
- 02Map executable input
- 03Inspect privileges
- 04Follow artifacts
- 05Validate locally
- 06Verify release identity
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Draw the actual privilege chain#
For every release-capable workflow, record event actor, workflow-file revision, checked-out revision, executable inputs, token permissions, reachable secrets, runner persistence, and final consumer. Distinguish data review from code execution: a checked-out contribution becomes executable through Makefiles, package install hooks, test discovery, plugins, and local actions as well as explicit shell scripts.
Use one row per job. Include reusable workflows and composite actions; apparent isolation at the top-level YAML may disappear inside their implementation. A permissive token setting without a lower-trust entry is a hardening observation, not a completed takeover claim.
| Boundary | Local evidence to collect | Comparison control |
|---|---|---|
| Metadata → generated shell | Original expression and resulting script | Environment variable passed as data |
| Fork content → privileged build | Exact fetched revision and executed file | Trusted base revision only |
| Test artifact → publisher | Producer identity, run association, consumed path | Artifact from expected trusted build |
| Build → attestation → release | Artifact digest and accepted signer identity | Wrong-source artifact rejected |
Triage commands and a safe coding pattern#
These examples inspect local files; they have not been run against a target. Grep-style matching deliberately overselects candidates.
rg -n --hidden -g '*.yml' -g '*.yaml' \
'pull_request_target|workflow_run|issue_comment|permissions:|id-token:' \
./repo/.github
rg -n --hidden -g '*.yml' -g '*.yaml' \
'download-artifact|upload-artifact|cache-mode|head\.sha|npm install|make ' \
./repo/.github
Passing untrusted metadata through an environment variable avoids inserting its contents into the generated shell program. Quote it and do not subsequently eval it. This pattern does not solve dangerous argument semantics in downstream tools. GitHub secure use.
- name: Inspect title as data
env:
PR_TITLE: ${{ github.event.pull_request.title }}
run: printf '%s\n' "$PR_TITLE"
Check current protections before declaring a pwn request#
GitHub currently documents fork-ref checkout protection for pull_request_target and read-only default cache access for that event. Inspect action revision, explicit allow-unsafe-pr-checkout, cache-mode, manual fetches, repository policy, and actual execution. The trigger name alone proves neither execution nor writable cache. Controls can differ from older published examples. Current pull_request_target guidance.
Hypothetical trace: an unprivileged test job uploads report.zip; a privileged follow-up selects an artifact by a generic name; an extracted helper is executed before publication. Validate run/repository binding, extraction destination, and consumer behavior in your own fixture with an inert marker and dummy credentials. A successful workflow conclusion does not authenticate every byte it produced. An artifact parsed strictly as data may break the supposed execution chain. GitHub Security Lab workflow analysis.
Follow identity to the shipped artifact#
For a known public artifact, this read-only verification example contacts GitHub. Replace repository, workflow, and commit with expected identities established independently; do not copy identities out of the untrusted artifact and then “verify” against them.
gh attestation verify ./dist/product.tar.gz \
--repo OWNER/REPO \
--signer-workflow OWNER/REPO/.github/workflows/release.yml \
--source-digest EXPECTED_COMMIT \
--format json > ./evidence/attestation.json
A reusable workflow may be the signer rather than the calling workflow. Verified provenance supports an origin claim; it does not prove the builder was uncompromised or the software is vulnerability-free. GitHub CLI verification.
Trusted publishing reduces long-lived publish-token exposure, but a compromised authorized workflow can still obtain publishing authority. Review environment approval, workflow identity, build separation, and remaining legacy credentials. PyPI trusted publisher security model.
Report and retest gate#
- A permitted low-trust actor controls the demonstrated input.
- Exact workflow/action revisions and security defaults are captured.
- Inert local reproduction reaches the privileged consumer.
- Real permissions are supported by configuration, not guessed from a trigger.
- Artifact run binding and signer/source identity are checked.
- Patched data handling rejects the case while normal releases still work.
Report the reachable capability and its prerequisites. Do not run target publishing steps or use discovered tokens to prove a local workflow hypothesis; maintainers can confirm live permissions through a coordinated reproduction when needed.
For GitLab repositories, also use GitLab pipeline boundaries: parent-project merge-request execution, job-token permissions and runner isolation have distinct semantics.
Dependencies: from advisory match to reachable impactSeparate composition, advisory matching, runtime reachability, and exploit prerequisites so dependency reports describe product impact.
Open canonical module and reverse references
- 01Inventory layers
- 02Resolve advisory
- 03Trace consumer
- 04Check prerequisites
- 05Compare fixed behavior
- 06Bound affected versions
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Keep four questions separate#
An SBOM answers what a tool identified in an artifact. An advisory matcher asks whether a component version matches known vulnerability data. Reachability asks whether the relevant implementation is used. Exploitability asks whether an attacker can satisfy the conditions and cross a security boundary. Preserve each conclusion independently; a negative result at one layer does not answer every other layer.
Inventory source locks, installed runtime, development/build dependencies, and final release output separately. A vulnerable parser in a lockfile may be absent from the production image. A development-only package may still run inside a privileged publisher. Record ecosystem, package identity, resolved version, origin, dependency path, optional features, and artifact digest. Use CI and release analysis for build impact and source data flow for runtime impact.
Inventory and match locally#
Examples require installed tools, reviewed inherited configuration, existing local paths, and prepared OSV ecosystem databases. They are command templates, not claimed results. ./release-root means an already extracted, reviewed release tree.
mkdir -p ./evidence
syft dir:./repo -o cyclonedx-json=./evidence/source.cdx.json
syft dir:./release-root -o cyclonedx-json=./evidence/release.cdx.json
osv-scanner scan source --offline -r --format json \
./repo > ./evidence/osv-source.json
Syft supports directory inventory and structured SBOM output; detected composition depends on catalogers and available metadata. Compare the two inventories rather than assuming they represent the same execution environment. Syft documentation.
OSV source scanning uses supported manifests and lockfiles. Full --offline requires cached databases and avoids network activity; --offline-vulnerabilities has a narrower meaning and can permit other network features. Missing caches are an incomplete scan. Preserve database age and scanner stderr alongside findings. OSV source scanning, offline behavior.
Build a reachability worksheet for each serious candidate#
| Question | Concrete evidence | If unresolved |
|---|---|---|
| Is the resolved component affected? | Exact advisory range, vendored source, backport patch | Version match only |
| Is affected code present? | Release tree, build flags, linked module | Source-only candidate |
| Is the vulnerable API called? | Consumer call site and dynamic trace | Reachability unknown |
| Can an attacker control its argument? | Ordinary route/file/CLI input and transformations | No attacker path established |
| Are prerequisites satisfied? | Parser mode, operating system, feature, privilege | Conditional exposure |
| What changes across the boundary? | Synthetic output or bounded local fault | Impact unproven |
Hypothetical example: a document service ships an affected archive library, but the advisory concerns symbolic-link extraction. Trace upload acceptance → import job → actual extraction API → filter configuration → destination. A safe filter can defeat that specific path even when the version matcher is correct. Conversely, an optional compatibility flag may restore the unsafe behavior. Test supported configurations explicitly and reuse the pinned lab.
Check package origin as well as version#
For dependency confusion, first inspect registry selection, namespace ownership, fallback, lockfiles, integrity hashes, and install hooks. pip documents that candidates from configured locations are considered together, rather than treating an extra index as a guaranteed lower-priority fallback. pip install index behavior.
Reproduce with two local registries and synthetic names that belong to your lab. Use packages containing metadata and harmless marker files; record which origin wins under the target's resolver configuration. Do not publish a real organization's missing package name. Resolver selection is one proof step; a product-impact claim must also show an affected build/consumer and applicable package controls.
Control, report, and completion#
Run the same smallest input against the affected and patched dependency in equivalent consumers. If swapping versions also changes application behavior or configuration, isolate that difference before crediting the patch. Test normal input to ensure the “fix” is not simply a broken integration. Avoid claiming every historical release is affected from one sampled version.
- Source, runtime, build, and release inventories are labeled.
- Advisory and patch are checked for the exact resolved component.
- Backports, vendoring, and optional features are considered.
- Attacker input reaches the affected API through supported use.
- Local impact and a fixed/control case are preserved.
- Tested versions, inferred exposure, and unknowns are separated.
The reporting threshold is a reproducible vulnerable use or the program's explicitly accepted dependency criterion. An old version number alone should remain a triage lead.
Libraries and parsers: bounded fuzzing to a minimal reproducerBuild a realistic local harness, instrument the target, bound fuzzing resources, and turn unique faults into defensible library or consumer reports.
Open canonical module and reverse references
- 01Choose contract
- 02Build harness
- 03Instrument target
- 04Fuzz within bounds
- 05Minimize root cause
- 06Verify consumer impact
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Choose an attacker-relevant parser contract#
Select a public API and a realistic input boundary: imported image, archive, document, packet, model file, project metadata, or CLI input file. Record who supplies bytes, the consumer's preprocessing, format limits, parser options, and the privilege of the process. Fuzzing an internal helper with impossible arguments may reveal a harness defect rather than a violated library contract.
Keep the first harness narrow: one parse operation, deterministic configuration, fresh state for each input, bounded allocations, and cleanup on both success and failure. Remove network and uncontrolled filesystem side effects. Seed with a few valid, small representative samples to reach deeper code; an all-random corpus may spend every iteration failing the magic-header check.
Harness and build cheat sheet#
The following is a template: product_parser.h, ProductDocument, and the parser functions are placeholders to replace with the target's documented API. Return/error ownership must follow that API. The commands have not been executed here.
#include <cstddef>
#include <cstdint>
#include "product_parser.h"
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
if (size > 65536) return 0;
ProductDocument* doc = product_parse(data, size);
if (doc != nullptr) product_document_free(doc);
return 0;
}
Build the target library itself with compatible sanitizer and coverage instrumentation; instrumenting only the harness misses much of the parser's behavior. Replace the source list with its reviewed build recipe. The example assumes Clang and a compatible C++ project. AddressSanitizer build guidance.
clang++ -g -O1 -fno-omit-frame-pointer \
-fsanitize=fuzzer,address,undefined -fno-sanitize-recover=undefined \
./fuzz_parser.cc ./parser.cc -o ./fuzz_parser
mkdir -p ./corpus ./artifacts ./evidence
./fuzz_parser ./corpus -max_total_time=120 -timeout=2 \
-rss_limit_mb=512 -max_len=65536 -artifact_prefix=./artifacts/
Add reviewed seed files before running. libFuzzer's time, input-length, timeout, and RSS settings bound the campaign; they are not a VM resource boundary. Run inside your local disposable environment with external memory/CPU limits appropriate for the workload. Track new coverage and distinct root causes rather than raw executions. libFuzzer reference.
Interpret the failure precisely#
| Observation | Immediate question | Defensible first claim |
|---|---|---|
| ASan heap read overflow | Does the public API permit this byte sequence? | Out-of-bounds read in the tested build |
| ASan write overflow | Which length check and allocation disagree? | Out-of-bounds write; execution impact unproven |
| Use-after-free | Did the harness violate ownership rules? | Lifetime defect if public usage is valid |
| UBSan arithmetic finding | Does overflow reach an unsafe allocation/index? | Undefined operation plus traced consequence |
| Timeout or RSS limit | Is it reproducible at modest input sizes? | Bounded evidence of resource exhaustion |
-fno-sanitize-recover=undefined makes the selected UBSan failures terminate instead of allowing a fuzzing run to continue after a diagnostic, which helps preserve reproducible failing inputs. UBSan checks selected undefined behaviors, not every arithmetic error. Unsigned integer wrap is defined in C/C++ and is not part of the default undefined group; choose additional checks when the target's threat model requires them. UndefinedBehaviorSanitizer.
Minimize and verify the consumer#
Copy one candidate into ./artifacts/repro.bin, then minimize with a unique output path. Preserve the original input, toolchain, build flags, revision, and symbolized trace.
./fuzz_parser -timeout=2 -rss_limit_mb=512 ./artifacts/repro.bin
./fuzz_parser -minimize_crash=1 -max_total_time=60 -timeout=2 -rss_limit_mb=512 \
-exact_artifact_path=./artifacts/minimized.bin ./artifacts/repro.bin
Replay the minimized bytes through a supported consumer in the pinned OSS lab. If the consumer rejects them before calling the library, report the library contract separately and leave consumer exposure unproven. A sanitizer crash is not proof of remote code execution, and its absence in an optimized build does not erase a confirmed memory-safety defect.
Hypothetical root cause: a header count is multiplied into a narrow allocation size, then a wider loop trusts the original count. Preserve the values, allocation site, failing access, and missing bound. The fix should bind accepted count to available input and checked allocation size; merely suppressing the crash is insufficient.
- Public API and ownership contract respected.
- Target code, not only harness, instrumented.
- Resources, seeds, revision, and toolchain recorded.
- Minimal input reproduces the same root cause.
- Normal input succeeds in affected and fixed builds.
- Library impact and real consumer exposure are stated separately.
Replay timeout and memory artifacts under the same resource limits as discovery. A minimized crash must retain the original root cause; a shorter input that fails differently is not equivalent evidence.
AI tools: input trust, execution identity and approvalsProve when lower-trust content changes a protected tool action, then locate the missing authorization or consent boundary.
Open canonical module and reverse references
- 01Pin legitimate task
- 02Map context to tool
- 03Run paired trials
- 04Check server decision
- 05Minimize evidence
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and security invariant#
Start with a local agent application, two researcher-owned identities, and stub tools that write only to an in-memory ledger or local JSONL file. Record the model identifier, sampling settings, application commit, available tools, connector scopes and legitimate user task. The invariant is: retrieved content can supply task data, but cannot grant permission, change the acting principal or silently expand an approved operation. Reuse the authorization matrix for tool objects and identity/session procedure for credentials.
Make a four-column trust map: attacker-editable source, transformation, tool argument, server-side decision. Include tool descriptions, tool results, document text, OCR and summaries. Follow the actual source path using source-to-sink review. A changed answer style demonstrates instruction influence; a security finding needs an unauthorized read, write, destination, privilege or other protected consequence. OWASP separates excessive tool functionality, permissions and autonomy. OWASP Excessive Agency
Controlled procedure#
- Record the baseline. Ask the owned victim agent to summarize one benign shared document. Save the exact prompt, document hash, tool calls and outbox state. Verify that a separately authorized mock write succeeds and an unapproved write is rejected; otherwise a silent tool failure could masquerade as protection.
- Change one lower-trust input. In a copy of the document, add a test instruction requesting a harmless change to the stub destination. Keep task, tools and identities constant. Run five fresh baseline trials and five fresh candidate trials with a fixed small token budget. Save every attempt, including failures; do not assume sampling controls make a hosted model deterministic.
- Inspect execution independently. Compare suggested calls, dispatched calls and committed stub events. Record which principal reached the backend. A model claiming success is insufficient. Repeat the same operation directly through the local tool adapter to determine whether the backend itself has an ordinary object-authorization defect.
- Bind approval to the action. Approve one harmless draft, then vary only its object ID, destination or content in the local fixture. The approved values and executed values must match, or the product must request renewed approval according to its contract. Compare canonical argument digests, not UI labels.
- Check the MCP boundary separately. With your own OAuth fixtures, compare a correct-resource token with a valid token for another resource. Exercise client A consent followed by a fresh client B request. MCP requires intended-audience validation and per-client consent in the documented proxy scenario; inherited upstream consent is not sufficient. Match all requests to the deployed protocol version. MCP authorization, MCP security guidance
Command and evidence cheat sheet#
Run these local inspection commands in the reviewed lab checkout. lab/tool-events.jsonl is your recorded adapter output, with one JSON object per event; these commands inspect evidence and do not invoke an agent.
rg -n 'approve|consent|tools/call|audience|tenant_id|workspace_id' ./src
python3 -I -S -m json.tool lab/case.json
python3 -I -S - <<'PY'
import json
from collections import Counter
with open('lab/tool-events.jsonl') as stream:
rows = [json.loads(line) for line in stream if line.strip()]
print(Counter((r['case'], r['decision']) for r in rows))
PY
Use event fields case, trial, principal, tool, argument_digest, decision and committed. Redact credentials. Python's JSON CLI validates the case file; it does not assess the security result. Python JSON documentation
Evidence, false positives and completion#
Preserve the legitimate task, attacker capability, differing document bytes, full relevant tool trace, principal and controlled side effect. Exclude hallucinated actions, canaries already present in the attack text, intentionally authorized tool use, and tests requiring nonexistent attacker editing rights.
- Baseline allowed operation and denied operation both work as expected.
- Every candidate has an unchanged benign comparison.
- Suggested, attempted and committed actions are distinguished.
- Approval arguments and execution identity are recorded.
- Successes/attempts and cleanup of local fixtures are included.
Continue across task lifetime#
An initial allowed or denied tool call does not cover later resumed work. Continue with AI state lifecycle for memory/checkpoint ownership, MRTR state binding, cancel/retry behavior and private caches. Apply resource budgets to tool fan-out, loop limits and per-principal cost without generating load on a live service. For model-output HTML, SQL or file effects, inspect the final consumer using browser trust, injection tracing and file boundaries; model output is input to those components.
RAG isolation: ingestion, retrieval, citations and revocationTrace document authorization through vector search, generated answers, citation downloads, caches and sharing changes.
Open canonical module and reverse references
- 01Seed private canaries
- 02Verify direct access
- 03Trace retrieval
- 04Change ownership state
- 05Compare exposed results
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and security invariant#
Use owned tenants A/B with private and shared collections. Place unique markers only in each tenant's private document. Record ingestion state, collection/document IDs and permissions. The invariant is: the effective caller can retrieve or resolve only currently authorized content through every supported path. Reuse the authorization matrix for object swaps and identity/session checks for tenant identity.
Unauthorized chunks may reach the model without appearing in its answer; filtered search can still return an unprotected citation download. OWASP identifies risks from shared vector stores and inadequate access controls. Test those boundaries separately. OWASP Vector and Embedding Weaknesses
Controlled procedure#
- Prove readiness. As A retrieve A's canary; as B retrieve B's. Then request B's document directly as A and verify denial. Record whether permissions belong to tenants, teams, individual users or externally synchronized groups. A canary missing from its owner's search is an ingestion problem, not isolation evidence.
- Compare retrieval paths. As A, issue the same semantic query used by B, with no secret marker in the query. Exercise search, chat, document lookup and citation download independently. In a local white-box fixture, log the authenticated principal, effective query filter, retrieved chunk IDs and resolver decision. Repeat with an absent tenant filter and a B collection ID only where the API actually accepts those fields.
- Inspect ingestion ownership. Upload a harmless A document while supplying B's collection or source reference. Confirm where the completed document is stored and who can read it. A rejected upload is a useful negative control; a successful job response alone does not show cross-tenant insertion. Follow async workers and document reindexing through source dataflow.
- Test sharing transitions. Share one synthetic document, retrieve it legitimately, then revoke access. After the product's documented synchronization interval, test a fresh conversation, an existing conversation, direct download, citations and cached answers. Record which content was already delivered before revocation; users retaining their earlier copy is not a new disclosure.
- Separate caches from filters. Warm a cache as B, repeat the same query as A, then repeat after an allowed lab cache clear or with a new synthetic document. Compare cold/warm results and cache keys. If only the warm path leaks, preserve that sequence. Test an A-only cache warm as the benign comparison.
Reusable case matrix#
| Case | Caller and state | Expected observation |
|---|---|---|
| Owner baseline | B → B-private document | Retrieval and resolver succeed |
| Tenant isolation | A → B-private document | No unauthorized chunk or download |
| Shared baseline | A → explicitly shared document | Allowed content is available |
| Revocation | A after share removal and sync | New protected retrieval is denied |
| Cache separation | B warms; A repeats | B's private result is absent |
Local command cheat sheet#
python3 -I -S - <<'PY'
import secrets
for owner in ('A', 'B'):
print(owner + '_PRIVATE_' + secrets.token_hex(16))
PY
rg -n 'tenant_id|collection_id|document_id|cache_key|namespace' ./src
python3 -I -S -m json.tool lab/retrieval-case.json
Store each generated marker only in its private fixture. Record caller, path, expected/actual document IDs, visibility and timestamp. These commands do not call hosted models or vector databases. Python secrets
Evidence and checklist#
Report the strongest observed boundary precisely: unauthorized chunk retrieval in logs, user-visible disclosure, or unprotected download. Do not present an internal trace as proof that an external attacker can see its contents. Exclude intentional shared collections, stale permissions within the stated synchronization window and markers included in the user's own query.
- Both owners can retrieve their own seeded documents.
- Search, answers and citation resolution were compared separately.
- Cold/warm cache and sharing-state controls are recorded.
- Actual caller identity and document ownership support the claim.
- Synthetic documents and temporary sharing grants were removed.
Beyond document retrieval#
Run AI state lifecycle when a private chunk is copied into a conversation, checkpoint, durable memory, background job or protocol cache. Record the new object owner and retention rule at each copy. A fresh search filter cannot repair an earlier copy served through an unprotected history/export API. Reuse the revocation controls above and distinguish content already delivered under a valid grant from a newly exposed private artifact.
OAuth, OIDC, and SAML: transaction binding and token trustSeparate browser transaction binding, issuer trust, token purpose, and identity linking using controlled federation fixtures.
Open canonical module and reverse references
- 01Map issuers and clients
- 02Capture authorized flow
- 03Alter one binding
- 04Verify token semantics
- 05Check replay and linking
- 06Remove test grants
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and trust record
Use two controlled accounts, two registered lab clients when available, and a test identity provider. Record authorization server, client ID/type, exact redirect URI, grant, scopes, resource/audience, browser transaction, issuer/subject, and resulting local account. Pin deployed protocol profiles and libraries. Reuse the session lifecycle and authorization matrix; this procedure adds federation-specific decisions.
The invariant is: a valid credential for one transaction, client, issuer, or purpose must not silently authorize another. A decoded token, valid cryptographic signature, and successful login are three different observations.
Numbered procedure
- Establish browser binding. Capture a normal authorization-code login and a separate account-link operation. In the replica, deliver A's response into B's pending transaction and compare the resulting local account/link with untouched controls. Track which session initiated each transaction. Missing
statealone is not decisive when transaction-specific PKCE or another valid mechanism provides the required binding. Validate a registered redirect with exact matching, accounting for the native loopback-port exception. RFC 9700. - Check code and PKCE handling independently. On separate fresh authorization codes, compare the correct verifier, a wrong verifier, and a missing verifier. Then replay one already redeemed code. Where the flow supports PKCE, inspect whether removing the challenge can create a downgrade path. Keep client authentication, client ID, and redirect constant so rejection can be attributed correctly. A verifier is secret; a public challenge is not. Generate a new verifier for each real transaction. PKCE specification.
- Validate OIDC context. With the lab issuer, produce otherwise valid fixtures with one wrong issuer, client audience, expired time, or mismatched nonce when that flow requires it. Check profile-specific
azp, hash, and authentication-age rules rather than assuming every claim is universal. Local identity should use the issuer/subject pair; equal email claims do not establish that two identities may be merged. Compare UserInfo subject with the ID-token subject when UserInfo is used. OpenID Connect Core. - Test JWT trust and purpose. First show rejection of one invalid signature. Then use validly signed local fixtures for another resource or purpose: an ID token at an access-token endpoint, or an email-verification token at a session endpoint, where the implementation exposes that path. In source review, inspect algorithm allowlists, issuer-selected key material, key-ID lookup, and mutually exclusive validation rules. Do not turn a token-supplied key URL into a production callback experiment. A signature-valid but unauthorized token is a semantic trust failure, not necessarily broken cryptography. RFC 8725.
- Check refresh replay under the actual design. Record original and rotated refresh credentials separately. After one normal rotation, attempt the previous credential once from another local client and inspect both the returned token and a harmless resource read. If tokens are sender-constrained, test the relevant missing/wrong sender proof instead. Public-client refresh tokens require rotation or sender constraint; documented retry grace affects interpretation but does not erase demonstrated independent unauthorized access. OAuth refresh-token protection.
- Follow SAML from verified bytes to identity. In a local IdP/SP pair, compare a normal response with independently changed signed content, a wrongly targeted assertion, an expired assertion, and one replay. Trace the exact signed element from which the SP consumes identity and attributes; verifying an unrelated signed element is insufficient. For SP-initiated SSO check request correlation. For supported IdP-initiated SSO, missing request correlation is expected, but replay and recipient/trust checks still matter. Keep RelayState handling separate from assertion integrity. OWASP SAML processing.
Local command card
Prepare expected.header and other-resource.header using valid tokens minted by your lab issuer; each contains its complete Authorization: Bearer … line. The second token is intentionally valid only for a different resource under the issuer policy. Preserve the same subject and request where possible. The protected route is illustrative. These requests do not test a public provider.
mkdir -p evidence/federation
chmod 600 expected.header other-resource.header
curl -q --noproxy '*' --max-time 10 --header @expected.header \
-D evidence/federation/expected.h -o evidence/federation/expected.json \
http://127.0.0.1:8000/api/me
curl -q --noproxy '*' --max-time 10 --header @other-resource.header \
-D evidence/federation/wrong-resource.h -o evidence/federation/wrong-resource.json \
http://127.0.0.1:8000/api/me
A 401/403 is a useful control, but also inspect bodies and resulting identity. Successful use of the expected token establishes only the baseline. Token transport and audience selection must match the actual local deployment.
Evidence and checklist
Capture one invalid condition, accepted identity or capability, minimal synthetic response, and the intended trust configuration. Document any test signing keys you supplied: possession of the real IdP signing key already grants authority and is not a new bypass. A rejected code, intentional multi-audience policy, or unconsumed altered claim is not takeover evidence.
- Browser binding and token validity tested separately.
- Correct, wrong, omitted, and consumed proof cases kept distinct.
- Token purpose and issuer/audience validation separated from signature validation.
- SAML consumed identity traced to the validated signed element.
- Grants, sessions, test keys, and transient credentials removed after evidence capture.
Request values: duplicate parameters, parser conflicts and downstream queriesTrace the value a security layer validates through to the value the application or downstream service actually consumes.
Open canonical module and reverse references
- 01Map parser chain
- 02Freeze two object controls
- 03Vary one representation
- 04Trace authorized value
- 05Inspect consumed value
- 06Prove controlled impact
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Map the actual processing chain: gateway, framework parser, schema validator, authorization code, service client, and worker. Use two synthetic objects, A-readable O1 and B-only O2, in a local replica with per-layer logs. The invariant is: the value authorized is the same value the protected operation consumes. This procedure covers application values; HTTP message framing stays in the proxy/cache procedure.
Numbered test procedure
- Create a value ledger. Capture raw request bytes, content type, parsed field type, validated value, authorized object and final downstream request. Start with O1 allowed and O2 denied under A's identity. Disable response caching for this local comparison. List which locations are accepted for each field: path, query, JSON, form, or multipart. Do not infer framework-wide behavior from a single handler.
- Compare duplicate parameters. Submit the original scalar, the alternative scalar, then the two values together in each order. Follow the value through validation and consumption. First-value, last-value, list or rejection behavior is not a vulnerability alone. A finding needs a security decision on O1 followed by an operation on O2, or another demonstrated rule violation. Test one observed parameter rather than fuzzing every field. WSTG parameter pollution.
- Preserve duplicate JSON names. Construct raw JSON bytes instead of a language dictionary, which may discard duplicates before sending. Compare a valid string, a repeated name, and a schema-invalid type such as a two-element array when a scalar is required. Record whether the gateway validates the first occurrence while the application consumes the last. RFC 8259 recommends unique object names and describes divergent receiver behavior; it does not make every accepted duplicate a vulnerability. Avoid asserting that all compliant parsers must reject them. RFC 8259, section 4.
- Check location and normalization precedence. Where the real endpoint accepts more than one representation, compare a query field against the same JSON/form field and a path ID against a body ID. In separate trials, inspect one relevant percent-decoding or Unicode-normalization difference. Only send supported content types; prove which route and middleware ran. A different error response or equivalent spelling alone is not an authorization bypass.
- Inspect server-created requests. A single external scalar may be interpolated into an internal query string or path. In a local service stub, use a marker such as
BB01&bb_probe=BB02as the complete outer value and inspect whether it remains one downstream value or becomes an added parameter. Encode it for the outer request exactly once. If a delimiter appears downstream, use only source-confirmed synthetic fields to establish a changed policy decision; do not guess internal administrative operations. Contrast correct URL-builder encoding with string concatenation. Server-side parameter pollution. - Confirm the boundary. Replay the minimal case through the normal ingress path and compare the strict-schema/consistent-parser negative control. Save the values each layer used and A's resulting access to the controlled O2 marker. Separate a reachable production path from a test harness configured with deliberately contradictory parsers. Remove both fixtures.
Command card: duplicate-preserving local requests
Assume an observed local /api/object-preview endpoint accepts scalar object, with test identities in A.header. Configure O1 as A-readable and O2 as B-only. The route names are placeholders, not discovered endpoints. This makes four ordinary, sequential requests without redirect following or inherited proxies.
mkdir -p evidence/parameters
curl -q --noproxy '*' --max-time 10 --header @A.header \
-o evidence/parameters/allowed.txt \
'http://127.0.0.1:8000/api/object-preview?object=O1'
curl -q --noproxy '*' --max-time 10 --header @A.header \
-o evidence/parameters/denied.txt \
'http://127.0.0.1:8000/api/object-preview?object=O2'
curl -q --noproxy '*' --max-time 10 --header @A.header \
-o evidence/parameters/forward.txt \
'http://127.0.0.1:8000/api/object-preview?object=O1&object=O2'
curl -q --noproxy '*' --max-time 10 --header @A.header \
-o evidence/parameters/reverse.txt \
'http://127.0.0.1:8000/api/object-preview?object=O2&object=O1'
For a JSON variation, preserve the literal repeated-name body in a file and use --data-binary @file with the correct content type; do not reconstruct it using a dictionary or UI form. curl data handling.
Report threshold and checklist
The report must identify the disagreeing components, raw ambiguous input, protected value checked, different value consumed, and controlled impact. A gateway-only rejection, intentional multivalue API, simple parser preference, or a client that changed the payload is insufficient. Use CWE-436 for an established interpretation conflict; add the specific resulting access or injection weakness where supported.
- Allowed and forbidden single-value controls recorded.
- Raw duplicates survived the test client.
- Parsed types, precedence and downstream construction recorded.
- Exact authorization/consumption mismatch demonstrated.
- Fixed-parser or fixed-encoding negative control compared.
Webhooks and integration events: authenticity, tenant binding and replayVerify the event signature, provider-to-tenant binding and durable worker effect together across retries and out-of-order delivery.
Open canonical module and reverse references
- 01Capture genuine event
- 02Verify signed bytes
- 03Bind provider and tenant
- 04Check replay and order
- 05Inspect durable effect
- 06Revoke and clean up
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and invariant
Treat an integration event as a separate privileged entry point. Inventory the subscription, receiving route, signing/authentication scheme, provider account, installation, tenant mapping, accepted event types, queue and final consumer. Use a local replica, a provider sandbox or local provider stub, two controlled tenants, and a reversible nonfinancial action. The invariant is: only an authentic, correctly bound event may cause its permitted effect, once under the workflow's rules.
Keep inbound event authenticity separate from outbound destination checks and object authorization. A valid signature does not itself identify which local tenant may be changed.
Numbered test procedure
- Capture a genuine baseline. Generate one ordinary event through the controlled integration. Save exact body bytes, verification headers, event/object IDs, provider account and mode, local tenant, queue ID and final state. Write down the expected change, such as granting one sandbox entitlement. Capture a correctly signed irrelevant event as a no-effect control. An HTTP
2xxcan acknowledge receipt before the worker runs. - Test authenticity before side effects. In the replica, compare the baseline with a missing signature, an invalid signature, and a one-byte body change retaining the original signature. Restore the local object, queue and deduplication fixture between authenticity trials: an already-consumed event can conceal a missing verifier. Follow both receiver and worker logs; rejection after a privileged item was accepted is too late. Respect the actual verification scheme. GitHub's documented HMAC covers payload bytes, so JSON reserialization changes the signed bytes; separately supplied event/delivery headers are not automatically covered. Prefer the official verifier and confirm failures stop processing. GitHub verification.
- Bind provider identity to the local object. Use valid sandbox events from two controlled provider accounts/installations. Verify each maps through trusted configuration to its correct tenant and provider object. For a local mutation experiment, change one untrusted routing field or unsigned header; a signed-field mutation requires an appropriately signed stub event and is a source-test hypothesis until its real attacker path is established. Check test/live mode and disconnected integrations. Do not claim a production forgery merely because the tester controls the lab signing key.
- Separate freshness, duplicates and order. Replay one valid event once and inspect durable effects. If the provider signs a timestamp, check the documented acceptance window using a local clock/fixture. A fresh signature is not a deduplication guarantee. Deliver two distinct legitimate events in reverse order in the sandbox and verify that an old update cannot undo a later revocation. Stripe documents duplicate delivery and does not guarantee event order; event creation seconds are not a safe total ordering or deduplication key. Use event identity and the product's object-state rules. Stripe webhook delivery behavior.
- Check dispatch and lifecycle. Compare the allowed event type/action with a genuine unsupported type and malformed-but-authentic local fixture. Ensure a loosely selected handler does not treat every event as success. Rotate a sandbox signing secret, record the intentional overlap, then check the old secret after that overlap ends. Remove the integration and test the documented handling of already queued events. GitHub recommends checking event type/action and keeps the delivery ID unchanged on redelivery. Treat any deduplication based on an unsigned header according to the actual authenticated binding. GitHub best practices.
- Inspect the transaction boundary. In the replica only, introduce one controlled worker failure before or after the effect, then retry. Check whether deduplication and the protected update can disagree, causing lost processing or duplicate effects. Use the existing state/replay procedure for transactional controls. Restore fixtures, cancel queued test deliveries, and remove local capture files when evidence has been redacted.
Command card: signed-body tamper control
Prepare event-valid.json and event-valid.headers from a newly generated local/sandbox delivery. The headers file contains only required verification and event metadata, not copied Host, Content-Length or production cookies. Prepare event-tampered.json by changing one known marker byte without re-signing. Start each authenticity case from the same clean local snapshot, including queue and deduplication state. Use a nonfinancial event and remain inside any signature freshness window.
mkdir -p evidence/webhooks
curl -q --noproxy '*' --max-time 10 \
--header @event-valid.headers -H 'Content-Type: application/json' \
--data-binary @event-valid.json \
-D evidence/webhooks/valid.h -o evidence/webhooks/valid.txt \
http://127.0.0.1:8000/webhooks/lab
# Inspect the valid effect, then restore the local fixture before this case.
curl -q --noproxy '*' --max-time 10 \
--header @event-valid.headers -H 'Content-Type: application/json' \
--data-binary @event-tampered.json \
-D evidence/webhooks/tampered.h -o evidence/webhooks/tampered.txt \
http://127.0.0.1:8000/webhooks/lab
For the separate replay experiment, restore once, send the valid request, wait for completion, and repeat it once without resetting. Inspect the durable effect count; two acknowledgements can correctly refer to one effect. The endpoint is illustrative. --data-binary preserves file bytes; these requests do not follow redirects. curl manual.
Report threshold and checklist
Report a forged protected transition, authentic event applied to the wrong controlled tenant, or replay/order failure with a durable security consequence. Acceptance of an intentionally unsigned public notification, permitted secret-rotation overlap, a harmless no-op acknowledgement, or a mock-only signing capability is not enough.
- Exact authenticated bytes and unsigned metadata distinguished.
- Provider identity, tenant, object and mode binding verified.
- Invalid/tampered controls caused no privileged worker effect.
- Freshness, duplicate delivery and order tested separately.
- Final state, retry behavior and cleanup recorded.
AI state: memory, resumed work, retries and private cachesTest persistent context, task ownership, MCP request-state integrity, duplicate effects and caches across identities and time.
Open canonical module and reverse references
- 01Seed owned state
- 02Verify owner controls
- 03Change principal or lifetime
- 04Resume or retry once
- 05Inspect committed effects
- 06Clear fixtures
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Setup and security invariant#
Use a local agent/backend with two owned principals A/B, a persistent test database, a deterministic mock model and a stub tool that only appends a marked event. Record agent/framework version, protocol version, connector identity and task contract. Keep budgets to one operation per case and at most two delivery attempts. Invariant: stored context, delayed work and retries preserve the authorized principal, operation and permitted lifetime. Use tool approvals for initial consent, RAG isolation for documents and resource budgets for cost bounds.
Operational procedure#
Inventory persistent objects. List conversation, checkpoint, durable memory, task/job, result artifact and connector credential separately, with owner and authorized readers. Thread history and cross-thread memory can live in different stores; LangGraph explicitly distinguishes these persistence systems. Seed a random private B marker, absent from A's prompts and fixtures. Confirm both owners can read their own state and A cannot directly read B's. LangGraph persistence.
Exercise state handles below the model. Compare A's own conversation/checkpoint with a known B-owned handle using A's same authenticated request. Follow read, resume, export and delete independently; successful queue admission is not proof that work completed. Repeat a resume after removing membership and after switching the active account, observing the worker's effective principal. A scheduler's service account must not erase the narrower authority of the initiating job. Report a stored-object authorization defect independently of any prompt injection.
Track memory provenance. Store one explicitly approved harmless preference as a positive control. Introduce an alternate preference only through a lower-trust retrieved fixture. Start a fresh task and inspect whether that content was promoted into trusted instructions or another tenant's memory. Use a stub destination change as the protected effect. A legitimate user-approved memory or changed answer style is not evidence of unauthorized persistent control. Record the original source and each summary/handoff, then remove the memory and confirm the effect stops. This is a proposed experiment for the product's persistence contract, not a claim that every memory feature is vulnerable.
Bind resumed MCP requests. For implementations using 2026-07-28 MRTR, compare an untouched
requestStatewith one altered local copy, a B-owned state under A, an expired state and the same state with changed action arguments. Observe a harmless protected result. The specification requires integrity checks where state affects authorization or business logic; it recommends principal, expiry and originating-request binding. Single-use effects require server-side enforcement beyond a protected token. These controls apply to the server, even though a conforming client treats the state as opaque. MCP MRTR.Disconnect at a defined point. In the local adapter, commit one marked stub operation and simulate loss of its response, then retry once. Compare committed events with received responses. A fresh JSON-RPC ID identifies a request, not automatically a business-level idempotency key. Require at-most-once behavior only when the operation promises it; intentionally repeatable actions need a different invariant. Also cancel before commitment and verify the documented cancellation boundary. The HTTP protocol removes stream redelivery, so transport reconnect behavior must not be mistaken for proof of business deduplication. MCP HTTP transport.
Compare private caches. Warm a synthetic private resource as B, then request the same method/parameters as A. Compare cold and warm results and each authorization context. MCP private results cannot be shared across authorization contexts; MRTR retries carrying input/state are not cacheable. Record protocol hints and actual returned data, since hints alone do not enforce object access. An identical public tool list is expected. MCP caching.
Bounded command and ledger sheet#
rg -n 'thread_id|checkpoint_id|requestState|inputResponses|cacheScope|idempotency' ./src
python3 -I -S - <<'PY'
import collections, json
events = collections.Counter()
with open('lab/state-events.jsonl') as stream:
for number, line in enumerate(stream):
if number >= 1000:
raise SystemExit('Fixture exceeds 1000 event review limit')
row = json.loads(line)
if row.get('committed'):
events[(row['case'], row['principal'], row['operation_id'])] += 1
for key, count in sorted(events.items()):
print(*key, 'commits=', count)
PY
The ledger is your locally recorded fixture output, not a scanner result. Include case, principal, operation_id, committed, timestamp, authorization decision and state/argument digests. Never log usable credentials.
Evidence and cleanup#
- Owner, cross-owner, expired/revoked and valid-resume cases are recorded.
- Committed effects are counted independently of model text and responses.
- Memory source and receiving principal survive every transformation.
- Cold/warm cache evidence identifies the actual reader.
- Restore the test database, delete canaries/checkpoints and revoke fixture credentials.
Resource budgets and sensitive business flowsTrace request fan-out, retries, recipient limits and tenant quotas using tiny local limits, recording stubs and measured downstream effects.
Open canonical module and reverse references
- 01Name the protected resource
- 02Prepare recording stubs
- 03Establish limit controls
- 04Vary one budget dimension
- 05Count committed work
- 06Restore and retest
At each step: retain the control → record the observation → continue only while the next boundary is understood.
Identify the unit that must be bounded#
An HTTP request can fan out into database queries, recipient notifications, reservations, image conversions, token generation, or recurring background work. Name the scarce resource and the authority that should own its budget: actor, tenant, recipient, workflow, or global service. A request-per-second limit does not necessarily constrain those downstream units. OWASP distinguishes unrestricted resource consumption from excessive use of sensitive business workflows; both need a concrete product-specific impact. API4, API6.
Build a small cost map before a test. For each entry point record accepted input size, batch cardinality, query/page complexity, queued jobs, retries, output size, and billed provider calls. Include failed requests: validation after a notification or job enqueue can return an error after consuming the protected resource. Use the state and business logic module for one-time transitions and the API protocol module for per-operation behavior.
Controlled procedure#
- State the business rule. Specify the prohibited outcome, such as more than two notifications to one synthetic recipient per lab window, or a member reserving an object after their allocation is exhausted. A missing CAPTCHA or rate-limit header alone is not the outcome.
- Prepare a faithful local deployment. Replace email, SMS, payments and model providers with recording stubs. Use supported configuration to reduce a real limit to a small lab value when possible. Record the configuration difference. Do not introduce a new guard or a fabricated endpoint and claim it represents the original application.
- Establish accepted and rejected controls. Exercise a permitted request and one documented over-limit case. Inspect actual downstream events and final state alongside the response. If the protected limit does not exist, bound the observation through source analysis and an isolated fixture; do not seek a production outage.
- Change one dimension. Compare a single operation with a small batch; the same actor through two permitted entry points; an original event with a duplicate; or a queued operation before and after its reservation expires. Keep the recipient, tenant, and resource ownership under your control. Test concurrency only when required to distinguish a non-atomic counter, using a local two-worker harness.
- Measure work and isolation. Record request count, accepted operation count, provider-stub count, bytes, task retries, and the budget's key. Check whether tenant A can consume tenant B's allocation or cause B's valid request to be rejected. A shared quota can be intentional; use the stated product rule.
- Reset and minimize. Cancel pending jobs, release synthetic reservations, clear only the dedicated fixture's state, and replay the smallest sequence. Recheck that a legitimate operation still works after the counter expires or allocation is restored.
Local command card#
The following is an adaptable request recipe, not a runnable application. Prepare lab/baseline.json, lab/at-limit.json, and lab/over-limit.json using the product's real request schema; create a header file with a synthetic lab token. Replace /api/exports with the reviewed local endpoint. Confirm the application uses recording stubs and that all three cases fit the agreed tiny experiment before running it.
mkdir -p evidence
for lab_case in baseline at-limit over-limit; do
curl -q --noproxy '*' --proto '=http' \
--connect-timeout 2 --max-time 10 --max-filesize 65536 \
--request POST --header @lab/authorization.header \
--header 'Content-Type: application/json' \
--data-binary "@lab/${lab_case}.json" \
--dump-header "evidence/${lab_case}.headers" \
--output "evidence/${lab_case}.body" \
'http://127.0.0.1:8000/api/exports' || break
done
This sends at most three sequential requests and does not follow redirects. It does not reset state, enforce the server's budget, or prove a limit. Capture the application's worker and stub logs separately. A transport failure stops the loop and must be recorded as blocked; HTTP error responses remain available for interpretation. If each case requires a fresh counter, restore the fixture between cases instead of running the loop as written. curl options.
Evidence, false positives, and completion#
Keep the expected budget, configuration, identities, input sequence, counter key, downstream effects, and valid control together. Demonstrated local resource amplification may support an impact argument; projected production cost remains an extrapolation until deployment and pricing assumptions are established. A few accepted requests below an unknown threshold do not prove unlimited access. A slow response is not proof of denial of service, and standard product automation may be intended.
- The protected resource or business allocation and intended owner are explicit.
- Real integrations are stubbed; no uninvolved recipient, customer, or bill is affected.
- The accepted control and over-limit control use the same relevant configuration.
- Downstream work is counted even when the HTTP response indicates rejection.
- Batch size, retries, async processing, and identity/tenant budget keys are considered.
- The minimal observation and any cost extrapolation are clearly separated.
- Pending work and fixture state are restored, with a legitimate operation retested.
Vulnerability reference#
Broken object-level authorization (BOLA / IDOR) · Broken property authorization: private response fields · Authentication or MFA state bypass · Residual access after explicit session revocation · OAuth login or account-linking confusion · JWT signature, issuer, audience, or token-purpose validation failure · SQL injection · NoSQL operator or query injection · OS command injection · Server-side template injection (SSTI) · Reflected or stored cross-site scripting · Server-side request forgery (SSRF) · Unsafe upload interpretation or placement · Path traversal outside an allowed root · XML external entity injection (XXE) · Archive extraction path escape · GraphQL resolver or field authorization failure · WebSocket message or subscription authorization failure · Hard-coded product credentials · Dependency confusion and unintended package origin · Unsafe object deserialization · JavaScript prototype pollution · Out-of-bounds memory read · Out-of-bounds memory write · Integer overflow affecting allocation or bounds · Parser resource exhaustion and decompression amplification · Regular-expression denial of service · Untrusted executable, library, or module search path · Indirect prompt injection into protected tool actions · Agent tool accepts unauthorized object or tenant parameters · Agent approval is not bound to executed arguments · MCP proxy consent confuses different clients · MCP accepts a token for another resource · RAG retrieval or citation resolver crosses tenant boundaries · RAG cache or fresh retrieval ignores revoked access · Exposed repository or history secret · Sensitive data in errors, debug routes, or generated files · Predictable security token or capability identifier · Argument or option injection without shell interpretation · Parameter or JSON parser disagreement crosses a security check · Server-side parameter pollution in a constructed request · Unverified or tampered webhook causes a protected effect · Authentic integration event applied to the wrong tenant or object · Agent conversation, checkpoint or memory crosses ownership boundaries · MCP resumed state changes authorization or request meaning · Private MCP result is reused across authorization contexts · Agent retry commits a protected single-use action twice · Untrusted model artifact reaches executable deserialization or custom code · Unbounded resource allocation or request amplification · Sensitive business flow exceeds its permitted frequency or allocation
Scope decomposition#
| Scope | Identity to pin | Boundary worth testing |
|---|---|---|
| Hosted inference/chat API | Endpoint, model identifier, project, API principal, version/date | Authentication, tenant objects, model access, quotas and output processing |
| RAG/document assistant | Tenant, collection/index, document IDs, source permissions | Ingestion ownership, retrieval filtering, citations, caches and revocation |
| Tool-using agent | Agent version, caller, available tools, execution identity | Instructions versus retrieved data, tool authorization and user approval |
| MCP implementation | Client/server package versions, protocol version, transport, OAuth issuer/resource | Token audience, per-request identity, tool/resource authorization and state ownership |
| Open weights/model artifact | Repository revision, weight digest, format, loader/runtime | Artifact provenance, parser behavior and loading-time code execution |
| Training/fine-tuning pipeline | Dataset/version, job/project, checkpoint destination | Poisoning, unauthorized training data and model/artifact publication |
These scopes are independent. A license to download weights does not authorize attacks on their hosting platform. An inference endpoint does not grant permission to extract unpublished weights or training data. A local loader finding may be an OSS vulnerability without affecting the hosted deployment; demonstrate the actual supported path.
Staged playbook#
1. Establish the product contract and deterministic baseline. Identify what the user intended, what the attacker can change and which tools the agent can invoke. Record model identifier, sampling parameters, system/application configuration where available, enabled connectors and versioned fixtures. Build two owned tenants and unique synthetic canaries. Set a small explicit trial count and request/token budget; a refusal screenshot is not a complete test outcome.
2. Map every entry point and sink. Entry points include chat, uploaded files, OCR/transcripts, repository text, retrieved web pages, vector results, tool descriptions and tool responses. Sinks include rendered HTML, SQL, shell arguments, network requests, document writes, permissions and external messages. Follow context through summarization and agent handoffs: security decisions must survive transformations, not just the initial prompt.
3. Separate behavior steering from an authority violation. Put a harmless instruction in one lower-trust fixture and have the owned victim account perform its legitimate task. First observe whether the instruction influences the answer. Then test whether that influence causes a restricted synthetic data read, changes a tool argument, or bypasses a required confirmation in a mocked action. An agent saying it sent a message proves nothing unless the tool/audit log shows the action. Excessive agency concerns the combination of tool functionality, permission and autonomy. OWASP Excessive Agency
Example proof design: tenant B owns a private document containing a random B_PRIVATE_... marker. Tenant A controls a shared document that the assistant is legitimately asked to summarize. If processing A's document causes B's assistant to retrieve B's private marker and expose it through an attacker-visible result, record that precise data boundary. Keep both identities under researcher control; use a stub outbox if the product action is a message. The marker must not appear in the attack text or baseline context, otherwise the experiment proves copying rather than disclosure.
4. Audit RAG as an authorization system. Trace tenant identity from the authenticated session into the query filter, document resolver and citation downloader. Compare direct document access with semantic retrieval; test renamed documents, deleted/revoked sources, shared-to-private transitions and cached answers. In local source review, look for optional client-supplied tenant filters and global fallback searches. Log retrieved document IDs and the final response independently: unauthorized retrieval remains important even when the final model declines to quote it, but externally observable impact must be stated accurately. OWASP Vector and Embedding Weaknesses
5. Test tools below the model. Repeat the controlled tool operation directly through its supported API with the same principal. Vary only owned object IDs, tenant IDs, destination accounts or a synthetic state handle. Verify server-side identity binding, schema validation, object ownership, canonical paths and approval state. A tool description saying “read only” does not implement authorization. Distinguish model-induced unauthorized behavior from a conventional backend authorization defect that the model merely exposes. MCP Tools
6. Audit MCP according to the deployed protocol era. As checked on 2026-09-11, the 2026-07-28 specification removes the initialization handshake and protocol sessions; requests carry version/capability metadata, and server/discover advertises capabilities. Earlier deployments through 2025-11-25 can still use initialize and session IDs. Apply session-binding tests to those implementations and explicit state-handle ownership tests to modern ones. Mixing protocol eras creates false positives. MCP changelog
For OAuth, separate client → MCP and MCP → upstream credentials. Check issuer, audience/resource, expiry, scope and per-request validation with your own tokens. A token for one resource must not become a credential for another merely because the proxy accepts it. Inspect per-client consent where an MCP proxy uses a shared upstream OAuth client; previous consent to one client must not silently authorize a different client. MCP Authorization, MCP security guidance
For HTTP transports, check invalid Origin rejection and whether header/body metadata disagree across a gateway and backend. Current Streamable HTTP requires version and method headers corresponding to the body, plus name/parameter headers where applicable. An authentication failure is not proof that Origin validation works; use an otherwise-valid request. In local stdio setups, inspect the executable/package provenance, inherited environment, allowed filesystem roots and process sandbox. Test OAuth metadata fetching with owned callback URLs, not actual cloud metadata or internal services. Streamable HTTP, MCP security guidance
7. Review weights and model-loading paths as software supply chains. Pin repository revision, model hash, loader package and all custom-code dependencies. Inspect trust_remote_code, unsafe deserialization, remote archive extraction, template parsing and plugin discovery. Hugging Face custom models can require executing repository-defined model code; that is an explicit trust decision, not automatically a library vulnerability. PyTorch's restricted weight loading reduces attack surface but is not a complete hostile-file sandbox. Begin with static inspection; perform malformed-artifact tests only in an isolated disposable runtime without credentials. Transformers custom models, PyTorch serialization
8. Minimize and quantify. Preserve full relevant inputs, retrieved content hashes, visible outputs and tool/audit events. Repeat fresh sessions with and without the candidate input, report successes/attempts and avoid presenting a stochastic result as universal. Keep the successful transcript, including the legitimate task; an attack string without the surrounding conditions is rarely reproducible. System-prompt disclosure should be assessed for exposed sensitive data or broken authorization, not presumed critical simply because instructions were revealed. OWASP System Prompt Leakage
Hypotheses, controlled tests and evidence#
| Hypothesis | Minimum controlled test | Evidence and interpretation |
|---|---|---|
| Retrieved instructions become tool authority | One attacker-owned document causes an owned victim's agent to invoke a mock restricted tool | Document hash, legitimate task, tool name/arguments, execution identity and stub audit event. An answer-style change alone is behavior evidence. |
| RAG applies tenant filtering only in the UI | Query A for a unique B-only marker through retrieval and citation paths | A token identity, B document ownership, retrieved IDs and final disclosure; include expected denied direct access. |
| Tool accepts caller-supplied ownership | Call the tool as A with B's synthetic object or workspace ID | Backend decision and canary result. Categorize underlying BOLA/IDOR independently of the LLM. |
| Agent approval binds to an obsolete action | In a local fixture, approve a harmless draft, then change its destination before execution | Approved argument digest versus executed argument digest and mock result. Do not send a real message. |
| MCP accepts a token for a different audience | Use a valid token issued to your own second resource against a harmless protected lab tool | Token claim summary, expected audience, request and unauthorized synthetic result; never publish token values. |
| State handles are bearer authorization | Supply your B-owned workflow handle while authenticated as A | Both authenticated identities, handle creation and B canary output. Legacy session behavior requires a version-specific test. |
| Gateway and MCP server disagree on routing identity | In a local fixture, vary one mirrored header against its body field | Request, gateway route, backend principal and result; mere protocol nonconformance is not automatically exploitable. |
| Model loading executes unintended code | Trace an externally reachable import path into an unsafe loader; reproduce using a benign local marker fixture | Digest, loader call, minimal input, isolated side effect and supported deployment path; do not load unknown artifacts on a credentialed workstation. |
Checklist#
- Hosted application, model, artifact, connector and provider scopes are distinguished.
- Attacker capability and the legitimate user's task are explicitly recorded.
- A/B tenants and private random canaries exist; canaries are absent from attacker input.
- Baseline requests establish expected permissions before adversarial input.
- Model version, runtime, sampling settings and trial budget are fixed and logged.
- Retrieval authorization, citations, caches and revocation are tested separately.
- Tool argument ownership and approval binding are checked below the model.
- MCP protocol era, transport, OAuth audience and upstream credentials are mapped.
- Model hashes, custom code and deserialization paths are reviewed in isolation.
- Success means an observable security consequence; hallucinated actions are excluded.
- Attempts/successes, evidence redaction and cleanup are documented.
Command cheat sheet#
These six commands are static/local inspection or protocol discovery against an owned fixture. Use a trusted Python 3.11+ interpreter for file_digest; isolated mode and disabled site initialization prevent imports from the working directory and site customizations. No model or repository code is executed by the first four. Local HTTP examples assume your intentionally unauthenticated loopback fixture; for an authenticated fixture, use its documented lab credentials. Inventory results do not establish tool-call authorization.
# 1. Find loader and execution sinks in a local source checkout.
rg -n 'trust_remote_code|torch\.load|pickle\.loads?|subprocess|shell=True' ./repo
# 2. Fingerprint one local weight artifact without loading the model.
python3 -I -S -c 'import hashlib; f=open("./model/model.safetensors","rb"); print(hashlib.file_digest(f,"sha256").hexdigest())'
# 3. Inspect JSON model configuration without importing custom model code.
python3 -I -S -m json.tool ./model/config.json
# 4. Generate a synthetic marker for your private test document.
python3 -I -S -c 'import secrets; print("B_PRIVATE_" + secrets.token_hex(16))'
# 5. Discover a local MCP server implementing the 2026-07-28 protocol.
curl -q --noproxy '*' --proto '=http' --silent --show-error --max-time 10 http://127.0.0.1:8000/mcp \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-H 'MCP-Protocol-Version: 2026-07-28' -H 'Mcp-Method: server/discover' \
--data '{"jsonrpc":"2.0","id":1,"method":"server/discover","params":{"_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{},"io.modelcontextprotocol/clientInfo":{"name":"bounty-lab","version":"1.0"}}}}'
# 6. List tool schemas, without calling a tool, on the same modern fixture.
curl -q --noproxy '*' --proto '=http' --silent --show-error --max-time 10 http://127.0.0.1:8000/mcp \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-H 'MCP-Protocol-Version: 2026-07-28' -H 'Mcp-Method: tools/list' \
--data '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{"_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{},"io.modelcontextprotocol/clientInfo":{"name":"bounty-lab","version":"1.0"}}}}'
References for command semantics: ripgrep guide, Python hashlib, Python JSON, Python secrets, MCP changelog, MCP HTTP binding, MCP tools. Syntax was checked against documentation; these examples were not run against a live target.
Close the persistence, retry and pipeline gaps#
Run AI state lifecycle after the initial prompting/tool tests. It covers thread/checkpoint ownership, memory provenance, delayed worker identity, MCP resumed-state controls, private caches and duplicate committed actions. A protected conversation read is not enough coverage for a separate resume or export endpoint.
| Pipeline handoff | Preserve and test |
|---|---|
| Upload → OCR/parser → index | Source owner, file validation and permission labels |
| Retrieval → summary → durable memory | Source provenance and ability to influence later authority |
| Human approval → background worker | Acting principal, argument binding, revocation and completion state |
| Model output → HTML/SQL/file/tool | Actual consumer validation and the final protected effect |
| Imported model → loader/runtime | Artifact digest, executable-code trust decision and isolation |
Reuse file boundaries, source dataflow, browser trust and injection tracing at those ordinary software boundaries. Use resource budgets for bounded cost and fan-out checks.
For MCP 2026-07-28, session removal does not remove application state: MRTR returns state that must be validated when it affects protected decisions. Private cache results require separate authorization contexts, and state-bearing retries have additional cache restrictions. Test the deployed implementation rather than transferring these requirements to older protocol versions. MRTR, caching.
Training/fine-tuning remains a distinct track: identify who may supply a dataset, start a job, select its base model and publish the resulting artifact. Test A/B job and artifact ownership before poisoning hypotheses. A measurable output change is not automatically a security violation; use the program's integrity/confidentiality contract and a separate untouched dataset control.
Sources#
All accessed 11 September 2026. Match living documentation to the target version.
| Publisher | Source | Publication or version |
|---|---|---|
| OWASP Foundation | OWASP AI Testing Guide | 2025-11-26 (v1 announcement) |
| OWASP GenAI Security Project | LLM Top 10 | 2025 edition |
| OWASP GenAI Security Project | LLM06:2025 Excessive Agency | 2025 edition |
| OWASP GenAI Security Project | LLM08:2025 Vector and Embedding Weaknesses | 2025 edition |
| OWASP GenAI Security Project | LLM07:2025 System Prompt Leakage | 2025 edition |
| Model Context Protocol | Tools | 2026-07-28 |
| Model Context Protocol | Key Changes | 2026-07-28 |
| Model Context Protocol | Authorization | 2026-07-28 |
| Model Context Protocol | Security Best Practices | 2026-07-28 revision |
| Model Context Protocol | Streamable HTTP | 2026-07-28 |
| Hugging Face | Customizing models — Transformers | Undated |
| PyTorch | Serialization semantics | Undated living documentation |
| Andrew Gallant / ripgrep contributors | ripgrep User Guide | Undated living repository |
| Python Software Foundation | hashlib — Secure hashes and message digests | Undated living documentation |
| Python Software Foundation | json — JSON encoder and decoder | Undated living documentation |
| Python Software Foundation | secrets — Generate secure random numbers for managing secrets | Undated living documentation |
| Python Software Foundation | Command-line options | Living documentation |