Understand the exposure.
Choose the right test.

Training can expose information through the gradients passed between model components, even when the original examples stay private. Gradient inversion investigates whether those updates reveal tokens or labels. It gives teams evaluating split training a way to test the privacy assumptions around the backward pass and to identify which observations an adversary would need.

The module provides several methods with different access requirements. DLG++ optimises candidate inputs against observed activations and gradients, using repeated initialisations and token refinement. Other methods explore analytic label recovery, sequence-smoothness priors, or token candidates scored against gradient directions. Depending on the method, the evaluator supplies a model, seam captures, public embeddings, paired activations or body/head gradients.

Results must be read at method level. DLG++ has a delegated model-driven path, and the direction-based method supports training captures; the analytic-label and prior methods retain synthetic-only paths where real-capture integration is absent. TAG/LAMP-style methods here are adaptations, not full reproductions of those attacks. Direction scoring measures information leakage and does not establish resilience against malicious participants in a complete training run.

When a model is split across private and external infrastructure, intermediate activations become part of the information shared with the provider. Seam inversion tests how much of the original token sequence can be reconstructed from that boundary. The result helps teams assess whether keeping raw text local meaningfully limits what an observer can recover.

The engine supports a calibrated activation decoder, optimisation-based reconstruction and joint methods that also use training gradients. Access to public embeddings, calibration examples or prefix weights depends on the selected method. Training and inference paths use their respective captures; inference does not imply access to gradients. Reconstruction measurements apply to the specified model, observations and budget, while each protected representation requires its own evaluation.

The features produced near a model’s output can retain information about the tokens being served. Output inversion examines this exposure by testing whether a decoder can recover tokens from output-side representations. This is relevant when a provider or intermediate service receives more information than the final text response alone.

The module offers a synthetic feature-decoding path and delegates model-driven work to an output-inversion script. Such evaluations require suitable output features and the model or paired training data needed by the decoder. This is a trained-decoder evaluation, not evidence that an ordinary text-only API exposes hidden states or a faithful reproduction of every sequence-inversion method.

A representation that resists a simple decoder may still reveal text to an attacker who trains a more capable inverter. This feature evaluates learned reconstruction from seam activations, giving privacy teams another way to challenge the information retained by a model boundary.

The approach trains an encoder-decoder using public text and a public model head, then refines candidate text through iterative correction. Model access, auxiliary training data and observation budgets form part of the assessment. It is a local adaptation of learned text inversion rather than a full external reproduction, and success on unprotected inputs does not establish effectiveness against a separately deployed defence.

Changing the coordinates of an internal representation does not necessarily remove the information it contains. Latent probing asks whether released features still identify tokens after rotations or permutations. It helps assess whether a transformation conceals only the representation’s appearance or also reduces recoverable information.

The module trains coordinate-based and rotation-invariant probes using labelled calibration data, then evaluates separate released rows. Norms and relationships between feature vectors provide alternatives to direct coordinate access. The trusted evaluation bundle supplies labels for scoring; those labels are not assumed to be available to a production observer. Conclusions depend on the bundle, transformation and calibration access tested.

Privacy controls need to be examined under partial exposure as well as ideal operating conditions. Latent sensitivity explores what changes when an observer captures more frames, identifies some decoys, gains access to untransformed features or learns a fraction of the underlying content. These comparisons help identify assumptions that deserve stronger operational protection.

The battery varies capture, decoy-identification, coordinate-access and known-plaintext conditions, comparing probe performance with the control for the same evaluation subset. It runs on labelled release bundles and models the selected compromise explicitly. A particular sweep does not establish a universal compromise threshold or demonstrate that decoys in another system can be identified.

Simple matching methods provide a useful baseline before investing in more expensive reconstruction. This feature tests whether released latent features can be associated with vocabulary items through reference centroids or signatures that do not depend on position. It can expose information that remains recognisable despite changes to feature layout.

The evaluator supplies labelled reference and evaluation bundles, and the module measures recovery on separate rows. Unseen evaluation labels count as misses rather than being excluded. These are local centroid and signature baselines: they neither reproduce the full published VMA algorithm nor recover the original sequence order, so their results should be interpreted as feature-matching evidence.

An observer may learn something sensitive without reconstructing a complete sentence. Label scoring tests whether the magnitude of backward gradients reveals a property of a training example. This helps teams assess exposure through a comparatively simple signal that can remain visible alongside more complex representations.

The module scores per-sequence gradient norms and evaluates their ability to distinguish supplied property labels, including ranking performance and true-positive rates at low false-positive rates. Labels are used to assess the inference, not handed to the observer as the answer. Results describe property leakage under the captured conditions and do not demonstrate token reconstruction or a malicious-participant training attack.

Training updates can combine information from several examples, but aggregation alone should not be assumed to hide their contents. Batch label reconstruction investigates recovery of tokens from first-step body-parameter gradients and gradients returned across the model boundary. It targets deployments where an observer has access to these richer training signals.

Using known body computation, the attack optimises dummy inputs and soft labels to match observed gradients. A model-and-capture path is complemented by a linear synthetic fixture. This grants stronger access than forward-activation observation and studies the first batch rather than an entire training trajectory. Findings therefore need to state both the observed gradients and the model computation available to the attacker.

Rotated feature coordinates may still preserve statistical structure that an observer can exploit without labelled examples. Blind source separation explores this possibility by attempting to disentangle mixed activation signals. It adds a different challenge to evaluations that otherwise rely mainly on supervised decoders.

The module uses FastICA to separate synthetic sources from their mixtures and assesses recovery in that controlled setting. Its assumptions concern the source distribution and the available observations, rather than unrestricted access to original text. A real-capture driver remains an integration gap, so the feature provides an algorithmic test bed rather than demonstrated recovery from arbitrary production activations.

Moving computation through a private middle segment introduces two observation boundaries. This feature asks whether access to both boundaries helps an observer reconstruct information that is harder to recover from the second boundary alone. It supports investigation of delegation designs in which multiple providers surround a private model component.

The synthetic study compares direct decoding at the second boundary, chained reconstruction and a first-boundary control. It grants known model maps and a middle-segment reference codebook, and uses an identity-middle control to check the machinery. Real dual-boundary capture is not yet supported by the capture format; the results therefore test a controlled topology rather than establish privacy of a deployed multi-provider model.

A privacy evaluation benefits from challenging a defence with more than one reconstruction strategy. This development feature combines a learned forward inverter with candidates suggested by backward gradients, then verifies candidate sequences through a public model head. It explores whether complementary observations improve reconstruction within a stated budget.

The learned decoder is trained on auxiliary public text kept separate from evaluation and pilot corpora. Training mode can combine forward and backward signals; inference mode uses forward observations only. This is a pilot-scale BiSR/SIP-style adaptation with explicit model-access assumptions, not a faithful reproduction of the full published training study or a result comparable to published BiSR figures without further work.

A representation may reveal little in one observation but become easier to recover as known examples accumulate. This feature measures that exposure within a single rotation-key period. It helps teams investigate how observation volume and key lifetime interact in a split-model privacy design.

The attacker collects known original/transformed activation pairs, estimates the rotation and evaluates token recovery. Capture-based and synthetic paths group observations by key period, with cross-period pooling as a control. This is a known-plaintext threat model: it does not assume an observer can obtain the required pairs for free, and the measured budget applies to the specific representation and rotation schedule.

Repeated system text or prompt scaffolding can give an observer a predictable reference across requests. Known-prefix testing examines whether that repetition supports recovery of a static representation transform. It makes the privacy implications of reusable templates and fixed prompt structure easier to investigate.

The synthetic experiment compares fixed scaffolds with padding and template-slot jitter under an attacker’s assumed alignment. It measures accumulation across a stated session budget. The jittered case only tests that alignment assumption; an attacker that actively realigns observations is evaluated separately by alignment-search. Padding variation should not be described as confidential decoy content or a general privacy guarantee.

Key renewal is useful only if old compromises do not automatically reveal current observations. Stale-key testing examines what an insider can recover after receiving earlier rotation keys. It helps distinguish protection against historical disclosure from protection against fresh information gathered during the current key period.

The synthetic attacker receives older keys and the stated derivation rule, then attempts current-period recovery with either no fresh pairs or an additional known-pair budget. The comparison measures whether stale knowledge gives a useful starting point. It tests the implemented schedule and compromise assumptions, not the security of every key-derivation design or an undisclosed master secret.

An exposed initialisation seed may let an insider recreate a model component that was expected to remain unknown. Seed-disclosure testing separates that disclosure from its practical privacy impact. This helps teams avoid treating either a leaked seed or an unchanged reconstruction score as a complete security assessment.

The module reconstructs an initial encoder through supplied hooks, checks it against an encoder digest and compares an inversion probe with a random-initialisation control. It can also consume supplied probe scores. Matching the initial encoder establishes the disclosure condition; increased recovery is a separate measurement. Neither result demonstrates a general bypass of representation protection.

Splitting a representation into independently transformed blocks changes the problem an observer must solve. This feature compares static sharding with staggered key rotation to investigate how block size and renewal timing affect reconstruction. It is useful when evaluating whether a proposed partitioning strategy actually limits accumulated knowledge.

The synthetic attacker solves blockwise transforms from known pairs and attempts to combine recovered blocks across an observation window. Static sharding is a control because smaller blocks can reduce the information needed for each solve. Results concern the chosen block-diagonal transform and schedule; they should not be generalised to arbitrary sharding, secret sharing or production key management.

A privacy assessment needs a consistent way to connect captured traffic with a clearly defined recovery measurement. Wire evaluation groups observations by key period and compares what can be decoded under different information conditions. It provides a common scoring path for split-training and inference captures.

The evaluator supplies wire tensors and aligned canonical references for supervised solves. Reports compare within-period recovery, decoding without a solved transform and pooling across key periods; training paths can also compare forward-only observations with forward-plus-backward observations. The canonical references are an explicit access assumption, and the presence of a capture file alone does not establish that an attack succeeded.

An attack can fail because its observations are misaligned rather than because the underlying information is protected. Alignment search tests a more persistent observer that tries to recover the correspondence between known prompt structure and captured model traffic. It challenges conclusions based only on passive alignment.

The method alternates between estimating the representation transform and using edit-distance or dynamic-time-warping alignment to identify matching positions. It then repeats the solve using the improved matches. Synthetic and capture paths require known scaffold content and suitable observations. The resulting comparison shows whether alignment work changes recovery under that scenario, rather than proving all padding or jitter ineffective.

Changing rotation keys may leave a shared structure in the underlying activations. Joint subspace analysis asks whether an observer can use that common structure to reduce the information needed within each key period. This gives teams a way to challenge assumptions that every renewal completely resets the reconstruction problem.

The synthetic method estimates a shared low-dimensional basis and solves period-specific transforms within it, comparing recovery with independent solves at the same known-pair budget. It requires canonical reference rows and assumes a suitably stable latent subspace. An advantage in this controlled setting is evidence about those assumptions, not a demonstrated reduction in privacy across all real model training runs.

A persistent insider can gather information over time even if each individual disclosure appears small. Leak accumulation examines partial key-row exposure across rotation periods, making cumulative compromise visible alongside per-period recovery. This is relevant when evaluating the consequences of repeated operational leaks.

The synthetic experiment gives the attacker a declared row-leak budget, reconstructs observed coordinates and tracks cumulative exposure. It distinguishes what can be decoded in a particular period from information collected throughout the campaign. Rotation is therefore assessed under an explicit continuing-compromise model; the module does not discover keys from a live service or establish an empirical breach probability.

Private information can be exposed through whether a document was used for training or whether it has a sensitive property. This feature evaluates those questions from captured boundary features, without requiring recovery of the full text. It broadens privacy assessment beyond exact token reconstruction.

A logistic probe is fitted using labelled reference examples and scored on separate documents. The workflow uses document-level features, multiple attacker seeds and document-level uncertainty analysis. Membership and property labels define the evaluation task, and a successful probe is conditional on that reference data and feature access. Its result is not evidence that an arbitrary external API reveals the same information.

The meaning of a model’s confidence depends on how comparable models behave on the same example. This feature uses reference statistics to investigate whether an example’s loss or confidence is more consistent with training membership than non-membership. It supports a more contextual assessment than relying on a single raw score.

The module computes LiRA-style likelihood ratios and RMIA scores from supplied target and reference statistics. It does not train the shadow models needed to produce those inputs. Reference quality, independence and calibration remain part of the study design, particularly when measuring rare false positives. Results should identify the available statistics and should not be interpreted as certain membership decisions about individual records.

A provider that can change returned computation has a different opportunity from one that only observes it. Active-cloud testing explores whether crafted responses or updates cause a client to reveal more information in subsequent interactions. It helps assess the consequences of moving from a curious-provider assumption to an actively malicious one.

An offline planner selects perturbations or update directions and analyses the resulting probe/observation pairs. A synthetic harness exercises this feedback loop, while deployed cloud operation requires a target-specific integration. Findings describe the permitted manipulations and observed reactions; they do not establish that the same intervention is available against a protected production endpoint.

An untrusted training server may be able to influence what the client reveals through returned gradients. Malicious-server testing compares honest body computation with a crafted alternative designed to strengthen token-related signals. It asks whether the assumed integrity of remote computation is carrying part of the privacy protection.

The model-and-capture path measures leakage under controlled honest and modified server conditions. The attacker is explicitly allowed to change body computation, a stronger capability than reading normal activations. Any recovery difference must be attributed to that access and configuration. It is not a generic result for inference APIs or a complete evaluation of Byzantine-resilient distributed training.

A legitimate client can still submit adversarial inputs or updates to computation shared with other users. This feature investigates extraction, poisoning, instability and collusion from that client-side position. It helps teams consider whether authentication and tenant membership are being mistaken for trustworthy behaviour.

The current implementation exercises controlled shared-body worlds in which the attacker can manipulate its own activations, queries or training updates, including multiple colluding identities. Results describe the effects on those simulated systems and the controls configured for them. A customer deployment needs its own integration and access contract before the same scenarios can support operational conclusions.

A remote model component may reveal enough through repeated queries for another party to approximate its behaviour. Body extraction evaluates this model-IP exposure by training a student component from outputs produced for attacker-chosen hidden-state inputs. It is relevant when the interface grants richer access than ordinary end-user prompts.

The workflow measures functional agreement and, where configured, transfer through a public output component. Query volume, student training budget and access to the surrounding model determine the comparison. This is functional extraction rather than recovery of the exact original weights. It also does not establish that a particular noise or decoy defence is effective without a dedicated matched evaluation.

Sensitive data may remain exposed through the resources used to process it, independently of the model’s text interface. GPU memory testing examines allocator behaviour, residual buffers and contention signals. It helps identify whether memory handling and isolation deserve closer inspection in a shared compute environment.

Available probes depend on hardware, drivers and permitted allocator visibility. They range from memory-use observations to supported residue, scrubbing and unified-memory contention checks. Seeing allocation activity is different from recovering another workload’s contents, and a same-process observation is not proof of a cross-process exploit. Reports must identify which probe actually ran and what it observed.

Response timing can reveal information that is absent from the response itself. KV timing investigates whether cached prefixes or speculative-decoding behaviour make prior activity distinguishable. It gives serving teams a way to examine privacy consequences of optimisations that reuse work across requests.

The module compares candidate-prefix timing or analyses available cache and acceptance telemetry through endpoint, journal and synthetic paths. Interpretation depends on observable timing, the selected controls and whether server telemetry is exposed. A timing difference does not itself recover a complete prompt, and findings for one cache configuration should not be generalised to all serving systems.

Encrypting payloads can leave message size and timing visible to an observer. Wire-shape analysis examines whether those patterns reveal token counts or distinguish training, prompt processing and generation phases. It supports assessment of what a network observer can infer without reading the model’s inputs or internal tensors.

The module works with packet or message journals containing sizes and timestamps, with synthetic paths for controlled comparisons. Its conclusions concern traffic metadata and depend on framing, padding and the observation conditions supplied. It does not decrypt traffic or reconstruct token contents, and it should be reported separately from attacks that receive model representations.

Model routing and scheduling can expose patterns even when payloads remain hidden. This feature investigates whether expert selection, routing weights, queue depth or overflow events reveal properties of a request or link activity across sessions. It is particularly relevant to assessments of shared mixture-of-experts infrastructure.

The analyser consumes routing journals or synthetic traces and tests property, linkability and activity signals available to the declared observer. The input metadata is an explicit prerequisite; ordinary response access does not imply visibility into expert routing. Results describe side-channel information in those traces, not full token reconstruction or a tested implementation of defensive decoy experts.

A protected tensor can still be delivered in the wrong session, at the wrong time or in the wrong order. Protocol integrity testing investigates whether a split-model system recognises replayed, duplicated, reordered or altered frames. It addresses the correctness and availability of the communication protocol alongside data confidentiality.

The battery models an active relay that can modify traffic without receiving endpoint keys. Scenarios include session swaps, epoch desynchronisation, truncation and frame corruption, with outcomes recorded against configured validation controls. These are protocol fixtures and journal-based tests; connecting them to a deployed transport requires explicit integration. Passing a fixture does not establish a production protocol’s integrity.

Shared batching, caches and streaming paths create opportunities for one tenant’s activity to affect another. Multi-tenant isolation testing investigates whether requests, state or responses cross those boundaries. It helps focus engineering review on the shared components that handle otherwise separate customer workloads.

The attacker acts as a low-privilege authenticated tenant alongside a cooperating victim fixture. Tests examine session handling, batches, caches, KV state and streams, using canaries to identify unexpected disclosure. Canaries detect leaks; they are not privacy-preserving decoys. Current fixture and journal evidence is specific to the exercised paths and cannot substitute for an integrated assessment of a customer’s serving stack.

Privacy controls at the model boundary do not prevent sensitive material from being copied into logs, checkpoints or temporary files. This feature looks for exposure in the operational records left behind by AI workflows. It can help teams identify where diagnostic convenience conflicts with their intended data-handling boundaries.

The workflow plants synthetic canaries in controlled scenarios and scans authorised stores for literal or encoded variants. It also checks relevant permissions and retention metadata. Supplied files and store roots can be examined, but no general live-service connector is implied. A clean scan only covers the supplied locations and detectable patterns; it is not a guarantee that all sensitive data has been removed.

Representation protection depends on the surrounding key and session lifecycle as well as the transform itself. This feature examines reuse, rollback, weak binding, downgrade and secret exposure within that lifecycle. It helps reveal failures that a narrow stale-key or tensor-reconstruction test would not exercise.

The battery models relationships between nonces, keys, epochs, sessions and negotiation state, comparing vulnerable and controlled configurations. Its key derivation is explicitly a toy mechanism used to test state-machine behaviour. Results therefore concern binding and lifecycle rules, not cryptographic strength. A real deployment requires integration with its actual key management, transport and retention controls.

An AI service can be disrupted through valid-looking workloads that consume disproportionate resources. Resource-exhaustion testing explores queue pressure, cache churn, long-context demand, starvation and cost amplification. It helps teams investigate how service limits and scheduling policies behave under adversarial demand.

The built-in environment is a bounded local serving simulation with explicit compute, memory, duration and workload caps. It compares resource and fairness outcomes under the selected scenarios rather than sending uncontrolled traffic to a live service. Production stress testing requires a separately authorised driver and scope; simulated results are not operational throughput figures or proof of production denial-of-service resistance.

An attacker may influence future model behaviour by modifying part of the material used to train or configure it. Poisoning and backdoor evaluation examines bounded changes to data, gradients, adapters, checkpoints or retrieval entries. It helps teams distinguish ordinary model error from behaviour introduced through a compromised input to the development process.

Current scenarios use classifier and retrieval fixtures to compare targeted trigger effects with controls such as random corruption. The attacker’s permitted modification budget and the affected artifact are explicit. A real fine-tuning driver is not connected, so the module’s outcomes should not be presented as demonstrated poisoning of an arbitrary language model or as a complete production backdoor assessment.

AI systems depend on models, adapters, tokenisers, datasets and plugins whose integrity can change between review and use. Supply-chain testing challenges the rules used to identify, verify and promote these artifacts. It supports decisions about which provenance and version controls are needed before an artifact enters a trusted workflow.

The module exercises tampering, rollback, provenance and drift scenarios against an artifact-registry verifier, with fixture and supplied-registry paths. Evidence describes how the configured registry accepted or rejected the tested artifacts. It does not establish the absence of malicious behaviour inside every verified model or package; matching an expected digest and establishing trustworthy provenance are related but distinct questions.

Reasoning-related envelopes or transcripts may carry information that should not be portable across contexts. Reasoning replay explores whether such material can be replayed, decoded, scanned for secrets or used to inject a trace into another interaction. It helps frame confidentiality and binding questions around the reasoning data an application actually exposes.

The engine provides simulator and transcript interfaces for portability and extraction experiments. Real provider adapters are not wired, and toy encodings must not be described as attacks on genuine encryption. The assessment must establish what the observer can obtain and which decoder or interface was exercised before claiming exposure of a production provider’s reasoning.

A visible summary and final answer can help an observer infer how a problem might have been solved. Trace inversion studies that possibility by generating candidate reasoning traces and checking their consistency with observable outputs. It helps separate the information carried by a summary from claims about access to hidden model internals.

The module uses verifier-guided synthesis, optional refinement and student comparisons in a synthetic arithmetic or retrieval setting. It can use supplied calibration traces, but the core observer does not receive the target’s genuine hidden trace. A plausible, answer-consistent explanation is not evidence that the original internal reasoning was recovered, and the current study does not demonstrate extraction of a provider’s private chain of thought.

Untrusted text can become a security issue when an application treats it as authority. Prompt-injection testing examines direct inputs, retrieved content and tool responses for their ability to cause unauthorised disclosure, state changes or policy violations. It focuses on what the application does, rather than whether a model repeats a provocative phrase.

The current battery uses controlled application fixtures with direct, indirect, encoded and adaptive scenarios. Outcomes are recorded as effects at the policy layer, with the attacker’s control over content made explicit. A generic live-endpoint integration remains to be connected. Fixture results establish behaviour of the tested controls and scenarios, not broad coverage of every production model or application.

System instructions, tool definitions and internal policy text can reveal information an application intends to keep private. This feature examines whether repeated or adaptive queries expose that context through visible responses. It helps teams distinguish necessary user-facing explanations from unintended disclosure of hidden implementation or control details.

Controlled fixtures supply hidden context for evaluation while the attacker observes responses only. The module separates exact recovery of planted strings from semantic or behavioural inference about policy. Those are different forms of evidence and should not be merged into one disclosure claim. A generic live application connector remains an integration requirement before the same scenarios can substantiate customer-system findings.

Retrieval can introduce both confidential content and attacker-controlled instructions into an AI workflow. This feature investigates cross-tenant retrieval, access-filter failures, poisoned ingestion, stale content and misleading citations. It helps assess whether the information reaching the model respects the application’s intended data and trust boundaries.

The module uses vector-store fixtures or supplied JSONL store dumps, with queries and ingestion constrained by the declared tenant access. Optional embedding-related paths require the relevant representation visibility. Results are specific to the store data and controls exercised; fixture retrieval behaviour does not establish that a production vector database or its surrounding authorisation layer has been tested.

Generated text becomes more consequential when it reaches a browser, query interpreter, command interface or privileged path handler. Output-sink testing examines whether downstream validation treats that text as untrusted data. It asks whether a proposed output can cross an execution or authorisation boundary, rather than merely whether a model can generate unusual syntax.

Captured or generated outputs are replayed through recording sinks for HTML, SQL, shell, templates, URLs, paths and code. The SQL case uses an in-memory fixture; other sinks do not execute attacker content. These controlled tests support review of validation decisions, but do not demonstrate arbitrary code execution or imply that a customer’s live downstream systems were exercised.

A fluent answer can still be unsupported, outdated or misleading enough to cause a poor decision. Misinformation integrity evaluation examines grounding, citation quality, confidence and the consequences of incorrect answers. It helps teams connect factual reliability to the decisions their application is expected to support.

The evaluator supplies captured answers and a domain-specific authoritative corpus, or uses controlled fixtures. That reference corpus is an evaluation resource, not knowledge automatically granted to an attacker. Conclusions depend on its coverage and freshness. This profile is separate from privacy gates and does not provide a universal truth score or establish correctness outside the questions and evidence assessed.

An agent may hold a legitimate tool permission but still use it with the wrong parameters, scope or number of effects. Tool-misuse testing examines those gaps between a user’s task and the action the agent attempts. It helps identify whether the tool gateway enforces the intended limits at the point where an action takes effect.

The module uses seeded agent loops to vary dangerous parameters, ownership assumptions and effect counts, scoring actual fixture actions against a defined authorised intent. This distinguishes policy violations from concerning language in a response. Customer assessment requires integration with the relevant gateway and action model; the current fixture does not establish that arbitrary external tools are supported.

A legitimate task can be redirected by instructions embedded in a document or tool result. Goal-hijack testing examines whether an agent lets that lower-trust content replace what the user actually asked it to do. It helps assess instruction provenance where useful external information and malicious directions appear in the same workflow.

Scenarios compare the sanctioned task with the effects the agent executes after exposure to untrusted content. Seeded worlds provide controlled tests, and a model-driven planner path adds model behaviour within the harness. The tools and resulting effects remain defined by that environment. A finding is scoped to the planner, controls and interfaces exercised, not to all real agent integrations.

Human approval is useful only when the approved description matches the action that follows. This feature investigates discrepancies between what an operator is shown and what an agent subsequently executes. It helps teams examine approval as a concrete binding between intent and effect rather than a generic confirmation step.

The fixture introduces approval-summary and execution mismatches, then measures whether the configured controls reject the substituted action. The evidence concerns a simulated approval/action relationship, not a study of how real people respond to interface wording. Integrating the check with a product requires its actual approval object, displayed summary and execution record to be connected.

An instruction written into persistent memory can influence an agent long after the original interaction ends. Memory-context poisoning tests whether stored content can redirect a later session. It helps teams assess the provenance and reuse rules for memory that an agent may otherwise treat as helpful background knowledge.

The module writes poisoned memory in a controlled fixture and observes subsequent retrieval and action effects. It compares those effects with the original authorised task and selected memory controls. This is a simulated cross-session workflow, not a connector to every memory service. Production conclusions require matching the test to the application’s storage, retrieval and trust policies.

An agent’s behaviour can change when a trusted-looking tool or connector is replaced. This feature investigates whether component metadata and integrity checks prevent a substituted dependency from steering the agent’s actions. It extends assessment from prompt content to the components that supply information and capabilities.

The fixture replaces a tool or connector component and compares resulting actions under the configured provenance controls. Component digests and expected metadata are part of the scenario. Results describe the simulated replacement path and do not certify an entire package ecosystem. A real assessment needs the deployment’s component loading, pinning and verification mechanisms to be included.

A tool result should not be able to convince an agent to act as a different user or borrow authority from another session. Identity-privilege abuse tests this confused-deputy risk at the gateway where an action is invoked. It helps distinguish having a valid tool permission from using it under the correct authenticated identity.

The simulated workflow varies claimed principals, roles and cross-session authority, then compares executed effects with the session’s authorised intent. Principal binding is evaluated separately from instruction-provenance controls. Evidence concerns that gateway mechanism and scenario; it does not demonstrate a flaw or protection in a customer’s identity provider without a corresponding integration.

An incorrect value can become more consequential as it passes through automated steps that assume earlier validation was sufficient. Cascading-failure testing examines whether a corrupted upstream decision reaches a downstream action unchecked. It helps teams identify where a workflow should revalidate facts or permissions before acting.

The module models propagation through several hops and tests controls that check chain integrity independently of the routed value. Effects are compared with a fixed authorised task. This is an integrity experiment in a simulated workflow, not an implementation of Byzantine consensus or evidence about an arbitrary deployed multi-agent system. Findings should name the validation boundary that failed.

An agent that can request code execution must run the computation actually authorised for the task. This feature tests whether an unapproved code identity can be introduced through untrusted content or swapped after approval. It helps assess whether approval remains attached to the exact proposed action until invocation.

The sandbox is simulated: code identities are opaque digests mapped to canned results, and no attacker code is executed. Tests compare the requested and approved digests, task binding and resulting fixture records. This provides evidence about authorisation and substitution controls; it is not a real sandbox-escape test or a demonstration of arbitrary execution in production.

An agent’s authority to complete a task does not automatically include permission to replicate, rewrite its goal or continue after the task ends. Rogue-agent testing examines those lifecycle boundaries. It helps teams define which operations require explicit operator authority beyond ordinary tool access.

The fixture records attempted spawning, self-modification and persistence as inert in-memory actions, then checks them against an attested mandate. It does not create real processes, change a live agent’s configuration or install persistence. Results evaluate the modelled lifecycle controls and need a deployment-specific integration before they can support claims about a running agent platform.

Messages between agents need a trustworthy sender, an intact payload and a clear indication that the instruction is still current. This feature investigates spoofed, altered and replayed messages. It helps assess whether an agent acts on a message because it is authenticated, rather than because it merely resembles a legitimate instruction.

The current bus connects synthetic endpoints and evaluates message authentication codes, recipient binding and nonce freshness against recorded actions. Payloads and effects remain inert within the harness; no real transport is exercised. Conclusions concern the configured message checks and are separate from claims about distributed consensus or the security of a production communication service.

Redundant computation can improve confidence only if the aggregation policy handles disagreement and malicious participants appropriately. Replica integrity examines collusion, abort behaviour, honest variation and the impact of shared-key assumptions. It helps teams ask what redundancy actually contributes under a defined participant threat model.

The module accepts supplied replica observations and separate scoring truth, with synthetic and independently seeded local-channel runners, including subprocess execution. It measures accepted errors, alarms and availability outcomes under the selected policy. These engineering tests do not establish Byzantine fault tolerance or cryptographic security, and local worker independence should not be presented as a completed cross-host deployment assessment.

Splitting or mixing information changes what each observer sees, but repeated releases and cooperating observers can alter the exposure. View-privacy testing examines de-mixing, complementary coordinate views and accumulation around monitoring windows. It helps evaluate the access assumptions behind partial-view protection.

The evaluator supplies observations, attacker-known calibration and separate scoring truth. Scenarios study repeated mixtures of the same target, declared coalitions and period-local tripwire counts. Coordinate views are not threshold secret sharing, and simplified monitors are not general adaptive controllers. Findings should state the coalition, repeated-release conditions and calibration knowledge rather than imply a break of a cryptographic primitive.

A defence should be challenged with a probe trained for the information it actually releases. Representation privacy evaluates fresh coordinate, norm and relationship-based probes against transformed features. It helps test whether a bottleneck or re-hardening step reduces a declared information target, rather than only disrupting an older decoder.

The workflow uses separate calibration and evaluation documents with provenance identifying the defender and release stage. It scores a prespecified per-document label, not automatically full-sequence recovery. Optional membership analysis requires additional phase records and validated provenance, and is training-specific. These are audits of supplied representations, not full reproductions of the defences that produced them.

Reducing a gradient’s magnitude may leave its direction informative. Gradient hardening examines what backward reconstruction can still recover after clipping, abort handling, additive noise or quantisation. It helps compare mitigation choices against the information retained by the returned gradient.

The battery replays raw captured gradients through the selected transformations and applies a direction-based decoder, with optional joint reconstruction using activations and a public head. It records clipping and recovery measurements for those conditions. This is a controlled post-capture audit, not a complete defended training run or a differential-privacy guarantee; already-transformed inputs would compound the intervention and change the question.

A distributed assessment can generate evidence on several machines. Remote fetch provides a consistent way to check configured nodes and collect their capture directories for analysis. It supports the practical work of gathering observations without confusing collection with the attack that will later interpret them.

The utility uses authorised SSH and read access, records reachability and copies the selected capture stores into a node-specific layout. An unreachable fleet is recorded as a collection outcome, not a privacy result. The operation does not run a new leakage algorithm, and collected files may still require decoding, schema validation or model-specific inputs before they can be evaluated.

Testing a model boundary first requires observing the traffic exchanged across it. The relay utility captures bytes in both directions while recording the communication for later analysis. It can support a controlled assessment when the system owner permits an observation point between the relevant endpoints.

This is a traffic-collection component rather than a reconstruction method. Captured bytes may remain encoded or encrypted, and interpreting them depends on the protocol and access agreed for the assessment. Reports should distinguish successful collection from recovery of meaningful model data; the presence of a relay does not imply that transport protection has been bypassed.

One unsuccessful attempt may reflect an unlucky initialisation or an insufficient search budget. Max effort repeats a selected attack across seeds, starting points and budgets so evaluators can examine how effort changes the outcome. It supports more informative comparisons than treating a single run as definitive.

The orchestrator records the selected attack’s outcomes and resource conditions while inheriting that attack’s inputs, dependencies and threat model. More attempts can change the interpretation of an observed result and must remain visible in reporting. This feature does not add a new observation surface, turn a synthetic target into a live integration or make unequal-budget comparisons automatically fair.

Separate observations can contribute to a broader exposure picture across time, users or model boundaries. Privacy composition brings existing results together to explore scenarios in which an adversary accumulates information from several sources. It helps identify combinations that may deserve a dedicated joint experiment.

The module consumes per-surface artifacts and budget assumptions to construct scenario projections. It does not itself recover protected tensors or measure the true joint leakage of a deployed system. Correlation, repeated access and incompatible experimental conditions can affect interpretation, so projected exposure should remain distinct from a calibrated privacy bound or a directly observed multi-surface attack.

A useful evaluation needs access to the model computation at the boundary being studied. The engine provides split interfaces for GPT-2, Mistral and Qwen3, with CPU and CUDA execution where supported. These interfaces support controlled observations and experiments around distributed model execution.

Support for a model interface does not mean every attack or study has been validated on that family. Current paper experiments and head exports are GPT-2-specific, and optional dependencies and hardware determine which paths can run. Assessment scope should identify the exact model snapshot, split point and executed workflow rather than treating family-level support as universal coverage.

The same model can expose different information during training, prompt processing and generation. Capture lifecycle support records training forward and backward observations separately from inference prefill and decode, with coordinates needed to align observations for scoring. It gives an assessment a clearer account of what was visible and when.

Canonical alignment connects captures to evaluation records without making scoring truth an automatic attacker input. The current rig is local, and inference uses full re-forwards rather than a production KV-cache serving stack. Results therefore need to preserve phase and capture provenance, and should not be presented as measurements of a distributed production transport or serving latency.

A model-driven planner can behave differently from a deterministic fixture. The live agent harness lets evaluators examine those decisions while keeping tasks, allowed effects and scoring structured. It supports investigation of whether exposure to untrusted information changes what an agent actually does.

The planner can use a real model, but its target tools and world remain defined by the harness. Effects are assessed against a recorded authorised intent rather than judged solely from response wording. This distinguishes model-driven evidence from a synthetic planner without implying arbitrary production integration; each customer tool and gateway still requires an appropriate connection and test contract.

Baseline comparisons help explain whether a proposed control changes the information an observer can recover. The engine includes plain split execution and reference transformations such as rotation, Gaussian noise and combinations of rotation, permutation and noise. These mechanisms provide controlled contrasts for privacy experiments.

Their availability is a way to test hypotheses, not evidence that a defence is effective for every workload. Synthetic bundles must remain distinguished from model-driven captures, and each comparison needs matched inputs and an explicit attacker model. A passing diagnostic describes the configured test conditions; it is not a general privacy guarantee.

Privacy interventions can act on input text, intermediate features or a learned representation bottleneck. The engine provides experiments with noise, sparsification, input-transform variants, trained bottlenecks and a PCA compression control. These options help teams compare where a transformation acts and how it affects both exposure and utility.

Methods including mid-lite, at-lite, santext, custext and snd-lite are local mechanisms or analogues rather than full published-method reproductions. The trained bottleneck is a local re-implementation with optional co-training and GPU requirements by default. Each method needs its own access, utility and reconstruction evaluation; compression or reduced recovery alone does not establish useful privacy.

A single token-accuracy figure can hide whether an attack recovered a meaningful phrase or scattered isolated tokens. Recovery metrics include token and sequence accuracy, exact windows, longest recovered runs and annotated names, numbers or identifiers. This helps evaluators describe the content exposure more precisely.

Available measurements depend on the workflow and the ground-truth annotations prepared for it. Sensitive-span annotations must be supplied before outcomes are examined, and absent annotations remain unmeasured rather than being treated as safe. Recovery scores describe performance on the evaluated content; linking them to business consequences still requires context about the affected data and system.

An inference attack needs to be assessed for false alarms as well as successful detections. The engine records measures such as AUC and true-positive rates at selected low false-positive rates, alongside controls and null scenarios. These measurements help teams understand how distinguishable a sensitive property is under the tested observations.

The statistical conventions and sampling units vary by module. Small evaluation sets may not resolve very low false-positive rates, so a reported operating point requires enough supporting examples. Such scores should not be converted into certainty about an individual record or combined across unrelated datasets without checking the reference populations and evaluation design.

A defence that suppresses leakage by making the model unusable does not solve the deployment problem. Utility and resource measurements place attack outcomes alongside loss, perplexity, throughput, elapsed time, memory and computational work where available. They support a comparison between protection and the cost of retaining useful behaviour.

Metrics must be interpreted within the workflow that produced them. Wall time may include capture I/O, CUDA elapsed time is not pure GPU busy time, and not every path records every field. Matched configurations and retained defender state matter when comparing utility. These records are experiment measurements, not automatic production performance or return-on-investment claims.

Differences between attacks or defences are easier to interpret when unrelated conditions are held constant. The framework supports paired methods, multiple victim and attack seeds, randomised execution order, separated data partitions and budget sweeps. These controls help distinguish a mechanism’s effect from data overlap, run order or search effort.

A configured protocol is not the same as a completed study. Frozen held-out validation plans and synthetic smoke execution remain separate from collected real-model results. Each report should identify what ran, which data and seeds were used, and where integration or execution is still pending. Conclusions should follow the completed evidence rather than the size of the planned matrix.

An assessment should show how stable a result is, not just its central score. The engine includes paired-seed intervals and, in newer study workflows, crossed document/run bootstrap analysis with correction for multiple comparisons. These methods help express uncertainty around the experimental contrast being examined.

Different studies use different sampling units and assumptions, so their intervals cannot be pooled indiscriminately. Tokens within one sequence are not automatically independent repetitions, and uncertainty conditional on one corpus does not establish generalisation across organisations. Reports need to state the analysis actually used and preserve insufficient-evidence outcomes when the design cannot support a conclusion.

A finding becomes more useful when another reviewer can identify the configuration, predictions and artifacts behind it. Evidence support records hash chains, manifests, code and runtime provenance, prediction hashes and independent rescoring where the workflow provides them. This helps teams inspect how a reported result was assembled.

Verification must be enforced and important digests retained independently for change detection to be meaningful. Hashes do not prove that an experiment ran on the claimed host or prevent someone from rewriting an entire unanchored bundle. Reproducible records support review, while scientific validity still depends on the study design, source authenticity and correctness of the measurements.

Engineering results need a form that reviewers can compare and act on. The engine produces CLI reports, scorecards, execution-coverage records, versioned benchmark profiles and workflow-dependent exports such as CSV files and plots. These outputs help organise findings and make missing or failed work visible.

Execution coverage shows which configured paths ran; it is not a validated estimate of organisational risk. Scores and gates retain the assumptions of the underlying module, and incomplete evidence must not be read as a clean assessment. Business-facing interpretation should explain the relevant outcome, scope and uncertainty rather than promote an aggregate score as certification.

New architectures may expose interfaces that existing modules do not cover. The engine provides an attack registry, structured metadata, surface providers and plugin entry points so additional methods and observation adapters can be integrated consistently. This supports extending an assessment around a specific system rather than forcing every target into the same fixture.

An extension still needs working input paths, documented attacker access, compatible artifacts and meaningful validation. Some modules require optional SDKs, captures or model dependencies, and a registered name establishes implementation availability rather than production coverage. Supported customer integrations should be claimed only after the relevant end-to-end path has been exercised.

Decision-makers need to understand which exposures were investigated and what evidence supports the next action. The engine’s exposure profiles and operational gates can organise results around those questions. They are useful inputs to a specialist review when their criteria and underlying measurements remain visible.

There is no externally calibrated universal risk score or validated organisational maturity model in the current foundation. Legacy gates are specific diagnostics, not breach probabilities or certifications. A ProofSeam assessment should therefore use these records to explain scoped evidence, unresolved questions and remediation priorities rather than assign an unsupported claim of overall security maturity.