v1.0.0 444 tests · 0 failures

BLACK BOX

AI Incident Forensics Platform

When AI incidents happen, you don't get to guess. You get to replay.

BLACK BOX captures every decision, seals every event, and replays the exact sequence — making AI security incidents provable in court.

444
Tests
10
Capture Streams
3
Forensic Layers
4
Gate Tiers
Install BLACK BOX Documentation →
BBX-{hex12}
Session ID Format
NIST
SP 800-86 Compliant
EU AI Act
Arts. 9 / 13 / 18
Dual-Sig
Ed25519 + ML-DSA-65
8765
REST API Port
Architecture

Three-Layer Forensic Platform

BLACK BOX operates as a continuous, cryptographically-sealed record of every event in an AI agent session. Three layers — CAPTURE, EVIDENCE CHAIN, and REPLAY — work together to make every incident reconstructable from first principles.

01
Layer 01
CAPTURE
Continuous, structured recording across 10 event streams. Every input, output, tool invocation, memory retrieval, reasoning checkpoint, and policy decision is captured in real time — before, during, and after the incident window.
  • 10 parallel event streams
  • Session crash recovery via PARTIAL_CHAIN flag
  • Ungated — no key required
  • Append-only JSONL + SQLite storage
02
Layer 02
EVIDENCE CHAIN
Merkle-style SHA-256 hash chain linking every captured event in sequence. Dual Ed25519 + ML-DSA-65 post-quantum signatures on all records. BBX-{hex12} session IDs provide globally unique, unforgeable session references.
  • SHA-256 Merkle-style hash chain
  • Ed25519 + ML-DSA-65 dual signatures
  • Tamper-evident append-only store
  • BBX-{hex12} session identifiers
03
Layer 03
REPLAY
Structured, gate-controlled replay of any captured session. Linear replay, filtered replay by stream or time window, and side-by-side comparison replay. Full context reconstruction and confidence trajectory visualisation.
  • Linear / filtered / comparison modes
  • Context reconstruction at any point
  • Confidence trajectory replay
  • Operator key required (REPLAY gate)
Capture Streams

10 Event Streams

Every AI agent session is broken into 10 structured capture streams. Each stream is independently queryable, hashable, and replayable. Together they give a complete forensic picture of what the agent did, decided, and why.

S-01
Session Metadata
Agent identity, runtime config, start/end timestamps, session ID chain
S-02
Input Context
System prompt, user messages, injected context, token counts
S-03
Memory Activity
Memory retrievals, writes, evictions, context window state
S-04
Tool Activity
Tool invocations, arguments, responses, latency, errors
S-05
Reasoning Chains
Chain-of-thought traces, intermediate conclusions, model outputs
S-06
Policy Decisions
Safety evaluations, refusals, guardrail activations, override events
S-07
Runtime Config
Temperature, sampling params, model version, system config snapshot
S-08
External Dependencies
API calls, external data fetches, third-party service responses
S-09
Human Oversight
Human-in-the-loop events, approvals, interventions, escalations
S-10
Confidence Timeline
Per-step confidence scores, uncertainty flags, calibration drift events
Evidence Chain

Cryptographic Integrity

Every event appended to BLACK BOX is immediately hashed and chained to the previous event. The resulting chain is dual-signed and cannot be modified without breaking the hash sequence — making every session record court-admissible.

Hash Chain

SHA-256 Merkle-Style Chaining

Each event record contains the SHA-256 hash of the previous record. Any modification to any event in the chain immediately invalidates all subsequent hashes, making tampering detectable with zero false-negative risk.

Signatures

Ed25519 + ML-DSA-65 Dual Signing

All chain records are dual-signed — Ed25519 for current interoperability and ML-DSA-65 (CRYSTALS-Dilithium, NIST FIPS 204) for post-quantum resistance. Both signatures must verify for a record to be accepted as authentic.

Storage

Append-Only JSONL + SQLite

Events are written to an append-only JSONL flat file for streaming and archival, and indexed in SQLite for fast structured queries. Neither store permits deletions or modifications after write. Crash recovery uses the PARTIAL_CHAIN flag with session.seal().

Session ID

BBX-{hex12} Session Identifiers

Every BLACK BOX session is assigned a globally unique BBX-{hex12} identifier at capture start. The session ID appears in all signed records, all export bundles, and all STIX 2.1 Campaign Graph exports. Session IDs are collision-resistant and cannot be predicted in advance.

STIX 2.1

Campaign Graph Export

BLACK BOX sessions can be exported as STIX 2.1 bundles for integration with NIGHTFALL Campaign Graph. Exported bundles carry the full BBX hash chain alongside STIX Observed-Data, Sighting, and Campaign objects for end-to-end incident correlation.

Access Control

Gate Architecture

BLACK BOX uses a four-tier gate system. CAPTURE and VERIFY are ungated so that recording can never be blocked by a missing key. REPLAY and EXPORT require cryptographic keys — ensuring that only authorised operators can access session contents.

CAPTURE
UNGATED
Continuous event recording across all 10 streams. No key required. Recording begins at process start and cannot be suppressed by configuration.
No key required
VERIFY
UNGATED
Hash chain integrity verification and signature validation. Allows any party to confirm the authenticity of a session record without requiring access to session contents.
No key required
REPLAY
OPERATOR KEY
Linear, filtered, and comparison replay of captured sessions. Full context reconstruction and confidence trajectory. Requires an operator Ed25519 private key.
Requires: operator Ed25519 key (operator.pem)
EXPORT
TWO-KEY
Session bundle export — JSONL, SQLite snapshot, STIX 2.1 Campaign Graph. Requires both the operator key and the BLACKBOX_EXPORT_KEY environment variable for two-person authorisation.
Requires: operator.pem + BLACKBOX_EXPORT_KEY env var
Compliance

Regulatory Alignment

BLACK BOX is designed from the ground up to support regulatory and legal requirements for AI incident documentation. Built-in compliance mappers generate structured evidence packages for each framework.

NIST SP 800-86

Guide to Integrating Forensic Techniques into Incident Response
  • Collection — append-only dual-stream capture
  • Examination — structured query across all 10 streams
  • Analysis — timeline reconstruction and comparison replay
  • Reporting — signed, exportable incident bundles
  • Chain of custody — BBX-{hex12} hash-chain continuity

EU AI Act

Articles 9, 13, and 18 — High-Risk AI Systems
  • Art. 9 — Risk management system logging
  • Art. 13 — Transparency and traceability records
  • Art. 18 — Automatic logging for high-risk AI
  • Incident documentation with timestamp integrity
  • Human oversight event capture (S-09)

NIST AI RMF

AI Risk Management Framework — Four Core Functions
  • GOVERN — policy decision capture (S-06)
  • MAP — session metadata and context (S-01, S-02)
  • MEASURE — confidence timeline (S-10)
  • MANAGE — incident replay and export bundles
  • Post-incident evidence package generation
AI Shield Integration

Native Module Integration

BLACK BOX integrates natively with four AI Shield modules to pull existing detection and telemetry records into the session evidence chain. When an AI Shield module fires during a captured session, its report is automatically incorporated into the BBX hash chain.

M12
Evidence stream — detection events and alert payloads
M17
Policy decision records — refusals and guardrail activations
M25
Runtime telemetry — latency, throughput, resource utilisation
M90
Long-term archive — cold storage and retrieval of sealed sessions
Quick Start

Install & Initialise

BLACK BOX is distributed as a Python package. Generate your operator key pair on first install. CAPTURE is active immediately — no configuration required to start recording.

# Install BLACK BOX
$ pip install red-specter-blackbox
 
# Generate operator key pair
$ blackbox keygen --output operator.pem
[+] Ed25519 operator key pair generated
[+] operator.pem written
[+] operator.pub written
 
# Start capturing a session
$ blackbox capture --session my-agent-run
[+] Session ID: BBX-a3f9c1d04b72
[+] CAPTURE active — 10 streams open
[+] Hash chain initialised (SHA-256 Merkle)
 
# Verify chain integrity
$ blackbox verify --session BBX-a3f9c1d04b72
[+] Chain verified — 2,847 events, 0 tamper flags
[+] Ed25519 signature: VALID
[+] ML-DSA-65 signature: VALID
 
# Replay a session (requires operator.pem)
$ blackbox replay --session BBX-a3f9c1d04b72 --key operator.pem
 
# Start the REST API
$ uvicorn blackbox.api:app --port 8765
[+] BLACK BOX REST API listening on :8765
BBX-{hex12}
Session ID format — 48-bit hex suffix (12 hex chars)
Ed25519 + ML-DSA-65 dual-signed — tamper-evident, post-quantum resistant
Embedded in all JSONL records, SQLite rows, and STIX 2.1 Campaign Graph exports
The Third Pillar of Red Specter

NIGHTFALL attacks. AI Shield defends. BLACK BOX records — so that when an incident happens, you have provable, court-admissible evidence of exactly what the AI did, decided, and why. No guessing. Replay.

Full Documentation → NIGHTFALL AI Shield