Auditing Access Events in an Immutable Ledger

An ISO 16363 audit does not ask whether your repository can restrict access — it asks whether you can prove, months after the fact, exactly who read which archival object and when. Most repositories fail that test not because access is uncontrolled but because the record of access is a plain log file that any operator can edit, truncate, or lose. This walkthrough sits under the Digital Preservation Security Policies area and shows how to record every read against an Archival Information Package (AIP) or Dissemination Information Package (DIP) into an append-only, tamper-evident ledger where each entry is cryptographically bound to the one before it. The goal is a trail an auditor can independently verify: not “trust our logs,” but “re-walk the chain and see for yourself that nothing was altered or removed.”

# Root-Cause Analysis of Weak Audit Trails

A trail that satisfies day-to-day curiosity (“who touched this last week?”) routinely collapses under the evidentiary standard ISO 16363 expects. Four concrete defects account for nearly every unusable access record:

1. Mutable logs. The access record is a text file, a rotating syslog stream, or a database table with UPDATE and DELETE grants. Any administrator, any compromised service account, or any careless log-rotation policy can rewrite history without leaving a trace. An auditor cannot distinguish an untouched log from one that was quietly edited last night, so the entire trail is presumed unreliable.
2. Gaps. Log rotation discards old segments, a result-backend TTL expires the evidence, or a crash drops the buffered tail. Because entries carry no notion of sequence beyond a timestamp, a missing block between 02:00 and 04:00 is invisible — nothing in the surviving records reveals that anything was removed.
3. No linkage to identity. The record says a file was read but not who read it, under what role, or through which dissemination request. An event without a bound actor and role cannot answer the access-review questions ISO 16363 poses, and it cannot corroborate the rights constraints emitted alongside restricted material.
4. Logs stored with less protection than the objects. The AIPs live on WORM storage with object-lock and offline replicas; the access log lives on the application server’s local disk with world-writable permissions. The evidence about the crown jewels is protected worse than the jewels — the exact inversion an attacker exploits to cover their tracks after exfiltrating a restricted collection.

The unifying flaw is that a conventional log records events but does not record the integrity of the sequence of events. Fixing that requires two independent properties: every entry must be bound to its predecessor so that altering one entry invalidates every entry after it, and the storage must physically forbid rewriting what has already been written. The diagram below shows how a hash chain delivers the first property.

Each record binds the prior record’s hash into its own body, so altering or deleting any entry breaks every hash after it — and an independent verifier re-walking the chain catches the break.

# The Hash-Chain Relation

The tamper-evidence comes from one recurrence. Let $R_n$ be the canonical serialization of the $n$-th access record — actor, role, object identifier, action, and timestamp — and let $H_{n-1}$ be the hash stored in the previous entry. The hash of record $n$ is:

$$ H_n = \mathrm{SHA256}!\left(H_{n-1} ,\Vert, R_n\right), \qquad H_0 = \mathrm{SHA256}(\text{genesis seed}) $$

where $\Vert$ denotes byte concatenation and $H_0$ is a fixed genesis anchor recorded out of band. Because $H_n$ depends on $H_{n-1}$, which in turn depends on $H_{n-2}$, every hash transitively commits to the entire prior history. Editing $R_k$ changes $H_k$, which changes $H_{k+1}$, and so on to the head of the chain — so a single altered field is detectable by recomputing any one downstream hash. Deleting a record is equally visible: the surviving successor’s stored prev-hash no longer equals the recomputed hash of its actual predecessor. This is the same append-only discipline used when Designing an Immutable Audit Ledger for preservation actions; here it is applied to the Access functional entity rather than to Preservation Planning.

# Step-by-Step Resolution

The fix has five parts: capture a complete event, chain it to the prior hash, append it under write-once storage, verify independently, and treat any break as an incident. The record schema captures the identity linkage that plain logs omit — every event names an actor and the role under which they acted, which is exactly the role vocabulary defined by Implementing Role-Based Access Control for Digital Archives.

from __future__ import annotations

import hashlib
import json
import logging
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
from pathlib import Path

logger = logging.getLogger("archival.access.ledger")

GENESIS_HASH = hashlib.sha256(b"archival-access-ledger:genesis:v1").hexdigest()


@dataclass(frozen=True)
class AccessEvent:
    """A single read/dissemination event against an AIP or DIP."""

    actor: str          # authenticated principal, e.g. "b.rivera"
    role: str           # RBAC role in force at access time
    object_id: str      # AIP/DIP identifier, e.g. "aip:0f3c"
    package_type: str   # "AIP" or "DIP"
    action: str         # "read", "download", "render", "export"
    timestamp: str      # RFC 3339, UTC


def _canonical(event: AccessEvent) -> bytes:
    """Deterministic byte serialization so the hash is reproducible."""
    return json.dumps(asdict(event), sort_keys=True, separators=(",", ":")).encode("utf-8")


def _record_hash(prev_hash: str, event: AccessEvent) -> str:
    """H_n = SHA256(H_{n-1} || R_n)."""
    digest = hashlib.sha256()
    digest.update(prev_hash.encode("ascii"))
    digest.update(_canonical(event))
    return digest.hexdigest()


def _last_hash(ledger_path: Path) -> str:
    """Return the hash of the current head record, or the genesis anchor."""
    if not ledger_path.exists():
        return GENESIS_HASH
    last = GENESIS_HASH
    with ledger_path.open("r", encoding="utf-8") as handle:
        for line in handle:
            if line.strip():
                last = json.loads(line)["record_hash"]
    return last


def append_access_event(ledger_path: Path, event: AccessEvent) -> str:
    """Chain an access event to the ledger head and append it, write-once."""
    prev_hash = _last_hash(ledger_path)
    record_hash = _record_hash(prev_hash, event)
    entry = {"prev_hash": prev_hash, "event": asdict(event), "record_hash": record_hash}

    # Open append-only; on a WORM/object-lock volume the OS forbids rewrites.
    with ledger_path.open("a", encoding="utf-8") as handle:
        handle.write(json.dumps(entry, sort_keys=True, separators=(",", ":")) + "\n")

    logger.info(
        "access_event_appended",
        extra={
            "actor": event.actor,
            "role": event.role,
            "object_id": event.object_id,
            "action": event.action,
            "record_hash": record_hash,
        },
    )
    return record_hash

The frozen=True dataclass makes an in-memory event immutable, and _canonical fixes a single byte representation so two verifiers agree on the hash. The append mode is the application-level half of write-once; the storage-level half is an object-lock bucket (S3 Object Lock in compliance mode, an Azure immutable blob policy, or a physical WORM tier) that refuses PUT-overwrite and DELETE for the retention period. Both halves matter: append mode alone is defeated by anyone with a text editor, while object-lock alone still lets an attacker append a forged continuation unless the hash chain binds each entry to real history.

# Independent Verification

Verification must be performed by a process that has no write access to the ledger and re-derives every hash from scratch. It re-walks the chain from the genesis anchor, recomputing $H_n$ for each record and comparing it against the stored value; the first mismatch localizes the tampering or the gap.

from __future__ import annotations

import json
import logging
from pathlib import Path

logger = logging.getLogger("archival.access.verifier")


def verify_ledger(ledger_path: Path) -> bool:
    """Re-walk the hash chain; return True only if every link verifies."""
    expected_prev = GENESIS_HASH

    with ledger_path.open("r", encoding="utf-8") as handle:
        for line_no, line in enumerate(handle, start=1):
            if not line.strip():
                continue
            entry = json.loads(line)
            event = AccessEvent(**entry["event"])

            if entry["prev_hash"] != expected_prev:
                logger.error(
                    "ledger_chain_broken",
                    extra={"line": line_no, "reason": "prev_hash_mismatch",
                           "expected": expected_prev, "found": entry["prev_hash"]},
                )
                return False

            recomputed = _record_hash(entry["prev_hash"], event)
            if recomputed != entry["record_hash"]:
                logger.error(
                    "ledger_record_tampered",
                    extra={"line": line_no, "reason": "record_hash_mismatch",
                           "object_id": event.object_id, "actor": event.actor,
                           "expected": recomputed, "found": entry["record_hash"]},
                )
                return False

            expected_prev = entry["record_hash"]

    logger.info("ledger_verified", extra={"head_hash": expected_prev, "status": "intact"})
    return True


if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s %(message)s")
    path = Path("access_ledger.jsonl")

    append_access_event(path, AccessEvent(
        actor="b.rivera", role="reading-room", object_id="aip:0f3c",
        package_type="DIP", action="read",
        timestamp=datetime.now(timezone.utc).isoformat()))
    append_access_event(path, AccessEvent(
        actor="svc-access", role="dissemination", object_id="aip:9b71",
        package_type="DIP", action="download",
        timestamp=datetime.now(timezone.utc).isoformat()))

    verify_ledger(path)  # -> logs ledger_verified

    # Simulate tampering: rewrite the first record's actor in place.
    lines = path.read_text(encoding="utf-8").splitlines()
    first = json.loads(lines[0])
    first["event"]["actor"] = "attacker"
    lines[0] = json.dumps(first, sort_keys=True, separators=(",", ":"))
    path.write_text("\n".join(lines) + "\n", encoding="utf-8")

    verify_ledger(path)  # -> logs ledger_record_tampered and returns False

The verify_ledger function distinguishes the two failure modes explicitly. A prev_hash_mismatch means a record was inserted or removed — the chain’s linkage no longer lines up. A record_hash_mismatch means a record’s own fields were edited in place — the stored hash no longer matches the recomputed one. Both return False, and both emit a structured error carrying the offending line and object, so the outcome is machine-readable and can drive an alert rather than sitting in a log nobody reads.

# Validation and Verification

Confirming the ledger is trustworthy means checking the mechanism, not just running the happy path:

• Round-trip a known chain. Append a fixed sequence of events into a scratch ledger, record the head hash, and assert verify_ledger returns True. The head hash is a compact fixity value for the entire access history — publish it to an out-of-band notary (a signed daily digest, a second repository) so a wholesale replacement of the ledger is detectable.
• Prove tamper-evidence. Edit one byte of any record and confirm the verifier returns False at the expected line. Delete a record and confirm the successor’s prev_hash_mismatch fires. These negative tests are the actual ISO 16363 evidence — that the control detects violation, not merely that it records events.
• Reconcile against rights. Cross-check that every action against a restricted object corresponds to an allowed role. The rights constraints themselves are declared by Emitting PREMIS Rights Statements for Restricted Collections; the ledger is the evidence that those constraints were honored at read time.
• Verify on a schedule and on demand. Run the verifier as a scheduled job and again immediately before any audit export. Store the verification outcome — head hash, timestamp, result — as its own durable record.

# Edge Cases and Gotchas

Two subtleties deserve emphasis. First, a hash chain proves integrity and order, not authenticity of origin — it shows no one altered the sequence, but it does not by itself prove which server wrote an entry. For high-assurance repositories, sign the periodic head-hash digest with an offline key so a full-ledger substitution is caught. Second, treat any verification failure as a reportable security incident, not a bug to quietly patch: the whole point of tamper-evidence is that a detected break is information, and suppressing it discards the accountability the ledger exists to provide. Wire the verifier’s False return into the same incident path that governs a fixity failure, and preserve the failing ledger as evidence rather than regenerating it.

For the cryptographic primitive, the Python hashlib documentation specifies the SHA-256 interface used throughout; the accountability requirement itself is set by the trustworthy-repository audit criteria of ISO 16363.

# Frequently Asked Questions

# Why hash-chain the log instead of just making it read-only?

Read-only permissions are enforced by the same administrators who could be the threat, and they say nothing about whether a record was altered before the permission was applied or after it was lifted for maintenance. A hash chain moves the guarantee from policy to mathematics: because each entry commits to the entire prior history, any edit or deletion invalidates every subsequent hash, and an independent verifier detects it without trusting the storage layer at all. Read-only storage and hash-chaining are complementary — object-lock stops the write, the chain proves none slipped through.

# What exactly should each access record contain?

At minimum the authenticated actor, the role under which they acted, the object identifier and its package type (AIP or DIP), the action performed, and a UTC timestamp. The actor-plus-role pairing is what turns a bare “file was read” line into evidence that answers an access review, and binding it into the hash means the identity cannot be edited out later. Keep the schema canonical and stable, because the byte serialization of these fields is what the hash is computed over.

# Does recording every read hurt performance?

The marginal cost of an append is one SHA-256 over a few hundred bytes plus a line write, which is negligible next to serving the object itself. The real cost to manage is contention: appends must be serialized so two workers do not fork the chain from the same head hash. Route access events through a single writer or a short-held lock, and the ledger keeps pace with the Access functional entity without becoming a bottleneck.

# How is this different from PREMIS preservation event logging?

PREMIS events record actions the repository takes on objects — fixity checks, migrations, replications — as provenance. This ledger records access to objects by agents for accountability. They share the append-only, tamper-evident discipline but answer different audit questions: one proves the repository preserved the bits faithfully, the other proves it disclosed them only to authorized readers. A complete ISO 16363 posture needs both.

# Related

• Digital Preservation Security Policies — the parent area: the access-control, rights, and audit policies this ledger operationalizes.
• Implementing Role-Based Access Control for Digital Archives — the role vocabulary every ledger entry references when it records the role in force at access time.
• Designing an Immutable Audit Ledger — the same hash-chained, append-only pattern applied to preservation-action provenance rather than read access.