Deployment & Key Custody

Shomei Governed Memory is designed for customer-VPC deployment with complete data residency and key control. Your data and keys never leave your environment.

Storage Model

Data is stored in per-tenant SQLite databases on a volume you control, with per-row encryption and managed key custody via AWS KMS (BYOK).

At-Rest Encryption: Per-Row DEK Envelope

Each memory's payload and embedding cells are encrypted under a per-row Data Encryption Key (DEK), wrapped by the workspace Key Encryption Key (KEK):

Memory record
├─ Payload (ciphertext under DEK_row)
├─ Embedding (ciphertext under DEK_row)
└─ Wrapped DEK_row (encrypted under workspace KEK)

• DEK envelope: the wrapped DEK is stored alongside the row; the plaintext DEK never lives at rest. The wrapping key is a domain-separated subkey of the workspace KEK (HKDF-SHA256), so the raw KEK is never used directly as an encryption key.
• Workspace KEK: 32 bytes, KMS-wrapped under your Customer-Managed Key (CMK). The plaintext KEK never touches disk.
• Governance metadata (memory ids, timestamps, tenant_id_hash, wrapped-DEK blobs) is plaintext at rest — full database page encryption via SQLCipher is not shipped.
• Crypto-erasure: DELETE /v1/memories/{id} and POST /v1/forget destroy per-row DEKs irreversibly; even with raw volume access, the affected plaintext is unrecoverable.

Honest Boundary

This is per-row DEK envelope encryption, NOT SQLCipher. Say so in docs; never market this path as "SQLCipher." The proof boundary is provable governance (deletion + signed receipts), not encryption coverage. Receipts are content-free: they attest authorship of a governed operation, not that copies or backups held elsewhere were also erased.

Key Custody Options

AWS KMS (Production BYOK)

The workspace KEK is stored as a KMS-wrapped ciphertext blob on disk and unwrapped on-demand:

Your AWS Account
├─ Customer-Managed CMK (your root key)
│  └─ Wraps the workspace KEK (32-byte plaintext never touches disk)
└─ EC2/ECS instance with minimal IAM policy
   └─ kms:Encrypt, kms:Decrypt, kms:DescribeKey only

Kill-switch model:

• Per-memory erasure: DELETE /v1/memories/{id} and POST /v1/forget destroy the per-row DEK (immediate, no CMK action needed).
• Workspace erasure: POST /v1/admin/erase-tenant is the whole-tenant kill switch. In the default shared-CMK / BYOK posture (SHOMEI_KMS_PER_TENANT=shared), it destroys the local KMS-wrapped KEK blob and zeroizes the in-process key cache — it does not schedule your CMK for deletion (deleting a shared CMK would brick every other tenant). The CMK is only scheduled for deletion (AWS ScheduleKeyDeletion, with its mandated 7–30 day PendingDeletion window) when the service owns a per-tenant CMK it minted itself (SHOMEI_KMS_PER_TENANT=auto).
• The instance role does NOT need kms:ScheduleKeyDeletion for BYOK. Tenant erasure crypto-disables the workspace via the local wrapped blob; deciding to destroy your CMK is a separate, deliberately-authorized admin action that the data-plane process should never be able to trigger.

Honest custody boundary. destroy_root / destroy_local is an at-rest control: it governs cold unwrap of the KEK, not a key already unsealed and held live in a process. Per-row DEK crypto-erasure is the granular control; the workspace kill switch is the whole-workspace root nuke. Under a CMK that is scheduled for deletion, the key is recoverable via KMS CancelKeyDeletion until the AWS window elapses; only after that is it irreversible.

File KEK Provider (Dev Only)

For local development without AWS:

SHOMEI_KEK_PROVIDER=file       # generates/persists a 32-byte key under SHOMEI_DATA_DIR
SHOMEI_ALLOW_DEV_CUSTODY=1     # required to start

• The key file must hold exactly 32 bytes with no group/other permission bits; reads use O_NOFOLLOW and validate the file on the same descriptor to close symlink-redirect / TOCTOU windows.
• destroy_root overwrites-then-unlinks the file (best-effort; not a hardware-erasure primitive).
• Restart survival: the key persists on disk and reattaches on process restart.

In-Memory KEK Provider (Tests Only)

SHOMEI_KEK_PROVIDER=inmemory

• Random 32-byte key generated per process, held in a zeroizable bytearray.
• Ephemeral: receipt signers change on restart, pinned keys become stale.
• destroy_root zeroizes the key in memory.
• Dev/test only — do not use in production.

Both file and inmemory are dev-custody providers. The service refuses to start on either (or on a weak/short/default SHOMEI_COMMIT_SECRET) unless SHOMEI_ALLOW_DEV_CUSTODY=1 is explicitly set. Dev custody is dev-only; it is not a production posture.

Deployment Topology

┌─ Your VPC / Account ──────────────────────────────────────┐
│                                                           │
│  Shomei Memory Service                                   │
│  (Docker / systemd)                                      │
│  ├─ Per-tenant SQLite DBs (per-row DEK-sealed)           │
│  │  └─ SHOMEI_DATA_DIR (encrypted volume)               │
│  │                                                       │
│  └──→ AWS KMS (Decrypt/Encrypt)                          │
│       └─ Your Customer-Managed CMK                       │
│           └─ Workspace KEK wrapped                       │
│                                                          │
│  Egress: signed receipts only (no PII, no keys)          │
│  ┌─→ S3 / audit / transparency log (optional)            │
│  └─→ Your downstream / DPO                              │
│                                                          │
└──────────────────────────────────────────────────────────┘

Environment Variables

The canonical prefix is SHOMEI_. Legacy MOCHI_* names are still honored as a fallback for backward compatibility (SHOMEI_* wins if both are set), but new deployments should use SHOMEI_*. Any secret variable also accepts a <NAME>_SSM form, which resolves an AWS SSM SecureString parameter (KMS-decrypted at boot) — prefer the *_SSM form for all secrets.

Core (Production)

Embedder

Graph-edge proposer

For a zero-model-cost entity-graph seed, also keep SHOMEI_EXTRACTOR_PROPOSER=deterministic and SHOMEI_EMBEDDER=hash; see Entity Memory Graph demo seeding.

No-Resurrection Ledger

Custody Version

AWS KMS Setup (BYOK)

1. Create a Customer-Managed CMK

In your AWS account (the region where the service runs):

aws kms create-key \
  --description "Shomei Memory workspace KEK root" \
  --region us-east-1

# Create an alias for convenience:
aws kms create-alias \
  --alias-name alias/shomei-workspace \
  --target-key-id <cmk-id> \
  --region us-east-1

2. Grant the Service Instance Role Minimum Permissions

The EC2/ECS instance role needs only wrap and unwrap:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "arn:aws:kms:us-east-1:111122223333:key/12345678-1234-1234-1234-123456789012"
    }
  ]
}

If using SHOMEI_EMBEDDER=bedrock for semantic embeddings, also grant:

{
  "Effect": "Allow",
  "Action": ["bedrock:InvokeModel"],
  "Resource": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"
}

3. Do NOT Grant kms:ScheduleKeyDeletion to the Service Role

Under the default shared-CMK / BYOK posture, the service never schedules your CMK for deletion — tenant erasure works by destroying the local KMS-wrapped KEK blob, so kms:Encrypt / kms:Decrypt / kms:DescribeKey are sufficient. Destroying the CMK itself is the ultimate kill switch and must be a deliberate, separately-authorized admin action; never let the data-plane process destroy your root key. Per-memory erasure (crypto-erase of per-row DEKs) is decoupled and requires no CMK action at all.

4. Use IMDSv2 Only

On the EC2 instance, enforce IMDSv2 with a hop limit of 1 (disable IMDSv1):

aws ec2 modify-instance-metadata-options \
  --instance-id <id> \
  --http-tokens required \
  --http-put-response-hop-limit 1

5. Store Secrets in SSM SecureString

Instead of plaintext env files:

aws ssm put-parameter \
  --name /shomei/prod/api-keys \
  --type SecureString \
  --value "key1:tenantA,key2:tenantB" \
  --key-id arn:aws:kms:us-east-1:111122223333:key/... \
  --region us-east-1

aws ssm put-parameter \
  --name /shomei/prod/commit-secret \
  --type SecureString \
  --value "$(openssl rand -hex 32)" \
  --key-id arn:aws:kms:... \
  --region us-east-1

Then point the service to these parameters:

SHOMEI_API_KEYS_SSM=/shomei/prod/api-keys
SHOMEI_COMMIT_SECRET_SSM=/shomei/prod/commit-secret

The instance role needs ssm:GetParameter scoped to /shomei/prod/*:

{
  "Effect": "Allow",
  "Action": ["ssm:GetParameter"],
  "Resource": "arn:aws:ssm:us-east-1:111122223333:parameter/shomei/prod/*"
}

A complete Terraform module (CMK, IAM, SSM, compute, user-data) ships at deploy/shomei-memory/terraform/.

Docker Compose Quickstart

# 1. Copy the example config
cp deploy/shomei-memory/.env.example deploy/shomei-memory/.env

# 2. Edit .env: set your BYOK CMK, API keys, and commit secret
# (or use SSM secret parameters: SHOMEI_API_KEYS_SSM, SHOMEI_COMMIT_SECRET_SSM)
nano deploy/shomei-memory/.env

# 3. Build and start
docker compose -f deploy/shomei-memory/docker-compose.yml up -d --build

# 4. Check health
curl -fsS http://127.0.0.1:8088/v1/healthz

The compose file:

• Builds the service from the repo root (context: ../..)
• Mounts a persistent volume at /var/lib/shomei-memory for tenant DBs and KEK material
• Publishes port 8088 on loopback only (127.0.0.1:8088:8088) — never to the internet
• Runs with no-new-privileges:true
• Checks /v1/healthz every 30 seconds

Systemd Deployment (VM)

For a non-Docker VM (e.g., EC2):

# 1. Install Python 3.10+ and dependencies (the project floor is >=3.10)
sudo apt-get install python3.10 python3.10-venv

# 2. Create venv and install (the [aws] extra pulls in boto3 for KMS/SSM/S3 paths)
# NB: use .venv (with the leading dot) — the shipped systemd unit's ExecStart
# points at /opt/shomei-memory/.venv/bin/python.
python3.10 -m venv /opt/shomei-memory/.venv
/opt/shomei-memory/.venv/bin/pip install -e '.[aws]'

# 3. Create config directory and env file
sudo mkdir -p /etc/shomei-memory
sudo touch /etc/shomei-memory/service.env
sudo chmod 600 /etc/shomei-memory/service.env

# 4. Fill in env vars (use SSM parameters for secrets)
sudo tee /etc/shomei-memory/service.env >/dev/null <<EOF
SHOMEI_API_KEYS_SSM=/shomei/prod/api-keys
SHOMEI_KEK_PROVIDER=aws-kms
SHOMEI_KMS_KEY_ID=arn:aws:kms:us-east-1:111122223333:key/...
SHOMEI_AWS_REGION=us-east-1
SHOMEI_COMMIT_SECRET_SSM=/shomei/prod/commit-secret
SHOMEI_EMBEDDER=bedrock
SHOMEI_BIND_HOST=127.0.0.1
SHOMEI_PORT=8088
SHOMEI_TLS_CERT=/etc/shomei-memory/tls/cert.pem
SHOMEI_TLS_KEY=/etc/shomei-memory/tls/key.pem
EOF

# 5. Install systemd unit
sudo cp deploy/shomei-memory/systemd/shomei-memory.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now shomei-memory

# 6. Check status
sudo systemctl status shomei-memory
curl -fsS https://localhost:8088/v1/healthz --insecure  # or via your load balancer

Setting both SHOMEI_TLS_CERT and SHOMEI_TLS_KEY makes the service speak HTTPS on SHOMEI_PORT. There is no separate HTTPS port; TLS rides on whatever SHOMEI_PORT is set to.

HTTP API Endpoints

All routes require a bearer token (Authorization: Bearer <api-key>) mapping to a tenant, except /v1/healthz and the self-serve signup routes. Destructive and control-plane routes additionally require an admin-scoped key (a data-scoped key gets 403). This table covers the deployment-relevant surface; see the HTTP API reference for the complete route list and response shapes.

Admin-scoped routes require an API key issued with admin scope. Keys embedded in client apps should be data-scoped only (they receive 403 on destructive/control-plane actions).

Verifying Deployment

1. Liveness

curl -fsS http://127.0.0.1:8088/v1/healthz
# -> {"status": "ok"}

2. Governance Smoke Test

Run the customer acceptance test (black-box: governed add/recall, crypto-erasure, legal hold, Art.18 restriction, subject export/forget, tenant isolation, offline receipt verification):

python acceptance/customer_acceptance.py

3. Verify a Deletion / Forget Receipt Offline

When you call DELETE /v1/memories/{id} the response includes a signed_receipt; POST /v1/forget returns per-memory proofs under signed_receipts (a list of {id, signed_receipt}) plus a subject-level signed_closure_receipt when a commitment key is configured. Every signed-receipt envelope has exactly these fields:

{
  "schema": "shomei.vector_delete.v1",
  "receipt_hash": "sha256:<hex over canonical body>",
  "signer_id": "shomei-receipt:<keyed tenant id>",
  "public_key_hex": "<ed25519 pubkey hex>",
  "signature_hex": "<ed25519 signature hex>",
  "body": { ... content-free receipt body ... }
}

Verify offline with the open-source verifier. It takes a parsed receipt dict (not a JSON string), and the authentication gate is authenticated (signature consistent and matched against the key you pinned), not valid alone:

from shomei_memory_verify import verify_signed_receipt
import json

receipt = json.load(open("signed_receipt.json"))   # the signed_receipt object, parsed
result = verify_signed_receipt(
    receipt,
    expected_public_key_hex="<your-pinned-tenant-key>",
)
print(result)  # VerifyResult(valid=True, authenticated=True, reason='ok', ...)

Or from the command line:

python -m shomei_memory_verify signed_receipt.json --pin <your-pinned-tenant-key>
# exit 0 only when authenticated against the pinned key

Pin the tenant's public key out-of-band: retrieve it from the startup log line or call GET /v1/account/signer-key once (returns {"tenant": ..., "signer_public_key_hex": ...}) and store it securely.

Hardening Checklist

• [ ] Custody: SHOMEI_KEK_PROVIDER=aws-kms with a strong SHOMEI_COMMIT_SECRET (≥16 chars or use _SSM). The service refuses to start on a dev-custody provider or the dev commit-secret default unless SHOMEI_ALLOW_DEV_CUSTODY=1 is set — never set that flag in production.
• [ ] IAM: Instance role has kms:Encrypt, kms:Decrypt, kms:DescribeKey only. No kms:ScheduleKeyDeletion. Also grant ssm:GetParameter (scoped) if using *_SSM secret parameters, and bedrock:InvokeModel (scoped) if using the bedrock embedder.
• [ ] Network: IMDSv2 enforced (hop limit 1), no public SSH, security group allows only your ingress. The service port is never internet-facing.
• [ ] TLS: Terminated here via SHOMEI_TLS_* (both cert and key set) or at an ingress/ALB; enforce HTTPS client-side.
• [ ] Storage: SHOMEI_DATA_DIR on an encrypted volume (EBS with encryption, etc.). Treat backups as secret material.
• [ ] Ledger separation: The no-resurrection forbidden ledger (<db>.forbidden.db) must not be included in per-tenant data backups — a whole-volume restore would roll it back. Back it up separately or use S3 Object Lock (SHOMEI_FORBIDDEN_LEDGER=s3) for restore immunity.
• [ ] Secrets: All secrets via SSM SecureString (*_SSM vars), never plain env files. Prevent SHOMEI_API_KEYS and SHOMEI_COMMIT_SECRET from appearing in /proc/<pid>/environ.
• [ ] Monitoring: Alert on any non-200 from /v1/healthz.
• [ ] Strict embedder: If using bedrock or ollama, keep SHOMEI_EMBEDDER_STRICT=1 so a backend outage fails loud (503) instead of silently degrading to non-semantic hash recall.
• [ ] (Optional) Read-only root filesystem: Run the container with a read-only root + tmpfs /tmp for additional defense.

What Leaves Your Environment

Only receipts — content-free, signed attestations of add/delete/forget/restrict/export operations. No memory content, no embeddings, no raw subject identity, no keys. Receipts prove authorship of the governed operation; they do not prove that copies or backups held outside this deployment were erased.

Egress destinations (optional, customer-controlled):

• Audit / transparency log (if enabled)
• DPO / data subject (subject access export)
• Downstream systems (via webhook, if configured)

Verify receipts offline with shomei_memory_verify and a pinned signer key.

Verifiability vs. attestation tier (honest scope). Today's signed receipts are operator-verifiable (Tier-2): you cryptographically verify that this operator authored the deletion/restriction/export, and the offline verifier checks the signature against the pinned signer key. They do not yet prove, against a dishonest operator, that the operator's enumeration of a subject's records was complete — that completeness-under-adversary guarantee requires the measured-enclave (TEE) attestation track, which is roadmap, not shipped. Key custody is the proof boundary: the per-tenant signer is derived from your own KEK, so no shared secret can forge another tenant's receipts.

Scaling Notes

• Single-writer SQLite per tenant → single-node, vertical scale. One service instance owns a tenant's DB file.
• Horizontal scale = container-per-tenant or sharded deployments (e.g., tenant foo → service instance A, tenant bar → service instance B).
• Do not imply elastic horizontal scale (no shared-DB cluster without explicit refactoring).

Receipt Signing & Custody Versions

By default (v2 custody, SHOMEI_CUSTODY_VERSION=v2-kek-hkdf), each tenant's receipt signer is derived from that tenant's own KEK with domain separation (HKDF-SHA256). One tenant's KEK roots only that tenant's receipts — no single shared secret forges every tenant's proofs.

Pin the receipt signer's public key (hex):

• Logged at startup, one line per tenant: [shomei] receipt signer (<src>) tenant=<t> public_key=<hex>
• Also available from GET /v1/account/signer-key (auth required)
• Pass it to python -m shomei_memory_verify <receipt> --pin <key-hex> to verify receipts offline

Legacy v1 custody (shared-signer derivation across all tenants) is available via SHOMEI_CUSTODY_VERSION=v1-commit-secret but deprecated; v2 is the default. Under v2, pinning an explicit SHOMEI_RECEIPT_SIGNING_KEY is forbidden — the per-tenant derived signer is used instead.