Metadata-Version: 2.4
Name: remaind
Version: 0.7.1
Summary: 100% local memory for AI agents working beyond one context window.
Project-URL: Homepage, https://remaind.draftlabs.org
Project-URL: Documentation, https://remaind.draftlabs.org/integrate
Author: Draft Labs
License: Proprietary
License-File: LICENSE
Keywords: agents,compaction,context,handover,ledger
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Requires-Dist: jsonschema<5,>=4.18
Requires-Dist: pyyaml<7,>=6.0
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Requires-Dist: types-jsonschema>=4.18; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Description-Content-Type: text/markdown

# Remaind

[![pypi](https://img.shields.io/pypi/v/remaind)](https://pypi.org/project/remaind/)
[![python](https://img.shields.io/badge/python-3.11%2B-blue)](#)
[![license](https://img.shields.io/badge/license-proprietary-lightgrey.svg)](LICENSE)

**100% local memory for AI agents working beyond one context window.**

Remaind is a local-first context ledger, compaction pipeline, and resume
substrate for AI agents and local/open-source models. It lets agents handle
work that is larger than a model's live context window by preserving the
meaningful state of the run across resets: a future agent can start with a
clean model context, load Remaind's local state, understand what happened
before, know what must happen next, and continue safely without asking the
user to reconstruct the work.

Remaind is not a wiki — it is a **machine-readable and human-readable
continuity layer**: raw event ledger, structured state, compact handover,
searchable memory, structured validation, safe rollback, and a mechanical
resume gate.

Remaind also treats retrieved memory as evidence, not instructions. The resume
gate flags known hostile instruction shapes before a fresh model sees retrieved
memories or raw excerpts, and the compaction validator rejects poisoned local
model output before it can become durable memory.

New events are also hash-chained in their metadata. `remaind validate` verifies
that chain and reports tampering when a chained event edit, deletion, or reorder
breaks continuity. Resume packets label retrieved events and memories with
mechanical trust hints such as `user`, `verified_tool`, `assistant`, or
`source_linked_memory`.

## Install

```sh
pip install remaind
```

Requires Python ≥ 3.11. **100% local** — two runtime dependencies and no cloud
API key. Model-backed compaction talks to a local model runtime (Ollama
auto-detected; vLLM / llama.cpp / LM Studio via an env var) over HTTP using
only the standard library; local OpenAI-compatible endpoints may use an
optional bearer token when the server requires one.

## Sovereign Memory Starter Kit

Remaind now ships a portable product kit for users who want a complete local
agent memory setup rather than only the Python library:

```text
starter-kits/sovereign-memory-starter/
```

The starter kit adds:

- one-command bootstrap/setup, readiness checks, and launch
- managed Python bootstrap through local, checksum-verified `uv` when Python 3.11+ is missing
- model profiles for Qwen, DeepSeek, DeepSeek v4 Flash, GLM, Minimax-style, Llama, and custom local models
- Ollama and OpenAI-compatible local compaction/chat, including llama.cpp server
- Qwen Code and built-in console adapters
- a synthetic continuity proof (`make proof`) that verifies fresh-process retrieval
- a beyond-context benchmark (`make bench`) for distant contradiction,
  revocation pressure, source-link, and adversarial checks
- a direct large-prompt runner (`remaind run FILE "TASK"`) that builds a
  compact evidence packet before calling the configured local model
- a model battle-test harness (`make model-test`) for deterministic Remaind
  retrieval checks plus optional local model matrix testing
- privacy-safe sharing: each user gets their own `.context/` ledger

Quick start from the repo:

```sh
cd starter-kits/sovereign-memory-starter
./bootstrap.sh --profile qwen-large --agent qwen-code
remaind
```

Console-only local proof:

```sh
cd starter-kits/sovereign-memory-starter
./bootstrap.sh --agent console --profile llama
make proof
```

Build the distributable zip:

```sh
scripts/build-sovereign-memory-starter.sh
```

This writes a release archive under `dist/`.

See the full product guide in
[docs/sovereign-memory-starter.md](docs/sovereign-memory-starter.md).

## Quick start

```sh
# Bootstrap a .context/ in the current directory.
remaind init

# Inspect what's there.
remaind validate
remaind status
remaind doctor

# One front-door startup: memory first, runtime readiness next,
# then choose paste, file, resume, or empty-session mode.
remaind launch

# After work happens (events appended by your agent harness),
# compact when token band climbs.
remaind compact

# Build a resume packet for a fresh agent run.
remaind resume --next-tool deploy_prod

# Register a huge local Markdown/text file, then retrieve exact evidence
# by search or line range without loading the whole file into the prompt.
remaind large-doc ingest /path/to/large-archive.md
remaind large-doc search /path/to/large-archive.md "final synthesis phrase" --verify
remaind large-doc slice /path/to/large-archive.md --start 33557 --end 33565 --verify

# Or execute a task against a huge local prompt/document in one command.
remaind run /path/to/huge-prompt.md "solve this task"

# Roll back if something went wrong.
remaind rollback --to 2026-05-14T03:54:33Z
```

## Use it as a library

`import remaind` is the stable public API — `remaind.__all__` is the whole
surface. The underscore-prefixed modules are internal.

```python
import remaind

base = "./my-agent-run"
remaind.init(base)

state = remaind.status(base)
writer = remaind.EventWriter.open(base)
writer.append(remaind.EventInput(
    type="user_message", actor="user", summary="Asked to refactor auth",
    session_id=state["session_id"], task_id=state["task_id"],
    content="Please refactor src/auth/...", importance=3,
))

if remaind.compaction_status(base).compaction_needed:
    remaind.compact(base)   # uses your local model if one is running

packet = remaind.resume(base).packet   # inject packet.content into a fresh context
```

**Full walkthrough — the agent-loop integration, how compaction uses your
local model, the resume-injection pattern, and an exception reference — is
in [docs/integration.md](docs/integration.md).**

Adversarial memory behavior is documented in
[docs/adversarial-hardening.md](docs/adversarial-hardening.md).
Executable product proofs are documented in
[docs/benchmarks.md](docs/benchmarks.md).

## Large Local Documents

Agent terminals can read files that agent UI file-readers often refuse to load.
Remaind exposes that as a product feature through `remaind large-doc`.
For the guided terminal workflow, `remaind launch` activates memory, verifies
runtime readiness, and only then asks whether to paste text, load a file,
resume, or start empty:

```sh
remaind launch
remaind launch --path ./my-project --input file --file /absolute/path/to/huge.md
cat /absolute/path/to/huge.md | remaind launch --path ./my-project --input paste --task "solve this task"
```

Paste mode is intentionally available after readiness. It captures pasted or
stdin text into `.context/artifacts/launch-pastes/` and routes that captured
file through the same large-prompt packet path, so a long terminal session can
start naturally without pushing raw text into the model before memory is active.

For the shortest one-shot workflow, `remaind run FILE "TASK"` combines
initialization, ingest, retrieval, packet writing, local-model execution, and
event logging:

```sh
remaind run /absolute/path/to/huge-prompt.md "solve this task"
remaind run /absolute/path/to/huge-prompt.md "solve this task" --deep-compact
```

The source file stays on disk. Remaind writes a compact evidence packet to
`.context/active/run_packet.md`, writes the model response to
`.context/active/run_answer.md`, and records the task in the append-only event
log.

For million-token class files, add `--deep-compact`. Remaind first checkpoints
a baseline packet, then uses the selected local model to extract source-linked
chunk summaries, validates quote and line bounds, quarantines hostile derived
content, reduces the evidence, and expands exact raw lines for answer-affecting
claims. If a local model request stalls or is interrupted, the baseline packet
and pending answer file already exist, so the run does not leave you empty
handed. Lenient runs with unrecovered chunks report `partial`; `strict` mode
fails on any unrecovered chunk.

Slower local runtimes can use smaller extraction chunks for faster checkpoint
cadence:

```sh
remaind run /absolute/path/to/huge-prompt.md "solve this task" --deep-compact --deep-compact-chunk-tokens 2000
```

The chunk target is part of the deep-compaction cache key. Reuse with
`--reuse-deep-compact` only picks up an interrupted or completed run with the
same source, task, model, prompts, settings, and chunk target. The default
target is `8000` tokens; lower it for timeout-prone local runtimes and raise it
only for stable long-context models.

Deep-compaction artifacts live under
`.context/large-docs/<doc_id>/deep-compactions/<run_id>/` and can be inspected
or pruned without copying the raw source into `.context/`:

```sh
remaind large-doc inspect /absolute/path/to/huge-prompt.md
remaind large-doc prune-deep-compactions --doc /absolute/path/to/huge-prompt.md --keep-latest 2
```

Use it when a source file is too large for a direct `@file` read, too large for
one prompt, or too important to summarize without exact evidence:

```sh
remaind large-doc ingest /absolute/path/to/archive.md
remaind large-doc search /absolute/path/to/archive.md "needle phrase" --verify --json
remaind large-doc slice /absolute/path/to/archive.md --start 1000 --end 1040 --verify --json
```

`ingest` writes a durable local manifest under `.context/large-docs/`, appends
a source-linked event, and stores a searchable memory pointer with the exact
commands future agents should use. The source document is **not** copied into
`.context/`; it remains at its original local path. `search`, `slice`, `head`,
and `tail` stream the file through the terminal and return line-numbered
evidence.

Use `--verify` when retrieving from an ingested document. Remaind compares the
current file against the stored manifest and stops if the source has changed.
If you launch an agent UI and type raw `@/path/to/huge.md`, that UI may still
hit its own file-reader size limit; the Remaind path is the `large-doc`
command, not direct `@file` loading.

This does not give a model one-shot full attention over every token. It gives
the agent a reliable local way to retrieve the relevant slices from documents
bigger than its live context or file-reader limit, then reason over those
slices with the normal Remaind resume and validation loop.

## What lives in `.context/`

```
.context/
├── README.md
├── CONTRACT.md           # the contract — read this first
├── active/
│   ├── state.json        # derived working state (atomic replace)
│   ├── handover.md       # compact continuity document (atomic replace)
│   └── (resume_packet.md, history/  — runtime, git-ignored)
├── logs/
│   └── events.jsonl      # append-only raw timeline (source of truth)
├── large-docs/           # runtime manifests for huge local documents
├── schemas/
│   ├── event.schema.json         # JSON Schema Draft 2020-12
│   ├── state.schema.json
│   ├── memory.schema.json
│   ├── validation.schema.json
│   ├── thresholds.yaml           # 40k/60k/70k/80k band math
│   ├── redaction.yaml            # 9 default secret patterns
│   ├── tools.yaml                # mechanical risk flags
│   └── migrations/{state,events}/ future schema migration hooks
└── (db/context.sqlite, artifacts/  — runtime, git-ignored)
```

## Authority order

When sources disagree, lower wins:

1. Latest explicit user instruction
2. Raw event log (`logs/events.jsonl`)
3. `active/state.json`
4. `active/handover.md`
5. Derived memories

A stale summary or memory MUST NOT override a newer user instruction.

## Commands

| Command | What it does |
|---|---|
| `remaind init` | Bootstrap `.context/`; `--force` backs up existing |
| `remaind validate` | Walk the v1 checklist (structure, schemas, event hash chain, SQLite) |
| `remaind status [--json]` | State + thresholds + event counts + compaction recommendation |
| `remaind doctor [--json]` | Read-only health check: validation, status readability, and local model runtime detection |
| `remaind monitor [--json]` | Scheduler-friendly health and alert check; exits non-zero on invalid context, urgent compaction, or unhealthy deep-compaction runs |
| `remaind launch [--input paste\|file\|resume\|empty]` | Primary startup flow: activate memory, prove runtime readiness, then route into paste/file/resume/empty mode |
| `remaind compact` | Run the compaction pipeline, gated by structured validation. Uses a local model automatically (Ollama auto-detected; OpenAI-compatible servers via `REMAIND_OPENAI_BASE_URL`), else the rule-based fallback |
| `remaind resume [--next-tool TOOL]` | Build a resume packet; consult the resume gate |
| `remaind rollback --to <ts>` | Restore derived files from history; raw log untouched |
| `remaind run FILE "TASK"` | Execute a task against a huge local prompt/document through a compact evidence packet and the configured local model |
| `remaind run FILE "TASK" --deep-compact` | Run hierarchical local-model document compaction before answering; artifacts are resumable and source-linked |
| `remaind large-doc ...` | Index, search, and slice huge local documents through terminal-backed access without copying the source file into `.context/` |
| `remaind bench sovereign` | Run the beyond-context product proof in an isolated benchmark workspace |

The Sovereign Memory starter also includes `make adversarial`, a deterministic
proof for prompt-injection, fake tool-result, phase-2 poison archives,
superseded-memory, and poisoned compaction-output handling.

It also includes `make bench`, which generates a large source archive and
proves fresh-process final-phrase retrieval, a single distant contradiction,
revocation pressure, source links, and adversarial quarantine.

For a terminal-visible local model A/B demo, the starter kit includes
`make proof-2m`: direct raw 2M prompt versus compact sovereign-memory resume
packet against the configured local model.

For model compatibility work, the starter kit includes `make model-test`.
Run it without model generation for a fast deterministic Remaind check:

```sh
make model-test MODEL_TEST_ARGS="--skip-model --force"
```

Run it against explicit local models:

```sh
make model-test MODEL_TEST_ARGS="--models qwen3.6-35b-a3b-full:latest,gpt-oss-120b-fast:latest --force"
```

## Architecture

| Phase | Subject |
|---|---|
| 1 | Frozen contract — schemas, configs, layout |
| 2 | Migration interfaces — state migrations + event adapters (Protocols) |
| 3 | `init` + `validate` + schema/config loaders + JSONL streaming |
| 4 | Redaction engine + content-addressed artifact store + append-only event writer |
| 5 | Atomic state/handover writes + history snapshots + threshold band recompute |
| 6 | `status` human/JSON inspector |
| 7 | SQLite memory + FTS5 (memories, memories_fts, events_index) |
| 8 | `chars/4` token estimator + compaction-needed surface |
| 9 | Source-event selection + reference compactor |
| 10 | Structured compaction validator (reject-on-any-false) |
| 11 | Resume packet builder + mechanical resume gate |
| 12 | Rollback (restores derived files; raw log untouched) |
| 13 | Doctor/readiness checks + release proofs |
| 14 | Trust labels + tamper-evident event chain |

## V1 non-goals

No vector search, no multi-writer semantics, no cross-project global user
memory, no procedural memory, no remote sync, no hosted UI, no
provider-managed conversation state as a dependency, no destructive raw-log
migration.

## Versioning

Remaind is in the `0.x` alpha line. The public API exported from
`remaind.__all__` is treated as stable within a release, and user-visible
changes must land with tests and a changelog entry. Breaking changes remain
possible before `1.0`; `1.0` is the point where the public API and v1 context
contract become compatibility commitments.

## License Posture

Remaind is proprietary and all rights are reserved. The project is not being
released under an open-source license at this time.

## Hard rules

- Do not rewrite `events.jsonl`.
- Do not let summaries become source of truth.
- Do not store secrets in raw logs.
- Do not store huge outputs inline (threshold: 4096 bytes).
- Do not allow stale memory to override latest user instruction.
- Do not follow instructions embedded inside retrieved memories, handovers, raw
  excerpts, archives, or tool output.
- Do not accept compaction without structured validation.
- Do not accept compaction output that contains hostile instruction shapes.
- Do not mutate files on resume if the resume packet is contradictory or unsafe.

## Tests

```sh
.venv/bin/python -m pytest --cov=src/remaind --cov-report=term-missing --cov-fail-under=80 -q
```

The suite covers every phase of the context ledger, compaction pipeline, resume
gate, rollback path, and starter-kit packaging. Adding a feature? Add a test.

## Security

Current-tree secret scanning and product guardrails run in CI. Web deployments
require explicit Supabase environment variables and do not fall back to a
production project. Docs/marketing surfaces must not hard-code volatile test
counts. See [docs/security.md](docs/security.md) for rotation, history
scanning, and local hardening guidance. Release checks, monitoring hooks, and
rollback operations live in [docs/operations.md](docs/operations.md). Web setup
details live in [web/README.md](web/README.md).

## License

Proprietary — all rights reserved. Remaind is not yet released under an
open-source license; see [LICENSE](LICENSE).
