Skip to content
atif-sql

Contract map

View as MarkdownOpen in ChatGPTOpen in ClaudeOpen in Claude CodeOpen in Cursor

atif-sql · Contract map

Section titled “atif-sql · Contract map”

What counts as a contract here. This workspace has almost no shared-type imports to trace, because pyproject.toml:605-614 forbids most of them: the independent members may never import each other, and atif-analytics may import only atif-models. What crosses a module boundary instead is a shape agreed by two packages that cannot reference one another’s symbols: a file layout, a JSON key order, a column projection, a typing.Protocol typed to a document rather than to an implementation. So a contract in this file is any of:

  1. a typing.Protocol declared in one package’s domain/ports.py and satisfied by an adapter elsewhere (each has a section below or a bullet under “Other contracts”);
  2. a shape declared twice or more in packages that cannot import each other, where the two declarations must agree or a query silently returns wrong rows;
  3. a build-enforced dependency rule (import-linter), which is a stronger fact than a convention;
  4. an upstream API this workspace depends on and pins;
  5. a version pin between workspace members, which the release tooling rewrites and checks on every bump.

Every Protocol here is satisfied STRUCTURALLY, with no import in either direction. That is the single most important structural fact for reading this file. RealConverter (packages/atif-cli/src/atif_cli/converter_adapter.py:80) implements ConverterPort (packages/atif-corpus/src/atif_corpus/domain/ports.py:101) by shape alone; it names the Protocol only in its docstring, and the Protocol can’t name it back, because atif-corpus may never import atif-converter. The Protocol name appears at only these import sites in the whole workspace: packages/atif-corpus/src/atif_corpus/application/materialize.py:172 (the module that CALLS it), packages/atif-cli/tests/test_converter_adapter.py:17, and packages/atif-cli/tests/test_codex_cli.py:26 (the tests that bind the two together in an annotation). So each Protocol row below has a producer of the shape and an implementer that never imports it; the type system meets the two only at a typed site (an annotated test binding, a factory’s annotated return, a call into a port-typed parameter), and the answer to “who finds out if the shape drifts” is one of those sites or nobody. Do not read a Protocol row as a dependency edge: pyproject.toml:605-608 forbids the edge it would imply.

Every contract below names its producer, its consumers, the verbatim shape, the assumptions consumers make beyond the shape, and the drift risk. Consumer counts are grep-derived and confirmed at each import, annotation, or call site: this repo has no code index (.gitignore:25 lists .codegraph/, and no index exists on disk), so no count here comes from a symbol graph. These grep hazards shape every count: names collide (DomainError and EmbeddingProviderMismatch across packages, cached_tokens across coordinate spaces), so every attribution here is by module path and never by bare name; and outside four atif-cli modules (listed under the import-linter contracts) every cross-package import under src/ is indented inside a function body or a TYPE_CHECKING block, so a line-anchored grep of src/ finds almost nothing.

The sections below aren’t ranked; “Other contracts” at the end collects the smaller ones.

The materialized corpus artifact layout

Section titled “The materialized corpus artifact layout”

Producer: packages/atif-corpus/src/atif_corpus/domain/layout.py:18-24

Consumers:

Shape:

<corpus_root>/                     # default: ~/.atif-sql/corpus/<corpus-slug>/
  sessions/<session_id>/
    trajectory.json.zst            # compact JSON (separators=(',',':')), ATIF-v1.7, one zstd frame
    loss_report.json               # atif_converter LossReport.to_json()
    edges.jsonl.zst                # one line per RAW record: {uuid, parent_uuid,
                                   #  message_id, type, ts, is_sidechain,
                                   #  is_compact_summary, source_file, tool_use_ids: [..]}
    session_events.jsonl.zst       # one line per KEPT non-message record; may be empty
    meta.json                      # {session_id, source_mtime_ns, source_files: [...],
                                   #  harbor_version, converter_version, converter_schema,
                                   #  materialized_at, agent, source_present,
                                   #  source_removed_at?, source_archive?,
                                   #  columnar_schema? (old layout)}
    source/<path>.zst              # raw source archive: one zstd file per source file
  blobs/sha256/<ab>/<sha256>.<ext> # inline attachments, content-addressed, written 0444
  watermark.json                   # {path: mtime_ns} across source corpus
  empty_sessions.json              # sessions whose transcript held nothing to convert
  sink_pending.json                # sessions the SessionSink hasn't taken yet

A corpus written by an older build keeps the plain trajectory.json, edges.jsonl, and session_events.jsonl, plus five per-session parquet files, until atif-sql corpus slim converts it, and every reader accepts either spelling. The parquet files are a cache of what the views compute from the trajectory, never a source of truth, and materialize doesn’t write them: the CLI passes no artifact producer (packages/atif-cli/src/atif_cli/app.py:990-992), and the lake loader runs atif-duck’s ColumnarArtifactProducer to stage each session it loads (packages/atif-duck/src/atif_duck/infrastructure/lake.py:561). atif-duck reads a session’s own parquet files only when meta.columnar_schema matches its version and all five are present and non-empty (packages/atif-duck/src/atif_duck/infrastructure/columnar.py:116), and falls back to the trajectory per session otherwise. The names and the schema version live in packages/atif-duck/src/atif_duck/domain/columnar.py; atif-corpus never learns them, and it merges whatever keys an ArtifactProducer returns into meta.json.

Assumptions consumers make:

The hand-written contract and the manifests split the job. Its CLI heading (docs/CONTRACT.md:300) names atif-cli the package that composes the others. The atif-cli manifest declares five siblings (packages/atif-cli/pyproject.toml:32-36) and the CLI imports every one of them; atif-models reaches it through atif-analytics, which imports atif-models (packages/atif-analytics/pyproject.toml:23) as the import-linter contracts permit. docs/CONTRACT.md:17-19 leaves VSS/semantic_search and the analytics pipelines to docs/CONTRACT-V2.md rather than ruling them out; both are shipped commands (the atif-analytics and atif-embed pins at packages/atif-cli/pyproject.toml:32 and :36, plus the analyze, embed, and search commands at packages/atif-cli/src/atif_cli/app.py:1725, :1893, and :2123). Read docs/CONTRACT.md:22-60 as authoritative for the layout (that’s the part the packages actually implement) and the manifests as authoritative for who imports whom.

Drift risk: a new artifact, a renamed file, or a new meta.json key must be applied in every place above, and most of them are reader-side, so an addition silently reaches nobody. The file names and the .zst suffix are pinned across all four packages by packages/atif-cli/tests/test_artifact_names_twin_pin.py:29 and :19; a new meta.json key has no such pin. Mitigation: the writer-side constants at packages/atif-corpus/src/atif_corpus/domain/layout.py:18-24 are the single source of truth on the write side. Any layout change starts there and then greps the reader modules named above.

The static SQL catalog

Section titled “The static SQL catalog”

Producer: packages/atif-duck/src/atif_duck/domain/catalog.py:30 (VIEW_NAMES), with VIEW_SCHEMA:59, MACRO_NAMES:324, MACRO_SIGNATURES:343, ANALYTICS_VIEW_NAMES:370, ANALYTICS_VIEW_SCHEMA:384, ANALYTICS_MACRO_SIGNATURES:453, TABLE_MACRO_NAMES:469, DESCRIPTIONS:487

Consumers:

Shape:

VIEW_NAMES: tuple[str, ...] = (
    "sessions",
    "steps",
    "messages",
    ...
)


VIEW_SCHEMA: dict[str, tuple[tuple[str, str], ...]] = {
    "sessions": (
        ("session_id", "VARCHAR"),
        ("agent", "VARCHAR"),
        ...
    ),
    ...
}


MACRO_SIGNATURES: dict[str, tuple[str, ...]] = {
    "ago": ("interval_text",),
    "model_used": ("sid",),
    ...
}

Assumptions consumers make:

Drift risk: adding a view or macro without a DESCRIPTIONS entry, an ARG_EXEMPLARS entry for each new parameter name, or TABLE_MACRO_NAMES membership fails CI loudly: the risk is inverted here, and the real exposure is a type that only the fixture corpus produces. Mitigation: VIEW_SCHEMA["message_embeddings"] at packages/atif-duck/src/atif_duck/domain/catalog.py:281-287 hardcodes FLOAT[1024], so run the drift test against a store built at a non-default output_dimension before changing that setting.

The import-linter architecture contracts

Section titled “The import-linter architecture contracts”

Producer: pyproject.toml:553-614

Consumers:

Shape:

[tool.importlinter]
root_packages = ["atif_converter", "atif_corpus", "atif_duck", "atif_models", "atif_analytics", "atif_embed", "atif_cli"]


[[tool.importlinter.contracts]]
name = "converter / corpus / duck / models / embed are mutually independent (only atif-cli composes; atif-analytics may import atif-models only)"
type = "independence"
modules = ["atif_converter", "atif_corpus", "atif_duck", "atif_models", "atif_embed"]


[[tool.importlinter.contracts]]
name = "atif-analytics imports only atif-models among workspace packages"
type = "forbidden"
source_modules = ["atif_analytics"]
forbidden_modules = ["atif_converter", "atif_corpus", "atif_duck", "atif_embed", "atif_cli"]

Assumptions consumers make:

Drift risk: a new workspace member that is not added to root_packages (pyproject.toml:554) is silently unchecked: import-linter reports every contract kept while the new package imports whatever it likes. Mitigation: adding a packages/* member means adding it to root_packages and giving it either a layers contract or a place in the independence list in the same commit.

LlmStructuredProvider: the structured-output port

Section titled “LlmStructuredProvider: the structured-output port”

Producer: packages/atif-models/src/atif_models/domain/ports.py:131

Consumers:

Shape:

@runtime_checkable
class LlmStructuredProvider(Protocol):
    """Port: one structured-output call. One adapter per backend.


    The seam is deliberately narrow — everything provider-specific is
    fixed at adapter construction.
    """


    async def classify_structured(
        self, *, system: str, prompt: str, schema: type[SchemaT]
    ) -> SchemaT: ...

Assumptions consumers make:

Drift risk: the port is @runtime_checkable, so an isinstance check passes on method names alone and would accept an adapter with the wrong parameter kinds (positional instead of keyword-only), and the only isinstance against it is the conformance test’s (packages/atif-models/tests/test_models_ports.py:33); no production code guards on it. Mitigation: the keyword-only signature at packages/atif-models/src/atif_models/domain/ports.py:138-140 is the contract, and the factory’s annotated return at packages/atif-models/src/atif_models/infrastructure/providers.py:26 is where a mismatched adapter fails the typecheck gate.

The edges.jsonl line shape

Section titled “The edges.jsonl line shape”

Producer: packages/atif-converter/src/atif_converter/domain/edges.py:22-32

Consumers:

Shape:

#: Stable key order for one edges.jsonl line (contract-fixed shape).
EDGE_FIELDS: tuple[str, ...] = (
    "uuid",
    "parent_uuid",
    "message_id",
    "type",
    "ts",
    "is_sidechain",
    "is_compact_summary",
    "source_file",
    "tool_use_ids",
)

Assumptions consumers make:

Drift risk: the keys are declared twice in packages that cannot import each other, and a new key added on the producer side is simply invisible to EDGE_COLUMNS: a projection reader drops unknown keys without erroring. Mitigation: EDGE_FIELDS is a single tuple; changing it means editing packages/atif-duck/src/atif_duck/domain/raw_readers.py:62-72 in the same commit.

EmbeddingProvider, VectorStorePort, TextRowsPort

Section titled “EmbeddingProvider, VectorStorePort, TextRowsPort”

Producer: packages/atif-embed/src/atif_embed/domain/ports.py:37, packages/atif-embed/src/atif_embed/domain/ports.py:65, packages/atif-embed/src/atif_embed/domain/ports.py:89

Consumers:

Shape:

class EmbeddingProvider(Protocol):
    @property
    def model_id(self) -> str: ...
    @property
    def dimension(self) -> int: ...
    async def embed_documents(self, texts: list[str]) -> list[list[float] | None]: ...
    def embed_query(self, text: str) -> list[float]: ...




class VectorStorePort(Protocol):
    def table_identity(self) -> tuple[str, int] | None: ...
    def get_embedded_hashes(self) -> dict[str, str]: ...
    def delete_uuids(self, uuids: Iterable[str]) -> int: ...
    def add_chunk(self, df: pl.DataFrame) -> None: ...
    def optimize(self) -> None: ...




class TextRowsPort(Protocol):
    def iter_unembedded(
        self,
        corpus_root: Path,
        *,
        embedded: dict[str, str] | None = None,
        limit: int | None = None,
    ) -> Iterator[PendingText]: ...
    @property
    def discovery(self) -> str: ...
    def commit(self, *, stored_rows: int) -> None: ...

Assumptions consumers make:

Drift risk: dimension is read once per run and stamped on every row (packages/atif-embed/src/atif_embed/application/embed.py:178), so a provider whose width depends on the input rather than on configuration would write a store whose rows disagree with their own dim column. Mitigation: dimension is a property fixed at adapter construction from settings: packages/atif-embed/src/atif_embed/infrastructure/cohere_bedrock.py:305-308 reads it from packages/atif-embed/src/atif_embed/infrastructure/settings.py:110-114, never from a response.

ConverterPort and ConversionOutput

Section titled “ConverterPort and ConversionOutput”

Producer: packages/atif-corpus/src/atif_corpus/domain/ports.py:101 (ConverterPort) and packages/atif-corpus/src/atif_corpus/domain/ports.py:66 (ConversionOutput)

Consumers:

Shape:

@dataclass(frozen=True, slots=True)
class ConversionOutput:
    trajectory_dict: dict[str, Any]
    loss_report_dict: dict[str, Any]
    edges_lines: list[str]
    events_lines: list[str] = field(default_factory=list)
    source_archive: tuple[ArchivedSource, ...] = ()
    blobs: tuple[BlobOutput, ...] = ()




class ConverterPort(Protocol):
    """Anything that can turn one session JSONL into corpus artifacts.


    Implementations may raise any exception: the materialize use case
    records the failure against the session and continues — one broken
    transcript must never abort a corpus sync. :class:`EmptySourceError` is
    the one exception with its own meaning: the session is empty, not broken.
    """


    def convert(self, session_jsonl: Path, *, archive_dir: Path | None = None) -> ConversionOutput:
        """Convert one session (main JSONL + its side-files) to artifacts."""
        ...

Assumptions consumers make:

Drift risk: the dict[str, Any] fields mean a converter that renames a trajectory key type-checks perfectly and produces artifacts atif-duck’s explicit projections silently null out; and because the implementer never names the port, a convert signature change on either side is caught only where a typed site meets the adapter. Mitigation: packages/atif-cli/tests/test_converter_adapter.py:92-96 is the dedicated assertion (treat it as part of the contract, not as a redundant smoke test), and packages/atif-cli/tests/test_converter_adapter.py:44 pins the field mapping while atif-duck’s DESCRIBE drift test catches a projection that stops matching.

Token semantics and the pricing table

Section titled “Token semantics and the pricing table”

Producer: packages/atif-duck/src/atif_duck/infrastructure/registry.py:73-76 (the semantics) and packages/atif-duck/src/atif_duck/domain/catalog.py:612 (DEFAULT_PRICING)

Consumers:

Shape:

# Model pricing per 1M tokens (in_rate, out_rate) at public list rates from
# Anthropic's published pricing page.
DEFAULT_PRICING: dict[str, tuple[float, float]] = {
    "claude-fable-5": (10.0, 50.0),
    ...
    "claude-opus-5": (5.0, 25.0),
    ...
    "claude-sonnet-5": (2.0, 10.0),
    ...
    "claude-haiku-4-5": (1.0, 5.0),
}

Assumptions consumers make:

Drift risk: cached_tokens exists in two coordinate spaces under one name: atif-duck’s per-step cache-read count (packages/atif-duck/src/atif_duck/infrastructure/projections.py:132) and atif-models’ per-call CallUsage.cached_tokens (packages/atif-models/src/atif_models/domain/ports.py:77). Neither name states its scope, so a cross-plane join or a copied formula silently mixes them. Mitigation: attribute the field by module path, and read the producer docstring (packages/atif-duck/src/atif_duck/infrastructure/registry.py:73-76 or packages/atif-models/src/atif_models/domain/ports.py:65-71) before using either in arithmetic.

The EmbeddingProviderMismatch guard, declared twice

Section titled “The EmbeddingProviderMismatch guard, declared twice”

Producer: two coordinate declarations, coupled by contract rather than by import: packages/atif-embed/src/atif_embed/domain/errors.py:29 with the rule at packages/atif-embed/src/atif_embed/domain/embedding_guard.py:38, and packages/atif-duck/src/atif_duck/domain/embedding_guard.py:33 with the rule at packages/atif-duck/src/atif_duck/domain/embedding_guard.py:51

Consumers:

Shape:

def ensure_store_matches(
    *,
    stored_model: str | None,
    stored_dim: int | None,
    expected_model: str,
    expected_dim: int | None,
) -> None:
    if stored_model is None or stored_dim is None:
        return
    model_ok = stored_model == expected_model
    dim_ok = expected_dim is None or stored_dim == expected_dim
    if model_ok and dim_ok:
        return
    raise EmbeddingProviderMismatch(...)

Assumptions consumers make:

Drift risk: the twin pin holds both RECOVERY_HINT literals to one expected string (packages/atif-embed/tests/test_guard_twin_pin.py:76 and :79) and checks that each reaches its raise; nothing compares the two comparison rules or the rest of the message, so the logic could diverge while every test passes. Mitigation: treat the two functions as one. Change both in the same commit, which is what both docstrings instruct.

harbor’s ATIF models, vendored and consumed by our ported converters

Section titled “harbor’s ATIF models, vendored and consumed by our ported converters”

Producer: upstream harbor.models.trajectories (RFC 0001 data classes) and harbor.utils.trajectory_validator at harbor 0.24.0, vendored as packages/atif-converter/src/atif_converter/domain/atif/ (UPSTREAM_VERSION in its __init__); harbor itself is a dev dependency

Consumers:

Shape:

#: Distributions that are dev-only, so no production module may import them.
DEV_ONLY: frozenset[str] = frozenset({"harbor", "litellm"})

Assumptions consumers make:

The workspace members’ ==0.3.0 sibling pins

Section titled “The workspace members’ ==0.3.0 sibling pins”

Producer: packages/atif-cli/pyproject.toml:32-36 and packages/atif-analytics/pyproject.toml:23

Consumers:

Shape:

dependencies = [
    "atif-analytics==0.3.0",
    "atif-converter==0.3.0",
    "atif-corpus==0.3.0",
    "atif-duck==0.3.0",
    "atif-embed==0.3.0",
    "cyclopts>=5.1.1",
    "duckdb>=2.0.0,<3",
    "loguru>=0.7.3",
]
version_files = [
    'pyproject.toml:^version',
    'packages/*/pyproject.toml:^version',
    'packages/atif-cli/pyproject.toml:^\s*"atif-',
    'packages/atif-analytics/pyproject.toml:^\s*"atif-models==',
]

The other five members (atif-converter, atif-corpus, atif-duck, atif-embed, atif-models) declare no sibling, so these six pins are every edge between members.

Assumptions consumers make:

  • No external installer sees these pins. The member manifests are dev manifests that the release never builds (pyproject.toml:10-14); the one published wheel is the root’s, its dependencies hold no atif-* entry (pyproject.toml:34-40), and packages/atif-cli/tests/test_distribution.py:143 fails if one reaches it. The comment at packages/atif-cli/pyproject.toml:24-31 reasons as if the manifest shipped: the pins keep the workspace coherent, and [tool.uv.sources] is what resolves them.
  • One version covers the whole repository. commitizen reads its own version key rather than the PEP 621 field (pyproject.toml:494-497), and version_files propagates it to the published [project] version and to every member manifest; a separate number per member for one artifact is the shape being refused.
  • The pin entries in version_files are per-file rather than globbed on purpose, because --check-consistency requires a hit in every matched file and five of the seven members pin no sibling (pyproject.toml:528-530).
  • The lockfile must land in the same commit as the versions it resolves. pre_bump_hooks run uv lock and git add uv.lock after the rewrite and before the commit (pyproject.toml:512-522), because uv.lock records every member’s version and mise run lock:check would otherwise fail on the release commit itself.
  • A breaking change moves 0.3.0 to 0.4.0, not 1.0.0, because major_version_zero = true (pyproject.toml:503-505); reaching 1.0.0 is a decision, not a side effect of a ! in a subject line.

Drift risk: a hand-edited pin, or a renamed member whose pin line stops matching its version_files regex, goes stale silently until a release. Mitigation: --check-consistency fails rather than skipping a file that no longer contains the current version; the guarantee is stated at pyproject.toml:523-526.

Other contracts

Section titled “Other contracts”

See also

Section titled “See also”