---
title: Code index operations
---

This page covers how to enable the code index workers, configure repositories and keep
the index current; the code index runs only when an operator enables its workers and
configures repositories. Every `code_*` worker except `code_coverage_notify` and
`code_backport_gaps` is off by default. What the `code.*` operations answer is in the
[code index overview](/docs/zh-CN/features/code-index/overview); how the index is built is in
[code index architecture](/docs/zh-CN/architecture/code-index-architecture).

One code namespace indexes one branch of one repository. There are two ways to add one:

| Way             | How                                         | Tiers                  | Page                                                   |
| --------------- | ------------------------------------------- | ---------------------- | ------------------------------------------------------ |
| Self-service    | dashboard **Repositories → Add repository** | parser (`code_ingest`) | [repositories](/docs/zh-CN/features/code-index/repositories) |
| Operator config | `LOOM_CODE_REPOS_JSON`                      | parser + compiler      | this page                                              |

An operator entry for a namespace a member added from the page is skipped
([adding repositories](/docs/zh-CN/features/code-index/repositories)).

Run `loom code` inside the container that runs the workers (`loom-workers` in
`dev/docker-compose.cloud.yml`, `loom-server` on single-container stacks), so it uses the
deployment's `.env`, git mirrors and ClickHouse:

```bash
docker exec loom-workers loom code stats --namespace code-myrepo
```

`--org` / `--namespace` default to `LOOM_CODE_ORG` / `LOOM_CODE_NAMESPACE`. Pass both
when the deployment indexes more than one repository.

## Prerequisites

- **The image.** It includes the parser grammars (the `code` extra), Go with scip-go,
  Node with scip-typescript and scip-python, and npm, pnpm, Yarn and Bun for
  automatic TypeScript dependency preparation. Docker users do not install these
  on the host. ClickHouse is the only required running service.
- **Native server toolchains.** Install Python 3.12+, Git and Node.js 22+ with npm,
  then run these commands from the Loom checkout before starting the server:

  ```bash
  pip install -e ".[code]"
  sh scripts/install-code-toolchains.sh
  export PATH="$PWD/var/code-toolchains/bin:$PATH"
  ```

  The script installs pinned TypeScript/Python indexers and package managers without
  requiring root. Keep that `PATH` in the environment of the Loom server, including
  its service manager. Native Go indexing additionally needs Go and scip-go:
  `go install github.com/scip-code/scip-go/cmd/scip-go@v0.2.7` (put Go's bin directory
  on the server's `PATH`). C/C++ needs scip-clang and the project's build setup.

- **An embedder.** Chunks are embedded with the server's `LOOM_EMBEDDING_MODEL`.
  `loom code backfill --embed` refuses the fake embedder unless you pass
  `--allow-fake-embedder` (development only).
- **Writable git mirrors.** The workers run as uid 999 and fetch into the bare clones
  under `var/code_repos` (bind-mounted into the containers). The host operator shares
  that directory with them through a group, gid 999 here. Prepare it once:

  ```bash
  sudo chgrp -R 999 var/code_repos && sudo chmod -R g+rwX,o-rwx var/code_repos
  sudo find var/code_repos -type d -exec chmod g+s {} +
  ```

  The setgid bit keeps new entries in the group. Then add the host operator to the
  group and set `LOOM_CODE_REPOS_SHARED_GID=999` in `.env`, which the containers and
  the host's `loom code` commands both read. A mirror or checkout directory Loom
  creates in that group is mode 2770; every other one is 0700, open to the server's
  user only, so nothing is shared until the setting names the group. With the
  setting, the server and every `loom code` command run with umask 007: the
  directories and lock files they create are writable by the group, git's files give
  the group what they give the owner (its objects stay read-only), and other users get
  nothing. Run a harness by hand on a checkout under the root through
  `loom code entry-lock`, which runs it under the same umask. Loom changes an existing
  directory's mode only for a restricted entry: run the commands again for files an
  older build or another tool created.

  The group reads every mirror, restricted ones included, so it must hold only the
  users that run Loom: the host operator and the containers' user (`getent group 999`).
  When gid 999 is already a system group with other members (`systemd-journal` on many
  distributions), run the containers with a dedicated group instead (Compose
  `user: "999:<gid>"`) and use that gid in the commands and the setting. Keep every
  path the host and the containers share (`code_repos_root`, `code_scip_checkout_dir`
  and each entry's `git_dir` and `scip_checkout_dir`) inside the prepared directory: a
  directory or lock file Loom creates elsewhere takes its parent's group under a
  setgid parent and the creating process's group otherwise, and with the umask that
  group could write it.

  The umask covers every file those processes write, outside the root too, where a
  file takes the writer's primary group: the host operator's primary group must be
  its own (a user private group, the default on most distributions), not one shared
  with others such as `users`.

  Upgrade the host checkout that runs `loom code` jobs together with the image, and
  run the commands above again once a colour deployment's previous colour has stopped.

- **A GitHub token** (`LOOM_CODE_GITHUB_TOKEN`, or `github_token` per entry) for
  merged-PR history and for private remotes.

## Configure the entry

`LOOM_CODE_REPOS_JSON` is a JSON array with one object per indexed branch. Fields an
entry leaves out fall back to the scalar `LOOM_CODE_*` settings; an entry for a
repository other than the scalar `LOOM_CODE_REPO` derives its `git_remote`
(`https://github.com/<owner>/<name>.git`) and `git_dir` (`<name>.git` beside the scalar
clone, or under `var/code_repos` when there is no scalar repository) from `repo`.

```bash
LOOM_CODE_REPOS_JSON='[
  {"org": "acme", "namespace": "code-myrepo", "repo": "acme/myrepo",
   "git_branch": "main", "public": true},
  {"org": "acme", "namespace": "code-myrepo-release-2", "repo": "acme/myrepo",
   "git_branch": "release-2", "embed_reuse_from": "code-myrepo"}
]'
```

| Field                                | Meaning                                                                                                                       |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
| `namespace`, `org`                   | where the index lives                                                                                                         |
| `repo`                               | `owner/name` on GitHub                                                                                                        |
| `git_dir`, `git_remote`              | the bare clone and its remote; branches of one repository share one clone                                                     |
| `git_branch`                         | the branch this namespace tracks                                                                                              |
| `path_prefixes`                      | comma-separated prefixes to index; an entry without it indexes the whole tree minus the built-in denylist                     |
| `exclude`                            | globs never indexed (list or comma-separated)                                                                                 |
| `exclude_generated`                  | refuse files whose content marks them generated (`Code generated … DO NOT EDIT.`, `@generated`)                               |
| `include_contrib_paths`              | vendored `contrib/` paths to index anyway, so their definitions bind the repository's call sites                              |
| `depends_on`                         | other code namespaces this code calls into; a definition missing here is looked up there                                      |
| `embed_reuse_from`                   | a namespace whose embeddings are reused when the chunk text is identical (a release branch of the same repository)            |
| `public`                             | the repository is public on its host: anyone in the org can join its index as a reader at once                                |
| `classification`                     | `restricted` for a private repository: a maintainer approves each reader, and it is never org-readable                        |
| `scip_producer`, `scip_checkout_dir` | the compiler-tier indexer (`clang`, `typescript`, `python`, `go`; unset, the tree's build files pick) and its working tree    |
| `scip_extra_producers`               | further compiler-tier producers over the same tree (a Python repository with a TypeScript frontend), each in its own checkout |
| `git_extra_branches`                 | globs of other remote branches the shared mirror should carry                                                                 |
| `github_token`                       | per-entry override of `LOOM_CODE_GITHUB_TOKEN`                                                                                |
| `git_proxy_identity`                 | the user name its git presents to an operator's [git proxy](#git-through-a-forward-proxy); unset, the deployment's default    |

Name a `git_remote` host in ASCII, as punycode for an internationalised name: a page
add refuses any other, so the two spell a host alike.

An entry whose `git_remote` is a local path or `file://` mirror is known by its `repo`,
which a page add of the same repository must match: name a mirror of a repository on a
host other than GitHub as `host/owner/name` in `repo`.

`public` is read when `loom code init` first registers the namespace; editing it later
does not change the join policy, which a maintainer changes with `code.repo.set_access`.
A configured entry starts with no maintainer. `POST /v1/admin/namespace.adopt` refuses a
restricted namespace, and no route or command installs the first direct grant on a
configured restricted namespace, so a private repository is set up in this order:

1. Configure the entry without `classification`, with `public` false (the default) and no
   `org_readable`, and leave `LOOM_CODE_ORG_READABLE` at its default, `false`. Until step
   3 the namespace is standard: a grant there, or a token whose claim covers it (a `*`
   claim included, not only one naming it), reads it.
2. Run `loom code init` for it, which writes the namespace's directory row (adopt answers
   `NAMESPACE_NOT_FOUND` before that), and adopt it for its maintainer. Do not run
   `loom code backfill` (or any `loom code backfill-*` command) yet: a configured entry
   gets no first index, PR history, backport verdicts or extra branches
   (`code.repo.add_branch` answers `REPO_NOT_INDEXED`) until the backfill writes its
   `git_head` watermark, so nothing of the repository is indexed until you run it. The
   entry's git mirror is cloned meanwhile, and its history reads pass the same gate, so
   finish step 3 before handing out `*` tokens of the org.
3. The maintainer restricts it with `code.repo.set_access` (`classification: "restricted"`
   and `join_policy: "approval"` or `"closed"`; both are required); then set
   `classification: "restricted"` on the entry as well, so either input restricts, and run
   `loom code backfill`. Mirrors and compiler-tier checkouts are created at mode 2770
   when they belong to `code_repos_shared_gid`, open to the server's
   user and that group only, and 0700 otherwise (see "Writable git mirrors"); one an
   older build created, or whose mode was changed by hand, is set to that mode at once
   on the host that serves the call, and on every other host that holds it at that
   host's next mirror round, once the host reads the namespace's directory row as
   restricted (within half a minute). Only the directory itself changes: what an older
   build created inside it needs the commands in "Writable git mirrors".

The restriction is per branch: each branch's index is its own namespace. A person whose
own token reads the repository on its host can still add another of its branches with
`code.repo.add`, into a new standard namespace; that person can already read the
repository. Set up each configured branch of a private repository in the order above.

An entry that names no `scip_producer` keeps `scip_checkout_dir` for C/C++ (see
[Compiler tier](#compiler-tier)), so name the producer when you prepare that tree for
another language.

The repository's `.gitattributes` `linguist-generated` / `linguist-vendored` patterns
and a `.loomignore` file at the indexed commit are honoured as excludes. A dependency
vendored as a git submodule is not in the bare mirror; give it its own entry and name it
in `depends_on`. The gauge `loom.code.repos_config_errors` reports entries the server
could not use (invalid JSON, a nameless entry, a namespace with `.` or `/`, a duplicate
namespace); 0 means the value parsed clean. Validate the JSON before a restart
with `docker compose --env-file .env -f dev/docker-compose.cloud.yml config`.

Who can read an index is governed by the namespace's join policy and grants
([tokens and grants](/docs/zh-CN/guides/tokens-and-grants)). `LOOM_CODE_ORG_READABLE=true`
makes every code namespace readable by every principal in the org without a grant; the
default is off.

## First index

```bash
docker exec loom-workers loom code init     --namespace code-myrepo --org acme   # optional: bare clone + index state
docker exec loom-workers loom code backfill --namespace code-myrepo --org acme --embed
docker exec loom-workers loom code backfill-prs --namespace code-myrepo --org acme --months 12
```

- `init` is optional: the ingest pass clones a missing mirror in the background, for an
  operator-configured entry too, and logs the path it clones into (`loom doctor`'s
  `code_mirrors` line names an operator `git_dir` that is missing, so a mistyped path
  is not a silent multi-GB clone). The mirror is a cache of the remote ClickHouse
  records: every serving host clones the mirrors of indexed entries at boot
  (`LOOM_CODE_MIRROR_WARMUP`) and, every `LOOM_CODE_MIRROR_FETCH_INTERVAL_S`, fetches
  the refs the leader's pass recorded in ClickHouse that it lacks (no `ls-remote`, and
  no network call when it is current). Until a host has the mirror, its history reads
  answer `no_git_mirror: cloning <entry>`; `/v1/health?include=mirrors` counts the host's
  mirrors under `mirrors`. Disk: every host holds a copy of every mirror (N replicas, N copies), and
  an operator `git_dir` must be writable by the `loom` user. Network: a host with an
  empty disk makes one full clone per indexed repository; keep `LOOM_CODE_REPOS_ROOT`
  on a persistent volume so a rollout does not re-clone every mirror on every pod.
- `backfill` parses, chunks, extracts symbols and edges, and embeds each file. It is
  resumable: re-run it after an interruption. `--phase index` then `--phase embed` splits
  the work, since the unembedded rows are the embed phase's queue. `--commit <sha>` pins
  the tree (and implies `--no-fetch`); `--concurrency N` parallelises files; `--limit N`
  indexes a sample.
- The first full backfill of a namespace also re-derives every file's edges after the
  name table is complete, so cross-file references bind (`--no-edges-second-pass` skips
  it). `--dialect-folder-pass` lets SQL files of undecided dialect take the dialect their
  directory agrees on.
- `backfill-prs` loads merged-PR history from GitHub, resumable by cursor and bounded by
  GitHub's rate limit (`LOOM_CODE_PR_BACKFILL_MONTHS`, default 12). `backfill-reviews`
  adds inline review comments, `backfill-assignees` requested reviewers, and
  `backfill-items` issues and PRs in any state.
- **The worker does not start a first index for a configured entry.** An entry without a
  `git_head` watermark is skipped with the note "run `loom code backfill` first". (It
  does build self-service first indexes, `LOOM_CODE_BOOTSTRAP_FILES_PER_PASS` files per
  pass.)

Then:

- `POST /v1/code.calibrate` (`ADMIN`) derives the code index's abstention thresholds from
  `bench/fixtures/code_calibration_probes.jsonl` (shipped in the image).
- `loom code describe --top-pct 20` (optional, LLM cost) writes one-line descriptions for
  the most central chunks.
- `loom code stats` prints the watermarks (`git_head`, `scip_commit`, …) and live row
  counts; `POST /v1/code.index_status` answers the same from the API.

`scripts/ops/code_intel_bootstrap.sh` runs this sequence for the configured
`LOOM_CODE_ORG` / `LOOM_CODE_NAMESPACE` on the Loom host (init, both backfill phases, PR
history, calibrate, then the `bench/code_intel_qa.py` gates), refusing a fake embedder
or a server whose `/v1/health` does not answer; `--dry-run` runs only that read-only
preflight and prints the commands. It reads `LOOM_BASE_URL` and `LOOM_ADMIN_TOKEN`.

## Steady state

Enable the workers in `.env` on the workers container and recreate it. Every worker's
cadence setting and default is in the [workers reference](/docs/zh-CN/reference/workers).

| Worker                 | Flag                                     | Does                                                                                                                                                                                                                                                                                                                                                                                                 |
| ---------------------- | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `code_ingest`          | `LOOM_CODE_INGEST_ENABLED`               | every `LOOM_CODE_INGEST_INTERVAL_SECONDS`: fetch, then per first-parent commit re-chunk only changed content, refresh symbols and edges, tombstone deletions, build the merged PR's context pack, advance `git_head`. Hourly it also finds indexed files whose chunks predate their content at head and heals up to `LOOM_CODE_RECONCILE_HEAL_PER_PASS` per pass. Builds self-service first indexes. |
| `code_embed_recovery`  | `LOOM_CODE_EMBED_RECOVERY_ENABLED`       | embeds chunks left without a vector (provider outage, `--phase index`)                                                                                                                                                                                                                                                                                                                               |
| `code_pr_history`      | `LOOM_CODE_PR_HISTORY_ENABLED`           | syncs newly merged PRs from GitHub                                                                                                                                                                                                                                                                                                                                                                   |
| `code_graph_stats`     | `LOOM_CODE_GRAPH_STATS_ENABLED`          | weekly: PageRank over the include graph, per-file degrees, the test ↔ source map                                                                                                                                                                                                                                                                                                                     |
| `code_summary`         | `LOOM_CODE_SUMMARY_ENABLED`              | weekly: LLM file and module cards for `code.map` (cost-capped per namespace)                                                                                                                                                                                                                                                                                                                         |
| `code_scip_apply`      | `LOOM_CODE_SCIP_APPLY_ENABLED`           | applies compiler-tier deltas uploaded through `code.scip_ingest`                                                                                                                                                                                                                                                                                                                                     |
| `code_coverage_notify` | `LOOM_CODE_COVERAGE_NOTIFY_ENABLED` (on) | a task event when a watched commit's coverage is ready in every tier                                                                                                                                                                                                                                                                                                                                 |
| `code_backport_gaps`   | `LOOM_CODE_BACKPORT_GAPS_ENABLED` (on)   | the nightly backport report; runs only with `code_pr_history`                                                                                                                                                                                                                                                                                                                                        |

The per-language compiler-tier workers are in the next section.

**Push-triggered refresh.** A CI job can call `POST /v1/code.refresh` (`WRITER`) right
after a merge. It runs the same ingest pass immediately for that namespace on the replica
that received it, under the namespace's lease (`code_refresh:<org>/<namespace>`), so
triggers landing on different replicas coalesce into one pass. It answers 404 for a
namespace no entry configures, so the pusher never decides what gets indexed. It works
with the poller disabled.

Open pull requests are not indexed; the working tree stays the client's own.

## Compiler tier

The parser tier answers from tree-sitter. Where a compiler-grade (SCIP) index exists, its
symbols and references win. Each language has a worker that indexes the commits between
its own watermark and the branch head in the entry's mirror (fetched by `code_ingest`)
and applies the delta in-process, on whichever process holds the workers lease. A
compiler tier does not wait for the parser tier's `git_head`; only an entry's first
index does.

| Language   | `scip_producer` | Worker flag                                                   | Host needs                                                                                                          |
| ---------- | --------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| C / C++    | `clang`         | `LOOM_CODE_SCIP_DELTA_ENABLED`, `LOOM_CODE_SCIP_FULL_ENABLED` | clang 21, CMake and ninja (in the image) and scip-clang (in the amd64 image; elsewhere `LOOM_CODE_SCIP_CLANG_PATH`) |
| TypeScript | `typescript`    | `LOOM_CODE_SCIP_TS_DELTA_ENABLED`                             | Node and the pinned indexer/package managers (included in the image); dependencies are prepared automatically       |
| Python     | `python`        | `LOOM_CODE_SCIP_PY_DELTA_ENABLED`                             | scip-python (in the image)                                                                                          |
| Go         | `go`            | `LOOM_CODE_SCIP_GO_DELTA_ENABLED`                             | Go and scip-go (in the image)                                                                                       |

An entry that names no `scip_producer` gets every tier whose build file its tree
carries: `go.mod` (at any depth) for Go; `tsconfig.json`, `jsconfig.json` or
`package.json` at the root for TypeScript; a root `CMakeLists.txt` for C/C++;
`pyproject.toml`, `setup.py` or `requirements.txt` at the root for Python.
`code.index_status` lists them as `compiler_candidates`. Name the producer where a build
file is incidental, such as a C++ repository whose `pyproject.toml` only serves its test
tooling; `scip_extra_producers` adds tiers beside a named one. Each tier works in its own
tree: the named producer (C/C++ when the entry names none) in `scip_checkout_dir`, every
other one in `<scip_checkout_dir>.<producer>`.

All the workers are enabled by default, and the Docker image carries every indexer:
scip-python, scip-typescript, scip-go, and for C/C++ clang 21, lld, CMake, ninja and
scip-clang. Upstream builds scip-clang for x86_64 Linux only, so the arm64 image has
the rest of the C/C++ toolchain but no scip-clang: point `LOOM_CODE_SCIP_CLANG_PATH` at
your own build, or the C/C++ tier records `scip-clang missing` as a skip. Each worker
initializes a missing or empty working tree directory as a Git repository and fetches
each commit into it from the entry's mirror, so a new leader starts from nothing.
An existing checkout must be the root of a standalone Git working tree, with its
`.git` directory inside the checkout. Compiler Git jobs need writable metadata
there. Linked worktrees, external Git metadata, regular files, bare repositories and
nonempty directories without their own working tree are refused without replacing
their contents. Use a separate empty path for the compiler checkout, or point
`scip_checkout_dir` at an existing standalone checkout root.
A worker whose project prerequisites are absent records a skip in `workers.status`; the Tree-sitter tier
remains available. TypeScript projects have their dependencies installed
automatically in their checkout.

Compiler reconciliation batches file paths and symbol keys by both item count and
serialized UTF-8 size. Long qualified names and multibyte paths therefore split into
smaller reads within the default ClickHouse HTTP form-field limit. A symbol key
larger than one field is reconstructed from bounded parameters; names and identifiers
remain exact.

The C/C++ harness runs the repository's CMake configure (compile database only, no
build) on the first run in a tree, or when its `CMakeCache.txt` is missing, was made in
another directory, or names a compiler the host no longer has. It runs the configure
again on the existing cache before a full index and for a delta whose window adds a
`.cpp`, `.cc` or `.cxx` file or adds, changes or deletes a `CMakeLists.txt` or `.cmake`
file, because the compile database lists only the translation units of the tree it was
configured on. A reconfigure that fails fails the run with CMake's last lines, and the
entry's `clang` record (`clang_full` for a full index) reads failing with kind `cmake`
(`disk` when the disk is full). A failed delta moves no watermark, so its window, with
the translation units it adds, is retried every pass; a full index is retried 6 hours
later, as any failed full run is.

A large C++ tree is the heavy case: a ClickHouse-sized checkout with its submodules
and build tree is about 30 GB beside the mirror, and scip-clang holds a translation
unit's AST per job (a few GB each), so `LOOM_CODE_SCIP_CLANG_JOBS` should fit the
memory the workers process has. The full index of such a tree takes about an hour.

- **C/C++ delta cap.** The worker indexes changed translation units, including those
  that include a changed header, up to `LOOM_CODE_SCIP_DELTA_MAX_TUS` (200); above the
  cap it indexes only the changed `.cpp` files. A delta that reconfigures CMake runs the
  configure inside the same `LOOM_CODE_SCIP_DELTA_TIMEOUT_SECONDS` (3600) as the index;
  a timed-out delta leaves `scip_commit` where it was, so the next pass retries a larger
  window.
- **Delta caps.** The server reads a compiler tier's delta (every tier, delta or full
  run) only when it is a regular file, not a link, and applies it a round of files at a
  time (`server/code/app/index/delta_file.py`). It first counts the delta's `{`, `[`
  and `,` bytes, then parses every value once and checks each file entry, and only
  then reopens it and reads the files again a round at a time for the apply. So a
  delta over a cap, or one that does not parse, fails the entry's pass before anything
  is applied. The caps hold as read when the delta is opened; a changed value reaches
  the next pass.
  - `LOOM_CODE_SCIP_DELTA_FILE_MAX_MB` (180 MiB) bounds the file's bytes and
    `LOOM_CODE_SCIP_DELTA_MAX_NODES` (6,000,000) its counted bytes, which bound the JSON
    values the parses build. They bound the work of a pass. Its memory is the round
    below plus a record of each file entry and deleted path: about 70 B each, as many
    as the node cap allows (a minimal entry costs 2 counted bytes, a deleted path 1:
    about 3 million entries, some 200 MiB, or 6 million deleted paths, some 400 MiB,
    at the default), and their paths' text (at most 4,096 bytes each), which the byte
    cap bounds. A large C++ tree's full run writes over 500 MiB in entries of tens of
    KB, a few MiB of bookkeeping: raise both for it.
  - `LOOM_CODE_SCIP_DELTA_ROUND_MAX_NODES` (3,000,000) bounds one round's memory: the
    counted bytes one apply round parses. A round holds at most 100 files, and is
    released before the next is parsed; one file entry alone may fill it, and a file
    entry over it fails the pass, naming the entry. Any other value that large (a
    deleted path, a header value, an ignored key) reads `delta invalid`, since the
    harness never writes one; the deleted paths are read one at a time. A fixed cap of
    64 million characters, not a setting, bounds one value's and one round's text the
    same way; a real entry has about 32 characters per counted byte, so for one it binds
    near 2,000,000 counted bytes. The largest file entry of a large C++ tree's full run
    holds about 600,000 counted bytes (some 100,000 edges), about a fifth of the round
    budget and a third of the character cap. The densest round repository code can write
    (minimal symbols) peaks at about 1.8 GiB resident while it is read at the default
    budget (measured with `ru_maxrss`; about 330 B per counted byte of Python objects,
    the rest the allocator's); a value's text near the character cap (up to 4 bytes a
    character) may be held up to three times while the read buffer grows, so the worst
    round's read is about 2.5 GiB, whatever the delta's size. Its apply holds that
    round's rows on top (the live rows read for its paths and the new rows to insert),
    about as much again for a round of new symbols, so size the worker for about twice
    the read. A real file entry of a C++ delta has one counted byte per about 32 bytes:
    a round is its 100 files long before it reaches the budget. To measure the read of
    the densest default round from a source checkout:

    ```bash
    .venv/bin/python -c "import resource, tempfile, pathlib; from server.settings import settings; from server.code.app.index.delta_file import open_delta; p = pathlib.Path(tempfile.mkdtemp()) / 'delta.json'; n = settings.code_scip_delta_round_max_nodes // 2 - 10; p.write_text('{\"commit\":\"c\",\"files\":[{\"path\":\"a\",\"symbols\":[' + ','.join(['{\"qualified_name\":\"\"}'] * n) + ']}]}'); list(open_delta(p).rounds(100, lambda x: True)); print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss >> 10, 'MiB peak RSS')"
    ```

  An invalid delta fails the pass with a short `delta invalid: <reason>` naming its
  first problem (for an entry, `file entry <i>: …`), not pydantic's list of every bad
  item. `--keep-delta` keeps the delta as the harness wrote it, before it is parsed
  (the byte cap still applies), as a file only the user that ran the command can read
  (mode 0600).

- **C/C++ full index.** `code_scip_full` re-indexes one entry per pass: an entry whose
  `scip_commit` is not seeded yet (the delta skips it with "watermarks not seeded"), and
  every entry once its last applied full run is `LOOM_CODE_SCIP_FULL_EVERY_HOURS` (168)
  old, which heals capped deltas. A run that fails is retried 6 hours after it failed,
  and one killed with its pod (a rollout) 6 hours after it started. A
  base-unknown delta clears `scip_commit`, so the entry gets a full index next. The pass
  runs under the entry lock with `LOOM_CODE_SCIP_FULL_TIMEOUT_SECONDS` (8 hours) for
  the harness and six more for the checkout move, the apply and the repack.
  `loom code scip-full` runs the same index by hand (`--keep-delta` saves the delta so an
  interrupted apply can be re-run with `--apply-delta`), and `loom code scip-delta` one
  delta pass, which prints a `RESULT:` line.
- **C/C++ submodules.** The tier fetches submodule pins with the entry's access
  token, so a private submodule whose `.gitmodules` URL is an absolute https URL under
  the remote's owner needs no other credential. The owner is matched as written, case
  included, so an owner spelled in another case is fetched without the token. A
  relative URL resolves to a local path in the checkout, which has no remote; the tier
  refuses the `file` transport, so that submodule update fails. Whoever can commit to
  the repository's `.gitmodules` can point a submodule at any repository of that owner
  the token reads, so give an entry a token scoped to the repositories its submodules
  need.
- **SQL ↔ C++ crossref.** `code_crossref` rebuilds `code_test_map`'s content-reference
  rows (test queries to the files that register the functions and settings they use)
  for each C/C++ entry whose `git_head` moved since its last run. It reads the parser
  tier's chunks only.
- **Checkout housekeeping.** The full C/C++ pass (under the entry lock) and the
  Python, TypeScript and Go passes repack their entry's checkout with
  `git gc --no-prune` every `LOOM_CODE_MIRROR_GC_INTERVAL_H`, under the same rules as
  the mirrors (below). Every git command a tier runs in a checkout has auto-gc and
  background maintenance off, so no fetch starts a repack.
- **Entry locks.** A full pass and a delta never work on the same entry at once: each
  takes the entry's lock file in `LOOM_CODE_SCIP_LOCK_DIR` (`var/cron/locks`),
  `scip.<org>.<namespace>.lock` with both names percent-encoded
  (`loom code scip-lock-path --org ORG --namespace NS` prints it). A full pass waits for
  a delta already on its entry; a delta skips an entry a full pass holds and names it
  under `skipped` ("entry lock held (full pass running)"). Different entries run side by
  side. Every pass also takes the entry's key in ClickHouse (`scip_entry:<org>/<namespace>`
  in `loom._migration_lock`), so
  passes on two hosts exclude each other too; a dead holder's key expires after 120 s.
  A pass that cannot take the key (ClickHouse or Keeper unreachable) fails instead of
  running on the host's file alone. A holder counts its key lost after 60 s without a
  renewal (a pause included), before another host can take it, and writes no further
  index row or watermark: every such write checks the key first, and the pass ends with
  `EntryLockLost`. A manual harness run goes under
  `loom code entry-lock --org ORG --namespace NS -- <command>`, which takes both locks,
  exits 75 when another pass holds the entry, and kills the command (exit 3) if the key
  is lost. It hands the command the key's holder (`LOOM_ENTRY_LEASE_HOLDER`), which the
  harness's uploads carry: `code.scip_ingest` refuses a part (409 `ENTRY_LEASE_NOT_HELD`)
  once that holder no longer has the key, so a pass that lost its entry completes no
  batch for `scip_apply`. With `--tier <producer>` it also records the command's exit as
  that compiler tier's health on the entry (`tier_health` in `code.repos`), and relays the
  command's output, stderr included, to stdout.
- **TypeScript dependencies are automatic.** Fresh checkouts install dependencies
  before compiler indexing, including development dependencies containing type
  definitions. Loom uses npm, pnpm, Yarn (classic or modern) or Bun according to the
  lockfile, enforces existing lockfiles and disables lifecycle scripts and executable
  package-manager hooks. Plain scripts without a package manifest need no install.
  Dependency manifest/lockfile changes trigger a refresh before the next index; a failed
  or interrupted install is retried without advancing the compiler watermark.
  Downloads need network access to the repository's package registry. A private package
  registry, incompatible lockfile or generated types requiring a build can still need
  project-specific configuration. `workers.status` names the failed installer and its
  exit code, and the entry's tier-health record (`code.index_status`) holds the end of
  the installer's output. Server credentials are not passed into the installer environment.
  The server reads the checkout's `package.json` (up to 1 MiB) and each install root's
  `yarn.lock` (up to 64 MiB) only as regular files, never through a link; a symlinked one
  fails the TypeScript pass even when the link points inside the repository.
- **Repository files are size-checked before they are read.** The ingest asks the mirror
  for a file's size first: a file over `LOOM_CODE_INGEST_MAX_FILE_BYTES` is not read, and
  its `code_files` row records the reason (`code.index_errors`); 0 lifts the cap. Every
  manifest the import resolver reads (the TypeScript, Python and Go project and workspace
  files) is read from the mirror up to 1 MiB, a `go.mod` up to 64 KiB and at most 200 of
  them; a larger one is skipped once per commit with a warning naming the org and
  namespace, and the imports it would have resolved stay unresolved. Outside a worker
  pass the resolver reads a file from its indexed chunks instead, which hold only files
  within `LOOM_CODE_INGEST_MAX_FILE_BYTES` (any size at 0). A barrel's re-exports are read
  under the same file cap. A binary or denylisted file over the cap is tombstoned as
  one under it is, by its path and its first 8 KiB; a generated one gets the skipped
  row, with no declarations.
- **The repository's own exclusions are bounded and fail open.** `.loomignore` and
  `.gitattributes` are read up to 1 MiB and give at most 1,000 patterns between them. A
  larger file gives no patterns, and patterns past the 1,000th are dropped, so the paths
  they would have excluded are indexed; the log names the file or the count. The
  entry's own `exclude` globs are never cut, so a path that must stay out belongs
  there.
- **A failed apply** leaves its batch `failed` with the error;
  `loom code scip-retry <batch_id>` resets it to pending.
- **Manual and CI producers** live in `scripts/scip_harness/` (`run_delta.sh`,
  `run_typescript_delta.sh`, `run_python_delta.sh`, `run_go_monorepo_delta.sh`,
  `scip_to_delta.py`; see its README).

### Repository-job sandbox

**Build-based tiers execute repository code.** The C/C++ tier runs the repository's own
`cmake` configure (a `CMakeLists.txt` is a program), the Go tier runs `go mod download`
over the repository's module graph, and the TypeScript tier installs the packages its
lockfile names. Treat every compiler-tier run as running code the repository's authors
wrote, on the host that runs it.

Loom runs every repository job (clone, fetch and `ls-remote`, `git submodule update`,
dependency installs and the compiler-tier harnesses) inside one boundary built on
[bubblewrap](https://github.com/containers/bubblewrap)
(`server/code/app/index/sandbox.py`) when the host can run it:

- its own user, pid, network, mount, ipc and uts namespaces, no capabilities, no new user
  namespaces, and a seccomp filter that refuses `ptrace`, `mount`, `unshare`, `setns`,
  a `clone` with `CLONE_NEWUSER`, `bpf`, `perf_event_open`, kernel keyrings, module and
  kexec loading, `userfaultfd`, `open_by_handle_at` and `process_vm_readv`/`writev`
  (`clone3` answers `ENOSYS`, so the C library falls back to `clone`);
- only system directories, the toolchains and the job's own paths are mounted. A
  toolchain is mounted only where it exposes none of Loom's data: never a directory
  that is or holds `var/`, the mirrors and checkouts, the env file, the
  install root or `HOME`, nor one inside `var/` or under `HOME` (the default
  `var/bin/scip-clang` is mounted as that one file). Where one of those paths lies
  inside any other mounted directory (a mirror inside the server's Python
  prefix, `var/` under a system path), an empty mount covers it inside the sandbox.
  The install root is the exception, since the server's Python may run from it. A host
  whose Python prefix or a system path is, or sits inside, one of Loom's data paths
  runs no sandboxed job: no empty mount could hide that data, and `/v1/health` and
  `loom doctor` report `repo_sandbox_layout_refused`. The checkout (or the
  bare mirror, for a fetch) and the output directory are writable, the checkout's
  `.git` and the harness scripts are read-only, and `/tmp` and `HOME` are a fresh
  directory removed after the job. No server environment, credentials, host `HOME`,
  Docker socket or other host paths;
- rlimits, set by `prlimit`: `LOOM_CODE_SANDBOX_CPU_SECONDS` (7200),
  `LOOM_CODE_SANDBOX_MEMORY_MB` (16384) and `LOOM_CODE_SANDBOX_MAX_FILE_MB` (32768),
  set before bwrap starts, bound **each process** of a job (CPU time, data memory, file
  size), not the job's total. `LOOM_CODE_SANDBOX_MAX_PIDS` (2048) bounds the processes
  and threads of **one job**: it is set inside the job's user namespace, where the
  kernel counts only that job's tasks (Linux 5.14 and later; older kernels count the
  uid's tasks host-wide). Outside it, RLIMIT_NPROC would count every task of the uid on
  the host, other containers' included. The sandbox runs the `prlimit`
  found on the server's `PATH`, so it must sit under a mounted system path such as
  `/usr`. Loom sets no cgroup; give the container its own memory and pids limits for a
  hard total. The colours set `pids_limit: 8192` on each container as that backstop.
  Each job also has a wall-clock timeout that kills its whole tree; the job's pid namespace
  guarantees no process it started outlives it;
- bounded output in the server: a job's stderr, and the stdout of a build, install or
  clone, keep only their last 1 MiB; a command whose stdout Loom parses (`ls-remote`,
  fetch) may print up to 64 MiB, and past that the whole job is killed and fails. A file
  the server reads back from a job or a checkout is a regular file, never a link: a
  compiler tier's delta up to `LOOM_CODE_SCIP_DELTA_FILE_MAX_MB` and
  `LOOM_CODE_SCIP_DELTA_MAX_NODES`, the checkout's `package.json` up to 1 MiB and each
  install root's `yarn.lock` up to 64 MiB;
- no network, except for clones, fetches, submodules and dependency downloads, and then
  only through an egress proxy outside the sandbox. It accepts `CONNECT` to ports 443,
  22 and the remote's own port, resolves the name (from `/etc/resolv.conf` alone: not
  `/etc/hosts` or nsswitch, and no search domains, so a remote needs its full DNS name),
  refuses it unless **every** address
  is public (loopback, private ranges, link-local and the cloud metadata address are
  refused, including when reached through a redirect or a `.gitmodules` URL) and
  connects to the address it vetted. Plain HTTP has no route. One job holds at most 32
  tunnels at a time. A client inside the sandbox reaches the proxy through a local relay
  (its HTTPS proxy), which bridges at most 32 connections; one more waits until a bridge
  ends. Any other connection over the cap at the proxy is refused after
  `LOOM_CODE_EGRESS_SLOT_WAIT_S` (5 s by default), and so is a relay connection while
  ssh tunnels hold some of the slots. The proxy logs the first refusal, and the total
  when the job ends if there was more than one.

  A tunnel with no byte either way for `LOOM_CODE_EGRESS_IDLE_TIMEOUT_S` (5 minutes by
  default) is closed. Once its client has sent EOF, it is closed after
  `LOOM_CODE_EGRESS_HALF_CLOSE_TIMEOUT_S` (10 s by default) without a byte from the
  server; while bytes still wait for a slow client, the idle timeout applies instead. A
  tunnel whose client sent EOF before any byte moved (a connection it gave up on) is
  closed at once. Once the proxy's side of a tunnel has ended (the server closed or
  reset it, or the proxy closed it), the relay ends its connection after at least a
  second without a byte from the client. A client that never reads what it was sent
  keeps its relay connection, and so its relay slot, for up to the idle timeout.

Every git a repository job runs reads no system or global git config and allows only
the `https` and `ssh` transports (`protocol.allow=never`); `ext::` and `file://` are
refused, so are URL rewrites. The entry's access token reaches only its clone, its
fetches and the C/C++ tier's `git submodule update`, never a build or `go mod
download`, and git sends it only to URLs under the remote's owner
(`https://host/owner/`; the whole host for a remote with no owner path). Inside the
sandbox an `ssh` remote has no host SSH key: use an HTTPS remote with an access token.

The sandbox's egress proxy connects to the remote itself and is not chained through an
operator's proxy: a host that runs the sandbox runs no network git while
`LOOM_CODE_GIT_PROXY_URL` is set ([git through a forward proxy](#git-through-a-forward-proxy)).

**Which stacks run the sandbox.** The blue/green colours `redeploy.sh` deploys
(`dev/docker-compose.colour.yml`) set the container options below. The other Compose
files (`compose.yaml`, `dev/docker-compose.cloud.yml`) do not: an
AppArmor profile the host has not loaded stops a container from starting, so they keep
Docker's defaults, the probe fails and every repository job runs as a plain process (see
_Without the sandbox_). To run it there, once the host has the profile:

- copy the colour file's `security_opt` block and its `pids_limit` (the container's
  backstop). The `seccomp=` path is relative to the project directory (the compose
  file's folder unless `--project-directory` says otherwise): `./seccomp/…` works
  for the files in `dev/`, and `compose.yaml` needs
  `./dev/seccomp/loom-repo-sandbox.json`;
- the release's standalone bundle (`loom-stack.tar.gz`) ships neither the seccomp file
  nor the AppArmor profile: take both from the same release's source tree and point
  `seccomp=` at where you put the JSON;
- `redeploy.sh` checks only the colour compose. Run `dev/apparmor_check.sh <compose file>`
  by hand before starting another stack, and the throwaway-container check below.

A source run (`make server`) has the sandbox only where the host passes the same
requirements.

**Which hosts can load the profile.** The shipped profile targets AppArmor 4.0 or later
(`abi <abi/4.0>`). Only Ubuntu 24.04 is tested, by the CI `sandbox-profile` job with
`kernel.apparmor_restrict_unprivileged_userns=1`; Debian 13's 4.x parser accepts the
ABI, but the profile is untested there. Hosts without AppArmor, such as Docker Desktop
(on any OS) or SELinux distributions like Fedora and RHEL, cannot use the colours'
options safely: Docker ignores `apparmor=` there, so nothing replaces the `/proc` masks
that `systempaths=unconfined` drops (see _`/proc`_ below). Older parsers (Ubuntu 22.04,
Debian 12) reject the profile; those hosts can run the sandbox only with
`seccomp=unconfined` and `apparmor=unconfined` (the paragraph before _Without the
sandbox_), which lose the same `/proc` protection. The released stack runs on all of
these hosts without the sandbox.

Unless `LOOM_CODE_SANDBOX_REQUIRED` is set, every host still indexes the repositories
listed in `LOOM_CODE_REPOS_JSON`; only repositories added from the page need the
sandbox, or `LOOM_CODE_SELF_SERVICE_GIT_UNSANDBOXED` for their parser tier alone.
_Without the sandbox_ says what that runs, and the
[quickstart](/docs/zh-CN/getting-started/quickstart#optional-index-a-repository-on-the-released-stack)
has the released stack's commands.

**Requirements.** `bwrap` and `prlimit` (util-linux) on `PATH` (the image has both) and
unprivileged user namespaces. Docker's defaults block bwrap three ways, and the colour
compose relaxes each as narrowly as it can:

```yaml
security_opt:
  - seccomp=./seccomp/loom-repo-sandbox.json
  - apparmor=loom-repo-sandbox
  - systempaths=unconfined
```

- **seccomp.** Docker's default profile allows `clone` with namespace flags, `unshare`,
  `mount` and `umount2` only to a container with `CAP_SYS_ADMIN`, and `pivot_root` to no
  container. `dev/seccomp/loom-repo-sandbox.json` is Docker's default profile
  (moby/profiles `seccomp/v0.2.3`, the one Docker 29 ships) plus one rule allowing those
  five to every process in the container; bwrap needs all of them. Inside the sandbox,
  Loom's own filter refuses the job `unshare`, `mount`, `umount2`, `pivot_root` and a
  `clone` with `CLONE_NEWUSER`. A `clone` asking for any other new namespace passes the
  filter and fails for want of a capability, since bwrap runs the job with
  `--cap-drop ALL`.
- **AppArmor.** `docker-default` has `deny mount`. `apparmor=unconfined` does not help
  on hosts that restrict unprivileged user namespaces (Ubuntu 23.10 and later,
  `kernel.apparmor_restrict_unprivileged_userns=1`): there an unconfined process that
  creates a user namespace is moved into the `unprivileged_userns` profile, which
  refuses the mounts. In `deploy/apparmor/loom-repo-sandbox` every process in the
  container runs under `docker-default`, renamed, with `deny mount` kept and, on ABI
  4.0, no `userns` rule, so user namespaces are refused. Only an exec of
  `/usr/bin/bwrap` moves into the child profile `loom-repo-sandbox//bwrap`, which is the
  same plus `userns, mount, pivot_root`. bwrap and everything it starts stay in the
  child profile, so a repository job is allowed those by AppArmor and refused them by
  Loom's seccomp filter above. The boundary is the exec of bwrap: the server and the
  tools it runs outside the sandbox cannot unshare or mount, but any process in the
  container that can exec `/usr/bin/bwrap` can create user namespaces and mount inside
  them, without Loom's filter.
- **`/proc`.** Docker masks paths under `/proc` and `/sys`, and the kernel then refuses
  the fresh procfs mount the sandbox makes; `systempaths=unconfined` unmasks them. That
  also drops the read-only mounts over `/proc/sys`, `/proc/sysrq-trigger`, `/proc/irq`,
  `/proc/bus` and `/proc/fs`, and the masks over `/proc/kcore`, `/proc/keys`,
  `/sys/firmware` and the rest. In both profiles the `docker-default` rules still refuse
  writes to `/proc/sys`, `sysrq-trigger` and the other system files under `/proc` and
  `/sys`, and any access to `kcore` and `/sys/firmware`. On a host without AppArmor,
  Docker ignores the `apparmor=` option and nothing takes the mounts' place, so
  `redeploy.sh` refuses such a host.

**`--disable-userns`.** The child profile keeps `docker-default`'s `/proc/sys` write
denials, which refuse the sysctl write bwrap's `--disable-userns` makes. An allow rule
cannot override a deny, and narrowing the deny globs for both forms the path can take
would make them hard to review. So under this profile, production included, the probe is
expected to see bwrap name `max_user_namespaces`, log it at INFO and run bwrap without
the flag; Loom's filter refuses new user namespaces on its own. Any other probe failure
is logged at WARNING and leaves the sandbox off.

**Loading the profile.** Load it on the Docker host once; `/etc/apparmor.d` is loaded
again at every boot:

```bash
sudo install -m644 deploy/apparmor/loom-repo-sandbox /etc/apparmor.d/loom-repo-sandbox
sudo apparmor_parser -r -W /etc/apparmor.d/loom-repo-sandbox
```

**Checking it.** `apparmor_parser -Q -K` accepts the profile, but whether bwrap runs
under it is known only once a host has loaded it. CI is where that first happens: the
`sandbox-profile` job in `.github/workflows/docker-smoke.yml` loads the profile on
Ubuntu 24.04 with `kernel.apparmor_restrict_unprivileged_userns=1` left on and runs the
probe and the hostile suite in the production image with the colours' options. On a
deploy host, check it in a throwaway container from the build about to be deployed, not
the serving one (`True` means the sandbox runs):

```bash
docker build -q -t loom:sandbox-check .
docker run --rm --network none --user loom -w /app --entrypoint python \
  --security-opt seccomp=dev/seccomp/loom-repo-sandbox.json \
  --security-opt apparmor=loom-repo-sandbox --security-opt systempaths=unconfined \
  loom:sandbox-check -c 'from server.code.app.index import sandbox as s; print(s.sandbox_available())'
docker image rm loom:sandbox-check
```

**What `redeploy.sh` checks.** Before building a colour it stops when the profile is
not loaded, and on a host without AppArmor. After the new colour starts, and before the
proxy is swapped to it, `dev/sandbox_probe.sh` runs the same probe in each of the
colour's containers that runs under the profile (`loom-<colour>` and
`loom-<colour>-workers`). When one fails, or cannot be inspected, it prints why, removes
the new colour's containers and exits with the old colour still serving. Removed, not
stopped: the build already replaced the previous build in that colour, so it is no
rollback target, and its workers cannot take the lease and run jobs unsandboxed. That is
the first check of the real profile on a deploy that skipped the one above.

**Requiring the sandbox.** The probe gates the swap, but the new workers start before
it and may take the supervisor lease (on a first deploy, or when the old lease is free)
and start repository jobs. `LOOM_CODE_SANDBOX_REQUIRED=true`, which the colours set,
closes that window: where the sandbox cannot run, no repository job (clone, fetch,
dependency install, compiler tier, `loom code scip-full`'s harness) runs as a plain
process. The ingest pass and the compiler tiers skip every entry, operator ones
included, and name the skip on the pass note; `loom code scip-full` exits 1; any job
that still reaches the runner fails with `SandboxRequiredError`. `/v1/health` reports
`repo_jobs_unsandboxed` as an error. `loom code init`'s first bare clone is the
worker's clone, a repository job like the rest: it exits 1 with the refusal too.

**Turning it off.** To run the colours without the sandbox again, remove both the
`security_opt` block and `LOOM_CODE_SANDBOX_REQUIRED` from both services in
`dev/docker-compose.colour.yml`, then redeploy. Removing only `security_opt` leaves
every repository job refused and `/v1/health` degraded.

No `no-new-privileges`: under it the kernel refuses the exec transition into the child
profile, and bwrap would not start. The image's setuid binaries (`su`, `mount`,
`passwd` and the like) therefore still work in the container, as under Docker's
defaults; the profile's rules, not the read-only mounts, keep them off `/proc/sys`.

These options still weaken the container's own confinement: seccomp lets every process
in the container make the five calls (outside bwrap, AppArmor and the missing
`CAP_SYS_ADMIN` still refuse them); AppArmor lets bwrap and what it runs create user
namespaces and mount inside them, and any process that can exec bwrap gets that, a
repository job excepted (Loom's filter); and `/proc` and `/sys` rely on the profile's
rules instead of Docker's masks. Weigh that against running repository code without the
sandbox. `seccomp=unconfined` and `apparmor=unconfined` also work where the host does not
restrict user namespaces, but they drop the whole profile.

**Without the sandbox** Loom probes once per process, logs a warning and reports
`repo_jobs_unsandboxed` in `/v1/health` and `loom doctor`. With
`LOOM_CODE_SANDBOX_REQUIRED=true` every repository job is refused (above). Otherwise:

- operator repositories (`LOOM_CODE_REPOS_JSON`) keep every tier, and their build steps
  run as plain processes on the host, as before;
- self-registered repositories whose remote is a network URL are not cloned or fetched
  and get no compiler tier: without the egress proxy nothing would vet the hosts and
  ports a job connects to. The ingest pass note and `workers.status` name the skip; the
  entry resumes once the sandbox is available;
- self-registered repositories on a local path keep the parser tier and the static
  compiler tiers but get no build-based tier (C/C++);
- adding a repository or a branch from the page is refused with `SANDBOX_UNAVAILABLE`.

**Self-registered repositories without the sandbox.**
`LOOM_CODE_SELF_SERVICE_GIT_UNSANDBOXED=true` changes the second and the last item on a
host that cannot run the sandbox. `code.repo.remote_branches`, `code.repo.add` and
`code.repo.add_branch` are no longer refused for the missing sandbox, and the ingest pass
and history reads clone and fetch a self-registered repository whose remote is a network
URL, so it gets the parser tier. It still gets no compiler tier, build-based or static:
their dependency installs, module downloads and submodule fetches would run without the
egress proxy too. `workers.status` names the skip.

The git jobs this allows (`ls-remote`, the bare clone, fetches) run as plain processes.
They run no repository code: a clone or a fetch transfers objects and refs, never hooks
or configuration, the mirror is bare, and git reads no system or global configuration.
Nothing vets where they connect. Git reaches the host and port the remote names and a
host that one redirects to, private and link-local addresses included, and an `ssh`
remote runs `ssh` with the server's `HOME` and the keys in it. Whoever names a remote
therefore chooses what the server connects to, so Loom refuses to load the setting unless
`LOOM_CODE_SELF_SERVICE_PLATFORM_ADMIN_ONLY=true` leaves those three ops to platform
admins ([self-service repositories](/docs/zh-CN/features/code-index/repositories#self-service-repositories)).
With auth off that rule passes a call that carries no principal, so the setting is
refused as well there (`LOOM_AUTH_REQUIRED=false` or
`LOOM_IDENTITY_PROVIDER=single-user`). It is refused with `LOOM_MCP_AUTH_MODE=open` too,
where such a call can arrive with auth on: the rule refuses it, and the setting still
needs every caller authenticated. It is also refused together with
`LOOM_CODE_SANDBOX_REQUIRED=true`, under which no repository job runs as a plain process.

Where `LOOM_CODE_GIT_PROXY_URL` is set, those git jobs connect to the operator's proxy
alone: it decides what they reach, and an `ssh` remote is refused
([git through a forward proxy](#git-through-a-forward-proxy)). The rules for loading the
setting stay as they are.

The setting also resumes the git jobs of entries that members added before
`LOOM_CODE_SELF_SERVICE_PLATFORM_ADMIN_ONLY` was set. They keep the remotes their adders
named and are cloned and fetched the same way. Before turning it on, list the branches in
the registry (the active rows of `loom.code_repos` whose `row_kind` is not `access`;
`code.repos` lists only what its caller may read) and have those a platform admin would
not add removed with `code.repo.remove`.

`/v1/health` and `loom doctor` keep reporting `repo_jobs_unsandboxed` as a warning, with
a message that names the setting. The dashboard reads that warning, not the setting, and
offers no **Add repository** or **Add branch** while it stands: call `code.repo.add` and
`code.repo.add_branch` directly.

Each Go entry has its own module cache under `LOOM_CODE_SCIP_GO_MODCACHE`
(`<root>/<org>/<namespace>`): `go` does not re-check the sums of modules already in its
cache, so a cache one repository's job wrote is never another's.

### Git through a forward proxy

A deployment whose pods reach git remotes only through a forward proxy sets
`LOOM_CODE_GIT_PROXY_URL`. Every network git of the code index then connects through
that proxy: a mirror's clone, fetch and `ls-remote`, the `ls-remote` before an add, and
a checkout's `git submodule update`, in the workers and in every serving process, which
keep mirrors of their own. While the setting is set git never connects directly: a job
that cannot go through the proxy is refused. This holds with git 2.31 or later: Loom
hands git these settings through its environment (`GIT_CONFIG_COUNT`), and an older git
reads none of them, the proxy included.

| Setting                           | Meaning                                                                                             |
| --------------------------------- | --------------------------------------------------------------------------------------------------- |
| `LOOM_CODE_GIT_PROXY_URL`         | the proxy, `http://host:port` or `https://host:port`: the port named, no credentials, path or query |
| `LOOM_CODE_GIT_PROXY_TOKEN_FILE`  | a file holding the proxy password, read again for every git job, so a rotated token is taken up     |
| `LOOM_CODE_GIT_PROXY_CA_FILE`     | a PEM bundle git verifies the remotes with through the proxy                                        |
| `LOOM_CODE_GIT_PROXY_TLS_CA_FILE` | a PEM bundle git verifies an `https://` proxy's own certificate with                                |
| `LOOM_CODE_GIT_PROXY_IDENTITY`    | the proxy user of an entry that names no `git_proxy_identity`                                       |

The other four need the first, the proxy's own bundle an `https://` one, and a value
that does not fit is refused when the settings load.

**What the proxy receives.** Git opens `CONNECT <remote host>:443` with
`Proxy-Authorization: Basic` on its first request: the user is the entry's identity, the
password the file's content (empty without a file). No request comes without them. The
entry's access token goes to the remote, inside the tunnel, never to the proxy. A proxy
that terminates TLS for a remote can then decide, by the identity, which repository a
job reaches and with which credential. An `http://` proxy URL sends the identity and the
token to the proxy in clear on every job: use one only over a path you trust, or give
the proxy an `https://` URL, which git verifies before it sends them (_The CA bundles_
below).

**The identity.** An entry presents its own `git_proxy_identity`, else
`LOOM_CODE_GIT_PROXY_IDENTITY`: letters, digits and `. _ ~ @ -`, at most 128 characters.
An operator entry names it in `LOOM_CODE_REPOS_JSON`; an entry with a malformed one is
skipped. `code.repo.add` and `code.repo.remote_branches` take it from a platform admin
only: the caller `LOOM_CODE_SELF_SERVICE_PLATFORM_ADMIN_ONLY` asks for, with or without
that setting. A branch added with `code.repo.add_branch` inherits it, and adding one to
such an index takes a platform admin too. Branches of one repository share a mirror: the
ingest pass fetches them in one negotiation only where remote, token and identity are
the same, and each host's own fetch round fetches the mirror as its first entry.

**Repositories added with `code.repo.add`.** The proxy is used where repository jobs
run as plain processes, and there `code.repo.remote_branches`, `code.repo.add` and
`code.repo.add_branch` answer `SANDBOX_UNAVAILABLE` unless
`LOOM_CODE_SELF_SERVICE_GIT_UNSANDBOXED` is set (_Self-registered repositories without
the sandbox_). With it, the `ls-remote` of those ops and the entry's clone and fetches
go through the proxy under the entry's identity, so the proxy decides which host they
reach. An entry in `LOOM_CODE_REPOS_JSON` needs no such setting.

**The password.** It is in the environment of the one git process that uses it, read
from the file for that job: not in an argument, a remote URL, a stored git config, a
build's environment or a sandboxed job. Loom masks it in what git prints (as the file
holds it, percent-encoded and in the header's base64 form) before that output is logged
or recorded. That keeps it out of a build's sight, not out of its reach: on a host that
uses the proxy, repository jobs are plain processes, so a compiler tier's build runs as
Loom's user and can read the token file. Run compiler tiers there only for repositories
you trust.

**The CA bundles.** There are two, one for each TLS connection, and neither stands in
for the other. `LOOM_CODE_GIT_PROXY_TLS_CA_FILE` is git's `http.proxySSLCAInfo` (in git
since release 2.27, older than the git every repository job already needs): the bundle
git verifies an `https://` proxy's own certificate with, before it sends the identity
and the token. Without it git checks the proxy against its default roots, so a proxy
whose certificate comes from an authority of your own needs it. The certificate must
also name the host the URL spells, as a DNS name or, for an address, as an IP address:
git checks the name as well as the authority.
`LOOM_CODE_GIT_PROXY_CA_FILE` is git's `http.sslCAInfo`, which replaces git's default
roots for the remotes of these jobs: it must hold every authority the remotes present
through the proxy, the proxy's own for a remote whose TLS it terminates and the public
roots for a remote it tunnels untouched. Loom reads neither file: git does, and a path
that names no file fails the job there.

A certificate git cannot verify fails the job in git's own words, the same for the
proxy's certificate and a remote's (`SSL certificate problem: unable to get local issuer
certificate` with one TLS library; the wording is the library's). A certificate that
names another host fails the same way, in other words of the library. When it is the
proxy's, git ends the connection before its first request, so neither the identity nor
the token leaves. A fetch records git's words on the `fetch` tier, as an unclassified
error. A clone shows `git clone failed (exit 128)` with git's words in the server log,
and the `ls-remote` before an add shows `git ls-remote failed for <repository>`: check
the two bundles and the name on the proxy's certificate when an entry's first contact
fails that way.

**Refused jobs.** Each of these stops the job before any process starts, with a fixed
reason that the entry's `fetch` tier, its stage and the mirror counts in `/v1/health`
show:

- the password file cannot be read or holds no token;
- neither the entry nor `LOOM_CODE_GIT_PROXY_IDENTITY` names an identity;
- the remote is not an `https://` URL: the proxy setting carries no ssh, and a
  submodule with an ssh URL fails as a transport git does not allow;
- the host runs repository jobs in the sandbox, whose egress proxy is not chained
  through this one. `/v1/health` and `loom doctor` report `repo_git_proxy_sandboxed`:
  use the proxy where repository jobs run as plain processes (_Without the sandbox_).

A refusal by the proxy itself reads as kind `egress` on the tier
([when a tier fails](#when-a-tier-fails)), for an entry's clone as for its fetches. The
clone and the `ls-remote` before an add, which show a fixed reason in place of git's
words, name the proxy and the status it answered (`the operator's git proxy
(LOOM_CODE_GIT_PROXY_URL) refused the connection (407)`). A `407` means the proxy did
not take the identity or the token, any other answer is the proxy's own rule for them.

**What does not use it.** A compiler tier's dependency download (the TypeScript package
install, the Go tier's `go mod download`) runs repository code, so it gets neither the
proxy nor its password and connects as before. Loom's HTTP clients (the GitHub API for
PR history and fork lookups, issuer keys, document sources) do not read these settings.

[Kubernetes](/docs/zh-CN/operations/kubernetes#git-through-a-forward-proxy) shows the mounts for the
token and the CA bundles.

### Mirror housekeeping

Each serving process runs `git gc --no-prune` on its host's mirrors every
`LOOM_CODE_MIRROR_GC_INTERVAL_H` (low CPU and IO priority, only with room for a second
copy plus `LOOM_CODE_MIRROR_GC_MARGIN_GB`, 5 GB by default; without it the gc waits
another interval and logs a warning), and sweeps `LOOM_CODE_REPOS_ROOT` of the mirrors,
`checkout-*` trees and mirror lock files no active entry names, so a removed repository leaves
every host, not only the one `code.repo.remove` reached. The
`loom.host.disk.free_bytes` / `loom.host.disk.total_bytes` gauges are the disk alarm's input.

## Measured coverage

A repository whose CI runs an instrumented build with per-test line coverage can feed the
test map with what each test executed, not only what it names. The import reads the
newest complete run from the ClickHouse server that holds the CI's `checks_coverage_lines`
and `checks` tables, in `LOOM_CODE_COVERAGE_SHARDS` slices by file hash. One
slice must fit that server's per-query memory. A server with an hourly quota per statement
shape must also take a whole import (two statements per slice) plus a retry. A public
playground allows 100 statements per shape per hour; the default 16 slices use 16 of each,
so a dry run and a real run fit in the same hour.

It writes:

- one `code_test_map` row per (source file, test), `method = coverage`, when the test
  executed a line at most `LOOM_CODE_COVERAGE_SPECIFIC_MAX_TESTS` tests reach (a line every
  query runs names no test); `confidence` falls with how many tests share the pair's most
  specific line, and `evidence` holds the run's commit and the specific line ranges;
- one `code_coverage_files` row per source file with every range any test executed, so
  `code.coverage_claim` can tell "executed by many tests" from "executed by none".

Rows the newest run no longer has are tombstoned, and only after every slice was read.
`code.coverage_claim` ranks a verified measured row as `executed`, above every other tier,
when its ranges meet the subject's span. Verified measurements for every source file at
the current repository watermark can establish that no test executed the subject, with
the evidence in `measured`.

Test names can be repository-relative file paths or pytest node IDs such as
`tests/test_example.py::TestExample::test_value[param]|run`. The import removes the
function, class, parameter and setup/run/teardown context suffixes and leading `./`, then
requires the test file to exist in the namespace's live index. Paths relative to the
integration or query test directory also resolve through those layouts. Contexts that
map to one source/test-file pair contribute their union of executed lines; overlapping
lines count once, and the pair keeps its strongest measured specificity score.

The source must expose the per-test line records in `checks_coverage_lines` and each
run's commit in `checks`; a coverage database or JSON report needs conversion to those
tables. Re-run `loom code coverage-import --namespace <namespace> --dry-run` to check
`unmapped_tests`, then import without `--dry-run` to rebuild stored pairs from the run.
Updating the server alone does not rebuild imported coverage.

```bash
LOOM_CODE_COVERAGE_SOURCE_URL=https://<coverage server>
LOOM_CODE_COVERAGE_SOURCE_USER=<reader>
LOOM_CODE_COVERAGE_NAMESPACES=<namespace>[,<namespace>]
loom code coverage-import --namespace <namespace> --dry-run   # counts only, writes nothing
loom code coverage-import --namespace <namespace>
LOOM_CODE_COVERAGE_IMPORT_ENABLED=true                         # then daily, in the workers
```

### Coverage freshness

Line ranges describe the source measured by their run. A coverage row supports current
execution claims only when its commit matches the source file's `indexed_commit` and the
namespace has a `git_head` watermark. A symbol's selected spans must carry the same
indexing commit; missing or inverted line extents and lagging compiler metadata remain
unverified.
A missing symbol extent never expands to whole-file execution. The measured
test must also remain live at the matching indexing commit before its pair counts as
`executed`. An unchanged file keeps its indexing commit when the repository advances,
so advancing the head alone does not invalidate positive evidence.

Other coverage rows remain file associations with method `coverage_history`, tier
`file_reference`, and evidence marked `unverified`. Their old line numbers are never
intersected with current symbol spans. These rows alone produce verdict `unknown`.
Independent strong evidence can still establish `covered_by`.

`measured.freshness` is `verified`, `partial`, or `unverified`. Its `files` dictionary
reports each measurement commit, indexing commit, selected `symbol_commits`, and
freshness. `commits` lists all measurement commits; `commit` is empty when they differ.
`paths` and `hits` include only
verified measurements. `unverified_paths` names measurements without matching identity;
`unmeasured_paths` names source files absent from the import. A partial or unverified
measurement cannot establish that the current subject was not executed. A negative also
requires `run_current: true`: all measurement commits match `index_commit`, the current
repository watermark. Tests added after an older run could execute an unchanged source.

Different commits do not prove different file contents. The serving read compares commit
identities without reading historical repository objects, so an unchanged file measured
at another commit can remain unverified. Missing identity metadata also remains
unverified. Matching measurement, source, symbol-span, and test indexing commits verify
line evidence. The import retains historical evidence until a complete run replaces it.

## Another branch

A namespace follows one branch, and a branch is indexed only when an entry names it. Add
an entry with the same `repo` and a different `namespace` and `git_branch`; it shares the
bare clone. Set `embed_reuse_from` to the main branch's namespace so identical chunks
reuse embeddings. Then run `loom code init` and `loom code backfill` for the new
namespace, as for the first index.

## Repairs

`loom code reconcile --namespace NS` lists indexed paths whose chunks predate their last
change at the branch head; `--heal` re-ingests them without moving the watermark.

**After an extractor change.** The code of the modules in `EXTRACTOR_MODULES`
(`server/code/domain/extract/extractor_stamp.py`) and the grammar versions form the
index's version stamp. The stamp hashes code tokens only; comment and docstring edits do
not move it. When a deploy moves the stamp, `code_ingest` re-derives the live files of
every namespace in the language families the change touches,
`LOOM_CODE_EXTRACTOR_RECHUNK_PER_PASS` files per pass, and runs only the stages (chunk,
symbol, edge, guard) whose stamp moved. `code.index_status` shows progress in its
`extractor_stamp*` state keys. On a large index this takes hours.

**After a change outside the stamp** (a language lane, a facet, a new file type),
existing rows are not healed automatically. Re-ingest the affected paths at the
namespace's own watermark:

```bash
HEAD=$(docker exec loom-workers loom code stats --namespace "$NS" | awk '$1 == "git_head" {print $2}')
git --git-dir var/code_repos/<name>.git ls-tree -r --name-only "$HEAD" | grep -E '\.(sql|sh)$' > /tmp/$NS-paths.txt
docker cp /tmp/$NS-paths.txt loom-workers:/tmp/
docker exec loom-workers loom code backfill --namespace "$NS" --commit "$HEAD" --no-fetch --embed \
  --paths-file /tmp/$NS-paths.txt
```

`--paths-file` is repair mode: it re-ingests only the listed paths and leaves the
watermark alone. Files whose content is unchanged keep their chunks and embeddings;
symbols, facets and edges are re-derived. Add `--edges-second-pass` when many files gained
symbols at once.

The optional lanes are switched by settings; after turning one on, run the repair over
the matching paths:

| Setting                           | Default | Lane                                                                                |
| --------------------------------- | ------- | ----------------------------------------------------------------------------------- |
| `LOOM_CODE_SQL_STATEMENT_RECORDS` | off     | `.sql` files as one `statement` symbol per statement, with dialect and table facets |
| `LOOM_CODE_SHELL_ISLANDS`         | off     | SQL issued from shell scripts as statement records                                  |
| `LOOM_CODE_PY_SQL_ISLANDS`        | off     | SQL strings in Python calls as statement records                                    |
| `LOOM_CODE_HOST_ISLANDS`          | off     | CI workflows, Dockerfiles and Makefiles: jobs, stages and targets as symbols        |

`scripts/ops/code_edge_audit.py` is a read-only census of the parser-tier call graph's
two silent holes (declined call sites whose callee now has a definition, and low-grade
edges whose target is gone); run it with `LOOM_BACKGROUND_WORKERS=off` against a staging
copy.

## Maintenance commands

| Command                                      | Does                                                                                                                                                                                                                                                                                                                    |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `loom code compact [--table T] [--apply]`    | measures, and with `--apply` runs, `OPTIMIZE … FINAL` on a code vector table once its physical copies per key reach `LOOM_CODE_COMPACT_MIN_RATIO` (1.5), and waits for the merge; bulk passes do it themselves after `LOOM_CODE_COMPACT_AFTER_BULK_ROWS` rows ([maintenance](/docs/zh-CN/operations/maintenance#compaction-and-optimize)) |
| `loom code warm <questions.txt>`             | asks each question once so `code.ask` answers are cached for everyone (text lines or JSONL with `question`); hits are free, so re-running is safe                                                                                                                                                                       |
| `loom code answers-purge`                    | retires cached `code.ask` answers after a synthesis-model change (the cache otherwise invalidates only when a cited file changes); dry run until `--apply`                                                                                                                                                              |
| `loom code repack`                           | rebuilds context packs for recently merged PRs in the current shape                                                                                                                                                                                                                                                     |
| `loom code crossref`                         | the SQL/shell test → implementing C++ reference pass (idempotent)                                                                                                                                                                                                                                                       |
| `loom code coverage-import [--dry-run]`      | imports the newest per-test line coverage run of the configured source into the test map ([measured coverage](#measured-coverage))                                                                                                                                                                                      |
| `loom code reembed`, `loom code vector-twin` | write a namespace's chunks under a second embedding model's label, or fill a Matryoshka-prefix twin of the vectors, inside ClickHouse; resumable                                                                                                                                                                        |
| `loom code lint-instructions`                | lists paths and `Qualified::symbol` references in `CLAUDE.md`, `AGENTS.md`, `.claude/**` and skills that no longer resolve at head; `--fail-on-stale` for CI                                                                                                                                                            |
| `loom code export-matrix`                    | writes a "when this file changes, these usually change too" table from merged-PR co-change, for an instructions file                                                                                                                                                                                                    |
| `loom code export-pairs`                     | exports (query → acted-on unit, hard negatives) retrieval pairs from real traffic, for embedder or reranker tuning                                                                                                                                                                                                      |

`make bench-code-intel LOOM_BASE_URL=…` runs the code-index quality gate
(`bench/code_intel_qa.py`) against a server with real providers, for a repository that
has a gold question set; it is never run in CI, whose embedder is fake
([benchmarks](/docs/zh-CN/benchmarks/running)).

## Rollout and shutdown

- `LOOM_MCP_CODE_TOOLS` (default off) lists the `code.*` tools on the default MCP
  toolset. The `coding` toolset lists them regardless, and HTTP and the SDKs can always
  call them, so you can evaluate an index before exposing it to chat clients.
- `LOOM_CODE_INTEL_ENABLED=false` makes every `code.*` operation but `code.repo.remove`
  return a precondition error. It also stops the coverage import and notify workers and
  the mirror host loop; every other code worker keeps its own switch
  ([workers](/docs/zh-CN/reference/workers)).
- **Retire a repository.** A maintainer uses **Retire…** on the repository's Access tab
  (`code.repo.remove` with `confirm` set to the namespace). The entry goes inactive, an
  erasure saga purges every `loom.code_*` table for the namespace, and the bare mirror is
  deleted when no other branch uses it. Search telemetry is kept. An operator-configured
  entry is refused with `REPO_FROM_CONFIG`: remove it from `LOOM_CODE_REPOS_JSON`
  instead. With `LOOM_CODE_INTEL_ENABLED=false` the page cannot list repositories, but
  `code.repo.remove` over HTTP, MCP or the SDK is still served; the other replicas then
  sweep their copy of the mirror only once code intelligence is on again.
- **Erase an index.** `erasure.request` with `scope_type = namespace` on a code namespace
  also purges the `loom.code_*` tables. An entry that is still configured is re-indexed
  from scratch on the next pass, so retire it first.

## When a tier fails

Each tier that keeps an entry current (the mirror fetch, the parser ingest and every
compiler tier) records its last outcome on the entry: when it fails, the error with
credentials redacted, an error kind, the one fix for that kind, `failing_since` and the
failed passes in a row. The next pass that succeeds clears it. A tier run from cron
records the same way when it runs under `loom code entry-lock --tier <tier>`;
`loom code scip-delta` and `scip-full` record by themselves.

A Go module whose dependencies this host cannot fetch (Go reports a private repository,
often the repository's own modules through a vanity import path) is not a failure: the
Go tier indexes the rest and records the tier `unavailable`, kind
`dependency_unavailable`, with the fix (set `LOOM_CODE_SCIP_GO_PRIVATE` and give the
tier's user a git credential for that host; Go says the same of a path or version that
is wrong or gone). The note is written once, counts no passes, is never a
`code_tier_failing` problem and is not in `as_of.failing_tiers`, so an agent's code
answer does not flag those modules. It reflects the last run that built modules: a pass
that builds nothing leaves it, and the next run that builds without such a failure
clears it, so it comes back when a module that still cannot fetch its dependency next
changes. Under `entry-lock --tier go`, which sees only its command's exit and output, a
clean exit with no such failure clears it. A module that fails for any other reason
still fails the tier, and its record names the unreachable ones too.

The record is read where each reader already looks:

- an agent's code answers carry it in `as_of.failing_tiers`
  ([freshness](/docs/zh-CN/features/code-index/freshness#the-as_of-block));
- `code.index_status` and `code.repos` return it in `tier_health`, and the dashboard's
  repository card shows the failing tier, since when, how far behind and the fix, and
  any `unavailable` tier with its fix;
- once a tier has failed `LOOM_CODE_TIER_FAILING_WARN_PASSES` passes in a row or for
  `LOOM_CODE_TIER_FAILING_WARN_HOURS`, it is the `code_tier_failing` problem in
  `/v1/health`, the dashboard banner and `loom doctor`
  ([monitoring](/docs/zh-CN/operations/monitoring#v1health)): a line per failing entry and tier, up to
  five, with its reason, and each distinct fix once. `loom doctor`, which reads with the
  deployment's own credentials, also names each entry it lists, up to five, as
  repository@branch (org/namespace) with the error line that says why; `code.repos`
  names the rest. A platform admin's token reads the same for every entry that is not
  [restricted](/docs/zh-CN/architecture/security#restricted-namespaces); a restricted one reads
  as "a restricted entry", as does one whose classification cannot be read. Any other
  caller reads no entry, since `/v1/health` answers without a token;
- for push alerting, the `loom.code.tier_failing` and `loom.code.tier_lag_commits`
  gauges carry each tier the `code_tier_failing` problem reports
  ([telemetry](/docs/zh-CN/features/observability/telemetry)), with an alert in
  `deploy/dashboards/code_plane.json` and, for Prometheus, the chart's rule
  ([alert rules](/docs/zh-CN/operations/kubernetes#alert-rules)).

The kinds (`_KINDS`) and their fixes (`fix_for`) are in
[`server/code/domain/index/tier_health.py`](https://github.com/ClickHouse/loom/blob/main/server/code/domain/index/tier_health.py).
A failure that matches no kind reads "unclassified error" and carries the end of
the output to read.

## Failure model

| Failure                                                                                                             | Behaviour                                                                                                                                                                                                                                                                                | Recovery                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| ------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `git fetch` fails                                                                                                   | watermark unmoved; the entry's `fetch` tier records the error, its kind and fix (`tier_health`)                                                                                                                                                                                          | the next pass (supervisor backoff)                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| crash mid-commit                                                                                                    | the commit is re-run                                                                                                                                                                                                                                                                     | idempotent upserts                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| a file fails to parse                                                                                               | chunks without symbols, `code_files.index_error` set (a parse past `code_parse_timeout_s` or `code_parse_memory_mb` included, and one that could not run in its process)                                                                                                                 | `code.index_errors` lists them; fixed by the next change or a repair (raising either limit re-reads none)                                                                                                                                                                                                                                                                                                                                                                                      |
| every file fails to parse with `LookupError` "... is loaded from outside the interpreter's site-packages"           | chunks without symbols for every file: tree-sitter or a grammar is installed where the parse process does not see it                                                                                                                                                                     | install the tree-sitter packages into the server interpreter's own environment, not through `PYTHONPATH` or the user site                                                                                                                                                                                                                                                                                                                                                                      |
| embedding provider down                                                                                             | chunks stored without vectors; lexical search still serves                                                                                                                                                                                                                               | `code_embed_recovery`                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| compiler-tier apply fails                                                                                           | batch `failed` with the error                                                                                                                                                                                                                                                            | `loom code scip-retry <batch_id>`                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| a compiler-tier pass fails or hangs                                                                                 | the harness's timeout ends a hang, and the pass deadline (that timeout plus headroom for the git steps and the apply) cancels anything else; the entry lock is freed; a failure the pass reports (the harness's own timeout included) is recorded with its kind and fix in `tier_health` | the next pass; `workers.status` names the timeout                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| compiler-tier window base unknown (history rewritten, pruned mirror)                                                | every `scripts/scip_harness/` tier exits 3, never an empty delta; the Python, TypeScript and Go workers clear the tier's watermark                                                                                                                                                       | the next run is a full index for that tier (`code_scip_full` for C/C++)                                                                                                                                                                                                                                                                                                                                                                                                                        |
| a compiler-tier delta is a link, not a regular file, or oversize, or the Go harness writes more deltas than modules | the delta is refused (`FileRefused`, or `Go harness wrote more deltas than its N module(s)`) and the entry's pass fails; the watermark stays                                                                                                                                             | oversize: raise `LOOM_CODE_SCIP_DELTA_FILE_MAX_MB` and `LOOM_CODE_SCIP_DELTA_MAX_NODES` (they bound work and the path text kept); for a file entry over the round budget raise `LOOM_CODE_SCIP_DELTA_ROUND_MAX_NODES`, within the worker's memory; a file entry over the fixed character cap is named, and no setting changes that cap: the file it indexes is what to split; a link, a non-regular file or extra deltas recur every pass until the repository changes or the entry is removed |
| GitHub rate limit                                                                                                   | the pass ends, cursor unmoved                                                                                                                                                                                                                                                            | the next pass                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| branch force-pushed (watermark no longer an ancestor)                                                               | watermark reset to the new head, logged                                                                                                                                                                                                                                                  | `loom code backfill` to re-derive                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| weekly graph stats fail                                                                                             | last week's ranks keep serving                                                                                                                                                                                                                                                           | the next weekly pass                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |

The code index's gauges (`loom.code.index_lag_commits`, `loom.code.stale_files`,
`loom.code.tier_lag_seconds`, `loom.code.embed_backlog`, `loom.code.files_errored`) are
catalogued in [telemetry](/docs/zh-CN/features/observability/telemetry); what to alert on is in
[monitoring](/docs/zh-CN/operations/monitoring#what-to-alert-on), and the "Loom — Code Plane" dashboard is
defined in `deploy/dashboards/code_plane.json`.

## Related

- [Repositories](/docs/zh-CN/features/code-index/repositories): self-service repositories,
  maintainers and readers
- [Freshness](/docs/zh-CN/features/code-index/freshness): how a client reads index lag
- [Code index architecture](/docs/zh-CN/architecture/code-index-architecture)
