Design¶

This page records the decisions behind nxr and the alternatives they beat. The exact wire behavior lives in the protocol reference, the commands in the CLI reference, and the test evidence behind every guarantee in Conformance.

The curl model¶

Scripted artifact flows grow hand-rolled curl snippets: retries, stall detection, digest checks, resume, rewritten in every repository that needs them. nxr replaces those snippets with one binary whose every invocation is self-sufficient: URL in argv, credentials from -u or the environment, nothing else.

Config files were rejected for one reason: transferability. A pipeline must be able to reproduce any call from any machine that has the binary and the environment. An rc-file quietly breaks that property, because the call now depends on a state the URL does not carry. The cost is bounded: repeating a long URL is the caller's problem, solved with a shell alias or a CI variable, which is the caller's own tooling rather than a second configuration format.

Four layers, dependencies point one way¶

The crate is embeddable at four depths. Dependencies point one way, from UX down to transport, and no layer ever imports upward.

Each layer is blind on purpose. L0 knows HTTP, retries, stall detection, TLS and auth, and does not know that directories or markers exist. L1 knows scans, diffs, markers, workers and Range-resume, and does not know that versions or channels exist. L2 knows channels and listings, and does not know the CLI exists. L3 is the only layer allowed to talk to a human.

The point of the blindness: a wrapper that wants raw PUT-with-retry embeds L0 and nothing above it. One that wants verified directory sync without opinionated naming embeds L1. Both enter through the Rust facade.

Completion is bytes plus a marker¶

Checksum catalogs, name lists and lockfiles share one flaw: the catalog and the content are two objects with two lifecycles, and every divergence between them is a bug class. The sha-sibling puts the proof next to the bytes, in sha256sum -c format Unix tools already speak. A name is complete exactly when bytes and sibling both exist and the digest matches.

Markers are mandatory by default because a tool that writes bytes should not leave them uncertified. A Markerless object is unverifiable, which makes it exactly as useful as a missing object, only harder to notice. up generates missing local siblings before uploading and writes the remote sibling strictly after the bytes. --no-sha makes the opt-out an explicit, greppable decision recorded in the command line instead of a default anyone gets by accident. The result of that flag is Markerless objects the server never certifies: a deliberate downgrade, and the tests pin that it stays one.

One diff, two directions, refusals included¶

The same classification table answers "what is missing here?" for both directions: up asks before sending, down asks before receiving. Four states per side (Absent, Markerless, Broken, Complete) remove the two hardest bugs of scripted artifact flows: re-uploading what is already there, and consuming a half-written file. States are per side, and the decision is about the pair. The full table is in the protocol reference.

The gap between Absent and Markerless is the crash window: the bytes landed, the marker write is still ahead.

The table is conservative in a direction that surprises people once: divergent objects are never overwritten. A local Complete and a remote Complete with different digests is a mismatch: the transfer refuses and escalates the decision to a human with exit 1. The client does not pick a winner because it cannot know which side is the truth, and an artifact store is the wrong place to find out. The same refusal protects a Broken marker on either side, because a foreign object is never shadowed.

Mirror-delete was rejected for the same reason. A remote-only name found by up is a mismatch, not a to-do: acting on it would mean deleting on a server's behalf, and a client that deletes is a different tool. As far as this client is concerned, the repository stays append-only.

The core is resume, and resume is asymmetric on purpose¶

An upload is a pure function of the file content: replaying it after a break converges on the same result. That is why uploads need no resume protocol (no offsets, no sessions, no server state), and why the cost is bandwidth on re-transfer. Correctness never depends on the transport remembering anything.

Downloads can and do resume, because HTTP Range is already there. down keeps each name's bytes in a stable part file (.nxr-part-<hash>) by default, sends Range: bytes=N- when the part holds N bytes, appends on 206, and restarts from zero on a 200, the answer of a server that ignored the range. A resumed part that belongs to an older remote version fails the digest check and is discarded once, and the name restarts from zero under the same check, so a rerun self-heals after a remote update instead of refusing. --fresh starts every name over, and only a fresh download that still diverges refuses with exit 1. The asymmetry that remains is deliberate: the get primitive resumes only under --continue, because a primitive trusts the caller to know its part files, while down resumes by default, because a rerun of a directory transfer should converge.

416 means the part already holds the whole object: the previous run died between the last downloaded byte and the rename. The client finalizes the part, digest-checks it against the remote sibling, and renames or discards. No bytes are re-fetched for an already-complete part, and that exact absence of a range fetch is pinned by a conformance test.

One upstream quirk is documented honestly. The Nexus version this client targets answers 500 instead of 416 when a Range request arrives for a zero-byte object, because the range computation in its PartialFetchHandler builds a Guava Range.closed(0, -1) and crashes. nxr never ranges a zero-length object: a Range header goes out only when the part already holds bytes, and a complete part of a zero-byte object is empty, so the request is a plain unconditional GET. That makes the quirk upstream behavior worth knowing rather than a client contract.

Segmented transfers are an optional assembly¶

A single artifact can be split into N closed Range spans (--segments N, 2..=64). The spans stage as separate files under .nxr-segments-<32 hex>, assemble positionally into the usual part, and one pass hashes the result. The staging directory carries a manifest with the object size and the span count. A mismatch means the object changed between runs, so the staging is discarded instead of assembling a tiling that no longer matches. A finished span survives a killed run. An unfinished one starts over.

The worker budget still caps total concurrency: spans and whole artifacts acquire the same permits, so --segments changes the shape of a transfer but not the number of connections. The spans request Accept-Encoding: identity, because byte offsets only address the stored representation. A compressible object can therefore transfer more bytes than a compressed single stream. When a server answers a ranged GET with 200 and the whole body, the staged spans are discarded and the object falls back to the single stream. A zero-byte or unsplittable object takes the single stream from the start.

Segments are opt-in because the win depends on the link and the server: one large artifact on a high-latency path benefits, many small artifacts usually do not.

A clone cache is a hint, not a state file¶

clone re-reads the local tree on every run, so it can skip a file whose size, mtime and marker digest still match the last verified copy. That cache lives at <DIR>/.nxr-state.json and is an accelerator, never authority. A missing, corrupt or foreign cache means "hash everything", not "trust me". --verify ignores the cache for one run, and verify, audit and doctor never read it. The cache never decides existence: the local stat and the remote probes do.

The model stays curl-like because the cache is optional and local to a clone. No other command reads it, and no correctness rule depends on it.

Enumeration is explicit¶

A raw repository has no guaranteed directory listing. The search API varies between server releases, and HEAD-probing for likely names is guessing with extra requests. So down requires an enumeration source (explicit names, a name file, a file URL whose last segment is the name, or an explicitly requested best-effort --ls) and refuses with a hint when it has none.

The caller owns the name list: repeat --name, or write the list to a file and pass --names-from. Nothing is reserved: a publisher who wants a machine-readable index ships one as an ordinary artifact and documents its name, and nxr never reads it. A name list the client cannot know stays a hard refusal, the same philosophy as the exit codes: when the client cannot know, it says so and asks.

Channels generalize pointers¶

v0.2 reserved two pointer names. v0.3 reserves none. A channel is any token file (latest, stable, prod-1), and the meaning of a name belongs to the publisher rather than the tool. channel set writes the token, and --if-forward compares tokens in dotted-numeric version order, which keeps the one property that mattered: a late CI job cannot roll a release back.

What was rejected along the way is claim immutability. The old reserved pointer tried to be an immutable claim, but a dumb store cannot enforce immutability, so the guarantee was fictional and the special case leaked into every layer. The special case was deleted and replaced by a comparison the client can perform.

The transport trusts no single signal¶

Retries cover connect errors, timeouts, broken bodies and 5xx but never 4xx, where retrying would just re-authenticate a refusal. A stall timeout replaces a total timeout: a 100 MB artifact over a slow link may legitimately take hours, but no bytes for 30 s is a dead connection. Auth failures bypass the retry loop entirely, because hammering a 401 is never useful. Exact numbers live in the protocol reference, and the recovery story in Conformance.

A 429 with Retry-After gets the additive-increase, multiplicative-decrease treatment: retries honor the header's wait, and the worker pool halves in flight at once, then re-grows one worker per success streak. The design separates two clocks: the retry loop is per-request patience, the worker budget is per-run pressure, and a polite client under throttle needs both. The pool never exceeds --workers and never reaches zero, so a throttled transfer converges to the pace the server allows instead of failing or hammering.

Exit codes encode the operator's decision¶

Data problems (exit 1) need a human, invocation problems (exit 2) need a pipeline fix, and transport problems (exit 3) need another attempt later. Every error carries a hint: line so the decision is cheap at 3 a.m. The full mapping is in errors and exit codes, and when it breaks turns it into a procedure.

Read-only diagnostics¶

audit and doctor --deep are read-only by contract: no request either command makes mutates the repository. The guarantee is enforced by construction, not by politeness: the diagnostics run on the same transport as everything else, but their probe and finding paths contain only GET and HEAD, and the conformance tests assert that a full audit or deep-doctor run against mock-nexus issues no other method. Write rights are deliberately not probed by doctor: a PUT probe would itself be a mutation, so the absence of write access surfaces on the first real up, with its usual exit code and hint. The one deliberate extension to reading is audit --bytes, which downloads and hashes asset bytes: it reads more, and it still writes nothing.

What was deliberately left out¶

Config files and profiles, repacking, unarchiving, a daemon, a server component. Deletion is the one deliberate extension, and it stays narrow: rm deletes exactly the enumerated names of one version, point deletes exactly one pointer (the protocol). The repository stays a dumb, verifiable object store. The client stays a static binary whose dependencies of consequence are reqwest and tokio. Working within those walls: publishing a version, consuming artifacts, CI recipes.