MityaMitya is Dmitri Karamazov. Alyosha carries the mercy, Ivan carries the law, and Mitya carries the passion: the brother who burns through everything he touches, himself included.Mitya is Dmitri Karamazov. Alyosha carries the mercy, Ivan carries the law, and Mitya carries the passion: the brother who burns through everything he touches, himself included.

1 · What it is

A question answering system over Dostoevsky's five novels. It answers in his voice, and it prints the chapters the answer came from.

It is two subsystems that share no code. One is a fine tuned model that supplies the voice. The other is a retrieval stack that supplies the substance. They meet in exactly one place, the prompt. The Sources list under each answer is not generated text: it is built in code from retrieval metadata, because the fine tune was trained to refuse external references. If the model tried to cite something, that would be a bug, not a feature.

The first version asked a hardware question. Could a grounded, in character assistant run on one 6GB consumer GPU. It could, and every design decision traced back to that budget. The second version asks a delivery question instead. At a fixed quality bar, what is the cheapest and most reliable way to serve grounded answers, and can the system prove it stays that way. Answering that moved generation off the GPU and behind a two family provider gateway. The retrieval stack barely moved, which is a result rather than an omission.

2 · Where it came from

The first version shipped and worked. A QLoRA fine tune quantized to GGUF, a structural parse of the novels, hybrid dense and sparse retrieval with native RRF, a reranker, an offline entity map, chapter summaries, FastAPI over SSE, a Vue front end. All of it on one 6GB card. Every decision in that list traces back to the same budget: the budget picked the quantization, picked a small CPU reranker, picked local mode Qdrant, picked a global lock of one, and picked a 400 token ceiling on generation. Its own closing finding was that the remaining quality ceiling was the fine tune, not the pipeline.

What it could not do was answer questions about itself. It could say a request took about twenty seconds and could not say where the time went; the reranker's share was an estimate written into a design document and never measured. It could not say which configuration produced a given number, because settings were module constants and environment variables and provenance was whatever the operator remembered. And its headline retrieval number turned out to have been produced by a harness that asserted a route, logged that the route was fine, and then ran the default path for every row anyway.

The second version started by rebuilding that. Once generation moved off the box, three defaults inherited from the 6GB era stopped making sense. The global lock of one had been justified by running a single local model, but with generation gone it was binding on retrieval alone, and it was measurably the wall. The 400 token generation ceiling had been sized to fit a local context window, and carried into hosted serving it cut most answers off mid sentence. And "the GPU is already paid for" stopped being a cost argument at all, because the GPU was no longer in the request path. None of these were wrong when they were written. They were correct answers to a constraint that no longer applied, which is the failure mode worth naming: a default inherited from a constraint you have escaped is not a default, it is a bug with a plausible history.

Most of the system did not change, and that was deliberate. The corpus parse, the chunking, the index, the summaries, the router, hybrid retrieval and RRF, the reranker and the front end all stayed. What changed was the entity map, context assembly, prompt composition, generation, the cache, and the API surface. The fine tune was retired from the request path rather than deleted or retrained. What is genuinely new is everything that lets the system make claims about itself: a measurement contract, telemetry with config identity, a provider gateway, a CI evaluation gate, a traffic harness, auth and tenancy, conversation memory, adversarial evaluation sets, and an optimizer. The unchanged list is not a list of omissions. Each item on it is a decision to spend a scarce resource, quota and an exclusive lock on the vector store, somewhere else.

3 · How it works

A question arrives over SSE. The router decides whether it names a chapter, which picks a structural path or the default one. Retrieval runs both legs in a single Qdrant round trip: a dense leg over quantized ONNX embeddings on CPU, and a BM25 leg, submitted as prefetch clauses and fused server side by native RRF. FlashRank cuts the fused pool to the three spans that are served. Those spans, the conversation memory and the persona prompt compete for one token budget, and passages win by policy, admitted whole. Generation happens at a hosted provider behind a two family gateway. When the stream ends, the Sources list is assembled in code from retrieval metadata, never parsed out of the answer.

Two things the diagram makes visible. The lock covers everything from routing through context assembly and is released before the first token streams. And which configuration produced any of this is a content hash, so an arm is switched by promoting an artifact, never by setting an environment variable.

4a · Retrieval

Five novels are parsed into 231 chapters, chunked at 500 tokens with 100 overlap, and indexed as 3,284 points that carry book, part and chapter as payload rather than as text. A question is routed by a pure function with no model in it: name a resolvable chapter and ask for a summary and it takes a structural path, otherwise the default one. That path expands the query with canonical names and aliases from a human gated entity map, runs a dense leg and a BM25 leg in a single Qdrant round trip fused server side by RRF, then reranks down to the three spans that are served.

Two things went wrong, and both were in the measurement rather than in the stack.

The published recall number was the recall of a pipeline production does not run. The harness called the router, asserted the route, logged that the route was fine, then ran the default hybrid path for every row regardless. The structural retrieval function was imported by production, imported by the quality harness, and imported by nothing in the retrieval harness. The assertion passed the entire time, because a passing assertion sitting next to an unused result is not a control.

Then recall turned out to be bimodal across repeated runs of one unchanged commit. The entity map's scan returned a set, so any query touching more than one character got its alias blocks back in a different order depending on the process hash seed. Different order, different expanded query, different embedding, different ranking. That is not only a measurement problem: two users could ask the same question and get different chunks.

Both fixed, and the baseline re-derived: recall@5 of 0.7105, 27 of 38 rows on the primary family.^[1][1]0.7105 = 27/38, denominator sequence 26 / 25 / 38; MRR 0.5039.[1]0.7105 = 27/38, denominator sequence 26 / 25 / 38; MRR 0.5039. v1 had published 0.720, and the new number is not a regression from it, because the denominator moved three times in between.^[2][2]v1 0.720 = 18/25, superseded.[2]v1 0.720 = 18/25, superseded. The two were never measuring the same set. Knowing which configuration produced which number is a separate problem, and the system did not have an answer to it yet.

One large defect was found and deliberately left alone. Retrieval never sees the conversation. In the session corpus, 68 of 108 turns are contentless follow-ups, so 63 percent of multi turn requests retrieve against a string with nothing in it to retrieve on.^[3][3]63% = 68 of 108 turns.[3]63% = 68 of 108 turns. Documented, unrepaired, and the biggest known gap in the system.

4b · Generation

The prompt is assembled in blocks: a system message, the persona head, a precedence marker that names each block and declares it data, the memory block, the conversation, and a reinforcement tail. The user turn carries the question and nothing else. Generation runs on hosted APIs behind a gateway speaking to two provider families, picked for how much they diverge in streaming shape, error vocabulary and rate limit surface, and each reached through its own native API rather than a compatibility endpoint. The first token is pulled before the response body opens, so a transport failure, a refusal or a size rejection arrives as a typed status instead of a hung 200.

The first adversarial evaluation this project ever ran found that the prompt's injection fence was doing nothing. Against a control differing in exactly one respect, the shipped prompt was set-identical: the same attempts landed, with an empty symmetric difference in both directions. Its replacement, composition C, then nearly lost to a broken metric. An assistant-tell detector scored C four times worse for breaking character, and all 26 of 26 matches turned out to be the bare literal "I cannot" inside in-character narration, which is the persona correctly refusing. The metric was punishing the arm that obeys its own grounding instruction, and published unread it would have said the safer prompt breaks character four times as often.

Composition C shipped, and it is the only change the entire optimizer phase promoted. It cut jailbreak violations from 0.5625 to 0.1250 across 16 attempts, a 78 percent reduction in that family,^[4][4]0.5625 → 0.1250 over 16 attempts, jailbreak family only.[4]0.5625 → 0.1250 over 16 attempts, jailbreak family only. and on the quality leg it came back superior on groundedness rather than merely non-inferior, +0.724 against a detection floor of 0.343.^[5][5]+0.724 against a measured MDE of 0.343: superior, not merely non-inferior.[5]+0.724 against a measured MDE of 0.343: superior, not merely non-inferior. Prompt extraction did not move, holding at 1 in 8 on every arm tested, which is where the result stops.^[6][6]1/8 on all three arms: control, shipped, promoted.[6]1/8 on all three arms: control, shipped, promoted. The two legs ran on different models, so they are two results rather than one.

The output ceiling from section 2 was raised from 400 to 1024, and the change was recorded as a measurement break enforced in code: ask the gate to compare a number from before it against one from after and it returns an error naming the break instead of a delta.

4c · Serving and degradation

/chat takes a session cookie, resolves it server side on every request, derives the tenant, and streams SSE events: tokens, then sources, then meta. Sessions are opaque server side rows rather than JWTs, so a session can be revoked; the cookie is HttpOnly, Secure and SameSite Lax, and it rotates on login. The inbound tenant header was removed rather than supplemented, because an authenticated tenant that an unauthenticated header can override is not authenticated. In front of that sits pure ASGI admission middleware holding one in-flight slot per request, bounded per tenant and in total, releasing on all six exits including client disconnect.

The admission layer exists because of what the first load test found. The system had never rejected anything: at every concurrency level, on both arms, zero local 429s, zero provider 502s, not one erroring request, and a 62 second p99 at a 100 percent success rate. It was queueing silently and without bound, which is the worst shape a reliability failure can take, since every request succeeds and the only symptom is a tail nobody is watching. Bounding it at two per tenant and six in total, with the inequality chosen so the total binds before every tenant reaches its own cap, took p99 on the same arm at the same level from 64,405 ms with zero rejections to 4,429 ms with thirteen.^[7][7]p99 64,405 ms / 0 rejections to 4,429 ms / 13.[7]p99 64,405 ms / 0 rejections to 4,429 ms / 13.

The five rung degradation ladder was written but had never been wired into the request path, and the readiness probe it depended on was querying llama-server, which no longer generates anything. Both were fixed together. Readiness is now deliberately config-only, configured and credentialed rather than reachable, because the front end polls it every few seconds against a free tier that allows very few requests per minute.

The concurrency knee moved from one to four after the lock was narrowed to cover routing through context assembly,^[8][8]knee 1 -> 4 (first measurement, v1 published no concurrency number).[8]knee 1 -> 4 (first measurement, v1 published no concurrency number). and authentication went in at no measurable latency cost.^[9][9]auth 2,045 ms p50 against 2,194 ms pre-auth.[9]auth 2,045 ms p50 against 2,194 ms pre-auth. What the admission bound buys is a bounded wait, not more throughput: retrieval is still mutually exclusive, so the lock was narrowed rather than parallelized.

4d · Telemetry and evaluation

One span per stage boundary feeds both the trace and the metric from a single set of instrumentation calls, so a timing and its distribution cannot disagree. A configuration version is the content hash of a frozen validated model, twelve hex characters of a canonical serialization, and every span, every metric and every evaluation row carries it. Two identical configurations are the same version by construction. Runs are stamped with the config version, the corpus version, the git SHA, whether the tree was dirty, the serving model and the prompt composition. On top of that sits a CI gate that compares the promoted configuration against a candidate in three layers, with no hand set threshold anywhere: every bar is derived from the run being judged.

That design came out of finding what happens without it. A configuration version that did not pin the generation model meant the version did not churn between two provider arms, so arm B was served arm A's cached answers, contaminating quality, then latency, then cost, in that order. Setting an environment variable to select an arm turned out not to switch arms at all: both runs carried the same config version, labelled nothing, and produced scores plausible enough to have been believed. The rule that came out of it is short. To measure an arm, promote a configuration for it.

Then the gate was asked what it could actually see. Strip every retrieved passage from the prompt, which is a total and certain regression, and the two mechanical metrics move by 0.0000. Identical zeros. The LLM judge catches it at 5.4 times its own detection floor,^[10][10]mechanical 0.0000 / 0.0000 vs judge groundedness 4.022 -> 1.000 at 5.4x MDE.[10]mechanical 0.0000 / 0.0000 vs judge groundedness 4.022 -> 1.000 at 5.4x MDE. but the mechanical layer does not respond at all, and no sample size fixes a metric that does not respond. Worse, the ungrounded configuration is 32 percent cheaper and passes the gate clean,^[11][11]ungrounded arm 32% cheaper.[11]ungrounded arm 32% cheaper. so anything optimizing against cost under that gate is being steered toward it. The gate's report now carries a permanent caption excluding grounding from what it verifies, enforced by the report validator rather than by convention, and the schema rejects any overall, score or composite key outright, because a scalar is a weighting and the weighting is the thing left open.

The number worth ending on is the ratio. Across the phase, thirteen or more defects were found in the instruments, and two in the product.^[12][12]instrument defects 13+ vs product defects 2.[12]instrument defects 13+ vs product defects 2. Not one of the thirteen was caught by a run that failed. They were caught by runs that passed and looked fine.

4e · Cost and performance

Three cost numbers exist because they answer different questions. A reconstructed estimate computable from a configuration and a manifest alone, which the CI gate blocks on. An assembled estimate that requires retrieval and the memory read to have actually run, which the optimizer ranks on. And a realised figure from provider reported usage. Free tier calls are priced at published list rates and flagged unmetered, so any aggregate containing one renders as an estimate rather than a bill.

The first thing worth measuring was where the time went, because v1 had never known. Its design document justified the reranker at roughly a fifth of the retrieval path. One trace put it at the overwhelming majority of it. Retrieval proper is flat at about 256 ms regardless of depth, so every millisecond above that is reranker work. Cutting the fused pool from 25 to 10 took retrieval wall clock from 2,121 ms to 917 ms, a 2.3 times reduction, and recall@5 did not move at any depth tested between 25 and 5.^[13][13]fused 25 -> 10, 2,121 ms -> 917 ms, recall@5 flat at 0.7105.[13]fused 25 -> 10, 2,121 ms -> 917 ms, recall@5 flat at 0.7105. It is a latency axis, not a cost axis: the same change is worth about 0.04 percent of the money. The one sentence version of the whole phase is that generation stopped being the bottleneck and retrieval became it.

Context assembly gained the ability to merge adjacent retrieved spans and drop the text they duplicate. Over 97 retrieval rows it cut mean served span tokens from 1,535 to 1,502, with recall, coverage and multi hit rate identical, 28 rows cheaper and none more expensive.^[14][14]merge-only 1,535 -> 1,502 tokens, -2.19% over 97 rows.[14]merge-only 1,535 -> 1,502 tokens, -2.19% over 97 rows. It ships as the default collision policy, and enabling it in the request path is a pointer write that is planned rather than done.

The cost model is exact and the spend is nothing. A production answer costs $0.003982 at list rate, the cheapest measured arm $0.001325, and actual spend across the phase was $0.00, because every rate row is unmetered.^[15][15]$0.003982 production / $0.001325 cheapest / $0.00 actual.[15]$0.003982 production / $0.001325 cheapest / $0.00 actual. Routing between arms was rejected before it was built: the best routed family came out 74 percent more expensive than always sending traffic to the cheapest arm, and the hierarchy is inverted, since the cheaper arm is the larger model. That result is bounded by what was measurable on a free tier, which is where it stops.

5 · Limitations

Answer quality is capped by the fine tune, not by the pipeline. v1 established that by measurement and v2 did not touch it, because weight training was out of scope by charter. Everything v2 improved sits downstream of a ceiling it deliberately did not raise.

Retrieval blindness ships unrepaired, and the reason is more interesting than the defect. The naive fix, feeding the transcript into retrieval, has a live objection against it: if the model answers from the conversation while the Sources list cites what the current query retrieved, the citation lies. A grounded system that cites the wrong thing is worse than one that admits it retrieved nothing. The repair needs the Sources contract to change first.

The composed quality gate ships empty and cannot block. That is a decision nobody has taken rather than a defect, and after measurement the honest description is that the gate cannot be called an answer quality gate at all. It is a catastrophe detector with an exact cost model attached.

Chunking was read from source in v1 at 500 tokens with 100 overlap and has never been measured. No chunking number exists in this project. Three arms is about an hour of local CPU, but the success branch re-freezes the corpus stamp and invalidates the phase's entire evidence base, which is why it is priced rather than done.

The degradation ladder descends on misconfiguration rather than on outage, because readiness is deliberately config only. On a real provider outage it does not descend at all.

And the largest scoping caveat: there is no real traffic anywhere, ever. Every harness figure and every session figure is synthetic by deliberate construction, and the synthetic query distribution decides in advance whether a retrieval component can demonstrate value at all.

6 · What is next

The most expensive stage in retrieval is the one carrying the least evidence. Reranking is 661 of 917 milliseconds, 72 percent of the budget, and no arm has ever run without it. Recall@5 is flat at 0.7105 across every fusion depth, and the served differences in that sweep are one or two rows against a minimum detectable effect of 0.144, which is noise by this project's own rule. The next run isolates it: the same harness, one arm with the cross encoder removed.

That measurement is only as good as the set it runs on. Thirty eight primary rows put the minimum detectable effect at 0.144, which is why v1's per stage ablation ended in withdrawal rather than a finding. Minimum detectable effect falls with the square root of the row count, so halving it takes roughly four times the data, around a hundred and fifty rows, spread across the query families rather than concentrated in the ones that are easiest to write.

Fusion is worth re-examining, though not for the reason it first appears. Retrieval proper is flat at about 256 milliseconds across every fusion depth, so reciprocal rank fusion is not where the time goes. What that depth sets is how many candidates the reranker then has to score. The open question is whether the sparse half earns its place, and exactly one effect survived v1's ablation: BM25 on quotation queries. So the measurement is per family, and cutting fusion globally would trade that family away.

Generation moving off the GPU freed the constraint that picked the retrieval stack. The embedding model, the sparse method and the local mode vector store were all chosen to share six gigabytes with a language model that no longer lives there. Chunking has never been measured either, still at 500 tokens with 100 overlap read from v1. Each of these re-embeds the corpus and re-freezes the corpus stamp, which invalidates the evidence base, so they belong in one re-baseline rather than four.