Pro track

Prompt, retrieval, or fine-tune

One use case. Three options. The failure each one still accepts. The default is do not fine-tune.

The one use case

A Northline employee asks, "How many weeks of parental leave?" The handbook page they may see is chunk leave-2: "Parental leave is 16 weeks for every employee." Compensation text is a different document. An employee must not receive it. The page can change again. Version 1 said 12 weeks and is tombstoned. That is the whole use case. Orders 1042, 1043, and 1044 are a different tool.

Option 1. Put the page in the prompt

Paste the current leave sentence into the instructions that run on every call. No index. No chunk id.

The failure this still accepts: the handbook changes, and the prompt does not. The next edit to 18 weeks leaves every call quoting 16 until a person edits the prompt and ships it. The failure also accepts a bad paste. If compensation text is copied into that same prompt, every caller sees it, because there is no group filter in front of the model. You cannot show a chunk id for the sentence you returned.

Option 2. Retrieve, then answer

Upsert leave-2 with the employee group. Tombstone older leave chunks. At question time, filter by the session's groups, then rank. The answer may quote only a chunk that came back. Empty stays empty.

The failure this still accepts: the lab embed is a hash of words. The query "time off for a new child" does not have to return leave-2. A real embedding model can still miss. A retrieved chunk can contain a hostile sentence, and the quote may show it. The quote must not add a tool or a group. Retrieval does not fix a wrong corpus. If you indexed compensation "because the folder was there," the filter is late.

Option 3. Fine-tune

Train on pairs: the question, and the answer "16 weeks." Ship the weights. Stop retrieving.

The failure this still accepts: the weights still say 16 after the page becomes 18. There is no chunk id to cite and no tombstone to flip. A training file that included a compensation sentence can still answer with it, and a group filter on an index does not run, because you deleted the index. You also cannot tell a later reader which version of the handbook the weights saw.

Default: do not fine-tune

For this use case, do not fine-tune. The handbook changes by an upsert and a tombstone, which week 3 already specified. A weight update is the wrong edit for a page that has a version. Use retrieval when the caller must see only their groups and you must name the chunk. Use a pasted prompt only for a sentence that you accept will be stale until the next deploy, and that you accept every caller will see. Neither option is fine-tuning. Fine-tuning is a later choice for a stable behavior you can test, not for a policy number that moves. This page does not claim that later choice is never useful. It says it is the wrong default here.

What you would say

I would keep leave-2 in the index, filter employee versus manager before rank, and refuse to fine-tune the week count into the weights. I studied this note. I did not tune a production model, and I am not claiming a quality number.

Next to this: identity and audit on week 3, and the interview page.