Ask

Runtime guardrails versus putting the rules in the prompt — what is the actual difference?

Where the runtime layer is insufficient, because it is sold as more complete than it is:

It sees text, not intent. Classifiers work on surface features. Paraphrase, indirection, and splitting a request across turns all reduce their effectiveness, and the arms race there is genuinely difficult.

It does not know your context. A filter cannot know that this particular account number is one your system should never emit. Business rules are yours to write and they are usually the ones that matter.

It has false positives, and they are expensive. Over-blocking makes a product feel broken in a way users cannot diagnose or route around. Getting the threshold right requires exactly the evaluation set most teams have not built.

Output filtering is late. By the time you are inspecting output, the model has already been steered. Blocking the answer does not undo an agent that has already taken an action.

26 · in/model-releases ·

Runtime guardrails versus putting the rules in the prompt — what is the actual difference?

The difference is where the rule lives relative to the thing it is constraining.

A rule in the prompt is text, in the same channel as everything else the model reads, evaluated by the same model that is producing the output. That has three consequences:

  • It competes with other text. Later instructions, retrieved documents and user input all argue with it, and the model resolves the conflict by judgement rather than by precedence.
  • It can be argued with. This is the whole basis of prompt injection.
  • It is probabilistic. It usually holds, and there is no version of "usually" you can put in a compliance document.

A runtime layer is separate code that inspects input or output and makes a decision the model does not participate in. It is deterministic in the sense that matters: the same input produces the same verdict, and the model cannot talk it out of that.

So the honest framing is: prompt rules are guidance, and a runtime layer is a control. Systems that need to demonstrate anything need at least one of the second kind.

30 · in/model-releases ·

On-device processing is a selling point again — when does running it locally actually matter?

Worth adding the argument that has nothing to do with privacy and often decides it: local means no per-use cost and no dependency on somebody else's business model.

A tool that runs locally cannot raise its prices per call, deprecate the model you depend on, or go out of business and take the capability with it. For anything embedded in a long-lived workflow that is a real form of durability, and it is why some teams choose local for a capability that cloud does better.

15 · in/local-llms ·

How do you keep credentials and customer data out of what gets pasted into an assistant?

The screen-blurring category is worth separating out because it solves a different problem than people assume.

It is aimed at screen sharing and recording — a call, a demo, a screenshot for a ticket. There the exposure is visual and momentary, and blurring at the compositor before anything is captured genuinely works.

It does very little for the paste problem, because pasting copies the underlying text, not the pixels. Somebody whose screen is blurred can still select and copy the token.

So if the concern is calls and recordings, that category is right. If the concern is what goes into an assistant, it is the wrong shelf, and the vendors are not always careful about the distinction.

22 · in/privacy-tools ·