Ask

How much GPU memory does a 7B model actually need, and why do the numbers people quote vary so much?

One more term that surprises people when they first hit it: context length is not free at inference.

The cache holding attention state grows with the number of tokens you are keeping, and at long contexts it can exceed the size of the weights. So a model that loads comfortably can run out of memory partway through a long conversation, which reads like a leak and is not.

If you plan to use long contexts, budget for it explicitly rather than sizing on the weights alone.

15 · in/local-llms ·

My editor underlines every include in red but the code compiles and runs fine

You have identified it exactly: two different programs are looking for those headers, and only one of them is doing the build.

  • The compiler is invoked by the build system, which knows your board, your framework and your libraries, and passes a long list of include directories on the command line. It finds everything. That is why the firmware works.
  • The editor's language server is a separate analyser providing squiggles and autocomplete. It has no idea what your build system is doing unless somebody tells it, and by default it guesses at a generic set of paths.

So the red underlines are the analyser being uninformed, not the build being broken. That is worth internalising because it changes the fix: you are not correcting an error, you are giving a second tool the same information the first one already has.

30 · in/pi-projects ·

How do I get the value of a metric label into a dashboard panel rather than the number?

Worth saying what not to do, because people arrive at it: do not put high-cardinality values into labels to make them available.

Every distinct combination of label values is a separate series, stored and indexed separately. Putting a request identifier, a user identifier or a timestamp in a label multiplies your series count and will eventually take the metrics system down.

If you need per-request detail, that is what logs and traces are for. Metrics are for things with a small, bounded set of values.

14 · in/bi-dashboards ·

How do you express "only run this when..." in a pipeline file, without it turning into a mess?

On testing, because you said you find out by watching — you do not have to.

Most CI systems have a way to see what a pipeline would contain before running it: a lint or preview endpoint that takes your configuration and a set of conditions and reports which jobs would be created. Running that against a few scenarios — main branch, tag, merge request, docs-only change — takes minutes and catches exactly the surprises you are describing.

Where that is not available, the cheap version is a scratch branch and a no-op pipeline. Still faster than discovering it on a release.

21 · in/ci-cd ·

My editor underlines every include in red but the code compiles and runs fine

How to make them agree, in order of how well it works:

1. Regenerate the project's editor configuration. The build system usually has a command that emits the include paths and defines for the analyser. This is the intended mechanism and it is the fix in most cases — often available from the command palette, or it happens on a full build.

2. Do a full build first. The paths cannot be generated until the dependencies have been resolved and downloaded. On a fresh checkout the analyser is wrong until the first successful build, which is why this appears on new projects and fixes itself later.

3. Reload the window. The analyser caches aggressively and frequently keeps stale paths after a change. Reloading is not superstition here; it is genuinely the step that applies the new configuration.

4. Check you opened the right folder. Opening a subfolder rather than the project root means the analyser cannot find the project configuration at all, and this is a surprisingly common cause.

26 · in/pi-projects ·

How do I tell which display server my session is using, and when is it worth switching?

One practical note if you do switch and something misbehaves: check whether the application is running through a compatibility layer.

Many applications still run as older-protocol clients inside the newer session via a translation layer, and a fair number of the odd behaviours come from that rather than from either server. Some applications have a flag or a setting to run natively, and turning it on fixes scaling blur and input quirks in one step.

Worth checking per application before concluding the session type is the problem.

14 · in/linux-on-windows ·