Vural

There is arithmetic and it is simple enough to do in your head once you know the terms.

Start with the weights. Memory for weights is roughly parameters × bytes per parameter. So for seven billion:

• 16-bit (half precision): 2 bytes each → about 14 GB
• 8-bit quantised: 1 byte → about 7 GB
• 4-bit quantised: half a byte → about 3.5 GB

That is where the whole range in your question comes from, and it is the single biggest factor.

Then add overhead for inference: the runtime itself, and the key-value cache that grows with how much context you are holding. Budget roughly 1–3 GB on top for a modest context, more if you want a long one — the cache can become the dominant term at very long contexts.

So a 4-bit 7B model fits comfortably on an 8 GB card; a 16-bit one needs 16 GB and is tight.

Training is a completely different budget, which is the other half of why quoted figures vary so wildly. For full fine-tuning you need, roughly:

• the weights, plus
• the gradients, about the same size again, plus
• the optimiser state, which for the common optimiser is roughly twice the weights again, plus
• activations, which depend on batch size and sequence length.

That lands full fine-tuning of a 7B model somewhere around 60–80 GB — well beyond a single consumer card, which is why nobody does it that way at home.

What people actually do is parameter-efficient fine-tuning: freeze the weights and train a small number of added parameters. Then the gradient and optimiser terms shrink to almost nothing and the job fits alongside a quantised model on a single consumer card. If you see somebody fine-tuning a 7B model on one GPU, this is what they are doing.

Worth asking whether the commit needs to happen at all, because for a good share of these cases it does not.

Generated files that are only consumed by the build can be artefacts rather than commits. Documentation can be published from the build to wherever it is served, without going back into the repository. A version number can be derived from a tag at build time rather than written into a file.

Every one of those removes the credential, the loop and the noisy history at once. Committing back is occasionally the right answer and it is worth being sure it is one of those cases first.

One thing worth checking before any of this: whether the backend actually requires HTTPS or merely defaults to it.

A surprising number of services have a setting for serving plain HTTP that is not on by default, or expect a header to tell them they are behind a proxy that terminated TLS. In the second case, forcing the internal hop to HTTPS is solving the wrong problem — the service was trying to redirect you to HTTPS because it did not know the front already was.

That is the same forwarded-protocol header that fixes CSRF failures behind proxies, and it is worth setting for the same reason.