Pars

One structural suggestion if this keeps happening: switch that domain to a DNS-based challenge.

It proves control by writing a record rather than serving a file, which means it does not care about port 80, proxies, redirects, authentication, or whether the site is even running. It is also the only option for a wildcard certificate and for a host with no public web server.

The cost is that your client needs credentials for your DNS provider, so scope that credential to the one zone if the provider allows it.

Two more that produce the identical message and are not about your key at all.

You are authenticating as the wrong user. The remote URL for these services is almost always git@host:group/repo.git. If yours has your own username in front, the server is looking for a key belonging to a user that does not exist.

An older key type is no longer accepted. Servers and clients have both been retiring older algorithms, and a key generated years ago can stop working after an upgrade at either end with no warning. If -v shows the key being offered and refused, generating a modern one and registering it takes two minutes and rules this out.

Generate with ssh-keygen -t ed25519 unless you have a specific reason to want something else.

Your instinct is exactly right and it is the thing most teams have not internalised: anybody who can change the pipeline can read every secret the pipeline can reach. Masking hides a value in logs; it does not stop a step base64-encoding it and printing that.

So the defensible setup is not about hiding values, it is about limiting what each secret can do and who can trigger a run that sees it.

The layers that actually help:

• Protected secrets. Most systems let a variable be exposed only to runs on protected branches or tags. That stops somebody opening a merge request from a fork with a step that exfiltrates production credentials — which is the realistic attack, not a leaked log.
• Scope every credential down. A deploy token that can push one image to one repository is a much smaller problem than an account password. Registry credentials in particular should never be a human's password.
• Separate environments. Production secrets available only to the production job, not to every job in the file.
• Rotate on a schedule, and immediately when anybody with access leaves.

Whichever you choose, the things that make it survivable in production:

• Make jobs idempotent. Every system will run a job twice eventually — a retry after a timeout that actually succeeded is the classic. Design so that running it twice is harmless, and this whole category of incident disappears.
• Pass identifiers, not objects. Store a record ID and re-read it in the worker. A serialised object is a snapshot that is already stale by the time it runs, and it breaks when the class changes under it.
• Separate the queues. Fast and slow work in one queue means a batch of reports blocks every password reset email.
• Alert on queue depth and on age of the oldest job. Depth alone lies; a queue of ten jobs that have been waiting an hour is a worse signal than a thousand moving quickly.
• Have a visible dead-letter place for what failed permanently. Jobs that vanish after their last retry are the ones you find out about from a customer.

The reason your script fails intermittently is that "what changed" has no single answer — it depends on what the run is for, and the runs are not all the same kind.

The two questions people conflate:

• What did this push add? Compare the previous head to the new head. Correct for a push to a long-lived branch, meaningless for a new branch, since there is no previous head.
• What does this branch change relative to where it will land? Compare against the merge base with the target branch. This is the right question for anything reviewing a proposed change, and it is stable across re-runs and force pushes, which the first one is not.

Almost everybody wants the second and writes the first. That is the bug.

Get the merge base explicitly rather than assuming a parent:

git fetch origin main
base=$(git merge-base HEAD origin/main)
git diff --name-only "$base" HEAD