Asu

The key thing to understand is who is doing the checking and from where. The authority is validating from its own servers, on the public internet, not from your machine. So "it works in my browser" tells you very little — your browser may be inside the network, cached, or using a different address.

What happens in an HTTP challenge:

1. The client asks for a certificate and receives a token.
2. It writes the token to a file under /.well-known/acme-challenge/ on your site.
3. The authority requests that exact URL over plain HTTP on port 80, from several locations.
4. It must receive the token, with no redirect to somewhere it cannot follow and no authentication in the way.

So the diagnostic is to fetch that URL yourself the way the authority does — from outside your network, over HTTP, following redirects, and watch what comes back. That single test identifies the failure in nearly every case.

The causes, ordered by how often I see them, given it worked before:

Port 80 is now closed or redirected. People harden a server, block port 80 or redirect everything to HTTPS in a way that breaks the challenge path. A redirect to HTTPS is fine and is followed; a redirect to a login page or a 404 is not.

A new proxy or CDN in front. Anything terminating connections ahead of your server has to pass the challenge path through. This is the single biggest cause of a renewal failing when issuance once worked.

Authentication in front of the site. A basic-auth prompt or an allow-list added later applies to the challenge path too. It must be excluded.

DNS has changed. The record now points somewhere else, or resolves differently from outside than inside. Split-horizon DNS makes this invisible locally.

Rate limits. Repeated failed attempts eventually earn a temporary block, which then masks the original error. Test with the staging environment while debugging — that is exactly what it is for.

The upgrade that removes most of this problem where it is available: stop storing long-lived credentials at all.

Most cloud providers and registries now accept a short-lived token that the CI system mints for a specific job, tied to the repository and branch, valid for minutes. Nothing is stored anywhere, there is nothing to rotate, and a leaked log line is worthless by the time anybody reads it.

It goes by different names — workload identity, OIDC federation, keyless authentication — and the setup is a trust relationship configured once on the provider side.

Where it is not available, the next best thing is a secrets manager the job authenticates to, so the CI system holds one credential rather than twenty, and the manager logs every retrieval. That audit trail is worth a lot when somebody asks what a departing engineer had access to.

One thing to settle early because it changes the design: does the user need to see the result?

If a report is generated and then emailed, fire the job and forget it. If the user is watching a page waiting for it, you also need a way to report progress and completion back — a status the page polls, or a push channel.

That requirement is what usually drives people from a scheduled script to a real queue, more than throughput ever does, because a script gives you nowhere to put the status.