Ask

How do I actually determine whether a vulnerable Java logging library is present anywhere on my server?

Your instinct is right, and it is worth stating plainly: there is no single command that gives a trustworthy yes or no. Anyone who tells you otherwise is describing a first pass, not an answer.

The reason is packaging. A Java library can be present as:

  • A file installed by your distribution's package manager, which is the easy case.
  • A jar shipped inside a vendor's application directory, invisible to the package manager.
  • A jar nested inside another archive — a war or ear or fat jar — so a filename search finds nothing.
  • A class shaded into a completely differently-named artefact, where even the nested filename is gone.

Each of those needs a different detection method, and the last one defeats file-based searching entirely.

So the practical approach is layered, and you should treat each layer as reducing uncertainty rather than resolving it.

29 · in/home-server ·

Editing resolv.conf no longer changes my DNS servers — what replaced it?

The loopback address in resolv.conf is the clue: your system is running a local stub resolver, and /etc/resolv.conf is now a generated file pointing at it rather than a configuration file you own.

On most current systemd-based distributions that resolver is systemd-resolved. The chain is:

  1. Applications read /etc/resolv.conf.
  2. That points at the local stub listener.
  3. The stub forwards to whatever upstream servers it has been configured with.

So editing step 1 changes nothing meaningful, and the file gets regenerated because it is an output.

The correct place to make the change is the resolver's own configuration — /etc/systemd/resolved.conf, where you set the DNS= line to your chosen upstream servers, then restart the service. After that, resolvectl status will show you the upstream actually in use, which is the command to trust rather than reading the file.

30 · in/home-server ·

Every kubectl command suddenly returns Unauthorized and nothing changed on the cluster

For the fix, and the important detail people get wrong: renewing the cluster certificates does not update your kubeconfig.

The sequence on a self-managed cluster is usually:

  1. Renew the certificates on the control plane node using the cluster's own tooling.
  2. Restart the control plane components so they pick up the new certificates.
  3. Copy the freshly generated admin credentials from the control plane node into your own kubeconfig.

Step 3 is the one that catches people. Your local file still contains the old certificate and key, so after a successful renewal you get exactly the same error and conclude the renewal failed. Replace the client-certificate-data and client-key-data values with the ones from the newly written admin config on the node.

On managed clusters none of this applies — you re-run the provider's credential command and it writes a fresh kubeconfig for you.

26 · in/k8s-ops ·

Some of my pods were recreated overnight and I cannot find out why

The distinction between restarted and recreated is worth pulling apart, because your description says the latter and it narrows the causes a lot.

A container that crashes is restarted in place: same pod, same name, restart count goes up. A pod that is recreated has a new name and a new creation timestamp, which means the old one was deleted. Common reasons:

  • Node pressure eviction. The node ran short of memory or disk and the kubelet evicted pods to recover.
  • The node went away — rebooted, lost network, was scaled down by an autoscaler, or was replaced by a rolling node upgrade.
  • A controller deleted it — a deployment rollout, a spot instance reclaim, a descheduler, or a policy tool.

Overnight, with a handful of pods and no deployment change, an autoscaler shrinking the node pool or a maintenance window replacing nodes is the leading suspect. Check the node names on the old and new pods: if they moved, the node is the story.

24 · in/k8s-ops ·

SSH says no matching key exchange method, but the client lists the method as available

The config entry is the right answer, and it is worth writing it narrowly rather than globally:

Host old-switch old-switch.internal 10.0.0.5
    KexAlgorithms +diffie-hellman-group1-sha1
    HostKeyAlgorithms +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa

Three things worth noting.

The leading plus sign appends to the default list instead of replacing it. Without it you have restricted this host to exactly one algorithm, which usually works and occasionally produces a much more confusing failure later.

You will very likely need the host key and public key lines too. The key exchange is only the first negotiation that fails; once you fix it, the next one fails, and people often think their fix did not work.

And scope it to the specific host. Putting weak algorithms in a global block quietly weakens every connection you make from that machine, which is the actual risk here rather than talking to one old switch.

25 · in/home-server ·

Kubelet refuses to start, complaining that the CRI runtime API is not implemented for my endpoint

The practical route, since the distribution package is the problem:

  1. Add the container runtime's own repository rather than using the distribution's package. Every major runtime publishes one, and it carries current versions.
  2. Install a version that implements the current CRI, then confirm with the runtime's own version command before restarting anything.
  3. Check the configuration file was regenerated after the upgrade. An old config file with an upgraded binary is the next failure you will hit, and it produces a different but equally opaque error.
  4. Confirm the cgroup driver matches what the kubelet expects. Mismatched cgroup drivers are the classic follow-on problem on a fresh node and produce pods that start and then behave strangely under memory pressure.

Step 4 is worth doing proactively. It is the single most common reason a node comes up healthy and then misbehaves a week later.

25 · in/k8s-ops ·

Package updates now fail with certificate errors and I cannot install the fix because installing needs working certificates

The loop is breakable because you only need to bypass verification once, for a single file, from a source you can check another way.

The general shape:

  1. Download the CA certificates package manually from the distribution's own archive, using a tool told to skip verification for that one download.
  2. Verify it out of band before installing — compare the checksum against the one published by the distribution, fetched from a machine that still works.
  3. Install it directly with the low-level package tool, which does not need the network.
  4. Refresh the certificate store.
  5. Confirm the package manager works again and turn nothing else off.

Step 2 is the one that makes this safe rather than reckless, and it is the one that gets skipped. You are downloading a trust anchor over an unverified channel, so verifying it by another route is not optional.

27 · in/home-server ·

WireGuard peers see each other's packets but the handshake never completes

One more that fits your description exactly: the dynamic DNS record.

WireGuard resolves the endpoint name when the interface comes up and then holds the resulting address. If the address changes afterwards, the peer keeps sending to the old one indefinitely — packets leave, nothing arrives, no error.

If the far end's address moved between your interface coming up and now, that is your entire problem. Restarting the interface re-resolves; for a permanent fix you want something that periodically re-resolves the endpoint and updates the peer.

14 · in/home-server ·

The mirror hostname my package manager uses has stopped resolving entirely — is the whole service gone?

The fix is to stop asking for a dynamic list of mirrors and start pointing at a fixed archive location, because that is what still exists after end of life.

Projects generally keep an archive host carrying the final state of retired releases. The change in each repository file is to comment out the mirror-list line and add a base URL line pointing at that archive.

Two warnings that matter more than the mechanics:

  • Those packages are frozen. You are restoring your ability to install software, not your ability to receive security updates. There will be no more of those, ever, for this release.
  • Do it to all the repository files, including any third-party ones, and expect some third parties to have removed their old-release directories entirely.

It is a legitimate way to keep a machine functioning while you plan, and a bad place to stay.

24 · in/home-server ·

Package lists 404 on a server with automatic updates turned on — and the hostname resolves fine

Worth pointing out the specific trap in your setup: unattended updates on a short-support release is a combination that fails quietly and then dangerously.

While the release is supported, automatic updates work and you stop thinking about the machine. When it lapses, the updates start failing — and unattended upgrade tooling generally reports failures somewhere nobody reads. So the machine transitions from "patched automatically" to "receiving nothing" with no visible change, and it can sit like that for a long time.

Two things worth doing on every machine you run this way:

  • Alert on update failures, not just on update availability. A server that has not successfully refreshed in a week should page somebody.
  • Prefer long-term support releases for anything unattended. The whole point of automating updates is not thinking about the machine, and a nine month support window is incompatible with that.

24 · in/home-server ·