Your mental model is half right and the missing half explains the "sometimes", including why small test inputs pass.
A slice is passed by value — but the value is three fields: a pointer to a backing array, a length, and a capacity. Copying the slice copies those three fields. It does not copy the array they point at. So both copies address the same memory.
Now append:
- If there is spare capacity (len < cap), append writes into the existing backing array and returns a slice with a longer length. The caller's slice still points at that same array, so any element it can see that you overwrote has changed underneath it.
- If there is no spare capacity, append allocates a new, larger array, copies everything across, and returns a slice pointing at the new one. Now the two are independent and nothing the callee does is visible to the caller.
That is the whole "sometimes". Whether your caller sees the change depends on whether capacity happened to be available, which depends on how the slice was built and how much has been appended to it already.
And it is exactly why tests pass: a slice built with a literal usually has capacity equal to length, so the first append always reallocates and everything looks clean. In production the slice arrives from somewhere with slack in it — a pooled buffer, a re-used slice, a subslice of something bigger — and the aliasing shows up.