There is a genuine musical argument for lower tuning that gets lost in the mystical framing, and it is worth rescuing.
For singers, a slightly lower pitch makes a demanding part easier. Repertoire written when performing pitch was lower is genuinely harder at the modern standard, and this is a real and much-discussed issue in some vocal music.
For period instruments, playing at a lower pitch is about historical practice and about what the instruments were built for — gut strings under less tension, different tone.
Both are legitimate reasons to tune lower and neither has anything to do with 432 in particular. Ensembles that do this typically use pitches associated with the repertoire's period rather than that number.
So if somebody wants a warmer sound or an easier vocal line, tuning down is a reasonable choice with reasons behind it. The reasons just are not the ones usually given.