There are three separate costs and they are worth separating, because they have very different severities.
Immediate risk. This is the real one. Shifting the front under heavy load can drop the chain, and if you are standing and pushing on a climb, a dropped chain means your foot goes to the floor with nothing under it. People fall this way. A dropped chain can also wedge between chainring and frame and scar the frame, which on a carbon frame is not a cosmetic problem.
Wear. Real but slow. Shifting works by deliberately letting the chain climb sideways off one sprocket onto another, and doing that under tension grinds the ramps and the chain plates against each other harder. It accelerates wear on cassette, chainrings and chain — measurably over years, not noticeably over months.
Sudden failure. Rare on a well-adjusted modern drivetrain and much less rare on a worn or badly indexed one. A tired chain shifting under load is how bent derailleur hangers and snapped chains usually happen.
So it is mostly the first item. Not slow poison — an occasional acute risk.