There is a practical caveat that makes the strategy slightly worse than neutral, though, and it is worth knowing for a real garden.
Small tubers carry less stored energy. A seed potato feeds the shoot until it establishes roots and leaves. A very small one gives the plant a weaker start, which in poor conditions or a short season can mean a smaller crop — not for genetic reasons, purely because of the packed lunch.
The usual advice is to plant medium tubers, or to cut large ones into pieces each with a couple of eyes and let the cut surfaces dry before planting. That gets the energy without wasting the biggest potatoes on the ground.
So: eat the big ones by all means, and use medium rather than tiny ones for seed.