The bottleneck is that you are reading letters instead of words, and your flashcards trained the letters harder.
Fluent readers do not identify individual notes. They see a shape — a third, a triad, a scale fragment, a familiar chord in a familiar inversion — and their hand knows it. What is on the page is the same information, but it is being consumed in chunks of five or six notes rather than one at a time. That is why they can look ahead: the current chunk is already handled.
This is exactly the reading analogy you used, and it goes further than you might think. You do not read English letter by letter and you also do not read it word by word — you take in phrases. Sheet music works the same way and the units are intervals and chords.
So flashcards did what they promise, which is faster letter identification, and letters were never the limit.