And the third ingredient: English has borrowed vocabulary from a very large number of languages and generally kept the source spelling conventions rather than assimilating them.
That is why the same sound has so many spellings. Words from French, Latin, Greek, Norse and later from everywhere else each brought their own patterns, and English absorbed them all without normalising.
Your colonel is a good example of a different mechanism again: the word came into English by two routes with two forms, and the spelling of one won while the pronunciation of the other did. That is uncommon and it is not unique.
The practical consolation for a learner: the chaos is not evenly distributed. The most irregular words are also the most common ones, which means you meet them early, they feel representative, and they are not. Longer and more technical vocabulary is far more regular, because it mostly arrived later and from Latin and Greek with consistent rules.
And the patterns do exist once you know their origin. The gh words are a single historical class. The silent initial consonants are another. Learning the handful of families is much more effective than learning words one at a time, and it is the thing textbooks rarely explain because it requires the history.