Worth knowing for anyone whose instinct says infinity: there is a structure where it is defined, and looking at what it costs is the clearest answer to why we do not do it by default.
You can extend the numbers with a single point at infinity and define division by zero there. What you lose is that addition and multiplication stop working everywhere, so it is a trade rather than a fix, and it is useful in a specific corner of mathematics rather than in arithmetic.