What is the right way to open a serial connection to a board for debugging?

On when a separate terminal is worth it, the honest answer is that the built-in monitor is fine for most work and there are three cases where it is not.

You need the port while the editor has it. Only one program can hold a serial port. A standalone terminal you can close independently is easier than fighting the editor for it, and this is the most common reason people switch.

You need to see the board boot. The interesting output often appears in the first moments after reset, and by the time the editor's monitor opens it has been missed. A terminal already connected when you press reset catches it.

You need logging, or non-text data. Capturing to a file over hours, or looking at raw bytes in hexadecimal, is what the standalone tools are for.

For anything else, the built-in one is fewer moving parts.

Garbage characters is a very specific symptom and it almost always means one thing: the baud rate does not match.

The connection is working — bytes are arriving — and they are being sampled at the wrong speed, so you get plausible-looking rubbish. Nothing at all usually means the wrong port, or the board is not printing.

The settings that matter:

• Baud rate, which must match what the firmware set. Common values are 9600 and 115200, and many boards print their bootloader messages at a different rate from the program, which is why you sometimes get garbage followed by readable text.
• Line ending. If you send commands and the board ignores them, this is usually why — it is waiting for a newline you are not sending.
• The port, which changes between plugs.

And everything else is usually left at the defaults of eight data bits, no parity, one stop bit.

Two behaviours that confuse people the first time, and are not faults.

Opening the port resets the board on many designs. The control line the terminal asserts is wired to reset. So connecting restarts your program — which is convenient for seeing boot messages and surprising if you expected to attach to something already running. Some terminals let you disable that.

Closing the terminal can leave the board in an odd state, and unplugging while a program has the port open sometimes requires a replug before it is usable again.

On Linux there is also a permissions layer: your user needs to be in the group that owns the serial devices, otherwise the port exists and cannot be opened. Adding yourself and logging out and back in is the fix, and the error message rarely says so.

One habit that makes debugging much easier: have the firmware print its own settings at startup.

A line at boot saying the baud rate, the firmware version and the build date means the first readable thing you ever see confirms you are connected to the right board running the right code. It costs one line and it removes a whole category of confusion about whether the thing you are looking at is what you just flashed.