One framing that changes what you build: treat the agent as an untrusted client of your own API, not as part of your application.
You would not let an unknown caller hit a paid endpoint with no quota, no rate limit and no per-key accounting. Give the agent its own credential, its own quota and its own bill, exactly as you would an external integration.
That is what the newer tools in this space are essentially selling — a credential with an allowance attached rather than your production key. You can build the same thing in an afternoon with a proxy in front of your providers, and the proxy is worth having anyway because it is where the logging, the caching and the kill switch live.
The kill switch is the part people skip. One flag that makes every tool call fail immediately, reachable without a deploy.