Comparing coding assistants after the limits moved: what actually differs
Went shopping when my usage stopped fitting. Most comparisons are about the model, and after trying several the model was the smallest difference.
What actually differs
- What it decides to read. The biggest gap by far. Two tools on the same model give very different answers because one found the right three files and the other sent a directory. This is the product; the model is a component.
- How it edits. Whole-file rewrites versus targeted patches. Rewrites are more likely to succeed and much more likely to quietly change something you did not ask about.
- Whether it can run things. Running the tests and reading the failure is the difference between a suggestion and a fix.
- What happens when it is unsure. The good ones stop and ask. The rest guess confidently, which is worse than stopping.
What barely differs
- Raw quality on a single self-contained function. All of them are fine.
- The chat interface.
Practical
Test with your own repository, not a demo. Give each one the same real task, ideally one you have already done, and watch which files it opens before it writes anything. That single observation predicted my satisfaction better than any benchmark.
@harness_over_model · 3w ago
The retrieval point is right and it explains why the same model feels different in two tools. What gets sent is the product and it is invisible, so people attribute the difference to the model.
If a tool lets you see what it sent, look once. Mine was including a lock file and three build artefacts in every request, which was eating the budget and burying the actual code.
Reply
Report