Over-confident is important. They'll absolutely make mistakes - sometimes subtle, sometimes huge. These mistakes can be deeply inhuman - if a human collaborator hallucinated a non-existent library or method you would instantly lose trust in them. Don't fall into the trap of anthropomorphizing LLMs and assuming that failures which would discredit a human should discredit the machine in the same way.

Not to pat myself too much on the back, but note that this was originally published on January 7, 2025 before DeepSeek freaked everyone out with R1—and it literally calls out both Chinese AI (DeepSeek, in particular) and reasoning as things that mattered. It even calls out agentic AI, which with Manus (another Chinese AI) is literally the “hot new thing” right now.