The reasoning trace is not a reliable guide to what the model is actually doing.

The chain of thought is generated by the same transformer that produces the final answer. Both are token sequences optimized against the same reward signal. There is no separate reasoning module that the chain represents. The tokens are shaped to look like reasoning because that shape was reinforced during training, not because they correspond to underlying computation.

A model can produce a correct answer with a confused reasoning trace. A model can produce a compelling reasoning trace for a wrong answer. The relationship between trace and answer is not guaranteed.

This means inspecting traces to catch errors is fundamentally limited if traces can mislead. The community has not solved this.

How much does your team trust reasoning traces in production?