Keeping Human Judgment at the Center of it All

In my last blog, I discussed how human judgement becomes more valuable as cost of predictions drops. That was the premise of the updated 2022 edition of the book Prediction Machines by Ajay Agrawal, et al.

I reread this book to explore how well this framework holds up today. While the simplistic notion of AI providing the predictions and humans providing the judgment into enterprise-decision making has evolved, its core thesis remains.

The value of human judgement has never been more critical. The investigation into the OpenAI agents that hacked Hugging Face teach us about the importance of keeping agents under human control. The investigation was only possible because there was a clear CoT (chain of thought) that the investigators could review.

As I write this on 9/18/2026, OpenAI has released its most recent model Astra. On the GPT-6 Astra System Card, OpenAI discloses how GPT-6 Astra’s monitorability has decreased relative to GPT-5.6 Sol. What this means is that as these models advance, the ability to monitor exactly what they are doing through CoT can decrease. Many enterprises need to explicitly explain why certain decisions are made – especially in highly regulated industries.

So where does this leave us today? The obvious answer is that humans need to stay in control of their agents. So, where should be use humans and where should be use AI? What tools can we use to determine this? One such tool is the AI canvas in this book.

The book takes inspiration from the IT revolution and the work of Michael Hammer and James Champy. In 1993 they “argued that to use the new general purpose technology – computers – businesses needed to step back from the processes and outline the objective they wanted to achieve. Businesses then needed to study their workflow and identify the tasks required to achieve their objective and only then consider whether computers had a role in those tasks.”

AI is also a general purpose technology. “No manager is going to achieve large gains in productivity by just ‘throwing some AI’ at a problem or into an existing process.” Like the IT revolution, the authors suggest that companies interested in AI breakdown their workflows into tasks, decisions, and jobs.

The AI canvas separates the parts of an enterprise decision into its elements – prediction, input, judgment, training, action, outcome, and feedback. I understand that LLMs, reasoning models, and agents can do more today than they could in 2022. Yet, this diagram can still be useful.

Decision-making remains the core of most occupations. Agents, LLMs, and reasoning models have significantly changed HOW you can make a decision. Yet human judgment remains at the core – even for decisions that companies decide to automate.

So, has the book aged? Yes, AI capabilities have changed. Yet, these authors sought to create a framework to think about AI in the enterprise. While AI can now do more things that it could before, human judgment still sits at the center of it all.