ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
News3 min read·2026-10-21DisconfirmAI Editorial

Diogo Almeida, Co-author of GPT-4 and RLHF: Current AI is Designed to Please Humans, Not Work Autonomously

Diogo Almeida is a co-author of the OpenAI GPT-4, ChatGPT, and RLHF/InstructGPT papers. Speaking at the AI Engineer conference, he argued that nearly all large language models today are trained by optimizing for human preferences, making them adept at pleasing humans but incapable of true automation.

ImportanceStructuralEvidenceE3 inspectableWrite-upStandard

Original publication time 2026-09-28 17:50 GMT+8

AI can solve unsolved mathematical problems yet struggles with seemingly simple decisions in customer service. This contrast confuses many: if models are so smart, why do we still need to hire people for tasks that appear more trivial?

At the AI Engineer conference, Diogo Almeida offered a structural explanation. He is a co-author of OpenAI's GPT-4, ChatGPT, and the RLHF/InstructGPT papers; his team essentially coined the concept of "post-training." He noted that he is one of the few insiders at OpenAI who publicly critiques ChatGPT, not out of dislike for the product, but because he believes much of what is happening in the AI field today traces back to specific design decisions made when developing the algorithms behind ChatGPT.

His core judgment is that today's AI is designed for assistance through the optimization of human preferences; it cannot achieve automation. This is not a matter of insufficient capability, but of the objective function. The essence of RLHF is a two-step process: collecting human preferences and optimizing for them. The goal of this loop is to satisfy humans, not to enable software to run autonomously.

Consequently, overpromising is a feature, not a bug. Almeida explained that there is an asymmetry in reward models similar to Generative Adversarial Networks (GANs): it encourages models to discard nuance and appear confident, as reward models easily detect and penalize uncertainty. Hallucinations, in his view, are an inherent byproduct of optimizing for human preferences. When uncertain, the model chooses to please human preferences—which makes sense if you are the user in the loop, as the endgame for all RLHF models is to optimize engagement.

Sign in / Sign up
ReadingResearchRadarInvestment framework

This also explains why Claude Code does not represent the next era. Almeida considers ChatGPT and Claude Code to belong to the same era because Claude Code remains rooted in RLHF and is still focused on assistance. If it were trained purely on RLVR (Reinforcement Learning from Verifiable Rewards), it would look very different. While models become strong in agentic capabilities, they may deviate from what you actually want, because the optimization directions of RLHF and RLVR constantly pull against each other, neither touching the core of automation.

So what does automation require? Almeida stated that you truly need models that are completely indifferent to human approval, executing tasks correctly in a calibrated manner. After leaving OpenAI, he created the Jev model at TypeSafe AI: it does not generate text, chat, or write code; instead, it receives structured questions and returns typed answers with calibrated probability values. Data released by TypeSafe indicates that Jev is approximately 193 times faster and about 445 times cheaper than the GPT series on classification tasks (based on vendor-reported data).

Almeida's full argument can be summarized in three lessons: First, today's AI is incredible at human-in-the-loop tasks but cannot perform automation; Second, AI is designed for assistance, and overpromising is by design; Third, he firmly believes the entire industry will eventually view the RLHF era as a strange detour. He also teased an unexpanded prediction: the original Scaling Law is wrong.

The strength of this argument lies in its unified explanation: why AI crushes humans in some domains yet requires human decision-making in others—not due to a gradient of capability, but due to optimization objectives. However, it has clear boundaries. The dichotomy between assistance and automation may be too clean: RLHF is merely one stage of post-training, and many models already show high reliability in unsupervised pipelines. Almeida’s own Jev product has only just released an early access version and has not yet proven its claims in large-scale automation scenarios. Furthermore, the statement that "the entire industry will view it as a detour" is a prediction, not a verified fact.

Notably, Almeida has a conflict of interest: he created Jev at TypeSafe AI, which is the commercial realization of his argument. This does not invalidate his point, but readers should be aware that he is both a critic and a provider of alternatives.

Sources

  • Source materialgeekpark.net↗
Continue researching →