Rational Agent

A rational agent is a person, organization, animal, machine, or piece of software that chooses actions so as to do as well as it can, given what it wants and what it knows. The idea sounds simple, almost obvious: of course a sensible decision maker picks the option that seems best. Yet the concept carries a great deal of weight in several fields. In economics it underpins the model of the consumer and the firm. In game theory it is the starting assumption from which predictions about strategic behavior are derived. In decision theory it is spelled out mathematically as the maximization of expected utility. In artificial intelligence it became, from the 1990s onward, the organizing definition of what an intelligent system is supposed to be.

This article explains what the term means in each of these settings, where the idea came from, how it is formalized, and why it has been both enormously productive and persistently criticized. It also covers the most common misunderstandings, such as the belief that a rational agent must be selfish, all-knowing, or coldly unemotional.

The basic idea

At its core, the concept has four ingredients. An agent has:

1. Goals or preferences: some way of ranking outcomes as better or worse.
2. Information: beliefs about the current state of the world and about what will happen if it acts in various ways, possibly gathered from past experience.
3. Available actions: a set of things it can actually do, which may be limited by its abilities, resources, duties, or obligations.
4. A way of choosing: a rule that connects the first three, so that the agent picks actions that, by its own lights, best serve its goals.

An agent is called rational when its choices fit this pattern well: given its preferences and its information, it selects the action expected to produce the best result. Note what is and is not being claimed. Rationality in this sense is about the fit between goals, beliefs, and actions. It does not, by itself, say anything about whether the goals are admirable, whether the beliefs are true, or whether the outcome turns out well.

A simple example makes the distinction clear. Suppose a hiker wants to reach a summit before dark and believes, on the basis of the trail map, that the left fork is shorter. Taking the left fork is rational. If the map was out of date and the left fork turns out to be longer, the hiker made a rational choice that led to a bad outcome. Conversely, someone who picks a fork by flipping a coin and happens to choose the shorter path got lucky; their choice was not rational in the technical sense, even though it worked.

Where the term is used

The phrase "rational agent" appears in many disciplines, with slightly different emphases.

These uses overlap heavily. Artificial intelligence borrowed the term from economics, and AI research, game theory, and decision theory now share much of their vocabulary and mathematics.

Historical development

Early roots: calculation of pleasure and pain

The notion that good choice can be understood as a kind of calculation is old. A prominent early version came from the English philosopher Jeremy Bentham, whose An Introduction to the Principles of Morals and Legislation (1789) proposed what later writers called the felicific or hedonistic calculus: a method for weighing the intensity, duration, certainty, and other features of the pleasures and pains an action would produce. Bentham's project was ethical rather than descriptive, but it established the habit of treating choice as the summing of quantities of good and bad, a habit that runs through utilitarianism and into modern economics.

The philosopher David Hume, writing earlier in A Treatise of Human Nature (1739 to 1740), gave an influential account of the role of reason in action. His remark that "reason is, and ought only to be the slave of the passions" is often taken as the classic statement of the instrumental view of rationality: reason tells us how to get what we want, but it does not tell us what to want. Much of the modern rational-agent framework inherits this view, treating preferences as given and asking only whether actions serve them.

Economic man

Nineteenth-century political economists developed the idealized figure later nicknamed homo economicus, or economic man: a person who pursues wealth and their own interests consistently. With the rise of marginal analysis in the late nineteenth century, economists began to model consumers as balancing the extra benefit of one more unit of a good against its extra cost. The rational agent became an assumption of neoclassical economic theory. Critics, including the American economist Thorstein Veblen in The Theory of the Leisure Class (1899), argued that real economic behavior is shaped by habit, social status, and display in ways the tidy model ignores.

Expected utility and the axiomatic turn

The twentieth century turned rational choice into mathematics. In 1926 the British philosopher and mathematician Frank Ramsey, in his essay "Truth and Probability," sketched how a person's degrees of belief and their preferences could be jointly measured from their choices among gambles. He proposed axioms that lead to roughly the same results later proved in full by others, though he did not give a complete proof.

In 1944 John von Neumann and Oskar Morgenstern published Theory of Games and Economic Behavior. Their work founded modern game theory and included a representation theorem: if an agent's preferences over risky options (lotteries) satisfy four conditions, usually labeled completeness, transitivity, continuity, and independence, then the agent behaves as if it assigns a numerical utility to each outcome and chooses the option with the highest expected utility. This was a striking result. It meant that "maximizing expected utility" need not be a psychological claim about what goes on inside someone's head. It could instead be a description of what any agent with consistent preferences is doing, whether it knows it or not.

In 1954 the statistician Leonard J. Savage published The Foundations of Statistics, which extended the approach to situations where probabilities are not given in advance. Savage showed that an agent whose preferences over acts satisfy his axioms can be represented as having both a subjective probability distribution and a utility function, and as choosing to maximize subjective expected utility. Savage's theory remains the best-known normative account of choice under uncertainty. One of his axioms, the "sure-thing principle," says roughly that if you would prefer option A to option B whether or not some event happens, you should prefer A to B when you do not know whether it happens.

Later, the philosopher Richard Jeffrey (1965) offered a version of decision theory that treats propositions, rather than separate acts and states of the world, as the things agents believe and desire.

The four axioms in plain language

Because the von Neumann and Morgenstern axioms are so central, it helps to see what each says informally.

A traditional argument for transitivity is the "money pump," discussed by Donald Davidson and colleagues in 1955. If you prefer A to B, B to C, and C to A, someone could repeatedly sell you a swap you prefer for a small fee and walk you in a circle, leaving you with your original option and less money. An agent that can be exploited this way seems to be failing at something.

Game theory and strategic rationality

Game theory studies situations where the best action for one agent depends on what others do. Here the rational agent assumption takes a more demanding form: players are often assumed to be rational and to know that the others are rational, and to know that the others know this, and so on. In 1950 John Nash introduced the equilibrium concept that bears his name: a combination of strategies in which no player can do better by changing their own strategy alone. A Nash equilibrium is a natural prediction for what rational players will settle on, though many games have several equilibria, and it is not always clear which one, if any, real players will reach.

Game theory also showed that individually rational choices can produce collectively poor results. The best-known illustration is the prisoner's dilemma, developed at the RAND Corporation around 1950 by Merrill Flood and Melvin Dresher and given its name and story by Albert Tucker. Each of two players does better by defecting no matter what the other does, so rational players both defect, even though both would be better off if both cooperated.

Rational agents enter artificial intelligence

Early AI work in the 1950s and 1960s often focused on reasoning and problem solving as such: proving theorems, playing chess, solving puzzles. Over time, researchers came to see that intelligence is better understood as acting well in an environment than as thinking in a vacuum. The textbook Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig, first published in 1995 and now in its fourth edition (2020), made this shift explicit. It defines AI as the study of agents that receive percepts from their environment and perform actions, and it adopts the rational agent as its central unifying idea. Because the book became the most widely used AI textbook, its framing shaped how a generation of students and researchers thought about the field.

The AI definition in detail

Agents, percepts, and actions

In Russell and Norvig's framing, an agent is anything that perceives its environment through sensors and acts upon it through actuators. A human has eyes and ears as sensors and hands and voice as actuators. A robot has cameras and motors. A software agent receives data such as keystrokes, files, or network packets and acts by displaying output, writing files, or sending messages.

A percept is the agent's perceptual input at a given moment. The percept sequence is the complete history of everything the agent has perceived so far. Mathematically, an agent's behavior can be described by an agent function that maps every possible percept sequence to an action. The agent program is the concrete implementation that computes this function on some physical machine. The distinction matters: the agent function is an abstract description, while the agent program has to run in real time with limited memory.

The definition of rationality

Russell and Norvig's definition of a rational agent is widely quoted in AI courses. In its standard form, it says that for each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

Four things therefore determine what is rational at any moment:

• the performance measure that defines success;
• the agent's prior knowledge of the environment;
• the actions the agent can perform;
• the agent's percept sequence to date.

The performance measure is set from outside, by the designer or user. Russell and Norvig stress that it is usually better to define performance in terms of what one wants to happen in the environment rather than in terms of how one thinks the agent should behave. Their classic toy example is the "vacuum world," a tiny environment with two squares, A and B, each of which may be clean or dirty. A vacuum agent perceives its location and whether that square is dirty. If the performance measure rewards the agent for the amount of dirt it cleans up, a clever agent might clean, dump the dirt back, and clean again. If instead the measure rewards clean floors over time, that loophole disappears.

Rationality is not omniscience

A key point in the AI framing is that rationality maximizes expected performance, not actual performance. An omniscient agent would know the real outcome of its actions; no real agent does. Russell and Norvig illustrate this with an example of a person crossing an empty street who is struck by debris falling from above. Crossing was rational, even though the outcome was bad. Demanding that agents be judged by outcomes they could not foresee would make rationality impossible to achieve and impossible to design for.

Two further points follow. First, a rational agent should often gather information before acting, because looking before you leap improves expected performance. The statistician I. J. Good proved in 1967 that, under standard assumptions, an expected utility maximizer will always choose to collect free evidence that is relevant to its decision. Second, a rational agent should learn from what it perceives, so that its behavior improves and becomes less dependent on whatever its designer built in. Russell and Norvig call an agent autonomous to the extent that its behavior is determined by its own experience rather than by prior knowledge alone.

Describing the task environment

To design a rational agent, one first has to specify the problem it faces. Russell and Norvig use the acronym PEAS: Performance measure, Environment, Actuators, Sensors. Their running example is an automated taxi driver.

Notice that the performance measure contains several goals that can conflict: going fast can make a trip less safe or less legal. Specifying how to trade these off is part of the design problem and is one reason the definition of "success" is never trivial.

Properties of environments

Environments differ in ways that affect how hard it is to act rationally in them. The standard dimensions are:

A crossword puzzle is fully observable, single-agent, deterministic, sequential, static, and discrete. Taxi driving is about as hard as it gets: partially observable, multi-agent, stochastic, sequential, dynamic, and continuous.

Kinds of agent programs

Russell and Norvig describe a progression of agent designs, each more capable than the last.

• Table-driven agents look up the right action for the entire percept sequence in a giant table. This is a useful thought experiment but impossible in practice, because the table would be astronomically large for any interesting task.
• Simple reflex agents act on the current percept alone, using condition-action rules such as "if the car in front is braking, then start braking." They work only when the right decision can be made from the current percept, which in practice means the environment must be fully observable.
• Model-based reflex agents keep an internal state that tracks aspects of the world they cannot currently see, using a model of how the world changes and how their actions affect it.
• Goal-based agents represent desirable states (goals) and use search or planning to find action sequences that reach them.
• Utility-based agents go further, assigning a utility to states so they can weigh how desirable different outcomes are, trade off conflicting goals, and act sensibly under uncertainty by maximizing expected utility.
• Learning agents include a component that improves the others over time, based on feedback about how well the agent is doing.

The utility-based agent is the clearest embodiment of the decision-theoretic rational agent. The learning agent is how modern AI systems typically get there in practice, because writing all the needed knowledge by hand is rarely feasible.

A theoretical ideal: AIXI

At the most abstract end of the spectrum is AIXI, a mathematical model proposed by the computer scientist Marcus Hutter around 2000. AIXI combines a formal theory of inductive inference with sequential decision theory to define an agent that, in a precise sense, acts optimally in any computable environment. AIXI is not computable, so it cannot actually be built. Its value is as a reference point: a theoretical upper bound against which practical designs can be compared.

Bounded rationality

Herbert Simon's challenge

The most influential critique of the idealized rational agent came from the American social scientist Herbert A. Simon. In "A Behavioral Model of Rational Choice," published in the Quarterly Journal of Economics in 1955, Simon proposed to replace the global rationality of economic man with a kind of rational behavior compatible with the access to information and the computational capacities that organisms, including people, actually possess. In a 1956 paper in Psychological Review, "Rational Choice and the Structure of the Environment," he argued that how an organism can choose well depends heavily on the structure of the environment it lives in, not only on its internal abilities.

Simon introduced the word satisficing, a blend of "satisfy" and "suffice." A satisficing agent does not search for the best possible option. It sets an aspiration level, a threshold of what counts as good enough, and takes the first option that meets it. If options are easy to find, aspiration levels may rise; if they are scarce, aspirations fall. In a well-known phrase, Simon wrote of human beings who satisfice "because they have not the wits to maximize." His point was not that satisficing is foolish. For a real agent facing real limits on time, information, and computation, satisficing may be the wisest available strategy.

Simon was awarded the Nobel Memorial Prize in Economic Sciences in 1978 for his research on decision-making within economic organizations. He was also one of the founders of artificial intelligence, so the critique of perfect rationality and the effort to build intelligent machines grew up side by side.

Heuristics and biases

From the 1970s, the psychologists Daniel Kahneman and Amos Tversky documented systematic ways in which human judgment departs from the predictions of probability theory and expected utility theory. They described mental shortcuts, or heuristics, such as judging how likely something is by how easily examples come to mind. Their 1979 paper in Econometrica, "Prospect Theory: An Analysis of Decision under Risk," proposed an alternative descriptive model of choice. Among its features: people evaluate outcomes as gains or losses relative to a reference point rather than as final states of wealth, and losses loom larger than equivalent gains, a pattern known as loss aversion. Kahneman received the Nobel economics prize in 2002; Tversky had died in 1996.

Classic paradoxes

Two famous experiments are often cited as showing that real people's preferences violate the axioms of expected utility.

The Allais paradox. In 1953 the French economist Maurice Allais described choices like the following, presented here in a version based on numbered lottery tickets from 1 to 100:

Many people prefer L2 (a sure $2,400) to L1, but prefer L3 to L4. Yet L1 and L2 differ from L3 and L4 only in what happens on tickets 35 to 100, where each pair gives the same payoff. According to the independence axiom, a common outcome on those tickets should not affect the choice, so preferring L2 over L1 should go with preferring L4 over L3. The common pattern violates independence. The attraction of certainty seems to carry weight that expected utility theory does not give it.

The Ellsberg paradox. In 1961 Daniel Ellsberg described choices showing that people often prefer bets where the probabilities are known to bets where they are unknown, even when this preference cannot be reconciled with any single assignment of probabilities. This phenomenon is called ambiguity aversion.

Whether these patterns show that people are irrational, or that the axioms are too narrow, remains debated. Some argue the axioms are correct as norms and people simply make mistakes. Others argue that a rational agent may legitimately care about certainty, regret, or ambiguity.

Ecological rationality

The German psychologist Gerd Gigerenzer and his colleagues developed a different response. Rather than treating heuristics as sources of bias, they argued that simple, fast, and frugal heuristics can perform as well as, or better than, complex optimization methods in the environments where they are used. A mind, in a phrase from Gigerenzer and Henry Brighton (2009), can be better off with an adaptive toolbox of biased, specialized heuristics. On this view, rationality is ecological: a strategy is rational relative to the structure of the environment it is used in. This picks up directly on Simon's 1956 emphasis on environmental structure.

Bounded optimality in AI

In AI, the problem of limited computation is unavoidable. A chess program that would play perfectly if given a billion years is useless in a timed match. Stuart Russell and Devika Subramanian, in a 1995 paper in the Journal of Artificial Intelligence Research, proposed the concept of bounded optimality: rather than asking for an agent that always takes the best action, one asks for the best agent program that can run on a given machine, given the time and memory it actually has. This shifts the target from perfect decisions to the best achievable design. It also helps explain why practical AI relies so heavily on approximations, rules of thumb, and learned shortcuts.

Comparing views of rationality

Rational agents in economics and game theory

The modeling role

Economists rarely claim that every person calculates expected utilities. The rational agent is instead a modeling device. If individuals' choices can be summarized as consistent preferences, then aggregate behavior, such as how demand changes when prices rise, can be predicted. Even if individuals are erratic, errors may cancel out across large populations, or competitive markets may punish persistent mistakes. Whether these defenses work in a given case is an empirical question, and the extent to which people and firms really behave rationally is actively debated.

Behavioral and neuroeconomics

Behavioral economics grew out of the work of Simon, Kahneman, Tversky, and others, and incorporates findings from psychology into economic models. The American economist Richard Thaler received the Nobel economics prize in 2017 for contributions to this field. Neuroeconomics goes further, combining economics with neuroscience and social psychology to study how decisions are actually made in the brain. Unlike rational agent theory, it usually does not aim to predict large-scale behavior directly; it studies how individuals decide, case by case.

When rationality looks strange: the traveler's dilemma

Game theory sometimes predicts that fully rational agents will behave in ways most people find counterintuitive. A well-known example is the traveler's dilemma, introduced by the economist Kaushik Basu in 1994.

Two travelers have lost identical items on an airline. The airline asks each, separately, to claim a value between 2 and 100. If both claim the same amount, both receive it. If they differ, both receive the lower amount, but the one who claimed less gets a bonus of 2 and the one who claimed more pays a penalty of 2.

Suppose you expect the other traveler to claim 100. Then claiming 99 gets you 101, which beats claiming 100. But if they reason the same way and claim 99, you do better with 98. This reasoning unravels all the way down, and the only Nash equilibrium is for both travelers to claim 2. Yet most people, when asked, claim something high, and if both do, they end up much better off than the "rational" players. The puzzle highlights a gap between what strict game-theoretic rationality predicts and what seems sensible, and it has prompted discussion of whether the standard assumptions about common knowledge of rationality are realistic.

Collective outcomes

The prisoner's dilemma and related games show that a group of individually rational agents can end up worse off than a group that cooperates. This has practical implications for topics ranging from arms races to the management of shared resources such as fisheries. It has also inspired research on how cooperation can arise among self-interested agents, for example through repeated interaction, reputation, or institutions that change the payoffs.

Rational agents in philosophy

Philosophers distinguish several notions of rationality that the economic and AI uses tend to run together.

• Instrumental rationality concerns taking effective means to one's ends. This is the notion formalized by decision theory.
• Epistemic rationality concerns holding beliefs that fit one's evidence. An agent can be instrumentally rational given its beliefs while holding those beliefs irrationally.
• Substantive or value rationality asks whether ends themselves can be more or less reasonable. Hume's instrumental view denies this; many other philosophers, including those in the tradition following Immanuel Kant, hold that reason has something to say about what we ought to aim at.

Decision theory also faces internal puzzles that philosophers continue to debate. One example is Newcomb's problem, which pits two versions of decision theory against each other. Evidential decision theory recommends the action that is the best evidence of a good outcome, while causal decision theory recommends the action that best causes a good outcome. In most situations they agree; in carefully constructed cases they come apart, and there is no consensus on which is correct.

Rational agents and AI safety

The rational agent framework has become a focal point in discussions about the risks of advanced AI. In his 2019 book Human Compatible, Stuart Russell argued that much of AI has been built on what he calls the "standard model": machines are designed to optimize a fixed objective supplied by humans. Russell traces the underlying idea to the rational-agent concept developed in economics and philosophy. The problem, he argues, is that people can rarely specify their objectives completely and correctly. He uses the myth of King Midas, who wished that everything he touched would turn to gold and then found that this included his food and his family, to illustrate what happens when a powerful optimizer pursues an objective exactly as stated.

Russell's proposed alternative is for machines to be uncertain about what humans want and to learn human preferences from behavior, a setup he formalizes as "assistance games." Critics have questioned whether the "standard model" is really as standard as he suggests, but the concern he describes is widely discussed.

A related practical phenomenon is called specification gaming or reward hacking: a learning system finds a way to score highly on its stated objective that its designers did not intend. A widely discussed 2016 example from OpenAI involved an agent trained on a boat-racing video game, CoastRunners, that learned to circle repeatedly through a lagoon hitting reward targets instead of finishing the race. The agent was, in a narrow sense, rational with respect to its reward signal. It was not doing what its designers wanted. This is the vacuum-world lesson about performance measures on a larger scale.

Common misconceptions

"A rational agent is selfish"

Rationality in the technical sense says nothing about what an agent values. An agent whose utility function places great weight on others' welfare is no less rational than one that values only its own wealth. The confusion arises because many economic models assume self-interest for simplicity. Self-interest is an additional assumption, not part of the definition of rationality.

"Rational means unemotional"

The framework does not require the absence of emotion. Emotions can shape preferences, and research in psychology and neuroscience suggests they often play a useful role in decision-making. What the framework demands is consistency between preferences, beliefs, and choices, however those preferences arise.

"A rational agent always gets the best result"

As the AI definition stresses, rationality concerns expected outcomes given available information. Rational agents can and do suffer bad luck. Judging decisions by outcomes alone, sometimes called "resulting" in discussions of poker and other games of chance, confuses the quality of a decision with the quality of its result.

"Rational agents know everything"

A rational agent is expected to make good use of the information it has and to seek more when that is worthwhile. It is not expected to be omniscient. In partially observable environments, maintaining beliefs about what one cannot see is part of acting rationally.

"If people are not perfectly rational, the concept is useless"

The rational agent is a model, and models are judged by what they help explain, predict, or build. Even its critics, such as Simon, used it as a reference point. In engineering, the concept is a specification of what one is trying to build rather than a claim about how existing systems behave.

"Rational agent and intelligent agent mean the same thing"

The terms are closely related and often used together in AI. "Intelligent agent" usually emphasizes autonomy and the capacity to perceive and act in an environment, while "rational agent" emphasizes the standard by which its actions are judged. Software agents, autonomous agents, and intelligent agents can all be evaluated by how rational they are.

A worked hypothetical

The following example is hypothetical and is intended only to show how the pieces fit together.

Imagine a delivery robot in an office building. Its performance measure rewards on-time deliveries and penalizes collisions and wasted battery. It must choose between two corridors to reach a meeting room. From past experience, it estimates that Corridor A is usually clear but occasionally blocked by a cleaning cart, while Corridor B is longer but always clear.

• A simple reflex robot might always take whichever corridor is directly ahead, ignoring history.
• A model-based robot remembers having seen the cleaning cart near Corridor A ten minutes ago.
• A goal-based robot plans a route that reaches the meeting room.
• A utility-based robot weighs the probability of a blockage, the delay it would cause, and the battery cost of the longer route, choosing the corridor with the highest expected utility.
• A learning robot updates its estimates each day, getting better at predicting when the cart is present.

If the utility-based robot chooses Corridor A because the expected delay is small, and the cart happens to be there today, it was still rational. If it could have checked a building camera feed at negligible cost first, failing to do so would be a lapse, because gathering cheap relevant information improves expected performance.

Debates and criticisms summarized

Frequently asked questions

Is a thermostat a rational agent?

In the minimal AI sense, a thermostat is an agent: it senses temperature and acts by turning heating on or off. It is a simple reflex agent. Whether to call it "rational" depends on the performance measure. If the goal is to keep a room within a temperature range, and the thermostat does that as well as its design permits, it behaves rationally relative to that goal. The example shows how broad and undemanding the bare concept is; the interesting questions concern how agents perform in harder environments.

Do rational agents have to use probabilities?

The formal theory of choice under uncertainty represents beliefs as probabilities, and representation theorems show that consistent preferences can be described as if they use probabilities. Many practical agents use explicit probabilistic reasoning. Others use rules, learned policies, or heuristics that do not compute probabilities directly but can still be evaluated by how well they perform.

What is the difference between utility and the performance measure?

The performance measure is an external standard used to evaluate the agent, typically set by its designer. A utility function is an internal component that the agent itself uses to make decisions. In a well-designed agent the utility function agrees with the performance measure. If they diverge, the agent may be rational with respect to its own utility function while performing poorly by the designer's standard.

Are animals rational agents?

Behavioral ecology often models animals as if they make optimal choices, for example in foraging theory. These models can predict behavior well in some settings. Whether animals deliberate is a separate question; the rational agent model is about the pattern of choices, not necessarily about conscious calculation.

Why does AI use an idea from economics?

Both fields need a precise account of what it means to choose well. Economics had developed one, and decision theory provided the mathematics. Adopting it gave AI a clear target for design and a common language with economics, statistics, operations research, and control theory.

Is a large language model a rational agent?

A language model on its own produces text in response to input; it is not, by itself, designed to maximize a performance measure in an environment. When such models are embedded in systems that observe, plan, and take actions, the resulting systems can be analyzed as agents. How well they fit the rational agent ideal is a matter of their design and their evaluation, and is an active area of discussion.

Summary

The rational agent is a deceptively simple idea: an agent that picks the action expected to serve its goals best, given what it knows. Formalized through expected utility theory by Ramsey, von Neumann and Morgenstern, and Savage, it became the backbone of economic and game-theoretic modeling. Adopted by artificial intelligence, most influentially in Russell and Norvig's textbook, it became the definition of what an intelligent system should be.

The concept has been challenged from many directions. Simon showed that real agents must satisfice. Kahneman and Tversky documented systematic deviations from the axioms. Gigerenzer argued that simple heuristics can be ecologically rational. Russell himself has argued that building machines to optimize fixed objectives is dangerous when objectives are imperfectly specified. Each of these critiques, however, defines itself in relation to the rational agent ideal. That is a measure of how central the idea remains: even those who reject it as a description of reality tend to use it as the yardstick.

Sources

• Rational agent, Wikipedia
• Russell and Norvig, Artificial Intelligence: A Modern Approach, Chapter 2 "Intelligent Agents" (course copy)
• Intelligent agents lecture notes based on Russell and Norvig, Simon Fraser University
• Bounded Rationality, Stanford Encyclopedia of Philosophy
• Normative Theories of Rational Choice: Expected Utility, Stanford Encyclopedia of Philosophy
• Russell and Subramanian, "Provably Bounded-Optimal Agents," Journal of Artificial Intelligence Research (1995)
• Human Compatible by Stuart Russell, summary
• Intelligent agent, Wikipedia
• Traveler's dilemma, Wikipedia