The problem of humanity is not technology but the use of technology ~ Norbert Wiener
This essay continues the argument developed in Part I, Intelligence Too Cheap to Meter: Artificial Intelligence and the Rise of Intelligence as Infrastructure, and Part II, Intelligence Too Cheap to Meter: Max Weber and the Rationalization of Cognition, which together examined how artificial intelligence may reshape institutions and extend the long historical expansion of rational systems across modern societies.
Part I explored how artificial intelligence may transform jobs, organizations, and institutions as intelligence itself begins to function as infrastructure embedded within economic and administrative systems. Part II placed that transformation within a longer historical arc by suggesting that artificial intelligence may represent the rationalization of cognition itself, extending the reach of calculation and optimization into the domain of reasoning.
A question now follows naturally from that trajectory: if intelligence becomes abundant and rational systems become pervasive across institutions, infrastructures, and decision processes, who ultimately decides what those systems should optimize for?
Over the past several centuries, rationalization steadily expanded the reach of calculation across social life as science rationalized the understanding of nature, industry rationalized production, bureaucratic institutions rationalized administration, and digital computing rationalized the organization and processing of information.
Artificial intelligence may extend this trajectory even further by enabling systems capable of analyzing, simulating, and optimizing decisions across increasingly complex domains of activity that once required human judgment and interpretation.
In such a world, optimization becomes easier, decisions can be evaluated across vast datasets, and systems can continuously adjust themselves in pursuit of defined objectives.
Yet efficiency alone does not resolve a deeper philosophical and political question.
Optimization always requires an objective toward which optimization is directed.
Rational systems can determine the most efficient means to achieve a goal, yet the choice of the goal itself lies outside the system.
This distinction between means and ends sits at the center of the challenge posed by increasingly powerful optimization systems.
Artificial intelligence can recommend which medical treatment maximizes survival probabilities, which transportation network minimizes congestion across a metropolitan region, which investment strategy maximizes expected financial returns, or which policy instrument stabilizes economic volatility across complex markets.
Yet the decision about which outcomes should matter most remains fundamentally a political and moral question shaped by values rather than by calculation alone.
Health systems must balance longevity against quality of life and access to care. Cities must balance efficiency against equity and social inclusion. Economic policy must balance growth against stability and distribution. Technological systems must balance convenience against privacy, security, and individual autonomy.
These decisions involve judgments about priorities that cannot be resolved by optimization alone.
They require societies to articulate values.
Max Weber emphasized this limitation when he wrote about the boundaries of rationalization, arguing that scientific and administrative reasoning could illuminate consequences, clarify trade-offs, and reveal the likely outcomes of different courses of action, yet such reasoning could not determine the ultimate values that societies choose to pursue.
Artificial intelligence does not remove this limitation.
Instead, it renders the limitation more visible.
As systems become capable of optimizing increasingly complex domains of activity, the question of defining the goals embedded in those systems becomes more consequential.
A society equipped with abundant intelligence may therefore discover that its central challenge lies not in generating analysis, but in determining which objectives should guide that analysis.
Questions of legitimacy consequently move to the center of the discussion.
Who defines the goals embedded in algorithmic systems that shape economic activity and social coordination? How are trade-offs between competing values negotiated within institutions responsible for governing these systems? Which political processes determine the priorities that guide technological infrastructures capable of optimizing entire sectors of society?
These questions belong to politics, governance, and public deliberation rather than to computation.
They shape how societies translate intelligence into collective action.
Decisions about goals are rarely neutral because they reflect the distribution of power within societies, the institutions through which authority is exercised, and the values that communities ultimately choose to prioritize.
The question of who defines the ends of these systems does not resolve into a single actor, because it emerges from the interaction of multiple forces within modern societies. Governments establish legal boundaries and institutional frameworks that confer legitimacy on decisions, companies design and deploy the systems that embed specific objectives and trade-offs into operational reality, markets shape behavior through incentives that reward certain outcomes over others, and society expresses values through norms, public discourse, and political pressure that influence both institutions and firms. The resulting process is dynamic and often contested, as goals are defined, negotiated, and revised over time rather than determined in a single place.
Moments of this kind have appeared before in the history of modern societies, particularly when new capabilities reshaped economic and institutional structures at scale. The industrial revolution required the creation of labor laws, regulatory institutions, and new political arrangements to manage the power of industrial capital. The expansion of financial systems led to the development of central banking frameworks and mechanisms for economic stabilization. The rise of digital networks prompted new debates around privacy, data ownership, and platform governance.
In each case, the underlying pattern remained consistent, as expanding capabilities compelled societies to renegotiate the rules, institutions, and norms through which those capabilities were governed.
Artificial intelligence may represent a similar moment, though one that operates at the level of cognition itself, where the systems being governed shape not only production and coordination but also the processes through which decisions are made.
When intelligence becomes abundant, the scarcity shifts toward wisdom about what intelligence should serve.
This shift places new demands on institutions responsible for governance and coordination.
Governments must develop regulatory frameworks capable of supervising increasingly complex decision systems embedded in economic infrastructure. Organizations must ensure that automated optimization aligns with legal norms, ethical expectations, and social trust. Public discourse must grapple with trade-offs that technical systems can illuminate but cannot resolve on their own.
Artificial intelligence deepens a tension that has long existed within modern societies.
Rationalization expands the capacity of societies to organize, calculate, and optimize their activities while leaving untouched the most fundamental questions concerning the choice of values, the definition of legitimate goals, and the political processes through which those choices are negotiated.
The arrival of abundant intelligence does not remove the problem of governance but amplifies it.
The ability to optimize systems across entire economies, infrastructures, and institutions makes the question of who defines the ends of those systems more consequential than at any earlier moment in the history of rationalized societies.
Part I described how intelligence may become infrastructure embedded within institutions. Part II argues that artificial intelligence extends the rationalization of cognition itself. Part III brings the argument to its final implication.
Artificial intelligence may automate rationality, yet societies must decide the ends that rationality should ultimately serve.