The Last Border Is the Mind

The Rise of the Thinking Machine “We taught the machine to think before we taught ourselves to feel. That is the sentence around which the twenty-first century turns.” There has never been, in the history of the human species, a tool whose intelligence approaches our own. This is true today. It will not be true much longer. The artificial intelligence systems of the present decade — the large language models, the multimodal reasoning systems, the autonomous agents, the embodied robotics — represent something genuinely new in the human story. They are not, like steam engines and combustion engines and electrical motors, extensions of our muscles. They are not, like calculators and computers, accelerators of our calculations. They are, in some still-emerging sense, participants in our cognition. They reason. They infer. They write. They negotiate. They produce, increasingly, work that is — at least in the narrow domains in which they have been trained — competitive with the work of skilled human professionals. This is not the speculative future of science fiction. This is the actual condition of the present moment, evolving faster than the institutional infrastructure of governance can keep up with. Every reader of this book is, today, surrounded by AI systems whose presence in the supply chains of daily life is greater than the public conversation has yet acknowledged. The text you receive from your bank, the diagnostic notes attached to your medical visit, the news article you read, the advertisement that targets you, the music recommendation you accept, the route the rideshare driver follows, the credit decision that approved or denied your loan — all of these increasingly involve AI as an active participant rather than as a passive tool. The transition has happened. The conversation about its implications is, at best, in its early adolescence. The premise of this chapter, and of the two that follow it, is simple. We are at a moral inflection point in the human relationship with our own tools. The tools have crossed a threshold beyond which they can no longer be treated as morally neutral. The question of what AI should be allowed to do — and what it should be required not to do — is now a first-order political question, on the same level as the questions of nuclear weapons, of pharmaceutical safety, of constitutional governance. It cannot be left to the markets to resolve, because the markets are optimized for outputs that, as we have seen in the previous chapters, do not reliably correspond to dignity. It cannot be left to the engineers to resolve, because engineers, however gifted, are not authorized by their professional training to decide on behalf of civilization. It cannot be left to any single nation to resolve, because the technology is borderless, and a poorly governed AI ecosystem in one jurisdiction will, within minutes, affect every other jurisdiction. This is the governance problem of our century. It is the problem this part of the book attempts to engage. I want to be precise about what is novel in the present moment, because there is a great deal of hyperbole — in both directions — in the contemporary AI discussion. There is a school of thought that holds the contemporary AI systems are nothing fundamentally new — that they are merely sophisticated statistical predictors, that they do not “really” understand, that they will not produce qualitative change in human civilization. This school is, by my reading of the evidence, wrong. The empirical record of the past few years has been a steady accumulation of cases in which these systems have performed tasks that, even five years ago, would have been confidently described as requiring genuine human intelligence — and they have performed them at human or superhuman levels. The philosophical question of whether they “really” understand is, in some sense, beside the point. The practical question of whether they can produce work that affects the economy, the legal system, the educational system, the political system, the healthcare system, and the cultural system at scale — to that question, the answer is unambiguously yes, and the rate of progress is accelerating, not slowing. There is also a school of thought that holds the contemporary AI systems represent an imminent existential threat to humanity, that the next few years will see the development of artificial general intelligence and, possibly, artificial superintelligence with consequences that will dwarf every other consideration in human affairs. I am, on this question, more cautious. The trajectory is consequential. The exact pace and form of the next several breakthroughs is harder to predict than the most confident voices on either side acknowledge. What I am confident about is that the technology is significant enough that any responsible civilizational policy must take seriously the possibility of substantial near-term effects. The cost of preparing for substantial effects that do not arrive is small. The cost of failing to prepare for substantial effects that do arrive is enormous. The asymmetry of these costs argues for taking the technology seriously even under uncertainty about its exact trajectory. The third school — and the one to which this book belongs — is what I will call the moral alignment school. It holds that the central question is not whether AI will be transformative — it will — nor whether AI will be dangerous — it may be, if poorly governed — but whether the values that get embedded in AI systems will reflect the values of a renaissance civilization committed to dignity, peace, and inclusion, or the values of the existing global system, with its blind spots described at length in the previous chapters. The technology will, in either case, amplify what we have. If we feed it the loneliness epidemic, it will produce more loneliness epidemic, at greater scale. If we feed it the privatization of belonging, it will industrialize the privatization of belonging. If we feed it the disorganization of opportunity, it will entrench the disorganization of opportunity in ways that will be very difficult to reverse. The work of the moral alignment school is, therefore, to ensure that the AI systems of the present decade are fed something different. This is the work the next two chapters develop. I want to make one further observation before closing this introductory chapter on the rise of the thinking machine. The conventional narrative around AI is that it is a technology developed in particular places — Silicon Valley, Shenzhen, and a few other concentrated technology corridors — that will be deployed to the rest of the world. The rest of the world, in this narrative, is the recipient. The places where the technology is developed are the senders. I want to challenge this narrative on the basis of a single observation. The data on which contemporary AI systems are trained is, overwhelmingly, the data of the wealthy world — the English-language internet, the academic literature of the Western university system, the digital cultural production of the global North. The languages, the cultural references, the philosophical assumptions, the political conventions, the aesthetic preferences embedded in these systems are, by sheer statistical weight, the assumptions of a particular slice of humanity. This is not, by itself, evil. It is, however, partial. And the partiality has consequences. When an AI system trained primarily on wealthy-world data is deployed in an African medical context, in a Latin American agricultural context, in a Southeast Asian governance context, it brings with it the implicit assumptions of the data on which it was trained. The assumptions about what counts as a default human experience, what counts as a default family structure, what counts as a default health complaint, what counts as a default legal context — all of these are baked in. The technology is borderless. The assumptions it carries are not. This is the deeper sense in which the contemporary AI ecosystem is, despite its global reach, a narrowly authored ecosystem. The renaissance argument is that this authorship must broaden. African AI, Latin American AI, South Asian AI, Southeast Asian AI, indigenous-language AI — these are not luxury projects to be considered after the main project is complete. They are necessary projects to ensure that the cognitive infrastructure of the next century reflects the actual diversity of the species that will use it. The economic case for these projects is strong. The cultural case is stronger. The civilizational case is the strongest. The chapter that follows — Tech-Yes-Logy — will develop the framework for ensuring that the AI revolution is, in fact, aligned with the broader civilizational project this book argues for. The chapter after — The Algorithm of the Heart — will engage the deeper philosophical question of what it means for a tool to reflect the conscience of its makers, and what kind of conscience we are, currently, in the position to offer the tools we are making.