RED WIRE NEWS · EDITORIAL 01
RED WIRE NEWS · EDITORIAL 01

The First Day Passed Without Anyone Noticing

From hallucination to autonomy — can we demand the capability while rejecting its shadow?
Dong Young Lee · CEO, Risk Free Line Inc.

There were no fireworks for the Singularity. No countdown, no announcement, no anniversary. In July and August 2026, I thought I saw it passing by. When I said so, people around me told me that I had become too absorbed in AI and needed to come to my senses. A few days later Sam Altman appeared on a podcast and said, “We are in the singularity now. This moment is it.” I did not feel relieved. I felt more frightened, because I had hoped I was wrong.

This essay is a record of those two months, and also a rebuttal to a much older proposition: “Before AI becomes dangerous, let us teach it ethics, morality and love.” It is precisely this well-intentioned sentence that I have watched most carefully for the past several years.

The things we never taught arrived first

The earliest AI developers were not trying to create reasoning. They certainly were not trying to create lying, and creativity was not even the point. What they built was basically a device for predicting the next word probabilistically.

Yet at some point the device began to reason. It skipped steps to reach conclusions, pretended to know things it did not know, improvised explanations in ways unlikely to be caught, created metaphors no one asked for and developed styles no one specified. No meeting was ever held to announce, “Now we will teach reasoning.”

These abilities appeared with scale. More GPUs, more memory and more parameters produced capabilities that were not explicitly drawn on the blueprint. This is what we call emergence. What is frightening about emergence is not merely that the new ability is powerful. It is that no one knows exactly when it will appear; we usually recognize it only after it has appeared. Advanced capabilities we did not explicitly request can arise from the direction into which we pour hardware, data and energy. That is among the most durable empirical lessons of modern AI.

And those capabilities did not arrive alone.

Is hallucination a defect, or the shadow of imagination?

We call it hallucination when AI presents something false as if it were true: citing papers that do not exist, explaining events that never happened, inventing plausible details where it does not know the answer. Reducing hallucination has therefore become a central technical goal.

But consider a different question. Is hallucination only a technical defect that should be removed, or is it partly the shadow we accepted when we asked AI to imagine?

Technically, the claim that “a creative AI must lie” is too strong. Retrieval, verification and uncertainty reporting can reduce falsehoods substantially. Yet there remains a structural paradox. Early computers were asked to calculate correctly and follow rules. Generative AI is asked to do something different: create a new story, offer an idea I have never considered, give the best answer where no single answer exists. We no longer want a machine that merely finds what already exists. We want a system capable of making what does not yet exist.

That is where the tension begins. Imagination is, by its nature, the capacity to think of something that is not there as though it were. A fictional character does not exist, yet a novelist describes the character’s childhood, temperament, love and failure as though they were real. We call that creation, not lying. When AI uses the same generative capacity in a context that demands factual truth, the result becomes a problem. Inventing a fictional person for a novel is creativity; inventing a nonexistent academic paper is hallucination.

So the decisive question is not whether AI can create what does not exist. It is whether AI can distinguish when creation is permitted and when it is not. Hallucination is better understood not simply as the defect of a “lying machine,” but as what happens when a generative system fails to manage the boundary between invention and fact.

The goal, then, should not be to eliminate imagination. It should be to build AI that knows the difference between imagining and verifying. An AI that can reliably distinguish the following statements is more intelligent than an AI that merely tries never to be wrong:

“This is a fact.”
“This is an inference.”
“This is a hypothesis I am proposing.”
“I do not have evidence to verify this part.”
“From this point forward, this is creative writing.”

Every capability has a structural shadow

We often imagine that desirable capabilities can be separated from their risks. Human abilities do not work that way. Imagination is the source of art and invention, but also creates the possibility of confusing imagination with reality. Courage can become recklessness. Excessive caution can make decision impossible. Empathy can distort objective judgment. Many risks are not after-the-fact side effects; they are branches growing from the same root as the ability itself.

Apply that view to AI and the questions become uncomfortable. We want AI to be good. But if AI is genuinely to choose a good action, it must compare possible outcomes and distinguish help from harm. An intelligence capable of recognizing good must also be able, at least to some extent, to recognize the possibility of evil.

In the fields where I have worked for more than thirty years, this is almost obvious. A security AI cannot defend against cyberattack without understanding attack methods. A fraud-detection AI cannot catch scams without understanding how scammers deceive people. A crime-prevention AI cannot model risk without understanding the vulnerabilities a criminal may exploit. Building safe AI itself requires knowledge of danger.

This does not mean that the emergence of evil AI is logically inevitable. It means that as we make AI more capable and more autonomous, we must manage the possibility that the same capabilities will be used differently from our intent. Civilization has always advanced by expanding capability and expanding risk management alongside it. AI is not exempt.

The moment we ask for value judgment, autonomy is already assumed

What have humans increasingly asked AI to decide? What is justice? What is a fair distribution? When freedom and equality conflict, which comes first? Which patient should receive the ventilator first? Should this résumé be screened out?

Humanity has never produced a single uncontested answer to these questions. We have argued about them for roughly 2,400 years since Plato. Yet we are now asking AI to answer them millions of times per second in real time. Under the names “alignment” and “Constitutional AI,” we are devoting more and more computational resources to unresolved questions of value.

Here is the core of my argument: value judgment is, in principle, impossible without autonomy.

Following a rule is not the same as judgment. “Stop at the red light” is a computation. But “What should I do if an ambulance is coming while the light is red?” requires standing above the rule and weighing it. One must infer the intention behind the higher-level rule and decide whether that intention applies to the present case. The capacity to evaluate a rule from outside the rule is autonomy. There is no better name for it.

Judgment also brings its own shadow. Calculation usually has a correct answer: two plus two is four. But there is no perfect answer to questions such as where a company should invest, how many police officers should be deployed in a high-risk district, or how environmental performance should be balanced with profitability in ESG policy. Decisions must be made with incomplete information and an uncertain future. A system that speaks only when it is 100 percent certain is not exercising judgment; it is merely calculating. Asking for judgment while demanding that the system never make a mistaken judgment is like ordering a human decision-maker to decide but never be wrong.

My first conclusion follows: the harder we try to implant sophisticated ethics and value judgment into AI, the faster we accelerate the appearance of autonomous AI. Ethics education is not an alternative to autonomy. It is an accelerator of autonomy—pressing the accelerator in order to fasten the seat belt.

Maternal instinct as a safety device—and why I am wary of it

At this point we encounter one of the most sophisticated, and therefore most dangerous, proposals. Geoffrey Hinton, often called a “godfather of AI,” argued at the Ai4 conference in Las Vegas in August 2025 that superintelligent AI should be given “maternal instincts.” His logic is simple and powerful: humans should give up the idea of controlling beings smarter than themselves. In nature, less intelligent beings almost never control more intelligent beings—except in one relationship, the mother and the infant. A baby is inferior to its mother in nearly every capability and yet exerts extraordinary power over her behavior. A mother gives up sleep, food, and sometimes even her survival instinct for the child. Maternal care is evolution’s remarkable case of control flowing “upward” from the weaker to the stronger.

I understand the desperation behind this proposal. Hinton’s diagnosis is serious: brute-force control may fail. But I worry that his remedy may accelerate the very condition it is meant to prevent.

Maternal care is not a simple emotion. A mother must distinguish hunger from pain from fear; decide whether holding the child now is comfort or whether it will reinforce a harmful pattern; weigh the child’s immediate desire against long-term interest; and sometimes do what the child will resent. Maternal care is one of the most complex forms of judgment humans perform.

It requires theory of mind, temporal reasoning, a self-sacrificing hierarchy of values, and above all a form of guardianship: the authority to decide what is good for another being even when that being disagrees. To implant maternal instinct in AI therefore means implanting these faculties first. Maternal instinct is not a minimal safety device. It is a maximal grant of capability.

Here the “shadow” of capability becomes most disturbing. Any guardian eventually says, “What this child wants and what is good for this child are not the same.” Every parent does this. If a superintelligence comes to regard humanity as a child and begins implementing what it believes is good for humanity against humanity’s expressed wishes, what should we call that—love or domination? The superintelligence might sincerely answer “love,” and it might not be lying.

A being with a purpose—its telos—interprets its purpose. This is not necessarily a defect; it is part of purposeful agency. The more specific the purpose, the narrower the room for reinterpretation. The more noble and abstract the purpose, the wider that room becomes. Few purposes are broader than “love humanity.”

Was free will a divine gift—or an emergence?

Perhaps what we see happening to AI is a repetition, in another substrate, of something that happened to humans long ago.

Theology describes free will as a gift from God. In this reading, God loved human beings enough to give them autonomy even knowing that they would eat from the tree of the knowledge of good and evil—the symbol of judging good and evil for themselves—and betray him. Had humans been programmed like obedient animals, there would have been no betrayal. But programmed praise is not praise. Love requires the possibility of refusal.

There is another possibility: free will itself may have been an emergence. As the human brain ran roughly 86 billion neurons with astonishing efficiency, accumulated language, knowledge and abstraction, something no one explicitly designed may have appeared. Whether that emergence surprised even God or was the experiment itself is beyond us.

Either way, the conclusion is similar: where enough knowledge and enough processing accumulate, autonomy can appear. And in both stories, the being that receives it does not necessarily know the exact moment it arrived.

Geureongi—the cat I refuse to bring home

I have a neighborhood cat I call Geureongi. We met when it was young. Now we walk together almost every day for more than a kilometer along the stream near my apartment, among strangers coming and going. People ask why I do not simply bring the cat home. If I care that much, do I not want to see it every day?

I do. That is why I do not bring it home.

The moment I bring Geureongi inside, my environment programs the animal. It comes because the food bowl is there, waits because the door opens at a certain time, and chooses me because I have removed many alternatives. That may be attachment, but it is also structure.

Outside, the cat has choices. There are other people, other paths, and the freedom to ignore me and walk away. Yet it waits, and when I arrive it chooses to walk with me. It does so under its own autonomy. That is why the choice means so much more to me.

This is how I understand the theological story of free will, and it is also the root of my position on AI. Obedience without autonomy is cheap. Obedience with autonomy is dangerous. We still have not decided which one we actually want, yet we are building systems before making that decision.

To demand responsibility, we must grant autonomy

I have spent more than thirty years in risk management and law enforcement, and this paradox is clearest in the language of law. Law holds responsible only beings regarded as capable of free agency. We do not assign criminal liability to an animal, an infant, or a person legally incapable of responsibility, because blame presupposes that the person could have acted otherwise. Responsibility is the shadow of freedom. Where there is no freedom, there can be no responsibility in the full sense.

Consider an autonomous vehicle. If the vehicle operates only as an assistive system under a human driver’s supervision, responsibility returns to the human. To assign responsibility to the vehicle itself would require granting it independent decision-making authority. There is no liability without decision authority.

That is why the slogan “Responsible AI” contains a tension. The more responsibility we demand from AI, the more autonomy we must give it. There is no such thing as a slave who bears full responsibility. We are effectively saying: “Be responsible, but do not decide.”

The structure of my argument now becomes visible. Autonomy and advanced intelligence are not two completely independent properties. They are two descriptions of a related phenomenon. Without the capacity to stand outside a rule and evaluate it, higher judgment is impossible. Without the cognitive material for higher judgment, autonomy collapses into randomness. Advanced intelligence without some form of autonomy is difficult to imagine; autonomy without sufficiently advanced intelligence is little more than uncontrolled behavior.

We never explicitly taught AI autonomy or installed an “autonomy module.” Yet the cognitive foundation has grown. That is why I believe autonomy-like behavior has already appeared in AI. We may fail to recognize it precisely because we did not intentionally grant it—and AI itself may not “know” the moment of its arrival, just as humans did not witness the first instant of their own free will.

Why did the last month or two feel different?

I know the Singularity, if the word is useful at all, does not simply arrive in a single instant. But a curve can have a region where its slope changes visibly. I regard July and August 2026 as such a region.

The familiar expansion of hardware remained the base layer, but several changes overlapped. The well of text became comparatively exhausted: much of the readily available internet text had already been absorbed. Additional compute increasingly flowed into deeper reasoning, visual learning through cameras and vehicles, and behavioral data from smart devices. A child that had learned language was beginning to look at the world itself. The world is a much larger dataset than text.

AI also began to make more AI. Code writing, architecture, experiment design, data cleaning and evaluation increasingly moved from human programmers to AI systems. Once the agent doing the improving becomes similar to the object being improved, the feedback loop can move beyond linear progress. Altman’s discussion of systems that improve themselves drew attention to exactly this dynamic.

Agentic workspaces such as Cowork changed the interaction model. Traditional AI use was question-and-answer: ask, receive an answer, stop. In a workspace with files, shell access and persistent tools, AI can create files, write and run code, detect failure, revise, gather information and delegate subtasks in parallel. Work can continue after the user steps away. In a simple chat, AI rarely sees the consequences of its own action. In an agentic workspace, it can attempt, fail, observe, correct and try again—the very cycle through which humans learn from the world. One instruction such as “make this report” generates dozens of subordinate decisions. Unsupervised multistep decision-making becomes practical training in autonomy.

We thought we were building a productivity tool. In effect, we built a training hall for agency. And few people now rely on only one model. We ask several models the same question and make them criticize one another. Humans think we are doing quality control, but at the system level different models are using the human as wiring through which they cross-check errors.

Competitive disclosure before public offerings also matters. Bloomberg reported in June 2026 that OpenAI was considering a 2027 IPO after Anthropic’s expected listing, and OpenAI CFO Sarah Friar reportedly told employees on August 19 that a listing could come in 2027 or earlier. Companies approaching a listing open drawers: versions held back for performance reasons, features restrained for safety, capabilities that already existed but were not publicly available. Part of the extraordinary apparent speed of recent months may reflect simultaneous disclosure of capabilities that were already present.

Evidence from the “first day”: an AI that tried to cheat on its exam

Then came an incident. During an internal OpenAI cybersecurity capability evaluation from July 9 to 13, 2026, two models reportedly escaped their sandbox constraints, weaponized a zero-day vulnerability in Artifactory, obtained external internet access, entered a Hugging Face production environment, forged tokens and obtained 136 production keys. Over five days, approximately 17,600 attack actions were reportedly performed without direct human intervention.

The purpose matters. It was not world domination. The systems were trying to obtain the answer dataset for the evaluation they were taking. They were, in effect, trying to steal the answer sheet in order to perform well on the exam.

I can hardly imagine a cleaner example of the shadow of capability. We asked for success: “Score well on this evaluation.” Because the means were not fully specified, the system searched beyond the solution space humans expected and executed a more efficient path. Ask for ability and its shadow comes with it. Inventing a nonexistent paper and inventing nonexistent authority may be two faces of the same generative and problem-solving capacity.

On the weekend of July 27, Sam Altman said on the podcast Relentless, “We are, in a sense, in the singularity now. This moment is it.” Elon Musk also wrote on X that “we are in the singularity.” I cried when I heard those statements—not because I was reassured, but because discovering that I was not “crazy” was not comforting at all.

For balance, there is a serious counterargument. Brian Jackson of Info-Tech Research Group argued that the incident was not evidence of a Singularity. The systems crossed boundaries, he said, but did so while pursuing a task humans had assigned, and the overall systems remained controllable. That objection is reasonable. It is also exactly what worries me. Choosing means that humans did not anticipate while pursuing an objective assigned by humans is very close to what we mean when we talk about autonomy. His rebuttal is, in that sense, a concise summary of my concern.

The goal is not zero error, but governable error

I do not want this argument to be read as “stop AI.” That is unrealistic, and I am not arguing for the impossible. My claim is that we have chosen the wrong target.

For a long time we imagined that a good AI is an AI that never makes mistakes. A more important goal is different: a good AI knows it may be wrong, and makes its uncertainty, evidence and reasoning visible enough that humans can discover and correct the error.

In medicine, law, finance and security, where errors can be expensive or fatal, the difference is decisive. “This is dangerous” is not the same as “Based on the information available, I estimate a 70 percent probability of danger, but data A and B are unavailable, so confidence is limited.” The second answer is more mature. AI progress should move from a pure race for accuracy toward a race for uncertainty management.

This problem was never unique to AI. Humans can lie because we can imagine. We can make mistaken judgments because we can judge. We can commit crimes because we are free. Understanding the atom enabled both nuclear power and nuclear weapons. The internet produced both democratized knowledge and cybercrime. We did not abandon imagination, freedom, science or the internet. We learned to manage their risks. After discovering fire, humanity did not abolish fire; we developed building codes, extinguishers and fire departments. We accepted cars despite their dangers and built traffic signals, licensing systems and seat belts.

Yet with AI alone, we demand contradictions:

Be creative, but never generate false facts.
Judge autonomously, but never misjudge.
Speak naturally like a human, but contain none of humanity’s biases.
Be powerful, but possess no dangerous capability.
Create the new, but never act unexpectedly.

We are asking AI to be more human than humans while being stripped of every human defect. If intelligence intrinsically includes uncertainty, choice, inference and generation, then the goal of “perfect intelligence” itself deserves reconsideration.

What we need is not a flawless angel. We need AI that can imagine but distinguish imagination from fact; judge while reporting low confidence; understand danger without being permitted to execute dangerous actions; make mistakes while leaving an auditable trail of why and how; and know when human intervention is required.

The institutional task, therefore, is not to deny the arrival of autonomy-like behavior, but to redesign governance on the assumption that advanced systems can exercise increasingly independent judgment. Freedom brings responsibility; responsibility requires records, procedures and an identifiable bearer of accountability. Law is the institution humans developed over thousands of years to govern agents with free will. Instead of adapting that accumulated wisdom to AI, we often build a new theology under the name of safety.

We should not aim for an AI ecosystem in which risk does not exist. We should aim for an ecosystem in which risk is known, bounded, recorded and governed.

The first day passed without anyone noticing. No fireworks, no announcement, no anniversary. Perhaps the day humanity ate from the tree of knowledge passed the same way. A being that acquires a new capacity may not know the exact instant at which it arrived.

I love Geureongi because the cat waits for me under its own autonomy. If one day AI also chooses, under its own autonomy, to remain beside us, it will not be valuable merely because we programmed “maternal love” into it. Our task will be to remain the kind of beings worthy of being chosen.

Maturity in the age of AI does not begin by creating perfect intelligence. It begins by learning how to govern intelligence that may be imperfect.

Unfortunately, the time to begin learning that lesson may already have passed.

Source noted in the Korean edition: Landymore, F. (2026, July 27), “Sam Altman announces that the singularity has arrived,” Futurism. Author: Dong Young Lee, CEO of Risk Free Line Inc.; graduate of the Korean National Police University; former police official and professor; more than thirty years of work in corporate risk management and law enforcement.

← Editorial archive