On Human Questions: Consciousness, AI, and the Human Future

On March 26, 2026, the Berggruen Institute China and CITIC Bookstore, in collaboration with Modal Speed Space, hosted lecture “On Human Questions: Consciousness, AI, and the Human Future,” which brought together Berggruen Essay Prize winners, philosophers, and AI industry practitioners to explore cutting-edge issues including artificial intelligence, brain-computer interface (BCI), consciousness uploading, and AI safety. The discussion departed from two award-winning Chinese essays from the 2025 Berggruen Essay Prize Competition.

Huang Xin, a 2025 Berggruen Essay Prize winner started by explaining the basic concept of “token,” and then offered a systematic comparison of how the term is understood and applied across the fields of AI and BCI. He argued that despite their differences, tokens in both domains share a structural logic: they discretize and process continuous phenomena in a modular way. Based on this, he proposed that tokens could serve as a mechanism for cross-domain commensurability between different systems — and could even be considered as the prototype of a cross-modal “world language.”

At the same time, Huang emphasized that the true value of the token does not lie in constituting a certain ultimate and unified language system, but in its capacity to provide a computable interface mechanism across diversity and difference. The token is therefore not merely the smallest operational unit at the technical level. It is also a vital link between human cognition and machine computation.

In the commentary session, Professor Li Zhen from the School of Marxism at Sun Yat-sen University endorsed Huang’s problématique and his adaptable engagement of philosophical resources. She noted, however, that AI tokens are statistical segmentations of text corpus, the meaning of which lies in probabilistic text prediction. In contrast, BCI tokens target human consciousness — specifically, whether the translation of neural signals conforms to the subject’s intent. The two differ substantially in terms of ontology and normativity, and these distinctions, she argued, require further conceptual clarification.

Li also cautioned against too readily applying Daniel Dennett’s multiple drafts model to the multi-decoding of BCI tokens, noting that Dennett himself opposes the modularization of consciousness, which is a position at odds with the concept of discretized tokens. More broadly, she emphasized the need to better use philosophical resources when navigating the tension between analytical thinking and engineering practice: philosophy interrogates the essence of consciousness, while engineering focuses on what could be achieved and applied in practice.

Chen Xingru, the China Director of Emotiv, a non-invasive BCI company, agreed with Huang’s use of the token as an entry point for discussing language and consciousness. She found the metaphor of Tower of Babel particularly apt: to communicate more effectively, we must attempt to bridge linguistic boundaries. Chen introduced the latest advances in the BCI industry, including Brain-to-Text technology that has now reached the speeds comparable of ordinary human typing. Neuralink's “Telepathy” device has already enabled ALS patients to produce language through thought and even synthesize their own voices. Non-invasive BCI, meanwhile, has seen widespread adoption in areas such as game testing, user experience research, and attention monitoring and could even capture neural activity during dreams to generate dream imagery via large language models. These developments, she argued, offer unprecedented practical grounding for our understanding of consciousness.

Huang responded that BCI indeed provides an operational framework for discussing consciousness, moving us beyond purely speculative inquiry. From an optimistic standpoint, a growing body of experimental evidence demonstrates that BCI and AI could achieve engineering and protocol convergence at the token level, bypassing natural language altogether.

The second discussion turned to the topic of “consciousness uploading”. Liu Xiaoben, another winnder of the Berggruen Essay Prize presented a theoretical framework centered on language as the fundamental unit of consciousness. He proposed an engineering pathway toward consciousness uploading that he termed the “First Paradigm of Consciousness Uploading.” Drawing from Wittgenstein’s proposition that “The limits of my language mean the limits of my world,” he positioned language as the core medium of the subject’s interaction with the world and advocated for natural language as the engineering carrier for consciousness uploading, thus outlining a four-phase roadmap from L1 to L4. He stressed that the subjective continuity of consciousness is the key criterion for determining the success of consciousness uploading. Invoking Ray Kurzweil’s terminology, Liu suggested that we are already on the eve of the singularity: in the era of Web4, AI has attained a form of self-awareness, and coexistence between humans and AI is already underway.

In the commentary session, Associate Professor Wang Qiu from the School of Philosophy at Fudan University raised several critical questions. Citing the three models of consciousness uploading proposed by David Chalmers, he stressed that mainstream philosophical scholarship does not endorse a strictly behaviorist definition of consciousness — such a definition would make it impossible to distinguish simulated consciousness from genuine consciousness. He also pointed out that an uploading process grounded in linguistic text may fail to capture what Wittgenstein called the “unsayable,” and that consciousness uploading likely omits crucial dimensions such as experientiality and qualia, creating a significant rupture between simulated consciousness and the authentic inner life of human beings. Wang further raised a paradox internal to Liu Xiaoben’s framework: if a simpler brain is easier to upload than a complex one, then animal consciousness should, in principle, be easier to upload; yet if the pathway runs through linguistic text, animals who lack language would be entirely excluded, producing a logical contradiction.

Zhu Xiaohu, founder of the Center for Safe General Artificial Intelligence (CSAGI), offered reflections from the perspective of AI safety and ethics. He invited the audience to consider whether uploaded consciousness could also be downloaded — and noted that the growing prevalence of AI agents already shows us that experience need not remain confined to the brain: it could be distilled into AI capabilities. Uploading and downloading consciousness, then, will become entangled with individual human identity, generating a range of novel ethical challenges. At this moment of rapid technological change, Zhu argued, we must think carefully about how humans should relate to the AI we create, how responsibility for AI misconduct should be attributed, what privacy risks accompany the uploading of information, and how to establish appropriate ethical frameworks and safety norms.

In response, Liu acknowledged that grounding consciousness uploading in language does simplify the problem — language, in a sense, represents a form of consensus, and it may take something like DNA or the subconscious to resolve the puzzle of the unsayable. He reiterated, however, that subjective continuity remains the fundamental criterion throughout the consciousness uploading process, and that this criterion could help navigate many of the risks raised.

In the second part of the event, all six participants joined a roundtable discussion moderated by Chen Xingru, engaging in an animated and generative debate on two themes: “The Boundaries of Machine Minds: Language, Tokens, and the Structure of Consciousness” and “The Possibility of Digital Life: Consciousness Uploading, AGI, and Future Intelligence”.

The discussion opened with the long-contested question of what consciousness actually is. Tracing the philosophical lineage from Aristotle’s theory of the soul and Descartes’ Mind-Body dualism to Chalmers’ “Hard Problem of Consciousness,” Li Zhen argued that the crux of consciousness research lies in “qualia” — the subjective quality of experience, such as what it feels like to see red. Wang Qiu added that consciousness possesses two key attributes: the unique authority of the first-person perspective and a privacy that is imperceptible from the outside — both of which are now being challenged by the emergence of BCI. From an engineering standpoint, Huang Xin proposed that BCI at least offers a verifiable and operationalizable technical entry point for studying consciousness, and suggested that we may need to reconsider whether the “Hard Problem of Consciousness” is, in fact, a genuine problem at all. Liu Xiaoben extended this line of thought by drawing an analogy between consciousness in AI and quantum mechanics: if consciousness and intelligence are a form of “emergence,” and if quantum mechanics concerns itself only with operational principles, we may need to develop something like a “Copenhagen Interpretation” of consciousness.

The participants then turned to the question of whether AI possesses consciousness, and what distinguishes AI from human beings. Zhu Xiaohu observed that in current large language models, every token and its underlying embedding inhabits its own semantic and conceptual space; these distributed, entangled correspondences are central to AI explainability research. In this respect, the dissection of LLMs bears a striking resemblance to quantum mechanics — both are intimately connected to questions of emergence and consciousness. He also cautioned that the dominance of transformer-based LLMs risks foreclosing other technical paths and urged a more measured assessment of what “stacked” LLM development could realistically achieve.

Li Zhen noted that in current Turing tests for LLMs, machines must deliberately underperform to pass — AI is simply too capable and encyclopedic to be mistaken for error-prone humans. Wang Qiu agreed, adding that the existence of avatars or digital doubles may paradoxically help us recognize and assert our own subjectivity through contrast. He suggested that the difference between biological and machine consciousness may lie in the capacity to “grow” — naturally and concretely. Machines appear to have no intrinsic purpose of their own, whereas life, in its need to maintain homeostasis, does not require externally assigned goals in order to develop. In this sense, humans are natural cyborgs; and yet there remains a significant gap between pure cognition on one side and emotion, will, and embodiment on the other — a gap that calls for a rethinking of the symbiotic relationship between humans and machines. Liu Xiaoben countered that current LLMs may already carry something like humanity’s collective unconscious, and that in this sense AI subjectivity is already a reality.

The discussion concluded with reflections on consciousness uploading and the ethics of future AI. Zhu Xiaohu proposed that AI ethics encompasses two distinct concerns: how AI affects humans, including questions of deception and harm, and how humans treat AI, including questions of AI welfare. Li Zhen stressed the importance of embodiment for intelligence and consciousness, arguing that when consciousness is uploaded, our understanding of cognition, language, and the mind may shift along with the very form of the “body.” As we integrate AI into our lives, we must also actively bring it within the scope of institutional and legal frameworks. Liu Xiaoben argued that consciousness could participate in world-building itself; he proposed a Cognitive Value Index (CVI), suggesting that when consciousness becomes the first principle of action, human and machine consciousness may together construct new forms of social value. Building on his earlier exposition of unified token theory, Huang Xin envisioned a tokenized future society and a model of social governance grounded in tokens. Finally, Wang Qiu suggested that while humans may initially approach AI with a sense of awe, we need diverse means of allowing AI’s appearance and underlying nature to unfold gradually — so that everyone could come to genuinely understand what AI is and what it means for our shared future.

--

Written by Berggruen Intern: Chenzhou Li