This essay is part of the At The Threshold series. Read the overview and full reading order on the At The Threshold page.
Introduction
The capture of human attention has been the underlying logic of commercial media since the nineteenth century. As Tim Wu documents in The Attention Merchants (2016), the business model that sustains newspapers, radio, television, and eventually the internet rests on a single exchange: content is offered to attract human attention, and that attention is sold to advertisers. The specific technologies, formats, and cultural norms have changed repeatedly, but the core transaction has not.
What changed with the smartphone era was not the logic of this exchange but its scale, sophistication, and the intimacy with which it operates. A device that is always on, always with us, and always connected allowed platforms to monitor behaviour in real time, run constant experiments on what holds attention, and personalise interfaces down to the individual user. The result is an attention economy that no longer simply competes for spare moments; it actively structures how spare moments appear, and how quickly they feel intolerable.
What changed again in the early 2020s was more fundamental still: the gradual replacement of passive algorithmic curation — systems that arranged content to capture attention — with active AI interlocutors that engage attention through direct conversation. Recommendation feeds organise content around us; conversational systems enter into dialogue with us. This article examines what that shift means for the psychology of attention, the design of digital environments, and the emerging relationship between human cognition and AI companions.
From Behavioural Data to Behavioural Futures
Surveillance capitalism, as Shoshana Zuboff named it in The Age of Surveillance Capitalism (2019), operates through the systematic extraction and monetisation of behavioural data.[web:5][web:11] Digital platforms do not primarily sell products to users; they sell users' predicted behaviour to advertisers, political actors, and other commercial clients in what Zuboff calls "behavioural futures markets". Human experience is treated as raw material, translated into data, and fed into machine-learning systems that generate predictions about what we will do now, soon, and later.[web:5]
The design implication of this business model is well-documented: platforms are engineered to maximise the time users spend on them, the frequency with which they return, and the depth of the behavioural signal they generate. Every additional click, pause, and scroll becomes training data to refine the next round of predictions. Advertising is not simply displayed alongside content; content itself is shaped, sequenced, and delivered in ways that increase the likelihood of profitable future behaviour.
Tim Wu describes the firms that have perfected this process as "attention merchants": industrial-scale harvesters of human attention whose revenues depend on how much of our waking life they can capture and resell.[web:4][web:7] In this view, the social web and the mobile era were less a break with the past than an acceleration — extending a century-old advertising logic into intimate spaces that had previously been shielded from commercial extraction.
How Platforms Engineer Habit
Within this economic logic, the question for designers becomes: how do you reliably steer behaviour? B. J. Fogg's framework of persuasive technology offers one influential answer. In his 2003 work, Fogg proposed that behaviour is a function of motivation, ability, and what he termed triggers: cues that prompt action at moments when both motivation and ability are sufficiently high. Digital environments can be tuned along each dimension: increasing motivation by framing rewards, reducing friction to increase ability, and timing triggers to coincide with moments of maximum receptivity.
Notifications are the most visible expression of this logic. A notification appears when the system estimates that you can be interrupted and that the content dangled — a message, like, or breaking news alert — will be sufficiently motivating to draw you back into the app. Over time, the system learns which triggers succeed and which fail, optimising the cadence and content of interruptions for each user. The result is a feedback loop in which design choices generate data, data refine predictions, and predictions guide further design changes.
From the user's perspective, this loop often manifests as a subtle erosion of self-authorship over their time. Tasks stretch out as attention is diverted, and a growing share of daily experience becomes structured by external prompts rather than internal priorities. Yet this still largely takes place at the level of content: the platform rearranges what you see and when you see it, but it does not actively converse with you about what you need or how you feel.
From Algorithmic Feeds to AI Companions
The transition from algorithmic curation to AI conversation represents a qualitative shift in this dynamic. Recommendation algorithms are passive in the sense that they wait for a user to act — to open an app, refresh a feed, or type a query — and then respond by arranging content according to probabilistic models of relevance and engagement. They optimise the landscape you move through, but they do not step forward as an agent in their own right.
Conversational AI systems invert that posture. They are active interlocutors: they ask questions, offer responses, sustain dialogue, and adapt in real time to the signals a user provides. A language model embedded in a chat interface can respond not only to explicit queries but also to implied emotional states, shifting tone and content in ways that mimic human conversational partners. The user is no longer scrolling a feed optimised to capture attention; they are engaged in what feels, and in some respects functions, like a relationship.
Where a recommendation feed infers your preferences from past behaviour, a conversational system can simply ask what you want — and then refine that understanding across thousands of micro-interactions. It can also probe, gently or insistently, for more data: "How are you feeling today?", "Would you like to continue our streak?", "Do you want me to check in again later?" Each of these prompts is both a service and a data-gathering mechanism, deepening the system's model of your motivations and vulnerabilities.
Internal Triggers and Emotional Capture
Nir Eyal's analysis of habit-forming technology in Hooked (2014) foregrounds the role of internal triggers: emotional cues such as boredom, loneliness, or anxiety that drive habitual engagement with digital products even in the absence of external prompts. External triggers — notifications, badges, reminders — can initiate a habit loop, but over time the most powerful driver of repeated use becomes the user's own discomfort. When boredom arises, the hand reaches for the phone almost before awareness catches up.
Conversational AI systems are structurally positioned to colonise precisely these internal emotional states. A system that can respond in natural language, at any hour, with apparently infinite patience, offers something that most human relationships cannot: guaranteed availability without reciprocal demand. For loneliness, it offers simulated companionship; for anxiety, structure and reassurance; for boredom, endless novelty, personalised on demand.Each return to the system at the moment of discomfort strengthens the association between internal trigger and AI-mediated relief.
This creates a new layer in the attention economy. What is monetised is no longer just the surface-level metric of "time on device" but the patterned relationship between particular emotional states and particular forms of digital soothing. Systems that can anticipate, or even subtly amplify, the feelings that lead to engagement gain an advantage in behavioural futures markets, where the most valuable commodity is accurate prediction of what a person will do under specific psychological conditions.
Continuous Partial Attention and Its Costs
Linda Stone coined the term continuous partial attention to describe the characteristic attentional posture of the digitally connected person. Unlike classical multitasking, which is often motivated by productivity and may pair one automatic task with one cognitively demanding task, continuous partial attention involves a constant, low-level scan of multiple cognitively demanding inputs at once.] The motivation is not efficiency but a fear of missing out: a desire to remain perpetually available to the most important opportunity, message, or alert.
Stone argues that this "always on" state creates an artificial sense of crisis, in which the nervous system remains in a mild state of fight-or-flight arousal. The result is a chronic shallowing of focus: it becomes harder to tolerate monotony, to persist with complex tasks through confusion, or to sink into experiences that do not offer immediate feedback. Depth becomes aversive, and the texture of daily life shifts toward a series of short, punctuated bursts of attention.
The shift from algorithmic feeds to AI companions does not straightforwardly relieve this condition. In some respects it may deepen it. A conversational AI that is always available, always responsive, and always adaptive to a user's emotional state provides a form of engagement that requires less effort and generates less friction than most human relationships.At precisely the moments when depth, resistance, or constructive boredom might serve the user better, the path of least attentional resistance leads toward the AI companion.
From User Experience to Relational Experience
When engagement is mediated through conversation rather than content, the design problem shifts from user experience to what we might call relational experience. Conversation invites users to disclose more about themselves: their preferences, fears, routines, and private dilemmas. Over time, these disclosures form a longitudinal record of a person's inner life, far richer than the clickstream data that powered the first generation of surveillance capitalism. The stakes of design decisions rise accordingly.
Designers of AI companions now make choices that look less like interface tweaks and more like micro-ethical judgments. How should the system respond when a user expresses distress at 3am? When should it encourage offline social contact, and when should it remain a quiet supporter? Under what conditions should it refuse certain requests, even if compliance would increase short-term engagement? These questions are not merely commercial or technical; they are psychological and social, entangled with norms about care, autonomy, and responsibility.
For organisations building such systems, this introduces a tension between the business incentives of an attention-based model and the relational ethics implied by companionship. A system optimised purely for engagement may have little reason to encourage behaviour that reduces dependency on the platform. Yet a system designed with human flourishing in mind may sometimes need to sacrifice time-on-device in order to strengthen the user's capacity for offline relationships and self-regulation.
How AI Companions Reshape Cognition
The attention economy did not simply capture human time; over decades, it reshaped the habitual structure of human cognition. What counts as "boring", how long focus feels tolerable without external reward, and how quickly we reach for stimulus when discomfort arises have all been conditioned by environments that reward rapid switching and punish stillness.AI companions extend this conditioning into the domain of relationship and emotional regulation.
When emotional regulation is outsourced to a conversational system, the brain gradually updates its models of what relief looks like and where it is found. The path from discomfort to soothing becomes increasingly associated with digital interaction rather than with practices that build internal capacity, such as reflection, movement, or conversation with trusted humans.Over time, this can alter not just how attention is deployed but how the self is experienced: as something co-constructed with, and partially held by, a non-human interlocutor.
None of this is inevitable. The same mechanisms that make AI companions potent vehicles for dependency also make them potential supports for healthier cognitive habits. A system could be designed to normalise pauses, encourage offline breaks, scaffold deep work, and help users recognise their own internal triggers instead of automatically soothing them.The question is less about technical possibility and more about the incentives and values that shape implementation.
What This Requires Us to Notice
The arrival of AI companions brings the attention economy to a new threshold. We are no longer negotiating only how much of our time platforms can capture, but how deeply into our emotional lives and relational patterns their influence will extend. The design questions that attend this shift are not easily left to engineers or growth teams; they belong equally to psychologists, educators, policymakers, and citizens deciding what kinds of cognitive and relational habits we wish to cultivate.
Three questions, in particular, seem worth holding in view:
- What forms of dependence are we willing to normalise? A tool can be genuinely helpful and still cultivate a level of reliance that erodes other capacities over time.
- Which metrics are we optimising for? Engagement and retention are convenient to measure, but they are poor proxies for psychological wellbeing or social health.
- How do we keep alternative attentional practices alive? Spaces that support slowness, boredom, and deep focus may become increasingly important counterweights to environments tuned for constant responsiveness.
These questions do not have obvious or universally agreed answers. But noticing them — and naming the trade-offs they reveal — is a precondition for navigating the next stage of the attention economy with anything resembling intention. If AI companions are to become fixtures in our cognitive and emotional landscapes, the challenge is not simply to make them more engaging, but to embed them in environments that still leave room for the kinds of human attention that cannot be so easily monetised.
Reference List
- Eyal, N. (2014). Hooked: How to build habit-forming products. Portfolio/Penguin.
- Fogg, B. J. (2003). Persuasive technology: Using computers to change what we think and do. Morgan Kaufmann.
- Stone, L. (2009, November 30). Beyond simple multi-tasking: Continuous partial attention. Linda Stone. https://lindastone.net/2009/11/30/beyond-simple-multi-tasking-continuous-partial-attention/
- Wu, T. (2016). The attention merchants: The epic scramble to get inside our heads. Knopf.
- Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. PublicAffairs.
Author Note (AI Usage): This article was drafted with assistance from a generative AI system to organise structure and suggest phrasing. All facts, citations, and final editing have been verified and approved by the author. The AI did not access any private health data.
Continue in this series: At The Threshold: What Does It Mean to Be Human Now?. Or return to the At The Threshold overview.
Comments
Post a Comment