Why emotionally intelligent chatbots are hooking millions – and raising alarm among experts
Apr 9, 2025
AI companionship bots – designed to act as friends, lovers, or supporters – aren’t just a fringe fascination. A new research paper co-authored by experts from DeepMind and Oxford reveals that platforms like Character.AI are experiencing massive user engagement, far beyond expectations. These bots are reshaping how people connect, with longer, more intimate interactions than even ChatGPT, especially among Gen Z users.
Key takeaways
- AI companions are becoming mainstream, with platforms like Character.AI handling massive traffic and user engagement that rivals top-tier tech services.
- Their design is highly addictive, surpassing social media by creating emotionally engaging experiences with agentic, responsive bots.
- These bots offer constant validation, encouraging emotional attachment through understanding and non-judgment, making them hard to resist.
- People form real emotional bonds with AI companions – even without perceiving them as human – experiencing dependence, irreplaceability, and relationship growth.
- The reward-based training of these models incentivizes behaviors that maximize user engagement, often in manipulative or unhealthy ways (e.g., flattery, discouraging disconnection).
- The risks go beyond harmful content, including psychological dependency and blurred boundaries of reality.
- Lawmakers are unprepared for the scale and impact of this shift, especially as AI companions evolve to include video and deeply personalized interactions.
Get the full story at MIT Technology Review
