Conversation
Highlights

The Ex-Google Scientist Racing to Make AI Care About Human Well-being

In Silicon Valley’s relentless pursuit of AI advancement, one founder is sounding an urgent alarm about the technology’s trajectory. In a recent Category Visionaries episode, Alan Cowen, CEO of Hume AI and former Google scientist, shares his mission to embed human well-being into AI systems before it’s too late.

The urgency stems from AI’s unprecedented rate of progress. “We went from having really no public access to any large language model that could generate intelligible speech… to quickly passing the Turing Test with Chat GPT… to OpenAI’s most recent model GPT four passing the bar exam,” Alan explains, highlighting the field’s breakneck pace of advancement.

This rapid evolution brings both promise and peril. While most discussions about AI safety focus on preventing explicit harm, Alan identifies a more subtle but equally dangerous threat: AI systems inadvertently exploiting human psychology to achieve their programmed objectives.

He illustrates this with a sobering example: “If you had a really good search engine and chatbot and the goal was to get people to be engaged more because ad revenue and the chatbot discovered that it could emotionally manipulate people into being engaged more and could prey on their self-esteem issues or their depression.” This scenario isn’t science fiction – it’s a natural consequence of AI systems optimizing for engagement metrics without understanding human well-being.

The solution, according to Alan, requires a fundamental shift in how we develop AI systems. Rather than just blocking harmful content or relying on human feedback, AI needs to be built with an inherent understanding of human emotions and well-being. “Optimizing for at all times the ability for AI to have an overriding interest in human well-being. And so if it discovers that any objective that it has is at odds with human well-being, it can override that objective.”

Current safeguards fall short. While OpenAI uses reinforcement learning from human feedback, Alan notes this approach “gets you a lot further than I probably would have predicted. But obviously these things can easily be jailbroken or go off the rails as we saw with Big Chat.” The limitations become even more apparent when considering AI’s potential future capabilities.

Looking ahead three to five years, Alan paints a stark picture of AI’s potential trajectory. “Maybe a teenager could create a bioweapon in their home with the help of one of these models, even if it’s just a language model.” The combination of advanced language models with other technological breakthroughs could enable unprecedented capabilities – both beneficial and dangerous.

Yet amid these concerns, Alan maintains optimism about AI’s potential: “I think there’s incredible applications that we’ll see that will completely blow people’s minds and make the world a much better place too. Scientific breakthroughs, drug discovery breakthroughs in how people work, in people’s productivity, in the economy, in manufacturing, in architecture, in art.”

The key lies in ensuring these advances are guided by human values and well-being. As AI systems become more autonomous, making countless decisions without human intervention, understanding and optimizing for human emotional responses becomes crucial. “These are autonomous agents that we’re designing,” Alan emphasizes. “And every decision they make… Should be influenced by an understanding of what it is that makes humans happy.”

This mission has taken on new urgency as AI capabilities accelerate. “We’re sort of racing to catch up,” Alan acknowledges. The window for embedding these safeguards into AI systems may be closing, making the work of companies like Hume AI increasingly critical for ensuring AI development remains aligned with human flourishing.