AI Safety Advocates Urge Caution in Technology Development

“Move cautiously and be prepared for change” — unfortunately, it doesn’t have the same punch as “move fast and break things.” However, three AI safety advocates made it clear to startup founders that rushing ahead could lead to ethical problems in the long run.

“We’re at a tipping point where huge resources are flowing into this space,” said Sarah Myers West, Executive Director of the AI Now Institute, on stage at TechCrunch Disrupt 2024. “I’m really concerned that there’s this rush to release products into the world without thinking about the world we actually want to live in and how the technologies we’re creating serve that world or actively harm it.”

This conversation is especially timely as AI safety has never been more crucial. In October, the family of a child who died by suicide sued Character.AI, alleging that the company’s chatbots played a role in the child’s death.

“This story really underscores the high stakes of the rapid deployment we’re seeing in AI technologies,” Myers West said. “Some of these are longstanding, nearly unsolvable issues, like content moderation and online abuse.”

But aside from life-and-death issues, the stakes in AI remain high, from misinformation to copyright violations.

“We’re creating something with enormous power that can truly, truly affect people’s lives,” said Jingna Zhang, founder of the Cara social platform for artists. “When you’re talking about something like Character.AI, which is deeply emotionally engaging, it makes sense to set some limits on how the product is created.”

Zhang’s platform, Cara, gained popularity after Meta made it clear that it could use public posts from users to train its AI. For artists like Zhang, this policy is a slap in the face. Artists need to post their work online to build followings and attract potential clients, but doing so means their work could be used to train AI models that may one day threaten their livelihood.

“Copyright is what protects us and allows us to make a living,” Zhang said. “Just because something is available online doesn’t mean it’s free. For example, digital news outlets need to license images from photographers to use them.” “As generative AI becomes more widespread, we’re seeing that it’s not aligning with the laws we’re used to, and if they want to use our work, they need to license it.”

Artists may also be impacted by products like ElevenLabs, a voice-cloning company worth over a billion dollars. As the head of security at ElevenLabs, Aleksandra Pedrashevska must ensure that the company’s advanced technologies aren’t used for unauthorized deepfakes, among other things.

“I think red-teaming models and understanding unintended consequences of any new release that a generative AI company makes are really becoming [a top priority],” she said. “ElevenLabs now has 33 million users. That’s a huge community, and any changes we make to our product affect them.”

Pedrashevska noted that one way people in her position can more actively ensure platform safety is by building stronger relationships with user communities.

“We can’t just be at the extremes — either fully against AI or trying to convince people there’s no need for regulation in this space,” she said. “I think we need to find a middle ground when it comes to regulation.”