AI and Data Control

AI can infer your risk of dying from your face but Who controls what it does with your data?

Imagine your friend has just been diagnosed with cancer. The doctor goes over the treatment options, but everyone is still worried about one thing: the risk of dying. Then, someone you know takes a photo of your friend's face. In just a few seconds, an AI system estimates your friend's risk of dying in the next three months. It's hard to believe and feels shocking.

But researchers have created an AI system called FAHR-FaceSurvival. It looks at facial photos to estimate survival risk for cancer patients. The model isn't perfect, but it did find that people with higher risk scores were more likely to die. Nevertheless, these findings are remarkable but breakthroughs in image recognition are challenging as you can see from last week’s stories.

• One company used AI to make food photos that didn’t match what customers actually got.
• An artist found that an AI-generated video seemed to use her identity without asking.
• An investigation showed how a personal AI assistant could build detailed profiles about people’s relationships, preferences, and even their vulnerabilities.

Let’s take a closer look at what happened.

1. AI turns breakfast into a glass of cola with bacon and eggs floating in cola

Just Eat Takeaway.com used AI to create menu images for restaurants, but the technology sometimes got dishes wrong.

For example, one cafe's sausage soda bread breakfast dish was illustrated as a pint of cola with meat floating in it. Just Eat apologized, took down the inaccurate images, and paused the rollout.

AI-generated images can look professional, polished and convincing. But that doesn't mean they accurately represent the product, service, or information customers rely on.

2. When AI uses someone's identity without consent

A Manchester-based musician says an AI-generated video that appears to feature her likeness was shared on Israeli government social media accounts. The artist publicly objected, saying she had not consented to the use of her identity or image. The posts were subsequently deleted.

The leadership question is no longer just, "Can AI create this?" It must also be:

“Do we have the right to use this data, and have the people involved given their consent?”

What Organizations are doing: Establishing boundaries for AI-generated content

Vinted, a resale marketplace, shows how companies can set clear rules for AI-generated content because users of the second-hand seller app Vinted have recently been discussing a seeming uptick in the use of AI-generated photos to sell products. The company has made it clear that using AI is not always banned. For instance, sellers can use AI for things like removing backgrounds from photos. But sellers still need to use real photos that show the actual items for sale.

AI cannot be used to make an item look different from its real appearance, condition, or authenticity. Vinted also uses moderation teams, member feedback, and detection tools to spot misleading content. If rules are broken, they may remove listings or suspend accounts. Instead of pausing or banning AI or letting people use it however they want, companies can set clear rules about what is allowed and make sure those rules are followed.

This week's lesson for Leaders

Good AI governance begins by telling the difference between:

• What AI is technically able to do?
• What the company should let it do?
• What data the system is using with permission?
• What checks and systems the company has to verify and protect its processes?

The Bottomline

Clear principles and governance don't always limit new ideas. They differentiate an organization's successes and failures.