Facebook’s Outage Reveals How AI Tags Photos

“One person, beard” – “3 people smiling, standing, indoor” – “6 people, indoor” – “child, closeup, indoor”. Instead of photos, tens of thousands of Facebook users got these texts displayed on Wednesday. The reason: in one of the databases there were problems with an automated system, which is responsible for configuration values and was unable to perform its task correctly due to an error (more here).

The result: users of Facebook, Instagram and WhatsApp could no longer see photos. The downtime lasted around two and a half hours and hit mainly users in the US and Europe. Instead of the images now empty fields with short texts were displayed. And these picture descriptions have it all. They show how Facebook uses Artificial Intelligence to mark images of users.

One nice thing about Facebook being utterly borked right now is it gives you a chance to see what its (very nice!) accessibility feature auto-alting images does.https://t.co/a3w9PcOosR pic.twitter.com/dnTXOih8Hp

— Dieter Bohn (@backlon) July 3, 2019

Since 2016, Facebook uses machine learning to read the content of photos. It’s been a long time since users have the feature that allows them to actively tag objects, places and people seen in the photos. But with its ML algorithms, Facebook also succeeds in recognizing image content that users do not actively describe themselves.

Purpose: The automatically generated image descriptions should give users with visual impairments or blind people the opportunity to actively use the platform. So-called “screen readers” can read displayed texts to them and tell them what’s on the pictures.

Oh yeah! I forgot Facebook uses machine learning to tag our photos with what it sees in the picture.

To be fair, "one person, beard" is pretty much a spot-on description of me. pic.twitter.com/fCpydUxtpz

— Zack Whittaker (@zackwhittaker) July 3, 2019

Currently, it is not known whether Facebook uses the data, which is collected automatically via photos, also for advertising purposes. In any case, it is known that the social network uses AI to automatically detect and, if necessary, delete dangerous or prohibited contents. For this purpose, “billions of public photos” (also from Instagram) are used to train the systems.

Go to:

Help Us Give International Visibility to More Local Innovators & Businesses

Our micro but diligent Trending Topics team of crazy enthusiasts has been working hard to deliver relevant and reliable insights into the Southeastern European innovation and startup ecosystems. Thank you for your trust and for being one of our 235K+ readers. And huge thanks to our 65 donors, who support us with their recurring monthly donations.

We’d like to believe, our independent community-based media gives international visibility and makes a positive difference for a whole generation of entrepreneurs, first-movers and forward-thinking leaders in SEE. This is why we are convinced that access to high-quality journalism must remain free.

However, the production of independent, fact-based, well-researched articles on an international scale comes with significant costs. To keep crafting it, we need your financial support. With each contribution - no matter how big or small - you will be helping us to continue playing our part, together with yours, in shaping a brighter future for our region.