WR 21: Kate's breaking the AI manipulated media monopoly
Weekly Reckoning for the week of 18/3/24
Welcome back to the Ethical Reckoner. It’s a real grab bag this week, with news about geoengineering, college financial aid applications, AI memes in India, and a data generation company leaving Kenyan workers high and dry. Then, I try to justify why I’ve spent so much time this past week reading about Kate Middleton with some slightly pessimistic musings on what it says about the health of our information ecosystem.
This edition of the WR is brought to you by… ferret filibusters.
The Reckonnaisance
The tech bros are coming for climate change
Nutshell: A geoengineering start-up is pumping seawater over Arctic ice to try to thicken it, to the chagrin of many scientists.
More: As Arctic sea ice melts, it allows the ocean to absorb more solar energy, which causes warming. Real Ice, a geoengineering start-up, is experimenting with drones that bore into the ice and pump seawater onto the top of the ice, thickening it and hopefully preventing it from melting. The company acknowledges that it’s at best part of a climate solution, but many scientists are opposed to it happening at all, calling it “insane” and “dangerous” both for its potential ecosystem impacts and its lack of scalability, with an estimated 10 million pumps needed to cover 10% of the Arctic. They do deserve kudos for their deferral to local Indigenous communities, but most scientists think their whole mission is a distraction from decarbonization.
Why you should care: Climate change is real, it’s coming for us all (but especially marginalized and vulnerable communities in the Global South), and we shouldn’t put our hopes into Hail Marys that encourage us to delay decarbonization.
Basic technical incompetence puts US college financial aid into jeopardy
Nutshell: The notoriously complicated US federal financial aid application is being “overhauled” to make it simpler, but technical issues might prevent students from getting aid.
More: The FAFSA application has been undergoing a revamp for the last three years,1 and the new application finally launched late last year. But glitches and basic incompetence—like not testing the system, not adjusting a key income formula, and forgetting about 70,000 emails with additional details from students (most from immigrant families)—means that millions of students are still waiting on their financial aid packages, and millions more may miss out altogether, with applications lagging 34% behind last year. Fingers are pointing every which way, but in the meantime, the May 1 deadline to respond to college admission offers loom.
Why you should care: While the Biden administration has canceled a ton of student debt for graduates, this bungled roll-out is putting prospective students’ college careers at risk, and is mostly impacting low-income students and students from immigrant households. It’s also a reminder that even in an age of AI and super-advanced tech, there are still legacy systems that massively impact our lives—even more so if they aren’t competently managed.
AI memes take over India’s elections
Nutshell: In the run-up to India’s spring elections, an “AI-enabled meme war” is spreading synthetic content without disclosures.
More: Both parties are using AI to create meme images and clips shared on official Instagram, Facebook, and YouTube accounts. Because of platform policies, the fact that they’re AI-generated mostly isn’t disclosed. While they’re not the kind of hyperrealistic malicious deepfake that most worries election observers, it’s still concerning that they can spread synthetic content from official party channels without disclosing it to viewers. (According to the government, this is a privilege only for them, with several citizens arrested for creating AI political satire.)
Why you should care: This election is taking place in India, where you, dear reader, likely aren’t, but it shows how unprepared platforms are for this bumper year of elections. This “meme war” highlights again the deficiencies in Meta’s synthetic content rules, which require advertisers but not political pages to disclose when they use AI, and also how content moderation is still worse in non-Western contexts. Across the board, generative AI platforms are also falling down on countering election disinformation, with AI photo generators easily used to produce misleading political images.2 The images being shared by Indian politicians are mostly silly memes, but existing policies and tech won’t protect voters from more mendacious ones.
Data annotation company leaves Kenyan workers high and dry
Nutshell: Data annotation platform Remotasks, used by AI companies to label training data, suddenly pulled out of Kenya, putting task workers out of a job.
More: No reason was given for the suspension (though that hasn’t stopped people from blaming the government, the platform, and even a student who told the president about the site). Data annotation platforms operate in developing countries because they can pay people less, and while the work is often exploitative and traumatic, it’s also a key source of income for many people, and families are now left on the verge of eviction.
Why you should care: Putting all of one’s eggs in a single platform basket is a risky choice—as we discussed last week in relation to the TikTok ban—but sometimes people have no choice. The vulnerable human labor that makes artificial intelligence possible is too often overlooked, and to use and discard people like gadgets is inhumane.
Extra Reckoning
Kate Middleton! Depending on what posts you’re reading, she’s either totally fine, in a coma, dead, puffy, growing out bangs, recovering from a BBL, leaving William (possibly because he’s having an affair), or refusing to cooperate with the royal family (possibly because she’s leaving William). If you have no idea what I’m talking about, congratulations for living under a rock not being terminally online. Let me get you up to speed with a very, very condensed timeline (for the full insanity, try here):
Kate Middleton (Princess Catherine of Wales) had “abdominal surgery” in January. Kensington Palace said she’d be out of commission until around Easter.
What they didn’t say is that she’d be totally incommunicado, and in late February, all of Twitter realizes at the same time that she hadn’t been seen in months, and rumors and conspiracy theories start swirling.
In an apparent attempt to calm the speculation, Kensington Palace issued a photo of Kate and her children on March 10th, British Mother’s Day. All seemed well, until…
Everyone on Twitter realized that the photo had clearly been Photoshopped. The AP and other news outlets issued “kill notices” and withdrew the photo.
The next day, a tweet from Kensington Palace’s account, signed by Kate, said she “occasionally experiment[s] with editing” and apologized for “any confusion caused” (while not actually directly saying she had edited the photo). Kensington Palace refuses to release the original photo.
In the absence of any more information, speculation continues apace.
Why are we talking about this in a tech ethics newsletter? Well, while it’s been extraordinarily entertaining to watch a PR meltdown unfold in real time, this also tells us a lot about the state of our media ecosystem.
We’re supposed to be able to trust the images we see from official media sources, and this is why news outlets are so mad at Kensington Palace. Global News Director of AFP Phil Chetwynd declared that they’re no longer a “trusted source,” helpfully throwing in that “kill notices” are usually reserved for sources like the North Korean or Iranian news agencies. Photo distribution agencies rely on the photos they get from trusted sources—called “handouts”—to be unmanipulated, and while there is a vetting procedure, it’s clearly more cursory for trusted sources. Downstream news organizations then rely on those photo agencies to provide accurate photos. In this case, the whole system broke down. In an age where images can be undetectably manipulated or wholesale generated by AI, we’re already struggling with trust in the media. This image seemed to be manipulated with conventional Photoshop, but it adds to the general atmosphere of uncertainty—which is understandable when images like these can be generated by publicly available tools.
News agencies have rules about image manipulation, but those rules are resting on a shakier and shakier foundation. In the film days, pictures went from negative to print to paper. Digital photography and Photoshop introduced some uncertainty; you had to trust that the news agencies were working with the raw files and not manipulating them, but people generally did. Social media came about and made it so that we couldn’t necessarily trust what was being shared in ads and by random accounts, but we had those reliable news outlets to go back to. Trust stretched thinner and thinner, exacerbated by political polarization, and now that distrust of news media is at historic lows (and trust in news from social media increasing), it seems that it’s stretched to a breaking point. People aren’t trusting the news from reliable outlets where it’s least likely to be manipulated, but are trusting it from social media platforms where it’s most likely to be manipulated, and at the very moment when news outlets are trying to turn the tide (especially while anticipating potential election disinformation on social media), a manipulated picture gets into the supposed-to-be-trustworthy ecosystem. And I doubt this will be the last of it. Image manipulation is so accessible and easy—and now with AI potentially undetectable—that it’s going to be very tempting for trusted sources to use tweak/alter/wholesale manipulate photos. If this keeps happening, the breakdown in trust in the news media will, unfortunately, be justified. And then, if we can’t trust official sources, who can we trust? We’re left with three options:
Uncritically believe everything we see and fall into misinformation and conspiracy.
Believe nothing and fall into cynicism and conspiracy.
Rigorously, independently fact-check each piece of information we see, or rely on a trusted third party to do it for us.
Of these, 3 sounds like the best of bad options. But it’s essentially recreating the same problem—no one can verify everything themselves, so we need a trusted upstream source to verify information. There are organizations that do this, but they can’t operate at the scale of “everything online.” Platforms aren’t doing well at this—Twitter has been delegating content moderation to users with mixed results, and pretty much all social media platforms are failing to adequately label AI-generated content.
I’ll close with a small bit of hope. Most people are normal, most people are not consuming that much disinformation, and most people (I hope) can critically think when something doesn’t look quite right. But as the amount of time I spent on #KateGate this past week tells me, we can all be sucked in.3 And more importantly, we can all be influenced by small tweaks in our information environment, especially when we don’t feel like we can trust it. And then we can’t trust ourselves to judge what’s true. So, if we can’t trust the platforms, or our own judgement, we have to trust each other. Even if not every piece of information we encounter is true, we can be good digital citizens and look out for our loved ones, fact checking what we know and having them do the same for us. Be skeptical, but trust that most media outlets are trying to get it right, and while they will increasingly be wrong, we can all do our part to shore up our ecosystem of trust.
I Reckon…
that climate change killing chocolate ought to be a good enough motivator for us to get our act together.
The old system runs on 1 million lines of COBOL. *shudder*
There’s been a lot of buzz about a study about how text-based generative AI is creating misinformation, but the researchers used the APIs rather than the chat interfaces, so it’s unclear how their conclusions generalize to the services most people are using.
Don’t google the Marchioness of Cholmondeley… (it’s “Chumley”)
Thumbnail generated by DALL-E 3 via ChatGPT with the initial prompt “Paint a hazy abstract Impressionist painting in the style of an Impressionist master about the concept of royalty” (it refused to do Monet).