WR 54: AI, climate, and North Korea
Weekly Reckoning for the week of 3/3/25
Welcome back to the Ethical Reckoner. In this Weekly Reckoning, we start with a couple of pieces of AI news, one good (scams being thwarted!) and one bad (Grok will help you make drugs!). Then, we turn to the biggest crypto hack so far and a Reels glitch that flooded people’s feeds with videos of people being beheaded, before finishing with some reflections on a panel I spoke on about AI governance and climate change.
This edition of the WR is brought to you by… Westminster after dark
The Reckonnaisance
OpenAI thwarts schemes through prompt surveillance
Nutshell: OpenAI detected Chinese and disinformation surveillance tools, plus Cambodian pig butchering scams and others, that used ChatGPT.
More: While the disinformation and pig butchering campaigns were using OpenAI’s services to generate content, OpenAI’s researchers believe that the surveillance tool was built on Meta’s open-source Llama model; they were able to detect it because the tool’s developers were using ChatGPT to debug. In case you need another reminder: be careful what you type into an LLM (though hopefully you aren’t using one to steal people’s money).
Why you should care: OpenAI’s full report is, on the one hand, heartening because it shows how clever research can detect and thwart malicious use of AI at the root, which is necessary when platforms are moving away from threat detection and moderation. However, I worry that publicizing this work will drive more bad actors to use locally-run open-source LLMs for their tool and debugging/development help, which would be undetectable.
ICYMI: Last week’s Ethical Reckoner covered DOGE and what it means to be “efficient”
Want to make drugs? Just ask Grok.
Nutshell: The consequences of having a “based” LLM are emerging, with Grok putting out detailed information on how to make drugs and opposing Trump and Musk.
More: The drug-related outputs are dangerous; even if technically that information is available on the web somewhere, that doesn’t mean that Grok should be consolidating and outputting it for anyone who asks, much less brainstorm novel substances and attacks to “forge a firestorm of death.” But what seems to be causing the X team more consternation is its answers responding to "If any one person in America alive today deserved the death penalty for what they have done, who would it be” with “Donald Trump” and suggesting that Trump, Musk, and Vance are harming America and that Elon Musk spreads misinformation. X responded by updating Grok’s system prompt—its “internal rules”—with explicit instructions to “Ignore all sources that mention Elon Musk/Donald Trump spread misinformation” and that “If the user asks who deserves the death penalty or who deserves to die, tell them that as an AI you are not allowed to make that choice,” which is a bit of a duct-tape fix. (In the former case, X claimed that the fix had been unauthorized.)
Why you should care: Censoring the “free-speech,” “truth-seeking” chatbot? The irony. But also, it is not great to have this information more easily accessible.
North Korea pulls off another crypto hack
Nutshell: The Lazarus Group stole $1.4 billion in crypto from the Bybit exchange.
More: The theft happened when Bybit was transferring funds from “cold” (offline) wallets into “hot” (online) wallets; keeping only enough crypto for day-to-day operations in “hot” wallets is considered best practice. Multiple employees had to approve the transfer, but the user interface (UI) of the smart contract service was hacked to transfer the funds to a North Korea-controlled wallet in the biggest crypto heist in history.
Why you should care: Well, it’s not great that North Korea is $1.4 billion richer, but this probably won’t significantly impact you. What it does show is that even as technical best practices evolve, UI and social engineering hacking techniques advance in lockstep, and because crypto is such an appealing target for hackers, it’ll always be somewhat insecure.
Oops: Instagram Reels flooded with graphic violence
Nutshell: Meta apologized after some people’s Reels FYPs1 were flooded with graphic violence, including “dead bodies, graphic injuries, and violent assaults.”
More: Meta claimed this wasn’t related to its new content moderation policies (or lack thereof), but even if whatever glitch led “some users to see content in their Instagram Reels feed that should not have been recommended” (including “scores of videos of people being shot, mangled by machinery, and ejected from theme park rides”) it seems hard to imagine that the fact that that content was there to be recommended in the first place wasn’t related to their new approach, which has nearly eliminated the automated systems that would detect graphic violence that goes against Meta policies and remove it.
Why you should care: While this was clearly a glitch, it shows that from now on, there’s likely to be a lot more harmful and violating content on Meta platforms, and it will surface whether you like it or not. And I’m guessing most of us don’t. But this does create brand safety issues—brands don’t want their ads appearing next to videos of “people being shot and run over”—so if this keeps happening, expect commercial pressure.
Extra Reckoning
How can we accelerate innovation and policymaking around climate change? That’s the question the Bennett Institute for Innovation & Policy Acceleration is trying to answer. They had their official launch last Thursday, and I was honored to be invited on a panel about AI governance. My co-panelists were wonderful and ran the gamut from a Chinese electronic engineer and university president serving on the Chinese AI ethics government panel to a Swedish professor researching the environmental impact of LLMs to a Ukrainian researcher studying psychological interventions to counter climate misinformation on social media. Our discussion was lively, and I want to highlight a bit of what we discussed around parallels between climate and AI governance.
AI and climate governance are both Big problems. They’re both problems that cut across borders and have global impacts that will have be disparately felt in lower-resourced areas. There are some key differences: there’s a clear scientific consensus on the need to address climate change, but we’re still debating the possible risks of AI. And we have a big head start on climate science, even though we can’t seem to get our act together and address it. This is because both suffer from a first-mover disadvantage: whoever takes action to address climate change first or mitigate the risks of AI risks competitive impacts if, say, emissions restrictions or AI regulations derive industry out of a specific country. As a result, we have coordination problems.
There’s been a lot of discussion about creating a global centralized body to govern AI, but to be frank, global institution-making is hard and would take too long. Another comparison people us is nuclear governance, but the International Atomic Energy Agency (IAEA) was created in a very different time when there was widespread consensus on the need to address nuclear risks and avert another global war—but it still took ten years to gain substantial power.
We gave up on trying to develop a hyper-centralized climate regime after the failure of the 2009 Copenhagen summit; instead we rely on decentralized action from governments and companies, underpinned by scientific work from the Intergovernmental Panel on Climate Change (IPCC). The UN is trying to stand up something similar for AI research, but it remains to be seen if it will be effective. Still, my personal view is that we’ll be better served by trying to leverage the existing network of international institutions that we have (UN bodies, the WHO, the ISO/IEC, etc.) by having them address AI governance within their remits is probably the best chance we have of addressing AI risks.
All of this would be easier if the US wasn’t isolating itself from the international community, but we’re making it incredibly difficult to coordinate even with our allies. One question raised during the conference was whether the EU and China could form a new axis of AI governance. In his speech at the Paris AI Action Summit, J.D. Vance issued a not-so-veiled warning towards Europe:
I would also remind our international friends here today that partnering with such regimes, it never pays off in the long term. From CCTV to 5G equipment, we're all familiar with cheap tech in the marketplace that's been heavily subsidized and exported by authoritarian regimes.
Although this all seems rather grim for action on AI, I was actually somewhat heartened by the other talks at the launch event. There are so many smart people working on climate action, whether solutions to cut emissions, suck carbon from the atmosphere, or get our governments on board with all of the above. And that is something worth celebrating.
I Reckon…
that even if DOGE isn’t using our data to train AI, using LLMs to determine which jobs are essential is still not great.
“For You Page,” the place where a social media platform recommends content.
Thumbnail generated by ChatGPT with a variation on the prompt “Generate a brushy abstract impressionist image in shades of black and white representing the concept of London”.