LibGuides: Artificial Intelligence (AI) and the Law: Content Moderation AI

Introduction

"For categories of prohibited content where AI is known to perform with high accuracy, such as spam, the content is automatically removed. But most types of policy violations are more subtle, so potentially violating posts identified automatically need to be sent to human content moderators to review."

Narayanan, Arvind; Kapoor, Sayash. AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference (pp. 181-182). Princeton University Press. Kindle Edition.

Harms

Facebook’s Content Moderation Failures in Ethiopia: Facebook has failed to moderate content in underserved countries. Facebook and other social media companies must invest more in local content moderation, instead of relying on global AI systems.
Council on Foreign Relations 4/19/22
Facebook did little to moderate posts in the world’s most violent countries
Politco 10/25/21
FACEBOOK RESPONSE: Sri Lanka Human Rights Impact Assessment
5/12/2020
Myanmar: The social atrocity: Meta and the right to remedy for the Rohingya
Amnesty International 9/29/22. Facebook was too slow to take down hate posts that encouraged violence.
Overrun by influencers, historic sites are banning TikTok creators in Nepal
Rest of the World 7/18/22
Refugees help power machine learning advances at Microsoft, Facebook, and Amazon Big tech relies on the victims of economic collapse.
Using local workers can also result in abuse of human content moderators due to low pay, exposure to toxic content, and exploitation of vulnerable populations.

Law Review Article

Content Moderation as Systems Thinking
Douek, Evelyn, 136 Harv. L.-Rev. 526 (2022)

Laws

EU Artificial Intelligence Act
Disclosure requirements
EU Digital Markets Act
aimed at increasing competition in online platforms, such as by disallowing large online platforms from self-preferencing their own results.
EU Digital Services Act
Covers lots of topics, including content moderation. It requires more transparency about automated content moderation tools and better oversight of moderation decisions.
EU General Data Protection Regulation
Privacy rules for how companies use and store personal data.
The EU General-Purpose AI Code of Practice
Voluntary

AI for Translation

AI-Generated Translation Devices: We Tested 3 So You Don’t Have To
Certified Languages International, 10/2/24
If AI is so good, why are there still so many jobs for translators?
NPR.org 6/18/24
The pros and cons of Artificial Intelligence in translations
XTM.cloud 6/12/25
Using AI for Translation: (When) is it Safe?
American Translators Association 4/17/24

Self-Regulation

The Facebook Oversight Board: Creating an Independent Institution to Adjudicate Online Free Expression
Facebook’s response to the Oversight Board’s first decisions
The Santa Clara Principles On Transparency and Accountability in Content Moderation
SantaClaraPriniciples.org
Unpaid social media moderators perform labor worth at least $3.4 million a year on Reddit alone
Northwestern Now 5/31/22. Reddit volunteers perform content moderation, with some AI tools.

Recommendation Algorithms Work at Cross Purposes to Content Moderation

Engagement, User Satisfaction, and the Amplification of Divisive Content on Social Media
Milli S, Carroll M, Wang Y, Pandey S, Zhao S, Dragan AD. ArXiv. Last revised 12/7/24.
Understanding Social Media Recommendation Algorithms
Arvind Narayanan, Knight First Amendment Institute at Columbia University, 3/9/23

Artificial Intelligence (AI) and the Law: Content Moderation AI