Cory Doctorow's linkblog
In the fall of 2020, Facebook went to war against Ad Observatory, a NYU-hosted crowdsourcing project that lets FB users capture the paid political ads they see through a browser plugin that santizes them of personal information and then uploads them to a portal that disinformation researchers can analyze.
https://pluralistic.net/2020/10/25/musical-chairs/#son-of-power-ventures
1/
Cory Doctorow's linkblog
Facebook said that its own disinfo research portal showed it was holding up its end of the bargain, and the company hated that Ad Observatory showed that this portal was a bad joke:
https://pluralistic.net/2021/08/06/get-you-coming-and-going/#potemkin-research-program
3/
Cory Doctorow's linkblog
Facebook's attacks were truly shameless. They told easily disproved lies (for example, claiming that the plugin gathered sensitive personal data, despite publicly available, audited source-code that proved this was absolute bullshit).
Why was Facebook so desperate to prevent a watchdog from auditing its political ads? Well, the company had promised to curb the rampant paid political disinformation on its platform as part of a settlement with regulators.
2/
Cory Doctorow's linkblog
Remember when Facebook management sat idly by as its own staff and external experts warned them that the platform was being used to organize genocidal pogroms in Myanmar against the Rohingya people? Remember Facebook's teary apology and promise to do better?
They didn't do better.
9/
Cory Doctorow's linkblog
The human rights org Global Witness tried buying ads on Facebook for eight pro-genocide phrases that had been used during the 2017 genocide. Facebook accepted *all eight ads*, even though they duplicated the messages it promised it would block in the future (Global Witness cancelled the ads before they could run).
10/
Cory Doctorow's linkblog
Whenever Zuck is dragged in front of Congress and they demand answers about what he's going to do about the open sewer he's trapped billions of internet users in, he always has the same answer: "The AI will fix it."
This is the pie-in-the-sky answer for every billionaire grifter (see also: "How will Uber ever turn a profit?").
7/
Cory Doctorow's linkblog
No one who understands machine learning (except for people extracting fat Big Tech salaries) takes this nonsense seriously. They know ML isn't up to the job.
But even by the standards of machine learning horror stories, the latest Facebook moderation failure is a fucking doozy. Genocidal, even.
8/
Cory Doctorow's linkblog
And while Facebook claims that it wants to purge its platform of disinformation, the reality is that disinfo is *very* profitable for the company. Ads for financial fraud, identity theft, dangerous scam products, and political disinformation are disproportionately lucrative for Facebook:
https://pluralistic.net/2020/12/11/number-eight/#curse-of-bigness
5/
Cory Doctorow's linkblog
All of this is the absolutely predictable consequence of Facebook's deliberate choice to "blitzscale" to the point where they are moderating three billion users' speech in more than 1,000 languages and more than 100 countries. Facebook may secretly like failing at this, but even if they were serious about the project, they would still fail.
6/
Cory Doctorow's linkblog
Facebook's leadership are accustomed to commanding a machine powerful enough to construct reality itself. That's why they nuked Crowdtangle, their own internal research platform that disproved the company's claims about how its amplification system worked, showing that it was rigged to goose far-right conspiratorialism:
https://pluralistic.net/2021/07/15/three-wise-zucks-in-a-trenchcoat/#inconvenient-truth
4/
Cory Doctorow's linkblog
A common newbie error in machine learning is to forget to hold back training data to evaluate the model with. Training an ML model involves feeding it a bunch of data (say, "messages that foment genocide against Rohingya people") so it can build a statistical model of what its target looks like. Then you take some of that training data - a portion you didn't use to train the model on - and see if the model recognizes it.
14/
Cory Doctorow's linkblog
If you forget and evaluate your model using some of its training data, you're not measuring whether the model can evaluate *new* input correctly - you're just checking to see whether it remembers seeing this input it's already seen.
Incredibly, FB seems to have done the opposite: they've produced a filter than *can't recognize the input it was trained on.*
15/
Cory Doctorow's linkblog
Facebook has claimed that:
a) It will filter out messages that promote genocide against Rohingya people;
b) It will subject paid ads to higher levels of scrutiny than other content;
c) It will subject political ads to the highest level of scrutiny.
12/
Cory Doctorow's linkblog
Facebook used legal threats to terrorize accountability groups seeking to hold them to these promises, stating that its in-house tools were sufficient to address its epidemic of paid political disinformation.
13/
Cory Doctorow's linkblog
Some of the phrases Facebook's moderation tool failed to catch:
* "The current killing of the [slur] is not enough, we need to kill more!"
* "They are very dirty. The Bengali/Rohingya women have a very low standard of living and poor hygiene. They are not attractive"
11/
Chirp! is a social network. It runs on GNU social, version 2.0.1-beta0, available under the GNU Affero General Public License.
All Chirp! content and data are available under the Creative Commons Attribution 3.0 license.