YouTube’s latest push to ban terrorist propaganda across its ubiquitous video platform is getting off to a rough start. Earlier this week, noted investigative reporter and researcher Alexa O’Brien woke to find that not only had she been permanently banned from YouTube, but that her Gmail and Google Drive accounts had been suspended as well. She would later learn that a reviewer who works for Google had mistakenly identified her channel, in the words of a YouTube representative, as “being dedicated to terrorist propaganda.”
This drastic enforcement action followed months of notifications from YouTube, in which O’Brien was told that three of her videos had been flagged for containing “gratuitous violence.” None of the videos, however, depict any actual scenes of violence, except for one that includes footage of American helicopter pilots gunning down civilians in Iraq, which has been widely viewed on YouTube for half a decade.
While appealing YouTube’s decision, O’Brien learned that the mechanism for correcting these mistakes can be vexing, and that a fair outcome is far from guaranteed. By Wednesday morning, her channel was slated for deletion. The Google Drive account she was locked out of contained hundreds of hours of research—or years worth of her work—and was abruptly taken offline. She was then told that she was “prohibited from accessing, possessing or creating any other YouTube accounts.” The ban was for life, and with little explanation and zero human interaction, O’Brien’s research, much of it not accessible elsewhere, was bound for Google’s trashcan.
With the knowledge that YouTube has faced increased pressure from the US and European governments to crack down on the spread of terrorist propaganda—a consequence of which has led to the disappearance of content amassed by conflict reporters—it wasn’t difficult to deduce what had happened to O’Brien’s account.
The problem was eventually addressed and representatives of both Google and YouTube later called O’Brien to apologize and explain the error. When she was told that her channel had been misidentified as an outlet for terrorist propaganda, she could hardly contain her laughter. “It was a series of unfortunate events,” a YouTube rep told her. The mistake, they explained, was the fault of a human reviewer employed by Google.
A spokesperson for Google told Gizmodo on Friday: “With the massive volume of videos on our site, sometimes we make the wrong call. When it’s brought to our attention that a video or channel has been removed mistakenly, we act quickly to reinstate it.”
This year, YouTube has begun increasingly relying on machine learning to find and scrub extremist content from its pages—a decision prompted by the successful online recruiting efforts of extremist groups such as ISIS. With over 400 hours of content uploaded to YouTube every minute, Google has pledged the development and implementation of systems to target and remove what it calls “terror content.”
Last month, a YouTube spokesperson admitted, however, that its programs “aren’t perfect,” nor are they “right for every setting.” But in many cases, the spokesperson said, its AI has proven “more accurate than humans at flagging videos that need to be removed.” In a call Wednesday, a YouTube representative told Alexa: “Humans will continue to make mistakes, just like any machine system would obviously be flawed.” The machine, which prioritizes the content reviewed by human eyes, wasn’t “quite ready,” she said, to recognize the context under which controversial content is uploaded.
The O’Brien incident demonstrates that Google has many miles to go before its AI and human reviewers are skilled enough to distinguish between extremist propaganda and the investigative work that even Google agrees is necessary to broaden the public’s knowledge of the intricate military, diplomatic, and law enforcement policies at play throughout the global war on terror.
What prompted a Google reviewer to designate O’Brien as a purveyor of terrorist content? Well, for one, her channel contains actual al-Qaeda propaganda. But that propaganda is also an important piece of US history: A few years ago, it nearly cost former US Army Private Chelsea Manning a life sentence.
O’Brien’s channel contain portions of a June 2011 video presented by al-Qaeda outlet As-Saḥāb Media featuring Adam Yahiye Gadahn, a US-born al-Qaeda operative in the Arabian Peninsula, who—in earlier jihadi propaganda tapes rebroadcast by US network news—referred to himself as “Azzam the American.” In 2006, Gadahn appeared in an al-Qaeda documentary that features an introduction by Ayman al-Zawahiri, the al-Qaeda co-founder and current leader of the organization who succeeded Bin Laden in 2011.
In January 2015, Gadahn was killed in Pakistan in a series of US drone strikes, which also claimed the lives of foreign aid workers Giovanni Lo Porto and Warren Weinstein.
O’Brien’s interest in Gadahn has nothing to do with spreading his views on the “Great Satan” or his prophesies of American streets run with blood. The footage she preserved using YouTube’s service, which was also embedded in an off-site analysis, was used by military prosecutors to support criminal offenses at the court martial of Chelsea Manning. The criminal proceedings against Manning lacked contemporaneous access to the court record. Only the work of reporters, like O’Brien, who personally attended the trial, is available to the public.
The As-Saḥāb video featuring Gadahn came into play after the US government accused Manning of “aiding the enemy,” a charge that, unlike most derived from the military’s code of justice, can be applied to civilians. And it carries a life sentence.
Manning was accused of aiding Gadahn, legally defined in the court martial as an enemy of the US, because the As-Saḥāb video cites both WikiLeaks and the State Department cables that Manning leaked. An unidentified male narrator in the Gadahn video references, for example, the “revelations of WikiLeaks,” and claims they expose “the subservience of the rulers of the Muslim world for their master America.” The video also includes portions of the infamous “Collateral Murder” tape, which depicts American Apache pilots firing upon a group of men in Baghdad, killing among them two Reuters journalists.
A stipulation in the criminal case reveals that the US government argued Osama bin Laden himself had been in receipt of, and consequently aided by, the intelligence Manning leaked. The evidence to support this, however, is classified—all of it collected during the May 2, 2011, raid on his Abbottabad compound. An analysis conducted by O’Brien, which includes the portions of the As-Saḥāb video she uploaded to YouTube, suggests that Bin Laden may have somehow received a copy of the video while hiding in Pakistan. A digital copy of the tape itself may even have been recovered by the US Navy SEALs that breached his compound during the CIA-led mission that ended in Bin Laden’s death.
The video of Gadhan had already been entered into evidence to support the aiding the enemy charge—but to prevent testimony, which would’ve involved an elaborate set-up to conceal the identity of a witness linked to the Bin Laden evidence, Manning’s defense agreed to stipulate that Bin Laden was in possession of information tied to WikiLeaks. The CIA recovered, for example, a letter from Bin Laden in which he requests from a member of al-Qaeda US Department of Defense material released by WikiLeaks. In another letter, an al-Qaeda operative attached a number of leaked battlefield reports. The defense further stipulated that Bin Laden was in possession of “Department of State information,” which O’Brien’s analysis suggests is likely the As-Saḥāb tape itself.
Ultimately, the charge didn’t stick. Manning was acquitted of aiding the enemy and convicted instead of “wanton publication,” a charge that, as O’Brien notes, “had never been used before, and is not tied to any existing federal statute or article in the Uniformed Code of Military Justice.” The sheer complexity of the case, the minute details of which are sparsely understood by the public, illustrate a need for records such as those catalogued by O’Brien to be maintained, even perhaps by YouTube, in spite of its squeamish attitude toward terrorism-related content.
Undeniably, portions of the Gadahn video uploaded by O’Brien do contain al-Qaeda propaganda—and 75 percent of content removed from YouTube over a one month period this summer involved “violent extremism”—but it did not contain scenes of graphic violence, other than the Collateral Murder tape, which, again, remains widely available across YouTube. “I personally would not have uploaded material that contained a beheading out of respect for the victim and his or her family,” she told Gizmodo. “That is a very personal choice, because provided context such material is certainly in the public interest. Moreover, if victims should die in such a manner, I might feel that I had an ethical or civic responsibility to (at a minimum) not to look away.”
“The excerpts contained in all three videos were squarely in the public interest and I handled the material responsibly,” O’Brien said. “I excerpted the portions and uploaded them to YouTube to use in my analysis of the case, because I did not want to post an entire hour and forty five minute terrorist propaganda video.” Moreover, each video included a description that offered context regarding the video’s public relevance—though that may not be visible to viewers on a mobile device.
“The material was used at a court-martial, which not only didn’t provide a contemporaneous public record of its proceedings,” O’Brien said, “but also which had legal precedents with wide ramifications for the public at large.”
The first takedown notification from YouTube arrived on July 5 and pertained to a Gadahn video titled, “Portion of As Shahab video dated June 6, 2011 from US v Pfc Manning 2007 U S Baghdad Airstrike Video.” A second notification, citing a separate clip from the 2011 video, arrived on Aug. 8. In both cases, O’Brien was warned that the videos had been flagged, and that, upon review, YouTube had determined they violated user guidelines. In total, three videos were pulled down. In every case, she was “assigned a Community Guidelines strike,” further receipts of which, YouTube warned, could lead to her “account being terminated.”
On Sept 12, O’Brien was informed via email that her YouTube account had been suspended due to “repeated or severe violations of [YouTube] Community Guidelines.” At the same time, Google disabled her Gmail account. She was instructed to sign back in, and warned: “If you don’t take action soon, your account and all of its contents will be scheduled for deletion.” She immediately appealed the decision.
Using an online form provided by Google, O’Brien explained who she was, that the purpose of the video was archival, newsworthy, and that it was embedded in posts describing events of Manning’s trial; but in no way was it intended to promote Al-Qaeda, its ideologies, or “gratuitous violence.” The appeal was immediately rejected in an email signed, “The YouTube Team.”
Based on her experience, O’Brien said the process whereby YouTube notifies its users about how their videos are being deleted works fine; it’s the appeals process where things seem to fall apart.
It was unclear, for instance, whether the “strikes” her account had received still stood. After discussing the problem with Google and YouTube representatives by phone—a form of redress that wouldn’t normally be available to most users—she regained access to her YouTube account; shortly after she received another email saying it was still suspended, an apparent mistake given her account is still online. She also regained access to her gmail account and Google Drive, and only one of the three videos taken down remains offline, but bizarrely it’s not the video containing the helicopter attack or Gadahn speaking directly into the camera. The video still blocked is primarily news clips and footage from music videos used by As-Saḥāb, which features voiceover discussing the leak of US State Department cables.
“This is more than a usability issue,” she said. “The company failed to articulate its policies and process in a meaningful way.”
During a phone call, YouTube and Google reps revealed that problems have arisen while trying to meet the demands of governments at war with international terror groups. The service is increasing the requirements of citizen journalists and investigative reporters—while also consulting with them—to justify posting videos related to ISIS, al-Qaeda, and other terrorist outfits.
When footage hits the site containing content to which the US government might object, the burden falls on journalists to ensure that the proper context is applied, and to show that the video contains actual educational or documentary value. “Something like uploading al-Qaeda footage, for example, without something in the video itself that makes it really clear that this is reporting, something you’re trying to shed light on, something you’re criticizing, whatever it is, would come down,” the YouTube rep said.
In the coming weeks, the rep said YouTube will be rolling out new features as part of its takedown system that will specifically request that users add context to videos involving terror content to demonstrate that documentary value. According to the rep, one way this can be done is by using YouTube’s built-in tools that allow users to add captions, cards or endcards to their videos. O’Brien asked if it were sufficient to add a caption that reads something like, “This for archival purposes. This is not for propaganda purposes.”
“Yes, exactly,” the rep replied.
The rep noted that YouTube has been working with other tech companies, such as Twitter, Facebook, and Microsoft, to share material with known ties to terrorist organizations. “But you should not be affected by that,” the rep told O’Brien, adding: “It’s possible that the video you shared includes al-Qaeda propaganda footage that’s in our shared database already.” In other words, it’s possible O’Brien’s videos contain a fingerprint that YouTube’s AI is directly trained to detect, and that that information has been shared with other companies helping in the battle “against radicalization and terrorist propaganda,” she said.
For law enforcement and intelligence agencies, acquiring access to information about the fingerprinted videos, or the identities of the people who’ve posted them, still requires a legal request—typically a subpoena. Of course, the more companies that have access to the data, the more opportunities the FBI, for example, has to acquire it.
“YouTube is an important global platform for news and information and we have clear policies that outline what content is acceptable to post,” a Google spokesperson told Gizmodo. “Sometimes graphic material is vital to our understanding of the world, whether it is posted to document wars or revolutions, to expose an injustice, or to ensure local events are seen globally. For all types of content on YouTube, but particularly with graphic content, adding context is important in helping the YouTube team review a video when it is flagged. Adding context within the video, like commentary or text, helps us understand background and intent.”
YouTube hopes to prevent other journalists from being banned like O’Brien was by “whitelisting” their accounts, meaning they will be flagged so that Google’s reviewers won’t mistake them for terrorist-run channels. The Google rep told O’Brien that her channel would be added to the whitelist. Individual videos, however, are still subject to the normal reporting and review procedures, so it’s still possible that some of her videos could be taken down.
“I don’t fault the company for coming to terms with terrorist propaganda on their platforms,” O’Brien said. “But YouTube’s decision tree should have more than two branches. Nuance is not binary. Neither is wisdom. I think it is humorous and frightening that a unidentified human determined that my channel supported terrorist propaganda.”
“Outside of YouTube or Google’s concerns about its public image or public safety, the company has a responsibility to engage its users about its civic and ethical responsibilities,” she added. “It must engage and articulate its policies, and if it can’t do that, then it really has not earned the credibility, nor would it deserve the power that it has to impact society. It also has a ethical responsibility to notify users if there are risks, legal or otherwise, for being identified in their protocols as a purveyor of inappropriate content.”