YouTube has been criticized through they years over the way it recommends videos to people — more specifically, the opaque nature of the Google-owned platform’s algorithms. While we don’t really understand how it all works, we do know that it relies on a real-time feedback loop to suggest new videos to watch, with the suggestions differing for each viewer based on their individual viewing activity.
Studies have suggested that YouTube plays a big part in fueling conspiracy theories, spreading misinformation, and “radicalizing” society, and the company has apparently made some efforts to reel in the monster it created. However, with a looming U.S. presidential election, this offers a golden opportunity to investigate how ideas spread online, which is Mozilla today lifted the lid on a new research initiative designed to better understand how YouTube algorithms actually work.
With RegretsReporter, a browser add-on for Firefox and Chrome, Mozilla is inviting users to submit reports to Mozilla whenever they are recommended “harmful” videos. Once installed, a user can click on the “frowning” RegretsReporter icon in their browser bar, which only works when an active YouTube video is on screen .
This will surface the video-viewing history of that session, and allow users to hit a “report video and history” button.
Before submitting, the user is asked to provide more information on what they found regrettable about the video.
According to Mozilla, it hopes to gain insights into the patterns around frequency or severity of harmful content, and whether there are specific usage patterns that lead to such recommendations being made.
The true value of RegretsReporter will, of course, lie in the quality and quantity of data it garners. According to Mozilla, each report submitted will include YouTube browsing history for up to 5 hours prior to submitting the report — this will include what kind of YouTube pages that were visited, and how they were reached. Users can also manually delete any specific video from their history that they don’t want to submit, though this would presumably compromise the integrity of the data.
Moreover, it’s pretty clear that YouTube recommendations are based on viewing patterns dating back much further than 5 hours, so this raises questions on how insightful this research is likely to be — even if it does manage to achieve meaningful scale in terms of installations. Mozilla acknowledges this shortfall, but still thinks there is enough value in the data it does to obtain meaningful takeaways.
“Many of our research questions can be studied without knowledge of a user’s full viewing data,” Brandi Geurkink, senior campaigner at Mozilla, told VentureBeat. “For example, are there identifiable patterns in terms of frequency or severity of reported content? And does the report frequency increase for users after they send their first report? That said, we know that the insights we glean won’t be comprehensive. We aim to identify areas where more examination is needed, and then build momentum to enable that deeper examination.”
And then there is the issue of data privacy, a thorny subject in today’s internet age. In addition to the data it collects at every form submission, Mozilla’s privacy T&Cs note that the add-on will gather “periodic, aggregated information” about a user’s YouTube usage, such as how often they visit YouTube and for how long — but not what they watch or search for on the platform.
So there is some trade-off here in terms of how willing the public are to participate in a program that seeks to shine a light on YouTube’s murky AI smarts, and the data they’re willing to divulge to Mozilla.
From Mozilla’s perspective, RegretsReporter constitutes part of an ongoing campaign against YouTube, which has included research and recommendations for YouTube to fix its problem based on Mozilla’s inaugural “YouTube regrets” campaign last year. Among its suggestions at the time were for YouTube to open up more of its data and commit to working with independent researchers. In lieu of that, RegretsReporter goes some way toward taking control out of Google’s hands, albeit with limited scope.
“We were shocked by the number of responses to our initial YouTube Regrets campaign, which indicates that this is a widespread problem,” Geurkink added. “We hope that this will translate into thousands of downloads of the extension.”
Based on its findings, Mozilla said that it may work with other researchers, journalists, policymakers, and “even engineers within YouTube” on solutions to the YouTube algorithm problem, and it plans to publish the results of its research for public consumption — thought it stopped short of giving any timescales.
“It’s hard to say [when Mozilla will publish results] without first an understanding of how users will use the extension and an average rate of regrets,” Geurkink continued. “But we will be performing regular analysis from the time that we launch, rather than waiting a fixed period of time. So we plan to share findings soon after we uncover them.”