March 5, 2021
by Rina Diane Caballar
photo by David Kelly Crow
Manipulative user interfaces — or dark patterns as they’re commonly known in the user experience design world — are web interfaces that have been designed to deceive or mislead users, such as charging them a recurring fee under the pretense of a free trial, or signing them up for memberships or subscriptions without their knowledge or consent after purchasing a product. In 2010, user experience specialist Harry Brignull coined the term “dark patterns” to describe the deceptive nature of these interfaces (making them “dark”) and their frequent occurrence (hence the “pattern”).
In his dissertation, Arunesh Mathur, a 2020 computer science Ph.D. graduate from Princeton and now a postdoctoral research fellow at the university’s Center for Information Technology Policy (CITP), examined the extent of manipulative user interfaces in three areas: influencer endorsements on social media, shopping websites, and political campaign emails. Mathur not only identified dark patterns and their characteristics, but he also quantified their prevalence using a combination of web automation tools and machine learning techniques.
“These interfaces are increasingly being discussed in the public sphere, yet there’s very little information about how they actually appear on the web — what they look like, what their varieties look like,” Mathur said. “We wanted to build knowledge that can help journalists, policymakers, and other researchers better understand the implications of these interfaces, and to shed some light on these practices.”
An example of visual interference, a type of manipulative interface in a shopping website uncovered by Mathur in his dissertation. The opt-out option is grayed out to indicate it is disabled or cannot be clicked, when it can. Image courtesy of Arunesh Mathur.
For his work, Mathur won the 2021 Outstanding Dissertation Award from the Association for Computing Machinery’s Special Interest Group on Computer-Human Interaction (SIGCHI). The SIGCHI Awards “identify and honor leaders and shapers of the field of human-computer interaction,” and the SIGCHI Outstanding Dissertation Award “recognizes excellent thesis research by recent Ph.D. recipients in human-computer interaction.” Winners of the award receive a $1,000 honorarium and are invited to give a talk about their dissertation at the CHI conference.
In order to measure the different forms of manipulative user interfaces at scale, Mathur first needed a way to gather the necessary data. Because each area has a set structure — for instance, all YouTube videos have a description below them, while all shopping websites have pages for product viewing and checkout — Mathur was able to leverage these structural similarities to automate the data gathering process, developing web bots that mimic user interactions and collect data.
“We had to come up with some clever techniques to build web bots that can browse shopping websites or help us sign up for emails,” said Mathur. “It certainly requires a lot of engineering work, and that was a big part of the challenge in discovering many of these patterns.”
Once he had the data, Mathur used machine learning to help him perform analysis at scale, training supervised machine learning classifiers to identify dark pattern types in the data set. He then filtered the patterns into a form where he could manually annotate and qualitatively analyze them. This is where Mathur’s human-computer interaction expertise came in, using qualitative data analysis techniques to pull out the different dark patterns he was seeing.
“If you’re only doing qualitative data analysis, you would be limited to finding dark patterns at a small scale. If you’re only using machine learning, you may not get that fine-grained level of detail into the kinds of patterns that exist. It’s this interdisciplinary approach that makes Arunesh’s contributions much stronger,” said Marshini Chetty, Mathur’s dissertation adviser who was previously at Princeton and is now an assistant professor at the University of Chicago.
The results from Mathur’s research were astounding. Of the 500,000 YouTube videos and 2 million Pinterest pins analyzed, around 90% of endorsements go undisclosed, making users believe they’re viewing unbiased, non-advertising content. Meanwhile, 1,818 dark patterns were present on 1,254 of the 11,000 shopping websites analyzed, in which these patterns misled users into making more purchases or disclosing more information than they would otherwise. And of the more than 100,000 emails from U.S. political campaigns and organizations during the 2020 election cycle, an estimated 40% contain manipulative interfaces that nudge recipients to click and read emails. Moreover, email address sharing among campaigns and organizations was found to be widespread, with this sharing not disclosed to users in a transparent way.
“The fact that we’ve been able to discover these many dark patterns tells us that it’s going to be hard for consumers to get away from them, just because of how prevalent they are,” Mathur said. “This paints a challenging picture, and there needs to be more awareness and better protection from these interfaces.”
Mathur has made the data and findings of his research on shopping websites and political emails available to the public. Code from his studies can also be found on GitHub, making it more accessible for software engineers, researchers, and other groups to explore these problems in more detail and adapt them to their use case.
Going a step further, Mathur has presented his research to the Federal Trade Commission, discussing how they should think about and regulate dark patterns. Mathur also notes that his research has been cited in many of the reports that European authorities have been drafting about the rise of dark patterns and regulation of platforms, asking companies and other bodies to stop using these practices and creating these interfaces.
“Arunesh made sure his work had a broad impact. He went further than just publishing about how different forms of manipulation are occurring online. He did a lot of outreach that was beyond what was necessary to achieve his PhD,” Chetty said. As she worked with Mathur on his dissertation at Princeton, Chetty got to see him develop as a researcher. “He was always two or three steps ahead of me. Anything I thought of in terms of research rigor or questions I had, he often had already thought about, whether related to methodological questions or just other things to consider in the research.”
According to Arvind Narayanan, Mathur’s collaborator on his doctoral work as well as at CITP, Mathur’s work is exceptional because he has the right mix of creativity and perseverance. “For a highly creative person, it can be frustrating to roll up one’s sleeves and work on methodical data collection for a year or more before reaching the more rewarding parts of a research project,” Narayanan said. “Arunesh has always been willing to do this, which is why he has been able to publish several groundbreaking empirical studies.”
At CITP, Mathur is continuing his research on manipulative user interfaces, this time focusing on online advertising. After that, he plans to conduct user studies to better characterize the effects of dark patterns on users. “Dark patterns are not going anywhere,” Mathur said. “It’s important for policymakers and regulators to realize some of this research, and I think we’ll continue to see some action and enforcement in this area in the coming years.”