Placebo Effect of Control Settings in Feeds Are Not Always Strong

2025-04-24

Towards A Global AI Auditing Framework: Assessment and Recommendations

2025-02-01 · 2 citations

A high-level précis of the Synthesis Report can be found in the Summary for Policymakers Recommendations for a Global AI Auditing Framework: Summary of Standards and Features. The growing integration of artificial intelligence (AI) into critical sectors of society, from healthcare to education, has the potential to support widespread social transformation and progress. However, AI systems also have the power to perpetuate biases, deepen inequalities, and cause environmental harm. Accurately evaluating the risks and benefits of an AI system requires a careful audit. Current approaches to auditing, however, rarely involve independent auditors, provide sufficient evidence, or account for global impacts. Policymakers urgently need a comprehensive global framework for AI audits that validates genuine benefits and risks. The IPIE’s Scientific Panel on Global Standards for AI Audits set out to independently establish global, cross-disciplinary scientific consensus on what makes an audit effective and trustworthy. The Panel consisted of 16 experts from computer science, the social sciences, and the humanities, with expertise spanning generative AI systems, algorithmic auditing, indigenous data sovereignty, data journalism, and the relationship between civil society, human rights, and AI.

Global Approaches to Auditing Artificial Intelligence: A Literature Review

2024-08-01 · 2 citations

This Synthesis Report is a literature review outlining the regulatory, industry, and academic approaches to AI audits. We review 78 articles published in peer-reviewed journals and as preprints, 21 documents from industry associations and standard-setting organizations, and national policy documents and regulations from 20 countries. Based on this review, we identify three key takeaways about the landscape of AI auditing: 1. To accurately assess the potential risks, and impacts of AI systems, we need a trustworthy audit ecosystem with complementary approaches from internal, external, and community auditors. 2. Auditors need better access to data and audit artifacts from the developers and deployers of AI systems. Comprehensive auditability requires documentation and disclosure of an AI system’s model and data components, associated risks and impacts, and easily understandable explanations of its outcomes. 3. Given that the development and use of AI systems impacts communities across the world, audit regimes must account for their global effects. Most existing audits have been conducted in North America, Europe, and other regions of the ‘global north’, with their results typically published in English and focused on effects within these regions. The impacts of AI systems, however, include shifts in social and environmental conditions beyond the immediate development or application contexts of a system.

Recommendations for a Global AI Auditing Framework: Summary of Standards and Features

2024-12-01

This Summary for Policymakers provides a high-level précis of the Synthesis Report Towards A Global AI Auditing Framework: Assessment and Recommendations. The growing integration of artificial intelligence (AI) into critical sectors of society, from healthcare to education, has the potential to support widespread social transformation and progress. However, AI systems also have the power to perpetuate biases, deepen inequalities, and cause environmental harm. Accurately evaluating the risks and benefits of an AI system requires a careful audit. Current approaches to auditing, however, rarely involve independent auditors, provide sufficient evidence, or account for global impacts. Policymakers urgently need a comprehensive global framework for AI audits that validates genuine benefits and risks. The IPIE’s Scientific Panel on Global Standards for AI Audits set out to independently establish global, cross-disciplinary scientific consensus on what makes an audit effective and trustworthy. The Panel consisted of 16 experts from computer science, the social sciences, and the humanities, with expertise spanning generative AI systems, algorithmic auditing, indigenous data sovereignty, data journalism, and the relationship between civil society, human rights, and AI.

Chapter 10 The Structural Problems of the Internet for Cultural Policy

New York University Press eBooks · 2022-07-18

Auditing Algorithms: Understanding Algorithmic Systems from the Outside In

Foundations and Trends® in Human–Computer Interaction · 2021 · 110 citations

• Computer Science
• Computer Science
• Algorithm

Algorithms are ubiquitous and critical sources of information online, increasingly acting as gatekeepers for users accessing or sharing information about virtually any topic, including their personal lives and those of friends and family, news and politics, entertainment, and even information about health and well-being. As a result, algorithmically-curated content is drawing increased attention and scrutiny from users, the media, and lawmakers alike. However, studying such content poses considerable challenges, as it is both dynamic and ephemeral: these algorithms are constantly changing, and frequently changing silently, with no record of the content to which users have been exposed over time. One strategy that has proven effective is the algorithm audit: a method of repeatedly querying an algorithm and observing its output in order to draw conclusions about the algorithm’s opaque inner workings and possible external impact. In this work, we present an overview of the algorithm audit methodology, including the history of audit studies in the social sciences from which this method is derived; a summary of key algorithm audits over the last two decades in a variety of domains, including health, politics, discrimination, and others; and a set of best practices for conducting algorithm audits today, contextualizing these practices using search engine audits as a case study. Finally, we conclude by discussing the social, ethical, and political dimensions of auditing algorithms, and propose normative standards for the use of this method.

Auditing Algorithms: Understanding Algorithmic Systems from the Outside In

2021-01-01 · 22 citations

Algorithms are ubiquitous and critical sources of information online and increasingly act as gatekeepers for users accessing or sharing information about virtually any topic. This includes information about their personal lives and those of friends and family, news and politics, entertainment, and even health and well-being. As a result, algorithmically-curated content is drawing increased attention and scrutiny from users, the media, and lawmakers alike. Studying such content poses considerable challenges as it is both dynamic and ephemeral. One strategy that has proven effective is the algorithm audit: a method of repeatedly querying an algorithm and observing its output in order to draw conclusions about the algorithm’s opaque inner workings and possible external impact. In this work, the authors present an overview of the algorithm audit methodology. They include the history of audit studies in the social sciences from which this method is derived; a summary of key algorithm audits over the last two decades in a variety of domains such as health, politics, and discrimination; and a set of best practices for conducting algorithm audits today. The authors conclude by discussing the social, ethical, and political dimensions of auditing algorithms, and propose normative standards for the use of this method.

"At the End of the Day Facebook Does What ItWants"

Proceedings of the ACM on Human-Computer Interaction · 2020-10-14 · 74 citations

Interest has grown in designing algorithmic decision making systems for contestability. In this work, we study how users experience contesting unfavorable social media content moderation decisions. A large-scale online experiment tests whether different forms of appeals can improve users' experiences of automated decision making. We study the impact on users' perceptions of the Fairness, Accountability, and Trustworthiness of algorithmic decisions, as well as their feelings of Control (FACT). Surprisingly, we find that none of the appeal designs improve FACT perceptions compared to a no appeal baseline. We qualitatively analyze how users write appeals, and find that they contest the decision itself, but also more fundamental issues like the goal of moderating content, the idea of automation, and the inconsistency of the system as a whole. We conclude with suggestions for -- as well as a discussion of the challenges of -- designing for contestability.

Auditing Race and Gender Discrimination in Online Housing Markets

Proceedings of the International AAAI Conference on Web and Social Media · 2020 · 59 citations

• Computer Science
• Sociology
• Internet privacy

While researchers have developed rigorous practices for offline housing audits to enforce the US Fair Housing Act, the online world lacks similar practices. In this work we lay out principles for developing and performing online fairness audits. We demonstrate a controlled sock-puppet audit technique for building online profiles associated with a specific demographic profile or intersection of profiles, and describe the requirements to train and verify profiles of other demographics. We also present two audits using these sock-puppet profiles. The first audit explores the number and content of housing-related ads served to a user. The second compares the ordering of personalized recommendations on major housing and real-estate sites. We examine whether the results of each of these audits exhibit indirect discrimination: whether there is correlation between the content served and users' protected features, even if the system does not know or use these features explicitly. Our results show differential treatment in the number and type of housing ads served based on the user's race, as well as bias in property recommendations based on the user's gender. We believe this framework provides a compelling foundation for further exploration of housing fairness online.

Communicating Algorithmic Process in Online Behavioral Advertising

2018-04-20 · 161 citations

Advertisers develop algorithms to select the most relevant advertisements for users. However, the opacity of these algorithms, along with their potential for violating user privacy, has decreased user trust and preference in behavioral advertising. To mitigate this, advertisers have started to communicate algorithmic processes in behavioral advertising. However, how revealing parts of the algorithmic process affects users' perceptions towards ads and platforms is still an open question. To investigate this, we exposed 32 users to why an ad is shown to them, what advertising algorithms infer about them, and how advertisers use this information. Users preferred interpretable, non-creepy explanations about why an ad is presented, along with a recognizable link to their identity. We further found that exposing users to their algorithmically-derived attributes led to algorithm disillusionment---users found that advertising algorithms they thought were perfect were far from it. We propose design implications to effectively communicate information about advertising algorithms.