publications

2027

1. IEEE S&P

   

   Janus: a Playground for User-Involved Agentic Permission Management

   Natalie Grace Brigham, Eugene Bagdasarian, Tadayoshi Kohno, and 1 more author

   IEEE Symposium on Security and Privacy (S&P), Montreal, May 2027

   Abs arXiv PDF

   AI agents that autonomously execute tool calls on a user’s behalf raise pressing questions about permission management: what role could users play, and what role should they play? Despite many proposed approaches, the user’s role in agentic permission management remains under explored. We introduce Janus, a playground system for implementing and evaluating user-involved agentic permission management designs. Janus consists of two components: Janus-Core, a modular agentic system supporting a diverse spectrum of permission management designs, and Janus-Harness, an automated evaluation framework. Grounded in a conceptual model that identifies key design axes for user involvement, we implement six permission assistants spanning the design space and evaluate them across three scenarios and three synthetic responders. We demonstrate that user input is critical and can significantly strengthen privacy and security, that AI augmentation of user decisions can help reduce cognitive load, and that realistic user behavior including permission fatigue must be accounted for in system design. No single design performs optimally across all contexts, motivating a more principled and context-sensitive approach to deploying permission assistants in agentic systems. Janus is publicly available to support future investigation into this dimension of agentic system design.

2026

1. preprint

   

   Threat Modeling Intimate Partner Abuse in Agentic Systems

   Natalie Grace Brigham, Eugene Bagdasarian, Franziska Roesner, and 1 more author

   Oct 2026

   Abs arXiv PDF

   Agentic systems are AI-powered technologies capable of autonomous action within digital environments, such as sending emails or making reservations. They are deeply connected to external tools as well as users’ data and resources, making them vulnerable to attacks which exploit their access and capabilities. In this work, we apply the technology-facilitated abuse threat model, in the context of intimate partner violence and surveillance, to these systems, revealing threats that require deliberate investigation and mitigation. We categorize attacks based on system ownership: (1) Abuser-as-owner attacks demonstrate how agentic systems make existing forms of technology-facilitated abuse far easier and more scalable. (2) Victim-survivor-as-owner attacks show how agentic systems create a new attack surface which abusers can exploit. In light of these risks, we outline research and development directions for effective mitigation strategies.

2. JOTS & TSRC

   

   Trust and Safety of Human- and AI-Powered Visual Description: A Case Study of Be My Eyes

   Natalie Grace Brigham and Tadayoshi Kohno

   Journal of Trust and Safety (JOTS) & Trust and Safety Research Conference, Stanford, Oct 2026

   Abs DOI PDF

   Be My Eyes provides a volunteer-powered visual description service that connects over one million blind and low vision users with more than ten million volunteers for everyday visual assistance tasks as well as Be My AI, an AI-powered counterpart. In this work, we qualitatively analyze 1,428 one-, two-, and three-star reviews from the Apple App Store and Google Play Store to holistically examine the concerns and harmful experiences of both users and volunteers. Our analysis surfaces online harassment and abuse (trolling, cyberflashing, reporting failures), privacy concerns beyond information exposure (data use for model training), and accessibility/safety problems (inaccurate descriptions, paternalistic guardrails). We additionally analyze platform policies to examine whether and how these issues are addressed by them. Based on challenges we find related to accessibility, security, privacy, and AI, we provide recommendations for strengthening trust and safety on visual description services that serve blind and low vision people’s access needs.

3. AIES

   

   Examining Risks Through a Characterization of the AI Companion Application Ecosystem: A Stratified Sample from the Apple App Store and Google Play Store

   Natalie Grace Brigham, Lucy Qin, and Tadayoshi Kohno

   AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), Malmö, Oct 2026

   This work was also presented as a poster at the Symposium on Usable Privacy and Security (SOUPS) 2026.

   Abs DOI arXiv PDF Poster

   While computer systems that allow users to interact through conversational natural language (i.e., chatbots) have existed for many years, various types of applications offering AI companionship (e.g., Character AI, Replika) have proliferated in recent years due to advancements in large language models. To better understand this application ecosystem, we identified 489 unique apps from the Apple App Store and Google Play Store that advertised AI companionship with social or relational capabilities (e.g., an AI romantic partner). We then systematically conducted and analyzed walkthroughs of a stratified sample of 30 apps, focusing on two distinct risk categories: potential harms posed to users by AI companion apps, and potential harms enabled by malicious users exploiting app features. Through our analysis, we categorize broader ecosystem trends that provide context for understanding risks and identify specific risks related to sensitive data collection and sharing, anthropomorphism, engagement mechanisms, sexual content, as well as the ingestion and reconstruction of likeness, including the potential for generating synthetic nonconsensual intimate imagery (synthetic NCII). We conclude with a discussion of paths for different key stakeholders to mitigate the identified risks.

2025

1. AIES

   

   Biased AI Outputs Can Impact Humans’ Implicit Bias: A Case Study of the Impact of Gender-Biased Text-to-Image Generators

   Mattea Sim, Natalie Grace Brigham, Tadayoshi Kohno, and 2 more authors

   AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), Madrid, Oct 2025

   Abs DOI PDF

   A wave of recent work demonstrates that text-to-image generators (i.e., t2i) can perpetuate and amplify stereotypes about social groups. This research asks: what are the implications of biased t2i for humans who interact with these systems? Across three human-subjects studies, 1,881 participants engaged in a simulated t2i interaction in which the output was controlled to appear either stereotypic, gender-balanced, or counter-stereotypic, via the ratio of perceived women and men in the output of occupation prompts (e.g., a physicist). We then measured people’s implicit gender bias using a gender-brilliance implicit association task (IAT), a bias that both relates to stereotypic occupation output in t2i and that has implications for women’s representation in different fields. Participants who interacted with neutral t2i output (including only gender-neutral objects, e.g., DVDs) showed relatively high implicit gender-brilliance bias at baseline. Stereotypic t2i output did not increase implicit gender bias relative to this baseline (Study 1). However, participants exposed to counter-stereotypic t2i output had significantly lower implicit gender bias than participants exposed to only gender-neutral output (Studies 1 and 2). Although counter-stereotypic t2i may reduce implicit gender bias amongst users, less than 5% of participants actually preferred the counter-stereotypic representations of women and men. Instead, most participants preferred representations that accurately reflect gender distributions in society or that are more gender-balanced (Study 3). This work demonstrates a novel approach to studying human-AI interaction and reveals important insights for designing generative AI that seeks to mitigate harm. In particular, these findings have implications for understanding the impact of stereotypic t2i on human users, bias mitigation strategies via counter-stereotypic t2i output, and how these impacts (mis)align with people’s preferences for t2i representations.

2. USENIX

   

   Analyzing the AI Nudification Application Ecosystem

   Cassidy Gibson, Daniel Olszewski, Natalie Grace Brigham, and 5 more authors

   USENIX Security Symposium, Seattle, Aug 2025

   Awards Abs arXiv PDF

   Internet Defense Prize Runner Up, CSAW 2025 Social Impact Winner

   Given a source image of a clothed person (an image subject), AI-based nudification applications can produce nude (undressed) images of that person. Moreover, not only do such applications exist, but there is ample evidence of the use of such applications in the real world and without the consent of an image subject. Still, despite the growing awareness of the existence of such applications and their potential to violate the rights of image subjects and cause downstream harms, there has been no systematic study of the nudification application ecosystem across multiple applications. We conduct such a study here, focusing on 20 popular and easy-to-find nudification websites. We study the positioning of these web applications (e.g., finding that most sites explicitly target the nudification of women, not all people), the features that they advertise (e.g., ranging from undressing-in-place to the rendering of image subjects in sexual positions, as well as differing user-privacy options), and their underlying monetization infrastructure (e.g., credit cards and cryptocurrencies). We believe this work will empower future, data-informed conversations—within the scientific, technical, and policy communities—on how to better protect individuals’ rights and minimize harm in the face of modern (and future) AI-based nudification applications.

2024

1. SoLaR @ NeurIPS

   

   Developing Story: Case Studies of Generative AI’s Use in Journalism

   Natalie Grace Brigham, Chongjiu Gao, Tadayoshi Kohno, and 2 more authors

   Workshop on Socially Responsible Language Modelling Research (SoLaR), Vancouver, Dec 2024

   Abs arXiv PDF Poster

   Journalists are among the many users of large language models (LLMs). To better understand the journalist-AI interactions, we conduct a study of LLM usage by two news agencies through browsing the WildChat dataset, identifying candidate interactions, and verifying them by matching to online published articles. Our analysis uncovers instances where journalists provide sensitive material such as confidential correspondence with sources or articles from other agencies to the LLM as stimuli and prompt it to generate articles, and publish these machine-generated articles with limited intervention (median output-publication ROUGE-L of 0.62). Based on our findings, we call for further research into what constitutes responsible use of AI, and the establishment of clear guidelines and best practices on using LLMs in a journalistic context.

2. SOUPS

   

   “Violation of my body:” Perceptions of AI-generated non-consensual (intimate) imagery

   Natalie Grace Brigham, Miranda Wei, Tadayoshi Kohno, and 1 more author

   Symposium on Usable Privacy and Security (SOUPS), Philadelphia, Aug 2024

   Abs arXiv PDF Video Slides

   AI technology has enabled the creation of deepfakes: hyper-realistic synthetic media. We surveyed 315 individuals in the U.S. on their views regarding the hypothetical non-consensual creation of deepfakes depicting them, including deepfakes portraying sexual acts. Respondents indicated strong opposition to creating and, even more so, sharing non-consensually created synthetic content, especially if that content depicts a sexual act. However, seeking out such content appeared more acceptable to some respondents. Attitudes around acceptability varied further based on the hypothetical creator’s relationship to the participant, the respondent’s gender and their attitudes towards sexual consent. This study provides initial insight into public perspectives of a growing threat and highlights the need for further research to inform social norms as well as ongoing policy conversations and technical developments in generative AI.