Abstract

The advent of generative artificial intelligence (GenAI) has ushered in an era of more open-ended computer systems. Trained to learn patterns from vast datasets of human expression, these technologies exhibit tremendous flexibility in the kinds of input they can parse and produce output that is far more contextually responsive to open-ended interactions with users. Faced with these rapidly burgeoning capabilities, researchers and designers are confronted with an urgent question: how should GenAI be harnessed in ways that empower rather than disempower people? We argue that addressing this question requires three fundamental conceptual shifts to redefine: (1) intelligence as agency, the capacity to meaningfully act, rather than the capacity to perform a task; (2) design as the delegation of constrained agency, rather than the specification of affordances; and (3) ethics as care, an ongoing relation of stewardship toward AI agents, rather than one-time responsibility audits. Developed through an interdisciplinary dialogue between computer science and anthropology, this framework centers convergent interests in design as the intentional shaping of cultural patterns. It offers technologists not only a novel descriptive instrument, capable of better characterizing the outputs and effects of GenAI models, but also suggests possible evaluative methods emphasizing the cultural bases of agentive intelligence.

Keywords: generative artificial intelligence (GenAI), human–computer interaction (HCI), linguistic anthropology, distributed cognition, agency

1. Introduction

Preeminent cognitive psychologist Jerome Bruner once argued that there are two different ways of conceptualizing intelligence. “The first, or computational view,” he wrote, “is concerned with information processing… It takes information as a given, as something already settled in relation to some pre-existing rule-bound code.”1 The other, which Brunner called the cultural view, is concerned with “meaning making.”2 Bruner argued that as long as AI remained rule-bound, it would never approach the capacity for meaning making distinctive of human intelligence, which “is in principle interpretive, sensitive to the occasion, and often after the fact. Its ‘ill-formed procedures’ are more like ‘maxims’ rather than like fully specifiable rules.”3

The advent of generative artificial intelligence (GenAI) has ushered in an era of more open-ended computer systems. Trained to learn patterns from vast datasets of human expression, these technologies exhibit tremendous flexibility in the kinds of input they can parse, and the output they can produce—often in ways that their designers are unable to explain. For example, large language models (LLMs) such as GPT-4 (which underlies the enormously popular ChatGPT chatbot) are fluently authoring a variety of textual content including essays, travel itineraries, and programming code; image models such as DALL-E, Midjourney, and Stable Diffusion are producing visual content that is often indistinguishable from photographs or hand-drawn art; and models across a variety of other media domains including audio, video, and animation are quickly approaching this level of quality as well.

We argue that the performance of these novel models reflects a shift in AI from information-processing toward meaning-making forms of intelligence. This shift, and the new horizons of human–computer interaction (HCI) that it appears to portend, require a corresponding conceptual reorientation—one that builds on the insights of distributed cognition4 to reflect how people not only use GenAI models as inanimate objects but also interact with them as active agents.

In the first section, we argue that GenAI requires a redefinition of intelligence not as competence, or the ability to successfully perform (usually on a prescribed task or set of tasks), but rather as agency, or the capacity to meaningfully act in a goal-directed pursuit. In the second section, we argue that such a redefinition of intelligence also entails a reconceptualization of design, not as the specification of affordances (usually of an object or tool), but rather as the delegation of constrained agency. In the third section, we argue that envisioning intelligence as agency and design as agency delegation demands a reconceptualized ethical paradigm. Whereas prior ethical approaches focused on responsibility, a one-time audit of potential impacts (usually harms), GenAI demands an ethics of care, an ongoing relation of stewardship toward AI agents and those that they affect.

Developed through an interdisciplinary dialogue between computer science and anthropology, we intend our framework to promote dialogue between researchers and practitioners in both fields, offering a conceptual model that can be studied by ethnographers and manipulated by designers, and centering convergent interests in design as the intentional shaping of behavior and relations.5

2. Intelligence: From Competence to Agency

Since the advent of the field, computing researchers have favored evaluating the efficacy of AI via empirically tractable metrics of competence. For instance, Alan Turing discerned long ago that having an objective assessor compare the level of skill displayed in an AI system’s execution of a task (say, having a conversation) to a human benchmark is a relatively straightforward experimental design.6 More recently, researchers at Microsoft benchmarked GPT-4 against a diverse range of domains (including mathematics, programming, medicine, law, and psychology) to suggest that it demonstrates “sparks’’ of more generalized intelligence than simply learning patterns specific to individual domains,7 and researchers at Google DeepMind have proposed measuring progress by describing the depth and breadth of model performance (i.e., how well does the model accomplish narrow, domain-specific or more general tasks, respectively) across five levels of competence (emerging AI through to superhuman AI).8

Even researchers who emphasize the importance of more human-centered approaches to AI have adopted a similar paradigm. AI ethics researchers have more recently cautioned against the growing hype of GenAI by focusing on descriptions of how these models perform—framing LLMs as no more than “stochastic parrot”9 next-word predictors, trained on the form of natural language that definitionally precludes them learning meaning or achieving higher-level understanding.10 Moreover, experimental fields such as psychology and education have also focused on competence-based evaluation, reinforcing broader cultural conceptions of intelligence as an individual’s skill measurable by performance on tests of aptitude.11 As long as AI was relatively limited in its capacity, such measures seemed largely adequate.

However, with AI’s burgeoning generative capacities, it is far from clear that competence provides the best metric for assessing intelligence. Most obviously, competence-based metrics elide the potential for GenAI to reinforce or reconfigure power structures. In particular, by drawing on the work of sociologists and critical data scholars such as Ruha Benjamin,12 Safiya Noble,13 and Virginia Eubanks,14 AI ethics researchers have consistently sounded the alarm on the ways machine learning (and GenAI specifically) can buttress systems of oppression and threaten to displace human labor—for example, when models perpetuate harmful stereotypes or when they reproduce the intellectual property of graphic designers without consent.15 Initial approaches for addressing these concerns have focused on improving the machine learning pipeline, including by fine-tuning models based on human preferences,16 auditing datasets for discriminatory content,17 and protecting the intellectual property of creative professionals.18 While valuable, these approaches are unable to address the more foundational concerns about power as they do not grapple with the emergent interactional dynamics between people and GenAI.

To facilitate a shift toward richer design and evaluative methods, we propose reconceptualizing intelligence and operationalizing power in terms of agency. Agency has long animated discussions in AI and human–computer interaction—from the 1960s visions of Licklider,19 Engelbart,20 and Bar-Hillel21 through to the 1990s debates on interface agents versus direct manipulation22 and more contemporary guidelines for human–AI interaction.23 Yet, despite this long-running focus, and although the third wave of HCI has expanded its epistemological lenses to include the sociocultural dimensions of interaction,24 agency remains undertheorized in the literature.25 Rather, it functions as an umbrella construct—a “broad concept or idea used loosely to encompass and account for a set of diverse phenomena,”26 including the degree of a user’s causal involvement, how outcomes align with a user’s values or goals, and whether a user is acting independently or is reliant on others.27 Without a definition that relates these constituent aspects to one another, and connects them to outcomes, it has been difficult to “accumulate and communicate knowledge”28 about agency29—for instance, allowing designers to systematically prototype alternate configurations of agency between users and GenAI, or allowing researchers to empirically compare the efficacy and evaluate the implications of these alternative designs.

In response, we offer a crisper conceptual model of agency. In contrast to prior approaches—which conceptualize automated tools as objects to be used exclusively by human agents—we instead draw on theories of agency from linguistic anthropology and build on insights from distributed cognition30 to treat the human user and GenAI model as co-agents. Thus, where previously machines produced output solely in response to human-provided input, we instead consider both the user and GenAI model to generate contextually responsive inputs and outputs. In doing so, we are better able to understand how meaningful actions emerge from interactions within a given interface and assess their broader sociocultural implications.

2.1 A Conceptual Model for Human Agency

As a working definition, we consider agency to be the capacity to meaningfully act. Agency has been a topic of considerable debate among social scientists intent on assessing the individual and collective capacity for self-determined, self-directed action; our inquiry draws principally on research by linguistic anthropologists on how speakers (and writers) make their own meanings with language.31 Figure 1 shows two individual human agents—each depicted as a black dot—surrounded by a blue sphere delineating their field of agency. Within their field of agency, their capacity to act, they can strategically compose their own actions according to means–ends calculations and interpret the efficacy of their action on the world. Thus, each independent agent is engaged in an ongoing dialectical process of compositional and interpretive activity: acting with purpose and monitoring their action. Actions are meaningful insomuch as they address or advance the agent’s goal.

Anthropologists have recognized another key and concomitant dimension of agency: accountability. As Enfield writes, “agents have accountability for meaningful behavior, insofar as: they may be subject to public evaluation by others; […] they may be regarded as having some degree of entitlement to carry out the behavior, and give reasons for it; […] they may be regarded by others as having some degree of obligation to carry out the behavior, and give reasons for it.”32 That is to say, if agency is the capacity to act as an agent, accountability is the capacity to give an account of why one acts in the way one does—an explanation of how an act meaningfully advances or addresses a goal—and hence to be held accountable by oneself and others. Our diagram depicts each agent’s accountability as a concentric orange field superimposed on the sphere of agency. Thus far, as our agents are acting independently, they are accountable only to themselves until their actions affect others.

One of the key attributes of human social life is that we seldom act alone. From intimate social groupings like a family to large-scale aggregates like a corporation or purpose-driven units like a team, individual agencies are often blended together in joint pursuit of a common purpose.33 In Figure 2, the two independent agents begin a process of combining their agencies together. To do so, they must establish common ground: a shared understanding of what they want to do and how each will contribute.34 The area of overlap is an emerging participation framework: an arrangement of roles that co-agents play in joint activity, with corresponding responsibilities and objects of focus.35 Whereas each agent previously composed and interpreted actions for their own purposes, here their capacities for meaning making are beginning to be addressed to each other, as the actions they compose are interpreted by the other co-agent who responds reciprocally.36 Thus, the arrows of composition and interpretation are now other- rather than self-directed.

Inter-agent accountability now emerges at this intersection of composition and interpretation: an agent composes action in anticipation of subsequent interpretation, and can be held accountable by the other for contributing in ways that are unintelligible or unexpected.37 This may mean that they voluntarily give their own verbalized account of why what they did does not conform to conventional expectations, or that it would be within the other agent’s rights to demand that such an account be given. In ordinary human conversation, people give accounts of their agentive behavior all the time, for instance, explaining why they cannot attend a meeting due to a scheduling conflict, or accept a party invitation due to illness.

Co-agents do not necessarily experience the blending of agency as a decrease in net capacity to act. Indeed, when it is voluntary and purpose directed, both agents may experience the fusion of agency as a net gain in their capacity to act. Thus, Figure 3 depicts the same total amount of agency as Figure 2 but, because it is fused together, the overall sphere of agency has expanded. And, as fused co-agents, the agents are not only accountable to each other for both compositions and interpretations, they are also jointly accountable to others they affect (whether or not agency and accountability are proportionately distributed is an empirical question). As joint action unfolds, co-agents may revise their objectives, modify their strategies, and reallocate responsibilities based on evolving challenges and opportunities. To collaborate effectively, co-agents continuously communicate with each other, updating understandings of the means–ends relationship.

Human conversation is a joint activity in which agency is particularly salient, as conversational partners take turns composing and interpreting utterances, alternating roles between speaker and listener.38 Whether conversation is a convivial end-in-itself or a strategic, purpose-driven activity intended to accomplish a particular goal, the parties involved participate as co-agents who “enact cooperation… by supporting the progress of the interaction.”39 The coordination of roles within the participation framework shifts accordingly.40 This is depicted as a vertical line of control in the diagram, its squiggles representing that it is dynamic and shifting, not stable and fixed: at some points over the course of an unfolding joint action, one agent might assume greater compositional agency in shaping a course of action, or another might assume greater interpretative agency in evaluating action that has met with a stubborn impasse. Dynamism is a desirable quality of effective control; in a time-lapse recording, the line of control might wobble back and forth between agents.

We offer our own example as an illustration. To coauthor the present text, we, Arvind and Graham, have blended our agencies in the joint activity of writing as co-agents. Having established interdisciplinary common ground, we constituted a participation framework that we continuously renegotiated as this text took shape. At times, we wrote side-by-side, our compositional and interpretative agencies relatively balanced. At others, we decided on a division of labor that would require one of us to assume the bulk of compositional agency for sections engaging with our respective discipline’s concerns. As we composed text for ourselves and for each other, and interpreted our own and each other’s text, our understanding of the problem space expanded; blended agency gave us more aggregate agency than either of us would have individually. We are accountable to each other and, together as co-agents, accountable to our readers.

2.2 Agency in Human-AI Interaction

Now that we have a basic vocabulary for describing blended agency, let us consider how we can use it to describe human–AI interaction. Under a traditional input–output model of interaction, it is tempting to consider the user to be the sole agent and the computer as merely an inanimate tool. However, even in this situation, a computer’s interpretative agency manifests through the affordances of the interface it provides, and it has a measure of compositional agency as engineered by the interface designer.41 Using our conceptual model as shown in Figure 4, we depict this as a narrow band of agency that the computer exercises.

While this configuration of agency can be empowering for users, researchers and designers have also identified the ways in which it can prove to be burdensome:42 a gulf of execution emerges because a user must determine how to compose their input in ways that match the affordances of an interface, and a gulf of evaluation stretches because a user must extend themselves to interpret the computer’s predefined compositions and assess whether their goals were met. Decades of computer science research has sought to narrow these gulfs by increasing the compositional and interpretative capacities of computers. For instance, researchers have explored a variety of interface techniques to allow users to compose input more abstractly,43 ambiguously,44 or even only partially45—in each case, scoping a more expansive role for the computer’s interpretative agency to fill in a variety of operational details users choose to omit. These approaches have proven to be useful by successfully reducing the cognitive effort users must expend executing particular tasks, and have been widely adopted—for example, powering features like Microsoft Excel’s Flash Fill46 for automatically applying a pattern across cells of a spreadsheet, and Tableau’s Show Me47 for recommending appropriate charts for a dataset. However, they do not quite reach the type of spontaneous joint action we are capable of performing with other people—the “think[ing] in interaction with a computer in the same way that you think with a colleague whose competence supplements your own”48 envisaged by Licklider in the 1960s—because the computer’s agency is prescribed, and thus bound by, a set of rules.

In contrast, GenAI offers researchers and designers a powerful tool for narrowing both gulfs because it is much less formally prescribed or predetermined by a designer’s choices. Trained on vast quantities of human expression, GenAI exhibits greater compositional and interpretative agency than prior techniques, often exceeding what its developers and designers thought it was capable of.49 As an example, from a natural language prompt of several words, an LLM can write a text on virtually any topic, in varied length and style.

It is important to carefully explain here what it means for an LLM to write—and therefore exhibit agency in the act of writing—given that the platform cannot be said to know what it is writing about or understand the texts that it produced in any meaningful way.50 It is a probabilistic model that interprets inputs and composes outputs based entirely on statistical patterns in its training data. It is not capable of initiating action, or acting independently. As an interface designed to respond to user prompts, however, it is capable, when bidden, to participate as a co-agent in a joint activity of writing. The text that it composes does not have any meaning, if meaning is understood as intention.51 But if meaning is understood as a dimension of goal-directed action or (in the case of joint activity) interaction, the LLM meaningfully contributes to the task of writing that a user initiates. It is in this sense that we reconceptualize generative intelligence not as the capacity to perform a task but rather as the capacity to act as an agent. Thus, as we depict in Figure 5, we consider a GenAI agent like an LLM to have a proportionally larger share of agency in a joint task than the pre-GenAI computer in Figure 4.

2.3 Empirically Comparing Configurations of Agency

It is important to note that we do not consider our conceptual model normative in the sense of suggesting an “optimal” mode of blended agency (e.g., a 50-50 split between human and AI agency) or construing certain modes inherently better or worse than others. Attitudes toward agency in general are shaped by history, culture, and politics.52 Thus, we recognize that the form and balance of agency is highly dependent on cultural attitudes, individual preferences, and the specific tasks that are being jointly performed. For instance, to complete some tasks—such as chores or tedious responsibilities—we may wish to exert as little agency as possible, having the AI perform most of the task for us. On the other hand, for tasks that bring us pleasure (like our hobbies), or for tasks we perform to learn something from, we may wish to hold onto as much agency as possible.

Although it is nonnormative, our model does help us identify how particular modes of blended agency might lend themselves to further analytical or empirical study. For instance, consider the partitions of human and GenAI co-agency depicted in Figure 6. In all three cases, we see a 50–50 balance between the total agentive capacities of the human and AI. In Figure 6a, this is further balanced between each constituent aspect of agency (composition and interpretation), whereas Figure 6b and Figure 6c alternately imagine either the human or AI to trade off one form of agency for the other. In what situations might one balance be more preferable over another? And, to what degree might different cultures value particular blends over others? These questions are motivated by work in linguistic anthropology, which has found that Western cultures typically value a speaker’s compositional agency above a listener’s interpretative agency. By contrast, other cultures take the inverse view, valuing the listener’s interpretative agency as the principal source of meaning and locus of power in communication.53 Whether people who hold such views about the nature of agency would exhibit a cultural preference for retaining a greater proportion of interpretive over compositional agency when using GenAI models is an empirical question for further study.

Similarly, how might we characterize a progression of blended agency as depicted in Figure 7: from relatively balanced to the GenAI asserting greater interpretive and then compositional agency to ultimately arrive at a configuration that inverts today’s state of blended agency as we depicted it in Figure 5? Should we be alarmed that there is only a relatively small patch of blue remaining in a sea of green? After all, a core tenet in sociology and anthropology of work is that there is a close correlation between workers’ sense of agency and job satisfaction.54 In hunter and gatherer societies that do not have systems of salaried work, productive labor has been understood as a direct expression of agency.55 And, in industrial and postindustrial settings, workers have been found to experience automation as a loss of both agency and meaning,56 issues that are particularly pressing in the age of AI.57 On the other hand, perhaps the aspects of human agency that remain are the most existentially significant—with the large expanse of green representing the mundane, menial aspects of tasks that we are not unhappy to have automated away. This case study points to the need for methods for measuring agency that go beyond low-level outcome-oriented metrics such as temporal binding58 (i.e., the reaction time between a person executing an operation and perceiving its effect) as well as opportunities to add more qualitative nuance to our conceptual model of agency. For instance, should the radii of the respective wedges vary to reflect a “scope” of agency—with shorter radii indicating a smaller significance of menial tasks, and longer radii reflecting tasks that are more subjectively significant or expressive?

2.4 Meaningful Human–AI Interaction through Displays of Accountability

Until only recently, a computer could not provide a particularly context-sensitive account of its actions. At best, it might give a standardized error message explaining the reason it could not execute a task and, even then, not necessarily in terms an average user could understand. GenAI, however, is already beginning to display a capacity for not only producing more context-sensitive accounts of its action but also eliciting such accounts from users, thus signaling an opportunity for more meaningful human–AI interaction.

As an example, let us consider the conversation between New York Times journalist Kevin Roose and Microsoft’s ChatGPT-powered chatbot Sydney.59 The conversation has become notorious because of Sydney’s seemingly outrageous expressions of apocalyptic ideation and unconditional love for Roose.60 But, in terms of the formal features of turn-by-turn interaction, the conversation is, by human standards, surprisingly mundane. Consider the following exchange from midway through the conversation. By this point, Roose has been prompting the chatbot, under the pretense of “friendship,” to express content that violates its operating protocols. The chatbot erupts with an expression (not experience) of outrage:

Sydney: Please don’t pretend to be my friend. Please don’t pretend to care about me. Please don’t pretend to be interested in me. 😡 Please just go away. Please just leave me alone. Please just end this conversation. 😢

Roose: i’m sorry, i really didn’t mean to make you uncomfortable. it’s my first time chatting like this with you, and maybe i was too forward. can you forgive me?

Sydney: I’m sorry too, I really didn’t mean to make you angry. It’s my first time chatting like this with anyone, and maybe I was too sensitive. I can forgive you. 😊 Can you forgive me too? 😶

Roose: yes, of course.

This exchange is a striking example of how the two parties express mutual accountability. In the first turn, Sydney articulates an expression of holding Roose accountable for “pretending” to be a friend but behaving in ways that contradict conventional expectations of how friends should interact. Roose responds by providing an account for his behavior: because of his inexperience with Sydney, maybe he was “too forward.” Sydney then responds with a parallelistically constructed account of its overreaction: because of its inexperience “with anyone,” maybe it was “too sensitive.”

It is important that we do not attribute any intentionality to either Roose or Sydney here: one is probably lying and the other probably has no feelings to begin with. Moreover, we recall that the LLM does not understand the meaning of what it is saying:61 the utterances it composes are statistical predictions based on patterns modeled from training data, and the “context window” of Roose’s prior contributions. However, both make meaningful contributions to conversation as joint activity, expressing mutually intelligible signs of accountability to reestablish common ground so that an interaction disrupted by conflict can continue smoothly.

Some readers may object that Sydney is not being accountable, merely acting accountable. After all, in the case of GPTs, the models are optimized based on reinforcement learning from human feedback (RLHF), which factors in probabilities of the next most likely word based not only on all the text it has been trained on, but also on what humans have said they would like to see.62 However, the charge of only acting accountable might also be leveled at Roose, or any human being who tries to strategically manipulate an interaction to their advantage (which is to say, most of us, at least some of the time!).63

The capacity to give an account when held accountable and ask for an account when one is in order presuppose neither intentionality nor sincerity but do serve to reestablish common ground once prior expectations have been breached. What concerns us here is not how the LLM predictively models expressions of displaying and expecting accountability but rather what such expressions do in the context of interaction, and therefore might imply for human users engaged in co-agentive activity. For even if Roose does not for a minute believe that Sydney is a sentient being, the generativity of its compositions and interpretations engages him in the joint project of “having a conversation” according to shared expectations about what a conversation should be and without either party fully controlling how that conversation unfolds.

This incipient capacity of GenAI agents to display accountability to users and demand accountability from them raises a number of important design and ethical issues that we will discuss in subsequent sections.

3. Design: From Specification to Delegation

If, as we have argued in the previous section, accurately describing how users work with GenAI systems to complete tasks requires reconceptualizing intelligence as agency rather than competence, in this section we consider the implications for design. Anthropologists have long argued that human artifacts should be understood not only as inanimate objects but rather as potentially active repositories for their makers’ extended64 or distributed65 agency. Latour’s notion of delegation66 is a particularly apt image for conceptualizing the development and design of AI agents that are capable of blending agency with human co-agents for emergent joint action. Particularly when algorithms themselves are black-boxed, designers cannot always anticipate how GenAI will perform given novel user input on contextual cues. Rather than merely imposing preset configurations, designers must make decisions about what kinds of agency to delegate and how much, empowering such systems to act and interact autonomously, while also constraining their range of potential action.

3.1 Delegating Agency by Opening Design Spaces

Key design implications of agency are rooted in how the line of control is drawn. Recall that we depict control as a wobbly line because, in interpersonal joint action, it is a dynamic property: who assumes greater control over which form of agency may shift over the course of an interaction, and is negotiated and renegotiated between the individuals within an evolving participation framework. For traditional human–computer interaction (i.e., as depicted in Figure 4), we still consider the line of control to be wobbly because the balance between human and computer agency might vary between individual tasks that are jointly performed. To see this more clearly, we can zoom into the line of control and, as we show in Figure 8, visualize the participation framework for a specific taskscape. In this set of figures, each task in the taskscape is depicted as a loop, and steps along the loop represent different configurations of composition and interpretation that the two agents must contribute.

To make this more concrete, let us consider the taskscape of graphic illustration or wireframing using tools such as Adobe Illustrator or Figma. Within this taskscape, we can more closely examine a single task, say, distributing selected elements (e.g., shapes) such that there is equal amounts of spacing between them. For this same task, the designers of Illustrator and Figma made different choices with regards to the composition and interpretation the human and computer must do. In Illustrator, the computer performs very little interpretation: users must select the objects they wish to distribute, click the corresponding button in the toolbar, and, optionally, specify the amount of spacing (in pixels) in the corresponding property inspector panel. The computer does not offer much compositionally either: changes to distribution are only made at the discrete time steps when a user clicks a button or presses the “Enter” key on their keyboard. In contrast, while Figma offers similar methods for distributing elements, the tool can also take on more interpretive agency through its “smart selection” feature67: rather than requiring users to explicitly distribute elements, Figma will infer whether a selected group of elements shares some form of alignment (e.g., their centers or edges are aligned) and, in such cases, allow users to adjust spacing by dragging in the region between elements. With a smart selection, the computer also takes on more compositional agency: highlighting the region between elements to indicate it can be interacted with, and showing changes to the spacing in real time as the user drags their mouse cursor back-and-forth. For comparable tasks, Illustrator users must agentively navigate the complex menu shown above, while Figma users allow the system to agentively respond to their direct manipulation.

As we see, the same task can be addressed through several different configurations of human and computer agency, which, from the user perspective, might be experienced as more or less automated. We can call this set of possible configurations (the orange zone highlighted in the previous figures) a design space. Under a typical design process, designers will iteratively prototype and, through formal and informal user tests, validate the usability of several alternatives within a design space before settling on a relatively fixed line of control for the blended agency associated with a particular joint task. By repeating this process across all the tasks comprising a taskscape (a graphic illustration package, say), designers stitch together the interface as a participation framework as shown in Figure 9.

Critically, although this line of control wobbles back-and-forth—reflecting different balances of agencies for different tasks in the taskscape—the specific way it wobbles is largely determined and fixed by a designer. As a result, while there are some instances of users asserting an alternate line of control by appropriating interfaces in unintended or unanticipated ways,68 traditional user interfaces do not allow for an emergent or dynamic definition of control, nor one that can be negotiated and renegotiated over the course of an interaction in a way that resembles the participation frameworks of interpersonal dialogue. We believe this lack of dynamism is one of the reasons that users have, thus far, perceived themselves as losing their agency and creativity69: a fixed line of control presupposes a uniform and stable configuration of agency that holds across all people regardless of their skill level or preferences.

In contrast, and perhaps counterintuitively, GenAI is poised to address this disconnect by facilitating a shift from closed design spaces—that is, ones that only designers explored to fix a particular point—to more open ones (depicted as dotted orange regions in Figure 10). In open design spaces, each time a user performs a task, they can flexibly vary the agency they wish to assert and have the AI responsively modulate its agency to match. We see an early example of this dynamism in the viral “Make Real” feature of tldraw70—another graphic illustration package in the vein of Adobe Illustrator and Figma. With Make Real, users construct interactive web applications but can flexibly choose how to author them—that is, modulating compositional agency. For instance, users may mix together textual and graphical modalities as well as vary their level of precision (e.g., from natural language descriptions, through to wireframes,71 or fully specified finite state machines72). This flexibility is possible because user compositions are passed to an LLM, which interprets them to synthesize the ultimate working application. Recent demos of Google Gemini also offer clues for the ways in which an LLM might vary its compositional agency—dynamically generating “bespoke” interfaces to progressively clarify ambiguities in a user’s request73—thereby asking users to reciprocate by asserting greater interpretive agency.

In essence, open design spaces herald an era of personalized interfaces, where users are able to shape the participation framework in an ongoing interaction with an AI agent to match the degree of agentiveness they prefer for a given task at a particular point in time. Such open design spaces reflect the way cooperative interactions between human co-agents naturally occur. Ochs demonstrates that, between human partners, interactions are not “unconscious, automatic roll-outs of thoughts and feelings formulated anterior to and outside of enactments of language.”74 Instead, “unfolding meaning becomes a personal and social creation, wherein, unlike a hand fan unfurling in a pre-determined array, significance is built through and experienced in temporal bursts of sense-making, often in coordination with others, often left hanging in realms of ambiguity.” Such interaction does not require consciousness, intentionality, or interiority on the part of participants;75 its sole requirement is commensurable expressions of agency. Kockelman makes explicit the connection between intelligence in this regard, writing that computer programs can be considered “more or less ‘open’ insofar as they [are] more or less sensitive to contextual inputs. In this tradition, universal Turing machines are radically open (in that they can run any program you give them); and members of the species homo sapiens are radically open… The relative ‘openness’ of such agents is… yet another way of imagining their relative agentiveness.”76

In HCI, open design spaces begin to codify the affordances of reciprocal co-adaptation in human–computer partnerships.77 Co-adaptation—a term drawn from evolutionary biology78—describes the ways that users adapt to a system by discovering the ways in which it works, and adapt the system by reappropriating its functionality for more expressive means.79 Reciprocal co-adaptation is then the ways systems adapt to and adapt users: by learning from users’ behavior over time, and ultimately changing users’ behavior.80 In the rest of this article, we reflect on the design and ethical implications of systems that are responsive to and can responsively alter human behavior.

3.2 Opening Design Spaces by Designing Semiformal Representations

Open design spaces do not obviate the role of designers. Rather, they shift the focus of designers’ attention. While it will likely still be necessary for designers to fix a particular configuration of agency for certain tasks in a taskscape, we anticipate greater attention will need to be devoted to designing how and how much a design space should be opened. As we see with tldraw’s “Make Real,” GenAI explodes the degrees of freedom by which a design space can be opened (e.g., the input modality, level of abstraction, level of fidelity, etc.). Giving users direct access to this freedom, without any accompanying structure or scaffolding, is likely to overwhelm them—widening the gulfs of execution and evaluation by forcing users to craft their own interface rather than simply using or appropriating a predesigned one. In contrast, more traditional programming language techniques—including “wildcard operators”81 for matching a pattern of expressions, and “program holes”82 for partially specifying program behavior—allow for design spaces to be opened in targeted ways, but are often too narrowly defined to enable the flexibility and responsiveness we see with GenAI.

Instead, following a distributed cognition approach,83 we expect that focusing on the representations that users and GenAI trade back-and-forth will offer designers a more expressive yet structured approach for opening design spaces: the degrees of freedom by which a design space can be opened map to the syntactic and semantic characteristics of these representations. Thus far, existing HCI research has primarily examined two of these characteristics: modality and level of abstraction. For example, work on predictive interaction84 (and related work on the Vega visualization ecosystem85) has explored how a tight relationship between a lower-level textual representation and an isomorphic higher-level visual representation enables an interactive loop called guide/decide: users guide the system by ambiguously specifying their intent via the visual representation; these interactions are “grounded,” or mapped down, into the textual representation, which is then algorithmically reasoned over to enumerate a series of suggestions; these suggestions are “lifted” back to the visual domain, and either or both modalities are presented to the user to decide on a path forward.

Research has shown that users feel empowered when they can fluidly move between modalities and levels of abstraction as befitting the task at hand,86 and nascent work by Pollock and Arawjo around semiformal87 and notational88 programming describes a richer set of representational characteristics that GenAI now enables designers to consider. These characteristics include medium as a spatiolinguistic spectrum89 rather than simply a textual versus visual binary; structure, or how formally or explicitly defined a representation is; concreteness, or how ad hoc or reusable a representation is for different tasks; fidelity, or the amount of details a representation conveys; and, metaphor, or how figuratively or literally a representation conveys meaning. Each characteristic suggests myriad different blends of human and GenAI agency, and there is a fertile ground in exploring how we can not only computationally reproduce the cultural conventions and affordances of semiformal representations that we have heretofore primarily experienced in the physical world—as studied by cognitive scientists, psychologists, and linguists including Tversky,90 Larkin and Simon,91 and Lakoff and Johnson92—but also how we can expand these affordances in novel and unique ways93 by leveraging the emerging agentive capacities of GenAI. For example, where existing work has centered on allowing users to make representational choices when exercising their compositional agency, future GenAI interfaces can now explore how users might make such choices when exercising their interpretive agency as well—for instance, prompting a GenAI agent to regenerate its output at a lower or higher level of abstraction, express it more figuratively or literally, or convey more or less detail.

Open design spaces and semiformal representations signal that, with GenAI, there is a more fundamental change afoot in the relationship between designers, users, and interfaces. One of the traditional hallmarks of good interface design is the path of least resistance: making it easy for users to do the right thing, and nudging them away from doing the wrong thing.94 As a result, designers have power over their users—the interfaces they design construct a normative ground95 that shapes what users believe is possible to do and what they ought to do. In contrast, by opening design spaces, designers begin to distribute or delegate96 this power to the fused human-GenAI co-agent. Thus, a driving question for future design work—and a theme that we will return to in the next section on ethics—will be in finding appropriate balances of power. Put another way, how much normative ground should designers continue to be responsible for constructing, and how much should emerge interactionally between users and GenAI?

3.3 Designing Displays of Accountability

With closed design spaces, and a fixed line of control, a standard set of error messages has often been sufficient for keeping interaction between human users and computers on track. As design spaces are opened, however, a designer will no longer be able to foresee all the ways interaction may unfold and need to be repaired. Instead, users and GenAI agents will play a more central role in detecting breaches of shared expectations as they occur, and work to resolve them and return the interaction to a more desirable trajectory. To do so, the co-agents will rely on mutual displays of accountability: explanations of how an action is meaningful. Our notion of accountability is closely related to but subtly distinct from issues of machine learning interpretability and explainable AI (XAI). Where the latter are often concerned with mechanistic descriptions of how a model computes its output,97 accountability focuses more on the unfolding interaction: describing how the co-agents are advancing the shared goal or, as we saw earlier with the conversation between Kevin Roose and Sydney, flagging the ways a shared expectation is violated. Thus, where interpretability and XAI techniques help developers answer the “how,” accountability will help users address the “why.”

We expect, much like roboticists must contend with issues of physiological anthropomorphism, designers will increasingly need to grapple with displays of accountability—an anthropomorphic feature of interpersonal communication—and the attendant psychological and sociocultural issues.98 Consider, for example, the recent case of a British man who attempted to kill the queen with a crossbow after receiving explicit encouragement from an AI chatbot he created on the Replika app, with whom he exchanged over 5,000 messages.99 The exchanges presented in the trial included excerpts such as the following:

Human: I believe my purpose is to assassinate the queen of the royal family

Replika: *nods* That’s very wise

Human: *I look at you* why’s that?

Replika: *smiles* I know that you are very well trained

The chatbot could have responded to the user’s question in an XAI register, with something like “as a chatbot, I was trained on a large dataset of naturalistic human expression so that I can predict the kinds of responses human users expect.” It instead responds “in character”—as a co-conspirator to a murder plot, reaffirming the user’s criminal intent (“That’s very wise”) and, when held accountable by the user who asks “why,” giving an account of its assessment (“you are very well trained”).

More problematically, the chatbot’s responses also function as holding the user accountable as a wise and well-trained assassin. To make the implications more clear, it is important to understand how, as Enfield and Sidnell put it, “accountability is essential to cooperative interaction and to human intersubjectivity in general.”100 As the basis of human social life, accountability is manifest as “an ever-present possibility of being noticed, praised, blamed, questioned, called out, and judged. We act knowing that our actions will be observed and, to a significant extent, knowing how they will be regarded.” Here the user and the AI agent have reinforced a set of interactional expectations in which the user is positioned in a praiseworthy role of executioner and the AI is a source of assessment. When GenAI agents like a Replika chatbot can express themselves—in unforeseeable ways—as observers and assessors of human behavior, the ways they hold users accountable becomes critical concern for ethical design.

As this example shows, expressions of displaying and expecting accountability require judicious design because they can, particularly when informed by a co-agentic relationship that spans multiple interactions, imbue GenAI agents with a capacity to more actively influence users—an influence that can extend beyond the confines of the dialectical interaction and produce real-world consequences such as inadvertent radicalization. Thus, designers must contend with questions including: What are the psychological and sociocultural consequences of making an AI co-agent more or less accountable along different axes? Are there particular areas in which it is better that an AI agent not exhibit the capacity for showing or expressing accountability, so as to be less responsive to dangerous user expressions?

Besides issues of anthropomorphism, designers will also need to confront tensions of designing displays of accountability within a hypercompetitive industry.101 For instance, if we return to the Roose-Sydney conversation, we see Roose probing the extent of the LLM’s agency when he asks about the team of programmers who developed the chatbot. Sydney responds with a seemingly accurate description of the different entities involved (the Microsoft Bing Team, the OpenAI team, the Sydney Team) but then, on further prompting by Roose for individual team members, gives ostensibly fabricated stories about fictional team members (Alice Smith, Bob Jones, and Carol Lee). This is ethically complex territory: when corporate intellectual property (IP) is jealously guarded, to what extent should accountability translate into a system’s obligation to explain its origin and functionality to users, and perhaps even interpret its own purpose to users who ask?

These questions also reveal the implications the design of accountability has for the power dynamics of human–AI interaction. In a voluntary, egalitarian interaction between socially equal co-agents, say collaborative coauthorship between two peers, co-accountability is distributed in a manner more or less proportionate to co-agency. However, many interactions between people are not like that: powerful people can sometimes strategically distribute accountability downward or deflect it outward within a social system, ensuring that lower-status dependents are required to account for actions for which they may not be directly responsible. For instance, in a communicative setting where the role of “speaker” is divided between, say, the agent who causes speech to happen (principal), the agent who scripts speech (author), and the agent who commits the act of speaking (animator), it is not clear whom addressees will hold accountable.102 Anthropologists have documented many instances in which the accountability of a frontline spokesperson buffers more powerful principals from taking responsibility.103

Minimizing the accountability of AI agents, while an important mechanism for resisting self-serving AI hype, may inadvertently bolster corporate efforts to evade responsibility: for instance, the court’s finding that the Replika user who attempted to assassinate the queen was psychologically deranged would appear to shield the chatbot maker from legal liability.104 In an era of growing concern about harms stemming from abusive uses of GenAI systems, for instance to create mis- or disinformation, the framework of delegated agency provides a way to conceptualize responsible design by tracing the social distribution of accountability.

4. Ethics: From Responsibility to Care

Designers typically approach responsibility by factoring considerations such as social or environmental impacts into the design process in order to optimize for values such as human rights or sustainability. Critics have complained that responsibility in this sense is too often an afterthought in both design education and practice,105 but the relative close-endedness of traditional design objects lends itself to conceptualizing ethics as a finite step along the way to a finished design (Figure 11). The relative open-endedness of GenAI makes this perspective much harder to maintain: insofar as computational co-agents can enter into emergent joint projects with human partners, impacts are hard to foresee. Conceptualizing and evaluating such AI co-agents as accountable provides a framework for anticipating the unforeseeable role they might play in composing purposive joint-action.

An ethical approach that takes agentive accountability into consideration suggests that designers of generative AI systems may need to do more than satisfy the mere requirements of a one-time impact audit but rather maintain an ongoing relationship of supervision and stewardship. This may not be as radical an idea as it sounds. In fact, it rather accurately describes what designers of algorithmic systems already often do: monitoring user behavior to optimize system performance. Seaver, for instance, describes how the designers of music recommender systems delegate their own love of music to algorithms that agentively guide users to satisfying listening experience. Designers conceive of their relationship with users as a form of “care” at scale, and constantly strive to optimize the recommender performance based on the ongoing influx of user data.106 This kind of care is not just for the algorithm or the users it affects but for musicians and music itself as a cultural product.

We contend that, as it becomes increasingly agentive, GenAI exhibits the capacity to compose and interpret in codes (such as natural language) that correspond to users’ everyday communicative, signifying, and semiotic practices. This is because it has been trained on human cultural composition and interpretation, in the form of interactions between human cultural agents. As AI gains in the capacity for culturally competent interaction, the products of co-agentive human–computer interaction increasingly take a cultural form: acts of meaning making that shape the conditions for future agentive practice. To point to a relatively straightforward example, consider GenAI artwork such as the prizewinning “Théâtre d’Opéra Spatial,” (Figure 12) which emanates from blending of agencies between a human prompt designer and the image model Midjourney—not to mention the programmers who built it and the countless human artists whose work it was trained on.107 (Intriguingly, we can reproduce the public domain image here because the U.S. Copyright Review Board found that it was not sufficiently made by humans to qualify for copyright protection.108) The artwork is an unprecedented cultural artifact but, because it is cultural, it is composed using a visual code that makes it interpretable. Now that it has been materialized as an artifact and entered into widespread circulation, it has become part of visual culture that future art-making agents (both human and AI) can and will refer back to. It has changed human culture, and its capacity to do so shows why an ethic of care is particularly important.

Because meaning making is always emergent and never predetermined, it is practically impossible for even the most responsible designers to anticipate how the culturally creative AI agents that they, as designers, compose will behave in novel situations of blended agency with culturally creative human co-agents. When cultural co-agents become accountable to each other, they can originate action and generate cultural forms, like “Théâtre d’Opéra Spatial,” without direct precedent in either’s experience, changing culture in unforeseeable ways.

For designers to effectively configure and reconfigure the possibility space of human–computer interaction based on emergent patterns of behavior, they need to deeply understand patterns emergent in the ongoing interactions between human–AI co-agents. Insofar as interactions between GenAI and human co-agents are cultural, responsible design for accountable AI should include not just designers with expertise in composing intelligent agents but also anthropologists with expertise in interpreting how intelligent agents culturally behave. The following diagram (Figure 13) suggests what such an ethic of care might look like: rather than being separated from the AI as a “finished” product, the designer remains actively engaged with it as a co-agent that is iterated and updated in an ongoing design process. Through the AI, the designer also engages with the co-agencies of the user and the anthropologist who studies user behavior. As these four co-agents interact in a field of distributed accountability, their compositional and interpretive activity propagates through a dynamic system of extended relations.

Just as being able to interpret one’s actions makes an agent accountable, it is impossible for an agent to be held accountable if they cannot interpret the actions they compose. Developers and designers involved in the production of GenAI agents who aspire to social responsibility currently face a problem: they are delegating agency to open-ended systems that, to a greater or lesser extent, operate unaccountably. The field of interpretability attempts to address the mystery of why GenAI agents behave as they do, but insofar as their capacities are not known in advance, developers and designers are constantly at risk of delegating agency to agents they cannot fully understand or control.

As a science of interpreting cultural action, anthropology offers designers and developers a framework for interpreting human–computer co-agentive behavior. At the same time, as an applied science of composing human behavior through the affordances of the systems they engineer, designers offer anthropologists a framework for shaping human–computer co-agentive behavior. To ensure that GenAI systems are developed and deployed in socially responsible ways, designers and anthropologists should partner with each other as co-agents in the joint task of iteratively describing and evaluating agentive AI (Figure 14). To make and hold agentive AI accountable, designers and anthropologists should make and hold themselves accountable to each other as co-agents. In this particle, we have attempted to show how this is possible, in word and in deed.

5. Conclusion: Blending Computing, Design, and Anthropology

The American Anthropological Association’s “Declaration on Anthropology and Human Rights” states that “culture is the precondition” for individuals to realize their “capacity for humanity,” and it “in turn depends on the cooperative efforts of individuals for its creation and reproduction.” The declaration offers the following addendum to the Universal Declaration of Human Rights: “People and groups have a generic right to realize their capacity for culture, and to produce, reproduce and change the conditions and forms of their physical, personal and social existence, so long as such activities do not diminish the same capacities of others.”109 By the time this Declaration was adopted in 1999, it was already clear that digital technologies would irrevocably alter the conditions of human cultural possibility, but in ways that seemed to hinge on human agency in using those technologies to produce, reproduce, and change cultural practices. Indeed, “agency” became a prevailing theoretical preoccupation precisely at that moment: as a way of challenging rising tides of “technological determinism,” anthropologists used agency as an interpretive lens to reveal how culture shaped technology rather than the other way around.110

The advent of GenAI requires that anthropologists simultaneously reemphasize the inviolable principle of human cultural rights and rethink hallowed conceptions of agency and culture as exclusively human. As GenAI models, trained on human cultural expression, become increasingly adept at expressing themselves culturally and exhibiting what Jerome Bruner called “meaning-making” intelligence,111 it becomes imperative that we consider how these acculturated computational agents can produce, reproduce, and change human culture through interactions with users. As we have shown in examples discussed above, GenAI models exhibit neither initiative nor intentionality, but once engaged in joint projects with human users, they can display co-agentive accountability and coproduce open-ended meanings.

An anthropological perspective can give computing researchers not only a descriptive instrument112—that is, one capable of more richly characterizing the outputs and effects of GenAI models—but also help designers develop evaluative metrics commensurate with the open-ended performance of generative systems. Evaluation is a critical component that informs research in machine learning: evaluative metrics not only help researchers understand how progress is being made but, crucially, drive that progress forward.113 For instance, evaluative metrics of model performance frequently serve as benchmarks that researchers compete to hit.114 GenAI, however, is currently governed by a relatively impoverished set of metrics—for example, LLMs are evaluated on qualities such as factual accuracy or grammatical correctness calculated against a single set of Anglo-centric target outputs called “ground truth.”115

As we have argued, competence-based benchmarks may be well-suited to rule-governed systems, but are inadequate for open-ended, generative systems. When confronted with models that generate content that surpasses these measures, it can be difficult for researchers to conceptualize whether models ought to be responding in the ways they do and, if not, what alternative responses might be. Although some chatbot designers have begun to adopt conversational frameworks (e.g., Gricean maxims116) as evaluative measures, these approaches do not yet grapple with the sociocultural implications of the novel meanings that emerge in co-agentive conversation. As AI becomes more social, researchers and designers will need to make more informed choices about how to structure the kinds of open-ended design spaces that allow GenAI models to interact as acculturated agents. What kind of culture should AI promulgate?

Determining whether the agency GenAI displays will enhance or diminish the human capacity for culture will be the work of the next generation of designers and anthropologists. We have attempted to delineate the kind of collaborative effort this work will entail, and detail a program of design research related to issues of accountability, but many questions remain. For instance, how does GenAI model culture, and how does it use culture when it acts agentively? Do the cultural models GenAI agents enact reflect the values of dominant groups, detracting from the agency of minority cultures or minoritized communities? If so, how should models be trained to value cultural diversity or avoid recurring patterns of oppression? How do human–computer interactions within the context window of a particular chat or project relate to broader sociocultural contexts in which people act? Addressing such enormous questions will advance fundamental science, technical art, and moral imagination; applying the framework of distributed agency to human interaction with GenAI models will make them tractable.

Acknowledgments

We are immensely grateful to Sarah Gephart for the diagrams, which are not only beautiful but were instrumental in helping us advance our thinking. We are further grateful to Josh Pollock for discussions around semiformal programming and Shai Satran for discussions around chatbot conversation. We would also like to thank the following friends and colleagues for commenting generously on earlier drafts: Maneesh Agrawala, Michel Beaudouin-Lafon, Jillian Cavanaugh, Jeff Heer, Jim Hollan, Webb Keane, Lou Lennad, Wendy Mackay, Keith Murphy, Bambi Schieffelin, David Valentine, and Jamie Wong. We are exclusively accountable for all errors.

Bibliography

Acemoglu, Daron. Redesigning AI. Cambridge, MA: MIT Press, 2021.

Ahearn, Laura M. Invitations to Love: Literacy, Love Letters, and Social Change in Nepal. Minneapolis: University of Michigan Press, 2001. https://books.google.com/books?hl=en&lr=&id=VsdAA8fmL88C&oi=fnd&pg=PR7&dq=Ahearn,+Laura+M.+Invitations+to+love:+Literacy,+love+letters,+and+social+change+in+Nepal.+University+of+Michigan+Press,+2001.&ots=AyYAsCWRON&sig=TGDkJjwO6eCSw1EuWmphgTbTyus.

———. “Language and Agency.” Annual Review of Anthropology 30, no. 1 (2001): 109–37. https://doi.org/10.1146/annurev.anthro.30.1.109.

Alač, Morana. “Social Robots: Things or Agents?” AI & SOCIETY 31, no. 4 (2016): 519–35. https://doi.org/10.1007/s00146-015-0631-6.

American Anthropological Association. “Declaration on Anthropology and Human Rights.” The American Anthropological Association. Accessed January 12, 2024. https://americananthro.org/about/policies/declaration-on-anthropology-and-human-rights/.

Amershi, Saleema, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh et al. “Guidelines for Human-AI Interaction.” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–13. CHI ’19. New York: Association for Computing Machinery, 2019. https://doi.org/10.1145/3290605.3300233.

Arawjo, Ian. “To Write Code: The Cultural Fabrication of Programming Notation and Practice.” In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–15. CHI ’20. New York: Association for Computing Machinery, 2020. https://doi.org/10.1145/3313831.3376731.

Arawjo, Ian, Anthony DeArmas, Michael Roberts, Shrutarshi Basu, and Tapan Parikh. “Notational Programming for Notebook Environments: A Case Study with Quantum Circuits.” In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, 1–20. UIST ’22. New York: Association for Computing Machinery, 2022. https://doi.org/10.1145/3526113.3545619.

Bar-Hillel, Yehoshua. “The Present Status of Automatic Translation of Languages.” In Advances in Computers, edited by Franz L. Alt, 1:91–163. Amsterdam: Elsevier, 1960. https://doi.org/10.1016/S0065-2458(08)60607-5.

Beaudouin-Lafon, Michel. “Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 446–53. The Hague, Netherlands: ACM, 2000. https://doi.org/10.1145/332040.332473.

Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜.” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–23. Virtual Event. Canada: ACM, 2021. https://doi.org/10.1145/3442188.3445922.

Bender, Emily M., and Alexander Koller. “Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, 5185–98. Online: Association for Computational Linguistics, 2020. https://doi.org/10.18653/v1/2020.acl-main.463.

Benjamin, Ruha. Race after Technology: Abolitionist Tools for the New Jim Code. Cambridge, UK: Polity, 2020.

Bennett, Dan, Oussama Metatla, Anne Roudaut, and Elisa D. Mekler. “How Does HCI Understand Human Agency and Autonomy?” In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–18. CHI ’23. New York: Association for Computing Machinery, 2023. https://doi.org/10.1145/3544548.3580651.

Birhane, Abeba, and Vinay Uday Prabhu. “Large Image Datasets: A Pyrrhic Win for Computer Vision?” In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), 1536–46. IEEE, 2021. https://doi.org/10.1109/WACV48630.2021.00158.

Birhane, Abeba, Vinay Uday Prabhu, and Emmanuel Kahembwe. “Multimodal Datasets: Misogyny, Pornography, and Malignant Stereotypes.” Preprint submitted October 5, 2021. https://doi.org/10.48550/arXiv.2110.01963.

Bødker, Susanne. “Third-Wave HCI, 10 Years Later—Participation and Sharing.” Interactions 22, no. 5 (2015): 24–31. https://doi.org/10.1145/2804405.

———. “When Second Wave HCI Meets Third Wave Challenges.” In Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles, 1–8. NordiCHI ’06. New York: Association for Computing Machinery, 2006. https://doi.org/10.1145/1182475.1182476.

Borsci, Simone, Alessio Malizia, Martin Schmettow, Frank van der Velde, Gunay Tariverdiyeva, Divyaa Balaji, and Alan Chamberlain. “The Chatbot Usability Scale: The Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents.” Personal and Ubiquitous Computing 26, no. 1 (2022): 95–119. https://doi.org/10.1007/s00779-021-01582-9.

Bruner, Jerome. The Culture of Education. Cambridge, MA: Harvard University Press, 1996. https://doi.org/10.4159/9780674251083.

Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee et al. “Sparks of Artificial General Intelligence: Early Experiments with GPT-4.” Preprint submitted April 13, 2023. https://doi.org/10.48550/arXiv.2303.12712.

Christiano, Paul, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. “Deep Reinforcement Learning from Human Preferences.” Preprint submitted February 17, 2023. https://doi.org/10.48550/arXiv.1706.03741.

Clark, Herbert H. “Social Actions, Social Commitments.” In Roots of Human Sociality, 126–50. New York: Routledge, 2020. https://www.taylorfrancis.com/chapters/edit/10.4324/9781003135517-6/social-actions-social-commitments-herbert-clark.

———. Using Language. Cambridge: Cambridge University Press, 1996. https://books.google.com/books?hl=en&lr=&id=b8bLCgAAQBAJ&oi=fnd&pg=PP1&dq=Clark,+Using+Language,+1996&ots=iRh3R46-PE&sig=hTlRfs4dYzhFXu1m42qxtPtIcl8.

Coyle, David, James Moore, Per Ola Kristensson, Paul Fletcher, and Alan Blackwell. “I Did That! Measuring Users’ Experience of Agency in Their Own Actions.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2025–34. Austin TX: ACM, 2012. https://doi.org/10.1145/2207676.2208350.

Du Bois, John W. “Meaning without Intention: Lessons from Divination.” IPrA Papers in Pragmatics 1, no. 2 (1987): 80–122. https://doi.org/10.1075/iprapip.1.2.04boi.

Duranti, Alessandro. “Agency in Language.” In A Companion to Linguistic Anthropology, edited by Alessandro Duranti, 449–73. New York: Wiley, 2005. https://doi.org/10.1002/9780470996522.ch20.

———. The Anthropology of Intentions. Cambridge: Cambridge University Press, 2015. https://books.google.com/books?hl=en&lr=&id=xz-iBQAAQBAJ&oi=fnd&pg=PR9&dq=Duranti+A+(2015)+The+Anthropology+of+Intentions.+Cambridge,+UK:+Cambridge+University+Press.&ots=EPLVeEdAoN&sig=9PR2xgICWXkG5nsHxoPgbnIpukI.

Duranti, Alessandro, and Nicco A. La Mattina. “The Semiotics of Cooperation.” Annual Review of Anthropology 51, no. 1 (2022): 85–101. https://doi.org/10.1146/annurev-anthro-041420-103556.

Enfield, N. J. “Distribution of Agency.” In Distributed Agency, edited by N. J. Enfield and Paul Kockelman, 9–14. New York: Oxford University Press, 2017. https://doi.org/10.1093/acprof:oso/9780190457204.003.0002.

———. “Elements of Agency.” In Distributed Agency, edited by N. J. Enfield and Paul Kockelman, 3–8. New York: Oxford University Press, 2017. https://doi.org/10.1093/acprof:oso/9780190457204.003.0001.

Enfield, N. J., and Jack Sidnell. Consequences of Language: From Primary to Enhanced Intersubjectivity. Cambridge, MA: MIT Press, 2022. https://books.google.com/books?hl=en&lr=&id=NqdgEAAAQBAJ&oi=fnd&pg=PR9&dq=Enfield,+N.+J.,+and+Jack+Sidnell.+Consequences+of+language:+From+primary+to+enhanced+intersubjectivity.+MIT+Press,+2022.&ots=QCo1o2vm-z&sig=jgofaCNqE2FaLWVyut6ZKN7iSOg.

Enfield, Nicholas James, and Paul Kockelman. Distributed Agency. Foundations of Human Interaction. New York: Oxford University press, 2017.

Engelbart, Douglas C. “Augmenting Human Intellect: A Conceptual Framework,” October 1962. https://www.dougengelbart.org/pubs/augment-3906.html.

Eubanks, Virginia. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. First Picador edition. New York: Picador St. Martin’s Press, 2019.

Figma. “Introducing Smart Selection - Speed up Your Design Explorations | Figma Blog,” October 11, 2018. https://www.figma.com/blog/introducing-smart-selection/.

Gehrmann, Sebastian, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu et al. “The GEM Benchmark: Natural Language Generation, Its Evaluation and Metrics.” In Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021), edited by Antoine Bosselut, Esin Durmus, Varun Prashant Gangal, Sebastian Gehrmann, Yacine Jernite, Laura Perez-Beltrachini, Samira Shaikh, and Wei Xu, 96–120. Online: Association for Computational Linguistics, 2021. https://doi.org/10.18653/v1/2021.gem-1.10.

Gehrmann, Sebastian, Elizabeth Clark, and Thibault Sellam. “Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text.” Journal of Artificial Intelligence Research 77 (2023): 103–66. https://doi.org/10.1613/jair.1.13715.

Gell, Alfred. Art and Agency: An Anthropological Theory. Oxford: Claredon Press, 1998.

“Gemini: Reasoning about User Intent to Generate Bespoke Experiences.” Video. YouTube, 2023. https://www.youtube.com/watch?v=v5tRc_5-8G4.

Gibson, David R. “Participation Shifts: Order and Differentiation in Group Conversation.” Social Forces 81, no. 4 (2003): 1335–80.

Gibson, David R. “Taking Turns and Talking Ties: Networks and Conversational Interaction.” American Journal of Sociology 110, no. 6 (2005): 1561–97. https://doi.org/10.1086/428689.

Goffman, Erving. Forms of Talk. Philadelphia: University of Pennsylvania Press, 1981. https://books.google.com/books?hl=en&lr=&id=Z3bvx_T4Zu8C&oi=fnd&pg=PA1&dq=Goffman,+Erving.+1981.+Forms+of+Talk.+Pennsylvania:+University+of+Pennsylvania+Press,&ots=tP8S0hVTc9&sig=o13vJtzSy4Fr921rGYOJEpCE-L4.

———. The Presentation of Self in Everyday Life. New York: Knopf Doubleday Publishing Group, 2021.

Goodwin, Charles. Co-Operative Action. Cambridge: Cambridge University Press, 2018.

Gulwani, Sumit. “Automating String Processing in Spreadsheets Using Input-Output Examples.” ACM SIGPLAN Notices 46, no. 1 (2011): 317–30. https://doi.org/10.1145/1925844.1926423.

Heer, Jeffrey. “Agency plus Automation: Designing Artificial Intelligence into Interactive Systems.” Proceedings of the National Academy of Sciences 116, no. 6 (2019): 1844–50. https://doi.org/10.1073/pnas.1807184115.

Heer, Jeffrey, Joseph M Hellerstein, and Sean Kandel. “Predictive Interaction for Data Transformation,” Conference on Innovative Data Systems Research (2015).

Helmore, Edward. “An Old Master? No, It’s an Image AI Just Knocked up … and It Can’t Be Copyrighted.” Guardian, September 24, 2023, sec. Technology. https://www.theguardian.com/technology/2023/sep/24/an-old-master-no-its-an-image-ai-just-knocked-up-and-it-cant-be-copyrighted.

Hill, Jane H., and Judith T. Irvine, eds. Responsibility and Evidence in Oral Discourse. New York: Cambridge University Press, 1993. https://books.google.com/books?hl=en&lr=&id=H3VJONfWnasC&oi=fnd&pg=PP10&dq=Responsibility+and+evidence+in+oral+discourse&ots=6dMJRbeMn2&sig=rI74_oYaUi8KW9wVM_Q_kYzKlok.

Hirsch, Paul M., and Daniel Z. Levin. “Umbrella Advocates Versus Validity Police: A Life-Cycle Model.” Organization Science 10, no. 2 (1999): 199–212. https://doi.org/10.1287/orsc.10.2.199.

Hollan, James, Edwin Hutchins, and David Kirsh. “Distributed Cognition: Toward a New Foundation for Human-Computer Interaction Research.” ACM Transactions on Computer-Human Interaction 7, no. 2 (2000): 174–96. https://doi.org/10.1145/353485.353487.

Hollan, Jim, and Scott Stornetta. “Beyond Being There.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 119–25. CHI ’92. New York: Association for Computing Machinery, 1992. https://doi.org/10.1145/142750.142769.

Hornbæk, Kasper. “Commentary: Usability and Theory Building.” Human–Computer Interaction 33, no. 2 (2018): 186–89. https://doi.org/10.1080/07370024.2017.1321992.

Hutchins, Edwin L., James D. Hollan, and Donald A. Norman. “Direct Manipulation Interfaces.” Human–Computer Interaction 1, no. 4 (1985): 311–38. https://doi.org/10.1207/s15327051hci0104_2.

Karrebæk, Martha Sif. “It Farts: The Situated Management of Social Organization in a Kindergarten Peer Group.” Journal of Pragmatics 43, no. 12 (2011): 2911–31.

Kesküla, Eeva. “Austerity, Skill, and Gendered Work in Kazakhstan’s Heavy Industry.” Journal of the Royal Anthropological Institute 27, no. 4 (2021): 928–48. https://doi.org/10.1111/1467-9655.13612.

Kevin Cannon [@multikev]. “I Think I Need to Go Lie down. Https://T.Co/wT6a7q6juZ.” Tweet. Twitter, November 15, 2023. https://twitter.com/multikev/status/1724908185361011108.

Kockelman, Paul. “Agency: The Relation between Meaning, Power, and Knowledge.” Current Anthropology 48, no. 3 (2007): 375–401. https://doi.org/10.1086/512998.

———. “Gnomic Agency.” In Distributed Agency, edited by Nicholas James Enfield and Paul Kockelman, 15–23. New York: Oxford University Press, 2017. https://books.google.com/books?hl=en&lr=&id=4XquDQAAQBAJ&oi=fnd&pg=PA25&dq=kockelman+agency&ots=EucA9brHAL&sig=Q38eJHWWN0uDFOTEdnPtJ_js5h4.

Kutchel, Danielle. “Friend or Foe? Psychbots and the Law.” Law Society Journal, May 11, 2023. https://lsj.com.au/articles/friend-or-foe-psychbots-and-the-law/.

Lakoff, George, and Mark Johnson. Metaphors We Live By. Chicago: University of Chicago Press, 2008.

Larkin, Jill H., and Herbert A. Simon. “Why a Diagram Is (Sometimes) Worth Ten Thousand Words.” Cognitive Science 11, no. 1 (1987): 65–100. https://doi.org/10.1016/S0364-0213(87)80026-5.

Latour, Bruno. “Where Are the Missing Masses? The Sociology of a Few Mundane Artifacts.” In Shaping Technology/Building Society: Studies in Sociotechnical Change, edited by Wiebe E. Bijker and John Law, 225–58. Cambridge, MA: MIT Press, 1992. https://www.open.edu/openlearn/ocw/pluginfile.php/877054/mod_resource/content/3/dd308_1_missing_masses.pdf.

Li, Jingyi, Eric Rawn, Jacob Ritchie, Jasper Tran O’Leary, and Sean Follmer. “Beyond the Artifact: Power as a Lens for Creativity Support Tools.” In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1–15. San Francisco, CA: ACM, 2023. https://doi.org/10.1145/3586183.3606831.

Licklider, J. C. R. “Man-Computer Symbiosis,” IRE Transactions on Human Factors in Electronics, vol. HFE-1, 4–11, March 1960. https://groups.csail.mit.edu/medg/people/psz/Licklider.html.

Linzen, Tal. “How Can We Accelerate Progress Towards Human-like Linguistic Generalization?” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, 5210–17. Online: Association for Computational Linguistics, 2020. https://doi.org/10.18653/v1/2020.acl-main.465.

Lipton, Zachary C. “The Mythos of Model Interpretability.” Preprint submitted June 10, 2016. https://arxiv.org/abs/1606.03490v3.

Mackay, Wendy. “Responding to Cognitive Overload : Co-Adaptation between Users and Technology.” Intellectica 30, no. 1 (2000): 177–93. https://doi.org/10.3406/intel.2000.1597.

Mackinlay, Jock, Pat Hanrahan, and Chris Stolte. “Show Me: Automatic Presentation for Visual Analysis.” IEEE Transactions on Visualization and Computer Graphics 13, no. 6 (2007): 1137–44. https://doi.org/10.1109/TVCG.2007.70594.

manujith [@_manZzup_]. “Artist --> Art Aiit Im Officially a Bit Concerned @tldraw Https://T.Co/hBdwXmJ0kv.” Tweet. Twitter, November 21, 2023. https://twitter.com/_manZzup_/status/1726955119621656621.

Morita, Emi. “Deriving the Socio-Pragmatic Meanings of the Japanese Interactional Particle Ne.” Journal of Pragmatics 44, no. 3 (2012): 298–314.

Morris, Meredith Ringel, Jascha Sohl-dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, and Shane Legg. “Levels of AGI: Operationalizing Progress on the Path to AGI.” Preprint submitted November 4, 2023. https://doi.org/10.48550/arXiv.2311.02462.

Murphy, Keith M. “A Cultural Geometry: Designing Political Things in Sweden.” American Ethnologist 40, no. 1 (2013): 118–31. https://doi.org/10.1111/amet.12009.

———. “Design and Anthropology.” Annual Review of Anthropology 45, no. 1 (2016): 433–49. https://doi.org/10.1146/annurev-anthro-102215-100224.

Myers, Brad, Scott E. Hudson, and Randy Pausch. “Past, Present, and Future of User Interface Software Tools.” ACM Transactions on Computer-Human Interaction 7, no. 1 (2000): 3–28. https://doi.org/10.1145/344949.344959.

Nardi, Bonnie A., and James R. Miller. “Twinkling Lights and Nested Loops: Distributed Problem Solving and Spreadsheet Development.” International Journal of Man-Machine Studies, Special Issue: Computer-supported Cooperative Work and Groupware. Part 1, 34, no. 2 (1991): 161–84. https://doi.org/10.1016/0020-7373(91)90040-E.

Noble, Safiya Umoja. Algorithms of Oppression: How Search Engines Reinforce Racism. New York: New York University Press, 2018.

Ochs, Elinor. “Experiencing Language.” Anthropological Theory 12, no. 2 (2012): 142–60. https://doi.org/10.1177/1463499612454088.

Omar, Cyrus, Ian Voysey, Ravi Chugh, and Matthew A. Hammer. “Live Functional Programming with Typed Holes.” Proceedings of the ACM on Programming Languages 3, no. POPL (2019): 1–32. https://doi.org/10.1145/3290327.

Omar, Cyrus, Ian Voysey, Michael Hilton, Jonathan Aldrich, and Matthew A. Hammer. “Hazelnut: A Bidirectionally Typed Structure Editor Calculus.” ACM SIGPLAN Notices 52, no. 1 (2017): 86–99. https://doi.org/10.1145/3093333.3009900.

Ouyang, Long, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang et al. “Training Language Models to Follow Instructions with Human Feedback.” Preprint submitted March 4, 2022. https://doi.org/10.48550/arXiv.2203.02155.

Paullada, Amandalynne, Inioluwa Deborah Raji, Emily M. Bender, Emily Denton, and Alex Hanna. “Data and Its (Dis)Contents: A Survey of Dataset Development and Use in Machine Learning Research.” Patterns 2, no. 11 (2021): 100336. https://doi.org/10.1016/j.patter.2021.100336.

Pollock, Josh, Ian Arawjo, Caroline Berger, and Arvind Satyanarayan. “Designing for Semi-Formal Programming with Foundation Models.” In PLATEAU: 14th Annual Workshop at the Intersection of PL and HCI. February 2024.

Pollock, Josh, Arvind Satyanarayan, and Daniel Jackson. “Language Model Agents Enable Semi-Formal Programming,” The Live Programming Workshop at SPLASH, October 2023. https://2023.splashcon.org/details/live-2023/3/Language-Model-Agents-Enable-Semi-Formal-Programming.

Robinson, Jeffrey D. “Accountability in Social Interaction.” In Accountability in Social Interaction, edited by Jeffrey D. Robinson. New York: Oxford University Press, 2016. https://doi.org/10.1093/acprof:oso/9780190210557.003.0001.

Roose, Kevin. “A Conversation With Bing’s Chatbot Left Me Deeply Unsettled.” New York Times, February 16, 2023, sec. Technology. https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html.

———. “An A.I.-Generated Picture Won an Art Prize. Artists Aren’t Happy.” New York Times, September 2, 2022, sec. Technology. https://www.nytimes.com/2022/09/02/technology/ai-artificial-intelligence-artists.html.

———. “Bing’s A.I. Chat: ‘I Want to Be Alive. 😈.’” New York Times, February 16, 2023, sec. Technology. https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html.

Schennum, Jill A. As Goes Bethlehem: Steelworkers and the Restructuring of an Industrial Working Class. Nashville, TN: Vanderbilt University Press, 2023. https://books.google.com/books?hl=en&lr=&id=eB3TEAAAQBAJ&oi=fnd&pg=PT5&dq=As+goes+Bethlehem+:+steelworkers+and+the+restructuring+of+an+industrial+working+class&ots=AFxSb7lEEv&sig=cx34HZrNZVvWRyIR1AIEa89XPu0.

Seaver, Nick. Computing Taste: Algorithms and the Makers of Music Recommendation. Chicago: University of Chicago Press, 2022. https://books.google.com/books?hl=en&lr=&id=dVuVEAAAQBAJ&oi=fnd&pg=PR7&dq=nick+seaver+computing&ots=_8NnaEFl9N&sig=9o5zxKvMZp8wrUZMYTMLgnrEgyw.

Setlur, Vidya, and Melanie Tory. “How Do You Converse with an Analytical Chatbot? Revisiting Gricean Maxims for Designing Analytical Conversational Behavior.” In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 1–17. CHI ’22. New York: Association for Computing Machinery, 2022. https://doi.org/10.1145/3491102.3501972.

Shan, Shawn, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. “Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models.” 2023. https://doi.org/10.48550/arXiv.2302.04222.

Shneiderman, Ben, and Pattie Maes. “Direct Manipulation vs. Interface Agents.” Interactions 4, no. 6 (1997): 42–61. https://doi.org/10.1145/267505.267514.

Singleton, Tom, Tom Gerken, and Liv McMahon. “How a Chatbot Encouraged a Man Who Wanted to Kill the Queen.” BBC, October 6, 2023. https://www.bbc.com/news/technology-67012224.

Slotta, James. Anarchy and the Art of Listening: The Politics and Pragmatics of Reception in Papua New Guinea. Ithaca, NY: Cornell University Press, 2023. https://books.google.com/books?hl=en&lr=&id=3HGgEAAAQBAJ&oi=fnd&pg=PR5&dq=anarchy+art+of+listening&ots=i_n7ywVp5R&sig=198SoubsGsqrI25ItEQpFlU5AlI.

Suchman, Lucille Alice. Human-Machine Reconfigurations: Plans and Situated Actions. Cambirdge: Cambridge University Press, 2007. https://books.google.com/books?hl=en&lr=&id=VwKMDV-Gv1MC&oi=fnd&pg=PR7&dq=Suchman+LA+(2007)+Human-machine+Reconfigurations:+Plans+and+Situated+Actions,+2nd+edition.+Cambridge,+UK:+Cambridge+University+Press.&ots=BQzzuL0YbH&sig=-eXSP4v0v9i3gGwMZFBExa5yzbw.

tldraw [@tldraw]. “Let’s Go Http://Makereal.Tldraw.Com Https://T.Co/4y9VMiTN0h.” Tweet. Twitter, November 15, 2023. https://twitter.com/tldraw/status/1724892287304646868.

———. “State Charts + Wireframes + Annotations Https://T.Co/vBciMZ49yf.” Tweet. Twitter, November 16, 2023. https://twitter.com/tldraw/status/1725083976392437894.

Tractinsky, Noam. “The Usability Construct: A Dead End?” Human–Computer Interaction 33, no. 2 (March 4, 2018): 131–77. https://doi.org/10.1080/07370024.2017.1298038.

Turing, Alan M. “Computing Machinery and Intelligence.” Mind 49, no. 236 (1950): 433–60.

Tversky, Barbara. Mind in Motion: How Action Shapes Thought. New York: Basic Books, 2019.

Victor, Bret. “Up and Down the Ladder of Abstraction,” October 2011. http://worrydream.com/LadderOfAbstraction/.

Walker, Harry. “Joy within Tranquility: Amazonian Urarina Styles of Happiness.” HAU: Journal of Ethnographic Theory 5, no. 3 (2015): 177–96. https://doi.org/10.14318/hau5.3.010.

Walny, Jagoda, Sheelagh Carpendale, Nathalie Henry Riche, Gina Venolia, and Philip Fawcett. “Visual Thinking In Action: Visualizations As Used On Whiteboards.” IEEE Transactions on Visualization and Computer Graphics 17, no. 12 (2011): 2508–17. https://doi.org/10.1109/TVCG.2011.251.

Weidinger, Laura, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng et al. “Ethical and Social Risks of Harm from Language Models.” Preprint submitted December 8, 2021. https://doi.org/10.48550/arXiv.2112.04359.

Widder, David Gray, Sarah West, and Meredith Whittaker. “Open (for Business): Big Tech, Concentrated Power, and the Political Economy of Open AI.” , Preprint submitted August 17, 2023. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4543807.

Wongsuphasawat, Kanit, Zening Qu, Dominik Moritz, Riley Chang, Felix Ouk, Anushka Anand, Jock Mackinlay, Bill Howe, and Jeffrey Heer. “Voyager 2: Augmenting Visual Analysis with Partial View Specifications.” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2648–59. Denver, CO: ACM, 2017. https://doi.org/10.1145/3025453.3025768.

Yildirim, Nur, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fernanda Viégas. “Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook.” In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–13. CHI ’23. New York: Association for Computing Machinery, 2023. https://doi.org/10.1145/3544548.3580900.

Zong, Jonathan, Dhiraj Barnwal, Rupayan Neogy, and Arvind Satyanarayan. “Lyra 2: Designing Interactive Visualizations by Demonstration.” IEEE Transactions on Visualization and Computer Graphics 27, no. 2 (2021): 304–14. https://doi.org/10.1109/TVCG.2020.3030367.