Abstract

With recent leaps in the accessibility and capabilities of generative artificial intelligence (GenAI), we believe it is time for college instructors and professional trainers to take seriously the possible uses of GenAI-enabled teaching tools. As instructors in multiparty negotiation courses at the Massachusetts Institute of Technology (MIT), the authors set out to experiment with the integration of GenAI into the development of multiple kinds of ‘negotiation coaching bots’ that can help students prepare more effectively for and learn from their involvement in role-play simulations (which are used extensively in negotiation instruction at both the undergraduate and graduate levels around the world). Based on our first round of experiments, we believe that coaching bots can, indeed, play a key part in teaching about and learning the skills of negotiation. The lessons we have identified thus far can, we believe, be applied in many other teaching contexts.

1. Can GenAI Be Used to Improve Teaching Human-to-Human Interactions?

Over the past several decades, research has shown that individuals who must deal with interpersonal conflict, in the workplace and everyday life, often tend to interact in counterproductive ways. They do not know how to engage in joint problem-solving, often escalating low-level disagreements into full-blown disputes. Many times, this happens because the individuals involved do not capitalize on proven negotiation theories and practices that would enable them to ‘create value’ and improve relationships while meeting their most important objectives.

In 2023, with the widespread availability of ChatGPT and other generative models, educators, including negotiation instructors, began exploring possible uses of AI to enhance negotiation instruction.1 At the same time, the president and provost of MIT initiated a multidisciplinary university-wide effort (with a call for papers) to explore the impact and application of GenAI, encouraging research, experimentation, and publication.2 We were invited to be part of that effort.

The two lead authors of this paper have been at the forefront of developing both the theory and the practice of negotiation instruction for many decades through the MIT–Harvard–Tufts interuniversity Program on Negotiation (PON).3 We were eager to see whether and how Generative AI in its currently available forms might be used to enhance negotiation instruction. In our view, any attempt to incorporate GenAI into negotiation pedagogy must build on what is already known about negotiation theory and practice, and it must design and implement ways of applying this knowledge in a clinical context (i.e., realistic role-play simulations).

At MIT, the authors teach negotiation courses that cover both one-on-one negotiation and multiparty negotiation. This is an important distinction. The AI experiments we had in mind focused on multiparty negotiation since that portion of the field is still underdeveloped. In all cases, our teaching focuses on helping negotiation students improve their personal theories of practice.4

In recent years, we have integrated various online learning platforms and asynchronous tasks into our curricula using videos and self-study assignments in a ‘learning loop’ that emphasizes what it takes to prepare for a negotiation, the steps required to enhance situational awareness at ‘the table,’ strategies for handling surprises in an unfolding negotiation, and techniques for reflecting on each negotiation to learn as much as possible in preparation for future negotiations.5

With that background, we postulated several experiments with AI negotiation bots. This followed a first effort to introduce a personalized ‘negotiation assistant’ into our MIT Entrepreneurial Negotiation course. We decided to focus on two of the steps in the learning loop: helping students prepare for an upcoming multiparty negotiation and encouraging our class to debrief privately (i.e., reflect and learn from their negotiation experience) based on their involvement in specific role-play simulations.

We hypothesized that GenAI might help pre-professionals prepare more effectively for upcoming negotiations like those common in business, law, complex technical projects, international diplomacy, and public management. We found some social media coverage of similar negotiation instruction incorporating AI bots of various kinds, but all of them focused entirely on two-party rather than multiparty situations.

Expanding into our ongoing graduate-level classes at MIT (during the 2024 academic year), we organized multiple opportunities for our students to use GenAI to prepare and debrief in the context of two multiparty role-play simulations. We hope this short summary of our findings will trigger broader interdisciplinary discussions about the roles that GenAI can play in improving classroom learning, especially regarding the best ways of helping students improve on their human-to-human interactions.

1.1. Building Generative AI Tools for Multiparty Negotiation

Our starting point was a brainstorming session at which we envisioned what GenAI tools we might be able to build and test within the nine-month academic calendar. We decided to focus on two distinct cases that we often use in our two courses. We also decided to collaborate with the experienced team at iDecisionGames (iDG), an online learning platform that has been part of PON’s instructional efforts for years.6 We needed to hire AI subject matter experts (i.e., advanced graduate students) as research and teaching assistants for this project. We were careful to document our work process and solicited lots of structured feedback from students.

One byproduct of our work is a seven-step approach to designing GenAI tools to accompany well-known teaching role-plays in the negotiation field. Another is a four-aspect approach to AI prompt design in this context.

1.2. The Seven Steps to Building GenAI Tools

We used a seven-step process to ensure that the tools we developed were based on our usual pedagogical assumptions (i.e., what we expect students to learn and how we expect them to learn through clinical experience). We applied this seven-step process to creating ‘preparation’ coaching bots for each role-play simulation. Then, we created ‘debriefing’ coaching bots for each of the two role-plays. The seven steps are as follows: (1) choose the teaching case (or role-play simulation), (2) identify the key learning objectives or goals in each case, (3) map (and test) the intended user experience, (4) understand and accept the limitations of the current technology, (5) undertake multiple revisions of prompt design, (6) integrate revised prompts into the actual learning platform, and (7) repeat these steps, making refinements as necessary.

1.2.1. Step 1: Pedagogy Process

Choose carefully the case you want to implement, in the context of your class syllabus, and be clear about your pedagogical objectives.

Given our strong motivation to utilize GenAI in teaching multiparty negotiation—specifically to enhance the human-to-human interaction—it was crucial that our experimental design build on the in-person clinical negotiation experience that we have relied on for many years. Thus, we aimed to ensure that the introduction of new AI tools did not get in the way of, or detract from, the learning process as we know it. We chose two complex multiparty cases that we would normally use to teach a long list of specific negotiation lessons (i.e., it is important to understand the interests of your negotiating partners, it is essential that the parties in a multiparty negotiating situation agree on ground rules that will govern their interaction, parties should focus on building ‘winning’ and/or ‘blocking’ coalitions, ‘packaging’ or ‘trading across issues’ are essential to creating value, etc.).

Harborco is a six-party, multi-issue, scoreable negotiation over a proposed new deep-water port that would be built on the east coast of the United States.7 Hydropower in Santales centers on a six-party, multi-issue, non-scoreable, mediated negotiation regarding the construction of a new hydropower plant in a fictitious South American country.8 Both role-play packages include a teacher’s guide, general instructions that all the players share, and confidential role instructions unique to each role-player. By delivering class materials through the iDG system, we could control the timing of each stage of the participants involvement.

1.2.2. Step 2: Pedagogy Objectives

Define specific tasks and patterns of engagement for each of the coaching bots.

Each type of coaching bot must reflect the teaching objectives of the instructor. Not only are the objectives specific to the case being taught, but the learning objectives may be different in each case for different roles. While a preparation coaching bot will pose some set of similar questions to each role-player, some will be different. For example, in the Harborco simulation, all roles must consider who their allies and opposition might be, but each role has different levels of interests (or urgency) in reaching a consensus decision versus blocking disadvantageous decisions.

In designing the preparation coaching bots, we prioritized (1) reinforcing each student’s understanding of the background case materials, (2) clarifying their motives and priorities in the upcoming negotiation, and (3) anticipating contingent responses to a range of tactics and strategies that might be used by the other parties.9 We specifically selected parts of required course readings to incorporate into the questions and responses preparation bots were ready to emphasize in their interactions with different players.

Effective preparation is pivotal in all multiparty negotiation. Thorough engagements with assigned materials help students avoid misinterpreting their priorities or overlooking critical strategic options, which could derail their plans. In an unassisted class, instructors cannot possibly coach students simultaneously. Our AI tool carefully balances providing individualized guidance, at a pace dictated by each student, with an effort to be sure that students give careful consideration to the assigned reading in the class and what they have retained from earlier clinical engagements. By achieving this balance, the coaching bot can not only enhance student preparedness but also provide an instructor with detailed insights into each student's engagement and understandings.10

For the debrief coach, our focus shifted to aiding students in reflecting on various aspects of the negotiation dynamics they had just experienced. Post-negotiation reflections are crucial, as they encourage students to learn from their own experiences. Following their personal reflections, students are usually drawn into a class-wide debriefing in which the instructor talks simultaneously with everyone in the class so they can hear from and integrate what others have learned into their own evolving personal theory of negotiation. The debrief coaches are designed to gather and synthesize information from each student’s negotiation experience, thereby informing the instructor’s approach to the classroom discussion.11 This is particularly advantageous in situations in which instructors cannot observe every group of six students doing the same negotiation separately but at the same time.

1.2.3. Step 3: User Experience Design

Describe in detail how you intend for students to interact with and learn from the coaching bots.

Based on years of teaching and coaching experience, we tried to imagine what typical coaching sessions might sound like. Traditional negotiation course assignments and learning platforms used a set of prescribed homework assignments (e.g., negotiation preparation checklists, negotiation debriefing prompts, etc.). As a teaching team, we tried to imagine what would encourage students to engage with the coaching bots (as an alternative to the usually written homework assignments) and what style of communication would be most comfortable for most students. Given the current technology, we realized that the coaching session would have to take the form of back-and-forth text messages between the student and the coaching bot. We considered who should start ‘speaking’ and what the introductory statements on both sides might sound like. We drafted possible exchanges, taking into account how to handle breaks or interruptions. We concluded by thinking about the ending of each coaching session: what summary items or accounts ought to be captured (e.g., main takeaways), what questions should be asked before ending (about the case, or about the coaching session itself), and how to verify with the student that they wished to end the session. We drafted imagined scripts for every individual coaching bot for each role in each case.

One key thing we realized is that just as each human coach has a personality, we should assign each bot a communication style or personality.12 We found that letting students choose the personality and communication style they preferred in their coach was crucial.13 They reported forming stronger connections to AI bots that exhibited certain human-like interactions that made them more comfortable.

1.2.4. Step 4: Integration Planning

Plan your integration and testing based on a thorough understanding of the requirements and constraints of the current technologies

A significant aspect of integrating these AI tools involved aligning them with the operational and technological constraints of the advanced online learning platform we had selected.14 We needed to fully understand the strengths and weaknesses of current technologies, the learning platform structure and its limitations, the state of current GenAI tools, and the available integration interfaces. Indeed, we had to be willing to revise our earlier choices and aspirations to ensure they were compatible.

Among the limitations of the learning platform and its integration capabilities were restrictions on the length of messages, the length of interactive sessions, the amount of material that can be used as input to a prompt, and moving the learning experience from one stage to the next (such as from the preparation stage to the negotiation stage to the debriefing stage).

We decided early in our discussions to forgo the inclusion of GenAI coaching bots during the actual face-to-face negotiations. When negotiations are live in class, we often try to capture (via video) as much of the exchange as possible to use in subsequent reflection and debriefing. Speakers tend to interrupt each other or speak in parallel while talking around the table as a group. Negotiators also have side conversations by whispering quietly to each other or by moving away from the table to caucus in smaller groups. These communications and alliance-building dynamics are an important focus of our pedagogy but hard to fully capture for GenAI use.15 Future developments would enhance the capabilities of our AI tools, making them even more effective in capturing the real-time nuances of multiparty negotiations.

1.2.5. Step 5: Prompt Design

Develop and test each coaching bot’s AI prompt in full detail by using direct engagement with a selected large language model (LLM). Commit to continuous revisions.

LLMs, which underpin our coaching bots, are primarily driven by sophisticated prompt engineering. Prompt engineering has evolved significantly, with numerous studies illuminating the subtleties of prompt structure and its important role in eliciting effective AI responses.16 We leveraged our prompt developers’ extensive experience with ChatGPT to craft prompts that not only served our educational purposes but also ensured the integrity of the saved interactions. We used ChatGPT-4 Turbo.

Prompt design is at the core of negotiating coaching bots. We now believe that there are four distinct aspects of prompt design focusing on (1) the substance of the conversation, (2) the style of the conversation, (3) communication session dynamics, and (4) the bot’s ‘memory.’ Each role in the two negotiation scenarios had its own tailored preparation coaching bot and a debrief coaching bot. Separation prevented cross-role information leakage.

1.2.6. Step 6: Integration Debugging

Integrate your tested prompts into the learning system’s interface and perform integration testing.

After rigorous direct (single-bot-prompt-only) testing with GPT-4 Turbo to confirm the effectiveness and security of each set of bot interactions, we integrated all the prompt sequences into the iDG platform. Our integration ensured that we could immediately access summaries of each student’s interactions. For testing, we used staff and volunteers to ensure each role in the group exercise worked correctly for all stages of the exercise.

1.2.7. Step 7: Refinement

The development team evaluates and adapts its earlier choices.

The development of our negotiation coaching bots was inherently iterative, requiring numerous rounds of testing and refinement by the full teaching team.17 After ensuring basic functional integrity, we continued fine-tuning each of the prior step elements, from pedagogy to prompts and integration.18

Now that we have covered the seven-step process, we want to dig a bit deeper into the process of prompt design.

1.3. Prompt Design—The Key to Building Effective Negotiation Coaching Bots

We developed a four-aspect lens to define what we needed from each bot prompt. We named these substance, style, session, and memory. While we were using ChatGPT-4 Turbo, these concepts translate to any other LLM that might be used as the backend of the system. Naturally, if switching LLMs (or upgrading to a new version of an existing one), we would want to test the system to make sure that the ‘behaviors’ have not been significantly modified.

Each bot session prompt was developed by a designated prompt developer. The developer wrote the prompt and tried it out in dialogue with ChatGPT-4 in a standalone session. That session was then reviewed by the designer and an experienced instructor who often suggested corrective actions. The developer could then repeat the process until each prompt produced the desired effect. The key to good prompt design is being very precise. It is a craft that one gets better at with experience.19 Then, the developer must ‘stress test’ the prompt to see how it handles various user behaviors and, where needed, continue to refine the prompt. Once the prompt is deemed acceptable, it is then loaded onto the learning platform in the right place in the flow. Then, the developer must test how the various prompts work together.

Figure 1 shows the four aspects of bot development, and Figure 2 (pictured later) shows the relationship with other session and platform modules.

1.3.1. Substance

This aspect consists of the substance of the conversation: what information to communicate (and what not to), and what information to ask about.

For the preparation coaching bot, we created detailed prompts incorporating both the general instructions for the simulation and role-specific guidelines (from the PON case and adapted).20 In addition, we provided clear preparation objectives driven by the core negotiation concepts regarding preparation presented in the syllabus. We used summaries of key terms and concepts, keeping text length short (i.e., instead of providing an entire book or article, we offered a summary of a key concepts). These prompts were very carefully designed to guide the AI in delivering focused advice and encouraging step-by-step strategy formulation.

1.3.2. Style

The style aspect is the style of the conversation, or the ‘personality’ of the bot.

As mentioned in the user experience step, each bot needs to be guided as to style and tone. ChatGPT tends to be very polite and accommodating, so specific effort needs to be employed to make it more direct and assertive.

To accommodate diverse learning preferences, we introduced three distinct coaching styles within the preparation coach:

• Inquisitive coach: encourages students to ask questions to deepen their understanding of the negotiation material and their roles.

• Suggestive coach: offers advice based on student questions and the specifics of the negotiation scenario.

• Critical coach: challenges students, prompting them to reconsider what they have in mind.

1.3.3. Session Dynamics

Session dynamics consist of the progression of the conversation.

Each session’s user experience should include instruction for the GenAI on how to open the conversation, how to proceed interactively, and under what situation it should move to end the conversation and how to do so.

Before concluding the dialogue session, for both the preparation and debrief coach prompts, we included a segment that summarizes the interaction between the coach and a student. This was useful for the student and served as our ‘memory’ on the learning platform.

1.3.4. Memory Connections

This aspect enables interface connections between the current prompt to prior or future prompts.

The prompt instruction to summarize the current conversation, including how to do so (e.g., how long the summary should be, whether in first or third person), enables future prompts to use this summary. For example, summarizing a student’s preparation coaching session enables it to be referred to later by the debrief coaching bots (as well as by the instructors.)

When a prompt includes instructions to incorporate certain reference material that has been prestored on the system from prior sessions, it enables connecting those prior sessions to the current one. Indeed, we enriched the AI of the individual debrief coach with summarized background data from the student’s earlier interactions (from the preparation phase) as well as the actual outcomes of their multiparty negotiations. This allowed the debrief coach to facilitate a reflective discussion informed by the student’s original intentions versus their actual reported performance.

2. Our Experimental Results: Student Engagement and Test Negotiations

The participants of the experiments were enrolled in “Multistakeholder Negotiation for Technical Experts” at the MIT School of Engineering, taught by Samuel Dinnar, and “Negotiation and Dispute Resolution in the Public Sector” in the MIT Department of Urban Studies and Planning, taught by Lawrence Susskind. In total, we had five groups for the experimental runs of the two games.21

Here are the steps taken by both the students and the instructors to get ready for and following the in-class experiment (see Figure 2):22

• A—Self-assessment baseline survey: the self-assessment baseline survey measured levels of negotiation skills among participants before they engaged with the AI coaching bots.23

• B—Role assignment and individual student preparation: Students were assigned roles and given access to the simulation via the learning platform. This is how they accessed their instructions and their custom coaching bots.

• C—Pre-negotiation: Individual preparation coaching session: The pre-negotiation preparation coaching bot for each role had all the general information about the case, role-specific information, instructions on how the coach should interact with the learner, and instructions on how to summarize the conversation at the end of each coaching session. The bot was provided with three different personalities that students could choose from at the beginning of each coaching session. We found that some students requested (and the GenAI allowed for) an additional coaching session with a different style choice, a creative blend of two different coaching types as requested, or even changing their initial selection midway through.

• D—Pre-negotiation: A note on back-table coaching: At the last minute, we also created a third type of coaching bot for each role in each simulation. We call this a back-table coaching bot—it offered, in a time-limited way, an opportunity for each role-player to have a brief conversation to learn more about each of the other roles’ interests and intents. Instead of structuring this as a coaching conversation, we decided to simulate a short conversation with someone from the other role’s back table (i.e., someone from the organization of the other role but not the empowered negotiator).24

• E—In-person negotiation: After students completed the required individual negotiation preparation, they proceed to have the multiparty, in-person, face-to-face negotiation in class.25 One student was assigned each role.26 The outcome of each negotiation (i.e., whatever agreement was reached) was added by the students to the iDG platform, including some interim stage outcomes (e.g., voting rounds in Harborco or an agreement on ground rules in Santales).27

• F—Post-negotiation debrief: Individual debrief coach: Shortly after the negotiation ended, the post-negotiation individual debrief coach provided an opportunity for each student to reflect on their performance and the results of their negotiation. This role-specific coach interaction with each student is intended to be a short (fifteen-minute or so) debriefing that could be done in the classroom or shortly thereafter.28 The individual debriefing coaching bot prepared a summary of the conversation at the end of the individual debriefing for students to use in their written reflections.29 The students’ level of engagement with this debriefing bot was uniformly high.

• G—Instructor summaries and in-class debrief: Summaries of each student’s levels of engagement, as well as conversation summaries, are available on the system along with summaries of each test group’s negotiation results. These were used by the instructor for the in-person, in-class debrief.30 In addition, the conversation summaries generated by the various negotiation coaching bots are now being analyzed by our teaching team to determine exactly how students are using the coaching bots’ assistance. We may design instructor summary bots to help distill all the student learning we have tracked, which could help instructors process the large amounts of information on their class learning during a semester.31

• H—Self-assessment post-class: The post-negotiation assessment survey aimed to measure the effectiveness of AI-assisted tools in enhancing negotiation skills after participants have completed the exercise. This survey was structured to compare against the baseline established pre-negotiation.

3. Findings and Implications for the Use of AI in Teaching Negotiations

Our experiential experiences with the use of GenAI tools in our negotiation courses has demonstrated quite clearly that tailored coaching bots of several kinds can reinforce key learning points in negotiation. While we did not have a control group for these experiments, we have taught negotiation for many years using these same role-play exercises. Since our focus is on how effectively negotiation students are able to build their personal theories of negotiation practice (i.e., their mastery of key negotiation concepts and strategies, their levels of confidence in applying in practice what they are learning in theory, and their ability to learn from their own experience), student self-assessments (i.e., before and after) were key.

3.1. Results From the Harborco Role-Play

In the Harborco scenario, which involves clear scoring metrics aimed at optimizing numerical outcomes, students engaged actively with the AI coaches. These negotiation coaching bots helped students leverage their understanding of each players’ priorities to guide students in implementing strategies for voting, coalition building, and caucusing. Feedback from the students was overwhelmingly positive. For example, in comparing pre- and post-negotiation reports regarding the use of negotiation skills, the students’ results were as follows:

• Discovering other parties’ interests: improved from an average and median of moderately confident (4) to confident (5).

• Dealing with strong emotions: improved from an average and median of moderately confident (4) to confident (5).

• Dealing with bad actors: improved from an average of somewhat insecure (3) and median of insecure (2) to average and median moderately confident (4).

From their experience with the coaching bots and what they found most helpful, students reported the following:

• 82% felt better prepared in advance of the negotiation.

• 77% experienced improvement in their ability to discover others' interests.

• 59% found it easier to express their positions or priorities.

Students appreciated the AI’s guidance in navigating negotiation complexities, with many highlighting how the coaches’ prompts pushed them to consider negotiation elements they had previously overlooked.32 As one student noted: “The act of articulating my thoughts to the bot significantly sharpened my critical thinking.”

3.2. Results from the Santales Role-Play

The Santales simulation, which doesn’t use numerical scoring and features significant power differentials among the parties, presented a different set of negotiation challenges. Student engagement with both the preparation and debriefing coaching bots focused heavily on process ground rules and strategy. Participants valued the AI’s assistance in managing the ambiguity and open ended aspect of the scenario, forcing them to prioritize their goals and priorities more effectively. This led to more creative and adaptive negotiation strategies, as reflected in student feedback: “Engaging with the AI coaches taught me that negotiation is less about rigid adherence to pre-prepared arguments and rules and more about being adaptable and responsive to the flow of the dialogue.” This feedback underscores the AI’s role not only in enhancing students’ mastery of key concepts but also in fostering a deeper understanding of negotiation as a dynamic and fluid process.

3.3. General Learnings (from Both Cases)

After conducting both exercises in class, we again surveyed the students about their overall experience. In general, twenty-two students reported that AI-assisted coaching significantly improved their ability to draw lessons from their own experience. A remarkable 100% of students found the preparation coaching bots extremely beneficial for negotiation preparation, and 75% appreciated the support from the debrief coaching bots.33

The students noted that the coaches were especially helpful in boosting their confidence and preparedness, understanding other parties’ interests, and formulating negotiation strategies. Despite these advantages, some also acknowledged challenges in translating coaching bot interaction into achieving mutually beneficial outcomes during the actual negotiations. The preparation bot was particularly valuable in getting them to consider the interests of all parties, suggesting creative solutions, providing a systematic framework for preparation, and allowing private conversations with a bot to explore information regarding the pressures put on their negotiating partners by their back tables. We were not able to assess the impact of AI-assisted coaching on emotional management during negotiations.

The students who worked with the negotiation debriefing bots said that their coaches provided a useful structured reflection but could help by being even more critical. One student remarked, “The debrief bots, both in-class and at home, were incredibly valuable for me in several ways. […] This real-time feedback was invaluable in helping me understand my strengths and weaknesses, allowing me to make adjustments and improve my negotiation skills on the spot.”

The coaches were a valuable complement to firsthand (experiential) practice. Contrary to initial concerns that students might use the AI assistance to reduce the time they needed to put into preparations, we perceived that they increased their investment of personal time in negotiation preparation through their interactions with the coaching bot.

3.4. Reactions from Professional Colleagues

After experimenting with the bot-assisted role-plays and debriefing our students and teaching assistants, we had an opportunity to present our preliminary findings to a large group of thirty experienced negotiation professors and instructors gathered at a PON Negotiation Pedagogy dinner. This Pedagogy at PON session helped us refine our statement of findings as well as deepen our understanding of what will be required before others will feel able to incorporate what we have learned into their own teaching.34

Our colleagues wondered most what they would have to know and be able to do on their own in order to use the negotiation coaching bots we have developed. Indeed, they wanted to know if we would immediately make the coaching bots available (with documentation) through PON’s Teaching Negotiation Research Center (TNRC).35

Their questions centered around whether other instructors would have to completely remake our coaching bots to reflect their own teaching emphases and theory, or could they just use what we have created the way they stand? Our answer was, “Both.” If they desire to use our own specifically ‘enhanced’ role-play simulations, then yes, we would be glad to share what we have produced.36 If their intention is to develop other bots or retrain our bots to work with other role-play simulations from TNRC, that’s fine, too, but we are not sure yet how to share what we have produced. There’s no question, though: we want to encourage others to design their own GenAI tools, tailored to their own pedagogical priorities using other role-play simulations.

Some instructors asked whether starting with two-party negotiations, incorporating only quantitative measures of success, would be the better starting point for building and testing the idea of GenAI negotiation coaches. We answered that we are aware of others trying to do that, but our focus is on multiparty negotiation in which optimization is not the only or primary measure.37 The negotiation dynamics we are teaching our students to handle lean in the direction of how to pursue alliances, whether and how to build blocking or winning coalitions, building consensus in the face of disagreements, dealing with missing (not-at-the-table) stakeholders, and facilitating the implementation of multiparty agreements (including setting ground rules, conversational turn taking, collaborative decision-making, and mediation).

Some professors in the audience shared that their students appear to be using publicly available online GenAI to complete their assignments faster, by “letting ChatGPT do their preparation for them.” We indicated that our experience was the opposite, both for the preparation and the debriefing coaching bots.38

We highlighted for our PON colleagues several additional innovations we had incorporated into the two role-play simulations we selected and by using the iDG online learning platform. The innovation that generated the most excitement was our addition of an extra pre-negotiation step during which students were able to interact with simulated ‘back-table’ representatives of the other parties they were about to negotiate with. We will explain more about how we developed and implemented these back-table player bots in an accompanying paper that we intend to publish describing the use and potential of this innovation.39

Another interesting thread in the discussion focused on the emotional dimension of negotiation and whether the way learners interact with the coaching bot decreases or increases their attention to it. We indicated that we had prompted both the preparation and the debriefing bots to ask questions in a way that emphasized the emotional dimension of negotiation theory and practice and that a lot more could be done if a more specific focus is desired.40

One final aspect of our PON discussion concerned the ability of coaching bots to carry forward specific knowledge gathered from interactions with a particular student in a semester-long class and move that knowledge from one role-play simulation to another. We believe this can be done, although we have not yet done it.41 By tracking the full body of learning from a sequence of sessions over a semester, it would be possible to enable individual coaching bots to help students develop a longitudinal sense of their changing theory of practice.

4. Implications of Our Findings

We can now demonstrate that developing and implementing tools using GenAI for multi-stakeholder negotiation coaching is feasible. We also feel that rapid developments in GenAI technology and various generative models will make it possible in the very near future to develop even better tools to accomplish our instructional goals.42 Current bot interactions with learners (e.g., including both the preparation bot and the debriefing bot) were completed using back-and-forth text messaging. Technology is already available to convert speech to text and text to speech. This would enable students to interact with their coaching bots in natural language verbal conversation. This could include conversations using an avatar with an emotive facial expressions and tone. We can also imagine this being complemented by an interpretation of the student’s facial expressions and tone by the coaching bot.

Another current limitation specific to (and significant in) multiparty negotiation is speaking attribution and conversation turn taking.43 Finally, another of the shortcomings we encountered is that current LLMs occasionally have trouble handling math (i.e., complex; but sometimes even simple math was impacted by the model’s ‘hallucinations’). We are confident that LLM designers are working hard to overcome this shortcoming.

There are others in academia (as we heard from our PON colleagues) who want to see for themselves what the quantitative impacts of negotiation bots are. The issue of how to measure the effectiveness of teaching specific negotiation concepts and methods using particular role-play simulations has been debated for quite some time (including recently).44 That’s even before Gen AI assistants were added.

We are eager to continue to broaden the use of negotiation coaching bots in several ways:

1. We would like to transform the two role-play simulation packages we have developed into commercial products. What we have in hand allowed us to test negotiation coaching bots in our classes using two well-known role-play simulations. We need to finalize the design of the three types of coaching bots described above, taking account of what we have learned so far. And, we need to work with TNRC to figure out how best to distribute them.

2. We want to be able to share the materials we have developed on the iDG learning platform. For this to happen, we will need to prepare complete teaching packages (i.e., teaching notes, instructional guides, etc.) for those who want to use them as is. For those who want to tailor them further—to fit their own pedagogical objectives—we need to prepare separate instructions or how-to guides.

3. We want to continue our experimentation at MIT by increasing the number of test groups that use our current generation of negotiation coaching bots in future semesters.

4. We want to work with TNRC to use other role-play simulations designed to teach different negotiation learning points.

5. We are eager to explore the use of negotiation coaching bots that emphasize additional elements of negotiation instruction in multiparty situations, particularly the roles of facilitators and mediators.

6. We want to experiment with negotiation coaching bots that carry documentation (‘the memory’) of the interaction with a student from one exercise to the next, enabling extended reflections on the evolution of the learning that happens during a full semester.

7. We want to continue exploring the use of GenAI bots to help students in other kinds of courses (i.e., leadership and entrepreneurship classes).

5. Closing

We believe our preliminary experiments have proven that negotiation coaching bots can be used to enhance pre-professional instruction in a number of ways. With current and upcoming improved capabilities and a greater accessibility of GenAI, we believe it is possible for college instructors and professional trainers to use GenAI tools to enhance how people learn to negotiate and how they can learn from their clinical negotiation experience. We have listed future directions we want to continue to pursue. And, finally, we have a basis for encouraging instructors in other fields, in which person-to-person interaction is the focus, to imagine using coaching bots to enhance their instructional effectiveness.

Authors

Lawrence Susskind is the Ford Professor of Urban and Environmental Planning at MIT, cofounder of PON at Harvard Law School, and the founder of the Consensus Building Institute in Cambridge, Massachusetts. His email address is [email protected]. 

Samuel ‘Mooly’ Dinnar is a lecturer at the MIT Graduate Engineering Leadership Program. As an aerospace engineer, computer scientist, entrepreneur, and business executive, he developed and taught many negotiation and mediation courses at PON and globally. His email address is [email protected].

Ololade O. Olaleye is a graduate of MIT where she earned a Master of Science in Electrical Engineering and Computer Science and an MBA through the Leaders for Global Operations program. Her email address is [email protected].

Leroy K. Sibanda is a graduate and researcher with MIT and works as a multidisciplinary engineer and computer scientist focused on AI and decision-making in society. He has focused his research and entrepreneurship specifically on human-centered AI and human-centered design. His e-mail address is [email protected].

Bibliography

• Dinnar, Samuel, and Lawrence Susskind. Entrepreneurial Negotiation: Understanding and Managing the Relationships That Determine Your Entrepreneurial Success. New York: Springer, 2018.

• Dinnar, Samuel, and Lawrence Susskind. “The Eight Big Negotiation Mistakes that Entrepreneurs Make.” Negotiation Journal 34, no. 4 (October 2018): 401-13. https://doi.org/10.1111/nejo.12244.

• Gallagher, Mary Beth. “Second Round of Seed Grants Awarded to MIT Scholars Studying the Impact and Applications of Generative AI.” MIT News, March 28, 2024. https://news.mit.edu/2024/mit-scholars-awarded-second-round-seed-grants-generative-ai-0328.

• Hatch, Jennie, and Lawrence Susskind. “Hydropower in Santales.” Program on Negotiation. Accessed August 16, 2024. https://www.pon.harvard.edu/shop/hydropower-in-santales/.

• Madigan, Denise, Thomas Weeks, and Lawrence Susskind. “Harborco.” Program on Negotiation. Accessed August 16, 2024. https://www.pon.harvard.edu/shop/harborco/.

• OpenAI. “GPT-4 Technical Report.” arXiv (March 2023). https://doi.org/10.48550/arXiv.2303.08774.

• Program on Negotiation. “Pedagogy at PON.” Accessed August 16, 2024. https://www.pon.harvard.edu/category/research_projects/negotiation-pedagogy-program-on-negotiation/.

• Program on Negotiation. “Teaching Materials and Publications.” Accessed August 16, 2024. https://www.pon.harvard.edu/store/about-our-materials/.

• Shapiro, Daniel L. “Teaching Students How to Use Emotions as They Negotiate.” Negotiation Journal 22, no. 1 (January 2006): 105-9. https://doi.org/10.1111/j.1571-9979.2006.00088.x.

• White, Jules, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C. Schmidt. “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT.” arXiv (February 2023). https://doi.org/10.48550/arXiv.2302.11382.