1. Motivation

The objective of this whitepaper is to identify opportunities, issues, and challenges facing equitable education pathways for careers in computing and the particular role that generative artificial intelligence (AI) could play to support postsecondary education at minority-serving institutions (MSIs) and community colleges (CCs).

The team of coauthors draw from our collective insights, practices, and lessons learned at our respective educational institutions, as well as exploration of larger trends in large language models (LLMs) tutors in computing education. We share a philosophy that innovative technologies should be codesigned with stakeholders, not for them.

With this in mind, this paper offers highlights from conversations with two groups. One is a focus group study with students at Georgia State University (GSU) who interact with state-of-the-art LLM tutors for Introduction to Computing to characterize their preferences and experiences. The other is from a set of interviews we conducted with industry experts to understand how generative AI, specifically code generation, is shifting professional software development and its implications for educational pathways to computing. We hope that the insights from this work contribute to the design and use of generative AI to improve equitable access and student success to introductory computing in post-secondary education, as well as inform the equitable design and development of LLM tutors for other topics.

This whitepaper is the result of a collaboration between the Massachusetts Institute of Technology (MIT; in Cambridge, Massachusetts), GSU (in Atlanta, Georgia), and Quinsigamond Community College (QCC; in Worcester, Massachusetts) supported by funding from MIT and Axim Collaborative. It had many contributions from stakeholders who teach introductory computing courses at GSU, QCC, and MIT, along with experts in pedagogy, generative AI, other types of machine learning, workforce education, user-centered design, education researchers, administration, human-computer interaction, and AI-powered tutor technologies.

2. Background: Post-Secondary Challenges for Equitable Student Success

The laudable goal of equitable post-secondary education in the United States refers to a system of higher education that is fair, just, and accessible to all individuals, regardless of their socioeconomic background, race, gender, or other demographic factors. BIPOC (Black, Indigenous, and other people of color) students constitute a vast majority of students at CCs, MSIs, and historically Black colleges and universities (HBCUs). Numerous disparities and inequalities still exist. Efforts to successfully address them is a complex and evolving issue, and the status of equitable education varies from state to state and among different colleges and universities.

This paper focuses on the central question of how generative AI could be used to advance equitable educational pathways in computing. However, the challenges underserved students face extend beyond learning about computing. For our purposes, acknowledging the broad factors that impede student success (Section 2) and how technology has been used to address them (Section 3) is important.

2.1. Challenges That Impede Student Success

2.1.1. Limited Access to Financial Resources

Nearly 33% of Black families and 31% of Hispanic families have negative net wealth. This economic disparity becomes more evident when considering the cost of college education in relation to family income. These students’ families often struggle to provide even the most fundamental educational tools such as computers, consistent internet access, and textbooks (Hsieh et al 2008). And this burden escalates for independent students with dependents, surging to an average of around $17,112 (Cahalan et al. 2022, 135). This startling economic reality overshadows the educational journey that under-resourced students face. Many students at MSIs and HBCUs must enter the workforce to make financial ends meet, juggling positions with multiple employers while attending school. Quite often, these work commitments interfere with students’ use of limited but needed educational support such as office hours, lab time, and tutoring services (Warschauer and Matuchniak 2010). This challenge is especially pronounced in areas like tuition and fees, where MSIs typically struggle to offer adequate financial assistance due to their own resource constraints (Espinosa et al. 2019, 65).

2.1.2. Limited Access to a Support Community

A student’s support community encompasses the institution, student clubs, friends, family, and other students, and it can have a profound effect on the student’s attitude toward education, their motivation to succeed, and their ability to anticipate and cope with challenges. Students at MSIs and HBCUs often chart new territory in academia as first-generation college attendees. (At GSU, 21% of undergraduate students are first-generation.1) Beyond financial difficulties, students often lack essential nonfinancial support—missing important guidance that family or peers with college experience typically provide. Also, having to spend significant time at work can interfere with a student’s ability to grow their support community. This gap can hinder their educational journey, as they may struggle with fundamental aspects of academic life, including navigating administrative procedures, developing effective study practices, or establishing essential professional networks.

2.1.3. Unmet Culturally Inclusive Learning Needs and Constraints

Understanding students’ learning needs is essential for creating inclusive educational experiences. Students from diverse backgrounds, based on their unique backgrounds and experiences, may have different levels of language proficiency and digital literacy, as well as distinct communication norms, participation preferences, and information processing methods. Recognizing and respecting these nuances is essential to ensure that classroom instruction aligns with students’ social and cultural identities, making the learning experience more relatable and engaging (Gumbel 2020).

By addressing socioeconomic, community, and cultural dimensions of student educational experience, educators and institutions can foster more equitable and accessible teaching strategies, ensuring that every student has the opportunity to thrive academically and reach their full potential (Tinto 2012).

These various challenges to success often come together in a particular academic context, as this whitepaper discusses next: Introduction to Computing.

2.2. Challenges in Equitable Educational Success through Computing Pathways

Introduction to Computing courses serve as the foundation for students venturing into the world of coding and are the gateway to educational pathways to computing careers. Although these courses offer invaluable knowledge and skills, several challenges are associated with them beyond the more systemic issues outlined in the previous section. Addressing them effectively is crucial to ensure the success and inclusivity of all students in educational pathways in computing. The US Department of Education reported that the attrition rate for computer science students at CCs is 71.8% (National Center for Education Statistics 2012)

Introduction to Computing courses’ fast pace can be overwhelming for students with no prior coding experience. This can lead to confusion and hinder their understanding of the material. These courses also come with a substantial workload, at least 10 hours per week, including weekly assignments, bi-weekly quizzes, and exams. These assessments are critical for learning but can be overwhelming for students. Access to required software, computers, reliable broadband, and educational platforms can pose additional barriers.

Students attending HBCUs, MSI, and CC’s have limited access to a small number of TAs, if any, due to funding constraints (Bettinger and Baker 2011). To support active learning, many Introduction to Computing courses use autograders, i.e., applications that students can use to run their coding assignments and receive immediate feedback in real time, twenty-four seven (24/7). For instance, applications like Zybooks (used at QCC) also provide students with interactive content, including animations and quizzes, and caters to various learning preferences to enhance student engagement. But the autograders are not without their flaws. Many students find it difficult to understand the feedback and debug highlighted errors.

In the classroom, instructors in Introduction to Computing courses often need to strike a delicate balance between catering to the needs of different student groups. It is not uncommon for high-achieving students to request more advanced exercises. Conversely, students who find the material challenging may seek more examples and clarification during class. Collaborative coding projects are not uncommon and help students learn important teamwork and communication skills. But although working together in principle can enhance the learning experience, it can also lead to uneven learning experiences if teamwork is uneven and a subset of students end up doing most of the work.

2.3. Key Insights

In summary, the challenge for computing programs at MSIs, HBCUs, and CCs is to provide adequate support and flexible learning options to their students, helping them navigate these complexities within and beyond the classroom, without compromising their educational aspirations.

These are the key insights that inform possible pathways to increasing equitable student success in computing:

• Support student learning beyond the classroom in a way that is flexible given students’ often complex school/work schedule. Tools and resources need to provide a good user experience.

• Personalize and adapt to address student learning experiences to meet their needs, to the extent that is appropriate and possible.

• Early prediction of students who are at risk of not meeting learning objectives should be met with timely and relevant support, at the administrative, classroom, and student level.

• Learning should happen within an inclusive and supportive human community. This is not only important for providing helpful guidance and advice, but also socioemotional support. Social isolation is a major risk factor in academic achievement and mental health, particularly for underprivileged students.

• Build student confidence to promote success. Socioemotional challenges facing underserved students can lead them to feel overwhelmed, have low esteem, and have low self-efficacy.

• Support culturally inclusive and diverse learning needs within the classroom.

Each of those insights can be addressed, at least in part, with the help of innovative technology, as examined next.

3. The Impact of Technology on Equitable Learner Success

3.1. Lessons from Educational Technology

Educational technology (EdTech) has the potential to improve learning outcomes for underserved students, but its impact varies. If done poorly, education technology can serve to widen achievement gaps rather than close them. Historically, most EdTech tools, including Massive Open Online Courses (MOOCs), have been designed to support an instructionist pedagogical approach (Granić 2022). Instructionism emphasizes a teacher-directed, passive learning process where knowledge is transmitted to students.

These types of educational technologies suffer from low completion rates due to lack of access to fast internet connectivity, social isolation, cultural misalignment with audiences, and insufficient motivation (Reich 2020). In-person, formal school settings, in contrast, provide the structure and social environment of instructors and peers to support student engagement success. Even primarily online colleges have increasingly recognized that human advisors and coaches are essential to supporting the persistence and success of online learners.

Constructionist approaches to learning, on the other hand, promote active, learner-centered, and socially interactive learning where students construct their understanding through exploration, engagement, and learning-through-making in student-driven projects. Learning about computational thinking and skills, like coding, have been made successful through innovative technologies and platforms like Scratch and MIT App Inventor. These platforms connect millions of learners of all ages within a global community of peers and mentors who design, share, and remix projects. An overarching design value is to offer a highly accessible coding platform for creative expression with a ‘low floor, high ceiling, and wide walls.’

These lessons from EdTech suggest application areas for intelligent tools.

3.2. Intelligent Tools for Equitable Student Success at Educational Institutions

Turning now to an in-person university context, the use of digital tools by students, educators, and administrators have taken root at MSIs, given increased reliable broadband access to the Internet for students and advances in AI. At GSU, for instance, AI coupled with other digital tools have been applied with measurable success to improve equitable education through three main intelligent application areas: administrative tools, student life enhancement tools, and academic engagement tools.

3.2.1 Intelligent Administrative Tools

The strategic application of capturing and using personalized data enhances institutional operations and signifies a pivotal shift at universities from being reactive to proactive in helping students. By anticipating student needs and acting accordingly, AI administrative tools can reduce the chances of minor missteps becoming major obstacles for the students (Page and Gehlbach 2018). Consider the case of how GSU addressed the challenge of ‘summer melt’—without the right guidance, students can easily miss crucial steps in their transition to college and fall between the proverbial cracks to fail to successfully enroll. This problem hits underprivileged or first-generation students the hardest. In response to this, GSU introduced Pounce, an AI chatbot named after their mascot, designed to aid high school graduates who have accepted their offer of admission. Pounce sends tailored text messages to students, helping them with key tasks such as submitting financial aid forms and school transcripts. As another example, GSU was able to proactively distribute about $25 million for student support within 24 hours of receiving CARES Act funding because it had invested in digital infrastructure and predictive analytics to estimate the unmet financial needs of individual students. Students can interact with Pounce 24/7 on their smartphones. As institutions become proactive, so do students. They often take the initiative to address other concerns. Similarly, QCC also uses data analysis and predictive analytics to identify at-risk students and provide timely interventions to support their success such as Starfish from EAB.2 This proactive approach not only elevates the overall administrative performance but also cultivates a sense of trust and positivity among students.

3.2.2. Intelligent Student Life Enhancement Tools

Intelligent student life enhancement tools foster a more equitable education system by, for example, helping guide students in selecting suitable majors and courses that align with their strengths and interests. They can help ensure students excel academically and transition to the workforce. This data-driven approach reduces the need for frequent changes of their majors. This not only enriches their overall educational experience but also minimizes the cost of wasted credit hours, ultimately making higher education more affordable and accessible.

In addition, AI-enabled platforms like Handshake, with their job search capabilities, and tools like VMock, which assist with mock interviews, are critical in providing equitable access to career development opportunities. They level the playing field by offering all students, regardless of their backgrounds, the guidance and support needed to transition effectively into the professional world.

3.2.3. Intelligent Academic Engagement Tools

Intelligent academic engagement tools are designed to offload mundane tasks from course instructors and promote student engagement and success in coursework . A wide variety of intelligent computer tutors or intelligent tutoring systems have been developed over the past several decades (Zawacki-Richter et al. 2019). Inspired by Bloom’s ‘2-sigma effect’ (Bloom 1984) on the effectiveness of personalized tutoring over classroom instruction, these systems attempt to emulate the role of adept human tutors who skillfully adapt to each learner’s unique needs. For instance, Jill Watson is an AI teaching assistant designed at the Georgia Institute of Technology for an online graduate-level computer science class in knowledge-based artificial intelligence (Goel 2018). Although not as skillful as a masterful human tutor, Jill helped to offload common tasks of grading, providing feedback, and answering student questions (Goel and Polepeddi 2018). Jill has enabled faculty to dedicate more time to the crucial and equitable aspects of student engagement. Even when used in conjunction with classroom instruction with teachers, however, their overall effectiveness depends on careful design, content quality, learner engagement, and the specific educational context of use.

3.3. Key Insights

We see similar themes emerge in this and the previous sections about important design considerations for any technology to promote holistic student success. We see examples where AI coupled with other digital tools have been applied with measurable success to improve equitable education through at the administrative, student life, and academic levels.

• Learning is social, and technology should support, not replace, human relationships.

• Learning is personal, and technology should adapt to address student’s particular learning needs to cultivate skill and mastery.

• Students who might benefit most from technological support may not have sufficient access.

• Design needs to align with the cultural and contextual characteristics of the target user community to enhance adoption.

• Data-driven early prediction of students who are at risk of not meeting learning objectives should be met with timely and relevant support: potentially at the administration, classroom, and student level.

• Student outcomes do not just rely on academic learning but also addressing student motivation, self-efficacy, and their connection to a supportive community.

• Quality content across multiple dimensions and pedagogies is necessary for successful application of technology-enabled solutions, especially in learning about computing.

4. Generative AI, the Future of Computing, and Computing Education

Generative AI represents a significant shift in AI’s capabilities, moving from systems that primarily analyze and interpret existing information, to ones that can create new, original content. In this section, we explore the uses of generative AI in education, how it is shaping the computing workforce, and how Introduction to Computing courses may leverage and adapt to these advances.

4.1. Background on Generative AI: Foundation Models

Foundation models in machine learning refer to neural network models trained on large-scale datasets, for example internet-scale text corpora, that can solve general tasks. These are tools whose use is new to general teaching and learning but, like earlier tools, can be expected to have unequal outcomes in both education and industry.

In the language domain, language models (LLMs) can solve a range of tasks specified by user prompts. These foundation models exhibit certain understanding of general text and image data and thus provide an opportunity for efficiently processing diverse types of education materials (including textbooks, lecture notes, recordings, and slides) and student inquiries (mathematical formulae, code blocks, references to homework questions, or even hyperlinks to web pages such as Wikipedia). These models can be fine-tuned with additional datasets in order to perform better in more specialized areas, such as coding, chat, and image description. These models can be powerful, safe, and robust. However, using them in specific domains often requires substantial research and engineering efforts.

Efficient solutions generally involve designing a retrieval mechanism to extract the material relevant to user input, whereas alignment requires additional effort to help ensure reliability of the model outputs; LLMs trained to model uncurated text data from the web may produce inappropriate or hallucinated responses when used naively. Their behavior must be tuned to be aligned with human-defined guidelines, including safety, robustness, correctness, and harmlessness.

A number of companies are developing foundational models with different performance characteristics and different costs associated with accessing and using these models. Costs for these models can range into the hundreds of millions of dollars and, not surprisingly, can be expensive to use and fine-tune. Currently, many foundation models are proprietary and developed by private companies (GPT-4 by OpenAI, Gemini by Google, Claude by Anthropic, etc.). Such models are only generally available via application programming interfaces, also known as APIs, which come with limitations on speed of responsiveness, usage (un)friendliness, significant cumulative financial cost, reliability, and privacy. Despite these limitations, these models (GPT-4, in particular) often provide the best performance and are the go-to choices for many LLM-based solutions developed by and used in education, non-profits, and industry.

An alternative is open-source models. Using the open-source approach of Llama series LLMs from Meta, the developer community has developed ways to efficiently use LLMs in a variety of use cases. While generally not as powerful as proprietary models, these open-source models provide greater flexibility to developers and have an active and growing community. Open-source models can be far more affordable to use and fine-tune.

These cost and performance trade-offs are a key issue for these models’ use in education.

4.2. Generative AI and Education: Promise and Concern

There is broad exploration in applying LLMs across numerous applications, including education by offering innovative ways to enhance learning experiences, assist educators, and personalize education (Klopfer et al. 2024). For instance, generative AI tools are being applied to assist teachers in producing tailored educational content, such as customized worksheets, quizzes, and learning modules. Generative AI can create personalized AI tutors capable of more flexible dialogues with students, informed by the student’s progress and performance. Coursera, for instance, has announced features using generative AI such as personalized job-aligned learning to help learners find the right educational content to build industry-relevant skills.

Technologies like LLM tutors utilize adaptive mechanisms to adjust the complexity and delivery style of content in real time, based on ongoing assessments of users’ learning progress, preferences, behaviors, and needs (Ma et al. 2014; Xu et al. 2013). Moreover, empathetic personalization can focus on responding to learners’ emotional states, offering real-time motivational feedback and support (Ma et al. 2014). Personalization could also include adapting learning experiences to align with students’ socioeconomic backgrounds, personality traits, academic histories, and communication styles.

In computing education, GenAI has brought changes, including the potential of the technology to transform teaching methodologies and learning experiences (Becker et al. 2022; Finnie-Ansley et al. 2022). Harvard’s Introduction to Computer Science MOOC, has recently integrated an AI chatbot tutor to assist online students with learning to code (Liu et al. 2024). Perhaps the best-known next-generation AI tutor is Khanmigo from Khan Academy. Designed to redefine K-12 online education, the product vision is “a tutor for every student, and a teaching assistant for every teacher” offering support in subjects such as math, writing, and computer science. The agent’s scaffolded dialogue is designed to encourage students to actively engage in the learning process rather than simply giving answers—with the goal of guiding students toward solutions, nurturing critical thinking, and developing problem-solving skills (Andriole 2023).

The promise of generative AI and its potential to revolutionize learning and education motivates numerous efforts by companies, nonprofits, and research universities but with some caveats (Denny et al. 2024). Their efficacy on improving learning outcomes across different subjects has yet to be formally assessed at scale, in particular for underserved or under-resourced students. Importantly, the use of generative AI in education has also raised a number of ethical concerns that are important to address:

• Student privacy: The prompts that students ask chatbots are stored by companies and can be used for future training, which comes with the risk of student prompts being unintentionally leaked or used verbatim in chatbot responses.

• Safety: It is hard to predict when generative algorithms will output harmful or inappropriate content. In educational settings, what is considered appropriate depends on age and grade level.

• Cost and access: Access to these models is dependent on access to the Internet and becomes expensive to use proprietary ones. This raises concerns about affordable, equitable access at scale.

• Biased outputs: Although (or because) these models are trained on huge datasets available on the Internet, books, transcribed videos, etc., these data are not yet adequately representative. The outputs of AI can perpetuate and reinforce prejudices and biases.

• Equity: If the datasets are not adequately representative of diverse learners, AI-powered educational tools may favor certain learning approaches or abilities.

• Incorrect answers: Although the generative nature of its algorithmic design supports its flexible, adaptive, and emergent properties of the content it produces, generative AI tools can unpredictably produce incorrect content in the form of ‘hallucinations.’

• Over-reliance: Educators have expressed concerns that overreliance by students may compromise the development of foundational student skills.

• Academic integrity: Students may claim the outputs of generative AI tools as their own original work.

5. Exploring State of the Art: A Comparison of LMM Tutors for Computing

We examined students’ experiences and preferences with two state-of-the-art LLM tutors at GSU, a large urban R1 minority-serving university with a nationally top-ten–ranked Information Systems program (Rai et al., forthcoming). The first tutor was Khanmigo,3 which was developed by Khan Academy and is a refined GPT-4 model that caters to students, teachers, and parents across various subjects including math, physics, and programming (Extance 2023). It is not free, costing $4/month or $44/year and is primarily useful for engaging with Khan Academy lessons and content. It is not limited to computer science. The other LLM tutor was CS50.ai, developed by Harvard University. It is a personal AI assistant for students in Harvard’s introductory computer science course, CS50.4 CS50.ai utilizes retrieval-augmented–generation technology to provide answers based on course lectures and supports programming languages such as C, Python, SQL, JavaScript, CSS, and HTML (Liu et al. 2024).

We designed and ran a focus group study to evaluate the effectiveness of these two popular LLM tutors with students at GSU. Participants were undergraduate students in a mandatory introductory Python programming course. Many students juggle part-time or full-time jobs to manage their tuition and living expenses. The course offered two virtual lab sessions per week led by human tutors, but attendance was low due to scheduling conflicts or concerns about being judged by the tutors.

5.1. Khanmigo vs. CS50.ai

Although both LLM tutors support introduction to programming in Python, they differ in important ways.

Khanmigo leverages a conversational and supportive tutoring style reminiscent of the Socratic method. It also incorporates empathetic expression and offers a choice of personas (e.g., Ada Lovelace, Albert Einstein). Although its interface includes accessibility features like speech-to-text for input and text-to-speech for output, it primarily offers a chat-like experience.

The CS50.ai is integrated into Harvard University’s online CS50 course. CS50.ai is free for the broader educational community. It has been well-received by Harvard students, leading to a reduction in human tutors. In contrast to Khanmigo’s conversation style, CS50.ai is much more direct, opting to provide succinct, complete answers to student queries. It integrates seamlessly with the Harvard Ed5 forum platform and Visual Studio Code integrated development environment (IDE) for real-time student engagement.

Table 1. Types of impacts of generative AI on software systems development.

STT = speech to text, TTS = text to speech.

5.2. Focus Group Participants and Design

Our study involved a diverse group of 60 students at GSU, providing insight into their backgrounds and the challenges they face that may affect their interaction with learning technologies. Of these students, 40.7% were male, and 59.3% were female. Additionally, 61% of the students were eligible for Pell grants—federal grants for undergraduate students with significant financial need. The racial composition of the sample included 54% African American students, 27.1% Asian students, 1.7% American Indian students, and 8.5% White students. Of the students, 18.6% identified as Hispanic. Notably, among the sampled students, 37.1% were non-native English speakers navigating academic challenges in a second language. Furthermore, 63.9% were first-generation college students, possibly facing unique educational and socioeconomic challenges. Employment is a significant factor for our participants: 14 students worked full time, and 28 held part-time jobs. These employment commitments indicate that nearly 70% of the participants are balancing their studies with work.

Four learning objectives, corresponding to lower to higher levels of learning in Bloom’s Taxonomy (Adams 2015, Bloom et al. 1964, Krathwohl 2002), were identified for the Introduction to Computing courses: concept comprehension debugging, quiz preparation, and program development (Starr et al. 2008). Four focus groups, each with 11–21 students, were established. Each group was assigned one of the following learning objectives by interacting with the LLM tutors: concept comprehension, specifically through comprehending the concept of dictionary; debugging, specifically detecting and resolving errors in loops; quiz preparation, specifically how to apply loops to new problem settings; and program development, specifically developing programs with loops. Participants took a pretest to gauge their initial understanding of the topic. They interacted with the tutors in a randomly assigned sequence, completed tasks, and subsequently evaluated their experiences through a survey. This process was repeated for the second tutor. After working with both tutors, participants compared them on their relative advantages, disadvantages, and overall preferences. Finally, a moderated group discussion allowed participants to share their experiences and insights on the effectiveness and impact of the two LLM tutors compared with human tutors.

5.3. Focus Group Insights and Implications

We analyzed responses to open-ended questions and feedback from group discussions to identify students’ likes and dislikes by functions and LLM tutors. The focus groups reveal differences in student preferences and interactions with the LLM tutor based on the learning method of the tutor, the expression of empathy by the tutor, and the prior experience and knowledge of the student.

5.3.1. Pedagogical Style

The focus groups surfaced a sharp contrast between the tutors with respect to the underlying pedagogical style (Socratic vs. didactic).

Khanmigo employs a Socratic method to guide students with questions to encourage critical thinking. For example, Khanmigo might ask, “The code should print ‘Prime’ at the end no matter what, right? So, why might it not be printing anything at all?” This approach fosters deep reflection but can frustrate students seeking quicker solutions. Khanmigo also suggests follow-ups to break down problems into manageable steps, such as ‘how to identify if a character is lowercase’ and ‘how to use islower() in my program.’ These guided prompts seek to help students explore the problem further. Additionally, Khanmigo seeks to boost cognitive engagement with the use of analogies. For instance, instead of explaining a Python dictionary technically, Khanmigo uses an analogy: “A dictionary in Python is like a magical book! Imagine a book where you look up a word (the ‘key’) and find a meaning (the ‘value’). In Python, a dictionary works the same way! Can you think of an example where you might use a dictionary in Python?”

In contrast, CS50.ai employs a didactic method, offering direct answers, identifying specific issues, and guiding students on solving problems, which the majority of the students liked. For example, CS50.ai might respond with, “It seems like you’re trying to print all prime numbers up to a certain number. However, there are a few issues with your code,” followed by specific directions to address the problems. Unlike Khanmigo’s analogical explanations, CS50.ai typically explains concepts in a straightforward, technical manner. For instance, CS50.ai would describe a Python dictionary as “Each item in a dictionary consists of two elements: a key and a value. The key identifies the item, and the value is the data associated with the key.”

5.3.1.1. Student Preferences Based on the LLM Tutor’s Learning Method

The focus groups revealed significant differences in student interaction styles, both within and between tools for the different functions, i.e., concept comprehension, debugging, quiz preparation, and program development. Students who sought to deepen their conceptual understanding, find iterative problem-solving to be engaging, and an empathetic approach to be supportive tended to favor Khanmigo’s method.

In contrast, students who sought quick answers and actionable feedback with technical precision tended to prefer CS50.ai. The analysis of the focus group chat logs revealed that the clarity of CS50.ai’s responses depends significantly on the specificity of the student’s prompts, with more detailed queries receiving more explicit guidance. Some students who do not possess an adequate understanding of the basic concepts struggled with formulating questions for debugging and program development, but those who provided specific prompts received more detailed responses.

5.3.2. Empathetic vs. Factual Approach of the Tutors

As a complement to the Socratic method, the focus groups also revealed that Khanmigo attempts to offer emotional support and encouragement, such as affirmations and motivational comments, during student interactions. As an example, when a student expressed frustration with understanding the concept of a function, Khanmigo responded, “Ah, no worries! A function is like a mini program within a program.” In contrast, CS50.ai uses a factual approach, using a direct and precise information delivery.

5.3.2.1. Student Preferences Based on LLM Tutor’s Expression of Empathy

Although some students appreciated empathy expressed by Khanmigo for enhancing confidence, others found it less credible, especially when praise is given for incorrect efforts. For example, when a student struggled to initiate writing a program correctly, Khanmigo responded, “Nice start!”

CS50.ai’s matter-of-fact style, devoid of expressed empathy, was not seen as a drawback by many students, who value its precision and to-the-point interactions, particularly when efficiency is prioritized over a detailed exploration of challenging concepts. However, some students observed that they found this style to be less engaging.

5.3.3. Differences Based on Prior Experience and Knowledge

Although students with limited prior experience with computing and programming viewed Khanmigo’s Socratic approach as beneficial, students with programming experience, including those exposed to robotics through extracurricular activities, felt it hindered their learning pace. High performers, who scored well on the midterm exam before participating in the focus groups, generally responded positively to tutor guidance, actively implementing corrections and exploring advanced features. Low performers, while generally receptive, often needed repeated clarifications of basic concepts.

5.4. Key Insights

Overall, students value the nonjudgmental nature and access to the two AI tutors. Despite this, dissatisfaction arises with both tutors due to a lack of code visualization, insufficient customization to individual needs, and the absence of integration with IDEs for coding tasks. Importantly, the focus groups revealed technical and functional challenges that frustrate students, distract them from learning, and dampen their motivation.

5.4.1. Technical Challenges

Students reported difficulties due to the lack of integration between the tutors and the IDE used for programming. This absence made transferring content between the tutoring environment and the IDE tedious, impacting efficiency and overall learning. Additionally, the user interface itself posed challenges, with cumbersome navigation further affecting the overall user experience.

5.4.2. Functional Challenges

Two key functional challenges emerged from the focus groups:

1. Response time: Students expressed dissatisfaction with the operational performance of both tutors, particularly noting slow response times. These delays were significant hindrances to learning.

2. User throttling for session control: CS50.ai allocates a finite number of ‘hearts’ that are consumed with each prompt. Khanmigo uses an ‘AI battery,’ indicating consumption level, to limit daily message interactions. Throttling frustrated many students, particularly when hearts or battery capacity were spent due to misunderstandings or misinterpretations by the tutor. The depletion of usage capacity restricted their ability to engage fully with the learning material. A suggestion from the focus groups was to enhance the LLM tutors by allowing students to earn usage capacity for good performance.

In sum, each tutor’s learning method and approach to empathy expression may not benefit all learners, depending on their learning needs and resource constraints. The tutors also are inextricably linked to their specific course materials and thus are not general-purpose tools for AI tutors in computing courses. Technical challenges related to response time and functional challenges arising from lack of IDE integration and constraints on rate of use can frustrate students and adversely impact their motivation and learning. Given the high cost of state-of-the-art LLM models, these challenges underscore the need to anticipate and mitigate the risks that AI tutors can deepen inequity in education. Collectively, the insights inform how to design and evaluate LLM tutors to effectively meet diverse learning objectives and cater to various types of learners in computing education, accommodating a wide range of student needs.

6. Looking Ahead: Generative AI and the Computing Workforce

Higher education prepares students to enter the workforce, and code LLMs and chat LLMs are already changing the process of software development. In this section, we explore how generative AI is impacting computing practice, tools, and skills across different businesses. This workforce perspective informs how computing courses may need to adapt to prepare students for entry level computing positions in the near future.

We interviewed ten software professionals across a range of industries: information technology, food, health, consulting, and others. We asked how generative AI is being used in their businesses and how it might change the skills and mindsets companies look for. See Table 1 for a summary.

Table 2. List of interviewees.

CTO = Chief Technology Officer, UPMC = University of Pittsburgh Medical Center.

As with any technology disruption in the workforce, generative AI will disrupt some software job skills, demand new ones, and raise the importance of others. Based on our interviews, it is unlikely that all of software development will be automated with AI. Rather, the workforce will need to be highly skilled in how to use state-of-the-art AI tools to develop robust, trustworthy, safe, and responsible software solutions. In “The Turing Trap,” Erik Brynjolfsson makes the point that designing AI to augment the human workforce creates new capabilities and new products and services, generating far more value than simply trying to replace what people do (Brynjolfsson 2022). Three ways that generative AI can transform knowledge work include reducing cognitive load by offloading structured and repetitive elements of knowledge work to generative AI tools; boosting cognitive capabilities by enhancing creativity, critical thinking, and knowledge sharing; and aiding in learning through regular feedback and mentoring (Alavi and Westerman 2023). All three methods apply to the important knowledge work of software development.

As software developers harness next-generation AI tools to improve productivity, these tools still require careful human oversight. People need to verify the correctness of code generated by AI to ensure it follows best practices and security guidelines, as well as tailor the AI’s outputs to the specific needs of the task and business needs.

6.1. AI in Pair Programming

AI pair programming is one of the most provocative ways that code LLMs are changing software development. Inspired by pair programming between people, this is a collaborative approach that involves a human developer working alongside an AI-powered coding assistant in an interactive coding environment (e.g., Open AI’s Codex or GitHub’s co-Pilot). As the developer writes code, the AI assistant can help in a number of ways such as providing real-time suggestions, offering code completions, automating testing, generating code snippets, assisting in debugging and documenting code, supporting code review, refactoring code, or even writing functions based on the developer’s input. In cases where the AI tool uses machine learning, continuously training the model with developers’ specific coding styles and preferences can improve its effectiveness over time.

AI pair programming can help enhance software developers’ productivity by offloading routine tasks to the AI. Developers may enhance creative problem solving by leveraging AI tools as a collaborator—posing problems to the AI, evaluating and verifying the candidate solutions it generates, refining them as necessary, and using this back-and-forth to inspire new ideas. In a human-AI team environment, it’s important to maintain open communication about the use of AI in pair programming so everyone understands the role of AI and how to interact with it effectively. Finally, people need to be vigilant to ensure that the use of AI-generated code adheres to the ethical standards of their respective organizations and is consistent with AI policy and regulations. There are also concerns that developers may become too trusting or reliant on LLM coding agents (Pearce et al. 2022).

6.2. How Organizations Are Future-Proofing Their Programming Workforce

In fast-paced industries, companies are investing more in the continued education and training of their employees, especially in software development. How companies engage in workforce education can also inform higher education pathways to computing careers. Beyond learning how to interact with generative AI software development tools, our interviewees agreed that entry-level coders will continue to need the same skills that talented coders currently need. They need enough knowledge of coding and technical environments so that they can understand whether a module’s logic is appropriate for the task and how they can adjust or debug it for the particular use case. They need to know technical environments (e.g., Ruby, Github, front-end and back-end tools, etc.). They will still need to understand agile software development methods and the value of standards and microservice architectures. Broadly speaking, history has shown that as tools develop and evolve in sophistication, coders will still need to understand computational thinking concepts and know how to code.

In addition to technical skills, software developers also need to understand the bigger picture. As agile methods and automated code generation take hold, this may require coders to know more about how business works and what users want and need. They will need to understand the broader problem being solved, their software solution’s responsible design and ethical use, and how to monitor and adjust their deployed solutions to serve the needs of their users and the business. Coders will need to know exactly what problems they are trying to solve and then how to pull the right information in the right way to assemble the right solution. They will need the capability to ‘work backward from the solution,’ rather than working forward on a prescribed set of tasks.

Because technical environments vary across companies and technologies change so frequently, companies are looking to hire people who can learn and adapt quickly. Finally, because coders will typically work in teams, they need the human skills (Stump 2020) to work effectively with others in an organization.

Below are some recommendations, which many companies are already using for their workforce in response to AI advancements.

6.2.1. Upskilling and Reskilling

Invest in comprehensive upskilling and reskilling programs to enable current employees to adapt to new technologies and methodologies in AI. This includes:

• Advanced technical skills: Develop advanced technical skills in AI, machine learning, and data science to ensure the workforce is equipped to handle future technological challenges. Promote specialization in niche areas of AI and machine learning to stay competitive and innovative in specific sectors.

• Problem-solving skills: Develop problem-solving skills with a focus on leveraging AI tools and techniques.

• Experimentation and innovation: Promote a culture of experimentation and innovation by encouraging employees to explore new ideas and approaches in AI.

• Collaboration skills: Develop collaboration skills to work in multidisciplinary AI projects, ensuring diverse perspectives and expertise.

• Business skills: Understand how business works in general, as well as particular functions the person is supporting, will make them more valuable in the short term and provide greater opportunity for career growth.

6.2.2. Ethical and Human-Centric Practices

Invest in programs that help employees develop, deploy, monitor, and evaluate responsible human-centered solutions.

• Ethics and bias training: Implement ethics and bias training to ensure the development of fair, responsible, and unbiased AI systems.

• Human-centric skills: Emphasize the importance of human-centric skills such as empathy and emotional intelligence in the AI development process.

• Human-AI augmentation: Educate employees on viewing AI as a tool that augments human capabilities, rather than a replacement for human roles.

• Regulatory and legal knowledge: Provide training in regulatory and legal aspects of AI to ensure compliance and informed decision-making in AI applications.

6.2.3. Organizational Learning and Culture of Adaptability

• Continuous learning: Support ongoing education and professional development in AI and related fields.

• Hiring for adaptability and AI-readiness: Prioritize adaptability and AI-readiness in the hiring process.

• Implementing AI governance: Establish/evolve a framework for AI governance to guide responsible use and management of AI technologies within the organization.

6.3. Key Recommendations

Because generative AI is already impacting professional software development and will shape the skills and knowledge needed to be successful in the computing workforce, it holds the potential to transform computing education. In this landscape, the responsible application of generative AI in computing education should be carefully considered and evaluated to achieve educational equity and professional opportunity.

• Introduction to Computing courses should adapt tools, content, and material to help put students on a solid path to computing careers where the ability to leverage generative AI adeptly and responsibly will be valuable.

• The fundamentals that students learn in computing courses are still necessary and valued by industry. Generative AI is not expected to replace human coders but rather augment them.

• Human-centered skills—teamwork, curiosity, critical thinking, lifelong learning, creativity, problem solving, and ethics—remain important to cultivate.

• Students should gain the skills to see the bigger picture, to ‘think back from the solution we need’ rather than following a prescribed set of steps or specs.

• A knowledge of business can help coders—even at entry level—understand the context that they are serving to communicate about problems being solved and suggest new solutions.

7. Conclusion

Higher education has long had paradoxical outcomes, both for its students and for the industries it looks to supply with a thriving workforce. It’s possible for practices that provide opportunities to also limit them. Holding a job while studying, for example, can make college more affordable but, when too large a burden, may also make it difficult to complete a degree on time—and thus it becomes more expensive to finish and more difficult for employers to find workers.

As the focus groups in this whitepaper demonstrate, the use of generative AI in education will not avoid these same contradictions without careful consideration of the tools’ designs, access, use cases, and pedagogical adaptations. Likewise, the interviews with employers show they are optimistic about generative AI’s potential to help employees be more productive and collaborative but unsure about its implications for everything from biased content to whether entry-level coders will enter the workforce with traditional competencies. Together these conversations and resulting recommendations should help others lay equitable educational pathways in computing.

Acknowledgments

This work was supported by an internal grant from MIT and an award from Axim Collaborative titled “Defining a Technical/Pedagogical/Evaluation Roadmap with Multi-Stakeholder Perspectives for GenAI Tutors for Equitable Student Success in Post-Secondary Computing Education.” There were many people who participated in workshops, regular meetings, and extended conversations to shape the ideas and work reported in this paper. We gratefully acknowledge Sharifa Alghowinem, Safinah Ali, Jacob Andreas, Ana Bell, Gargi Chug, Alex Gu, Sheng Huang, Phillip Isola, Yoon Kim, Don Lee, Parker Malachowsky, Timothy Rennick, Alexis Ross, Philipp Schmidt, Shannon Shen, and Tongzhou Wang.

Additional Reading

Hamilton, Scott, and Norman L. Chervany. 1981. “Evaluating Information System Effectiveness - Part I: Comparing Evaluation Approaches.” MIS Quarterly 5, no. 3 (September): 55–69.

Luckin, Rose, Wayne Holmes, Mark Griffiths, and Laurie B. Forcier. 2016. Intelligence Unleashed: An Argument for AI in Education. London: Pearson.

Sarma, Sanjay, and Aikaterini Bagiati. 2022. “Current Innovations in STEM Education and Equity Needs for the Future.” In Imagining the Future of Undergraduate STEM Education Symposium: Proceedings of a Virtual Symposium, 50–69. Washington, DC: The National Academies Press. https://www.nationalacademies.org/event/10-21-2020/docs/D56046592B2BE49E08E498462F8DF93DA3808D65290C.

Steele, C. M. 1997. “A Threat in the Air: How Stereotypes Shape Intellectual Identity and Performance.” American Psychologist 52, no. 6 (June): 613–629.

Xie, Haoran, Hui-Chun Chu, Gwo-Jen Hwang, and Chun-Chieh Wang. 2019. “Trends and Development in Technology-Enhanced Adaptive/Personalized Learning: A Systematic Review of Journal Publications from 2007 to 2017.” Computers and Education 140:103599.