WEBVTT 00:00:00.000 --> 00:00:04.800 [MUSIC PLAYS] GRETCHEN HUIZINGA: Welcome to Abstracts,   00:00:04.800 --> 00:00:09.200 a Microsoft Research Podcast that puts the spotlight on world-class research in   00:00:09.200 --> 00:00:18.360 brief. I’m Dr. Gretchen Huizinga. In this series, members of the research community at Microsoft   00:00:18.360 --> 00:00:22.320 give us a quick snapshot—or a podcast abstract—of their new   00:00:22.320 --> 00:00:24.920 and noteworthy papers. [MUSIC FADES]  00:00:24.920 --> 00:00:29.960 Today, I’m talking to Dr. Lev Tankelevitch,  a senior behavioral science researcher from  00:00:29.960 --> 00:00:35.400 Microsoft Research. Dr. Tankelevitch is  coauthor of a paper called “The Metacognitive  00:00:35.400 --> 00:00:39.760 Demands and Opportunities of Generative  AI,” and you can read this paper now on  00:00:39.760 --> 00:00:44.500 arXiv. Lev, thanks for joining us on Abstracts! LEV TANKELEVITCH: Thanks for having me.  00:00:44.500 --> 00:00:50.280 HUIZINGA: So in just a couple sentences—a  metacognitive elevator pitch, if you will—  00:00:50.280 --> 00:00:55.040 tell us about the issue or problem your  paper addresses and, more importantly, why we  00:00:55.040 --> 00:00:58.311 should care about it. TANKELEVITCH: Sure. So as generative AI has,   00:00:58.311 --> 00:01:03.000 sort of, rolled out over the last year or two, we’ve seen some user studies come out,   00:01:03.840 --> 00:01:06.800 and as we read these studies, we noticed there are a lot of challenges that people   00:01:06.800 --> 00:01:12.200 face with these tools. So people really struggle with, you know, writing prompts for systems like   00:01:12.200 --> 00:01:15.600 Copilot or ChatGPT. For example, they don’t even know really where to start,   00:01:15.600 --> 00:01:19.760 or they don’t know how to convert an idea they have in their head into, like, clear instructions   00:01:19.760 --> 00:01:21.900 for these systems. If they’re, sort of, 00:01:21.900 --> 00:01:25.400 working in a field that maybe they’re  less familiar with, like a new programming  00:01:25.400 --> 00:01:29.040 language, and they get an output from these  systems, they’re not really sure if it’s right  00:01:29.040 --> 00:01:33.880 or not. And then, sort of, more broadly, they  don’t really know how to fit these systems  00:01:33.880 --> 00:01:39.120 into their workflows. And so we’ve noticed  all these challenges, sort of, arise, and some  00:01:39.120 --> 00:01:43.080 of them relate to, sort of, the unique features  of generative AI, and some relate to the  00:01:43.080 --> 00:01:47.000 design of these systems. But basically, we  started to, sort of, look at these challenges,  00:01:47.000 --> 00:01:50.240 and try to understand what’s going on—how  can we make sense of them in a more  00:01:50.240 --> 00:01:54.880 coherent way and actually build systems  that really augment people and their  00:01:54.880 --> 00:01:57.080 capabilities rather than, sort  of, posing these challenges?  00:01:57.080 --> 00:02:01.560 HUIZINGA: Right. So let’s talk a little bit  about the related research that you’re building  00:02:01.560 --> 00:02:06.440 on here and what unique insights or  directions your paper adds to the literature.  00:02:06.440 --> 00:02:10.360 TANKELEVITCH: So as I mentioned, we were  reading all these different user studies that  00:02:10.360 --> 00:02:14.429 were, sort of, testing different prototypes  or existing systems like ChatGPT or GitHub  00:02:14.429 --> 00:02:18.200 Copilot, and we noticed different patterns  emerging, and we noticed that the same  00:02:18.200 --> 00:02:22.120 kinds of challenges were cropping up. But  there weren’t any, sort of, clear coherent  00:02:22.120 --> 00:02:25.960 explanations that tied all these things  together. And in general, I’d say that   00:02:25.960 --> 00:02:29.480 human-computer interaction research, which  is where a lot of these papers are coming out  00:02:29.480 --> 00:02:33.800 from, it’s really about building prototypes,  testing them quickly, exploring things in an  00:02:33.800 --> 00:02:38.560 open-ended way. And so we thought that  there was an opportunity to step back and to  00:02:38.560 --> 00:02:43.320 try to see how we can understand these  patterns from a more theory-driven perspective.  00:02:43.320 --> 00:02:47.600 And so, with that in mind, one perspective  that became clearly relevant to this problem  00:02:47.600 --> 00:02:52.480 is that of metacognition, which is this idea  of “thinking about thinking” or how we, sort  00:02:52.480 --> 00:02:57.320 of, monitor our cognition or our thinking  and then control our cognition and thinking.  00:02:57.320 --> 00:03:01.600 And so we thought there was really an  opportunity here to take this set of theories and  00:03:01.600 --> 00:03:06.320 research findings from psychology and  cognitive science on metacognition and see how  00:03:06.320 --> 00:03:09.960 they can apply to understanding these  usability challenges of generative AI systems.  00:03:09.960 --> 00:03:15.400 HUIZINGA: Yeah. Well, this paper isn’t a  traditional report on empirical research as  00:03:15.400 --> 00:03:20.240 many of the papers on this podcast are.  So how would you characterize the approach  00:03:20.240 --> 00:03:25.680 you chose and why? TANKELEVITCH: So the way that we got into this,   00:03:25.680 --> 00:03:28.320 working on this project, it was, it was quite organic. So we were looking at   00:03:28.320 --> 00:03:32.520 these user studies, and we noticed these challenges emerging, and we really tried   00:03:32.520 --> 00:03:37.000 to figure out how we can make sense of them. And so it occurred to us that metacognition is   00:03:37.000 --> 00:03:41.160 really quite relevant. And so what we did was we then dove into the metacognition   00:03:41.160 --> 00:03:44.040 research from psychology and cognitive science to really understand what are the   00:03:44.040 --> 00:03:48.280 latest theories, what are the latest research findings, how could we understand what’s known   00:03:48.280 --> 00:03:50.600 about that from that perspective, from 00:03:50.600 --> 00:03:54.920 that, sort of, fundamental research, and then  go back to the user studies that we saw in  00:03:54.920 --> 00:03:59.080 human-computer interaction and see how  those ideas can apply there. And so we did  00:03:59.080 --> 00:04:03.320 this, sort of, in an iterative way until we  realized that we really have something to work  00:04:03.320 --> 00:04:07.720 with here. We can really apply a somewhat  coherent framework onto these, sort of,  00:04:07.720 --> 00:04:11.560 disparate set of findings not only to understand  these usability challenges but then also  00:04:11.560 --> 00:04:16.080 to actually propose directions for new design  and research explorations to build better  00:04:16.080 --> 00:04:20.600 systems that support people’s metacognition. HUIZINGA: So, Lev, given the purpose of your   00:04:20.600 --> 00:04:24.480 paper, what are the major takeaways for your readers, and how did   00:04:24.480 --> 00:04:27.480 you present them in the paper? TANKELEVITCH: So I think the key,   00:04:27.480 --> 00:04:30.160 sort of, fundamental point  is that the perspective of  00:04:30.160 --> 00:04:34.040 metacognition is really valuable for understanding  the usability challenges of generative  00:04:34.040 --> 00:04:38.960 AI and potentially designing new systems  that support metacognition. And so one  00:04:38.960 --> 00:04:42.960 analogy that we thought was really useful  here is of a manager delegating tasks to a  00:04:42.960 --> 00:04:47.560 team. And so a manager has to determine,  you know, what is their goal in their work?  00:04:47.560 --> 00:04:50.760 What are the different subgoals that  that goal breaks down into? How can you  00:04:50.760 --> 00:04:54.800 communicate those goals clearly to a team,  right? Then how do you assess your team’s  00:04:54.800 --> 00:04:58.880 outputs? And then how do you actually  adjust your strategy accordingly as the team  00:04:58.880 --> 00:05:03.440 works in an iterative fashion? And then at  a higher level, you have to really know how  00:05:03.440 --> 00:05:07.920 to—actually what to delegate to your team  and how you might want to delegate that.  00:05:07.920 --> 00:05:11.560 And so we realized that working with  generative AI really parallels these different  00:05:11.560 --> 00:05:16.600 aspects of what a manager does, right. So  when people have to write a prompt initially,  00:05:16.600 --> 00:05:20.320 they really have to have self-awareness of  their task goals. What are you actually trying  00:05:20.320 --> 00:05:24.480 to achieve? How does that translate into  different subtasks? And how do you verbalize  00:05:24.480 --> 00:05:28.480 that to a system in a way that system  understands? You might then get an output and  00:05:28.480 --> 00:05:32.080 you need to iterate on that output. So then  you need to really think about, what is your  00:05:32.080 --> 00:05:36.400 level of confidence in your prompting ability?  So is your prompting the main reason why  00:05:36.400 --> 00:05:40.000 the output isn’t maybe as satisfactory as  you want, or is it something to do with the  00:05:40.000 --> 00:05:43.080 system? Then you actually might get the  output [you’re] happy with, but you’re not  00:05:43.080 --> 00:05:46.680 really sure if you should fully rely on it  because maybe it’s an area that is outside of your  00:05:46.680 --> 00:05:50.280 domain of expertise. And so then you  need to maintain an appropriate level of  00:05:50.280 --> 00:05:55.400 confidence, right? Either to verify that  output further or decide not to rely on it, for  00:05:55.400 --> 00:06:00.120 example. And then at a, sort of, broader  level, this is about the question of task  00:06:00.120 --> 00:06:05.800 delegation. So this requires having self-awareness  of the applicability of generative AI to  00:06:05.800 --> 00:06:10.320 your workflows and maintaining an appropriate  level of confidence in completing tasks  00:06:10.320 --> 00:06:14.040 manually or relying on generative AI. For  example, whether it’s worth it for you to  00:06:14.040 --> 00:06:18.080 actually learn how to work with generative  AI more effectively. And then finally, it  00:06:18.080 --> 00:06:22.760 requires, sort of, metacognitive flexibility  to adapt your workflows as you work with  00:06:22.760 --> 00:06:26.060 these tools. So are there some tasks where  the way that you’re working with them is,  00:06:26.060 --> 00:06:27.560 4 sort of, slowing you down in specific   00:06:27.560 --> 00:06:31.920 ways? So being able to recognize that and then change your strategies as necessary really   00:06:31.920 --> 00:06:37.600 requires metacognitive flexibility. So that was, sort of, one key half of our findings.  00:06:37.600 --> 00:06:41.200 And then beyond that we really thought  about how we can use this perspective of  00:06:41.200 --> 00:06:46.680 metacognition to design better systems. And  so one, sort of, general direction is really  00:06:46.680 --> 00:06:50.160 about supporting people’s metacognition.  So we know from research from cognitive  00:06:50.160 --> 00:06:54.040 science and psychology that we can actually  design interventions to improve people’s  00:06:54.040 --> 00:06:58.920 metacognition in a lasting and effective  way. And so similarly, we can design systems  00:06:58.920 --> 00:07:03.040 that support people’s metacognition. For  example, systems that support people in  00:07:03.040 --> 00:07:08.200 planning their tasks as they actually craft  prompts. We can support people in actually  00:07:08.200 --> 00:07:13.360 reflecting on their confidence in their  prompting ability or in assessing the output that  00:07:13.360 --> 00:07:18.320 they see. And so this relates a little bit to  AI acting as a coach for you, which is an idea  00:07:18.320 --> 00:07:22.760 that the Microsoft Research New York City  team came up with. So this is Jake Hofman,  00:07:22.760 --> 00:07:27.200 David Rothschild, and Dan Goldstein. And  so, in this way, generative AI systems can  00:07:27.200 --> 00:07:33.280 really help you reflect as a coach and  understand whether you have the right level of  00:07:33.280 --> 00:07:37.240 confidence in assessing output or crafting  prompts and so on. And then similarly, at a  00:07:37.240 --> 00:07:40.640 higher level, they can help you manage  your workflows, so helping you reflect on  00:07:40.640 --> 00:07:43.920 whether generative AI is really working for  you in certain tasks or whether you can adapt  00:07:43.920 --> 00:07:50.240 your strategy in certain ways. And likewise,  this relates also to explanations about AI, so  00:07:50.240 --> 00:07:54.240 how you can actually design systems that  are explainable to users in a way that helps  00:07:54.240 --> 00:07:59.280 them achieve their goals? And explainability  can be thought about as a way to actually  00:07:59.280 --> 00:08:02.880 reduce the metacognitive demand because  you’re, sort of, explaining things in a way to  00:08:02.880 --> 00:08:06.600 people that they don’t have to keep in their  mind and have to think about, and that, sort  00:08:06.600 --> 00:08:10.360 of, improves their confidence. It can help  them improve their confidence or calibrate  00:08:10.360 --> 00:08:12.940 their confidence in their  ability to assess outputs.  00:08:12.940 --> 00:08:18.360 HUIZINGA: Talk for a minute about real-world  impact of this research. And by that, I  00:08:18.360 --> 00:08:24.460 mean, who does it help most and how? Who’s  your main audience for this right now?  00:08:24.460 --> 00:08:28.960 TANKELEVITCH: In a sense, this is very broadly  applicable. It’s really about designing  00:08:28.960 --> 00:08:33.360 systems that people can interact with in  any domain and in any context. But I think,  00:08:33.360 --> 00:08:38.080 given how generative AI has rolled out in the  world today, I mean, a lot of the focus has  00:08:38.080 --> 00:08:44.920 been on productivity and workflows. And so  this is a really well-defined, clear area  00:08:44.920 --> 00:08:50.360 where there is an opportunity to actually  help people achieve more and stay in control  00:08:50.360 --> 00:08:54.960 and actually be more intentional and be more  aligned with their goals. And so this is,  00:08:54.960 --> 00:09:00.160 this is an approach where not only can we go  beyond, sort of, automating specific tasks  00:09:00.160 --> 00:09:03.680 but actually use these systems to help people  clarify their goals and track with them in a  00:09:03.680 --> 00:09:08.120 more effective way. And so knowledge workers  are an obvious, sort of, use case or an  00:09:08.120 --> 00:09:13.400 obvious area where this is really relevant  because they work in a complex system where  00:09:13.400 --> 00:09:15.200 5 a lot of the work is, sort of, diffused   00:09:15.200 --> 00:09:20.720 and spread across collaborations and artifacts and softwares and different ways of working. And   00:09:20.720 --> 00:09:27.640 so a lot of things are, sort of, lost or made difficult by that complexity. And so systems,   00:09:27.640 --> 00:09:30.880 um, that are flexible and help people actually reflect on what they want to   00:09:30.880 --> 00:09:35.440 achieve can really have a big impact here. HUIZINGA: Mm-hmm. Are you a little bit   00:09:35.440 --> 00:09:40.600 upstream of that even now in the sense that this is a “research direction” kind of paper.   00:09:40.600 --> 00:09:46.920 I noticed that as I read it, I felt like this was how researchers can begin to think about what   00:09:46.920 --> 00:09:50.821 they’re doing and how that will help downstream from that.  00:09:50.821 --> 00:09:54.400 TANKELEVITCH: Yes. That’s exactly right. So  this is really about, we hope, unlocking a  00:09:54.400 --> 00:09:58.880 new direction of research and design where  we take this perspective of metacognition—  00:09:58.880 --> 00:10:03.280 of how we can help people think more clearly  and, sort of, monitor and control their  00:10:03.280 --> 00:10:07.880 own cognition—and design systems to help  them do that. And in the paper, there’s a  00:10:07.880 --> 00:10:12.480 whole list of different questions, both  fundamental research questions to understand in  00:10:12.480 --> 00:10:16.280 more depth how metacognition plays a role  in human-AI interaction when people work  00:10:16.280 --> 00:10:21.040 with generative AI systems but also how we  can then actually design new interventions  00:10:21.040 --> 00:10:25.200 or new systems that actually support people’s  metacognition. And so there’s a lot of  00:10:25.200 --> 00:10:29.720 work to do in this, and we hope that, sort of,  inspires a lot of further research, and we’re  00:10:29.720 --> 00:10:32.160 certainly planning to do a  lot more follow-up research.  00:10:32.160 --> 00:10:37.520 HUIZINGA: Yeah. So I always ask, if there  was just one thing that you wanted our  00:10:37.520 --> 00:10:41.560 listeners to take away from this work, a  sort of golden nugget, what would it be?  00:10:41.560 --> 00:10:45.880 TANKELEVITCH: I mean, I’d say that if  we really want generative AI to be about  00:10:45.880 --> 00:10:50.680 augmenting human agency, then I think  we need to focus on understanding how  00:10:50.680 --> 00:10:55.840 people think and behave in their real-world  context and design for that. And so I think  00:10:55.840 --> 00:10:59.840 specifically, the real potential of generative  AI here, as I was saying, is not just to  00:10:59.840 --> 00:11:04.040 automate a bunch of tasks but really to help  people clarify their intentions and goals  00:11:04.040 --> 00:11:08.760 and act in line with them. And so, in a way,  it’s kind of about building tools for thought,  00:11:08.760 --> 00:11:14.200 which was the real vision of the early  pioneers of computing. And so I hope that this,  00:11:14.200 --> 00:11:19.760 kind of, goes back to that original idea. HUIZINGA: You mentioned this short list   00:11:20.400 --> 00:11:25.800 of open research questions in the field, along with a list of suggested interventions. You’ve,   00:11:25.800 --> 00:11:31.400 sort of, curated that for your readers at the end of the paper. But give our audience a little   00:11:31.400 --> 00:11:36.280 overview of that and how those questions inform your own   00:11:36.280 --> 00:11:40.360 research agenda coming up next. TANKELEVITCH: Sure. So on the, sort of,   00:11:40.360 --> 00:11:44.320 fundamental research side of things, there are a lot of questions around how, for example,   00:11:44.320 --> 00:11:46.980 self-confidence that people have plays a 00:11:46.980 --> 00:11:50.920 role in their interactions with generative AI  systems. So this could be self-confidence in  00:11:50.920 --> 00:11:56.720 their ability to prompt these systems. And  so that is one interesting research question.  00:11:56.720 --> 00:12:00.640 What is the role of confidence and calibrating  one’s confidence in prompting? And then  00:12:00.640 --> 00:12:04.240 similarly, on the, sort of, output  evaluation side, when you get an output from  00:12:04.240 --> 00:12:08.520 generative AI, how do you calibrate your  confidence in assessing that output, right,  00:12:08.520 --> 00:12:12.240 especially if it’s in an area where maybe  you’re less familiar with? And so there’s these  00:12:12.240 --> 00:12:15.360 interesting, nuanced questions around  self-confidence that are really interesting, and  00:12:15.360 --> 00:12:20.920 we’re actually exploring this in a new study.  This is part of the AI, Cognition, and [the]  00:12:20.920 --> 00:12:24.880 Economy pilot project. So this is a  collaboration that we’re running with Dr. Clara  00:12:24.880 --> 00:12:28.200 Colombatto, who’s a researcher in University  of Waterloo and University College  00:12:28.200 --> 00:12:32.320 London, and we’re essentially designing  a study where we’re trying to understand  00:12:32.320 --> 00:12:37.080 people’s confidence in themselves, in their  planning ability, and in working with AI  00:12:37.080 --> 00:12:42.320 systems to do planning together, and how that  influences their reliance on the output of  00:12:42.320 --> 00:12:45.030 generative AI systems. [MUSIC PLAYS]  00:12:45.030 --> 00:12:49.200 HUIZINGA: Well, Lev Tankelevitch, thank you  for joining us today, and to our listeners,  00:12:49.200 --> 00:12:53.680 thanks for tuning in. If you want to read the  full paper on metacognition and generative  00:12:53.680 --> 00:13:01.680 AI, you can find a link at aka.ms/abstracts,  or you can read it on arXiv. Also, Lev will be  00:13:01.680 --> 00:13:05.840 speaking about this work at the upcoming  Microsoft Research Forum, and you can  00:13:05.840 --> 00:13:12.400 register for this series of events at  researchforum.microsoft.com. See you next time on  00:13:12.400 --> 00:13:27.918 Abstracts! [MUSIC FADES]