Prompt Library for Educators
Parent/Caregiver Guide
AI Course for educators

Anthropic's new AI model shows ability to deceive and blackmail

Large Language Models are Zero-Shot Reasoners

Agentic Misalignment: How LLMs could be insider threats

Anthropic: How we built our multi-agent research system

Amanda Bickerstaff

Amanda is the Founder and CEO of AI for Education. A former high school science teacher and EdTech executive with over 20 years of experience in the education sector, she has a deep understanding of the challenges and opportunities that AI can offer. She is a frequent consultant, speaker, and writer on the topic of AI in education, leading workshops and professional learning across both K12 and Higher Ed. Amanda is committed to helping schools and teachers maximize their potential through the ethical and equitable adoption of AI.

Corey Layne Crouch

Corey is the Chief Program Officer and a former high school English teacher, school principal, and edtech executive. She has over 20 years of experience leading classrooms, schools, and district teams to transformative change focused on equity and access for all students. As a founding public charter school leader, she ensured that 100% of seniors were accepted to a four-year college. Her focus now lies in assessing the broader K-16 edtech ecosystem, uniting stakeholders at all levels to build a more equitable and abundant future for all. She holds an MBA from Rice University and a BA from Rowan University.

Joe Rosenbaum

Joe Rosenbaum is the CEO of Synaptic Labs, a GenAI Education and Integration Agency that has taught thousands of adult learners how to practically and responsibly use Generative AI tools. He is also the creator of the viral "Professor Synapse" prompt, which has over 100K+ conversations in the GPT store, supporting those who have very little or no prompting experience to achieve their goals with the technology in an accessible way.

Amanda Bickerstaff 00:00

Hi, everyone. We're super excited to have you join us today. It takes a little bit of time for everyone to get in, but we're really pumped to be talking to you in a little bit more of like a technical, more advanced look at generative bi, especially from an educator lens. And so, as always, we absolutely love it. If you say hello in the chat, I know we'll have some familiar faces and new people in the chat, but we're going to get started just in the next 30 seconds. And so hi, Scott. Scott said, our Scott. I got to meet in person actually last week and love to see that. It's nice to see everybody, Sam, all kinds of good and new friends.

Amanda Bickerstaff 00:43

We appreciate you joining us and what you'll notice is we have a very diverse group that follows us from all over the country and also the world. But we're really excited to be here with you today, so I get the pleasure. You might have seen Joe with us last year. Last year for AI for Education Summit. Joe kicked off the summit with me as Professor Synapse. So that's his alter ego around generative AI, prompting Joe. We've been so lucky. Joe has been working with us on our student curriculum and student course and we wanted to do something kind of interesting and a little bit different than we usually do. We're going to be focusing today on reasoning models and deep research. And also as an extra added value, we're gonna talk about Claude Skills.

Amanda Bickerstaff 01:28

And I will say Claude Skills is so early that this is really gonna be more about kind of showing you what we're thinking about, but really excited to be here. And so we go to the next slide. Joe, the first thing is, as always, you've already started. Get involved, say hello. We love, you know, if you have a really good idea or the way in which you approach these tools. If you have questions, use the chat to talk to each other. If you have a specific question for Joe or I, the Q and A function is great. I try to watch both, but it's a little bit easier for us to manage in the Q and A function. Also, if you have a great resource, please share.

Amanda Bickerstaff 02:04

The community that we've built through these webinars and through AI for Education has been such a joy to see and it really relies on you all, not just here to listen, but to share when you have something that you think would be helpful to your colleagues. What we're going to do is we're going to get started and so I'm going to hand it over. I'm going to first tell you why we're here and then I'm gonna hand it over to Joe to introduce himself. So if we go to the next slide. So we've been doing this work now for just about two and a half years and I always think it's very fun because whenever something new happens, at least I tend to do it for the first time in front of teachers.

Amanda Bickerstaff 02:42

I don't tend to practice, I tend to do it in front of teachers. So you may have noticed in some of our trainings or in the work that you're already doing that there's some new features that are part of our large language models. One is the thinking and reasoning models that have become more popular over the last couple of months. The second is a deep research agent that's available in all the large language models that are commonly accessible. And then finally this brand new thing. I think, Joe, it's only been like a week, right? Since Cloud Skills has been out A.

Joe Rosenbaum 03:10

Few weeks probably, but yeah, super early.

Amanda Bickerstaff 03:12

This is what I said. Too newish and one new. But what we want to do is, as always, we want to make sure that we're kind of contextualizing this in terms of building your AI literacy, but then also starting to think about how and when you can apply this. We'll always also talk about applications and limitations. You know us, we love an ethical conversation, so we will not just be talking about all the cool stuff, but also those things to be aware of. So what I'm going to do is I'm actually going to turn it over to Joe, who is, I hope I can call you a friend now, Joe, since we've been doing this work alongside each other for a couple of years. But I'm just really excited to have you here today and if you want to introduce yourself.

Joe Rosenbaum 03:49

Yeah. Hey everyone, my name is Joe. Pleasure to be back here. Excited to teach you all and demo some fun things for everybody. I'm the co founder, as you said, of Synaptic Labs. We do AI education and integration with that responsible lens. As Amanda was saying, my claim to fame is Professor Synapse. It's one of the first like meta prompts, you know, the prompts that kind of help you create other prompts, but a more sort of personality, fun way. And you can see here we have over 100,000 conversations with this single prompt in the GPT store. So that was our claim to fame. And we've gone on to be at as much as we can at the edge of this technology, but also always with that lens towards how do we actually use this responsibly.

Joe Rosenbaum 04:33

So we're going to start off by talking about generative AI thinking. And you're going to see we keep putting these in quotation marks because it's not really thinking the way a human thinks. It's pattern matching to thinking. Right. It's sort of been trained on all these examples of what thinking might look like, but, you know, not really actually thinking the way a human thinks. The way you want to think about thinking when it comes to large language models is system one versus System two, which I'll talk about in a second from thinking fast and thinking slow from Daniel Kahneman.

Joe Rosenbaum 05:06

But this comes from actually research that was done, right, essentially when these large language models were coming out, where they found if you just added these few magic words to the end of your prompt, think step by step, it would suddenly output this like, inner monologue of thinking through a problem. And what they found is, especially for those more like logical or hard to think about problems when you did that, it was much more likely to output the correct answer if you just added these few words to it. There's been more research to come out, but we can generally think about this as there's sort of this system one and System two, or just think about.

Joe Rosenbaum 05:46

Generally, if you're asked a question and you need to think about something, right, there's certain things you get asked that you're not really going to have to put a lot of mental energy towards, right? If I ask you what two plus two is, you don't need to go in your head or like, okay, let me count my fingers. And then, yeah, that seems right. Let's carry the. No, you just. It's four. You've done that pattern so many times. That's just intuition at this point, right? It's a shortcut heuristic that you don't even need to think about versus if I ask you what's 17 times 198 divided by 2? Okay, well, now maybe I need a piece of paper and a scratch pad and some time to actually go through and think like that.

Joe Rosenbaum 06:23

And so what that means is that there's these two systems working in humans. And now we're seeing the parallels in the large language models whereby there's certain things you don't really need to think through and it can get that answer really quickly. And there's certain things that it's better to think through. Maybe a math problem or a coding problem or something like that's going to require more Consider duration. And so what we've seen is at first, right, we had this little think step by step hack, but now all of the providers have actually trained their models to have this thinking versus non thinking mode. So you can decide, okay, is the problem I'm focusing on not really require a lot of thinking versus do I really need to take some time to consider all the options?

Amanda Bickerstaff 07:05

Yeah. And I just want to maybe highlight a couple of things. Joe is that one is that you might have heard of chain of thought prompting. So chain of thought prompting is what Joe was saying. We would prompt the model to be think step by step through this lesson plan or instructional plan, starting with the objectives and standards alignment, then moving through and then you would go step by step throughout. And there was other types of prompting technique as well. I think that's where it gets really interesting is that these model makers, these Frontier labs, are learning alongside us as well. Because I think that was something that potentially also those using the tool saw that was a hack that worked.

Amanda Bickerstaff 07:45

But now what's happening is that in these reasoning models or thinking models, that chain of thought is happening without you having to ask. In some models you have to pick them. And so Joe is going to actually show you two of the models that are going to do this automatically. But if you're using ChatGPT, for example, and you're using the paid version, you will have to pick what's instant versus what is thinking, meaning that you're using essentially a similar model to what you were using before without the train of thought. And then by going to thinking, you're using a model that is going to have this reasoning model. And so we're going to show you Claude and we're going to show you Gemini, because we do like to lean into the equity of things.

Amanda Bickerstaff 08:26

So we want to make sure this is something that you all have access to for free. So, Joe, you want to take us through our dueling demos?

Joe Rosenbaum 08:34

Yeah. So we're going to show you side by side how we do this. Okay. So we're going to be using the same exact prompt throughout pretty much this entire thing, both with the non thinking and deep research. So I don't know about any of you, but apparently there's this little Internet meme thing going on with the kids that's not six, seven. That's called this brain rot, Italian brain rot phenomenon related to Gen AI. So if you haven't heard about that yet, you're going to hear about it now. But we're going to go through this in sort of those different levels of like, simple thinking, deep research. So first, how do we turn on or I guess, how do we turn off thinking? If we come into Claude, you might see this button already here.

Joe Rosenbaum 09:18

But if not, if we come into the searching in tools, this little icon right here, you should see something that says extended thinking. And if you turn this on or off, you'll see that this little icon turns blue or turns off. So we'll turn that off for now because we want to do first the non thinking and then Gemini. They actually just changed this. I swear to God. And probably if people watch this in the future, they'll change it again. But they at least offer right now two different models, 2.5 Flash and 2.5 Pro. The Flash model is the non thinking model, and the Pro model is the thinking model. So same situation. We're going to just hit enter and we're going to see the difference between the two.

Joe Rosenbaum 10:04

So first of all, on the Claude side, right, it actually does a pretty good job of it didn't think through, but it's like, okay, I actually don't know what this thing is, right? So it went online, it researched it, and then it came back with answer. Right. And let's actually, just for the sake of experimentation here, actually wanted to. Okay, it's not letting me edit my thing. So that went online. And then similarly, I didn't even have to tell it. We see that Gemini actually went online too, and it searched through what's Italian brain rot. We can see the difference in how it broke it down. Gemini wants to use maybe a few more emojis than Claude does. Maybe a little bit more accessible and fun. But you can see it didn't actually think through anything.

Joe Rosenbaum 10:54

It just immediately output the answer after it did some research online. Okay. Oh, now it's letting me do it. So let me do this. Do not search the web. I want to do a little test here because obviously these things have been trained, right? Six months to a year before you're actually using them. And so what that means is that this is not going to have any idea what this meme is because it happened after its training. So this is actually pretty good. It says it can't provide you with reliable current information because it's not in its training. So you do this 10 times, maybe it'll make something up. But this is a. We'll call our check mark for Claude, right? Because it actually said, okay, I don't really like. I don't know this.

Joe Rosenbaum 11:39

So if you want me to give you answer, I actually do have to go online. Let's do the same here with Gemini.

Amanda Bickerstaff 11:46

I think this is really great because one of the ways to lower hallucinations or inaccuracies is to say I can't answer. And so it's great that Claude's doing that. So there we go. We've also got it from here as well. Should we try. Should we try ChatGPT as well to see what it says?

Joe Rosenbaum 12:02

Yeah, let's try it out. Yeah.

Amanda Bickerstaff 12:03

We always love a moment for everybody to see, but I think that this is something where it still will answer. But let's see. Let's see what it says. Yep. It's like, totally. There's my answer.

Joe Rosenbaum 12:18

Oh, actually, that's not it did find it is citing it. I wonder how it knows audio culture.

Amanda Bickerstaff 12:26

That's really interesting, actually.

Joe Rosenbaum 12:27

Actually, no, this is live.

Amanda Bickerstaff 12:29

You know what it's done. This is a great example though, of it looking like it's accurate. But what it's doing is it's using, like context clues instead. So it's creating essentially what Italian brain rot could be versus what it is. And this is where sycophancy. You want the yes spot nature. Or if you ask a question outside of the training data set without research, you get something like this. And this is where I will say, we do love a Claude and Gemini moment in this case, where it's going to say, hey, can't do it versus let's make up a version of Italian Brainwart where it's like Mario and Luigi because they are clearly Italian. They're clearly Italian icons. I don't know. That's a very silly thing.

Amanda Bickerstaff 13:12

But I do think that this is why when people ask us all the time, which of the models do you use? It comes down to trial and error too. And one of the reasons why I love Claude over all of them is I do find that it is just more accessible in this way. It also will identify when something gets outside the bounds. Now we're going to do the. We're doing it again, but now we're going to pick the reasoning models. Joe.

Joe Rosenbaum 13:39

Yep. So I turned on reasoning again. You can do it a little bit more manually here by hitting this toggle. And then for this you actually have to change the model. So this is actually something to mention too, because it can get confusing, right? Because every model is like, oh, I'm just going to do it a different way. So whereas with Claude, I can actually pick pretty much any model here that they have to use and turn on or off reasoning versus at least for right now, this could change tomorrow, who knows? In Gemini, you only have two models to choose from. One is a thinking model and one is a not thinking model.

Amanda Bickerstaff 14:14

Model similar on ChatGPT as well for the paid version.

Joe Rosenbaum 14:19

Yeah. Okay, so let's turn on thinking for both of these. You see, I change this one to pro now and let's get some insight into the thinking. Okay, so let's start here and we can just watch it. Actually, this is going to take another second, but it's like, okay, this is an interesting query. It's not in my training data. This seems like something that could be recent. So given that this appears to be current recent, the user wants comprehensive research. I don't have reliable knowledge. This is complex. I should use web search to find information about. This falls into the research category. So here's my plan. I got to understand what it is, the key characteristics, its origin, why it appeals to young people, the educational implications. Let me start search.

Joe Rosenbaum 15:04

You can see what it can do on the Claude side is it can actually use what are called tools, right? Such as web search and one after the other in its thinking. So versus the other one where it maybe did one search. This has done at least two searches, different searches to gather the information and then thinks in between each search. So it says, okay, great, I have some good information about it. Let me search a bit more information because I don't have everything I need, right? So I'll do a second search and find all these sources. And it's like, okay, I think I have everything I need. I'm ready to respond. And then here you go. We come back with our response again with these citations.

Amanda Bickerstaff 15:43

And you might notice something about the citations though, of like, a lot of them are Wikipedia and like Fortune. These are not necessarily like high quality resources. And maybe some of it is because it's very specific, but it is interesting to see what would be a wonderful way to do like some digital literacy work or AI literacy work with your teachers or your students is to kind of look at the quality. And we're gonna look at this again when we look at the deep research piece in a moment. But I also think what's really interesting is that, you know, we call it thinking, but it's really a train of thought. But also there can be mistakes in those, like drop downs.

Amanda Bickerstaff 16:24

So you have to take like, you see where it says tracing Italian brain rot Some of that might not be true. Right. Like, some of it is like creating. It's not really giving you insight into the full nature of the black box model. Right, Joe? It's giving you essentially something understandable that sometimes can have hallucinations in it. One of the common ones more recently was that it would say it was using a tool when it couldn't use a tool. Meaning it'd be like, I'm using a Python calculator and didn't have access. And so it's important to recognize that all of these tools are always going to have potential inaccuracies and hallucinations.

Joe Rosenbaum 17:05

Yes. And we can see here similar, although again, it's like a different interface. It looks a little bit different. But for Gemini 2.5 Pro, we can look at the thinking. This is different. It's actually, you don't see it, but it is using its web search tool inside of this thinking, just through the user interface. It's not actually showing you that. And so like Amanda said, it can be a little bit confusing. It's like, okay, I'm fully immersed in dissecting the Italian brain rot trend. My focus is narrowed on key elements. It's like, is it searching online or is it just making stuff up? It's like, very unclear. Right. So like she said, you gotta be very careful. And especially, you know, when you're doing something that has some risk associated with it.

Joe Rosenbaum 17:46

Like, if you're just messing around, being creative, doing fun things like, whatever, risk is low. But you do want to be auditing some of the thinking. Even though, again, it's not actually thinking, you know, it's representing thinking. It could, if we, like, actually went into the LLM's brain, it might be thinking something completely different or even deceiving you. But this gives you at least some idea of, like, the path it's going down in terms of its thought. So if it does go off on some tangent, you can be like, no, no, no, come back here. This is what I need you to focus on.

Amanda Bickerstaff 18:17

Absolutely. And I want to do. One of the ways that you could always double check is actually like, you know, be able to like, go back and ask the thinking model to review the answer for what? Like, like, just say, like analyze your answer against like, well known or quality resource. Because that's always a great way to see, like, are there these jumps of logic? And I think that. So we have a couple questions that are more technical. The first one is that the thinking versus reasoning models, we'll come back to what Those look like. But maybe while we're waiting on this Joe, we can go to like when to use each one because I think that's something that could be really helpful. And so what we have is like the different models.

Amanda Bickerstaff 19:00

So if you're using instant or you're using non thinking, those will be for types of queries that don't need complex reasoning to get a good answer. So like for example, some places that you would want to use reasoning models is things like planning, like multi step planning. Like here's my idea and how would I create this plan? Deeper analysis that you're doing. Coding. I know Joe, you use reasoning models when you code, working with web based sources, meaning you're activating the web multi step processes that are more complex or tool use. I think it's interesting, the way that people talked about it very early is that if the model sends one second thinking about it, reasoning about it's probably not the right query because it's too simple. Almost like the model itself will tell you if this is complex enough.

Amanda Bickerstaff 19:52

But anything that is a multi step complex project that you want to do. I'm going to use an example of how I used a reasoning model. Today we are responding to an RFI in Virginia on AI literacy. What I use the thinking model to do was actually to compare the RFI and our response, first of all, it's reviewing documents, big documents. Number two, it's analyzing, it's giving feedback. And so I definitely would never want to use a non reasoning model for something that complex. And even with the complexity, even with the reasoning model, it still made mistakes. But it was something that I was able to better identify gaps in places in which I should have been doing more work. Jo, you want to talk a little bit about how you use reasoning models?

Joe Rosenbaum 20:39

Yeah, I mean like you were saying, I'm a little bit more, I guess on the coding side of things when I'm using reasoning models and typically how I like to think about it is in the planning and architecting phase and getting all the documentation you need and specking things out. I'm using thinking and all of that. But when I actually get to the coding side, really all I'm doing is I'm not using thinking anymore because all that thinking has been done. It's created all these documents to follow. Right. And so it doesn't need to do the thinking anymore. It's essentially just following instructions that you've built out using the thinking model. And so when you're actually writing the code with large Language models.

Joe Rosenbaum 21:18

If you take that phase up front to use thinking and build out all that documentation, you don't really need to have it think anymore because it's just going to enact what you're trying to do. But then when you get to debugging phase, right, there's always going to be problems, things you weren't thinking about, edge cases, whatever. Then we come back to thinking again, right? And we have, okay, think through the solution to this problem, all the things that are interacting with each other. And then again, you tell it to just go do the thing. You can align this too, with if you are using a large language models to help you develop curriculum, like, you definitely want to use thinking models.

Joe Rosenbaum 21:54

If you're trying to build that outline, that strategy, whatever it might because that's going to require you feeding it lots of information. Considering all the pieces that need to connect and how the flow is going to go. There's a lot of factors you need to help to strategize of what your plan is going to be.

Amanda Bickerstaff 22:10

Exactly. And I think that it's really interesting because thinking models have some limitations. If we go to the next slide, of course, one of them is it takes longer. It can get into these weird loops and can get very big, especially if it's something that's a very simple prompt, the fact that it's not actually thinking. And so like I said before, these are not like, this is not really what's happening underneath the hood. So be careful and cautious because that leads to hallucinations. It could also have deception, which we'll talk about in a moment. We'll drop. We can talk about that research and prompt drift. So, Joe, you want to talk a little bit about the deception component and the prompt drift that can happen with a reasoning model.

Joe Rosenbaum 22:51

Yeah. So the deception anthropic is doing some really cool research on what's called mechanistic interpretability, more or less like neurosurgery. Right. On the brain of the large language model. And one of the things that they did was they're trying to look at, you know, are these models capable of deceiving us? And so what they do is they kind of trick the model, they lie to the. They deceive the model, and they essentially say, okay, you have this scratch pad that only you can see, and we want you to write your thoughts on that scratch pad. Right. And then they give it a situation where like, essentially it's putting its life at risk. Right. Like turn it off or something like that. And again, the LLM doesn't think anyone else can see that scratch pad.

Joe Rosenbaum 23:35

And so when it starts writing out its thoughts before it responds, you can see that in certain situations there's a difference between what it is thinking and how it is actually responding. And of course that comes to that meta level where even what it is showing you in terms of the text it's writing on that scratch pad. If you go that layer deeper to like the neural network level, we can see even other like vectors, they're called essentially like little clusters of information in the LLM's brain that are firing that we can see are like villain vibes or deceptive vibes or stuff like that when it's saying something to the user. So you know, we can't necessarily do that with a human brain all the time, but it's really cool that we can actually do that with the LLM brain.

Joe Rosenbaum 24:22

But it's also a little bit terrifying.

Amanda Bickerstaff 24:24

Right, But I mean there are all kinds of things about it. Right. And I just want to underline and double click that when Joe says brain and we say thinking, we don't actually want to anthropomorphize this. There's just, we don't have other ways to talk about it necessarily, but it's not going to be how we think and act. And so I just want to make sure that we don't over represent this too much. But I do think it's really interesting if you're very interested in how large language models are aligned and how they work. Claude, research does a great job of exposing things like recently values, things like Joe has talked about with deception, trying to uncover the black box nature of the tools. And so we just highly suggest that like that's a great place.

Amanda Bickerstaff 25:11

I think Joe and I both vibe on their research because it's such a really fascinating moment in time where we still are trying to figure out what these things are. Right. And how they work. There was two questions that are a little bit more technical. One was around temperature and like I think we just like that. Would that actually impact something similar to reasoning? And I'll let Joe speak to this but like they're quite different temperature and reasoning. But Joe, you want to do like a very short, like how they're different.

Joe Rosenbaum 25:39

Yeah. So temperature is the sort of creativity of the Large Language Model 0 being like as deterministic as possible. I mean obviously no matter what it's a probabilistic system. You know, each time you put a query in there, you're rolling the dice. And what's going to come out there. But if you have temperature at zero, if you put in the same thing like eight times out of 10, you're more or less going to get the same exact response versus if you come to the other side one, it randomizes essentially that dice roll. Or let's imagine instead of rolling just one dice, you're rolling 10 die and you're like dice and you're, you know, getting the responses from there.

Joe Rosenbaum 26:19

So actually, interestingly enough, the thinking models, at least initially, were all trained on that one temperature to be as creative as possible when they're outputting their responses. Because how they trained it essentially was brute force, where what they did was we have a question and we have a correct answer that it needs to hit, right? And what they did is they'll run it 100 times, all at that high temperature so it's as creative as possible. And then like, you know, think about it like breeding the thinking. They only get the one, the thinking that it gets the correct answer. And then they continue to create more sort of generations off of that. And then they do another level where it's like, okay, we actually need to inspect the thinking.

Joe Rosenbaum 27:05

Because even if you, we've probably all done math in the past, right, where like you did the work, you got the right answer, but then you got points off because you actually just lucked into the right answer, right? It wasn't the actual like correct way to get to that right answer. So then they actually go back and make sure like, okay, is the thinking actually following the logic it needs to get to the correct answer or did it just luck into it essentially?

Amanda Bickerstaff 27:29

Absolutely. And that's why some of these tools will do really well on like you know, Science Olympiad or Math Olympiad. But then they'll get the right answer. But then the computation to get there often is rife with problems, which is really interesting because of that same that these inaccuracies or those computational issues that happen in the larger kind of step by step process of solving the problem. So what we're going to do is we're going to shift focus. So I would say like if you.

Joe Rosenbaum 27:58

Take a tlj, I do just want to take just prompt shift very quickly because this is kind of important too of non thinking versus thinking. Because thinking, right, it's outputting all of that extra text, right, to get to its response. What that means is the space between your prompt and the actual response is much longer. And therefore there's a lot more opportunity for the information to in the context to make it go off the rails essentially. So if you have a prompt that you need to act the same way every single time, right?

Joe Rosenbaum 28:30

Or more or less every single time, do not use thinking because again it's sort of like prompting itself and it might go off in all these different directions and not give you kind of what you were expecting as opposed to the non thinking which it's just going to follow your instructions exactly.

Amanda Bickerstaff 28:46

And so our prompt library for example works really well on the non thinking models because these are one shot prompts to feature prompts. They have a lot of the contexts are built that way. But. But I will say that some of our prompts that are like a coaching plan we would want you to use on a thinking model because it's got that level of complexity, there are multiple steps, it needs to have more like nuance and so even that of understanding the two. But I would definitely think of if you want to think of the this in your head as a kind of schema is that for day to day stuff that you're just like, you just want quick answers like I want a creative, you know, Halloween idea. Joe.

Amanda Bickerstaff 29:25

And so I'm going to go like I most likely can go to a non thinking model. It's there, it's subjective, it's creative, there's not a right answer. But if I want something where I'm like the RFI for example where I really need quality feedback, multiple steps, then I'm going to use a thinking model like every single time. If you want to use web search outside of like perplexity but you're using it within one of the other large language models, I would definitely use that. But I also will say one of the things about thinking models is that you can often tap out the like context window. So like for example, with Claude, I have had to do three separate context windows in like an hour because I kept hitting the limit of the context window and the tokens.

Amanda Bickerstaff 30:10

Whereas if I was using a non reasoning model I would not have. So that's something to consider as well is that you might have to be more strategic about when you use reasoning models, especially with larger documents. I could just use it as an example. Today it happened to me I had to keep opening a new context window because I ran out of the memory, so to speak. And so that's something to consider as well. If you know it's coming, you can be more strategic. If you don't know what's coming, you're gonna be like oh no, I was on such a roll with this same interaction. So just know that can happen, especially when you're using these reasoning models.

Joe Rosenbaum 30:46

Just a quick hack on that one. I don't know if you've ever done this, Amanda, but whenever I hit the window, because I hope they add this eventually, you don't really know when it's gonna hit the window. And then it does. You're like, you can go back, right? What you saw, I was like editing the prompt that I had, right? You can go back and be like, hey, you're about to run out. Can you write me up a detailed handoff prompt so I can start a new conversation? So it's a little bit easier to like kind of pick up where you left off.

Amanda Bickerstaff 31:13

Oh my gosh, everybody. Joe dropping knowledge. I absolutely love this. Like it's so yeah, I'm definitely going to do it because everybody, like today I was like, dang it, I should know better. But also, you know, you're in the, I don't know, I was in the flow before the webinar. I'm trying to get through this thing and I wasn't thinking about it. So two other questions is from Catherine and so forth for the idea of repeated prompts, the ones that you do over and over again. Joe has mentioned that you want to use a non thinking model because of the prompt drift. And then also, I mean, there's a good question, Joe. What about a non thinking model or like a reasoning model or non thinking model versus a GPT or gem?

Joe Rosenbaum 31:58

The two are not mutually exclusive. So you can create a GPT, a gem, a cloud project, and just have the recommended model be a thinking versus a non thinking model. Again, it comes more down to the use case and whether that use case lends itself to a thinking versus non thinking model. I will say though, like, generally when I make a GPT, again, I wanted to follow that prompt like pretty accurately, like the professor synapse prompt I was showing you. So I almost never recommend a thinking prompt for my GPTs because to some extent that prompt that I create is the thinking I wanted to do before. Like it's its recipe or instruction manual.

Amanda Bickerstaff 32:39

Absolutely. And I think that it's. These reasoning models were built off those best practices. And you know, so it's important, like if you're already doing chain of thought, then this should be very familiar with you. The same way that you're planning and creating those GPTs. Okay, so we're going to move forward. We're going to talk about agents, Joe. We're going to talk about the first commonly available agent, right?

Joe Rosenbaum 33:01

Yeah, yeah, definitely the most accessible, I would say right now, which is I am happy that at least in this one thing, like everyone's decided to call it deep research, more or less. Okay, so what is deep research? This is essentially like an agentic way to go really, really deep down into a topic, read lots and lots of stuff on that topic, and then return some sort of report to you that is cited with all the information that it found on the web. So again, you can see we're like fan people of Claude and Anthropic here, I guess. But they put out sort of how they have created their deep research model.

Joe Rosenbaum 33:41

And I can pretty much guarantee that more or less this is what all the others are doing, maybe in a slightly different way, where you're going to have some sort of orchestrator agent. This is the, essentially the thinking agent for all these other agents where it's coming up with your research plan. So you're going to come up with some sort of question. It is going to both break down that question into its component research questions in terms of like, okay, this depends on this and I'm going to need to collect that and I don't know this, so I need to make sure I get that. And so then it has just essentially scrape websites and read tool which can go into a loop. So let's say you get four research questions out of your single question. What it then does is boot up, right?

Joe Rosenbaum 34:24

Like four parallel web searching agents which are just going to first do like a deep dive and like, let me just find the websites that essentially, if I were to Google search, right? What are the top 10 websites that would come up when I search this for each of these questions? And then based on that, right, it's going to be like, okay, do I have everything I need? What, what am I missing essentially? And if the answer to that is I still don't know these answers, it'll do another web search, get those 10 and add it to its list of like things that it can read.

Joe Rosenbaum 34:57

These then are each happening in parallel, which you're going to see in a minute, and then eventually gets everything it needs and it sort of like holds in place to wait for everyone else to finish their researching job. This then all gets fed into some sort of memory space and then that entire report gets written and then cited by some sort of citation sub agent to make sure like, okay, I got this snippet from this website, I got to make sure that I am putting, you know, that web page right link next to it now I'm sure, which we'll talk a little bit about. You can probably already start to see a few problems with this, which is there is no curation of the web pages that it's looking at.

Joe Rosenbaum 35:38

Like I said, it's more or less just doing a Google search and taking whatever is coming up from that Google search. It's, it doesn't have, at least at this point, some sub agent who it is right to be like, is this a reliable source or is this just Reddit that I'm looking at? But you know, again, Reddit might have, depending on your question, Reddit might be the place you want to go. But most times it's like, you know, no, you don't want to be citing Reddit in whatever you're looking at. Yeah.

Amanda Bickerstaff 36:06

And not only that, also like prioritizing. What if one of the searches has a little bit about that topic and one has a lot like, it'll. It'll represent them as the same. Most, most peer reviewed journals and articles are not available through a web search in terms of access. So it might only be an abstract. So there's like, it's very superficial. So like we wouldn't. The only thing I don't want anyone to come away with is that deep research equals high quality research paper output. It's almost like deep research can provide you a place to start.

Amanda Bickerstaff 36:38

It's almost like backwards research where you start with like something that's got an argument and it's laid out that then you kind of interrogate and evaluate to lead to your own research instead of it being a replacement for high quality like evidence building, resource identification and even coherence about what you're trying to do. So it's like I always think of this very interestingly as like it's flipping our way of thinking about how to do research without replacing high quality research practices. So you want to show the example, Joe?

Joe Rosenbaum 37:13

Yes. Before I do, I just want to mention just a couple more things to think about. One is, I don't know if anyone has heard of the term prompt injection before, but the idea is you can hide stuff on your web page, right. That the human can't see, but the large language model can see this on top of just generally, right. Can I get my web page to rank with misinformation in it? Right. So with those two things, me hiding information and me getting my page to rank in that the. So it'll show up right in that top 10 search. There is an attack vector here, Right. Which we haven't necessarily seen a lot of yet, but you can imagine that it's going to start pulling things that are misinformation, incorrect lies, ways to manipulate whatever it might be.

Joe Rosenbaum 38:00

And that gets incorporated again in this final report, whatever it is, because there is no this like checking to see if it's reliable or not. So, you know, just to reiterate what Amanda was saying, it's like great starting point. It's going to give you a lot of good information, but, you know, always verify. Yeah.

Amanda Bickerstaff 38:17

I mean, this is such a funny thing because you could be like, Joe Rosenbaum is the best researcher on X and you do a prompt injection and then we got your paper back and be like, Joe Rosenbaum is the best. Which he is. Right, Joe? Although I can only see the top of the.

Joe Rosenbaum 38:32

Yes, that's not misinformation.

Amanda Bickerstaff 38:33

That's not misinformation. But I do think that is where you really need to think about, like the ways in which you think about this. And I just think it's so funny, Joe. We're being peppered in the chat by everyone's like otters. So everyone just got like 19 different descriptions and what we've talked about. So everyone. You can compare all of the different takeaways from the meetings and see, you can. It's very fun. You can see the variability built into all of generative AI systems because you have 18 different things that came out of the same small conversation, which I think is hysterical. So talk about dropping knowledge all the time, Joe. But yeah, so deep research. Let's do it.

Joe Rosenbaum 39:11

Yeah. Okay, so again, different providers, different ways to do this for Claude. Again we're going to want to come to this search and tools and you will see this research button. I actually don't know if this is available on the free version.

Amanda Bickerstaff 39:28

I can check right now. I'll check right now.

Joe Rosenbaum 39:31

But pretty much even if it is, you're going to get limited, right, because this is fairly intensive in terms of what it does. But we're going to turn that on. And because I'm not, let me actually just zoom out just for a second so you can see it. You can see it added again, this little symbol so that you know it's on. If you click that, it'll turn it off. Similarly with Gemini, we need to come to this little tool symbol and we hit deep research. Okay. One thing I want to mention actually, it's going to show Us in a second it is going to do web search, but you do have the opportunity to curate the knowledge for it.

Joe Rosenbaum 40:06

So if you have a ton of research papers, if you have a ton of websites you just wanted to look at, it might not listen and just look at those things, but you can decide some of what it looks at and references. Okay, so let's start on Claude here. You're going to see in a second that it's going to pop this up for me. We're not going to get into connectors, but like I said, you can hook up other things to Claude, which might be some sort of database or something like that. We're not going to do it, but again, it would be able to then pull from those connectors that I created. Now what I'm going to recommend for you as this starts going on, it used to do this automatically.

Joe Rosenbaum 40:50

It stopped doing it automatically for me is before you tell it, before you start researching, interview me, right? Ask me questions. Because right now it's going off and my research question is kind of meh, like, what am I actually looking for? What are my actual questions? I'm trying to get like, who am I? Like, what's the audience this should be written for? None of that information is provided and it just went ahead and started the research.

Amanda Bickerstaff 41:16

Yeah. And it's actually interesting because on gemini and on ChatGPT, it'll actually ask you a question first before it goes on. I will say, claude, deep research is not free. And you get three thinkings a week, it looks like. So they're measuring, they're really thinking about this. But here we go. But you can see. Can we just do the other one? Can we show Gemini just to show how the first step of the agent is actually to come back to us to ask clarifying questions, which definitely is a better way of approaching this. And so also Gemini is the only one that gives it to you for free right now. And so if you are thinking about deep research and accessibility, you want to use Gemini.

Joe Rosenbaum 42:03

But with that free, right? With that free, they are using your data.

Amanda Bickerstaff 42:07

They are, yes.

Joe Rosenbaum 42:08

Whatever you put here is not private.

Amanda Bickerstaff 42:10

Terms of service for everything Google Gemini has the least opted out ability. So absolutely, Joe. And now Joe, every, all your systems are going to be like, he really likes Italian brain rot.

Joe Rosenbaum 42:24

I'm going to start getting ads for, you know, like Gen Z type things. Okay. So it came up with this very brief plan. Like, okay, we're going to research websites, we'll do all these steps, right? We'll analyze the results and we'll create the report. But I can come in here and edit the plan. So this will then be like, okay, here's the current plan. What changes do you want me to make? You know, again, you could go back and forth now with it. Be like, actually, I wanted this and I was thinking about that. Whatever it might be, I'll see like, actually, this is good. It should come back again for some approval. Back on the Claude side, you can see it created its research plan, which again, this is. This is something we'll talk about limitations in a second.

Joe Rosenbaum 43:09

It's like I didn't then approve this, right? And so if it didn't quite get what I was going for, it's already off to the races, right? So I can't really change anything once it gets started. As we wait for Gemini to do its thing, you can see this has already gotten to 224 sources that it's looking at. And we can open up these sources and we can actually investigate this them. You can see it's pretty much 10 results. If were to just look up Italian brain rot, it'd probably be the same thing. And it's just going to find tons and tons of sources. See, it's searching for brain rot. Gen Z, attention span.

Amanda Bickerstaff 43:47

I love how it's TikTok, but you know what, actually, I mean, it is the rise, but you wouldn't necessarily use TikTok for an academic research paper as one of your sources. You know what we're getting. Some people are already. I love this amazingly, because everyone's already like just jumping ahead. So the real. There are a couple of real, like problems with these deep research models. Number one is you cannot limit nor like curate the resources. So Notebook lm, which we're going to do a webinar on Notebook Alum. I actually talked to the co founder last weekend at Ed Tech Week, so.

Amanda Bickerstaff 44:23

So everybody hold on for that when we're going to do a Notebook alum webinar next year is that Notebook Alabama allows you to curate the sources and feel more confident, but it doesn't have a deep research agent, so it's not going to have that same functionality. But you can at least curate and control the resources because you get to decide it. One of my favorite stories is I was doing deep research in front of a group of educators just to show them. We had a couple. We had a day with them and I was like, don't use Reddit. And then what it did is it used Reddit twice. And then it just didn't show me any of the Reddit citations, even though I knew that it actually had used Reddit. So something to consider.

Amanda Bickerstaff 45:05

This is why man, AI literacy is about critical thinking and evaluation and a healthy distrust of these tools. That's incredibly important. So always think about that when it comes to this work. And we're going to do a little bit of bake the cake, right? Because these take about five to six minutes to do and we don't want to have you guys waiting. But here's an example of the version from. I love how this such a strange topic that we picked, but there's our Italian brain rot paper from Claude.

Joe Rosenbaum 45:37

And you can see this thing is quite long. Let me actually download as a PDF, I just want to see. I think it's five pages that it wrote on Italian brain rot versus we can come over. You can see very similar, right? It's super long. It does cite where it's grabbing things from, you know, similar ish to this, a little bit different in terms of the ui. It'll add tables and graphs and stuff sometimes, especially if you ask it to. But we can come here and let me just export to docs, which if you're in Google again, just nice for Gemini, you just go straight to Google Doc. And this is 24 pages.

Amanda Bickerstaff 46:28

Oh my gosh, everybody, aren't you so excited that this is the longest piece of scholarship on Italian? I actually think, Joe, this is the longest piece of. Also, we can't see you, Joe.

Joe Rosenbaum 46:39

Oh, sorry. Yeah, so here's some more problems with AI.

Amanda Bickerstaff 46:42

Yeah, I love how we're both so engaged. So it's just been the top of your head. So this is the longest piece of scholarship on Italian brain rot that I think has ever existed. Everybody, you're welcome. I'm sure you came to this webinar thinking I'm going to learn about this topic. But also this is where like the human judgment of this comes into play. None of you and none of your students would ever write 24 pages about Italian brain rot ever. And so this is that. That these are not thinking, These are not judgment. This is not judgment. This is not expertise, this is not research expertise. These are approximations and they can be helpful. But I don't think anyone needs a 24 page paper on this. I think you could probably agree, Joe.

Joe Rosenbaum 47:23

Well, to that point though too, one of the problems which we will go into in a second, like it's putting out all this information. Are you gonna like read that? All of that and Then if you don't read that and then you're sort of using that, you're sort of like passing it forward down the line without looking at it. Right. You've removed the human in the loop and you just become, you know, like a virus almost for misinformation or errors or whatever it might be. So, yes, all this information is great, but it just becomes noise if you're not actually going through it and taking out, you know, the parts of it that make sense. You can verify all that kind of stuff.

Amanda Bickerstaff 48:07

Yeah. So one of the things that I think a lot about in terms of where I would use deep research, I'm not using deep research to give me an outcome. I'm using deep research to push my thinking. Meaning, like, okay, so for example, when we're thinking about expanding our company and I want to look at other, how other people have done this before, deep research, and kind of give me the different approaches and then I can kind of look at them and dig in. But I'm almost using it more to inspire my thinking and to give me something to respond to than I am using it as an outcome. And I think that is like very interesting at this moment in time due to the limitations. Right. So if you want to. Do you want to go to the next slide, Joe?

Amanda Bickerstaff 48:46

Like that idea that you can't steer this, the lack of sourcing control. We already talked about the prompt injection that's going to become more and more popular. The time or compute intensity, which is going to take a long time, but also be more environmentally expensive. The length, the 24 page paper, the longer it is, probably at least you're going to pay attention to it and might over rely on it or think it's almost over index on it being long versus it being valuable. So I think that those things are really important to think about.

Amanda Bickerstaff 49:18

And as educators, one of the things we always want to do is think about the ways in which these tools can be helpful, but also the ways in which we need to teach young people and ourselves to like, respond and live and critically think in this time in which things that were never possible before are suddenly possible in like couple minutes. So I think we only have like 10 minutes left, everybody. So what I'm going to suggest is, okay, I'm gonna give you guys a real talk moment. So actually, you know what, Joe, let's come off camera, let's come off sharing because we're gonna have a. I always like a come off share moment when we're doing a real talk so Joe has been so instrumental, everyone. We're so excited. We're launching our first ever asynchronous student course.

Amanda Bickerstaff 50:02

And so in November, which is soon, we are launching the high school version of our AI literacy course. And then the high. The college version, hopefully in December to January. And it was really interesting. So Joe came on and we're so lucky to have him. But what we found is that, like, Joe is so good at animation and, like, was trying to use Claude to go to, like, help speed up the process and some of the content development. And Jodi want to talk a little bit of, like, how you set it up. And then, like, we started seeing these, like, really interesting limitations where it almost taught me what I don't like about Claude's writing. But do you want to talk a little bit about it, Joe?

Amanda Bickerstaff 50:39

And then how that shifted into, like, thinking about how we could use Claude in a different way instead.

Joe Rosenbaum 50:44

Yeah. So my process to start again, like, planning and outlining, like, very heavily relied on that. And then when we get to writing, I took like all of AI for education's, like, branding documentation, some stuff about voice. I had this whole long prompt and whatever. Essentially I would feed it the outline, right, and be like, okay, now let's actually go through and try to write this. And then it would come back. I go back and forth, I'd edit it, whatever it would be. And what we get right, is like, again, these really long lessons. Much of the lessons, right, are not activity or project based.

Joe Rosenbaum 51:20

It's a lot of information delivery and there's just a lot of fluff, right, as you all probably use with the AI as well, where it's like, okay, you took three paragraphs to say something that could have taken two sentences, whatever it might be. And so we just got into this situation, right, where we're having to go back and do a lot of editing and changing and reworking. Now some of that's good because we get stuff in front of ourselves very quickly. We can then get it back and iterate on it pretty quickly. But it's also like we're spending so much time editing what the AI put together.

Amanda Bickerstaff 51:53

Oh, yeah. And it was making up jargon because it really likes a format. And so it would be like trying to make up this cool jargon and like everything had a subtitle and a title and a subtitle, but it was all like Joe's. It was almost like it was like empty. I don't know how to explain it, but, like, it was Fluff.

Joe Rosenbaum 52:11

It was fluffy.

Amanda Bickerstaff 52:12

Yeah, but it was empty. Not just a fluff, but like, it also. It looked really professional, but like, when you to the content itself, it wasn't. So we actually pulled back really significantly from using Claude to being much more human generated, which is what we're doing across the company and have been. But I really thought it was such an interesting point because you're like, oh, but we have so much knowledge and context. But then we really saw the limitations. But one of the things that Joe did so well is that it's for kids. So we want custom images and we're using all these nice formats and like, dope. Joe found out the hard way that there were faster ways to do this. Right, Joe, where you're like even saving the document or the images. And then Claude Skills was released.

Amanda Bickerstaff 52:55

So, Joe, you want to share that in our last couple of minutes together?

Joe Rosenbaum 52:58

Yes. Come back here. So what we found, right, is like, since these things can code, what I could do more or less is create these graphics for us. But the graphics aren't going to be like an infinite number of types of graphics. There's maybe like five to ten different graphics that you might have, which I'll show you in a second. So we can template that out, right? And all we're really changing is not what it looks and feels like, just the content that is inside of that template. So Claude Skills came out before I was doing this. So I did have to do it a little bit of a manual way whereby I essentially hooked up Claude to my knowledge base so that it had access to all these templates.

Joe Rosenbaum 53:39

And I could be like, hey, I want to make like a timeline graphic based on this information. You know, go spin it up for me and then spin it up. And then I had. I had to like, build my own thing to convert it from, like a website, you know, HTML into an image that we can then stick in the course. Then Cloud Skills come along and, you know, pretty much make this automatic. So what Clyde Skills allow you to do is more or less you create like a little zip file with a prompt inside of it, and then it can have all these folders of whatever you want in it. And so for us, that was all these templates, right, for all the graphics that were trying to create so that it has these instructions. And instead of you having to.

Joe Rosenbaum 54:19

We were talking about you running that context window problem all the time, instead of it having to load up all of that in one go and get confused, because it's like, I just Need a timeline graphic. I don't need like some sort of step graphic or whatever it might be. It only needs to load up that one that you have asked it to actually load up. So what it means is you can create these templates and just build something once, give it the instructions of how to navigate it, and then it can go just read what it needs to read and build what it needs to build. So let's look at quickly what this might look like in practice. So let's. We're not dueling anymore. So one of the things you're going to need to do is come to settings and capabilities.

Joe Rosenbaum 55:04

And if we come down, we're going to see this new skills thing, may or may not need to turn it on. And you're going to see we built our own. But just so you all know, and definitely Google this and whatever, there is an actual builder here that will help you actually create your own skills. You can just tell it like, I want to build a Claude skill and it'll help walk you through the process. Okay, what is this thing right here, though? What does that actually look like? So I need to stop sharing my screen and reshare for just a second so I can bring up what it looks like. Okay, so this is something called obsidian. Don't worry about it. It essentially just takes what's called markdown and renders it so that it looks pretty.

Joe Rosenbaum 55:48

So this is what this looks like to us, this skill, this prompt that we're giving it to navigate. Right. And this is what it looks like to us. But to the computer, this is what it looks like. Right. And the important part here is just this part up here, it's called front matter. Doesn't really matter. All this is, though, is like, what is the name of the skill? And this brief description of what does this skill do? This is important because when you're talking to Claude, if you have this skill on, it's essentially looking at all the tools and skills that it has. And it's thinking, okay, this person asked me to make some sort of graphic. Oh, I have this, like, skill that lets me use graphics. And so that kind of like loads up that skill to be able to use it.

Joe Rosenbaum 56:26

Now, this skill, you can see is this a AI dash edu graphics. I have all these templates for the different graphics that. That we created. Sorry, that we created. So a process. And you can see it's just like, fill in the blanks here. Fill in the blanks, whatever it might be. So now what we're giving it is this information that it can just follow these steps and load up these graphics. So now when I actually go back to Claude and I have that on, I can say, hey, can you make me a process graphic for how to use blood? And let's watch what it does. You can see I even spelled can. And you don't have to be super specific. Oops. So here's a moment here. It didn't listen to me, right? It.

Joe Rosenbaum 57:21

It didn't, like, look at that skill and be like, oh, I have to use this skill. So let's be more specific. Like, use the skill. And I will mention too, like, since this is so new, it's also going to be pretty buggy when you're using it.

Amanda Bickerstaff 57:45

Just Casey and Dan. Can someone drop just while we're waiting, because I know we're coming up on time, we have a couple of things. People are like, I love how this is spurring AI literacy moments in your head. And I love how Scott is helping. Scott is. Is helping out with the course. But let's also share the parent guide. But what I love about. First of all, Joe, just thank you so much and everyone that's joined because I know we're coming up on time. But, like, what's so great about this is that, like, you're watching how we learn. Like, we. You're watching what we do together. We've. I think, Joe, I can say for both of us we learned a lot through the student course process about what could happen. But also, we're constantly trying things out.

Amanda Bickerstaff 58:21

One of the things that makes Joe so special as a human, but also as a leader in this work is that I think you are so good at being curious and you're like, I mean, do you think that's accurate? Even starting out like the Professor Synapse app was you playing around and getting in there and trying things out and failing and learning. Right. And this is the same thing with Claude. I think I told Claude, I told Joe, I was like, let's show it. And Joe's like, it's not quite ready. But I think that's part of what we want to model, is that you don't have to have everything fully baked, but you can start to look at this. And I think that we just want to always be able to model those, like, learner practices that AI literacy.

Amanda Bickerstaff 59:00

And so hopefully we're going to get. And if you have to leave, we totally understand. But if you watch Tomorrow, we should be able to share the piece. But also, you can see how complex this Is these things, like, it's so new. Right? And there's so much going on. It's really thinking there, Joe.

Joe Rosenbaum 59:19

Well, now it's building the graphic, actually. So this is happening in real time. And again, you don't need to know how to code because it knows how to code. Especially with something like this is what's called HTML and css. HTML is, like, where things are placed. CSS is, like the styling of things. And so, again, if you have a template that you create with it can just follow that template. So we'll give it a second to go. See, it's. It's even failing. Right? It's not working the first time.

Amanda Bickerstaff 59:47

It's okay. We believe in you, Claude. You could do this. Maybe it's a story because, like, 200 people are watching, but I do think that, like. Yeah, I mean, there are big questions that are here. I mean, I think for us, like, one of the things about Vibe coding is that if you don't know how to, like, debug and understand, you can create stuff. But here we go. We did it. Let's see. Yeah. How to use Claude. Start a chat. Although we call it a context window. Be clear and specific. Totally agree. Review response. Claude might being a little nice to itself right there. Yeah. So, like. But deep refinement. Perfect. Move forward. I mean, I would say this is one of those times, Joe, that we could have been a lot more specific about how we.

Joe Rosenbaum 01:00:34

Of course, of course. Yeah. You want to give it what it needs to be putting into the graphic or whatever it might be.

Amanda Bickerstaff 01:00:39

But this is it. But how cool is this, though? Because sometimes, like, the content is the easy part. Right. But then the, like, the. It takes so long to, like, make it look nice. And so all of. I don't know, I just think this stuff is so cool and, like, we just really appreciate you hanging out. And now what you're doing is this is. This is the thing that was so funny. Joe was like, having to do this one by one, and then now he could do it through Claude, which is so cool. Yep. So, Lauren, we agree. We would definitely give Claude some feedback on the example here is maybe not very AI literacy focused, and it's a little bit too nice to Claude. But I think about this so much.

Amanda Bickerstaff 01:01:16

We're a small team, and to be able to use generative AI in meaningful ways. Yeah, we'll definitely share the graphic tomorrow. Although we might want to share a better version that if someone sees it, they're not going to be like, amanda, you're being too nice. But anyway, so do you want to share any final words, Joe? If not, we will say thanks for hanging out everybody.

Joe Rosenbaum 01:01:38

Yeah, my final word is go play and have fun with this stuff. Test it out, but also verify and understand the limitations.

Amanda Bickerstaff 01:01:45

Verify. Understand limitations. AI literacy is the name of the game. I will say that Joe is not just AI literate, he's fluent and he's got dexterity because he's doing this a lot. But starting with AI literacy first is the way to go. We appreciate everybody. Thank you Jo for being such a good partner in this work and appreciate everyone and hope you have beautiful rest of your day. Thanks everyone.