WEBVTT 00:00.840 --> 00:02.800 Now I want to talk about resources. 00:02.800 --> 00:08.880 So resources are a way that you can get more out of your agents, that you can equip your agents to 00:08.880 --> 00:13.080 be able to solve your problems better, along with tools that will come to you in a second. 00:13.280 --> 00:20.320 And resources is really just a fancy way of saying that you can improve the effectiveness of an LM by 00:20.320 --> 00:24.760 providing it with more context, more information to improve its expertise. 00:24.800 --> 00:31.480 And the way that you do that is just grab some relevant data to the question and shove it in the prompt. 00:31.480 --> 00:36.880 So it's just a matter of saying, like, if this is going to be a question that's going to be particularly, 00:36.920 --> 00:41.720 we're going to prompt the LM to ask something about the company that we work at. 00:41.760 --> 00:46.800 Maybe it's to be a flight airline, uh, customer support agent. 00:46.920 --> 00:50.600 We could shove into the prompt all the information about ticket prices. 00:50.600 --> 00:54.760 And then when it's answering a question, it would be able to refer to any of that information as part 00:54.760 --> 00:55.960 of giving its answer. 00:55.960 --> 00:58.120 That would be an example of a resource. 00:58.160 --> 01:04.880 And so it's nothing more fancy than just saying we can just put extra context, extra information in 01:04.880 --> 01:05.320 the prompt. 01:05.360 --> 01:08.090 We send an LM and think of that as a resource. 01:08.090 --> 01:09.650 We're providing the LLM. 01:10.250 --> 01:13.730 And when I say it's nothing more fancy than that, there is a bit more fancy than that. 01:13.730 --> 01:19.130 You can use clever techniques that will allow you to rather than just shoving all the ticket prices, 01:19.130 --> 01:22.090 just put the ticket prices relevant to that question. 01:22.090 --> 01:28.450 So coming up with clever tricks for ways to figuring out the best, most relevant context tricks perhaps 01:28.450 --> 01:31.090 that can even use other llms to help with that. 01:31.130 --> 01:33.090 That is all part of the the whole field. 01:33.090 --> 01:34.890 That's called rag retrieval. 01:34.890 --> 01:36.530 Augmented generation. 01:36.530 --> 01:39.090 That's about retrieving relevant context. 01:39.090 --> 01:40.890 It is, of course, a super hot topic. 01:40.890 --> 01:42.290 It's covered on my other course. 01:42.290 --> 01:46.530 It's not relevant for this course particularly, but it's very interesting. 01:46.530 --> 01:53.210 But that whole side of things, figuring out how to pick relevant extra resource information to put 01:53.210 --> 01:56.890 in the prompt falls under the the heading of resources. 01:56.890 --> 01:58.930 And we'll be doing some of that today. 01:59.810 --> 02:05.170 And with that, let's turn to the main topic of the day, which is tools. 02:05.570 --> 02:07.210 So tools. 02:07.210 --> 02:08.570 We've said it a few times now. 02:08.570 --> 02:16.510 It's clear that this is really at the heart of a gigantic AI Tools is when you give an LLM the power 02:16.510 --> 02:21.790 to do something, to use a tool, and it's really can use that tool at its discretion. 02:21.790 --> 02:27.310 And so it's one of the, the, the, the key tricks to giving an LLM some autonomy. 02:27.470 --> 02:29.270 So what does this actually mean. 02:29.430 --> 02:36.430 So we're going to give an LLM the the power the ability to carry out different different actions like 02:36.670 --> 02:41.710 do a SQL query of a database or send a message to another LLM to do something. 02:42.190 --> 02:46.830 So the first time that you heard about this, if you've if you've heard about tools and function calling, 02:46.990 --> 02:48.990 it might sound like it's kind of crazy. 02:48.990 --> 02:50.950 It's kind of it's sort of creepy. 02:51.110 --> 02:57.030 We're going to be interacting with open AI on a cloud, and we're going to say, hey, you have a tool 02:57.030 --> 03:04.630 you can use, you can query my database, and it's going to be able to like, connect back to my computer 03:04.870 --> 03:08.350 and query the database as part of giving its response. 03:08.590 --> 03:10.110 That's what tools are all about. 03:10.150 --> 03:12.630 And it's like, okay, how exactly is it going to do that? 03:12.670 --> 03:14.630 Like that sounds crazy. 03:14.950 --> 03:18.550 Well, the truth is actually rather more mundane. 03:18.550 --> 03:24.880 And people that know about tools know this already, but it's sadly it has a sense of being very magical. 03:24.880 --> 03:29.880 But the reality is a bit of a conjuring trick, as I will now show you. 03:30.200 --> 03:33.480 So in theory, what's going on with tool calling? 03:33.520 --> 03:35.120 We've written some code. 03:35.440 --> 03:42.880 Our code is sending a prompt to an LLM to like OpenAI, like GPT four or mini running on the cloud, 03:43.240 --> 03:50.440 and that LLM is being given the ability to execute a tool of some sort, which might be a SQL query, 03:50.480 --> 03:56.080 it might be a calculator, it might be a tool which turns on and off the lights that are stringing around 03:56.080 --> 03:57.800 me from the first episode. 03:58.080 --> 04:00.800 So that's the kind of thing that it's able to do. 04:00.840 --> 04:03.480 And based on that, it will give its response. 04:04.120 --> 04:05.240 So this is it. 04:05.480 --> 04:13.520 In theory, the practice has just one tiny deviation from this, but it reveals everything. 04:13.960 --> 04:16.240 This is what actually happens. 04:16.760 --> 04:22.240 You prompt the LLM and you say, you know, I want to turn on my lights or whatever. 04:22.400 --> 04:26.850 And you say, but I would like you to tell me. 04:27.050 --> 04:32.850 If you want me to take care of a few actions that I can do on your behalf, and one of them is called 04:32.890 --> 04:33.930 turn on the lights. 04:34.210 --> 04:40.450 And if you reply turn on the lights, then I will do that and I will get back to you with I turned on 04:40.450 --> 04:41.090 the lights. 04:41.410 --> 04:48.890 So basically in the prompt to the LM, you just list out everything that it's able to to ask for, and 04:48.890 --> 04:51.450 you tell it to respond in JSON. 04:51.650 --> 04:55.730 And in that JSON response it should say what it wants to do. 04:56.290 --> 04:59.890 Now the tool calling code sort of packages all that away from me. 04:59.890 --> 05:03.490 So you don't need to worry about the fact that it's being called and responding in JSON. 05:03.490 --> 05:04.970 But that's what's really happening. 05:04.970 --> 05:06.450 It's all that's happening. 05:06.490 --> 05:13.730 It's really just clever stuff about getting JSON back from the LM if it wants you to do something. 05:13.730 --> 05:18.650 And at the end of the day, you have to write an if statement in your code. 05:18.890 --> 05:21.850 You can be fancier than an if statement, but it's like an if statement. 05:21.850 --> 05:31.130 That's like, if the LM wants to do X, then do X, and then call the LM a second time with the results. 05:31.130 --> 05:37.580 And so that's that's what it comes down to, asking the LM if it wants to run a tool and if so, then 05:37.580 --> 05:38.780 running the tool. 05:38.980 --> 05:43.220 So if you knew that already, then I'm just emphasizing what you already know. 05:43.260 --> 05:48.860 If you didn't know it, it might be a kind of aha moment when you realize that that that actually it's 05:48.860 --> 05:50.540 not that, that that clever. 05:50.580 --> 05:54.220 It's, it's pretty pedestrian, but it works and it works really well. 05:55.020 --> 05:58.460 And just to show you, just to make this concrete for you. 05:58.740 --> 05:59.860 Check this out. 06:00.180 --> 06:04.100 Look at this conversation that I had with GPT four. 06:04.820 --> 06:07.740 I said you're a support agent for an airline. 06:07.780 --> 06:09.900 You answer users questions. 06:09.900 --> 06:12.980 You have the ability to query ticket prices. 06:12.980 --> 06:14.300 Just respond. 06:14.460 --> 06:19.740 Use tool to fetch ticket price for London, to retrieve the ticket price for London, or for a city 06:19.740 --> 06:20.580 that you name. 06:20.740 --> 06:22.500 Here's the user question a user. 06:22.500 --> 06:23.700 I'd like to go to Paris. 06:23.740 --> 06:24.860 How much is a flight? 06:25.260 --> 06:27.540 And look at what ChatGPT replies. 06:27.540 --> 06:29.700 Use tool to fetch ticket price for Paris. 06:30.460 --> 06:32.100 That's all there is to it. 06:32.500 --> 06:35.260 You write code that prompts a bit like this. 06:35.980 --> 06:41.600 You get back, use tool to fetch ticket price for Paris, and then you send the question a second time, 06:41.600 --> 06:45.320 but with the answer to that included in the question. 06:45.720 --> 06:49.280 And you've just seen tool use in action. 06:49.920 --> 06:56.200 So the way that we can interpret this, of course, is to say that we are giving ChatGPT the autonomy 06:56.200 --> 07:00.920 to decide that it wants to make use of this tool should it wish to. 07:00.960 --> 07:09.080 And that's all sounds very mystical and and powerful, but at the end of the day it's JSON and if statements. 07:09.080 --> 07:10.280 That's how it works. 07:11.320 --> 07:15.840 Okay, so with that it's now time to go back to the lab. 07:15.880 --> 07:17.120 But I have bad news. 07:17.120 --> 07:19.680 We're not actually going to do a tool today. 07:19.680 --> 07:21.480 Even though we've just been talking about tools. 07:21.480 --> 07:23.640 We are going to be using resources. 07:23.640 --> 07:25.600 So we are going to tie to that. 07:25.800 --> 07:30.680 And in the next lab tomorrow we are going to actually use tools. 07:30.680 --> 07:35.520 So for now I want you to put on hold that thinking about tool use and if statements because we will 07:35.520 --> 07:36.360 come to that. 07:36.360 --> 07:40.520 But for now we're going to be building something with resources and it's going to be fun and it's going 07:40.560 --> 07:44.080 to set things up for the tool use that will come next time. 07:44.080 --> 07:45.240 I'll see you in the lab.