WEBVTT 00:00.160 --> 00:07.280 Okay, so I am going to start by taking us over to the tools which we're in right here. 00:07.640 --> 00:14.280 So the way that the tools module works is it's just a module that's going to load in all of the tools 00:14.280 --> 00:16.520 that we've worked with in the past. 00:16.760 --> 00:19.720 Um, it starts with a loading in the dot env. 00:19.920 --> 00:24.080 Um, because we'll need the API keys, we set up a few constants for pushover. 00:24.400 --> 00:29.560 We create the Google Sherpa API wrapper so that we can use that API. 00:30.000 --> 00:34.200 Um, and then we have uh, the playwright tools. 00:34.200 --> 00:39.640 This is the tools for navigating for driving the browser, taking exactly from the notebook that we 00:39.640 --> 00:40.680 were looking at before. 00:40.960 --> 00:48.600 Uh, we, we use the async code to start the playwright instance, and we launch Playwrights Chromium, 00:48.840 --> 00:53.920 and then we get the toolkit, and then we return the tools from the toolkit. 00:54.040 --> 00:58.600 We also return this browser and the playwright object itself. 00:58.600 --> 01:03.540 And we do that because we need to clean them up at the end of it when it's finished with resources. 01:03.540 --> 01:06.300 And this is something that I'm not sure I. 01:06.620 --> 01:09.500 It remains to be seen whether this is properly cleaning up resources. 01:09.500 --> 01:10.700 I'm checking to see if it does. 01:10.740 --> 01:15.620 Sometimes I see that the playwright the chromium browser, hangs around for longer than I was expecting, 01:15.620 --> 01:16.460 but I think it does. 01:16.460 --> 01:17.900 It does close it down properly. 01:18.100 --> 01:22.980 Uh, but uh, yeah, we'll see if it if it ever causes like a leak of browser windows being, being 01:22.980 --> 01:23.740 left open. 01:24.380 --> 01:24.780 All right. 01:24.780 --> 01:30.100 And then this you should recognize because this is the push notification that we've used many times. 01:30.300 --> 01:35.460 Uh, it's just a simple function that sends a push notification using requests using the push over URL. 01:35.860 --> 01:39.300 And then there's also one here that's new. 01:39.340 --> 01:41.500 This is get file tools. 01:41.620 --> 01:49.020 And this is using the file management toolkit which is from right here from Langshan community. 01:49.180 --> 01:55.820 And this is an example again of one of the things that is so amazing about the Lang chain ecosystem. 01:56.020 --> 01:56.940 It's so popular. 01:56.940 --> 02:01.270 So many people have used it that there are tons and tons of tools that you can tap into. 02:01.510 --> 02:06.270 Now in the in week six, we're going to get very excited about Mchp, which of course has taken the 02:06.270 --> 02:07.030 world by storm. 02:07.070 --> 02:13.270 And Mchp is so amazing because it's sort of unleashed the ability to connect together with different 02:13.270 --> 02:14.710 tools all over the place. 02:14.990 --> 02:21.230 But already people who are part of the long chain ecosystem have had the advantage of a lot of of tools 02:21.230 --> 02:29.270 that can form to lang chains tools format, which in itself is like a sort of a mini MCP, but but just 02:29.270 --> 02:34.790 for people in in Lang chain and so we can take advantage of this now we'll have more in week six, but 02:34.790 --> 02:40.510 for now we have access to all of the ones that come in the Lang Chain Toolkit, which includes this 02:40.510 --> 02:48.670 file management, one which will give tools for our LM to be able to to mess around in a directory that 02:48.670 --> 02:50.110 I'm setting sandbox. 02:50.110 --> 02:53.670 And it will be it will be forced to stay in that root directory. 02:53.870 --> 02:54.950 So that's good. 02:55.830 --> 03:01.570 And then so so I'm going to put together these different tools into this, this um, uh, this this 03:01.570 --> 03:02.490 collection of tools here. 03:02.490 --> 03:03.610 So we're going to take a push tool. 03:03.610 --> 03:07.330 We're going to define a tool around that, that push notification. 03:07.330 --> 03:09.730 So this is like a homegrown tool to do that. 03:10.010 --> 03:13.170 Um, there's file tools that we've just talked about. 03:13.490 --> 03:17.290 There's uh a tool to run the Serpa search. 03:17.290 --> 03:20.570 So this is again us manually creating a tool around the search. 03:21.050 --> 03:23.530 There's something called the Wikipedia tool. 03:23.770 --> 03:26.730 Uh, so we create a Wikipedia API wrapper. 03:27.010 --> 03:27.890 Uh, we create that. 03:27.890 --> 03:32.090 And that needed me to install a Wikipedia Python package in our environment, which I did. 03:32.370 --> 03:40.530 And then this tool is able to call and collect Wikipedia pages using Wikipedia's API, which is freely 03:40.530 --> 03:41.730 available to everybody. 03:41.730 --> 03:46.690 And so that gives our LLM expertise about stuff through Wikipedia. 03:47.010 --> 03:52.250 And then I also create here a Python tool. 03:52.370 --> 03:54.410 Maybe it'd be nicer just to show that separately. 03:54.450 --> 03:57.130 Python wrapper. 04:02.350 --> 04:04.910 So this means this is slightly nicer. 04:05.430 --> 04:15.670 This means that we are giving our, our, uh, LLM the ability to run Python code, much as if you just 04:15.670 --> 04:21.630 typed Python at the command line in one of these kinds of interfaces where it can put in some code and 04:21.630 --> 04:23.110 get back the answer. 04:23.390 --> 04:25.630 So we're giving it that power. 04:25.790 --> 04:29.190 And this this is something which isn't sandboxed. 04:29.190 --> 04:35.350 So it's unlike when we did this with crew when we ran Python code within a Docker container. 04:35.350 --> 04:37.430 So it was somewhat insulated from the world. 04:37.430 --> 04:38.950 This is not insulated. 04:38.950 --> 04:40.750 And so this should be used with caution. 04:40.750 --> 04:43.830 And if you're not comfortable with it then you should comment this out. 04:44.110 --> 04:49.110 Uh, and so just just by or just remove it from here, I mean remove it from these tools. 04:49.110 --> 04:54.630 If you're not happy with your LLM being able to run Python code on your computer. 04:54.910 --> 05:03.080 Uh, and, uh, but but for me, I'm, I'm especially as I'm using GPT four mini, I'm quite comfortable 05:03.080 --> 05:04.400 that it's going to be sensible. 05:04.400 --> 05:07.800 And besides, I'll monitor it and be be be careful with it. 05:07.800 --> 05:11.440 So as long as you're careful, as long as you show caution, it should be completely fine. 05:11.640 --> 05:14.080 Um, but do be aware of what we're doing there. 05:14.080 --> 05:20.080 And if you're not comfortable, if you have any doubts at all, then remove that tool from the list. 05:20.360 --> 05:21.360 You have been warned. 05:21.960 --> 05:27.520 And you can also, of course, just, uh, remove the the the, um, playwright tools. 05:27.520 --> 05:30.880 If you don't like it, just have it return something empty there instead. 05:31.160 --> 05:33.760 Uh, instead of returning those, uh, those tools. 05:34.800 --> 05:43.320 Okay, so that is our set of tools, all the tools that we want to arm our coworker, uh, sidekick 05:43.320 --> 05:43.840 with. 05:44.040 --> 05:47.160 And you can just add more tools to this other tools list. 05:47.160 --> 05:50.400 Anything you put in there will just automatically get used by our sidekick. 05:50.440 --> 05:55.680 You can just keep putting more and more and more things in there, and you can Google or ask ChatGPT 05:55.800 --> 06:03.340 about some of the tools that are in the long chain tools and the community folders like community utilities 06:03.340 --> 06:05.860 and experimental tools and tools. 06:05.860 --> 06:08.060 You can look at these or you can just look at them on chains. 06:08.060 --> 06:08.940 Documentation. 06:08.940 --> 06:12.500 There's a lot to choose from and you can just put them all in here. 06:12.500 --> 06:17.100 You can have a tool that can look at your Google Calendar and attach to to Google, so that it could 06:17.140 --> 06:18.540 schedule things for you. 06:18.540 --> 06:20.180 You can have all sorts of tools. 06:20.220 --> 06:21.300 There's so many. 06:21.340 --> 06:27.860 And that's the beauty of this project that you can just keep giving more and more capabilities to your 06:27.860 --> 06:30.020 LLM, that to your agent. 06:30.060 --> 06:31.100 That's the idea. 06:31.140 --> 06:31.500 All right. 06:31.540 --> 06:34.540 Next up we're going to go to the big class which is sidekick. 06:34.740 --> 06:36.500 So now I've gone to the sidekick module. 06:36.540 --> 06:37.540 Here it is. 06:37.540 --> 06:43.100 So the good news is this should all be familiar to you, because it's just the same code we had in the 06:43.100 --> 06:48.700 lab just moved into a Python module, which shows how, again, if I can do another of these, like 06:48.740 --> 06:54.380 a pitch for, for using, uh, these notebooks, you can iterate on something in a notebook, you can 06:54.380 --> 07:00.120 prototype, iterate, perfect your prompts, and then move to a module and people from an engineering 07:00.120 --> 07:02.720 background will say, well, that's not the way that we write software. 07:02.720 --> 07:03.920 We have things like TDD. 07:03.960 --> 07:06.920 We have we have a very different kind of process. 07:07.200 --> 07:11.840 The thing about this kind of work is that it is much more experimental by nature. 07:11.840 --> 07:17.920 The mindset of an AI engineer is more about crafting prompts, trying different ideas, seeing how they 07:17.920 --> 07:18.400 work. 07:18.520 --> 07:23.120 And so because by its very nature, it is a bit more trial and error. 07:23.120 --> 07:29.800 And and it means that it lends itself very well to a notebook interface initially until you've got things 07:29.800 --> 07:35.200 better down, and then you move to sort of productionizing something in Python code like this. 07:35.200 --> 07:43.360 So anyways, this is the the module we define the state, the typed dict that is our state. 07:43.360 --> 07:49.560 It has the messages field which is the annotated field that that is a list that will use this reducer. 07:49.600 --> 07:55.000 And messages we have success criteria, feedback success criteria met and user input needed. 07:55.160 --> 08:00.300 These these are the things that we get back from our assessment and talking about our assessment. 08:00.300 --> 08:04.100 Our evaluation here is the structured outputs schema. 08:04.140 --> 08:08.180 The the schema for the output that we get from our LM. 08:08.580 --> 08:13.820 And we want feedback whether or not the success criteria is met and whether or not user input is needed. 08:13.820 --> 08:15.140 That's what we want back. 08:15.180 --> 08:20.780 And these descriptions are what will be provided to the LM so that it populates the structured output. 08:20.780 --> 08:22.220 Well okay. 08:22.260 --> 08:24.580 And now we have a class called sidekick. 08:24.580 --> 08:26.580 And it's a it's quite a quite a big one. 08:26.620 --> 08:29.940 And maybe this could be could could warrant some refactoring. 08:29.940 --> 08:32.580 But it's basically everything we had in the notebook. 08:33.020 --> 08:40.620 Um, there's one fussy thing about working with async code, which is that the init method, when we 08:40.660 --> 08:43.700 create this, we don't want that to be to be async. 08:43.980 --> 08:49.820 Um, but we need to be able to do some initialization that will be async, like setting up our graph. 08:49.980 --> 08:56.180 And so we have to have like a separate async, uh I can say async method, but a coroutine uh, that, 08:56.180 --> 08:58.980 that is going to be handling that part of it. 08:59.110 --> 09:04.430 And we're going to need to make sure when we when we initialize a sidekick that we can first instantiate 09:04.430 --> 09:07.590 it and then call this setup asynchronously. 09:07.830 --> 09:15.550 Um, so first the first thing I do in this setup is I call that playwright tools function coroutine 09:15.550 --> 09:19.150 that we saw in the other, um, in the, in the sidekick tools. 09:19.150 --> 09:25.390 And then I populate my tools, my browser and playwright, and then I add into tools the other tools 09:25.390 --> 09:28.710 that we also put together in the other module. 09:29.150 --> 09:29.430 Okay. 09:29.430 --> 09:33.870 And then I create my worker LM, which is in this case GPT four mini. 09:33.910 --> 09:40.390 But feel free to switch it up and bind it to tools and store that as an instance variable. 09:40.550 --> 09:41.910 And then an evaluator. 09:41.910 --> 09:44.990 LM and I also store that as an instance variable. 09:44.990 --> 09:47.830 And then I call build graph. 09:48.550 --> 09:55.350 That's going to be the the the big part of that's what we've always said is the five steps that need 09:55.350 --> 09:59.430 to happen before you can actually run your graph and do your super steps.