WEBVTT 00:00.160 --> 00:05.160 Well, I'm back, although it took slightly longer than I expected, so the run that I had just kicked 00:05.160 --> 00:09.480 off actually ended up taking several minutes, and it appeared to lose its way. 00:09.520 --> 00:13.840 Going back again and again to get more and more emails generated. 00:13.840 --> 00:17.080 So I actually cancelled it and ran it a second time and it ran fine. 00:17.080 --> 00:18.120 But it is interesting. 00:18.160 --> 00:24.800 It's a it's a cautionary tale to reflect on the fact that by their very nature, these autonomous agents. 00:24.800 --> 00:30.280 I made this statement here, and this was the answer to the, the challenge from, from last time that 00:30.280 --> 00:34.280 I say you can use the tools multiple times if you're not satisfied with the results. 00:34.280 --> 00:40.360 And that's really giving this kind of autonomy and potentially an infinite loop to the agent. 00:40.360 --> 00:44.720 And sure enough, we I don't know if it was infinite, but we certainly had some kind of a loop going 00:44.720 --> 00:45.040 on. 00:45.040 --> 00:47.400 But I ran it a second time and it was fine. 00:47.560 --> 00:50.960 If I bring up the trace, here come the traces. 00:51.000 --> 00:53.480 You this this is the one that was going wrong. 00:53.480 --> 00:59.120 You can see it had it had run 14 tools and it was taking more than 300 seconds. 00:59.120 --> 01:03.430 If I bring it up, you'll see that it was going through sales agents one, two and three and then back 01:03.430 --> 01:04.830 to one, two and three again. 01:04.830 --> 01:06.710 And I can load more and on. 01:06.870 --> 01:08.150 So it kept going. 01:08.510 --> 01:11.070 Now you'll see of course, that agent one is deep seek. 01:11.110 --> 01:12.310 Agent two is Gemini. 01:12.350 --> 01:14.990 Agent three is llama 3.3. 01:15.230 --> 01:19.950 One other thing that I didn't point out last time that I would like to mention is that you can see that 01:19.950 --> 01:22.070 these are running in parallel. 01:22.070 --> 01:27.830 They are running, of course asynchronously because again, because they're very IO bound, they are 01:27.830 --> 01:34.070 doing they're waiting for information to come back from the API call that allows async IO to be running 01:34.070 --> 01:35.550 multiple in parallel. 01:35.550 --> 01:39.990 But of course the three run and then it needs to sort of wait for. 01:40.030 --> 01:46.510 Then the sales manager will review those three runs and decide whether it wants to run them again, 01:46.550 --> 01:48.910 which apparently it did lots of times. 01:48.950 --> 01:52.750 And llama 3.3 for some reason, took a really long time on one of them. 01:52.750 --> 01:59.390 So you can see there is inherent instability with these autonomous agent frameworks. 01:59.390 --> 02:02.550 And that's something that you need to code explicitly for. 02:02.550 --> 02:02.620 Four. 02:03.460 --> 02:04.060 Okay. 02:04.100 --> 02:07.860 But now let's let's just go back and look at the trace that did better. 02:07.980 --> 02:09.620 So this is the trace that just ran. 02:09.620 --> 02:12.260 And it took a rather more pleasing amount of time. 02:12.300 --> 02:15.940 And you can see that this time it only ran each agent once. 02:15.980 --> 02:18.980 Deep seek Gemini and Llama 3.3. 02:19.180 --> 02:20.460 Each one ran. 02:20.500 --> 02:22.580 Each one came back with a response. 02:22.820 --> 02:26.060 That response was of course went to our handoff. 02:26.100 --> 02:32.660 Went to the email manager that wrote the subject, wrote the email, and then an email arrived right 02:32.660 --> 02:34.420 here in my inbox. 02:34.660 --> 02:37.740 So all has worked well in the end. 02:38.660 --> 02:42.220 So I went through that code very fast because it's something that we saw before yesterday, so hopefully 02:42.220 --> 02:43.460 it was somewhat familiar. 02:43.500 --> 02:48.940 The new angle was that we were calling multiple models and it was very easy because they are all models 02:48.940 --> 02:51.860 which have open AI compatible endpoints. 02:52.100 --> 02:57.620 Now, you may have noticed that there is one model that was missing there in the form of Anthropic's 02:57.620 --> 03:02.740 Claude and anthropic does not offer an open AI compatible endpoint. 03:02.740 --> 03:10.100 And so I understand that there isn't, as of today, an easy way to use Anthropic's Claude in lieu of 03:10.260 --> 03:13.260 those other models that have an OpenAI compatible endpoint. 03:13.500 --> 03:15.020 But there are plenty of workarounds. 03:15.020 --> 03:19.860 So there is, for example, a third party provider called Open Router that people are probably a lot 03:19.860 --> 03:26.340 of people use that, and it allows you to to set up your keys for other models, including Anthropic's 03:26.340 --> 03:26.980 Claude. 03:26.980 --> 03:30.380 And it has an open AI compatible endpoint. 03:30.380 --> 03:32.860 So you can go through open router to use Claude. 03:33.260 --> 03:39.060 Also, anthropic has the incredible MCP protocol, which we'll be talking about in the last week. 03:39.060 --> 03:43.900 And that gives us another route to be using anthropic through MCP. 03:44.220 --> 03:45.980 So more will come on that later. 03:46.020 --> 03:51.500 In the meantime, the easiest way to do this is to use models other than anthropic for this purpose, 03:51.500 --> 03:52.900 and perhaps quite soon. 03:52.940 --> 03:56.420 Anthropic's Claude will be available through the OpenAI SDK direct. 03:57.020 --> 04:03.020 Anyway, the next thing we're going to do is talk about both structured outputs and guardrails. 04:03.020 --> 04:04.220 We're going to do them together. 04:04.220 --> 04:10.970 Other, and guardrails are ways that you can put a constraint around your agent platform. 04:10.970 --> 04:12.690 It's a test that you will do. 04:12.730 --> 04:18.290 And the cool thing about guardrails, I mean, you might think if guardrails were simply like a check 04:18.330 --> 04:23.650 of, of like, um, checking that something has given the right kind of results, you could just write 04:23.650 --> 04:24.530 that in Python code. 04:24.530 --> 04:30.170 You can just obviously look test whether the results have a certain property in them or not, and throw 04:30.170 --> 04:31.370 an exception if they don't. 04:31.810 --> 04:37.450 The special thing about guardrails here is that guardrails can themselves be agents, which means that 04:37.450 --> 04:43.330 you can use an LLM to be checking that things look good at any point in your flow. 04:43.610 --> 04:44.850 Well, actually, that's not quite true. 04:44.850 --> 04:51.930 When I say at any point, guardrails actually can only be applied either to the input at the very beginning, 04:51.930 --> 04:56.850 the input of the first agent or the output of the last agent. 04:56.890 --> 04:59.770 You can't insert guardrails all over the place. 04:59.770 --> 05:01.730 You just have them at the very beginning or the very end. 05:01.730 --> 05:07.640 And they're designed to protect your model against getting an input, which is inappropriate or not, 05:07.680 --> 05:14.000 not what it's intended for, and also to protect it against producing an output which should not be 05:14.000 --> 05:15.240 shown to the user. 05:15.320 --> 05:16.840 So it's those two extremes. 05:16.840 --> 05:18.920 That's where you can implement guardrails. 05:18.920 --> 05:21.520 And we're going to implement an input guardrail right now. 05:21.520 --> 05:25.520 But as you'll see it's it's basically exactly the same thing either way. 05:25.520 --> 05:28.360 And I'll leave a challenge for you to add an output one afterwards. 05:29.000 --> 05:36.160 So the first thing I do because we're going to be working with structured outputs is I define an a class 05:36.200 --> 05:38.280 called name Check Output. 05:38.440 --> 05:43.200 We're going to be we're going to be checking for names for people's names because we don't want people's 05:43.200 --> 05:44.040 names in our emails. 05:44.040 --> 05:47.400 We decided so we're going to have a name check output. 05:47.400 --> 05:50.320 And this is one of these pedantic objects. 05:50.320 --> 05:56.240 If you remember this is where you have objects classes which are designed to reflect a particular schema 05:56.240 --> 05:57.080 of data. 05:57.240 --> 06:02.960 So what we want is that we want the name check output to be something which has two fields is name in 06:03.000 --> 06:09.470 message, which is a boolean where true should mean that there is a name in this message and name. 06:09.510 --> 06:12.830 A string would be the name if there is a name in the message. 06:13.310 --> 06:17.990 Okay, so this is all we're doing here is we're sort of defining we're defining a schema. 06:17.990 --> 06:23.190 We're saying this is a particular structure of class that that we want to to think of. 06:23.630 --> 06:27.910 And now we're going to define an agent and I'm naming it guardrail agent. 06:27.910 --> 06:28.990 You can name it whatever you want. 06:29.030 --> 06:31.270 We are going to use this in a guardrail. 06:31.270 --> 06:34.670 But this is just as of now, just another agent we're creating. 06:34.830 --> 06:36.790 Its name is going to be Name check. 06:37.030 --> 06:43.390 The instructions are check if the user is including someone's personal name in what they want you to 06:43.390 --> 06:43.870 do. 06:44.550 --> 06:46.710 The output type this. 06:46.710 --> 06:48.670 This is the structured outputs piece. 06:48.710 --> 06:54.710 We are telling this agent we don't want you to output text, which is what you normally do by default. 06:54.710 --> 07:02.110 All of these are outputting just strings of data, but rather we want you to output a name check output. 07:02.150 --> 07:07.110 We want you to output an object that conforms to this schema. 07:07.390 --> 07:09.620 And you can use this for any of your agents. 07:09.620 --> 07:13.700 So we could go back and use this as a way to structure our emails better. 07:13.700 --> 07:18.940 So we could use this so that rather than emails being just strings, they could actually already have 07:18.980 --> 07:21.140 a subject, a recipient, a body. 07:21.180 --> 07:22.940 We could do it that way from the get go. 07:22.940 --> 07:24.100 And that would be much nicer. 07:24.100 --> 07:26.140 And that is in the challenges at the end. 07:26.580 --> 07:29.820 But for here we specify the output type is name check output. 07:29.820 --> 07:34.300 And as before we specify a model GPT four mini. 07:34.460 --> 07:37.820 And that brings us simply to the guardrail. 07:38.020 --> 07:42.220 So a guardrail is it's like an asynchronous. 07:42.260 --> 07:43.300 It's a coroutine. 07:43.420 --> 07:44.380 It's going to say a function. 07:44.380 --> 07:47.020 But it is of course strictly speaking a coroutine. 07:47.340 --> 07:54.820 And you simply decorate it with input guardrail to say, I'm going to be using this as an input guardrail. 07:54.860 --> 07:57.340 It's a bit like decorating something to be a tool. 07:57.740 --> 08:03.180 And there's a completely analogous one called output guardrail, which is what you would use if it was 08:03.180 --> 08:04.940 to be used as a guardrail for the output. 08:05.460 --> 08:11.700 You can call it whatever you want and it takes the context is a bunch of data that you can pass between 08:11.700 --> 08:15.380 agents that we we haven't made use of the agent itself. 08:15.540 --> 08:18.460 And this is the input, the input that's coming in. 08:18.500 --> 08:19.580 I've called it message. 08:19.820 --> 08:25.380 And so what we can do right here is we can as part of implementing this guardrail, we can actually 08:25.420 --> 08:27.980 kick off this guardrail agent. 08:27.980 --> 08:30.020 We can run this guardrail agent. 08:30.540 --> 08:33.620 And it will it will be giving it these instructions. 08:33.620 --> 08:39.540 It will be passing in the message and it will be expecting a name check output to come out of that. 08:40.220 --> 08:43.420 So this is basically going to do just what it says on the tin. 08:43.420 --> 08:49.340 It's going to check if someone's personal name is being included in the message. 08:50.140 --> 08:52.300 And then you have to return. 08:52.300 --> 08:58.100 The rule about writing these guardrails is you have to return something called a guardrail function 08:58.140 --> 08:58.820 output. 08:59.180 --> 09:05.220 That's that's always a thing you return, which is informing the system whether or not there is a problem. 09:05.420 --> 09:07.140 And it has two two things. 09:07.180 --> 09:14.490 Output info is just a dictionary of stuff that might be useful for tracing and trip wire triggered is 09:14.490 --> 09:15.530 a boolean. 09:15.570 --> 09:16.410 It's true. 09:16.410 --> 09:19.130 If there's a problem you pass in true here. 09:19.130 --> 09:25.610 If something has happened which should trip the trip wire and alert the guardrail has not been met and 09:25.610 --> 09:26.890 there is a problem. 09:26.890 --> 09:34.010 So what we're doing here is we're saying if this agent that we just ran, this agent produced an object 09:34.170 --> 09:41.730 of type name check output if that has is name and message set to true, which means that this agent 09:41.730 --> 09:49.090 reckons that there's someone's personal name in the instructions, then that should trigger the trip 09:49.090 --> 09:49.610 wire. 09:49.770 --> 09:52.810 That should cause this guardrail to fail. 09:52.810 --> 09:54.290 And there should be a problem. 09:54.730 --> 09:56.050 That's that's the key. 09:56.050 --> 09:57.210 That's how it works. 09:57.210 --> 09:58.290 I hope that makes sense. 09:58.290 --> 10:00.530 And you could use exactly this structure. 10:00.530 --> 10:04.610 And then of course for output guardrails, it's pretty much the same thing. 10:04.610 --> 10:09.010 But instead of message you would put the output there otherwise exactly the same. 10:09.490 --> 10:10.130 All right. 10:10.130 --> 10:11.210 Let's give this a try.