WEBVTT 00:00.160 --> 00:04.160 Next we're going to peel back the onion a little bit more on workflows. 00:04.320 --> 00:11.840 Looking at different types of workflow, anthropic identifies five different design patterns that you'll 00:11.840 --> 00:16.320 find when building a genetic systems that have workflows in them. 00:16.320 --> 00:18.840 And let's go through each of them and talk about them. 00:19.280 --> 00:20.240 This is the first one. 00:20.240 --> 00:21.960 They call it prompt chaining. 00:21.960 --> 00:23.160 And it's a simple one. 00:23.400 --> 00:26.800 What you're seeing here in this diagram obviously the in and the out on the left and the right is the 00:26.800 --> 00:28.360 beginning and the end of the workflow. 00:28.720 --> 00:32.440 The yellow boxes represent calls to models to Llms. 00:32.440 --> 00:37.680 And the blue boxes is where you potentially have just some code that you've written, some some software 00:37.960 --> 00:38.920 and it's optional. 00:39.240 --> 00:44.800 And what you'll see there is that this is simply saying you could have an LLM carry out some task and 00:44.800 --> 00:49.520 then potentially based on some code, you could then pass that to a second LLM and that output could 00:49.520 --> 00:50.680 go to a third LLM. 00:50.680 --> 00:52.320 And that could be the conclusion. 00:52.520 --> 00:55.480 And actually this is really similar to what you just did. 00:55.520 --> 00:58.040 This is similar to that commercial question. 00:58.040 --> 01:00.560 When we ask an LLM first pick a sector. 01:00.560 --> 01:04.240 And then given that sector pick a pain point and then pick a solution. 01:04.240 --> 01:06.020 And obviously it doesn't need to be three LMS. 01:06.020 --> 01:08.220 It can be as many as you want, but this is the idea. 01:08.220 --> 01:14.020 You are chaining a series of LM calls decomposing into a fixed set of subtasks. 01:14.220 --> 01:21.500 And the reason you might want to do that is because you can take care to frame each LM call very precisely 01:21.500 --> 01:27.100 to get the the best, the most effective LM response based on that prompt. 01:27.140 --> 01:32.700 So it allows you to to really work on each of those tasks being very effective. 01:32.700 --> 01:39.300 And then keep your whole process, your workflow on guardrails by taking it step by step through a sequence 01:39.300 --> 01:41.300 of well-defined tasks. 01:41.940 --> 01:44.260 Now I want to make a point that you might have already thought of. 01:44.740 --> 01:52.220 Whilst anthropic is quite clear to define this as a workflow and not an agent pattern, it is perfectly 01:52.220 --> 01:59.740 possible for prompt chaining workflow like this to be giving the LM some discretion on what activities 01:59.740 --> 02:05.060 happen, because just as we did in our example, the first call to the LM might come up with a topic, 02:05.060 --> 02:09.570 and it's that topic that gets worked on by LMS two and three. 02:09.770 --> 02:16.490 So in some ways, the way that anthropic distinguishes between workflows and agents, it's a little 02:16.530 --> 02:17.730 bit artificial. 02:17.770 --> 02:20.090 There is definitely a blurred line between them. 02:20.090 --> 02:26.810 It's perfectly possible for there to be an element of autonomy about a workflow pattern like this. 02:27.530 --> 02:28.330 Onwards. 02:28.330 --> 02:33.970 The second design pattern is called routing, and this is where an input comes in. 02:34.210 --> 02:42.810 And an LLM has the task of deciding which of multiple possible models are selected to carry out this 02:42.810 --> 02:43.450 function. 02:43.690 --> 02:47.170 And the idea is that you might have specialist models in here. 02:47.170 --> 02:49.250 They're shown by LLM one, two and three. 02:49.290 --> 02:51.530 And they're each good at different tasks. 02:51.530 --> 02:57.930 And the router's job is to classify the task, understand which of the specialists will be best equipped 02:57.930 --> 02:59.330 to tackle this task. 02:59.490 --> 03:05.570 And it allows for, as I say here, separation of concerns for being able to have different LMS that 03:05.570 --> 03:12.990 have different levels of expertise and have them and have an LLM decide how to route to those experts. 03:13.390 --> 03:15.350 So this is a powerful model. 03:15.350 --> 03:16.790 It's very common. 03:16.950 --> 03:22.990 And it's something which again, I would say I would argue there is some it's a bit artificial to say 03:22.990 --> 03:28.990 that there's no autonomy here, because clearly that router is able to make some decisions, albeit 03:29.030 --> 03:29.950 on guardrails. 03:29.990 --> 03:33.190 Again, it still has to follow a given workflow. 03:33.990 --> 03:36.350 And on to the third design pattern. 03:36.390 --> 03:41.270 Parallelization, which is at first blush, might look quite similar to what we just talked about. 03:41.310 --> 03:43.390 But there is a lot in common with these design patterns. 03:43.390 --> 03:45.830 But the idea here is that you have. 03:46.230 --> 03:49.830 Remember, the blue boxes represent code R code, not an LM. 03:49.830 --> 03:56.470 So we write some code that takes a task and breaks it down into multiple pieces that should all run 03:56.470 --> 03:57.430 in parallel. 03:57.430 --> 04:01.430 So in the previous design pattern it was an LM doing the routing. 04:01.470 --> 04:06.870 And it was one of these three experts that was given the task in this design pattern. 04:06.870 --> 04:07.950 It's code. 04:07.990 --> 04:11.910 It's let's say Python code that's deciding what to do or how to coordinate. 04:11.910 --> 04:18.900 And in parallel, it's being sent to three llms to carry out three different activities concurrently. 04:18.900 --> 04:20.900 And then there's more code. 04:20.900 --> 04:24.580 Maybe it's Python code that takes those answers and stitches them together. 04:24.780 --> 04:29.940 And actually, anthropic makes the point that these tasks don't necessarily need to be different tasks. 04:29.940 --> 04:36.140 You could imagine a situation where you send the same task and have it be done three times, and you 04:36.140 --> 04:39.180 use the aggregator, perhaps to take the average or something like that. 04:39.180 --> 04:43.140 So there are various situations where this might be the same thing that you're doing three times doesn't 04:43.140 --> 04:49.340 need to be different, but I think most commonly it's when there are multiple subtasks happening concurrently. 04:49.500 --> 04:51.660 That is the parallelization. 04:51.700 --> 04:54.420 It's hard to say of the design patterns. 04:54.900 --> 04:55.460 Okay. 04:55.500 --> 04:58.780 And now this one orchestrator worker. 04:59.180 --> 05:07.060 This is when a difficult, challenging task is broken down and recombined. 05:07.420 --> 05:10.260 And you might be thinking, hang on, isn't that just what we just did? 05:10.300 --> 05:11.260 I just flick between them. 05:11.260 --> 05:14.300 Don't these two slides, these two design patterns are really similar? 05:14.340 --> 05:17.980 Well, you'll notice a subtle color change along with the change in terminology. 05:18.180 --> 05:25.280 The key point is that this is exactly like the prior design pattern, except it's no longer code that's 05:25.280 --> 05:26.600 doing the orchestration. 05:26.600 --> 05:27.720 It's an LLM. 05:28.040 --> 05:35.880 So you are using a model to break down a complex task into smaller steps, and then you are using a 05:35.880 --> 05:38.520 model to combine the results. 05:38.520 --> 05:45.920 And so this is a much more dynamic kind of system where the orchestrator can choose how to divvy up 05:45.920 --> 05:46.680 the task. 05:46.800 --> 05:54.200 And again, I would argue that it's quite artificial to categorize this as a workflow rather than an 05:54.240 --> 05:59.560 agent pattern that we'll look at in a minute, because clearly that orchestrator has discretion over 05:59.560 --> 06:04.920 how it divvies up the tasks, and it could choose how many different llms get assigned the activity. 06:04.920 --> 06:09.480 So saying that this is just a fixed workflow is perhaps a stretch. 06:09.480 --> 06:12.000 So, you know, I think you get the overall idea. 06:12.000 --> 06:17.680 There's certainly more constraints on this than when we come to the more flexible agent flows. 06:17.680 --> 06:25.950 But but generally the idea here LLM breaks down the task Llms carry out each expert task, LM synthesizes 06:25.950 --> 06:30.750 the task for an output that is pattern for the orchestrator worker. 06:31.270 --> 06:34.030 And now on to pattern five. 06:34.070 --> 06:38.710 The final workflow pattern and the one that I use most commonly. 06:38.710 --> 06:45.310 This is one that I come against all the time, and it's called an evaluator optimizer by anthropic. 06:45.350 --> 06:49.510 I tend to call them just evaluators or validation validation agents. 06:49.510 --> 06:52.990 You hear that a lot, but the idea is it's very simple. 06:53.150 --> 06:55.590 You have an LM that's doing your job. 06:55.590 --> 06:57.470 Let's call it the LM generator. 06:57.510 --> 06:58.710 It's it's doing something. 06:58.710 --> 07:02.190 And it comes up with a solution shown in this white arrow here. 07:02.470 --> 07:07.390 And you have a second LM that's playing the role of evaluator. 07:07.390 --> 07:10.390 It's there to check the work of the first LM. 07:10.510 --> 07:16.590 And it's given any extra information, any context, everything to, to arm itself to, to not be trying 07:16.590 --> 07:21.550 to generate content, but check the work of a prior LM. 07:21.910 --> 07:26.470 And based on that it can choose to either accept or reject the work. 07:26.510 --> 07:30.140 If it accepts it, then that's it goes to the output. 07:30.140 --> 07:31.260 If it rejects it. 07:31.300 --> 07:32.860 It should come up with a reason. 07:32.860 --> 07:36.940 And the rejection and the reason goes back to the LM generator. 07:36.940 --> 07:41.620 And that can then choose to come up with another solution which comes back here. 07:41.940 --> 07:45.540 And so you can see this, this sort of feedback loop setup. 07:45.660 --> 07:47.180 It's very powerful. 07:47.300 --> 07:55.700 Of course, one of the key concerns of building production systems with LMS is about accuracy, predictability, 07:55.740 --> 07:59.940 robustness of the responses and having validation agents. 07:59.940 --> 08:07.940 Having these kinds of evaluator optimizer flows in your LMS solutions is a really powerful way to increase 08:07.940 --> 08:11.540 the accuracy and build more guarantees. 08:11.580 --> 08:17.020 There's never full guarantees with LMS, but build a higher level of guarantee around the quality of 08:17.060 --> 08:18.300 the final output. 08:18.300 --> 08:21.260 So this is a really effective pattern. 08:21.300 --> 08:26.860 I use it all the time, and of course we'll have plenty of examples on the course, and you should be 08:26.860 --> 08:31.620 able to think about how you could apply this to LM solutions in your day jobs right away.