WEBVTT 00:00.080 --> 00:05.120 And so here we are back in cursor and in the number five folder for Autogen. 00:05.120 --> 00:08.560 And going to lab for Autogen Core Distributed. 00:08.560 --> 00:14.560 I'm only giving a teaser of what this is about because, uh, I'm to be honest, I'm I'm not even sure 00:14.560 --> 00:16.960 how relevant this is to, to many, many of you. 00:17.000 --> 00:19.400 So I, you know, I'm going to give a flavor for it. 00:19.400 --> 00:22.800 If you do want me to go into more detail, if you want more content, then please do. 00:23.040 --> 00:25.240 Do, uh, send me a note, let me know. 00:25.280 --> 00:29.600 I can I can always go deeper into this, but I don't want to hold, uh, others up. 00:29.840 --> 00:35.960 Um, and, um, I think that particularly as Microsoft is saying, that it's, it's still experimental. 00:35.960 --> 00:37.920 It's better to get a flavor for it. 00:37.960 --> 00:38.440 Okay. 00:38.600 --> 00:44.120 So I've done this and there's actually two different ways that I want to show you this working. 00:44.360 --> 00:47.320 Uh, one of them is what I'm calling all in one worker. 00:47.360 --> 00:49.560 And one of them is with multiple workers. 00:49.720 --> 00:53.400 And so I've got this flag here that you can set from true to false. 00:53.400 --> 00:56.360 And we're going to go through it once with it true, and then once with it at false. 00:56.360 --> 00:57.960 And you'll see what I mean. 00:57.960 --> 01:01.680 So we do some imports and we load the dot env. 01:02.040 --> 01:08.970 And we start with our message, uh, class as as before, which I'm going to clear all outputs and restart. 01:09.010 --> 01:09.690 Do this again. 01:09.690 --> 01:10.130 Sorry. 01:10.570 --> 01:10.730 Uh. 01:10.730 --> 01:11.530 Run this. 01:11.810 --> 01:14.970 I start as before, with our message class defining this thing. 01:14.970 --> 01:20.810 It's the sort of analogy to the state with with Landgraf, although it's to to describe how we interact 01:20.810 --> 01:22.970 between our agents. 01:23.250 --> 01:23.770 Okay. 01:24.090 --> 01:28.650 And now I mentioned that the distributed runtime consists of these two things. 01:28.650 --> 01:30.130 One of them is the host. 01:30.130 --> 01:41.170 And this is how you create the host Autogen runtimes, gRPC, the gRPC worker agent runtime host. 01:41.170 --> 01:49.330 So this is something which uses gRPC, the remote procedure called technique to be able to to send messages. 01:49.330 --> 01:55.130 And this host will run on my local host on port 551. 01:55.130 --> 01:57.210 And I will start it with this. 01:57.210 --> 02:00.090 So this is now going to be a running host. 02:00.090 --> 02:08.580 And gRPC of course, is a cross-language, uh, approach for sending, uh, A function calls between 02:08.580 --> 02:11.660 different languages, and it's super powerful. 02:11.660 --> 02:14.860 It's used in many different, different places. 02:14.860 --> 02:16.060 You can think of it. 02:16.100 --> 02:23.220 It's like it's like making rest HTTP calls, except you're able to call directly from one function to 02:23.260 --> 02:30.780 another, and gRPC is used all over the place where, uh, where interactive messaging needs to be implemented. 02:30.780 --> 02:32.900 That crosses process boundaries. 02:33.420 --> 02:34.660 So that is gRPC. 02:34.700 --> 02:37.820 We've started a host and it is running. 02:38.060 --> 02:42.380 And what we're now going to do is, first of all, reintroduce an old friend. 02:42.420 --> 02:48.420 We're going to bring back the Autogen Sirpa tool using our Sirpa search API. 02:48.860 --> 02:53.540 And the reason we're the way we're doing it is through we're we're doing it through Lang Chain. 02:53.540 --> 03:00.060 So we're introducing a few things from, from before, uh, so we uh, we create the Google Serp API 03:00.060 --> 03:00.620 wrapper. 03:00.620 --> 03:04.540 We create a Lang chain tool for internet search. 03:04.740 --> 03:11.980 Uh, this is the same tool that we used last week, and now we wrap that in a long chain tool wrapper 03:12.180 --> 03:14.660 so that it becomes an autogen tool. 03:14.940 --> 03:21.060 And there it is, an autogen tool for searching the internet using the Sirpa API. 03:21.420 --> 03:25.980 So originally I was going to have this play rock paper, scissors in a distributed way, but I realized 03:25.980 --> 03:31.100 that it's a bit frivolous and we should be trying to at least have some commercial footing here. 03:31.100 --> 03:35.260 So I was going to make it do the the stock price comparison. 03:35.260 --> 03:38.500 And I thought, you know, we've already done some, some of them and we've got a whole week coming 03:38.500 --> 03:38.860 on that. 03:38.860 --> 03:43.460 So instead I've gone with this trying to we're trying to make a business decision. 03:43.660 --> 03:50.100 And let's say that business decision is whether we should use Autogen in a new AI agent project. 03:50.300 --> 03:58.860 And we want to have a two different agents, one agent research the pros of autogen by using web searches 03:58.860 --> 04:03.420 and the other agent research the cons, the negatives, the drawbacks. 04:03.700 --> 04:09.460 Uh, and so the two agents will go off and do their analysis, do their searching, and then they will 04:09.460 --> 04:15.030 come together and we will have a judge agent that must make a decision whether to use Autogen for a 04:15.030 --> 04:15.710 project. 04:15.950 --> 04:23.070 Uh, and, uh, it must be based purely on research from its team, from its agent team. 04:23.230 --> 04:25.150 Respond with the decision. 04:25.190 --> 04:26.270 Brief rationale. 04:26.510 --> 04:29.550 So this is sort of by analogy with the rock paper, scissors. 04:29.550 --> 04:33.510 We've got instruction one for player one, instruction two for player two. 04:33.510 --> 04:36.190 And the judge that will make the call. 04:36.470 --> 04:36.990 All right. 04:36.990 --> 04:38.670 And so now we have our agents. 04:38.670 --> 04:41.310 And this is again by analogy with last time. 04:41.310 --> 04:42.710 It's really very similar. 04:42.910 --> 04:48.190 Player one I've kept calling it player one because I you know, this really is a copy paste job basically. 04:48.390 --> 04:54.470 Uh, except I'm using GPT four or mini for both because I thought it wasn't fair to use, uh, a llama, 04:54.470 --> 04:56.990 we didn't get the same the same quality. 04:57.310 --> 05:06.230 Uh, so we've got GPT four or mini, um, and, uh, so we are this is the the the the player one routed 05:06.230 --> 05:16.640 agent, and we are passing in, uh, here we're using the Autogen gen super tool so that that is what 05:16.640 --> 05:17.320 we're doing. 05:17.320 --> 05:21.400 I realize even looking at this, that we don't actually need a player one and player two class. 05:21.400 --> 05:24.080 This could be done in a simpler way with just one. 05:24.360 --> 05:27.280 Uh, so uh, that's an obvious point. 05:27.280 --> 05:33.800 But anyways, for whatever reason, we we've got two, two agents here, two different types of agent, 05:34.000 --> 05:38.760 uh, that uh, but we're going to prompt it for whether it should find the pros or cons, but you could 05:38.760 --> 05:42.560 imagine you could switch in a different model here if you wanted, if you wanted to have deep seek, 05:42.600 --> 05:44.200 do one of the research. 05:44.200 --> 05:47.200 So I'll keep it as two separate agents in case you choose to do that. 05:47.640 --> 05:48.480 Um, okay. 05:48.600 --> 05:50.400 So other than that, this is exactly the same. 05:50.440 --> 05:56.320 Other than supplying the search tool, reflecting on tool use, everything here is the same. 05:56.600 --> 06:01.520 Uh, it delegates to its underlying LM and the judge is the same. 06:01.560 --> 06:08.920 It also has a delegate, an underlying LM, and in its message handler, the the method that's decorated 06:08.920 --> 06:14.600 message handler, it collects the two messages that we set up above. 06:15.000 --> 06:19.690 It puts them, it finds the two agents player one and player two. 06:19.970 --> 06:26.090 It uses this lookup and then it calls send message and this code is identical. 06:26.170 --> 06:31.410 This is all exactly the same as the code we just used yesterday with the runtime. 06:31.410 --> 06:32.690 That was that was local. 06:32.890 --> 06:37.450 And so the reason I do this, I want to show you that without changing anything I didn't. 06:37.490 --> 06:39.290 There's nothing here about it being distributed. 06:39.290 --> 06:41.170 It doesn't know that it's distributed. 06:41.330 --> 06:43.650 This this could still be doing rock, paper, scissors. 06:43.650 --> 06:44.890 It would be the same thing. 06:44.890 --> 06:47.330 We're just calling self dot send message. 06:47.330 --> 06:52.410 And what we don't realize is this is running remotely and it's running on a, on a runtime that's running 06:52.410 --> 06:53.610 on a port here. 06:53.930 --> 07:01.490 And uh, it's going to be the, the Autogen core is going to be handling calling the right function 07:01.490 --> 07:03.450 in the right, uh, agent. 07:03.450 --> 07:10.170 So these calls that appear to just be simply, uh, I'm calling send message, uh, right here, I call 07:10.210 --> 07:10.810 send message. 07:10.810 --> 07:15.130 And that is going to result in this handle my message getting called. 07:15.370 --> 07:20.900 And before that was just directly like some if statements in Python that we're just making that call. 07:21.100 --> 07:28.660 Now this is going to be happening using gRPC remotely orchestrated by this distributed runtime. 07:28.860 --> 07:32.340 But that is completely unknown to us as far as we're concerned. 07:32.340 --> 07:34.140 We're just doing exactly the same thing. 07:34.380 --> 07:40.060 And that is the power of Autogen core distributed that we don't have to worry about the fact that these 07:40.060 --> 07:44.660 are different processes running, and they could be written in different computer programming languages. 07:44.660 --> 07:50.780 And all of the stitching together of messages is happening for us, just based on looking up player 07:50.820 --> 07:52.260 one and player two. 07:52.580 --> 07:54.220 Um, everything is happening. 07:54.220 --> 07:55.940 So enough prattle. 07:56.100 --> 07:57.060 Let's run that. 07:57.380 --> 07:57.700 Okay. 07:57.740 --> 08:01.780 And this is where this is where the meat happens. 08:01.860 --> 08:04.420 So I've got two different implementations that I want to show you. 08:04.420 --> 08:07.500 And we're starting with all in one worker. 08:07.620 --> 08:08.900 And here's how it works. 08:09.020 --> 08:17.340 We we, uh, first of all, say that we want to create a new worker agent runtime, and we point it 08:17.340 --> 08:18.580 at our host. 08:18.620 --> 08:19.980 This is the host. 08:19.980 --> 08:26.180 So this will be a new runtime connecting to that host and we will start that worker. 08:26.420 --> 08:31.620 We are then going to register, uh, three agents with that worker. 08:31.940 --> 08:34.060 Agent one a player one, player two. 08:34.100 --> 08:35.020 And the judge. 08:35.180 --> 08:35.620 Here they are. 08:35.620 --> 08:36.100 Player one. 08:36.100 --> 08:43.340 Player two, and the judge all being registered with this worker with this gRPC worker agent runtime 08:43.500 --> 08:44.540 at that host. 08:45.060 --> 08:45.900 And there we go. 08:45.940 --> 08:48.700 There is a player one, player two. 08:48.820 --> 08:50.380 And the, the the judge. 08:50.380 --> 08:53.020 These are the factories that will create them. 08:53.300 --> 09:00.340 And we are now um, now we, uh, collect the agent ID of the judge. 09:00.700 --> 09:01.580 So I'm going to run this. 09:01.580 --> 09:06.420 And because this is set to true, only this code here is going to run and it's done. 09:06.900 --> 09:09.100 And now this is the same as before. 09:09.100 --> 09:15.780 It's just the same thing we're going to send go the go message to the agent ID that is that I've just 09:15.820 --> 09:19.420 that I set here to the judge, uh, and let's see what happens. 09:21.220 --> 09:23.900 So it's thinking stuff is happening. 09:23.900 --> 09:26.750 I'm still thinking. 09:27.710 --> 09:28.990 Messages are flying. 09:30.150 --> 09:35.310 It's what we hope is happening is that OpenAI, GPT four, and Minnie is coming up with pros and cons 09:35.310 --> 09:37.190 through two separate agent calls. 09:37.190 --> 09:40.590 And then a judge has put it all together, and this is what we get back. 09:40.630 --> 09:41.750 Pros of Autogen. 09:41.790 --> 09:45.310 Here are some advantages of using Autogen in your AI agent projects. 09:45.430 --> 09:47.510 The cons are right here. 09:47.870 --> 09:54.830 Uh, limited AI capabilities, less customizable, less structured potential bugs and performance issues 09:54.830 --> 09:56.230 with lower end models. 09:56.270 --> 10:02.630 Okay, but based on this, basically there's also some, uh, some strengths that appears, uh, memory, 10:02.630 --> 10:07.710 coherent context, ease of development, scalability, which is what we're experiencing and versatile 10:07.710 --> 10:08.670 applications. 10:08.870 --> 10:12.870 And the decision is recommend using Autogen. 10:12.870 --> 10:17.270 So the uh based purely on this research, that is the decision. 10:17.510 --> 10:23.390 And with that we will then stop our workers and we stop our host. 10:23.390 --> 10:27.830 And we have just experienced distributed autogen core.