WEBVTT 00:00.440 --> 00:01.480 Well, hello. 00:01.520 --> 00:02.240 I'm back. 00:02.240 --> 00:03.960 And you're back too, which is a great thing. 00:04.000 --> 00:07.920 It means it's time for us to start day three of our journey together. 00:08.200 --> 00:13.040 And day three, we've already done building an agentic workflow for the first time. 00:13.200 --> 00:14.480 And we've talked about patterns. 00:14.480 --> 00:19.520 And now we're going to talk about orchestrating between LMS and many of them. 00:20.000 --> 00:22.960 So this is going to be a practical day. 00:22.960 --> 00:24.080 You'll be pleased to hear. 00:24.080 --> 00:25.920 It's about time we got some more coding. 00:25.920 --> 00:28.320 And we're going to be calling a lot of LMS. 00:28.600 --> 00:31.560 And I just want to of course, say a few things up front. 00:31.800 --> 00:37.280 We're going to be calling both paid APIs and also open source models, and we're going to be calling 00:37.280 --> 00:42.360 them both in the cloud and also open source models locally and doing it throughout this course. 00:42.560 --> 00:48.400 And I want to be clear that you have complete flexibility to decide which models you pick, at which 00:48.400 --> 00:53.600 point I'm going to have coded it one way, but a great exercise is to take what I've done and apply 00:53.600 --> 00:54.880 it to other models. 00:54.880 --> 00:57.640 And if you don't want to spend a dime, you don't need to. 00:57.680 --> 01:00.280 You can do this all using local models. 01:00.280 --> 01:05.050 Although performance might vary, you can see the results I get from the models I use, and try and 01:05.050 --> 01:08.770 see what you can achieve with free open source models as well. 01:09.250 --> 01:15.290 And if if you want more information on what it's like to select models, whether they're open or closed 01:15.290 --> 01:19.290 source, apply them and deploy them, then you should take a look at my other course. 01:19.330 --> 01:23.610 We're not going to be covering that in detail here, because otherwise I feel like it's going to be 01:23.610 --> 01:26.410 too much of a of a rabbit hole and we'll get distracted. 01:26.410 --> 01:31.690 So I'm assuming you're coming into this knowing about the different kinds of models and having a sense 01:31.690 --> 01:34.650 of what makes sense for closed source, open source, and so on. 01:34.890 --> 01:38.970 If you don't have that, you can either just go along with it, just just sort of pick it up as we go. 01:39.210 --> 01:43.330 Or of course, you could always turn to look at my other course, or I'll try and put some more background 01:43.330 --> 01:46.170 information in the guides as well, because that won't be required. 01:46.170 --> 01:50.650 You can just just go, go with the flow in terms of how we pick models. 01:51.330 --> 01:56.490 So let's talk about the cast of characters, the different models that we're going to be experiencing 01:56.490 --> 01:56.890 now. 01:57.290 --> 02:00.050 So the first model needs no introduction really. 02:00.090 --> 02:05.090 It is of course, the model GPT four mini Many from open AI. 02:05.290 --> 02:07.530 We've already used it in a couple of calls. 02:07.530 --> 02:11.090 It's for sure the most well known of the models out there. 02:11.090 --> 02:15.370 And of course, there's also GPT four, the bigger cousin of GPT four. 02:16.050 --> 02:21.330 And then there are the reasoning models, which are models that have been trained to think through their 02:21.330 --> 02:26.530 steps in an Agentic like way, in like a workflow of thinking through the different steps before they 02:26.530 --> 02:27.650 arrive at their conclusion. 02:27.650 --> 02:33.330 Because it turns out that when you ask an LLM to think through its steps, you get much better outcomes. 02:33.810 --> 02:39.330 So we may take a look at some point at oh one and oh three mini, but it's less essential for this course. 02:39.330 --> 02:40.010 Most of the time. 02:40.010 --> 02:45.810 We're going to be sticking with GPT four mini now and OpenAI's great rival. 02:45.810 --> 02:48.370 Of course, their competitor is anthropic. 02:48.410 --> 02:53.170 That was actually started by a couple of people from OpenAI originally, and we'll be looking at some 02:53.170 --> 02:53.730 of their models. 02:53.730 --> 02:58.730 But Claude 37 sonnet is the model that I will spend most time on. 02:58.850 --> 03:04.290 And if you want to have a cheaper version, you can go with the Claude three haiku, which is significantly 03:04.290 --> 03:05.210 lower cost. 03:05.620 --> 03:09.100 But but Sonet is also fairly cheap for Google. 03:09.100 --> 03:12.180 We're going to be using Gemini two zero flash. 03:12.220 --> 03:15.300 There is also the Pro version of that too, but I think we'll stick with flash. 03:15.300 --> 03:20.980 And as of right now, flash is actually free at least as long as you use it within certain usage limits. 03:21.020 --> 03:25.940 I don't know how long that will be the case for, but by all means, if you want to use an open frontier 03:25.980 --> 03:29.820 model without paying for it, then Gemini might be your path. 03:29.860 --> 03:33.180 Do look into that deep Sikh. 03:33.180 --> 03:40.100 Deep Sikh, of course, is the Chinese upstart startup that shocked us all by coming up with such a 03:40.100 --> 03:43.580 powerful model in the form of Deep Sikh V3 and R1. 03:43.580 --> 03:48.660 And it's important to understand that what made Deep Sikh so sensational was not necessarily that their 03:48.660 --> 03:51.380 model was the strongest in the world, because it wasn't. 03:51.420 --> 03:58.020 It was slightly behind the latest from open AI, but that they developed such powerful techniques to 03:58.060 --> 04:06.260 train Deep Sikh to be that good, that it cost them a fraction of the spend that OpenAI had spent to 04:06.350 --> 04:09.870 train GPT four and train GPT 40101. 04:10.150 --> 04:16.110 Deep seek was able to achieve very similar performance, pretty much comparable at a fraction. 04:16.110 --> 04:18.830 I think it's like 30 times less spend. 04:18.870 --> 04:20.590 That was the true innovation. 04:20.590 --> 04:22.590 That's the remarkable thing about deep seek. 04:22.670 --> 04:25.350 And also that they open sourced the model so that you can use it. 04:25.350 --> 04:32.470 But the main the major model has 671 billion parameters, which means it's far too big for anyone to 04:32.510 --> 04:33.950 run that on their computers. 04:34.150 --> 04:40.070 But there are versions of it, small versions of it called the distilled versions, which are in fact 04:40.070 --> 04:42.230 themselves just smaller models. 04:42.230 --> 04:48.950 They're versions of Llama and Quen, two different models that have been fine tuned on data generated 04:48.950 --> 04:56.710 by the big Deep Seek and those smaller distilled versions of Deep Seek for sure available free of charge. 04:57.150 --> 04:57.910 Grok. 04:57.950 --> 05:02.510 We will also be using grok, and there are two groks. 05:02.910 --> 05:10.040 Confusingly, if you don't know this grok spelt with a k at the end is the name of the model that comes 05:10.040 --> 05:12.200 from from the company formerly known as Twitter. 05:12.240 --> 05:16.600 Now x x is grok and we might use grok with a k at some point as well. 05:16.640 --> 05:20.120 Not not in today's lab, but grok with a Q is something different. 05:20.160 --> 05:29.560 Grok with a Q is a company that has come up with a really cheap, fast way to run inference runtime 05:29.560 --> 05:37.040 models like llama 3.3, which is the massive version of llama with 70 billion parameters. 05:37.040 --> 05:44.960 So you can run llama 3.3 really fast, really low cost on Groks infrastructure and along with many other 05:44.960 --> 05:48.280 open source models including Deep Seek variants. 05:48.280 --> 05:51.000 So grok is great to use for that. 05:52.280 --> 05:53.880 And then Olama. 05:53.880 --> 05:58.280 So Olama is itself a more of a platform. 05:58.280 --> 06:05.360 It's something that you can use to run something locally that provides endpoints locally that are consistent, 06:05.360 --> 06:11.560 very similar to the endpoints that OpenAI and other models here have so that you can make local calls 06:11.560 --> 06:18.080 to an API, which is in fact going to just run an open source model locally on your computer in high 06:18.080 --> 06:22.360 performance optimized cplusplus using a library called llama CPP. 06:22.880 --> 06:26.040 And so a llama is something that we will use as well. 06:26.760 --> 06:30.400 Now, if some of these terms are unfamiliar to you, terms like inference. 06:30.440 --> 06:34.600 If you're not sure about that, if you don't know, if you don't fully understand the difference between 06:34.600 --> 06:37.760 running something over a llama or grok, then I can. 06:37.760 --> 06:43.200 I can suggest a background materials, look in the guides, and also consider whether you'd like to 06:43.200 --> 06:45.960 look at my LLM engineering course, which does cover all of this. 06:46.720 --> 06:53.640 And the final point that I will make is that you may know that I'm something of a fanatic on leaderboards, 06:53.640 --> 06:58.960 on places you can go to read about metrics and performance of different models. 06:58.960 --> 07:05.320 So much so that I was called a leaderboard, uh, humorously by my my great friend John Crone on his 07:05.320 --> 07:06.680 Super Data Science podcast. 07:07.000 --> 07:12.730 But there is a website called called leaderboard, which I've given the web address right there. 07:12.730 --> 07:18.570 And that is a great resource because it compares a number of the leading closed source and open source 07:18.570 --> 07:20.570 models together side by side. 07:20.570 --> 07:26.730 And it has things like the costs and it has the context window size, if you're familiar with that, 07:26.850 --> 07:31.210 and it has the cost, you can calculate based on the number of input and output tokens. 07:31.450 --> 07:37.810 And it's also got the performance, the results of key benchmarks across a number of dimensions. 07:37.810 --> 07:41.410 So I strongly encourage people to go and check out the vellum leaderboard. 07:41.410 --> 07:43.650 And there'll be a link in the resources. 07:43.650 --> 07:46.050 And it's great to have that bookmarked up. 07:46.090 --> 07:51.810 And as you go through different APIs, use that to gauge the the costs and capabilities associated with 07:51.810 --> 07:52.130 them. 07:53.010 --> 07:56.890 And now I'm going to put up one more time something that I said to you in the first lecture. 07:56.890 --> 08:03.130 But just to really double down on this, I want to remind you that there are great resources all over 08:03.130 --> 08:04.090 the place for this course. 08:04.130 --> 08:07.330 There's a resource for the whole course with videos and links. 08:07.330 --> 08:10.290 There's GitHub, the repo that has guides. 08:10.290 --> 08:11.570 It has the troubleshooting. 08:11.570 --> 08:14.500 And I'm always updating the labs to keep them up to date. 08:14.500 --> 08:18.140 I might add on new models, the models that we just went through a moment ago. 08:18.300 --> 08:20.700 Those are the models that right now are the latest and greatest. 08:20.700 --> 08:23.660 But as new versions come out, I will update the labs. 08:23.660 --> 08:25.380 So you've got the latest in there. 08:25.380 --> 08:29.980 And so, you know, what you've heard is, is is one story, but you'll get an even better story when 08:29.980 --> 08:31.900 you go through the labs yourselves. 08:32.300 --> 08:37.180 And I will also urge you to keep a thick skin. 08:37.460 --> 08:38.260 There's that. 08:38.260 --> 08:39.420 You will hit roadblocks. 08:39.420 --> 08:40.460 There will be problems. 08:40.460 --> 08:45.740 That's one thing I can say for sure, but see them as this is where real learning happens. 08:45.740 --> 08:46.700 It's in debugging. 08:46.700 --> 08:51.620 It's in diagnosing and figuring things out, even if it's painful to start with. 08:51.660 --> 08:57.020 It's super satisfying when you fix it and get it on track, and that is the way to do it. 08:57.340 --> 09:04.980 And and if all else fails, or as I say, or if not all else fails, you just want to then reach out 09:04.980 --> 09:06.020 and contact me. 09:06.060 --> 09:07.060 Of course, one more time. 09:07.060 --> 09:10.180 I've got my LinkedIn right there, but also you can email me. 09:10.220 --> 09:13.900 I've got my details all over the place and I like hearing from people. 09:13.900 --> 09:15.020 I'm responsive. 09:15.060 --> 09:19.940 But most importantly, I don't want you to be suffering in pain with problems. 09:19.940 --> 09:21.220 I want to be fixing them. 09:21.220 --> 09:22.460 So keep this in mind. 09:22.460 --> 09:28.540 And also, I mean ask ChatGPT you do tend to get great, great answers. 09:28.540 --> 09:30.700 Honestly, it's amazing and I tend to. 09:30.700 --> 09:36.380 When I have a really difficult question, I will often ask ChatGPT and Claude, both because, uh, 09:36.420 --> 09:41.420 sometimes you get answers that are too long winded or take you in too many different directions, and 09:41.420 --> 09:43.780 so asking a couple of different models can help. 09:43.820 --> 09:48.020 And you can also basically have like an agent workflow that you do manually. 09:48.020 --> 09:54.100 You can ask a question to ChatGPT and then to Claude, you can say, I've got this response to my question. 09:54.140 --> 09:54.820 Do you agree? 09:54.820 --> 09:55.900 Is this accurate? 09:55.940 --> 10:01.540 You can have it be the evaluator, like the evaluator optimizer pattern that we looked at a moment ago. 10:01.540 --> 10:03.980 So that's a that's really cool that you can do it manually. 10:03.980 --> 10:09.700 And it's a great technique to get good answers from Llms when you're stuck yourself. 10:10.300 --> 10:10.820 Okay. 10:11.100 --> 10:14.500 With that preamble, it's time for our next lab. 10:14.660 --> 10:15.700 Let's get to it.