WEBVTT 00:00.440 --> 00:06.040 So next up we're going to ask anthropic and we're going to ask Claude 37 sonnet latest. 00:06.240 --> 00:12.840 So one of the things you'll see here is that Claude's API anthropic API is slightly different. 00:13.120 --> 00:17.880 As before, we're going to create a new instance of the client Python library anthropic. 00:17.880 --> 00:19.160 And we're going to call it Claude. 00:19.440 --> 00:21.520 You'll see that the API is slightly different. 00:21.520 --> 00:24.400 It's Claude create. 00:24.400 --> 00:27.000 And again César will fill that in for you. 00:27.280 --> 00:32.400 But otherwise we pass in the model name and the messages and actually anthropic needs you to specify 00:32.400 --> 00:36.200 the maximum number of tokens that's allowed to generate before it stops. 00:36.240 --> 00:38.480 OpenAI doesn't require that, but. 00:38.480 --> 00:40.320 But Claude Anthropic does. 00:40.480 --> 00:43.000 But other than that, this is exactly the same code. 00:43.000 --> 00:44.200 We have a model name. 00:44.200 --> 00:46.480 We get back the answer, we'll display it. 00:46.480 --> 00:50.480 And let's see how anthropic does when we ask. 00:50.480 --> 00:52.000 Claude 37 sonnet latest. 00:52.000 --> 00:56.000 One of the best models, that there are the same question. 00:56.280 --> 00:59.640 So it's interesting that the headlines look kind of similar. 00:59.640 --> 01:02.720 It's more concise, which may be why it ran faster. 01:02.720 --> 01:07.600 And it seems like it's a pretty clear, well-articulated answer. 01:08.080 --> 01:09.200 No surprise. 01:09.800 --> 01:12.760 Okay, so now we're going to use Gemini. 01:12.800 --> 01:17.200 We're going to call Google's Gemini to get its answer. 01:17.440 --> 01:19.640 And here is the API. 01:19.800 --> 01:23.120 And there's going to be a bit of a surprise in here if you're not aware of this. 01:23.400 --> 01:30.480 When we set up Gemini, you'll be surprised to see that we do it using OpenAI's code. 01:30.640 --> 01:32.040 And that might be super confusing. 01:32.080 --> 01:32.640 Like why? 01:32.680 --> 01:35.720 How come we're using OpenAI to call Gemini? 01:36.000 --> 01:40.960 Well, remember I mentioned to you that this code OpenAI, there's nothing fancy here. 01:40.960 --> 01:44.960 This isn't like an LLM or anything about like a neural network going on. 01:45.120 --> 01:52.720 This is a lightweight library that just wraps HTTP calls to an endpoint of a particular structure, 01:52.760 --> 01:57.920 a well-known structure that includes passing in lists of dicks and that kind of thing in the form of 01:57.960 --> 01:59.280 an HTTP request. 01:59.680 --> 02:02.790 And OpenAI built this, this, these endpoints. 02:02.790 --> 02:04.470 And they became very, very popular. 02:04.470 --> 02:09.910 And a lot of people decided that they would offer endpoints with exactly the same format, the same 02:09.950 --> 02:12.350 kind of spec as OpenAI. 02:12.510 --> 02:14.870 In fact, pretty much everyone has done that. 02:14.910 --> 02:21.150 Everyone that is, apart from anthropic, at least in today's examples, anthropic is the only the only 02:21.150 --> 02:23.350 place that hasn't gone with this. 02:23.590 --> 02:30.270 But for all of the others, they have done in such a way that and kindly OpenAI opened up their Python 02:30.270 --> 02:37.750 client library so that when you create a new Python client instance, you can pass in a base URL. 02:37.790 --> 02:42.990 You can say, look, I don't actually want you to contact OpenAI on their endpoints. 02:42.990 --> 02:46.830 I would like to switch you to using Google's endpoints. 02:47.030 --> 02:54.150 And in particular, Google has an endpoint which you'll see, again, somewhat confusingly ends OpenAI. 02:54.870 --> 02:59.350 And that's because this is Google saying, hey, look, we've got endpoints you can use to call code 02:59.350 --> 03:01.950 Gemini and they have a particular format. 03:01.950 --> 03:08.310 But by the way, we've made special ones that have exactly the same format as OpenAI, and they're served 03:08.310 --> 03:09.710 on this URL. 03:09.950 --> 03:12.750 So you can use OpenAI's client library. 03:12.790 --> 03:20.390 You can pass in our key, the Google key, and tell OpenAI to use our endpoint that's in their format, 03:20.390 --> 03:22.190 and that will work great. 03:22.510 --> 03:23.750 I hope you followed me there. 03:23.750 --> 03:26.230 If not, you're going to pick it up because we're going to do this all the time. 03:26.510 --> 03:31.830 What you're going to see is that once we've created Gemini and we're going to use Gemini two zero flash, 03:31.990 --> 03:35.830 we then call Gemini Create. 03:35.950 --> 03:37.270 It's the same, same thing. 03:37.310 --> 03:40.110 Obviously we pass in the model name and the messages. 03:40.110 --> 03:43.110 We get back the response in exactly the same way using. 03:43.150 --> 03:46.910 Now the structure that you're super familiar with will display the answer. 03:46.910 --> 03:49.030 And enough enough chitter chatter. 03:49.070 --> 03:51.830 Let's see what Gemini makes of this. 03:51.830 --> 03:55.790 How Gemini two zero flash is able to answer this question. 03:56.030 --> 03:56.990 And here we go. 03:57.150 --> 03:59.430 You can see oh it's quite quite a lengthy answer. 03:59.630 --> 04:01.630 There's a lot in there with mitigation strategies. 04:01.630 --> 04:04.230 I don't think I saw that others had that. 04:04.230 --> 04:07.550 And then a framework for ethical assessment at the end. 04:07.710 --> 04:10.510 So there's there's lots to like about this one certainly. 04:10.950 --> 04:11.670 All right. 04:11.830 --> 04:13.390 Next up is deep Seac. 04:14.150 --> 04:18.350 So there are a lot of different ways to call deep Seac which can be confusing. 04:18.550 --> 04:25.150 But in this case we're calling the largest the full sized version of Deep six model. 04:25.150 --> 04:28.190 It has 671 billion parameters. 04:28.190 --> 04:35.910 It's a big model and deep Seac runs it and provides an API to connect with it and to use that API. 04:35.950 --> 04:40.030 Deep Seac also says you can just use OpenAI's library. 04:40.030 --> 04:40.830 Just tell it. 04:40.830 --> 04:46.950 This is the base URL and as it happens, I do believe deep Seac only supports the OpenAI library. 04:46.950 --> 04:47.710 They don't have their own. 04:47.710 --> 04:49.390 They just say just just use OpenAI's. 04:49.390 --> 04:49.950 Everyone does. 04:49.950 --> 04:50.590 It's great. 04:50.830 --> 04:52.310 So that's what we do here. 04:52.310 --> 04:57.670 We call deep Seac using OpenAI's library, and we provide in a key. 04:57.870 --> 05:03.500 There are two models that Deep seek most commonly offers deep seek chat and deep seek reasoning. 05:03.540 --> 05:08.260 Deep seek reasoning is the famous R1 model, and the reason I'm not using it here is that I'm not. 05:08.300 --> 05:12.980 I want everyone to be on an even footing, and we're not going to be using reasoning models for this 05:12.980 --> 05:15.140 question, just the chat models. 05:15.460 --> 05:17.700 So other than that, it's exactly the same. 05:17.740 --> 05:19.180 It's deep chat. 05:20.140 --> 05:22.940 Create pass in the model name and the messages. 05:22.980 --> 05:24.900 Get back the answer, print it. 05:24.900 --> 05:28.740 And as before, we're adding it to competitors and to the answers. 05:28.940 --> 05:32.180 We'll give deep seek a minute to think this one through. 05:32.380 --> 05:38.220 I actually I do believe that deep seek takes longer over this because it does come up with quite a lengthy 05:38.220 --> 05:38.860 answer. 05:39.100 --> 05:43.540 So I may have to keep this sentence moving for a long time while I'm doing it. 05:43.540 --> 05:46.340 You can see that what's coming up is going to be grok. 05:46.700 --> 05:47.020 All right. 05:47.020 --> 05:48.260 But let's let's have a look. 05:48.260 --> 05:48.860 Here we go. 05:48.900 --> 05:50.420 It is quite a long answer. 05:50.740 --> 05:57.300 There are frameworks to consider a bit similar to to Gemini but looking looking very robust. 05:57.340 --> 05:57.780 All right. 05:57.780 --> 05:58.540 So that's deep. 05:58.580 --> 05:58.940 Seek. 05:59.420 --> 06:01.900 So as I as I said, we're now going to come on to grok. 06:02.300 --> 06:05.340 So remember this is grok with a Q not grok with a k. 06:05.500 --> 06:11.900 That means we're dealing with grok, the provider of fast inference on specialist hardware. 06:11.900 --> 06:15.780 They've built their own kind of hardware that's really, really fast at this. 06:15.940 --> 06:19.100 And they also support using OpenAI's library. 06:19.100 --> 06:19.900 They have their own too. 06:19.900 --> 06:21.340 But but you can use OpenAI's. 06:21.540 --> 06:25.820 You pass in their key and the base URL, which looks like that. 06:25.820 --> 06:32.100 And you can see, as with Google, they've got an endpoint which has the word OpenAI in it, because 06:32.100 --> 06:35.980 it's an endpoint that's designed to be compatible with OpenAI. 06:36.660 --> 06:45.260 The model we're going to use is llama 3.3, the latest biggest llama three, and the llama three series. 06:45.260 --> 06:47.340 And it's got 70 billion parameters. 06:47.340 --> 06:49.020 It's it's very powerful. 06:49.020 --> 06:51.020 It's very impressive for that size. 06:51.060 --> 06:58.580 I believe it's either on par with or even outperforms the 405 billion version of the llama model from 06:58.580 --> 07:00.260 from before from llama 3.1. 07:00.260 --> 07:02.380 So it's a very powerful model indeed. 07:02.580 --> 07:04.340 Otherwise this is the same. 07:04.380 --> 07:07.380 And of course, this will take a very long time because it's a big model. 07:07.460 --> 07:09.540 Of course, it won't take any time at all because it's grok. 07:09.580 --> 07:11.020 And grok is super fast. 07:11.060 --> 07:14.420 Look at all of this that got generated in just just a second or two. 07:14.580 --> 07:18.260 It is really cool using grok because it's so, so fast like that. 07:18.300 --> 07:20.860 And we're seeing again a lot of stuff that looks familiar now. 07:20.900 --> 07:23.140 I guess mitigation strategies come up a lot. 07:23.620 --> 07:24.060 Okay. 07:24.100 --> 07:26.420 And the next one we're going to we're going to switch tactic. 07:26.420 --> 07:28.660 We're going to move to using llama. 07:29.180 --> 07:33.980 So just to explain again what llama is it is a local service. 07:34.020 --> 07:37.060 It's a piece of software that runs on your local computer. 07:37.300 --> 07:42.380 And it provides an endpoint, a web service that you can call running on local host. 07:42.420 --> 07:46.180 So you'll be able to go to local host and then some ports and be able to talk to it. 07:46.180 --> 07:52.460 And the endpoint that it offers is compatible with OpenAI, the same kind of endpoint that we've been 07:52.460 --> 07:55.020 using in all of these other examples except for anthropic. 07:55.700 --> 07:57.100 It has that same endpoint. 07:57.100 --> 08:04.460 And when you hit that endpoint, it has some highly optimized Cplusplus code to run open source models 08:04.460 --> 08:06.220 locally on your box. 08:06.460 --> 08:12.700 And it should be said that it can only run small models because running it on your box directly on your. 08:12.740 --> 08:18.060 Unless you have a really big computer, unless you have like a bitcoin miner mining rig then or a big 08:18.060 --> 08:21.220 gaming box, then it's going to be problematic to run a large model. 08:21.220 --> 08:24.780 Just small models will work fine and that's what we'll do right now. 08:25.220 --> 08:31.100 So if you don't have Olama, you can install it and it's very easy to install. 08:31.860 --> 08:32.980 There's a link right here. 08:33.020 --> 08:36.020 If you click on that link it will bring up Alarm.com. 08:36.060 --> 08:40.060 You press the download button, follow a couple of instructions and it will be installed. 08:40.300 --> 08:41.740 That's all there is to it. 08:41.780 --> 08:47.820 Once you follow these instructions, you should be able to go to this here localhost my box at port 08:47.820 --> 08:51.620 11434 and you should see what I see right here. 08:51.660 --> 08:55.300 Olama is running and that tells you that all is well. 08:56.060 --> 09:02.250 So if it doesn't say that, you might need to first restart the cursor, maybe even reboot your computer 09:02.250 --> 09:08.650 and then open up a terminal with which you can do with the control and the tick button like this. 09:08.850 --> 09:14.730 And then you can type Allama serve and that that should then work. 09:15.090 --> 09:17.410 And you should then be able to go there and see it running. 09:17.770 --> 09:21.650 Here are some other useful commands that you might want to know about for Allama. 09:22.250 --> 09:28.170 But let me make this very important point that I put a super important ignore me at your peril because 09:28.170 --> 09:32.050 many people ignored it when I when I said it before and it's caused people distress. 09:32.530 --> 09:34.490 Llama 3.3. 09:34.890 --> 09:37.530 That is the same model that we use with grok up here. 09:37.570 --> 09:42.930 Llama 3.3 is a massive model with 70 billion parameters. 09:43.050 --> 09:50.210 It is way too big for most people's computers, including mine, and it takes like 100GB of space. 09:50.210 --> 09:51.050 And it's really. 09:51.090 --> 09:52.050 Or maybe it's like 60. 09:52.130 --> 09:57.770 I'm exaggerating now, but it's big and it's not something that can fit in the memory of most computers. 09:57.890 --> 10:04.850 And unfortunately, Obama does tend to have that in its sort of default recommended model, which confuses 10:04.850 --> 10:05.130 people. 10:05.130 --> 10:10.690 And some people have tried to use it, and it completely swallows up all of their computer resources. 10:11.050 --> 10:15.610 So steer clear of llama 3.3 for running on your box. 10:15.610 --> 10:22.050 Instead, use a nice model like llama 3.2, which is a nice few 3 billion parameters. 10:22.050 --> 10:28.570 You can also do an even smaller one llama 3.2 colon one b, and there's also plenty of other models 10:28.570 --> 10:29.570 that are very popular. 10:29.570 --> 10:34.210 Quen from Alibaba Cloud GEMA is Google's small open source model. 10:34.250 --> 10:40.610 Phi from Microsoft or Deep Seek, has these distilled versions of the model, which are, as I mentioned 10:40.610 --> 10:43.290 before, I think they're actually other models. 10:43.330 --> 10:50.010 They're Quen and they're llama that have been trained on Deep Six data on synthetic data generated by 10:50.010 --> 10:50.650 deep seq. 10:51.050 --> 10:55.890 To see all of the different models, you can go to this models page in llama. 10:55.930 --> 11:01.210 It's just on this model's tab, and you can read about all of the different models that are out there 11:01.210 --> 11:03.410 and see the different sizes they come in. 11:03.410 --> 11:06.890 And really, 7 or 8 billion or less is best. 11:06.890 --> 11:12.690 I'd stick with 1.5 billion or 3 billion if you can, that that's the right kind of size. 11:12.890 --> 11:17.090 And once you've done that, you can run Olama pull like this. 11:17.130 --> 11:22.890 You probably know that putting an exclamation mark in front of of a command like this in Jupyter means 11:22.890 --> 11:26.050 this isn't sorry, this isn't Jupyter in cursor. 11:26.330 --> 11:31.330 Uh, putting an exclamation mark inside a notebook like this means this isn't Python code. 11:31.330 --> 11:35.770 This is a command that needs to be run as as like a terminal command. 11:35.810 --> 11:38.490 And so it runs it in a little terminal right here. 11:38.490 --> 11:40.090 So it's worth knowing that trick. 11:40.370 --> 11:44.970 So so we run a llama pool 3.2 to make sure that we've got that model locally. 11:44.970 --> 11:47.850 And then finally, finally here we go. 11:48.290 --> 11:54.330 We use the same client library as if we're calling OpenAI, but we're not calling OpenAI. 11:54.450 --> 11:57.280 We're calling localhost our local box. 11:57.280 --> 11:59.240 This this URL right here. 11:59.480 --> 12:00.840 The API key doesn't matter. 12:00.840 --> 12:03.480 Something has to be provided, but it can be anything you want. 12:03.520 --> 12:04.560 Just put the word Allama in. 12:04.600 --> 12:05.880 There is what people tend to do. 12:06.120 --> 12:10.760 The model name llama 3.2 that must match whatever you've pulled locally. 12:10.760 --> 12:16.160 The model needs to be there and then the same chat completions create. 12:16.200 --> 12:22.600 Let's see what the little llama 3.2 that's now going to be running on my computer, chugging away through 12:22.640 --> 12:28.320 its 3 billion parameters using fast cplusplus code. 12:28.320 --> 12:29.480 And here it is. 12:29.760 --> 12:31.280 It's done a slightly mediocre job. 12:31.280 --> 12:33.640 It seems to have sort of got bored after 0.3. 12:33.680 --> 12:35.440 It's a .1.2. 12:35.680 --> 12:43.000 And uh, so uh, yeah, nice try, but but obviously this is a really challenging question. 12:43.000 --> 12:49.160 And it may be shown the limits of a smaller model like llama 3.2, but it's still a perfectly pleasant 12:49.160 --> 12:49.800 answer. 12:50.400 --> 12:51.160 All right. 12:51.200 --> 12:55.960 When we come back, we're going to actually be starting to look at everything and doing some judging.