WEBVTT 00:00.120 --> 00:02.360 And I've come into Ladysmith one more time just to show you. 00:02.360 --> 00:08.560 I think I've read from the wrong place that the final output from GPT was right down here. 00:08.560 --> 00:11.800 I have sent you a push notification with the current USD GP exchange rate. 00:11.800 --> 00:16.600 So that's where you saw the end of the whole of the trace right down at the bottom there. 00:17.160 --> 00:20.480 Okay, so here we are back in this chat. 00:20.480 --> 00:22.000 And now let me just show you something. 00:22.320 --> 00:29.240 Despite its incredible intelligence with this conversation, what we can now say is, uh, my name's 00:29.560 --> 00:30.000 Ed. 00:31.320 --> 00:32.200 Uh, nice to meet you, Ed. 00:32.200 --> 00:33.160 How can I assist you today? 00:33.160 --> 00:34.480 And I can say, what's my name? 00:37.720 --> 00:40.400 And it says I don't have access to your personal information. 00:40.560 --> 00:45.640 Uh, so I use this as a way of showing you that, uh, despite the fact that we have all of this state 00:45.640 --> 00:51.160 management, despite the fact that we've we've built up this, this, um, these reducers and this clever 00:51.200 --> 00:54.080 structure, it doesn't have any memory. 00:54.120 --> 00:58.560 It doesn't have the ability to remember between the different conversations. 00:58.560 --> 01:02.520 And you might be thinking, well, I guess you're not thinking because I explained it already, but 01:02.520 --> 01:04.480 but if I hadn't explained it, you might be thinking. 01:04.480 --> 01:05.480 But but hang on. 01:05.680 --> 01:06.320 Uh, why? 01:06.560 --> 01:08.120 Uh, we've got all these reducers. 01:08.120 --> 01:11.800 We've got all of this stuff to handle state and to add in state. 01:11.840 --> 01:15.560 Then why, why isn't it, why isn't it just naturally remembering what's happening? 01:15.560 --> 01:17.880 And yes, it's all down to super steps. 01:18.080 --> 01:25.320 The point is that every invocation, every time we call this graph, it's like a separate fresh invocation 01:25.520 --> 01:29.760 and state is managed and maintained within any one invocation. 01:29.760 --> 01:31.520 That's what that's what it's there for. 01:31.560 --> 01:34.960 It's to handle so that you could have multiple nodes running in parallel. 01:35.000 --> 01:39.800 You could have all sorts of stuff happening, and the state would be managed through the whole graph 01:39.920 --> 01:41.760 in a, in a reproducible way. 01:41.920 --> 01:44.400 That's the power of the state management part. 01:44.680 --> 01:48.400 But that doesn't help you between separate super steps. 01:48.520 --> 01:51.960 Each super step is a different invocation of the graph. 01:52.120 --> 01:56.760 And the way that you do this is you use checkpointing. 01:56.760 --> 01:58.520 That's how you add memory. 01:58.840 --> 02:03.240 Well, I have to tell you, I have some misgivings about Landgraaf from time to time. 02:03.240 --> 02:06.960 How heavyweight it can be on occasion with some of the things we have to learn. 02:07.320 --> 02:14.040 But this is one case where I am very impressed indeed, and it's a moment for me when I get it. 02:14.040 --> 02:19.040 I realise why people like this so much and I can really appreciate the power of it. 02:19.080 --> 02:26.200 Checkpointing is very elegant, very simple and it leads to robust, repeatable processes. 02:26.200 --> 02:27.360 And you will see that too. 02:27.400 --> 02:31.000 And I imagine you'll be impressed like me at the end of it. 02:31.560 --> 02:36.000 So we begin by creating a new object called memory saver. 02:36.160 --> 02:40.600 And a memory saver doesn't mean like a saving memories. 02:40.600 --> 02:46.480 It's talking about saving to to to in-memory stores, to in-memory, rather than to disk or to to a 02:46.480 --> 02:47.200 database. 02:47.240 --> 02:48.160 It's a bit confusing. 02:48.160 --> 02:50.520 So that's what Memory Saver means then. 02:50.520 --> 02:52.040 This is the same code we had before. 02:52.080 --> 02:53.160 Exactly the same code. 02:53.160 --> 02:57.040 I put in a print statement so we can see what it's doing and otherwise. 02:57.040 --> 03:03.690 The only change is when we compile the graph, we pass in check pointer equals memory. 03:03.730 --> 03:05.490 We're passing in that memory object. 03:05.490 --> 03:06.290 But there's our graph. 03:06.290 --> 03:08.370 It looks identical exactly the same. 03:08.810 --> 03:12.770 And now this is where we actually run the code. 03:12.930 --> 03:18.410 There's a little bit of stuff here which is a bit hacky a bit hokey, but but the the idea is you have 03:18.410 --> 03:20.330 to create this object called config. 03:20.650 --> 03:27.250 It's a dictionary with one field configurable, and then that field you have to put in a thread ID and 03:27.250 --> 03:31.650 thread ID doesn't mean like a technical thread, it's meant to be like a conversation thread, like 03:31.650 --> 03:37.930 it's something that refers to this, this thread of memories that needs to be connected together. 03:37.930 --> 03:43.530 And so that's how you specify that, that this is this particular grouping in memory. 03:44.010 --> 03:48.770 And then when you invoke the graph you have to pass in that config. 03:48.890 --> 03:53.050 That's how you make sure that when you're invoking the graph, it's being associated with the right 03:53.090 --> 03:55.090 sort of slot in memory. 03:55.370 --> 03:57.130 And that is all there is to it. 03:57.130 --> 03:59.850 So with that, let's bring this up and we say hi there. 04:01.650 --> 04:02.050 Hello. 04:02.090 --> 04:03.010 How can I assist you today? 04:03.370 --> 04:04.170 My name is Ed. 04:06.370 --> 04:07.410 Nice to meet you, Ed. 04:07.450 --> 04:08.530 How can I help you? 04:08.930 --> 04:09.930 What's my name? 04:12.490 --> 04:13.450 I think it's gonna get it. 04:14.330 --> 04:15.210 Your name is Ed. 04:15.250 --> 04:16.450 How can I help you further? 04:16.570 --> 04:17.010 Ha, ha. 04:17.410 --> 04:22.410 And no surprise, you can see down here the print statement that I mentioned is printing out the the, 04:22.570 --> 04:23.770 uh, the inputs. 04:23.770 --> 04:30.210 And we're getting the full history each time, and that is all there is to it. 04:30.250 --> 04:37.610 That is checkpointing in action, maintaining memory between each of the calls, each of the super steps. 04:37.650 --> 04:38.770 Now, I know what you're thinking. 04:38.770 --> 04:40.930 You're thinking, okay, what's the big deal? 04:40.930 --> 04:42.490 Why are you so impressed by this? 04:42.530 --> 04:46.650 I mean, it's just like you could also get there just by storing that list of disks. 04:46.650 --> 04:51.890 Or we could take it from Gradio UI here and put it in a global link and keep playing it in. 04:52.010 --> 04:54.130 Well, look, look at some of this stuff. 04:54.250 --> 04:57.850 So you can call graph the compile graph paragraph. 04:58.130 --> 04:59.250 Getstate. 04:59.410 --> 05:06.490 Given that the the config and we get back this thing called a state snapshot, and this has in it the 05:06.570 --> 05:10.850 messages and then the complete history the conversation so far. 05:11.250 --> 05:11.410 Yeah. 05:11.450 --> 05:11.770 All right. 05:11.810 --> 05:14.930 So so it's like a, like a global that that has this in it. 05:15.170 --> 05:20.610 But there's more you can get get state history for our config. 05:20.810 --> 05:28.610 And what we get now is each step in time, every super step, every time we invoke the graph, the complete, 05:28.850 --> 05:34.770 uh, snapshot at that moment, starting from most recent at the top and then going back in time. 05:35.490 --> 05:37.490 And this is where it gets cool. 05:37.490 --> 05:41.650 And it's not like it's super sophisticated, but it's definitely a cool construct. 05:41.890 --> 05:46.930 Langshan allows you to step back in time to any prior moment. 05:46.930 --> 05:53.250 When you're passing in that config, the configurable thread ID you can pass in a checkpoint ID to kind 05:53.250 --> 05:57.690 of rewind to a previous moment, and then replay that through the graph. 05:57.970 --> 06:03.650 Uh, and this, this gives you this ability to basically do what they call a time travel. 06:03.650 --> 06:09.010 That's the Landgraf official name for it, which is really to be able to move back, get your snapshot 06:09.010 --> 06:12.010 at any point in time and be able to rerun it from there. 06:12.250 --> 06:18.250 And this is this is really great because it allows you to build systems that are repeatable and robust. 06:18.290 --> 06:23.170 If something falls over, you can restart it from any snapshot, any point in time. 06:23.170 --> 06:26.890 And you've got this kind of full tracking on everything that happened. 06:27.050 --> 06:30.530 So yeah, it's it's it's simple but it's elegant. 06:30.530 --> 06:34.450 And this is a case of, of an abstraction that really makes sense to me. 06:34.650 --> 06:38.410 Uh, and is definitely something that I think is very valuable. 06:38.450 --> 06:43.610 And I just want to show you as well that this has got nothing to do with, with gradio and with kind 06:43.610 --> 06:44.330 of variables. 06:44.330 --> 06:51.450 I can come back in and I can I can just relaunch my Gradio UI right here and I can say like, hi there. 06:53.850 --> 06:55.890 It says hi again, and I was going to say, what's my name? 06:55.890 --> 06:56.700 But I don't need to. 06:56.740 --> 06:59.020 You can see right away it's got the memory. 06:59.020 --> 07:00.420 It's still got that memory. 07:00.420 --> 07:05.980 It's got nothing to do with the the information stored in radio's chat UI, which is what we've always 07:05.980 --> 07:12.220 used in the past with, with gradio, but rather it is the memory, the checkpointing that's happening 07:12.420 --> 07:15.860 inside this, uh, this memory object. 07:16.860 --> 07:23.620 And then I can actually I can come back and if I recreate, I can I can create a fresh new memory saver 07:23.620 --> 07:31.420 object and come in and then I have to re rebuild my graph with, with, uh, the new memory right there 07:31.420 --> 07:33.980 and then re bring up my chat interface. 07:33.980 --> 07:37.020 And now if I say uh hi there. 07:39.260 --> 07:40.500 And what's my name. 07:41.780 --> 07:43.580 It's obviously not going to know it. 07:45.140 --> 07:50.380 So that just explains that by recreating a new memory object we've reset everything. 07:50.420 --> 07:53.140 And that's, that gives you a good sense of what's going on. 07:53.420 --> 07:55.300 My name's Ed. 07:56.620 --> 07:57.220 Nice to meet you. 07:57.420 --> 07:58.540 How can I assist you today? 07:58.980 --> 08:00.020 Uh, what's my name? 08:02.380 --> 08:04.140 Just to prove the point, your name is Ed. 08:04.260 --> 08:06.500 And then let me just show one more thing here. 08:06.500 --> 08:10.020 We can also change the thread ID to two. 08:10.300 --> 08:14.380 Let's rerun this and, uh, take a look here. 08:14.940 --> 08:15.780 Say hi there. 08:17.340 --> 08:17.580 Hello. 08:17.620 --> 08:18.220 How can I assist you? 08:18.220 --> 08:19.140 What's my name? 08:24.060 --> 08:27.060 And it says it doesn't know because the thread is two. 08:27.220 --> 08:30.020 If we now go back to the thread is one. 08:30.060 --> 08:32.220 I know you get this, but it's worth saying. 08:32.500 --> 08:32.980 And now. 08:33.060 --> 08:33.660 Hi there. 08:35.060 --> 08:35.620 Hello again. 08:36.060 --> 08:36.500 Ha ha. 08:37.100 --> 08:37.620 So there. 08:37.620 --> 08:38.140 You see it? 08:38.180 --> 08:40.300 It's a nice, elegant. 08:40.340 --> 08:43.700 It's it's relatively lightweight and simple, but very powerful. 08:43.900 --> 08:46.780 And that's a nice combination that I can get behind. 08:46.780 --> 08:51.340 And you can come in and look of course at the different checkpoints for either thread number one or 08:51.340 --> 08:52.500 thread number two. 08:52.780 --> 08:56.140 Uh, using the same this, this same structure.