Skip to content
Talk to us →︎
←︎ All articles

Live from the Lab: memory and context, the highlights

Live from the Lab 02: Memory and context, with Ross Pepperell. Watch the replay.

On Friday I ran the second Live from the Lab, on memory and context. If you missed it, or you were there and want the key bits again, here are the highlights in just over two minutes.

Read the transcript

So here's a question for you. Does AI learn from what you type? So if you were to tell it the name of your bank, for example, does the model know it? So if you thought the answer was yes, it's a very easy assumption to make. However, the model stays exactly as it was when training was finished, for you and for everybody else. So it only changes when a maker releases a new version.

So how does a new chat seem to actually know you? Well, most tools have an added feature called memory. So you can think of that as a sort of sheet of paper sitting next to the AI. As you chat, the tool writes down the things about you that seem useful. And at the start of your next chat, it reads that note back in. So in other words, it's not the memory the model has. It's kind of like a note the tool keeps.

The key thing is, though, you want to fix what's wrong and then delete what's out of date. So there are two things that tend to catch people out. Firstly, turning memory off isn't the same thing as deleting it. Off stops it being used. But what's saved obviously stays saved. Secondly, deleting a chat doesn't empty the note.

On a work account, your chats are company records. So admins can usually search and export them for compliance with tools like Microsoft Purview and Google Vault. So that isn't your manager necessarily browsing your chats. However, it is sensible to treat them as if someone could read them. And thirdly, memory doesn't change what you're allowed to share. Your organisation's AI policy is the thing that obviously still decides that.

[The highlights skip the live demo here. The same task was run twice: once on its own, and once with two short pieces of text pasted in first, one about a made-up company and one about how it writes.]

And those two pieces of text are what we call kind of context blocks. And each one is about one thing. They're short, typically kind of a couple of hundred words at most. So a person can actually read and check it. And it's also plain text. So it kind of goes with you with any tool. So in other words, that's your own kind of copy.

The model doesn't learn. It's the system around it that does. And that's really valuable. So if you just do one thing this week, then just have a go at looking at the memory and maybe just fix one thing, and then start a new chat and ask it again and see what the difference is.

01The model doesn't learn from you

This is the one most people get wrong, and it's an easy assumption to make. Tell your AI the name of your bank and the model itself doesn't take it in. It stays exactly as it was when training finished, for you and for everyone else, until its maker releases a new version.

So every new chat starts blank. What makes it feel like it knows you is a feature most tools now have, called memory.

02Memory is a note the tool keeps

The picture I use is a sheet of paper sitting next to the AI. As you chat, the tool writes down things about you that seem useful. At the start of your next chat, it reads that note back in.

Two things catch people out:

  • Off isn't deleted. Turning memory off stops it being used. What's already saved stays saved.
  • Deleting a chat doesn't empty the note. The note is kept separately from your chats.

And if you use AI at work, remember your chats are usually company records. Your admins may decide whether memory is on at all, and your organisation's AI policy still decides what you're allowed to share, memory or not.

03Keep your own copy

We used to tell people to leave memory off, because you couldn't see what was in it. That's changed. The main tools now let you read most of the note and edit it, so it's worth having on, and worth tidying.

The catch is that it stays in that one tool. So keep your own copy of what matters.

In the session I showed why with a live demo. I asked the AI for a welcome message for three new starters, twice. The first time it got the task on its own, and the welcome could have come from anyone. The second time I pasted in two short pieces of text first, one about the company (a made-up one) and one about how it writes. That message knew who we were and sounded like us.

Those two pieces of text are context blocks. Each is about one thing, short enough for a person to read and check, and plain text, so it works in any tool. Write the shared ones once, give them an owner, and the whole team gets the update when they change.

04Next Friday

Live from the Lab is free, every Friday from 12:00 to 12:30. Next week Jeremy takes What it can do, on Friday 27 November. Register on the events page, and if you're new, here's how a session runs.

Want help putting this to work?

We help organisations turn ideas like this into everyday practice, with training, a platform and practitioners who do the work alongside you.