ClarityWorkflows
Perspective 04
Yesterday's
Truth
Why memory needs time — and what AI systems miss when they remember correctly but reason from the wrong moment.
Clarity Workflows Perspectivesclarityworkflows.com
Why memory needs time — and what AI systems miss when they remember correctly but reason from the wrong moment.
AI systems are becoming increasingly good at remembering. The more interesting question is whether they know when what they remember was true. This Perspective explores temporal orientation — our largely invisible capacity to locate ourselves in time — and what happens when memory exists without it.
There is a brief moment after waking from a particularly deep sleep when I sometimes have no idea what is going on.
Not in an alarming way. More a temporary failure to assemble the world.
The room is familiar enough. I know where I am physically. But for a few seconds the rest of it is missing. What day is it? Is it morning? Why has the alarm gone off? Am I meant to be somewhere? Was there something important today?
Then pieces start returning. It is Wednesday. Ralph needs to get up for school. I have something at ten. I stayed up too late. The ordinary machinery of life starts moving again and the moment passes. I had never given much thought to what was actually happening during those few seconds. I was locating myself. Not on a map — I already knew where the bed was. I was locating myself in time.
It is something most of us do so continuously that there is little reason to notice it. We know roughly where today sits. Yesterday is behind us. Tomorrow is coming. Christmas was months ago or is three weeks away. Somebody told us something last Thursday. We have not heard from them since. Time is sitting underneath almost everything, quietly organising it.
Perhaps that is why Oliver Sacks's account of Jimmie R. in The Lost Mariner is so disturbing. Sacks met Jimmie in 1975. He was forty-nine years old, intelligent, good-humoured and capable of talking with extraordinary detail about his younger life. He remembered growing up in Pennsylvania. He remembered old telephone numbers. He could talk about science and mathematics and his service as a radio operator aboard submarines during the Second World War.
Then, somewhere around the end of the war, the continuity seemed to disappear. When Sacks asked him what year it was, Jimmie thought it was 1945. He thought he was nineteen. He was sitting in front of Sacks as a grey-haired man of forty-nine. Internally, he had not arrived there.
Sacks described him as having lost his past and his “moorings in time.”[1] I keep coming back to that word. Moorings. It suggests something slightly different from memory. A boat can contain everything it contained before and still drift if it has nothing holding it in relation to where it is. Jimmie had memories — many of them astonishingly vivid. What he seemed unable to do was use them to reach the present.
Sacks had realised that Jimmie genuinely believed himself to be a young man — not joking, not confused about a date, not being vague about his age. The world he experienced was, in an important sense, still the world of his youth.
Sacks showed him a mirror. Jimmie saw an older man looking back. He was horrified.
It is difficult to know what that moment could have felt like. We age too slowly to experience our own ageing as a sudden revelation. I can look at an old photograph of myself and think I look ridiculously young, but I have the intervening years. However incomplete my memory is, there is a route from him to me. Jimmie did not appear to have that route. Sacks quickly realised what he had done, distracted him and took him to the window. There were children outside playing baseball. A few minutes later Sacks left the room and returned. Jimmie greeted him as though they had not met.[1]
The terror had gone because the event which produced the terror had gone with it. Jimmie could think. He could converse. Sacks found plenty of intellectual ability still intact. Yet the present did not reliably attach itself to the moment before.
Normally one moment hands something to the next. We barely notice the handover. Jimmie makes it visible because, for him, so much of the handover had disappeared.
I started thinking about my own memories after rereading Sacks. Not the important ones first. Just ordinary ones. My first school. A house I lived in. An old job. A holiday. Somebody I used to know. Very few of those memories come with a date attached. Instead I know where they belong. Before I moved. After university. When Ralph was little. When I was working there. Before Dad died. The summer after that happened.
Human beings seem to construct whole private dating systems like this. They are imprecise, but they work. Someone says, “Do you remember that pub?” I cannot tell you whether it was 2007 or 2008, but I remember who I was with, which house I lived in, which version of my life the pub belongs to. The memory has a position. That position changes the memory.
“First day at school” contains more than the school. The adult remembering it knows what came afterwards. The child in the memory does not. Both people are me. That fact is so familiar it hardly sounds remarkable — but I think it is. I can picture a younger version of myself and know that I was him without feeling that I am him now. I know things he did not know. Some of what worried him no longer matters. Other things turned out to matter far more than he imagined. Time creates that relationship.
Endel Tulving used the term autonoetic consciousness when writing about episodic memory: the peculiar human capacity to experience ourselves across subjective time. Episodic memory is not simply knowing that an event happened — it can involve mentally returning to something that happened to me, from the position of the person I am now.[2] Later work on what is often called mental time travel extends the same idea forwards as well as backwards. That phrase sounds like something involving a machine covered in flashing lights. Mostly it means thinking about next Tuesday.
We use the future constantly. It affects what we do now despite not existing yet. If I promise somebody I will do something tomorrow, tomorrow acquires an obligation before it arrives. If I have an appointment next week, an imaginary future event changes where I go and what I do today. Hope is temporal. So is dread. So, for that matter, is procrastination — the future is extremely useful when I want to give something unpleasant to another version of myself.[3]
Jimmie's story is heartbreaking partly because that ordinary stretch across time has been so severely damaged. He is not simply missing entries from an internal diary. The diary itself no longer seems to behave like a continuous thing.
Once I started noticing temporal orientation, I began seeing the little devices we use to support it everywhere. Clocks. Calendars. Birthdays. School terms. Anniversaries. Seasons. Old emails. The year somebody got married. Before Covid. After Covid. When the children were small. The house before this one. We are constantly pinning things onto time.
I once found myself staring at an old photograph, disproportionately irritated because I could not work out when it was taken. I knew the people. I knew the place. Nothing important depended on solving it. Still, I started inspecting it for clues. How old did everybody look? What was I wearing? Was that before I had that car? Had we moved by then? The photograph itself had not changed. What bothered me was that I could not place it. Once I worked out roughly when it was, the irritation disappeared.
An undated memory floats. Give it a position and it reconnects with everything around it.
Jimmie's life had not literally stopped in 1945. Things happened to him afterwards. He remained in the Navy for years. His brother married. Decades passed. Yet much of that later life could not be summoned as a coherent autobiographical sequence. The fragments did not reliably join together into a history he could inhabit.[1]
This is where the filing-cabinet metaphor for memory starts to fail. Something goes in. Later we retrieve it. The drawer contains childhood; another contains names. But a life is not a collection of filled drawers. Things lead to other things. One decision changes what becomes possible afterwards. Something that happened twenty years ago can still affect how I react today, even when I have not consciously thought about the original event for years.
There is a difference between possessing memories and possessing a past.
Suppose somebody tells me: “We are thinking about moving house.” There is nothing uncertain about that statement. At that moment it is true. Six months later they may be living somewhere else. The earlier statement has not become false — they really were thinking about moving. It has changed category. It is now history.
The same thing happens constantly. “She is waiting to hear whether she got the job” is true on Monday and potentially nonsense on Friday. “We haven't decided which supplier to use” is true when the meeting begins and possibly obsolete before everybody has finished the biscuits.
We handle this without thinking very much about it because the passage of time is already built into our understanding of conversation. If somebody tells me in January that they are considering leaving their job and I see them again in September, it would be slightly odd to carry on from the exact point at which we stopped. I would ask what happened. Perhaps they left. Perhaps they stayed. Perhaps the whole crisis lasted a weekend and they had forgotten about it by February. I do not know. That is the point. I remember what was true in January. I know it is now September. Between those two things sits the possibility of change.
Not all information ages at the same speed. If somebody tells me where they were born, I am not likely to check next week in case it has changed. If they tell me they are waiting for an email, I might want an update by lunchtime. We do not carry a formal expiry date beside everything we know. Human beings somehow manage something messier: we know, imperfectly, which kinds of things change. We understand that the fact somebody was angry yesterday does not prove they are angry today. We know a plan is not the same as an outcome.
Time changes how much confidence we place in old information without necessarily changing the historical truth of the information itself. I had never thought of that as a form of intelligence. Then I started using AI systems with memory, and I noticed something odd.
I did not read Sacks, think about temporal consciousness and then wonder whether it might apply to artificial intelligence. I got annoyed with ChatGPT. That is considerably less elegant, but it is what happened.
I had been using AI across long-running conversations — some related to work, others to research and projects. The memory became increasingly useful. Instead of explaining everything again, I could sometimes return and carry on. The system knew the people involved, the earlier decisions, the documents, what we had been trying to achieve. That felt much more like collaboration.
Then occasionally something would happen that was difficult to describe at first. The system would remember correctly and still get the situation wrong. It might remember that I was waiting for somebody to reply — I had been, three weeks earlier. Or it would remember that I was considering a particular option when the decision had since been made. Nothing in the recalled information was fabricated. There was no obvious hallucination to point at. It just belonged to an earlier state of affairs.
The people were right. The project was right. The problem was familiar. The assistant seemed to know exactly what we had been talking about. Only the world outside the conversation had continued moving.
I eventually described this as an illusion of continuity: the conversation can appear coherent while the remembered context is no longer safely current.[4] That phrase came later. At first I mostly thought: why does it keep doing that?
I want to be careful with the comparison to Jimmie. An AI system is not a person with amnesia. It does not wake up frightened because years have disappeared. As far as I know, it does not experience Tuesday at all. Jimmie should not be turned into a metaphor for artificial intelligence — that would flatten a human life into a convenient technology example, which is almost the opposite of what Sacks was doing.
The connection is narrower. Jimmie exposes something about memory that is normally difficult to see: remembered information needs orientation. An AI can retrieve something from the past and reason about it extremely well — quote it, summarise it, connect it to another document and use it in a sophisticated argument. But none of those abilities necessarily answers a simpler question: where does this belong? If the system does not reliably distinguish a historical state from a current one, it can reason beautifully from premises that are no longer current. The machine can be wrong without inventing a single fact.
There is an obvious response: give AI better memory. I agree. Having to reconstruct the same project from scratch every time is tedious and inefficient. Better memory is useful. But there is a slightly odd consequence: the better the memory gets, the larger the past becomes.
Imagine an AI with access to years of email, meeting transcripts, CRM notes, policies, project documents and customer correspondence. That sounds enormously powerful. It also sounds like an archaeological dig. Somewhere in there is the first version of the proposal. The revised version. The version sent by mistake. The meeting where everybody thought they had agreed. The email sent two days later explaining that they had not. The old policy. The unofficial workaround that became normal practice. The new policy nobody has read yet.
Anyone who has worked inside an organisation will recognise this landscape. Giving a machine access to all of it solves the retrieval problem. It does not automatically solve the question of what the retrieved thing means now.
A correct fact from the wrong yesterday has credentials. It looks trustworthy because, in isolation, it is trustworthy.
One failure during my own experimentation made this particularly clear. I had two separate conversations involving similar technical subjects. The AI retrieved a timestamp from the wrong one. It was a real timestamp — it really belonged to me, it really related to the general subject. It was also the wrong evidence.[4] A fabricated date is an obvious failure. A genuine date from the wrong place has credentials. It looks trustworthy because, in isolation, it is trustworthy.
The longer I worked with the problem, the more questions appeared around it. What do I remember? When was it true? Where did I get it from? Does it belong to this conversation? Has anything happened since? And then the awkward question AI systems are not always naturally inclined to ask: do I actually know?
There is a difference between searching properly and finding nothing, and being unable to search properly. One person searches the archive and says, “I cannot find it.” Another cannot get through the archive door. Those are not the same result. AI can collapse that distinction because a clean answer is usually more satisfying than an unresolved one. “I don't know” carries information when it is accurate. So does: “I know that was true then, but I do not know whether it still is.”
My practical response to all of this began with two words: Time Trigger. Typing them told the AI, in effect, to stop and work out where the conversation was in time before we carried on. The protocol became much more elaborate during testing, because almost everything becomes more elaborate once you start trying to make it reliable. That story sits in a separate thought paper.[4] The interesting part for me now is much simpler: I had found myself explicitly instructing an AI system to perform something I would expect a human collaborator to do almost unconsciously. Notice that time has passed. That was it. The rest of the protocol was engineering around those five words.
I thought I knew where this Perspective had come from.
The story I had been telling myself went like this: the AI problem led me to Time Trigger, Time Trigger led me back to Sacks, and somewhere in the process I had arrived at a useful argument about memory and time. A tidy enough origin story.
It was not the true one.
I had finished the first draft when I went looking for some notes I had written six years earlier, at the funeral of my cousin Billy. I found them. And I realised that the question I thought I had arrived at through frustration with a chatbot had in fact been sitting in my own memory since August 2020, waiting.
I was seven or eight years old when Billy first took me fishing. I woke up one morning to Mum telling me I was not going to school — I was going with Billy instead. He was an avid fisherman. We ended up at a small pond somewhere in South London, though the journey there was half the story: we went by red double-decker bus, which remains the first and last time I have arrived at a fishing location by that particular means of transport. Billy told me to go for a walk around the pond. When I came back, I caught a fish before long.
It would be approximately fifteen years before I found out that Billy had put it on the line himself. He wanted me to have an enjoyable first experience, so he quietly arranged one. What makes the story funny rather than merely touching is that it worked — fishing did become something I went back to as a teenager, with friends, quite regularly. The manufactured experience was the beginning of something real.
That memory is over thirty years old. It has stayed with me.
Standing at the funeral in August 2020, with that memory in mind, I found myself thinking about what time actually is and what we do with it.
My son Ralph had been nine years old when he put it to me as a practical problem: why does time go so quickly when he is playing computer games but so slowly when he is at school? It is a genuinely good question. An hour is still an hour, but it almost never feels like one. Golfers know the feeling — a fantastic round disappears; a nightmare round stretches considerably. My own experience of golf has generally been the latter. I certainly get my money's worth.
We kill time. We save it. We rob and get robbed of it, lose it, and sometimes feel we have all of it in the world. But none of us is powerful enough to stop the march of time, or slow it down, or speed it up.
I believe God is. I believe that for the eternal God, time simply stands still — that he looks down on creation and sees everything at once, the beginning and the end, without standing between them as we do. With the Lord a day is like a thousand years, and a thousand years are like a day. [2 Peter 3:8] I am not going to turn this Perspective into a theological argument, but I am not going to edit that out simply because it does not fit neatly into an essay about AI. That was where my thinking genuinely went, standing at a funeral in August 2020, and pretending otherwise would be its own small dishonesty.
The people left behind at a funeral are not standing outside time. They are very much inside it, with their grief and their unanswered questions. At the time I was working with people who had suffered brain injuries — people living with loss, anger, fear, uncertainty, the realisation of how fragile life can be. One thing I had learned from that work was that difficult emotions do not resolve themselves by being pushed down. They delay and compound. The most important thing is to recognise how you are feeling. To know that it is okay not to be okay.
We are only on this earth for a fleeting moment. So I invited everyone there to use the funeral not only to say goodbye to Billy, but to let go — of unanswered questions, of things left unsaid — and to think about how they use the time they still have.
Then we said goodbye.
Into the freedom of wind and sunshine, Billy, we let you go.
Into the dance of the stars and planets, Billy, we let you go.
We love you. We miss you.
Go softly. Go onwards on your journey.
Those words were written for that day, not for an essay. When I found them again in 2026, I also found something unexpected: I had been thinking about time — seriously, personally — six years before the AI problem arose, for reasons that had nothing to do with technology. The question I thought I had arrived at through frustration with a chatbot had in fact been carried with me the whole time, unacknowledged, waiting to become relevant again.
This is also where Sacks's account matters in a way that goes beyond the neurological. He did not finish with Jimmie's deficit. He watched him over time and noticed that there were activities in which the fragmentation seemed to recede — music, gardening, art, nature, religious worship. In those moments Jimmie could appear settled and deeply present in a way that the clinical picture did not capture.[1]
That changes the story. Jimmie cannot simply be reduced to a broken memory mechanism. He is still a man. He can be moved, attend, belong. A person is more than an autobiography. Music can hold us in a way that has very little to do with knowing what year it is. Time is clearly not the whole of human existence. And yet Sacks's word still feels right. Something fundamental loosens when enough temporal continuity disappears.
The fishing trip demonstrates this quietly. At seven or eight, it was the memory of catching a fish. Fifteen years later it became the memory of discovering a kindness. At the funeral it became part of a goodbye. In 2026 it became part of an essay about memory and time. Nothing about the morning at the pond has changed. What keeps changing is its relationship to everything around it — because I keep arriving at it from a different point in my own life, and each time it means something slightly different.
That is not a failure of memory. It is what memory does.
AI systems will remember more. I think that will generally be a good thing. But what interests me now is what happens after the remembering. Does the system know whether the information describes the present? Can it distinguish the meeting where we considered something from the later meeting where we abandoned it? And occasionally — can it recognise that enough time has passed for the sensible response to be simply: “What happened?”
After six months apart, I bump into a colleague I know was applying for a job. I do not ask how the application is going. I ask what happened with it. The difference is tiny — one question assumes the past has continued, the other knows it has not and leaves room for whatever occurred in between. Nobody would call that a skill. Between two people over coffee, it is just conversation. Perhaps that is why its absence in AI can feel so disorienting. The system remembers. It sounds coherent. And yet something small and completely fundamental is missing.
The machine remembers the impressive part and misses the ordinary part.
I began this Perspective believing I was writing about a problem I had encountered with AI memory. Somewhere along the way I discovered I had been thinking about the same question since August 2020, at a funeral in South London, having arrived there on a red double-decker bus thirty years earlier in my memory. The route back to an idea can run through places you had entirely forgotten.
Time does something almost invisible to memory.
It gives it somewhere to stand.
And between yesterday and now,
whether we notice it or not,
life moves.
Clarity Workflows Perspectives — Perspective 04 of an ongoing series
Clarity Workflows Perspectives offers short-form thinking on human-led, AI-assisted work, workflow design and the changing relationship between people and technology.