Meiji-mura open air architectural museum, near Nagoya. Shown here is a fragment of the Kawasaki Bank building.

Meiji-mura open air architectural museum, near Nagoya. Shown here is a fragment of the Kawasaki Bank building.

Fast Essay: "Spatial Computing, the Perfect Modality for Building a Memory Palace" 
Arlo Emerson, 2019 

TL;DR 

I have a deep need and general desire to memorize things. Things like: Japanese words (I speak a little), programming language syntax, and trivial type of information (e.g. street names) along with personal interaction details (e.g. your name). Practical stuff. Somewhere along the way I learned about a technique called a Memory Palace. This is a powerful concept for cementing mental symbols. 

A good memory palace should be extremely vivid, like a lucid dream. It should also be like a home, with a sense of permanence and familiarity. These are your memories, after all, and we don’t want things drifting off into the dark void. I remember hearing about memory palaces some years ago, and it seemed not so much esoteric, but rather like a lot of work to maintain. It would take a lot of time and effort to generate the mental symbols/objects, and then more time to revisit these symbols within the palace. This is basically a form of meditation, as it takes place primarily in the mind. 

Cutaway architectural model of Charles Garnier’s Opera House, Musee d’Orsay, Paris, France.

Cutaway architectural model of Charles Garnier’s Opera House, Musee d’Orsay, Paris, France.

The method we, as humans, normally employ to memorize things is to interact with them in the actual physical world, over and over and over, until abstract concepts become natural components of our awareness. Yes, you could argue that all of this interaction takes place in your mind, and with your eyes open most of the time, and that we project the world we experience. But let’s table that philosophical debate for a moment and briefly touch on the different modalities one can employ to create a memory palace IN THE REAL WORLD. A few come to mind:  

• Index cards 

• Post it notes on a wall 

• Pictures with labels pinned to a board (mood board) 

• Google Street View with labels 

• 3D immersive experience, say, in Unity  

I’ve tried all these methods except the last, to build an immersive app that houses my memory palace. The Google Street View method is interesting but doesn’t FEEL immersive. It’s fun, at first, but it wears off quickly because the experience doesn’t feel special. And it’s tedious work to set up. For example, I’ve labeled things (e.g. Japanese words) on objects on a street I lived on in Nagoya. I should say I lived on this street for exactly two weeks, so I had enough time to soak in many details but not enough to permanently lock everything. Moving through this same street in Street View helped to bring back memories, sure, but it lacked the visceral effects. The sounds, the temperatures, the humidity, the smells. The strange details were all missing, and those are the important ones to capture. A memory palace depends on the power of the symbols it houses. Weak symbols will appear as nothing more than labels, at which point you are better off doing flash cards. Walking through one’s memory palace needs to feel very special, very personal. Like a lucid dream. 

An Instagram post from the author’s account, this image being an attempt to capture the surreal-ness of those flickering moments where memories are permanently cemented in the mind.

An Instagram post from the author’s account, this image being an attempt to capture the surreal-ness of those flickering moments where memories are permanently cemented in the mind.

In my case, I’m interested in learning Japanese beyond the overly-formal “business Japanese” that I’ve gotten away with so far. I’ve concluded that not only do I need vivid imagery, I need avatars to interact with and talk to. I’ll use a simple example: When I buy something at, say, Lawson, the cashier will spend a few valuable seconds to count out the change for this gaijin that has suddenly appeared. We gaijin actually slow down Japanese in their day-to-day, so it would be good for me to practice this interaction and reduce the friction. Thus, I shall build a little world for these interactions to happen. Imagine this: We start with a “memory city”. This becomes our town, and for me this town is Nagoya, specifically the neighborhood of Fushimi. I’ve got everything I need here: a subway line, a train station, restaurants, the Lawson on the corner, a small, quiet room in a hotel with a view of a concrete wall, the NHK weather report on the TV. There’s a park nearby with homeless people, to one of them I hand a few coins. There’s a Japanese guy on a subway who talked my ear off at eleven PM one night. There’s the dive jazz clubs I spent so much time in. This little neighborhood is where my palace is.   

Ok, good. Now what I really need is a way to edit this world. It’s all built in 3D (written in Unity, probably), and I’m going to need magical interactions to modify content. These interactions should almost be like a rite, or ritual, and to an outside observer it will appear as such. I draw a symbol with my finger and gold light spills into the space before me. I associate this symbol with this place, this experience, this object. Whatever it may be. An avatar is associated with this particular “scene”, and this avatar is more or less there to help me remember. In a way, the avatar is an extension of me. The cashier at the Lawson has been programmed to count out change from a 100-yen bill. The guy on the subway has been programmed to ask me seemingly random questions. Hopefully there’s a way to get a bit of conversation going in this world. I go to the train station where the swarm envelops me and carries me along within its flocking algorithm. 

This kind of personal world can be custom built for each of us, and the creation tools to edit must be built alongside. I say these tools ought to feel like magic or feel like play, or something else that resonates with the user. Maybe a node editor.  Maybe it’s Lego blocks. Maybe it’s mud.

Regardless of the tools we build, spatial computing will become deeply integrated into society, and we, as the creators of this future, are well poised to enhance and extend the old tools into this brave new world. 

Shopping district in Sakae, Nagoya.

Shopping district in Sakae, Nagoya.