Multimodal Learning - Unleashing the human memory

October 27, 2023

Author: Dave Cohen

Below is the blog version of a talk I gave on memorization techniques.

Other formats:

Outline:

Why this talk?
What we’ll explore
My story - hitting a wall of text
Rote learning takes up to 160 repetitions
A new approach - drawing pictures
The Art of Memory
Why text is the default
Imaginative powers - senses, faces, time, spaces
Mnemonics
Elaborative Encoding
Elaborative Encoding Example - Remembering the C4 model
Method of Loci - Walking through a memory palace
Choosing memory palaces
Final thoughts
Resources

Why this talk?

The goal of this talk is to introduce some learning and memory techniques. I’m hoping that you’ll find them fun and interesting and you’ll want to try them out.

Question: what should people learn in an age of chatgpt?

Learning, in general, has always been fascinating to me. These days, I’ve had to ask myself what we still need to learn and actually remember. When I chatted with AI about it, the bots reminded me that curiosity and creativity (which are both supported by learning and memorization) are more relevant than ever.

What we’ll explore

The state of learning today, how we got stuck on a narrow way of learning (text)
How multimodal memory techniques can broaden how we learn; make it more fun and efficient

My story - hitting a wall of text

How I’m Building My Second Brain

A while back, I discovered the “second brain” methodology of note-taking. The goal of a “second brain” is to create a digital network of knowledge - and that’s done by connecting notes. The more notes you create and link together (using “backlinks”), the more your network of nodes expands and frankly, the results just look really cool. So, I did just that - I made summaries of books, articles, and podcasts. I was proud of my ability to be a human summarizer.

// big .md notes collection

Projects
[[how-to-x]]
[[how-to-y]]

x
[[all-about-x1]]
[[all-about-x2]]

I read and re-read my hand-crafted notes but didn’t notice myself being able to recall things (that well or at all) given how much time I spent. You could say I hit a wall - a wall of text. I wasn’t even close to being a kickass walking encyclopedia.

Rote learning takes up to 160 repetitions

~Rote Memorization and Alternatives to Learn Faster and More - A Tutor

Looking back, by reading, and re-reading my notes, I was essentially trying to learn by rote.

Our brains reserve resources for interesting events, they’re primarily concerned with survival. Just exposure to abstract concepts (in text form especially) isn’t nearly interesting enough to take up precious mental space.

So, clearly, that wasn’t working.

A new approach - drawing pictures

I started drawing pictures of concepts I was trying to learn.

As we all know, a picture is worth a thousand words, and I was able to remember things a lot better.

I also remembered that 50% of our brains are involved in visual processing.

Thinking visually results in a significant gain in brain capacity utilization!

The Art of Memory

As I dug further, I uncovered memory techniques that were used by the ancient greeks and romans. They were able to remember impressive amounts of information. They gave speeches from memory, engaged in lively debate, and demonstrated the potential of human memory. Books weren’t prevalent, so the main place human knowledge was stored was in people’s heads.

See Ars Memoriae - 2500 year old memory techniques

Why text is the default

Text is the dominant form of communication. It’s convenient and storage-friendly. Conversations start in Slack or email and then make things official with (Google Docs, and other text collaboration platforms).

Another example is a conversation with ChatGPT. Text goes in as a question and we get an answer in text.

But what about the in-between bits - the thought processes behind coming up with the question and how we process the answer?

I see text as the end result. But it’s a handicap to think only in text - thinking benefits from going beyond text.

Imaginative powers - senses, faces, time, spaces

So, let’s lay the groundwork for enhancing our memories and learning better as a result.

Senses:

We’ll first get in touch with our imaginative powers. By that, I mean conjuring something in the mind, usually associated with one of the 5 senses. Associating what we’re trying to learn with one or more of the senses helps create a richer, more vivid learning experience.

Faces:

A significant part of the visual part of our brains can remember faces in great detail, so faces can be great learning aids.

Time:

We also tend to remember momentous times in our lives. (In the talk is a picture of my nephew who was born in 2005.)

Spaces (the birth of the memory palace technique)

Another “sense” (as it were) is spatial memory - our brains are evolved to remember locations of important resources. A Greek poet in the 6th century BC used this fact to his advantage. He was having dinner with colleagues, left the building, and the next thing he knew, the building collapsed, killing everyone inside. He noticed that he was able to recall each person’s locations in the dining hall without having made an effort to do so. (This is the origin story of the Method of Loci, which I’ll talk about in a bit.)

Mnemonics

Let’s see some memory techniques which use our imaginative powers. Mnemonics, which you’re probably familiar with, are any conceptual device that helps us remember something.

The way I learned to remember which months have 30 days is through song and utilizes our auditory imagination.

That stuck with me really well but we can condense it into an acronym - SANJ (as long as you differentiate between June and the other J months.) This adds a sense of taste to the mnemonic.

There’s also a visual and tactile mnemonic to remember that - using your knuckles and the spaces between them to represent the months.

Image credit: Mnemonic - Wikipedia

These mnemonics engage our senses which makes it easier to remember what we’re trying to remember.

Elaborative Encoding

The next technique, Elaborative Encoding, uses mnemonics.

By elaborating new information with what’s already-established, it becomes easier to learn and recall this new information.

We’re linking the new to the old:

the link should be conceptually related to the new
the link should be something we already know and can recall easily
the more sensory, vivid, wild, outlandish the link is, the more effective it will be

This augmenting process makes it something our brain will be much more likely to pay attention to.

Let’s see an example.

Elaborative Encoding Example - Remembering the C4 model

The C4 model is an easy to learn, developer friendly approach to software architecture diagramming. … It’s a way to create maps of your code, at various levels of detail, in the same way you would use something like Google Maps to zoom in and out of an area you are interested in.

~C4 model

I was looking for a way to remember the four similar sounding C words of the C4 model.

Context
Container
Component
Code

Using visual, sound, taste, and other associations, each word is elaborated to help encode it into my memory.

Context - sounds like “Kant-text”
Container - uses “Kant” and him drinking out of a container
Component - sounds like “Computer” and I think of its constituent parts
Code - code on a screen (no need to over-complicate it)

Method of Loci - Walking through a memory palace

One of the ancient memory techniques used by memory experts today the Method of Loci - Walking through a memory palace.

The method of loci is a strategy for memory enhancement, which uses visualizations of familiar spatial environments in order to enhance the recall of information.

~Wikipedia

Image credit: https://www.monitask.com/en/blog/best-memorization-techniques

Image credit: https://themdjourney.com/studying-with-memory-palace-in-medical-school/

Method of Loci Example - Putting the C4 model in a memory palace

As I mentally travel through the room, I note each point of interest in the numbered order.

Context - The first point is the TV - there’s a documentary playing on it about how Kant wrote his most influential works.
Container - Kant is resting on the couch drinking a milkshake.
Computer - Kant was fascinated by the computer on the dining table and he took it apart.
Code - On the easy chair is a monitor with code displayed on it.

Choosing memory palaces

Places you’re very familiar with:

Childhood home(s)

Places you associate with learning:

School
Coding bootcamp

Etc:

Street routes
Famous places
Fictitious places
any place with interesting spatial qualities that can be easily recalled

Final thoughts

Learning this way is fun, engaging, and motivating. It’s personalized and creative, not just text on a page or screen.

(These techniques would’ve been extremely helpful when I was in school.)

I think you’ll learn more faster by drawing, generating mnemonics, and using a memory palace:

you’ll use more of your brain (>50%)
less time re-reading (160x faster)
1 picture, 1000 words (1000x more efficient)

Resources

I highly recommend checking out Moonwalking with Einstein: The Art and Science of Remembering Everything by Joshua Foer. It goes into great depth on how to become a memory expert.

Memorization / Learning

Integration

Visualization

Comprehension

Motivation

Other

The C4 model for visualising software architecture
Guitar Chord Finder | oolimo.com
Essentialism by Greg McKeown - Summary & Notes
Steven L. Franconeri - Faculty - Kellogg School of Management - Courses - Visualization for Persuasion