When Matthew Blakeslee shapes hamburger patties with his hands, he experiences a vivid bitter taste in his mouth. Esmerelda Jones (a pseudonym) sees blue when she listens to the note C sharp played on the piano; other notes evoke different hues--so much so that the piano keys are actually color-coded. And when Jeff Coleman looks at printed black numbers, he sees them in color, each a different hue. Blakeslee, Jones and Coleman are among a handful of otherwise normal people who have synesthesia. They experience the ordinary world in extraordinary ways and seem to inhabit a mysterious no-man's-land between fantasy and reality. For them the senses--touch, taste, hearing, vision and smell--get mixed up instead of remaining separate.
Modern scientists have known about synesthesia since 1880, when Francis Galton, a cousin of Charles Darwin, published a paper in Nature on the phenomenon. But most have brushed it aside as fakery, an artifact of drug use or a mere curiosity. About seven years ago, however, we and others began to uncover brain processes that could account for synesthesia. Along the way, we also found new clues to some of the most mysterious aspects of the human mind, such as the emergence of abstract thought and metaphor.
A common explanation of synesthesia is that the affected people are simply experiencing childhood memories and associations. Maybe a person had played with refrigerator magnets as a child, and the number 5 was red and 6 was green. This theory does not answer why only some people retain such vivid sensory memories, however. You might think of cold when you look at a picture of an ice cube, but you probably do not feel cold, no matter how many encounters you may have had with ice and snow during your youth.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Another prevalent idea is that synesthetes are merely being metaphorical when they describe the note C sharp as red or say that chicken tastes pointy--just as you and I might speak of a loud shirt or sharp cheddar cheese. Our ordinary language is replete with such sense-related metaphors, and perhaps synesthetes are just especially gifted in this regard.
We began trying to find out whether synesthesia is a genuine sensory experience in 1999. This deceptively simple question had plagued researchers in the field for decades. One natural approach is to start by asking the subjects outright: Is this just a memory, or do you actually see the color as if it were right in front of you? When we asked this question, we did not get very far. Some subjects did respond, Oh, I see it perfectly clearly. But a more frequent reaction was, I kind of see it, kind of dont or No, it is not like a memory. I see the number as being clearly red, but I also know it isnt; it's black. So it must be a memory, I guess.
To determine whether an effect is truly perceptual, psychologists often use a simple test called pop-out or segregation. If you look at a set of tilted lines scattered amid a forest of vertical lines, the tilted lines stand out. Indeed, you can instantly segregate them from the background and group them mentally to form, for example, a separate triangular shape. Similarly, if most of a background's elements were green dots and you were told to look for red targets, the red ones would pop out. On the other hand, a set of black 2's scattered among 5's of the same color almost blend in [see box on page 81]. It is hard to discern the 2's without engaging in an item-by-item inspection of numbers, even though any individual number is just as clearly different from its neighbors as a tilted line is from a straight line. We thus may conclude that only certain primitive, or elementary, features, such as color and line orientation, can provide a basis for grouping. More complex perceptual tokens, such as numbers, cannot.
We wondered what would happen if we showed the mixed numbers to synesthetes who experience, for instance, red when they see a 5 and green with a 2. We arranged the 2's so that they formed a triangle.
When we conducted these tests with volunteers, the answer was crystal clear. Unlike normal subjects, synesthetes correctly reported the shape formed by groups of numbers up to 90 percent of the time (exactly as nonsynesthetes do when the numbers actually have different colors). This result proves that the induced colors are genuinely sensory and that synesthetes are not just making things up. It is impossible for them to fake their success.
Visual Processing
CONFIRMATION that synesthesia is real brings up the question, Why do some people experience this weird phenomenon? Our experiments lead us to favor the idea that synesthetes are experiencing the result of some kind of cross wiring in the brain. This basic concept was initially proposed about 100 years ago, but we have now identified where and how such cross wiring might occur.
An understanding of the neurobiological factors at work requires some familiarity with how the brain processes visual information. After light reflected from a scene hits the cones (color receptors) in the eye, neural signals from the retina travel to area 17, in the occipital lobe at the back of the brain. There the image is processed further within local clusters, or blobs, into such simple attributes as color, motion, form and depth. Afterward, information about these separate features is sent forward and distributed to several far-flung regions in the temporal and parietal lobes. In the case of color, the information goes to area V4 in the fusiform gyrus of the temporal lobe. From there it travels to areas that lie farther up in the hierarchy of color centers, including a region near a patch of cortex called the TPO (for the junction of the temporal, parietal and occipital lobes). These higher areas may be concerned with more sophisticated aspects of color processing. For example, leaves look as green at dusk as they do at midday, even though the mix of wavelengths reflected from them is very different.
Numerical computation, too, seems to happen in stages. An early step also takes place in the fusiform gyrus, where the actual shapes of numbers are represented, and a later one occurs in the angular gyrus, a part of the TPO that is concerned with numerical concepts such as ordinality (sequence) and cardinality (quantity). When the angular gyrus is damaged by a stroke or a tumor, the patient can still identify numbers but can no longer perform multiplication. After damage to another nearby region, subtraction and division may be lost, while multiplication may survive (perhaps because it is learned by rote). In addition, brain-imaging studies in humans strongly hint that visually presented letters of the alphabet or numbers (graphemes) activate cells in the fusiform gyrus, whereas the sounds of the syllables (phonemes) are processed higher up, once again in the general vicinity of the TPO.
Because both colors and numbers are processed initially in the fusiform gyrus and subsequently near the angular gyrus, we suspected that number-color synesthesia might be caused by cross wiring between V4 and the number-appearance area (both within the fusiform) or between the higher color area and the number-concept area (both in the TPO).
Other, more exotic forms of the condition might result from similar cross wiring of different sensory-processing regions. That the hearing center in the temporal lobes is also close to the higher brain area that receives color signals from V4 could explain sound-color synesthesia. Similarly, Matthew Blakeslee's tasting of touch might occur because of cross wiring between the taste cortex in a region called the insula and an adjacent cortex representing touch by the hands. Another synesthete with taste-induced touch describes the flavor of mint as cool glass columns.
Taste can also be cross-wired to hearing. For example, one synesthete reports that the spoken Lord's Prayer tastes mostly of bacon. In addition, the name Derek tastes of earwax, whereas the name Tracy tastes like a flaky pastry.
Assuming that neural cross wiring does lie at the root of synesthesia, why does it happen? We know that synesthesia runs in families, so it has a genetic component. Perhaps a mutation causes connections to emerge between brain areas that are usually segregated. Or maybe the mutation leads to defective pruning of preexisting connections between areas that are normally connected only sparsely. If the mutation were to be expressed (that is, to exert its effects) in some brain areas but not others, this patchiness might explain why some synesthetes conflate colors and numbers, whereas others see colors when they hear phonemes or musical notes. People who have one type of synesthesia are more likely to have another, and within some families, different members will have different types of synesthesia; both facts add weight to this idea.
Although we initially thought in terms of physical cross wiring, we have come to realize that the same effect could occur if the wiring--the number of connections between regions--was fine but the balance of chemicals traveling between regions was skewed. So we now speak in terms of cross activation. For instance, neighboring brain regions often inhibit one another's activity, which serves to minimize cross talk. A chemical imbalance of some kind that reduces such inhibition--for example, by blocking the action of an inhibitory neurotransmitter or failing to produce an inhibitor--would also cause activity in one area to elicit activity in a neighbor. Such cross activation could, in theory, also occur between widely separated areas, which would account for some of the less common forms of synesthesia.
Support for cross activation comes from other experiments, some of which also help to explain the varied forms synesthesia can take. One takes advantage of a visual phenomenon known as crowding [see box on opposite page]. If you stare at a small plus sign in an image that also has a number 5 off to one side, you will find that it is easy to discern that number, even though you are not looking at it directly. But if we now surround the 5 with four other numbers, such as 3's, then you can no longer identify it. It looks out of focus. Volunteers who perceive normally are no more successful at identifying this number than mere chance. That is not because things get fuzzy in the periphery of vision. After all, you could see the 5 perfectly clearly when it was not surrounded by 3's. You cannot identify it now because of limited attentional resources. The flanking 3's somehow distract your attention away from the central 5 and prevent you from seeing it.
A big surprise came when we gave the same test to two synesthetes. They looked at the display and made remarks like, I cannot see the middle number. It's fuzzy, but it looks red, so I guess it must be a 5. Even though the middle number did not consciously register, it seems that the brain was nonetheless processing it somewhere. Synesthetes could then use this color to deduce intellectually what the number was. If our theory is right, this finding implies that the number is processed in the fusiform gyrus and evokes the appropriate color before the stage at which the crowding effect occurs in the brain; paradoxically, the result is that even an invisible number can produce synesthesia for some synesthetes.
Another finding we made also supports this conclusion. When we reduced the contrast between the number and the background, the synesthetic color became weaker until, at low contrast, subjects saw no color at all, even though the number was perfectly visible. Whereas the crowding experiment shows that an invisible number can elicit color, the contrast experiment conversely indicates that viewing a number does not guarantee seeing a color. Perhaps low-contrast numbers activate cells in the fusiform adequately for conscious perception of the number but not enough to cross-activate the color cells in V4.
Finally, we found that if we showed synesthetes Roman numerals, a V, say, they saw no color--which suggests that it is not the numerical concept of a number, in this case 5, but the grapheme's visual appearance that drives the color. This observation, too, implicates cross activation within the fusiform gyrus itself in number-color synesthesia, because that structure is mainly involved in analyzing the visual shape, not the high-level meaning, of the number. One intriguing twist: Imagine an image with a large 5 made up of little 3's; you can see either the forest (the 5) or focus minutely on the trees (the 3's). Two synesthete subjects reported that they saw the color switch, depending on their focus. This test implies that even though synesthesia can arise as a result of the visual appearance alone--not the high-level concept--the manner in which the visual input is categorized, based on attention, is also critical.
But as we began to recruit other volunteers, it soon became obvious that not all synesthetes who colorize their world are alike. In some, even days of the week or months of the year elicit colors.
The only thing that days of the week, months and numbers have in common is the concept of numerical sequence, or ordinality. For certain synesthetes, perhaps it is the abstract concept of numerical sequence that drives the color, rather than the visual appearance of the number. Could it be that in these individuals, the cross wiring occurs between the angular gyrus and the higher color area near the TPO instead of between areas in the fusiform? If so, that interaction would explain why even abstract number representations, or the idea of the numbers elicited by days of the week or months, will strongly evoke specific colors. In other words, depending on where in the brain the synesthesia gene is expressed, it can result in different types of the condition--higher synesthesia, driven by numerical concept, or lower synesthesia, produced by visual appearance alone. Similarly, in some lower forms, the visual appearance of a letter might generate color, whereas in higher forms it is the sound, or phoneme, summoned by that letter; phonemes are represented near the TPO.
We also observed one case in which we believe cross activation enables a color-blind synesthete to see numbers tinged with hues he otherwise cannot perceive; charmingly, he refers to these as Martian colors. Although his retinal color receptors cannot process certain wavelengths, we suggest that his brain color area is working just fine and being cross-activated when he sees numbers.
In brain-imaging experiments we conducted with Geoffrey M. Boynton of the Salk Institute for Biological Studies in San Diego, we obtained evidence of local activation of the color area V4 in a manner predicted by our cross-activation theory of synesthesia. (The late Jeffrey A. Gray of the Institute of Psychiatry in London and his colleagues reported similar results.) On presenting black and white numbers and letters to synesthetes, brain activation increased not only in the number area--as it would in normal subjects--but also in the color area. Our group also observed differences between types of synesthetes. Subjects with lower synesthesia showed much greater activation in earlier stages of color processing than did control subjects. In contrast, higher synesthetes show less activation at these earlier levels.
Floating Numbers
GALTON DESCRIBED another intriguing form of synesthesia, in which numbers seem to occupy specific locations in space. Different numbers occupy different locations, but they are arranged sequentially in ascending order on an imaginary number line. The number line is often convoluted in an elaborate manner--sometimes even doubling back on itself so that, for example, 2 might be closer to 25 than to 4. If the subject tilts his head, the number line also may tilt. Some synesthetes claim to be able to wander the number landscape and are even able to shift vantage point, to inspect hidden parts of the line or see it from the other side so the numbers appear reversed. In some individuals, the line even extends into three-dimensional space. These strange observations reminded us of neuroscientist Warren S. McCulloch's famous question, What is a number, that a man may know it, and a man, that he may know a number?
How do we know the number line is a genuine perceptual construct, not something the subject is just imagining or making up? One of us (Ramachandran), working in collaboration with U.C.S.D. graduate student Shai Azoulai, tested two number-line synesthetes. We presented 15 numbers (out of 100) simultaneously on the screen for 30 seconds and asked the subjects to memorize them. In one condition (called the congruent condition), the numbers fell where they were supposed to on the virtual number line. In the second condition, the numbers were placed in incorrect locations (the incongruent condition). When tested after 90 seconds, the subjects memory for the numbers in the congruent condition was significantly better than in the incongruent condition. This is the first objective proof, since Galton observed the effect, that number lines are genuine in that they can affect performance in a cognitive task.
In a related experiment, we used the well-known numerical distance effect. When normal people are asked which of two numbers is bigger, they respond faster when the numbers are farther apart (for example, 4 and 9) than when they are close together (say, 3 and 4). This phenomenon implies that the brain does not represent numbers in a kind of look-up table, as in a computer, but rather spatially in sequence. Adjacent numbers are more easily confused, and therefore more difficult to make comparisons with, than numbers that are farther apart. The astonishing thing is that in one subject with a convoluted number line we found that it was not the numerical distance alone that determined performance, but spatial distance on the synesthetic screen. If the line doubled back on itself, then 4 might be more difficult to tell apart from, say, 19 than from 6! Here again was evidence for the reality of number lines.
Number lines can influence arithmetic. One of our subjects reported that even simple arithmetic operations such as subtraction or division were more difficult across the kinks or inflections of the line than across straight sections. This result suggests that numerical sequence (whether for numbers or calendars) is represented in the angular gyrus of the brain, which is known to be involved in arithmetic.
Why do some people have convoluted number lines? We suggest the effect occurs because one of the main functions of the brain is to remap one dimension onto another. For instance, numerical concept (size of the number) is mapped in a systematic manner onto the sequentiality represented in the angular gyrus. Usually this effect is a vague left-to-right, straight-line remapping. But if a mutation occurs that adversely influences this remapping, a convoluted representation results. Such quirky spatial representations of numbers may also enable geniuses like Albert Einstein to see hidden relations between numbers that are not obvious to lesser mortals like us.
A Way with Metaphor
OUR INSIGHTS into the neurological basis of synesthesia could help explain some of the creativity of painters, poets and novelists. According to one study, the condition is much more common in creative people than in the general population.
One skill that many creative people share is a facility for using metaphor (It is the east, and Juliet is the sun). It is as if their brains are set up to make links between seemingly unrelated domains--such as the sun and a beautiful young woman. In other words, just as synesthesia involves making arbitrary links between seemingly unrelated perceptual entities such as colors and numbers, metaphor involves making links between seemingly unrelated conceptual realms. Perhaps this is not just a coincidence.
Numerous high-level concepts are probably anchored in specific brain regions, or maps. If you think about it, there is nothing more abstract than a number, and yet it is represented, as we have seen, in a relatively small brain region, the angular gyrus. Let us say that the mutation we believe brings about synesthesia causes excess communication among different brain maps--small patches of cortex that represent specific perceptual entities, such as sharpness or curviness of shapes or, in the case of color maps, hues. Depending on where and how widely in the brain the trait was expressed, it could lead to both synesthesia and a propensity toward linking seemingly unrelated concepts and ideas--in short, creativity. This might explain why the apparently useless synesthesia gene has survived in the population.
In addition to clarifying why artists might be prone to experiencing synesthesia, our research suggests that we all have some capacity for it and that this trait may have set the stage for the evolution of abstraction--an ability at which humans excel. The TPO (and the angular gyrus within it), which plays a part in the condition, is normally involved in cross-modal synthesis. It is the brain region where information from touch, hearing and vision is thought to flow together to enable the construction of high-level perceptions. For example, a cat is fluffy (touch), it meows and purrs (hearing), it has a certain appearance (vision) and odor (smell), all of which are derived simultaneously by the memory of a cat or the sound of the word cat.
Could it be that the angular gyrus--which is disproportionately larger in humans than in apes and monkeys--evolved originally for cross-modal associations but then became co-opted for other, more abstract functions such as metaphors?
Consider two drawings, originally designed by psychologist Wolfgang Khler [see box on opposite page]. One looks like an inkblot and the other, a jagged piece of shattered glass. When we ask, Which of these is a bouba, and which is a kiki? 98 percent of people pick the inkblot as a bouba and the other as a kiki. Perhaps that is because the gentle curves of the amoebalike figure metaphorically mimic the gentle undulations of the sound bouba, as represented in the hearing centers in the brain as well as the gradual inflection of the lips as they produce the curved boo-baa sound.
In contrast, the waveform of the sound kiki and the sharp inflection of the tongue on the palate mimic the sudden changes in the jagged visual shape. The only thing these two kiki features have in common is the abstract property of jaggedness that is extracted somewhere in the vicinity of the TPO, probably in the angular gyrus. In a sense, perhaps we are all closet synesthetes.
So the angular gyrus performs a very elementary type of abstraction--extracting the common denominator from a set of strikingly dissimilar entities. We do not know exactly how it does this job. But once the ability to engage in cross-modal abstraction emerged, it might have paved the way for the more complex types of abstraction.
When we began our research on synesthesia, we had no inkling of where it would take us. Little did we suspect that this eerie phenomenon, long regarded as a mere curiosity, might offer a window into the nature of thought.
THE AUTHORS
VILAYANUR S. RAMACHANDRAN and EDWARD M. HUBBARD collaborate on studies of synesthesia. Ramachandran directs the Center for Brain and Cognition at the University of California, San Diego, and is adjunct professor at the Salk Institute for Biological Studies. He trained as a physician and later obtained a Ph.D. from Trinity College, University of Cambridge. Hubbard received his Ph.D. from the departments of psychology and cognitive science at U.C.S.D. and is now a postdoctoral fellow at INSERM in Orsay, France. A founding member of the American Synesthesia Association, he helped to organize its second annual meeting at U.C.S.D. in 2001.