Have you ever watched TV while lying sideways on the couch? If so, have you ever felt a desire to rotate the TV to better match the orientation of your head? The desire to align incoming visual information to your own orientation is natural and relates to your brain's specialized networks for processing upright information.
Source: Pexels Over the course of a lifetime, we all develop categories of perceptual expertise. By the time we are adults, most of us are experts at perceiving faces, reading words, recognizing animals, and interpreting complex scenes. However, that expertise is constrained to upright (or nearly upright) stimuli. For example, while most of us are great at recognizing faces, we are severely impaired at recognizing upside-down faces. This is demonstrated most clearly with the "Thatcher Illusion" discovered by Peter Thompson in 1980. The manipulated image of Margaret Thatcher looks a bit odd when it is upside-down, but looks absolutely grotesque when shown upright. This is because the brain regions that process faces can only efficiently process faces that are upright (or slightly tilted). We have very limited expertise for perceiving faces—or other visual stimuli—that are not upright.
Given these constraints, how is it that we are able to watch TV or other media while lying sideways? Our research team recently investigated this question by devising a clock-reading task that participants completed while sitting upright or lying sideways (Davidenko, Cheong, Waterman, Smith, Anderson, & Harmon, 2018). Participants were asked to report the time on rotated analog clocks by pressing a number on a keypad corresponding to the hour indicated on the clock. In some trials, participants were sitting upright; in other trials, they were lying sideways on their right or left side.
Our predictions were as follows: when sitting upright, participants should be fastest to respond to upright clocks (no surprise there). However, when lying sideways, we predicted participants would integrate multiple orientation cues and respond fastest to clocks at an intermediate orientation between the participant's internal upright (i.e. the orientation of their head) and the external upright (i.e. the orientation of the room and gravity). In other words, we predicted that participants would be fastest to respond to tilted clocks that are "partially" upright with respect to both the internal and external reference frames.
And this is exactly what we found. As shown in the figure below, reaction times followed sine-like curves. When participants were sitting upright (blue curve), the reaction time curve reached its minimum very close to 0 degrees relative to the observer. When participants were lying on their left side (red curve), reaction times were shortest when clocks appeared at around 35 degrees clockwise from the observer (toward the externally defined upright). Likewise, when participants were lying on their right side (yellow curve), reaction times were shortest when clocks appeared at around 45 degrees counter-clockwise from the observer (again, toward the externally defined upright).
Results of study 1 Source: Nicolas Davidenko These results support the idea that our visual system does not rely solely on an internal reference frame to process upright and rotated stimuli. Instead, the brain combines internal cues and external cues to arrive at a “compromise” reference frame—a skewed reference frame that lies somewhere in between the internal upright and the external upright defined by vestibular and visual-environmental cues.
To test this idea further, we conducted a second experiment using Virtual Reality to dissociate the different external cues available to participants. Participants were assigned to one of four groups: Group A participants completed the clock task while sitting upright and immersed in an upright virtual environment; group B participants sat upright but were immersed in a sideways virtual environment. Group C participants lied on their right side while immersed in a sideways virtual environment, and group D participants lied on their right side while immersed in an upright virtual environment (see the figure below). We were interested in knowing the optimal clock angle leading to the fastest response. If participants incorporate external reference frames based on vestibular cues alone, we should find significant results whenever participants are lying sideways (groups C and D), regardless of the orientation of the virtual environment. If participants incorporate external reference frames based on visual information alone, we should find significant results whenever participants are immersed in a sideways virtual environment (groups B and D).
What we found was that only participants in group D showed a significant influence of the external reference frame. That is, the external reference frame only played a role when both vestibular and visual cues were available to indicate which way was up.
Results of study 2 Source: Nicolas Davidenko Taken together, our results suggest that the brain integrates visual and vestibular cues to estimate the direction of gravity, and uses this information to select an intermediate reference frame to process visual information.
So how does this pertain to watching TV while lying sideways on the couch? Our results suggest that if you could tilt the TV screen, a tilt of about 35-45 degrees away from the internal upright (toward the external upright) may lead to the best TV watching experience. Although it may not be practical (or even safe) to literally rotate your TV set, this information could be useful when watching media on head-mounted or mobile devices. Depending on your head and body orientation, a slight tilt of your mobile screen may actually improve your ability to comprehend and enjoy what you are watching.