As the world has shut down in the wake of the Covid-19 pandemic, more and more of us have turned to remote working, with Zoom meetings becoming ubiquitous, whether for work or for more social activities. The almost totally virtual nature of life over the past year has led to growing concerns about “Zoom fatigue” as people grow tired of the lack of human interaction and the unique nature of video conferencing brings unique challenges to our engagements.
New research from Carnegie Mellon also suggests that using Zoom for team interactions may not even be particularly effective and might actually be making our teams less intelligent. The study suggests that whereas we might assume that technology that aims to replicate our face-to-face interactions via video links will be most effective, the reality is that non-visual communication may be more effective, especially when synchronized with audio cues.
Boosting our collective intelligence
Previous research from MIT explored the way collective intelligence forms in online communities. They reasoned that things like our ability to empathize and interact with others are key to successful teamwork, but both are difficult to measure online as we don’t have access to in-person cues.
The key is the “Theory of Mind” (ToM), which describes our ability to understand someone else’s mental state, and indeed how it may differ from our own. The ToM has been used to explore the collective intelligence of groups in real-life settings, and the authors hypothesized that it could also work online.
It emerged that the level of communication coupled with the ToM abilities within the group was a good indicator of the collective wisdom of the team. Crucially, the medium (ie the online chat room) was no hindrance to the ability of the group to interpret the emotions of their peers or to contribute fully to the tasks at hand.
Getting on the same page
The Carnegie Mellon team focused on non-verbal cues and their role in effective teamwork. They wanted to test how video conferencing tools affect our ability to effectively hold conversations and share ideas. They argue that in real-life, non-verbal cues are crucial in helping to mediate conversations so that we know when it’s our turn to speak.
The study focused specifically on synchrony of facial expression and prosodic synchrony. While I’m sure we can all grasp what facial expression synchrony is, prosodic synchrony may require a bit more explanation. It basically revolves around capturing things such as our tone, intonation, stress, and rhythm of speech.
The hypothesis was that during Zoom calls we have access to both audio and visual cues and that this would encourage us to rely more on facial expression synchrony, whereas if we only have audio cues, we rely more on prosodic synchrony to develop collective intelligence.
Dumbing down
“We found that video conferencing can actually reduce collective intelligence,” the researchers say. “This is because it leads to more unequal contribution to conversation and disrupts vocal synchrony. Our study underscores the importance of audio cues, which appear to be compromised by video access.”
Of course, audio cues don’t provide the whole picture. For instance, research from Tampere University, in Finland highlights the importance of eye contact for collaboration, in a similar way to the study from MIT mentioned earlier.
The study found that eye contact during video calls triggers the same kind of pscychophysiological responses as eye contact face to face. The findings emerged after the researchers examined the physical reactions to eye contact in a range of situations, including face-to-face and via a live video call.
Responses were measured via skin conductance and activation of their facial muscles. It’s believed that changes in our skin conductance is a reflection of the level to which our autonomous nervous system is activated. The activation of our facial muscles reflects the positivity or negativity of this effect.
Eye contact in a face-to-face setting was found to elicit a heightened autonomic arousal response, which is consistent with previous studies. Where this study is interesting, however, is that similar results were found when people engaged via a video chat.
Changing our communication
All of which makes it perhaps understandable that our communication style changes when we engage via video conferencing. Recent research from Florida Atlantic University reveals that our gaze is often altered during video conferencing, precisely because we believe the other person can see us, and we’re highly sensitive to the gaze direction of other people.
Indeed, even children as young as 2 prefer it when people look directly at them. It’s a phenomenon known as gaze cueing, and it provides a powerful signal to help us orient attention.
This is a natural consequence of human history, with conversations always being conducted face-to-face. This assumption has been broken since the invention of the telephone, but video conferencing promises to make virtual communication more personal again.
Interestingly, in real-time conditions, it was more common for participants to display avoidant fixation behaviors. The lack of time spent on the eyes of the other person suggests that the extra time spent looking at the mouth during the pre-recorded conversation wasn’t done at the expense of eye contact, but rather reduced time spent looking elsewhere.
“Regardless of the specific mechanisms underlying the observed differences in fixation patterns, results from our study suggest participants were taking social and attentional considerations into account in the real-time condition,” the researchers conclude. “Given that encoding and memory have been found to be optimized by fixating the mouth, which was reduced overall in the real-time condition, this suggests that people do not fully optimize for speech encoding in a live interaction.”
This matters because a major factor in our ability to judge the mental state of others is through their eyes. It’s something commonly measured in a test to determine our ability to gauge the Theory of Mind, called Reading of the Eyes. The test asks participants to try and judge the mental state of others through nothing more than looking at a photo of their eyes.
What’s clear is that video platforms are not exactly replicating our communication methods in a face-to-face setting, and it’s important that we understand this so don’t assume that they do.
