How was your conference? Did attendees pay attention during the educational sessions? How many tuned out? When did they tune out? How many laughed at your keynote speaker’s jokes? How many found them distasteful?
If the answer to these questions is, “We don’t have the post-event survey back yet,” or “We didn’t know how to ask that in the survey,” then it’s clear that relying on event surveys alone to gauge the emotions of your audience is not enough. While post-event surveys are a chance for attendees to be heard, there are obvious limitations.
1. The feedback lag. Whether you ask for feedback right away or send your survey out in a week, all responses are post event.
2. Response rates are never 100 percent, even if you offer a gift card or a discount code.
3. Feedback can be open to interpretation. On a one-to-five scale, getting mostly ones for an educational session tells you a lot. But what do mostly threes tell you? Was the topic good but the speaker mediocre? Was the speaker great but the topic irrelevant?
4. Post-event surveys are only as accurate as the respondents’ honesty and memory.
So, how can attendee approval and engagement be measured and acted on during the conference? How can a conference be more like a focus group?
Emotional Energy
Capturing and analyzing the facial expressions of audience members is emerging as a reliable and affordable tool for conference planners thanks to rapid improvements in biometric technology and artificial intelligence. Companies such as Paris-based Datakalab and Houston-based Zenus are adapting the kinds of technologies now common in market research to the meetings industry in order to better understand an audience.
Unlike the biometric scanning being developed to speed conference registration and access control, which require attendees to opt in, the new audience engagement tools are anonymous. They simply identify an attendee’s emotions, without identifying the face. CWT Meetings & Events is an early adopter. The company began partnering with Datakalab in February 2018, and to date has used the technology for 15 events in the U.S. and France.
“As humans, we sit in meetings and we have a general sense of how things are going, but to actually scan an audience of 200 or 300 people in real time and see who is tuned out is a great tool,” says Beau Ballin, vice president of business development at CWT Meetings & Events. He describes the process as “simple and inexpensive” and the client response so far as “over the moon.” The technology, he says, will not replace pre- and post-event surveys, but is used in addition.
How It Works
In a Datakalab setup, video cameras (typically one per 100 attendees) are mounted at the side of the stage to film the facial expressions of the audience. The video feed is run through a software that uses artificial intelligence to instantly recognize and track facial micro expressions. The expressions, explains Anne-Marie Gaultier, president and cofounder of Datakalab, are based on the psychologist Paul Ekman’s work from the 1970s, which identified seven universal emotions: anger, fear, disgust, joy, sadness, surprise, and contempt.
“We can see if people are watching the stage and what emotion they are expressing,” Gaultier says, “and we map this second by second.” Although meeting planners can be provided with a live feed tracking roughly how many audience members are paying attention and how many are displaying positive or negative emotions, the processed data takes a short time to analyze and is then presented in an easy-to-understand format.
Emotional peaks representing surprise or joy are shown within the time frame of the presentation so that meeting planners can see where the audience was shocked by a statistic or laughed at a joke. “A one-hour speech takes around one hour to process,” Gaultier says. “If the client requests it, the data can show male and female responses separately.” Once the cameras are set up, Datakalab processes the video remotely.
Gaultier says that while it’s interesting for meeting planners to see whether attendees are paying attention, the bigger payoff is in understanding if they are responding emotionally to the content. “Emotions are anchors for memory. People remember things better when emotions stimulate the hippocampus,” she says. “Feelings help to encode information from short-term memory to long-term memory,” meaning that if your attendees are passively watching a presentation without any emotional engagement, they won’t remember it a few days later. As Ballin says, “When you are flying people across the country to train them, the cost is high so organizations are keen on finding a way to make it count.”
To avoid privacy issues, signs at the event explain that the audience is on camera but will not be identified. If an attendee doesn’t want to be part of data tracking, there are seats set up off camera. Ballin says that when CWT has used the technology, 99 percent of the audience chose to be on camera.
Emotional Engagement in Practice
CWT’s first tryout for the technology was during an internal conference, training, and trade show event in France for around 200 attendees. Two cameras were mounted unobtrusively near the stage and the organizers were able to watch raw data curves of the audience’s response to each presentation in real time, an experience Ballin likens to watching a stock market ticker. Analyzed data became available a short time afterward and confirmed what the planners suspected. For example, audience engagement was predictably higher during a seven-minute introduction and a 30-minute presentation than for a 56-minute session.
“Once you get over one hour, engagement drops no matter how compelling the content is,” Ballin says. However, he suggests that armed with information on the actual moment when the audience begins to tune out, planners can arrange ahead of time to signal the speaker to introduce something interactive that will re-engage the audience, or even just change a lighting cue so that the room is brighter.
CWT has made several changes to its events based on Datakalab data, including rescheduling sessions and guiding how speakers present content. The data also provides another metric beyond the inexact attendee survey responses when choosing both presenters and content. Ballin says, “If you spend $25,000 on a speaker, you need to be able to validate that more reliably than a score from one to five. And if the data tells you that a large percentage of your audience looked at their phones at the same time for 12 minutes during an education session, you need to be able to tell that story differently.”
Not Just for Meeting Planners
Speakers can also benefit from audience engagement data, especially when the subject matter is difficult. Sophie Stanton, marketing, communications, and citizenship executive with IBM France, experimented with Datakalab technology at a training event for marketers—and took a bit of a risk because the session she chose to film and analyze was presented by one of her seniors at the company. The talk focused on preventing sex trafficking and included a video, anecdotes from victims, information on how to prevent trafficking, and steps to help victims. After processing the data, the team from Datakalab went up on stage to present it. Typically, a meeting planner would receive the data privately, but Stanton wanted the marketing professionals to see how the tool worked for future projects. She says, “If you are scared to face the data, you will never innovate. You have to take risks.”
Fortunately, the data showed the audience was completely engaged and indicated there were four major emotional peaks. The first coincided with an opening joke by the speaker, which was received well despite the somber topic, and the audience also reacted during the video and personal histories. Stanton says, “It was helpful for the speaker to see the engagement and positive responses for future presentations.”
For an organization, this type of unaffected feedback can be very useful. Consider, for example, a company-wide meeting held by the CEO to announce changes. Attendees may try to appear positive in order to be perceived as team players, but the video would capture their immediate, unfiltered responses and provide a heads-up about how the information was really received. As Ballin says, “You can’t trick the data set.”
Happiness Analysis
Zenus, a Texas-based company, also has an emotion tracking product aimed at the events industry. Panos Moutafis Ph.D., the co-founder of Zenus, describes the product as a smart camera specialized for artificial intelligence tasks that scans event attendees to track positive emotions. “The camera analyzes 15 frames per second to capture someone smiling even for just a split second and returns the data to us as statistics, with no identifying images,” he says.
At a pilot program at a large event before Christmas, the cameras—about the size of an iPhone 6—were mounted discreetly inside trade show booths to monitor where attendees were happiest. One booth hosted by a travel company was split into a business-information side and a games side hosted by flight attendants. The cameras showed that attendees were extremely happy and stayed much longer at the gaming side, absorbing more information about the company from the positive experience.
This type of data can be collected on the first day of a trade show and then used on subsequent days to optimize the booth based on attendee response. Moutafis says that reports generated from the data include a “Happy Map” on the parts of an event that sparked the most joy. This is a metric which meeting planners can use to show exhibitors how successful they were, and also help them design future events based on what factors contributed to attendee happiness.
The Future May Not Be Anonymous
The availability of data on audience engagement will almost certainly lead to a better experience for attendees, providing more accurate information for meeting planners to work with. But the next step, creating a pattern of responses based on gender, ethnicity, age, or any other factor, is technically feasible now.
If an attendee decides to be identified and provides a photograph during registration, the identification process can be automated, making it cheap and seamless. In the next couple of years meeting planners will have the choice to implement tools that provide pinpoint accuracy on how successful the event was for each attendee, and the metrics to distribute or expand budgets accordingly. On the flip side, organizations that pay for employee attendance at an event may decide they need that data for their own return on investment. Will the future include a post-event report card for attendees?
