Does A/B testing have a place in meetings?

I was reading an article about how A/B testing is dominating Web design and generally changing how businesses evolve in Wired magazine last night, and it made me wonder if some version of it isn't what we already do in meetings.

What A/B testing is, according to Wired: "Using A/B, new ideas can be essentially focus-group tested in real time: Without being told, a fraction of users are diverted to a slightly different version of a given web page and their behavior compared against the mass of users on the standard site. If the new version proves superior—gaining more clicks, longer visits, more purchases—it will displace the original; if the new version is inferior, it’s quietly phased out without most users ever seeing it. A/B allows seemingly subjective questions of design—color, layout, image selection, text—to become incontrovertible matters of data-driven social science."

If you can use this technology to maximize your meeting (and organizational) Web site, wouldn't that just be the bee's knees? But what's easy to do on the Web is a little harder in real life. You can't control all the variables or truly randomize the testing in the same way. But still, it's the same concept that has meetings professionals use when they break their marketing program into pieces to address their niches, isn't it? And when they tweak a piece of the program, introduce something new in formatting without doing away with the old first, then gauge which "works" best (with "works" meaning whatever it means for you, be it learning achieved, new behaviors cemented, information shared, etc.)?

While it makes me uncomfortable to think that it's not important to understand why certain things work better than others, as the article notes, maybe I need to just let go of that and trust that if the data says something is better, it just is even if I can never figure out why? That has to be better than relying on HiPPOs, right? (HiPPOs being "highest-paid person’s opinion," as defined in the Wired article.)

Like it or not, we're living in an increasingly data-driven world, and that's not necessarily a bad thing. I think we just have to be careful not to lose the meaning behind the numbers. Anyway, it's a fascinating read, and a reminder that, whether we know it or not, we're all both the scientists and the guinea pigs in the great experiment of life.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.