How should you do a neuromarketing test? I’m increasingly being asked whether the scales from the Emotiv EPOC Affective™ Suite system can be used to assess cognitive and emotional responses in e.g. customers. After all, it would be really appealing if we could use a full box set with responses such as Engagement, Meditation, Frustration and Excitement. I also see that some new up and coming companies use this system more or less from the box. After all, who wouldn’t just pay $5,000 for a neuromarketing study rather than the more expensive studies that require whole teams and specialists?
I use the Emotiv system myself for many of my studies. For recording EEG it works well. But when it comes to their emotional scales, the truth is that they are, at best, just a black box with many unknowns. Quite realistically, it’s more like a can of worms.
From one of my own data, it’s easy to check whether the Emotiv scales are distinct. They are produced very nicely through the export function in iMotions’ Attention Tool – the best scalable neuromarketing suite I know and can think of. Best of all, it allows export of time-synced raw EEG data, along with data from eye-tracking, GSR, facial coding and much more.
So how do the Emotiv scales fare? Let’s take the example from “Meditation” and “Frustration”. After all, we should expect that scores for “Meditation” would be quite different from “Frustration” scores, right? Well, the truth is that these scales are highly correlated:
So unless you think that meditation is really about frustration (it may be to some…), you should be skeptical towards the Emotiv scales. Some of the other scales seem to fare better, such as the Engagement score. My own studies so far suggest that the Engagement score is related to working memory load, but this is indeed still a heterogenous construct, and much too premature to draw any conclusions. I do not yet know how specific Engagement scores are to working memory…and should it then be called “Engagement”?
The basic problem remains, however, that the Emotiv system is still a black box solution, and more or less impossible to determine how the scales are made. And to be honest, I would not trust a 22-year old graduate who has taken a course or two on neuroscience, nor any company using the Emotiv solution up front. Just in the same way I would not trust the same persons to perform my bypass surgery or repair my car. And yes, you may call it ad hominem attacks, but in the tradition of science this is how it is. When you speak bullshit, you are called a bullshitter.
As you can guess, I am not a proponent of black boxing, particularly not in neuromarketing where we should be able to converge on the same solutions. Quite the contrary. I simply do not understand the need for secrecy among neuromarketing companies. The science is already out there, so why make up new scales? It opens up the possibility of cheating, snake oil production and what is less. Think about the strategic blunders that may be made based on erroneous and unscientific hand waving.
If you want to do neuromarketing studies, make sure you do it right from the beginning. There is already too much hype and BS in this industry, so let’s start being self-corrective.