In “Ecological Evaluation of Persuasive Messages Using Google AdWords” the authors show how internet advertising systems can be used to test and explore the mechanism of persuasion in different ways, for example for exploring natural language processing.
In recent years there has been a growing interest in crowdsourcing methodologies to be used in experimental research for NLP tasks. In particular, evaluation of systems and theories about persuasion is difficult to accommodate within existing frameworks. In this paper we present a new cheap and fast methodology that allows fast experiment building and evaluation with fully-automated analysis at a low cost. The central idea is exploiting existing commercial tools for advertising on the web, such as Google AdWords, to measure message impact in an ecological setting. The paper includes a description of the approach, tips for how to use AdWords for scientific research, and results of pilot experiments on the impact of affective text variations which confirm the effectiveness of the approach.
A similar approach is described in the book Supercrunchers and has been used by political consultants to do message testing in a fast and relatively cheap way. As we proceed down the road I think secondary research uses can be found for a lot of the services online. The secondary uses of data allow for a wide variety of creative uses that explore language, social structures and economics. The data shadow of the web is the next frontier for many of these empirical sciences.
Besides exploring the theoretical notion the authors provide a great hands-on description of how to set up your experiment. Very intriguing.
(Full disclosure – I work at Google, but write here in a private capacity. The study was done in cooperation with Google and was funded by our research programme Google Research Awards to a part. I still think it is really, really interesting – but of course you should know those two things when reading this post!)