AdWords
We at IPA have recently been delving into the world of online advertising to help us spread the gospel of rigorous impact research. Being who we are, we could not resist this opportunity to run a field experiment. We designed one that would help us optimize our advertising strategy while also settling an important score: which academic institution's rep pulls the most weight in cyberspace? Our ad was simple:
Poverty Research
Breakthroughs to Fight Poverty
By [randomized] Researchers
Inside the brackets in the third line, Google ads then randomly inserted one of nine university names, one of three acronyms (IPA, JPAL, or FAI) , one of three "impostor" acronyms (ITA, GTAM, and MAI) that were phonetically similar to the real acronyms, or one of three generic words (university, top, and academic).
Needless to say, we got some interesting results. Surprisingly, Nobel factory UChicago fared the worst among all university brand names that we tested. Interestingly, albeit we will admit this was not our prior intended subsample analysis, turns out that the cities of New York and Chicago were both the laggards in this race; NYU, Columbia, Northwestern, and UChicago had the lowest response rates among universities tested. The top ones were Dartmouth, MIT, Yale and Harvard, in order.
As you can see, generic keywords "university," "top" and "academic," were significantly more effective than the ones naming schools or organizations.
Among the acronyms, FAI fared the best, outperforming Princeton, Harvard, and Yale. J-PAL narrowly edged out its counterpart IPA, and was right on the heels of Ivies Princeton and Harvard. But before any acronym organization gets too excited, we must note there was no significant difference between real and impostor acronyms. Although J-PAL and FAI marginally outperformed their impostors, IPA actually had a slightly lower click-through rate than "impostor" ITA. These findings unfortunately temper our enthusiasm at IPA and its fellow research groups apparently having as big a rep in cyberspace as the top Ivies.
|
To increase the probability that we would offend someone with this post, we also ran a descriptive regression examining the correlation between select university attributes and its apparent popularity in cyberspace. Its results come with the disclaimer that there are only nine data points and no identification strategy whatsoever, so readers are warned to interpret this regression at their own risk:
Coef. |
Std. Err. |
T |
P>|t| |
95% Conf. |
|||
rank |
0.003283 |
0.00462 |
0.71 |
0.529 |
0.01142 |
0.017985 |
|
selectivity |
-0.00767 |
0.004306 |
-1.78 |
0.173 |
0.02138 |
0.006031 |
|
endowment |
-0.00214 |
0.003241 |
-0.66 |
0.556 |
0.01246 |
0.008175 |
|
attractiveness |
0.080691 |
0.059113 |
1.37 |
0.266 |
0.10743 |
0.268816 |
|
drug use |
0.069523 |
0.06215 |
1.12 |
0.345 |
0.12826 |
0.267311 |
|
Constant |
0.718469 |
0.282063 |
2.55 |
0.084 |
0.17918 |
1.61612 |
|
N (# obs.) |
9 |
The dependent variable is the percentage of ad views that actually resulted in clicks. "Rank" indicates the 2010 US News and World Report ranking of each school. "Selectivity" is the 2010 undergraduate acceptance rate in percentage points, endowment is endowment size in billions, "attractiveness" is an index measuring how attractive the undergraduate females were (according to a survey of students and alumni from collegeprowler.com), and "drug use" is an index measuring the prevalence of undergraduate drug use (higher scores indicate lower prevalence, again from collegeprowler.com). Interestingly, selectiveness is the most significant right-hand variable; the negative point estimate indicates that more exclusive schools got higher click rates. Second is female attractiveness, which is positively correlated with performance. Low reported prevalence of drug use is correlated with more clicks.
And, as we are now learning, adding 2010 to your headline makes a bigger difference than anything we tested here.