“As you can see, this is basically kitten war for ideas,” said Princeton sociologist Matthew Salganik in introducing the wiki survey tool All Our Ideas at the Collective Intelligence 2012 conference.
Instead of a simple poll of which cat is cuter — like at kittenwar.com — participants are asked the equally direct: Which idea is better?
Other design features include: Participants can contribute ideas but cannot campaign for them among friends or followers, because paired presentation of candidate ideas is set by the All Our Ideas algorithm. And for any individual idea, no information is provided about contributor or current vote tally.
Salganik discussed the case of the PlaNYC project conducted by Mayor Bloomberg’s office. Its wiki survey question was: Which do you think is a better idea for creating a greener, greater New York City?
Staff seeded the survey with 25 ideas; 464 ideas were contributed (239 in another slide, the discrepancy is unclear); and 28,829 votes were cast. Of the top-ten vote getters, eight were contributed by citizen participants.
According to Salganik, top vote-getters included both alternative framings of ideas that mayor’s office staff were aware of (“Keep NYC’s drinking water clean by banning fracking in NYC’s watershed;”) and novel ideas that staff later verified as potentially effective (“Plug ships into electricity grid so they don’t idle in port — reducing emissions equivalent to 12000 cars per ship.”)
Salganik explained:
Your intuition might be: If we open this up and anyone in New York can upload something, we’re going to get a lot of junk. And that is true; you get a lot of junk. The worst ideas are uploaded by users. It’s also the case, as I said, that some of the best ideas are uploaded by users. So the ideas uploaded by users have a much higher variance. And if you combine variance with high volume, then you’re likely to find these extreme cases. …
Usually our intuition is variance is bad. But if we’re only looking for extreme cases, and we have a good filtering mechanism, then this variance, plus the volume, is the place where we’re going to find these extreme cases. To the extent that the variance is sufficient and the volume is sufficient, the ideas that are uploaded by users will always end up scoring better than many of the initial ideas.
Video below, slides here (pdf), and paper here. More videos from the Collective Intelligence 2012 conference.
