Who Twitter Got?

The Baltimore Ravens and the San Francisco 49ers battle it out on Sunday. Snacks have been acquired. Storylines have been established. Someone, somewhere is carefully dusting a new 72″ plasma. We’re ready for our biggest national holiday.
However, the crucial question of who Twitter users are rooting for hasn’t been answered. It turns out that figuring out the answer to this question is pretty complicated:
Why not just search for “rooting Ravens” and “rooting 49ers” and compare? Unfortunately, Twitter doesn’t report the number of search results for different searches, so we wouldn’t know how many people tweeted “rooting Ravens” or “rooting 49ers”. As we’ll see, this isn’t an accurate metric in any case.
Why not used an automated tool? Unfortunately, automated sentiment analysis tools can’t answer this type of question accurately. To understand why, try to guess how automated tools are likely to assess the following examples:
Good Gawd “@BreeOlson: Who’s rooting for the #Ravens this #SuperBowl ? I am! http://t.co/k2UKnApR“
Follow if you’re rooting for the 49′ers! RT if you’re rooting for the Ravens! #SUPERBOWL
“@989RadioNow: who are you rooting for in the #SuperBowl ? my vote goes to the #RAVENS” 49ers here!!!
These aren’t isolated examples; in order to express themselves in 140 characters or less, people often write in ways that are difficult for machines to interpret. Luckily, CrowdFlower has a better tool for the job.. We pulled over 1,000 tweets referencing the Super Bowl and “rooting”, and had the crowd decide if the tweet was rooting for the 49ers, the Ravens, or neither. It took about fifteen minutes to set up a CrowdFlower task that got thousands of answers from hundreds of human beings all over the world.
So, what was the answer?
In news that will surprise no one, Twitter, like God, is rooting for …
SF 49ers logoOk, maybe we let our hometown bias get the better of us there. Here’s how the data came out:

Number rooting for the 49ers vs the Ravens

Chart showing Twitter rooting for the Ravens over the 49ers by about two to one, with a large number of Neutral votes
So Twitter users who mention the Super Bowl and “rooting” in their tweet are rooting for the Ravens about twice as often as they’re rooting for the 49ers. Let’s pass over this embarrassing fact as quickly as possible. Why all the tweets that didn’t express an opinion?
We took a closer look at those tweets. A lot of them looked like this:
The Superbowl is this Sunday! Who are you rooting for?
So we formed a hypothesis: a lot of the neutral tweets were promotional: companies or other organizations trying to start a conversation with their followers about the Super Bowl. To answer this question, we had the crowd look at the Twitter profiles and tweets of everyone whose tweet had been rated Neutral. We asked them to decide if the user was a company or a person. Again, it took about fifteen minutes to set up a task that got thousands of answers on our dataset from all over the world. Here’s what we got:

Number of Neutral tweets from Companies and from People

Chart showing that about 40% of the accounts are companiesAbout 40% of the Neutral tweets came from companies. That means that there were a very high number of tweets in the original dataset from companies. It makes sense that companies would try to start conversations with their customers around the Super Bowl (those who live in glass houses … ), but it’s still surprising to get so many in a mostly random pull of Twitter data. One explanation might be that there are fewer companies than people on Twitter, but companies tweet more often. Food for future research.
So in summary:
  • Twitter’s rooting for the Ravens
  • A lot of Twitter traffic about the Super Bowl comes from company accounts
  • The people who are most excited about Sunday are all Beyonce fans (you’ll have to trust me on this)
Have fun Sunday.
And if you’re interested in analyzing Twitter with the crowd, check out our dead-simple, crowd-powered tool Senti.

The Accuracy of Apple Maps Listings: Reality Check

Update: this piece has been featured on Mashable.
map pin Apple Maps, which replaced Google Maps for all iPhone and iPad users in the most recent version of iOS, has been receiving a lot of attention. Things died down after a while after Tim Cook’s apology, but flared up again last week when travelers in Australia were stranded when they were directed into the Outback. While we all enjoy taking potshots at the cool kids, as data nerds, it bothers us that all we’ve had to go on in the Apple Maps discussion is anecdotal evidence. So we decided to assemble a dataset that would let us take potshots with authority give some solid answers.
Apple Maps has been criticized for a few things, from how it interprets your search to the directions it gives. We asked a simpler question: when you ask for a restaurant, hotel or other business, how often do you get the right location?
Extremely Brief Summary
Apple Maps in the US is bad enough to be noticeable: you probably won’t throw away your iPhone, although you may miss a dinner reservation. Those of you using Apple Maps in the UK, however, might want to keep emergency food and water in the car.
For more detail and statistics, read on …
How We Did It
We started with a list of 1,000 US businesses in our database, then added 100 UK businesses to give some idea of international differences. We had our crowd find the official websites of these businesses and extract current address information from the website. This was our reference data.
We had the crowd pull the same information from Apple Maps, Google Maps, and Bing. In order to replicate the search experience of a typical user, we had people search for “business name, city” first before trying different variations.
Then, we compared the results. In the case of major errors we investigated carefully – many UK addresses lack street numbers and can be described multiple ways.
Results
Our results have two parts: first, if I search for a business, will I get a result? Second, will that result be accurate?
Percentage of businesses found
Percent of businesses found in the databases of three search providers
First we tested how many of the businesses we could find on each service. In results that will surprise no one, Google Maps has listings for the most businesses in the US and UK, with 89% coverage in the US and 91% coverage in the UK. Apple Maps is credible in the US with 74% of businesses found, but with 47% coverage in the UK the phone book starts to look like a real option. Bing is somewhat better than Apple but not great, with 79% coverage in the US and 57% coverage in the UK.
Not being able to find a business is one thing. Getting an incorrect listing might be even worse.
Percentage of listings with major errors
Graph of error rate of business listings for three map providers
We consider a major error to be anything that puts you a block or more from your intended destination. With a 3.4% major error rate in the US (compared to 1.1% for Google Maps and 1.3% for Bing) there’s a decent chance Apple Maps will send you in the wrong direction. Even small errors can be frustrating: witness the difference between the Apple Maps result for Nick’s Crossroad Cafe in Albuquerque with the actual result. (We’re using a Google Maps screenshot for convenience, but our address data comes from Nick’s official website.)
The difference between Central Ave NW and Central Ave NE
Map showing distance between Apple Maps result and actual result
In the UK, not only are the more incorrect listings, but you might find yourself even further off-course. Here’s the Apple Maps result for Advance Gym in Reading, UK compared to the actual result. You’ll also notice that the Apple Maps version is missing a lot of detail compared to the Google Maps version.
About four miles
Map example showing distance between Apple Maps result and actual result Note that we tried to be forgiving in making our evaluation: for example, if Apple Maps had multiple listings and at least one was valid, we accepted that as valid.
Will I get where I want to go?
So when you search for a business, what are the chances that you’ll find the business you want AND the address will be accurate?
With Apple maps you have a 71% shot, compared to 88% for Google Maps and 78% for Bing. But with a 3.4% error rate, you’re three times as likely to be sent on a wild goose chase with Apple Maps.
In the UK, the situation is dicier: you’ll get a good listing 33% for Apple Maps, compared to 88% for Google Maps and 55% for Bing. And 30% of the time, you’ll get a listing, but it’ll be incorrect.
Local data is hard. Small businesses are closing and starting up all the time. Streets are being re-routed. And Apple is just getting into this game. We’ll be very interested to see how Apple’s data improves over time.
Want to know more about how CrowdFlower handles business data? Get in touch or read about how we provide high-quality SMB data for sales and marketing.

Announcing Our New Cloud Module Partnership with Parse

Today we are very proud to announce a partnership with Parse, the leading mobile and desktop application platform, to bring crowdsourced photo moderation to Parse-powered applications with the CrowdFlower Cloud Module.
As Parse writes today on their blog:
“Cloud Modules are the easiest way to integrate a Parse app with third-party services and libraries like the already released Twilio, Mailgun, and SendGrid Cloud Modules. The CrowdFlower Cloud Module, the latest addition to the Parse ecosystem, gives Parse application developers access to RTFM or Real Time Foto Moderator. Many applications include photo sharing amongst friends, family, and the world. Moderating these images can take a lot of time and resources. By crowdsourcing this tedious task, developers that create applications that include photo-sharing, like the Parse-powered TOMS app, don’t have to create a complex workflow to manage moderation. Parse developers now have the power of millions of eyes moderating their content with just a few lines of code.
Here’s a look at how the Crowdflower Cloud Module works:
var crowdflower = require("crowdflower");
crowdflower.initialize("crowdflowerKey");

Parse.Cloud.define("myCrowdFlowerFunction", function(request, response) {
  crowdflower. submitImage({
    image : {
      url: request.params.url
      metadata: {
        objectId: request.params.objectId
      }
    }
  }, {
    success: function(httpResponse) { response.success("Image submitted!"); },
    error: function(httpResponse) { response.error("Uh oh, something went wrong"); }
  });
});
To learn about this new Cloud Module, take a look at the Parse documentation page.”
At CrowdFlower, we’re celebrating this partnership for many reasons. When we initially designed and released RTFM, we did so specifically with app developers in mind. We know that mobile apps will continue to create large amounts of user-generated content. Our market research indicated that content moderation can be a challenge for app developers, particularly small teams. Most developers struggle with the decision whether or not to moderate content; those who do moderate have to decide whether or not to devote internal resources to developing a proprietary moderation tool. We see the power of crowdsourcing every day, and we wanted to make it more accessible to the developer community. Now developers who moderate photos via the Parse Cloud Module can easily integrate photo moderation while leveraging our experience with crowd management and quality control.
Our partnership with Parse has been taking shape for most of 2012, and we are thrilled that this Cloud Module is now available. If you have suggestions for improvement or ideas for other partnerships, please drop us a note at: rtfm at crowdflower dot com.

Participation & Patience: Making Crowdsourcing Communities Sustainable

CrowdConf is nearly here and we are excited to welcome back Shelley Kuipers, Founder and CEO of Chaordix, as a speaker at this year’s conference. With CrowdConf just around the corner, here are some great insights from Shelley about crowdsourcing and participation, to get you excited for the big event. Looking forward to seeing you Tuesday!
Forward thinking brands such as Patagonia, LEGO, Muji and their fans have long understood the value of social spaces and events (first physical, then digital) where they could meet, interact and co-create the future together. Today, companies, organizations, and geographic communities need to become fully oriented toward embracing transparent and authentic collaboration with their crowds.
Although some early attention has been given to shifting the role customers can play in brand communities and innovation processes, this open, participatory and collaborative shift can also engage crowds of employees, suppliers, alliances and citizens. From the foundation of social media and early wins in open innovation, the social enterprise and the open government movement have emerged. I believe that a very important aspect of brand and citizen participation will also involve work in developing nations to give voice to emergent participants both social and commercial; this holds much promise for brands and communities who want to step up and meet the authentic and clearly-articulated needs and wants of this wave of new voices.
True, sustainable and meaningful communities can emerge from crowdsourcing when the collective wisdom of their participants is respected and the brand understands that they’re building long-term, collaborative relationships together, rather than drive-by idea-hunting forums. When communities and brand leaders accept that fostering customer and employee participation in a brand comes with positive and negative feedback, they’ll see that a well-managed (and transparently resolved) criticism can be as valuable as a positive net-new idea.
To be fully engaged, returning community members need a compelling participant experience, regular feedback and encouragement, not just badgering with periodic one-way surveys and idea contests. Brands will reap the respect and attention that they sow. Participants need to be rewarded intrinsically and (when appropriate) extrinsically for thoughtful and helpful contributions – it should be baked into the culture and design of any crowdsourcing community.
The loudest people don’t always have the best insights, but wisdom can ultimately be found in a crowd, if you’re open and patient.

Why You Need to Know How to Build A Crowd Business Model

Ross Dawson is leading a Crowd Business Models Summit the day before CrowdConf and we are excited to hear what he plans to cover in this intensive half-day program. Come learn about relevant business models for your company and maximize your time at CrowdConf, especially for those visiting from out of town. Here is a preview from Ross Dawson of what to expect at the Summit.
New applications of crowdsourcing are providing major new opportunities for business, society, and government. More broadly, they are fundamentally disrupting the traditional work model as we see a fundamental global shift to distributed work and value creation.
One of the most important aspects of this shift is the rise of ‘crowd business models,’ that is based on the creation of value by the crowd.
In my recent book, Getting Results From Crowds, I take a look at these various business models and analyze their benefits and challenges.
Some models, such as design competitions, have being used for many decades, drawing on the participation of many to find the best. While newer crowd-based models have being enabled by the rapid rise of connectivity and bandwidth. A few of the better known examples include Wikipedia which consists of reference content generated by many, service marketplaces such as Freelancer.com and Odesk which bring together supply and demand for talent, prediction markets that aggregate opinions into insights, and crowd design platforms such as Quirky that enable ideas from crowds to be captured and brought to life.
I would argue that a majority of business models are incorporating elements of crowdsoucing such as ratings, product feedback, and social media sharing. While crowd business models are increasingly important for entrepreneurs, investors, and corporate executives, so far relatively little attention has been paid to the idea and its applications.
At the Crowd Business Models Summit we will bring together industry leaders who have experience and insights to share in the field of crowd business models. We will delve into the success factors for crowd business models, in attracting and rewarding contributions, creating value, and monetizing that value creation. We will also look into the future of crowd business models, where they are going, where the opportunities lie, and what entrepreneurs need to do to seize that value.
The event will be highly participatory, helping to generate insights not just for those present, but also the broader community, in developing and refining useful frameworks for crowd business models and how to make them successful.
It promises to be a not-to-be-missed event, packed into an intense half-day session, to help kickoff CrowdConf the following day. I hope you can join us!
-Ross Dawson