CrowdControl, which launched in November with the goal of improving the accuracy of crowdsourcing projects by analyzing results against a set of artificial intelligence techniques, has raised $ 2 million from Greycroft Partners and RTP Ventures. It’s not big data in terms of size — the output or activity of any given worker produces a relatively small amount of data — but it is a unique approach to the big data mission of improving human activity with algorithms and hard math.
What CrowdControl does, essentially, is partner with crowdsourcing services like Amazon Mechanical Turk (a very close partner, in fact) to provide a layer of quality assurance between the remote workers and the client needing work done. CrowdControl provides a worker-management interface, determines pricing and — most critically — uses its software to determine whether the work product is accurate.
Here’s how I described CrowdControl’s theory and methodology back in November:
Sentiment analysis already is becoming big business for companies such as IBM and SAS that are turning their predictive analytics engines on social media streams. But CrowdControl Founder and CEO Max Yankelevich says there are two big problems in the space right now. One is that current natural-language-processing technologies are better suited to identifying keywords than they are to deciphering true sentiment. The other is that humans, whose brains are inherently better at looking at text in context and working around abbreviations and poor grammar, have a tendency to underperform.
…
To cure that problem, CrowdControl contains more than 15,000 rules to determine how accurate workers are in completing their tasks. Those rules comprise much of the company’s secret sauce, but Yankelevich explained the methods for “adjudication,” the process of judging accuracy, at a high level. A big one is called “plurality,” which entails either assessing a worker’s answer in relation to everyone else’s answer on the same question, or giving the same question multiple times and looking for the same response. Another is “gold answers:” The tester continuously inserts questions to which it knows the answer and calculates how often the worker gets it right.
The company recently completed a survey of half of Amazon Mechanical Turk workers, which should help it further refine its algorithms and processes. As Yankelevich told me in November, where workers live and what they do for a living can have a big effect on how they perform. Presently, CrowdControl says its ideal use cases are sentiment analysis, data cleansing and data normalizing, which is the process of adding consistent structure to unruly data sets.
Image courtesy of CrowdControl.
Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.
- Defining work in the digital age: an analysis by GigaOM Pro
- Dissecting the data: 5 issues for our digital future
- Siri: Say hello to the coming “invisible interface”
