NetBase and Gnip – Maintaining Streams of Quality Content
When I describe my job, I borrow from my childhood near Pittsburgh and explain that if NetBase were a steel mill, I’d be the guy making sure we have trains and barges bringing in high-quality iron ore and the other raw materials we need to make high-quality steel. Gnip, whose Plugged In program we are proud to have joined from its launch, operates one of the biggest, fastest “trains” delivering raw data to us. Out of the roughly 50 million social media postings we analyze and index each day, the majority come from Twitter and we rely on Gnip’s ability to operate a TGV-scale delivery platform. The TGV, if you don’t know that term, is France’s Tres Grande Vitesse, the fastest railway in the world. We also depend on Gnip for Disqus blog comments and much more. But they’re not just filling up buckets for us – Gnip adds value.
Although the TGV has a switching system, its scale is microscopic compared with the “switching” we need from our content suppliers. Although in many cases we tell our supplier “Give us everything you have,” we are able to rely on some of them, including Gnip, to filter the content to ensure that our customers receive 100 percent of the relevant tweets, posts and articles and as little spam and junk as possible. That makes them more a supplier of raw material; it makes them a partner. We also rely on them to put in place the technical and business processes to ensure that we are in compliance with licensing and other requirements created by the owners and distributors of the data they supply – and to help our customers do the same. Just keeping up with those processes and requirements is a challenge as the social media ecosystem evolves and matures.
The fast pace of change means we also need partners who aren’t reluctant to wrestle with the tough challenges. At its Big Boulder conference and our private conversations, the Gnip team has shown a remarkable willingness to engage on opportunities, ethics and obstacles. The company has a culture of transparency that we value, with patience and flexibility. And did I mention fun? Gnip even teases itself about embracing a “Boulder way of life” that is familiar to those of us who now call Silicon Valley home – work hard, play hard. Of course, that might be because the Gnippians, as we call them, seem to spend half their time in California! And they should, because Silicon Valley is a center of this revolution in the way that my hometown, Pittsburgh, was a center of the industrial revolution. Keep those trains and barges coming!