Data Platforms Set Off a Cambrian Explosion

The Jan. 18, 2014, edition of The Economist featured “A Cambrian Moment,” a special report on tech start-ups. The report discusses how the digital marketplace is experiencing a Cambrian explosion of products and services brought to market by an ever-increasing plethora of new start-ups. Precipitating this explosion is the modern “digital platform” – an assembly of basic building blocks of open-source software, cloud computing, and social networks that is revolutionizing the IT industry. This digital platform has enabled technology start-ups to rapidly design and bring to market a raft of new products and services.


A similar evolution is taking form in the world of data. Anabelle Gawker, a researcher at Imperial College Business School, has argued that “platforms” are a common feature of highly evolved complex systems, whether economic or biological. The emergence of such platforms is the ultimate result of evolving exogenous conditions that force a recombination and rearrangement of the building blocks of such systems. In the data world, this rearrangement is taking place thanks to the falling cost of information processing, the standardization of data formats, and maturation of large-scale connectivity protocols. The emergence of such data platforms has significant implications for a range of industries, not the least of which are data-intensive industries like healthcare and financial services. Like the digital platform, the data platform will give rise to an explosion of data-enabled services and products.

Early evidence already can be seen in the pharmaceutical and drug manufacturing industries. Drug manufacturers need thousands of patients with certain disease states for late-stage clinical trials of drugs under development. This patient recruitment process can take years. The Wall Street Journal reported that to speed up the recruitment process, drug manufacturers have turned to entities such as Blue Chip Marketing Worldwide, a drug-industry contractor that provides data-enabled solutions based on consumer data sets obtained from Experian, a data broker and provider of consumer data. Blue Chip uses sophisticated big data analyses and algorithms to identify individuals with potential disease states. Blue Chip’s services have already enabled a drug manufacturer to cut patient recruitment time from years to months.

Trading is another industry where early indications of such movements can be seen. Also reported in The Wall Street Journal, in their never-ending quest to gather market-moving information quicker, traders have turned to outfits such as Genscape, a player in the growing industry that employs sophisticated surveillance and data-crunching technology to supply traders with nonpublic information about topics including oil supplies, electric-power production, retail traffic, and crop yield. Founded by two former power traders, Genscape crunches vast amounts of sensor data, video camera feeds, and satellite imagery to draw patterns and make predictions on potential movements in supply and demand of commodities such as oil and electricity.

We are in the early days of this evolutionary process. The evolution of the data platform will most likely mirror the evolution of the digital platform, although it is expected to proceed at a faster pace. As competing technologies and solutions evolve and mature, we will see consolidation and emergence of just a few data platforms that will benefit from the tremendous horizontal economies of scale. Information industry incumbents such as Google will be in the pole position to be leaders in the data platform space. For example, Google already has a massive base of web and social interaction data thanks to Google Search, Android, and Google Maps, and it is making aggressive moves to expand into the “Internet of things,” the next frontier of big data.

Concurrently, we will witness a broad emergence of a long tail of small vendors of industry-oriented data products and services. Blue Chip and Genscape are first movers in a nascent market. As the economics of harvesting and using data become more attractive, an increasing number of industry players will want to leverage third-party data services, which in turn will create opportunities for data entrepreneurs. The Cambrian explosion will then be complete and the start-up garden of the data world will be in full bloom.


Google’s Lessons Applied to The Consulting Profession

I recently read a review of a newly released book “How Google Works” by Eric Schmidt and Jonathan Rosenberg (Don’t Be Modest).  This is an interesting book by the architects of Google on what makes Google so successful, and how can other companies emulate its success factors and experience the growth rate it has over the past fifteen years.  The authors describe three critical factors that are central to Google’s success: 1) Thinking Big 2) Failing Fast and 3) Leveraging  Data.  As I was reading the review of the authors’ treatment of the subject, I could not help but think the applicability of these success factors to the management consulting profession.  When I come across successful management consulting professionals in my day to day professional life, I see them possessing all these success factors.

First and forhow-google-works-2014-10-08emost, “Thinking Big”, or in the parlance of Silicon Valley, the “moonshot”. Google’s management is not happy with employees striving for incremental improvements; they want employees to think up ideas that will deliver 10x gains.  And so it is with management consulting professionals, or at least the good ones.  Consultants are paid big bucks not to provide advice that merely enhances existing strategies, but to think up big ideas that change the nature of the game itself.  Truly successful consultants however do not stop there: they are actually able to convince their clients to launch such game-changing strategies, and subsequently also help them execute those strategies.  “Thinking Big” is applicable to the more mundane and normal as well.  If the client has asked for a cost reduction strategy that delivers 10% cost improvement, there is no reason not to strive to overachieve and over-deliver for a 15% reduction if possible.  And of course, “Thinking Big” in a consulting context means having the big picture.  Many a time consultants get too frequently caught up in the weeds and the nitty-gritty detail of the data analyses, and fail to extend their findings to truly altering the big picture. Thinking Big sets a high bar to over-achieve and over-deliver, and what can be better than that for someone in a services industry such as consulting.

Second, “Failing Fast”, which refers to the cultural aspect encouraging people to rapidly experiment to test the viability of their ideas, adapt and evolve the ideas based on the results, or move on if those ideas are not workable.  “Iteration is the most important part of the strategy”, as the authors of the book advise.  Google is well known for its ability to push out hundreds of incremental product updates every day as part of this iterative strategy.  This agile iterative approach is in Google’s DNA.  And the significance of this piece of advice can be seen if one considers how Agile based approaches have been successfully applied to managing complex projects and delivering new ground-breaking products.  Iteration is at the heart of Agile. An agile iterative approach to problem solving is an absolutely necessary tool in any good consultant’s toolbox.  This is particularly important in a strategy and advisory setting, where often information at hand is incomplete and the problem to be solved is ambiguous. Even the sharpest consultants will fail if they cannot rapidly adapt their approach as conditions change and new information emerges during the course of the project.

Finally, “Leveraging Data”, which refers to the primacy of data over a HIPPO, group think or collective experience, intuition or gut.  Google conducts hundreds of data-driven experiments to gauge how a particular product feature or enhancement would be received by the market place.  Such data-driven decision-making is at the core of a number of successful companies in other industries.  This exactly is the essence of the fact-based hypothesis-driven approach that all good management consultants swear by.  All good scientific approaches to solving management problems start with defining hypotheses, and then subsequently collecting and analyzing data to test those hypotheses.  Good consultants may start with their favorite hypotheses, but will be the first ones to throw them out the window if the data show those hypotheses to be untenable, no matter how much effort has been spent in defining the hypothesis and no matter how near and dear the hypothesis is to them.  Data, no matter how insignificant a role it plays, lends a certain credibility to strategic analysis that plain intuition and “experience” cannot.  Distinctive consulting skill comes from the ability to frame up the data analysis in a way that is effective and efficient, as well as the ability to interpret the results in a way that is convincing and yet simple.

Providing the foundation for these critical factors is Google’s culture of employee empowerment and enfranchisement.  Google hires the best and the brightest, but goes to great extent to ensure that employees are happy and engaged.  And culture perhaps is the greatest factor that distinguishes great consulting firms from the ones that are mediocre.  A culture that encourages healthy debate and intellectual challenge to authority, no matter how low down the totem pole you are, and even one that goes beyond to make it a mandatory requirement, an “obligation to dissent”.  A culture that is fair and ruthlessly meritocratic, yet one that is fun.  But most importantly, a culture that patiently cares about the development of the company’s most important asset by way of providing strong mentorship, support and professional guidance at the right place and at the right time.