Big Data Gets Bigger
Nearly 14 billion years ago, massive forces of gravity and friction converged to spark the big bang that gave life to the universe as we know it. Today, the same type of convergence is occurring in the business world.
After years of technological innovation and aggressive IT investment, companies now sit atop a vast volume of data on their business activities. The sea of structured data on production, marketing, sales and pricing, HR, finance, facilities and operations, and other internal matters is matched by transaction-level data from supplier, customer, and partner relationships. Best-in-class companies are deriving extraordinary value from this data by mining it for hidden insights on their customers, products, and business activities.
But the convergence of major technology shifts like cloud computing, mobility, and social and business networks has sparked an explosion of a new class of “unstructured” data – texts, tweets, blog posts, web-based videos, and other social postings – that now greatly exceeds traditional data types found within most organizations. According to a study by independent research firm Aberdeen Group, the vast majority – more than 83% – of the information companies now have is unstructured. And companies that effectively harness such information stand poised to gain unique insights that give them significant advantage over their peers.
Online networks have been key enablers of the first-wave of globalization, making it as easy and transparent to conduct business with a partner on the other side of the world as with one across the street. Consumers tap into personal networks like Facebook, Twitter and Amazon.com to learn, share and shop better. Leading companies leverage business networks to collaborate more efficiently with their employees, customers and other trading partners.
But networks are about more than just connecting companies, people and processes. Their real power lies in what goes on inside them – all the interactions, transactions and commentary - and the massive amounts of unstructured data that they generate. It is from this data that the next wave of innovation and business productivity will come. Traditional relational or structured data may serve as the foundation for analytic efforts. But by combining it with unstructured information, companies can gain additional insights that enable them to make better business decisions. And with the right technologies, they can do it in real time.
Consider how consumers are already benefiting from the convergence of structured and unstructured data:
- Amazon.com harvests the buying patterns (transactions) of its customers to recommend complementary products for up-sale. It also uses community-generated ratings and tips to further guide buying decisions.
- Twitter and Facebook mine unstructured comments to develop psychographic profiles of users and deliver highly targeted advertisements (e.g., promoted Tweets) that have much higher conversion rates than traditional advertising approaches based on demographic segmentation alone.
- The Facebook “Like” button – now pervasive on websites ranging from online stores to leading news media outlets to political blogs- offers far more accurate (and quicker) insights into buyer preferences and public opinion than traditional focus groups and straw polls.
- New online and mobile banking options from startups like Square are capitalizing on the convergence of mobile, cloud, and social to foster entirely new payment models, where a consumer’s mobile device can, using geo-location information, detect when its owner is in his favorite coffee shop, use the Cloud to automatically place an order, and virtual financial networks to settle out between the customer, the shop’s register and the bank. If the consumer is so inclined, he can include a tip and even post a rating or comment on his visit via integrated social channels.
This new “Internet of things” is not only making our lives more efficient, it’s unleashing a host of new data that can be used for everything from social research to targeted marketing to post-sale service. Leveraging the hundreds of billions of dollars of financial transactions and transactional data along with relationship history that resides in business networks, for instance, buyers and sellers can make more informed decisions by detecting changes in buying patterns or pricing trends and provide confidence and qualifying information on a potential – yet unfamiliar – trading partner. And, when combined with community-generated ratings and content, they can glean not only real-time insights, but also recommended strategies for moving their businesses forward.
So things like performance ratings where buyers rate suppliers and suppliers rate buyers. Others in the community can use this information to help determine who to do business with or to help detect risk in their supply chain. By accessing the real-time insights into invoice approval status married with historical data on payment patterns of given buyers that business networks provide, banks and other service providers can remove the risks from receivables financing, allowing them to offer more competitive rates and new services to network members that increase revenue.
Innovative companies are already on the bandwagon, harvesting the information inside the communities they participate in to deliver new insights and capabilities. For example:
- When looking for alternative sources of supply, Plaid Enterprises, a mid-sized manufacturer of do-it-yourself products used a business network to uncover potential suppliers. The network not only provided a directory of suppliers that met Plaid’s requirements, but offered up insights into each supplier – such as how many other buyers the supplier was doing business with on the network; how many RFPs it had been invited to and won within the past year; and how other buyers rated the performance of each supplier – all drawn from structured transactions and unstructured comments and ratings from other network members.
- Accessing real-time insights into invoice approval status available on a business network, Mediafly, a fast-growing mobile marketing solution provider, has not only been able to view its outstanding invoices and know when they’ll get paid, but make offers to secure early payment. Mediafly CFO John Evarts says such network-based ‘dynamic discounting’ has helped him manage the business differently: “We can now get access to capital at favorable rates when we need it, allowing us to hire new developers so we can take on new projects and grow the business.” Evarts says the transparency and control afforded by the network has even helped Mediafly put off the need to take a new round of funding.
Cases like this make clear that the so-called “big data” served up by networks will inherently change the way and speed at which business gets done. Armed with the right tools, companies can get the right information to the right people in real time on any device, fueling better business decisions and results.
Demand for such real-time insights is at an all-time high. A growing number of business executives are fed up with lagging indicators and feel the lack of access to timely information is negatively impacting their personal decisions and the performance of their business. Case in point: 35% of the companies surveyed as part of the Aberdeen study say data is too slow to access. 47% indicated that they need information within an hour of a business event, but that they achieve this goal just 71% of the time.
To fill this void, a new breed of technology has emerged. Known as in-memory computing, it increases processing speed by 100% or greater by loading data directly into Random Access Memory of a server. When combined with a mobile platform, it enables companies to serve up the right information to the right people in the right places in real time.
It’s a powerful proposition. And companies that embrace it will be able to make the most of their big data by bringing structure to the unstructured and gleaning insights that enable them to deliver game-changing gains in collaboration, productivity and insight that vault and keep them ahead of the competition.
Yes, big data is getting bigger. But so too are the possibilities for businesses that effectively harness it.