From big data to data lakes
Westcon Imagine 2014 - Research shows the more data-driven an organisation becomes, the greater its success.
Gartner states that organisations integrating new, high-value and diverse information types and sources into a coherent information management infrastructure will outperform their industry peers financially by more than 20%. Additionally a SAP survey revealed that 90% of organisations that employ predictive analytic software agreed it had given them a competitive advantage.
“However, many companies are struggling to handle, process and interpret the overwhelming amount of data – both existing and new – in a timely and effective fashion,” Arron Patterson, Chief Technology Officer, EMC New Zealand, says.
“The opportunity is to redefine traditional data architectures characterised by rigid, difficult-to-scale, proprietary configurations to a modern ‘data lake’, characterised by agile, scale-out, open architectures via on-premises or cloud delivery models.”
Patterson defines a data lake as ‘a platform that provides for storage, management and analysis of Big Data by bringing together real time, near real time or batch data from disparate sources across multiple protocols’. He says a key requirement for data lakes are comparatively high levels of processing performance with simple, cost-effective and secure management of massive volumes of data.
He says leveraging big data effectively opens up numerous exciting opportunities for customers, including gaining deeper customer insights for optimised targeting, decreasing operational costs through predictive/forecasting based applications and gaining more accurate insight in detecting fraud and assessing risk. It also opens the doors to introducing new products and services, monetising information, changing industry dynamics or making positive societal impact.
But in order to fully capitalise on big data, organisations need to move from siloed IT infrastructure to a data lake where data is immediately analysed and value created - then stored if required.
“Many companies start a few big data projects and gain incremental value, but the keys to success are to maximise opportunities through the transformation of people, processes and technology from your current situation to a data driven enterprise. The path to become data driven starts with a business case, or alternatively a business driven process to use big data as a means to meet clear, tangible goals. The more agile you are in capturing big data, deriving insight from it and creating or updating applications to reflect newer insights, the more data driven you will become. This allows you to grow your business through rapid iteration with the ability to quickly add new features or application logicbased on analytics.”
Patterson says the first step to becoming a data driven enterprise is a well trained and experienced staff who can collaborate with the business and IT to uncover an optimal use case for business value.
“Often you can get a big data project started by focusing on a key business process that is already supported by your existing business intelligence and data warehouse environment, and leverage big data to take that business process to the next level. Create a reproducible methodology that’s right for you where the business and IT work together to ask the right questions to uncover big data use cases. Organisations that have started this journey recognise that its not enough to just collect big data in Hadoop, but to continuously ask the right questions to solve business problems – that’s the real value.”
The EMC Federation
The EMC Federation offers a low-risk, best-of-breed approach to create a Big Data strategy that addresses the technology, people, and processes required to successfully complete a Big Data project, optimise IT infrastructures, and become a data-driven organisation. With minimal up-front costs and by leveraging existing IT investments, the EMC Federation ensures organisations identify the right Big Data use cases for fast return on investment, backed by an efficient Data Lake solution that significantly lowers the economics of managing data and adapts to diverse processing and storage demands. In order to maximise value across the organisation, the EMC Federation offers agile application development and Platform-as-a-Service hosting to quickly create new applications that operationalise insight.
For more information visit www.emc.com/campaign/bigdata/index.htm
