Big data demand soars as companies become data driven businesses

Big data continues to be big business with IDC forecasting strong growth across big-data related infrastructure, software and services.

The analyst firm is predicting the big data technology and services market to grow at a compound annual growth rate of 23.1% over the 2014-2019 forecast period with annual spending reaching $48.6 billion in 2019.

All three major big data submarkets – infrastructure, software and services – are expected to see solid growth over the next five years.

Infrastructure, which consists of computing, networking, storage infrastructure and other data center infrastructure-like security – will grow at a 21.7% CAGR.

Software, which consists of information management, discovery and analytics, and applications software – will grow at a CAGR of 26.2% and services, which includes professional and support services for infrastructure and software, will grow at a CAGR of 22.7%.

IDC says infrastructure spending will account for roughly one half of all spending throughout the forecast period.

As big data matures, IDC expects its share of the larger business analytics market to increase.

However, year-over-year growth within the big data market is forecast to gradually slow. The reduction in overall growth will be largely caused by increased price pressures for infrastructure and higher rates of commercialisation of open source software. The availability and skill level of big data IT and analytics talent will also have a direct impact on the market, IDC says.

Driving the growth over the next five years, is the acceleration by businesses in their transformation into data-driven companies, across a range of industries.

IDC says while the business drivers, barriers and potential benefits from deploying big data initiatives vary between industries, the efforts are primarly focused on delivering a better customer experience, supporting product and service innovation and optimising business processes.

The company says common barriers include security and privacy concerns and the related challenges of collecting, using and managing customers' personal data. A shifting regulatory environment is also an issue.

Jessica Goepfert, program director for IDC's global technology and industry research organisation, says the ability to leverage big data and analytics to develop an integrated view of customer activities and business operations will provide competitive differentiation to companies across industries.

"However, in addition to the huge opportunities, big data presents some significant risks and liabilities to organisations," Goepfert says.

"Line of business and IT executives will need to approach these ongoing challenges with awareness, flexibility, adaptability, and responsibility."

IDC says the largest industries for big data spending include discrete manufacturing ($2.1 billion in 2014), banking ($1.8 billion in 2014) and process manufacturing ($1.5 billion in 2014). The industries with the fastest growth rates include securities and investment services (26% CAGR), banking (26% CAGR), and media (25% CAGR).