The data dilemma
Every modern company relies on a wide variety of systems, from a key-card reader to enter the front door, to the application server that the company website runs on. These systems are vital, constantly changing and almost every day they introduce new dynamics that impact the business technology ecosystem.What most executives aren’t thinking about is that each of these systems creates a unique stream of data that contains a definitive record of who’s interacting with their company, what time they stopped by, when and what they’re looking for and any obstacles that prevented them from accomplishing their desired outcome. But all this data lives in different buckets – network logs, clickstream data, firewalls, OS logs, Java garbage collection stats. The list goes on. These disparate systems reside on different servers, sometimes in different geographies and have never traditionally been thought of to provide vital business information.This is the IT data dilemma – making sense of the volumes of data generated by systems, new devices and transient and disparate sources. Today IT faces an increasingly difficult challenge managing all of its systems, much less seeing and understanding the relation of one set of data to another.However, once you’re able to overcome these obstacles, you can begin to gain visibility across an organisation and discover the insights needed to optimise IT and the business. This is what we call operational intelligence – harnessing unstructured, machine-generated IT data so that IT and the business can make faster and more informed decisions. As companies start to discover the value of their IT data, they reveal a wealth of new insights that were previously unattainable.It’s time for a massive shift in how we capture and analyse IT data, and this starts with understanding the nature of the challenge.Harnessing IT Data: The 4 main obstacles1. Exponential growth: We’re creating 1,200 exabytes of data this year, up from 150 exabytes in 2005. New devices, applications, and trends like cloud computing and virtualisation are adding to the data deluge and continue this explosive growth curve. Gartner estimates that 80-90% of this growth is represented by unstructured data. this explosive growth curve. Gartner estimates that 80-90% of this growth is represented by unstructured data.2. Diversity of sources: Every new mobile device, application, wireless connection, browsing session, shopping cart, creates a new stream of data to be captured.3. Lack of a standard interface: Analysing IT data has traditionally meant buying a specialised point tool to import data into a database and analysing it with a reporting solution – this simply isn’t feasible with the diversity and volume of data found in today’s IT infrastructures.4. No historical context: Data is constantly being created and it’s constantly changing. Specific data can unlock a trend or pattern critical to the business; sometimes it’s only revealed when it’s analysed in a historical context. The vast majority of systems that monitor or manage IT data persist the data for a short time before discarding it. Once this happens the ability to apply a historical context, discover complex patterns and identify trends, is lost forever.The traditional approach to managing data – defining a schema and organising analysis around a fixed set of known questions – is failing us. However, technology has started to catch up and we now have the opportunity to capture IT data and create new levels of operational visibility that enable better business decision making.OpportunityFor the channel, the IT data dilemma presents an opportunity to deepen relationships with organisations that struggle with managing and monitoring IT. In addition, it opens a strategic conversation about IT data and how to harness it to provide IT and the business the insight to make better decisions. To do that, channel partners need to understand the elements of successfully utilising IT data.Keys to successfully harnessing IT data:
- Real-time and historical data – Some data springs to life immediately as it streams across a real-time monitoring dashboard. Some data is more obscure and doesn’t reveal its value until it’s put in historical context. A solution needs to capture everything – real-time and historical IT data – to make sure valuable context isn’t lost.
- One solution, many uses – An IT data solution needs to address use cases across IT instead of working within disparate silos. The solution needs to be usable by everyone in IT and flexible enough to provide custom dashboards or views to the rest of the company.
- Compatibility with any IT data source – A solution can’t be limited by the ability to analyse some IT data but not others; operational visibility is about leveraging all the IT data generated across an organisation.
- Scalability – Keeping up with the exponential growth of data means that an IT data solution has to be fast so you can search thousands or billions of events in seconds; and it has to be scalable enough to handle both departmental data volumes and global, distributed deployments with terabytes of IT data a day.
