Operational Intelligence, Getting from Data to Insights
Businesses must become smarter, more intelligent. In order to keep up with the pace of competition, they need to be able to have full insight and visibility into its business operations to deliver better results. With a sharp eye on managing overhead costs, companies have turned to technology to unlock cost savings and improve efficiency within their organisation. Increasingly, they require technology to be fast, immediate, and easy to use.
An organization’s data is its definitive source of intelligence. It is a categorical record of the organisation’s activities, including user transactions, customer behaviour, machine behaviour, security threats and fraudulent activity.
With over 10,000 customers around the world, US-based Splunk is an operational intelligence software company providing technology that searches, monitors, digests and analyses real time and and machine-generated Big Data. It captures and indexes the data in real-time and uses it to create alerts, charts, graphs, dashboards and visualisation. These are then combined and condensed into a searchable form that is readily available and easy to understand. Ultimately, Splunk enables users to develop valuable insights into innovating new services, as well as analysing trends and customer behaviours.
Key Factors Affecting Splunk’s Performance
One key advantage for Splunk is the need to produce output on-demand, which is critical to generate and display useful insights in real-time. If a Splunk implementation suffers from low performance, then the information it generates may be outdated or provided too late to deliver value. After all, receiving delayed alerts are typically of little use to you.
One key factor that may affect Splunk’s performance is the number of concurrent searches Splunk is running at any one time. Each search consumes CPU and memory and can significantly affect Splunk’s performance with heavy use.
Another factor is the effect of indexing on storage. Data is indexed as it is processed; the more data is ingested, the larger the index files become, leaving less space on disk. This in turn slows down write speeds due to additional time taken to search for storage space for the data. Having a flexible, expandable high-performance storage system ensures that Splunk’s performance is not impacted as indexes grow.
Although Splunk can run on commodity x86 servers, the growing volume of data use puts increasing pressure on storage. However, configuring and building of what Splunk refers to as “Capacity Buckets” can help tune its performance. With all the data tiers in a single Shared-Nothing DAS (Direct Attached Storage), network latency is reduced as the buckets roll from one data tier to the next.
Splunk Enterprise Deserves an Enterprise Storage Infrastructure
When the use of Splunk for operational intelligence grows from a pilot program to full deployment, its operational integrity becomes critical. Splunk performs best with a storage infrastructure that promotes optimal and consistent performance with minimal maintenance and expense.
NetApp is a specialist in data management and has built reference architectures for Splunk. Through this collaboration, NetApp’s Splunk Enterprise storage solution has demonstrated consistently faster static and streaming searches against similar commodity server architectures with internal disks drives. Purpose-built scalable high-performing disk arrays are a critical consideration over DAS for sustained performance in a growing Splunk environment.
Superior Performance
For many operations, the most compelling reason to power your Splunk environment with NetApp storage is the sheer performance advantage that you can realize versus the performance of commodity servers with internal storage.
Recent testing that closely simulated real-world Splunk search performance showed conclusively that operations have much to gain from NetApp’s storage approach. Searches were significantly faster with NetApp’s storage solutions (on average 111% faster) when compared to commodity servers with internal storage. Dense Searching was up to 22% faster and Rare Search performance was up to 200% faster on average.
The partnership between NetApp and Splunk includes the development of apps for the NetApp storage platform portfolio, as well as add-ons to complement the reference architectures to monitor NetApp Storage: the SANtricity Performance App and the Technology Add-on for SANtricity. This synergy allows product solutions from Splunk and NetApp to work fluidly together and facilitate better use of resources, thereby supporting a more secure overall operation for businesses.
It is no doubt that machine-generated big data is the secret for success in IT, security and business operations. The need to accumulate, maintain, and coordinate the massive amounts of data that organizations around the world are continuously producing has never been greater. However, to drive real value from these data, business must look towards solutions that would help deliver the required business value, while also taking into account important operational considerations such as performance, reliability, cost and convenience which Splunk and NetApp provides.
To be clear and understand more on how NetApp can help you with Splunk Performance, click here!
