Anodot Provides Anomaly Detection and Operational Intelligence

A new company has emerged in the market for real-time analytics software. Anodot came out of stealth mode in late 2015 with $3 million in funding. It is led by three founders: CEO David Drai, whose company Cotendo was acquired by networking company Akamai Technologies in 2012; Ira Cohen, chief data scientist, who previously held that position at Hewlett-Packard; and Shay Lang, who serves as VP of R&D. Unlike most vendors in the space, the company is delivering anomaly detection and operational intelligence through software as a service (SaaS).

Anodot’s technology uses pattern recognition to find anomalous data patterns and send alerts when something is out of the norm. Its customers currently are in three markets: the Internet of Things (IoT), Advertising Technology (AdTech) and e-commerce. We associate real-time analytics with operational intelligence, but increasingly it overlaps into other categories such as predictive analytics and machine learning. According to our predictive analytics benchmark research, only 7 percent of companies constantly update their predictive models, but automation of analytic model building, also known as machine learning, is a key consideration in pattern recognition systems required for operational intelligence and IoT systems.

In the world of connected devices called IoT, one Anodot customer, a printing services company, is using the technology to look at printer failure before it happens by understanding anomalous patterns in usage, temperature, engine speed, voltage, error patterns and the like. This saves money by proactively servicing the printers (potentially remotely), limiting on-site visits from technicians and streamlining inventory management. Furthermore, since the service business is a recurring revenue model, it is able to meet SLAs and eliminate downtime, both of which could impact the revenue line.

For AdTech, one customer utilizes Anodot as part of the marketing optimization engine for its digital analytics and advertising platform. One of the challenges with large digital platforms is the enormous amount of real-time data funneling through from different platforms; that data must be analyzed in real time to produce appropriate recommendations. If the data fed through the system is bad, the recommendations will also be bad. Anodot helps address these data integrity issues by monitoring multiple streams of data and alerting of anomalous patterns that can represent bad data flowing through the system.

E-commerce is the third area where Anodot has multiple customers. One of them processes a half a billion events per day. E-commerce and Web-based businesses, similar to AdTech, need to analyze vast amounts of data in real time to keep sites running effectively and users satisfied. Furthermore, large e-commerce retailers can have many products that have to be matched to pricing without errors or the company can potentially lose significant money. With real-time error detection, such issues can be identified and system rules devised to prevent them.

Anodot’s product has some capabilities that should benefit it in the market. Its patented machine-learning algorithms enable the system to look at different types such as sparse data, irregular data, smooth data, multimodel data and discrete data, and apply the correct algorithm for the specific pattern. This is important because different categories of data exhibit different characteristics, and the correct algorithm must be applied to produce effective results. Based on these algorithms, the company provides thresholds for alerts that change as the data changes. When a threshold, or variable parameter, is reached, an alert can be sent for a human or machine to follow up. This approach is in contrast to the static thresholding that has been used for alerting. With static thresholds a calculation is made and variable parameters are set in advance. Such an approach may delay alerts or, worse, miss anomalies altogether.

A second interesting characteristic of Anodot is that it is offered as software-as-a-service. Among its advantages is not needing to store the underlying data but only the patterns of the data. Storing of data patterns enables the system to identify longer-term patterns, adjust algorithms and refine its accuracy in different scenarios over time.

Since the algorithms are prebuilt, users of the application do not need a data scientist to build models; business intelligence users or application developers can do that. This helps address a major demand in software evaluation, according to both our operational intelligence and predictive analytics research: usability.

SaaS also relieves companies of maintaining their own hardware and software for anomaly detection; they can simply rent the application and save on licensing and administration. Anodot prices usage of the product on a monthly basis based on the number of metrics tracked, which typically ranges from $3,000 to $10,000 per month. In the past event-based systems have been expensive due to the necessities of installing them on-premises and training users. Anodot’s pricing model helps address the two most common barriers to the business case for operational intelligence: lack of resources, mentioned by 41 percent of research participants, and lack of budget, mentioned by 40 percent.

The platform’s cloud architecture enables it to stream events in a variety of time-series metric formats such as Graphite, StatsD, Coda Hale Metrics, and collectd. It also can connect to various applications and data sources using RESTful APIs and JSON. The ability to connect to many sources is a key consideration for such systems. According to our operational intelligence research, the most common barriers to implementation are incomplete event data sources (for 39% of organizations) and too many event data sources (35%).

We recommend that companies in AdTech, e-commerce and IoT, consider Anodot for some of the uses outlined above. OEMs and analytic service providers also may want to evaluate the system in the areas those mentioned above as well as in security and fraud prevention. Given its cloud-based architecture, Anodot fits well within developer operations (DevOps) frameworks and embedded analytics scenarios. To test the system, the vendor offers a free one-month trial.

Regards,

Ventana Research

Pentaho Poised for Exploiting Internet of Things

PentahoWorld 2015, Pentaho’s second annual user conference, held in mid-October, centered on the general availability of release 6.0 of its data integration and analytics platform and its acquisition by Hitachi Data Systems (HDS) earlier this year. Company spokespeople detailed the development of the product in relation to the roadmap laid out in 2014 and outlined plans for its integration with those of HDS and its parent Hitachi. They also discussed Pentaho’s and HDS’s shared intentions regarding the Internet of Things (IoT), particularly in telecommunications, healthcare, public infrastructure and IT analytics.

Pentaho competes on the basis of what it calls a “streamlined data refinery” that enables a flexible way to access, transform and integrate data and embed and present analytic data sets in usable formats without writing new code. In addition, it integrates a visual analytic workflow interface with a business intelligence front end including customization extensions; this is a differentiator for the company since much of the self-serve analytics market in which it competes is still dominated by separate point products.

Pentaho 6 aims to provide manageable and scalable self-service analytics. A key advance in the new version is what Pentaho calls “virtualized data sets” that logically aggregate multiple data sets according to transformations and integration specified by the Pentaho Data Integration (PDI) analytic workflow interface. This virtual approach allows the physical processing to be executed close to the data in various systems such as Hadoop or an RDBMS, which relieves users of the burden of having to continually move data back and forth between the query and the response systems. In this way, logical data sets can be served up for consumption in Pentaho Analytics as well as other front-end interfaces in a timely and flexible manner.

One challenge that emerges when accessing multiple integrated and transformed data sets is data lineage. Tracking its lineage is important to establish trust in the data among users by enabling them to ascertain the origin of data prior to transformation and integration. This is particularly useful in regulated industries that may need access to and tracking of source data to prove compliance. This becomes even more complicated with events and completely sourcing them along with the large number of them as found in over a third of organizations in our operational intelligence benchmark research that examined operational centric analytics and business intelligence.

Similarly, Pentaho 6 uses Simple Network Management Protocol (SNMP) to deliver application programming interface (API) extensions so that third-party tools can help provide governance lower in the system stack to further enable reliability of data. Our benchmark research consistently shows that manageability of systems is important for user organizations and in particular for big data environments.

The flexibility introduced with virtual tables and improvements in Pentaho 6.0 around in-line modeling (a concept I discussed after last year’s event are two critical means to building self-service analytic environments. Marrying various data systems with different data models, sometimes referred to as big data integration, has proven to be a difficult challenge in such environments. Pentaho’s continued focus on big data integration and providing an integration backbone to the many business intelligence tools (in addition to its own) are potential competitive differentiators for the company. While analysts and users prefer integrated tool sets, today’s fragmented analytics market is increasingly dominated by separate tools that prepare data and surface data for consumption. Front-end tools alone cannot automate the big data integration process, which Pentaho PDI can do. Our research into big data integration shows the importance of eliminating manual tasks in this process: 78 percent of companies said it is important or very important to automate their big data integration processes. Pentaho’s ability to integrate with multiple visual analytics tools is important for the company, especially in light of the HDS accounts, which likely have a variety of front-end tools. In addition, the ability to provide an integrated front end can be attractive to independent software vendors, analytics services providers and certain end-user organizations that would like to embed both integration and visualization without having to license multiple products.

Going forward, Pentaho is focused on joint opportunities with HDS such as the emerging Internet of Things. Pentaho cites established industrial customers such as Halliburton, Intelligent Mechatonic Systems and Kirchoff Datensysteme Software as reference accounts for IoT. In addition, a conference participant from Caterpillar Marine Asset Intelligence shared how it embeds Pentaho to help analyze and predict equipment failure on maritime equipment. Pentaho’s ability to integrate and analyze multiple data sources is key to delivering business value in each of these environments, but the company also possesses a little-known asset in the Weka machine learning library, which is an integrated part of the product suite. Our research on next-generation predictive analytics finds that Weka is used by 5 percent of organizations, and many of the companies that use it are large or very large, which is Pentaho’s target market. Given the importance of machine learning in the IoT category, it will be interesting to see how Pentaho leverages this asset.

Also at the conference, an HDS spokesperson discussed its target markets for IoT or what the company calls “social innovation.” These markets include telecommunications, healthcare, public infrastructure and IT analytics and reflect HDS’s customer base and the core businesses of its parent company Hitachi. Pentaho Data Integration is currently embedded within major customer environments such as Caterpillar, CERN, FINRA, Halliburton, NASDAQ, Sears and Staples, but not all of these companies fit directly into the IoT segments HDS outlined. While Hitachi’s core businesses provide a fertile ground in which grow its business, Pentaho will need to develop integration with the large industrial control systems already in place in those organizations.

The integration of Pentaho into HDS is a key priority. The 2,000-strong global sales force of HDS is now incented to sell Pentaho, and it will be important for the reps to include it as they discuss their accounts’ needs. While Pentaho’s portfolio can potentially broaden sales opportunities for HDS, big data software is a more consultative sale than the price-driven hardware and systems that the sales force may be used to. Furthermore, the buying centers, which are shifting from IT to lines of business, can be significantly different based on the type of organization and their objectives. To address this will require significant training within the HDS sales force and with partner consulting channels. The joint sales efforts will be well served by emphasizing the “big data blueprints” developed by Pentaho over the last couple of years and developing of new ones that speak to IoT and the combined capabilities of the two companies.

HDS says it will begin to embed Pentaho into its product portfolio but has promised to leave Pentaho’s roadmap intact. This is important because Pentaho has done a good job of listening to its customers and addressing the complexities that exist in big data and open source environments. As the next chapter unfolds, I will be looking at how the company integrates its platform with the HDS portfolio and expands it to deal with the complexities of IoT, which we will be investigating in upcoming benchmark research study.

For organizations that need to use large-scale integrated data sets, Pentaho provides one of the most flexible yet mature tools in the market, and they should consider it. The analytics tool provides an integrated and embeddable front end that should be of particular interest to analytics services providers and independent software vendors seeking to make information management and data analytics core capabilities. For existing HDS customers, the Pentaho portfolio will open conversations in new areas of those organizations and potentially add considerable value within accounts.

Regards,

Ventana Research