Digital Technology Agenda for Business in 2016


Technology innovation is accelerating faster than companies can keep up with. Many feel pressure to adopt new strategies that technology makes possible and find the resources required for necessary investments. In 2015 our research and analysis revealed many organizations upgrading key business applications to operate in the cloud and some enabling access to information for employees through mobile devices. Despite these steps, we find significant levels of digital disruption impacting every line of business. In our series of research agendas for 2016 we outline the areas of technology that organizations need to understand if they hope to optimize their business processes and empower their employees to handle tasks and make decisions effectively. Every industry, line of business and IT department will need to be aware of how new technology can provide opportunities to get ahead of, or at least keep up with, their competitors and focus on achieving the most effective outcomes.

Let’s review the digital disruptions that are impacting businesses of every size in any industry.

Analytics is at the top of the list. It has become indispensable not just for measuring performance and efficiency but also for guiding effective actions that make critical differences in an organization. Once just an ad hoc part of business intelligence efforts, analytics now can have a continuous role in Untitledstreamlining business processes. Historical analysis – measuring the past to inform the present – is no longer sufficient; looking forward with predictive analytics can help organizations anticipate future behavior and outcomes. Our benchmark research on predictive analytics shows that nearly half (49%) of organizations expect to gain significant impact from utilizing it, and another one-third (32%) said it can have transformational impact.

However, to develop continuous analytics organizations first must prepare for use the data to be analyzed. Typically this requires significant amounts of time from analysts, data and vr_DAC_20_justification_for_data_preparationoperations professionals – time that could be used more productively. Today they can regain that time by using data preparation tools designed for this purpose. In 2016 we will perform in-depth market research on data preparation to assess the variety of ways in which it is used and where it can offer the greatest benefit to analytics and operations. Our research in 2015 found that preparing data for analysis is the most common impediment in the analytics process for more than half (55%) of organizations, as it has been for the past five years. We also will conduct and publish new research on the role of analytics in the sales, finance and human resources functions. In these and other lines of business we assert that organizations must develop competencies in analytics and begin using them continuously to improve their performance and competitiveness. Look at your own organization to determine if you have made analytics a priority and is being used effectively.

Another precursor to continuous analytics is collecting and processing what is commonly called big data, the huge volumes and broad variety of data that organizations encounter. Advances in computing technology including in-memory processing and storage of big data are now cost-effective and can be readily accessed and used through cloud computing. Because not all data is the same, ranging from structured data to unstructured content, documents and text, how businesses vr_DAC_07_importance_of_external_data_sourcesmanage their information assets is just as important as the guidance they receive from analyzing them. To further investigate the impact of big data on business, we will perform new benchmark research to determine where investment can have the greatest impact in terms of value and time savings. Managing data effectively enables organizations to optimize their information, and there are other sources of data that can add to what they know. My colleague Robert Kugel has named this “cryptic data”; typically it is out of reach of business users, tucked away on the Internet and in external sources, but accessing it could enrich the value of existing information and analytics. Last year our data and analytics in the cloud benchmark research found that Internet information sources are important to 42 percent of organizations.

It is important to remember that big data is not just about data management but also about how it is interconnected and used for business purposes. Industry jargon that isolates it in “data lakes” and other ridiculous terms do a disservice to its full potential for analytics and any range of applications, extending even to advances in the Internet of Things (IoT), which connects whole networks and myriad devices to each other.   Beyond this data science has intersected with expert systems to produce cognitive computing systems that can learn from past and present decisions and interactions to answer questions in natural language and guide decisions to optimal results.

In the excitement over big data and analytics it’s easy to forget that they are useful only when people work with them, and businesses rely on their people to interact and collaborate to reach agreement or better understand opportunities and situations to be resolved. Through a new generation of digital technologies, workers and managers alike can engage in discussions interactively online, through videoconferences that can share applications and presentations and with mobile technologies that make it simpler to collaborate at any time from any place. Our next-generation learning management benchmark research found that social collaboration is critical for more than half of organizations to share learning socially through activity streams. Technology enables even digital “town hall” meetings in which workers anywhere on the planet join in interactive scdiscussions. But collaborative technologies must be used in context of business processes that rely on business applications in which information must be shared, assessed and acted upon to achieve specific goals. Thus the idea of embedding collaboration in business application is taking hold among large application providers, although some just make it available separately. Our research in the past several years has identified collaboration as one of the most widely recognized digital technologies to advance business processes; for example, more than one-third (38%) of participants in our data and analytics in the cloud research are using it, although fewer than that (30%) are satisfied with how they collaborate, which is not surprising when many are still using email as the primary mode of collaboration. The good news is that new methods are gaining traction: Almost half (47%) are planning to use or are evaluating discussion forums, and nearly as many (48%) are interested in wall posting. New research we’ll produce in 2016 will identify the further adoption of collaboration and best practices in contact centers, sales, human resources and finance groups. Furthermore, to better engage workers in the organization, a new generation of digital feedback techniques used in consumer applications for easy-to-rate feedback is migrating into business. In general collaboration using digital techniques is still one of the most underutilized methods in organizations, but it can have large returns on investment since it engages and should motivate workforces to interact with others and management.

Another major new digital technology that has reshaped the way organizations use information is cloud computing, which enables applications or services to operate beyond an enterprise’s own premises. It can help organizations simplify access to and use of software by removing barriers of resources and vr_DAC_04_widespread_use_of_cloud_based_analyticsskills, allowing any size of organization to exceed its previous computing capabilities. Simplifying the ability to onboard a range of software whether business applications or other tools and to manage them easily in any area of business in conjunction with IT policies provides a radically faster time to value. We have also seen this in the use of analytics, as almost half of organizations in our data and analytics in the cloud research already use cloud-based analytics in some manner and another one-fifth (19%) will use it in the next year. Now organizations are shifting to integrating business applications in the cloud and in the enterprise, a process that requires integration software designed to help streamline interoperability. Underlying this transition of business computing is a movement toward the platform as a service (PaaS) and messaging that interconnects business and consumers in a range of cloud environments – public, private and hybrid. The enterprise architecture of the future is centered in the cloud; much of the software industry has shifted to this approach, and business organizations will be required to adapt or be left using and managing their own software. It is only a matter of time until they will not have a choice as new applications are rapidly becoming available only in the cloud.

Until recently many businesses have worried about the security of systems they don’t deploy and control themselves. Our data and analytics in the cloud benchmark research vr_DAC_13_impediments_to_deploying_cloud_based_analyticsshows that lack of confidence in security is still the most frequent impediment to deployment cloud-based analytics, in more than half (56%) of organizations. Arising from these worries is new digital technology designed to ensure cybersecurity and protect intellectual assets (systems, internal data and customer information) from being hacked and compromised. More than a few large-scale incidents have shown that such attacks can significantly impact not only financial profitability but an organization’s credibility. Alert organizations now realize that just protecting the network that connects their computers and systems is insufficient to ensure that the full range of threats is mitigated. For example, most organizations have not effectively inventoried and assessed their IT assets to identify outdated software that might have known cyber exposures that can create wormholes that work from inside the organization to the outside. Building on IT asset management is the ability to identify legacy systems that increase threats and put data at risk in databases or from systems and tools that access them from more than one location. Such vigilance requires a sophisticated set of technology that not only detects and responds to threats but can recommend and even act on cyber exposures before situations reach crisis levels. The data within databases and analytics also needs to be secured. This challenge will require a new generation of cyberintelligence that is managed directly by the CIO’s office and understood by business management.

As if all this was not complicated enough, now we have the Internet of Things (IoT) emerging. Devices, machines and networks that are interconnected to the Internet through sensors and messaging are no longer just for monitoring but also for interactive dialogues that notify and take action on threats or malfunctions. As we evolve to this technologically sophisticated world, even things we wear, from watches to certain types of clothing, also can provide information on business and personal activities that range from responding to requests to the wellness of individuals. The underlying connectivity comes from the use of Bluetooth and RFID for cellular or WiFi connections directly onto the Internet. As we find ways to miniaturize and embed sensors and related technology that can provide data, we also find that the processing is operating at the edge of the network and within machines, even automobiles. These Internet-level bots do not just operate at the edge of the network but can also transport themselves to where processing needs to happen. IoT will require applications that can monitor systems and also be used to manage monetization as in subscription to services and interact across any range of services. Such a change will require advanced skills in IoT analytics and capabilities for real-time processing; we call this the next generation of operational intelligence and are conducting new market research to determine the rate of innovation and emerging best practices in adoption of the technologies.

As we all can see, smartphones and tablets are vr_DAC_17_mobile_access_to_cloud_based_analyticseverywhere, connecting people and the Internet. The potential for businesses is enormous, and it will be a necessity for them to equip and support their workers and managers with applications that can easily operate on these devices. Unfortunately so far many business software applications and tools provide only lip service to using their capabilities; few of these vendors have a “mobile first” approach to supporting workforce effectiveness. Working across devices from Apple or Android has plenty of nuances, and many applications require a lot of “pinching” to interact with them rather dynamically sizing in response to the device on which it operates. Additionally, a new generation of notebooks that operate through touch screens and tablets that use Microsoft Windows is emerging. Giving ineffective software to mobile-enabled workers can lead to employee dissatisfaction and become a factor in why they leave an organization. Ensuring that mobile apps provide a contemporary user experience and easy usability is more important than just the app’s capabilities; don’t listen to analyst firms that rate them on the number of customers or amount of revenue they have generated. Such recommendations have led many organizations to select the wrong software and weaken themselves for years to come. With new research in 2016 we will continue our decade-long analysis of the mobile revolution and its impact on business; we advise that embracing mobile-ready applications is essential to maximize the value of the workforce.

Mobile technology advances have paved the way for a new generation of wearable devices, most evidently the new kind of watch, which is now ready for businesses to use to consume and act on information and make decisions. Wearables can support business productivity by increasing the responsiveness of individuals in any role. A new generation of smart watches that are easier for technology providers to integrate with business applications is available and will begin to establish new workflow and interactivity capabilities. Our upcoming research into the new generation of human resources management systems (HRMSs) and into workforce management will assess the demand for these applications. This generation of wearables will come with location information that can be used to promote situational awareness and be optimized for a variety of uses. For many organizations and workers, using wearables provides immediate visibility on the wellness of individuals that not just helps the individual maintain personal health but helps organizations ensure that workers are able to conduct their job responsibilities in ways that minimize risk and ensure safety.

As you see, this will be a big year for technology and potentially just as big a one for business in learning to take advantage of these advances. We have put together a formalized set of research agendas covering all of these areas for more depth on our direction in 2016. Please rely on Ventana Research to help guide you in understanding the challenges and making the decisions that will serve your organization best.

Regards,

Mark Smith

CEO and Chief Research Officer

Cryptic Data: Challenges and Rewards in Finding and Using It


Using information technology to make data useful is as old as the Information Age. The difference today is that the volume and variety of  available data has grown enormously. Big data gets almost all of the attention, but there’s also cryptic data. Both are difficult to harness using basic tools and require new technology to help organizations glean actionable information from the large and chaotic mass of data. “Big data” refers to extremely large data sets that may be analyzed computationally to reveal patterns, trends and associations, especially those related to human behavior and interaction. The challenges in dealing with big data include having the computational power that can scale to the processing requirements for the volumes involved; analytical tools to work with the large data sets; and governance necessary to manage the large data sets to ensure that the results of the analysis are accurate and meaningful. But that’s not all organizations have to deal with now. I’ve coined the term “cryptic data” to focus on a different, less well known sort of data challenge that many companies and individuals face.

Cryptic data sets aren’t easy to find or aren’t easily accessed by people who could make use of them. Why “cryptic?” As a scuba diver, I donate time to Reef Check by doing scientific species counts in and around Monterey Bay, Calif. Cryptic organisms are ones that hide out deep in the cracks and crevices of our rocky reefs. Finding and counting them accurately is time-consuming and requires skill. Similarly, it’s difficult to locate, access and collect cryptic data routinely. Because it’s difficult to locate or access routinely, those who have it can gain a competitive advantage over those who don’t. The main reason cryptic data is largely untapped is cost vs. benefits: The time, effort, money and other resources required to manually retrieve it and get it into usable form may be greater than the value of having that information.

By automating the process of routinely collecting information and transforming it into a usable form and format, technology can expand the range of data available by lowering the cost side of the equation. So far, most tools, such as Web crawlers, have been designed to be used by IT professionals. Data integration software, also mainly used by IT departments, helps transform the data collected into a form and format where it can be used by analysts to create mashups or build data tables for analysis to support operational processes. Data integration tools mainly work with internal, structured data and a majority have little or no capability to support data acquisition in the Web. Tools designed for IT professionals are a constraint in making better use of cryptic data because business users are subject matter experts. They have a better idea of the information they need and are in a better position to understand the subtleties and ambiguities in the information they collect. To address this constraint, Web scraping tools (what I call “data drones”) have appeared that are designed for business users. They use a more visual user interface design and hide some of the complexity inherent in the process. They can automate the process of collecting cryptic data and expand the scope and depth of data used for analysis, alerting and decision support.

Cryptic data can be valuable because when collected, aggregated and analyzed, it provides companies and individuals with information and insight that were unavailable. This is particularly true of data sets gathered over time from a source or combination of sources that can reveal trends and relationships that otherwise would be difficult to spot.

Cryptic data can exist within a company’s firewall (typically held in desktop spreadsheets or other files maintained by an individual as well as in “dark” operational data sets), but usually it is somewhere in the Internet cloud. For example, it may be

  • Industry data collected by some group that is only available to members
  • A composite list of products from gathered from competitors’ websites
  • Data contained in footnotes in financial filings that are not collected in tabular form by data aggregators
  • Tables of related data assembled through repetitive queries of a free or paid data source (such as patents, real estate ownership or uniform commercial code filings).

Along these lines, our next-generation finance analytics benchmark research shows that companies have limited access to information about markets, industries and- economies. vr_NG_Finance_Analytics_17_accessibility_of_external_dataOnly 14 percent of participants said they have access all the external data they need. Most (63%) said they can access only some of it, and another 14 percent said they can’t access any such data. In the past, this lack of access was even more common, but the Internet changed that. And this type of external data is worth going after, as it can help organizations build better models, perform deeper analysis or do better in assessing performance, forecasting or gauging threats and opportunities.

Cryptic data poses a different set of challenges than big data. Making big data usable requires the ability to manage large volumes of data. This includes processing large volumes, transforming data sets into usable forms, filtering extraneous data and code data for relevance or reliability, to name some of more common tasks. To be useful big data also requires powerful analytic tools that handle masses of structured and unstructured data and the talent to understand it. By contrast, the challenge of cryptic data lies in identifying and locating useful sources of information and having the ability to collect it efficiently. Both pose difficulties. Whereas making big data useful requires boiling the ocean of data, cryptic data involves collecting samples from widely distributed ponds of data. In the case of cryptic data, automating data collection makes it feasible to assemble a mosaic of data points that improves situational awareness.

Big data typically uses data scientists to tease out meaning from the masses of data (although analytics software vendors have been working on making this process simpler for business users). Cryptic data analysis is built on individual experience and insight. Often, the starting point is a straightforward hypothesis or a question in the mind of a business user. It can stem from the need to periodically access the same pools of data to better understand the current state of markets, competitors, suppliers or customers. Subject matter expertise, an analytical mind and a researcher’s experience are necessary starting capabilities for those analyzing cryptic data. These skills facilitate knowing what data to look for, how to look for it and where to look for it. Although these qualities are essential, they not sufficient. Automating the process of retrieving data from sources in a reliable fashion is a must because, as noted above, the time and expense required to acquire the data manually are greater than its value to the individual or organization.

Almost from the dawn of the Internet, Web robots (or crawlers) have been used to automate the collection of information from Web pages. Search engines, for example, use them to index Internet pages while spammers use them to collect email addresses. These robots are designed and managed by information technology professionals. Automating the process of collecting cryptic data requires software that business people can use. To make accessing cryptic data feasible, they need “data drones” that can be programmed by users with limited training to fetch information from specific Web pages. Tools available from Astera ReportMinerConnotate, Datawatch, import.io, Kofax Kapow and Mozenda are great examples on where you can get started for leveraging cryptic data. I recommend that everyone who has to routinely collect information from Internet sites or from internal data stores that are hard to access or who thinks that they could benefit from using cryptic data investigate tools available for collecting it.

Regards,

Robert Kugel – SVP Research