Data Analytics

Exploiting Analytics in the Cloud Without Moving All Your Data There

Actian Corporation

July 29, 2019

Your company has competing data needs. You need IT systems optimized for real-time transaction processing to serve as the engine for digitally transformed business processes. You also need your data aggregated and optimized for analytics to generate both real-time insights and perform deep data-mining activities. It can be difficult to balance these competing needs in your solution architecture if the only approach you are considering is “find a middle-of-the-road solution that minimally satisfies both needs.”

As one might imagine from the way that approach is phrased, it is a compromise solution, one in which both needs are satisfied during the short term, but no one is happy and the company can’t excel during the long term. The alternative (and recommended) approach is to develop separate, specialized and optimized solutions for each business challenge. Enabling specialization not only leads to better performance but also a path to long-term scalability and business agility.

Your transactional systems must be optimized for real-time processing. They may reside on-premises, in the cloud or be a Software-as-a-Service (SaaS) solution that third parties manage. They each have their own underlying data structure and processing capabilities that have been carefully crafted to provide optimum performance for the business processes that rely on them. Let these systems do the job for which they were designed and don’t slow them to satisfy analytics needs.

Your analytics systems must aggregate, analyze and present data in a consumable format. Digital business processes rely on this data to inform operators and staff what actions must be taken to keep the company running smoothly and to take advantage of business opportunities.

The challenge of effective real-time analytics is putting the data in a common place for analysis, and then having enough processing power to convert raw data into meaningful information and actionable insights in real-time. The cloud provides an ideal solution to support your analytics needs, because of its massively parallel-processing (MPP) capabilities, consumption-based billing models and access to low-cost, shared infrastructure resources.

The question many solution architects and IT leaders ask is “If I want to do analytics in the cloud, then must I move all of my data there?” The answer is “No.” The data you have moved to the cloud should be processed in the cloud. The data you have stored on-premises or in your private cloud should similarly be processed there. Most analytic processing occurs in memory, which means you can simply pull in the data you need from source systems when you are ready to process it.

It is important to co-locate the data and processing as moving masses of data can be slow and expensive. Once the analysis is complete, you can discard the raw data from the analytics system. This approach lets you leverage the cloud-processing power and scale for processing without moving all your data.

The key to an effective cloud-analytics solution is having a set of data-management capabilities that enable you to manage data connections to source systems, tools to aggregate data into a common hybrid data model. Actian provides multi-cloud and on-premises analytics with a robust distributed query capability that lets you put the compute wherever the data lies.

Actian is the leading provider of hybrid data-management solutions to support real-time, cloud-based and on-premises analytics and operational data processing. To learn more, visit www.actian.com/data-platform.

About Actian Corporation

Actian empowers enterprises to confidently manage and govern data at scale. Actian data intelligence solutions help streamline complex data environments and accelerate the delivery of AI-ready data. Designed to be flexible, Actian solutions integrate seamlessly and perform reliably across on-premises, cloud, and hybrid environments. Learn more about Actian, the data division of HCLSoftware, at actian.com.

Actian Life

New Actian India Office

Pamela Fowler

July 26, 2019

We would like to make everyone aware that on July 16, 2019, we announced the launch of our new Actian India facility hosted by HCL. This is very exciting news for the team in India. The team has grown to the point that a new and larger facility was needed. As well, we needed to provide a more robust infrastructure for the team (long overdue).

As a result, we are moving our location to the Cessna Business Park in Bangalore. This new facility follows along the lines of what we did with our new Austin facility and what we want to do with all of our facilities in the future. Also for the India team, this now provides them with all the added benefits of all of our other Actian office locations such as the same hardware (laptops, monitors, etc.) as well as a major technology upgrade which includes:

Much larger Internet Pipe (100Mb).
Same wireless (Actian-Corp, Actian-Guest) as other sites.
Computers joined to Actian domain, primary ids will be user@actian.com.
Local IT Management, under Actian Global IT supervision.
Still have work-at-home ability with VPN.
Will be testing SD-WAN capabilities from Aryaka in September to help solve latency issues.
Meraki advanced networking technology that provides us with significantly enhanced security.

Also as part of this, Durgaprasad Das, who has been leading our operations along with some other companies operations under Indecomm, will be moving into a full-time role working exclusively for Actian leading and building our team.

Wednesday, July 17, 2019 we took the team to tour the new facility and we then all participated in a celebration of our new facility and our 1-year anniversary of Actian’s relationship with HCL.

We are very excited for the team in India and we look forward to the improved productivity and the growth opportunities this will provide.

As well, this continues our efforts towards making the team feel more like ‘Actian’ as we continue to drive on One Actian as a company in support of our future growth plans.

About Pamela Fowler

Pamela Fowler is SVP - Customer Success at Actian, overseeing worldwide support infrastructure. Pamela leads strategic direction and tactical execution for customer support and managed services, ensuring high customer satisfaction. She's a champion of 'voice of the customer,' aligning feedback with Actian's operational roadmap. Pamela's blog articles highlight customer advocacy, support best practices, and service innovation. Browse her posts to learn how robust support can elevate user experiences.

Data Intelligence

The Chief Data Officer’s Evolution to a Data Democracy Sponsor

Actian Corporation

July 22, 2019

Under the pressure of digital transformation, Chief Data Officers (CDO) have appeared within large companies. According to Gartner, 90% of large companies will have a CDO by the end of 2019.

The thousands of CDOs appointed in the course of the past few years were in charge of improving efficiency and capacity to create value for their organization’s information ecosystem. That is to say, they were invited to direct their organization in processing and exploiting information with the same discipline as the other, more traditional, assets.

Companies that valorize their information assets surpass their rivals in using them to reinvent, digitize, or eliminate existing processes or products.

The CDO’s missions can be summarized as exploiting and finding uses for corporate data, as well as being in charge of developing the use of and trust of employees regarding internal enterprise data. As we have seen, these missions often collide with the powerful cultural restraints within organizations.

How Have the Chief Data Officer’s Missions Evolved?

The CDO has many responsibilities. Gartner identified the main responsibilities of a CDO during their 2019 Data & Analytics event in London. These are, among others:

Defining a data and analytics strategy in their organization.
Supervising operational initiatives in response to the established upstream strategy.
Ensuring information made available on the data is trustworthy and valuable.
Constructing data governance.
Creating business value in data analytics.
Managing efforts regarding data science.
Operating and maintaining efforts in infrastructure in response to data analysis needs.

We believe that this impressive list of responsibilities is complemented by another, which could serve as a common thread for all the others and facilitate them: promoting Data Democracy and supporting cultural changes.

At first, CDOs had to lead a mission to convince interest organizations to exploit data. The first few years of this mission were often supported by the construction of a data universe adapted to new uses, often in the form of a Data Lake or Data Mart. The investments agreed upon to construct these data universes were significant but often reserved to specialists. In brief, organizations had more so implemented Data Aristocracies rather than Data Democracies.

The CDO Towards a New Role

With the exponential development of data, the role of the CDO took a new scope. From now on CDOs must reconsider the organization in a cross-functional and globalizing way. They must become the new leaders in Data Democracy within companies and respond to the call of numerous data citizens who have understood that the way in which data is processed must change radically. The new CDOs must break the bonds of data silos.

In order to obtain the support for data initiatives from all employees, they must not only support them in understanding data (original context, production, etc.) but also help them to invest in the production strategy and the exploitation of data.

From now on, the involvement of stakeholders in the exploitation of data must extend to all levels of the enterprise. It is by facilitating understanding, exchanges, and access around data that organizations will become data-driven.

About Actian Corporation

Data Architecture

Managing Data Across Distributed Data Warehouses

Actian Corporation

July 17, 2019

Data integration, like the digital transformation initiatives it supports, is a journey and not a destination. Every company is somewhere on a path from the past to a desired state of future integration they’d like to achieve. If your company has existed for a number of years, then you likely have multiple databases, data marts and data warehouses, developed for independent business functions, that now must be integrated to provide the holistic perspective that digitally transformed business processes require.

You may have the ambition to merge all of your data into a single data warehouse (a potentially multi-year effort); you might also decide to keep your legacy systems; or you could choose to restructure your data in a new way and distribute it across regional data warehouses. There is a commonality to any of these scenarios – you will be managing distributed queries for quite a while (if not indefinitely).

IT leaders are increasingly accepting the necessity of distributed queries, but, simultaneously, are becoming more concerned about performance implications for both operational systems and analytics that must leverage data from distributed warehouses. Cloud-based integration platforms and hybrid data warehouses are providing an answer to some of these challenges.

Why are Distributed Queries Problematic?

Distributed queries that span multiple data warehouses are a performance challenge because of the latency caused by remote joins, iterative operations and large data sets that (in addition to being processed within a database) must also traverse network infrastructure. As analytics become more complex and the underlying data sets increase (both common situations), the query requires more time to process.

This isn’t a big deal for batch-processing operations, but the use cases for integrated data are increasingly focused on providing real-time insights. This may be part of a transactional process an agent is performing (such as checking order history, stocking or compiling a 360-degree view of the customer, etc.) or it may be real-time monitoring and analytics to drive operational decision-making/process-tuning. Either way, end users have a low tolerance for data-processing delays and expect the data to be correct, robust and current (real-time).

How to Address the Distributed Queries Challenge?

IT teams have a few available options to address the performance issues of distributed queries.

Merge distributed data warehouses into a single instance – While this seems like the most efficient solution, it often is not a cost-effective alternative due to legacy infrastructure investments, migration costs and business disruption.
Separate the queries and perform aggregation processing in the application layer – Many small companies have used this approach, relying on either Web services or client applications to combine data from distributed sources. While it avoids the distributed query problems, application infrastructure typically has less processing capacity and speed than database infrastructure and as a result, desired performance gains are rarely achieved.
Add an operational data warehouse aggregation layer to the solution architecture – This approach is proving to be the most effective method to achieve high-performance data processing at enterprise scale. Leave the data in the source systems and replicate the info you need for queries into a specialized data warehouse layer that is purposely built for aggregation and servicing real-time query requests. These systems often can be implemented with minimal modification to application code and can be scaled using cloud services to support even large enterprise data sets.

Distributed data warehouses are likely to be a part of the IT ecosystem of companies for many years. Making informed decisions about how to manage data across these warehouses and support real-time, distributed-query operations is essential to helping your company move from basic digital transformation towards real-time, data-driven decision-making and enterprise business agility.

Actian cloud-based data management platforms can help. In addition to providing a hybrid cloud-based integration platform based on Actian DataConnect, Actian also enables organizations to deploy data warehouses across cloud platforms and on-premises.

About Actian Corporation

Data Intelligence

Understanding the Different Data Cultures

Actian Corporation

July 16, 2019

Just like corporate or organizational culture, each enterprise that deals with data has its own data culture. We believe that what distinguishes Web Giants isn’t the structure of their governance, but the culture that irrigates and animates this organization.

We believe in putting in place a Data Democracy. It refers to corporate culture, an open model where freedom rhymes with responsibility.

To better understand Data Democracy, it is necessary to compare it to other data cultures. Here are the main data cultures:

Data Anarchy

In this system, operational professionals feel poorly served by their IT departments, and each one develops its clandestine base (shadow IT), which serves their immediate interests while freeing them from all control regulations and conformity to standards. In 2019, this culture brings sizable risks: data leaks, contravention of ethical regulations, service quality degradation, reinforcement of silos, etc.

Data Monarchy

This system translates to a very strong asymmetry in data access depending on the hierarchical position. Data, here, is very strictly controlled; its consolidation level is carefully aligned with the organizational structure, and its distribution is very selective.

This monarchical culture prevailed for a long time in Business Intelligence (BI) projects: data collected in data warehouses was carefully controlled, then consolidated in reports where access was reserved to a few select people who were close to decision-making bodies. This method promotes a “top-down” approach and willingly encourages a defensive strategy, where rules, restrictions, and regulations insulate data. Its main theoretical benefit is the almost infallible control over corporate data, but that translates into very limited access to data, only reserved for certain privileged groups.

Data Aristocracy

A Data Aristocracy is characterized by a more significant degree of freedom than in Data Monarchy, but which is solely reserved to a very select subset of the population, mainly expert profiles such as Data Engineers, Data Analysts, Data Scientists, etc. This aristocratic approach is often the one that brings the most successful data governance projects to the surface.

Such a culture can be favorable to more offensive strategies, as well as to heterogeneous one, combining top-down and bottom-up. However, it deprives the majority of employees access to data and thus, a certain number of possible innovations and valorizations.

Data Democracy

Data Democracy’s main objective is to make a company’s data widely accessible to the greatest number of people, if not to all. In practice, every employee is able to pull data values at any level. This freedom of access offers maximum opportunities to create value for the company; it provides each employee with the ability, at their level, to use all accessible and compatible resources within their needs in order to produce locally, and through a trickle effect, it will benefit the entire company.

This freedom only works if the regulations and the basic tools are implemented, and each employee is responsible for how they use their data. Therefore, the distribution of necessary and sufficient information is required to allow employees to make proper use of it while adhering to regulations.

About Actian Corporation

Data Integration

Address Integration Challenges With the Proliferation of IoT Devices

Actian Corporation

July 16, 2019

During the past few years, IoT devices have evolved from being a consumer novelty to a core part of next-generation IT ecosystems for businesses of all sizes. IoT devices (such as smartwatches, fitbits, home sensors, RFID point-of-sale scanners, heartbeat monitors, etc) have become attractive and useful because of their inexpensive price point, ease of setup/administration and their diverse capabilities. IoT devices are everywhere and used in a variety of industries and use cases.

Now that business users are experiencing the value of IoT devices in their personal lives, they are understandably asking for similar capabilities at work. As companies embark on digital transformation journeys, they expect IoT to be one of the key technologies to serve as a source of new operational insights and performance gains. Small, inexpensive devices connected to a company’s network provide real-time telemetry and monitoring of business processes, operations, delivery logistics, facilities issues and much more.

The characteristics that make IoT so attractive to business users is also leading to a rather large headache for IT staff. In the personal environment, smartphone applications or a commercial digital assistant (such as Siri or Alexa) manage the connections to individual IoT devices, unify the interaction experience and aggregate the data from all of a user’s IoT devices.

In the business context, where there can be thousands (or more) IoT devices deployed and operating across the IT ecosystem, consistently managing connectivity and integration can be challenging. One of the biggest areas of concern for IT staff (and business users) is integrating the data from IoT devices into a company’s data ecosystem and providing real-time insights based on that data.

IoT Data Integration and Digital Transformation

Most companies have embarked on some sort of digital-transformation journey during the past few years – seeking to re-invent business processes to leverage modern technology at a deeper level. IoT has been critical to enabling digital transformation by providing a cost-effective set of solutions for monitoring end-to-end business and operations processes and controlling physical infrastructure.

Applications for IoT have included such diverse scenarios as monitoring manufacturing quality, optimizing power consumption in company facilities and tracking the flow of customers through retail stores. As more companies move towards digital business processes, IoT will be even more crucial to enabling centralized monitoring and control, and with real-time process optimization.

Leveraging IoT effectively within a business environment requires a scalable and effective means of collecting, connecting and delivering data to each of the deployed IoT devices. The issue with managing connectivity with IoT devices is the sheer volume of individual devices that are deployed as well as variety of data that is collected . Where many more sophisticated IT infrastructure devices (such as wireless access points) can be loaded with standardized configurations and administration profiles, IoT devices are often simple and lacking the robust and typical administrative features of IT environments. There is also very little API standardization across IoT vendors and as a result, a centralized service independent of the IoT devices must typically perform connection management and data aggregation.

The Value of an Integration Platform for IoT Solutions

In the consumer space, a smartphone or digital assistant may be all that is needed to manage integration across a handful of IoT devices and a few users. Businesses with hundreds or thousands of IoT devices and thousands of employees and the need to leverage them require an enterprise-scale set of integration capabilities to be successful. That is the reason organizations need a hybrid data-integration platform like Actian DataConnect, that supports both batch as well as real-time data integration capabilities required by IoT use cases. Real time data integration is crucial for IoT applications.

Driven by business use cases, IoT solutions may require data processing to be brought close to the edges — to the “things” where data is generated. So you need a data integration platform that integrates data at rest as well as data in motion. So you need an integration platform that really spans from the core of the network to the edge. They may also sometimes require aggregation of data in IoT platforms and other data stores, where it can be analyzed in bulk. New IoT data must be reconciled with traditional enterprise master data, for example. Interacting with and reusing data that is constantly in motion and available in various locations throughout highly distributed IoT infrastructures, and working with data within IoT operational flows, will push integration work to the edges.

Integration platforms built for business scale have been optimized for the central administration of large numbers of data endpoints. With many IoT devices, saving a couple of minutes of admin time on each device can add-up quickly. Add new devices quickly, monitor device connectivity status, orchestrate the flow of IoT data to data warehouses and analytics tools and manage who is using your IoT data in a central place. With Actian DataConnect, you don’t need to worry about IoT proliferation bloating your administrative costs. Instead, you can focus on deploying the IoT capabilities your business needs and be confident of your IT staff’s ability to support them.

About Actian Corporation

Data Intelligence

Data Stewardship and Governance: The Data Steward’s Multiple Facets

Actian Corporation

July 11, 2019

Where Stewardship refers to the taking care of and the supervision of a specific property or organization, Data Stewardship refers to data supervision. Initially, the idea was that a domain expert would be in charge of qualifying and documenting data from their professional standpoint. Data Stewards are those who work closest to where the data is collected; they are often those who best understand the different aspects of data and the standards to which they must adhere.

Data Stewardship and Governance: The Responsibilities

In practice, Data Stewardship covers a wide range of responsibilities, depending on the maturity level of the organization. We can organize these responsibilities in four broad categories:

Operational Supervision and Quality

This refers to monitoring and supervising the complete life cycle of a dataset.

More specifically, Data Stewards must define and implement the necessary processes for the acquisition, storage, and distribution of datasets.

They must also ensure that the data produced fulfills the quality criteria that were defined (values, gaps, completeness, freshness, etc.) and that the procedures are put into place to evaluate and correct potential quality problems.

Documentation

A Data Steward is in charge of defining and documenting data and creating a glossary of industry-specific terms.

They must ensure that each element of a dataset possesses a clear definition and a specific use.

The documentation constitutes a collection of technical and functional metadata according to a meta model in common principle.

Conformity and Risk Management

Data protection and the management of regulatory risks or ethics is one of the most challenging aspects of the Data Steward’s role.

The regulatory environment around data is more restrictive and shifting. It’s up to them to ensure that the proliferation of data is framed by a collection of protocols ensuring conformity with the applicable standards – especially regarding privacy protection.

Security and Access Control

Finally, Data Stewards must define the rules governing data access.

These include the different levels of confidentiality and procedures, allowing the authorization of a person or group to access data.

How Does Data Democracy Strengthen Agile Data Governance?

Orchestrated by a Data Management division, implemented by different types of Data Stewards, data governance must be deployed in an organization. To ensure this deployment, several operational models are conceivable in theory – decentralized, federated, centralized, etc. We think what distinguishes organizations is not the structure of their governance but the underlying culture of this organization. This culture has a name: Data Democracy.

About Actian Corporation

Data Management

Will Data Management in Operational Technology Finally Be Standardized?

Actian Corporation

July 11, 2019

Rows of virtual files in a data catalog, contributing to powerful data management

The proliferation of Operational Technology (OT) within companies is increasing, which has led many to ask whether this will lead to standardization of how operational technology-generated data is managed. The answer is “it’s complicated.”

There is unlikely to be standardization of the data individual operational technology devices generate, but there will be new capabilities for interoperability, data aggregation, and unified analysis.

What is Operational Technology?

Before examining the standardization issue, it is important to understand the definition of “operational technology.” OT is an umbrella term that describes technology components used to support a company’s operations – typically referring to traditional operations activities, such as manufacturing, supply chain, distribution, field service, etc. (Some companies are relying on operational technology to support, for example, marketing, sales and digital delivery of services, but that is the topic of a future article.)

Operational technology includes, for example, embedded sensors within manufacturing equipment, telemetry from operations components deployed in the field (e.g., oil pipelines, traffic signals, windmills, etc.); industrial IoT devices; location-enabled tablets, which field service personnel use; and much more – the list is long. This is important because OT is not a single classification of technology, it is a descriptor of how technology components are used.

The Push for Interoperability

Some efforts are occurring within the industry to drive interoperability amongst IT and OT components. Open Platform 3.0 (OP3) from The Open Group is a good example. What this standard and others like it seek to do is enable components from different manufacturers to co-exist and work better together within a company’s technology ecosystem. They aren’t seeking to standardize the data coming from individual OT systems or how that data is managed. That challenge is being left to individual companies and the data sciences profession to address.

Data science professionals have been working with companies and individual technology providers for many years to determine a scalable and efficient method to aggregate data from diverse data sources. Efforts to standardize data models and interfaces have been largely unsuccessful due to the desire of some large players in the market to develop and defend closed technology ecosystems.

In light of this, most of the recent developments have been centered on the use of data warehouses to aggregate diverse data and then applying machine learning and artificial intelligence to reconcile differences.

Why Operational Technology Data Management May Never be Standardized

The biggest challenge to standardizing OT data management is managing change. It would be entirely possible to design and deploy a standardized solution to manage all the data generated from OT systems today. The problem is that the technology in this space is continuously evolving and the data being generated is changing too.

Neither technology suppliers nor the companies consuming OT have any desire to slow the pace of technological innovation or constrain it through standardization. New OT innovations will be the driving force behind the next generation of business modernization and companies are eager to consume new capabilities as soon as they can be made available.

How Companies Are Integrating Operational Technology Data

Even though companies don’t have a desire to standardize the data coming from various OT source systems, they have a very critical business need to combine data and analyze it as part of an integrated data set. That is where data management tools, such as Actian, come into play.

Actian’s suite of products, including DataConnect, Actian Data Platform and Zen, provide companies with a platform to manage the ingestion of data from all of their OT data sources, reconcile it in real-time using cloud-scale analytics and machine learning, and then apply the robust statistical analysis (e.g., time series and correlation analysis) to translate data into meaningful insights in an operations context.

The operational technology space is poised to be one of the most important sectors of the IT industry during the next few years. New components will enable companies to generate data from almost all facets of their operations and robust data management solutions, such as Actian, will enable them to interpret this data in real-time to generate valuable operational insights.

While standardization is unlikely, component interoperability is improving and emerging technologies, such as AI, are making data analytics easier. To learn more about how Actian can support your OT efforts, visit www.actian.com/zen.

Exploiting Analytics in the Cloud Without Moving All Your Data There

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

New Actian India Office

About Pamela Fowler

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

The Chief Data Officer’s Evolution to a Data Democracy Sponsor

How Have the Chief Data Officer’s Missions Evolved?

The CDO Towards a New Role

About Actian Corporation

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Managing Data Across Distributed Data Warehouses

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Understanding the Different Data Cultures

Data Anarchy

Data Monarchy

Data Aristocracy

Data Democracy

About Actian Corporation

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Address Integration Challenges With the Proliferation of IoT Devices

IoT Data Integration and Digital Transformation

The Value of an Integration Platform for IoT Solutions

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Data Stewardship and Governance: The Data Steward’s Multiple Facets

Data Stewardship and Governance: The Responsibilities

Operational Supervision and Quality

Documentation

Conformity and Risk Management

Security and Access Control

How Does Data Democracy Strengthen Agile Data Governance?

About Actian Corporation

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Will Data Management in Operational Technology Finally Be Standardized?

What is Operational Technology?

The Push for Interoperability

Why Operational Technology Data Management May Never be Standardized

How Companies Are Integrating Operational Technology Data

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Discover more

TechDay Data Brasil

TechDay Data LaTam

Actian Data Observability

Ready to Get Started?