Data Intelligence

What is a Chief Data Officer?

Actian Corporation

June 17, 2019

According to a Gartner study presented at the Data & Analytics conference in London in 2019, 90% of large companies will have a CDO by 2020.

With the arrival of Big Data, many companies find themselves with colossal amounts of data without knowing how to exploit them. In response to this challenge, a new function is emerging within these large companies: the Chief Data Officer.

The Chief Data Officer’s Role

Considered as data “gurus”, Chief Data Officers (CDO) play a key role in an enterprise’s data strategy. They are in charge of improving the organization’s overall efficiency and the capacity to create value around their data.

In order for CDOs to fulfill their missions, they must reflect on providing high-quality, managed, and secure data assets. In other words, they must find the right balance between an offensive and defensive data governance strategy that matches the enterprise’s needs.

According to the Gartner study, presented at their annual Data & Analytics event in London in March 2019, the CDO, among other things, has several important responsibilities within a company:

Define a Data & Analytics Strategy

What are the short, medium, and long-term data objectives? How can I implement a data culture within my enterprise? How can I democratize data access? How can I measure my data assets quality? How can I attain internal and/or legal regulatory compliance? How can I empower my data users?

There are so many questions that CDOs must ask themselves in order to implement a data & analytics strategy in their organization.

Once the issues have been identified, it is time for operational initiatives. A CDO acts as a supervisor so that the efforts made in providing data information are trustworthy and valuable.

Their role takes shape over time. They must become the new “Data Democracy” leaders within their companies and maintain the investment provided for its infrastructure and organization.

Build Data Governance

Implementing data governance must successfully combine compliance with increasingly demanding regulatory requirements and the exploitation of as much data as possible in all areas of an enterprise. To achieve this goal, a CDO must first ask themselves a few questions:

What data do I have in my organization?
Are these data sufficiently documented to be understood and managed by my collaborators?
Where do they come from?
Are they secure?
What rules or restrictions apply to my data?
Who is responsible for them?
Who uses my data? And how?
How can my collaborators access them?

It’s by building agile data governance in the most offensive way possible that CDOs will be able to facilitate data access and ensure their quality in order to add value to them.

Evangelize a “Data Democracy” Culture

Data Democracy refers to the idea that if each employee, with full awareness, can easily access as much data as possible, an enterprise as a whole will reap the benefits. This right to access data comes with duties and responsibilities, including contributing to maintaining the highest level of data quality and documentation. Therefore, governance is no longer the sole preserve of a few, but becomes everyone’s business.

To achieve this mission, the Actian Data Intelligence Platform connects and federates teams around data through a common language. Our data catalog allows anyone – with the allotted allowances – to discover and trust in an enterprise’s data assets.

Are You a Chief Data Officer Looking for a Data Governance Tool?

In order for Chief Data Officers achieve their objectives, they need to be equipped with the right tools. With the Actian Data Intelligence Platform’s data catalog, CDOs can identify their data assets, make them accessible and usable by their collaborators in order to be valorized.

Easy to use and intuitive, our data catalog is the CDO’s indispensable tool for implementing agile data governance. Contact us for more information.

About Actian Corporation

Actian empowers enterprises to confidently manage and govern data at scale, streamlining complex data environments and accelerating the delivery of AI-ready data. The Actian data intelligence approach combines data discovery, metadata management, and federated governance to enable smarter data usage and enhance compliance. With intuitive self-service capabilities, business and technical users can find, understand, and trust data assets across cloud, hybrid, and on-premises environments. Actian delivers flexible data management solutions to 42 million users at Fortune 100 companies and other enterprises worldwide, while maintaining a 95% customer satisfaction score.

Data Management

Delivering the Right Data to the Right User in the Right Format

Actian Corporation

June 13, 2019

The data consumers within your organization aren’t a homogeneous group, so you must adapt your data management solutions accordingly. At one end of the spectrum is the ideal, “one-size-fits-all” solution. At the other end is a customized set of capabilities for each individual user that seems ideal, but is unworkable. How then do you incrementally approach the ideal solution?

The most effective data solutions are formed by generalizing your user community into a few key groups by skillset and role objectives, data consumption habits and applied tools associated with those data needs, and then deploying the right set of solutions for each group. While each company may be a little different, the following user segments are where to start for most companies.

Data Professionals – These are your data scientists, data analysts and statisticians with specialized training in how to manage and analyze data. They typically need access to detailed raw data and a set of robust capabilities to aggregate, analyze and interpret it. They may employ newer capabilities, such as machine learning and AI, to identify trends and make projections. It’s imperative they are able to ingest text, time-series and structured and unstructured data through large-scale data ingestion, for example, Spark; leverage their favorite programming language APIs, for example, C++ or Python, favorite IDEs, such as Jupyter, and algorithm libraries, such as R-Lib and Tensorflow, as well as visualization tools, such as Tableau.

Business Analysts – These are the business users with data-intensive job roles, such as finance, operations planning, HR and administration. Business analysts typically don’t need access to raw data, but instead data feeds for the specific data sets with which they are working. They typically use analytics tools, such as Microsoft Excel, Tableau, Qlik Sense and Power BI, to sort, categorize and summarize data and provide insights to inform business decisions related to business operations.

Data Curators – These are the people within your organization who generate reports and dashboards for executive consumption. Like business analysts, they typically leverage data feeds and the tools they use are often focused on understanding data quality; managing control thresholds; and creating data visualizations, dashboards and reports. They are seen as closer to the business and program execution.

Executives – The leaders and decision-makers in your organization are focused on understanding the big picture, so they can direct the organization’s resources effectively. They often have little experience creating, analyzing and curating data, but have extensive experience interpreting data, graphs and reports to determine their relevance to and impact on your organization. Recently, many executives have shifted from using tools, such as flip-charts and PowerPoint slides with embedded Excel Spreadsheets, to interactive dashboards as their primary means of consuming and interacting with company data.

Deploy the Right Solution for Each Target Audience

Each of your target-user communities will have a unique set of solution needs. It is important to keep in mind that a data solution must include both the data at scale and with extensive variations as well as the tools for consuming and working with the data. Since many of the user communities will interact with each other, forming a data value chain that transforms raw and stages of processed data into actionable business insights, it is important, therefore, the tools and data solutions interact well with each other across the entire organization.

For example, executives must be able to consume easily the trends and projections your data scientists generate and the analytics your business teams create must be aggregated into enterprise dashboards. Based on executive feedback and direction, those algorithms must then be deployed downstream at the point of action embedded in and aligned with the business operations to streamline and improve them. You can learn more about the range of Actian data management solutions here.

About Actian Corporation

Data Integration

Connecting to Actian Data Platform via JDBC

Actian Corporation

June 12, 2019

Actian delivers an operational data warehouse as a managed service in the cloud. Its developer-friendly high-performance analytics engine has been proven to outperform other popular analytics platforms without specialized hardware or complicated software development. Requires minimal setup, provides automatic tuning to reduce database administration effort, and enables highly responsive end-user BI reporting.

This is how to connect to Actian from a few popular BI tools using the Actian JDBC driver.

You can also connect to Actian from Pentaho, using the Actian JDBC driver.

A prerequisite is to have the Actian Data Platform JDBC driver downloaded and available in the environment where the tools run. The JDBC Driver is available via the Actian Data Platform, formerly Avalanche, console Drivers and Tools link.

Looker

Because Looker is a web-based tool, it already has the driver installed, so there is no need to download the driver.

This is how to connect from Looker to Actian Data Platform.

If you have a Looker-hosted instance, you will need to put the Looker IP addresses on the Allow List in your Actian Data Platform console (Manage > Update Allow List IPs).

More information about the Looker IP addresses is available at Enabling Secure Database Access.

Go to Admin > Database > Connections.

Fill out the connection information (from the Actian Data Platform console Manage > Connect):

• Select Vector for the Dialect.
• Enter the host and the JDBC port in the Host:Port fields.
• Type db for the Database.
• Enter you database username and password in the Username and Password fields.
• In the Additional Params field type encryption=on.

Example:

Click on Test These Settings to confirm that the connection works.

DBVisualizer

Setting up DB Visualizer to connect to Actian Data Platform can be done by following these steps:

1. Add the Actian Data Platform JDBC driver to DBVisualizer.

Go to Tools > Driver Manager…

Add a new driver by clicking on the plus sign (+). Provide a name for the new driver, e.g. Actian Data Platform Driver. Use the folder button to add the jar file (iijdbc.jar) from where it was downloaded and uncompressed.

Select the driver class from the Driver Class drop down box. It should be com.ingres.jdbc.IngresDriver.

Exit the Driver Manager window.

2. Create a new database connection

Click on Create new database connection.

In the wizard, select the newly created driver, then edit the user name, password, and URL.

The URL should be taken from your Actian Data Cloud console (Manage > Connect).

Example:
jdbc:ingres://01c77d46010046ec7.vpaasstage.actiandatacloud.com:27839/db;encryption=on;

Once the connection configuration is completed, the database is available for querying.

To learn more about Actian Data Platform, our fully managed cloud data warehouse service, visit https://www.actian.com/data-platform/.

About Actian Corporation

Data Management

Democratization of Data for Users of Varied Skill Levels and Charters

Actian Corporation

June 11, 2019

If you’re a data engineer or a data architect, then you’re probably kept awake at night wondering how to design your data integration, management and analytics support platforms, so your DBAs and IT Ops colleagues can easily manage them while a varied set of data users are able to consume them simultaneously. These users range from new, demanding, hands-on users, such as developers and data scientists, to those who traditionally use the data through SQL and ad hoc queries, such as business analysts, as well as those who interact with the data indirectly through business applications.

Not everyone in your company is a data scientist and, given how scarce they are, you’d be in quite a small company if they were the majority of staff members. At the risk of over-generalizing the role of data scientists, they tend to need data that supports designing and training algorithms that can be deployed downstream, embedded in other applications and used by other users. Data scientists often need large and varied sets of data, but it seldom needs to be real-time, yet freshness is a paramount requirement as they iterate heuristic training of their models.

Application developers, like data scientists tend to interact with their data through programming APIs. The data sets on which they operate tend to be smaller, or time-series and real-time, embedded directly in the business process instead of informing it as is often the case with what a data scientist is doing. For business analysts, the needs are yet again different and for end-users, the point is to make the data invisible to their operations – even if it’s integral and essential to those operations. The point here is that designers of data systems must be able to make data available, but to several different factions that don’t have the same skill sets, roles and responsibilities or interest levels when it comes to data.

What mandate does this place on data engineers or data architects? Simple. Make data usable for people of varied skill levels to consume what they need, when they need it and in ways that are most useful. Okay, maybe not so simple. How do you avoid siloed sets of data, managed by bespoke systems if you narrowly cater to each of these constituencies?

Understand Your User Community and How it is Using Data

Everyone within your company has a unique set of data needs, both in terms of the type of data and tools he or she needs to use and how this data use is deemed effective. You may have some users who need access to a very specific dataset to perform a focused job task while other users may need big-picture data for planning and strategic decision-making, for example. Some of your users will need detailed raw data, while others need curated dashboards, reports and visualizations. In many cases, the same user may fit into each of the scenarios above, but during different phases of a project. In other cases, these different scenarios leverage the same data in different forms or manipulated in different ways and in combination with other sets of data.

Understanding how to make your data users successful is a function of understanding consumers’ skill levels and the tools and datasets they will need. For example, in addition to the data sets referenced above, data scientists tend to spend much time preparing data and hand-coding algorithms or using libraries for AI and ML, such as TensorFlow. Conversely, business analysts are more inclined to leverage SQL for reporting and popular BI and Visualization tools on those queried datasets. Power users on the business side may be able to handle simpler queries, but are most comfortable manipulating spreadsheets, such as your finance and planning staff. Each of these users has a unique set of needs not just for the data, but also for the tools that actually define how they leverage data to do their jobs. You can learn more about the range of Actian data management solutions here.

About Actian Corporation

Data Integration

Data Management Trends and How They Affect Integration

Actian Corporation

June 10, 2019

Many of the trends of the past few years have successfully made the transition from emerging concepts to mainstream solution approaches. While trends are important building blocks of how companies approach their data management today, they are also providing insights into future capabilities to incorporate the individual pieces into a holistic, integrated solution.

The Cloud is Becoming the First Choice for New IT Systems

Companies have been moving applications and IT components to the cloud for a few years to save costs, improve performance and achieve solution scalability. During 2019, new developments are emerging in the cloud migration trend where most companies are looking to cloud solutions first for new IT systems and only considering on-premises solutions if the cloud isn’t feasible for some reason. This includes both ready-to-use SaaS solutions as well as cloud-based infrastructure (IaaS and Paas) for various needs, such as data warehouses and in-house developed applications. One of the latest cloud trends is integration-platform-as-a-service (IPaaS), which is a set of cloud-based capabilities for integrating both cloud-native and on-premises components for seamless interoperability.

Data Warehouse Migration to the Cloud

During the past few years, Hadoop has been the big trend in data warehouses in the cloud. Many companies have implemented Hadoop on-premises and are now facing increasing operational costs as well as challenges to integrating with cloud-native solutions. The trend of 2019 is for companies to migrate data warehouses to the cloud where they can be operated at a lower cost and with greater performance and scalability. As this migration is occurring, many companies are examining their integration strategies and capabilities to evaluate how to bring more SaaS data sources and streaming data from IoT systems into the data warehouse for analysis.

Data Lake Reboot

The unstructured nature of data lakes made them all the rage during the past 5 years as business users and others marveled at the flexibility of being free from the constraints of relational data structures. Earlier (and small-scale) data lakes seemed like the perfect solution for organizations seeking agility. What companies learned though is sustainable competitive advantage requires some level of structure and their data lakes were quickly devolving into chaos. During 2019, many companies are re-launching their data lakes, adding data governance and support for relational data structures as a means of supporting agility, yet enabling enterprise-scale analytics and the sustainability of data solutions.

The Data Catalog and Metadata to Drive Consumption

One of the biggest pieces of feedback from data users (both data professionals and business consumers) is that they know the data they seek exists, but as enterprise data increases, it is becoming increasingly difficult to find that data. This is driving an increased investment in data catalogs and metadata that not only provides traditional content tagging, but also includes, for example, data quality, age and trustworthiness scores. The catalog and metadata being collected is being used to drive search capabilities as well as new AI-enabled capabilities for data correlation and advanced analysis.

IoT as the Next Wave of Big Data

Big data is no longer an emerging challenge – it is an operational reality in most companies. The new development in the big-data space is the origination of the data. More companies are deploying IoT devices, mobile apps and embedded sensors within machinery that (instead of providing large aggregated data sets) are generating large volumes of independent data streams that must be managed and reconciled. The challenge companies are facing is managing the connections between each of these independent data sources and a scalable way of integrating the data in the data warehouse. This is one of the key use-cases for IPaaS.

Hybrid Data Management

On-premises systems are not disappearing any time soon, so companies must determine how to make their on-premises and cloud data co-exist and interoperate. Many companies are addressing this by keeping their core, report-oriented data warehouses on-premises to avoid impacts to end-users while moving data staging, special data for analytic, and other pieces to a cloud-based solution. During 2019, there will be a large focus on how to manage the integrations both between different components of the data management solution as well as the interfaces with data sources and consuming applications.

Each of these trends provides a building block that will form the foundation of companies’ next generation of hybrid data management solutions. Now that individual facets of the picture have matured, companies are shifting their focus to data and solution integration, leveraging options, such as the Actian DataConnect IPaaS solution, and advanced metadata and catalog techniques to make enterprise data more integrated and more accessible to users. To learn more, visit DataConnect.

About Actian Corporation

Data Intelligence

How Does Data Democracy Strengthen Agile Data Governance?

Actian Corporation

June 10, 2019

In 2018, we published our first Whitepaper, “Why Start an Agile Data Governance?”. Our goal was to present a pragmatic approach to the attributes of such data governance, one that is capable of rising to the challenges of this new age of information:

We advocate for it to be bottom-up, non-invasive, automated, and iterative. In a word, agile.

In this second edition, we decided to tackle the organization of this new agile data governance and its scaling process using the same mindset.

We believe that what distinguishes Web Giants in their approach to their data isn’t the structure of their governance but the culture that irrigates and animates their organization. This culture has a name: Data Democracy.

Assessing Data Governance

Very centralized, and sometimes bureaucratic, they focus on data control and conformity, often resulting in limiting data access among all company employees.

The Concept of a Data Democracy

To understand what Data Democracy is, it is important to know that it is not a governance model. Data Democracy refers to a corporate culture, an open model where liberty goes hand in hand with responsibility.

Data Democracy’s main objective is to make a company’s data widely accessible to the greatest number of people, if not to all. In practice, every employee is able to pull data values at any level.

A democratic approach presents an interesting challenge to balance: on the one hand, you must ensure that the right to use data can truly be exercised, and on the other hand you must counterbalance this right with a certain number of duties.

Building a Data Democracy

The adoption of a data culture can only work if everyone benefits, hence the importance of communication previously mentioned when discussing rights and responsibilities. The balance between the two must be positive in the end, and governance must not introduce more restrictions than gains. Finally, the results must be made easier.

To enable everyone to find the necessary information. That is the main objective of a data catalog, which must, even more so than its basic function (referencing data and associated metadata), offer simplicity of use in order to navigate through an ocean of information.

The New Roles of Agile Data Governance

Under the pressure of digital transformation, new roles appear within large companies.

The Chief Data Officer: The Data Democracy Sponsor

Among them, there is the Chief Data Officer, or CDO. They are in charge of improving efficiency and the capacity to create value for the information ecosystem of their organization.

With the exponential development of data, the role of the CDO took on a new scope.

From now on, CDOs must reconsider the organization in a cross-functional and globalizing way, and governance and corporate data management technology in enterprises.

They must become the new leaders in “Data Democracy” within companies and must respond to the call of numerous “data citizens” who have understood that the way in which data is processed must change radically. The new CDOs must break the bonds of data silos.

Are we all Data Stewards?

The concept of Data Stewardship stems from a much more traditional model. The organizations that already have Data Stewards tend to be quite large and established.

Everyone who uses sensitive data engages their responsibility regarding the way they use it. The regulations for the protection of sensitive data – regulatory or internal – must be applied in the same manner for all those who enter contact with it.

This dedication to involving everyone helps distribute responsibility for data, giving a broader sense of ownership, which encourages users to explore data themselves, and lastly decompartmentalizes data.

About Actian Corporation

Data Integration

Actian DataConnect v11.5 is Here!

Actian Corporation

June 7, 2019

It’s with great excitement that we announce the release of Actian DataConnect 11.5.

This release includes a new set of robust and intuitive features to make the integration and design experience simpler and faster. From mapping and process design enhancements in DataConnect Studio to new and improved connectivity tools in UniversalConnect^TMand user-management features in Integration Manager, DataConnect has the capabilities you need to connect anything, anytime, anywhere.

DataConnect Studio

At Actian, we understand how important it is to provide you with the tools to create integrations quickly, manage them as your environment changes and evolve them to keep your organization agile. With continued focus on the integration design experience, DataConnect Studio 11.5 is full of features to improve productivity and simplify the integration design experience.

Guided workflows help users quickly leverage components in Process Designer while maintaining visual context, limiting the need to “plan ahead,” and reducing the number of clicks required to configure steps.
New automatic debugging support virtually eliminates the need to write your own logging scripts to troubleshoot variables and message objects.
The Map Designer interface has been enhanced, so users can be more efficient by allowing them to: configure events and actions from the “simple mapping” view, bulk-edit field properties and receive immediate interactive feedback on expressions.
A new Extract editor enables visual parsing and extraction of data elements from semi-structured files. Pre-built Template-process designs also accelerate onboarding for new users as “out-of-box” quick start, best-practice accelerators.

UniversalConnect™

The ecosystem of data sources that companies need to connect continues to expand. UniversalConnect has been enhanced in this release to support new connectivity and platform features, including:

ServiceNow Map connector – A new Map designer connector for mapping to and from ServiceNow objects (including custom objects).
NetSuite connector updates – Support the current WSDL version and token-based authentication.
SMTP Email invoker enhancements – Support HTML content, recipient lists and multiple attachments.
ZEN, Vector and Vector Delimited file connectors have also been updated.

Integration Manager

Actian DataConnect Integration Manager is a Web application that allows operations and support staff to configure, schedule, execute and monitor easily all deployed integrations from a single pane of glass. Integration Manager aims to create a streamlined and seamless hand-off experience from designers to production users as integration packages are deployed. This release builds on the core capabilities already in place within Integration Manager while enhancing deployment features and updating secure access capabilities.

As your company’s data environment continues to evolve, Actian is here to provide you with the tools you need to bring your enterprise data together to achieve true enterprise insights and agility. The latest release of Actian DataConnect includes an enhanced set of tools to help your integration designers be more productive and efficient when managing data connections across the enterprise. To learn more, visit DataConnect.

About Actian Corporation

Data Integration

A Guide to Adopting Hybrid Integration Platform

Actian Corporation

June 6, 2019

There are three key trends that have motivated a rapid yet radical transition in how organizations manage their information environments, namely:

As-a-Service Platforms, such as software-as-a-service (SaaS) and platform-as-a-service (PaaS) environments, that allow organizations to transition core business and data management functionality to external service providers.
Cloud Computing, which provides a broad array of low-cost hosted computing, storage, and application services.
Modernization, the desire to consider ways to re-engineer existing business applications to better meet both current and anticipated business requirements.

Together, these trends have triggered waves of data and process migration out of the traditional on-premises data center toward a variety of hosted and cloud-based environments. Yet this migration cannot take place in one fell swoop. Modernization must be planned and incrementally executed, resulting in the growing complexity of what could be called a hybrid environment. These hybrid environments conceptually incorporate data and system services across a variety of platforms including on-premises, a variety of hosted cloud environments, as well as a growing pool of operational systems or Internet of Things (IoT) devices that stream data.

A hybrid integration platform (HIP) is a suite of capabilities that address the challenges of integration across a variety of platforms, sources, and applications.

Gartner specifies that a true HIP should support:

Personas (constituents): Integration specialists, ad hoc integrators, citizen integrators and digital integrators.
Integration Domains: Application, data, B2B and process.
Endpoints: On-premises devices, the cloud, mobile devices and IoT devices.
Deployment Models: Cloud (potentially across multiple environments), on-premises, hybrid (cloud and on-premises) and embedded in IoT devices [as well as XaaS platforms and applications]”.

Although you need the right tools and technology for a HIP, there are some important concepts to focus on if you plan to embrace this kind of environment. This checklist report highlights some of these concepts and provides a step-by-step guide for adopting a hybrid integration platform.

Please click on this link to download this checklist.

About Actian Corporation

Data Intelligence

How Artificial Intelligence Enhances Data Catalogs

Actian Corporation

June 5, 2019

Can machines think? We are talking about Artificial Intelligence, “the biggest myth of our time”.

A simple definition for AI could be: “a set of applied theories and techniques to create machines capable of simulating intelligence.” Among those AI functions, there is deep learning, an automated learning method used to process data.

Data must be understood and accessible. It’s with the help of an intelligent data catalog that data users, such as data scientists, can easily research and efficiently choose the right datasets for their machine learning algorithms.

Let’s see how.

Search Engine: Facilitation Dataset Research

By connecting to all of an enterprise’s data sources, a data catalog can efficiently pull up a maximum amount of documentation (otherwise known as metadata) from its storage systems.

This information, indexed and filterable in the Actian Data Intelligence Platform’s search engine, allows data users to quickly attain the data sets needed for their information systems.

Recommendation System

Guiding Data Scientists in Their Choices

An intelligent data catalog is a tool that rests on “fingerprinting” technology. This intelligent feature gives recommendations to data users as to what data sets are the most relevant for their projects based on, among others:

How the data is used.
The quality and scoring of the documentation.
Its previous searches.
What other users search for.
Give more meaning to their datasets.

This feature offers data users that are responsible for a particular data set some suggestions as for its documentation. These recommendations can, for example, be associated with tags, contacts, or even business terms of other data sets based on:

The analysis on the data itself (statistical analysis).
The schema resembling other data sets.
The links on the other data set’s fields.
Automatically contextualizing data sets in a data catalog allows for any data user to work with data that is understood and appropriate for their use cases.

Automatic Dataset Linking: Visualizing Your Data’s Life Cycle

As mentioned above, with fingerprinting technology, a data catalog can recognize and connect to other data sets. We are talking about data lineage: a visual representation of data life cycles.

Automatic Error Detection: Be Aware of Errors in Datasets

In order to overcome potential data interpretation problems, an intelligent data catalog must be able to automatically detect errors or misunderstandings in the quality and documentation of any data.

This key feature, based on the analysis of data or its documentation, must alert data users of its integrity.

GDPR Notification: Be Notified of Sensitive Information

An intelligent data catalog must be able to detect personal/private data in any given data set and report it on its interface. This feature helps enterprises respond to the different GDPR demands put into place in May 2018, and also to alert potential users on the sensitivity level as well as the use of their data.

About Actian Corporation

Data Platform

A Cost/Benefit Guide to the Data Warehouse

Actian Corporation

June 4, 2019

Implementing a data warehouse is a big investment for most companies and the decisions you make now will impact both your IT costs and the business value you are able to create for many years.

This concise data warehouse cost/benefit guide will help you understand what to expect, so you can make informed decisions about what solution is best for your company. There are many options available in the marketplace, from on-premises installed solutions that run in your company’s data center to hosted cloud services and Data Warehouse-as-a-Service (DWaaS). The solution you select will determine what costs you incur and what benefits you can realize.

Data Warehouse Cost

Your data warehouse is the centralized repository for your company’s data assets. Data from all of your IT systems will be copied into the warehouse where it can be aggregated, sorted, stored, analyzed and curated into the reports and insights that decision makers and operational staff need, so your business runs smoothly.

The large volume of data created as a result of your day-to-day business operations means your data warehouse is likely to be your largest single IT system (and, perhaps, your most valuable one too). A combination of size, deployment option (on-premises or cloud) and the sophistication of analytics tools that come with the warehouse will drive the cost of your data warehouse. You should plan for these main cost categories:

Setup – These are the costs to acquire and configure the data warehouse solution. If your solution will run on-premises, then setup costs may include, for example, data center hardware, purchase/licensing of solutions and staff or consultants to configure the solution and establish connections with each of your source IT systems. If you are choosing a cloud data warehouse, then you may be able to avoid many of the up-front infrastructure costs, but you are likely to still need staff or contractors to configure the cloud service to integrate into your IT environment.

Data Migration – If you are replacing an existing data warehouse or consolidating data from your current databases, then you will need to migrate data from the old systems into the new warehouse. The migration itself isn’t difficult, but this is often when you discover data quality issues that must be addressed. Both the amount of data you are moving and the amount of remediation that is required to achieve an acceptable level of quality drive data cleanup and migration costs.

Compute and Storage Capacity – If your solution will run on-premises, then you must determine how much storage and compute capacity you will need in advance, so you can allocate your infrastructure investments appropriately. If you are using a cloud data warehouse or Data Warehouse-as-a-Service, then your compute and storage capacity will likely be a combination of a baseline subscription fee, Performance SLAs and the amount of resources your company uses. You should expect these costs to increase during the future (for both on-premises and cloud) as a function of your data volume.

Administration – Data warehouses (whether on-premises or hosted in the cloud) require active administration to: ensure data feeds are processing as expected, monitor analytics batch jobs for errors and manage user access to data. Most solutions include administration tools to assist in these tasks, but you will still need a person to be responsible for overseeing the continuous operations of your data warehouse.

Data Hygiene and Maintenance – This is one of the most overlooked costs of a data warehouse. The value of your company’s realized business insights is directly dependent on the quality of data with which you are working. To maximize your potential value, continuous data maintenance is needed. This includes purging old records, reconciling conflicts among data feeds, refining the data model, validating data for completeness and accuracy, and ensuring data is properly categorized and tagged, so users can find it easily.

In addition to these cost categories, you should also expect costs related to onboarding new data sources, supporting changes to source systems, implementing advanced analytics capabilities, such as AI, and training your user community how to use the data warehouse effectively.

Operational Benefits

The costs outlined above may seem overwhelming and lead you to question whether implementing a data warehouse (or upgrading your current one is a good idea). It is important to consider the value and benefits it will provide your company when assessing the investment choices, as different deployment options yield different levels of benefits. Expect your data warehouse solution to benefit from the following attributes:

Performance – The main reason you are implementing a data warehouse is because you must manage large amounts of data and run resource-intensive queries, but you don’t want the transactional source systems that your company is using for day-to-day business to perform slowly. The data warehouse provides a dedicated environment designed for these resource-intensive activities. Large-scale infrastructure in your data center powers your on-premises solutions, while cloud data warehouses use an elastic-demand model, drawing from a pool of resources to support your storage and processing needs.

Scalability – Your company’s data assets will continue to increase as a result of normal business operations and the impact of digital transformation initiatives on your operations. A data warehouse is one of the best available tools for managing data growth by enabling archival, aggregation and analysis of data from many different data sources. Data warehouses (particularly new cloud-based solutions) are built on highly scalable architectures and infrastructure platforms to enable full-featured data warehouse capabilities to be deployed at a small scale, and then expanded as the company’s needs increase.

Cost Benefits – Cloud-based data warehoused, in particular, provide companies with a number of cost benefits related to IT expense management. Cloud service providers benefit from immense economies of scale and buying power, which means they are able to acquire and manage the underlying hardware your data warehouse uses at much less cost than you could purchase it yourself. The service provider also manages the depreciation of hardware assets, simplifying your company’s ITAM activities. The biggest benefit of cloud-based data warehouses is the avoidance of underutilized capacity with a demand/utilization-based billing model in which your company pays only for the resources you consume rather than paying for much more expensive hardware up front and it sitting idle some of the time.

Resiliency – Data warehouses, both on-premises and in the cloud, provide a layer of resiliency to your company data by maintaining a complete copy of your operational data separate from the source systems. In case of a data breach, hardware failure or disaster scenario, your data is still available. Cloud-based data warehouses have additional resiliency features, including service provider-managed maintenance and security patching and script-based deployment that can be leveraged during a disaster recovery scenario.

Strategic Benefits

Most of the operational benefits of a data warehouse are related to your IT department cost structure, technology assets and overhead required to administer the system. More importantly, data warehouses provide a set of strategic benefits to your company that (although they are difficult to quantify) are very important to consider.

The Cost of Poor Decisions – Your data warehouse is a decision-support system intended to help your company leaders and operational staff make informed business decisions. Without the data it provides, you are at a greater risk of making potentially catastrophic decisions based on false assumptions.
The Speed of Insights – Even without a data warehouse, your company likely has all the data building blocks needed to understand what is occurring within your organization. The data warehouse provides a way to mine, refine and harvest actionable insights faster – increasing the amount of time available to realize the benefits from those insights in both exploiting opportunities and mitigating risk.
Capability to Change in the Future – Your business and IT environment is continuously changing, with new solutions replacing legacy systems, cloud service providers and business processes due to re-organization. A data warehouse provides your company with the capability to isolate your data from the impacts of these changes, maintaining a consistent record of your business, regardless of what people and tools are being used.

The investment in a data warehouse is one of your company’s most important IT decisions. Whether you choose an on-premises solution or a new cloud data warehouse offering, such as Actian Data Platform, your company will benefit from the impacts of this decision for many years. Understanding the costs and benefits related to this decision is essential when making an informed investment that aligns with your company’s data management goals. Learn about the benefits of deploying your own Data Warehouse in the cloud by downloading the Whitepaper here.

About Actian Corporation

Actian Life

Employee Spotlight: Marc Potter

Rae Coffman-Bueb

June 3, 2019

At Actian, we have outstanding employees from a variety of backgrounds. We believe that your life experiences make you the unique person you are today and influence what you bring to the amazing culture here. We have chosen to feature those incredible individuals with a blog post dedicated to them – I am pleased to introduce to you, our Employee Spotlight.

To kick off the series, we met with our new CRO, Marc Potter.

Growing up, Marc taught disabled children to ski – something he picked up from his father, an Olympic downhill skier and disabled ski instructor. Upon graduating college in 1992 from the University of New Hampshire, he moved to Vail Colorado to ski while he supported himself as a short-order cook at the top of Vail Mountain. When summer arrived, he moved to Park City Utah for mountain biking and then onto Wyoming to continue his chase for the perfect slopes. One day on the way to work in Jackson Hole Wyoming, he hit a cow with his car and broke the axle. When Marc later met his wife, he told her, “I was a ski bum that wanted to go find myself and what I found after 2 years is that I was broke and needed to find a job.”

Marc started his career as an Inside Sales Rep in IT network sales. From there, he grew to manage Sales and Operations for an office outside of Frankfurt, Germany. He later became the Director of Regional Sales and found himself moving back to the US, where he settled in Atlanta so that he could go to the Olympic games being held there that year. Which luckily, he did – since that is where he met his wife, Janeen! They had a daughter named Chloe in 2002 and planted roots in Atlanta, Georgia where he has continued to climb the leadership ranks in software sales. He was most recently VP of Sales at Oracle, before joining us here at Actian. As Chief Revenue Officer, Marc brings over 20 years of enterprise sales and leadership experience in information technology. To put it in Marc’s own words;

“I’m excited about the opportunity to lead the Actian sales team through this transition from on-premises solutions to a market-leading Cloud company. Our customers are on this same Cloud journey and looking for a data-driven technology company that can deliver price and performance at enterprise scale. I look forward to working as a team to become the market leader in Cloud data management and analytics.”

In his free time, Marc enjoys golfing and spending time with his family on the lake. His biggest passion is volunteering in his community and specifically working with Habitat for Humanity.

We couldn’t be more excited to have Marc join the Actian family!

About Rae Coffman-Bueb

Rae Coffman-Bueb is Director of Employee Experience at Actian, dedicated to enhancing organizational culture. With a background in People Operations, Rae has implemented global best practices that empower teams and streamline HR processes. She provides guidance on talent development, onboarding, and cross-functional collaboration. Rae's blog posts focus on employee engagement, internal communications, and HR innovations. Check them out for tips on boosting workplace satisfaction.

Data Integration

Digital Transformation Begins With Integration

Actian Corporation

June 3, 2019

Almost every company is embarking on some sort of digital transformation initiative – reshaping the relationships between employees and business processes and the technology that supports them. At the heart of any digitally transformed operation is data – data about your company’s operations, data about your customers and suppliers and data about your environment.

What makes digital transformation so powerful is that it optimizes the way people interact with data by using technology to integrate different data sources in support of processes and decision-making. A successful digital transformation initiative begins with data integration.

The Role of Data Before Digital Transformation

Before digital transformation, there was a clear delineation between the activities people performed in support of business processes and the technological processes and systems that supported them. Data was created in the physical world and recorded in the digital world. The interfaces where these worlds intersected (forms, screens, Web pages, etc.) were very specific in their designs to capture data from one world and transmit it to the other. While this worked okay to enable people to use technology in support of business processes, it also served to reinforce the separation. In that environment, the flow of data both between people and machines and across the organization became the ultimate constraint to productivity and process performance.

Digital Transformation Changes Everything

The main concept behind digital transformation is removing the preconceived notions of barriers between people and technology being necessary and, instead, re-imagining an environment where humans and technology freely interact in support of business processes. With digital transformation, there are no business processes and IT processes, just processes. The structured interfaces (forms, screens, Web pages, etc.) between the physical world and digital world are replaced with a new set of capabilities enabling users to interact freely with data in an immersive manner. Within this new environment, data silos that were built for functional IT systems where data is stored are replaced with enterprise data, which policies and role-based access controls manage.

Why Integration is Important for Digital Transformation

Data integration is an essential part of your digital transformation journey. Integration is the tool to help you eliminate the silos and re-factor user interfaces. Without doing this, you won’t be able to achieve your company’s vision of modernized technology-embedded processes. By addressing integration at the start of your digital-transformation initiative, you will be able to create a new/modern data platform upon which to build your new processes. Because you are integrating instead of consolidating source systems, your legacy and new processes can co-exist through the transformation process.

Integration and Business Agility

Once your digital-transformation initiative is complete, the focus of your organization will shift from transformation to achieving business agility to sustain continuous evolution and the re-invention of your processes. Data integration helps you achieve agility by decoupling your processes from individual IT systems, applications or components. New systems can be added to your environment and old ones can be removed freely, but your enterprise data can remain stable and secure. Business agility also refers to making on-the-spot business decisions that are informed by real-time information insights. Data integration enables your operations staff and company leaders to see what is occurring across the company, monitor for changes and identify areas rapidly that require immediate attention.

Digital transformation is an ambitious goal for any company – moving beyond the constraints of the past and reimagining an environment where people, processes, data and technology interact freely in support of your company’s goals. Data integration is an important part of taking the vision of digital transformation and making it a reality. Actian DataConnect can help by giving you the tools you need to connect anything, anytime and anywhere. To learn more, visit DataConnect.

What is a Chief Data Officer?

The Chief Data Officer’s Role

Define a Data & Analytics Strategy

Build Data Governance

Evangelize a “Data Democracy” Culture

Are You a Chief Data Officer Looking for a Data Governance Tool?

About Actian Corporation

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Delivering the Right Data to the Right User in the Right Format

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Connecting to Actian Data Platform via JDBC

DBVisualizer

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Democratization of Data for Users of Varied Skill Levels and Charters

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Data Management Trends and How They Affect Integration

The Cloud is Becoming the First Choice for New IT Systems

Data Warehouse Migration to the Cloud

Data Lake Reboot

The Data Catalog and Metadata to Drive Consumption

IoT as the Next Wave of Big Data

Hybrid Data Management

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

How Does Data Democracy Strengthen Agile Data Governance?

Assessing Data Governance

The Concept of a Data Democracy

Building a Data Democracy

The New Roles of Agile Data Governance

The Chief Data Officer: The Data Democracy Sponsor

Are we all Data Stewards?

About Actian Corporation

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Actian DataConnect v11.5 is Here!

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

A Guide to Adopting Hybrid Integration Platform

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

How Artificial Intelligence Enhances Data Catalogs

Search Engine: Facilitation Dataset Research

Recommendation System

Guiding Data Scientists in Their Choices

Automatic Dataset Linking: Visualizing Your Data’s Life Cycle

Automatic Error Detection: Be Aware of Errors in Datasets

GDPR Notification: Be Notified of Sensitive Information

About Actian Corporation

Subscribe to the Actian Blog

Subscribe