Blog | Data Integration | July 10, 2019 | 6 min read

4 Steps to Build a Capability-Based Data Integration Strategy

By Actian Corporation

#Data Integration #Data Integration Strategy #Data Warehouse
data integration strategy

By Actian Corporation

#Data Integration #Data Integration Strategy #Data Warehouse

Data integration will be one of the most important strategic challenges any company’s IT organization will face during the next few years. As rapidly evolving market environments and the drive for business agility compel company leaders to demand better-integrated data during near-real time, IT executives and their architects will be faced with an increasingly larger challenge to deliver the capabilities their companies need, both for the present and the future.

A capability-based data integration strategy is likely the answer. By mapping what business and technical capabilities will be needed (and when), IT decision-makers can make strategic appraisals about when to invest in the deployment of new technical solutions and when to focus on using the existing solutions more effectively (increasing adoption). Here are the four simple steps to build a capability-based data integration strategy for your company.

1. Determine Your Business Needs

This is perhaps the biggest challenge for most IT organizations. IT teams are good at observing technological trends (what’s new and cool in the industry). They are also experts at responding to the stated needs (and demands) of business users. The struggle for many IT organizations is anticipating how a business’ capability needs will evolve during the next few years and how that aligns with the pace of new technology being developed in the marketplace and adopted within a company. The result is IT focuses resources on addressing yesterday’s problems or deploying technology a business doesn’t actually need.

Developing a business-centric mindset, where technologists collaborate openly with business leaders to understand how the business operations work and how they must evolve, will give IT staff the ability to focus on solving actual business problems. It will also provide IT with the opportunity to avoid proactively business issues that impede operational productivity (and company profitability).

With data integration, it is important to understand from business teams what data they will need, with whom they will need to collaborate and where real-time vs. aggregated data will be needed. The goal of this relationship is to develop a clear list of a business’s operational (and technological) needs during the next few years and when those needs are expected to become critical.

2. Assess Your Current Capabilities

Don’t assume you know your current technical capabilities throughout your company. In most organizations, IT systems are fragmented, siloed and, in some cases, duplicated across business functions. Before you start shopping for new systems, assess your current parts and pieces and their future potential (not just how they are being used today). During this assessment, review how your business and operations teams are using the capabilities you’ve deployed during the past. Are they using all of their potential? Is adoption an issue? Do they understand what data is available to them and how to access it?

If you want to solve your company’s integration challenges successfully, then you can’t just focus on deploying technology – you must look at the whole system, which includes people, processes, data, technology and culture. You can deploy the best technology in the world, but if people don’t know how to use it or you don’t have a culture willing to embrace data-driven decision-making, then achieving the desired results will be difficult.

3. Determine What Capabilities Are Needed to Address Your Business Needs

Once you understand your business needs and the existing components in your organization’s environment, the next step is to connect the two. The best method is to ask a simple question, “If we want to solve this problem (or opportunity), then what components will we need?” You will notice the question is framed around the technology (“How can we convince more people to adopt these systems?”). Capability-based planning is the path to solving problems.

When you initiate a capability-based integration strategy, it is imperative you focus on solving concrete business problems. This may require new data sources, a more scalable methodology to connect and ingest data, better data-access-control policies, data warehousing or analytics tools and user training.  Each organization is different and the variety of technological components from which you can choose is continuously expanding. Your IT staff can add tremendous value at this point by understanding leading-edge technological trends and applying them to your business problems.

4. Align Investments to Deliver New Capabilities When They Are Needed

Once you thoroughly understand what components are needed to deliver each business capability, you may start to feel overwhelmed at the size and cost of the components on the list. This is the ideal time to discuss with business leaders their vision of how the business may change during the next few years. You must also be aware of the often-large gap between the level of importance and the level of urgency of an individual capability before you can proceed with integration. Your goal is to understand both, so you can develop a delivery roadmap that makes the essential capabilities available just before your business needs them (not too early or too late).

Most IT budgets are limited, so it is not likely you will be able to fulfill all of your business’ needs as quickly as they are required. You can manage this situation by either prioritizing individual requests or looking for common needs that shared investments can fulfill. Examples include acquiring a new external data source that multiple functions can use or implementing a data-integration platform to manage streaming data and IoT devices. Remember, your goal is to fulfill as many business needs as possible (to maximize value), not just deploy new systems.

Applying a capability-based integration strategy can be an effective means of increasing the value of your IT investments. By focusing on your business problems, not just today’s, but also during the next few years, your IT staff will develop a deeper business understanding that will enable them to better support your company’s strategic needs. Data integration will be one of the biggest strategic challenges your company faces during the next few years and an ideal situation to apply a capability-based planning approach.

Blog | Data Integration | July 9, 2019 | 4 min read

The Importance of Data Stewardship Agreements

By Actian Corporation

#Data #Data Stewardship
Data stewardship

By Actian Corporation

#Data #Data Stewardship
In this post:

There has been much talk recently about the technical implications of leveraging SaaS solutions and the benefits of integrating data across supply-chain networks. Today, most companies aren’t vertically integrated and, therefore, rely on a network of suppliers and distribution partners to transform raw materials into the finished goods and services customers consume.

It makes sense in the era of digital transformation initiatives that integration of IT systems and data sharing across companies have become high-priority strategic initiatives. Integration is great, but it raises a new set of issues about data quality, security, and governance that companies must now address.

If you are considering cross-company integration, then you should also be establishing formal data stewardship agreements with the companies with whom you integrate.

What is a Data Stewardship Agreement?

Data stewardship is a term that has been used in the data management profession for many years and refers to the set of activities that are performed to manage and monitor the quality of company data and control how the data is used. Enterprise data can be quite voluminous and diverse. It isn’t practical for one person or team to understand, “own” and manage the end-to-end data set.

Data stewards are the people who understand the company’s data, how it is created, where it is modified, when it can be archived/deleted, and who should be using it. Their job is to safeguard the integrity of the data and ensure it is used appropriately.

Each IT system and/or business function has unique data related to its operations and with its stewardship considerations. Examples include customer data, sales records, HR data, supplier orders, etc.

For cross-company integrations, data stewardship becomes more complicated. Instead of relying on a person to be a data steward, contractual documents called data stewardship agreements are used to outline each organization’s responsibilities in managing data, constraints about how data is used or shared, and disclosure requirements for any data-policy violations.

Why are Data Stewardship Agreements Important?

Efficient supply-chain processes require the free flow of information between companies, but it also creates a significant potential for risk and misuse of data. For example, a component supplier may need visibility of the pipeline of new products and services being developed, but it should be constrained from sharing that information with competitors. A marketing company may be hired to do customer-segmentation analysis, but the personally identifiable customer data it uses must be protected from potential data breaches. An IT-support provider may be granted access to ITSM data about outages, security vulnerabilities and bugs, but this information could be embarrassing to the company if released to the public.

Data stewardship agreements are important because they enable companies to extend their enterprise-data sets to include data created and aggregated across the entire supply chain while providing a set of enforceable guidelines and controls to mitigate risk. It is important your data stewardship agreements be robust yet unambiguous. They should be in writing and included in the contractual documents that govern the relationships with your company’s trading partners. These agreements can also be used to establish liability and potential reparations in the event of a data breach (which is happening much too often recently).

Modern technology makes integration of business processes across companies and IT systems much easier. It is resulting in new business models, greater specialization and access to global markets that weren’t within reach for most companies a few years ago. Sharing data is important, but it should also be approached carefully. Data stewardship agreements are an important and effective tool for brokering your data-sharing relationships across companies.

Blog | Data Management | July 8, 2019 | 4 min read

Are You Collecting the Right Data Sources?

By Actian Corporation

#Data #Data Sources #Data Warehouse
abstract concept of connecting data sources with confidence

By Actian Corporation

#Data #Data Sources #Data Warehouse
In this post:

Quantity, as well as quality, are so important to your data, but there is a third dimension to your data puzzle that is even more important – content.

As your company evaluates new data sources and improvements to your existing data, ask yourself, “Are we collecting the right data?” Here are three tips to determine if you are acquiring the data you actually need, or just creating data clutter.

Point-of-View vs. Perspective

Each data set you acquire provides you with a unique point of view of your business operations and external environment. While insightful on their own, a single data set rarely (if ever) provides the complete picture. There are gaps, blind spots, bias and many other issues you will be forced to address. Similar data sets or those from similar sources are likely to have similar defects.

You can avoid this with data diversity. By aggregating data from different sources, you can assemble multiple points of view of your operations, which leads to a more holistic perspective.

The best methods to identify what new data sources you need are to look for gaps in your current data as well as areas where data sources always agree with each other. Some level of data conflict is good because it indicates you are gathering different points of view that describe unique facets or dimensions to your company.

Duplication and Redundancy

While those words may seem to mean the same, in the context of selecting new data sources, there is an important distinction to understand. Duplicate data (that is the same) can usually be traced back to the same source system, even if it is acquired through different channels. A good example may be a list of products obtained from the marketing system vs. the manufacturing system.

If the lists are the same, then either one of the two systems is the system of record and the data has been copied into the other, or the data is sourced from somewhere else entirely. This is important because adding duplicate data doesn’t create additional value for your company – you already have that data set.

Redundant data (data sets that are different, but overlap) is highly valuable because it reflects different perspectives. In the marketing and manufacturing example, the list of products from manufacturing may contain those products your company builds or is in the process of building.

The marketing product list may contain products that you resell from 3rd parties (but don’t build yourself), but may not include new products that R&D is still developing. Some of the data between these data sets is the same, but the pieces that are different are very insightful.

Data That is Up-to-Date

Every piece of data you collect has a time stamp of when it was created or observed. Data starts aging from the time it is created, not when it is collected and added to a data warehouse. It is important to understand when your data was collected and how current the data is you ingest from different data sources. Digital business processes require real-time data to be effective.

To ensure you are collecting the most current data, trace where your data originated. You ideally want to collect data directly from the source system where it is first created and not some downstream system that only refreshes data periodically.

Data time stamps are particularly important in situations where you must perform time-series analysis to identify operational trends and quality issues or forecast future events. The sooner you can acquire the data, the sooner you will be able to analyze it and update your operational reports and forecasts – leading to more agile business operations.

Your company is continuously evolving, both your operations and your environment. Continual refinement of your data sources to ensure you are obtaining a holistic perspective that generates actionable insights and provides real-time visibility is essential if you want to succeed in a highly competitive business environment. It isn’t just about collecting more data, or better-quality data – you must collect the right data.

You can learn more about Actian data management products here.

Blog | Data Platform | June 24, 2019 | 4 min read

Top 5 Issues in Data Warehousing for IT Leaders

By Actian Corporation

#Data #Data Warehouse #Enterprise Data
exclamation marks on a red backdrop

By Actian Corporation

#Data #Data Warehouse #Enterprise Data
In this post:

With more and more companies embarking on digital transformation journeys, the focus on enterprise data is greater than ever. Data warehousing is not a new concept, but recent developments in the industry are generating a new wave of executive interest and the need to modernize both the approach and solutions for how enterprise data is managed. Here are the top 5 issues that IT leaders are facing when charting a future course for data warehousing capabilities.

Security

It is no surprise that information security and risk management are top of mind for most IT leaders. In the data warehousing space, this means not only physical security against hacking, data loss, etc., but also securing the company’s intellectual property against unauthorized use. Data is increasingly becoming a strategic asset that companies can use to develop and sustain a competitive advantage in the marketplace. Benefiting the most from your data involves making it easily accessible for those people who need to use it while maintaining the confidentiality of sensitive company data and personal data about customers.

Performance

Data warehouse performance has elevated into a top issue since 2018 with the proliferation of digital transformation initiatives that more deeply embed technology and data within transactional business processes. These new digitally enabled processes depend on real-time data to provide staff with the insights they need both to sustain productivity as well as support informed decision-making. The data warehouse is very important to the support of a company’s data consumption needs and does so by providing a set of capabilities for aggregating data across multiple sources, performing large-scale analytics and refinement processes, and making data available to consuming users and IT applications. Cloud-based technologies and in-memory computing enable modern data warehouses to move massive amounts of data extremely fast – creating an information engine for modern business.

Flexibility

Companies’ IT environments are not homogeneous, nor are they stable. There are a great variety of infrastructure configurations, including, for example, on-premises data centers, public cloud, private cloud, and SaaS solutions that are involved in a company’s data value stream. Not only are IT leaders faced with ecosystem diversity, but they are also challenged with a rapidly accelerating pace of technological change. For data warehousing, this means companies need the flexibility to support the various deployment modalities that exist today within their IT environments without being locked into a specific solution that would impede their ability to change course in the future.

Cost

Data growth within modern companies is accelerating faster than ever. IoT, cloud services, mobile apps, and 3rd party data feeds are just some of the examples of new data sources that are causing the growth curve of data warehouse capacity to exceed significantly forecasts from a few years ago. This creates a troubling cost challenge for IT leaders who are tasked with enabling new business capabilities while simultaneously reducing IT operational costs. Many companies are in the process of migrating their data warehouse capabilities to the cloud, where storage is cheaper and more scalable, eliminating the operating costs of on-premises data centers and storage hardware that become obsolete and depreciate in value rapidly.

Speed of Deployment

Businesses are evolving faster than ever and as they do, executives expect IT systems to adapt just as quickly to support emerging needs and opportunities. For data warehouse systems, this translates to speed of deployment – both in terms of new data sources for ingestion as well as new consuming systems (such as AI and advanced analytics systems). Companies are faced with a shrinking window of time to maximize the value from opportunities and mitigate risks/threats. Since data is so integral to informing business decisions, data warehouse solutions (both new systems and changes) must be quick to deploy and easy to change.

Enterprise data has never been more important to companies than it is today. IT leaders face a big challenge to evolve their technological capabilities rapidly to support business needs, and the data warehouse is recognized as the core of the modern IT ecosystem. Actian provides companies with a set of modern capabilities for ingesting data from all your data sources, processing it quickly and effectively at scale, and then making data easily accessible to those people and systems within the organization that need to use it. To learn more, visit www.actian.com/data-platform.

Blog | Actian Life | June 18, 2019 | 2 min read

Ready, Set, Dough

By Actian Corporation

#Company Culture
Teambuilding - Ready, Set, Dough

By Actian Corporation

#Company Culture

Actian Employees Run for the Gold..en Donuts

All work and no play make for a hungry team according to Actian employees, or at least that’s what we found out during a local community event.

With our new location nestled in the quiet little city of Round Rock, it was time to venture out and find new ways to embrace the new surroundings. We caught wind of a new tradition that was going to take place and it involved donuts soooo… what were we waiting for?

A small group of us decided to sign up for the 1st Annual Round Rock Donut Dash 5K. If you’ve never heard of Round Rock Donuts, where have you been? This famous establishment has been in business since 1926 and has been featured on national platforms such as USA Today, The Food Network, and The Jimmy Kimmel Show just to name a few.

So, what made this 5K different than all the rest? To summarize – a lot! The competitor category challenged runners to run the first half of the race to a pitstop where 12 Round Rock donuts would be waiting to be demolished before being able to complete the second half of the race, all in under an hour. It seemed like a small feat at first, but little did our team know just how difficult this challenge would be. With all of that being said, Daniel, Tony, and I were not daunted by the task at hand. We went, we ate, we conquered.

We are happy to report that no Actian employees were harmed during their attempt at this event and all participants that entered the challenger category successfully completed the challenge.

Blog | Data Platform | June 17, 2019 | 4 min read

Too Much Data, Too Few Actionable Insights…Too Late!

By Actian Corporation

#Actionable Insights #Data #Data Warehouse
researcher contemplating data findings

By Actian Corporation

#Actionable Insights #Data #Data Warehouse

Data is the lifeblood of a modern company. It provides operations staff with visibility of company operations, marketing and product development teams with insights about what’s occurring in the marketplace, and company leaders with the input they need to make informed business decisions.

Unfortunately, most companies are struggling to translate their data into true business value and a sustainable competitive advantage. They have too much data and are generating too few actionable insights, and by the time they do harvest a valuable nugget, the window of opportunity to act on it has passed. Here are some of the most common challenges from business leaders regarding operational data.

We have more data than we know what to do with.

Companies have been collecting large amounts of data for years in their ERP, CRM, HR, ITSM systems, and many others. With the digital transformation of business processes, even more data is being generated about operational processes. Now that IoT and mobile devices are a mainstay of business, streaming data is becoming a real issue.

Intuitively, companies know that data is valuable, but they continue to experience data silos and a perception that all the data they are creating is going into a black hole never to be seen again. For companies that want to develop a sustainable competitive advantage, they must start aggregating, organizing, and refining their massive data stores into real business intelligence that leads to better decisions and more efficient operations.

We are struggling to convert data into actionable insights.

Data is a raw material, not a finished good. The source of its value is a combination of quality inputs and the process of data refinement. Many company leaders forget the importance of the refinement process when aggregating, reconciling, organizing, filtering, analyzing, and interpreting their data – they assume the solution for too few actionable insights is to collect more data.

The key to converting data into actionable insights is having the right set of tools and a structured method for processing data through a value stream to generate progressive levels of refinement. For most companies, data moves from the source, into a data warehouse and is then distributed to users in the form of reports and dashboards.

There are only 2 levels of refinement (aggregation into the warehouse and curation into reports) occurring. This isn’t enough to manage the data complexity of modern business. A 4–5 tier refinement process that includes filtering data at the source, aggregating it in an operational data warehouse, normalizing it into an enterprise data model, segmenting it into functional views and then curating it into role-based dashboards and reports will provide data consumers with better quality and actionable information insights.

By the time we harvest insights, it’s too late to act on them.

Business agility is only achievable with near real-time information insights. Batch processing of data from source systems and nightly refreshes of reports and dashboards, for example, simply isn’t sufficient to support the demands of modern businesses where minutes can represent the difference between an opportunity captured or lost and a risk becoming a crippling major incident. Digitally transformed business processes rely on real-time data to enable staff to make decisions and keep processes within the company operating smoothly.

Data management solutions, such as the Actian Data Platform, can help resolve the processing delay by providing a set of scalable, high-performance and cloud-based capabilities to ingest data from all your source systems in real-time and perform the necessary processing tasks to transform your raw data into actionable insights. Don’t miss another business opportunity because you have too much data and too few insights delivered too late. Visit www.actian.com/data-platform to learn more.

Blog | Data Intelligence | June 17, 2019 | 4 min read

What is a Chief Data Officer?

By Actian Corporation

#Data Governance
what is a chief data officer

By Actian Corporation

#Data Governance
In this post:

According to a Gartner study presented at the Data & Analytics conference in London in 2019, 90% of large companies will have a CDO by 2020.

With the arrival of Big Data, many companies find themselves with colossal amounts of data without knowing how to exploit them. In response to this challenge, a new function is emerging within these large companies: the Chief Data Officer.

The Chief Data Officer’s Role

Considered as data “gurus”, Chief Data Officers (CDO) play a key role in an enterprise’s data strategy. They are in charge of improving the organization’s overall efficiency and the capacity to create value around their data.

In order for CDOs to fulfill their missions, they must reflect on providing high-quality, managed, and secure data assets. In other words, they must find the right balance between an offensive and defensive data governance strategy that matches the enterprise’s needs.

According to the Gartner study, presented at their annual Data & Analytics event in London in March 2019, the CDO, among other things, has several important responsibilities within a company:

Define a Data & Analytics Strategy

What are the short, medium, and long-term data objectives? How can I implement a data culture within my enterprise? How can I democratize data access? How can I measure my data assets quality? How can I attain internal and/or legal regulatory compliance? How can I empower my data users?

There are so many questions that CDOs must ask themselves in order to implement a data & analytics strategy in their organization.

Once the issues have been identified, it is time for operational initiatives. A CDO acts as a supervisor so that the efforts made in providing data information are trustworthy and valuable.

Their role takes shape over time. They must become the new “Data Democracy” leaders within their companies and maintain the investment provided for its infrastructure and organization.

Build Data Governance

Implementing data governance must successfully combine compliance with increasingly demanding regulatory requirements and the exploitation of as much data as possible in all areas of an enterprise. To achieve this goal, a CDO must first ask themselves a few questions:

  • What data do I have in my organization?
  • Are these data sufficiently documented to be understood and managed by my collaborators?
  • Where do they come from?
  • Are they secure?
  • What rules or restrictions apply to my data?
  • Who is responsible for them?
  • Who uses my data? And how?
  • How can my collaborators access them?

It’s by building agile data governance in the most offensive way possible that CDOs will be able to facilitate data access and ensure their quality in order to add value to them.

Evangelize a “Data Democracy” Culture

Data Democracy refers to the idea that if each employee, with full awareness, can easily access as much data as possible, an enterprise as a whole will reap the benefits. This right to access data comes with duties and responsibilities, including contributing to maintaining the highest level of data quality and documentation. Therefore, governance is no longer the sole preserve of a few, but becomes everyone’s business.

To achieve this mission, the Actian Data Intelligence Platform connects and federates teams around data through a common language. Our data catalog allows anyone – with the allotted allowances – to discover and trust in an enterprise’s data assets.

Are You a Chief Data Officer Looking for a Data Governance Tool?

In order for Chief Data Officers achieve their objectives, they need to be equipped with the right tools. With the Actian Data Intelligence Platform’s data catalog, CDOs can identify their data assets, make them accessible and usable by their collaborators in order to be valorized.

Easy to use and intuitive, our data catalog is the CDO’s indispensable tool for implementing agile data governance. Contact us for more information.

Blog | Data Management | June 13, 2019 | 4 min read

Delivering the Right Data to the Right User in the Right Format

By Actian Corporation

#Data #Data Architect #Data Scientist
data management image

By Actian Corporation

#Data #Data Architect #Data Scientist

The data consumers within your organization aren’t a homogeneous group, so you must adapt your data management solutions accordingly. At one end of the spectrum is the ideal, “one-size-fits-all” solution. At the other end is a customized set of capabilities for each individual user that seems ideal, but is unworkable. How then do you incrementally approach the ideal solution?

The most effective data solutions are formed by generalizing your user community into a few key groups by skillset and role objectives, data consumption habits and applied tools associated with those data needs, and then deploying the right set of solutions for each group. While each company may be a little different, the following user segments are where to start for most companies.

Data Professionals – These are your data scientists, data analysts and statisticians with specialized training in how to manage and analyze data. They typically need access to detailed raw data and a set of robust capabilities to aggregate, analyze and interpret it. They may employ newer capabilities, such as machine learning and AI, to identify trends and make projections. It’s imperative they are able to ingest text, time-series and structured and unstructured data through large-scale data ingestion, for example, Spark; leverage their favorite programming language APIs, for example, C++ or Python, favorite IDEs, such as Jupyter, and algorithm libraries, such as R-Lib and Tensorflow, as well as visualization tools, such as Tableau.

Business Analysts – These are the business users with data-intensive job roles, such as finance, operations planning, HR and administration. Business analysts typically don’t need access to raw data, but instead data feeds for the specific data sets with which they are working. They typically use analytics tools, such as Microsoft Excel, Tableau, Qlik Sense and Power BI, to sort, categorize and summarize data and provide insights to inform business decisions related to business operations.

Data Curators – These are the people within your organization who generate reports and dashboards for executive consumption. Like business analysts, they typically leverage data feeds and the tools they use are often focused on understanding data quality; managing control thresholds; and creating data visualizations, dashboards and reports.  They are seen as closer to the business and program execution.

Executives – The leaders and decision-makers in your organization are focused on understanding the big picture, so they can direct the organization’s resources effectively. They often have little experience creating, analyzing and curating data, but have extensive experience interpreting data, graphs and reports to determine their relevance to and impact on your organization. Recently, many executives have shifted from using tools, such as flip-charts and PowerPoint slides with embedded Excel Spreadsheets, to interactive dashboards as their primary means of consuming and interacting with company data.

Deploy the Right Solution for Each Target Audience

Each of your target-user communities will have a unique set of solution needs. It is important to keep in mind that a data solution must include both the data at scale and with extensive variations as well as the tools for consuming and working with the data. Since many of the user communities will interact with each other, forming a data value chain that transforms raw and stages of processed data into actionable business insights, it is important, therefore, the tools and data solutions interact well with each other across the entire organization.

For example, executives must be able to consume easily the trends and projections your data scientists generate and the analytics your business teams create must be aggregated into enterprise dashboards. Based on executive feedback and direction, those algorithms must then be deployed downstream at the point of action embedded in and aligned with the business operations to streamline and improve them. You can learn more about the range of Actian data management solutions here.

Blog | Data Management | June 11, 2019 | 4 min read

Democratization of Data for Users of Varied Skill Levels and Charters

By Actian Corporation

#Data #Data Architect #Data Engineer #Data Scientist
Making data usable

By Actian Corporation

#Data #Data Architect #Data Engineer #Data Scientist

If you’re a data engineer or a data architect, then you’re probably kept awake at night wondering how to design your data integration, management and analytics support platforms, so your DBAs and IT Ops colleagues can easily manage them while a varied set of data users are able to consume them simultaneously. These users range from new, demanding, hands-on users, such as developers and data scientists, to those who traditionally use the data through SQL and ad hoc queries, such as business analysts, as well as those who interact with the data indirectly through business applications.

Not everyone in your company is a data scientist and, given how scarce they are, you’d be in quite a small company if they were the majority of staff members. At the risk of over-generalizing the role of data scientists, they tend to need data that supports designing and training algorithms that can be deployed downstream, embedded in other applications and used by other users. Data scientists often need large and varied sets of data, but it seldom needs to be real-time, yet freshness is a paramount requirement as they iterate heuristic training of their models.

Application developers, like data scientists tend to interact with their data through programming APIs. The data sets on which they operate tend to be smaller, or time-series and real-time, embedded directly in the business process instead of informing it as is often the case with what a data scientist is doing. For business analysts, the needs are yet again different and for end-users, the point is to make the data invisible to their operations – even if it’s integral and essential to those operations. The point here is that designers of data systems must be able to make data available, but to several different factions that don’t have the same skill sets, roles and responsibilities or interest levels when it comes to data.

What mandate does this place on data engineers or data architects? Simple. Make data usable for people of varied skill levels to consume what they need, when they need it and in ways that are most useful. Okay, maybe not so simple. How do you avoid siloed sets of data, managed by bespoke systems if you narrowly cater to each of these constituencies?

Understand Your User Community and How it is Using Data

Everyone within your company has a unique set of data needs, both in terms of the type of data and tools he or she needs to use and how this data use is deemed effective. You may have some users who need access to a very specific dataset to perform a focused job task while other users may need big-picture data for planning and strategic decision-making, for example. Some of your users will need detailed raw data, while others need curated dashboards, reports and visualizations. In many cases, the same user may fit into each of the scenarios above, but during different phases of a project. In other cases, these different scenarios leverage the same data in different forms or manipulated in different ways and in combination with other sets of data.

Understanding how to make your data users successful is a function of understanding consumers’ skill levels and the tools and datasets they will need. For example, in addition to the data sets referenced above, data scientists tend to spend much time preparing data and hand-coding algorithms or using libraries for AI and ML, such as TensorFlow. Conversely, business analysts are more inclined to leverage SQL for reporting and popular BI and Visualization tools on those queried datasets. Power users on the business side may be able to handle simpler queries, but are most comfortable manipulating spreadsheets, such as your finance and planning staff. Each of these users has a unique set of needs not just for the data, but also for the tools that actually define how they leverage data to do their jobs. You can learn more about the range of Actian data management solutions here.

Blog | Data Integration | June 10, 2019 | 4 min read

Data Management Trends and How They Affect Integration

By Actian Corporation

#Cloud #Cloud Data Warehouse #Data #Data Management #Data Trends #Data Warehouse
Data management Trends

By Actian Corporation

#Cloud #Cloud Data Warehouse #Data #Data Management #Data Trends #Data Warehouse
In this post:

Many of the trends of the past few years have successfully made the transition from emerging concepts to mainstream solution approaches. While trends are important building blocks of how companies approach their data management today, they are also providing insights into future capabilities to incorporate the individual pieces into a holistic, integrated solution.

The Cloud is Becoming the First Choice for New IT Systems

Companies have been moving applications and IT components to the cloud for a few years to save costs, improve performance and achieve solution scalability. During 2019, new developments are emerging in the cloud migration trend where most companies are looking to cloud solutions first for new IT systems and only considering on-premises solutions if the cloud isn’t feasible for some reason. This includes both ready-to-use SaaS solutions as well as cloud-based infrastructure (IaaS and Paas) for various needs, such as data warehouses and in-house developed applications. One of the latest cloud trends is integration-platform-as-a-service (IPaaS), which is a set of cloud-based capabilities for integrating both cloud-native and on-premises components for seamless interoperability.

Data Warehouse Migration to the Cloud

During the past few years, Hadoop has been the big trend in data warehouses in the cloud. Many companies have implemented Hadoop on-premises and are now facing increasing operational costs as well as challenges to integrating with cloud-native solutions. The trend of 2019 is for companies to migrate data warehouses to the cloud where they can be operated at a lower cost and with greater performance and scalability. As this migration is occurring, many companies are examining their integration strategies and capabilities to evaluate how to bring more SaaS data sources and streaming data from IoT systems into the data warehouse for analysis.

Data Lake Reboot

The unstructured nature of data lakes made them all the rage during the past 5 years as business users and others marveled at the flexibility of being free from the constraints of relational data structures. Earlier (and small-scale) data lakes seemed like the perfect solution for organizations seeking agility. What companies learned though is sustainable competitive advantage requires some level of structure and their data lakes were quickly devolving into chaos. During 2019, many companies are re-launching their data lakes, adding data governance and support for relational data structures as a means of supporting agility, yet enabling enterprise-scale analytics and the sustainability of data solutions.

The Data Catalog and Metadata to Drive Consumption

One of the biggest pieces of feedback from data users (both data professionals and business consumers) is that they know the data they seek exists, but as enterprise data increases, it is becoming increasingly difficult to find that data. This is driving an increased investment in data catalogs and metadata that not only provides traditional content tagging, but also includes, for example, data quality, age and trustworthiness scores. The catalog and metadata being collected is being used to drive search capabilities as well as new AI-enabled capabilities for data correlation and advanced analysis.

IoT as the Next Wave of Big Data

Big data is no longer an emerging challenge – it is an operational reality in most companies. The new development in the big-data space is the origination of the data. More companies are deploying IoT devices, mobile apps and embedded sensors within machinery that (instead of providing large aggregated data sets) are generating large volumes of independent data streams that must be managed and reconciled. The challenge companies are facing is managing the connections between each of these independent data sources and a scalable way of integrating the data in the data warehouse. This is one of the key use-cases for IPaaS.

Hybrid Data Management

On-premises systems are not disappearing any time soon, so companies must determine how to make their on-premises and cloud data co-exist and interoperate. Many companies are addressing this by keeping their core, report-oriented data warehouses on-premises to avoid impacts to end-users while moving data staging, special data for analytic, and other pieces to a cloud-based solution. During 2019, there will be a large focus on how to manage the integrations both between different components of the data management solution as well as the interfaces with data sources and consuming applications.

Each of these trends provides a building block that will form the foundation of companies’ next generation of hybrid data management solutions. Now that individual facets of the picture have matured, companies are shifting their focus to data and solution integration, leveraging options, such as the Actian DataConnect IPaaS solution, and advanced metadata and catalog techniques to make enterprise data more integrated and more accessible to users. To learn more, visit DataConnect.

Blog | Data Intelligence | June 10, 2019 | 4 min read

How Does Data Democracy Strengthen Agile Data Governance?

By Actian Corporation

#Data Governance
data democracy white paper

By Actian Corporation

#Data Governance
In this post:

In 2018, we published our first Whitepaper, “Why Start an Agile Data Governance?”. Our goal was to present a pragmatic approach to the attributes of such data governance, one that is capable of rising to the challenges of this new age of information:

We advocate for it to be bottom-up, non-invasive, automated, and iterative. In a word, agile.

In this second edition, we decided to tackle the organization of this new agile data governance and its scaling process using the same mindset.

We believe that what distinguishes Web Giants in their approach to their data isn’t the structure of their governance but the culture that irrigates and animates their organization. This culture has a name: Data Democracy.

Assessing Data Governance

Very centralized, and sometimes bureaucratic, they focus on data control and conformity, often resulting in limiting data access among all company employees.

The Concept of a Data Democracy

To understand what Data Democracy is, it is important to know that it is not a governance model. Data Democracy refers to a corporate culture, an open model where liberty goes hand in hand with responsibility.

Data Democracy’s main objective is to make a company’s data widely accessible to the greatest number of people, if not to all. In practice, every employee is able to pull data values at any level.

A democratic approach presents an interesting challenge to balance: on the one hand, you must ensure that the right to use data can truly be exercised, and on the other hand you must counterbalance this right with a certain number of duties.

Building a Data Democracy

The adoption of a data culture can only work if everyone benefits, hence the importance of communication previously mentioned when discussing rights and responsibilities. The balance between the two must be positive in the end, and governance must not introduce more restrictions than gains. Finally, the results must be made easier.

To enable everyone to find the necessary information. That is the main objective of a data catalog, which must, even more so than its basic function (referencing data and associated metadata), offer simplicity of use in order to navigate through an ocean of information.

The New Roles of Agile Data Governance

Under the pressure of digital transformation, new roles appear within large companies.

The Chief Data Officer: The Data Democracy Sponsor

Among them, there is the Chief Data Officer, or CDO. They are in charge of improving efficiency and the capacity to create value for the information ecosystem of their organization.

With the exponential development of data, the role of the CDO took on a new scope.

From now on, CDOs must reconsider the organization in a cross-functional and globalizing way, and governance and corporate data management technology in enterprises.

They must become the new leaders in “Data Democracy” within companies and must respond to the call of numerous “data citizens” who have understood that the way in which data is processed must change radically. The new CDOs must break the bonds of data silos.

Are we all Data Stewards?

The concept of Data Stewardship stems from a much more traditional model. The organizations that already have Data Stewards tend to be quite large and established.

Everyone who uses sensitive data engages their responsibility regarding the way they use it. The regulations for the protection of sensitive data – regulatory or internal – must be applied in the same manner for all those who enter contact with it.

This dedication to involving everyone helps distribute responsibility for data, giving a broader sense of ownership, which encourages users to explore data themselves, and lastly decompartmentalizes data.

Blog | Data Integration | June 7, 2019 | 3 min read

Actian DataConnect v11.5 is Here!

By Actian Corporation

#DataConnect
Actian DataConnect v11.5

By Actian Corporation

#DataConnect

It’s with great excitement that we announce the release of Actian DataConnect 11.5.

This release includes a new set of robust and intuitive features to make the integration and design experience simpler and faster. From mapping and process design enhancements in DataConnect Studio to new and improved connectivity tools in UniversalConnectTM and user-management features in Integration Manager, DataConnect has the capabilities you need to connect anything, anytime, anywhere.

DataConnect Studio

At Actian, we understand how important it is to provide you with the tools to create integrations quickly, manage them as your environment changes and evolve them to keep your organization agile. With continued focus on the integration design experience, DataConnect Studio 11.5 is full of features to improve productivity and simplify the integration design experience.

  • Guided workflows help users quickly leverage components in Process Designer while maintaining visual context, limiting the need to “plan ahead,” and reducing the number of clicks required to configure steps.
  • New automatic debugging support virtually eliminates the need to write your own logging scripts to troubleshoot variables and message objects.
  • The Map Designer interface has been enhanced, so users can be more efficient by allowing them to: configure events and actions from the “simple mapping” view, bulk-edit field properties and receive immediate interactive feedback on expressions.
  • A new Extract editor enables visual parsing and extraction of data elements from semi-structured files. Pre-built Template-process designs also accelerate onboarding for new users as “out-of-box” quick start, best-practice accelerators.

UniversalConnect™

The ecosystem of data sources that companies need to connect continues to expand. UniversalConnect has been enhanced in this release to support new connectivity and platform features, including:

  • ServiceNow Map connector – A new Map designer connector for mapping to and from ServiceNow objects (including custom objects).
  • NetSuite connector updates – Support the current WSDL version and token-based authentication.
  • SMTP Email invoker enhancements – Support HTML content, recipient lists and multiple attachments.
  • ZEN, Vector and Vector Delimited file connectors have also been updated.

Integration Manager

Actian DataConnect Integration Manager is a Web application that allows operations and support staff to configure, schedule, execute and monitor easily all deployed integrations from a single pane of glass. Integration Manager aims to create a streamlined and seamless hand-off experience from designers to production users as integration packages are deployed. This release builds on the core capabilities already in place within Integration Manager while enhancing deployment features and updating secure access capabilities.

As your company’s data environment continues to evolve, Actian is here to provide you with the tools you need to bring your enterprise data together to achieve true enterprise insights and agility. The latest release of Actian DataConnect includes an enhanced set of tools to help your integration designers be more productive and efficient when managing data connections across the enterprise. To learn more, visit DataConnect.