Data Management

The Consequences of Poor Data Quality: Uncovering the Hidden Risks

Actian Corporation

June 23, 2024

Costly Consequences of Poor Data Quality

Summary

Poor data quality quietly drains millions in revenue, productivity, and trust. This blog outlines the hidden financial, operational, and compliance risks that stem from inaccurate or incomplete data.

The average business loses $15 million annually due to poor data quality; in the U.S., this impact reaches $3.1 trillion across the economy.
Employees spend up to 27% of their time correcting bad data, slowing decision-making, and increasing operational costs.
Poor data undermines compliance efforts, damages brand reputation, and leads to missed market opportunities.

The quality of an organization’s data has become a critical determinant of its success. Accurate, complete, and consistent data is the foundation upon which crucial decisions, strategic planning, and operational efficiency are built. However, the reality is that it is a pervasive issue, with far-reaching implications that often go unnoticed or underestimated.

Defining Poor Data Quality

Before delving into the impacts of poor data quality, it’s essential to understand what constitutes subpar data. Inaccurate, incomplete, duplicated, or inconsistently formatted information can all be considered poor data quality. This can stem from various sources, such as data integration challenges, data capture inconsistencies, data migration pitfalls, data decay, and data duplication.

The Hidden Costs of Poor Data Quality

Loss of Revenue
Poor data quality can directly impact a business’s bottom line. Inaccurate customer information, misleading product details, and incorrect order processing can lead to lost sales, decreased customer satisfaction, and damaged brand reputation. Gartner estimates that poor data quality costs organizations an average of $15 million per year.
Reduced Operational Efficiency
When employees waste time manually correcting data errors or searching for accurate information, it significantly reduces their productivity and the overall efficiency of business processes. This can lead to delayed decision-making, missed deadlines, and increased operational costs.
Flawed Analytics and Decision-Making
Data analysis and predictive models are only as reliable as the data they are based on. Incomplete, duplicated, or inaccurate data can result in skewed insights, leading to poor strategic decisions that can have far-reaching consequences for the organization.
Compliance Risks
Stringent data privacy regulations, such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), require organizations to maintain accurate and up-to-date personal data. Failure to comply with these regulations can result in hefty fines and reputational damage.
Missed Opportunities
Poor data quality can prevent organizations from identifying market trends, understanding customer preferences, and capitalizing on new product or service opportunities. This can allow competitors with better data management practices to gain a competitive edge.
Reputational Damage
Customers are increasingly conscious of how organizations handle their personal data. Incidents of data breaches, incorrect product information, or poor customer experiences can quickly erode trust and damage a company’s reputation, which can be challenging to rebuild.

Measuring the Financial Impact of Poor Data Quality

Annual Financial Loss: Organizations face an average annual loss of $15 million due to poor data quality. This includes direct costs like lost revenue and indirect costs such as inefficiencies and missed opportunities (Data Ladder).
GDP Impact: Poor data quality costs the US economy approximately $3.1 trillion per year. This staggering figure reflects the extensive nature of the issue across various sectors, highlighting the pervasive economic burden (Experian Data Quality) (Anodot).
Time Wasted: Employees can waste up to 27% of their time dealing with data issues. This includes time spent validating, correcting, or searching for accurate data, significantly reducing overall productivity (Anodot).
Missed Opportunities: Businesses can miss out on 45% of potential leads due to poor data quality, including duplicate data, invalid formatting, and other errors that hinder effective customer relationship management and sales efforts (Data Ladder).
Audit and Compliance Costs: Companies may need to spend an additional $20,000 annually on staff time to address increased audit demands caused by poor data quality. This highlights the extra operational costs that come with maintaining compliance and accuracy in financial reporting (CamSpark).

Strategies for Improving Data Quality

Addressing poor data quality requires a multi-faceted approach encompassing organizational culture, data governance, and technological solutions.

Fostering a Data-Driven Culture
Developing a workplace culture that prioritizes data quality is essential. This involves establishing clear data management policies, standardizing data formats, and assigning data ownership responsibilities to ensure accountability.
Implementing Robust Data Governance
Regularly auditing data quality, cleaning and deduplicating datasets, and maintaining data currency are crucial to maintaining high-quality data. Automated data quality monitoring and validation tools can greatly enhance these processes.
Leveraging Data Quality Solutions
Investing in specialized data quality software can automate data profiling, cleansing, matching, and deduplication tasks, significantly reducing the manual effort required to maintain data integrity.

The risks and costs associated with poor data quality are far-reaching and often underestimated. By recognizing the hidden impacts, quantifying the financial implications, and implementing comprehensive data quality strategies, organizations can unlock the true value of their data and position themselves for long-term success in the digital age.

About Actian Corporation

Actian empowers enterprises to confidently manage and govern data at scale. Actian data intelligence solutions help streamline complex data environments and accelerate the delivery of AI-ready data. Designed to be flexible, Actian solutions integrate seamlessly and perform reliably across on-premises, cloud, and hybrid environments. Learn more about Actian, the data division of HCLSoftware, at actian.com.

Data Integration

Introducing Actian’s Enhanced Data Quality Solutions

Actian Corporation

June 18, 2024

Introducing Actian's enhanced data quality solutions

We are pleased to announce that data profiling is now available as part of the Actian Data Platform. This is the first of many upcoming enhancements to make it easy for organizations to connect, manage, and analyze data. With the introduction of data profiling, users can load data into the platform and identify focus areas, such as duplicates, missing values, and non-standard formats, to improve data quality before it reaches its target destination.

Why Data Quality Matters

Data quality is the cornerstone of effective data integration and management. High-quality data enhances business intelligence, improves operational efficiency, and fosters better customer relationships. Poor data quality, on the other hand, can result in costly errors, compliance issues, and loss of trust.

Key Features of Actian’s Enhanced Data Quality Solutions

Advanced Data Profiling
Our advanced data profiling tools provide deep insights into your data’s structure, content, and quality. You can quickly identify anomalies, inconsistencies, and errors by analyzing your data sources and leveraging pre-defined rule sets to detect data problems. Users can also create rules based on the use case to ensure data is clean, correct, and ready for use.
Data Cleansing and Enrichment
Actian’s data cleansing and enrichment capabilities ensure your data is accurate, complete, and up-to-date. Our automated processes isolate data that does not meet quality standards so data teams can act before data is moved to its target environment.
Data Quality Monitoring
With real-time data quality monitoring, you can continuously assess the health of your data. Our solution provides ongoing validation, enabling you to monitor deviations from predefined quality standards. This continuous oversight helps you maintain data integrity for operational and analytics use.
Flexible Integration Options
Actian’s data quality solutions seamlessly integrate with various data sources and platforms. Whether you’re working with on-premises databases, cloud-based applications, or hybrid environments, our tools can connect, cleanse, and harmonize your data across all systems.
User-Friendly Interface and Dashboards
Our intuitive interface makes managing data quality tasks easy for users of all skill levels. Detailed reporting and dashboards provide clear visibility into data quality metrics, enabling you to track improvements and demonstrate compliance with data governance policies.

Transform Your Data into a Strategic Asset

Actian’s enhanced Data Quality solutions empower you to transform raw data into a strategic asset. Ensuring your data is accurate, reliable, and actionable can drive better business outcomes and gain a competitive edge.

Get Started Today

Don’t let poor data quality hold your business back. Discover how Actian’s enhanced Data Quality solutions can help you achieve your data management goals. Visit our Data Quality page to learn more and request a demo.

Stay Connected

Follow us on social media and subscribe to our newsletter for the latest updates, tips, and success stories about data quality and other Actian solutions.

About Actian Corporation

Data Management

Actian Zen 16.0 Introduces New Data Sync Utility

Emma McGrattan

June 17, 2024

We are thrilled to announce the general availability of Actian Zen 16.0, delivering up to 50% faster query processing, flexible cloud deployment options, improved developer productivity, and a new data synchronization utility called EasySync.

More than 13,000 organizations across the globe trust Actian Zen as their embedded edge database for making fast, confident decisions. With this release, Actian is committed to helping businesses simplify edge-to-cloud data management, boost developer productivity, and build secure, distributed IoT apps.

Actian Zen’s latest release solidifies its position as the go-to database for building low-latency embedded applications. These applications enable real-time data access, optimize operations, and deliver valuable insights faster than ever before. The Zen 16.0 release helps embedded edge developers bring more efficiency at the edge with the following capabilities:

Faster query processing—up to 50% faster.
Flexible deployment options using Zen Container SDK.
Improved developer experiences with a Btrieve 2 Python package.
Real-time data streaming with Zen and Apache Kafka.
Simplified data movement and synchronization using Zen EasySync.

Curious how the new capabilities can help? Let an Actian representative show you!

Let’s dive into the ways Zen 16.0 empowers users with the new capabilities.

Execute Queries Up to 50% Faster With Actian Zen

You can run faster, smarter applications on edge devices with Zen 16.0. Zen accesses frequently used data that’s stored in the L2 cache, speeding up results for queries using this data. Common queries, such as those for frequently used reports or analysis, will experience significantly faster results.

Another technique boosting query performance is page read-ahead, which makes it much faster to scan large data files. When a query is executed, the Zen MicroKernel engine anticipates the data and preloads pages from the data file into memory. This optimization mechanism allows the database engine to not read from the disk as often, enabling faster results.

Having ultra-fast data retrieval is perfect for applications requiring immediate insights from edge devices. This capability ensures real-time analytics and decision-making, enhancing the overall efficiency and responsiveness of your operations. For example, Tsubakimoto Chain Company, a global machinery manufacturer, relies on Actian Zen as the embedded database, sorting up to 10,000 items per hour on their high-speed material handling systems.

Deploy Your Way With Zen Container SDK

With containerization, developers can quickly set up and use Actian Zen, running in Docker containers, with Kubernetes orchestration and Helm Chart configuration. This makes deployment and management across various environments, including on-premises, cloud, and hybrid, much easier.
The containerization of Zen supports ARM 32 and ARM 64 processors for wider deployment options. The ARM architecture is increasingly prevalent in various devices, from smartphones to Internet of Things (IoT) gadgets. Container support for ARM allows developers to target a broader range of platforms with their applications.

Elevate Developer Experiences Leveraging a Btrieve 2 Python Package

The Btrieve2 Python SDK has gained popularity within the Python community. With this release, developers can now leverage the performance and flexibility of Btrieve databases from Python using the Btrieve2 Python package:

Simplified Btrieve integration. The Btrieve2 Python package streamlines the process of working with Btrieve databases from Python applications. Developers can leverage familiar Python syntax for database operations, reducing the learning curve and development time.
Broader developer reach. Availability on PyPI makes the Btrieve2 package easily discoverable and installable using the familiar pip command. This expands the potential user base for Btrieve-compatible applications.
Simplified distribution and management. PyPI provides a centralized repository for package distribution and version management. You can easily share and update your Btrieve2 package, ensuring users have access to the latest version.

Zen 16.0 also boosts developer productivity with features such as LIKE with ESCAPE syntax and literal matching for concise, readable queries. Additionally, Zen now supports JSON nested-object queries to simplify data retrieval from JSON formats, allowing developers to focus on core logic and accelerate development cycles. Lastly, SQL query logging improves performance debugging effectiveness by revealing database interactions, aiding in identifying bottlenecks and optimizing query performance.

Enable Real-Time Data Streaming With Zen and Apache Kafka

Real-time data streaming – particularly in Kafka – is a popular method for moving data from the edge to the cloud, and vice versa. Zen support for Kafka allows you to benefit from streaming-based edge applications.

Combining Zen replication features with Apache Kafka can create a real-time data pipeline. Zen acts as the source database, replicating changes to a secondary database for analytical workloads. Kafka serves as a high-throughput messaging system, efficiently streaming data updates to analytics engines for immediate processing and insights.

Zen’s support for Kafka also allows you to build apps for real-time data processing. This is crucial for scenarios requiring immediate responses to data updates such as fraud detection or sensor data analysis.

Move and Sync Data Easier Using Zen EasySync

A pre-built data synchronization utility called EasySync saves time and effort compared to custom replication logic, allowing you to focus on core application functionality. EasySync lets you move and copy data easier than ever:

Data consistency and availability. Zen offers a robust replication mechanism for ensuring data is kept synchronized across multiple servers or geographically dispersed locations. This minimizes downtime and data loss risk in cases of hardware failures, network outages, or planned maintenance.
Reduced development complexity. By providing a pre-built data synchronization solution, Actian Zen saves you time and effort compared to implementing custom replication logic from scratch or paying for a separate data sync solution. This allows you to focus on core application functionality.

In Industrial IoT (IIOT) environments, the ability to replicate data to the database from handheld devices without requiring a gateway and without creating new code opens new use cases and opportunities. This enables real-time data collection and faster decision-making for process control, remote monitoring, and field service.

Drive Better Outcomes at the Edge

Zen simplifies edge-to-cloud data management with secure, scalable storage and seamless cloud synchronization. We listened to customer feedback and looked at market trends to ensure Zen continues to deliver new and sustainable value for your IoT and edge devices.
For example, you asked us to create longer index keys with more descriptive names. We delivered with index keys longer than 255 characters, enabling you to create more granular indexes that target the data needed for specific queries. You benefit from improved query speed, especially for complex searches or filtering, while being able to create data models with more expressive and descriptive field names to improve code readability and maintainability.

You can use Zen Mobile, Zen Edge, and Zen Enterprise to support modernization efforts, optimize embedded apps, and simplify edge-to-cloud data management. The surge in data from IoT and edge devices, alongside rapidly growing data volumes, makes extracting actionable insights a key differentiator.

Empower your team to achieve embedded edge intelligence with Zen 16.0. Packed with productivity-boosting features and flexible deployment options, Zen 16.0 helps you build the future of IoT.

Get started today!

About Emma McGrattan

Emma McGrattan is CTO at Actian, leading global R&D in high-performance analytics, data management, and integration. With over two decades at Actian, Emma holds multiple patents in data technologies and has been instrumental in driving innovation for mission-critical applications. She is a recognized authority, frequently speaking at industry conferences like Strata Data, and she's published technical papers on modern analytics. In her Actian blog posts, Emma tackles performance optimization, hybrid cloud architectures, and advanced analytics strategies. Explore her top articles to unlock data-driven success.

Data Intelligence

Data Shopping Part 1 – How to Shop for Data Products

Actian Corporation

June 17, 2024

Businessman using a computer to document management concept, online documentation database and digital file storage system or software, records keeping, database technology, file access, doc sharing.

Just as shopping for goods online involves selecting items, adding them to a cart, and choosing delivery and payment options, the process of acquiring data within organizations has evolved in a similar manner. In the age of data products and data mesh, internal data marketplaces enable business users to search for, discover, and access data for their use cases.

In this series of articles, get an excerpt from our Practical Guide to Data Mesh and discover all there is to know about data shopping as well as the Actian Data Intelligence Platform’s Data Shopping experience in its Enterprise Data Marketplace:

How to shop for data products.
The Data Shopping experience.

As mentioned above, all classic marketplaces offer a very similar “checkout” experience, which is familiar to many people. The selected products are placed in a cart, and then, when validating the cart, the buyer is presented with various delivery and payment options.

The actual delivery is usually done outside the marketplace, providing tracking functionalities. Delivery can be immediate (for digital products) or deferred (for physical products). Some marketplaces have their own logistics system, but most of the time, delivery is the responsibility of the seller. The delivery time is an important element of customer satisfaction – the shorter it is, the more satisfied users are.

How does this shopping experience translate into an Enterprise Data Marketplace? To answer this question, we need to consider what data delivery means in a business context and, for that, focus on the data consumer.

The Delivery of Data Products

A data product offers one or more consumption protocols – these are its outbound ports. These protocols may vary from one data product to another, depending on the nature of the data – real-time data, for example, may offer a streaming protocol, while more static data may offer an SQL interface (and instructions for using this interface from various programming languages or in-house visualization tools).

For interactive consumption needs, such as in an application, the data product may also offer consumption APIs, which in turn may adhere to a standard (REST, GraphQL, OData, etc.). Or simply download the data in a file format.

Some consumers may integrate the data product into their own pipelines to build other data products or higher-level uses. Others may simply consume the data once, for example, to train an ML model. It is up to them to choose the protocol best suited to their use case.

Whatever protocols are chosen, they all have one essential characteristic: they are secure. This is one of the universal rules of governance – access to data must be controlled, and access rights supervised.

With few exceptions, the act of purchase therefore simply involves gaining access to the data via one of the consumption protocols.

Access Rights Management for Data Products

However, in the world of data, access management is not a simple matter, and for one elementary reason: consuming data is a risky act.

Some data products can be desensitized – somehow removing personal or sensitive data that poses the greatest risk. But this desensitization cannot be applied to the entire product portfolio: otherwise, the organization forfeits the opportunity to leverage data that is nonetheless highly valuable (such as sensitive financial or HR data, commercial data, market data, customer personal data, etc.). In one way or another, access control is therefore a critical activity for the development and widespread adoption of the data mesh.

In the logic of decentralization of the data mesh, risk assessment and granting access tokens should be carried out by the owner of the data product, who ensures its governance and compliance. This involves not only approving the access request but also determining any data transformations needed to conform to a particular use. This activity is known as policy enforcement.

Evaluating an access request involves analyzing three dimensions:

The data themselves (some carry more risk than others) – the what.
The requester, their role, and their location (geographical aspects can have a strong impact, especially at the regulatory level) – the who.
The purpose – the why.

Based on this analysis, the data may be consumed as is, or they may require transformation before delivery (data filtering, especially for data not covered by consent, anonymization of certain columns, obfuscation of others, etc.). Sometimes, additional formalities may need to be completed – for example, joining a redistribution contract for data acquired from a third party, or complying with retention and right-to-forget policies, etc.

Technically, data delivery can take various forms depending on the technologies and protocols used to expose them.

For less sensitive data, simply granting read-only access may suffice – this involves simply declaring an additional user. For sensitive data, fine-grained permission control is necessary, at the column and row levels. Most modern data platforms support native mechanisms to apply complex access rules through simple configuration – usually using data tags and a policy enforcement engine. Setting up access rights involves creating the appropriate policy or integrating a new consumer into an existing policy. For older technologies that do not support sufficiently granular access control, it may be necessary to create a specific pipeline to transform the data to ensure compliance, store them in a dedicated space, and grant the consumer access to that space.

This is, of course, a lengthy and potentially costly approach, which can be optimized by migrating to a data platform supporting a more granular security model or by investing in a third-party policy enforcement solution that supports the existing platform.

Data Shopping in an Internal Data Marketplace

In the end, in a data marketplace, data delivery, which is at the heart of the consumer experience, translates into a more or less complex workflow, but its main stages are as follows:

The consumer submits an access request – describing precisely their intended use of the data.
The data owner evaluates this request – in some cases, they may rely on risk or regulatory experts or require additional validations – and determines the required access rules.
An engineer in the domain or in the “Infra & Tooling” team sets up the access – this operation can be more or less complex depending on the technologies used.

Shopping for the consumer involves triggering this workflow from the marketplace.

For the Actian Data Intelligence Platform’s marketplace, we have chosen not to integrate this workflow directly into the solution but rather to interface with external solutions.

In our next article, discover the Actian Data Intelligence Platform Data Shopping experience and the technological choices that set us apart.

About Actian Corporation

Databases

Embracing Database Modernization: Future Proofing Your Business

Actian Corporation

June 12, 2024

Embracing Database Modernization with Actian

Modernizing your database and apps to align with evolving business needs can improve efficiency, security, compliance, and user experiences while reducing costs and enhancing scalability.

Modernizing your IT infrastructure is the process of updating and even transforming your technologies, systems, and processes to better meet your current and future needs. Taking a strategic approach to modernization, including modernizing your database and apps, can deliver a range of benefits that include:

Informed Decision-Making. The ability to integrate all relevant data for transactional processing and timely, accurate insights for better decision-making.
Improved Efficiency. Automation, cloud computing, and advanced analytics can be leveraged to reduce operational costs and improve productivity.
Simplified Compliance. Modern systems are typically better equipped than legacy technologies to meet compliance and regulatory requirements.
Robust Security. Modern IT systems often offer enhanced security features and provide regular updates that protect against threats, helping ensure your data is secure.
Easy Scalability. Modern infrastructures, especially those that are cloud-based, provide immediate scalability to handle changing workloads.
Improved Customer Experiences. State-of-the-art IT systems support chatbots, personalized marketing, and real-time transactions for enhanced customer experiences.
Reduced Costs. Upfront investments to modernize can be significant, but they reduce technical debt and offer longer-term savings due to efficiency, less maintenance, and reduced downtime.

A strategic approach to modernization that aligns with evolving business and IT needs helps ensure you can capture and optimize the right data and make it usable across your organization. The recently enhanced Actian Ingres can play a pivotal role in your modernization journey.

Innovation Demands Modernization

Organizations like yours must have a solid data foundation for driving innovation—and innovate at a pace that allows you to seize trends, meet shifting customer preferences, and offer breakthrough products and features before your competitors do. This requires a high-performance database that delivers trusted, rapid insights without expecting you to use a multitude of different tools.

A modern approach to your database—and your IT infrastructure as a whole—can open new opportunities, such as bringing increased levels of automation, easier system integrations, and the ability to modernize at your pace in the environment you choose, whether it’s on-premises, in the cloud, or a hybrid setting.

The problem with many legacy systems is that they can’t easily integrate with other systems, making it difficult to seamlessly share data, are not scalable to handle growing data volumes, and require IT help to add data pipelines and utilize the data. All of these issues create barriers to rapid insights and limit your ability to take a data-driven approach to innovation, decision-making, personalized customer engagement, and other business areas.

The linchpin for success ultimately comes down to your approach to data. And it’s why a modern database that supports better management and utilization of data—without ongoing IT help—is required.

Benefits of Database Modernization

A database with modern features and capabilities delivers benefits such as fast data querying, high levels of efficiency, advanced security, and seamless data manageability without requiring advanced skills. Database modernization can deliver:

Flexible Modernization Paths. Your platform should give you the flexibility and agility to modernize according to your needs. For example, if you choose, you should be able to modernize in-place for better performance and security, or migrate data workloads to the cloud or multiple clouds to meet company mandates for a cloud-first approach to data.
Phased Approach to the Cloud. If you want to move to the cloud, it should be at your pace, allowing you to migrate as you’re ready. This way, you can move data backup and recovery capabilities to the cloud, which is a common cloud use case, but keep other workloads on-premises until you’re ready to move them to the cloud. A phased approach supports a smooth transition with minimal disruption.
Advanced Capabilities. Modernization entails more than upgrading technologies. It encompasses aligning technology with business priorities to enable you to reach desired outcomes faster—and have confidence in the results. A modern database with user-friendly capabilities lets you deliver new value across your organization while fostering a data-driven culture.
Optimized User Experiences. A modern database provides fast, reliable access to data. Features such as automated scaling and easy integration with various applications, along with the ability to support complex queries and large datasets, lead to more engaging user experiences and increased productivity.

Bridging the Skills Gap

Modernization efforts simplify data access and management while reducing the time spent on manual tasks such as wrangling and prepping your data. A successful modernization approach also bridges the skills gap for analysts and other data users by making data easy to access and use. The result is more people throughout the company being able to utilize data, which helps unlock the full potential of your data.

A modern approach to building apps complements a modern database by enabling rapid app development, scalability, and integration with cloud services for increased agility and a faster path to innovation. Modernizing your database in addition to app building processes can help you better predict market changes, shorten the timeframe to market, accelerate data-driven innovation, and maintain a competitive advantage.

Actian offers a solution to deliver modern apps quickly. OpenROAD is a database-centric, object-oriented, 4GL rapid application development tool for developing and deploying mission-critical, n-tier business applications. It simplifies app modernization by letting you reuse your existing business logic, making it much easier to offer modern user interfaces.

Trusted Support for Database Modernization

Modernizing your database and applications delivers myriad benefits, yet you must take an eyes-wide-open approach. Complex interdependencies, your data infrastructure, operating systems, and hardware can pose risks when modernizing, so you must consider how they will be impacted.

Actian offers professional services tailored to modernization needs through our Ingres NeXt Initiative to transform your mission-critical Actian Ingres database and OpenROAD applications into open, extensible platforms while reducing risk and accelerating modernization. The expert support ensures a smooth modernization journey while preserving existing investments.

About Actian Corporation

Data Management

Actian Ingres 12.0: Modernize Your Way – Trusted, Reliable, and Adaptable

Douglas Dailey

June 11, 2024

Our trusted and reliable database delivers performance and flexibility, empowering customers to modernize at their own pace.

As the director of product management for Actian, I’m thrilled to share first-hand insights into the latest enhancements to Actian Ingres. This major release embodies our commitment to customer-driven innovation and reinforces our position as a trusted technology partner.

Actian Ingres 12.0 builds upon the core strengths that have made Ingres a go-to transactional database for decades. We’ve invested heavily in performance, security, and cloud-readiness to ensure it meets customers’ modernization needs.

Actian Ingres offers modernization at your own pace, in a low-risk fashion, using any of the following options:

Choice and Flexibility

This release is all about giving customers the power of choice. Whether you’re committed to on-premises deployments, ready to embrace the cloud, or are looking for a hybrid solution, Actian Ingres 12.0 adapts to your modernization strategy.

We have options for Lift/Shift to VM, containerization via Docker and Kubernetes, and plans for bring your own license (BYOL) on the AWS Marketplace. If customers want to take a phased approach, customers have several options. Customers can move first to Linux on-premises, then to virtual machines (VMs) in the cloud, and finally to containers. We’re here to help and want customers to know we have a cloud story to help them in their journey.

Core Enhancements

We understand that familiarity and reliability are crucial to our users. That’s why Actian Ingres 12.0 strengthens core capabilities alongside exciting new features. We’ve doubled down on investments in these areas to ensure that Ingres remains a database that delivers new and sustainable value; this commitment keeps it relevant for the long term.

Reliability and security are paramount for our customers. Ingres 12.0 strengthens our ability to prevent brute force and Denial of Service (Dos) cyber-attacks, and DBMS security for user privileges to better protect users, roles, and groups.

We’ve added User Defined Function (UDF) support for Python and Javascript, offering a powerful way to extend the functionality of a database or streamline processes. The use of containers offers an isolated execution environment to keep the DBMS secure.

The X100 analytics engine attracts attention for its superior performance where users have seen significant performance gains for OLAP related activities through the use of X100 tables by emphasizing their speed and efficiency.

X100 Analytics Table

Most notably, we introduced table and schema cloning in this release. This translates into a huge savings for warehouse-oriented customers and eliminates overhead for storage and latency without data duplication. Imagine a simple SQL-based table clone command that can clone not just one, but many tables in a single executed statement, and opens new possibilities for future data sharing and analytics down the line.

Cloud Enablement

Cloud adoption can be complex, but we’re here to make the journey smooth. Migrations can be challenging, which is why we provide support every step of the way. Ingres 12.0 is more adaptive to meet current and emerging business challenges while helping customers who want to move to the cloud to do so at their own pace.

This release brings a long-awaited backup to cloud capability for Actian Ingres that appeals to most data protection strategies. For many organizations, the ability to backup and restore data as part of an off-site disaster recovery strategy is their first objective. This type of backup strengthens business continuity.

Users already deploy Ingres on Linux using Docker and leverage Kubernetes to simplify orchestration. With Ingres 12.0 we now support disaster recovery using IngresSync, a DR utility formerly only available through Professional Services. IngresSync allows users to set up a read-only standby server. Yet another reason to have more confidence stepping into the cloud knowing you can distribute workloads and have disaster recovery options.

Performance Matters

Our development team was granted 5 patents with an additional 3 currently pending. This is the type of innovation that helps to differentiate us in areas of performance optimization. These patents touched advances in User Defined Functions (UDFs), index optimization, and continued differentiation with the in-memory storage, tracking, and merging of changes stored in X100 Positional Delta Trees (PDT). This is a tremendous show of passion for perfection by our amazing developers.

We invested in additional performance testing and standardization on industry TPC-H, TPC-DS, and TPC-C benchmarks, making strides release over release, and even more so, when it comes to complex X100 queries. These types of investments uncover various edge cases and costing scenarios that we can improve so users of any workload type can benefit. Of course, mileage varies.

Customers also benefit from more efficient workload management tailored to their specific business needs. Workload Manager 2.0 offers the capability to establish priority-driven queues, enabling resources to be allocated based on predefined priorities and user roles. During peak workload periods, the system can intelligently handle incoming queries by prioritizing specific queues and users, guaranteeing that important tasks are handled promptly while upholding overall system performance and efficiency.

For example, if business leaders require immediate information for a quarterly report, their queries are prioritized accordingly. Conversely, in situations where real-time transactions are crucial, prioritization is adjusted to maintain system efficiency.

Modernize With Confidence

Modernizing applications can be daunting. OpenROAD, a database-centric rapid application development (RAD) tool for developing and deploying business apps, continues to make this process easier with improvements to abf2or and WebGen utilities shipped with the product.

Empowering customers to transform their apps and up-level them for the web and mobile helps them stay current in a rapidly evolving developer space. This area of work can be the most challenging of all but having the ability to convert “green screen” applications to OpenROAD, and then on to web/mobile is a great starting point.

OpenROAD users can expect to see a new gRPC-based architecture for the OpenROAD Server. This architecture helps to reduce administration, enhance concurrency support, and is more lightweight because of its use of HTTP/2 and protocol buffers. Our developers were excited to move forward with this project and see it as a big jump from COM/DCOM.

The new gRPC architecture is also microservices-friendly and able to be packaged into a separate container. Because of this, we’ve got our sights set on containerized deployment of the Server in the cloud. In the meantime, we’ve distributed Docker files with this release so that customers can do some discovery and exploration.

Driven by Customer Feedback

Actian Ingres 12.0 can help customers expand their data capabilities footprint, explore new use cases, and reach their modernization goals faster. We’ve focused on enabling customers to strategically grow their business using a trusted database that keeps pace with new and emerging business needs.

We want customer feedback as we continue to innovate. Many of the database enhancements are based on direct customer input. We talked with users across industries about what features and capabilities customers like, and what customers wanted to see added. Their feedback was incorporated into our product roadmap, which ensures that Ingres continues to meet their evolving requirements. Plus, with our commitment to best-in-class support and services, every customer can be assured that we’re here to help them, no matter where customers are on their modernization journey.

Ingres is more than just a database. It’s a trusted enabler to help customers become future-fit and innovate faster without barriers. Whether you’re up leveling your version to 12.0 for the new capabilities and improvements, migrating to the cloud, modernizing applications, or leveraging built-in X100 capabilities for real-time analytics against co-located transactional data, Ingres 12.0 has something for everyone.

Additional Resources:

Read the Actian Ingres 12.0 Launch blog for more detailed information about the latest release.
Explore Actian Ingres 12.0 in Actian Academy, watch the “What’s New” video.
Read the Actian Ingres 12.0 Press Release.

About Douglas Dailey

Douglas Dailey is Director of Product Management for Actian's cloud and on-prem databases, tools, and connectivity solutions. Over 15 years, Doug has built data virtualization platforms and steered technology investments in IBM Netezza, DB2 Replication, and Informix. He's led sessions at major data events (e.g., IBM's Think conference) and authored Whitepapers on data hub and data fabric topologies. On the Actian blog, Doug focuses on hybrid data strategies, replication, and emerging use cases. Check out his latest articles to understand modern data architectures.

Data Intelligence

Building a Marketplace for Data Mesh: Domain Data Catalogs – Part 3

Actian Corporation

June 10, 2024

person touching a screen to look at data mesh

Over the past decade, data catalogs have emerged as important pillars in the landscape of data-driven initiatives. However, many vendors on the market fall short of expectations with lengthy timelines, complex and costly projects, bureaucratic data governance models, poor user adoption rates, and low-value creation. This discrepancy extends beyond metadata management projects, reflecting a broader failure at the data management level.

Given these shortcomings, a new concept is gaining popularity: the internal marketplace, or what we call the Enterprise Data Marketplace (EDM).

In this series of articles, get an excerpt from our Practical Guide to Data Mesh where we explain the value of internal data marketplaces for data product production and consumption, how an EDM supports data mesh exploitation on a larger scale, and how they go hand-in-hand with a data catalog solution:

Facilitating data product consumption through metadata.
Setting up an enterprise-level marketplace.
Feeding the marketplace via domain-specific data catalogs.

Structuring data management around domains and data products is an organizational transformation that does not change the operational reality of most organizations: data is available in large quantities, from numerous sources, evolves rapidly, and its control is complex.

Data catalogs traditionally serve to inventory all available data and manage a set of metadata to ensure control and establish governance practices.

Data mesh does not eliminate this complexity: it allows certain data, managed as data products, to be distinguished and intended for sharing and use beyond the domain to which they belong. But each domain is also responsible for managing its internal data, which will be used to develop robust and high-value data products – its proprietary data, in other words.

Metadata Management in the Context of an Internal Marketplace fed by Domain-Specific Catalogs

In the data mesh, the need for a Data Catalog does not disappear, quite the contrary: each domain should have a catalog allowing it to efficiently manage its proprietary data, support domain governance, and accelerate the development of robust and high-value data products. Metadata management is thus done at two levels:

At the domain level – in the form of a catalog allowing the documentation and organization of the domain’s data universe. Since the Data Catalog is a proprietary component, it is not necessary for all domains to use the same solution.
At the mesh level – in the form of a marketplace in which the data products shared by all domains are registered; the marketplace is naturally common to all domains.

With a dedicated marketplace component, the general architecture for metadata management is as follows:

data marketplace architecture

In this architecture, each domain has its own catalog – which may rely on a single solution or not – but should be instantiated for each domain to allow it to organize its data most effectively and avoid the pitfalls of a universal metadata organization.

The marketplace is a dedicated component, offering simplified ergonomics, and in which each domain deploys metadata (or even data) for its data products. This approach requires close integration of the different modules:

Domain catalogs must be integrated with the marketplace to avoid duplicating efforts in producing certain metadata – especially lineage, but also data dictionaries (schema), or even business definitions that will be present in both systems.
Domain catalogs potentially need to be integrated with each other – to share/synchronize certain information, primarily the business glossary but also some repositories.

Data Catalog vs. EDM Capabilities

When we look at the respective capabilities of an Enterprise Data Marketplace and a Data Catalog, we realize that these capabilities are very similar:

Data Catalog Vs Enterprise Data Marketplace

In the end, on a strictly functional level, their capabilities are very similar. What distinguishes a modern Data Catalog from an EDM are:

Their scope – The Data Catalog is intended to cover all data, whereas the marketplace is limited to the objects shared by domains (data products and other domain analytics products).
Their user experience – The Data Catalog is often a fairly complex tool, designed to support governance processes globally – it focuses on data stewardship workflows. The marketplace, on the other hand, typically offers very simple ergonomics, heavily inspired by that of an e-commerce platform, and provides an experience centered on consumption – data shopping.

The Practical Guide to Data Mesh: Setting up and Supervising an Enterprise-Wide Data Mesh

Written by Guillaume Bodet, our guide was designed to arm you with practical strategies for implementing data mesh in your organization, helping you:

Start your data mesh journey with a focused pilot project.
Discover efficient methods for scaling up your data mesh.
Acknowledge the pivotal role an internal marketplace plays in facilitating the effective consumption of data products.
Learn how the Actian Data Intelligence Platform emerges as a robust supervision system, orchestrating an enterprise-wide data mesh.

Get the eBook.

About Actian Corporation

AI & ML

Your Company is Ready for GenAI. But is Your Data?

Dee Radh

June 5, 2024

The buzz around Generative AI (GenAI) is palpable, and for good reason. This powerful technology promises to revolutionize how businesses like yours operate, innovate, and engage with customers. From creating compelling marketing content to developing new product designs, the potential applications of GenAI are vast and transformative. But here’s the kicker: to unlock these benefits, your data needs to be in tip-top shape. Yes, your company might be ready for GenAI, but the real question is—are your data and data preparation up to the mark? Let’s delve into why data preparation and quality are the linchpins for GenAI success.

The GenAI Foundation: Data Preparation

Think of GenAI as a master chef. No matter how skilled the chef is, the quality of the dish hinges on the ingredients. In the realm of GenAI, data is the primary ingredient. Just as a chef needs fresh, high-quality ingredients to create a gourmet meal, GenAI needs well-prepared, high-quality data to generate meaningful and accurate outputs.

Garbage In, Garbage Out

There’s a well-known adage in the data world: “Garbage in, garbage out.” This means that if your GenAI models are fed poor-quality data, the insights and outputs they generate will be equally flawed. Data preparation involves cleaning, transforming, and organizing raw data into a format suitable for analysis. This step is crucial for several reasons:

Accuracy

Ensuring data is accurate prevents AI models from learning incorrect patterns or making erroneous predictions.

Consistency

Standardizing data formats and removing duplicates ensure that the AI model’s learning process is not disrupted by inconsistencies.

Completeness

Filling in missing values and ensuring comprehensive data coverage allows AI to make more informed and holistic predictions.

The Keystone: Data Quality

Imagine you’ve meticulously prepared your ingredients, but they’re of subpar quality. The dish, despite all your efforts, will be a disappointment. Similarly, even with excellent data preparation, the quality of your data is paramount. High-quality data is relevant, timely, and trustworthy. Here’s why data quality is non-negotiable for GenAI success:

Relevance

Your GenAI models need data that is pertinent to the task at hand. Irrelevant data can lead to noise and outliers, causing the model to learn patterns that are not useful or, worse, misleading. For example, if you’re developing a GenAI model to create personalized marketing campaigns, data on customer purchase history, preferences, and behavior is crucial. Data on their shoe size? Not so much.

Timeliness

GenAI thrives on the latest data. Outdated information can result in models that are out of sync with current trends and realities. For instance, using last year’s market data to generate this year’s marketing strategies can lead to significant misalignment with the current market demands and changing consumer behavior.

Trustworthiness

Trustworthy data is free from errors and biases. It’s about having confidence that your data reflects the true state of affairs. Biases in data can lead to biased AI models, which can have far-reaching negative consequences. For example, if historical hiring data used to train an AI model contains gender bias, the model might perpetuate these biases in future hiring recommendations.

Real-World Implications

Let’s put this into perspective with some real-world scenarios:

Marketing and Personalization

A retail company leveraging GenAI to create personalized marketing campaigns can see a substantial boost in customer engagement and sales. However, if the customer data is riddled with inaccuracies—wrong contact details, outdated purchase history, or incorrect preferences—the generated content will miss the mark, leading to disengagement and potentially damaging the brand’s reputation.

Product Development

In product development, GenAI can accelerate the creation of innovative designs and prototypes. But if the input data regarding customer needs, market trends, and existing product performance is incomplete or outdated, the resulting designs may not meet current market demands or customer needs, leading to wasted resources and missed opportunities.

Healthcare and Diagnostics

In healthcare, GenAI has the potential to revolutionize diagnostics and personalized treatment plans. However, this requires precise, up-to-date, and comprehensive patient data. Inaccurate or incomplete medical records can lead to incorrect diagnoses and treatment recommendations, posing significant risks to patient health.

The Path Forward: Investing in Data Readiness

To truly harness the power of GenAI, you must prioritize data readiness. Here’s how to get started:

Data Audits

Conduct regular data audits to assess the current state of your data. Identify gaps, inconsistencies, and areas for improvement. This process should be ongoing to ensure continuous data quality and relevance.

Data Governance

Implement robust data governance frameworks that define data standards, policies, and procedures. This ensures that data is managed consistently and remains high-quality across the organization.

Advanced Data Preparation Tools

Leverage advanced data preparation tools that automate the cleaning, transformation, and integration of data. These tools can significantly reduce the time and effort required to prepare data, allowing your team to focus on strategic analysis and decision-making.

Training and Culture

Foster a culture that values data quality and literacy. Train employees on the importance of data integrity and equip them with the skills to handle data effectively. This cultural shift ensures that everyone in the organization understands and contributes to maintaining high data standards.

The Symbiosis of Data and GenAI

GenAI holds immense potential to drive innovation and efficiency across various business domains. However, the success of these initiatives hinges on the quality and preparation of the underlying data. As the saying goes, “A chain is only as strong as its weakest link.” In the context of GenAI, the weakest link is often poor data quality and preparation.

By investing in robust data preparation processes and ensuring high data quality, you can unlock the full potential of GenAI. This symbiosis between data and AI will not only lead to more accurate and meaningful insights but also drive sustainable competitive advantage in the rapidly evolving digital landscape.

So, your company is ready for GenAI. But the million-dollar question remains—is your data?

Download our free GenAI Data Readiness Checklist shared at the Gartner Data & Analytics Summit.

About Dee Radh

As Senior Director of Product Marketing, Dee Radh heads product marketing for Actian. Prior to that, she held senior PMM roles at Talend and Formstack. Dee has spent 100% of her career bringing technology products to market. Her expertise lies in developing strategic narratives and differentiated positioning for GTM effectiveness. In addition to a post-graduate diploma from the University of Toronto, Dee has obtained certifications from Pragmatic Institute, Product Marketing Alliance, and Reforge. Dee is based out of Toronto, Canada.

Databases

Actian Ingres 12.0 Enhances Cloud Flexibility and Offers Faster Analytics

Emma McGrattan

June 4, 2024

Today, we are excited to announce Actian Ingres 12.0*, which is designed to make cloud deployment simpler, enhance security, and deliver up to 20% faster analytics. The first release I worked on was Ingres 6.4/02 back in 1992, and the first bug I fixed was for a major US car manufacturer that used Ingres to drive its production line. It gives me great pride to see that three decades later, Ingres continues to manage some of the world’s most mission-critical data deployments and that there’s so much affection for the Ingres product.

With this release, we’re returning to the much-loved Ingres brand for all platforms. We continue to partner with our customers to understand their evolving business needs and make sure that we deliver products that enable their modernization journey. With this new release, we focused on the following capabilities:

Backup to cloud and disaster recovery. Ingres 12.0 greatly simplifies these configurations for both on-premises and cloud deployments through the use of Virtual Machines (VMs) or Docker containers in Kubernetes.
Fortified protection automatically enables AES-256 encryption and hardened security to defend against brute force and Denial of Service (DoS) attacks.
Improved performance and workload management with up to 20% faster analytical queries using the X100 engine. Workload Manager 2.0 provides greater flexibility in the allocation of resources to meet specific user demand.
Elevated developer experiences in OpenROAD 12. We make it quick and easy to create and transform database-centric applications for web and mobile environments.

These new capabilities, coupled with our previous enhancements to cloud deployment, are designed to help our customers deliver on their modernization goals. They reflect Actian’s vision to develop solutions that our customers can trust, are flexible to meet their specific needs, and are easy to use so they can thrive when uncertainty is the only certainty they can plan for.

Customers like Lufthansa Systems rely on Actian Ingres to power their Lido flight and route planning software. “It’s very reassuring to know that our solution, which keeps airplanes and passengers safe, is backed up by a database that has for so many years been playing in the ‘premier league’,” said Rudi Koffer, Senior Database Software Architect at the Lufthansa Systems Airlines Operations Solutions division in Frankfurt Raunheim, Germany.

Experience the new capabilities first-hand. Connect with an Actian representative to get started. Below we dive into what each capability delivers.

A Database Built for Your Modernization Journey

Backup to Cloud and Disaster Recovery

Most businesses today have 24×7 data operations, so a system outage can have serious consequences. With Ingres 12.0 we’ve added new backup functionality to cloud and disaster recovery capabilities to dramatically reduce the risk of application downtime and data loss with a new component called IngresSync. IngresSync makes copies of a database to a target location for offsite storage and quick restoration.

Disaster recovery is now Docker or Kubernetes container-ready for Ingres 12.0 customers, allowing users to set up a read-only standby server in their Kubernetes deployment. Recovery Point Objectives are in the order of minutes and are user configurable.

Actian Ingres 12.0 Process to Disaster Recovery
Backup to cloud and disaster recovery are imperative for situations like:

Natural Disasters: When a natural disaster such as a hurricane or earthquake strikes a local datacenter, cloud backups ensure that a copy of the data is readily available, and an environment can be spun up quickly in the cloud of your choosing to resume business operations.
Cyberattacks: In the event of a cyberattack such as ransomware, having cloud backups and a disaster recovery plan are essential to establish a non-compromised version of the database in a protected cloud environment.

Fortified Protection

Actian Ingres 12.0 enables AES-256 bit encryption on data in motion by default. AES-256 bit is considered one of the most secure encryption standards available today and is widely used to protect sensitive data. The 256-bit key size makes it extremely resistant to attacks and is often used by governments and highly regulated industries like banking and healthcare.

In addition, Actian Ingres 12.0 offers user-protected privileges and containerized User Defined Functions (UDFs). These UDFs, which can be authored in SQL, JavaScript, or Python, safeguard against unauthorized activities within the company’s firewall that may target the database directly. Containerization of UDFs further enhances security by isolating user operations from core database management system (DBMS) processes.

Improved Performance and Workload Automation

Actian Ingres 12.0 customers can increase resource efficiency on transactional and analytic workloads in the same database. Workload Manager 2.0 enhances the data management experience with priority-driven queues, enabling the system to allocate resources based on predefined priorities and user roles. Now database administrators can define role-types such as DBAs, application developers, and end users, and assign a priority for each role-type.

The X100 engine, included with Ingres on Linux and Windows, brings efficiency improvements such as table cloning for x100 tables that allow customers to conduct projects or experiments in isolation from core DBMS operations.

Our Performance Engineering Team has determined that for analytics workloads, these enhancements make Actian Ingres 12.0 the fastest Ingres version yet with a 20% improvement over prior versions. Transactional workloads see improved release over release performance.

Elevated Developer Experiences

Actian OpenROAD 12.0, the latest update to the Ingres graphical 4GL, also sees some new enhancements designed to assist customers on their modernization journey. Surprisingly or not, we still have customers with forms-based applications and while many argue that these are the fastest and most reliable apps for data-entry, our customers want to deliver more modern versions of these apps mostly on tablet style devices. To facilitate this modernization and to protect the decades of investments in business logic, we have delivered enhanced versions of abf2or and WebGen in OpenROAD 12.0.

Additionally, OpenROAD users will benefit from the new gRPC-based architecture, which streamlines administration, bolsters concurrency support, and offers a more efficient framework, thanks to HTTP/2 and protocol buffers. The gRPC design is optimized for microservices and can be neatly packaged within a distinct container for deployment. The introduction of a newly distributed Docker file lays the groundwork for cloud deployment, providing production-ready business logic ready for integration with any modern client.

Leading Database Modernization and Innovation

These latest innovations join our recent milestones to solidify Actian’s position as a data and analytics leader. These achievements build on recent recognitions, including:

With this momentum, we are ready to accelerate solutions that our customers can trust, are flexible to their needs, and are easy-to-use.

Get hands-on with the new capabilities today. Connect with an Actian representative to get started.

*Actian Ingres includes the product formerly known as Actian X.

About Emma McGrattan

Data Intelligence

Building a Marketplace for Data Mesh: Enterprise-Level Marketplace – Part 2

Actian Corporation

June 3, 2024

women working and building a marketplace for data mesh

Given these shortcomings, a new concept is gaining popularity: the internal marketplace, or what we call the Enterprise Data Marketplace (EDM).

Facilitating data product consumption through metadata.
Setting up an enterprise-level marketplace.
Feeding the marketplace via domain-specific data catalogs.

As described in our previous article, an Enterprise Data Marketplace is a simple system in which consumers can search among the data product offerings for one or more eligible to perform a specific use case, become aware of the information related to these products, and then order them. The order materializes as access opening, physical data delivery, or even a request for data product evolution to cover the new use case.

Three Main Options for Setting up an Internal Data Marketplace

When establishing an internal data marketplace, organizations typically consider three primary approaches:

Develop It

This approach involves building a custom data marketplace tailored to the organization’s unique requirements. While offering the potential for a finely tuned user experience, this option often entails significant time and financial investment.

Integrate a Solution From the Market

Alternatively, organizations can opt for pre-existing solutions available in the market. Originally designed for data commercialization or external data exchange, these solutions can be repurposed for internal use. However, they may require customization to align with internal workflows and security standards.

Use Existing Systems

Some organizations choose to leverage their current infrastructure by repurposing tools such as data catalogs and corporate wikis. While this approach may offer familiarity and integration with existing workflows, it might lack the specialized features of dedicated data marketplace solutions.

The Drawbacks of Commercial Marketplaces

Although often offering a satisfying user experience and native support for the data product concept, commercial marketplaces often have significant drawbacks: highly focused on transactional aspects (distribution, licensing, contracting, purchase or subscription, payment, etc.), they are often poorly integrated with internal data platforms and access control tools. They generally require data to be distributed by the marketplace, meaning they constitute a new infrastructure component onto which data must be transferred and shared (such a system is sometimes called a Data Sharing Platform).

Actian Data Intelligence Platform’s Enterprise Data Marketplace

In a pragmatic approach, it is not desirable to introduce a new infrastructure component to deploy a data mesh – it seems highly preferable to leverage existing capabilities as much as possible.

Therefore, we’ve evolved our data discovery platform and data catalog to offer a unique solution, one that mirrors the data mesh at the metadata level to continually adapt to the organization’s evolving data platform architecture. This Enterprise Data Marketplace (EDM) integrates a cross-domain marketplace with private data catalogs tailored to each domain’s needs.

An approach that we detail in the next article of our series, made possible by what has long distinguished the Actian Data Intelligence Platform and differentiates it from most other data catalogs or metadata management platform vendors: an evolving knowledge graph.

In our final article, discover how an internal data marketplace paired with domain-specific catalogs, provides a comprehensive data mesh supervision system.

About Actian Corporation

Data Management

Actian Ingres Disaster Recovery

Emma McGrattan

May 31, 2024

Most production Actian Ingres installations need some degree of disaster recovery (DR). Options range from shipping nightly database checkpoints to off-site storage locations to near real-time replication to a dedicated off-site DR site.

Actian Ingres enterprise hybrid database that ships with built-in checkpoint and journal shipping features which provide the basic building blocks for constructing low-cost, efficient DR implementations. One such implementation is IngresSync, which utilizes Actian Ingres’ native checkpoint/journal shipping and incremental roll-forward capabilities to implement a cost-effective DR solution.

ingressync

IngresSync works on the concept of source and target Actian Ingres installations. The source installation is the currently active production environment. The target, or multiple targets if needed, kept current by an IngresSync job scheduled to execute on a user-defined interval. Each sync operation copies only journals created since the previous sync and applies those transactions to the targets. Checkpoints taken on the source node are automatically copied to and rolled forward on all targets.

Example

Suppose we have an environment where the production installation is hosted on node corp and we need to create two DR sites dreast and drwest.

The DR nodes each need:

An Ingres installation at the same version and patch level as corp.
Passwordless SSH configured to and from the other nodes.
Ingres/Net VNODE entries to the other nodes.

DR nodes for Ingress

To configure this environment, we must first designate the source and target hosts and apply the latest source checkpoint to the targets.

ingresSync --source=corp --target=dreast,drwest --database=corpdb --iid=II --ckpsync --restart

source and target hosts for Ingress

The two target installations are now synched with the source, and the target databases are in incremental rollforward (INCR_RFP) state. This state allows journals to be applied incrementally to keep the targets in sync with the source. Incremental rollforward is performed by:

ingresSync --hosts=corp,dreast,drwest --database=corpdb --iid=II --jnlsync

When executed, this will close the current journal on the source, copy new journals to the targets, and roll forward those journals to the targets. The journal sync step should be configured to execute at regular intervals using the system scheduler, such as cron. Frequent execution results in minimal sync delay between the source and targets.

The target installations at dreast and drwest are now in sync with the source installation at corp. Should the corp environment experience a hardware or software failure, we can designate one of the target nodes as the new source and direct client connections to that node. In this case, we’ll designate drwest as the new source and dreast will remain as a target (DR site).

ingresSync --target=drwest --database=corpdb --iid=II --incremental_done

This takes the drwest corpdb database out of incremental rollforward mode; the database will now execute both read and update transactions and is the new source. The dreast database is still in incremental rollforward mode and will continue to functioning as a DR target node.

drwest for ingress

Since the corp node is no longer available, the journal sync job must be started on either drwest or dreast. The journal sync job can be configured and scheduled to execute on all three nodes using the –strict flag. In this case, the job determines if it executes on the current source node; if so it will execute normally. If executing on a target, the job will simply terminate. This configuration allows synchronization to continue even as node roles change.

Once corp is back online it can be brought back into the configuration as a DR target.

ingresSync --source=drwest --target=corp --database=corpdb --iid=II --ckpsync --restart

dr target for Ingress

At some point, we may need to revert to the original configuration with corp as the source. The steps are:

Terminate all database connections to drwest

Sync

corp

 with

drwest

 to ensure

corp

 is current
ingresSync --source=drwest --target=corp --database=corpdb --iid=II

--jnlsync

Reassign node roles

ingresSync --target=corp --database=corpdb --iid=II --incremental_done

ingresSync --source=corp --target=drwest --database=corpdb --iid=II

--ckpsync --restart

revert to original corp as source for Ingress

Summary

IngresSync is one mechanism for implementing a DR solution. It is generally appropriate in cases where some degree of delay is acceptable and the target installations have little or no database user activity. Target databases can be used for read only/reporting applications with the stipulation that incremental rollforwards cannot run while there are active database connections. The rollforward process will catch up on the first refresh cycle when there are no active database connections.

The main pros and cons of the alternative methods of delivering disaster recovery for Actian Ingres are outlined below:

Feature	Checkpoint Shipping	IngresSync	Replication
Scope	Database	Database	Table
Granularity	Database	Journal	Transaction
Sync Frequency	Checkpoint	User Defined	Transaction
Target Database	Read/Write(1)	Read Only	Read/Write(2)

Target database supports read and write operations but all changes are lost on the next checkpoint refresh.
Target database supports read and write operations but there may be update conflicts that require manual resolution.

Note: IngresSync currently runs on Linux and Microsoft Windows. Windows environments require the base Cygwin package and rsync.

About Emma McGrattan

Databases

Types of Databases, Pros & Cons, and Real-World Examples

Dee Radh

May 30, 2024

Summary

This blog offers a comprehensive overview of major database models—including relational, NoSQL, in‑memory, graph, and hybrid types—highlighting their strengths, weaknesses, and real-world use cases to guide decision-makers in selecting the right database for their needs.

Relational (SQL): Ideal for structured, ACID-compliant workloads—great for transactions and complex queries—but can struggle with horizontal scaling and rigid schema.
NoSQL (Document, Key‑Value, Columnar): Offers high flexibility and horizontal scalability for large, unstructured data sets; may sacrifice consistency, require complex modeling, and incur training/development costs.
In‑Memory & Graph/Hybid Models: In‑memory databases deliver ultra-low latency; graph databases simplify relationship-heavy queries. Hybrid systems (like Actian’s) combine OLTP and OLAP strengths for real-world analytic performance.

Databases are the unsung heroes behind nearly every digital interaction, powering applications, enabling insights, and driving business decisions. They provide a structured and efficient way to store vast amounts of data. Unlike traditional file storage systems, databases allow for the organization of data into tables, rows, and columns, making it easy to retrieve and manage information. This structured approach, coupled with data governance best practices, ensures data integrity, reduces redundancy, and enhances the ability to perform complex queries. Whether it’s handling customer information, financial transactions, inventory levels, or user preferences, databases underpin the functionality and performance of applications across industries.

Types of Information Stored in Databases

Telecommunications: Verizon

Verizon uses databases to manage its vast network infrastructure, monitor service performance, and analyze customer data. This enables the company to optimize network operations, quickly resolve service issues, and offer personalized customer support. By leveraging database technology, Verizon can maintain a high level of service quality and customer satisfaction.

E-commerce: Amazon

Amazon relies heavily on databases to manage its vast inventory, process millions of transactions, and personalize customer experiences. The company’s sophisticated database systems enable it to recommend products, optimize delivery routes, and manage inventory levels in real-time, ensuring a seamless shopping experience for customers.

Finance: JPMorgan Chase

JPMorgan Chase uses databases to analyze financial markets, assess risk, and manage customer accounts. By leveraging advanced database technologies, the bank can perform complex financial analyses, detect fraudulent activities, and ensure regulatory compliance, maintaining its position as a leader in the financial industry.

Healthcare: Mayo Clinic

Mayo Clinic utilizes databases to store and analyze patient records, research data, and treatment outcomes. This data-driven approach allows the clinic to provide personalized care, conduct cutting-edge research, and improve patient outcomes. By integrating data from various sources, Mayo Clinic can deliver high-quality healthcare services and advance medical knowledge.

Types of Databases

The choice between relational and non-relational databases depends on the specific requirements of your application. Relational databases are ideal for scenarios requiring strong data integrity, complex queries, and structured data. In contrast, non-relational databases excel in scalability, flexibility, and handling diverse data types, making them suitable for big data, real-time analytics, and content management applications.

Types of databases: Relational databases and non-relational databases

1. Relational Databases

Strengths

Structured Data: Ideal for storing structured data with predefined schemas
ACID Compliance: Ensures transactions are atomic, consistent, isolated, and durable (ACID)
SQL Support: Widely used and supported SQL for querying and managing data

Limitations

Scalability: Can struggle with horizontal scaling
Flexibility: Less suited for unstructured or semi-structured data

Common Use Cases

Transactional Systems: Banking, e-commerce, and order management
Enterprise Applications: Customer Relationship Management (CRM) and Enterprise Resource Planning (ERP) systems

Real-World Examples of Relational Databases

MySQL: Widely used in web applications like WordPress.
PostgreSQL: Used by organizations like Instagram for complex queries and data integrity.
Oracle Database: Powers large-scale enterprise applications in finance and government sectors.
Actian Ingres: Widely used by enterprises and public sector like the Republic of Ireland.

2. NoSQL Databases

Strengths

Scalability: Designed for horizontal scaling
Flexibility: Ideal for handling large volumes of unstructured and semi-structured data
Performance: Optimized for high-speed read/write operations

Limitations

Consistency: Some NoSQL databases sacrifice consistency for availability and partition tolerance (CAP theorem)
Complexity: Can require more complex data modeling and application logic
Common Use Cases

Big Data Applications: Real-time analytics, IoT data storage
Content Management: Storing and serving large volumes of user-generated content

Real-World Examples of NoSQL Databases

MongoDB: Used by companies like eBay for its flexibility and scalability.
Cassandra: Employed by Netflix for handling massive amounts of streaming data.
Redis: Utilized by X (formerly Twitter) for real-time analytics and caching.
Actian Zen: Embedded database built for IoT and the intelligent edge. Used by 13,000+ companies.
HCL Informix: Small footprint and self-managing. Widely used in financial services, logistics, and retail.
Actian NoSQL: Object-oriented database used by the European Space Agency (ESA).

3. In-Memory Databases

Strengths
Speed: Extremely fast read/write operations due to in-memory storage
Low Latency: Ideal for applications requiring rapid data access

Limitations

Cost: High memory costs compared to disk storage
Durability: Data can be lost if not backed up properly

Common Use Cases

Real-Time Analytics: Financial trading platforms, fraud detection systems
Caching: Accelerating web applications by storing frequently accessed data

Real-World Examples of In-Memory Databases

Redis: Used by GitHub to manage session storage and caching.
SAP HANA: Powers real-time business applications and analytics.
Actian Vector: One of the world’s fastest columnar databases for OLAP workload.

Combinations of two or more database models are often developed to address specific use cases or requirements that cannot be fully met by a single type alone. Actian Vector blends OLAP principles, relational database functionality, and in-memory processing, enabling accelerated query performance for real-time analysis of large datasets. The resulting capability showcases the technical versatility of modern database platforms.

4. Graph Databases

Strengths

Relationships: Optimized for storing and querying relationships between entities
Flexibility: Handles complex data structures and connections

Limitations

Complexity: Requires understanding of graph theory and specialized query languages
Scalability: Can be challenging to scale horizontally

Common Use Cases

Social Networks: Managing user connections and interactions
Recommendation Engines: Suggesting products or content based on user behavior

Real-World Examples of Graph Databases

Neo4j: Used by LinkedIn to manage and analyze connections and recommendations.
Amazon Neptune: Supports Amazon’s personalized recommendation systems.

Factors to Consider in Database Selection

Selecting the right database involves evaluating multiple factors to ensure it meets the specific needs of your applications and organization. As organizations continue to navigate the digital landscape, investing in the right database technology will be crucial for sustaining growth and achieving long-term success. Here are some considerations:

1. Data Structure and Type

Structured vs. Unstructured: Choose relational databases for structured data and NoSQL for unstructured or semi-structured data.
Complex Relationships: Opt for graph databases if your application heavily relies on relationships between data points.

2. Scalability Requirements

Vertical vs. Horizontal Scaling: Consider NoSQL databases for applications needing horizontal scalability.
Future Growth: For growing data needs, cloud-based databases offer scalable solutions.

3. Performance Needs

Latency: In-memory databases are ideal for applications requiring high-speed transactions, real-time data access, and low-latency access.
Throughput: High-throughput applications may benefit from NoSQL databases.

4. Consistency and Transaction Needs

ACID Compliance: If your application requires strict transaction guarantees, a relational database might be the best choice.
Eventual Consistency: NoSQL databases often provide eventual consistency, suitable for applications where immediate consistency is not critical.

5. Cost Considerations

Budget: Factor in both initial setup costs and ongoing licensing, maintenance, and support.
Resource Requirements: Consider the hardware and storage costs associated with different database types.

6. Ecosystem and Support

Community and Vendor Support: Evaluate the availability of support, documentation, and community resources.
Integration: Ensure that the database can integrate seamlessly with your existing systems and applications.

Databases are foundational to modern digital infrastructure. By leveraging the right database for the right use case, organizations can meet their specific needs and leverage data as a strategic asset. In the end, the goal is not just to store data but to harness its full potential to gain a competitive edge.

The Consequences of Poor Data Quality: Uncovering the Hidden Risks

Defining Poor Data Quality

The Hidden Costs of Poor Data Quality

Measuring the Financial Impact of Poor Data Quality

Strategies for Improving Data Quality

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Introducing Actian’s Enhanced Data Quality Solutions

Why Data Quality Matters

Key Features of Actian’s Enhanced Data Quality Solutions

Transform Your Data into a Strategic Asset

Get Started Today

Stay Connected

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Actian Zen 16.0 Introduces New Data Sync Utility

Execute Queries Up to 50% Faster With Actian Zen

Deploy Your Way With Zen Container SDK

Elevate Developer Experiences Leveraging a Btrieve 2 Python Package

Enable Real-Time Data Streaming With Zen and Apache Kafka

Move and Sync Data Easier Using Zen EasySync

Drive Better Outcomes at the Edge

About Emma McGrattan

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Data Shopping Part 1 – How to Shop for Data Products

The Delivery of Data Products

Access Rights Management for Data Products

Data Shopping in an Internal Data Marketplace

About Actian Corporation

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Embracing Database Modernization: Future Proofing Your Business

Innovation Demands Modernization

Benefits of Database Modernization

Bridging the Skills Gap

Trusted Support for Database Modernization

About Actian Corporation

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Actian Ingres 12.0: Modernize Your Way – Trusted, Reliable, and Adaptable

Choice and Flexibility

Core Enhancements

Cloud Enablement

Performance Matters

Modernize With Confidence

Driven by Customer Feedback

About Douglas Dailey

Related Tags

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Ready to Get Started?

Building a Marketplace for Data Mesh: Domain Data Catalogs – Part 3

Metadata Management in the Context of an Internal Marketplace fed by Domain-Specific Catalogs

Data Catalog vs. EDM Capabilities

The Practical Guide to Data Mesh: Setting up and Supervising an Enterprise-Wide Data Mesh

About Actian Corporation

Subscribe to the Actian Blog

Subscribe

Thank you for subscribing to the Actian Blog!

Your Company is Ready for GenAI. But is Your Data?

The GenAI Foundation: Data Preparation

Garbage In, Garbage Out

Accuracy