Tip: When searching for a specific document number, prefix the number with 'KB-', e.g. 'KB-419873' or 'KB-261109'.
gtar, gzip and Compressing Ingres Checkpoints
Article ID: 417655 - Created: Feb 3rd, 2009 at 03:07 PM - Modified: Nov 14th, 2011 at 11:33 PM
Summary
Version: Ingres all, Unix
Problem
Resolution
On Linux, Gnu tar and gzip are the default tools and these can be installed on many Unix variants.
Compressibility
gzip compression can be controlled by the "-N" parameter, which determines the degree of compression to be used. N=1 gives you the least compression with the minimum CPU use; N=9 gives you the greatest compression with considerable extra CPU usage. The default is 6. It turns out that there is little benefit in compressing an Ingres database beyond N=1: you use more CPU and get little extra compression. Most Ingres databases are compressible to about 10% of their size with N=1. There is, however, some variation in the resonance between your data and the deeper compression algorithms.
Here are the results of a small set of tests run with N=1,6 and 9 on a real database directory.
Original size = 1242275840
gzip -N Compressed size Elapsed Time
N=1 9.7 % 26 s
N=6 7.6 % 54 s
N=9 7.4 % 5m 25 s
Implementation
Login as the owner of the installation (usually ingres).
Make a copy of $II_SYSTEM/ingres/files/cktmpl.def (say cktmpl.cmp).
Edit cktmpl.cmp
There are four lines in this file which must be changed:
Original
WSDD: cd %D; /bin/tar cf %A *
WRDD: cd %D; /bin/tar xf %A
WSDT: cd %D; /bin/tar cf %A %B
WRDT: cd %D; /bin/tar xf %A %B
Changed (sh version)
WSDD: cd %D; GZIP="-1"; export GZIP; /bin/tar cvzf %A *
WRDD: cd %D; /bin/tar xvzf %A
WSDT: cd %D; /bin/tar cvzf %A %B
WRDT: cd %D; /bin/tar xvzf %A %B
Finally, make this symbol table entry:
ingsetenv II_CKTMPL_FILE path-to-files-directory/cktmpl.cmp
This entry can be viewed with ingprenv and will be used by ckpdb (to make a checkpoint) and rollforwarddb (to restore a database from a checkpoint and journals). Your checkpoints will be compressed, and they will be successfully unzipped during restoration.
Notes
1. The GZIP="-1" passes this parameter to the gzip implicit in tar -z and controls the degree of compression.
2. The v parameters cause tar to list all the files passing into and out of the archive.
3. Adding a .gz suffix for the checkpoint files is optional. It serves to identify the files to casual inspection as gz files. gtar will recognize either filename; proprietary Unix tar may not recognize a compressed file unless it has a .gz extension.
4. This integrated tar command is preferred to tar | gzip because it will detect more types of failure and report them with a non-zero exit status.
5. The WRDT line is also changed so that you can perform table-level rollforward from a database-level checkpoint. The WSDT line is also changed to be compatible with the WRDT line, so that you can perform table-level rollforward from a table-level checkpoint.
6. If you have enough database files in any directory that you exceed the line-length limit for the command line interpreter, then there is an easy solution, but you will lose some functionality. If in doubt, use 6a.
6a. If you want to use table-level rollforward ONLY from database-level checkpoints:
WSDD: cd %D; GZIP="-1"; export GZIP; /bin/tar cvzf %A . # Note "." instead of "*"
WRDD: cd %D; /bin/tar xvzf %A
WSDT: cd %D; /bin/tar cvzf %A %B # Note no "./". In this context, %B is a list of files, and ./ applies only to the first file
WRDT: cd %D; /bin/tar xvzf %A ./%B # Note "./" to find the files in the tar archive.
The names of the files inside the database-level tar saveset are now prefixed with "./". The WRDT line is executed once for each table in the tablelist. You will NOT be able to use table-level rollforward from a table-level checkpoint (unless in preparation you remove the "./" from the WRDT line).
6b If you want to use table-level rollforward ONLY from table-level checkpoints:
WSDD: cd %D; GZIP="-1"; export GZIP; /bin/tar cvzf %A . # Note "." instead of "*"
WRDD: cd %D; /bin/tar xvzf %A
WSDT: cd %D; /bin/tar cvzf %A %B
WRDT: cd %D; /bin/tar xvzf %A %B # Note no "./"
6c If you want to use table-level rollforward from BOTH database-level and table-level checkpoints, use one of the above configurations for the most common rollforward case and for the other, change the WRDT line before you run rollforwarddb.
User Comments
You must be logged in to leave a comment.

mohsa01 (September 6th 2010)
This document has already been updated and the problem mentioned in the comments below ís resolved.
gerhard.hofmann@planat.de (July 7th 2010)
See also issue # 145607 in the service desk.
gerhard.hofmann@planat.de (July 7th 2010)
There is a bug in this document. When cktmpl.def is tweaked like described, an
alterdb dbname -delete_oldest_ckp
will *not* delete any checkpoints because it does not seem to handle the .gz extension.