Can anyone share a GC WFL? for a database, or dataset. Thank you

On 4/27/2022 2:36 PM, Val wrote:
> Can anyone share a GC WFL? for a database, or dataset. Thank you

Since nobody else replied, I'll give it a try.

Enterprise Database Server for ClearPath MCP Utilities Operations Guide
ClearPath MCP 18.0
April 2017 8600 0759-621

Explains the reorganization process, including Garbage collection, in
detail in section 7.
A few WFL and parameter samples are also provided.
Please let us know if any additional help is needed so I can send you a
more compact example.

One more thing, I was really surprised Paul K. didn´t reply to this
post. He is usually the first one to provide as detailed a reply as
anyone would ever need.

--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

-------- Original Message --------
Subject: Re: DMSII GC
From: TKosfeld <tkosfeld@gmx.net>
To:
Date: Tue May 10 2022 14:55:56 GMT-0700 (Pacific Daylight Time)

> On 4/27/2022 2:36 PM, Val wrote:
>> Can anyone share a GC WFL? for a database, or dataset.  Thank you
>
> Since nobody else replied, I'll give it a try.
>
> Enterprise Database Server for ClearPath MCP Utilities Operations Guide
> ClearPath MCP 18.0
> April 2017 8600 0759-621
>
> Explains the reorganization process, including Garbage collection, in
> detail in section 7.
> A few WFL and parameter samples are also provided.
> Please let us know if any additional help is needed so I can send you a
> more compact example.
>
>
> One more thing, I was really surprised Paul K. didn´t reply to this
> post. He is usually the first one to provide as detailed a reply as
> anyone would ever need.
>

Paul K. has been out of town and is just now getting caught up after the
trip.

There are two ways to do DMSII garbage collection. The first, and
easiest way is to use the <mix>SM GC command, where <mix> is the mix
number of the data base stack. This is an on-line GC and can run while
the data base is being updated by applications. It only reorganizes
index sequential sets (not data sets), however, and is limited to a
maximum of ten index sets per command. You cannot do a GC while an
on-line dump is in progress.

For example:

<mix>SM GC MASTERX, DETAILX, CODESX, CODESNAMEX

There are also TERMINATE GC and STATUS GC commands to abort and monitor
a GC.

You should read the entire "GARBAGE COLLECT Command" section in section
12 of the Enterprise Database Server Utilities and Operations Guide, as
there are a number of other caveats and considerations. It is best to do
a GC while the data base is undergoing light activity, as it does a lot
of I/O and needs to write a copy of each reorganized index table block
to the audit trail file, so the application load and GC could interfere
with each other. It also requires space on the same disk family(ies) as
the structure(s) being GC-ed.

It's an essential practice to do a data base dump immediately before
initiating the GC.

The second way is to use the DMSII Reorganization facility, which is
documented in section 7 of that same manual. This is a much more complex
tool, with multiple modes and lots of options. I'll try to summarize and
then give an example of a simple, straightforward case.

DMSII Reorganization always implicitly does some level of garbage
collection. GC by itself is simply Reorganization where you aren't doing
file format conversion (changing block size, area size, etc.) or record
format conversion (adding/deleting/changing field definitions in records).

Reorganization comes in three flavors:

1. Off-line reorganization -- this is the easiest to set up and
run, and is usually the fastest, but it requires that the data base be
completely shut down while it runs. Unless you absolutely cannot afford
the downtime, this is the one you should use.

2. On-line reorganization -- this is almost as easy as off-line
reorg to set up and run, and it can run while the data base is being
updated, but all applications must close the data base briefly at the
start of the reorg run. It can also take a long, long time to run and
can generate lots of audit trail data. I would avoid this unless the
structures being reorganized are relatively small.

3. "REORGDB" reorganization -- this is a newer form of on-line
reorg (although it's been around for quite a while) with efficiencies
that are much closer to that of off-line reorg. Unfortunately, I don't
have any experience with it, so can't give an example. My understanding
is that it is somewhat more complex to set up, but is less disruptive to
running applications.

One big benefit of DMSII Reorganization for GC is that it gives you lots
of options on where the working files are located and the level of
system resources that will be devoted to the process. For example, you
can direct it to build the reorganized structures on disk families other
than those used by the data base. It will then move the reorganized
files to their proper locations at the end of the reorg run, replacing
the original structures.

You really, really, really need to read the documentation in section 7
carefully before attempting to use any of these flavors. You also
really, really, really need to do data base backups before and after the
reorg.

I also highly recommend that you try out your reorg on a test data base
before doing so in production, especially if you haven't done this
before. Reorganization is like brain surgery -- you need to do it
carefully and you don't want to screw it up.

With that preamble, here is a WFL that will generate a simple, off-line
GC reorganization:

BEGIN JOB NYSPEXDB/GC;
QUEUE = 5;
BDNAME = REORG/NYSPEXDB;

TASK T;

RUN *SYSTEM/BUILDREORG [T];
FILE DASDL = (NYSPEX)DESCRIPTION/NYSPEXDB ON OPS;
DATA
$ RESET ZIP
%>> UPDATE;
GENERATE MESSAGEF, MESSAGEX, MESSAGETERMX, MESSAGEVISX,
MESSAGEDATEX, MESSAGEUSERX;
GENERATE DISTRIBUTIONF, DISTRIBUTIONX, DISTRDESTX, DISTRQX,
DISTRVISX;
GENERATE ATTACHMENTF, ATTACHMENTX;
GLOBAL
EXCLUSIVE;
OFFLINE;
INTERNAL FILES (FAMILYNAME=DBPACK);
TASKLIMIT = 10;
ALLOWEDCORE = 1000000;
?
IF T IS COMPLETEDOK THEN
BEGIN
T (STATUS=NEVERUSED);
COMPILE REORGANIZATION/NYSPEXDB WITH DMALGOL [T] LIBRARY;
ALGOL FILE TAPE = DATABASE/REORGSYMBOLIC;
ALGOL FILE PROPERTIES = DATABASE/PROPERTIES;
ALGOL FILE DASDL = DESCRIPTION/REORGANIZATION/NYSPEXDB;
ALGOL DATA
$ SET MERGE SINGLE LINEINFO
$ RESET LIST
?
IF T IS COMPILEDOK THEN
REMOVE DESCRIPTION/REORGANIZATION/NYSPEXDB;
END;
? END JOB

DMSII Reorganization generates a custom reorganization program. You then
run that reorganization program to do the actual data base reorg. You
can generate the reorg program well in advance of doing the actual
reorganization. If you are only doing GC, you can also reuse the reorg
program, but you likely will need to regenerate it after a DASDL update.
The generated program can be named anything, so in principle you could
have multiple reorg programs for different portions of the data base.

This WFL runs *SYSTEM/BUILDREORG, which reads a file of specifications
and outputs a special-purpose DESCRIPTION file. That DESCRIPTION file is
then used by the DMALGOL compiler to compile the customized reorg
program from the DATABASE/REORGSYMBOLIC source file that is part of the
standard system software release.

I prefer to run the BUILDREORG and compile in the same job, hence the
$RESET ZIP line. If you set $ZIP, BUILDREORG will initiate the compile
for you as a separate WFL job.

The UPDATE directive is explicitly commented out to show that this is a
not a reorg that is being done as a result of a DASDL update, which
would typically be needed if you were doing file format or record format
conversion. With UPDATE in effect, the generated reorganization program
would try to run *SYSTEM/DMCONTROL to update the data base control file
and exchange versions of the DMSUPPORT library. None of that should be
done for a GC-only reorg.

The GENERATE directives specify the data base structures to be
reorganized/garbage collected. Each GENERATE specifies structures
related to one disjoint data set. For this data base, structure names
ending in "F" represent data sets and names ending in "X" represent
index sets. You can omit the data set name if you want to reorg only
some or all of the index sets, but all structures in one GENERATE
directive must be related to the same disjoint data set.

The global EXCLUSIVE option prevents application programs from opening
the data base while the reorg program is running. You could specify
INQUIRYONLY instead, which for an off-line reorg will allow read-only
access to the structures not being reorganized in that run.

The global OFFLINE option specifies that this will be an off-line reorg.
Omitting this option makes it an on-line reorg. The global USEREORGDB
option specifies that it will be on-line and use the REORGDB techniques.

The global INTERNAL FILES option specifies the disk family where the
reorg program will place its temporary files. The default is DISK. You
may want to specify this as something other than the data base family,
both for space availability and to separate I/O paths.

The global TASKLIMIT option specifies the number of worker tasks the
reorg program will run simultaneously. More tasks generally results in a
faster run, but at some point additional tasks will offer diminishing
returns. More tasks may also interfere with the performance of other
tasks in the mix.