Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

I *____knew* I had some reason for not logging you off... If I could just remember what it was.


computers / comp.sys.tandem / Variable Length Record for an Entry-Sequenced File

SubjectAuthor
* Variable Length Record for an Entry-Sequenced FileJason Paradiso
+- Re: Variable Length Record for an Entry-Sequenced FileRandall
`- Re: Variable Length Record for an Entry-Sequenced FileBill Honaker

1
Variable Length Record for an Entry-Sequenced File

<3acb03ae-34c8-45d2-81d7-8cadfaffa99fn@googlegroups.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=671&group=comp.sys.tandem#671

 copy link   Newsgroups: comp.sys.tandem
X-Received: by 2002:ad4:4f52:0:b0:63d:25f:71d5 with SMTP id eu18-20020ad44f52000000b0063d025f71d5mr1702qvb.1.1690334454968;
Tue, 25 Jul 2023 18:20:54 -0700 (PDT)
X-Received: by 2002:a9d:7c88:0:b0:6b9:f343:acfd with SMTP id
q8-20020a9d7c88000000b006b9f343acfdmr1222070otn.5.1690334454771; Tue, 25 Jul
2023 18:20:54 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!proxad.net!feeder1-2.proxad.net!209.85.160.216.MISMATCH!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.sys.tandem
Date: Tue, 25 Jul 2023 18:20:54 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=173.21.141.221; posting-account=9NzBdAoAAABNni7vwG7dBJFf1VnA4kTB
NNTP-Posting-Host: 173.21.141.221
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <3acb03ae-34c8-45d2-81d7-8cadfaffa99fn@googlegroups.com>
Subject: Variable Length Record for an Entry-Sequenced File
From: alee...@gmail.com (Jason Paradiso)
Injection-Date: Wed, 26 Jul 2023 01:20:54 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
 by: Jason Paradiso - Wed, 26 Jul 2023 01:20 UTC

Hey, there, HPNonstoppers,

I'm in the process of conducting analysis for a project that requires to create a transaction log file to capture OLTP transactions. I read the eCOBOL manual and it suggested that an entry-sequenced file will be a good candidate for transaction logging file that stores data chronologically.

Our HPNS runs OS with J06.22+ that allows a max record length of 27576 bytes for an entry-sequenced file and I need to record size around 10k bytes on a Format 2. During our team brainstorming, someone asked if it would be better off to create variable length for this new transaction log.

I'm not quite familiar with how a cobol program would access a variable length entry-sequenced record so I did go through ecobol manual trying to find some info, but it did not completely answer all my questions. So, here I am with some of my unanswered questions:

1). From the ecobol manual, I understand that I'll need to include either a RECORD CONTAINS rec-1 TO rec-2 CHARACTERS or a RECORD IS VARYING IN SIZE FROM rec-1 TO rec-2 CHARACTERS clause, in the FD entry to declare a variable length record. Nevertheless, the file schema would still be set up with a fixed record length for the entry-sequenced file, say 12k bytes. How does declaring a variable length record for a fixed length record entry-sequenced file gonna buy us anything? Does it imply the fixed record length of 12k would potentially store more than 1 record based on the variable length (e.g.., 2 5k variable length records within 12k length)?

2). What would be the best way to estimate/determine the ideal record size to prevent block split and minimize the slack space? In the ecobol manual, it mentioned that when the run-time routines execute a REWRITE or WRITE statement referencing a file of variable-length records, the number of character positions in the record written is determined. Does that mean it would always cause block split if the program declares variable length for an entry-sequence file with a fixed record length (say, 12k bytes)?

3). Last but not the least, I guess the question I kept asking myself is - given an entry-sequenced file schema that defines a fixed length of record size (e.g., 12k), how would the records of variable length written into a block?

Thanks for your attention to my questions and any feedback or suggestion will be greatly appreciated!

Jason

Re: Variable Length Record for an Entry-Sequenced File

<e185d8ae-6001-4206-9d85-9c346c846a9dn@googlegroups.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=672&group=comp.sys.tandem#672

 copy link   Newsgroups: comp.sys.tandem
X-Received: by 2002:a05:620a:406:b0:767:404c:787f with SMTP id 6-20020a05620a040600b00767404c787fmr7272qkp.3.1690399830587;
Wed, 26 Jul 2023 12:30:30 -0700 (PDT)
X-Received: by 2002:a05:6808:128a:b0:3a2:6d07:ad48 with SMTP id
a10-20020a056808128a00b003a26d07ad48mr692765oiw.4.1690399830250; Wed, 26 Jul
2023 12:30:30 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.sys.tandem
Date: Wed, 26 Jul 2023 12:30:29 -0700 (PDT)
In-Reply-To: <3acb03ae-34c8-45d2-81d7-8cadfaffa99fn@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=2607:fea8:3fa9:4200:34b0:9522:108b:35bb;
posting-account=6VebZwoAAAAgrpUtsowyjrKRLNlqxnXo
NNTP-Posting-Host: 2607:fea8:3fa9:4200:34b0:9522:108b:35bb
References: <3acb03ae-34c8-45d2-81d7-8cadfaffa99fn@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <e185d8ae-6001-4206-9d85-9c346c846a9dn@googlegroups.com>
Subject: Re: Variable Length Record for an Entry-Sequenced File
From: rsbec...@nexbridge.com (Randall)
Injection-Date: Wed, 26 Jul 2023 19:30:30 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 4718
 by: Randall - Wed, 26 Jul 2023 19:30 UTC

On Tuesday, July 25, 2023 at 9:20:56 p.m. UTC-4, Jason Paradiso wrote:
> Hey, there, HPNonstoppers,
>
> I'm in the process of conducting analysis for a project that requires to create a transaction log file to capture OLTP transactions. I read the eCOBOL manual and it suggested that an entry-sequenced file will be a good candidate for transaction logging file that stores data chronologically.
>
> Our HPNS runs OS with J06.22+ that allows a max record length of 27576 bytes for an entry-sequenced file and I need to record size around 10k bytes on a Format 2. During our team brainstorming, someone asked if it would be better off to create variable length for this new transaction log.
>
> I'm not quite familiar with how a cobol program would access a variable length entry-sequenced record so I did go through ecobol manual trying to find some info, but it did not completely answer all my questions. So, here I am with some of my unanswered questions:
>
> 1). From the ecobol manual, I understand that I'll need to include either a RECORD CONTAINS rec-1 TO rec-2 CHARACTERS or a RECORD IS VARYING IN SIZE FROM rec-1 TO rec-2 CHARACTERS clause, in the FD entry to declare a variable length record. Nevertheless, the file schema would still be set up with a fixed record length for the entry-sequenced file, say 12k bytes. How does declaring a variable length record for a fixed length record entry-sequenced file gonna buy us anything? Does it imply the fixed record length of 12k would potentially store more than 1 record based on the variable length (e..g., 2 5k variable length records within 12k length)?
>
> 2). What would be the best way to estimate/determine the ideal record size to prevent block split and minimize the slack space? In the ecobol manual, it mentioned that when the run-time routines execute a REWRITE or WRITE statement referencing a file of variable-length records, the number of character positions in the record written is determined. Does that mean it would always cause block split if the program declares variable length for an entry-sequence file with a fixed record length (say, 12k bytes)?
>
> 3). Last but not the least, I guess the question I kept asking myself is - given an entry-sequenced file schema that defines a fixed length of record size (e.g., 12k), how would the records of variable length written into a block?
>
> Thanks for your attention to my questions and any feedback or suggestion will be greatly appreciated!
>
> Jason

I don't think the eCOBOL manual is the best place to look. There are other manuals - File Utilities, Guardian Programmer's Guide, Proc Calls, that have more details. FILE_WRITEX64_ is the call to examine.

As far as block splits, they are rare in entry-sequenced files where everything is effectively variable length whether you intend it or not. The more relevant question is will your log entries fit into a block or not. That is a min/max optimization question on minimizing dslack. What happens is you have slack, which you want to be small in a log file because you are not going to have subsequent inserts. A block split is more relevant in key sequenced files where a block can split either when an index causes a split, or when a rewrite can force the split.

I'm sure others have more info on this.

Re: Variable Length Record for an Entry-Sequenced File

<4463ci15l49jbc7a6ii52bde9ui4c2ogq6@4ax.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=673&group=comp.sys.tandem#673

 copy link   Newsgroups: comp.sys.tandem
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx17.iad.POSTED!not-for-mail
From: no_spam_...@x_i_d.com (Bill Honaker)
Newsgroups: comp.sys.tandem
Subject: Re: Variable Length Record for an Entry-Sequenced File
Organization: XID, Ltd.
Message-ID: <4463ci15l49jbc7a6ii52bde9ui4c2ogq6@4ax.com>
References: <3acb03ae-34c8-45d2-81d7-8cadfaffa99fn@googlegroups.com>
X-Newsreader: Forte Agent 6.00/32.1186
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Lines: 51
X-Complaints-To: abuse@easynews.com
X-Complaints-Info: Please be sure to forward a copy of ALL headers otherwise we will be unable to process your complaint properly.
Date: Wed, 26 Jul 2023 17:35:35 -0500
X-Received-Bytes: 5017
 by: Bill Honaker - Wed, 26 Jul 2023 22:35 UTC

On Tue, 25 Jul 2023 18:20:54 -0700 (PDT), Jason Paradiso <alee616@gmail.com> wrote:

>Hey, there, HPNonstoppers,
>
>I'm in the process of conducting analysis for a project that requires to create a transaction log file to capture OLTP transactions. I read the eCOBOL manual and it suggested that an entry-sequenced file will be a good candidate for transaction logging file that stores data chronologically.
>
>Our HPNS runs OS with J06.22+ that allows a max record length of 27576 bytes for an entry-sequenced file and I need to record size around 10k bytes on a Format 2. During our team brainstorming, someone asked if it would be better off to create variable length for this new transaction log.
>
>I'm not quite familiar with how a cobol program would access a variable length entry-sequenced record so I did go through ecobol manual trying to find some info, but it did not completely answer all my questions. So, here I am with some of my unanswered questions:
>
>1). From the ecobol manual, I understand that I'll need to include either a RECORD CONTAINS rec-1 TO rec-2 CHARACTERS or a RECORD IS VARYING IN SIZE FROM rec-1 TO rec-2 CHARACTERS clause, in the FD entry to declare a variable length record. Nevertheless, the file schema would still be set up with a fixed record length for the entry-sequenced file, say 12k bytes. How does declaring a variable length record for a fixed length record entry-sequenced file gonna buy us anything? Does it imply the fixed record length of 12k would potentially store more than 1 record based on the variable length (e.g., 2 5k variable length records within 12k length)?
>
>2). What would be the best way to estimate/determine the ideal record size to prevent block split and minimize the slack space? In the ecobol manual, it mentioned that when the run-time routines execute a REWRITE or WRITE statement referencing a file of variable-length records, the number of character positions in the record written is determined. Does that mean it would always cause block split if the program declares variable length for an entry-sequence file with a fixed record length (say, 12k bytes)?
>
>3). Last but not the least, I guess the question I kept asking myself is - given an entry-sequenced file schema that defines a fixed length of record size (e.g., 12k), how would the records of variable length written into a block?
>
>Thanks for your attention to my questions and any feedback or suggestion will be greatly appreciated!
>
>Jason

Hi, Jason,

Randall's post is correct with regards to block splits.

DDL has no concept of variable-length structures. It will generate fixed-length structures, and any FUP CREATE statements will have REC and BLOCK attributes set.

COBOL will write the data based on the FD. You are looking for the VARYING phrase on page 172 of the NonStop COBOL manual in the Data division section:

RECORD VARYING FROM n1 TO n2 CHARACTERS DEPENDING ON len-var.

n1 and n2 are literals defining the smallest and largest lengths. len-var is the name of an unsigned integer data item.

When writing variable length, you must set the len-var prior to the WRITE. Typically you would use the STRING verb and WITH POINTER option to know how long to write.
Alternatively, you might have to use a PERFORM loop looking from the end of a buffer for the last trailing blank and calculate the length.

When reading, you must READ and then check the value of len-var. You should use the len-var value in a reference modifier:

READ var-file INTO var-rec.
MOVE var-rec (1:len-var) TO tareget-buffer.

You can REWRITE an entry sequenced record but it's not usually done in Logging. You can't DELETE a record (altough you can REWRITE it with a zero-byte count).

As for your third (blocking) question, you should read look in the Enscribe Programmers manual and find "Figure 31 Block Format for Structured Format 2 Files".

It's not a simple case of '10 5,000 byte records fit in a 50,000 byte block'.

"Table 7 Comparison of Format 1 Versus Format 2 Files" tells you some maximums, but in the case of variable-length records you'd have to know statistics.
Which you won't know until th app has run for a while. You can just let the Disk Process manage it and it should be fine.

Good luck,
Bill

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor