On 8/24/2023 1:31 PM, Johnny Billquist wrote:
> On 2023-08-24 16:05, Dave Froble wrote:
>> RMS keys are defined as locations in a data record.  Not sure I'd
>> define that as "information about fields".  As far as I know, RMS
>> (other than key definitions) knows nothing about data
>> fields/columns/whatever.
>
> RMS keys have a type (obviously). You can also have some meta-data, even
> though I suspect it's mostly not used...

> Note both the name, and type information for each key. However, the name
> have no function as such within RMS. At least not under RSX. But you can
> put information there if you want to.

Pretty much same on VMS.

The manual for XABKEY lists among other fields:

XAB$B_DTP : key type and order
XAB$B_FLG : flags (incl. allowing duplicates or not)
XAB$L_KNM : name (only used for documentation)
XAB$W_POSn : position in record (only n=0 used if non segmented key)
XAB$B_SIZn : size (only n=0 used if non segmented key)
XAB$B_REF : key number (0 is primay key)

Arne

On 8/24/2023 9:04 AM, David Jones wrote:
> On Thursday, August 24, 2023 at 8:13:53 AM UTC-4, Jan-Erik Söderholm wrote:
>> $ help eoj
>>
>> EOJ
>>
>> Marks the end of a batch job submitted through a card reader.
>
> I got into the habit of using EOJ to logout as most sys admins that think
> it's useful to define a logout symbol aren't aware this alternative exists. It
> is a little cleaner than stop/id=0.

Same here.

It just got in my fingers a long time ago.

Arne

On Wednesday, August 23, 2023 at 10:47:38 AM UTC-4, David Jones wrote:
> To support any number of Unix applications reading of files, the C runtime
> includes fgets() to return what is logically a record. So what?

That fails woefully even on some text file formats. I probably sound like I'm beating a dead horse, but go look at ckvfio.c for an idea of what is really needed to get text lines out of RMS regardless of the file format. For those who say "RMS contains all of the necessary info", go look at how True BASIC stores the source code for user-written programs (hint: RMS undefined).

In addition to doing the C-Kermit work myself, I hit the Info-ZIP people with a clue-by-four to get rid of the abomination that was BILF.

On 2023-08-25 06:17, terry-...@glaver.org wrote:
> On Wednesday, August 23, 2023 at 10:47:38 AM UTC-4, David Jones wrote:
>> To support any number of Unix applications reading of files, the C runtime
>> includes fgets() to return what is logically a record. So what?
>
> That fails woefully even on some text file formats. I probably sound like I'm beating a dead horse, but go look at ckvfio.c for an idea of what is really needed to get text lines out of RMS regardless of the file format. For those who say "RMS contains all of the necessary info", go look at how True BASIC stores the source code for user-written programs (hint: RMS undefined).

Well, to be fair - RMS undefined is hardly a "text file format". :-)
That's a very weird choice for any program to use for source code.

Johnny

On Friday, August 25, 2023 at 12:18:00 AM UTC-4, terry-...@glaver.org wrote:
> On Wednesday, August 23, 2023 at 10:47:38 AM UTC-4, David Jones wrote:
> > To support any number of Unix applications reading of files, the C runtime
> > includes fgets() to return what is logically a record. So what?
> That fails woefully even on some text file formats. I probably sound like I'm beating a dead horse, but go look at ckvfio.c for an idea of what is really needed to get text lines out of RMS regardless of the file format. For those who say "RMS contains all of the necessary info", go look at how True BASIC stores the source code for user-written programs (hint: RMS undefined).

That rant is orthogonal to the point I was making. Unix applications run anywhere use fgets(), not just on OpenVMS.

On 8/25/2023 12:17 AM, terry-...@glaver.org wrote:
> On Wednesday, August 23, 2023 at 10:47:38 AM UTC-4, David Jones
> wrote:
>> To support any number of Unix applications reading of files, the C
>> runtime includes fgets() to return what is logically a record. So
>> what?
>
> That fails woefully even on some text file formats. I probably sound
> like I'm beating a dead horse, but go look at ckvfio.c for an idea of
> what is really needed to get text lines out of RMS regardless of the
> file format. For those who say "RMS contains all of the necessary
> info", go look at how True BASIC stores the source code for
> user-written programs (hint: RMS undefined).

RFM=UDF is not intended to be used for text files.

In general the VMS expectation is that operations
going through RMS should work transparently independent
of ORG and RFM to the extent that it makes logical
sense.

It seems to almost be the case.

Small test on VMS Alpha 8.4.

ffun.com:

$ type ffun_c.c
$ cc ffun_c
$ link ffun_c
$ type ffun_p.pas
$ pas ffun_p
$ link ffun_p
$ create ffun.txt
AA
BB
CC
$ $ call fcheck
$ convert/fdl="record; format stream_lf" ffun.txt ffun.txt
$ call fcheck
$ convert/fdl="record; format stream" ffun.txt ffun.txt
$ call fcheck
$ convert/fdl="record; format stream_cr" ffun.txt ffun.txt
$ call fcheck
$ convert/fdl="record; format vfc" ffun.txt ffun.txt
$ call fcheck
$ convert/fdl="record; format fixed; size 2" ffun.txt ffun.txt
$ call fcheck
$ convert/fdl="file; organization indexed; key 0; seg0_length 1;
seg0_position 0; type string" ffun.txt ffun.txt
$ call fcheck
$ exit
$!
$ fcheck: subroutine
$ write sys$output f$fao("org=!AS rfm=!AS eof=!UL ffb=!UL",
f$file("ffun.txt", "org"), f$file("ffun.txt", "rfm"), f$file("ffun.txt",
"eof"), f$file("ffun.txt", "ffb"))
$ on error then continue
$ write sys$output "C"
$ run ffun_c
$ write sys$output "Pascal"
$ run ffun_p
$ write sys$output "DCL"
$ open/read f ffun.txt
$ loop:
$ read/end=endloop f line
$ write sys$output "|" + line + "|"
$ goto loop
$ endloop:
$ close f
$ return
$ endsubroutine

output:

#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[])
{ FILE *fp;
char line[32768];
fp = fopen("ffun.txt", "r");
line[0] = '|';
while(fgets(line + 1, sizeof(line) - 1, fp))
{
if(line[strlen(line) - 1] == '\n') line[strlen(line) - 1] = 0;
strcat(line, "|");
puts(line);
}
fclose(fp);
return 0;
} program ffun(input,output);

var
f : text;
line : varying [32767] of char;

begin
open(f, 'ffun.txt', old);
reset(f);
while not eof(f) do begin
readln(f, line);
writeln('|' + line + '|');
end;
close(f);
end.
org=SEQ rfm=VAR eof=1 ffb=12
C |AA|
|BB|
|CC|
Pascal
|AA|
|BB|
|CC|
DCL
|AA|
|BB|
|CC|
org=SEQ rfm=STMLF eof=1 ffb=9
C |AA|
|BB|
|CC|
Pascal
|AA|
|BB|
|CC|
DCL
|AA|
|BB|
|CC|
org=SEQ rfm=STM eof=1 ffb=12
C |AA|
|BB|
|CC|
Pascal
|AA|
|BB|
|CC|
DCL
|AA|
|BB|
|CC|
org=SEQ rfm=STMCR eof=1 ffb=9
C |AA|
|BB|
|CC|
Pascal
|AA|
|BB|
|CC|
DCL
|AA|
|BB|
|CC|
org=SEQ rfm=VFC eof=1 ffb=18
C ||
||
||
Pascal
|AA|
|BB|
|CC|
DCL
|AA|
|BB|
|CC|
org=SEQ rfm=FIX eof=1 ffb=6
C |AA|
|BB|
|CC|
Pascal
|AA|
|BB|
|CC|
DCL
|AA|
|BB|
|CC|
org=IDX rfm=VAR eof=16 ffb=0
C |AA|
|BB|
|CC|
Pascal
%PAS-F-TEXREQSEQ, textfiles require sequential organization and access
File "F" Filename "DISK2:[ARNE]ffun.txt;8"
%TRACE-F-TRACEBACK, symbolic stack dump follows
image module routine line rel PC abs PC
PAS$RTL 0 00000000000200BC
000000007BFE80BC
PAS$RTL 0 00000000000361C8
000000007BFFE1C8
PAS$RTL 0 00000000000228E8
000000007BFEA8E8
PAS$RTL 0 00000000000200BC
000000007BFE80BC
PAS$RTL 0 0000000000021584
000000007BFE9584
ffun_p FFUN FFUN 8 0000000000000074
0000000000020074
0 FFFFFFFF80340964
FFFFFFFF80340964
%TRACE-I-END, end of TRACE stack dump
DCL
|AA|
|BB|
|CC|

It looks like Pascal text files has an explicit test requiring
ORG=SEQ instead of just calling SYS$GET no matter what. There
is probably a good reason for that.

And then C fgets fail on VFC. It looks like it read the
control bytes as data. WTF? I believe it has been revealed that
C IO does not use SYS$GET to read records but SYS$READ to read
blocks, so there is a reason why C is different. But I would
still consider it a bug.

Arne

On 8/26/2023 1:28 PM, Arne Vajhøj wrote:
> org=SEQ rfm=VFC eof=1 ffb=18
> C
> ||
> ||
> ||
> Pascal
> |AA|
> |BB|
> |CC|
> DCL
> |AA|
> |BB|
> |CC|

> org=IDX rfm=VAR eof=16 ffb=0
> C
> |AA|
> |BB|
> |CC|
> Pascal
> %PAS-F-TEXREQSEQ, textfiles require sequential organization and access
>   File "F"  Filename "DISK2:[ARNE]ffun.txt;8"
> %TRACE-F-TRACEBACK, symbolic stack dump follows
>   image    module    routine             line      rel PC           abs PC
>  PAS$RTL                                    0 00000000000200BC
> 000000007BFE80BC
>  PAS$RTL                                    0 00000000000361C8
> 000000007BFFE1C8
>  PAS$RTL                                    0 00000000000228E8
> 000000007BFEA8E8
>  PAS$RTL                                    0 00000000000200BC
> 000000007BFE80BC
>  PAS$RTL                                    0 0000000000021584
> 000000007BFE9584
>  ffun_p  FFUN  FFUN                         8 0000000000000074
> 0000000000020074
>                                             0 FFFFFFFF80340964
> FFFFFFFF80340964
> %TRACE-I-END, end of TRACE stack dump
> DCL
> |AA|
> |BB|
> |CC|

> And then C fgets fail on VFC. It looks like it read the
> control bytes as data. WTF? I believe it has been revealed that
> C IO does not use SYS$GET to read records but SYS$READ to read
> blocks, so there is a reason why C is different. But I would
> still consider it a bug.

But C IO must be using SYS$GET for ORG=IDX or that would have
failed as well.

Arne

On Friday, August 25, 2023 at 5:47:14 AM UTC-4, Johnny Billquist wrote:
> On 2023-08-25 06:17, terry-...@glaver.org wrote:
> > On Wednesday, August 23, 2023 at 10:47:38 AM UTC-4, David Jones wrote:
> >> To support any number of Unix applications reading of files, the C runtime
> >> includes fgets() to return what is logically a record. So what?
> >
> > That fails woefully even on some text file formats. I probably sound like I'm beating a dead horse, but go look at ckvfio.c for an idea of what is really needed to get text lines out of RMS regardless of the file format. For those who say "RMS contains all of the necessary info", go look at how True BASIC stores the source code for user-written programs (hint: RMS undefined).
> Well, to be fair - RMS undefined is hardly a "text file format". :-)
> That's a very weird choice for any program to use for source code.

I take it you've never encountered the religious fervor of latter-day
Kemeny & Kurtz 8-}

VMS C-Kermit had to support every possible file format users would
throw at it (VMS was WAY more common back then) and convert it to
either canonical ASCII or fixed-block binary. Frank da Cruz and I came
up with the new "Labeled" file type, which containerized arbitrary data
(in the VMS case, any and every possible RMS file type) for transport
with Kermit. The container could go through any system, including
ones with different character sets, and come out at the far end and be
reconstituted exactly. As I recall, the test was VAX -> DECsystem-20 ->
IBM VM/CMS -> MS-DOS -> VAX. There was also a standalone VMS
utility to reconstitute the file (for example, if it had been FTP'd back
to the VMS system).

There are all sorts of special cases, like Fortran carriage control, that
are converted by C-Kermit. My Kermit work was complicated enough
that it won me a DECUS award or two. 8-)

On 2023-08-26 20:57, terry-...@glaver.org wrote:
> On Friday, August 25, 2023 at 5:47:14 AM UTC-4, Johnny Billquist wrote:
>> On 2023-08-25 06:17, terry-...@glaver.org wrote:
>>> On Wednesday, August 23, 2023 at 10:47:38 AM UTC-4, David Jones wrote:
>>>> To support any number of Unix applications reading of files, the C runtime
>>>> includes fgets() to return what is logically a record. So what?
>>>
>>> That fails woefully even on some text file formats. I probably sound like I'm beating a dead horse, but go look at ckvfio.c for an idea of what is really needed to get text lines out of RMS regardless of the file format. For those who say "RMS contains all of the necessary info", go look at how True BASIC stores the source code for user-written programs (hint: RMS undefined).
>> Well, to be fair - RMS undefined is hardly a "text file format". :-)
>> That's a very weird choice for any program to use for source code.
>
> I take it you've never encountered the religious fervor of latter-day
> Kemeny & Kurtz 8-}

Nope. I have not. Don't even know who they are.

> VMS C-Kermit had to support every possible file format users would
> throw at it (VMS was WAY more common back then) and convert it to
> either canonical ASCII or fixed-block binary. Frank da Cruz and I came
> up with the new "Labeled" file type, which containerized arbitrary data
> (in the VMS case, any and every possible RMS file type) for transport
> with Kermit. The container could go through any system, including
> ones with different character sets, and come out at the far end and be
> reconstituted exactly. As I recall, the test was VAX -> DECsystem-20 ->
> IBM VM/CMS -> MS-DOS -> VAX. There was also a standalone VMS
> utility to reconstitute the file (for example, if it had been FTP'd back
> to the VMS system).
>
> There are all sorts of special cases, like Fortran carriage control, that
> are converted by C-Kermit. My Kermit work was complicated enough
> that it won me a DECUS award or two. 8-)

Oh. I know a bunch about the different formats RMS have. Not that I
played *that* much with it in VMS, but a lot in RSX, where I had my ftp
client and server be able to deal with all of that as well. The problem
is that when travelling over some other systems, it becomes pretty much
impossible to retain a bunch of meta-information unless you store it in
two files, and then recreate the file back on VMS using both of them. Or
else write some encode/decode program that you use on the VMS side which
creates a file that contains all that is needed to restore the file
again, and then you can move that file around as you wish. (Of course,
you could also add the meta-information in the actual file transferred,
but then it looks like some different file on all other systems). Lots
of different choices and compromises, but none is perfect.

Johnny

On Sunday, August 27, 2023 at 10:38:04 AM UTC-4, Johnny Billquist wrote:
> On 2023-08-26 20:57, terry-...@glaver.org wrote:
> > On Friday, August 25, 2023 at 5:47:14 AM UTC-4, Johnny Billquist wrote:
> >> On 2023-08-25 06:17, terry-...@glaver.org wrote:
> >>> On Wednesday, August 23, 2023 at 10:47:38 AM UTC-4, David Jones wrote:
> >>>> To support any number of Unix applications reading of files, the C runtime
> >>>> includes fgets() to return what is logically a record. So what?
> >>>
> >>> That fails woefully even on some text file formats. I probably sound like I'm beating a dead horse, but go look at ckvfio.c for an idea of what is really needed to get text lines out of RMS regardless of the file format.. For those who say "RMS contains all of the necessary info", go look at how True BASIC stores the source code for user-written programs (hint: RMS undefined).
> >> Well, to be fair - RMS undefined is hardly a "text file format". :-)
> >> That's a very weird choice for any program to use for source code.
> >
> > I take it you've never encountered the religious fervor of latter-day
> > Kemeny & Kurtz 8-}
> Nope. I have not. Don't even know who they are.

The original creators of Dartmouth BASIC. This probably explains it better than I can:
https://en.wikipedia.org/wiki/True_BASIC

> > VMS C-Kermit had to support every possible file format users would
> > throw at it (VMS was WAY more common back then) and convert it to
> > either canonical ASCII or fixed-block binary. Frank da Cruz and I came
> > up with the new "Labeled" file type, which containerized arbitrary data
> > (in the VMS case, any and every possible RMS file type) for transport
> > with Kermit. The container could go through any system, including
> > ones with different character sets, and come out at the far end and be
> > reconstituted exactly. As I recall, the test was VAX -> DECsystem-20 ->
> > IBM VM/CMS -> MS-DOS -> VAX. There was also a standalone VMS
> > utility to reconstitute the file (for example, if it had been FTP'd back
> > to the VMS system).
> >
> > There are all sorts of special cases, like Fortran carriage control, that
> > are converted by C-Kermit. My Kermit work was complicated enough
> > that it won me a DECUS award or two. 8-)
> Oh. I know a bunch about the different formats RMS have. Not that I
> played *that* much with it in VMS, but a lot in RSX, where I had my ftp
> client and server be able to deal with all of that as well. The problem
> is that when travelling over some other systems, it becomes pretty much
> impossible to retain a bunch of meta-information unless you store it in
> two files, and then recreate the file back on VMS using both of them. Or
> else write some encode/decode program that you use on the VMS side which
> creates a file that contains all that is needed to restore the file
> again, and then you can move that file around as you wish. (Of course,
> you could also add the meta-information in the actual file transferred,
> but then it looks like some different file on all other systems). Lots
> of different choices and compromises, but none is perfect.

Yup. I always disliked the idea of the Mac's Resource Fork. It is a concept
that hasn't been needed for years (OS X has been around for 25+ years at
this point). Yet I still encounter recent Zip archives (often containing
executables that will only run under Microsoft Windows) with resource
fork directories.

Grossly simplifying, files coming from VMS to another operating system
fall into 3 major types:

1) Text files
2) Binary files
3) Files that have no meaning outside of VMS

C-Kermit did a pretty good job of sorting them into those categories (after
quite a bit of development - VMS was popular enough back then that I got
a lot of "You don't support my oddball file format!" emails).

Those were handled with:

1) "set file type text"
2) "set file type binary"
3) "set file type labeled"

Other file transfer utilities implemented various other solutions, with more or
less user-transparency. I still get "STRU O VMS: command not understood."
from other FTP servers to this day.