On 10/13/2023 7:26 PM, Arne Vajhøj wrote:
> On 10/13/2023 2:33 PM, Simon Clubley wrote:
>> On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>> On 10/13/2023 8:52 AM, Simon Clubley wrote:
>>>> Because it's a BufferedReader, Java may be reading in larger chunks
>>>> than the C RTL.
>>>
>>> It probably does.
>>>
>>> I did not use BufferedReader to get that buffer, but to get the readLine
>>> method.
>>>
>>> Anyway that is a possible explanation of why Java is faster than C.
>>
>> One way to confirm this would be to record the current number of direct
>> I/Os before and after each instance of your tests. The DIRIO number for
>> Java _should_ be rather lower if this theory is correct. This is what I
>> am thinking of:
>>
>> $ write sys$output f$getjpi("", "DIRIO")
>
> The Java code would also get all the IO to load classes.
>
> But still interesting.

Results:

C with fgets : 6104 DIRIO
Java : 6262 DIRIO

Same (I assume the difference is due to Java classloading).

Arne

On 10/13/2023 7:26 PM, Arne Vajhøj wrote:
> On 10/13/2023 2:33 PM, Simon Clubley wrote:
>> On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>> On 10/13/2023 8:52 AM, Simon Clubley wrote:
>>>> Are you in a position to try direct QIO reads in C on both
>>>> architectures
>>>> so you cut out both the language RTL and RMS ?
>>>
>>> Reading blocks with SYS$QIO(W) and parsing lines manually is
>>> not so much fun. But it is certainly doable.
>>>
>>
>> How difficult would it be to directly call RMS in record mode instead
>> from your test ? That would eliminate most of the CRTL as a source of
>> overhead.
>
> SYS$GET is not that hard to use.

I don't know about "overhead".

C fgets 20.0s
sys$get 56.7s

(but it looks like Pascal readln may be going directly to sys$get!!)

Code below.

Arne

++++

long stat;
struct FAB fab;
struct RAB rab;
char buf[1000];
int n;
....
fab = cc$rms_fab;
fab.fab$l_fna = (char *)fnm;
fab.fab$b_fns = strlen(fnm);
fab.fab$b_fac = FAB$M_GET;
stat = sys$open(&fab, 0, 0);
if(!(stat & 1)) {
printf("sys$open stat = %d\n", stat);
}
rab = cc$rms_rab;
rab.rab$l_fab = &fab;
rab.rab$b_rac = RAB$C_SEQ;
stat = sys$connect(&rab, 0 ,0); n = 0;
if(!(stat & 1)) {
printf("sys$connect stat = %d\n", stat);
}
for(;;)
{
rab.rab$l_ubf = buf;
rab.rab$w_usz = sizeof(buf);
stat = sys$get(&rab, 0, 0);
if(stat == RMS$_EOF) break;
if(!(stat & 1)) {
printf("sys$get stat = %d\n", stat);
}
buf[rab.rab$w_rsz] = 0;
n++;
if(n == 10000000) break; // only read 100 MB of 1 GB
}
printf("%d lines\n", n);
stat = sys$disconnect(&rab, 0, 0);
if(!(stat & 1)) {
printf("sys$disconnect stat = %d\n", stat);
}
stat = sys$close(&fab, 0, 0);
if(!(stat & 1)) {
printf("sys$close stat = %d\n", stat);
}

On Thursday, October 12, 2023 at 5:28:57 PM UTC-7, Craig A. Berry wrote:
> getopt was one of several things that has made it difficult to
> contemplate building Perl with 64-bit pointers previously. I know some
> work has been done on the CRTL so maybe getopt() now deals with 64-bit
> pointers, at least on x86.

I got much farther along in building Perl 5.38 with C++ by making a few mods to the configure script, and now I'm able to work on some actual C++-related compile errors, like Sock_size_t being defined as size_t in config.h instead of socklen_t, causing a pointer type mismatch in doio.c.

Sadly, in order to generate config.h and descrip.mms, I had to compile the "vms/munchconfig.c" utility with the C compiler instead of C++ to avoid a read error I can't work around. The code uses fgets(), the same function people have been talking about on this thread. It also uses getopt(), and that exposed two different issues with the C++ compiler.

If I compile with no /Pointer_Size flag, then I get a pointer size mismatch error between argv[] and the second arg to getopt(). If I use /Pointer_Size=32, it compiles.

If I use /Pointer_Size=64=Argv, then I get a linker warning about a missing "decc$ga__optarg64" symbol. I can see in the header where it #defines "optarg" as "_optarg64" when using Clang under these circumstances, so I can workaround it by undefining "optarg".

That's not why I had to give up and build munchconfig.c with the C compiler.. The runtime error I'm getting when I build with the C++ compiler is that fgets() is returning NULL immediately when reading the first line of the second file given to it, causing NULL to be passed to strstr(), because the code has nonexistent error handling.

I patched the code to check the return value of fgets() and do some logging when that happens, and what I see is feof() returns false, ferror() returns false, and errno is 9 (EBADF). Apparently the first file reads in with no problems and then it fails reading the first line of the second file in the loop searching for a sentinel string.

Since the same code works correctly with the C compiler, and fails with Clang regardless of /Standard, /Pointer_Size, etc., and since it's reading two ASCII files line by line in a loop, I don't think there are any bugs in the code, except for the lack of error handling, so something must be broken in the C RTL, header files, or Clang.

Regards,
Jake Hamby

On 10/13/23 7:19 PM, Jake Hamby (Solid State Jake) wrote:

> That's not why I had to give up and build munchconfig.c with the C
> compiler. The runtime error I'm getting when I build with the C++
> compiler is that fgets() is returning NULL immediately when reading the
> first line of the second file given to it, causing NULL to be passed to
> strstr(), because the code has nonexistent error handling.
>
> I patched the code to check the return value of fgets() and do some
> logging when that happens, and what I see is feof() returns false,
> ferror() returns false, and errno is 9 (EBADF). Apparently the first
> file reads in with no problems and then it fails reading the first line
> of the second file in the loop searching for a sentinel string.

> Since the same code works correctly with the C compiler, and fails
> with Clang regardless of /Standard, /Pointer_Size, etc., and since it's
> reading two ASCII files line by line in a loop, I don't think there are
> any bugs in the code, except for the lack of error handling, so
> something must be broken in the C RTL, header files, or Clang.

Given that it sounds like fgets() is confused about which file it's
operating on, you could try moving the fclose() calls up from the end of
main() to right after it's done with each file in case that clears out
some context. Shouldn't be necesary, but it might get you past that
problem.

On Friday, October 13, 2023 at 4:04:54 PM UTC-7, Arne Vajhøj wrote:
> On 10/13/2023 6:46 PM, Arne Vajhøj wrote:
> > On 10/13/2023 4:25 PM, Vitaly Pustovetov wrote:
> >> пятница, 13 октября 2023 г. в 02:11:01 UTC+2, Arne Vajhøj:
> >>> But I think there is a problem with C I/O on VMS x86-64.
> >> In your benchmark, you compared the C function read() (+ a small
> >> overhead from Java buffering) with the C function fgets() which has
> >> much more overhead.
> >
> > They are very different in implementation.
> >
> > But C fgets and Java BufferedReader.readLine must be the standard
> > way to read lines from a file in C and Java respectively.
> >
> > And we can see that C fgets got much slower when moving
> > from Itanium to x86-64 and Java BufferedReader.readLine
> > got faster (as expected on faster hardware).
> >
> > That is a problem for C I/O. The users do not care
> > much about how the functions are implemented only about
> > the result.
> >
> > (whether it is an urgent problem for VSI to fix is another
> > question - maybe there are not that many actual C applications
> > reading millions of text lines on VMS)
> >
> >> For example, the fgets() starts its work with two system calls
> >> __PAL_PROBER and that's just the beginning...
> >
> > That is very interesting.
> >
> > I believe we have previously been told that PROBE is very
> > expensive on x86-64.
> >
> > So if fgets start with two PROBE then that may explain
> > a lot.
> >
> > That does not solve the problem, but identifying the cause
> > of a problem is always first step in solving a problem.
> I can confirm that fread is much better than fgets.
>
>
> Java 0.5s
> C fgets 20.0s
> C fread 512 byte blocks 6.2s
> C fread 5120 byte blocks 2.2s
> C fread 51200 byte blocks 1.8s
> C fread 512000 byte blocks 1.7s
>
> Code below.
>
> Arne
>
> ++++
>
> FILE *fp;
> char *buf;
> int n, bufsiz, i;
> ...
> buf = malloc(blksiz);
> fp = fopen(fnm, "r");
> n = 0;
> while((bufsiz = fread(buf, 1, blksiz, fp)) > 0)
> {
> for(i = 0; i < bufsiz; i++)
> {
> if(buf[i] == '\n') n++;
> if(n == 10000000) break; // only read 100 MB of 1 GB
> }
> }
> printf("%d lines\n", n);
> free(buf);
> fclose(fp);

I did some additional testing with PostMark 1.51, which is quite versatile and portable. I started with the "benchmark.pmrc" config file that Phoronix uses:

set transactions 250000
set size 5120 524288
set number 500
run
quit

The default settings are buffered stdio read/writes (fread/fwrite) with a 512-byte r/w size. That could explain why the VMS speed was only just over 18 MB/s.

Unfortunately, I wasn't able to improve the speed too much with this knowledge. My best results were with a 64K read/write size and unbuffered POSIX read/write. The speed was relatively close to this with reads/writes of 4K or larger.

$ mcr []postmark-1^.51.exe benchmark-64k-direct.pmrc
PostMark v1.51 : 8/14/01
Reading configuration from file 'benchmark-64k-direct.pmrc'
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1377 seconds total
1373 seconds of transactions (182 per second)

Files:
125608 created (91 per second)
Creation alone: 500 files (166 per second)
Mixed with transactions: 125108 files (91 per second)
125224 read (91 per second)
124584 appended (90 per second)
125608 deleted (91 per second)
Deletion alone: 716 files (716 per second)
Mixed with transactions: 124892 files (90 per second)

Data:
41835.69 megabytes read (30.38 megabytes per second)
42011.05 megabytes written (30.51 megabytes per second)

And here's the slightly slower 64K block size results using buffered I/O (fread/fwrite):

$ mcr []postmark-1^.51.exe benchmark-64k-buf.pmrc
PostMark v1.51 : 8/14/01
Reading configuration from file 'benchmark-64k-buf.pmrc'
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1500 seconds total
1496 seconds of transactions (167 per second)

Files:
125608 created (83 per second)
Creation alone: 500 files (250 per second)
Mixed with transactions: 125108 files (83 per second)
125224 read (83 per second)
124584 appended (83 per second)
125608 deleted (83 per second)
Deletion alone: 716 files (358 per second)
Mixed with transactions: 124892 files (83 per second)

Data:
41835.69 megabytes read (27.89 megabytes per second)
42011.05 megabytes written (28.01 megabytes per second)

There's definitely an overhead to using fopen() to open files on VMS vs. open(), but the create/open/delete numbers aren't great either way.

While I was at it, I tested some DECC$ settings and discovered that defining DECC$POSIX_COMPLIANT_PATHNAMES to "2" is slower than not setting it and using the original VMS/UNIX hybrid parsing code. When I enabled "POSIX-compliant pathnames", the file create/open/delete speed dropped to 77 files/sec. So that's a feature to avoid.

FWIW, I tried using SET RMS_DEFAULT to set /extend_quantity to a higher value like 16, or /buffer_count to 128 or the max of 255, and those attempts only made the PostMark results slower. That was the only advice in the VMS docs related to file I/O performance that I found that I could apply.

I'll start a new thread with the IPC benchmark results, which are quite similar to what I found on Itanium (2-CPU rx2620) when I tested there before the x86 port was available. Those numbers are much slower than the same x86 VM running Linux, but the fact that they're so similar to a much slower Itanium makes me wonder if there's something timing or locking-related about VMS that's not tied to raw CPU speed that limits the IPC performance.

Since I also posted some C++ benchmarks in this thread, I noticed a few subtests had unusually bad performance that threw off the final results, so I wanted to share some additional info on those.

The stepanov_vector test scored especially poorly on the "double pointer reverse reverse", "double vector reverse_iterator reverse", and "double vector iterator reverse reverse" subtests (30x slower than the first subtest), while on Linux all 8 subtests ran at ~4.65 sec / 1290.67 M, or about the same speed as the first VMS subtest.

$ mcr []stepanov_vector.exe
VMSVM1$DKA100:[home.jhamby.Benchmarks.CppPerf.CppPerformanceBenchmarks-master]stepanov_vector.EXE;1

test description absolute operations ratio with
number time per second test0

0 "double pointer verify2" 4.70 sec 1276.60 M 1.00
1 "double vector iterator" 10.72 sec 559.70 M 2.28
2 "double pointer reverse" 10.73 sec 559.18 M 2.28
3 "double vector reverse_iterator" 10.71 sec 560.22 M 2.28
4 "double vector iterator reverse" 10.71 sec 560.22 M 2.28
5 "double pointer reverse reverse" 144.76 sec 41.45 M 30.80
6 "double vector reverse_iterator reverse" 143.08 sec 41.93 M 30.44
7 "double vector iterator reverse reverse" 142.88 sec 41.99 M 30.40

Next, the mathlib test results were thrown off by the tgamma() results, which were a factor of ~3000x slower than the first subtest in the series. Something must be seriously wrong with the VMS implementation of tgamma(), perhaps due to known issues with Infinity and NaN?

61 "double tgamma" 976.64 sec 0.41 M 2872.47
55 "float tgamma" 990.51 sec 0.40 M 3095.34

Finally, in the atol benchmark, all of the sscanf results were much slower than the other subtests (this was also true in the Linux results to a lesser extent, especially sscanf "d" and "ll"):

VMSVM1$DKA100:[home.jhamby.Benchmarks.CppPerf.CppPerformanceBenchmarks-master]atol.EXE;1

test description absolute operations ratio with
number time per second test0

0 "atol" 0.43 sec 58.14 M 1.00
1 "atoi" 0.45 sec 55.56 M 1.05
2 "strtol" 3.73 sec 6.70 M 8.67
3 "strtoul" 3.47 sec 7.20 M 8.07
4 "sscanf d" 7.88 sec 3.17 M 18.33
5 "atoll" 0.42 sec 59.52 M 0.98
6 "strtoll" 3.44 sec 7.27 M 8.00
7 "strtoull" 2.99 sec 8.36 M 6.95
8 "sscanf ll" 7.57 sec 3.30 M 17.60
9 "simple_strtol" 0.28 sec 89.29 M 0.65

On Saturday, October 14, 2023 at 8:14:37 PM UTC-7, Jake Hamby (Solid State Jake) wrote:
>
> Next, the mathlib test results were thrown off by the tgamma() results, which were a factor of ~3000x slower than the first subtest in the series. Something must be seriously wrong with the VMS implementation of tgamma(), perhaps due to known issues with Infinity and NaN?
>
> 61 "double tgamma" 976.64 sec 0.41 M 2872.47
> 55 "float tgamma" 990.51 sec 0.40 M 3095.34

I nearly forgot to mention that I got a bunch of linker errors on missing math library functions if I compiled the mathlib C++ test with "-Ofast", but it worked with "-O3". BTW, here's a link to the OpenBenchmarking page for this C++ test suite. I'm using "-std=gnu++14" like in the Makefile.

https://openbenchmarking.org/test/pts/cpp-perf-bench

BTW, I noticed by using /Verbose that CXX is calling Clang with no optimization by default and "-Ofast" if you add /Opt. I believe the default /Opt level for Clang should be "-O3", to avoid breaking code that depends on correct FP behavior (denormalized numbers, infinity, etc.). GCC/Clang "-Ofast" turns on a bunch of unsafe math optimizations, and those are incompatible with the guarantees made by the default /IEEE_Mode=Denorm_Results.

Cheers,
Jake Hamby

On Saturday, October 14, 2023 at 11:29:19 PM UTC-4, Jake Hamby (Solid State Jake) wrote:
> On Saturday, October 14, 2023 at 8:14:37 PM UTC-7, Jake Hamby (Solid State Jake) wrote:
> >
> > Next, the mathlib test results were thrown off by the tgamma() results, which were a factor of ~3000x slower than the first subtest in the series. Something must be seriously wrong with the VMS implementation of tgamma(), perhaps due to known issues with Infinity and NaN?
> >
> > 61 "double tgamma" 976.64 sec 0.41 M 2872.47
> > 55 "float tgamma" 990.51 sec 0.40 M 3095.34
> I nearly forgot to mention that I got a bunch of linker errors on missing math library functions if I compiled the mathlib C++ test with "-Ofast", but it worked with "-O3". BTW, here's a link to the OpenBenchmarking page for this C++ test suite. I'm using "-std=gnu++14" like in the Makefile.
>
> https://openbenchmarking.org/test/pts/cpp-perf-bench
>
> BTW, I noticed by using /Verbose that CXX is calling Clang with no optimization by default and "-Ofast" if you add /Opt. I believe the default /Opt level for Clang should be "-O3", to avoid breaking code that depends on correct FP behavior (denormalized numbers, infinity, etc.). GCC/Clang "-Ofast" turns on a bunch of unsafe math optimizations, and those are incompatible with the guarantees made by the default /IEEE_Mode=Denorm_Results.
>
> Cheers,
> Jake Hamby
I'll have the CXX team review those settings. I've been watching all the posts and we're doing triaging now.

On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>
> But I could skip it in this particular cases where we
> know that the 10 million lines check will trigger first.
>
> A little test showed absolutely no difference.
>

Thanks for checking. So much about this is counter-intuitive that
it's now worth not making assumptions about anything.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
> On 10/13/2023 7:26 PM, Arne Vajhøj wrote:
>> On 10/13/2023 2:33 PM, Simon Clubley wrote:
>>> On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>> On 10/13/2023 8:52 AM, Simon Clubley wrote:
>>>>> Are you in a position to try direct QIO reads in C on both
>>>>> architectures
>>>>> so you cut out both the language RTL and RMS ?
>>>>
>>>> Reading blocks with SYS$QIO(W) and parsing lines manually is
>>>> not so much fun. But it is certainly doable.
>>>>
>>>
>>> How difficult would it be to directly call RMS in record mode instead
>>> from your test ? That would eliminate most of the CRTL as a source of
>>> overhead.
>>
>> SYS$GET is not that hard to use.
>
> I don't know about "overhead".
>
> C fgets 20.0s
> sys$get 56.7s
>
> (but it looks like Pascal readln may be going directly to sys$get!!)
>

Ok. That's the last time I make assumptions like that. :-)

Maybe I should start writing all my performance-critical code in Java... :-)

Thanks for doing the test.

On a more serious node, that does point to a serious overhead with
calling RMS on x86-64 VMS, because from your other tests, I do agree
that Pascal does indeed _appear_ to be calling sys$get() directly
as the numbers are just too close.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>
> I can confirm that fread is much better than fgets.
>
>
> Java 0.5s
> C fgets 20.0s
> C fread 512 byte blocks 6.2s
> C fread 5120 byte blocks 2.2s
> C fread 51200 byte blocks 1.8s
> C fread 512000 byte blocks 1.7s
>

Ouch!

Even when you start reading stupid sized blocks in C, Java is still
several times faster (and as you point out elsewhere, Java has to do
the character encoding conversion as well).

I would love to know what on earth is going on here.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On Mon, 2023-10-16 at 12:27 +0000, Simon Clubley wrote:
> Even when you start reading stupid sized blocks in C, Java is still
> several times faster (and as you point out elsewhere, Java has to do
> the character encoding conversion as well).
>
> I would love to know what on earth is going on here.

At a guess, the C/C++ RTL libraries linked in are probably going
through several layers of abstraction until it gets down to the VMS
libraries.
--
Tactical Nuclear Kittens

понедельник, 16 октября 2023 г. в 14:27:39 UTC+2, Simon Clubley:
> Even when you start reading stupid sized blocks in C, Java is still
> several times faster (and as you point out elsewhere, Java has to do
> the character encoding conversion as well).
Java uses the read() function rather than the fread() function.

On 2023-10-14 01:04, Arne Vajhøj wrote:
> I can confirm that fread is much better than fgets.
>
>
> Java                            0.5s
> C fgets                        20.0s
> C fread 512 byte blocks         6.2s
> C fread 5120 byte blocks        2.2s
> C fread 51200 byte blocks       1.8s
> C fread 512000 byte blocks      1.7s

Don't surprise me at all.
fread supposedly go directly from disk to user buffer. That can be done
nicely directly through RMS.

fgets on the other hand should read into internal buffers in libc,
processed, and then fed to the program. So there are multiple copies
going on, and processing of the characters in there in addition.
Although it must also be doing things in a rather inefficient way to be
*that* much worse, I suspect.

So I expect it should be possible to get it to perform better, but it
will always be much worse than fread.

Hmm. I wouldn't be surprised if fgets() in the end does a read and
processing one character at a time through several layers.

Johnny

On 10/16/2023 8:18 AM, Simon Clubley wrote:
> On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> On 10/13/2023 7:26 PM, Arne Vajhøj wrote:
>>> On 10/13/2023 2:33 PM, Simon Clubley wrote:
>>>> On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>>> On 10/13/2023 8:52 AM, Simon Clubley wrote:
>>>>>> Are you in a position to try direct QIO reads in C on both
>>>>>> architectures
>>>>>> so you cut out both the language RTL and RMS ?
>>>>>
>>>>> Reading blocks with SYS$QIO(W) and parsing lines manually is
>>>>> not so much fun. But it is certainly doable.
>>>>
>>>> How difficult would it be to directly call RMS in record mode instead
>>>> from your test ? That would eliminate most of the CRTL as a source of
>>>> overhead.
>>>
>>> SYS$GET is not that hard to use.
>>
>> I don't know about "overhead".
>>
>> C fgets 20.0s
>> sys$get 56.7s
>>
>> (but it looks like Pascal readln may be going directly to sys$get!!)
>
> Ok. That's the last time I make assumptions like that. :-)
>
> Maybe I should start writing all my performance-critical code in Java... :-)
>
> Thanks for doing the test.
>
> On a more serious node, that does point to a serious overhead with
> calling RMS on x86-64 VMS, because from your other tests, I do agree
> that Pascal does indeed _appear_ to be calling sys$get() directly
> as the numbers are just too close.

The Pascal numbers were already bad on Itanium. They just
got worse on x86-64.

I suspect that the concept in SYS$GET of switching from
user mode to executive mode for each line get worse
as physical IO get faster.

On a VMS 4.4 reading data from a RA81/RA82 disk it would have
taken forever to get the data from the plates to VMS, so N
switch from U to E was not that visible.

On a VMS 8.4/9.2 reading data from SSD or XFC cached HDD
getting the data i fast and the U to E overhead means a lot
more.

Disclaimer: there is still the possibility that Hein or
another RMS guru can give us the magic options that will
speed RMS up a lot.

Arne