On 10/12/2023 8:18 AM, Simon Clubley wrote:
> On 2023-10-11, Craig A. Berry <craigberry@nospam.mac.com> wrote:
>> On 10/11/23 2:50 PM, John Reagan wrote:
>>> If you use clang as a foreign-command, it will see the ".c" suffix and compile
>>> as a C program. The CXX command always assumes the source is C++
>>> regardless of the suffix (just like Alpha/Itanium).
>>
>> I'm not the expert here (John is), but I think the source *can* be valid
>> (albeit "dumb") C++ even if it's nominally C. For example, the Perl
>> sources are tested to build with C++ compilers because some people write
>> extensions in C++, but almost no one builds Perl core in C.
>
> There is one area I immediately know of in which that has not been true
> until recently (C++20) and that is C's designated initialisers. Until
> recently, they were not supported in C++.
>
> You need to compile such a program in C mode, not C++ mode. No prizes for
> guessing how I know all this. :-)

I believe the classic C vs C++ difference is:

#include <stdio.h>

int main()
{ printf("%d\n", (int)sizeof(' '));
return 0;
}

It compiles fine as both C and C++ but print 4 and 1 respectively.

Arne

On 10/12/23 1:38 PM, Simon Clubley wrote:
> On 2023-10-12, Jake Hamby (Solid State Jake) <jake.hamby@gmail.com> wrote:
>>
>> I think it's just a fact of life that Linux is going to be ~61x the speed of VMS for file creation and deletion, compared to a journaled filesystem like ext4. I have host-side disk caching enabled for both VMs, using the LsiLogic controller.
>>
>
> That would seem to mostly address any concerns about the different levels
> of filesystem caching in VMS and Linux.
>
>> The 61x performance gap in PostMark for reads and writes is more worrisome and I'll have to see if there's an easy way to modify the code to close the gap. Based on both my own experiences and reading the stories of others posted here, I'm convinced that tuning for performance is VMS's Achilles' heel, because there are so many system parameters and quota limits and buffer size adjustments that just aren't needed on Linux/UNIX, Mac, or Windows.
>>
>
> Yes it is. Are your tests going via RMS ?
>
> If so, do you have any tests that can do QIOs directly so you bypass RMS
> and go directly to the VMS kernel ?
>
> If you are going via RMS, we don't know from these numbers how much time
> is being lost in RMS and how much is being lost in VMS kernel mode code.

Probably neither, directly. One would have to know what the CRTL is
doing, generally I suspect RMS for fread()/fwrite() and QIO for
read()/write(), but I believe the reality is more complicated.

You'd especially need to know if the benchmark is tripping over any of
the known performance killers, such as closing and reopening a file for
each I/O when writing in append mode with shared access. Locale lookups
can be expensive and the CRTL does a lot more of them than it really
needs to. AND then there's handling filenames that may be in Unix or VMS
syntax such that the CRTL doesn't know if "foo" is the directory
foo.DIR;1 or just a plain file until it's done a lookup on it. If doing
network I/O there is the socket buffer that defaults to 255 bytes and
can't be changed without elevated privileges. These are the first few
that bubbled up in memory, but I'm sure there are more.

On 10/12/23 1:59 PM, Jake Hamby (Solid State Jake) wrote:
> On Wednesday, October 11, 2023 at 6:55:02 PM UTC-7, Craig A. Berry wrote:
>> As documented in the release notes, initializing long doubles is not
>> supported yet. You'll want to configure with something like this:
>>
>> @configure -"Dusedevel" -"DDEBUGGING" -"Duser_c_flags=/L_DOUBLE=64"
>> -"Dusevmsdebug" -"des"
>
> Adding "/L_DOUBLE=64" was exactly the workaround I needed, thanks!
>

> My build appears to be progressing now. I didn't use the debugging
On 10/12/23 1:59 PM, Jake Hamby (Solid State Jake) wrote:

Sounds like I need to fix the release notes. If you have a pointer to
where it said that, send it along. Shortening symbols has been required
for some time.

> Now I'm running into the same build failures due to missing targets
that I was seeing with Itanium. I think I have to deassign all of the
DECC$ logicals that the JDK startup script sets:

> $ define decc$fd_locking true
> $ define decc$efs_case_preserve enable ! Enable ODS-5 names
> $ define decc$readdir_dropdotnotype enable
> $ define decc$efs_charset enable
> $ define decc$argv_parse_style enable
>

Perl sets several of these at start-up for its own use. One that it
can't meaningfully set is DECC$ARGV_PARSE_STYLE, which is needed for
some of the tests to pass but the point at which it is added by Perl is
after the command line has already been processed.

> If decc$readdir_dropdotnotype is enabled, the build fails fairly
> early because it adds and extra "." into a directory name If I
> deassign that logical only, I get a build failure about not finding
> a rule to build DynaLoader. This is immediately after it builds
> perl.exe and is starting to build the shared Perl modules.
I have had to disable some in the past. Which ones depends on a variety
of constantly-changing factors.

> I started from a newly-extracted Perl 5.38.0 source tree and
> deassigned all my DECC$* logicals, and now I'm crossing my fingers
> that this will get me past the DynaLoader build failure. I doubt that
> "set proc/parse=extended" is incompatible, right?

That one should be ok. I have it on all the time.

On 10/12/23 2:34 PM, Jake Hamby (Solid State Jake) wrote:
> On Thursday, October 12, 2023 at 11:59:24 AM UTC-7, Jake Hamby (Solid State Jake) wrote:
>>
>> If decc$readdir_dropdotnotype is enabled, the build fails fairly early because it adds and extra "." into a directory name. If I deassign that logical only, I get a build failure about not finding a rule to build DynaLoader. This is immediately after it builds perl.exe and is starting to build the shared Perl modules.
>>
>> I started from a newly-extracted Perl 5.38.0 source tree and deassigned all my DECC$* logicals, and now I'm crossing my fingers that this will get me past the DynaLoader build failure. I doubt that "set proc/parse=extended" is incompatible, right?
>
> No luck. Here's the build failure I can't get past:
>
> MCR Sys$Disk:[]miniperl.exe "-I[.lib]" make_ext.pl "MAKE=MMS" "DynaLoader"
> Generating a MMS-style Descrip.MMS
> Writing Descrip.MMS for DynaLoader
>
>
> MCR DKA0:[local.perl-5^.38^.0]miniperl.exe;2 "-I[--.lib]" DynaLoader_pm.PL DynaLoader.pm
> MCR DKA0:[local.perl-5^.38^.0]miniperl.exe;2 "-I[--.lib]" -MExtUtils::Mksymlists -e "Mksymlists('NAME'=>'DynaLoader', 'DLBASE' => 'PL_DynaLoader', 'DL_FUNCS' => { }, 'FUNCLIST' => [], 'IMPORTS' => { }, 'DL_VARS' => []);"
> MCR DKA0:[local.perl-5^.38^.0]miniperl.exe;2 -e "print ""[--.lib.auto.DynaLoader]DynaLoader.olb/Include=DynaLoader\n[--.lib.auto.DynaLoader]DynaLoader.olb/Library\n"";" >>DynaLoader.opt
> MCR DKA0:[local.perl-5^.38^.0]miniperl.exe;2 -e "print qq{PerlShr/Share\n}" >>DynaLoader.opt
> MCR DKA0:[local.perl-5^.38^.0]miniperl.exe;2 "-I[--.lib]" "-MExtUtils::Command" -e "cp" "--" DynaLoader.opt [-.lib.auto.DynaLoader]DynaLoader.opt
> %MMS-F-GWKNOACTS, Actions to update DynaLoader.obj are unknown.
> %MMS-F-GWKNOACTS, Actions to update DynaLoader.obj are unknown.
>
>
> %MMS-F-GWKNOACTS, Actions to update DynaLoader.obj are unknown.
> %MMS-F-GWKNOACTS, Actions to update DynaLoader.obj are unknown.
>
>
> Unsuccessful make(ext/DynaLoader): code=1024 at make_ext.pl line 584.
> %MMS-F-NOMSG, Message number 00EE805C
> %MMS-F-ABORT, For target DynaLoader.obj, CLI returned abort status: %X00EE805C.
> -MMS-F-GWKNOACTS, Actions to update !AS are unknown.
>
> Any ideas what's going on here? I'm going to try renaming the build
directory to "perl-5_38" on the off chance that the "." in the build
directory name is a problem.

MMS doesn't work :-(. You'll have to use MMK. Something about the
generated rules is just too complicated for MMS to follow. Yeah, we
should document this better. But MMS used to work. And it might work
again if it gets fixed. So I'm reluctant to add docs that say not to use
something that people are likely to have already and that should work.

Another gotcha to keep an eye out for. One of the tests (sorry, don't
remember which one) tries to generate an infinity by multiplying a large
number by another large number in a tight loop. But infinities aren't
working on x86 yet. So that test gets stuck in a cpu-bound loop and you
have to kill the subprocess that's running it for the test suite to proceed.

On Thursday, October 12, 2023 at 1:37:01 PM UTC-7, Craig A. Berry wrote:
>
> MMS doesn't work :-(. You'll have to use MMK. Something about the
> generated rules is just too complicated for MMS to follow. Yeah, we
> should document this better. But MMS used to work. And it might work
> again if it gets fixed. So I'm reluctant to add docs that say not to use
> something that people are likely to have already and that should work.
>
> Another gotcha to keep an eye out for. One of the tests (sorry, don't
> remember which one) tries to generate an infinity by multiplying a large
> number by another large number in a tight loop. But infinities aren't
> working on x86 yet. So that test gets stuck in a cpu-bound loop and you
> have to kill the subprocess that's running it for the test suite to proceed.

I thought it might be MMS, so I compiled MMK and that got me past the failure. I wonder if MMS could be made to work if you used the /EXTENDED_SYNTAX option.

As for long symbol names, search the CXX release notes for:

"The compiler does not automatically truncate external names longer than 31
characters. External names can be of any length. There are symbol length
limits in the LIBRARIAN and in LINKER options files but in general the LINKER
doesn't care about symbol lengths. The /NAMES=TRUNCATED option can be used
to revert to the IA64 C++ behavior."

also:

"/NAMES=(TRUNCATED,SHORTENDED)
-names2={truncated|shortened} option
This option controls whether or not external names greater than
31 characers get truncated or shortened.
Two possible options are "truncated" and "shortened".
* "truncated" - Truncates long external names to first 31 characters.
* "shortened" - shortens long external names
A shortened name consists of the first 23 characters of the name
followed by a 7-character Cyclic Redundancy Check (CRC) computed
by looking at the full name, and then a "$".
By default, external names can be of any length. Itanium C++ defaults
to /NAMES=TRUNCATED."

There's a typo in "characters" too.

In addition to switching to MMK, I had to hack the Perl configure script to generate appropriate values for the long double type, since the 128-bit values, except for the detected size, are hardcoded in the script, even if long double is detected as 64 bits. The long double type should be changed to "0", for LONG_DOUBLE_IS_DOUBLE, and the inf and nan bit patterns and mantissa size should be the same as double.

I also had to add "/L_DOUBLE=64" to the "Checkcc" definition in the configure script or else it detects a 128-bit long double and I'm not sure if that affects the results of any of the other tests.

If this Perl build works, then I may try using CXX after that, but I know from an earlier quick test that it fails almost immediately with a pointer size mismatch on the second argument to getopt(). Since getopt() works with the C compiler, I assume it's some header file issue with C++.

The Perl configure script also defines "/Standard=Relaxed" for CXX, which isn't allowed, and the configure test C sources expect either "__DECC" or "__DECCXX" to be defined, and it appears neither of them are.

I fixed both of those issues, but most of the C configure tests are still failing with header files and functions not found, and I don't know enough DCL to figure out an easy way to have it output the build failures so I know what's going on.

Regards,
Jake Hamby

On 2023-10-12 19:53, Simon Clubley wrote:
> On 2023-10-12, Johnny Billquist <bqt@softjar.se> wrote:
>> On 2023-10-12 14:27, Simon Clubley wrote:
>>> On 2023-10-11, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>> On 10/11/2023 9:01 AM, Simon Clubley wrote:
>>>>> On 2023-10-10, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>>>> On 10/10/2023 3:13 PM, Jake Hamby (Solid State Jake) wrote:
>>>>>>> As promised, here's another post with Linux vs. VMS performance on OpenSSH, C++ microbenchmarks, PostMark, PerlBench, and PHPBench.
>>>>>>>
>>>>>>> First, OpenSSH, which suffers greatly from not being built for VMS
>>>>>>> with x86 assembler optimizations.
>>>>>>
>>>>>> Probably important.
>>>>>>
>>>>>
>>>>> Even allowing for that, and allowing for un-optimised compilers, those
>>>>> numbers are horrible.
>>>>
>>>> The difference between un-optimized and optimized can be
>>>> pretty big - like a factor 3-5.
>>>>
>>>> The difference between generic C code implementation
>>>> of algorithm and assembler using special instructions
>>>> could be even bigger - the AES-NI instructions for AES
>>>> are supposedly a factor 4-8 faster.
>>>>
>>>> So I don't see anything horrible in the numbers.
>>>>
>>>
>>> I think it's reasonable for the CPU tests to see what happens with
>>> finished compilers.
>>>
>>> However, look again at the filesystem numbers.
>>>
>>> Even with unoptimised compilers, they should not be _that_ bad.
>>
>> What kind of file system related operations do OpenSSH do???
>>
>
> If you actually read the posting Johnny, you will see there are many
> tests, not just OpenSSL ones. These include filesystem tests with
> hideous VMS performance results when compared to Linux.

Well, to me it appeared as if we were basically talking about the
OpenSSH numbers here, since the comment thread above really related
specifically to that bit.

And those numbers were really different on Linux and VMS.

So maybe you should have made the comments more on a thread about the DB
and fs parts then, if it was those numbers you were going after?

Johnny

On 10/12/2023 6:07 PM, Johnny Billquist wrote:
> On 2023-10-12 19:53, Simon Clubley wrote:
>> On 2023-10-12, Johnny Billquist <bqt@softjar.se> wrote:
>>> On 2023-10-12 14:27, Simon Clubley wrote:
>>>> On 2023-10-11, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>>> On 10/11/2023 9:01 AM, Simon Clubley wrote:
>>>>>> On 2023-10-10, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>>>>> On 10/10/2023 3:13 PM, Jake Hamby (Solid State Jake) wrote:
>>>>>>>> As promised, here's another post with Linux vs. VMS performance
>>>>>>>> on OpenSSH, C++ microbenchmarks, PostMark, PerlBench, and PHPBench.
>>>>>>>>
>>>>>>>> First, OpenSSH, which suffers greatly from not being built for VMS
>>>>>>>> with x86 assembler optimizations.
>>>>>>>
>>>>>>> Probably important.
>>>>>>>
>>>>>>
>>>>>> Even allowing for that, and allowing for un-optimised compilers,
>>>>>> those
>>>>>> numbers are horrible.
>>>>>
>>>>> The difference between un-optimized and optimized can be
>>>>> pretty big - like a factor 3-5.
>>>>>
>>>>> The difference between generic C code implementation
>>>>> of algorithm and assembler using special instructions
>>>>> could be even bigger - the AES-NI instructions for AES
>>>>> are supposedly a factor 4-8 faster.
>>>>>
>>>>> So I don't see anything horrible in the numbers.
>>>>>
>>>>
>>>> I think it's reasonable for the CPU tests to see what happens with
>>>> finished compilers.
>>>>
>>>> However, look again at the filesystem numbers.
>>>>
>>>> Even with unoptimised compilers, they should not be _that_ bad.
>>>
>>> What kind of file system related operations do OpenSSH do???
>>>
>>
>> If you actually read the posting Johnny, you will see there are many
>> tests, not just OpenSSL ones. These include filesystem tests with
>> hideous VMS performance results when compared to Linux.
>
> Well, to me it appeared as if we were basically talking about the
> OpenSSH numbers here, since the comment thread above really related
> specifically to that bit.
>
> And those numbers were really different on Linux and VMS.
>
> So maybe you should have made the comments more on a thread about the DB
> and fs parts then, if it was those numbers you were going after?

Yes.

Most usenet readers tend to assume that comments relate to the
stuff quoted - not the stuff not quoted.

Arne

On 10/12/2023 2:38 PM, Simon Clubley wrote:
> On 2023-10-12, Jake Hamby (Solid State Jake) <jake.hamby@gmail.com>
> wrote:
>> I think it's just a fact of life that Linux is going to be ~61x the
>> speed of VMS for file creation and deletion, compared to a
>> journaled filesystem like ext4. I have host-side disk caching
>> enabled for both VMs, using the LsiLogic controller.
>
> That would seem to mostly address any concerns about the different
> levels of filesystem caching in VMS and Linux.

Why?

There can be caches in disk, controller, OS and process.

That the cache is identical for controller does not prove that
cache situation is identical.

>> The 61x performance gap in PostMark for reads and writes is more
>> worrisome and I'll have to see if there's an easy way to modify the
>> code to close the gap. Based on both my own experiences and reading
>> the stories of others posted here, I'm convinced that tuning for
>> performance is VMS's Achilles' heel, because there are so many
>> system parameters and quota limits and buffer size adjustments that
>> just aren't needed on Linux/UNIX, Mac, or Windows.
>
> Yes it is. Are your tests going via RMS ?
>
> If so, do you have any tests that can do QIOs directly so you bypass
> RMS and go directly to the VMS kernel ?
>
> If you are going via RMS, we don't know from these numbers how much
> time is being lost in RMS and how much is being lost in VMS kernel
> mode code.

There are many I/O API's on VMS:
* language RTL
* RMS record I/O ($GET og $PUT)
* RMS block I/O ($READ and $WRITE)
* SYS$QIO(W)
* SYS$IO_PERFORM(W)
* mapping file to memory

I would expect only the two last to be serious candidates for
max IO throughput.

Given that the benchmarks come from Linux world, then it
seems fair to assume that it is C RTL, which again use
either RMS record I/O or RMS block I/O.

Definitely overhead. We have known that since forever.
The numbers seem bigger than expected though.

Arne

On 10/12/2023 1:57 PM, Jake Hamby (Solid State Jake) wrote:
> I snuck in one filesystem benchmark, PostMark, and left the reader to do the math. Here they are together:
>
> Time:
> Linux: 38, VMS: 2317 sec total (61.0x)
> Linux: 38, VMS: 2311 (60.8x) sec of transactions (Linux: 6578, VMS: 108 per sec) (36.5x)
>
> Files:
> 125608 created (Linux: 3305, VMS: 54 per second) (61.2x)
> Creation alone: 500 files (Linux: 500, VMS: 125 per second) (4.0x)
> Mixed with transactions: 125108 files (Linux: 3292, VMS: 54 per second) (61.0x)
> 125224 read (Linux: 3295, VMS: 54 per second) (61.0x)
> 124584 appended (Linux: 3278, VMS: 53 per second) (61.8x)
> 125608 deleted (Linux: 3305, VMS: 54 per second) (61.2x)
> Deletion alone: 716 files (Linux: 716, VMS: 358 per second) (2.0x)
> Mixed with transactions: 124892 files (Linux: 3286, VMS: 54 per second) (60.9x)
>
> Data:
> 41835.69 megabytes read (Linux: 1100.94, VMS: 18.06 megabytes per second) (61.0x)
> 42011.05 megabytes written (Linux: 1105.55, VMS: 18.13 megabytes per second) (61.0x)

> I think it's just a fact of life that Linux is going to be ~61x the
> speed of VMS for file creation and deletion, compared to a journaled
> filesystem like ext4. I have host-side disk caching enabled for both
> VMs, using the LsiLogic controller.
>
> The 61x performance gap in PostMark for reads and writes is more
> worrisome and I'll have to see if there's an easy way to modify the
> code to close the gap. Based on both my own experiences and reading
> the stories of others posted here, I'm convinced that tuning for
> performance is VMS's Achilles' heel, because there are so many system
> parameters and quota limits and buffer size adjustments that just
> aren't needed on Linux/UNIX, Mac, or Windows.

Large scale directory modification (file creation, file deletion) is
known to be slow on VMS. Creating thousands of files in a single
directory has always been a performance killer.

I don't think it is a big problem - having a need to create or delete
that many files is rarely a good design.

And anyway VSI can probably not fix it until we get a new file
system - and that will not happen in the near future.

The real worry is the x18 on read and write. Everybody is
expecting VMS IO to be slower than Linux IO. But x18 seems
like a lot. My Java example only had a factor x1.5
difference. Something is wrong here that VSI need to
investigate.

SSD and a controller cache obviously increase the factor
due to less time spent in storage HW, but x18 is still too much.

Arne

On 10/12/2023 7:36 PM, Arne Vajhøj wrote:
> On 10/12/2023 1:57 PM, Jake Hamby (Solid State Jake) wrote:
>> I snuck in one filesystem benchmark, PostMark, and left the reader to
>> do the math. Here they are together:
>>
>> Time:
>> Linux: 38, VMS: 2317 sec total (61.0x)
>> Linux: 38, VMS: 2311 (60.8x) sec of transactions (Linux: 6578, VMS:
>> 108 per sec) (36.5x)
>>
>> Files:
>> 125608 created (Linux: 3305, VMS: 54 per second) (61.2x)
>> Creation alone: 500 files (Linux: 500, VMS: 125 per second) (4.0x)
>> Mixed with transactions: 125108 files (Linux: 3292, VMS: 54 per
>> second) (61.0x)
>> 125224 read (Linux: 3295, VMS: 54 per second) (61.0x)
>> 124584 appended (Linux: 3278, VMS: 53 per second) (61.8x)
>> 125608 deleted (Linux: 3305, VMS: 54 per second) (61.2x)
>> Deletion alone: 716 files (Linux: 716, VMS: 358 per second) (2.0x)
>> Mixed with transactions: 124892 files (Linux: 3286, VMS: 54 per
>> second) (60.9x)
>>
>> Data:
>> 41835.69 megabytes read (Linux: 1100.94, VMS: 18.06 megabytes per
>> second) (61.0x)
>> 42011.05 megabytes written (Linux: 1105.55, VMS: 18.13 megabytes per
>> second) (61.0x)
>
>> I think it's just a fact of life that Linux is going to be ~61x the
>> speed of VMS for file creation and deletion, compared to a journaled
>> filesystem like ext4. I have host-side disk caching enabled for both
>> VMs, using the LsiLogic controller.
>>
>> The 61x performance gap in PostMark for reads and writes is more
>> worrisome and I'll have to see if there's an easy way to modify the
>> code to close the gap. Based on both my own experiences and reading
>> the stories of others posted here, I'm convinced that tuning for
>> performance is VMS's Achilles' heel, because there are so many system
>> parameters and quota limits and buffer size adjustments that just
>> aren't needed on Linux/UNIX, Mac, or Windows.
>
> Large scale directory modification (file creation, file deletion) is
> known to be slow on VMS. Creating thousands of files in a single
> directory has always been a performance killer.
>
> I don't think it is a big problem - having a need to create or delete
> that many files is rarely a good design.
>
> And anyway VSI can probably not fix it until we get a new file
> system - and that will not happen in the near future.
>
> The real worry is the x18 on read and write. Everybody is
> expecting VMS IO to be slower than Linux IO. But x18 seems
> like a lot. My Java example only had a factor x1.5
> difference. Something is wrong here that VSI need to
> investigate.
>
> SSD and a controller cache obviously increase the factor
> due to less time spent in storage HW, but x18 is still too much.

One potential reason for the big difference could be that
the device drivers on the Linux guest does something smart
knowing that it is running virtual while the device drivers
on VMS has no idea. I have no idea what Ubuntu in VirtualBox
on Ubuntu offers in that regard. It could obviously be tested
by running Ubuntu and VMS directly on metal.

But I think there is a problem with C I/O on VMS x86-64.

I have my little read and count all lines nano-benchmark
in both C and Java.

Reading 10 million lines of 10 bytes each.

C Java

VMS 8
old itanium 2.9s 1.5s
HDD

VMS 9
VMWare Player 20.0s 0.5s
Windows 10
PC
SSD

Something is wrong here.

And the code is bloody simple:

FILE *fp;
char line[1000];
int n;
....
fp = fopen(fnm, "r");
n = 0;
while(fgets(line, sizeof(line), fp))
{
n++;
if(n == 10000000) break; // only read 100 MB of 1 GB
}
printf("%d lines\n", n);
fclose(fp);

and:

try(BufferedReader br = new BufferedReader(new FileReader(fnm))) {
int n = 0;
String line;
while((line = br.readLine()) != null) {
n++;
if(n == 10000000) break; // only read 100 MB of 1 GB
}
System.out.printf("%d lines\n", n);
}

Arne

On 10/12/23 4:29 PM, Jake Hamby (Solid State Jake) wrote:
> On Thursday, October 12, 2023 at 1:37:01 PM UTC-7, Craig A. Berry wrote:
>>
>> MMS doesn't work :-(. You'll have to use MMK.
>
> I thought it might be MMS, so I compiled MMK and that got me past the failure. I wonder if MMS could be made to work if you used the /EXTENDED_SYNTAX option.

Good thought. I've never tried it.

> As for long symbol names, search the CXX release notes for:
>
> "The compiler does not automatically truncate external names longer than 31
> characters.

Ah, *those* release notes. I thought you meant Perl's release notes.
Shortened symbols are a requirement for Perl because they've been a
requirement for every compiler on VMS since ages up until this one. In
Perl we actually predict what the shortened names will be so we can
build the linker options file. Even if C++ can now produce long symbols,
and even when the rest of the VMS toolchain catches up, you wouldn't be
able to link with objects built with another compiler. Just use
/names=shortened for now; if and when all of the pieces are in place to
change that, it can be reconsidered.

> In addition to switching to MMK, I had to hack the Perl configure
script to generate appropriate values for the long double type, since
the 128-bit values, except for the detected size, are hardcoded in the
script, even if long double is detected as 64 bits. The long double type
should be changed to "0", for LONG_DOUBLE_IS_DOUBLE, and the inf and nan
bit patterns and mantissa size should be the same as double.
>
> I also had to add "/L_DOUBLE=64" to the "Checkcc" definition in the
configure script or else it detects a 128-bit long double and I'm not
sure if that affects the results of any of the other tests.

I haven't done these things to build with C and have had no trouble
building. And modifying the configuration to pretend that 128-bit
doubles are not available when they actually are (albeit not fully
implemented in the field test compiler) doesn't seem right.

> If this Perl build works, then I may try using CXX after that, but I
know from an earlier quick test that it fails almost immediately with a
pointer size mismatch on the second argument to getopt(). Since getopt()
works with the C compiler, I assume it's some header file issue with C++.
>

getopt was one of several things that has made it difficult to
contemplate building Perl with 64-bit pointers previously. I know some
work has been done on the CRTL so maybe getopt() now deals with 64-bit
pointers, at least on x86.

> The Perl configure script also defines "/Standard=Relaxed" for CXX, which isn't allowed,

Not allowed as in throws an error message? I'm skeptical that it
actually does that. The code to set that flag looks like this:

$ IF ccname .EQS. "CXX"
$ THEN
$ ccflags="/Include=[]/Standard=ANSI/Prefix=All/Obj=''obj_ext' ''ccflags'"
$ ENDIF

If you are getting Relaxed_Ansi rather than Ansi, then it's not
recognizing that you have a C++ compiler. There is probably a bug in
detecting a compiler it's never seen before.

> and the configure test C sources expect either "__DECC" or
> "__DECCXX" to be defined, and it appears neither of them are.

The documentation for all previous C++ compilers on VMS says __DECXX is
defined, so if it's not, there is a problem somewhere, but probably not
one the Perl configuration can fix short of reconsidering from the
ground up what a compiler invoked with "CXX" means.

> I fixed both of those issues, but most of the C configure tests are
still failing with header files and functions not found, and I don't
know enough DCL to figure out an easy way to have it output the build
failures so I know what's going on.
>

You don't have to do anything special to output the errors that lead to
build failures. I find it most convenient to submit a batch job and
review the log file, but if you're just running it in a terminal,
whatever caused the build failure should be readily apparent.

If you're talking about building the trial C programs that are used in
the configuration process to probe what the compiler can do, then
copy-pasting those programs in to little C files and compiling them is
probably your best bet.

On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
>
> I believe the classic C vs C++ difference is:
>
> #include <stdio.h>
>
> int main()
> {
> printf("%d\n", (int)sizeof(' '));
> return 0;
> }
>
> It compiles fine as both C and C++ but print 4 and 1 respectively.
>

I didn't know about that one, probably because you would not catch me
writing code that fragile. :-)

If I need to know how much space something like that takes up, _I_ set
the amount of allocated space per-whatever instead. As you well know :-),
there are data types which allow you to guarantee that they allocate a
specific size.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
> On 10/12/2023 2:38 PM, Simon Clubley wrote:
>> On 2023-10-12, Jake Hamby (Solid State Jake) <jake.hamby@gmail.com>
>> wrote:
>>> I think it's just a fact of life that Linux is going to be ~61x the
>>> speed of VMS for file creation and deletion, compared to a
>>> journaled filesystem like ext4. I have host-side disk caching
>>> enabled for both VMs, using the LsiLogic controller.
>>
>> That would seem to mostly address any concerns about the different
>> levels of filesystem caching in VMS and Linux.
>
> Why?
>
> There can be caches in disk, controller, OS and process.
>
> That the cache is identical for controller does not prove that
> cache situation is identical.
>

Yeah, although I did say "mostly", I still could have certainly qualified
that statement a lot better. :-)

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
> On 10/12/2023 1:57 PM, Jake Hamby (Solid State Jake) wrote:
>> I snuck in one filesystem benchmark, PostMark, and left the reader to do the math. Here they are together:
>>
>> Time:
>> Linux: 38, VMS: 2317 sec total (61.0x)
>> Linux: 38, VMS: 2311 (60.8x) sec of transactions (Linux: 6578, VMS: 108 per sec) (36.5x)
>>
>> Files:
>> 125608 created (Linux: 3305, VMS: 54 per second) (61.2x)
>> Creation alone: 500 files (Linux: 500, VMS: 125 per second) (4.0x)
>> Mixed with transactions: 125108 files (Linux: 3292, VMS: 54 per second) (61.0x)
>> 125224 read (Linux: 3295, VMS: 54 per second) (61.0x)
>> 124584 appended (Linux: 3278, VMS: 53 per second) (61.8x)
>> 125608 deleted (Linux: 3305, VMS: 54 per second) (61.2x)
>> Deletion alone: 716 files (Linux: 716, VMS: 358 per second) (2.0x)
>> Mixed with transactions: 124892 files (Linux: 3286, VMS: 54 per second) (60.9x)
>>
>> Data:
>> 41835.69 megabytes read (Linux: 1100.94, VMS: 18.06 megabytes per second) (61.0x)
>> 42011.05 megabytes written (Linux: 1105.55, VMS: 18.13 megabytes per second) (61.0x)
>

[snip]

> The real worry is the x18 on read and write. Everybody is
> expecting VMS IO to be slower than Linux IO. But x18 seems
> like a lot. My Java example only had a factor x1.5
> difference. Something is wrong here that VSI need to
> investigate.
>
> SSD and a controller cache obviously increase the factor
> due to less time spent in storage HW, but x18 is still too much.
>

Where are you getting the x18 from Arne ? I don't see that in the above data.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 10/13/2023 8:41 AM, Simon Clubley wrote:
> On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> On 10/12/2023 1:57 PM, Jake Hamby (Solid State Jake) wrote:
>>> I snuck in one filesystem benchmark, PostMark, and left the reader to do the math. Here they are together:
>>>
>>> Time:
>>> Linux: 38, VMS: 2317 sec total (61.0x)
>>> Linux: 38, VMS: 2311 (60.8x) sec of transactions (Linux: 6578, VMS: 108 per sec) (36.5x)
>>>
>>> Files:
>>> 125608 created (Linux: 3305, VMS: 54 per second) (61.2x)
>>> Creation alone: 500 files (Linux: 500, VMS: 125 per second) (4.0x)
>>> Mixed with transactions: 125108 files (Linux: 3292, VMS: 54 per second) (61.0x)
>>> 125224 read (Linux: 3295, VMS: 54 per second) (61.0x)
>>> 124584 appended (Linux: 3278, VMS: 53 per second) (61.8x)
>>> 125608 deleted (Linux: 3305, VMS: 54 per second) (61.2x)
>>> Deletion alone: 716 files (Linux: 716, VMS: 358 per second) (2.0x)
>>> Mixed with transactions: 124892 files (Linux: 3286, VMS: 54 per second) (60.9x)
>>>
>>> Data:
>>> 41835.69 megabytes read (Linux: 1100.94, VMS: 18.06 megabytes per second) (61.0x)
>>> 42011.05 megabytes written (Linux: 1105.55, VMS: 18.13 megabytes per second) (61.0x)
>
> [snip]
>
>> The real worry is the x18 on read and write. Everybody is
>> expecting VMS IO to be slower than Linux IO. But x18 seems
>> like a lot. My Java example only had a factor x1.5
>> difference. Something is wrong here that VSI need to
>> investigate.
>>
>> SSD and a controller cache obviously increase the factor
>> due to less time spent in storage HW, but x18 is still too much.
>
> Where are you getting the x18 from Arne ? I don't see that in the above data.

Good question.

I thought it was x18 but it says x61 all over.

My mistake. My apologies.

It certainly does not make it any better.

Arne

On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
>
> But I think there is a problem with C I/O on VMS x86-64.
>
> I have my little read and count all lines nano-benchmark
> in both C and Java.
>
> Reading 10 million lines of 10 bytes each.
>
> C Java
>
> VMS 8
> old itanium 2.9s 1.5s
> HDD
>
> VMS 9
> VMWare Player 20.0s 0.5s
> Windows 10
> PC
> SSD
>
> Something is wrong here.
>
> And the code is bloody simple:
>
> FILE *fp;
> char line[1000];
> int n;
> ...
> fp = fopen(fnm, "r");
> n = 0;
> while(fgets(line, sizeof(line), fp))
> {
> n++;
> if(n == 10000000) break; // only read 100 MB of 1 GB
> }
> printf("%d lines\n", n);
> fclose(fp);
>
> and:
>
> try(BufferedReader br = new BufferedReader(new FileReader(fnm))) {
> int n = 0;
> String line;
> while((line = br.readLine()) != null) {
> n++;
> if(n == 10000000) break; // only read 100 MB of 1 GB
> }
> System.out.printf("%d lines\n", n);
> }
>

Interesting.

Because it's a BufferedReader, Java may be reading in larger chunks
than the C RTL.

Can you recreate the code in Pascal and Fortran for both Itanium and
x86-64 so we are comparing the traditional compiled languages against
the other ?

Also, to negate the effects of the VMS read cache, you should probably
run each test 3 times in the order C/Java/C/Java/C/Java and make sure
they don't change significantly on each run.

Are you in a position to try direct QIO reads in C on both architectures
so you cut out both the language RTL and RMS ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 10/13/2023 8:52 AM, Simon Clubley wrote:
> On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>
>> But I think there is a problem with C I/O on VMS x86-64.
>>
>> I have my little read and count all lines nano-benchmark
>> in both C and Java.
>>
>> Reading 10 million lines of 10 bytes each.
>>
>> C Java
>>
>> VMS 8
>> old itanium 2.9s 1.5s
>> HDD
>>
>> VMS 9
>> VMWare Player 20.0s 0.5s
>> Windows 10
>> PC
>> SSD
>>
>> Something is wrong here.
>>
>> And the code is bloody simple:
>>
>> FILE *fp;
>> char line[1000];
>> int n;
>> ...
>> fp = fopen(fnm, "r");
>> n = 0;
>> while(fgets(line, sizeof(line), fp))
>> {
>> n++;
>> if(n == 10000000) break; // only read 100 MB of 1 GB
>> }
>> printf("%d lines\n", n);
>> fclose(fp);
>>
>> and:
>>
>> try(BufferedReader br = new BufferedReader(new FileReader(fnm))) {
>> int n = 0;
>> String line;
>> while((line = br.readLine()) != null) {
>> n++;
>> if(n == 10000000) break; // only read 100 MB of 1 GB
>> }
>> System.out.printf("%d lines\n", n);
>> }
>>
>
> Interesting.
>
> Because it's a BufferedReader, Java may be reading in larger chunks
> than the C RTL.

It probably does.

I did not use BufferedReader to get that buffer, but to get the readLine
method.

Anyway that is a possible explanation of why Java is faster than C.

It does not explain why C has become so much slower on x86-64 compared
to Itanium.

> Can you recreate the code in Pascal and Fortran for both Itanium and
> x86-64 so we are comparing the traditional compiled languages against
> the other ?

I have a VMS Pascal version, but the Itanium numbers are also
miserable so not sure how much point there is.

> Also, to negate the effects of the VMS read cache, you should probably
> run each test 3 times in the order C/Java/C/Java/C/Java and make sure
> they don't change significantly on each run.

I make 10 runs of each and the number above is where it stabilized.

(the Itanium run was just with 3)

> Are you in a position to try direct QIO reads in C on both architectures
> so you cut out both the language RTL and RMS ?

Reading blocks with SYS$QIO(W) and parsing lines manually is
not so much fun. But it is certainly doable.

What is the point? The Java numbers prove that numbers can be
better using whatever IO API that Java use. And I am pretty
sure that Java does go through some C call and RMS (in C you can
add extra RMS arguments to fopen/open - you can do the same
for Java via the logical JAVA$FILENAME_MATCH_LIST).

Arne

On 10/13/2023 9:06 AM, Arne Vajhøj wrote:
> On 10/13/2023 8:52 AM, Simon Clubley wrote:
>> On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>
>>> But I think there is a problem with C I/O on VMS x86-64.
>>>
>>> I have my little read and count all lines nano-benchmark
>>> in both C and Java.
>>>
>>> Reading 10 million lines of 10 bytes each.
>>>
>>>                          C          Java
>>>
>>> VMS 8
>>> old itanium           2.9s        1.5s
>>> HDD
>>>
>>> VMS 9
>>> VMWare Player        20.0s        0.5s
>>> Windows 10
>>> PC
>>> SSD
>>>
>>> Something is wrong here.

>> Can you recreate the code in Pascal and Fortran for both Itanium and
>> x86-64 so we are comparing the traditional compiled languages against
>> the other ?
>
> I have a VMS Pascal version, but the Itanium numbers are also
> miserable so not sure how much point there is.

What the heck.

C Pascal Java
VMS 8
old itanium 2.9s 31s 1.5s
HDD

VMS 9
VMWare Player 20.0s 53s 0.5s
Windows 10
PC
SSD

Seems like Pascal has a problem but a smaller problem than C
with x86-64.

Again super simple code:

type
string = varying[32767] of char;

....

var
f : text;
line : string;
n : integer;

....

open(f, fnm, old);
reset(f);
n := 0;
while (not eof(f)) and (n < 10000000) do begin
readln(f, line);
n := n + 1;
end;
writeln(n:1,' lines');
close(f);

Arne

On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
> On 10/13/2023 8:52 AM, Simon Clubley wrote:
>> On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>
>>> But I think there is a problem with C I/O on VMS x86-64.
>>>
>>> I have my little read and count all lines nano-benchmark
>>> in both C and Java.
>>>
>>> Reading 10 million lines of 10 bytes each.
>>>
>>> C Java
>>>
>>> VMS 8
>>> old itanium 2.9s 1.5s
>>> HDD
>>>
>>> VMS 9
>>> VMWare Player 20.0s 0.5s
>>> Windows 10
>>> PC
>>> SSD
>>>
>>> Something is wrong here.
>>>
>>> And the code is bloody simple:
>>>
>>> FILE *fp;
>>> char line[1000];
>>> int n;
>>> ...
>>> fp = fopen(fnm, "r");
>>> n = 0;
>>> while(fgets(line, sizeof(line), fp))
>>> {
>>> n++;
>>> if(n == 10000000) break; // only read 100 MB of 1 GB
>>> }
>>> printf("%d lines\n", n);
>>> fclose(fp);
>>>
>>> and:
>>>
>>> try(BufferedReader br = new BufferedReader(new FileReader(fnm))) {
>>> int n = 0;
>>> String line;
>>> while((line = br.readLine()) != null) {
>>> n++;
>>> if(n == 10000000) break; // only read 100 MB of 1 GB
>>> }
>>> System.out.printf("%d lines\n", n);
>>> }
>>>
>>
>> Interesting.
>>
>> Because it's a BufferedReader, Java may be reading in larger chunks
>> than the C RTL.
>
> It probably does.
>
> I did not use BufferedReader to get that buffer, but to get the readLine
> method.
>
> Anyway that is a possible explanation of why Java is faster than C.
>

One way to confirm this would be to record the current number of direct
I/Os before and after each instance of your tests. The DIRIO number for
Java _should_ be rather lower if this theory is correct. This is what I
am thinking of:

$ write sys$output f$getjpi("", "DIRIO")

(maybe store in a variable somewhere for later comparison).

There might also be another factor in _how_ RMS is being used. See below.

> It does not explain why C has become so much slower on x86-64 compared
> to Itanium.
>

No it doesn't, and that's something which needs fixing.

>> Are you in a position to try direct QIO reads in C on both architectures
>> so you cut out both the language RTL and RMS ?
>
> Reading blocks with SYS$QIO(W) and parsing lines manually is
> not so much fun. But it is certainly doable.
>

How difficult would it be to directly call RMS in record mode instead
from your test ? That would eliminate most of the CRTL as a source of
overhead.

> What is the point? The Java numbers prove that numbers can be
> better using whatever IO API that Java use. And I am pretty
> sure that Java does go through some C call and RMS (in C you can
> add extra RMS arguments to fopen/open - you can do the same
> for Java via the logical JAVA$FILENAME_MATCH_LIST).
>

Because the fact Java is twice as fast as C on Itanium caught my
attention and I absolutely was _not_ expecting that.

One suggestion I offer (with no evidence to back it up :-)) is that
C might be doing RMS record-level reads and Java might be doing RMS
block-level reads and doing its own unpacking of lines.

This by itself would result in a major reduction in the number of calls
to RMS if this is correct.

If Java is also doing larger block-level direct I/Os than the normal
RMS record-level code does, that might help explain these rather dramatic
differences, given that executive mode is emulated on x86-64 VMS.

As an aside, given that Rdb lives in executive mode, I now wonder what
Oracle's initial performance figures for Rdb on x86-64 VMS look like.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>
> What the heck.
>

Thank you. :-)

> C Pascal Java
> VMS 8
> old itanium 2.9s 31s 1.5s
> HDD
>
> VMS 9
> VMWare Player 20.0s 53s 0.5s
> Windows 10
> PC
> SSD
>
> Seems like Pascal has a problem but a smaller problem than C
> with x86-64.
>

I am surprised that Pascal is that slow on Itanium, even with things
like bounds checking turned on.

If you ignore whatever else is causing the Pascal overhead, the C
and Pascal difference is roughly comparable, with Pascal being a
few seconds slower:

C: 20.0 - 2.9 = 17.1 seconds
Pascal: 53 - 31 = 22 seconds

I notice you have an explicit eof() check in the Pascal loop. I wonder
if that is adding any overhead ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

пятница, 13 октября 2023 г. в 02:11:01 UTC+2, Arne Vajhøj:
> On 10/12/2023 7:36 PM, Arne Vajhøj wrote:

> But I think there is a problem with C I/O on VMS x86-64.
In your benchmark, you compared the C function read() (+ a small overhead from Java buffering) with the C function fgets() which has much more overhead.
For example, the fgets() starts its work with two system calls __PAL_PROBER and that's just the beginning...

On 10/13/2023 4:25 PM, Vitaly Pustovetov wrote:
> пятница, 13 октября 2023 г. в 02:11:01 UTC+2, Arne Vajhøj:
>> But I think there is a problem with C I/O on VMS x86-64.
> In your benchmark, you compared the C function read() (+ a small overhead from Java buffering) with the C function fgets() which has much more overhead.

They are very different in implementation.

But C fgets and Java BufferedReader.readLine must be the standard
way to read lines from a file in C and Java respectively.

And we can see that C fgets got much slower when moving
from Itanium to x86-64 and Java BufferedReader.readLine
got faster (as expected on faster hardware).

That is a problem for C I/O. The users do not care
much about how the functions are implemented only about
the result.

(whether it is an urgent problem for VSI to fix is another
question - maybe there are not that many actual C applications
reading millions of text lines on VMS)

> For example, the fgets() starts its work with two system calls __PAL_PROBER and that's just the beginning...

That is very interesting.

I believe we have previously been told that PROBE is very
expensive on x86-64.

So if fgets start with two PROBE then that may explain
a lot.

That does not solve the problem, but identifying the cause
of a problem is always first step in solving a problem.

Arne

On 10/13/2023 6:46 PM, Arne Vajhøj wrote:
> On 10/13/2023 4:25 PM, Vitaly Pustovetov wrote:
>> пятница, 13 октября 2023 г. в 02:11:01 UTC+2, Arne Vajhøj:
>>> But I think there is a problem with C I/O on VMS x86-64.
>> In your benchmark, you compared the C function read() (+ a small
>> overhead from Java buffering) with the C function fgets() which has
>> much more overhead.
>
> They are very different in implementation.
>
> But C fgets and Java BufferedReader.readLine must be the standard
> way to read lines from a file in C and Java respectively.
>
> And we can see that C fgets got much slower when moving
> from Itanium to x86-64 and Java BufferedReader.readLine
> got faster (as expected on faster hardware).
>
> That is a problem for C I/O. The users do not care
> much about how the functions are implemented only about
> the result.
>
> (whether it is an urgent problem for VSI to fix is another
> question - maybe there are not that many actual C applications
> reading millions of text lines on VMS)
>
>> For example, the fgets() starts its work with two system calls
>> __PAL_PROBER and that's just the beginning...
>
> That is very interesting.
>
> I believe we have previously been told that PROBE is very
> expensive on x86-64.
>
> So if fgets start with two PROBE then that may explain
> a lot.
>
> That does not solve the problem, but identifying the cause
> of a problem is always first step in solving a problem.

I can confirm that fread is much better than fgets.

Java 0.5s
C fgets 20.0s
C fread 512 byte blocks 6.2s
C fread 5120 byte blocks 2.2s
C fread 51200 byte blocks 1.8s
C fread 512000 byte blocks 1.7s

Code below.

Arne

++++

FILE *fp;
char *buf;
int n, bufsiz, i;
....
buf = malloc(blksiz);
fp = fopen(fnm, "r");
n = 0;
while((bufsiz = fread(buf, 1, blksiz, fp)) > 0)
{
for(i = 0; i < bufsiz; i++)
{
if(buf[i] == '\n') n++;
if(n == 10000000) break; // only read 100 MB of 1 GB
}
}
printf("%d lines\n", n);
free(buf);
fclose(fp);

On 10/13/2023 2:45 PM, Simon Clubley wrote:
> On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> C Pascal Java
>> VMS 8
>> old itanium 2.9s 31s 1.5s
>> HDD
>>
>> VMS 9
>> VMWare Player 20.0s 53s 0.5s
>> Windows 10
>> PC
>> SSD
>>
>> Seems like Pascal has a problem but a smaller problem than C
>> with x86-64.
>
> I am surprised that Pascal is that slow on Itanium, even with things
> like bounds checking turned on.

There is not that much bounds checking in the code.

> If you ignore whatever else is causing the Pascal overhead, the C
> and Pascal difference is roughly comparable, with Pascal being a
> few seconds slower:
>
> C: 20.0 - 2.9 = 17.1 seconds
> Pascal: 53 - 31 = 22 seconds

Maybe that means something.

> I notice you have an explicit eof() check in the Pascal loop. I wonder
> if that is adding any overhead ?

That is how a text file is read in pascal.

But I could skip it in this particular cases where we
know that the 10 million lines check will trigger first.

A little test showed absolutely no difference.

Arne

On 10/13/2023 2:33 PM, Simon Clubley wrote:
> On 2023-10-13, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> On 10/13/2023 8:52 AM, Simon Clubley wrote:
>>> On 2023-10-12, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>>> But I think there is a problem with C I/O on VMS x86-64.
>>>>
>>>> I have my little read and count all lines nano-benchmark
>>>> in both C and Java.
>>>>
>>>> Reading 10 million lines of 10 bytes each.
>>>>
>>>> C Java
>>>>
>>>> VMS 8
>>>> old itanium 2.9s 1.5s
>>>> HDD
>>>>
>>>> VMS 9
>>>> VMWare Player 20.0s 0.5s
>>>> Windows 10
>>>> PC
>>>> SSD
>>>>
>>>> Something is wrong here.
>>>
>>> Interesting.
>>>
>>> Because it's a BufferedReader, Java may be reading in larger chunks
>>> than the C RTL.
>>
>> It probably does.
>>
>> I did not use BufferedReader to get that buffer, but to get the readLine
>> method.
>>
>> Anyway that is a possible explanation of why Java is faster than C.
>
> One way to confirm this would be to record the current number of direct
> I/Os before and after each instance of your tests. The DIRIO number for
> Java _should_ be rather lower if this theory is correct. This is what I
> am thinking of:
>
> $ write sys$output f$getjpi("", "DIRIO")

The Java code would also get all the IO to load classes.

But still interesting.

>>> Are you in a position to try direct QIO reads in C on both architectures
>>> so you cut out both the language RTL and RMS ?
>>
>> Reading blocks with SYS$QIO(W) and parsing lines manually is
>> not so much fun. But it is certainly doable.
>>
>
> How difficult would it be to directly call RMS in record mode instead
> from your test ? That would eliminate most of the CRTL as a source of
> overhead.

SYS$GET is not that hard to use.

>> What is the point? The Java numbers prove that numbers can be
>> better using whatever IO API that Java use. And I am pretty
>> sure that Java does go through some C call and RMS (in C you can
>> add extra RMS arguments to fopen/open - you can do the same
>> for Java via the logical JAVA$FILENAME_MATCH_LIST).
>>
>
> Because the fact Java is twice as fast as C on Itanium caught my
> attention and I absolutely was _not_ expecting that.

Java is pretty fast!

:-) :-) :-)

> One suggestion I offer (with no evidence to back it up :-)) is that
> C might be doing RMS record-level reads and Java might be doing RMS
> block-level reads and doing its own unpacking of lines.

That is almost a given.

Java need to unpack the lines no matter how it does IO because
it need to convert from on disk encoding (default on VMS is
ISO-8859-1) to the UTF-16 encoding used in memory.

Arne