Now, before everyone piles on, I understand the procedure provides an
indicative/comparative/finger-in-the-air measurement of the relative
performance of a VMS CPU relative to "the original VAX processor".

It states as much in the prologue:

|$! Provides an estimate of system CPU performance on OpenVMS systems.
|$! Use at your own risk.

To clarify the platform of a particular run I added:

|$ write sys$output f$fao("!AS with !UL CPU and !ULMB running VMS !AS",-
|f$edit(f$getsyi("hw_name"),"compress,trim"),-
|f$getsyi("availcpu_cnt"),-
|(f$getsyi("memsize")*(f$getsyi("page_size")/512)/2048),-
|f$edit(f$getsyi("version"),"collapse"))

which provides the likes of:

|HP rx2660 (1.40GHz/6.0MB) with 4 CPU and 14335MB running VMS V8.4-2L1
|Digital Personal WorkStation with 1 CPU and 1536MB running VMS V8.4-2L1
|innotek GmbH VirtualBox with 2 CPU and 7574MB running VMS V9.2

It seems to be implemented as a tight DCL loop that executes almost
entirely in inner modes (I'm sure Brian can explain why).

$ start_cputime = f$getjpi(0,"CPUTIM")
$ loop_index = 0
$ 10$:
$ loop_index = loop_index + 1
$ if loop_index .ne. init_loop_maximum then goto 10$
$ end_cputime = f$getjpi(0,"CPUTIM")

|Combined for 2 CPUs 0 50 100 150 200
| Interrupt State | |
| MP Synchronization | |
| Kernel Mode 21 |▒▒▒▒ |
| Executive Mode 21 |▒▒▒▒ |
| Supervisor Mode 58 |▒▒▒▒▒▒▒▒▒▒▒ |
| User Mode | |
| Compatibility Mode | |
| Idle Time 99 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ |

100% (or all-but) of this execution appears to be in inner modes.
Although X86 (above) seems to to have much more Kernel than other
architectures, e.g. IA64 below). There is no USER mode displayed in either.

|Combined for 4 CPUs 0 100 200 300 400
| Interrupt State 1 | |
| MP Synchronization | |
| Kernel Mode 5 | |
| Executive Mode 18 |▒ |
| Supervisor Mode 78 |▒▒▒▒▒▒▒ |
| User Mode | |
| Not Available | |
| Idle Time 299 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒ |

There appear to be (at least) two versions of these procedures. The
later contains:

|$! Modified: MAY-2010: Code updated by Volker Halle to address the
|$! following issues:

and tweaks a few of the calculations.

There also appear to be earlier tweaks allowing for Alpha processors

|$ cpu_multiplier = 10 ! VAX = 10 - Alpha/AXP = 40
|$ cpu_round_add = 1 ! VAX = 1 - Alpha/AXP = 9

but none for Itanium.

Are the Alpha tweaks sufficient to allow relevance for all 64bit CPUs?

Are further tweaks required to make measurements on Itania relevant?

And of course the same question for the successor to all three
architectures?

--
Anyone, who using social-media, forms an opinion regarding anything
other than the relative cuteness of this or that puppy-dog, needs
seriously to examine their critical thinking.

Mark,

I've been using this procedure in the past (and fixed some problems with faster Alpha CPUs), to obtain CPU speed estimates during migration projects to the Stromasys CHARON emulators. In the beginning, those emulators did not perform - CPU-wise - well enough for emulating the faster Alphas (1 GHz or above). Never was a problem for VAX.

To get an estimate of the CPU speed of the customer's system, which should be migrated, getting a plain DCL procedure being executed was the easiest way to get an idea of the CPU performance. Yes, I know, system performance is more than CPU performance, but CPU speed of the early emulators was an expected bottleneck.

Itanium was never a target for running this procedure, as there was/is no Itanium emulator.

The procedure tries to execute a close DCL loop to prevent IOs as much as possible, they would disturbe the CPU speed estimate. This is why you just see Supervisor mode (or higher) and no user mode, as that would require an executable image.

There are certainly better and more exact CPU performance measurement tools, but a DCL procedure was the easiest thing to transfer to the customer's system and be run to provide a CPU speed estimate.

Volker.

On 2022-12-16, Mark Daniel <mark.daniel@wasd.vsm.com.au> wrote:
>
> It seems to be implemented as a tight DCL loop that executes almost
> entirely in inner modes (I'm sure Brian can explain why).
>

Unless CPU instructions execute at a different rate in inner modes,
that by itself should make no difference. However, there is one area
in which that could maybe matter for x86-64. See below.

> $ start_cputime = f$getjpi(0,"CPUTIM")
> $ loop_index = 0
> $ 10$:
> $ loop_index = loop_index + 1
> $ if loop_index .ne. init_loop_maximum then goto 10$
> $ end_cputime = f$getjpi(0,"CPUTIM")
>
>|Combined for 2 CPUs 0 50 100 150 200
>| Interrupt State | |
>| MP Synchronization | |
>| Kernel Mode 21 |???? |
>| Executive Mode 21 |???? |
>| Supervisor Mode 58 |??????????? |
>| User Mode | |
>| Compatibility Mode | |
>| Idle Time 99 |??????????????????? |
>
> 100% (or all-but) of this execution appears to be in inner modes.
> Although X86 (above) seems to to have much more Kernel than other
> architectures, e.g. IA64 below). There is no USER mode displayed in either.
>

And that's the difference (and what you are seeing is what I would expect).

Don't forget that Executive and Supervisor modes on x86-64 VMS are purely
an illusion and don't forget that effort is expended in Kernel mode to
switch to and from the emulation of those modes.

>
> Are the Alpha tweaks sufficient to allow relevance for all 64bit CPUs?
>
> Are further tweaks required to make measurements on Itania relevant?
>
> And of course the same question for the successor to all three
> architectures?
>

I think you are now seeing the limits of trying to do this in DCL.

Given all the Executive and Supervisor mode overheads in x86-64 VMS,
I think the only sensible solution is to implement this as a user-mode
compiled program so that you really are testing the relative CPU performance.

You are certainly going to see issues when trying to compare Intel with
AMD processors for example (due to the lack of PCID on AMD processors).

I was going to compare VUPS.COM to the BogoMIPS measurement used on Linux,
but there are issues addressed during the calculation of BogoMIPS that
cannot be addressed when trying to do this at DCL level. The following is
a pretty good summary of how BogoMIPS works:

https://en.wikipedia.org/wiki/BogoMips

BTW, there is one very interesting thing mentioned in the above article
that I had not considered until now when it comes to VMS on x86-64:

Does x86-64 VMS implement dynamic frequency scaling or does it run the
CPUs flat out at 100% of maximum speed all the time ?

As a final note, VUPS.COM only tests relative CPU performance. Should there
now be additional testing programs to test I/O subsystem performance ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 12/16/22 11:57, Mark Daniel wrote:
> Now, before everyone piles on, I understand the procedure provides an
> indicative/comparative/finger-in-the-air measurement of the relative
> performance of a VMS CPU relative to "the original VAX processor".

The spec.org site used to be the best place for that sort of thing
and where iirc, the original work to define 1 vup was done. Still
running and might be worth looking at. Truth is though, gains have
been incrementally minimal for single core, with multicore
taking up the banner since.

Here, in the days when I used the Tex package, a good rough guide
was the number of source pages compiled per minute. Microvax II,
about 4 ppm, First Sun workstation, about 20 ppm...

Chris

>
> It states as much in the prologue:
>
> |$! Provides an estimate of system CPU performance on OpenVMS systems.
> |$! Use at your own risk.
>
> To clarify the platform of a particular run I added:
>
> |$ write sys$output f$fao("!AS with !UL CPU and !ULMB running VMS !AS",-
> |f$edit(f$getsyi("hw_name"),"compress,trim"),-
> |f$getsyi("availcpu_cnt"),-
> |(f$getsyi("memsize")*(f$getsyi("page_size")/512)/2048),-
> |f$edit(f$getsyi("version"),"collapse"))
>
> which provides the likes of:
>
> |HP rx2660 (1.40GHz/6.0MB) with 4 CPU and 14335MB running VMS V8.4-2L1
> |Digital Personal WorkStation with 1 CPU and 1536MB running VMS V8.4-2L1
> |innotek GmbH VirtualBox with 2 CPU and 7574MB running VMS V9.2
>
> It seems to be implemented as a tight DCL loop that executes almost
> entirely in inner modes (I'm sure Brian can explain why).
>
> $ start_cputime = f$getjpi(0,"CPUTIM")
> $ loop_index = 0
> $ 10$:
> $ loop_index = loop_index + 1
> $ if loop_index .ne. init_loop_maximum then goto 10$
> $ end_cputime = f$getjpi(0,"CPUTIM")
>
> |Combined for  2 CPUs         0         50        100       150      200
> | Interrupt State             |                                       |
> | MP Synchronization          |                                       |
> | Kernel Mode              21 |▒▒▒▒                                   |
> | Executive Mode           21 |▒▒▒▒                                   |
> | Supervisor Mode          58 |▒▒▒▒▒▒▒▒▒▒▒                            |
> | User Mode                   |                                       |
> | Compatibility Mode          |                                       |
> | Idle Time                99 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒                    |
>
> 100% (or all-but) of this execution appears to be in inner modes.
> Although X86 (above) seems to to have much more Kernel than other
> architectures, e.g. IA64 below).  There is no USER mode displayed in
> either.
>
> |Combined for  4 CPUs         0         100       200       300      400
> | Interrupt State           1 |                                       |
> | MP Synchronization          |                                       |
> | Kernel Mode               5 |                                       |
> | Executive Mode           18 |▒                                      |
> | Supervisor Mode          78 |▒▒▒▒▒▒▒                                |
> | User Mode                   |                                       |
> | Not Available               |                                       |
> | Idle Time               299 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒          |
>
> There appear to be (at least) two versions of these procedures.  The
> later contains:
>
> |$! Modified: MAY-2010: Code updated by Volker Halle to address the
> |$! following issues:
>
> and tweaks a few of the calculations.
>
> There also appear to be earlier tweaks allowing for Alpha processors
>
> |$ cpu_multiplier = 10 ! VAX = 10 - Alpha/AXP = 40
> |$ cpu_round_add = 1 ! VAX = 1 - Alpha/AXP = 9
>
> but none for Itanium.
>
> Are the Alpha tweaks sufficient to allow relevance for all 64bit CPUs?
>
> Are further tweaks required to make measurements on Itania relevant?
>
> And of course the same question for the successor to all three
> architectures?
>

On 12/16/2022 6:57 AM, Mark Daniel wrote:
> Now, before everyone piles on, I understand the procedure provides an
> indicative/comparative/finger-in-the-air measurement of the relative
> performance of a VMS CPU relative to "the original VAX processor".

Yes.

I think the order of preference would be:
1) measuring the actual application in question
2) measuring a board suite of programs (SPEC style)
3) measuring using a simple thing like this

But even the latter can do if it is only the magnitude that is
relevant.

> It seems to be implemented as a tight DCL loop that executes almost
> entirely in inner modes (I'm sure Brian can explain why).
>
> $ start_cputime = f$getjpi(0,"CPUTIM")
> $ loop_index = 0
> $ 10$:
> $ loop_index = loop_index + 1
> $ if loop_index .ne. init_loop_maximum then goto 10$
> $ end_cputime = f$getjpi(0,"CPUTIM")
>
> |Combined for  2 CPUs         0         50        100       150      200
> | Interrupt State             |                                       |
> | MP Synchronization          |                                       |
> | Kernel Mode              21 |▒▒▒▒                                   |
> | Executive Mode           21 |▒▒▒▒                                   |
> | Supervisor Mode          58 |▒▒▒▒▒▒▒▒▒▒▒                            |
> | User Mode                   |                                       |
> | Compatibility Mode          |                                       |
> | Idle Time                99 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒                    |
>
> 100% (or all-but) of this execution appears to be in inner modes.
> Although X86 (above) seems to to have much more Kernel than other
> architectures, e.g. IA64 below).  There is no USER mode displayed in
> either.
>
> |Combined for  4 CPUs         0         100       200       300      400
> | Interrupt State           1 |                                       |
> | MP Synchronization          |                                       |
> | Kernel Mode               5 |                                       |
> | Executive Mode           18 |▒                                      |
> | Supervisor Mode          78 |▒▒▒▒▒▒▒                                |
> | User Mode                   |                                       |
> | Not Available               |                                       |
> | Idle Time               299 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒          |

Given that DCL runs in supervisor mode not user mode then supervisor
mode percentage being high and user mode being zero does not surprise
me.

I don't know why kernel and executive mode is higher on x86-64 than on
Itanium. But one possible explanation could be that kernel and executive
mode time is partly fixed while supervisor mode time (time spent
interpreting the execution of the loop) is strictly related to CPU
speed - which means higher K+E percentage and lower S percentage
on faster CPU's.

But a bit outside my area of expertise.

> There also appear to be earlier tweaks allowing for Alpha processors
>
> |$ cpu_multiplier = 10 ! VAX = 10 - Alpha/AXP = 40
> |$ cpu_round_add = 1 ! VAX = 1 - Alpha/AXP = 9
>
> but none for Itanium.
>
> Are the Alpha tweaks sufficient to allow relevance for all 64bit CPUs?
>
> Are further tweaks required to make measurements on Itania relevant?
>
> And of course the same question for the successor to all three
> architectures?

One would need to look at how those constants are used.

But if we assume that they make the test do more iterations
to provide a better test on faster systems (that is quite
common in speed tests), then I would expect Itanium
constants to be higher than Alpha constants and x86-64
constants to be higher than Itanium constants.

If they relate to something 32 bit vs 64 bit then the
previous would be totally wrong, but DCL is 32 bit math
AFAIK.

Arne

On 12/16/2022 9:11 AM, chris wrote:
> On 12/16/22 11:57, Mark Daniel wrote:
>> Now, before everyone piles on, I understand the procedure provides an
>> indicative/comparative/finger-in-the-air measurement of the relative
>> performance of a VMS CPU relative to "the original VAX processor".
>
> The spec.org site used to be the best place for that sort of thing
> and where iirc, the original work to define 1 vup was done. Still
> running and might be worth looking at. Truth is though, gains have
> been incrementally minimal for single core, with multicore
> taking up the banner since.
>
> Here, in the days when I used the Tex package, a good rough guide
> was the number of source pages compiled per minute. Microvax II,
> about 4 ppm, First Sun workstation, about 20 ppm...

SPEC is diffferent from VUPS.

But VUPS, SPEC 89 and SPEC 92 are all VAX 780 based.

SPEC 95 is SparcStation 10 and SPEC 2000 is Sparc Ultra 10 based.

(all according to old notes I made like 20 years ago)

Arne

On 12/16/22 14:49, Arne Vajhøj wrote:
> On 12/16/2022 9:11 AM, chris wrote:
>> On 12/16/22 11:57, Mark Daniel wrote:
>>> Now, before everyone piles on, I understand the procedure provides an
>>> indicative/comparative/finger-in-the-air measurement of the relative
>>> performance of a VMS CPU relative to "the original VAX processor".
>>
>> The spec.org site used to be the best place for that sort of thing
>> and where iirc, the original work to define 1 vup was done. Still
>> running and might be worth looking at. Truth is though, gains have
>> been incrementally minimal for single core, with multicore
>> taking up the banner since.
>>
>> Here, in the days when I used the Tex package, a good rough guide
>> was the number of source pages compiled per minute. Microvax II,
>> about 4 ppm, First Sun workstation, about 20 ppm...
>
> SPEC is diffferent from VUPS.
>
> But VUPS, SPEC 89 and SPEC 92 are all VAX 780 based.
>
> SPEC 95 is SparcStation 10 and SPEC 2000 is Sparc Ultra 10 based.
>
> (all according to old notes I made like 20 years ago)
>
> Arne
>
>

I should try looking again. Pretty much bang up to date afaics...

Chris

On 12/16/2022 8:31 AM, Simon Clubley wrote:
> On 2022-12-16, Mark Daniel <mark.daniel@wasd.vsm.com.au> wrote:
>>
>> It seems to be implemented as a tight DCL loop that executes almost
>> entirely in inner modes (I'm sure Brian can explain why).
>>
>
> Unless CPU instructions execute at a different rate in inner modes,
> that by itself should make no difference. However, there is one area
> in which that could maybe matter for x86-64. See below.
>
>> $ start_cputime = f$getjpi(0,"CPUTIM")
>> $ loop_index = 0
>> $ 10$:
>> $ loop_index = loop_index + 1
>> $ if loop_index .ne. init_loop_maximum then goto 10$
>> $ end_cputime = f$getjpi(0,"CPUTIM")
>>
>> |Combined for 2 CPUs 0 50 100 150 200
>> | Interrupt State | |
>> | MP Synchronization | |
>> | Kernel Mode 21 |???? |
>> | Executive Mode 21 |???? |
>> | Supervisor Mode 58 |??????????? |
>> | User Mode | |
>> | Compatibility Mode | |
>> | Idle Time 99 |??????????????????? |
>>
>> 100% (or all-but) of this execution appears to be in inner modes.
>> Although X86 (above) seems to to have much more Kernel than other
>> architectures, e.g. IA64 below). There is no USER mode displayed in either.
>>
>
> And that's the difference (and what you are seeing is what I would expect).
>
> Don't forget that Executive and Supervisor modes on x86-64 VMS are purely
> an illusion and don't forget that effort is expended in Kernel mode to
> switch to and from the emulation of those modes.
>
>>
>> Are the Alpha tweaks sufficient to allow relevance for all 64bit CPUs?
>>
>> Are further tweaks required to make measurements on Itania relevant?
>>
>> And of course the same question for the successor to all three
>> architectures?
>>
>
> I think you are now seeing the limits of trying to do this in DCL.
>
> Given all the Executive and Supervisor mode overheads in x86-64 VMS,
> I think the only sensible solution is to implement this as a user-mode
> compiled program so that you really are testing the relative CPU performance.
>
> You are certainly going to see issues when trying to compare Intel with
> AMD processors for example (due to the lack of PCID on AMD processors).
>
> I was going to compare VUPS.COM to the BogoMIPS measurement used on Linux,
> but there are issues addressed during the calculation of BogoMIPS that
> cannot be addressed when trying to do this at DCL level. The following is
> a pretty good summary of how BogoMIPS works:
>
> https://en.wikipedia.org/wiki/BogoMips
>
> BTW, there is one very interesting thing mentioned in the above article
> that I had not considered until now when it comes to VMS on x86-64:
>
> Does x86-64 VMS implement dynamic frequency scaling or does it run the
> CPUs flat out at 100% of maximum speed all the time ?
>
> As a final note, VUPS.COM only tests relative CPU performance. Should there
> now be additional testing programs to test I/O subsystem performance ?
>
> Simon.
>

Of course there can be more specific, and accurate, means of such measurements.
But as Volker mentioned, what was desired was a rough wild ass gestimate of what
one might expect. For example, if the same code run on system "B" was twice as
fast as on system "A", then one might reasonable expect that system "B" would do
more work than system "A". Many times that is adequate.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 2022-12-16 12:27:30 +0000, Mark Daniel said:

> Now, before everyone piles on, I understand the procedure provides an
> indicative/comparative/finger-in-the-air measurement of the relative
> performance of a VMS CPU relative to "the original VAX processor".

Around the introduction of Alpha, DEC punted on VUPS.

VUPS wasn't particularly representative across VAX. With Alpha, less so.

There was a while where SPEC was occasionally used, and IIRC
occasionally even some TPC benchmarks for database-related, but tended
to misrepresent system performance.

This was all part of the genesis of the DEC customer benchmarking and
test lab at DEC ZKO Nashua; test the actual customer apps on the actual
servers.

For those that do want pictures, some performance charts:
http://www.cs.columbia.edu/~martha/courses/3827/au14/advanced-topics.pdf
http://www.jcmit.net/cpu-performance.htm
http://www.roylongbottom.org.uk/whetstone.htm

And then there's this:
Raspberry Pi 2 at 1 GHz offers 4,744 Dhrystone MIPS, as compared with a
VAX-11/780 offering a single, solitary MIP.

--
Pure Personal Opinion | HoffmanLabs LLC

On 16/12/2022 10:27 pm, Mark Daniel wrote:
> Now, before everyone piles on, I understand the procedure provides an
> indicative/comparative/finger-in-the-air measurement of the relative
> performance of a VMS CPU relative to "the original VAX processor".
8< snip 8<

Thanks to all those who contributed to this thread.

The followup that caught my eye was from Simon Clubley who provided an
entirely convincing explanation for the elevated X86 Kernel mode.

Also his pointer to BogoMips. Most interesting. I read the FAQ and
accessed the github code. Quite straighforward. Might be a good
replacement as a general performance metric.

https://github.com/vitalyvch/Bogo/blob/BogoMIPS_v1.3/bogomips.c

So taking the example general use implementation, modified it for
building on VMS (and added some bits I find useful).

The only bit I am less sure of is the suppression of optimisation. The
above example uses a GCC-ism

| for (i = loops; !!(i > 0); --i)
| asm volatile ("" ::: "memory");

which I replaced with

|#pragma optimize save
|#pragma optimize level=0
|/* portable version */
|static void delay(long loops)
|{
| long i;
| for (i = loops; !!(i > 0); --i);
|}
|#pragma optimize restore

Anyway, the results were interesting (to say the least):

|$ mcr []bogomips
|HP rx2660 (1.40GHz/6.0MB) 4 CPUs 14335MB V8.4-2L1
|Calibrating delay loop.. ok - 692.73 BogoMips
|$ @vups
|HP rx2660 (1.40GHz/6.0MB) with 4 CPU and 14335MB running VMS V8.4-2L1
|INFO: Preventing endless loop (10$) on fast CPUs
|Approximate System VUPs Rating : 486.3 ( min: 483.8 max: 488.8 )

**
|$ mcr []bogomips
|Digital Personal WorkStation 1 CPU 1536MB V8.4-2L1
|Calibrating delay loop.. ok - 497.10 BogoMips
|$ @vups
|Digital Personal WorkStation with 1 CPU and 1536MB running VMS V8.4-2L1
|Approximate System VUPs Rating : 150.9 ( min: 149.4 max: 151.8 )

|$ mcr []bogomips
|AlphaServer DS20 500 MHz 2 CPUs 1536MB V8.4-2L2
|Calibrating delay loop.. ok - 488.06 BogoMips
|$ @vups
|AlphaServer DS20 500 MHz with 2 CPU and 1536MB running VMS V8.4-2L2
|Approximate System VUPs Rating : 250.7 ( min: 249.8 max: 251.2 )

|$ mcr []bogomips
|innotek GmbH VirtualBox 2 CPUs 7574MB V9.2
|Calibrating delay loop.. ok - 185.12 BogoMips
|$ @dvups
|innotek GmbH VirtualBox with 2 CPU and 7574MB running VMS V9.2
|Approximate System VUPs Rating : 275.9 ( min: 269.2 max: 286.6 )

For anyone interested my VMS bogomips.c can be found at

https://wasd.vsm.com.au/wasd_tmp/bogomips.c

** Interestingly, the PWS BogoMips matches the same model reported in
the mini-Howto list of results:

> 21164A/500 PWS 494.88 Kenny Gryp <gryp@_dakin.be>
> 21164A/500 PWS 497.02 Robert Harley <robert.harley@_inria.fr>

https://www.clifton.nl/index.html?bogomips.html

--
Anyone, who using social-media, forms an opinion regarding anything
other than the relative cuteness of this or that puppy-dog, needs
seriously to examine their critical thinking.

On 17/12/2022 10:13 am, Mark Daniel wrote:
> On 16/12/2022 10:27 pm, Mark Daniel wrote:
>> Now, before everyone piles on, I understand the procedure provides an
>> indicative/comparative/finger-in-the-air measurement of the relative
>> performance of a VMS CPU relative to "the original VAX processor".
> 8< snip 8<
>
> Thanks to all those who contributed to this thread.
>
> The followup that caught my eye was from Simon Clubley who provided an
> entirely convincing explanation for the elevated X86 Kernel mode.
>
> Also his pointer to BogoMips.  Most interesting.  I read the FAQ and
> accessed the github code.  Quite straighforward.  Might be a good
> replacement as a general performance metric.
>
> https://github.com/vitalyvch/Bogo/blob/BogoMIPS_v1.3/bogomips.c
8< snip 8<

For general VMS comparative usage something more VMS-measuring is
needed. I looked about the 'net and nothing sprang out. I wonder what
VSI are using for metrics on X86 development? Anything lurking in the
DECUS/Freeware repositories I missed?

Anyway, in the absence of anything else, I was thinking about what may
consume "non-productive" VMS cycles (i.e. non-USER mode crunching :-)
and all I could think of were the transitions between USER, EXEC and
KERNEL modes. As required by RMS, $QIO, drivers, etc., etc. No SUPER
modes measured here.

With this in mind I knocked-up a small program to repeatedly call a
function using $CMEXEC which calls a function using $CMKRNL and that is
that. It measures how much effort is required compared to the simple
USER mode loop and reports it as b[ogo]VUPs.

https://wasd.vsm.com.au/wasd_tmp/bogovups.c

The real disappointment is my X86 VM. The rest of the results seem in
line with expectations.

PS. Looking for ideas, suggestions, criticism(s), etc. here...

--
Anyone, who using social-media, forms an opinion regarding anything
other than the relative cuteness of this or that puppy-dog, needs
seriously to examine their critical thinking.

On 2022-12-19, Mark Daniel <mark.daniel@wasd.vsm.com.au> wrote:
>
> With this in mind I knocked-up a small program to repeatedly call a
> function using $CMEXEC which calls a function using $CMKRNL and that is
> that. It measures how much effort is required compared to the simple
> USER mode loop and reports it as b[ogo]VUPs.
>
> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>
> The real disappointment is my X86 VM. The rest of the results seem in
> line with expectations.
>

What numbers did you see ?

> PS. Looking for ideas, suggestions, criticism(s), etc. here...
>

The tests still "feel" rather artificial.

My suggested alternative would be to write actual files away to disk
using RMS sequential files and also indexed files. Maybe read them
back as well.

Repeat the sequential files test using direct QIO access and see what
performance difference that gives when you bypass the transition to
executive mode.

(Yes, I know, RMS will add its own fixed overheads, but you will still
be able to see a percentage difference across the various machines you
are testing on and whether x86-64 VMS imposes a much higher performance
overhead.)

One obvious problem is that you will have to address issues round
caching in your tests.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Den 2022-12-19 kl. 14:23, skrev Simon Clubley:
> On 2022-12-19, Mark Daniel <mark.daniel@wasd.vsm.com.au> wrote:
>>
>> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>>
>
> What numbers did you see ?

They are in the C file on the link above.

On Monday, December 19, 2022 at 8:23:23 AM UTC-5, Simon Clubley wrote:
> On 2022-12-19, Mark Daniel <mark....@wasd.vsm.com.au> wrote:
> >
> > With this in mind I knocked-up a small program to repeatedly call a
> > function using $CMEXEC which calls a function using $CMKRNL and that is
> > that. It measures how much effort is required compared to the simple
> > USER mode loop and reports it as b[ogo]VUPs.
> >
> > https://wasd.vsm.com.au/wasd_tmp/bogovups.c
> >
> > The real disappointment is my X86 VM. The rest of the results seem in
> > line with expectations.
> >
> What numbers did you see ?
> > PS. Looking for ideas, suggestions, criticism(s), etc. here...
> >
> The tests still "feel" rather artificial.
>
> My suggested alternative would be to write actual files away to disk
> using RMS sequential files and also indexed files. Maybe read them
> back as well.
>
> Repeat the sequential files test using direct QIO access and see what
> performance difference that gives when you bypass the transition to
> executive mode.
>
> (Yes, I know, RMS will add its own fixed overheads, but you will still
> be able to see a percentage difference across the various machines you
> are testing on and whether x86-64 VMS imposes a much higher performance
> overhead.)
>
> One obvious problem is that you will have to address issues round
> caching in your tests.
> Simon.
>
> --
> Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
> Walking destinations on a map are further away than they appear.
Why would you want I/O involved in a measurement of relative CPU power? That makes no sense. The VUPs rating has always been a relative CPU performance test. You can argue whether User mode vs other modes makes sense. I suppose that using Mark's updated program may make sense given that the newer versions of OpenVMS use software for some functions (modes really). I don't believe that this is accurate as it will compare hardware vs software between a few models.

I would be interested in (and will likely test) this new option in the emulated environment. It may be a more consistent comparison in this environment.

Dan

On 2022-12-19, Jan-Erik Söderholm <jan-erik.soderholm@telia.com> wrote:
> Den 2022-12-19 kl. 14:23, skrev Simon Clubley:
>> On 2022-12-19, Mark Daniel <mark.daniel@wasd.vsm.com.au> wrote:
>>>
>>> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>>>
>>
>> What numbers did you see ?
>
>
> They are in the C file on the link above.
>

Dammit. I somehow managed to miss that in the comments while looking at
the rest of the code. :-(

Sorry.

Thanks, Jan-Erik.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2022-12-19, abrsvc <dansabrservices@yahoo.com> wrote:
> On Monday, December 19, 2022 at 8:23:23 AM UTC-5, Simon Clubley wrote:
>> On 2022-12-19, Mark Daniel <mark....@wasd.vsm.com.au> wrote:
>> >
>> > With this in mind I knocked-up a small program to repeatedly call a
>> > function using $CMEXEC which calls a function using $CMKRNL and that is
>> > that. It measures how much effort is required compared to the simple
>> > USER mode loop and reports it as b[ogo]VUPs.
>> >
>> > https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>> >
>> > The real disappointment is my X86 VM. The rest of the results seem in
>> > line with expectations.
>> >
>> What numbers did you see ?
>> > PS. Looking for ideas, suggestions, criticism(s), etc. here...
>> >
>> The tests still "feel" rather artificial.
>>
>> My suggested alternative would be to write actual files away to disk
>> using RMS sequential files and also indexed files. Maybe read them
>> back as well.
>>
>> Repeat the sequential files test using direct QIO access and see what
>> performance difference that gives when you bypass the transition to
>> executive mode.
>>
>> (Yes, I know, RMS will add its own fixed overheads, but you will still
>> be able to see a percentage difference across the various machines you
>> are testing on and whether x86-64 VMS imposes a much higher performance
>> overhead.)
>>
>> One obvious problem is that you will have to address issues round
>> caching in your tests.
>>
> Why would you want I/O involved in a measurement of relative CPU power? That makes no sense. The VUPs rating has always been a relative CPU performance test. You can argue whether User mode vs other modes makes sense. I suppose that using Mark's updated program may make sense given that the newer versions of OpenVMS use software for some functions (modes really). I don't believe that this is accurate as it will compare hardware vs software between a few models.
>

Because Mark was reporting bad performance on x86-64 VMS compared to his
other machines, but are the results artifically bad ?

We already know there is going to be an overhead because of the need to
emulate executive and supervisor modes on x86-64 VMS, but Mark's results
don't cover the elapsed time taken by doing "real" work while in executive
mode.

Now that I have seen the numbers (thanks Jan-Erik :-)), they would appear
to be not only bad in general, but bad even when compared to Alpha (provided
Mark's VirtualBox instance isn't somehow constraining performance and
assuming he is running it on modern x86-64 hardware).

As such, doing real work in executive mode, and then comparing that
performance to direct QIO calls, might give better insights into more
real-world performance results.

Also, to help eliminate differences in disk hardware performance, might
it be a good idea to run those tests on a RAM disk instead of a physical
disk ?

Regarding your comment about testing in an emulated environment, don't
forget that VirtualBox is NOT an emulator, but a virtualisation mechanism.

That means operating systems running under it should be running at close
to native hardware performance speeds and it means you should be testing
the x86-64 performance against physical Alpha machines, not emulated ones.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

>
> Regarding your comment about testing in an emulated environment, don't
> forget that VirtualBox is NOT an emulator, but a virtualisation mechanism..
>
> That means operating systems running under it should be running at close
> to native hardware performance speeds and it means you should be testing
> the x86-64 performance against physical Alpha machines, not emulated ones..
> Simon.
>
I never said that the tests run by Mark were done on an emulated environment. Don't infer things that are not there.

What I suggested was that it would be interesting to see what the use of his "new" option would show on an emulated system. No mention of anything other than that. I still think it would be interesting to see if the correlation of "real" to emulated holds using Mark's test when compared to using the original VUPS procedure.

Dan

On 2022-12-19, abrsvc <dansabrservices@yahoo.com> wrote:
>
>>
>> Regarding your comment about testing in an emulated environment, don't
>> forget that VirtualBox is NOT an emulator, but a virtualisation mechanism.
>>
>> That means operating systems running under it should be running at close
>> to native hardware performance speeds and it means you should be testing
>> the x86-64 performance against physical Alpha machines, not emulated ones.
>> Simon.
>>
> I never said that the tests run by Mark were done on an emulated environment. Don't infer things that are not there.
>
> What I suggested was that it would be interesting to see what the use of his "new" option would show on an emulated system. No mention of anything other than that. I still think it would be interesting to see if the correlation of "real" to emulated holds using Mark's test when compared to using the original VUPS procedure.
>

Dan, you are _way_ too quick to read bad things into my comments. :-(

Read the quoted comment above again, and you will see I am talking about
the emulator tests _you_ are planning to do and I was pointing out that
Mark's tests should have been done at somewhere close to his native
hardware speed.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 12/18/22 11:01 PM, Mark Daniel wrote:
> On 17/12/2022 10:13 am, Mark Daniel wrote:
>> On 16/12/2022 10:27 pm, Mark Daniel wrote:
>>> Now, before everyone piles on, I understand the procedure provides an
>>> indicative/comparative/finger-in-the-air measurement of the relative
>>> performance of a VMS CPU relative to "the original VAX processor".
>> 8< snip 8<
>>
>> Thanks to all those who contributed to this thread.
>>
>> The followup that caught my eye was from Simon Clubley who provided an
>> entirely convincing explanation for the elevated X86 Kernel mode.
>>
>> Also his pointer to BogoMips.  Most interesting.  I read the FAQ and
>> accessed the github code.  Quite straighforward.  Might be a good
>> replacement as a general performance metric.
>>
>> https://github.com/vitalyvch/Bogo/blob/BogoMIPS_v1.3/bogomips.c
> 8< snip 8<
>
> For general VMS comparative usage something more VMS-measuring is
> needed.  I looked about the 'net and nothing sprang out.  I wonder what
> VSI are using for metrics on X86 development?  Anything lurking in the
> DECUS/Freeware repositories I missed?
>
> Anyway, in the absence of anything else, I was thinking about what may
> consume "non-productive" VMS cycles (i.e. non-USER mode crunching :-)
> and all I could think of were the transitions between USER, EXEC and
> KERNEL modes.  As required by RMS, $QIO, drivers, etc., etc.  No SUPER
> modes measured here.
>
> With this in mind I knocked-up a small program to repeatedly call a
> function using $CMEXEC which calls a function using $CMKRNL and that is
> that.  It measures how much effort is required compared to the simple
> USER mode loop and reports it as b[ogo]VUPs.
>
> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>
> The real disappointment is my X86 VM.  The rest of the results seem in
> line with expectations.
>
> PS.  Looking for ideas, suggestions, criticism(s), etc. here...

It's pretty hard to know what this means, if anything. I'll assume your
VirtualBox host is a speedy recent Intel machine with all the cpu
capabilities VMS prefers? (not an M1 Mac emulating x86 I hope!).

It may be that the cross compiler doesn't have optimizations turned on.
It may be that there is a ton of debugging code in the OS that will
eventually get turned off. It may be mode switching is genuinely slow
on x86.

If the latter is the main culprit, I suspect it will play out the same
way alignment faults did on Itanium. They were horrible for the things
affected, but not everything was affected, and not everything that was
affected was affected equally. And multiple different fixes to
compilers, products, and the OS itself eventually mitigated them to
where they became a non-problem for most people most of the time, but it
took a couple years.

The last time I remember anyone from VSI saying anything about
performance was that they hadn't even started looking at it yet. That
may have been over a year ago, but I suspect they still haven't -- just
too much else to do. The word "performance" does not appear on the roadmap.

Somehow I got the impression that enabling compiler optimizations would
be deferred until after native compilers were available. (I may have
that wrong, possibly from a vague memory of the order in which things
happened for previous ports). Not that one couldn't enable optimizations
in a cross compiler, but just that they are working in priority order,
and having native compilers (and more compilers, such as BASIC) is a
much higher priority than performance at this point. You can't even
port from Alpha to x86 right now without buying an Itanium, and I doubt
very many Alpha customers would consider doing that.

On 20/12/2022 5:40 am, Simon Clubley wrote:
> On 2022-12-19, abrsvc <dansabrservices@yahoo.com> wrote:
>> On Monday, December 19, 2022 at 8:23:23 AM UTC-5, Simon Clubley wrote:
>>> On 2022-12-19, Mark Daniel <mark....@wasd.vsm.com.au> wrote:
>>>>
>>>> With this in mind I knocked-up a small program to repeatedly call a
>>>> function using $CMEXEC which calls a function using $CMKRNL and that is
>>>> that. It measures how much effort is required compared to the simple
>>>> USER mode loop and reports it as b[ogo]VUPs.
>>>>
>>>> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>>>>
>>>> The real disappointment is my X86 VM. The rest of the results seem in
>>>> line with expectations.
>>>>
>>> What numbers did you see ?
>>>> PS. Looking for ideas, suggestions, criticism(s), etc. here...
>>>>
>>> The tests still "feel" rather artificial.

Certainly they are artificial. I really didn't want to write a test
suite, just a simple measuring stick for a quick comparison of
platforms. For performance testing the only real metric is "the
application" or a representative code base.

>>> My suggested alternative would be to write actual files away to disk
>>> using RMS sequential files and also indexed files. Maybe read them
>>> back as well.
>>>
>>> Repeat the sequential files test using direct QIO access and see what
>>> performance difference that gives when you bypass the transition to
>>> executive mode.
>>>
>>> (Yes, I know, RMS will add its own fixed overheads, but you will still
>>> be able to see a percentage difference across the various machines you
>>> are testing on and whether x86-64 VMS imposes a much higher performance
>>> overhead.)
>>>
>>> One obvious problem is that you will have to address issues round
>>> caching in your tests.
>>>
>> Why would you want I/O involved in a measurement of relative CPU power? That makes no sense. The VUPs rating has always been a relative CPU performance test. You can argue whether User mode vs other modes makes sense. I suppose that using Mark's updated program may make sense given that the newer versions of OpenVMS use software for some functions (modes really). I don't believe that this is accurate as it will compare hardware vs software between a few models.

Agreed. Not for a "simple measuring stick" (to quote myself).

Disagree. It compares *VMS* performance. Not necessarily just CPU
power (which is the thrust of BogoMips) but a combination of
hardware/bus/software.

> Because Mark was reporting bad performance on x86-64 VMS compared to his
> other machines, but are the results artifically bad ?

X86 actually performs quite responsively, so this is a fair question.

My EXEC mode function calling a KERNEL mode function is pretty intense.
Perhaps it would be better calling KERNEL mode every tenth time. Or
some combination such as that. I have no information on the respective
call rates apart from @VUPS.COM which on my Alpha shows

| Executive Mode 18 |▒▒▒▒▒▒▒
| Supervisor Mode 81 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

and on the X86 VM

Combined for 2 CPUs
| Kernel Mode 22 |▒▒▒▒
| Executive Mode 16 |▒▒▒
| Supervisor Mode 62 |▒▒▒▒▒▒▒▒▒▒▒▒
| Idle Time 100 |▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

Of course there is no SUPER mode in my code.

The numbers certainly support the idea that X86 inner-modes are
"emulated" in some way, mediated by KERNEL mode processing.

> We already know there is going to be an overhead because of the need to
> emulate executive and supervisor modes on x86-64 VMS, but Mark's results
> don't cover the elapsed time taken by doing "real" work while in executive
> mode.

Quite deliberately. It was intended to measure only the overhead of
inner-mode calls. Is this fair and equitable? Who knows?

> Now that I have seen the numbers (thanks Jan-Erik :-)), they would appear
> to be not only bad in general, but bad even when compared to Alpha (provided
> Mark's VirtualBox instance isn't somehow constraining performance and
> assuming he is running it on modern x86-64 hardware).

A pre-loved AU$300.00 Dell SFF bought via eBay -- Optiplex 9020,
Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz RAM 16.0 GB, Windows 10 Pro
Version 22H2 Installed 17/03/2021, VirtualBox 6.1.40. A nice little
system for the money.

> As such, doing real work in executive mode, and then comparing that
> performance to direct QIO calls, might give better insights into more
> real-world performance results.
>
> Also, to help eliminate differences in disk hardware performance, might
> it be a good idea to run those tests on a RAM disk instead of a physical
> disk ?

Not interested in I/O.

Doesn't meet the project criterion of quick (and dirty) platform
comparison (a la VUPS.COM). Perhaps this approach is fraught with all
sorts of limitations and attention should be redirected to VUPS.COM.

> Regarding your comment about testing in an emulated environment, don't
> forget that VirtualBox is NOT an emulator, but a virtualisation mechanism.

Certainly.

> That means operating systems running under it should be running at close
> to native hardware performance speeds and it means you should be testing
> the x86-64 performance against physical Alpha machines, not emulated ones.

As a performance comparison tool, why not?

> Simon.

Thanks for your (plural) interest.

--
Anyone, who using social-media, forms an opinion regarding anything
other than the relative cuteness of this or that puppy-dog, needs
seriously to examine their critical thinking.

On 12/19/2022 12:01 AM, Mark Daniel wrote:
> On 17/12/2022 10:13 am, Mark Daniel wrote:
>> Also his pointer to BogoMips.  Most interesting.  I read the FAQ and
>> accessed the github code.  Quite straighforward.  Might be a good
>> replacement as a general performance metric.
>>
>> https://github.com/vitalyvch/Bogo/blob/BogoMIPS_v1.3/bogomips.c
> 8< snip 8<
>
> For general VMS comparative usage something more VMS-measuring is
> needed.  I looked about the 'net and nothing sprang out.  I wonder what
> VSI are using for metrics on X86 development?  Anything lurking in the
> DECUS/Freeware repositories I missed?
>
> Anyway, in the absence of anything else, I was thinking about what may
> consume "non-productive" VMS cycles (i.e. non-USER mode crunching :-)
> and all I could think of were the transitions between USER, EXEC and
> KERNEL modes.  As required by RMS, $QIO, drivers, etc., etc.  No SUPER
> modes measured here.
>
> With this in mind I knocked-up a small program to repeatedly call a
> function using $CMEXEC which calls a function using $CMKRNL and that is
> that.  It measures how much effort is required compared to the simple
> USER mode loop and reports it as b[ogo]VUPs.
>
> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>
> The real disappointment is my X86 VM.  The rest of the results seem in
> line with expectations.
>
> PS.  Looking for ideas, suggestions, criticism(s), etc. here...

First, then I assume neither your code nor VMS
itself are optimized - I believe John Reagan said that
the cross compilers do not optimize much.

But besides that I am not convinced that the time spent to
do mode switches is a particular relevant test. It should
never be a large enough part of total CPU usage to
matter much.

In general the CPU bottlenecks should be in
user mode. So back to BogoMips or DhryStone/WhetStone
or SPEC or whatever.

If something in VMS should be tested then I think a more
relevant test would be to test what the scheduler can handle.
When does overhead start hurting throughput - 500 threads?
1000 threads? 2000 threads? 4000 threads? 8000 threads?

Arne

Years (decades?) ago I grabbed a copy of VUPS.COM out of the startup code for a layered product; Diskeeper IIRC. It's not very large, but I'm not sure if it's proper to share what must have been copyrighted code.
Results didn't exactly match published specs, but they helped fill in the gaps for systems that were never on the old VUPS list.

-Bob

Mark,

on my Intel i5-9600K @3.7 GHz with 6 Cores, Windows 10 Pro 22H2 with VMware Player 16, running my VUPS.COM reports about 730 VUPS with VSI OpenVMS x86-64 V9.2 with 2 vCPUs.

$ MONI MODE/INT=1/AVERAGE reports about 10% Kernel, 21% Exec and 51% Supervisor mode while running VUPS.COM

Running a simple LOOP.COM

$ i=0
$loop:
$ i=i+1
$ GOTO loop

consumes 22% Kernel, 33% Exec and 45% Supervisor mode

Running the same LOOP.COM procedure on an OpenVMS V8.2 rx2600 1.3 GHz (1 CPU) consumes: 8% Kernel, 31% Exec and 61% Supervisor mode.
Running VUPS.COM on that Itanium reports about 1762 VUPS with 5% Kernel, 17% Exec and 77% Supervisor mode.

The 'CPU work' is done in Supervisor mode in a small loop in VUPS.COM, Exec and Kernel could be considered (necessary) 'overhead' and this 'overhead' is more significant on x86-64.

AFAIK VSI has not yet publicized any performance data and may still be concentrating on function vs. performance.

Volker.

On 2022-12-19 20:10, Simon Clubley wrote:
> On 2022-12-19, abrsvc <dansabrservices@yahoo.com> wrote:
>> On Monday, December 19, 2022 at 8:23:23 AM UTC-5, Simon Clubley wrote:
>>> On 2022-12-19, Mark Daniel <mark....@wasd.vsm.com.au> wrote:
>>>>
>>>> With this in mind I knocked-up a small program to repeatedly call a
>>>> function using $CMEXEC which calls a function using $CMKRNL and that is
>>>> that. It measures how much effort is required compared to the simple
>>>> USER mode loop and reports it as b[ogo]VUPs.
>>>>
>>>> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>>>>
>>>> The real disappointment is my X86 VM. The rest of the results seem in
>>>> line with expectations.
>>>>
>>> What numbers did you see ?
>>>> PS. Looking for ideas, suggestions, criticism(s), etc. here...
>>>>
>>> The tests still "feel" rather artificial.
>>>
>>> My suggested alternative would be to write actual files away to disk
>>> using RMS sequential files and also indexed files. Maybe read them
>>> back as well.
>>>
>>> Repeat the sequential files test using direct QIO access and see what
>>> performance difference that gives when you bypass the transition to
>>> executive mode.
>>>
>>> (Yes, I know, RMS will add its own fixed overheads, but you will still
>>> be able to see a percentage difference across the various machines you
>>> are testing on and whether x86-64 VMS imposes a much higher performance
>>> overhead.)
>>>
>>> One obvious problem is that you will have to address issues round
>>> caching in your tests.
>>>
>> Why would you want I/O involved in a measurement of relative CPU power? That makes no sense. The VUPs rating has always been a relative CPU performance test. You can argue whether User mode vs other modes makes sense. I suppose that using Mark's updated program may make sense given that the newer versions of OpenVMS use software for some functions (modes really). I don't believe that this is accurate as it will compare hardware vs software between a few models.
>>
>
> Because Mark was reporting bad performance on x86-64 VMS compared to his
> other machines, but are the results artifically bad ?
>
> We already know there is going to be an overhead because of the need to
> emulate executive and supervisor modes on x86-64 VMS, but Mark's results
> don't cover the elapsed time taken by doing "real" work while in executive
> mode.

The overhead would only be something that hits a little at mode
switching. While just executing code, the cost of not having executive
and supervisor in hardware should be zero.

And I agree that doing I/O usually means you won't see much of what the
CPU performance actually looks like.

Johnny

Mark Daniel schrieb am Montag, 19. Dezember 2022 um 06:01:45 UTC+1:
> On 17/12/2022 10:13 am, Mark Daniel wrote:
> > On 16/12/2022 10:27 pm, Mark Daniel wrote:
> >> Now, before everyone piles on, I understand the procedure provides an
> >> indicative/comparative/finger-in-the-air measurement of the relative
> >> performance of a VMS CPU relative to "the original VAX processor".
> > 8< snip 8<
> >
> > Thanks to all those who contributed to this thread.
> >
> > The followup that caught my eye was from Simon Clubley who provided an
> > entirely convincing explanation for the elevated X86 Kernel mode.
> >
> > Also his pointer to BogoMips. Most interesting. I read the FAQ and
> > accessed the github code. Quite straighforward. Might be a good
> > replacement as a general performance metric.
> >
> > https://github.com/vitalyvch/Bogo/blob/BogoMIPS_v1.3/bogomips.c
> 8< snip 8<
>
> For general VMS comparative usage something more VMS-measuring is
> needed. I looked about the 'net and nothing sprang out. I wonder what
> VSI are using for metrics on X86 development? Anything lurking in the
> DECUS/Freeware repositories I missed?
>
> Anyway, in the absence of anything else, I was thinking about what may
> consume "non-productive" VMS cycles (i.e. non-USER mode crunching :-)
> and all I could think of were the transitions between USER, EXEC and
> KERNEL modes. As required by RMS, $QIO, drivers, etc., etc. No SUPER
> modes measured here.
>
> With this in mind I knocked-up a small program to repeatedly call a
> function using $CMEXEC which calls a function using $CMKRNL and that is
> that. It measures how much effort is required compared to the simple
> USER mode loop and reports it as b[ogo]VUPs.
>
> https://wasd.vsm.com.au/wasd_tmp/bogovups.c
>
> The real disappointment is my X86 VM. The rest of the results seem in
> line with expectations.
>
> PS. Looking for ideas, suggestions, criticism(s), etc. here...
> --
I ran your program in two of our own environments with the following rsults:

|AlphaServer ES45 Model 1B 4 CPUs 16384MB V8.4-2L2
|Calculating IPS...
|321.000000 cpu-ticks, 3.279087 seconds, 609925873 / second
|Calculating bVUPs...
|1645.000000 cpu-ticks, 16.614445 seconds, 370.8 bVUPs
|9157.000000 cpu-ticks, 91.599084 seconds, 10.8 bVUPs

|HP DL380 2.6 GHz VMware, Inc. 4 CPUs 15868MB V9.2
|Calculating IPS...
|459.000000 cpu-ticks, 4.589954 seconds, 435734205 / second
|Calculating bVUPs...
|7470.000000 cpu-ticks, 74.799252 seconds, 58.3 bVUPs
Sorry to say, but your bVUPs computation is not very useful.
This is because you divide dins [instructions/sec] by dticks [0.01 sec].
Insead of being dimensionless (like a good VUP ought to be)
your result has the physical unit of [ 1/sec²].
My own interpretation of the figures given above is:
X86 integer performance reaches 71% of ES45 (321/459 Ticks)
X86 change mode performance reaches 22% of ES45 (1645/7470 Ticks)

This not bad given the fact, that the C crosscompiler currently attempts no
optimization at all (as I know from an extraordinarily well informed source).
Andreas