I'm curious what to expect when moving OpenVMS from an Alpha / ES server to a virtual machine with an emulator. I'm mostly looking at the single threaded / single core floating point performance. Are there emulators that are significantly better / worse for this task? Can I expect a comparable running time on the fastest (clock speed) Intel Xeon money can buy today?

On 10/11/2022 4:20 PM, Nick K wrote:
> I'm curious what to expect when moving OpenVMS from an Alpha / ES
> server to a virtual machine with an emulator. I'm mostly looking at
> the single threaded / single core floating point performance. Are
> there emulators that are significantly better / worse for this task?
> Can I expect a comparable running time on the fastest (clock speed)
> Intel Xeon money can buy today?

I don't have an answer.

But two important questions must be:
1) does the Alpha emulator do JIT?
2) are you using F/G or S/T floats?

JIT and S/T could potentially be pretty fast.

Arne

On Tuesday, October 11, 2022 at 4:26:40 PM UTC-4, Arne Vajhøj wrote:
> On 10/11/2022 4:20 PM, Nick K wrote:
> > I'm curious what to expect when moving OpenVMS from an Alpha / ES
> > server to a virtual machine with an emulator. I'm mostly looking at
> > the single threaded / single core floating point performance. Are
> > there emulators that are significantly better / worse for this task?
> > Can I expect a comparable running time on the fastest (clock speed)
> > Intel Xeon money can buy today?
> I don't have an answer.
>
> But two important questions must be:
> 1) does the Alpha emulator do JIT?
> 2) are you using F/G or S/T floats?
>
> JIT and S/T could potentially be pretty fast.
>
> Arne

I will state that little can be gained from looking at the standard benchmarks unless you know the relationship of them to your own application. I have not found single threaded performance numbers to relate to overall application performance either. That being said, I am aware of many installation using the Charon emulators that perform at or exceed "real" hardware levels. A lot depends upon the application. Some emulators perform JIT, others prescan the code stream and generate the instructions in parallel to their execution. Your mileage may vary.

For full disclosure, I work for the company that created and supports the Charon product stream. I can not comment on the performance of other emulators.

Dan

On 2022-10-11, abrsvc <dansabrservices@yahoo.com> wrote:
>
> I will state that little can be gained from looking at the standard benchmarks unless you know the relationship of them to your own application. I have not found single threaded performance numbers to relate to overall application performance either. That being said, I am aware of many installation using the Charon emulators that perform at or exceed "real" hardware levels. A lot depends upon the application. Some emulators perform JIT, others prescan the code stream and generate the instructions in parallel to their execution. Your mileage may vary.
>

Dan is right. For these types of questions, the only meaningful answer is
the one obtained from the OP performing their own benchmarks.

Dan, do the emulator vendors support trial licences so the OP can perform
these tests before committing to purchase the product ?

If so, I would recommend to the OP that this is the path they go down.

> For full disclosure, I work for the company that created and supports the Charon product stream. I can not comment on the performance of other emulators.
>

Do you ever get user applications that have race conditions on your
emulators that were never exposed on their original, slower, hardware ?

No reason for asking. Just curious.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On Wednesday, October 12, 2022 at 9:12:10 AM UTC-4, Simon Clubley wrote:
> On 2022-10-11, abrsvc <dansabr...@yahoo.com> wrote:
> >
> > I will state that little can be gained from looking at the standard benchmarks unless you know the relationship of them to your own application. I have not found single threaded performance numbers to relate to overall application performance either. That being said, I am aware of many installation using the Charon emulators that perform at or exceed "real" hardware levels. A lot depends upon the application. Some emulators perform JIT, others prescan the code stream and generate the instructions in parallel to their execution. Your mileage may vary.
> >
> Dan is right. For these types of questions, the only meaningful answer is
> the one obtained from the OP performing their own benchmarks.
>
> Dan, do the emulator vendors support trial licences so the OP can perform
> these tests before committing to purchase the product ?
>
> If so, I would recommend to the OP that this is the path they go down.
> > For full disclosure, I work for the company that created and supports the Charon product stream. I can not comment on the performance of other emulators.
> >
> Do you ever get user applications that have race conditions on your
> emulators that were never exposed on their original, slower, hardware ?
>
> No reason for asking. Just curious.
>
> Simon.
>
> --
> Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
> Walking destinations on a map are further away than they appear.
RE: Race conditions: As with any "hardware" change, poor code can result in these types of conditions although these days most of these had been addressed many years ago. I do see performance profile changes in how the systems work on a regular basis. For an Alpha installation for example, real hardware has limits of both main memory and Ethernet throughput speeds. With modern hardware, the Alpha will think it is connected to a slower network, with newer speeds actually realized. Also (at least with Charon-AXP) more main memory is available than with real hardware. This can help with overall application performance sometimes.

Dan

Most of what I'm looking at is single-threaded software, think Fortran scientific applications that have been around for a really long time. The newer versions were compiled with the IEEE floating points (T), the older versions were not. If I narrow things down to a half dozen emulators I want to look into, and all vendors want to give me a trial, this would still take a week or two, for me to configure systems and run benchmarks (unfortunately, my boss has other ideas for my time). This is why I'm asking if something is already available, or heuristically known. SpecFP? FLOPS? Emulator vendor A's pet project is IO bound, emulator vendor B's pet project is Int bound, emulator vendor C's pet project is Fortran bound? Given that EmuVM and AVT have offices that are 5 km apart, I assume there must be some significant differences between all the various emulators.

On 2022-10-12 14:26:04 +0000, Nick K said:

> Most of what I'm looking at is single-threaded software, think Fortran
> scientific applications that have been around for a really long time.
> The newer versions were compiled with the IEEE floating points (T), the
> older versions were not. If I narrow things down to a half dozen
> emulators I want to look into, and all vendors want to give me a trial,
> this would still take a week or two, for me to configure systems and
> run benchmarks (unfortunately, my boss has other ideas for my time).
> This is why I'm asking if something is already available, or
> heuristically known. SpecFP? FLOPS? Emulator vendor A's pet project
> is IO bound, emulator vendor B's pet project is Int bound, emulator
> vendor C's pet project is Fortran bound? Given that EmuVM and AVT have
> offices that are 5 km apart, I assume there must be some significant
> differences between all the various emulators.

There's no way to know without running the apps under emulation. I've
met performance and timing and stability surprises when updating apps
to different native hardware of the same architecture, as well as with
emulated hardware.

Emulation is not going to get you FLOPS. An emulator with JIT will
normally do better than one without. But whether even a JIT is
beneficial for your particular case, and your mix of code? Test.

I've also met weird floating point bugs. Old Fortran code with floating
point data can be... interesting... to reproduce the same results,
should even slight changes to floating point implementations and
settings arise. Test.

If you have source code, it's likely the results of a native port to
OpenVMS x86-64 will be faster than the path of emulation. Native
Fortran is arriving later this year or sometime early next year, per
the current VSI roadmap. And, of course. test.

--
Pure Personal Opinion | HoffmanLabs LLC

Nick K schrieb am 12.10.2022 um 16:26:
> Most of what I'm looking at is single-threaded software, think Fortran scientific applications that have been around for a really long time. The newer versions were compiled with the IEEE floating points (T), the older versions were not. If I narrow things down to a half dozen emulators I want to look into, and all vendors want to give me a trial, this would still take a week or two, for me to configure systems and run benchmarks (unfortunately, my boss has other ideas for my time). This is why I'm asking if something is already available, or heuristically known. SpecFP? FLOPS? Emulator vendor A's pet project is IO bound, emulator vendor B's pet project is Int bound, emulator vendor C's pet project is Fortran bound? Given that EmuVM and AVT have offices that are 5 km apart, I assume there must be some significant differences between all the various emulators.

from your base note:
> I'm curious what to expect when moving OpenVMS from an Alpha / ES server to a virtual machine with an emulator.

"ES server" is a wide field with different CPUs and different performance.

Having said that, Stromasys SA does run the old SPECfp2000 (and
SPECint2000) benchmarks with their CHARON-AXP emulators, incidently with
an ES40 configuration. I'm not sure whether the results are published
openly, but a Stromasys partner can probably give you a copy. You then
can compare the results with the original results submitted by Compaq
around the year 2000.

Without going into details, the current CHARON-AXP version is faster
than a ES40 6/667, but a bit slower than the ES40 6/833 with the
SPECfp2000 tests. (Almost 40% faster than the 6/833 for SPECint2000).

These tests were run on a Xeon 5122 3,6 GHz from 2017. With a more
current CPU you'll probably see better results - compare ProLiant DL380
results for SPECspeed2017_fp with different CPUs.

Ask the other vendors whether they run some kind of benchmark to compare
the emulators with the real hardware.

Does this help?

Hans.

Disclaimer: my company is a Stromasys Sales & Support Partner.

On Monday, October 31, 2022 at 6:54:42 PM UTC-4, Hans Bachner wrote:
> Nick K schrieb am 12.10.2022 um 16:26:
> > Most of what I'm looking at is single-threaded software, think Fortran scientific applications that have been around for a really long time. The newer versions were compiled with the IEEE floating points (T), the older versions were not. If I narrow things down to a half dozen emulators I want to look into, and all vendors want to give me a trial, this would still take a week or two, for me to configure systems and run benchmarks (unfortunately, my boss has other ideas for my time). This is why I'm asking if something is already available, or heuristically known. SpecFP? FLOPS? Emulator vendor A's pet project is IO bound, emulator vendor B's pet project is Int bound, emulator vendor C's pet project is Fortran bound? Given that EmuVM and AVT have offices that are 5 km apart, I assume there must be some significant differences between all the various emulators.
> from your base note:
> > I'm curious what to expect when moving OpenVMS from an Alpha / ES server to a virtual machine with an emulator.
> "ES server" is a wide field with different CPUs and different performance..
>
> Having said that, Stromasys SA does run the old SPECfp2000 (and
> SPECint2000) benchmarks with their CHARON-AXP emulators, incidently with
> an ES40 configuration. I'm not sure whether the results are published
> openly, but a Stromasys partner can probably give you a copy. You then
> can compare the results with the original results submitted by Compaq
> around the year 2000.
>
> Without going into details, the current CHARON-AXP version is faster
> than a ES40 6/667, but a bit slower than the ES40 6/833 with the
> SPECfp2000 tests. (Almost 40% faster than the 6/833 for SPECint2000).
>
> These tests were run on a Xeon 5122 3,6 GHz from 2017. With a more
> current CPU you'll probably see better results - compare ProLiant DL380
> results for SPECspeed2017_fp with different CPUs.
>
> Ask the other vendors whether they run some kind of benchmark to compare
> the emulators with the real hardware.
>
> Does this help?
>
> Hans.
>
> Disclaimer: my company is a Stromasys Sales & Support Partner.
I don't have specific numbers, but Hans is correct. The CPU performance of the Charon-AXP emulator will depend upon the underlying Core speed as well as the implementation. I am not aware of any specific numbers that are published since there are too many variables to make them meaningful. Customers that are concerned about performance are usually requested to either provide testing programs or perform internal tests during a proof of concept trial. Please note that there are alternatives available in the emulated environment that can improve performance over "real" hardware. Memory size is one example. Emulation allows for more memory to be available than in some hardware models.

Dan
Stromasys Sr. Engineer.