On 8/26/23 00:22, Arne Vajhøj wrote:
> On 8/25/2023 9:03 AM, Simon Clubley wrote:
>> On 2023-08-25, Johnny Billquist <bqt@softjar.se> wrote:
>>> On 2023-08-25 14:18, Simon Clubley wrote:
>>>> It has now become _very_ clear to me that the use of ALGOL-based
>>>> languages
>>>> in OS development was very seriously widespread by the time DEC came to
>>>> design VMS. Pity DEC didn't join them.
>>>
>>> Including Multics. Not an outstanding success exactly...
>>> Most companies that went that way early was way less successful than
>>> DEC, so it is kindof strange to claim that DEC did it wrong and they did
>>> it right...
>>>
>>
>> In way too many cases, marketing success (at least in the short term,
>> and sometimes in the longer term) doesn't appear to be related to
>> technical elegance.
>>
>> After all, we all use Windows...
>>
>> OTOH, Unix with its portable HLL approach, has outlasted VMS...
>
> VMS still exist.
>
> If we limit Unix to kernel of Unix origin then I don't see it as
> unrealistic that VMS outlast Unix.
>
> Tru64 is dead. HP-UX is practically dead. Solaris is close to dead.
> AIX is not well. And FreeBSD/OpenBSD/NetBSD seem to increasingly
> get status of hobby OS for enthusiasts.
>
> Arne
>

Agreed, proprietary unix is more or less dead, because no single
company has the resources to match the dedicated and partly free
efforts of the open source movement. Small proprietary dev teams vs
potentially thousands of eyes for open source, can never match
that, on quality, delivery, or cost.

Very much FreeBSD here for some years, after decades first with dec,
then Sun. Forms the basic of at least some proprietary offerings, as
well as millions of embedded devices. Linux is still a unix,
and runs the majority of web sites of the world, so if anything,
unix has won the os wars...

Chris

On 2023-08-27, terry-...@glaver.org <terry-groups@glaver.org> wrote:
> On Sunday, August 27, 2023 at 10:27:33?AM UTC-4, Johnny Billquist wrote:
>> But actually, with the TK70, the UI and firmware side was mostly sorted,
>> and it actually can be used in a sane way. You still have the constant
>> cleaning to deal with, though.
>
> And then they came out with the TZ30. Rumors that the TZ30 was the result of a drunken bar bet between a DECie and a Mitsubishi upper manager are unconfirmed. 8-}
>

At least the TZ30 was reliable and just worked. :-) (Had one in a VAXstation).

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-08-28, chrisq <devzero@nospam.com> wrote:
>
> Very much FreeBSD here for some years, after decades first with dec,
> then Sun. Forms the basic of at least some proprietary offerings, as
> well as millions of embedded devices. Linux is still a unix,
> and runs the majority of web sites of the world, so if anything,
> unix has won the os wars...
>

Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
serious users... :-) ).

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On Thu, 1970-01-01 at 00:00 +0000, Simon Clubley wrote:
> > Very much FreeBSD here for some years, after decades first with
> > dec,
> > then Sun. Forms the basic of at least some proprietary offerings,
> > as
> > well as millions of embedded devices. Linux is still a unix,
> > and runs the majority of web sites of the world, so if anything,
> > unix has won the os wars...
> >
>
> Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
> serious users... :-) ).

Netflix picked FreeBSD as it could chuck out data at 400GB/s. Linux was
not even close.
--
Tactical Nuclear Kittens

On 2023-08-29, Single Stage to Orbit <alex.buell@munted.eu> wrote:
> On Thu, 1970-01-01 at 00:00 +0000, Simon Clubley wrote:
>> > Very much FreeBSD here for some years, after decades first with
>> > dec,
>> > then Sun. Forms the basic of at least some proprietary offerings,
>> > as
>> > well as millions of embedded devices. Linux is still a unix,
>> > and runs the majority of web sites of the world, so if anything,
>> > unix has won the os wars...
>> >
>>
>> Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
>> serious users... :-) ).
>
> Netflix picked FreeBSD as it could chuck out data at 400GB/s. Linux was
> not even close.

10 years from now (assuming the economy hasn't collapsed by then :-) ):

400GB/s ??? Is that all ??? Amateurs!!! :-)

On a more serious note, I wonder what the maximum rate VMS is capable
of emitting data at if it was using the fastest network hardware
available.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-08-29 19:25, Simon Clubley wrote:
> On 2023-08-29, Single Stage to Orbit <alex.buell@munted.eu> wrote:
>> On Thu, 1970-01-01 at 00:00 +0000, Simon Clubley wrote:
>>>> Very much FreeBSD here for some years, after decades first with
>>>> dec,
>>>> then Sun. Forms the basic of at least some proprietary offerings,
>>>> as
>>>> well as millions of embedded devices. Linux is still a unix,
>>>> and runs the majority of web sites of the world, so if anything,
>>>> unix has won the os wars...
>>>>
>>>
>>> Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
>>> serious users... :-) ).
>>
>> Netflix picked FreeBSD as it could chuck out data at 400GB/s. Linux was
>> not even close.
>
> 10 years from now (assuming the economy hasn't collapsed by then :-) ):
>
> 400GB/s ??? Is that all ??? Amateurs!!! :-)

Well. I have this friend of mine, who installed 40 Gb/s at his moms
place in 2007...

> On a more serious note, I wonder what the maximum rate VMS is capable
> of emitting data at if it was using the fastest network hardware
> available.

What a weird question. VMS in itself don't have any limits. It's all
always just about the hardware.
Some software might be able to squeeze more out of the same hardware,
but just spin up faster hardware, and you'll get higher throughput. But
any system will basically just be limited by the speed of the network
hardware, if that item is fixed. You can't go above that. But there are
no reasons why you wouldn't be able to get to that point.

These are the kind of questions that sometimes make me wonder if you
know anything at all about computers. But then you do some other posts
which clearly demonstrate that you do understand some stuff.

Johnny

On 8/29/2023 8:13 AM, Simon Clubley wrote:
> On 2023-08-28, chrisq <devzero@nospam.com> wrote:
>> Very much FreeBSD here for some years, after decades first with dec,
>> then Sun. Forms the basic of at least some proprietary offerings, as
>> well as millions of embedded devices. Linux is still a unix,
>> and runs the majority of web sites of the world, so if anything,
>> unix has won the os wars...
>
> Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
> serious users... :-) ).

It definitely has some but not as many as it once had.

20 years ago FreeBSD was sort of the free "high end" OS
and used by places where Windows and Linux was not considered
good enough.

The world has changed since then.

Linux has also squeezed FreeBSD market share.

Primarily for non-technical reasons:
- Linux got backing from IBM, Oracle etc.
- Easier to hire Linux expertise
- Many companies standardize on a Linux only strategy for applications
(exception for the stuff supporting PC's)
- Cloud vendors has pushed Linux
- Many companies are moving applications to Kubernetes on Linux (*)

*) I believe that FreeBSD got jails before Linux got containers and
jails should be just as good, but FreeBSD jails does not have
the eco-system that Linux containers has (Kubernetes, OpenShift etc.)

Arne

Johnny Billquist <bqt@softjar.se> writes:

> On 2023-08-29 19:25, Simon Clubley wrote:

[ snip ]

>> 400GB/s ??? Is that all ??? Amateurs!!! :-)

> Well. I have this friend of mine, who installed 40 Gb/s at his moms
> place in 2007...

The best part is that our mutual friend did it as a Midsommar hack, just
because he could.

--
Rich Alderson news@alderson.users.panix.com
Audendum est, et veritas investiganda; quam etiamsi non assequamur,
omnino tamen proprius, quam nunc sumus, ad eam perveniemus.
--Galen

On 8/29/2023 9:15 AM, Single Stage to Orbit wrote:
> On Thu, 1970-01-01 at 00:00 +0000, Simon Clubley wrote:
>>> Very much FreeBSD here for some years, after decades first with
>>> dec,
>>> then Sun. Forms the basic of at least some proprietary offerings,
>>> as
>>> well as millions of embedded devices. Linux is still a unix,
>>> and runs the majority of web sites of the world, so if anything,
>>> unix has won the os wars...
>>>
>>
>> Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
>> serious users... :-) ).
>
> Netflix picked FreeBSD as it could chuck out data at 400GB/s. Linux was
> not even close.

Yes. But.

NetFlix is running their general server load (node.js, Spring Boot,
Kafka, MySQL, Cassandra etc.) on Linux (supposedly Ubuntu)
in AWS.

NetFlix chose FreeBSD for their CDN appliance that they deploy
at ISP's.

Maybe not so important market share wise.

But certainly proof of FreeBSD's technical qualities. The default
choice would have been Linux, so FreeBSD must have proven to
be better to be selected.

Arne

On 8/29/2023 3:18 PM, Johnny Billquist wrote:
> On 2023-08-29 19:25, Simon Clubley wrote:
>> On 2023-08-29, Single Stage to Orbit <alex.buell@munted.eu> wrote:
>>> On Thu, 1970-01-01 at 00:00 +0000, Simon Clubley wrote:
>>>>> Very much FreeBSD here for some years, after decades first with
>>>>> dec,
>>>>> then Sun. Forms the basic of at least some proprietary offerings,
>>>>> as
>>>>> well as millions of embedded devices. Linux is still a unix,
>>>>> and runs the majority of web sites of the world, so if anything,
>>>>> unix has won the os wars...
>>>>>
>>>>
>>>> Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
>>>> serious users... :-) ).
>>>
>>> Netflix picked FreeBSD as it could chuck out data at 400GB/s. Linux was
>>> not even close.
>>
>> 10 years from now (assuming the economy hasn't collapsed by then :-) ):
>>
>> 400GB/s ??? Is that all ??? Amateurs!!! :-)
>
> Well. I have this friend of mine, who installed 40 Gb/s at his moms
> place in 2007...
>
>> On a more serious note, I wonder what the maximum rate VMS is capable
>> of emitting data at if it was using the fastest network hardware
>> available.
>
> What a weird question. VMS in itself don't have any limits. It's all
> always just about the hardware.

Not really. VMS has always been notoriously slow with I/O and I assume
that's what Simon was hinting at.

> Some software might be able to squeeze more out of the same hardware,
> but just spin up faster hardware, and you'll get higher throughput. But
> any system will basically just be limited by the speed of the network
> hardware, if that item is fixed. You can't go above that. But there are
> no reasons why you wouldn't be able to get to that point.

You are forgetting the overhead of the drivers and any underlying O/S
code that has to be called to do it.

>
> These are the kind of questions that sometimes make me wonder if you
> know anything at all about computers. But then you do some other posts
> which clearly demonstrate that you do understand some stuff.

I think it was just a subtle poke in the ribs for VMS.

bill

On 8/29/23 3:27 PM, bill wrote:
> On 8/29/2023 3:18 PM, Johnny Billquist wrote:
>> On 2023-08-29 19:25, Simon Clubley wrote:

>>> On a more serious note, I wonder what the maximum rate VMS is capable
>>> of emitting data at if it was using the fastest network hardware
>>> available.
>>
>> What a weird question. VMS in itself don't have any limits. It's all
>> always just about the hardware.
>
> Not really.  VMS has always been notoriously slow with I/O and I assume
> that's what Simon was hinting at.

Right, and differently so for different kinds of I/O. See posts from a
few years ago by (I think) Eric Johnson on performance testing of the
network stack. And I wish I could remember the name of the guy who
posted about slow disk I/O even longer ago (Dave something?) including
code to do the testing.

VSI has canceled two different file system projects, one of which was
GFS and one of which was "not Spiralog" by Andy Goldstein (I don't know
if it ever had a name but Clair Grant posted here that it inherited some
concepts but was not the same thing as Spiralog). Something will have to
be done eventually for disk I/O, and while the file system isn't the
whole enchilada, it's certainly one big part of it.

The network stack improvements described here:

http://www.vmsconsultancy.com/download/NL-VMSUpdate-2015/Vienna%20LAN%20Performance%20Improvements.pdf

will hopefully be revisited at some point. If they aren't, then VMS
will remain slower at network performance than other systems using the
same networking hardware. I totally get why the port had to take
precedence for a small company, but holding the line is not the same
thing as moving forward.

On 8/29/2023 6:49 PM, Craig A. Berry wrote:
> On 8/29/23 3:27 PM, bill wrote:
>> On 8/29/2023 3:18 PM, Johnny Billquist wrote:
>>> On 2023-08-29 19:25, Simon Clubley wrote:
>>>> On a more serious note, I wonder what the maximum rate VMS is capable
>>>> of emitting data at if it was using the fastest network hardware
>>>> available.
>>>
>>> What a weird question. VMS in itself don't have any limits. It's all
>>> always just about the hardware.
>>
>> Not really.  VMS has always been notoriously slow with I/O and I assume
>> that's what Simon was hinting at.
>
> Right, and differently so for different kinds of I/O.  See posts from a
> few years ago by (I think) Eric Johnson on performance testing of the
> network stack.  And I wish I could remember the name of the guy who
> posted about slow disk I/O even longer ago (Dave something?) including
> code to do the testing.

David Mathog?

Arne

On 8/29/2023 6:49 PM, Craig A. Berry wrote:
> On 8/29/23 3:27 PM, bill wrote:
>> On 8/29/2023 3:18 PM, Johnny Billquist wrote:
>>> On 2023-08-29 19:25, Simon Clubley wrote:
>>>> On a more serious note, I wonder what the maximum rate VMS is capable
>>>> of emitting data at if it was using the fastest network hardware
>>>> available.
>>>
>>> What a weird question. VMS in itself don't have any limits. It's all
>>> always just about the hardware.
>>
>> Not really.  VMS has always been notoriously slow with I/O and I assume
>> that's what Simon was hinting at.
>
> Right, and differently so for different kinds of I/O.  See posts from a
> few years ago by (I think) Eric Johnson on performance testing of the
> network stack.  And I wish I could remember the name of the guy who
> posted about slow disk I/O even longer ago (Dave something?) including
> code to do the testing.
>
> VSI has canceled two different file system projects, one of which was
> GFS and one of which was "not Spiralog" by Andy Goldstein (I don't know
> if it ever had a name but Clair Grant posted here that it inherited some
> concepts but was not the same thing as Spiralog). Something will have to
> be done eventually for disk I/O, and while the file system isn't the
> whole enchilada, it's certainly one big part of it.
>
> The network stack improvements described here:
>
> http://www.vmsconsultancy.com/download/NL-VMSUpdate-2015/Vienna%20LAN%20Performance%20Improvements.pdf
>
> will hopefully be revisited at some point.  If they aren't, then VMS
> will remain slower at network performance than other systems using the
> same networking hardware.  I totally get why the port had to take
> precedence for a small company, but holding the line is not the same
> thing as moving forward.

My expectation would be that:
* VMS network IO is slower than "modern whatever OS" network IO
* the difference for network IO is significant smaller than for
file IO
* the CPU's running VMS today are so fast and the network cards
supported by VMS so slow that OS overhead is not a huge problem

But I could be wrong.

Arne

On 8/29/2023 7:30 PM, Arne Vajhøj wrote:
> My expectation would be that:
....
> * the CPU's running VMS today are so fast and the network cards
>   supported by VMS so slow that OS overhead is not a huge problem
>
> But I could be wrong.

I took a quick look at:
https://www.islandco.com/hp-parts/hp-integrity-network-options

It seems like the most common netword card speed is 1 Gbit, but
that 10 Gbit cards are available.

I don't see 400 Gbps being possible. Not enough slots in an Itanium
to be able to put enough 10 Gbit cards in to achieve that.

Arne

On Thu, 1970-01-01 at 00:00 +0000, Arne Vajhøj wrote:
>
> But certainly proof of FreeBSD's technical qualities. The default
> choice would have been Linux, so FreeBSD must have proven to
> be better to be selected.

Yup.
--
Tactical Nuclear Kittens

On 8/29/2023 6:49 PM, Craig A. Berry wrote:
>
> On 8/29/23 3:27 PM, bill wrote:
>> On 8/29/2023 3:18 PM, Johnny Billquist wrote:
>>> On 2023-08-29 19:25, Simon Clubley wrote:
>
>>>> On a more serious note, I wonder what the maximum rate VMS is capable
>>>> of emitting data at if it was using the fastest network hardware
>>>> available.
>>>
>>> What a weird question. VMS in itself don't have any limits. It's all
>>> always just about the hardware.
>>
>> Not really.  VMS has always been notoriously slow with I/O and I assume
>> that's what Simon was hinting at.
>
> Right, and differently so for different kinds of I/O.  See posts from a
> few years ago by (I think) Eric Johnson on performance testing of the
> network stack.  And I wish I could remember the name of the guy who
> posted about slow disk I/O even longer ago (Dave something?) including
> code to do the testing.
>
> VSI has canceled two different file system projects, one of which was
> GFS and one of which was "not Spiralog" by Andy Goldstein (I don't know
> if it ever had a name but Clair Grant posted here that it inherited some
> concepts but was not the same thing as Spiralog). Something will have to
> be done eventually for disk I/O, and while the file system isn't the
> whole enchilada, it's certainly one big part of it.
>
> The network stack improvements described here:
>
> http://www.vmsconsultancy.com/download/NL-VMSUpdate-2015/Vienna%20LAN%20Performance%20Improvements.pdf
>
> will hopefully be revisited at some point.  If they aren't, then VMS
> will remain slower at network performance than other systems using the
> same networking hardware.  I totally get why the port had to take
> precedence for a small company, but holding the line is not the same
> thing as moving forward.
>

Well, to be honest, is network performance really that important for the
things VMS is used for? I doubt we are going to see a proliferation of
VMS run web sites.

bill

On 8/29/2023 8:05 PM, bill wrote:
> On 8/29/2023 6:49 PM, Craig A. Berry wrote:
>> The network stack improvements described here:
>>
>> http://www.vmsconsultancy.com/download/NL-VMSUpdate-2015/Vienna%20LAN%20Performance%20Improvements.pdf
>>
>> will hopefully be revisited at some point.  If they aren't, then VMS
>> will remain slower at network performance than other systems using the
>> same networking hardware.  I totally get why the port had to take
>> precedence for a small company, but holding the line is not the same
>> thing as moving forward.
>
>
> Well, to be honest, is network performance really that important for the
> things VMS is used for?  I doubt we are going to see a proliferation of
> VMS run web sites.

Probably not.

And note that many web sites does not need that crazy much bandwidth.

Some web service calls sending NNN byte HTTP headers and MMM byte
JSON in request and response does not require that much.

NN KB web pages and NNN KB graphics does not require that much.

Streaming HD and 4K video require a lot!

(I believe around respectively 5 and 25 Mbps per client)

Arne

On 2023-08-29 22:27, bill wrote:
> On 8/29/2023 3:18 PM, Johnny Billquist wrote:
>> On 2023-08-29 19:25, Simon Clubley wrote:
>>> On 2023-08-29, Single Stage to Orbit <alex.buell@munted.eu> wrote:
>>>> On Thu, 1970-01-01 at 00:00 +0000, Simon Clubley wrote:
>>>>>> Very much FreeBSD here for some years, after decades first with
>>>>>> dec,
>>>>>> then Sun. Forms the basic of at least some proprietary offerings,
>>>>>> as
>>>>>> well as millions of embedded devices. Linux is still a unix,
>>>>>> and runs the majority of web sites of the world, so if anything,
>>>>>> unix has won the os wars...
>>>>>>
>>>>>
>>>>> Yes, very much so. (And I can't believe Arne thinks the *BSDs have no
>>>>> serious users... :-) ).
>>>>
>>>> Netflix picked FreeBSD as it could chuck out data at 400GB/s. Linux was
>>>> not even close.
>>>
>>> 10 years from now (assuming the economy hasn't collapsed by then :-) ):
>>>
>>> 400GB/s ??? Is that all ??? Amateurs!!! :-)
>>
>> Well. I have this friend of mine, who installed 40 Gb/s at his moms
>> place in 2007...
>>
>>> On a more serious note, I wonder what the maximum rate VMS is capable
>>> of emitting data at if it was using the fastest network hardware
>>> available.
>>
>> What a weird question. VMS in itself don't have any limits. It's all
>> always just about the hardware.
>
> Not really.  VMS has always been notoriously slow with I/O and I assume
> that's what Simon was hinting at.

So? It just means that other systems might achieve higher rate of I/O
throughput than VMS on a specific piece of hardware. Nothing prevents me
from throwing faster hardware on the problem until I saturate the
network, no matter which OS I'm using.

>> Some software might be able to squeeze more out of the same hardware,
>> but just spin up faster hardware, and you'll get higher throughput.
>> But any system will basically just be limited by the speed of the
>> network hardware, if that item is fixed. You can't go above that. But
>> there are no reasons why you wouldn't be able to get to that point.
>
> You are forgetting the overhead of the drivers and any underlying O/S
> code that has to be called to do it.

Irrelevant. That just means I need to throw some more hardware on the
problem. Not that the OS somehow prevents me from achieving a higher
throughput.

>> These are the kind of questions that sometimes make me wonder if you
>> know anything at all about computers. But then you do some other posts
>> which clearly demonstrate that you do understand some stuff.
>
> I think it was just a subtle poke in the ribs for VMS.

I think it was a display of complete ignorance on behalf of Simon.
Possibly just not thinking before writing. Who knows...?

Johnny

On 8/29/23 6:06 PM, Arne Vajhøj wrote:
> On 8/29/2023 6:49 PM, Craig A. Berry wrote:
>> On 8/29/23 3:27 PM, bill wrote:
>>> On 8/29/2023 3:18 PM, Johnny Billquist wrote:
>>>> On 2023-08-29 19:25, Simon Clubley wrote:
>>>>> On a more serious note, I wonder what the maximum rate VMS is capable
>>>>> of emitting data at if it was using the fastest network hardware
>>>>> available.
>>>>
>>>> What a weird question. VMS in itself don't have any limits. It's all
>>>> always just about the hardware.
>>>
>>> Not really.  VMS has always been notoriously slow with I/O and I assume
>>> that's what Simon was hinting at.
>>
>> Right, and differently so for different kinds of I/O.  See posts from a
>> few years ago by (I think) Eric Johnson on performance testing of the
>> network stack.  And I wish I could remember the name of the guy who
>> posted about slow disk I/O even longer ago (Dave something?) including
>> code to do the testing.
>
> David Mathog?

Yes, he's the one.

On Tuesday, August 29, 2023 at 8:29:30 PM UTC-4, Johnny Billquist wrote:
> On 2023-08-29 22:27, bill wrote:
> > Not really. VMS has always been notoriously slow with I/O and I assume
> > that's what Simon was hinting at.
> So? It just means that other systems might achieve higher rate of I/O
> throughput than VMS on a specific piece of hardware. Nothing prevents me
> from throwing faster hardware on the problem until I saturate the
> network, no matter which OS I'm using.

I discovered massive speed differences way back when on a VAX-
11/780 with a TU78 tape drive - $ BACKUP/IMAGE made the tape
drive go "bloop... bloop... bloop" while a $ BACKUP/PHYSICAL made
the tape drive go "neeeeeeeeeeeeeee" with the same block size.
Same disk, same tape, filesystem overhead.

Since then, both speeds have gotten faster but VMS file-structured
I/O is still WAY slower than the physical hardware. I have an x86-64
system running here with a load of enterprise SSDs that give me a
sustained write performance of 1.8GByte/sec under FreeBSD 13.
I'm running an emulated Alpha (AlphaVM) on it as I haven't heard
anything from VSI since I (re) registered for their hobbyist program
quite a few months ago. But from what I've seen, emulated Tru64 is
a lot faster than VMS under the same AlphaVM release on the same
host OS / hardware.

Yes, that's disk I/O. But I would assume that network paths also have
high overhead (not that it really matters, as real-world high-bandwidth/
high-volume transfers likely involve filesystem data).

On 2023-08-30 10:04, terry-...@glaver.org wrote:
> On Tuesday, August 29, 2023 at 8:29:30 PM UTC-4, Johnny Billquist wrote:
>> On 2023-08-29 22:27, bill wrote:
>>> Not really. VMS has always been notoriously slow with I/O and I assume
>>> that's what Simon was hinting at.
>> So? It just means that other systems might achieve higher rate of I/O
>> throughput than VMS on a specific piece of hardware. Nothing prevents me
>> from throwing faster hardware on the problem until I saturate the
>> network, no matter which OS I'm using.
>
> I discovered massive speed differences way back when on a VAX-
> 11/780 with a TU78 tape drive - $ BACKUP/IMAGE made the tape
> drive go "bloop... bloop... bloop" while a $ BACKUP/PHYSICAL made
> the tape drive go "neeeeeeeeeeeeeee" with the same block size.
> Same disk, same tape, filesystem overhead.
>
> Since then, both speeds have gotten faster but VMS file-structured
> I/O is still WAY slower than the physical hardware. I have an x86-64
> system running here with a load of enterprise SSDs that give me a
> sustained write performance of 1.8GByte/sec under FreeBSD 13.
> I'm running an emulated Alpha (AlphaVM) on it as I haven't heard
> anything from VSI since I (re) registered for their hobbyist program
> quite a few months ago. But from what I've seen, emulated Tru64 is
> a lot faster than VMS under the same AlphaVM release on the same
> host OS / hardware.
>
> Yes, that's disk I/O. But I would assume that network paths also have
> high overhead (not that it really matters, as real-world high-bandwidth/
> high-volume transfers likely involve filesystem data).

Which still means, in the end, that VMS do not have a limitation on the
speed of I/O as such, and a question of "how fast can VMS push bits" is
really just a question of how fast your hardware is.

But your observation also raise another good point. I/O is sortof slow
in VMS, but it's not the actual I/O that is the main problem, but RMS.
The overhead here is pretty massive compared to something stupid like Unix.

Not sure how easy it is to dodge RMS under VMS. In RSX, you can just do
the QIOs to the ACP yourself and go around the whole thing, which makes
I/O way faster. Of course, since files still have this structure thing,
most of the time you are still going to have to pay for it somewhere.
But if you are happy with just raw disk blocks, the basic I/O do not
have near as much penalty. Admitted, the ODS-1 (as well as ODS-2)
structure have some inherent limitations that carry some cost as well.
So you could improve things some by doing some other implementation on
the file system level.
But mainly, no matter what the file system design is, you are still
going to have the pain of RMS, which is the majority of the cost. And
you'll never get away from this as long as you use VMS.

It's like if you were to always access all files in Unix via BDB. If
people were to do that, the numbers on Unix systems would also look very
different.

Johnny

On Wednesday, August 30, 2023 at 9:53:11 AM UTC+1, Johnny Billquist wrote:
> On 2023-08-30 10:04, terry-...@glaver.org wrote:
> > On Tuesday, August 29, 2023 at 8:29:30 PM UTC-4, Johnny Billquist wrote:
> >> On 2023-08-29 22:27, bill wrote:
> >>> Not really. VMS has always been notoriously slow with I/O and I assume
> >>> that's what Simon was hinting at.
> >> So? It just means that other systems might achieve higher rate of I/O
> >> throughput than VMS on a specific piece of hardware. Nothing prevents me
> >> from throwing faster hardware on the problem until I saturate the
> >> network, no matter which OS I'm using.
> >
> > I discovered massive speed differences way back when on a VAX-
> > 11/780 with a TU78 tape drive - $ BACKUP/IMAGE made the tape
> > drive go "bloop... bloop... bloop" while a $ BACKUP/PHYSICAL made
> > the tape drive go "neeeeeeeeeeeeeee" with the same block size.
> > Same disk, same tape, filesystem overhead.
> >
> > Since then, both speeds have gotten faster but VMS file-structured
> > I/O is still WAY slower than the physical hardware. I have an x86-64
> > system running here with a load of enterprise SSDs that give me a
> > sustained write performance of 1.8GByte/sec under FreeBSD 13.
> > I'm running an emulated Alpha (AlphaVM) on it as I haven't heard
> > anything from VSI since I (re) registered for their hobbyist program
> > quite a few months ago. But from what I've seen, emulated Tru64 is
> > a lot faster than VMS under the same AlphaVM release on the same
> > host OS / hardware.
> >
> > Yes, that's disk I/O. But I would assume that network paths also have
> > high overhead (not that it really matters, as real-world high-bandwidth/
> > high-volume transfers likely involve filesystem data).
> Which still means, in the end, that VMS do not have a limitation on the
> speed of I/O as such, and a question of "how fast can VMS push bits" is
> really just a question of how fast your hardware is.
>
> But your observation also raise another good point. I/O is sortof slow
> in VMS, but it's not the actual I/O that is the main problem, but RMS.
> The overhead here is pretty massive compared to something stupid like Unix.
>
> Not sure how easy it is to dodge RMS under VMS. In RSX, you can just do
> the QIOs to the ACP yourself and go around the whole thing, which makes
> I/O way faster. Of course, since files still have this structure thing,
> most of the time you are still going to have to pay for it somewhere.
> But if you are happy with just raw disk blocks, the basic I/O do not
> have near as much penalty. Admitted, the ODS-1 (as well as ODS-2)
> structure have some inherent limitations that carry some cost as well.
> So you could improve things some by doing some other implementation on
> the file system level.
> But mainly, no matter what the file system design is, you are still
> going to have the pain of RMS, which is the majority of the cost. And
> you'll never get away from this as long as you use VMS.
>
> It's like if you were to always access all files in Unix via BDB. If
> people were to do that, the numbers on Unix systems would also look very
> different.
>
> Johnny

QIO to files is a documented option in I/O User’s Reference Manual Chapter 1 and there's Chapter 10 on Fast I/O.
https://docs.vmssoftware.com/vsi-openvms-io-user-s-reference-manual/

Den 2023-08-30 kl. 10:53, skrev Johnny Billquist:
> On 2023-08-30 10:04, terry-...@glaver.org wrote:
>> On Tuesday, August 29, 2023 at 8:29:30 PM UTC-4, Johnny Billquist wrote:
>>> On 2023-08-29 22:27, bill wrote:
>>>> Not really.  VMS has always been notoriously slow with I/O and I assume
>>>> that's what Simon was hinting at.
>>> So? It just means that other systems might achieve higher rate of I/O
>>> throughput than VMS on a specific piece of hardware. Nothing prevents me
>>> from throwing faster hardware on the problem until I saturate the
>>> network, no matter which OS I'm using.
>>
>> I discovered massive speed differences way back when on a VAX-
>> 11/780 with a TU78 tape drive - $ BACKUP/IMAGE made the tape
>> drive go "bloop... bloop... bloop" while a $ BACKUP/PHYSICAL made
>> the tape drive go "neeeeeeeeeeeeeee" with the same block size.
>> Same disk, same tape, filesystem overhead.
>>
>> Since then, both speeds have gotten faster but VMS file-structured
>> I/O is still WAY slower than the physical hardware. I have an x86-64
>> system running here with a load of enterprise SSDs that give me a
>> sustained write performance of 1.8GByte/sec under FreeBSD 13.
>> I'm running an emulated Alpha (AlphaVM) on it as I haven't heard
>> anything from VSI since I (re) registered for their hobbyist program
>> quite a few months ago. But from what I've seen, emulated Tru64 is
>> a lot faster than VMS under the same AlphaVM release on the same
>> host OS / hardware.
>>
>> Yes, that's disk I/O. But I would assume that network paths also have
>> high overhead (not that it really matters, as real-world high-bandwidth/
>> high-volume transfers likely involve filesystem data).
>
> Which still means, in the end, that VMS do not have a limitation on the
> speed of I/O as such, and a question of "how fast can VMS push bits" is
> really just a question of how fast your hardware is.
>
> But your observation also raise another good point. I/O is sortof slow in
> VMS, but it's not the actual I/O that is the main problem, but RMS. The
> overhead here is pretty massive compared to something stupid like Unix.
>
> Not sure how easy it is to dodge RMS under VMS. In RSX, you can just do the
> QIOs to the ACP yourself and go around the whole thing, which makes I/O way
> faster. Of course, since files still have this structure thing, most of the
> time you are still going to have to pay for it somewhere. But if you are
> happy with just raw disk blocks, the basic I/O do not have near as much
> penalty. Admitted, the ODS-1 (as well as ODS-2) structure have some
> inherent limitations that carry some cost as well. So you could improve
> things some by doing some other implementation on the file system level.
> But mainly, no matter what the file system design is, you are still going
> to have the pain of RMS, which is the majority of the cost. And you'll
> never get away from this as long as you use VMS.
>
> It's like if you were to always access all files in Unix via BDB. If people
> were to do that, the numbers on Unix systems would also look very different.
>
>   Johnny
>

Note that Rdb does not use RMS for database I/O.

It only use RMS when there is some sequential file involved,
such as when creating a .RBF backup file or an "export" file.
The table "unload/load" probably also use RMS for the .UNL files.

But the whole core database operation does not use RMS, AFAIK...

Jan-Erik.

On Tuesday, August 29, 2023 at 6:49:51 PM UTC-4, Craig A. Berry wrote:
> On 8/29/23 3:27 PM, bill wrote:
> > On 8/29/2023 3:18 PM, Johnny Billquist wrote:
> >> On 2023-08-29 19:25, Simon Clubley wrote:
>
> >>> On a more serious note, I wonder what the maximum rate VMS is capable
> >>> of emitting data at if it was using the fastest network hardware
> >>> available.
> >>
> >> What a weird question. VMS in itself don't have any limits. It's all
> >> always just about the hardware.
> >
> > Not really. VMS has always been notoriously slow with I/O and I assume
> > that's what Simon was hinting at.
> Right, and differently so for different kinds of I/O. See posts from a
> few years ago by (I think) Eric Johnson on performance testing of the
> network stack. And I wish I could remember the name of the guy who
> posted about slow disk I/O even longer ago (Dave something?) including
> code to do the testing.
>
> VSI has canceled two different file system projects, one of which was
> GFS and one of which was "not Spiralog" by Andy Goldstein (I don't know
> if it ever had a name but Clair Grant posted here that it inherited some
> concepts but was not the same thing as Spiralog). Something will have to
> be done eventually for disk I/O, and while the file system isn't the
> whole enchilada, it's certainly one big part of it.
>
> The network stack improvements described here:
>
> http://www.vmsconsultancy.com/download/NL-VMSUpdate-2015/Vienna%20LAN%20Performance%20Improvements.pdf
>
> will hopefully be revisited at some point. If they aren't, then VMS
> will remain slower at network performance than other systems using the
> same networking hardware. I totally get why the port had to take
> precedence for a small company, but holding the line is not the same
> thing as moving forward.
Craig,

Indeed. There have been a number of projects in the I/O area, few of which have emerged into released form.

There are a number of issues. I dug into them rather deeply while writing my Ph.D. dissertation. OpenVMS has a good collection of them, as do essentially all of the other extant operating systems. IMHO, they can be remediated in many ways.

With disk I/O (more properly referred to as "mass storage" I/O these days), there are unnecessary serializations forced by driver processing. IMHO, they are a remnant of the days when kernel storage was far more constrained, e.g. IBM System/360 under OS/360 or DEC PDP-11 under RSX. Now that kernel memory is far more available, less serial approaches become viable.

Other mass storage issues can often be addressed, at least on sequential files, by adjusting buffering limits. The "as shipped" defaults were set back in the memory-constrained days of the VAX-11/780 and are kept at those settings for back compatibility. Changing them is straightforward, although one has to also change quotas in the UAF correspondingly. I have spoken for over thirty years on that particular performance issue, starting at NASA Marshall in the late 1980s. I have sample programs that have gone from less than 10% CPU utilization on a MicroVAX 3100 to 100%, merely by changing the RMS buffer factors.

The network stack has issues when it comes to high performance transfers. As I recall, several years ago there was an early-adoptor iSCSI kit, later withdrawn for a number of issues. The accompanying documentation included negative comments about performance. Efficiency of the IP stack would allow the use of increasingly popular iSCSI hardware.

At the User API-level, FAST I/O does reduce some overhead, but requires application-level changes. Fast I/O makes sense for high I/O libraries, e.g., Pathworks.

Some of these issues remains relevant when running as a virtualized guest. Some becomes significantly less relevant when one is paravirtualized as a guest on a virtual machine. However, paravirtualization is not a panacea. The unnecessary serialization of mass storage requests is at a level that it remains when the OS is paravirtualized. IMHO, the more levels between the application and the actual hardware, the more buffer factors become relevant.. The overhead of the network stack is in the way requests are processed, and needs to be improved.

IMHO, much of this is a remnant of the days when memory resources were far more limited. It is fixable.

Am in the process of publishing a monograph with the analysis, but it has been delayed by tune availability and the pandemic. I have published a paper at an IEEE conference with some of the material and relevant diagrams. The pre-print is at: http://rlgsc.com/r/20220506.html

- Bob Gezelter, http://www.rlgsc.com

On Wednesday, August 30, 2023 at 10:41:28 AM UTC+1, Bob Gezelter wrote:
> On Tuesday, August 29, 2023 at 6:49:51 PM UTC-4, Craig A. Berry wrote:
> > On 8/29/23 3:27 PM, bill wrote:
> > > On 8/29/2023 3:18 PM, Johnny Billquist wrote:
> > >> On 2023-08-29 19:25, Simon Clubley wrote:
> >
> > >>> On a more serious note, I wonder what the maximum rate VMS is capable
> > >>> of emitting data at if it was using the fastest network hardware
> > >>> available.
> > >>
> > >> What a weird question. VMS in itself don't have any limits. It's all
> > >> always just about the hardware.
> > >
> > > Not really. VMS has always been notoriously slow with I/O and I assume
> > > that's what Simon was hinting at.
> > Right, and differently so for different kinds of I/O. See posts from a
> > few years ago by (I think) Eric Johnson on performance testing of the
> > network stack. And I wish I could remember the name of the guy who
> > posted about slow disk I/O even longer ago (Dave something?) including
> > code to do the testing.
> >
> > VSI has canceled two different file system projects, one of which was
> > GFS and one of which was "not Spiralog" by Andy Goldstein (I don't know
> > if it ever had a name but Clair Grant posted here that it inherited some
> > concepts but was not the same thing as Spiralog). Something will have to
> > be done eventually for disk I/O, and while the file system isn't the
> > whole enchilada, it's certainly one big part of it.
> >
> > The network stack improvements described here:
> >
> > http://www.vmsconsultancy.com/download/NL-VMSUpdate-2015/Vienna%20LAN%20Performance%20Improvements.pdf
> >
> > will hopefully be revisited at some point. If they aren't, then VMS
> > will remain slower at network performance than other systems using the
> > same networking hardware. I totally get why the port had to take
> > precedence for a small company, but holding the line is not the same
> > thing as moving forward.
> Craig,
>
> Indeed. There have been a number of projects in the I/O area, few of which have emerged into released form.
>
> There are a number of issues. I dug into them rather deeply while writing my Ph.D. dissertation. OpenVMS has a good collection of them, as do essentially all of the other extant operating systems. IMHO, they can be remediated in many ways.
>
> With disk I/O (more properly referred to as "mass storage" I/O these days), there are unnecessary serializations forced by driver processing. IMHO, they are a remnant of the days when kernel storage was far more constrained, e.g. IBM System/360 under OS/360 or DEC PDP-11 under RSX. Now that kernel memory is far more available, less serial approaches become viable.
>
> Other mass storage issues can often be addressed, at least on sequential files, by adjusting buffering limits. The "as shipped" defaults were set back in the memory-constrained days of the VAX-11/780 and are kept at those settings for back compatibility. Changing them is straightforward, although one has to also change quotas in the UAF correspondingly. I have spoken for over thirty years on that particular performance issue, starting at NASA Marshall in the late 1980s. I have sample programs that have gone from less than 10% CPU utilization on a MicroVAX 3100 to 100%, merely by changing the RMS buffer factors.
>
> The network stack has issues when it comes to high performance transfers. As I recall, several years ago there was an early-adoptor iSCSI kit, later withdrawn for a number of issues. The accompanying documentation included negative comments about performance. Efficiency of the IP stack would allow the use of increasingly popular iSCSI hardware.
>
> At the User API-level, FAST I/O does reduce some overhead, but requires application-level changes. Fast I/O makes sense for high I/O libraries, e.g., Pathworks.
>
> Some of these issues remains relevant when running as a virtualized guest.. Some becomes significantly less relevant when one is paravirtualized as a guest on a virtual machine. However, paravirtualization is not a panacea. The unnecessary serialization of mass storage requests is at a level that it remains when the OS is paravirtualized. IMHO, the more levels between the application and the actual hardware, the more buffer factors become relevant. The overhead of the network stack is in the way requests are processed, and needs to be improved.
>
> IMHO, much of this is a remnant of the days when memory resources were far more limited. It is fixable.
>
> Am in the process of publishing a monograph with the analysis, but it has been delayed by tune availability and the pandemic. I have published a paper at an IEEE conference with some of the material and relevant diagrams. The pre-print is at: http://rlgsc.com/r/20220506.html
>
> - Bob Gezelter, http://www.rlgsc.com
I'm having trouble with this link http://rlgsc.com/r/20220506.html - page not found