On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
> On 10/7/2021 9:34 AM, Arne Vajhøj wrote:
>
>> I suspect that the variable length file output below will
>> surprise a few *nix developers.
>>
>
> Why do you post C code examples that confuse me and give me a headache?
>
>:-)
>
> Then again, Basic code examples might confuse Unix developers ...
>

Some of them might be aware of Basic.

Back in the later MS-DOS days, Microsoft used to ship a Basic
interpreter for free with MS-DOS and (apparently some Windows versions):

https://en.wikipedia.org/wiki/QBasic

I've just discovered there's a version of Microsoft QuickBasic for Linux:

https://en.wikipedia.org/wiki/FreeBASIC

which I did not know about.

Just been reminded that Gorillas.bas was released 30 years ago.

I am now depressed. :-)

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
> On 10/7/2021 8:59 AM, Simon Clubley wrote:
>> On 2021-10-07, Greg Tinkler <tinklerg@gmail.com> wrote:
>>> On Thursday, 7 October 2021 at 11:26:38 pm UTC+11, Simon Clubley wrote:
>>>
>>>> How do you find byte 12,335,456 in a variable length RMS sequential file
>>>> without reading from the start of the file ?
>>>>
>>>> That's why there are restrictions on RMS supported file formats in an
>>>> application in some cases.
>>>
>>> The same way it is done on Unix, calculate the block offset, go get it, and extract the byte. no difference and nothing to do with the underlying format.
>>>
>>
>> You don't know the block offset without scanning the file when it comes
>> to some RMS file formats.
>>
>> IOW, data byte 12,335,456 will not be the same thing as file byte 12,335,456
>> unless you restrict yourself to record formats that do not have embedded
>> record metadata.
>>
>
> I'm guessing Unix files don't have metadata and such. So the comparison
> is not valid.
>

No, Unix doesn't. At Unix filesystem level, files are just a stream of bytes.

The next layer up on Unix is the C RTL. There's nothing like RMS
between the filesystem and the C RTL on Unix.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 10/7/2021 1:53 PM, Simon Clubley wrote:
> On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
>> Then again, Basic code examples might confuse Unix developers ...
>
> Some of them might be aware of Basic.
>
> Back in the later MS-DOS days, Microsoft used to ship a Basic
> interpreter for free with MS-DOS and (apparently some Windows versions):
>
> https://en.wikipedia.org/wiki/QBasic

GW-Basic came with DOS 1-4 and QBasic with DOS 5-6 and early Windows I
believe.

GW-Basic source code is now available at:
https://github.com/microsoft/GW-BASIC

> I've just discovered there's a version of Microsoft QuickBasic for Linux:
>
> https://en.wikipedia.org/wiki/FreeBASIC
>
> which I did not know about.

I would still not expect many Linux people to know Basic.

And besides VMS Basic is somewhat different from MS Basic flavors.

Arne

On 10/7/2021 2:07 PM, Simon Clubley wrote:
> On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
>> I'm guessing Unix files don't have metadata and such. So the comparison
>> is not valid.
>
> No, Unix doesn't. At Unix filesystem level, files are just a stream of bytes.
>
> The next layer up on Unix is the C RTL. There's nothing like RMS
> between the filesystem and the C RTL on Unix.

The Unix file systems does not have meta data about how the
bytes are to be read/interpreted (like VMS: ORG, RFM, RAT,
MRS etc.). They do have some general meta data (owner,
protection, size, timestamp).

Arne

On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
>
> Don't know how far work had progressed on alternate file systems. Might
> or might not help to make RMS "just another capability". But, doing
> what you suggest would go a long way toward making VMS more useful in
> the future.
>
> I've got the suspicion that VMS clusters, while good, create some of the
> problems in attempting to add new capabilities to VMS. Need I mention
> "MOUNT"? Better segregation might help to add new and different
> capabilities. Not sure how easy that might be.
>

VMS clusters at conceptual level are not the problem. They offer
some very nice functionality that only recently is beginning to
appear elsewhere. They were literally a generation ahead of what
was available elsewhere when they were released.

The problem is how VMS was designed in those early days before
modular and layered computing really took off.

The VMS filesystem code, including MOUNT as you say, is a _horrible_
monolithic mass of closely interlinked code without any clear
boundaries between them that allow people (including end users) to
easily plug in new functionality and new filesystems.

The same is true for VMS CLIs BTW. DCL is tightly bound into VMS
in a horrible way it should not be. On Linux, both the command
shell and filesystem architectures are vastly cleaner and more
modular than they are on VMS.

However, if VMS had been designed in a later era, there would be
absolutely nothing stopping VMS having a cleaner internal architecture
_and_ also having world-leading cluster capabilities that are only
now just being equalled elsewhere.

IOW, it's not clustering that's the problem - it's the fact that
VMS wasn't implemented 5 to 10 years later than it was.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2021-10-07 17:16:03 +0000, Dave Froble said:

> On 10/7/2021 11:34 AM, Stephen Hoffman wrote:
>> On 2021-10-07 01:25:57 +0000, Greg Tinkler said:
>>
>>> My point is SSIO seems to be focused on just PostgreSQL, whereas an RMS
>>> solution is much much easier to program, uses well tested code, and is
>>> already cluster ready putting the team ahead of the game and not
>>> building issues for the future.
>>
>> ...
>>
>> Fix the existing RMS data corruption in 32-bit RMS and/or in the C
>> library, and get PostgreSQL available on OpenVMS soonest. I expect this
>> is the priority for VSI.
>
> Most likely.
>
>> Everything else is aspirational.
>>
>> Integrate stream file access support at the XQP and allow C and C++ and
>> other non-punched-card-style app designs and stream- and OO-focused
>> languages to optionally bypass RMS entirely.
>
> I don't use C, so I don't know much about it. But isn't this
> capability already available?

The C standard functions—the equivalent of the BASIC calls OPEN, READ,
WRITE, et al—are via RMS. There's no knob to tell C "don't do that".

The C default sequential file format creation format on OpenVMS is RMS
VFC, which has been a perpetual source of confusion and consternation
for users new to C on OpenVMS.

> Even RMS has the BLOCK I/O capability, at least from Basic.

C doesn't do sector I/O within the standard library, though the native
platform calls are easily available.

> As far as I know, QIO doesn't know a thing about RMS. Well, the
> directory structure does know RMS, and to an extent is RMS.

$qio (and $io_perform) offer sector access through RMS (virtual),
record access through RMS (virtual), or access to device through the
file system (IO$_ACPCONTROL XQP), or direct access to the device driver
and device (logical and physical I/O).

The VIRT_IO virtual I/O paths through RMS and through the XQP are
cluster-aware, while the LOG_IO logical and PHY_IO physical I/O paths
are not.

RMS provides record locking for cluster coordination, while the XQP
provides coordination for the on-disk file system.

>> Better integrate and document the existing range-locking support
>> available within DLM.
>
> Yes, for sure. And if needed, make it much better.
>
>> And in aggregate, stop trying to make the current 32-bit RMS NoSQL
>> database more complex than it already is, and re-architect such that
>> 32-bit RMS NoSQL database becomes just another available database, and
>> preferably while providing room for 64-bit RMS rather than trying
>> another OpenVMS Alpha V7.0-style 32-/64-bit or FAB/RAB/RAB64/NAM/NAML
>> design, and make 32- or (hypothetical) 64-bit RMS not the sole
>> persistent-storage "funnel" for structured file access for apps running
>> on OpenVMS, short of those few using XQP or LOG_IO or PHY_IO. Existing
>> RMS apps are already headed for "fun" as part of the upcoming 64-bit
>> LBN work for VSI and for apps, and a whole lot of those apps just won't
>> make it past messes similar to apps still tied to ODS-2 naming. I'd
>> wager that most existing apps don't yet fully support ODS-5 naming,
>> UTF-8 and all, too. Similar app messes with latent 32-bit RMS
>> dependencies.
>
> Oh, no, Steve. That is much too logical and reasonable. Can't have
> that. We must insure that things stay totally screwed up.

I'd prefer an approach where there's some opportunity to ease new work
and new APIs into production, and to also retire overtly-busted APIs.

Oracle Rdb was really good at that migration and for as far as that
went, but most other apps and OpenVMS itself have not managed to copy
that. Not successfully.

> Don't know how far work had progressed on alternate file systems.
> Might or might not help to make RMS "just another capability". But,
> doing what you suggest would go a long way toward making VMS more
> useful in the future.
>
> I've got the suspicion that VMS clusters, while good, create some of
> the problems in attempting to add new capabilities to VMS. Need I
> mention "MOUNT"? Better segregation might help to add new and
> different capabilities. Not sure how easy that might be.

Oracle Rdb and some other databases have cluster access locking,
whether using DLM or database-level locking.

Other databases can be single-host.

The SQLite port to OpenVMS supports DLM and clustering.

PostgreSQL has been adding replication and clustering:
https://www.postgresql.org/docs/9.5/different-replication-solutions.html

Whether an OpenVMS port of PostgreSQL can incorporate DLM calls is
fodder for future discussions, once the SSIO prerequisite becomes
available and a hypothetical future PostgreSQL port becomes stable. A
stable PostgreSQL will interest some folks, with adoptions depending on
both intrinsic interest and, um, potential extrinsic factors not yet in
evidence.

And no, you need not mention MOUNT, having necessarily (re)written what
MOUNT provides on several occasions.

--
Pure Personal Opinion | HoffmanLabs LLC

On 2021-10-07 18:28:04 +0000, Simon Clubley said:

> IOW, it's not clustering that's the problem - it's the fact that VMS
> wasn't implemented 5 to 10 years later than it was.

....Or that OpenVMS and its apps weren't later migrated to DEC MICA.
Which is kinda-sorta what you're referring to.

--
Pure Personal Opinion | HoffmanLabs LLC

On 10/7/2021 2:30 PM, Stephen Hoffman wrote:
> PostgreSQL has been adding replication and clustering:
> https://www.postgresql.org/docs/9.5/different-replication-solutions.html
>
> Whether an OpenVMS port of PostgreSQL can incorporate DLM calls is
> fodder for future discussions, once the SSIO prerequisite becomes
> available and a hypothetical future PostgreSQL port becomes stable. A
> stable PostgreSQL will interest some folks, with adoptions depending on
> both intrinsic interest and, um, potential extrinsic factors not yet in
> evidence.

PostgreSQL clusters are active/passive.

All updates and typical all reads goes to the active node
and updates get replicated from the active node to the passive nodes.

I believe it is possible to have the passive nodes support
reading.

But with only the active node taking updates then there
is no need for DLM.

(VMS people may not even call such a config a cluster, but ...)

Arne

On 2021-10-07 18:36:28 +0000, Arne Vajhj said:

> PostgreSQL clusters are active/passive. ...

For folks interested in this general topic area with PostgreSQL around
failover and replication, please see the PostgreSQL documentation for
details.

Here's an updated link from what I'd posted earlier:
https://www.postgresql.org/docs/14/different-replication-solutions.html

If there's interest in adding what OpenVMS calls clustering within any
hypothetical future PostgreSQL port, use of the DLM will undoubtedly be
considered.

nb: PostgreSQL uses the term "cluster" for something entirely different
and unrelated to OpenVMS clustering.

--
Pure Personal Opinion | HoffmanLabs LLC

On 10/7/2021 1:01 PM, Stephen Hoffman wrote:
> On 2021-10-07 16:18:28 +0000, Dave Froble said:
>
>> A while back we were discussing doing away with I/O to buffers, and
>> accessing the data in place. Slower access perhaps, but doing away
>> with the reading and writing to/from buffers. Haven't heard much
>> about that lately. I don't get out much.
>
> Ayup. Nonvolatile byte-addressable storage hardware is available now,
> and is in use in various applications.
>
> Compatible memory hardware will be rather more available for OpenVMS
> x86-64, for folks interested in investigating this for their apps.
>
> Carving out a hunk of persistent storage will be interesting topic for
> app developers on OpenVMS, though I can think of a couple of ways to try.
>
> Here's an HPE overview from a few years ago on the topic:
> https://www.pdl.cmu.edu/SDI/2016/slides/keeton-2016-10-19-memory-driven-computing.pdf
>
>
> I see some B-Tree work for this area in a newer paper, and a number of
> other discussions.
>
>> Such type of activity would really benefit from having the capability
>> of locking just the required data, and, would need the capability of
>> reading and writing just the required data.
>
> Locking access to the contents of a global section, or locking access to
> hardware-backed storage for external devices, is the same issue.
>
> Whether DLM overhead is too high for that to be workable is another
> discussion that the app developers will want to ponder.
>
>> I'm aware of how useful something like SSIO would be. I'm just
>> appalled by the design and implementation. As mentioned, it seems
>> aimed at just a few current uses, and totally ignores how useful it
>> would be for many more future uses. This is rather consistent with
>> the long time apathy with which VMS has been treated. It's more a
>> patch than an enhancement. This is what I lament.
>
> Alas, there's no other outcome when upward-compatibility is an
> overarching goal for the platform.

Now I''m just a dumb polock, wandered down out of the woods. But I just
don't see where upward compatibility has anything to do with
enhancements to the DLM. If existing calls continue to work as before,
and only when an optional extra parameter would enable new capabilities,
then upward compatibility just cannot be an issue. At least for this.

The optional parameter might be a "lock type", and if not present,
existing logic would be used, and if present, new code could be executed
to process the new lock type. Stuff a couple of quadwords into the
resource name for the numeric range. It would add one new piece of data
to the DLM data structure(s).

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/7/2021 1:25 PM, Arne Vajhøj wrote:
> On 10/7/2021 1:16 PM, Dave Froble wrote:
>> On 10/7/2021 11:34 AM, Stephen Hoffman wrote:
>>> Integrate stream file access support at the XQP and allow C and C++ and
>>> other non-punched-card-style app designs and stream- and OO-focused
>>> languages to optionally bypass RMS entirely.
>>
>> I don't use C, so I don't know much about it. But isn't this
>> capability already available? Even RMS has the BLOCK I/O capability,
>> at least from Basic.
>
> C/C++ and most newer languages have a "stream view" of files while
> RMS has a "record view" of files.
>
> If they used different file systems everything would be fine.
>
> If all text files are STMLF then it works and the "stream view"
> and the "record view" produces consistent results.
>
> But trying to mix on variable length or VFC files becomes
> a minefield.
>
> I know you don't like C, but try look at the example I posted.
> Some of the outputs are very weird.
>
> Arne
>

I have some understanding of RMS files. I've been known to do recovery
work on corrupted RMS files. Have to have some knowledge of RMS to do
that. But, it sure isn't any fun ...

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/7/2021 2:28 PM, Simon Clubley wrote:
> On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
>>
>> Don't know how far work had progressed on alternate file systems. Might
>> or might not help to make RMS "just another capability". But, doing
>> what you suggest would go a long way toward making VMS more useful in
>> the future.
>>
>> I've got the suspicion that VMS clusters, while good, create some of the
>> problems in attempting to add new capabilities to VMS. Need I mention
>> "MOUNT"? Better segregation might help to add new and different
>> capabilities. Not sure how easy that might be.
>>
>
> VMS clusters at conceptual level are not the problem. They offer
> some very nice functionality that only recently is beginning to
> appear elsewhere. They were literally a generation ahead of what
> was available elsewhere when they were released.
>
> The problem is how VMS was designed in those early days before
> modular and layered computing really took off.
>
> The VMS filesystem code, including MOUNT as you say, is a _horrible_
> monolithic mass of closely interlinked code without any clear
> boundaries between them that allow people (including end users) to
> easily plug in new functionality and new filesystems.
>
> The same is true for VMS CLIs BTW. DCL is tightly bound into VMS
> in a horrible way it should not be. On Linux, both the command
> shell and filesystem architectures are vastly cleaner and more
> modular than they are on VMS.
>
> However, if VMS had been designed in a later era, there would be
> absolutely nothing stopping VMS having a cleaner internal architecture
> _and_ also having world-leading cluster capabilities that are only
> now just being equalled elsewhere.
>
> IOW, it's not clustering that's the problem - it's the fact that
> VMS wasn't implemented 5 to 10 years later than it was.
>
> Simon.
>

You may have noticed that I didn't blame VMS clusters for the problem.
Rather how some things are so rigid, and more so because of support for
some things that involve clusters. Makes new stuff sometimes much
harder, as you mentioned.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/7/21 1:30 PM, Stephen Hoffman wrote:
> On 2021-10-07 17:16:03 +0000, Dave Froble said:

>> I don't use C, so I don't know much about it.  But isn't this
>> capability already available?
>
> The C standard functions—the equivalent of the BASIC calls OPEN, READ,
> WRITE, et al—are via RMS. There's no knob to tell C "don't do that".

You're pretending that you don't know about the foo="bar" options on the
CRTL open/fopen/creat calls. Yes, it's all via RMS, but you can tell it
to do or not do certain things. And the feature logicals, of course, but
it might be dinnertime in your time zone and I wouldn't want to give you
indigestion :-).

But from BASIC, yes, I think you have to write wrappers around the CRTL
functions and then call them from BASIC, or at least that's what I did
the one time I had to write stream files from BASIC.

On 10/7/2021 7:15 PM, Craig A. Berry wrote:
>
> On 10/7/21 1:30 PM, Stephen Hoffman wrote:
>> On 2021-10-07 17:16:03 +0000, Dave Froble said:
>
>
>>> I don't use C, so I don't know much about it. But isn't this
>>> capability already available?
>>
>> The C standard functions—the equivalent of the BASIC calls OPEN, READ,
>> WRITE, et al—are via RMS. There's no knob to tell C "don't do that".
>
> You're pretending that you don't know about the foo="bar" options on the
> CRTL open/fopen/creat calls. Yes, it's all via RMS, but you can tell it
> to do or not do certain things. And the feature logicals, of course, but
> it might be dinnertime in your time zone and I wouldn't want to give you
> indigestion :-).
>
> But from BASIC, yes, I think you have to write wrappers around the CRTL
> functions and then call them from BASIC, or at least that's what I did
> the one time I had to write stream files from BASIC.

I'd ask, why not call the RMS routines?

No, messing with FABs and RABs and such is not one of my favorite things
to do. But it sure is doable.

Now perhaps the naming doesn't mean the same thing, but:

OPEN
Syntax
[ FOR INPUT ]
OPEN file-spec1 [ FOR OUTPUT ] AS [ FILE ] chnl-exp1 [,
open-clause ]...

open-clause: { { VIRTUAL }
}
{ { UNDEFINED }
}
{ [ ORGANIZATION ] { INDEXED } [ STREAM ]
}
{ { SEQUENTIAL } [ VARIABLE ]
}
{ { RELATIVE } [ FIXED ]
}

Basic help seems to imply that stream files can be created ...

Perhaps I should actually try it, much as it entails work ...

Itanic> t zz.bas
1 Open "ZZ.ZZ" For Output as File #1%, &
Organization Sequential Stream, &
Recordsize 32767

Print #1%, Num1$(Z%) For Z% = 1% to 5%

Close #1%

End
Itanic> t zz.zz
1 2
3 4
5 Itanic> dir/full zz.zz

Directory DKB0:[DFE]

ZZ.ZZ;1 File ID: (6678,7,0)
Size: 1/16 Owner: [DFE]
Created: 7-OCT-2021 20:22:50.29
Modified: 7-OCT-2021 20:22:50.36 (1)
Expires: <None specified>
Backup: <No backup recorded>
Effective: <None specified>
Recording: <None specified>
Accessed: 7-OCT-2021 20:22:50.29
Attr Mod: 7-OCT-2021 20:22:50.36
Data Mod: 7-OCT-2021 20:22:50.29
Linkcount: 1
File organization: Sequential
Shelved state: Online
Caching attribute: Writethrough
File attributes: Allocation: 16, Extend: 0, Global buffer count: 0,
No version limit
Record format: Stream, maximum 32767 bytes, longest 1 byte
Record attributes: Carriage return carriage control
RMS attributes: None
Journaling enabled: None
File protection: System:RWED, Owner:RWED, Group:RE, World:
Access Cntrl List: None
Client attributes: None

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On Friday, October 8, 2021 at 4:37:06 AM UTC+13, osuv...@gmail.com wrote:
> Whatever happened to Compound Document Architecture (CDA)? It always
> struck me as an effort (now abandoned) toward an object oriented file structure.

Then there was Bento, which Apple was fond of for a while (back in the days of the OpenDoc-versus-OLE2 war).

Seems like nobody cares about live embedding and compound documents now. Probably turned out to be too complex for most users to handle.

One interesting modern trend is the use of ZIP archives as a document metaformat. For example, an ODF file (ISO 26300) is essentially a ZIP archive. There is this interesting convention that the first element of the archive shall be named “mimetype”, and its content shall be uncompressed. This allows file sniffers to pick up the MIME type info at a fixed offset near the start of the file.

On 10/7/2021 8:55 PM, Lawrence D’Oliveiro wrote:
> One interesting modern trend is the use of ZIP archives as a document
> metaformat. For example, an ODF file (ISO 26300) is essentially a ZIP
> archive.

ODF, OOXML, a bunch of Java stuff (jar, war, rar, ear) etc..

Arne

On Friday, October 8, 2021 at 4:34:50 AM UTC+13, Stephen Hoffman wrote:
> Integrate stream file access support at the XQP and allow C and C++ and
> other non-punched-card-style app designs and stream- and OO-focused
> languages to optionally bypass RMS entirely.

I assume you mean “bypass RMS for non-block-level I/O”, since it was always possible for nonprivileged code to do direct ACP/XQP calls like IO$_ACCESS, READ/WRITEVBLK and friends.

(You soon appreciate how much work $PARSE is doing for you...)

On Friday, October 8, 2021 at 6:19:02 AM UTC+13, Dave Froble wrote:
> As far as I know, QIO doesn't know a thing about RMS. Well, the
> directory structure does know RMS, and to an extent is RMS.

Last I checked*, on VMS, directories were just files, and there was no protection against processes with write access screwing up their contents. For some reason that was not considered to be a vital part of filesystem integrity.

RMS implements the full file/directory name syntax, but the management of name entries inside directories is an ACP/XQP function.

*decades ago, admittedly

On Friday, October 8, 2021 at 7:07:02 AM UTC+13, Simon Clubley wrote:
>
> On 2021-10-07, Dave Froble <da...@tsoft-inc.com> wrote:
>>
>> I'm guessing Unix files don't have metadata and such.
>>
> No, Unix doesn't. At Unix filesystem level, files are just a stream of bytes.

Some Linux filesystems have the concept of “extended attributes” <https://manpages.debian.org/buster/manpages/xattr.7.en.html>. Some are reserved for security purposes, others are user-defined.

On Friday, October 8, 2021 at 7:28:07 AM UTC+13, Simon Clubley wrote:
> The same is true for VMS CLIs BTW. DCL is tightly bound into VMS
> in a horrible way it should not be. On Linux, both the command
> shell and filesystem architectures are vastly cleaner and more
> modular than they are on VMS.

Fundamental difference in mindset: process creation in VMS is expensive and to be avoided if possible, while on *nix systems it’s something you do as naturally as breathing.

And of course the VMS mindset continued over into Windows NT...

> However, if VMS had been designed in a later era ...

Note that Unix predates VMS. Folks at DEC would have been aware of it right from the early days, since it was born on DEC hardware.

On 2021-10-07 21:03:53 +0000, Dave Froble said:

> On 10/7/2021 1:01 PM, Stephen Hoffman wrote:
>> On 2021-10-07 16:18:28 +0000, Dave Froble said:
>>
>>> I'm aware of how useful something like SSIO would be. I'm just
>>> appalled by the design and implementation. As mentioned, it seems
>>> aimed at just a few current uses, and totally ignores how useful it
>>> would be for many more future uses. This is rather consistent with the
>>> long time apathy with which VMS has been treated. It's more a patch
>>> than an enhancement. This is what I lament.
>>
>> Alas, there's no other outcome when upward-compatibility is an
>> overarching goal for the platform.
>
> Now I''m just a dumb polock, wandered down out of the woods. But I
> just don't see where upward compatibility has anything to do with
> enhancements to the DLM. If existing calls continue to work as before,
> and only when an optional extra parameter would enable new
> capabilities, then upward compatibility just cannot be an issue. At
> least for this.

I was building on the "long term apathy" and "more patch than
enhancement" comments, with the increasing difficulties even making
comparatively minor or isolated changes and updates.

Larger changes can be Really Difficult with ~40 years of accumunated
dependencies around, assuming the developers and schedule and funding
are all available. (q.v. Hyrum's Law.)

There are sections of OpenVMS that would best be ripped out and
replaced, or refactored, or re-architected, but that can't happen or
can't easily happen while staying compatible with existing apps.

DLM itself needs better abstractions as some of the more common tasks
are just absurdly involved to program within the existing API. Tasks
such as selecting a primary app server for a host or a cluster, for
instance. This is less of an issue for experienced OpenVMS programmers
and for those with access to examples (cost and schedule and budget and
ongoing support discussions aside), but this sequence is not something
at all obvious to less-experienced developers. And even with
experienced developers, mistakes still happen. And within a wider view,
this DLM primary support is building local process and job control
support, which is an omission I've commended on before.

--
Pure Personal Opinion | HoffmanLabs LLC

On 2021-10-07 23:15:19 +0000, Craig A. Berry said:

> On 10/7/21 1:30 PM, Stephen Hoffman wrote:
>> On 2021-10-07 17:16:03 +0000, Dave Froble said:
>
>
>>> I don't use C, so I don't know much about it.  But isn't this
>>> capability already available?
>>
>> The C standard functions—the equivalent of the BASIC calls OPEN, READ,
>> WRITE, et al—are via RMS. There's no knob to tell C "don't do that".
>
> You're pretending that you don't know about the foo="bar" options on
> the CRTL open/fopen/creat calls. Yes, it's all via RMS, but you can
> tell it to do or not do certain things. And the feature logicals, of
> course, but it might be dinnertime in your time zone and I wouldn't
> want to give you indigestion :-).

I need to pretend harder, or to forget harder, then. I'm sufficiently
familiar with those C options and with the acc routines and with my
always-favorite C feature logical names and the lib$initialize psect
fun to often prefer use of $qio or $io_perform when suppression of RMS
"helpfulness" is sought, yes. Even with all those knobs, "well, there's
egg and bacon; egg sausage and bacon; egg and RMS; egg bacon and RMS;
egg bacon sausage and RMS; RMS bacon sausage and RMS; RMS egg RMS bacon
and RMS; RMS sausage RMS bacon RMS..." "Do you have anything without
RMS (getting "helpful")?" https://vimeo.com/329001211

Too many C design quirks awaiting the unwary or the uninitiated here,
too; where you have to add options to remove platform-specific oddities
(see above), where basename works for Unix specs but not for OpenVMS
specs, how select only works for IP, where VFC is the default
sequential file creation format, the need for the moving target that is
lib$initialize, and the decc$to_vms and decc$from_vms calling
conventions. And suchlike.

This is part of why I'd prefer to see a new C standard library within
the port, and to relegate the existing standard library for use by
existing apps.

> But from BASIC, yes, I think you have to write wrappers around the CRTL
> functions and then call them from BASIC, or at least that's what I did
> the one time I had to write stream files from BASIC.

I'd expect there's BASIC code around that doesn't handle TCP streams
all that well, either. Similar issues. Punched cards are really
entrenched all through OpenVMS.

--
Pure Personal Opinion | HoffmanLabs LLC

On 2021-10-08 01:19:33 +0000, Lawrence D’Oliveiro said:

> On Friday, October 8, 2021 at 7:28:07 AM UTC+13, Simon Clubley wrote:
>> The same is true for VMS CLIs BTW. DCL is tightly bound into VMS in a
>> horrible way it should not be. On Linux, both the command shell and
>> filesystem architectures are vastly cleaner and more modular than they
>> are on VMS.
> Fundamental difference in mindset: process creation in VMS is expensive
> and to be avoided if possible, while on *nix systems it’s something you
> do as naturally as breathing.

VAX-era wisdom, and which is clung on. Creating new processes on
OpenVMS never got as light as Unix, but the overhead has become
negligible on modern systems for all but industrial-scale
creation-deletion.

Having looked at this back in the VAX era, the slow process creations
our apps were incurring were arising from inefficiencies within the DCL
spawn-related processing, and not from within the OpenVMS process
creation overhead.

Once that was identified and the obvious work-around implemented,
spawns were pretty speedy even VAX-era.

To Simon's comment, how DCL gets mapped into process address space is
just ugly, too. And hard to debug.

--
Pure Personal Opinion | HoffmanLabs LLC

On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
>
> Now I''m just a dumb polock, wandered down out of the woods. But I just
> don't see where upward compatibility has anything to do with
> enhancements to the DLM. If existing calls continue to work as before,
> and only when an optional extra parameter would enable new capabilities,
> then upward compatibility just cannot be an issue. At least for this.
>

Is there a version number on the current inter-node DLM messages ?

If not, how can you change the DLM message structure in a compatible way ?

If yes, what happens when an older node sees a later format DLM message ?
You would at least need a compatibility kit to be installed on the older
nodes.

> The optional parameter might be a "lock type", and if not present,
> existing logic would be used, and if present, new code could be executed
> to process the new lock type. Stuff a couple of quadwords into the
> resource name for the numeric range. It would add one new piece of data
> to the DLM data structure(s).
>

What about the DLM messages sent between nodes ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2021-10-07, Lawrence D?Oliveiro <lawrencedo99@gmail.com> wrote:
> On Friday, October 8, 2021 at 7:07:02 AM UTC+13, Simon Clubley wrote:
>>
>> On 2021-10-07, Dave Froble <da...@tsoft-inc.com> wrote:
>>>
>>> I'm guessing Unix files don't have metadata and such.
>>>
>> No, Unix doesn't. At Unix filesystem level, files are just a stream of bytes.
>
> Some Linux filesystems have the concept of ?extended attributes? <https://manpages.debian.org/buster/manpages/xattr.7.en.html>. Some are reserved for security purposes, others are user-defined.

That's true and I do use them. However, at filesystem level, the
file data itself is just a stream of bytes without any embedded metadata
(unlike on VMS).

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.