On 10/8/2021 10:51 AM, Stephen Hoffman wrote:
> On 2021-10-07 21:03:53 +0000, Dave Froble said:
>
>> On 10/7/2021 1:01 PM, Stephen Hoffman wrote:
>>> On 2021-10-07 16:18:28 +0000, Dave Froble said:
>>>
>>>> I'm aware of how useful something like SSIO would be. I'm just
>>>> appalled by the design and implementation. As mentioned, it seems
>>>> aimed at just a few current uses, and totally ignores how useful it
>>>> would be for many more future uses. This is rather consistent with
>>>> the long time apathy with which VMS has been treated. It's more a
>>>> patch than an enhancement. This is what I lament.
>>>
>>> Alas, there's no other outcome when upward-compatibility is an
>>> overarching goal for the platform.
>>
>> Now I''m just a dumb polock, wandered down out of the woods. But I
>> just don't see where upward compatibility has anything to do with
>> enhancements to the DLM. If existing calls continue to work as
>> before, and only when an optional extra parameter would enable new
>> capabilities, then upward compatibility just cannot be an issue. At
>> least for this.
>
> I was building on the "long term apathy" and "more patch than
> enhancement" comments, with the increasing difficulties even making
> comparatively minor or isolated changes and updates.
>
> Larger changes can be Really Difficult with ~40 years of accumunated
> dependencies around, assuming the developers and schedule and funding
> are all available. (q.v. Hyrum's Law.)

Hyrum's Law and such points to the need of good software architecture.
(I always have to use the spell checker on that word.)

If intelligent and structured use of something like VMS if followed,
enhancements should not be much of an issue. It is when people do
things they really should not the problems arise. Compatibility with
well designed tools should not be an issue. Going off on one's own, and
making assumptions about things, which are not guaranteed to remain
as-is is where such problems occur, for the most part.

As a simple example:

If Stat% and 1%

vs

If Stat% and SS$_NORMAL

That causes a problem, if the VMS developers decide that "1" is no
longer what it used to be. The problem is not compatibility, the
problem is not using the approved constant.

Now while breaking customers code can be bad for business, the dumb
polock can say "fuck 'em, enhance the product and break their erroneous
code".

:-)

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/8/2021 2:19 PM, Simon Clubley wrote:
> On 2021-10-07, Dave Froble <davef@tsoft-inc.com> wrote:
>>
>> Now I''m just a dumb polock, wandered down out of the woods. But I just
>> don't see where upward compatibility has anything to do with
>> enhancements to the DLM. If existing calls continue to work as before,
>> and only when an optional extra parameter would enable new capabilities,
>> then upward compatibility just cannot be an issue. At least for this.
>>
>
> Is there a version number on the current inter-node DLM messages ?

Good question, and if not, perhaps such could be implemented. However,
what I envision should not affect usage of the existing resource name lock.

> If not, how can you change the DLM message structure in a compatible way ?
>
> If yes, what happens when an older node sees a later format DLM message ?
> You would at least need a compatibility kit to be installed on the older
> nodes.

Perhaps.

>> The optional parameter might be a "lock type", and if not present,
>> existing logic would be used, and if present, new code could be executed
>> to process the new lock type. Stuff a couple of quadwords into the
>> resource name for the numeric range. It would add one new piece of data
>> to the DLM data structure(s).
>>
>
> What about the DLM messages sent between nodes ?
>
> Simon.
>

First, re-read what I posted. Specifically "if not present, existing
logic would be used".

Not sure what you're calling "DLM message".

The only data item I'd see added to the lock database would be the "lock
type", and that could be done in a manner such that it does not affect
lock database information that does not have the new structure definitions.

Perhaps it could be arranged that when using the new data structure(s),
that it would be mandatory to update all nodes in a cluster. Perhaps
some type of version would disallow usage of dissimilar versions.

Note that any node or cluster that wished to use numeric range locking
would have to have the enhancement installed. If not using it, then
nothing changes.

This could be done as a VMS DLM enhancement. I'm rather sure of that.
Whether the desire to do so might be a different issue.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 2021-10-08, Dave Froble <davef@tsoft-inc.com> wrote:
>
> Not sure what you're calling "DLM message".
>

DLM-related cluster traffic.

Anything you propose not only has to be compatible at API level,
but also in physical DLM messages on the wire.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On Saturday, October 9, 2021 at 4:37:26 AM UTC+13, Stephen Hoffman wrote:
> Having looked at this back in the VAX era, the slow process creations
> our apps were incurring were arising from inefficiencies within the DCL
> spawn-related processing, and not from within the OpenVMS process
> creation overhead.
>
> Once that was identified and the obvious work-around implemented,
> spawns were pretty speedy even VAX-era.
>
> To Simon's comment, how DCL gets mapped into process address space is
> just ugly, too. And hard to debug.

But the whole reason why DCL maps into a process in this way, so that user-mode code can be repeatedly loaded, run and then wiped from the same process, was precisely to avoid multiple process creations. Now you are saying that the DCL mechanism itself contributes to the overhead of process creations!

But “spawn” is still not the same as “fork”.. Sure, in *nix, the “fork” followed by “exec” idiom is common, but lots of forks are done without an accompanying exec (I’ve done a few myself). In the early days of Unix, the “vfork” hack was invented to speed things up in the fork+exec case, but this was later discovered to be unnecessary: not (so much) because hardware had become faster, but it was recognized that the bottleneck of giving the child process its own copy of non-shared writable memory could be avoided/postponed by just copying the relevant page table entries and setting a “copy-on-write” flag on them.

What do you know, vfork(2) was actually specified in POSIX, and Linux still supports it <https://manpages.debian.org/bullseye/manpages-dev/vfork.2.en.html>.

<snip>
>>> The optional parameter might be a "lock type", and if not present,
>>> existing logic would be used, and if present, new code could be executed
>>> to process the new lock type. Stuff a couple of quadwords into the
>>> resource name for the numeric range. It would add one new piece of data
>>> to the DLM data structure(s).

What would be useful is a name space for the lock e.g. RMS ...

Well there is the group id from UIC, and there is $set_resource_domain(), both can be useful but not a solution.

Having a name space that can be local machine or cluster wide could be very useful for some applications. But that is a much longer term idea.

At present the resource name is limited to 31 char, ok in the 32 era but in 64 bit era and looking at GFS2 for peta byte moving into the exabyte and possibly the Zettabyta range, if VMS is to survive the next 40 years it needs to prepare.

First lets move onto X86_64. Yes it would be good to have easier building of open source code, and the main issues as I understand it are

file IO, moving to RMS will fix most if not all of that
fork
for fork/exec - spawn is fine, no modern systems a bit of CPU...
for file access more of problem, but not often used
directory and filenaming
some work is in place for this

So the main issue is file IO, so change CRTL to use RMS.

gt

On 10/8/2021 4:36 PM, Simon Clubley wrote:
> On 2021-10-08, Dave Froble <davef@tsoft-inc.com> wrote:
>>
>> Not sure what you're calling "DLM message".
>>
>
> DLM-related cluster traffic.
>
> Anything you propose not only has to be compatible at API level,
> but also in physical DLM messages on the wire.
>
> Simon.
>

If the single new piece of data is not used, then nothing changes.

If it is in use, then the nodes in question would already have the
enhancement installed.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/8/2021 8:04 PM, Greg Tinkler wrote:
> <snip>
>>>> The optional parameter might be a "lock type", and if not present,
>>>> existing logic would be used, and if present, new code could be executed
>>>> to process the new lock type. Stuff a couple of quadwords into the
>>>> resource name for the numeric range. It would add one new piece of data
>>>> to the DLM data structure(s).
>
> What would be useful is a name space for the lock e.g. RMS ...
>
> Well there is the group id from UIC, and there is $set_resource_domain(), both can be useful but not a solution.
>
> Having a name space that can be local machine or cluster wide could be very useful for some applications. But that is a much longer term idea.
>
> At present the resource name is limited to 31 char, ok in the 32 era but in 64 bit era and looking at GFS2 for peta byte moving into the exabyte and possibly the Zettabyta range, if VMS is to survive the next 40 years it needs to prepare.
>
> First lets move onto X86_64. Yes it would be good to have easier building of open source code, and the main issues as I understand it are
>
> file IO, moving to RMS will fix most if not all of that
> fork
> for fork/exec - spawn is fine, no modern systems a bit of CPU...
> for file access more of problem, but not often used
> directory and filenaming
> some work is in place for this
>
> So the main issue is file IO, so change CRTL to use RMS.
>
> gt
>

Let me ask this as a question, because I really don't know.

Doesn't C already use RMS for file I/O ?

It has been my impression that all the VMS languages use RMS for file
I/O. But I don't get out much.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/8/2021 9:09 PM, Dave Froble wrote:
> On 10/8/2021 8:04 PM, Greg Tinkler wrote:
>> So the main issue is file IO, so change CRTL to use RMS.
>
> Let me ask this as a question, because I really don't know.
>
> Doesn't C already use RMS for file I/O ?
>
> It has been my impression that all the VMS languages use RMS for file
> I/O.  But I don't get out much.

It has previously been claimed that:

other languages use a very thin layer on top of SYS$GET and SYS$PUT

C use a much thicker layer on top of SYS$READ and SYS$WRITE

I don't know if it is true or not.

Arne

> So the main issue is file IO, so change CRTL to use RMS.
>
> gt

CRTL uses RMS for file I/O. But there is an issue with concurrent access of multiple processes to the same file in stream mode. And we had a choice - 1) rewrite half of Postgres by inserting file locking; 2) add a new SSIO (Shared Stream IO) service to VMS.

On Saturday, 9 October 2021 at 6:00:48 pm UTC+11, Vitaly Pustovetov wrote:
> > So the main issue is file IO, so change CRTL to use RMS.
> >
> > gt
> CRTL uses RMS for file I/O. But there is an issue with concurrent access of multiple processes to the same file in stream mode.
> And we had a choice - 1) rewrite half of Postgres by inserting file locking; 2) add a new SSIO (Shared Stream IO) service to VMS.

Sort of.

RMS worked and has been working of 40+years and does not have these concurrent access issues! NB there is no such thing at an OS level of stream anything, every thing is clumps of data being buffered is some way, the API that accesses that data from the higher levels may be stream based. In this case it is CRTL's role to translate the clumps of data into/from stream API.

So the other choice, 3), fix CRTL to use RMS correctly, and the problems will go away. Engineering effort would not be great. I don't have access to the code base, but assuming that stdio uses unixio then it is fixing 5 routines. This would also allow all the other ports to work with minimal changes in the file access area. If you what to know more contact me directly.

Longer term the SSIO may be useful to RMS, which is where it belongs.

Sorry if the above is a little blunt, I appreciate the efforts people have put in over the years, but some of us have using and coding VMS for a very long time, and I really want VMS to be successful and easy to port to. This has been a good opportunity for me to look more into CRTL and RMS, and see the problems that have been there for decades.

locking is an interesting area, I still feel the current DLM is more than capable of doing the 'lock a byte range' in a way that can be used with the current RMS locking. Longer term DLM needs some changes but they are about sizes of resource names, scoping of resource names, ability to scan for children resources by name.

gt

On Saturday, October 9, 2021 at 6:19:44 AM UTC-4, tink...@gmail.com wrote
> So the other choice, 3), fix CRTL to use RMS correctly, and the problems will go away. Engineering effort would not be great. I don't have access to the code base, but assuming that stdio uses unixio then it is fixing 5 routines. This would also allow all the other ports to work with minimal changes in the file access area. If you what to know more contact me directly.
>
> Longer term the SSIO may be useful to RMS, which is where it belongs.

I don't think the CRTL can do it with just the capabilities RMS gives it currently(or they would have fixed it already). Maintaining coherence of where end-of-file is for multiple writers is a difficult problem.

The crtl does not layer stdio file access on top of unixio primitives.

> RMS worked and has been working of 40+years and does not have these concurrent access issues!
No, you are wrong. RMS works fine with record-based files, but not streams. You can write a program even in MACRO, you will still have the same issues. This is a documented feature of RMS.

On 2021-10-09 10:19:42 +0000, Greg Tinkler said:

> On Saturday, 9 October 2021 at 6:00:48 pm UTC+11, Vitaly Pustovetov wrote:
>>> So the main issue is file IO, so change CRTL to use RMS.> >> > gt
>> CRTL uses RMS for file I/O. But there is an issue with concurrent
>> access of multiple processes to the same file in stream mode. And we
>> had a choice - 1) rewrite half of Postgres by inserting file locking;
>> 2) add a new SSIO (Shared Stream IO) service to VMS.
>
> Sort of.
> RMS worked and has been working of 40+years and does not have these
> concurrent access issues! NB there is no such thing at an OS level of
> stream anything, every thing is clumps of data being buffered is some
> way, the API that accesses that data from the higher levels may be
> stream based. In this case it is CRTL's role to translate the clumps
> of data into/from stream API.

RMS is a pretty good database, for its time. Alas, its become rather
more dated, with an API design that is complex and limiting, and in
competitive terms RMS is badly feature-limited.

If you need a key-value store and where the developer entirely owns the
fields used within the punched cards, and where y'all can fit your
files in 2 TiB (or bound volume sets, gag), RMS is still a fine choice.

For stream access to data, removing the punched-card assumptions and
file and cluster locking and the rest of effectively removes all of RMS
from the discussion; in such a case, RMS really isn't used either in
name, or in general.

As for whether or not there are streams of data, the abstraction
exists. The difference at the app level is whether the operating system
and its default file system enforces the use of a punched-card
abstraction. C does not expect that. Classic OpenVMS apps do.

> So the other choice, 3), fix CRTL to use RMS correctly, and the
> problems will go away. Engineering effort would not be great. I don't
> have access to the code base, but assuming that stdio uses unixio then
> it is fixing 5 routines. This would also allow all the other ports to
> work with minimal changes in the file access area. If you what to know
> more contact me directly.

Punched cards and punched-card-based assumptions are rather more
pernicious within OpenVMS and clustering, and mailboxes, and various
other areas, alas. For those of us steeped in OpenVMS, the effects of
these assumptions can be invisible.

> Longer term the SSIO may be useful to RMS, which is where it belongs.

Longer-term, SSIO belongs in XQP, and RMS needs a demotion to "just
another of the available databases on OpenVMS" atop the XQP, and/or
atop some replacement XQP and/or FUSE for different file systems, and
this with various other common databases present.

SSIO and similar work aside, that demotion of RMS and the related and
substantial investment in new file system and database work are not
going to happen any time soon. Getting the device drivers and XQP to
64-bit storage addressing was reportedly one part of the work involved
(and was once targeted for V8.5), while getting RMS to 64-bit
addressing was a separate and subsequent feature. Getting RMS to 64-bit
storage addressing was and is and will be a rather larger investment,
too. Both VAFS and GFS have been discussed here, but VSI has been busy
with and increasingly focused on the port and port-related work.

SSIO is unrelated to the other file system work pending here.

Back to RMS and SSIO, apps that don't expect punched-card semantics can
and variously do perform their own coordination, so sharing the
underlying files with apps that do expect punched cards is unnecessary,
and counterproductive.

> Sorry if the above is a little blunt, I appreciate the efforts people
> have put in over the years, but some of us have using and coding VMS
> for a very long time, and I really want VMS to be successful and easy
> to port to. This has been a good opportunity for me to look more into
> CRTL and RMS, and see the problems that have been there for decades.

Part of the problem was that thirty years ago, senior DEC leadership
and OpenVMS development leadership was unwilling or unable to foresee
the directions that computing was headed, and fallout from that era
continues to reverberate through to this day around C and IP and RMS.
And around where OpenVMS has found itself in recent years. As long as
we're being blunt.

One of the problems that OpenVMS has here is RMS. While RMS was and is
very useful, it's just not a competitive database in 2021, and too many
of its punched-card assumptions have permeated the platform. That, and
the primary RMS API is just bad for making any sort of significant
changes. This is another area very much like the addition of 64-bit
virtual addressing on OpenVMS; where providing compatibility for 32-bit
virtual apps makes an already complex environment (RMS) vastly more
complex (mixed 32-bit and 64-bit storage addressing within RMS).

> locking is an interesting area, I still feel the current DLM is more
> than capable of doing the 'lock a byte range' in a way that can be used
> with the current RMS locking. Longer term DLM needs some changes but
> they are about sizes of resource names, scoping of resource names,
> ability to scan for children resources by name.

C and DLM already implement range locking on OpenVMS.

--
Pure Personal Opinion | HoffmanLabs LLC

On 10/9/2021 6:19 AM, Greg Tinkler wrote:
> On Saturday, 9 October 2021 at 6:00:48 pm UTC+11, Vitaly Pustovetov wrote:
>>> So the main issue is file IO, so change CRTL to use RMS.
>>>
>>> gt
>> CRTL uses RMS for file I/O. But there is an issue with concurrent access of multiple processes to the same file in stream mode.
>> And we had a choice - 1) rewrite half of Postgres by inserting file locking; 2) add a new SSIO (Shared Stream IO) service to VMS.
>
> Sort of.
>
> RMS worked and has been working of 40+years and does not have these concurrent access issues!

What is RMS, in the current context?

Record Management System
========================

Notice the first word is RECORD. Not FILE, not FIELD, just RECORD.

As for concurrent access, RMS has used the VMS DLM since around 1984.
It is cooperating processes using the DLM that avoids concurrent access
issues.

DLM offers such services.
RMS does not.
CRTL does not.

> NB there is no such thing at an OS level of stream anything,

Not true, unless discussing block oriented devices, which we most likely
are. I do hope you are not suggesting that VMS cannot access memory in
any manner it chooses?

> every thing is clumps of data being buffered is some way, the API that accesses that data from the higher levels may be stream based. In this case it is CRTL's role to translate the clumps of data into/from stream API.

So, how does Pascal, Fortran, Cobol, Basic, and such do it?

>
> So the other choice, 3), fix CRTL to use RMS correctly, and the problems will go away.

3a, The CRTL may use RMS in the manner in which RMS is designed to work.

3b, What problems are there in RMS?

? Engineering effort would not be great. I don't have access to the
code base, but assuming that stdio uses unixio then it is fixing 5
routines. This would also allow all the other ports to work with
minimal changes in the file access area. If you what to know more
contact me directly.
>
> Longer term the SSIO may be useful to RMS, which is where it belongs.

What use does RMS have for any numeric range locking, at least for
anything besides records, which is what RMS is all about.

> Sorry if the above is a little blunt, I appreciate the efforts people have put in over the years, but some of us have using and coding VMS for a very long time, and I really want VMS to be successful and easy to port to. This has been a good opportunity for me to look more into CRTL and RMS, and see the problems that have been there for decades.
>
> locking is an interesting area, I still feel the current DLM is more than capable of doing the 'lock a byte range' in a way that can be used with the current RMS locking. Longer term DLM needs some changes but they are about sizes of resource names, scoping of resource names, ability to scan for children resources by name.
>
> gt
>

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/9/2021 12:47 PM, Stephen Hoffman wrote:

> C and DLM already implement range locking on OpenVMS.

I'd really like to see the documentation and how to use it.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

In article <sjsh2f$tf8$1@dont-email.me>, Stephen Hoffman
<seaohveh@hoffmanlabs.invalid> writes:

> RMS is a pretty good database, for its time. Alas, its become rather
> more dated,

From database to datedbase. :-)

On 10/9/2021 1:54 PM, Dave Froble wrote:
> On 10/9/2021 6:19 AM, Greg Tinkler wrote:
>> every thing is clumps of data being buffered is some way, the API that
>> accesses that data from the higher levels may be stream based.  In
>> this case it is CRTL's role to translate the clumps of data into/from
>> stream API.
>
> So, how does Pascal, Fortran, Cobol, Basic, and such do it?

They do not treat files as streams of bytes - they treat files
as sequences of records.

The underlying problem is that the two paradigms are pretty
incompatible. It is not easy for CRTL to translate a sequence
of records to a stream of bytes in a consistent and meaningful
manner.

Arne

On 10/9/2021 12:47 PM, Stephen Hoffman wrote:
> On 2021-10-09 10:19:42 +0000, Greg Tinkler said:
>> On Saturday, 9 October 2021 at 6:00:48 pm UTC+11, Vitaly Pustovetov
>> wrote:
>>>> So the main issue is file IO, so change CRTL to use RMS.> >> > gt
>>> CRTL uses RMS for file I/O. But there is an issue with concurrent
>>> access of multiple processes to the same file in stream mode. And we
>>> had a choice - 1) rewrite half of Postgres by inserting file locking;
>>> 2) add a new SSIO (Shared Stream IO) service to VMS.
>>
>> Sort of.
>> RMS worked and has been working of 40+years and does not have these
>> concurrent access issues!  NB there is no such thing at an OS level of
>> stream anything, every thing is clumps of data being buffered is some
>> way, the API that accesses that data from the higher levels may be
>> stream based.  In this case it is CRTL's role to translate the clumps
>> of data into/from stream API.
>
> RMS is a pretty good database, for its time.  Alas, its become rather
> more dated, with an API design that is complex and limiting, and in
> competitive terms RMS is badly feature-limited.
>
> If you need a key-value store and where the developer entirely owns the
> fields used within the punched cards, and where y'all can fit your files
> in 2 TiB (or bound volume sets, gag), RMS is still a fine choice.

Hoff I think you are muddying the water here.

This discussion has so far been about ORG:SEQ files.

ORG:IDX files are a Key Value Store. But that is a totally
different topic.

Arne

On 2021-10-09 18:22:02 +0000, Arne Vajhj said:

> On 10/9/2021 12:47 PM, Stephen Hoffman wrote:
>>
>> RMS is a pretty good database, for its time.  Alas, its become rather
>> more dated, with an API design that is complex and limiting, and in
>> competitive terms RMS is badly feature-limited.
>>
>> If you need a key-value store and where the developer entirely owns the
>> fields used within the punched cards, and where y'all can fit your
>> files in 2 TiB (or bound volume sets, gag), RMS is still a fine choice.
>
> Hoff I think you are muddying the water here.
>
> This discussion has so far been about ORG:SEQ files.
>
> ORG:IDX files are a Key Value Store. But that is a totally different topic.

And here I was trying to explicitly not slag on RMS and its
capabilities, as that'd solely serve provoke a torrent of folks quite
reasonably pointing out that RMS is perfect for {app}.

--
Pure Personal Opinion | HoffmanLabs LLC

On 2021-10-09 18:00:45 +0000, Dave Froble said:

> On 10/9/2021 12:47 PM, Stephen Hoffman wrote:
>
>> C and DLM already implement range locking on OpenVMS.
>
> I'd really like to see the documentation and how to use it.

Alas, entirely undocumented, per the previous comments around here.

--
Pure Personal Opinion | HoffmanLabs LLC

суббота, 9 октября 2021 г. в 21:02:59 UTC+3, Dave Froble:
> On 10/9/2021 12:47 PM, Stephen Hoffman wrote:
>
> > C and DLM already implement range locking on OpenVMS.
> I'd really like to see the documentation and how to use it.

"File Locking
The C RTL supports byte-range file locking using the F_GETLK, F_SETLK, and F_SETLKW
commands of the fcntl function, as defined in the X/Open specification. Byte-range file locking is
supported across OpenVMS clusters. You can only use offsets that fit into 32-bit unsigned integers.
When a shared lock is set on a segment of a file, other processes on the cluster are able to set shared
locks on that segment or a portion of it. A shared lock prevents any other process from setting
an exclusive lock on any portion of the protected area. A request for a shared lock fails if the file
descriptor was not opened with read access....."(c)VSI C Run-Time Library Reference Manual

On 2021-10-09 21:22:29 +0000, Vitaly Pustovetov said:

> суббота, 9 октября 2021 г. в 21:02:59 UTC+3, Dave Froble:
>> On 10/9/2021 12:47 PM, Stephen Hoffman wrote:>> > C and DLM already
>> implement range locking on OpenVMS.
>> I'd really like to see the documentation and how to use it.
>
> "File Locking
> The C RTL supports byte-range...

That comment was in reference to the DLM range-locking API; the
(un)documentation for what's implemented underneath those C calls
within CRTL and DLM.

--
Pure Personal Opinion | HoffmanLabs LLC

On 10/9/2021 2:18 PM, Arne Vajhøj wrote:
> On 10/9/2021 1:54 PM, Dave Froble wrote:
>> On 10/9/2021 6:19 AM, Greg Tinkler wrote:
>>> every thing is clumps of data being buffered is some way, the API
>>> that accesses that data from the higher levels may be stream based.
>>> In this case it is CRTL's role to translate the clumps of data
>>> into/from stream API.
>>
>> So, how does Pascal, Fortran, Cobol, Basic, and such do it?
>
> They do not treat files as streams of bytes - they treat files
> as sequences of records.
>
> The underlying problem is that the two paradigms are pretty
> incompatible. It is not easy for CRTL to translate a sequence
> of records to a stream of bytes in a consistent and meaningful
> manner.
>
> Arne

Which is why Steve's suggestion for ODS2/ODS5 becoming just another file
system.

Which is why Steve's suggestion for RMS to become just another database
product. Well, if ODS? wants to use it for directories, Ok.

But even if another "application" handles other files, there is still
the issue of today's disks being block based (Ok, punched card if you
must) devices.

Stream devices is alien enough to today's VMS that it would be much
better served by dedicated tools designed for that format. (And it sure
isn't RMS!)

Then there is the interesting question of what the next format to come
along might be.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/9/2021 4:57 PM, Stephen Hoffman wrote:
> On 2021-10-09 18:00:45 +0000, Dave Froble said:
>
>> On 10/9/2021 12:47 PM, Stephen Hoffman wrote:
>>
>>> C and DLM already implement range locking on OpenVMS.
>>
>> I'd really like to see the documentation and how to use it.
>
> Alas, entirely undocumented, per the previous comments around here.
>
>

Then, does it really exist?

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 10/9/2021 2:22 PM, Arne Vajhøj wrote:
> On 10/9/2021 12:47 PM, Stephen Hoffman wrote:
>> On 2021-10-09 10:19:42 +0000, Greg Tinkler said:
>>> On Saturday, 9 October 2021 at 6:00:48 pm UTC+11, Vitaly Pustovetov
>>> wrote:
>>>>> So the main issue is file IO, so change CRTL to use RMS.> >> > gt
>>>> CRTL uses RMS for file I/O. But there is an issue with concurrent
>>>> access of multiple processes to the same file in stream mode. And we
>>>> had a choice - 1) rewrite half of Postgres by inserting file
>>>> locking; 2) add a new SSIO (Shared Stream IO) service to VMS.
>>>
>>> Sort of.
>>> RMS worked and has been working of 40+years and does not have these
>>> concurrent access issues! NB there is no such thing at an OS level
>>> of stream anything, every thing is clumps of data being buffered is
>>> some way, the API that accesses that data from the higher levels may
>>> be stream based. In this case it is CRTL's role to translate the
>>> clumps of data into/from stream API.
>>
>> RMS is a pretty good database, for its time. Alas, its become rather
>> more dated, with an API design that is complex and limiting, and in
>> competitive terms RMS is badly feature-limited.
>>
>> If you need a key-value store and where the developer entirely owns
>> the fields used within the punched cards, and where y'all can fit your
>> files in 2 TiB (or bound volume sets, gag), RMS is still a fine choice.
>
> Hoff I think you are muddying the water here.
>
> This discussion has so far been about ORG:SEQ files.
>
> ORG:IDX files are a Key Value Store. But that is a totally
> different topic.
>
> Arne

No, it is not. The OP declared that RMS should be used for that.

You are correct that we're concerned about stream files, but claims
about RMS have been part of the discussion.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486