Hello Old$Boys!

Can u help me to search a reference to info about of RMS journaling? May be DEC Journal or something other ?

Den 2023-07-16 kl. 12:51, skrev d45 n0x8!7:
> Hello Old$Boys!
>
> Can u help me to search a reference to info about of RMS journaling? May be DEC Journal or something other ?

Maybe: https://docs.vmssoftware.com/rms-journaling-for-openvms-manual/

On Sunday, July 16, 2023 at 6:51:52 AM UTC-4, d45 n0x8!7 wrote:
> Hello Old$Boys!
>
> Can u help me to search a reference to info about of RMS journaling? May be DEC Journal or something other ?

For full understanding you should start with the documentation as per Jen-Erik's link.
It may be useful to point out that RMS Journaling is based on physical 'buckets', not logical records in the files.
The implication of that is that there is potentially much more data being journaled than there is data changed.
It's goal is to present a consistent file for a given point in time.
AI journaling to re-apply changing up to a selectable time starting from a backup.
BI journaling to 'undo' changes back to a selectable time starting from a live file
RU journaling to create transactional semantics across files allowing a single commit (or rollback) for multiple record changes.

Did you just want an explanation (how deep?) or do you have a business requirement for which you think RMS-Journaling might offer a solution? What is the exact problem you are trying to solve (if any).

A relatively common Journaling desire is Change Data Capture (CDC) which RMS Journaling does NOT offer.
The desire there being to have access to individual changed to records, either for auditing purposes (who changed what when) or to be able to forward such changes to another system or to a database.

There is a Commercial product called RMS-CDC owned by Brian Schenkenberger which can do full CDC for any and all file changes including before-images based on a magic RMS intercept.

VSI has a prototype CDC based on the standard SSI (System Service Intercept) which required the programs to cooperate and declare they want they actions to be intercepted. It has good potential but does not offer Before Images (which may or might not be critical).

Hope this helps,
Hein.

On Sunday, July 16, 2023 at 5:52:59 PM UTC+3, Hein RMS van den Heuvel wrote:
> Did you just want an explanation (how deep?) or do you have a business requirement for which you think RMS-Journaling might offer a solution? What is the exact problem you are trying to solve (if any).

As deep as possible.
I'm need to implement (in pure C of course, not in BLISS :-) ) a logic for atomic write for 4k block on several physical devices (with 512 bytes granularity) in Soft. Def. Stor. project.
In a couple words: I/O request for 4k block should be spread over 4-8 disks as "atomic" action.
There is "start TX" phase - and "end TX" phase. My driver accept an I/O request for the particular block and wait for "end TX" signal.

> Hope this helps,
> Hein.
Thanks a lot for answer.

PS: Sorry, but this project is for Snack Oil OS but not for OpenVMS.

On Sunday, July 16, 2023 at 3:32:01 PM UTC+3, Jan-Erik Söderholm wrote:
> Den 2023-07-16 kl. 12:51, skrev d45 n0x8!7:
> > Hello Old$Boys!
> >
> > Can u help me to search a reference to info about of RMS journaling? May be DEC Journal or something other ?
> Maybe: https://docs.vmssoftware.com/rms-journaling-for-openvms-manual/
Thanks!
In my past live as VMS-boys I used RMS journaling in programs, but ... never mind how it works internally.

On Sunday, July 16, 2023 at 9:09:53 AM UTC-7, d45 n0x8!7 wrote:
> On Sunday, July 16, 2023 at 3:32:01 PM UTC+3, Jan-Erik Söderholm wrote:
> > Den 2023-07-16 kl. 12:51, skrev d45 n0x8!7:
> > > Hello Old$Boys!
> > >
> > > Can u help me to search a reference to info about of RMS journaling? May be DEC Journal or something other ?
> > Maybe: https://docs.vmssoftware.com/rms-journaling-for-openvms-manual/
> Thanks!
> In my past live as VMS-boys I used RMS journaling in programs, but ... never mind how it works internally.

I know I'm going to regret this:

I thnk you're talking about some RAID configuration, not journaling. Besides, you're writing for Unix, not VMS. The filesystem tools are quite a bit .... different

On Sunday, July 16, 2023 at 9:08:06 AM UTC-7, d45 n0x8!7 wrote:
> On Sunday, July 16, 2023 at 5:52:59 PM UTC+3, Hein RMS van den Heuvel wrote:
> > Did you just want an explanation (how deep?) or do you have a business requirement for which you think RMS-Journaling might offer a solution? What is the exact problem you are trying to solve (if any).
> As deep as possible.
> I'm need to implement (in pure C of course, not in BLISS :-) ) a logic for atomic write for 4k block on several physical devices (with 512 bytes granularity) in Soft. Def. Stor. project.
> In a couple words: I/O request for 4k block should be spread over 4-8 disks as "atomic" action.
> There is "start TX" phase - and "end TX" phase. My driver accept an I/O request for the particular block and wait for "end TX" signal.

I forgot to add -
This looks like a roll-your-own 2-phase commit that's missing "commit / rollback" logic. What you're describing is an in-line protocol that's also missing a time-out, i.e. what happens when "end TX" never arrives?

anyway...
>
>
> > Hope this helps,
> > Hein.
> Thanks a lot for answer.
>
>
> PS: Sorry, but this project is for Snack Oil OS but not for OpenVMS.

On Sunday, July 16, 2023 at 9:08:06 AM UTC-7, d45 n0x8!7 wrote:
> On Sunday, July 16, 2023 at 5:52:59 PM UTC+3, Hein RMS van den Heuvel wrote:
> > Did you just want an explanation (how deep?) or do you have a business requirement for which you think RMS-Journaling might offer a solution? What is the exact problem you are trying to solve (if any).
> As deep as possible.
> I'm need to implement (in pure C of course, not in BLISS :-) ) a logic for atomic write for 4k block on several physical devices (with 512 bytes granularity) in Soft. Def. Stor. project.
> In a couple words: I/O request for 4k block should be spread over 4-8 disks as "atomic" action.
> There is "start TX" phase - and "end TX" phase. My driver accept an I/O request for the particular block and wait for "end TX" signal.
>
>
> > Hope this helps,
> > Hein.
> Thanks a lot for answer.
>
>
> PS: Sorry, but this project is for Snack Oil OS but not for OpenVMS.

For those playing along at home, it's "Snakeoil OS"

On Monday, July 17, 2023 at 2:42:58 PM UTC+3, plugh wrote:
> > I'm need to implement (in pure C of course, not in BLISS :-) ) a logic for atomic write for 4k block on several physical devices (with 512 bytes granularity) in Soft. Def. Stor. project.
> > In a couple words: I/O request for 4k block should be spread over 4-8 disks as "atomic" action.
> > There is "start TX" phase - and "end TX" phase. My driver accept an I/O request for the particular block and wait for "end TX" signal.
> I forgot to add -
> This looks like a roll-your-own 2-phase commit that's missing "commit / rollback" logic. What you're describing is an in-line protocol that's also missing a time-out, i.e. what happens when "end TX" never arrives?

Thanks!
I reading the RMS docs to refresh my knowledge about of possible variants ....
U r right there is missing "rollback" and i'm need to think about this.

Some other question: Is the RMS journaling allows nested transactions ?
Hein RMS (The) told that RMS journaling is performed at bucket level not for records, so ... how is resolved a situation when from other tread (FAB/RAB) is coming request to update the same bucket ?

On Monday, July 17, 2023 at 8:33:26 AM UTC-4, El SysMan wrote:
> Some other question: Is the RMS journaling allows nested transactions ?

No, one "RunUnit" per process

> Hein RMS (The) told that RMS journaling is performed at bucket level not for records, so ... how is resolved a situation when from other tread (FAB/RAB) is coming request to update the same bucket ?

I was sloppy, not knowing where the questions were going. The AI and BI journalling record buckets (up to the free byte mark).
RU journalling is record based and impacted Records stay locked for the duration of the transaction.
The picture in the Journaling manual under "5.2.4 Transaction states" may be helpful there.

Hein.

On Monday, July 17, 2023 at 5:50:07 PM UTC+3, Hein RMS van den Heuvel wrote:
> On Monday, July 17, 2023 at 8:33:26 AM UTC-4, El SysMan wrote:
> > Some other question: Is the RMS journaling allows nested transactions ?
> No, one "RunUnit" per process
> > Hein RMS (The) told that RMS journaling is performed at bucket level not for records, so ... how is resolved a situation when from other tread (FAB/RAB) is coming request to update the same bucket ?
> I was sloppy, not knowing where the questions were going. The AI and BI journalling record buckets (up to the free byte mark).
> RU journalling is record based and impacted Records stay locked for the duration of the transaction.
> The picture in the Journaling manual under "5.2.4 Transaction states" may be helpful there.
>
> Hein.

Can u recollect any related publications in the Digital journal or something else?

Thanks a lot!

On 2023-07-16 14:52:56 +0000, Hein RMS van den Heuvel said:

> On Sunday, July 16, 2023 at 6:51:52 AM UTC-4, d45 n0x8!7 wrote:
>> Hello Old$Boys!>> Can u help me to search a reference to info about of
>> RMS journaling? May be DEC Journal or something other ?
> {...snip...}
>
> A relatively common Journaling desire is Change Data Capture (CDC)
> which RMS Journaling does NOT offer.
> The desire there being to have access to individual changed to records,
> either for auditing purposes (who changed what when) or to be able to
> forward such changes to another system or to a database.
>
> There is a Commercial product called RMS-CDC owned by Brian
> Schenkenberger which can do full CDC for any and all file changes
> including before-images based on a magic RMS intercept.

The "intercept" isn't "magic", it's the only way to do this and do this
properly! ANY and ALL RMS operations that can affect a file's contents
must be captured. The record's data is obtained from the bowels of RMS.
Not everything to be recorded or "captured" is passed as an argument!
The BEFORE IMAGE is the real magic!

BEFOE IMAGE??? What's in the record BEFORE an $UPDATE? What WAS in the
record BEFORE a $DELETE? It's all magic from deep in the bowels of RMS.
Believe me, it's not simple and it's not as simple as performing a $GET
before doing an $UPDATE or a $DELETE. There's a lot of unpleasant
shhh..tuff in those bowels. Indexed files are expecially onerous
because a $GET, which would seem an obvious way to obtain the BEFORE
IMAGE, can and will corrupt the record pointers for the $UPDATE and or
$DELETE. It has/had to be done without affecting the actual RMS call's
functional operation. It's also done ***without*** performing
additional I/O!

> VSI has a prototype CDC based on the standard SSI (System Service
> Intercept) which required the programs to cooperate and declare they
> want they actions to be intercepted. It has good potential but does not
> offer Before Images (which may or might not be critical).

I can't see how that can/could work. SSI calls need to be conciously
incorporated into an/the image that would be doing the RMS calls.
Could/would this capture RMS from DCL, CONVERT, etc? without code
modification? What about the BEFORE IMAGE? Good luck there without
privileged code.

RMS-CDC works without any code modification needed. It's installed and
loaded, and then the files that are of interest for change data capture
are added to RMS-CDC's files-of-interest database. From that point on,
anything that can and will modifiy a file's contents is captured and
squirreled away in a log file for consumption for whatever purpose --
as Hein mentioned such as auditing or database or forwarding to other
systems not necessarily VMS systems -- deemed necessary.

If a system is in a cluster, RMS-CDC needs to be on every node of the
cluster because files can be accessed from every node in the cluster.
This requires special syncronization and sequencing to keep all CDC
data coordinated.

I know that Hein knows all of this because I'd worked closely with him
for several years to implement RMS-CDC. This reply is for the
enlightenment of others.

— VAXman