novaBBS - comp.os.vms - Volume Shadowing internals

Volume Shadowing internals

<a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>

https://www.novabbs.com/computers/article-flat.php?id=30894&group=comp.os.vms#30894

X-Received: by 2002:a05:620a:29c6:b0:775:78bb:5e8e with SMTP id s6-20020a05620a29c600b0077578bb5e8emr235112qkp.5.1698779899821;
Tue, 31 Oct 2023 12:18:19 -0700 (PDT)
X-Received: by 2002:a05:6870:3646:b0:1e9:dc1e:8384 with SMTP id
v6-20020a056870364600b001e9dc1e8384mr6741579oak.5.1698779899555; Tue, 31 Oct
2023 12:18:19 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!proxad.net!feeder1-2.proxad.net!209.85.160.216.MISMATCH!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.vms
Date: Tue, 31 Oct 2023 12:18:19 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=213.108.211.135; posting-account=9f7q1AoAAABDPsFEcWiRS6khkDURMqEt
NNTP-Posting-Host: 213.108.211.135
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
Subject: Volume Shadowing internals
From: aaa....@gmail.com (El SysMan)
Injection-Date: Tue, 31 Oct 2023 19:18:19 +0000
Content-Type: text/plain; charset="UTF-8"

by: El SysMan - Tue, 31 Oct 2023 19:18 UTC

Re: Volume Shadowing internals

<95a1c219-6fe9-4e2d-9b81-c49b1a206e16n@googlegroups.com>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=30897&group=comp.os.vms#30897

copy link Newsgroups: comp.os.vms

X-Received: by 2002:ae9:e010:0:b0:777:506b:39ee with SMTP id m16-20020ae9e010000000b00777506b39eemr200300qkk.6.1698787329225; Tue, 31 Oct 2023 14:22:09 -0700 (PDT)
X-Received: by 2002:a05:6808:1a02:b0:3a7:27a:86e3 with SMTP id bk2-20020a0568081a0200b003a7027a86e3mr6000078oib.3.1698787328988; Tue, 31 Oct 2023 14:22:08 -0700 (PDT)
Path: i2pn2.org!i2pn.org!newsfeed.endofthelinebbs.com!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!feeder.usenetexpress.com!tr2.iad1.usenetexpress.com!69.80.99.15.MISMATCH!border-1.nntp.ord.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.vms
Date: Tue, 31 Oct 2023 14:22:08 -0700 (PDT)
In-Reply-To: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=2a00:23c6:2201:c401:fc6e:23f4:25b7:1221; posting-account=7q_AmAoAAAB-_OKdNMKqm2-wLm5fUMWW
NNTP-Posting-Host: 2a00:23c6:2201:c401:fc6e:23f4:25b7:1221
References: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <95a1c219-6fe9-4e2d-9b81-c49b1a206e16n@googlegroups.com>
Subject: Re: Volume Shadowing internals
From: martinsw...@gmail.com (Martin Rushton)
Injection-Date: Tue, 31 Oct 2023 21:22:09 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 7

by: Martin Rushton - Tue, 31 Oct 2023 21:22 UTC

On Tuesday, October 31, 2023 at 7:18:21 PM UTC, El SysMan wrote:
> Hi All!
>
> Is there any publications is related to the Volume Shadowing ?
Your title specifies internals, but then you just ask if there are "any publications is related to the Volume Shadowing ?". Well I don't have anything on internals, but there is the VolShad guide online: https://docs.vmssoftware.com/vsi-openvms-volume-shadowing-guide/

Re: Volume Shadowing internals

<uhrvd1$184h3$1@dont-email.me>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=30898&group=comp.os.vms#30898

copy link Newsgroups: comp.os.vms

Path: i2pn2.org!i2pn.org!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: seaoh...@hoffmanlabs.invalid (Stephen Hoffman)
Newsgroups: comp.os.vms
Subject: Re: Volume Shadowing internals
Date: Tue, 31 Oct 2023 18:34:09 -0400
Organization: HoffmanLabs LLC
Lines: 26
Message-ID: <uhrvd1$184h3$1@dont-email.me>
References: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: dont-email.me; posting-host="874bdf326f21f19e1a77ba78c23672e4";
logging-data="1315363"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+AnNcLMT2MFotL1/hHMEBu6v4FZwum7Ko="
User-Agent: Unison/2.2
Cancel-Lock: sha1:PSnur3mWBd08jRS0aFKzgWFpA+0=

by: Stephen Hoffman - Tue, 31 Oct 2023 22:34 UTC

On 2023-10-31 19:18:19 +0000, El SysMan said:

> Is there any publications is related to the Volume Shadowing ?

Published internals for Host-based Volume Shadowing (HBVS); for the
OpenVMS clustered multi-host shared-write software RAID-1
implementation? No.

Internals documentation was never published for either HBVS Phase I or
HBVS Phase II.

The old internals and data structures did have a page or two on the
internal lock usage.

There might be a few pieces in passing in either the VMS File Systems
Internals and the Cluster Internals books, if you can source either of
those.

Of what was published, the high-level design of HBVS Phase II from
Digital Technical Journal is probably the most detailed:
https://vmssoftware.com/docs/dtj-v03-03-1991.pdf

--
Pure Personal Opinion | HoffmanLabs LLC

Re: Volume Shadowing internals

<uhs86j$19ct5$1@dont-email.me>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=30899&group=comp.os.vms#30899

copy link Newsgroups: comp.os.vms

Path: i2pn2.org!i2pn.org!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: FIRST.L...@vmssoftware.com (Robert A. Brooks)
Newsgroups: comp.os.vms
Subject: Re: Volume Shadowing internals
Date: Tue, 31 Oct 2023 21:04:19 -0400
Organization: A noiseless patient Spider
Lines: 12
Message-ID: <uhs86j$19ct5$1@dont-email.me>
References: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Wed, 1 Nov 2023 01:04:19 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="5e11a3dc942a2c01d42c89bb602d7390";
logging-data="1356709"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19Lej8IR0YSnNCtbitym+NjVgLuQYXVYWgtMCocDT/xbA=="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:43RfuJdLn2d9suNmyAFf1UjEtQw=
In-Reply-To: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
Content-Language: en-US
X-Antivirus-Status: Clean
X-Antivirus: Avast (VPS 231031-10, 10/31/2023), Outbound message

by: Robert A. Brooks - Wed, 1 Nov 2023 01:04 UTC

On 10/31/2023 3:18 PM, El SysMan wrote:
> Hi All!
>
> Is there any publications is related to the Volume Shadowing ?

Can you be a bit more specific regarding what aspect of Shadowing you are
interested in?

--- Rob (current shadowing maintainer)

Re: Volume Shadowing internals

<457fbb3e-7b60-4a66-ac57-c8b043562c11n@googlegroups.com>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=30903&group=comp.os.vms#30903

copy link Newsgroups: comp.os.vms

X-Received: by 2002:a0c:be88:0:b0:66d:8696:7216 with SMTP id n8-20020a0cbe88000000b0066d86967216mr293113qvi.11.1698860315724;
Wed, 01 Nov 2023 10:38:35 -0700 (PDT)
X-Received: by 2002:a9d:578b:0:b0:6ce:27a0:8658 with SMTP id
q11-20020a9d578b000000b006ce27a08658mr4564033oth.1.1698860315519; Wed, 01 Nov
2023 10:38:35 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!peer03.ams1!peer.ams1.xlned.com!news.xlned.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.vms
Date: Wed, 1 Nov 2023 10:38:34 -0700 (PDT)
In-Reply-To: <uhs86j$19ct5$1@dont-email.me>
Injection-Info: google-groups.googlegroups.com; posting-host=2a00:23c6:2201:c401:fc6e:23f4:25b7:1221;
posting-account=7q_AmAoAAAB-_OKdNMKqm2-wLm5fUMWW
NNTP-Posting-Host: 2a00:23c6:2201:c401:fc6e:23f4:25b7:1221
References: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com> <uhs86j$19ct5$1@dont-email.me>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <457fbb3e-7b60-4a66-ac57-c8b043562c11n@googlegroups.com>
Subject: Re: Volume Shadowing internals
From: martinsw...@gmail.com (Martin Rushton)
Injection-Date: Wed, 01 Nov 2023 17:38:35 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 1947

by: Martin Rushton - Wed, 1 Nov 2023 17:38 UTC

On Wednesday, November 1, 2023 at 1:04:23 AM UTC, Robert A. Brooks wrote:
> On 10/31/2023 3:18 PM, El SysMan wrote:
> > Hi All!
> >
> > Is there any publications is related to the Volume Shadowing ?
> Can you be a bit more specific regarding what aspect of Shadowing you are
> interested in?
>
> --
>
> --- Rob (current shadowing maintainer)
I've just checked the indexes of Internals and Data Structures (VMS 4.4) and File System Internals (VMS 5.0) and apart from a passing reference in the latter they don't mention volume shadowing at all. Probably need newer versions.

Re: Volume Shadowing internals

<uhufvb$1ptot$1@dont-email.me>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=30904&group=comp.os.vms#30904

copy link Newsgroups: comp.os.vms

Path: i2pn2.org!i2pn.org!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: seaoh...@hoffmanlabs.invalid (Stephen Hoffman)
Newsgroups: comp.os.vms
Subject: Re: Volume Shadowing internals
Date: Wed, 1 Nov 2023 17:29:15 -0400
Organization: HoffmanLabs LLC
Lines: 16
Message-ID: <uhufvb$1ptot$1@dont-email.me>
References: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com> <uhs86j$19ct5$1@dont-email.me> <457fbb3e-7b60-4a66-ac57-c8b043562c11n@googlegroups.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: dont-email.me; posting-host="bac5f280364b6f1e55bf198e1ca78b98";
logging-data="1898269"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+THhHDDU8NhwVlNfbXqgMQd8n+pphFPA4="
User-Agent: Unison/2.2
Cancel-Lock: sha1:AecodrcccW75/uF/RFMB2RKK+YI=

by: Stephen Hoffman - Wed, 1 Nov 2023 21:29 UTC

On 2023-11-01 17:38:34 +0000, Martin Rushton said:

> I've just checked the indexes of Internals and Data Structures (VMS
> 4.4) and File System Internals (VMS 5.0) and apart from a passing
> reference in the latter they don't mention volume shadowing at all.
> Probably need newer versions.

IDSM never documented HBVS. Not more than some passing references.
DEC never particularly documented HBVS internals. User docs, yes.
Internals, no.
Most detailed internal-ish description of HBVS I'm aware of is the DTJ
article linked earlier.

--
Pure Personal Opinion | HoffmanLabs LLC

Re: Volume Shadowing internals

<9c8db53f-09eb-497c-b6f1-488195db00b2n@googlegroups.com>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=30909&group=comp.os.vms#30909

copy link Newsgroups: comp.os.vms

X-Received: by 2002:a05:6214:514b:b0:66d:1308:1836 with SMTP id kh11-20020a056214514b00b0066d13081836mr182654qvb.2.1698925860484;
Thu, 02 Nov 2023 04:51:00 -0700 (PDT)
X-Received: by 2002:a05:6870:1689:b0:1e9:8932:9fe6 with SMTP id
j9-20020a056870168900b001e989329fe6mr9074687oae.4.1698925860271; Thu, 02 Nov
2023 04:51:00 -0700 (PDT)
Path: i2pn2.org!i2pn.org!nntp.comgw.net!weretis.net!feeder8.news.weretis.net!proxad.net!feeder1-2.proxad.net!209.85.160.216.MISMATCH!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.vms
Date: Thu, 2 Nov 2023 04:50:59 -0700 (PDT)
In-Reply-To: <uhufvb$1ptot$1@dont-email.me>
Injection-Info: google-groups.googlegroups.com; posting-host=213.108.211.135; posting-account=9f7q1AoAAABDPsFEcWiRS6khkDURMqEt
NNTP-Posting-Host: 213.108.211.135
References: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
<uhs86j$19ct5$1@dont-email.me> <457fbb3e-7b60-4a66-ac57-c8b043562c11n@googlegroups.com>
<uhufvb$1ptot$1@dont-email.me>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <9c8db53f-09eb-497c-b6f1-488195db00b2n@googlegroups.com>
Subject: Re: Volume Shadowing internals
From: aaa....@gmail.com (El SysMan)
Injection-Date: Thu, 02 Nov 2023 11:51:00 +0000
Content-Type: text/plain; charset="UTF-8"

by: El SysMan - Thu, 2 Nov 2023 11:50 UTC

Great thanks for references VMS OldBoys!

I have a small task to implement a cross-site replication of block devices. So, i simply made "full copy" of source device to target - it's acceptable for NVMEoF in the LAN, but in case of "cross-site" we need to reduce a traffic.
I have supposed that Volume Shadowing may contains some kind of optimization to exclude copying unallocated LBN-s for example or may be maintain a bitmap for updated pages\buckets from last "synchronization" ...

Re: Volume Shadowing internals

<ui0955$27cgp$1@dont-email.me>

copy mid

https://www.novabbs.com/computers/article-flat.php?id=30913&group=comp.os.vms#30913

copy link Newsgroups: comp.os.vms

Path: i2pn2.org!rocksolid2!news.neodome.net!news.mixmin.net!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: FIRST.L...@vmssoftware.com (Robert A. Brooks)
Newsgroups: comp.os.vms
Subject: Re: Volume Shadowing internals
Date: Thu, 2 Nov 2023 09:45:08 -0400
Organization: A noiseless patient Spider
Lines: 21
Message-ID: <ui0955$27cgp$1@dont-email.me>
References: <a24a2941-bdc2-4cdc-98d6-a70a123529can@googlegroups.com>
<uhs86j$19ct5$1@dont-email.me>
<457fbb3e-7b60-4a66-ac57-c8b043562c11n@googlegroups.com>
<uhufvb$1ptot$1@dont-email.me>
<9c8db53f-09eb-497c-b6f1-488195db00b2n@googlegroups.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Thu, 2 Nov 2023 13:45:09 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="1b1ba055721fedfc4a428bcffad6fb0e";
logging-data="2339353"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+OKhniMcDDEbaNAvN8dO3XLuZ596U5muDxz4RuEZjSkg=="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:lSbmO6596RNMf1ngQXtXxRvU+9A=
In-Reply-To: <9c8db53f-09eb-497c-b6f1-488195db00b2n@googlegroups.com>
X-Antivirus-Status: Clean
X-Antivirus: Avast (VPS 231030-4, 10/30/2023), Outbound message
Content-Language: en-US

by: Robert A. Brooks - Thu, 2 Nov 2023 13:45 UTC

On 11/2/2023 7:50 AM, El SysMan wrote:
> Great thanks for references VMS OldBoys!
>
> I have a small task to implement a cross-site replication of block
> devices. So, i simply made "full copy" of source device to target -
> it's acceptable for NVMEoF in the LAN, but in case of "cross-site" we
> need to reduce a traffic. I have supposed that Volume Shadowing may
> contains some kind of optimization to exclude copying unallocated
> LBN-s for example or may be maintain a bitmap for updated
> pages\buckets from last "synchronization" ...

Read up on mini copy, host-based minimerge, and $ INIT/SHAD for ways to
reduce copy and merge times.

-- Rob

--
This email has been checked for viruses by Avast antivirus software.
www.avast.com

"Elvis is my copilot." -- Cal Keegan

computers / comp.os.vms / Volume Shadowing internals

Subject	Author
Volume Shadowing internals	El SysMan
Re: Volume Shadowing internals	Martin Rushton
Re: Volume Shadowing internals	Stephen Hoffman
Re: Volume Shadowing internals	Robert A. Brooks
Re: Volume Shadowing internals	Martin Rushton
Re: Volume Shadowing internals	Stephen Hoffman
Re: Volume Shadowing internals	El SysMan
Re: Volume Shadowing internals	Robert A. Brooks