Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.

Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...

Thanks for any info

On 4/7/2022 7:24 PM, Rich Jordan wrote:
> Customer has decided to turn off "legacy" DECnet support on their network.
> They currently use DECnet for copies between two nonclustered integrity
> servers that are on different IP subnets/VLANs and have Cisco doing whatever
> magic it does to make DECnet appear local. THis is Phase IV, not DECnet over
> IP.

Install DECnet-Plus (IA64) or DECnet/OSI (Alpha), and enable DECnet over TCP/IP.

That's likely the easiest solution, since it'll require no application change.

Other than the initial configuration, it'll be "set it and forget it".

If you have non-transparent task-to-task applications, you'll need to understand
some of the differences between Phase IV and Phase V, mostly in terminology.
Phase IV "objects" are Phase V "session control applications". NCL vs. NCP
has a bit of a learning curve.

If your use of DECnet is limited to $ SET HOST and $ COPY, then the differences
between Phase IV and Phase V will barely be noticeable.

--
-- Rob

> [...] THis is Phase IV, not DECnet over IP.

> Install DECnet-Plus (IA64) or DECnet/OSI (Alpha), and enable DECnet
> over TCP/IP.

Yeah, what he said.

I've never worried about performance, but when I ran a SIMH VAX
emulator on a Mac, I was pleased with the DECnet-like behavior of
DECnet(-Plus) between my main IA64 system (ITS, 10.0.0.140, 1.140) and
the pseudo-VAX (WISP, 10.0.4.32, 1.32). Normal file access, like, say:
ITS $ directory /size wisp::
just worked.

In that case, the Mac was the router between the two IP subnets
(main Ethernet = 10.0.0.0/24 and Mac-emulator = 10.0.4.0/24).

On 4/7/2022 7:24 PM, Rich Jordan wrote:
> Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
>
> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
>
> Thanks for any info
>

Tell customer to re-think ...

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On Thursday, April 7, 2022 at 7:24:03 PM UTC-4, Rich Jordan wrote:
> Customer has decided to turn off "legacy" DECnet support on their network.. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
>
> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
>
> Thanks for any info
Rich,

As Robert noted, DECnet Phase V over IP is a viable choice. In situations where the client did not want to do that change, I have used sftp quite effectively, often in concert with ZIP/UNZIP.

As an example, in one quick test, I was able to transfer more than a terabyte of data in a reasonable time (I do not have the precise numbers in easy reach at the moment). The context was non-tape backup. The process was:
- Do BACKUP/IMAGE to a scratch device
- use ZIP "-V" to compress the BACKUP save set. Your mileage will vary, I was able to get approximately 90% size reduction.
- Use sftp to copy the resulting ZIP files to the destination machine
- UNZIP the transferred files
- use BACKUP to restore the saveset on the destination machine

The above was run in multiple BATCH jobs on both sides.

- Bob Gezelter, http://www.rlgsc.com

On Friday, April 8, 2022 at 6:24:37 AM UTC-5, geze...@rlgsc.com wrote:
> On Thursday, April 7, 2022 at 7:24:03 PM UTC-4, Rich Jordan wrote:
> > Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
> >
> > Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
> >
> > Thanks for any info
> Rich,
>
> As Robert noted, DECnet Phase V over IP is a viable choice. In situations where the client did not want to do that change, I have used sftp quite effectively, often in concert with ZIP/UNZIP.
>
> As an example, in one quick test, I was able to transfer more than a terabyte of data in a reasonable time (I do not have the precise numbers in easy reach at the moment). The context was non-tape backup. The process was:
> - Do BACKUP/IMAGE to a scratch device
> - use ZIP "-V" to compress the BACKUP save set. Your mileage will vary, I was able to get approximately 90% size reduction.
> - Use sftp to copy the resulting ZIP files to the destination machine
> - UNZIP the transferred files
> - use BACKUP to restore the saveset on the destination machine
>
> The above was run in multiple BATCH jobs on both sides.
>
> - Bob Gezelter, http://www.rlgsc.com

Thank you all for responses. I'm not excited about the thought of installing Phase V mainly because it has been 25+ years since I even looked at it. I can see if it is an option. At least that I can test here at our office..

Losing current DECnet transit is not negotiable apparently; we lost that argument but that choice is with the customer and their network vendor/support; we and the VMS boxes are a side issue that just needs to deal with the change.

As noted we already do some data moves by doing backup to local disk savesets (using the /DATA_FORMAT=COMPRESS option) then FTP'ing those sets to a PC. Fully tested so we know we can retrieve those savesets, fix the attributes, and restore them because doing even a zero compression encapsulating ZIP takes too much time for the larger savesets (ZIP would preserve the backup saveset file attributes).

The current VMS to VMS is done with DECnet copies of individual files, or backup of folders/trees or bulk files to remote saveset via DECnet that gets restored on the remote system.

> [...] backup to local disk savesets (using the /DATA_FORMAT=COMPRESS
> option) [...]

> [...] fix the attributes, and restore them because doing even a zero
> compression encapsulating ZIP takes too much time for the larger
> savesets (ZIP would preserve the backup saveset file attributes).

If all you wanted Zip+UnZip to do was preserve the attributes of a
BACKUP save set, then I wouldn't bother. There are enough DCL scripts
floating around which can restore them in near-zero time.

It might be worth a quick experiment to see whether BACKUP without
compression followed by Zip with compression makes any sense.

> The current VMS to VMS is done with DECnet copies of individual files,
> [...]

The principal advantage of new DECnet (ising IP) is that it works
like old DECnet.

On 2022-04-08 02:10, Robert A. Brooks wrote:
> On 4/7/2022 7:24 PM, Rich Jordan wrote:
>> Customer has decided to turn off "legacy" DECnet support on their
>> network.
>> They currently use DECnet for copies between two nonclustered integrity
>> servers that are on different IP subnets/VLANs and have Cisco doing
>> whatever
>> magic it does to make DECnet appear local. THis is Phase IV, not
>> DECnet over
>> IP.
>
> Install DECnet-Plus (IA64) or DECnet/OSI (Alpha), and enable DECnet over
> TCP/IP.
>
> That's likely the easiest solution, since it'll require no application
> change.

Meh. I wouldn't even go there. I'd just install Multinet and do DECnet
over IP, and staying on Phase IV. Phase V is just going to make a mess
for no gain.

Johnny

On 2022-04-08, Rich Jordan <jordan@ccs4vms.com> wrote:
>
> Thank you all for responses. I'm not excited about the thought of installing Phase V mainly because it has been 25+ years since I even looked at it. I can see if it is an option. At least that I can test here at our office.
>

Two potential problems with that:

1) They may not like the idea of you running an older less secure
protocol over what is really just a TCP/IP tunnel.

2) Does DECnet Phase V itself offer any form of encryption ?
If not, they may not like the idea of unencrypted traffic running
on their network.

> Losing current DECnet transit is not negotiable apparently; we lost that argument but that choice is with the customer and their network vendor/support; we and the VMS boxes are a side issue that just needs to deal with the change.
>

If they are reducing the number of protocols running on their network
for security reasons, and especially getting rid of the older less
secure ones for that same reason, that's a very reasonable thing for
them to decide to do.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2022-04-07, Dave Froble <davef@tsoft-inc.com> wrote:
> On 4/7/2022 7:24 PM, Rich Jordan wrote:
>> Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
>>
>> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
>>
>> Thanks for any info
>>
>
> Tell customer to re-think ...
>

The customer may be doing this precisely because they do want to
make their network more secure.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 4/8/2022 1:51 PM, Simon Clubley wrote:
> On 2022-04-07, Dave Froble <davef@tsoft-inc.com> wrote:
>> On 4/7/2022 7:24 PM, Rich Jordan wrote:
>>> Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
>>>
>>> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
>>>
>>> Thanks for any info
>>>
>>
>> Tell customer to re-think ...
>>
>
> The customer may be doing this precisely because they do want to
> make their network more secure.
>
> Simon.
>

Security or nice helpful tools, choose one ...

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On Friday, April 8, 2022 at 11:30:49 AM UTC-4, Rich Jordan wrote:
> On Friday, April 8, 2022 at 6:24:37 AM UTC-5, geze...@rlgsc.com wrote:
> > On Thursday, April 7, 2022 at 7:24:03 PM UTC-4, Rich Jordan wrote:
> > > Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
> > >
> > > Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
> > >
> > > Thanks for any info
> > Rich,
> >
> > As Robert noted, DECnet Phase V over IP is a viable choice. In situations where the client did not want to do that change, I have used sftp quite effectively, often in concert with ZIP/UNZIP.
> >
> > As an example, in one quick test, I was able to transfer more than a terabyte of data in a reasonable time (I do not have the precise numbers in easy reach at the moment). The context was non-tape backup. The process was:
> > - Do BACKUP/IMAGE to a scratch device
> > - use ZIP "-V" to compress the BACKUP save set. Your mileage will vary, I was able to get approximately 90% size reduction.
> > - Use sftp to copy the resulting ZIP files to the destination machine
> > - UNZIP the transferred files
> > - use BACKUP to restore the saveset on the destination machine
> >
> > The above was run in multiple BATCH jobs on both sides.
> >
> > - Bob Gezelter, http://www.rlgsc.com
> Thank you all for responses. I'm not excited about the thought of installing Phase V mainly because it has been 25+ years since I even looked at it. I can see if it is an option. At least that I can test here at our office.
>
> Losing current DECnet transit is not negotiable apparently; we lost that argument but that choice is with the customer and their network vendor/support; we and the VMS boxes are a side issue that just needs to deal with the change.
>
> As noted we already do some data moves by doing backup to local disk savesets (using the /DATA_FORMAT=COMPRESS option) then FTP'ing those sets to a PC. Fully tested so we know we can retrieve those savesets, fix the attributes, and restore them because doing even a zero compression encapsulating ZIP takes too much time for the larger savesets (ZIP would preserve the backup saveset file attributes).
>
> The current VMS to VMS is done with DECnet copies of individual files, or backup of folders/trees or bulk files to remote saveset via DECnet that gets restored on the remote system.
Rich,

If you are transferring trees or subtrees, the /IMAGE is completely unnecessary.

The reason I used ZIP was to preserve the OpenVMS file attributes when transferring. OpenVMS ZIP/UNZIP can preserve file attributes without problem.

How large are the data volumes being transferred.

- Bob Gezelter, http://www.rlgsc.com

> [...] OpenVMS ZIP/UNZIP can preserve file attributes without problem.

They can do it _with_ problems, too. There have been bugs, and there
are unsupported file-system features which BACKUP handles better.

help set file /enter

Also, idiosyncratic names like "a^.b." might get altered.
Everything's complicated.

On Friday, April 8, 2022 at 11:33:45 PM UTC-4, Steven Schweda wrote:
> > [...] OpenVMS ZIP/UNZIP can preserve file attributes without problem.
>
> They can do it _with_ problems, too. There have been bugs, and there
> are unsupported file-system features which BACKUP handles better.
>
> help set file /enter
>
> Also, idiosyncratic names like "a^.b." might get altered.
> Everything's complicated.
Steve,

Quite. Which is my preferred solution is to do a BACKUP to a saveset and then ZIP/UNZIP the saveset.

- Bob Gezelter, http://www.rlgsc.com

On Friday, April 8, 2022 at 10:56:00 AM UTC-5, Steven Schweda wrote:
> > [...] backup to local disk savesets (using the /DATA_FORMAT=COMPRESS
> > option) [...]
>
> > [...] fix the attributes, and restore them because doing even a zero
> > compression encapsulating ZIP takes too much time for the larger
> > savesets (ZIP would preserve the backup saveset file attributes).
> If all you wanted Zip+UnZip to do was preserve the attributes of a
> BACKUP save set, then I wouldn't bother. There are enough DCL scripts
> floating around which can restore them in near-zero time.
>
> It might be worth a quick experiment to see whether BACKUP without
> compression followed by Zip with compression makes any sense.
> > The current VMS to VMS is done with DECnet copies of individual files,
> > [...]
>
> The principal advantage of new DECnet (ising IP) is that it works
> like old DECnet.

did a couple of quick tests. For 25-50GB backup savesets on the RX2800s, run to local disks with backup parameters optimized (and RMS), composite time for compressed backup and ZIP with no compression was about 15% shorter than backup without compression and ZIP "-V6" (from 6 to 9 got very little more compression in our test but added a fair amount of ZIP time). The system disk backup done while booted from an alternate boot disk, which is our smallest disk, the times were almost even. So for now we're sticking with backup doing the compression.

On Friday, April 8, 2022 at 6:17:59 PM UTC-5, geze...@rlgsc.com wrote:
> On Friday, April 8, 2022 at 11:30:49 AM UTC-4, Rich Jordan wrote:
> > On Friday, April 8, 2022 at 6:24:37 AM UTC-5, geze...@rlgsc.com wrote:
> > > On Thursday, April 7, 2022 at 7:24:03 PM UTC-4, Rich Jordan wrote:
> > > > Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
> > > >
> > > > Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
> > > >
> > > > Thanks for any info
> > > Rich,
> > >
> > > As Robert noted, DECnet Phase V over IP is a viable choice. In situations where the client did not want to do that change, I have used sftp quite effectively, often in concert with ZIP/UNZIP.
> > >
> > > As an example, in one quick test, I was able to transfer more than a terabyte of data in a reasonable time (I do not have the precise numbers in easy reach at the moment). The context was non-tape backup. The process was:
> > > - Do BACKUP/IMAGE to a scratch device
> > > - use ZIP "-V" to compress the BACKUP save set. Your mileage will vary, I was able to get approximately 90% size reduction.
> > > - Use sftp to copy the resulting ZIP files to the destination machine
> > > - UNZIP the transferred files
> > > - use BACKUP to restore the saveset on the destination machine
> > >
> > > The above was run in multiple BATCH jobs on both sides.
> > >
> > > - Bob Gezelter, http://www.rlgsc.com
> > Thank you all for responses. I'm not excited about the thought of installing Phase V mainly because it has been 25+ years since I even looked at it. I can see if it is an option. At least that I can test here at our office.
> >
> > Losing current DECnet transit is not negotiable apparently; we lost that argument but that choice is with the customer and their network vendor/support; we and the VMS boxes are a side issue that just needs to deal with the change.
> >
> > As noted we already do some data moves by doing backup to local disk savesets (using the /DATA_FORMAT=COMPRESS option) then FTP'ing those sets to a PC. Fully tested so we know we can retrieve those savesets, fix the attributes, and restore them because doing even a zero compression encapsulating ZIP takes too much time for the larger savesets (ZIP would preserve the backup saveset file attributes).
> >
> > The current VMS to VMS is done with DECnet copies of individual files, or backup of folders/trees or bulk files to remote saveset via DECnet that gets restored on the remote system.
> Rich,
>
> If you are transferring trees or subtrees, the /IMAGE is completely unnecessary.
>
> The reason I used ZIP was to preserve the OpenVMS file attributes when transferring. OpenVMS ZIP/UNZIP can preserve file attributes without problem.
>
> How large are the data volumes being transferred.
> - Bob Gezelter, http://www.rlgsc.com

Sorry for delays, got pulled into someone else's mega-project for a while.

To be clear the image backups are made and transferred to a PC server so they go offsite. Data transfer from one VMS machine to the other is done using non-image backups, then transfer the saveset, or if there is only a small batch of files, backup with the target being the remote system using proper syntax, or even just copy.

Its not impossible that we might extremely occasionally transfer an image backup to the remote system (or make the image backup with the saveset on the remote system) but that would be a serious recovery operation, not daily data updates.

We only used ZIP on backup savesets to preserve saveset attributes when they were transferred to PC servers, not for any VMS to VMS usage.

I know ZIP can be used to create a local archive of files or folders/trees with all file attributes saved (then we transfer the archive); I guess its worth trying but there is the _occasional_ need to update one or two major files using /ignore=interlock to get them to the backup system. In these cases the file is held open by processing until their staff fixes the problem that caused the hold but no changes are occurring so the data is consistent. THis can happen when processing runs into a problem, and the customer's staff is alerted and probably working the issue but we still need to get that file moved. ZIP can't do that.

And no, telling their devs to change that isn't going to go anywhere. Gotta work with what we got.

Still wishing they'd jumped for a cluster license and shadowing.

On 14/04/2022 15:59, Rich Jordan wrote:
> On Friday, April 8, 2022 at 10:56:00 AM UTC-5, Steven Schweda wrote:
>>> [...] backup to local disk savesets (using the /DATA_FORMAT=COMPRESS
>>> option) [...]
>>
>>> [...] fix the attributes, and restore them because doing even a zero
>>> compression encapsulating ZIP takes too much time for the larger
>>> savesets (ZIP would preserve the backup saveset file attributes).
>> If all you wanted Zip+UnZip to do was preserve the attributes of a
>> BACKUP save set, then I wouldn't bother. There are enough DCL scripts
>> floating around which can restore them in near-zero time.
>>
>> It might be worth a quick experiment to see whether BACKUP without
>> compression followed by Zip with compression makes any sense.
>>> The current VMS to VMS is done with DECnet copies of individual files,
>>> [...]
>>
>> The principal advantage of new DECnet (ising IP) is that it works
>> like old DECnet.
>
>
> did a couple of quick tests. For 25-50GB backup savesets on the RX2800s, run to local disks with backup parameters optimized (and RMS), composite time for compressed backup and ZIP with no compression was about 15% shorter than backup without compression and ZIP "-V6" (from 6 to 9 got very little more compression in our test but added a fair amount of ZIP time). The system disk backup done while booted from an alternate boot disk, which is our smallest disk, the times were almost even. So for now we're sticking with backup doing the compression.
>

Have you tried zip -3 (or -2) ? You get a little less compression, but a
much lower CPU load/elapsed time

--
Chris

On Thursday, April 14, 2022 at 10:44:02 AM UTC-5, Chris Townley wrote:
> On 14/04/2022 15:59, Rich Jordan wrote:
> > On Friday, April 8, 2022 at 10:56:00 AM UTC-5, Steven Schweda wrote:
> >>> [...] backup to local disk savesets (using the /DATA_FORMAT=COMPRESS
> >>> option) [...]
> >>
> >>> [...] fix the attributes, and restore them because doing even a zero
> >>> compression encapsulating ZIP takes too much time for the larger
> >>> savesets (ZIP would preserve the backup saveset file attributes).
> >> If all you wanted Zip+UnZip to do was preserve the attributes of a
> >> BACKUP save set, then I wouldn't bother. There are enough DCL scripts
> >> floating around which can restore them in near-zero time.
> >>
> >> It might be worth a quick experiment to see whether BACKUP without
> >> compression followed by Zip with compression makes any sense.
> >>> The current VMS to VMS is done with DECnet copies of individual files,
> >>> [...]
> >>
> >> The principal advantage of new DECnet (ising IP) is that it works
> >> like old DECnet.
> >
> >
> > did a couple of quick tests. For 25-50GB backup savesets on the RX2800s, run to local disks with backup parameters optimized (and RMS), composite time for compressed backup and ZIP with no compression was about 15% shorter than backup without compression and ZIP "-V6" (from 6 to 9 got very little more compression in our test but added a fair amount of ZIP time). The system disk backup done while booted from an alternate boot disk, which is our smallest disk, the times were almost even. So for now we're sticking with backup doing the compression.
> >
> Have you tried zip -3 (or -2) ? You get a little less compression, but a
> much lower CPU load/elapsed time
>
> --
> Chris

Did, don't recall the numbers. 5 and 6 were the breakeven for time including the transfer after backup and ZIP were completed. But even -0 took quite a while even with the optimizations in place.

On Thursday, April 7, 2022 at 6:24:03 PM UTC-5, Rich Jordan wrote:
> Customer has decided to turn off "legacy" DECnet support on their network.. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
>
> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
>
> Thanks for any info

Well, tried various combinations of the TCP protocol settings (like NODELAY_ACK), the FTP logicals for initial and append allocations, the FTP window size, etc. FTP is doing 1.4 to 1.64Mbps (that last due to /NODELAY_ACK) and won't go any higher. SFTP is supposed to be slower but we'll try it anyway. For now going to locate iPerf ports for VMS and run some tests with it to see what actual throughput we are getting between the sites.

On 04/14/2022 11:31 AM, Rich Jordan wrote:
> On Thursday, April 7, 2022 at 6:24:03 PM UTC-5, Rich Jordan wrote:
>> Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
>>
>> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
>>
>> Thanks for any info
>
>
> Well, tried various combinations of the TCP protocol settings (like NODELAY_ACK), the FTP logicals for initial and append allocations, the FTP window size, etc. FTP is doing 1.4 to 1.64Mbps (that last due to /NODELAY_ACK) and won't go any higher. SFTP is supposed to be slower but we'll try it anyway. For now going to locate iPerf ports for VMS and run some tests with it to see what actual throughput we are getting between the sites.
>

We had to move large save sets over FTP when a system was replaced with
another system at a different location and we used the following setting
to speed up the transfers:

$ TCPIP
sysconfig -r socket sb_max=2000000
sysconfig -r socket somaxconn=10240
sysconfig -r socket sominconn=10240
sysconfig -r inet tcp_sendspace=300000 tcp_recvspace=300000
sysconfig -q socket
sysconfig -q inet tcp_sendspace tcp_recvspace

I know I didn't figure this out but I don't remember where I found these
settings.

Jeff
www.digitalsynergyinc.com

> We only used ZIP on backup savesets to preserve saveset attributes
> when they were transferred to PC servers, not for any VMS to VMS usage.

Just to repeat, using "zip -0" simply to preserve the attributes of a
BACKUP save set seems to me like excessive I/O to preserve a record
format and a record length, both of which could be restored in
approximately no time using SET FILE /ATTRIBUTE (or BACKUP /REPAIR).
DCL scripts exist to do it automatically. See, for example:

http://antinode.info/dec/sw/fixrec.html

On Thursday, April 14, 2022 at 1:52:54 PM UTC-5, Jeffrey H. Coffield wrote:
> On 04/14/2022 11:31 AM, Rich Jordan wrote:
> > On Thursday, April 7, 2022 at 6:24:03 PM UTC-5, Rich Jordan wrote:
> >> Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
> >>
> >> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
> >>
> >> Thanks for any info
> >
> >
> > Well, tried various combinations of the TCP protocol settings (like NODELAY_ACK), the FTP logicals for initial and append allocations, the FTP window size, etc. FTP is doing 1.4 to 1.64Mbps (that last due to /NODELAY_ACK) and won't go any higher. SFTP is supposed to be slower but we'll try it anyway. For now going to locate iPerf ports for VMS and run some tests with it to see what actual throughput we are getting between the sites.
> >
> We had to move large save sets over FTP when a system was replaced with
> another system at a different location and we used the following setting
> to speed up the transfers:
>
> $ TCPIP
> sysconfig -r socket sb_max=2000000
> sysconfig -r socket somaxconn=10240
> sysconfig -r socket sominconn=10240
> sysconfig -r inet tcp_sendspace=300000 tcp_recvspace=300000
> sysconfig -q socket
> sysconfig -q inet tcp_sendspace tcp_recvspace
>
> I know I didn't figure this out but I don't remember where I found these
> settings.
>
> Jeff
> www.digitalsynergyinc.com

Jeff, thanks will take a shot with those.

I'm currently testing remote VMS ftp to/from the PC server that the main system sends its nightly backup savesets to, and getting about 4.8 Mbps, so about 3x the VMS to VMS rate. The main system gets about 15Mbps to that PC but they are in the same datacenter. Still slow for gigabit but it fits their timing windows.

We'll see if the above parameters make a difference between the remote VMS server and the PC first; I can't make changes to the production box until the weekend, especially if a reboot is needed (will have to check)

Thanks again

On 04/14/2022 01:17 PM, Rich Jordan wrote:
> On Thursday, April 14, 2022 at 1:52:54 PM UTC-5, Jeffrey H. Coffield wrote:
>> On 04/14/2022 11:31 AM, Rich Jordan wrote:
>>> On Thursday, April 7, 2022 at 6:24:03 PM UTC-5, Rich Jordan wrote:
>>>> Customer has decided to turn off "legacy" DECnet support on their network. They currently use DECnet for copies between two nonclustered integrity servers that are on different IP subnets/VLANs and have Cisco doing whatever magic it does to make DECnet appear local. THis is Phase IV, not DECnet over IP.
>>>>
>>>> Is there any documentation on relative performance for bulk data transfer with preliminary data archiving (ie ZIP or backup to a local archive file so not transferring lots of individual files) on VMS using FTP/SFTP versus using NFS as the transfer mechanism? I know details matter and I don't have them yet but is one likely to be enough different that it is worth pursuing a test? Which won't be trivial given equipment availability...
>>>>
>>>> Thanks for any info
>>>
>>>
>>> Well, tried various combinations of the TCP protocol settings (like NODELAY_ACK), the FTP logicals for initial and append allocations, the FTP window size, etc. FTP is doing 1.4 to 1.64Mbps (that last due to /NODELAY_ACK) and won't go any higher. SFTP is supposed to be slower but we'll try it anyway. For now going to locate iPerf ports for VMS and run some tests with it to see what actual throughput we are getting between the sites.
>>>
>> We had to move large save sets over FTP when a system was replaced with
>> another system at a different location and we used the following setting
>> to speed up the transfers:
>>
>> $ TCPIP
>> sysconfig -r socket sb_max=2000000
>> sysconfig -r socket somaxconn=10240
>> sysconfig -r socket sominconn=10240
>> sysconfig -r inet tcp_sendspace=300000 tcp_recvspace=300000
>> sysconfig -q socket
>> sysconfig -q inet tcp_sendspace tcp_recvspace
>>
>> I know I didn't figure this out but I don't remember where I found these
>> settings.
>>
>> Jeff
>> www.digitalsynergyinc.com
>
> Jeff, thanks will take a shot with those.
>
> I'm currently testing remote VMS ftp to/from the PC server that the main system sends its nightly backup savesets to, and getting about 4.8 Mbps, so about 3x the VMS to VMS rate. The main system gets about 15Mbps to that PC but they are in the same datacenter. Still slow for gigabit but it fits their timing windows.
>
> We'll see if the above parameters make a difference between the remote VMS server and the PC first; I can't make changes to the production box until the weekend, especially if a reboot is needed (will have to check)
>
> Thanks again
>

No reboot necessary. I set up those commands to be executed on system
startup

Jeff

On 15/4/22 4:22 am, Jeffrey H. Coffield wrote:
>
>
> On 04/14/2022 11:31 AM, Rich Jordan wrote:
>> On Thursday, April 7, 2022 at 6:24:03 PM UTC-5, Rich Jordan wrote:
8< snip 8<> We had to move large save sets over FTP when a system was
replaced with
> another system at a different location and we used the following setting
> to speed up the transfers:
>
> $       TCPIP
> sysconfig -r socket sb_max=2000000
> sysconfig -r socket somaxconn=10240
> sysconfig -r socket sominconn=10240
> sysconfig -r inet tcp_sendspace=300000 tcp_recvspace=300000
> sysconfig -q socket
> sysconfig -q inet tcp_sendspace tcp_recvspace
>
> I know I didn't figure this out but I don't remember where I found these
> settings.

A quick consult with Dr Google shows lamentably few hits for

"openvms sysconfig -r socket sb_max"

and the rest (though some which may be of interest).

I also notice an online manual

"HP TCP/IP Services for OpenVMS Tuning and Troubleshooting"

available from various sites, e.g. (quoted to prevent wrapping)

> https://www.digiater.nl/openvms/doc/alpha-v8.3/83final/documentation/pdf/aa_rn1vb_te.pdf

which seems to be missing from the VSI collection

https://docs.vmssoftware.com/

Are the directives and recommendations still applicable to VSI TCP/IP
Services 5 and 6?

> Jeff
> www.digitalsynergyinc.com

--
Anyone, who using social-media, forms an opinion regarding anything
other than the relative cuteness of this or that puppy-dog, needs
seriously to examine their critical thinking.

On Thursday, April 14, 2022 at 7:12:07 PM UTC-5, Mark Daniel wrote:
> On 15/4/22 4:22 am, Jeffrey H. Coffield wrote:
> >
> >
> > On 04/14/2022 11:31 AM, Rich Jordan wrote:
> >> On Thursday, April 7, 2022 at 6:24:03 PM UTC-5, Rich Jordan wrote:
> 8< snip 8<> We had to move large save sets over FTP when a system was
> replaced with
> > another system at a different location and we used the following setting
> > to speed up the transfers:
> >
> > $ TCPIP
> > sysconfig -r socket sb_max=2000000
> > sysconfig -r socket somaxconn=10240
> > sysconfig -r socket sominconn=10240
> > sysconfig -r inet tcp_sendspace=300000 tcp_recvspace=300000
> > sysconfig -q socket
> > sysconfig -q inet tcp_sendspace tcp_recvspace
> >
> > I know I didn't figure this out but I don't remember where I found these
> > settings.
> A quick consult with Dr Google shows lamentably few hits for
>
> "openvms sysconfig -r socket sb_max"
>
> and the rest (though some which may be of interest).
>
> I also notice an online manual
>
> "HP TCP/IP Services for OpenVMS Tuning and Troubleshooting"
>
> available from various sites, e.g. (quoted to prevent wrapping)
>
> > https://www.digiater.nl/openvms/doc/alpha-v8.3/83final/documentation/pdf/aa_rn1vb_te.pdf
>
> which seems to be missing from the VSI collection
>
> https://docs.vmssoftware.com/
>
> Are the directives and recommendations still applicable to VSI TCP/IP
> Services 5 and 6?
>
> > Jeff
> > www.digitalsynergyinc.com
>
> --
> Anyone, who using social-media, forms an opinion regarding anything
> other than the relative cuteness of this or that puppy-dog, needs
> seriously to examine their critical thinking.

I actually did go through the TCPIP troubleshooting manual and tried a couple of the suggestions. Benefits were minimal and could have just been random impact of actual network load at the time of testing. Could not do jumbo packets (but also found no reference to indicate that FTP would benefit from jumbo packets) because the intermediate network doesn't support them.

Main VMS server and PC backup server on the same LAN, second VMS server at the remote site. The test saveset was one of the small ones; I'll need to get times on the three much larger ones.

Main VMS to local PC backup server transfers a 6.7M block backup file in 3 minutes 41 seconds
Main VMS to remote VMS transfers same file in 32 minutes 21 seconds (push or pull)
Remote VMS server pulls the same backup file from the PC backup server in 10 minutes 30 seconds.

So it is faster to relay backups (and presumably other data of any significant size) through the PC backup server than doing it directly VMS to VMS.

The sysconfig changes above did not make a measurable difference when set on either VMS system or both; maybe 2% difference in time that was likely more due to network usage.

Setting TCP protocol DELAY_ACK to disabled made a few percentage point difference overall but still nothing major.

For now I guess we'll have to live with it. I'll try setting up sftp/scp to test but everything I've read says those will be slower.