I've been trying to find a way to read tapes written on other systems
that use a block size somewhat larger than 64K (usually 128K). The 64K
limit on VMS *seems* to be caused by using a 2-byte field for the DMA
buffer size but perhaps there is more.

Any idea on how involved it would be to increase the maximum magtape
block size for custom code (I'm not worried about increasing it for
backup, etc.)?

Mark Berryman

On Thursday, May 19, 2022 at 5:27:29 PM UTC-4, Mark Berryman wrote:
> I've been trying to find a way to read tapes written on other systems
> that use a block size somewhat larger than 64K (usually 128K). The 64K
> limit on VMS *seems* to be caused by using a 2-byte field for the DMA
> buffer size but perhaps there is more.
>
> Any idea on how involved it would be to increase the maximum magtape
> block size for custom code (I'm not worried about increasing it for
> backup, etc.)?
>
> Mark Berryman

I would look for answers to two questions:
1) Is the blocksize known?
2) Is the size consistent for the entire tape?

If the answer is yes, mount the tape foreign and read a fixed blocksize that VMS can handle and build up the file as needed on the output side.
I recall having to do this with tapes many years ago and had a FORTRAN program to do it.

Dan

On 5/19/22 2:27 PM, Mark Berryman wrote:
> I've been trying to find a way to read tapes written on other systems
> that use a block size somewhat larger than 64K (usually 128K).  The 64K
> limit on VMS *seems* to be caused by using a 2-byte field for the DMA
> buffer size but perhaps there is more.
>
> Any idea on how involved it would be to increase the maximum magtape
> block size for custom code (I'm not worried about increasing it for
> backup, etc.)?

I have seen the same problem with backup tapes from Windows machines,
but our solution was to simply change the Windows software to not write
such big blocks. As I recall, the problem was that the VMS I/O driver
simply couldn't handle more than a 64K-2 byte record size, and it wasn't
worth the effort to re-write the driver and QIO format.

Back in 1970 I had a somewhat similar problem. The tapes (from some
sort of oil field equipment) had been written on a Kennedy incremental
recorder and one record filled the entire 600' tape. We had two tape
drives on our lab mini (a Varian 620/i), one 45 ips and one 75 ips. I
wrote a program to read the gapless tape on the 45 ips drive using
programed I/O, fill one of two buffers, then fire off a DMA write on the
75 ips drive. The higher speed allowed it to keep up while still
writing inter-record gaps.

Totally unrelated to that, but also interesting, was a tape whose
records had to be processed in reverse order. The program ran on
the big corporate mainframe, an IBM 360/65. All it did was skip
forward to EOF, then do Backspace-Backspace-Read until it got to
BOT. It worked perfectly, but it totally freaked out the operators
in the data center. The noise from the 2420 tape drive made them
think it was about to fly apart! After the first run we were asked
to please warn them, and only run it when certain operators were
on duty. The IBM FE wanted to declare it "abuse" but the drive
never failed.

Alan Frisbie

On 2022-05-20 00:01, abrsvc wrote:
> On Thursday, May 19, 2022 at 5:27:29 PM UTC-4, Mark Berryman wrote:
>> I've been trying to find a way to read tapes written on other systems
>> that use a block size somewhat larger than 64K (usually 128K). The 64K
>> limit on VMS *seems* to be caused by using a 2-byte field for the DMA
>> buffer size but perhaps there is more.
>>
>> Any idea on how involved it would be to increase the maximum magtape
>> block size for custom code (I'm not worried about increasing it for
>> backup, etc.)?
>>
>> Mark Berryman
>
> I would look for answers to two questions:
> 1) Is the blocksize known?
> 2) Is the size consistent for the entire tape?
>
> If the answer is yes, mount the tape foreign and read a fixed blocksize that VMS can handle and build up the file as needed on the output side.
> I recall having to do this with tapes many years ago and had a FORTRAN program to do it.

That won't help. You cannot read a block larger than 64K under VMS. And
with tapes, after reading a block, even if not complete, the drive then
moves to the start of the next block, and the remaining part of the
previous block is just dropped.

Mounting it foreign don't actually change anything about this.

Johnny

On 05/20/22 19:17, Johnny Billquist wrote:
> On 2022-05-20 00:01, abrsvc wrote:
>> On Thursday, May 19, 2022 at 5:27:29 PM UTC-4, Mark Berryman wrote:
>>> I've been trying to find a way to read tapes written on other systems
>>> that use a block size somewhat larger than 64K (usually 128K). The 64K
>>> limit on VMS *seems* to be caused by using a 2-byte field for the DMA
>>> buffer size but perhaps there is more.
>>>
>>> Any idea on how involved it would be to increase the maximum magtape
>>> block size for custom code (I'm not worried about increasing it for
>>> backup, etc.)?
>>>
>>> Mark Berryman
>>
>> I would look for answers to two questions:
>> 1) Is the blocksize known?
>> 2) Is the size consistent for the entire tape?
>>
>> If the answer is yes, mount the tape foreign and read a fixed
>> blocksize that VMS can handle and build up the file as needed on the
>> output side.
>> I recall having to do this with tapes many years ago and had a FORTRAN
>> program to do it.
>
> That won't help. You cannot read a block larger than 64K under VMS. And
> with tapes, after reading a block, even if not complete, the drive then
> moves to the start of the next block, and the remaining part of the
> previous block is just dropped.
>
> Mounting it foreign don't actually change anything about this.
>
> Johnny

Interesting. Again, we are reminded of the joy of unix, tar, dd etc...

Chris

On Thursday, May 19, 2022 at 6:43:38 PM UTC-7, Alan Frisbie wrote:

(snip)

> Back in 1970 I had a somewhat similar problem. The tapes (from some
> sort of oil field equipment) had been written on a Kennedy incremental
> recorder and one record filled the entire 600' tape. We had two tape
> drives on our lab mini (a Varian 620/i), one 45 ips and one 75 ips. I
> wrote a program to read the gapless tape on the 45 ips drive using
> programed I/O, fill one of two buffers, then fire off a DMA write on the
> 75 ips drive. The higher speed allowed it to keep up while still
> writing inter-record gaps.

There was a story once about someone writing such a tape
on an IBM S/360 machine. The OS won't do that, but if you
write your own channel program, and use data chaining, it is
supposed to be possible. Data chaining allows multiple
channel commands to write to a single data block.
You could even make a loop of channel commands to do it.

> Totally unrelated to that, but also interesting, was a tape whose
> records had to be processed in reverse order. The program ran on
> the big corporate mainframe, an IBM 360/65. All it did was skip
> forward to EOF, then do Backspace-Backspace-Read until it got to
> BOT. It worked perfectly, but it totally freaked out the operators
> in the data center. The noise from the 2420 tape drive made them
> think it was about to fly apart! After the first run we were asked
> to please warn them, and only run it when certain operators were
> on duty. The IBM FE wanted to declare it "abuse" but the drive
> never failed.

Someone goofed. All the S/360 tape drives I know of, have the
ability to read backwards. This was mostly used by sort
programs, that would write records to a temporary tape file,
and then immediately read them back. There are external sort
algorithms specifically designed to work that way.

The way read backwards works, is that the program supplies
the address of the end of the buffer, and it is then filled up from
the end, to the beginning. So, the data comes out in the right
way, but records come off the tape, last one first.

I am not sure how many more modern drives have this ability.
It is difficult for helical scan drives like DAT and Exabyte, so
I suspect that they don't do it. I believe that the 3480 and related
drives, and the Ultrium successors can, but haven't checked lately.

Otherwise, I believe that OS/360 limits blocksize to 32767, though
channel commands have a 16 bit unsigned count. There are no
instructions for 16 bit unsigned arithmetic for S/360, though it
isn't hard to get around. In the early OS/360 days, memory was
small, and every byte counted. On a 64K byte machine, there
wasn't much need to write larger blocks.

I believe that has been changed by the time of z/OS, but I am
not so sure how it is done. Many of the old control blocks,
such as DCB, are still there with 16 bit fields.

On 2022-05-19 21:27:25 +0000, Mark Berryman said:

> I've been trying to find a way to read tapes written on other systems
> that use a block size somewhat larger than 64K (usually 128K). The 64K
> limit on VMS *seems* to be caused by using a 2-byte field for the DMA
> buffer size but perhaps there is more.
>
> Any idea on how involved it would be to increase the maximum magtape
> block size for custom code (I'm not worried about increasing it for
> backup, etc.)?

I'd try calling IO$_DIAGNOSE directly, and passing along the necessary
SCSI commands, and reading the responses from the tape drive.

Tapes aren't particularly complicated, as SCSI goes.

You'll probably wedge something the first few times, so don't test on a
production server.

Related examples:

https://www.digiater.nl/openvms/decus/vms95a/net95a/gktest.c

https://www.digiater.nl/openvms/decus/vmslt07a/vu/mtools-vms_notes.txt

T10DEF has some of the related SCSI command and response data
structures, though is probably sparse on magtape commands:
librar/extr=T10DEF/out=x.x sys$share:sys$lib_c.tlb

--
Pure Personal Opinion | HoffmanLabs LLC

On 2022-05-21 00:10, chris wrote:
> On 05/20/22 19:17, Johnny Billquist wrote:
>> On 2022-05-20 00:01, abrsvc wrote:
>>> On Thursday, May 19, 2022 at 5:27:29 PM UTC-4, Mark Berryman wrote:
>>>> I've been trying to find a way to read tapes written on other systems
>>>> that use a block size somewhat larger than 64K (usually 128K). The 64K
>>>> limit on VMS *seems* to be caused by using a 2-byte field for the DMA
>>>> buffer size but perhaps there is more.
>>>>
>>>> Any idea on how involved it would be to increase the maximum magtape
>>>> block size for custom code (I'm not worried about increasing it for
>>>> backup, etc.)?
>>>>
>>>> Mark Berryman
>>>
>>> I would look for answers to two questions:
>>> 1) Is the blocksize known?
>>> 2) Is the size consistent for the entire tape?
>>>
>>> If the answer is yes, mount the tape foreign and read a fixed
>>> blocksize that VMS can handle and build up the file as needed on the
>>> output side.
>>> I recall having to do this with tapes many years ago and had a FORTRAN
>>> program to do it.
>>
>> That won't help. You cannot read a block larger than 64K under VMS. And
>> with tapes, after reading a block, even if not complete, the drive then
>> moves to the start of the next block, and the remaining part of the
>> previous block is just dropped.
>>
>> Mounting it foreign don't actually change anything about this.
>>
>> Johnny
>
> Interesting. Again, we are reminded of the joy of unix, tar, dd etc...

Why do you think it is any different there?
Try dd on a tape, giving bs=1024 for a tar tape, and observe how your
copy is missing 90% of the content.

There might be controllers/hardware that allow larger blocks, but it's
quite common to see a limit of 64K. Also, if block lengths vary, you're
in deep doo-doo in Unix... You need to write your own programs to deal
with it.

Johnny

On Tuesday, May 24, 2022 at 10:28:41 AM UTC-7, Johnny Billquist wrote:

(snip)
> Why do you think it is any different there?
> Try dd on a tape, giving bs=1024 for a tar tape, and observe how your
> copy is missing 90% of the content.
> There might be controllers/hardware that allow larger blocks, but it's
> quite common to see a limit of 64K. Also, if block lengths vary, you're
> in deep doo-doo in Unix... You need to write your own programs to deal
There are some interesting things that can be done with dd.

if you use bs=(large number) then dd reads blocks up to and including
(large number) bytes, and then writes them out again. You can make
a copy of a VB or VBS tape, without knowing the blocking.

If you use ibs=number1 obs=number2, then dd reads blocks of up
to number1 bytes, concatenates them together (if needed) and then
writes out blocks of number2 bytes (except for the last one).

This allows reblocking for some cases, especially FB data.