Completely escaped my notice, but last September was *HUMONGOUS*’s 10th Anniversary.

I’ve finally completed my clean up of the SIG/M disks. I focused on four things: complete, clean, uncorrupted, and "original”. None of the known sources had all volumes; many of the filenames were corrupted; and several hundred files didn’t match their CRCs. I’ve cleaned out all of the accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames, and fixed all but a handful of the CRC problems. My goal was to produce something as close to "original" as possible.

The collection is available in LRB, ARK and IBM-3740 disk image formats at http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.

I’ve encapsulated everything into a BASH shell script which is available at github.com/cpmarchives. Have a look there if you’re interested in all the details, or you can run the script to recreate everything yourself.

Nathanael, gHUMONGOUS* CP/M Archives.

On Thursday, 7 April 2022 at 13:05:13 UTC+1, Nathanael wrote:
> Completely escaped my notice, but last September was *HUMONGOUS*’s 10th Anniversary.
>
> I’ve finally completed my clean up of the SIG/M disks. I focused on four things: complete, clean, uncorrupted, and "original”. None of the known sources had all volumes; many of the filenames were corrupted; and several hundred files didn’t match their CRCs. I’ve cleaned out all of the accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames, and fixed all but a handful of the CRC problems. My goal was to produce something as close to "original" as possible.
>
> The collection is available in LRB, ARK and IBM-3740 disk image formats at http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
>
> I’ve encapsulated everything into a BASH shell script which is available at github.com/cpmarchives. Have a look there if you’re interested in all the details, or you can run the script to recreate everything yourself.
>
>
> Nathanael, gHUMONGOUS* CP/M Archives.
Nathanael
Browsing your BASH script I notice that you rename files post creation of lbr files. I am not sure whether you are aware but he current version of mklbr supports renaming files as they are included by providing the stored name as part of the recipe. This allows for _ to / mapping and for more obscure mappings e.g. reserved filenames in Windows and illegal Windows/Unix filename characters. It also supports adding explicit file and library time stamps. The README.md file on my github site provides details.
Mark

On Thursday, April 7, 2022 at 9:38:08 PM UTC+8, ogd...@gmail.com wrote:
> On Thursday, 7 April 2022 at 13:05:13 UTC+1, Nathanael wrote:
> > Completely escaped my notice, but last September was *HUMONGOUS*’s 10th Anniversary.
> >
> > I’ve finally completed my clean up of the SIG/M disks. I focused on four things: complete, clean, uncorrupted, and "original”. None of the known sources had all volumes; many of the filenames were corrupted; and several hundred files didn’t match their CRCs. I’ve cleaned out all of the accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames, and fixed all but a handful of the CRC problems. My goal was to produce something as close to "original" as possible.
> >
> > The collection is available in LRB, ARK and IBM-3740 disk image formats at http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> >
> > I’ve encapsulated everything into a BASH shell script which is available at github.com/cpmarchives. Have a look there if you’re interested in all the details, or you can run the script to recreate everything yourself.
> >
> >
> > Nathanael, gHUMONGOUS* CP/M Archives.
> Nathanael
> Browsing your BASH script I notice that you rename files post creation of lbr files. I am not sure whether you are aware but he current version of mklbr supports renaming files as they are included by providing the stored name as part of the recipe. This allows for _ to / mapping and for more obscure mappings e.g. reserved filenames in Windows and illegal Windows/Unix filename characters. It also supports adding explicit file and library time stamps. The README.md file on my github site provides details.
> Mark

I was not aware of that feature. That will simplify my script. Thanks.

To clarify, then, to rename SIG_M.LIB to SIG/M.LIB, an entry in the recipe file would look like:

SIG_M.LIB?SIG/M.LIB

Is that correct? (I’m not at home ATM, or I’d just try it myself).

On Friday, 8 April 2022 at 02:42:26 UTC+1, Nathanael wrote:
> On Thursday, April 7, 2022 at 9:38:08 PM UTC+8, ogd...@gmail.com wrote:
> > On Thursday, 7 April 2022 at 13:05:13 UTC+1, Nathanael wrote:
> > > Completely escaped my notice, but last September was *HUMONGOUS*’s 10th Anniversary.
> > >
> > > I’ve finally completed my clean up of the SIG/M disks. I focused on four things: complete, clean, uncorrupted, and "original”. None of the known sources had all volumes; many of the filenames were corrupted; and several hundred files didn’t match their CRCs. I’ve cleaned out all of the accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames, and fixed all but a handful of the CRC problems. My goal was to produce something as close to "original" as possible.
> > >
> > > The collection is available in LRB, ARK and IBM-3740 disk image formats at http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> > >
> > > I’ve encapsulated everything into a BASH shell script which is available at github.com/cpmarchives. Have a look there if you’re interested in all the details, or you can run the script to recreate everything yourself.
> > >
> > >
> > > Nathanael, gHUMONGOUS* CP/M Archives.
> > Nathanael
> > Browsing your BASH script I notice that you rename files post creation of lbr files. I am not sure whether you are aware but he current version of mklbr supports renaming files as they are included by providing the stored name as part of the recipe. This allows for _ to / mapping and for more obscure mappings e.g. reserved filenames in Windows and illegal Windows/Unix filename characters. It also supports adding explicit file and library time stamps. The README.md file on my github site provides details.
> > Mark
> I was not aware of that feature. That will simplify my script. Thanks.
>
> To clarify, then, to rename SIG_M.LIB to SIG/M.LIB, an entry in the recipe file would look like:
>
> SIG_M.LIB?SIG/M.LIB
>
> Is that correct? (I’m not at home ATM, or I’d just try it myself).
That should work, if not let me know and I will investigate why not and fix it.
Note the rename is optional and can be omitted if not needed.

I have a corresponding mlbr tool, also on github, that takes a lbr file and can extract it, expanding compressed files within it and optionally recreating it as a zip file. In this tool an info file is produced that records any files that were renamed to avoid illegal names or clashes. It will also extract embedded dates and comments from the compressed files if present. I have used this to highlight corrupt lbr files, including compressed content errors.
regards
Mark

On Friday, April 8, 2022 at 5:15:24 PM UTC+8, Mark Ogden wrote:
> On Friday, 8 April 2022 at 02:42:26 UTC+1, Nathanael wrote:
> > On Thursday, April 7, 2022 at 9:38:08 PM UTC+8, ogd...@gmail.com wrote:
> > > On Thursday, 7 April 2022 at 13:05:13 UTC+1, Nathanael wrote:
> > > > Completely escaped my notice, but last September was *HUMONGOUS*’s 10th Anniversary.
> > > >
> > > > I’ve finally completed my clean up of the SIG/M disks. I focused on four things: complete, clean, uncorrupted, and "original”. None of the known sources had all volumes; many of the filenames were corrupted; and several hundred files didn’t match their CRCs. I’ve cleaned out all of the accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames, and fixed all but a handful of the CRC problems. My goal was to produce something as close to "original" as possible.
> > > >
> > > > The collection is available in LRB, ARK and IBM-3740 disk image formats at http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> > > >
> > > > I’ve encapsulated everything into a BASH shell script which is available at github.com/cpmarchives. Have a look there if you’re interested in all the details, or you can run the script to recreate everything yourself.
> > > >
> > > >
> > > > Nathanael, gHUMONGOUS* CP/M Archives.
> > > Nathanael
> > > Browsing your BASH script I notice that you rename files post creation of lbr files. I am not sure whether you are aware but he current version of mklbr supports renaming files as they are included by providing the stored name as part of the recipe. This allows for _ to / mapping and for more obscure mappings e.g. reserved filenames in Windows and illegal Windows/Unix filename characters. It also supports adding explicit file and library time stamps. The README.md file on my github site provides details.
> > > Mark
> > I was not aware of that feature. That will simplify my script. Thanks.
> >
> > To clarify, then, to rename SIG_M.LIB to SIG/M.LIB, an entry in the recipe file would look like:
> >
> > SIG_M.LIB?SIG/M.LIB
> >
> > Is that correct? (I’m not at home ATM, or I’d just try it myself).
> That should work, if not let me know and I will investigate why not and fix it.
> Note the rename is optional and can be omitted if not needed.
>
> I have a corresponding mlbr tool, also on github, that takes a lbr file and can extract it, expanding compressed files within it and optionally recreating it as a zip file. In this tool an info file is produced that records any files that were renamed to avoid illegal names or clashes. It will also extract embedded dates and comments from the compressed files if present. I have used this to highlight corrupt lbr files, including compressed content errors.
> regards
> Mark

I updated the script and did some very brief testing with good results. Will play around with it some more this weekend. mlbr doesn’t seem to mind if source and target names are the same, which makes the scripting dead simple.

Does your zip utility produce CP/M-compatible ZIPs?

On Friday, 8 April 2022 at 11:11:23 UTC+1, Nathanael wrote:
> On Friday, April 8, 2022 at 5:15:24 PM UTC+8, Mark Ogden wrote:
> > On Friday, 8 April 2022 at 02:42:26 UTC+1, Nathanael wrote:
> > > On Thursday, April 7, 2022 at 9:38:08 PM UTC+8, ogd...@gmail.com wrote:
> > > > On Thursday, 7 April 2022 at 13:05:13 UTC+1, Nathanael wrote:
> > > > > Completely escaped my notice, but last September was *HUMONGOUS*’s 10th Anniversary.
> > > > >
> > > > > I’ve finally completed my clean up of the SIG/M disks. I focused on four things: complete, clean, uncorrupted, and "original”. None of the known sources had all volumes; many of the filenames were corrupted; and several hundred files didn’t match their CRCs. I’ve cleaned out all of the accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames, and fixed all but a handful of the CRC problems. My goal was to produce something as close to "original" as possible.
> > > > >
> > > > > The collection is available in LRB, ARK and IBM-3740 disk image formats at http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> > > > >
> > > > > I’ve encapsulated everything into a BASH shell script which is available at github.com/cpmarchives. Have a look there if you’re interested in all the details, or you can run the script to recreate everything yourself.
> > > > >
> > > > >
> > > > > Nathanael, gHUMONGOUS* CP/M Archives.
> > > > Nathanael
> > > > Browsing your BASH script I notice that you rename files post creation of lbr files. I am not sure whether you are aware but he current version of mklbr supports renaming files as they are included by providing the stored name as part of the recipe. This allows for _ to / mapping and for more obscure mappings e.g. reserved filenames in Windows and illegal Windows/Unix filename characters. It also supports adding explicit file and library time stamps. The README.md file on my github site provides details.
> > > > Mark
> > > I was not aware of that feature. That will simplify my script. Thanks..
> > >
> > > To clarify, then, to rename SIG_M.LIB to SIG/M.LIB, an entry in the recipe file would look like:
> > >
> > > SIG_M.LIB?SIG/M.LIB
> > >
> > > Is that correct? (I’m not at home ATM, or I’d just try it myself).
> > That should work, if not let me know and I will investigate why not and fix it.
> > Note the rename is optional and can be omitted if not needed.
> >
> > I have a corresponding mlbr tool, also on github, that takes a lbr file and can extract it, expanding compressed files within it and optionally recreating it as a zip file. In this tool an info file is produced that records any files that were renamed to avoid illegal names or clashes. It will also extract embedded dates and comments from the compressed files if present. I have used this to highlight corrupt lbr files, including compressed content errors.
> > regards
> > Mark
> I updated the script and did some very brief testing with good results. Will play around with it some more this weekend. mlbr doesn’t seem to mind if source and target names are the same, which makes the scripting dead simple.
>
> Does your zip utility produce CP/M-compatible ZIPs?
I don't know, I haven't tried the generated files on CP/M.
As it stores directory information, it may not.
regards
Mark

On 4/7/22 08:05, Nathanael wrote:
> Completely escaped my notice, but last September was *HUMONGOUS*’s 10th
> Anniversary.
>
> I’ve finally completed my clean up of the SIG/M disks. I focused on four
> things: complete, clean, uncorrupted, and "original”. None of the known
> sources had all volumes; many of the filenames were corrupted; and several
> hundred files didn’t match their CRCs. I’ve cleaned out all of the
> accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames,
> and fixed all but a handful of the CRC problems. My goal was to produce
> something as close to "original" as possible.
>
> The collection is available in LRB, ARK and IBM-3740 disk image formats at
> http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
>
> I’ve encapsulated everything into a BASH shell script which is available
> at github.com/cpmarchives. Have a look there if you’re interested in all
> the details, or you can run the script to recreate everything yourself.

There are no ISO images present under:

http://cpmarchives.classiccmp.org/downloads

A README mentions an alternate site:

http://cpm.jenandcal.familyds.org/downloads

But I'm unable to connect to that URL.

Woops. My bad. I’d taken them down a while back for something. They’re back up now.

On Sunday, April 10, 2022 at 10:18:28 PM UTC+8, Steven Hirsch wrote:
> On 4/7/22 08:05, Nathanael wrote:
> > Completely escaped my notice, but last September was *HUMONGOUS*’s 10th
> > Anniversary.
> >
> > I’ve finally completed my clean up of the SIG/M disks. I focused on four
> > things: complete, clean, uncorrupted, and "original”. None of the known
> > sources had all volumes; many of the filenames were corrupted; and several
> > hundred files didn’t match their CRCs. I’ve cleaned out all of the
> > accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames,
> > and fixed all but a handful of the CRC problems. My goal was to produce
> > something as close to "original" as possible.
> >
> > The collection is available in LRB, ARK and IBM-3740 disk image formats at
> > http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> >
> > I’ve encapsulated everything into a BASH shell script which is available
> > at github.com/cpmarchives. Have a look there if you’re interested in all
> > the details, or you can run the script to recreate everything yourself.
> There are no ISO images present under:
>
> http://cpmarchives.classiccmp.org/downloads
>
> A README mentions an alternate site:
>
> http://cpm.jenandcal.familyds.org/downloads
>
> But I'm unable to connect to that URL.

There is a README there at this time which says:
Due to space issues, all files have been removed from downloads/ and links updated to point to http://cpm.jenandcal.familyds.org/downloads.

Scott

On Monday, April 11, 2022 at 7:24:31 AM UTC-7, Nathanael wrote:
> Woops. My bad. I’d taken them down a while back for something. They’re back up now.
> On Sunday, April 10, 2022 at 10:18:28 PM UTC+8, Steven Hirsch wrote:
> > On 4/7/22 08:05, Nathanael wrote:
> > > Completely escaped my notice, but last September was *HUMONGOUS*’s 10th
> > > Anniversary.
> > >
> > > I’ve finally completed my clean up of the SIG/M disks. I focused on four
> > > things: complete, clean, uncorrupted, and "original”. None of the known
> > > sources had all volumes; many of the filenames were corrupted; and several
> > > hundred files didn’t match their CRCs. I’ve cleaned out all of the
> > > accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames,
> > > and fixed all but a handful of the CRC problems. My goal was to produce
> > > something as close to "original" as possible.
> > >
> > > The collection is available in LRB, ARK and IBM-3740 disk image formats at
> > > http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> > >
> > > I’ve encapsulated everything into a BASH shell script which is available
> > > at github.com/cpmarchives. Have a look there if you’re interested in all
> > > the details, or you can run the script to recreate everything yourself.
> > There are no ISO images present under:
> >
> > http://cpmarchives.classiccmp.org/downloads
> >
> > A README mentions an alternate site:
> >
> > http://cpm.jenandcal.familyds.org/downloads
> >
> > But I'm unable to connect to that URL.

On 4/11/22 10:24, Nathanael wrote:
> Woops. My bad. I’d taken them down a while back for something. They’re back up now.

Thanks. A couple of edits later I'm able to generate the volumes. Thanks for
your work on this! I was given a massive lot of 8" diskettes that include
SIGM. Wasn't looking forward to imaging them. This is a lot simpler.

> On Sunday, April 10, 2022 at 10:18:28 PM UTC+8, Steven Hirsch wrote:
>> On 4/7/22 08:05, Nathanael wrote:
>>> Completely escaped my notice, but last September was *HUMONGOUS*’s 10th
>>> Anniversary.
>>>
>>> I’ve finally completed my clean up of the SIG/M disks. I focused on four
>>> things: complete, clean, uncorrupted, and "original”. None of the known
>>> sources had all volumes; many of the filenames were corrupted; and several
>>> hundred files didn’t match their CRCs. I’ve cleaned out all of the
>>> accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames,
>>> and fixed all but a handful of the CRC problems. My goal was to produce
>>> something as close to "original" as possible.
>>>
>>> The collection is available in LRB, ARK and IBM-3740 disk image formats at
>>> http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
>>>
>>> I’ve encapsulated everything into a BASH shell script which is available
>>> at github.com/cpmarchives. Have a look there if you’re interested in all
>>> the details, or you can run the script to recreate everything yourself.
>> There are no ISO images present under:
>>
>> http://cpmarchives.classiccmp.org/downloads
>>
>> A README mentions an alternate site:
>>
>> http://cpm.jenandcal.familyds.org/downloads
>>
>> But I'm unable to connect to that URL.

Yes. A while back I was running into space issues, so I removed the downloads material and pointed it to my backup copy on my home server. Looks like space issues have been resolved, so I’ve moved everything back up to the main site.

I’ll delete the README.

On Tuesday, April 12, 2022 at 12:16:28 AM UTC+8, sc...@mischko.com wrote:
> There is a README there at this time which says:
> Due to space issues, all files have been removed from downloads/ and links updated to point to http://cpm.jenandcal.familyds.org/downloads." rel="nofollow" target="_blank">http://cpm.jenandcal.familyds.org/downloads.
>
> Scott
> On Monday, April 11, 2022 at 7:24:31 AM UTC-7, Nathanael wrote:
> > Woops. My bad. I’d taken them down a while back for something. They’re back up now.
> > On Sunday, April 10, 2022 at 10:18:28 PM UTC+8, Steven Hirsch wrote:
> > > On 4/7/22 08:05, Nathanael wrote:
> > > > Completely escaped my notice, but last September was *HUMONGOUS*’s 10th
> > > > Anniversary.
> > > >
> > > > I’ve finally completed my clean up of the SIG/M disks. I focused on four
> > > > things: complete, clean, uncorrupted, and "original”. None of the known
> > > > sources had all volumes; many of the filenames were corrupted; and several
> > > > hundred files didn’t match their CRCs. I’ve cleaned out all of the
> > > > accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames,
> > > > and fixed all but a handful of the CRC problems. My goal was to produce
> > > > something as close to "original" as possible.
> > > >
> > > > The collection is available in LRB, ARK and IBM-3740 disk image formats at
> > > > http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> > > >
> > > > I’ve encapsulated everything into a BASH shell script which is available
> > > > at github.com/cpmarchives. Have a look there if you’re interested in all
> > > > the details, or you can run the script to recreate everything yourself.
> > > There are no ISO images present under:
> > >
> > > http://cpmarchives.classiccmp.org/downloads
> > >
> > > A README mentions an alternate site:
> > >
> > > http://cpm.jenandcal.familyds.org/downloads
> > >
> > > But I'm unable to connect to that URL.

Could you do a favor for me?

There were hundreds of files in the extant sources whose CRCs no longer matched the values recorded in -CATALOG. This seems mostly to do with the CP/M 128-byte sector size issue. Original CRCs were generated against files that retained the extra padding, which was lost as the files passed through other file systems, resulting in mismatching CRCs.

There’s code in my script which pads files out to 128-bytes using 1AH. In most cases, this seems to have restored the proper CRCs, but there are a few dozens files that still don’t match. Particularly volumes 59 and 80. At a guess, this is because for those files, the extra padding consisted of something other than 1AH.

Could you run a CRC check against your copy of volume 59 or 80 and see if they match the CRCs in -CATALOG?

On Tuesday, April 12, 2022 at 6:34:39 AM UTC+8, Steven Hirsch wrote:
> On 4/11/22 10:24, Nathanael wrote:
> > Woops. My bad. I’d taken them down a while back for something. They’re back up now.
> Thanks. A couple of edits later I'm able to generate the volumes. Thanks for
> your work on this! I was given a massive lot of 8" diskettes that include
> SIGM. Wasn't looking forward to imaging them. This is a lot simpler.
> > On Sunday, April 10, 2022 at 10:18:28 PM UTC+8, Steven Hirsch wrote:
> >> On 4/7/22 08:05, Nathanael wrote:
> >>> Completely escaped my notice, but last September was *HUMONGOUS*’s 10th
> >>> Anniversary.
> >>>
> >>> I’ve finally completed my clean up of the SIG/M disks. I focused on four
> >>> things: complete, clean, uncorrupted, and "original”. None of the known
> >>> sources had all volumes; many of the filenames were corrupted; and several
> >>> hundred files didn’t match their CRCs. I’ve cleaned out all of the
> >>> accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames,
> >>> and fixed all but a handful of the CRC problems. My goal was to produce
> >>> something as close to "original" as possible.
> >>>
> >>> The collection is available in LRB, ARK and IBM-3740 disk image formats at
> >>> http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> >>>
> >>> I’ve encapsulated everything into a BASH shell script which is available
> >>> at github.com/cpmarchives. Have a look there if you’re interested in all
> >>> the details, or you can run the script to recreate everything yourself.
> >> There are no ISO images present under:
> >>
> >> http://cpmarchives.classiccmp.org/downloads
> >>
> >> A README mentions an alternate site:
> >>
> >> http://cpm.jenandcal.familyds.org/downloads
> >>
> >> But I'm unable to connect to that URL.

Glad to hear the script works on someone else’s system. Curious as to what the “couple of edits” were. Anything that I need to update?

On Tuesday, April 12, 2022 at 6:34:39 AM UTC+8, Steven Hirsch wrote:
> On 4/11/22 10:24, Nathanael wrote:
> > Woops. My bad. I’d taken them down a while back for something. They’re back up now.
> Thanks. A couple of edits later I'm able to generate the volumes. Thanks for
> your work on this! I was given a massive lot of 8" diskettes that include
> SIGM. Wasn't looking forward to imaging them. This is a lot simpler.
> > On Sunday, April 10, 2022 at 10:18:28 PM UTC+8, Steven Hirsch wrote:
> >> On 4/7/22 08:05, Nathanael wrote:
> >>> Completely escaped my notice, but last September was *HUMONGOUS*’s 10th
> >>> Anniversary.
> >>>
> >>> I’ve finally completed my clean up of the SIG/M disks. I focused on four
> >>> things: complete, clean, uncorrupted, and "original”. None of the known
> >>> sources had all volumes; many of the filenames were corrupted; and several
> >>> hundred files didn’t match their CRCs. I’ve cleaned out all of the
> >>> accretions (e.g., WILDCAT.BBS), restored "illegal" characters in filenames,
> >>> and fixed all but a handful of the CRC problems. My goal was to produce
> >>> something as close to "original" as possible.
> >>>
> >>> The collection is available in LRB, ARK and IBM-3740 disk image formats at
> >>> http://cpmarchives.classiccmp.org/ftp.php?b=cpm%2FSoftware%2FUserGroups%2FSIGM.
> >>>
> >>> I’ve encapsulated everything into a BASH shell script which is available
> >>> at github.com/cpmarchives. Have a look there if you’re interested in all
> >>> the details, or you can run the script to recreate everything yourself.
> >> There are no ISO images present under:
> >>
> >> http://cpmarchives.classiccmp.org/downloads
> >>
> >> A README mentions an alternate site:
> >>
> >> http://cpm.jenandcal.familyds.org/downloads
> >>
> >> But I'm unable to connect to that URL.

On 4/11/22 19:55, Nathanael wrote:
> Glad to hear the script works on someone else’s system. Curious as to what
> the “couple of edits” were. Anything that I need to update?
>

I had to edit in the ISO and output locations. Suggestion: Modify the script
to take these from the environment.

I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).

Not sure how I’d get that info from the environment?

On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
> On 4/11/22 19:55, Nathanael wrote:
> > Glad to hear the script works on someone else’s system. Curious as to what
> > the “couple of edits” were. Anything that I need to update?
> >
> I had to edit in the ISO and output locations. Suggestion: Modify the script
> to take these from the environment.

On Thursday, 14 April 2022 at 18:17:32 UTC+1, Nathanael wrote:
> I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).
>
> Not sure how I’d get that info from the environment?
> On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
> > On 4/11/22 19:55, Nathanael wrote:
> > > Glad to hear the script works on someone else’s system. Curious as to what
> > > the “couple of edits” were. Anything that I need to update?
> > >
> > I had to edit in the ISO and output locations. Suggestion: Modify the script
> > to take these from the environment.
Nathanael
I appreciate that most CP/M systems did not support date/time but some did, which lbr and some of the compression formats supported.
If you use mlbr, it will show any embedded dates or in some cases the comment fields that contain text date information.
It is probably not worth saving the files with these dates, but it may be of historic interest to some, so adding listings could be of interest.
Note if there are date/times (except those in comments) then mlbr will use these to set the timestamps of extracted files as follows
1) if in date/time is in a compressed file, this is used
2) else if in the lbr header time/date is set for a file, this is used

On a separate note, lbr files should have padding to 128 byte boundaries for the included files and if CRC is being used, the padding (1Ah) will be included in the calculation.
The lbr header's own CRC is calculated specially, in that it is calculated with the CRC initially assumed to be 0 then updated with calculated value

Mark

Sorry, not sure I’m following you. I don’t touch any of the LBRs in the SIG/M collection, so whatever internal information they contain is preserved as-is. I use mlbr to build the SIGMVxxx.LBR files in LBR/ only. I suppose I could examine each individual LBR in the collection to determine what date to set for that LBR, but even that only sets the lower bound.

If that’s not what you’re referring to, could you clarify?

—nathanael

On Friday, April 15, 2022 at 8:06:41 PM UTC+8, Mark Ogden wrote:
> On Thursday, 14 April 2022 at 18:17:32 UTC+1, Nathanael wrote:
> > I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).
> >
> > Not sure how I’d get that info from the environment?
> > On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
> > > On 4/11/22 19:55, Nathanael wrote:
> > > > Glad to hear the script works on someone else’s system. Curious as to what
> > > > the “couple of edits” were. Anything that I need to update?
> > > >
> > > I had to edit in the ISO and output locations. Suggestion: Modify the script
> > > to take these from the environment.
> Nathanael
> I appreciate that most CP/M systems did not support date/time but some did, which lbr and some of the compression formats supported.
> If you use mlbr, it will show any embedded dates or in some cases the comment fields that contain text date information.
> It is probably not worth saving the files with these dates, but it may be of historic interest to some, so adding listings could be of interest.
> Note if there are date/times (except those in comments) then mlbr will use these to set the timestamps of extracted files as follows
> 1) if in date/time is in a compressed file, this is used
> 2) else if in the lbr header time/date is set for a file, this is used
>
> On a separate note, lbr files should have padding to 128 byte boundaries for the included files and if CRC is being used, the padding (1Ah) will be included in the calculation.
> The lbr header's own CRC is calculated specially, in that it is calculated with the CRC initially assumed to be 0 then updated with calculated value
>
> Mark

On Friday, 15 April 2022 at 16:49:44 UTC+1, Nathanael wrote:
> Sorry, not sure I’m following you. I don’t touch any of the LBRs in the SIG/M collection, so whatever internal information they contain is preserved as-is. I use mlbr to build the SIGMVxxx.LBR files in LBR/ only. I suppose I could examine each individual LBR in the collection to determine what date to set for that LBR, but even that only sets the lower bound.
>
> If that’s not what you’re referring to, could you clarify?
>
> —nathanael
> On Friday, April 15, 2022 at 8:06:41 PM UTC+8, Mark Ogden wrote:
> > On Thursday, 14 April 2022 at 18:17:32 UTC+1, Nathanael wrote:
> > > I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).
> > >
> > > Not sure how I’d get that info from the environment?
> > > On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
> > > > On 4/11/22 19:55, Nathanael wrote:
> > > > > Glad to hear the script works on someone else’s system. Curious as to what
> > > > > the “couple of edits” were. Anything that I need to update?
> > > > >
> > > > I had to edit in the ISO and output locations. Suggestion: Modify the script
> > > > to take these from the environment.
> > Nathanael
> > I appreciate that most CP/M systems did not support date/time but some did, which lbr and some of the compression formats supported.
> > If you use mlbr, it will show any embedded dates or in some cases the comment fields that contain text date information.
> > It is probably not worth saving the files with these dates, but it may be of historic interest to some, so adding listings could be of interest.
> > Note if there are date/times (except those in comments) then mlbr will use these to set the timestamps of extracted files as follows
> > 1) if in date/time is in a compressed file, this is used
> > 2) else if in the lbr header time/date is set for a file, this is used
> >
> > On a separate note, lbr files should have padding to 128 byte boundaries for the included files and if CRC is being used, the padding (1Ah) will be included in the calculation.
> > The lbr header's own CRC is calculated specially, in that it is calculated with the CRC initially assumed to be 0 then updated with calculated value
> >
> > Mark
Nathanael
When an LBR file is created it will record timestamps associated with the files on the disk, or no timestamps if the system did not support them.
Some compressed files, contain a timestamp of when the compressed file was created, which is likely to give a more accurate view.

If you use my mlbr utility it will show the timestamp of any compressed file, if it exists and use it to set the timestamp of any extracted file.

regards
Mark

I’ve been playing around with this for a couple of hours and getting some odd results.

For many of the LBR files in SIG/M mlbr returns rather strange dates. Take as an example BYTYPE86.LBR on vol 229:

APC-DATE.LBR 1985-05-17
!! apc-date.lbr library CRC is missing
apc-date.lbr 18688 Library - 2157-06-06 08:04
date4.pas 6400 Stored - 1982-05-25 17:41
date4.cmd 12160 Stored - 2144-10-27 17:56

APCSERIO.LBR 1985-05-17
!! apcserio.lbr library CRC is missing
apcserio.lbr 16256 Library - 2157-06-06 08:04
serialio.a86 13056 Stored - 1978-01-03 00:00
terminal.a86 3072 Stored - <no date record>

The CRCs of both LBRs match the listing in -CATALOG.229, so the files themselves seem to be intact.

On Saturday, April 16, 2022 at 1:44:45 AM UTC+8, Mark Ogden wrote:
> On Friday, 15 April 2022 at 16:49:44 UTC+1, Nathanael wrote:
> > Sorry, not sure I’m following you. I don’t touch any of the LBRs in the SIG/M collection, so whatever internal information they contain is preserved as-is. I use mlbr to build the SIGMVxxx.LBR files in LBR/ only. I suppose I could examine each individual LBR in the collection to determine what date to set for that LBR, but even that only sets the lower bound.
> >
> > If that’s not what you’re referring to, could you clarify?
> >
> > —nathanael
> > On Friday, April 15, 2022 at 8:06:41 PM UTC+8, Mark Ogden wrote:
> > > On Thursday, 14 April 2022 at 18:17:32 UTC+1, Nathanael wrote:
> > > > I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).
> > > >
> > > > Not sure how I’d get that info from the environment?
> > > > On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
> > > > > On 4/11/22 19:55, Nathanael wrote:
> > > > > > Glad to hear the script works on someone else’s system. Curious as to what
> > > > > > the “couple of edits” were. Anything that I need to update?
> > > > > >
> > > > > I had to edit in the ISO and output locations. Suggestion: Modify the script
> > > > > to take these from the environment.
> > > Nathanael
> > > I appreciate that most CP/M systems did not support date/time but some did, which lbr and some of the compression formats supported.
> > > If you use mlbr, it will show any embedded dates or in some cases the comment fields that contain text date information.
> > > It is probably not worth saving the files with these dates, but it may be of historic interest to some, so adding listings could be of interest.
> > > Note if there are date/times (except those in comments) then mlbr will use these to set the timestamps of extracted files as follows
> > > 1) if in date/time is in a compressed file, this is used
> > > 2) else if in the lbr header time/date is set for a file, this is used
> > >
> > > On a separate note, lbr files should have padding to 128 byte boundaries for the included files and if CRC is being used, the padding (1Ah) will be included in the calculation.
> > > The lbr header's own CRC is calculated specially, in that it is calculated with the CRC initially assumed to be 0 then updated with calculated value
> > >
> > > Mark
> Nathanael
> When an LBR file is created it will record timestamps associated with the files on the disk, or no timestamps if the system did not support them.
> Some compressed files, contain a timestamp of when the compressed file was created, which is likely to give a more accurate view.
>
> If you use my mlbr utility it will show the timestamp of any compressed file, if it exists and use it to set the timestamp of any extracted file.
>
> regards
> Mark

On 4/14/22 13:17, Nathanael wrote:
> I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).
>
> Not sure how I’d get that info from the environment?

In shell coding, environment variables are just another global variable. So,
define two values, e.g. 'ISODIR' and 'OUTDIR', then have the user set those
before running the program:

$ export ISODIR=/path/to/iso
$ export OUTDIR=/path/for/output
$ ./buildvol XXX

> On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
>> On 4/11/22 19:55, Nathanael wrote:
>>> Glad to hear the script works on someone else’s system. Curious as to what
>>> the “couple of edits” were. Anything that I need to update?
>>>
>> I had to edit in the ISO and output locations. Suggestion: Modify the script
>> to take these from the environment.

On Saturday, 16 April 2022 at 14:33:07 UTC+1, Steven Hirsch wrote:
> On 4/14/22 13:17, Nathanael wrote:
> > I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).
> >
> > Not sure how I’d get that info from the environment?
> In shell coding, environment variables are just another global variable. So,
> define two values, e.g. 'ISODIR' and 'OUTDIR', then have the user set those
> before running the program:
>
> $ export ISODIR=/path/to/iso
> $ export OUTDIR=/path/for/output
> $ ./buildvol XXX
> > On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
> >> On 4/11/22 19:55, Nathanael wrote:
> >>> Glad to hear the script works on someone else’s system. Curious as to what
> >>> the “couple of edits” were. Anything that I need to update?
> >>>
> >> I had to edit in the ISO and output locations. Suggestion: Modify the script
> >> to take these from the environment.
Nathanael
I have posted an update to mlbr on github that ignores dates set to 0xffff.
There is still a problem in that it appears as though some of the dates have been set using a different base date rather than the DR one (or have garbage date info)
The lbr specification can be found at https://www.seasip.info/Cpm/ludef5.html. You will see that dates should be based from the number of days since December 31, 1977. If the day value used is based on a different epoch, e.g. unix, msdos or windows, mlbr will reflect a date many years different.

I also suspect that lbr files generated using LU86 have garbage date/time value. Some of these could be detected if the timestamps they represent are later than the timestamp of the lbr file itself.

Note the timestamps of some of the LBR files themselves are later than the release date of the SIG/M volume catalog info, possibly indicating that the files were copied without keeping the original dates/times.

Mark

On Saturday, April 16, 2022 at 9:33:07 PM UTC+8, Steven Hirsch wrote:
> $ export ISODIR=/path/to/iso
> $ export OUTDIR=/path/for/output

Doh. I thought you mean deriving them automatically from the environment. Yeah, setting environment variables is easy enough. I’ve updated the script as follows:

[[ -z “$iso_loc” ]] && iso_loc=$(pwd)
[[ -z “$buildroot” ]] && buildroot=$(pwd)

I know earlier releases of LU didn’t support CRCs and left garbage in the CRC field in the lbr headers. Wouldn’t one expect something similar for dates, since few people ran systems that supported them anyway.

If it were just a matter of using a different epoch number, you should see consistency, at least, in file dates, even if the dates are wrong.

On Saturday, April 16, 2022 at 11:32:18 PM UTC+8, Mark Ogden wrote:
> On Saturday, 16 April 2022 at 14:33:07 UTC+1, Steven Hirsch wrote:
> > On 4/14/22 13:17, Nathanael wrote:
> > > I’d forgotten to edit out my own local system values. I’ve replaced both with $(pwd).
> > >
> > > Not sure how I’d get that info from the environment?
> > In shell coding, environment variables are just another global variable.. So,
> > define two values, e.g. 'ISODIR' and 'OUTDIR', then have the user set those
> > before running the program:
> >
> > $ export ISODIR=/path/to/iso
> > $ export OUTDIR=/path/for/output
> > $ ./buildvol XXX
> > > On Thursday, April 14, 2022 at 9:49:46 PM UTC+8, Steven Hirsch wrote:
> > >> On 4/11/22 19:55, Nathanael wrote:
> > >>> Glad to hear the script works on someone else’s system. Curious as to what
> > >>> the “couple of edits” were. Anything that I need to update?
> > >>>
> > >> I had to edit in the ISO and output locations. Suggestion: Modify the script
> > >> to take these from the environment.
> Nathanael
> I have posted an update to mlbr on github that ignores dates set to 0xffff.
> There is still a problem in that it appears as though some of the dates have been set using a different base date rather than the DR one (or have garbage date info)
> The lbr specification can be found at https://www.seasip.info/Cpm/ludef5.html. You will see that dates should be based from the number of days since December 31, 1977. If the day value used is based on a different epoch, e.g. unix, msdos or windows, mlbr will reflect a date many years different.
>
> I also suspect that lbr files generated using LU86 have garbage date/time value. Some of these could be detected if the timestamps they represent are later than the timestamp of the lbr file itself.
>
> Note the timestamps of some of the LBR files themselves are later than the release date of the SIG/M volume catalog info, possibly indicating that the files were copied without keeping the original dates/times.
>
> Mark

On 4/16/22 12:30, Nathanael wrote:
> On Saturday, April 16, 2022 at 9:33:07 PM UTC+8, Steven Hirsch wrote:
>> $ export ISODIR=/path/to/iso
>> $ export OUTDIR=/path/for/output
>
> Doh. I thought you mean deriving them automatically from the environment. Yeah, setting environment variables is easy enough. I’ve updated the script as follows:
>
> [[ -z “$iso_loc” ]] && iso_loc=$(pwd)
> [[ -z “$buildroot” ]] && buildroot=$(pwd)

Yeah, that'll work. FYI: it's sort of customary to use all uppercase on
variables expected in the environment. But, I'm one of those Unix graybeard
types :-).

“it’s sort of customary to use all uppercase”

Yeah, I know. But that’d require me to do a find-and-replace. Twice.. Just don’t know if I’ve got that kind of time :)

On Sunday, April 17, 2022 at 3:34:32 AM UTC+8, Steven Hirsch wrote:
> On 4/16/22 12:30, Nathanael wrote:
> > On Saturday, April 16, 2022 at 9:33:07 PM UTC+8, Steven Hirsch wrote:
> >> $ export ISODIR=/path/to/iso
> >> $ export OUTDIR=/path/for/output
> >
> > Doh. I thought you mean deriving them automatically from the environment. Yeah, setting environment variables is easy enough. I’ve updated the script as follows:
> >
> > [[ -z “$iso_loc” ]] && iso_loc=$(pwd)
> > [[ -z “$buildroot” ]] && buildroot=$(pwd)
> Yeah, that'll work. FYI: it's sort of customary to use all uppercase on
> variables expected in the environment. But, I'm one of those Unix graybeard
> types :-).