Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Computer programmers do it byte by byte.


devel / comp.os.cpm / New version of my Xmodem program

SubjectAuthor
* New version of my Xmodem programMartin Eberhard
`* Re: New version of my Xmodem programPhillip Stevens
 `* Re: New version of my Xmodem programPhillip Stevens
  `* Re: New version of my Xmodem programTony Nicholson
   `* Re: New version of my Xmodem programPhillip Stevens
    `- Re: New version of my Xmodem programPhillip Stevens

1
New version of my Xmodem program

<126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=2906&group=comp.os.cpm#2906

 copy link   Newsgroups: comp.os.cpm
X-Received: by 2002:a05:622a:164c:: with SMTP id y12mr863190qtj.63.1636998881920;
Mon, 15 Nov 2021 09:54:41 -0800 (PST)
X-Received: by 2002:a9d:6195:: with SMTP id g21mr689108otk.289.1636998881665;
Mon, 15 Nov 2021 09:54:41 -0800 (PST)
Path: i2pn2.org!i2pn.org!aioe.org!news.uzoreto.com!news-out.netnews.com!news.alt.net!fdc2.netnews.com!peer02.ams1!peer.ams1.xlned.com!news.xlned.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.cpm
Date: Mon, 15 Nov 2021 09:54:41 -0800 (PST)
Injection-Info: google-groups.googlegroups.com; posting-host=69.27.189.115; posting-account=yyiXngoAAADSoqDiJDHNpP-BO9yBRgh5
NNTP-Posting-Host: 69.27.189.115
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com>
Subject: New version of my Xmodem program
From: mfeberh...@gmail.com (Martin Eberhard)
Injection-Date: Mon, 15 Nov 2021 17:54:41 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 2074
 by: Martin Eberhard - Mon, 15 Nov 2021 17:54 UTC

I've just released version 2.9 of my Xmodem file transfer program for CP/M. This is a minor tweak on version 2.8: the receiver is now flushed of any garbage characters before receiving a file. (Previous versions would get hung up due to leftover characters in the UART's buffers if you had aborted a file transfer before trying to transfer a new file.)

You can find the new version in my directory that's hosted on Mike Douglas's website, here:

https://deramp.com/downloads/mfe_archive/005-Documentation%20and%20Code%20by%20Martin/010%20Code%20by%20Martin/CPM%20Code/XMODEM/

There's also a handy little program for confirming the functionality of your serial link, called XMTERM, here:

https://deramp.com/downloads/mfe_archive/005-Documentation%20and%20Code%20by%20Martin/010%20Code%20by%20Martin/CPM%20Code/XMTERM/

XMTERM is a simple terminal program that uses the same configuration file (XMODEM.CFG) as does the Xmodem program, so it tests the serial link as well as the config file setup.

Cheers,
Martin

Re: New version of my Xmodem program

<002e0d6d-e7f5-44db-8c41-d95a18d08968n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=3210&group=comp.os.cpm#3210

 copy link   Newsgroups: comp.os.cpm
X-Received: by 2002:a05:622a:6:b0:2e1:e793:b93b with SMTP id x6-20020a05622a000600b002e1e793b93bmr3326239qtw.530.1647419187062;
Wed, 16 Mar 2022 01:26:27 -0700 (PDT)
X-Received: by 2002:a05:6870:a70b:b0:dd:9cf0:763e with SMTP id
g11-20020a056870a70b00b000dd9cf0763emr1792956oam.116.1647419186578; Wed, 16
Mar 2022 01:26:26 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.cpm
Date: Wed, 16 Mar 2022 01:26:26 -0700 (PDT)
In-Reply-To: <126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=203.76.225.173; posting-account=yre5vgoAAABwuYxQiQDdGcu9SyyRYxDD
NNTP-Posting-Host: 203.76.225.173
References: <126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <002e0d6d-e7f5-44db-8c41-d95a18d08968n@googlegroups.com>
Subject: Re: New version of my Xmodem program
From: phillip....@gmail.com (Phillip Stevens)
Injection-Date: Wed, 16 Mar 2022 08:26:27 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 26
 by: Phillip Stevens - Wed, 16 Mar 2022 08:26 UTC

> I've just released version 2.9 of my Xmodem file transfer program for CP/M. This is a minor tweak on version 2.8: the receiver is now flushed of any garbage characters before receiving a file. (Previous versions would get hung up due to leftover characters in the UART's buffers if you had aborted a file transfer before trying to transfer a new file.)

Hi Martin,

I've been using your new version 2.9 on my CP/M machine.

I assemble it with DRI MAC, and load with MLOAD.
Very simple build. Works great.

Because my CP/M BIOS serial handling doesn't touch BC or DE, I've removed that protection from GOBIOS (which takes quite a few cycles out of the transmit and receive loops).

What I'm finding is that the 2.9 version pauses and waits while sending to CP/M much more often than the 2.7 version. The actual throughput is quite a bit lower.

There are some notes in your 2.8 release related to this, and I wonder if by "speeding up" the GOBIOS routine I've in-fact broken the carefully choreographed timing of buffer read and write? Would that be the case?

Cheers, Phillip

Re: New version of my Xmodem program

<5fda4cdd-2485-4459-94d3-31f091113fc9n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=3211&group=comp.os.cpm#3211

 copy link   Newsgroups: comp.os.cpm
X-Received: by 2002:ac8:5dcf:0:b0:2e1:baf1:502d with SMTP id e15-20020ac85dcf000000b002e1baf1502dmr2336124qtx.635.1647491787998;
Wed, 16 Mar 2022 21:36:27 -0700 (PDT)
X-Received: by 2002:a4a:2f0f:0:b0:320:f7cd:58d2 with SMTP id
p15-20020a4a2f0f000000b00320f7cd58d2mr794352oop.85.1647491787456; Wed, 16 Mar
2022 21:36:27 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.cpm
Date: Wed, 16 Mar 2022 21:36:27 -0700 (PDT)
In-Reply-To: <002e0d6d-e7f5-44db-8c41-d95a18d08968n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=203.76.225.173; posting-account=yre5vgoAAABwuYxQiQDdGcu9SyyRYxDD
NNTP-Posting-Host: 203.76.225.173
References: <126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com> <002e0d6d-e7f5-44db-8c41-d95a18d08968n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <5fda4cdd-2485-4459-94d3-31f091113fc9n@googlegroups.com>
Subject: Re: New version of my Xmodem program
From: phillip....@gmail.com (Phillip Stevens)
Injection-Date: Thu, 17 Mar 2022 04:36:27 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 37
 by: Phillip Stevens - Thu, 17 Mar 2022 04:36 UTC

> > I've just released version 2.9 of my Xmodem file transfer program for CP/M. This is a minor tweak on version 2.8: the receiver is now flushed of any garbage characters before receiving a file. (Previous versions would get hung up due to leftover characters in the UART's buffers if you had aborted a file transfer before trying to transfer a new file.)

> I assemble it with DRI MAC, and load with MLOAD. Very simple build. Works great.

> Because my CP/M BIOS serial handling doesn't touch BC or DE, I've removed that protection from GOBIOS (which takes quite a few cycles out of the transmit and receive loops). What I'm finding is that the 2.9 version pauses and waits while sending to CP/M much more often than the 2.7 version. The actual throughput is quite a bit lower.

> There are some notes in your 2.8 release related to this, and I wonder if by "speeding up" the GOBIOS routine I've in-fact broken the carefully choreographed timing of buffer read and write? Would that be the case?

Hi Martin,

I've done some experiments with 2.9, by calculating the correct timings for CRTIME (from my bios) and adjusting for the revised GOBIOS cycles in CONTO (& RDRTO). I've tried with different CPUMHZ values. I've set WBPERX to 252.. No matter what I do, I'm still getting the "Retry 0: NAK on sector" message from my PC end (moserial 3.0.12) very often, and substantially more often than with 2.7.

Previously (2.7) the CP/M receive direction would fill RAM (approx 54kB) without showing a NAK and then write to disk and repeat, but now (2.9) it only does a few kB before showing repeated NAKs and is very choppy and slow. I replicate this on the same machine receiving the same file, so the only change is xmodem.

Any thoughts on what has changed which might cause this different behaviour?

Cheers, Phillip

Re: New version of my Xmodem program

<f78e8f72-39e7-4dd0-97e6-07656d2fe79bn@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=3213&group=comp.os.cpm#3213

 copy link   Newsgroups: comp.os.cpm
X-Received: by 2002:a05:620a:22db:b0:67d:72f5:bc60 with SMTP id o27-20020a05620a22db00b0067d72f5bc60mr4043477qki.633.1647556802062;
Thu, 17 Mar 2022 15:40:02 -0700 (PDT)
X-Received: by 2002:a05:6808:1b11:b0:2da:73df:2dbd with SMTP id
bx17-20020a0568081b1100b002da73df2dbdmr6780351oib.293.1647556801830; Thu, 17
Mar 2022 15:40:01 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.cpm
Date: Thu, 17 Mar 2022 15:40:01 -0700 (PDT)
In-Reply-To: <5fda4cdd-2485-4459-94d3-31f091113fc9n@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=2403:5807:8025:0:d005:b516:3018:40d6;
posting-account=5wnjpAoAAABrhNpKTP0a5hOqJUBnLT3d
NNTP-Posting-Host: 2403:5807:8025:0:d005:b516:3018:40d6
References: <126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com>
<002e0d6d-e7f5-44db-8c41-d95a18d08968n@googlegroups.com> <5fda4cdd-2485-4459-94d3-31f091113fc9n@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <f78e8f72-39e7-4dd0-97e6-07656d2fe79bn@googlegroups.com>
Subject: Re: New version of my Xmodem program
From: agn...@gmail.com (Tony Nicholson)
Injection-Date: Thu, 17 Mar 2022 22:40:02 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 22
 by: Tony Nicholson - Thu, 17 Mar 2022 22:40 UTC

On Thursday, March 17, 2022 at 3:36:28 PM UTC+11, Phillip wrote:

> Previously (2.7) the CP/M receive direction would fill RAM (approx 54kB) without showing a NAK and then write to disk and repeat, but now (2.9) it only does a few kB before showing repeated NAKs and is very choppy and slow. I replicate this on the same machine receiving the same file, so the only change is xmodem.
>
> Any thoughts on what has changed which might cause this different behaviour?

Phillip,

I found that XMODEM sometimes does a NAK if the CP/M file writing takes
too long (the timeout settings are hard to get right on faster Z80/Z280
systems). The workaround for me was to reduce the amount of in-memory
buffering.

Version 2.9 as a new setting /K that allows you to select this. I put /K20
in my XMODEM.CFG file to limit it to 20Kbytes on my problematic
system.

Tony

Re: New version of my Xmodem program

<add6a081-e9c1-4efc-b964-51841add2fden@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=3217&group=comp.os.cpm#3217

 copy link   Newsgroups: comp.os.cpm
X-Received: by 2002:a05:6214:246b:b0:435:418c:71b6 with SMTP id im11-20020a056214246b00b00435418c71b6mr6886947qvb.57.1647609832509;
Fri, 18 Mar 2022 06:23:52 -0700 (PDT)
X-Received: by 2002:a4a:e865:0:b0:318:4b66:ffe0 with SMTP id
m5-20020a4ae865000000b003184b66ffe0mr2794621oom.80.1647609831956; Fri, 18 Mar
2022 06:23:51 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.cpm
Date: Fri, 18 Mar 2022 06:23:51 -0700 (PDT)
In-Reply-To: <f78e8f72-39e7-4dd0-97e6-07656d2fe79bn@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=203.76.225.173; posting-account=yre5vgoAAABwuYxQiQDdGcu9SyyRYxDD
NNTP-Posting-Host: 203.76.225.173
References: <126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com>
<002e0d6d-e7f5-44db-8c41-d95a18d08968n@googlegroups.com> <5fda4cdd-2485-4459-94d3-31f091113fc9n@googlegroups.com>
<f78e8f72-39e7-4dd0-97e6-07656d2fe79bn@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <add6a081-e9c1-4efc-b964-51841add2fden@googlegroups.com>
Subject: Re: New version of my Xmodem program
From: phillip....@gmail.com (Phillip Stevens)
Injection-Date: Fri, 18 Mar 2022 13:23:52 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 41
 by: Phillip Stevens - Fri, 18 Mar 2022 13:23 UTC

> > Phillip wrote:
> > Previously (2.7) the CP/M receive direction would fill RAM (approx 54kB) without showing a NAK and then write to disk and repeat, but now (2.9) it only does a few kB before showing repeated NAKs and is very choppy and slow. I replicate this on the same machine receiving the same file, so the only change is xmodem.
> > Any thoughts on what has changed which might cause this different behaviour?

>Tony Nicholson wrote:
> I found that XMODEM sometimes does a NAK if the CP/M file writing takes too long (the timeout settings are hard to get right on faster Z80/Z280 systems). The workaround for me was to reduce the amount of in-memory buffering.. Version 2.9 as a new setting /K that allows you to select this. I put /K20 in my XMODEM.CFG file to limit it to 20Kbytes on my problematic system.

Hi Tony, thanks for the hints, but it seems that I've got a different issue..

Firstly, I've isolated it down to just z80/SIO/v2.9. Both z80/ACIA and 8085/ACIA are working as expected. And with the Z80/SIO the v2.7 is working perfectly too.

I've got the code from both v2.7 and v2.9 side by side, and I'm trying to decipher the new code flow. There have been many substantial changes.

I've also spent some time with the logic analyser and traces. The failing process begins when is a block is received and rather than being ACKed(0x06) it is NAKed (0x15). Perhaps noise, but I doubt that. It will be something programmatic causing the CRC to fail (or something). The sender then repeats the block but the receiver then goes into a long time out, before the receiver sends NAK again. At this point the sender then repeats the block a second time the receiver ACKs it, and things continue normally.

So, I'm thinking to try to reduce the timeout before sending that second NAK, to speed up reinitialisation of transmission. But that is a work around.
What I need to do is to find out what is causing the block errors in the first place (given the triage above).

Interesting problem...

Cheers, Phillip

Re: New version of my Xmodem program

<7e2b5755-fbbc-4153-8cde-89f2ae8de1c8n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=3222&group=comp.os.cpm#3222

 copy link   Newsgroups: comp.os.cpm
X-Received: by 2002:a05:620a:d87:b0:67b:311c:ecbd with SMTP id q7-20020a05620a0d8700b0067b311cecbdmr8057898qkl.146.1647682399712;
Sat, 19 Mar 2022 02:33:19 -0700 (PDT)
X-Received: by 2002:a05:6870:4790:b0:dd:cd0f:e00b with SMTP id
c16-20020a056870479000b000ddcd0fe00bmr3215929oaq.120.1647682399249; Sat, 19
Mar 2022 02:33:19 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.os.cpm
Date: Sat, 19 Mar 2022 02:33:19 -0700 (PDT)
In-Reply-To: <add6a081-e9c1-4efc-b964-51841add2fden@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=203.76.225.173; posting-account=yre5vgoAAABwuYxQiQDdGcu9SyyRYxDD
NNTP-Posting-Host: 203.76.225.173
References: <126afc3f-988f-4bab-a302-0e2789e2aa96n@googlegroups.com>
<002e0d6d-e7f5-44db-8c41-d95a18d08968n@googlegroups.com> <5fda4cdd-2485-4459-94d3-31f091113fc9n@googlegroups.com>
<f78e8f72-39e7-4dd0-97e6-07656d2fe79bn@googlegroups.com> <add6a081-e9c1-4efc-b964-51841add2fden@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <7e2b5755-fbbc-4153-8cde-89f2ae8de1c8n@googlegroups.com>
Subject: Re: New version of my Xmodem program
From: phillip....@gmail.com (Phillip Stevens)
Injection-Date: Sat, 19 Mar 2022 09:33:19 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 22
 by: Phillip Stevens - Sat, 19 Mar 2022 09:33 UTC

> Phillip Stevens wrote:

> I've also spent some time with the logic analyser and traces. The failing process begins when is a block is received and rather than being ACKed(0x06) it is NAKed (0x15). Perhaps noise, but I doubt that. It will be something programmatic causing the CRC to fail (or something). The sender then repeats the block but the receiver then goes into a long time out, before the receiver sends NAK again. At this point the sender then repeats the block a second time the receiver ACKs it, and things continue normally.

> So, I'm thinking to try to reduce the timeout before sending that second NAK, to speed up reinitialisation of transmission. But that is a work around. What I need to do is to find out what is causing the block errors in the first place (given the triage above).

> Interesting problem...

Well, by adjusting some timings the problem seems to have resolved itself. I'm not sure that these fixes make any substantial difference, but anyway there's nothing to see here.

Left a gist of the timing changes here.
https://gist.github.com/feilipu/73f84745d9019a20ee1ef15d1fee948c

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor