I want to start a new thread for the purpose of debugging Forth code
posted by Marcel Hendrix for the computation of SHA512 hash on a buffer
of data:

https://groups.google.com/g/comp.lang.forth/c/3TH1hHgia9Q/m/Jd_HPeSuAQAJ

The program is a translation of an earlier C program given at

https://aarongifford.com/computers/sha.html

While some discussion on optimization of the Forth code occurred on the
earlier thread, it was demonstrated that the Forth code does not yet
compute the correct SHA512 hash. The purpose of this thread is to
discuss the implementation errors in the Forth SHA512 code.

As a reference for correct implementation of the SHA512 hash algorithm,
we can use the C version from Aaron Gifford (linked above). For
debugging, the intermediate computation results for the test case, on a
string buffer containing "abc", is given in the following NIST document,

https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Standards-and-Guidelines/documents/examples/SHA512.pdf

The SHA512 hashing algorithm is also described in detail in FIPS
(Federal Information Processing Standards) PUB 180-4, "Secure Hash
Standards (SHS)", from NIST:

https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.180-4.pdf

Two variants of Marcel Hendrix's Forth program posting may be found at
the following links -- both aim to perform the same computation as
Marcel's originally posted code:

https://www.complang.tuwien.ac.at/forth/programs/sha512.4th
https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th

To reiterate, all of the above Forth implementations fail to correctly
compute the SHA512 hash, and with SHA512 being a core utility on
computing systems, it will be worthwhile to have a proper Forth
implementation. To this end, I hope we can work together here to fixing
the current code. As a start, I discovered the following apparent bug in
the ROUND512_x words:

From
https://groups.google.com/g/comp.lang.forth/c/3TH1hHgia9Q/m/Jd_HPeSuAQAJ
--------
: ROUND512_a ( u -- )
>R
W512[] R@ 1+ $0F AND CELL[] @ sigma0_512l ( s0)
W512[] R@ #14 + $0F AND CELL[] @ sigma0_512l ( s1) +
W512[] R@ 9 + $0F AND CELL[] @ + DUP W512[] R@ $0F AND CELL[] +! ( -- u )
h + e sigma1_512u + e f g Ch + K512[] R> CELL[] @ + DUP +TO d ( T1 )
a sigma0_512u + a b c Maj + TO h ; PRIVATE
----

Comparing with the C code on which it is based, the line,

W512[] R@ #14 + $0F AND CELL[] @ sigma0_512l ( s1) +

should read

W512[] R@ #14 + $0F AND CELL[] @ sigma1_512l ( s1) +

From sha2.c,

#define ROUND512(a,b,c,d,e,f,g,h) \
s0 = W512[(j+1)&0x0f]; \
s0 = sigma0_512(s0); \
s1 = W512[(j+14)&0x0f]; \
s1 = sigma1_512(s1); \
T1 = (h) + Sigma1_512(e) + Ch((e), (f), (g)) + K512[j] + \
(W512[j&0x0f] += s1 + W512[(j+9)&0x0f] + s0); \
(d) += T1; \
(h) = T1 + Sigma0_512(a) + Maj((a), (b), (c)); \
j++

If confirmed, all of the ROUND512_x words should be changed accordingly.

--
Krishna

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>W512[] R@ #14 + $0F AND CELL[] @ sigma0_512l ( s1) +
>
>should read
>
>W512[] R@ #14 + $0F AND CELL[] @ sigma1_512l ( s1) +

Thanks. Applied, and the result changed, but is still different from
the correct result.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2/2/22 11:13, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> W512[] R@ #14 + $0F AND CELL[] @ sigma0_512l ( s1) +
>>
>> should read
>>
>> W512[] R@ #14 + $0F AND CELL[] @ sigma1_512l ( s1) +
>
> Thanks. Applied, and the result changed, but is still different from
> the correct result.
>

That's what I found also.

-----
s" abc" SHA512_Data cr type

13CA833A9E7C792B7901AFF9A232DA3EB36B993E35D940CB1F1B32F0406A9AE62A4A810D5A9686945A327CA4141A26FB07B2F3361E3E3EC75BAFC4C312F0359
ok
-----

Clearly there are other issues as well. I'm going back to the FIPS 180-4
document, which has a clear description of the algorithm in section 6.4.
It's not clear where step 4 is being accomplished in the Forth code --
it should be performed in SHA512_Transform, but if so, it is not apparent.

--
Krishna

On 2/2/22 08:40, Krishna Myneni wrote:
> I want to start a new thread for the purpose of debugging Forth code
> posted by Marcel Hendrix for the computation of SHA512 hash on a buffer
> of data:

The Forth implementation is failing on

( e f g h a b c d ) 4 ROUND512_0_TO_15_e

The value of "e" is incorrect: B47B...

Per the NIST values for this round (t=3), the value of "e" should be

0B47...

--
Krishna

On 2/2/22 16:47, Krishna Myneni wrote:
> On 2/2/22 08:40, Krishna Myneni wrote:
>> I want to start a new thread for the purpose of debugging Forth code
>> posted by Marcel Hendrix for the computation of SHA512 hash on a
>> buffer of data:
>
> The Forth implementation is failing on
>
> ( e f g h a b c d )   4 ROUND512_0_TO_15_e
>
> The value of "e" is incorrect: B47B...
>
> Per the NIST values for this round (t=3), the value of "e" should be
>
> 0B47...
>

Red herring... the numbers are the same, just the formatting was different.

KM

On 2/2/22 08:40, Krishna Myneni wrote:
> I want to start a new thread for the purpose of debugging Forth code
> posted by Marcel Hendrix for the computation of SHA512 hash on a buffer
> of data:
>
> https://groups.google.com/g/comp.lang.forth/c/3TH1hHgia9Q/m/Jd_HPeSuAQAJ
>
....
> https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th
>

A version which computes the correct hashes for the example strings is
now available.

The revised version does not use the unrolled loops -- it is about 20%
slower under kforth64.

https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th

--
Krishna

-----------
$ kforth64
kForth-64 v 0.2.4 (Build: 2022-01-18)
Copyright (c) 1998--2022 Krishna Myneni
Contributions by: dpw gd mu bk abs tn cmb bg dnw imss
Provided under the GNU Affero General Public License, v3.0 or later

Ready!
include ans-words
ok
include modules
ok
include strings
ok
include utils
ok
include dump
ok
include sha512
ok
also sha512
ok
s" abc" SHA512_Data cr type

DDAF35A193617ABACC417349AE20413112E6FA4E89A97EA20A9EEEE64B55D39A
2192992A274FC1A836BA3C23A3FEEBBD454D4423643CE80E2A9AC94FA54CA49F ok
ok
S"
abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmnhijklmnoijklmnopjklmnopqklmnopqrlmnopqrsmnopqrstnopqrstu"
SHA512_Data TYPE
8E959B75DAE313DA8CF4F72814FC143F8F7779C6EB9F7FA17299AEADB6889018
501D289E4900F7E4331B99DEC4B5433AC7D329EEB6DD26545E96E55B874BE909 ok

---------
( output lines are split for easier reading )

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>On 2/2/22 08:40, Krishna Myneni wrote:
>> I want to start a new thread for the purpose of debugging Forth code
>> posted by Marcel Hendrix for the computation of SHA512 hash on a buffer
>> of data:
>>
>> https://groups.google.com/g/comp.lang.forth/c/3TH1hHgia9Q/m/Jd_HPeSuAQAJ
>>
>...
>> https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th
>>
>
>A version which computes the correct hashes for the example strings is
>now available.

Great. What were the bugs? A diff to the old version might be
sufficient.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On Thursday, February 3, 2022 at 12:39:58 AM UTC-6, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:

> >> https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th
> >>
> >
> >A version which computes the correct hashes for the example strings is
> >now available.
> Great. What were the bugs? A diff to the old version might be
> sufficient.

The program is substantially different. It uses the loop version of the algorithm, which is more directly comparable to the algorithm description provided in FIPS 180-4. Aaron Gifford's program provided two implementations of SHA512_Transform, one being the loop version and another which used the C macros ROUND512_0_TO_15 and ROUND512 to provide an unrolled implementation. M.H.'s posted code followed the unrolled implementation; however, his ROUND512_x (including the ROUND512_0_TO_15_x) words did not reproduce the same logic as the C macros. In particular, they did not shift the values in a--h per the algorithm on each round, e.g.

g TO h
f TO g
e TO f
etc.

The C macros achieved this through a re-ordering of parameters for their macros. But the Forth ROUND512_x words did not implement this. The logic of the unrolled version, particularly the Forth implementation, is hideously difficult to compare with the algorithm description -- the problem was only found by comparing the intermediate values of a--h after each round against the NIST test case documentation.

An additional problem with M.H.'s posted version was that it skipped a step in the algorithm, namely adding the computed a--h values to values accumulated for each pass through SHA512_Transform. This was apparently an oversight in the Forth version since the C versions of SHA512_Transform explicitly performed the accumulation.

It should be easy to go back now and unroll the working looped version, for those who are interested in extreme optimization. It may also be possible to patch the ROUND512_x words, but I think that will be more difficult than simply unrolling the working version.

--
Krishna

Krishna Myneni <km3611.2069@gmail.com> writes:
> M.H.'s posted code followed the unrolled implementation; however, his ROUN=
>D512_x (including the ROUND512_0_TO_15_x) words did not reproduce the same =
>logic as the C macros. In particular, they did not shift the values in a--h=
> per the algorithm on each round, e.g.
>
>g TO h
>f TO g
>e TO f
>etc.
>
>The C macros achieved this through a re-ordering of parameters for their ma=
>cros.

Marcel Hendrix achieved the rotation by having 8 copies of the two
ROUND512 words and rotating the usage of the a-h names. This logic
was so obvious to me that I rerolled it without problems. The result
is <https://www.complang.tuwien.ac.at/forth/programs/sha512.4th> and
it produces the same result as Marcel Hendrix' unrolled code.

>An additional problem with M.H.'s posted version was that it skipped a step=
> in the algorithm, namely adding the computed a--h values to values accumul=
>ated for each pass through SHA512_Transform. This was apparently an oversig=
>ht in the Forth version since the C versions of SHA512_Transform explicitly=
> performed the accumulation.

Yes, I don't see this in Marcel Hendrix' version.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2/4/22 12:06, Anton Ertl wrote:
> Krishna Myneni <km3611.2069@gmail.com> writes:
>> M.H.'s posted code followed the unrolled implementation; however, his ROUN=
>> D512_x (including the ROUND512_0_TO_15_x) words did not reproduce the same =
>> logic as the C macros. In particular, they did not shift the values in a--h=
>> per the algorithm on each round, e.g.
>>
>> g TO h
>> f TO g
>> e TO f
>> etc.
>>
>> The C macros achieved this through a re-ordering of parameters for their ma=
>> cros.
>
> Marcel Hendrix achieved the rotation by having 8 copies of the two
> ROUND512 words and rotating the usage of the a-h names. This logic
> was so obvious to me that I rerolled it without problems. The result
> is <https://www.complang.tuwien.ac.at/forth/programs/sha512.4th> and
> it produces the same result as Marcel Hendrix' unrolled code.
> ...

I came to the conclusion that his logic in the ROUND512_x words did not
work, i.e. it does not reproduce the computation of the SHA512 algorithm
at the end of the rounds. When I have some time I will trace the steps
to show why I came to this conclusion. The fact that your rolled results
agree with his does not mean that they agree with the SHA512
computation. If they did, a--h would have the same values after the 80th
round (t = 79) as shown in

https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Standards-and-Guidelines/documents/examples/SHA512.pdf

You can display the a--h values at the end of SHA512_Transform and
compare them with the a--h values shown on page 4, t=79, for the test
case, "abc".

--
Krishna

On 2/4/22 12:06, Anton Ertl wrote:
> Krishna Myneni <km3611.2069@gmail.com> writes:
>> M.H.'s posted code followed the unrolled implementation; however, his ROUN=
>> D512_x (including the ROUND512_0_TO_15_x) words did not reproduce the same =
>> logic as the C macros. In particular, they did not shift the values in a--h=
>> per the algorithm on each round, e.g.
>>
>> g TO h
>> f TO g
>> e TO f
>> etc.
>>
>> The C macros achieved this through a re-ordering of parameters for their ma=
>> cros.
>
> Marcel Hendrix achieved the rotation by having 8 copies of the two
> ROUND512 words and rotating the usage of the a-h names. This logic
> was so obvious to me that I rerolled it without problems.
I think you misconstrued my earlier statement, given below.

"The logic of the unrolled version, particularly the Forth
implementation, is hideously difficult to compare with the algorithm
description ..."

I'm not saying that the Forth implementation logic is hideous. I'm
saying it is quite difficult to tell whether the code is doing what the
algorithm is specifying.

--
Krishna

On 2/2/22 22:45, Krishna Myneni wrote:
> On 2/2/22 08:40, Krishna Myneni wrote:
>> I want to start a new thread for the purpose of debugging Forth code
>> posted by Marcel Hendrix for the computation of SHA512 hash on a
>> buffer of data:
>>
>> https://groups.google.com/g/comp.lang.forth/c/3TH1hHgia9Q/m/Jd_HPeSuAQAJ
>>
> ...
>> https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th
>>
>
> A version which computes the correct hashes for the example strings is
> now available.
>
> The revised version does not use the unrolled loops -- it is about 20%
> slower under kforth64.
>
> https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th
>
> --
> Krishna
>
> -----------
> $ kforth64
> kForth-64 v 0.2.4     (Build: 2022-01-18)
> Copyright (c) 1998--2022 Krishna Myneni
> Contributions by: dpw gd mu bk abs tn cmb bg dnw imss
> Provided under the GNU Affero General Public License, v3.0 or later
>
>
> Ready!
> include ans-words
>  ok
> include modules
>  ok
> include strings
>  ok
> include utils
>  ok
> include dump
>  ok
> include sha512
>  ok
> also sha512
>  ok
> s" abc" SHA512_Data cr type
>
> DDAF35A193617ABACC417349AE20413112E6FA4E89A97EA20A9EEEE64B55D39A
> 2192992A274FC1A836BA3C23A3FEEBBD454D4423643CE80E2A9AC94FA54CA49F ok
>  ok
> S"
> abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmnhijklmnoijklmnopjklmnopqklmnopqrlmnopqrsmnopqrstnopqrstu"
> SHA512_Data TYPE
> 8E959B75DAE313DA8CF4F72814FC143F8F7779C6EB9F7FA17299AEADB6889018
> 501D289E4900F7E4331B99DEC4B5433AC7D329EEB6DD26545E96E55B874BE909 ok
>
> ---------
> ( output lines are split for easier reading )
>

To go beyond the simple test cases above, and perform the SHA512 hashes
for arbitrary files, below is a definition of FILE_SHA512 which takes a
filename ( caddr u -- ) and displays its SHA512 hash. The output for a
file of over 0.5 GB length is also demonstrated below, using FILE_SHA512
and the standard Linux utility, sha512sum.

The computation under kforth64 is slow for such a large file size. It
will be interesting to see how the working version of sha512.4th or
optimized versions of it perform on different Forth systems.

--
Krishna

( sha512-test.4th )

include ans-words
include strings
include files
include utils
include dump
include modules
include sha512

also sha512

variable rfileid
create buf[] SHA512_BLOCK_LENGTH ALLOT

\ Read n bytes from input file, store at addr array
: bytes@ ( adr u -- ) rfileid @ READ-FILE 2DROP ;
: block@ buf[] SHA512_BLOCK_LENGTH bytes@ ;

: File_SHA512 ( caddr u -- )
R/O BIN OPEN-FILE SWAP rfileid !
ABORT" Invalid input file."
SHA512_Init \ Valid file, init transform
rfileid @ FILE-SIZE DROP ( ud ) \ Get bytesize of input file
CR ." Bytesize: " 2DUP UD.
SHA512_BLOCK_LENGTH UM/MOD ( rembytes nblocks ) \ Compute
nblocks & rembytes
0 ?DO
block@
buf[] SHA512_BLOCK_LENGTH SHA512_Update
LOOP \ Do n full blocks
buf[] swap 2dup bytes@
SHA512_Update
SHA512_End
CR TYPE CR \ Show SHA512 hash for file
rfileid @ CLOSE-FILE DROP
;

( end of file )

--- demonstration of File_SHA512

s" newsounds_4371.wav" File_SHA512

Bytesize: 582120044
1071FD92DF5DF8378212FFA1046C4B2922520DD12B47CC8BCE56B68C3EDDE428A6839529CEDF110245B12C69AB4905496B42669A2C7388F13A78CDE7009DF82D
---

--- comparison with Linux utility, sha512sum
$ sha512sum newsounds_4371.wav
1071fd92df5df8378212ffa1046c4b2922520dd12b47cc8bce56b68c3edde428a6839529cedf110245b12c69ab4905496b42669a2c7388f13a78cde7009df82d
newsounds_4371.wav
---

On 2/4/22 13:52, Krishna Myneni wrote:
> On 2/2/22 22:45, Krishna Myneni wrote:

> The computation under kforth64 is slow for such a large file size. It
> will be interesting to see how the working version of sha512.4th or
> optimized versions of it perform on different Forth systems.
>
....

A revised version of sha512.4th, with improved efficiency of 15--20%,
has been posted. The new version uses inlining of some of the functions
used by the algorithm and improves the Forth coding of the main hashing
word, SHA512_TRANSFORM.

https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th

Test results below under kforth64 and 64-bit Gforth. On my system, plain
Gforth appears to have a hashing speed of about 11.7 MiB/s and kforth64
a hashing speed of 3.5 MiB/s. These numbers indicate a practical use
limit for file sizes of < 100 MB with the present code, which is good
for hashing user documents (word processing, spreadsheet, presentations,
source code archives, compressed audio, and other user data), but not
really usable for file sizes of hundreds of megabytes to gigabytes of
data without additional optimizations.

The output hash for the test file (size 582,120,044 bytes) was verified
by the Linux utility, sha512sum.

--
Krishna

kforth64 Tests
----
$ kforth64
kForth-64 v 0.2.4 (Build: 2022-01-18)
Copyright (c) 1998--2022 Krishna Myneni
Contributions by: dpw gd mu bk abs tn cmb bg dnw imss
Provided under the GNU Affero General Public License, v3.0 or later

Ready!
include sha512-test

( clip file load messages from default kForth directory )
ok
SHASPEED

Processing 40 Mbytes ... 11225 ms elapsed
ok
ms@ s" newsounds_4371.wav" File_SHA512 ms@ swap - cr . .( ms)

Bytesize: 582120044
1071FD92DF5DF8378212FFA1046C4B2922520DD12B47CC8BCE56B68C3EDDE428A6839529CEDF110245B12C69AB4905496B42669A2C7388F13A78CDE7009DF82D

166417 ms ok
----

Gforth 0.7.9_20220120
----
$ ../gforth
Gforth 0.7.9_20220120
Authors: Anton Ertl, Bernd Paysan, Jens Wilke et al., for more type
`authors'
Copyright © 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<https://gnu.org/licenses/gpl.html>
Gforth comes with ABSOLUTELY NO WARRANTY; for details type `license'
Type `help' for basic help
include sha512-test.fs
utils.fs:41:43: warning: redefined table
table.fs:37:3: warning: original location
utils.fs:66:1: warning: redefined place
kernel/basics.fs:138:3: warning: original location ok
SHASPEED
Processing 40 Mbytes ... 3378 ms elapsed
ok
ms@ s" newsounds_4371.wav" File_SHA512 ms@ swap - cr . .( ms)
Bytesize: 582120044
1071FD92DF5DF8378212FFA1046C4B2922520DD12B47CC8BCE56B68C3EDDE428A6839529CEDF110245B12C69AB4905496B42669A2C7388F13A78CDE7009DF82D

49706 ms ok
----

To run sha512-test.fs under Gforth requires a set of compatibility words
provided in kforth-compat.fs, and the additional files, modules.fs,
strings.fs, and utils.fs (equivalent versions of kforth .4th files).

kforth-compat.fs
----
\ kforth-compat.fs
\ \ kForth compatibility defns. for gforth
\ \ Revised: 2022-02-05

utime 2constant start_time

base @
DECIMAL
: deg>rad ( F: r1 -- r2 ) PI f* 180.0e0 f/ ;
: rad>deg ( F: r1 -- r2 ) 180.0e0 f* PI f/ ;
: fround>s ( F: r -- ) ( -- s ) fround f>s ;
: ftrunc>s ( F: r -- ) ( -- s ) ftrunc f>s ;
: allot? ( u -- a ) here swap allot ;
: 2+ 2 + ;
: 2- 2 - ;
: ms@ ( -- u ) utime start_time d- 1000 um/mod nip ;
: us2@ ( -- ud ) utime start_time d- ;
: usleep ( u -- ) 1000 / 1 max ms ;
: nondeferred ( -- ) ;
synonym a@ @
synonym ptr value
base !
----

sha512-test.fs
----
\ Test file hashes for the Forth SHA512 implementation.
\

include kforth-compat.fs
include strings.fs
include utils.fs
include modules.fs
include sha512.fs

also sha512

variable rfileid
create buf[] SHA512_BLOCK_LENGTH ALLOT

\ Read n bytes from input file, store at addr array
: bytes@ ( adr u -- ) rfileid @ READ-FILE 2DROP ;
: block@ buf[] SHA512_BLOCK_LENGTH bytes@ ;

: File_SHA512 ( caddr u -- )
R/O BIN OPEN-FILE SWAP rfileid !
ABORT" Invalid input file."
SHA512_Init \ Valid file, init transform
rfileid @ FILE-SIZE DROP ( ud ) \ Get bytesize of input file
CR ." Bytesize: " 2DUP UD.
SHA512_BLOCK_LENGTH UM/MOD ( rembytes nblocks ) \ Compute
nblocks & rembytes
0 ?DO
block@
buf[] SHA512_BLOCK_LENGTH SHA512_Update
LOOP \ Do n full blocks
buf[] swap 2dup bytes@
SHA512_Update
SHA512_End
CR TYPE CR \ Show SHA512 hash for file
rfileid @ CLOSE-FILE DROP
; ----

( sha512.fs and sha512.4th are exactly the same file )

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>Test results below under kforth64 and 64-bit Gforth. On my system, plain
>Gforth appears to have a hashing speed of about 11.7 MiB/s and kforth64
>a hashing speed of 3.5 MiB/s.

Which CPU do you use?

>$ ../gforth

For performance I recommend gforth-fast

>include sha512-test.fs

sha512-test.fs is not (yet) in <https://github.com/mynenik/kForth-64>

Ah, here:

>sha512-test.fs
>----
>\ Test file hashes for the Forth SHA512 implementation.
....

Results later.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2/6/22 12:51, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> Test results below under kforth64 and 64-bit Gforth. On my system, plain
>> Gforth appears to have a hashing speed of about 11.7 MiB/s and kforth64
>> a hashing speed of 3.5 MiB/s.
>
> Which CPU do you use?
>

Intel Core i5-8400 2.8 GHz (6 cores)

>> $ ../gforth
>
> For performance I recommend gforth-fast
>

The hashing speed improves by ~2x over plain Gforth, to about 24.2 MB/s
(the units of my previously posted numbers should have been MB/s, not
MiB/s).

---
$ ../gforth-fast
Gforth 0.7.9_20220120
Authors: Anton Ertl, Bernd Paysan, Jens Wilke et al., for more type
`authors'
Copyright © 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<https://gnu.org/licenses/gpl.html>
Gforth comes with ABSOLUTELY NO WARRANTY; for details type `license'
Type `help' for basic help
include sha512-test.fs
utils.fs:41:43: warning: redefined table
table.fs:37:3: warning: original location
utils.fs:66:1: warning: redefined place
kernel/basics.fs:138:3: warning: original location ok
SHASPEED
Processing 40 Mbytes ... 1640 ms elapsed
ok
ms@ s" newsounds_4371.wav" File_SHA512 ms@ swap - cr . .( ms)
Bytesize: 582120044
1071FD92DF5DF8378212FFA1046C4B2922520DD12B47CC8BCE56B68C3EDDE428A6839529CEDF110245B12C69AB4905496B42669A2C7388F13A78CDE7009DF82D

24211 ms ok
---

>> include sha512-test.fs
>
> sha512-test.fs is not (yet) in <https://github.com/mynenik/kForth-64>
>
> Ah, here:
>
>> sha512-test.fs
>> ----
>> \ Test file hashes for the Forth SHA512 implementation.
> ...
>
> Results later.
>
----
A simplified sha512-test.fs for Gforth is the following. strings.fs and
utils.fs were only needed to provide the defn of TABLE which has been
added below. The modules.fs from the kForth repo may be used once
kforth-compat.fs has been included.

--
\ Test file hashes for the Forth SHA512 implementation.
\

include kforth-compat.fs
\ include strings.fs
\ include utils.fs
include modules.fs

: table ( v1 v2 ... vn n <name> -- | create a table of singles )
create dup cells allot? over 1- cells + swap
0 ?do dup >r ! r> 1 cells - loop drop ;

include sha512.fs

also sha512

variable rfileid
create buf[] SHA512_BLOCK_LENGTH ALLOT

\ Read n bytes from input file, store at addr array
: bytes@ ( adr u -- ) rfileid @ READ-FILE 2DROP ;
: block@ buf[] SHA512_BLOCK_LENGTH bytes@ ;

: File_SHA512 ( caddr u -- )
R/O BIN OPEN-FILE SWAP rfileid !
ABORT" Invalid input file."
SHA512_Init \ Valid file, init transform
rfileid @ FILE-SIZE DROP ( ud ) \ Get bytesize of input file
CR ." Bytesize: " 2DUP UD.
SHA512_BLOCK_LENGTH UM/MOD ( rembytes nblocks ) \ Compute
nblocks & rembytes
0 ?DO
block@
buf[] SHA512_BLOCK_LENGTH SHA512_Update
LOOP \ Do n full blocks
buf[] swap 2dup bytes@
SHA512_Update
SHA512_End
CR TYPE CR \ Show SHA512 hash for file
rfileid @ CLOSE-FILE DROP
; ----

Krishna

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>A simplified sha512-test.fs for Gforth is the following. strings.fs and
>utils.fs were only needed to provide the defn of TABLE which has been
>added below. The modules.fs from the kForth repo may be used once
>kforth-compat.fs has been included.

But I don't find kforth-compat.fs in the kForth-64 project.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2/6/22 16:44, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> A simplified sha512-test.fs for Gforth is the following. strings.fs and
>> utils.fs were only needed to provide the defn of TABLE which has been
>> added below. The modules.fs from the kForth repo may be used once
>> kforth-compat.fs has been included.
>
> But I don't find kforth-compat.fs in the kForth-64 project.
>

It is not there. I listed it in an earlier message in this thread (it is
reproduced below). I should probably add a compatibility folder to the
kForth-64 repo, to make it easier to run code from it in standard Forths
without modifications.

--
Krishna

kforth-compat.fs
---
\ kforth-compat.fs
\ \ kForth compatibility defns. for gforth
\ \ Revised: 2022-02-05

utime 2constant start_time

base @
DECIMAL
: deg>rad ( F: r1 -- r2 ) PI f* 180.0e0 f/ ;
: rad>deg ( F: r1 -- r2 ) 180.0e0 f* PI f/ ;
: fround>s ( F: r -- ) ( -- s ) fround f>s ;
: ftrunc>s ( F: r -- ) ( -- s ) ftrunc f>s ;
: allot? ( u -- a ) here swap allot ;
: 2+ 2 + ;
: 2- 2 - ;
: ms@ ( -- u ) utime start_time d- 1000 um/mod nip ;
: us2@ ( -- ud ) utime start_time d- ;
: usleep ( u -- ) 1000 / 1 max ms ;
: nondeferred ( -- ) ;
synonym a@ @
synonym ptr value
base !
---

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>I listed it in an earlier message in this thread (it is
>reproduced below). I should probably add a compatibility folder to the
>kForth-64 repo, to make it easier to run code from it in standard Forths
>without modifications.

Would be great. I have extended kforth-compat.fs to load on VFX64 and
iForth 5.1-mini, see below. Of course, thanks to VFX64 directory
handling, the result works only if the working directory is the one
that contains all the files.

Results on a Ryzen 7 5800X:

VFX Forth 64 5.11 RC2 [build 0112] 2021-05-02 for Linux x64
Processing 40 Mbytes ... 200 ms elapsed
Performance counter stats for 'vfx64 include sha512-test.fs shaspeed bye':
810,436,137 cycles
3,785,560,513 instructions # 4.67 insn per cycle

iforth 5.1-mini (no useful replacement for ms@ in the glossary,
so no useful Forth-internal timing result):
Performance counter stats for 'iforth include /nfs/a5/anton/forth/kForth-64/forth-src/sha512-test.fs shaspeed bye':
1,161,347,179 cycles
3,410,826,133 instructions # 2.94 insn per cycle

gforth-fast 0.7.9_20211216
Processing 40 Mbytes ... 1470 ms elapsed
Performance counter stats for 'gforth-fast -e include /nfs/a5/anton/forth/kForth-64/forth-src/sha512-test.fs shaspeed bye':
6,977,023,865 cycles
19,855,889,207 instructions # 2.85 insn per cycle

One interesting aspect here is that in many other programs, VFX has
lower "insn per cycle" IPC results than gforth-fast; basically, in
these other programs, the data flow of the computation we are
interested in has so many dependencies that the additional
instructions of Gforth are for a good part absorbed by the many
resources of wide cores like the Zen3 core of the Ryzen 7 5800X. This
program, however, apparently has so much instruction-level parallelism
that VFX64 can come close to the limits of what Zen3 can provide;
gforth-fast does not reach as much IPC, which indicates that data
dependencies through the threaded-code instruction pointer or
data-stack pointer limit the amount of instruction-level paralleism
that the CPU can exploit.

Below you find my enhanced kforth-compat.fs.

- anton

\ kforth-compat.fs
\ \ kForth compatibility defns. for gforth
\ \ Revised: 2022-02-05

[defined] utime [if]
utime 2constant start_time
[else]
[undefined] ticks [if]
0 constant ticks \ just a useless dummy
[then]
ticks constant start_time
[then]

[undefined] pi [if]
3.14159265358979e fconstant pi
[then]

base @
DECIMAL
: deg>rad ( F: r1 -- r2 ) PI f* 180.0e0 f/ ;
: rad>deg ( F: r1 -- r2 ) 180.0e0 f* PI f/ ;
: fround>s ( F: r -- ) ( -- s ) fround f>s ;
: ftrunc>s ( F: r -- ) ( -- s ) ftrunc f>s ;
: allot? ( u -- a ) here swap allot ;
: 2+ 2 + ;
: 2- 2 - ;
[defined] utime [if]
: ms@ ( -- u ) utime start_time d- 1000 um/mod nip ;
: us2@ ( -- ud ) utime start_time d- ;
[else]
: ms@ ( -- u ) ticks start_time - ;
: us2@ ( -- ud ) ms@ 1000 um* ;
[then]
: usleep ( u -- ) 1000 / 1 max ms ;
: nondeferred ( -- ) ;
[defined] synonym [if]
synonym a@ @
synonym ptr value
[else]
: a@ @ ;
: ptr value ;
[then]
base !

[undefined] ud. [if]
: ud. 0 ud.r space ;
[then]

On 2/5/22 22:38, Krishna Myneni wrote:
....
> A revised version of sha512.4th, with improved efficiency of 15--20%,
> has been posted. The new version uses inlining of some of the functions
> used by the algorithm and improves the Forth coding of the main hashing
> word, SHA512_TRANSFORM.
>
> https://github.com/mynenik/kForth-64/blob/master/forth-src/sha512.4th
>
....
> kforth64 Tests
> ----
> $ kforth64
> kForth-64 v 0.2.4     (Build: 2022-01-18)
> Copyright (c) 1998--2022 Krishna Myneni
> Contributions by: dpw gd mu bk abs tn cmb bg dnw imss
> Provided under the GNU Affero General Public License, v3.0 or later
>
>
> Ready!
> include sha512-test
>
> ( clip file load messages from default kForth directory )
>  ok
> SHASPEED
>
> Processing 40 Mbytes ...  11225 ms elapsed
>  ok
> ms@ s" newsounds_4371.wav" File_SHA512 ms@ swap - cr . .( ms)
>
> Bytesize: 582120044
> 1071FD92DF5DF8378212FFA1046C4B2922520DD12B47CC8BCE56B68C3EDDE428A6839529CEDF110245B12C69AB4905496B42669A2C7388F13A78CDE7009DF82D
>
>
> 166417 ms ok
> ----
....

The program now uses inlined fixed bit rotations to avoid unnecessary
arithmetic with the general ROR definition which takes the number of
bits as an argument.

:inline ROR1 DUP 1 RSHIFT SWAP 63 LSHIFT OR ;
:inline ROR7 DUP 7 RSHIFT SWAP 57 LSHIFT OR ;
:inline ROR8 DUP 8 RSHIFT SWAP 56 LSHIFT OR ;
:inline ROR14 DUP 14 RSHIFT SWAP 50 LSHIFT OR ;
:inline ROR18 DUP 18 RSHIFT SWAP 46 LSHIFT OR ;
:inline ROR19 DUP 19 RSHIFT SWAP 45 LSHIFT OR ;
:inline ROR28 DUP 28 RSHIFT SWAP 36 LSHIFT OR ;
:inline ROR34 DUP 34 RSHIFT SWAP 30 LSHIFT OR ;
:inline ROR39 DUP 39 RSHIFT SWAP 25 LSHIFT OR ;
:inline ROR41 DUP 41 RSHIFT SWAP 23 LSHIFT OR ;
:inline ROR61 DUP 61 RSHIFT SWAP 3 LSHIFT OR ;

In kforth64 this change increases the hashing rate from 3.5 MB/s to 4.1
MB/s, a sizeable improvement.

---
ms@ s" newsounds_4371.wav" File_SHA512 ms@ swap - cr . .( ms)

Bytesize: 582120044
1071FD92DF5DF8378212FFA1046C4B2922520DD12B47CC8BCE56B68C3EDDE428A6839529CEDF110245B12C69AB4905496B42669A2C7388F13A78CDE7009DF82D

142350 ms ok
---

--
Krishna

On 2/8/22 05:58, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> I listed it in an earlier message in this thread (it is
>> reproduced below). I should probably add a compatibility folder to the
>> kForth-64 repo, to make it easier to run code from it in standard Forths
>> without modifications.
>
> Would be great. I have extended kforth-compat.fs to load on VFX64 and
> iForth 5.1-mini, see below. Of course, thanks to VFX64 directory
> handling, the result works only if the working directory is the one
> that contains all the files.
>
> Results on a Ryzen 7 5800X:
>
> VFX Forth 64 5.11 RC2 [build 0112] 2021-05-02 for Linux x64
> Processing 40 Mbytes ... 200 ms elapsed
> Performance counter stats for 'vfx64 include sha512-test.fs shaspeed bye':
> 810,436,137 cycles
> 3,785,560,513 instructions # 4.67 insn per cycle
>
> iforth 5.1-mini (no useful replacement for ms@ in the glossary,
> so no useful Forth-internal timing result):
> Performance counter stats for 'iforth include /nfs/a5/anton/forth/kForth-64/forth-src/sha512-test.fs shaspeed bye':
> 1,161,347,179 cycles
> 3,410,826,133 instructions # 2.94 insn per cycle
>
> gforth-fast 0.7.9_20211216
> Processing 40 Mbytes ... 1470 ms elapsed
> Performance counter stats for 'gforth-fast -e include /nfs/a5/anton/forth/kForth-64/forth-src/sha512-test.fs shaspeed bye':
> 6,977,023,865 cycles
> 19,855,889,207 instructions # 2.85 insn per cycle
>
> One interesting aspect here is that in many other programs, VFX has
> lower "insn per cycle" IPC results than gforth-fast; basically, in
> these other programs, the data flow of the computation we are
> interested in has so many dependencies that the additional
> instructions of Gforth are for a good part absorbed by the many
> resources of wide cores like the Zen3 core of the Ryzen 7 5800X. This
> program, however, apparently has so much instruction-level parallelism
> that VFX64 can come close to the limits of what Zen3 can provide;
> gforth-fast does not reach as much IPC, which indicates that data
> dependencies through the threaded-code instruction pointer or
> data-stack pointer limit the amount of instruction-level paralleism
> that the CPU can exploit.
>

The timing on your system for gforth-fast is comparable to the timing on
my system, with an effective hashing rate of about 27 MB/s. VFX64
appears to have an effective hashing rate of 200 MB/s, which is quite
impressive. Please see my upgrade of sha512.4th from today, which should
further improve the hashing rate for most Forth systems. Perhaps Marcel
can provide an equivalent to ms@ for iForth.

> Below you find my enhanced kforth-compat.fs.
>

Thanks. There is a "forth-src/compat" folder in the kForth-64 repo now,
which includes kforth-compat.fs and modules.fs. I will update it with
your revised version later today.

--
Krishna

On Tuesday, February 8, 2022 at 1:39:02 PM UTC+1, Anton Ertl wrote:
[..]
> Results on a Ryzen 7 5800X:
>
> VFX Forth 64 5.11 RC2 [build 0112] 2021-05-02 for Linux x64
> Processing 40 Mbytes ... 200 ms elapsed
> Performance counter stats for 'vfx64 include sha512-test.fs shaspeed bye':
> 810,436,137 cycles
> 3,785,560,513 instructions # 4.67 insn per cycle
>
> iforth 5.1-mini (no useful replacement for ms@ in the glossary,
> so no useful Forth-internal timing result):
> Performance counter stats for 'iforth include /nfs/a5/anton/forth/kForth-64/forth-src/sha512-test.fs shaspeed bye':
> 1,161,347,179 cycles
> 3,410,826,133 instructions # 2.94 insn per cycle
[..]

Nice to see that it *is* possible to debug some else's Forth code :-E

FORTH> in sha512
Creating --- SHA-512 64-bit Version 0.01 ---

Try: S" abc" SHA512_Data .SHA512
= DDAF35A1 93617ABA CC417349 AE204131 12E6FA4E 89A97EA2
0A9EEEE6 4B55D39A 2192992A 274FC1A8 36BA3C23 A3FEEBBD
454D4423 643CE80E 2A9AC94F A54CA49F

S" abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmnhijklmnoijklmnopjklmnopqklmnopqrlmnopqrsmnopqrstnopqrstu" SHA512_Data .SHA512
= 8E959B75 DAE313DA 8CF4F728 14FC143F 8F7779C6 EB9F7FA1
7299AEAD B6889018 501D289E 4900F7E4 331B99DE C4B5433A
C7D329EE B6DD2654 5E96E55B 874BE909

SHAspeed -- test speed with a 40 MB buffer (>181 MB/sec).
ok
FORTH> S" abc" SHA512_Data .SHA512
DDAF35A1 93617ABA CC417349 AE204131 12E6FA4E 89A97EA2
0A9EEEE6 4B55D39A 2192992A 274FC1A8 36BA3C23 A3FEEBBD
454D4423 643CE80E 2A9AC94F A54CA49F ok
FORTH> S" abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmnhijklmnoijklmnopjklmnopqklmnopqrlmnopqrsmnopqrstnopqrstu" SHA512_Data .SHA512
8E959B75 DAE313DA 8CF4F728 14FC143F 8F7779C6 EB9F7FA1
7299AEAD B6889018 501D289E 4900F7E4 331B99DE C4B5433A
C7D329EE B6DD2654 5E96E55B 874BE909 ok
FORTH> SHAspeed
Processing 40 Mbytes ... 0.184 seconds elapsed. ok

FORTH> .ticker-info
AMD Ryzen 7 5800X 8-Core Processor, TICKS-GET uses iTSC at 4192MHz

Maybe I'll try to optimize it some more. The above is for the do-loop version.

-marcel

On Wednesday, February 9, 2022 at 3:23:05 AM UTC+1, Marcel Hendrix wrote:
[..]
> Maybe I'll try to optimize it some more. The above is for the do-loop version.

FORTH> S" abc" SHA512_Data .SHA512
DDAF35A1 93617ABA CC417349 AE204131 12E6FA4E 89A97EA2
0A9EEEE6 4B55D39A 2192992A 274FC1A8 36BA3C23 A3FEEBBD
454D4423 643CE80E 2A9AC94F A54CA49F ok
FORTH> SHAspeed many
Processing 40 Mbytes ... 0.149 seconds elapsed.
Processing 40 Mbytes ... 0.149 seconds elapsed.
Processing 40 Mbytes ... 0.149 seconds elapsed.
Processing 40 Mbytes ... 0.149 seconds elapsed.
Processing 40 Mbytes ... 0.149 seconds elapsed.
Processing 40 Mbytes ... 0.149 seconds elapsed. ok
FORTH> 1000 40 149 */ . 268 ok

-marcel

On Tuesday, February 8, 2022 at 9:06:03 PM UTC-6, Marcel Hendrix wrote:
> On Wednesday, February 9, 2022 at 3:23:05 AM UTC+1, Marcel Hendrix wrote:
> [..]
> > Maybe I'll try to optimize it some more. The above is for the do-loop version.
> FORTH> S" abc" SHA512_Data .SHA512
> DDAF35A1 93617ABA CC417349 AE204131 12E6FA4E 89A97EA2
> 0A9EEEE6 4B55D39A 2192992A 274FC1A8 36BA3C23 A3FEEBBD
> 454D4423 643CE80E 2A9AC94F A54CA49F ok
> FORTH> SHAspeed many
> Processing 40 Mbytes ... 0.149 seconds elapsed.
> Processing 40 Mbytes ... 0.149 seconds elapsed.
> Processing 40 Mbytes ... 0.149 seconds elapsed.
> Processing 40 Mbytes ... 0.149 seconds elapsed.
> Processing 40 Mbytes ... 0.149 seconds elapsed.
> Processing 40 Mbytes ... 0.149 seconds elapsed. ok
> FORTH> 1000 40 149 */ . 268 ok
>
> -marcel

The word SLIDE_WITH_ADD in the Loop version is actually a shift register
operation on a...h, with 'h' being shifted out on the right, and T1 + T2 being
shifted in on the left. Additonally, one of the elements, 'd', was summed
with T1 before being shifted to the position of 'e' during the shift operation.

Your original unrolled code did not reproduce this type of operation. There
were additional bugs also.

--
Krishna

On Wednesday, February 9, 2022 at 5:59:45 AM UTC+1, km361...@gmail.com wrote:
> On Tuesday, February 8, 2022 at 9:06:03 PM UTC-6, Marcel Hendrix wrote:
> > On Wednesday, February 9, 2022 at 3:23:05 AM UTC+1, Marcel Hendrix wrote:
> > [..]
> > > Maybe I'll try to optimize it some more. The above is for the do-loop version.
> > FORTH> S" abc" SHA512_Data .SHA512
> > DDAF35A1 93617ABA CC417349 AE204131 12E6FA4E 89A97EA2
> > 0A9EEEE6 4B55D39A 2192992A 274FC1A8 36BA3C23 A3FEEBBD
> > 454D4423 643CE80E 2A9AC94F A54CA49F ok
> > FORTH> SHAspeed many
> > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > Processing 40 Mbytes ... 0.149 seconds elapsed. ok
> > FORTH> 1000 40 149 */ . 268 ok
> >
> > -marcel
> The word SLIDE_WITH_ADD in the Loop version is actually a shift register
> operation on a...h, with 'h' being shifted out on the right, and T1 + T2 being
> shifted in on the left. Additonally, one of the elements, 'd', was summed
> with T1 before being shifted to the position of 'e' during the shift operation.
>
> Your original unrolled code did not reproduce this type of operation. There
> were additional bugs also.

Sorry if I was unclear. I consulted your repository, specifically
the earliest modification that produced correct results.
That version had a DO-LOOP, (no SLIDE_WITH_ADD, though) and
I wanted to see how that worked out, so kept it in place.

The version that I originally posted did not have the line-by-line
debugging code from the C version (too bulky for a post), but
with that in place I quickly got it working (at least the examples).

T1 and T2 are removed from the present code as they can be kept
on the stack (that is actually the 35ms speed up in my last result).
Given the unexpectedly high influence of T1 and T2, looking at
the fetch and store patterns might be beneficial.

-marcel

On Wednesday, February 9, 2022 at 9:52:33 AM UTC+1, Marcel Hendrix wrote:
> On Wednesday, February 9, 2022 at 5:59:45 AM UTC+1, km361...@gmail.com wrote:
> > On Tuesday, February 8, 2022 at 9:06:03 PM UTC-6, Marcel Hendrix wrote:
> > > On Wednesday, February 9, 2022 at 3:23:05 AM UTC+1, Marcel Hendrix wrote:
> > > [..]
> > > > Maybe I'll try to optimize it some more. The above is for the do-loop version.
> > > FORTH> S" abc" SHA512_Data .SHA512
> > > DDAF35A1 93617ABA CC417349 AE204131 12E6FA4E 89A97EA2
> > > 0A9EEEE6 4B55D39A 2192992A 274FC1A8 36BA3C23 A3FEEBBD
> > > 454D4423 643CE80E 2A9AC94F A54CA49F ok
> > > FORTH> SHAspeed many
> > > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > > Processing 40 Mbytes ... 0.149 seconds elapsed.
> > > Processing 40 Mbytes ... 0.149 seconds elapsed. ok
> > > FORTH> 1000 40 149 */ . 268 ok
[..]
> T1 and T2 are removed from the present code as they can be kept
> on the stack (that is actually the 35ms speed up in my last result).
> Given the unexpectedly high influence of T1 and T2, looking at
> the fetch and store patterns might be beneficial.

Further ( high-level ) tweaks ...

FORTH> SHAspeed many
Processing 40 Mbytes ... 0.114 seconds elapsed.
Processing 40 Mbytes ... 0.113 seconds elapsed.
Processing 40 Mbytes ... 0.113 seconds elapsed.
Processing 40 Mbytes ... 0.113 seconds elapsed.

FORTH> 1000 40 113 */ . 353 ok

353 MB/s, almost as fast as the non-working version.

-marcel