You can download it at http://thebeez.home.xs4all.nl/4tH/download.html

Changes:
* The words +TO, ;THEN, R"@, I', ?EXIT, [NAMES], EXCEPT, UNLESS,
THROW", >ZERO, RDROP, 2RDROP, CASE, OF, ENDOF, ;ENDOF, ENDCASE
and STOW were added.
* The preprocessor now supports single, double and floating point
FORTRAN formulas.
* Destructors and constructors were added to FOOS (Forth Object
Oriented Simple).
* An "expert mode" has been added to 4tH.
* The [NAMES] directive enables symbolic decompilation.
* The behavior of COMPARE can be influenced by issuing the
appropriate pragmas.
* The library files now support Intel hex format.
* Global variables can be converted to local variables.
* New binary search libraries were added.
* Dynamic string support was added.
* Dynamic array support was added.
* CCITT-2 and CRC32 support were added.
* A new priority queue library was added.
* A general purpose filename splitting library was added.
* Internationalization support (i18n) was added.
* A fast Levenshtein Distance library was added.
* A sequences library was added
* A floating point equivalent of >NUMBER was added.
* A fast >FLOAT implementation was added to ANS floating point.
* Fast single and double word exponentiation libraries were
added.
* A fast fixed point exponentiation and natural logarithm library
was added.
* Several mathematical functions were added to the fractions
library.
* The graphics suite now includes routines for drawing circles,
arcs, ellipses and different lines.
* The graphics suite now includes flood fill routines.
* MRG32k3a was added to the random number generators.
* Binary Insertionsort and a simple Timsort were added to the
sort algorithms.
* Preprocessing is 2.5x times faster, compilation is 25% faster
(on average).

Features:
* Comes with an extensive library, featuring hundreds
of words covering ANS-Forth compatibility, user defined stacks,
lists, XLS, LaTeX, RTF and HTML files, floating point,
databases, etc. etc.
* Almost 95% of the CORE wordset is supported.
* Comes with a nice, 600+ page manual.
* Over 250 examples and 500+ libraries.
* Turbo C and GCC like frontends.
* Native packages for MS-DOS, MS-Windows, Coherent, Linux,
including the Ben Nanonote and the Raspberry Pi.

On 2022-01-18, Hans Bezemer <the.beez.speaks@gmail.com> wrote:
> * An "expert mode" has been added to 4tH.

What is "expert mode" about?

Hans Bezemer wrote:
> You can download it at http://thebeez.home.xs4all.nl/4tH/download.html

Hi Hans,

great stuff, many thanks, deeply appreciating the libs, the source
organisation, the doc and the examples.
I copy source from the doc as a text-version from your homepage
to try some examples. 25 years ago i asked you for this service
and you published the text-version since then, extremely helpful.

4tH here is the most favoured program to explore and optimize
the organisation of many of my text tools because it's
tremendously easy to deconstruct and factorize functions and
do conceptually changes.
BTW my preferred editor is TSEpro, now freeware, with an
own build syntax highlighting and function list of 4tH source.

Just one question:
I often use defer and if i comment some out,
have i also to comment out the definition of :noname
or only the part after the closing
semicolon "is xyz" for that noname definition?

Compiling the not needed nonames costs me compile time
not space, right?
Commenting out defer and leaving the :noname as is, costs
me space but is not harmful otherwise, right?

Here you see parts of a function list. It
shows the begin and the end of :noname definitions.

defer shdcn2
\ defer splic
defer adrith
defer hdverg5

:noname \ hdverg5
; is hdverg5
:noname \ shdcn2
; is shdcn2
:noname \ splic
; \ is splic
:noname \ adrith
; is adrith

--
Salut
_)oachim

On Wednesday, January 19, 2022 at 10:58:31 AM UTC+1, j.merkel wrote:

> Just one question:
> I often use defer and if i comment some out,
> have i also to comment out the definition of :noname
> or only the part after the closing
> semicolon "is xyz" for that noname definition?

TL;DR version: BOTH.

Now for the real answer:

Well, I slightly changed your code by adding the [NAMES] directive to it, so you can see EXACTLY what is going on:

[names]

defer shdcn2
\ defer splic
defer adrith
defer hdverg5
\ etc.

4tH message: No errors at word 21
Object size: 21 words
String size: 0 chars
Variables : 3 cells
Strings : 0 chars
Symbols : 3 names
Reliable : Yes

Addr| Opcode Operand Argument

0| literal -9223372036854775808
1| to 0 shdcn2 \ yes, this simply expands to (error) to shdcn2
2| literal -9223372036854775808
3| to 1 adrith \ likewise for adrith
4| literal -9223372036854775808
5| to 2 hdverg5 \ likewise for hdverg5
6| literal 7 \ xt of :noname
7| branch 8 \ :noname definition
8| exit 0
9| to 2 hdverg5 \ now hdverg5 has that xt assigned to it
10| literal 11 \ xt of :noname
11| branch 12 \ :noname definition
12| exit 0
13| to 0 shdcn2 \ now shdcn2 has that xt assigned to it
14| literal 15 \ xt of :noname
15| branch 16
16| exit 0
17| literal 18 \ xt of :noname
18| branch 19
19| exit 0
20| to 1 adrith \ now shdcn2 has that xt assigned to it

So, let me get you in to some dirty 4tH secrets: neither VALUE nor DEFER is actually required. TO (or IS) will create that identifier when it is not there yet.

if ((y = GetSymbol (CurrentWord, W_SEARCH)) == MISSING)
AddSymbol (ValueToken, (val = Object->Variables++), CurrentWord);
else
{
if (Object->SymTable [y].Token != ValueToken)
Object->ErrNo = M4NOTYPE;
val = Object->SymTable [y].Value;
}

It simply queries the Symboltable. If it find it, it simply establishes whether the right thing is requested. Filling a predefined VALUE with IS isn't gonna cut it - so it bombs out with an error. If it doesn't find it, it adds the symbol to the Symboltable and continues. In both cases, the address of the VECTOR or the VALUE is known when compiling.

So, if you want to write tight 4tH code and don't mind portability or properly declaring all the stuff you require, you don't actually *need* to use DEFER ( and save two tokens). I don't actually recommend this practice for large programs, because when maintaining it you wonder where this darn thing actually came from. As you know I tend to declare all stuff beforehand and comment them.

The very reason 4tH behaves like this is that is saves tokens - which are a very limited resource. Using this scheme we can fold TO, IS, DEFER and VALUE into one single token - without sacrificing any ANS Forth compatibility. (ERROR) is a guarantee that you WON'T be able to use it before proper assignment - because it will simply bomb out.

Ok, second part. What does the standard say about :NONAME? Create an execution token xt, enter compilation state and start the current definition, producing colon-sys. ( C: -- colon-sys ) ( S: -- xt ) What this mumbo jumbo actually means it that the darn thing leaves its execution token AFTER the definition so that :NONAME ; EXECUTE actually works. And given gForth, we can see it do it:

Gforth 0.7.9_20211216
Copyright © 2021 Free Software Foundation, Inc.
Type `help' for basic help
:noname cr ." Hello world" cr ; dup . execute 140403878221608
Hello world
ok

So yes, the :NONAME left an 'xt' on the stack that we printed an executed. 4tH does essentially the same thing - it only drops the 'xt' right at the beginning, which is no problem since we jump over the definition anyways.

14| literal 15
15| branch 16
16| exit 0

That means you left a lonely 'xt' on the stack which is *not* consumed - and that is stack pollution. I consider that a problem. Want proof? Let's add a '.' at the very end of this program:

$ pp4th -x joachim.4th
15 $

And yes, that's the 'xt' that was left on the stack at operator #14. So, not commenting out the equivalent :NONAME definition gets you in much worse trouble than simply a little overhead. Personally, I'd rather use :TOKEN definitions, because that is equivalent to ANS Forth:

:NONAME ; CONSTANT bladibla \ which won't compile in 4tH

Let's see hoe what works out:

[names]

defer shdcn2
\ defer splic
defer adrith
defer hdverg5

:token (hdverg5) \ hdverg5
; (hdverg5) is hdverg5

:token (shdcn2) \ shdcn2
; (shdcn2) is shdcn2

:token (splic) \ splic
; \ (splic) is splic

:token (adrith) \ adrith
; (adrith) is adrith

depth .

And decompiled:

Addr| Opcode Operand Argument

0| literal -9223372036854775808
1| to 0 shdcn2
2| literal -9223372036854775808
3| to 1 adrith
4| literal -9223372036854775808
5| to 2 hdverg5
6| branch 7
7| exit 0
8| literal 6
9| to 2 hdverg5
10| branch 11contermination
11| exit 0
12| literal 10
13| to 0 shdcn2
14| branch 15
15| exit 0
16| branch 17
17| exit 0
18| literal 16
19| to 1 adrith
20| sp@ 0
21| . 0

Note I added the latter two to see which effect it has on the stack - and it's clean. It you remove them, the actual program is even a token shorter, because the stack contamination is solved. If you tend to use this practice, it is safer to use :TOKEN definitions.

However, definitions YOU DON'T USE will ALWAYS take up space, since 4tH compiles in one single pass and doesn't redo the whole shebang to see if there is any dead code there. Any optimization that is done is using a very, very tiny window.

A bit lengthier than you expected may be, but here it is..

Hans Bezemer

On Wednesday, January 19, 2022 at 5:54:12 AM UTC+1, meff wrote:
> On 2022-01-18, Hans Bezemer <the.bee...@gmail.com> wrote:
> > * An "expert mode" has been added to 4tH.
> What is "expert mode" about?
4tH is one of the 4tH incarnations, mimicking an early Turbo Pascal. After each command, it repaints the entire menu. Some people use the menu so frequently, they know it by heart and don't feel they need it. pressing 'X' will stop repainting the menu and simply leave only the prompt. An empty entry restores the menu - for those who accidentally enter the mode and don't know their way out.

Hans Bezemer

Hans Bezemer wrote:

> [...]
> If you tend to use this practice, it is safer to use :TOKEN
> definitions.

Now i understand :TOKEN and :NONAME better, i took :NONAME because
it was lesser typing.

> However, definitions YOU DON'T USE will ALWAYS take up space, since 4tH
> compiles in one single pass [...]
I use defer for high speed testing code, so i can have
the main stuff on top of the source file contrary to
the Forth style. This habit came from my older Basic programs.

It's not much work to comment it out with an editor macro --
otherwise i use a self defined library -- the best way to
hold the source readable.

> A bit lengthier than you expected may be, but here it is..

Thank you for hints again exploring the compiler stuff.

--
Salut
_)oachim

On Wednesday, January 19, 2022 at 7:35:13 PM UTC+1, j.merkel wrote:
> I use defer for high speed testing code, so i can have
> the main stuff on top of the source file contrary to
> the Forth style. This habit came from my older Basic programs.

Due to this use case I've implemented a new directive, called [FORWARD]. I think it makes your work a lot easier:

." This is the beginning" cr

[forward] test
test

: test ." Hello world!" cr ;
test

It allows you to defer the implementation of a colon definition. So, you can use it BEFORE it is defined. You later define the word as usual - nothing special. I've tried to break it in any way possible, but it seems rock solid.

It works, because a [FORWARD] declaration defines an empty BRANCH instruction, that is caught later on when you actually define it. If you fail to define it, it simply bombs out - exactly like an undefined DEFER.

Since the overhead is a single jump, I don't think it will break the bank. The thing is quite smart as well. After you've defined the colon definition subsequent use does NOT håve the overhead anymore (the jump is omitted).

Does this help?

Hans Bezemer

Addr| Opcode Operand Argument

0| ." 0 This is the beginning
1| cr 0
2| branch 3 \ here we got the [forward] declaration
3| branch 5
4| call 2 \ here we use the [forward] declaration
5| branch 8 \ this is the implementation of "TEST"
6| ." 22 Hello world!
7| cr 0
8| exit 0
9| call 5 \ now we call "TEST" directly

Hans Bezemer wrote
: > Due to this use case I've implemented a new directive, called [FORWARD].
[...]
> It allows you to defer the implementation of a colon definition.
[...]
> Does this help?

Sounds promising because renaming colon definitions to :noname etc.
becomes superfluous.

defer - :noname definitions primarily seems to be sort of a
library technique and [FORWARD] appropriately used in the main
program.

--
Salut
_)oachim

On Thursday, January 20, 2022 at 1:57:09 PM UTC+1, the.bee...@gmail.com wrote:
> Due to this use case I've implemented a new directive, called [FORWARD]. I think it makes your work a lot easier:

Absolutely useful. LMI Forth had similar words:

F: <name>
used to create a forward reference colon definition (builds just the header)

R: <name> ... ;
used to resolve the forward reference (contains the body of the word)

I used these words very often to create menus in DOS programs.

Henry

On 2022-01-21 12:16, Heinrich Hohl wrote:
> On Thursday, January 20, 2022 at 1:57:09 PM UTC+1, the.bee...@gmail.com wrote:
>> Due to this use case I've implemented a new directive, called [FORWARD]. I think it makes your work a lot easier:
>
> Absolutely useful. LMI Forth had similar words:
>
> F: <name>
> used to create a forward reference colon definition (builds just the header)
>
> R: <name> ... ;
> used to resolve the forward reference (contains the body of the word)
>
> I used these words very often to create menus in DOS programs.
>
> Henry

How are these better than the defer/is paradigm? Seems about the same to me.

On Friday, January 21, 2022 at 12:04:41 PM UTC+1, Ron AARON wrote:
> How are these better than the defer/is paradigm? Seems about the same to me.
In 4tH it is VERY different! DEFER creates a variable that when executed, retrieves the xt stored there and executes it. It can be reassigned, though - since in essence it just stores an xt. This method (which I had to change slightly for architectural reasons) creates a trampoline.

It compiles two BRANCH instructions: BRANCH (this location + 1) and subsequently BRANCH (some outrageous location). This is required, because 4tH's tail call optimization like:

: no1 ." Do something here" ;
: no2 no1 ;

Creates EXACTLY the same two BRANCH instructions MINUS the "outrageous location". This is what the forward counterpart makes it recognize as a valid trampoline for resolving the forward declaration. 4tH won't allow you to write code that results in such a "trampoline".

So under the hood it is a very different implementation. DEFER is quite an expensive one in 4tH terms, because it requires its own token. The trampoline can be settled in the compiler only and doesn't require a variable. The use cases are slightly different as well. Gotta test performance though..

Hans Bezemer

On Friday, January 21, 2022 at 12:23:56 PM UTC+1, Hans Bezemer wrote:
> Gotta test performance though..

Well, the figures are in:

DEFER: 109% versus a direct execution;
FORWARD: 111% versus a direct execution.

Hans Bezemer

Hans Bezemer <the.beez.speaks@gmail.com> writes:
>On Friday, January 21, 2022 at 12:23:56 PM UTC+1, Hans Bezemer wrote:
>> Gotta test performance though..
>
>Well, the figures are in:
>
>DEFER: 109% versus a direct execution;
>FORWARD: 111% versus a direct execution.

Gforth has had a FORWARD in forward.fs for three years, and, since
today, available from the start; this FORWARD works only for colon
definitions and should not cost any performance. Here's a
microbenchmark:

forward foo1
defer foo2
: bar1 foo1 ;
: bar2 foo2 ;
: foo1 ; \ resolves the FORWARD FOO1 above
' foo1 is foo2
: bar3 foo1 ; \ direct call of the colon definition FOO1
: bench1 100000000 0 do bar1 bar1 bar1 bar1 bar1 bar1 bar1 bar1 bar1 bar1 loop ;
: bench2 100000000 0 do bar2 bar2 bar2 bar2 bar2 bar2 bar2 bar2 bar2 bar2 loop ;
: bench3 100000000 0 do bar3 bar3 bar3 bar3 bar3 bar3 bar3 bar3 bar3 bar3 loop ;

The performance results on a Skylake (Core i5 6600K), measured with

for i in 1 2 3; do LC_NUMERIC=en_US.utf8 perf stat -B -e cycles:u -e instructions:u gforth-fast bench-forward.fs -e "bench$i bye"; done

are (cycles and instructions per bar* call):

cyc inst
8.6 25.1 bar1
11.7 28.1 bar2
9.6 25.1 bar3

It's not clear to me why BAR1 has a different performance from BAR3
(my guess is that code alignment played a role, this time with BAR1
being lucky), but in any case, you can see that the benefit of not
using DEFER is just 2 or 3 cycles and three instructions on Skylake.
Whether that benefit makes it worthwhile to use FORWARD rather than
DEFER is up to the reader to decide.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On Friday, January 21, 2022 at 12:04:41 PM UTC+1, Ron AARON wrote:
> How are these better than the defer/is paradigm? Seems about the same to me.

"F: <name>" (forward definition) and "R: <name>" (resolve definition) are not better
than DEFER / IS.

In fact, the DEFER / IS method is more powerful because it allows you to change the behavior
of the deferred action as often as you like. In contrast, "R: <name>" has only one fixed action,
namely the one that you specify when you resolve the definition, and you cannot change this
action later on.

For certain applications you want just this: A forward reference for a word, but this word
is supposed to have only one behavior. For example, you may want to design and test
the menu tree of software using top-down design. With forward references, the menu works
and can be tested, although all menu entries are still no-ops because the bodies have not
been resolved yet..

In such cases, "F: <name>" and "R: <name>" are very convenient. For each word that you
define, you only need to specify one name. In case of DEFER, you also need a name
for the variable. This is not a big deal, but the forward/resolve method is easier to use
and in my opinion gives the code more clarity.

It is similar to using a CONSTANT instead of a VARIABLE in cases where the stored value
will be specified only once and is never changed later on.

I have used F: and R: in my descriptions, but the same thing applies to similar methods
such as the [FORWARD] directive presented above by Hans Bezemer.

Henry

On 22/01/2022 18:54, Heinrich Hohl wrote:
> On Friday, January 21, 2022 at 12:04:41 PM UTC+1, Ron AARON wrote:
>> How are these better than the defer/is paradigm? Seems about the same to me.
>
> "F: <name>" (forward definition) and "R: <name>" (resolve definition) are not better
> than DEFER / IS.
>
> In fact, the DEFER / IS method is more powerful because it allows you to change the behavior
> of the deferred action as often as you like. In contrast, "R: <name>" has only one fixed action,
> namely the one that you specify when you resolve the definition, and you cannot change this
> action later on.
>
> For certain applications you want just this: A forward reference for a word, but this word
> is supposed to have only one behavior. For example, you may want to design and test
> the menu tree of software using top-down design. With forward references, the menu works
> and can be tested, although all menu entries are still no-ops because the bodies have not
> been resolved yet..
>
> In such cases, "F: <name>" and "R: <name>" are very convenient. For each word that you
> define, you only need to specify one name. In case of DEFER, you also need a name
> for the variable. This is not a big deal, but the forward/resolve method is easier to use
> and in my opinion gives the code more clarity.
>
> It is similar to using a CONSTANT instead of a VARIABLE in cases where the stored value
> will be specified only once and is never changed later on.
>
> I have used F: and R: in my descriptions, but the same thing applies to similar methods
> such as the [FORWARD] directive presented above by Hans Bezemer.
>
> Henry

Thank you for that insight.I'd considered some 'forward' implementation
in the past for 8th, but haven't yet because I couldn't see any
particular advantage over defer/is.

However, code clarity is a consideration. I've already got a "deferred:"
word which creates a deferred word with initial code, just because it's
"nicer" in some way.

On 21/01/2022 18:28, Hans Bezemer wrote:
> On Friday, January 21, 2022 at 12:23:56 PM UTC+1, Hans Bezemer wrote:
>> Gotta test performance though..
>
> Well, the figures are in:
>
> DEFER: 109% versus a direct execution;
> FORWARD: 111% versus a direct execution.
>
> Hans Bezemer

Ah, so no serious performance difference then. I'll bet real differences
would be on a per-CPU-class basis (older CPUs showing more difference
would be my guess).

On 23/01/2022 16:02, Ron AARON wrote:
>
>
> On 21/01/2022 18:28, Hans Bezemer wrote:
>> On Friday, January 21, 2022 at 12:23:56 PM UTC+1, Hans Bezemer wrote:
>>> Gotta test performance though..
>>
>> Well, the figures are in:
>>
>> DEFER: 109% versus a direct execution;
>> FORWARD: 111% versus a direct execution.
>>
>> Hans Bezemer
>
> Ah, so no serious performance difference then. I'll bet real differences
> would be on a per-CPU-class basis (older CPUs showing more difference
> would be my guess).

If speed matters, I'd be looking to defer at a higher level of abstraction
where the defer has negligible impact.

On 23/01/2022 7:53, dxforth wrote:
> On 23/01/2022 16:02, Ron AARON wrote:
>>
>>
>> On 21/01/2022 18:28, Hans Bezemer wrote:
>>> On Friday, January 21, 2022 at 12:23:56 PM UTC+1, Hans Bezemer wrote:
>>>> Gotta test performance though..
>>>
>>> Well, the figures are in:
>>>
>>> DEFER: 109% versus a direct execution;
>>> FORWARD: 111% versus a direct execution.
>>>
>>> Hans Bezemer
>>
>> Ah, so no serious performance difference then. I'll bet real differences
>> would be on a per-CPU-class basis (older CPUs showing more difference
>> would be my guess).
>
> If speed matters, I'd be looking to defer at a higher level of abstraction
> where the defer has negligible impact.

Such as (perhaps) conditionally compiling different versions?

On 23/01/2022 17:06, Ron AARON wrote:
>
>
> On 23/01/2022 7:53, dxforth wrote:
>> On 23/01/2022 16:02, Ron AARON wrote:
>>>
>>>
>>> On 21/01/2022 18:28, Hans Bezemer wrote:
>>>> On Friday, January 21, 2022 at 12:23:56 PM UTC+1, Hans Bezemer wrote:
>>>>> Gotta test performance though..
>>>>
>>>> Well, the figures are in:
>>>>
>>>> DEFER: 109% versus a direct execution;
>>>> FORWARD: 111% versus a direct execution.
>>>>
>>>> Hans Bezemer
>>>
>>> Ah, so no serious performance difference then. I'll bet real differences
>>> would be on a per-CPU-class basis (older CPUs showing more difference
>>> would be my guess).
>>
>> If speed matters, I'd be looking to defer at a higher level of abstraction
>> where the defer has negligible impact.
>
> Such as (perhaps) conditionally compiling different versions?

Were I at the point where the difference mattered, I suspect I'd be having
speed issues everywhere. Defer would be the least of my problems (nor the
solution).

On 23/01/2022 9:44, dxforth wrote:
> On 23/01/2022 17:06, Ron AARON wrote:
>>
>>
>> On 23/01/2022 7:53, dxforth wrote:
>>> On 23/01/2022 16:02, Ron AARON wrote:
>>>>
>>>>
>>>> On 21/01/2022 18:28, Hans Bezemer wrote:
>>>>> On Friday, January 21, 2022 at 12:23:56 PM UTC+1, Hans Bezemer wrote:
>>>>>> Gotta test performance though..
>>>>>
>>>>> Well, the figures are in:
>>>>>
>>>>> DEFER: 109% versus a direct execution;
>>>>> FORWARD: 111% versus a direct execution.
>>>>>
>>>>> Hans Bezemer
>>>>
>>>> Ah, so no serious performance difference then. I'll bet real
>>>> differences
>>>> would be on a per-CPU-class basis (older CPUs showing more difference
>>>> would be my guess).
>>>
>>> If speed matters, I'd be looking to defer at a higher level of
>>> abstraction
>>> where the defer has negligible impact.
>>
>> Such as (perhaps) conditionally compiling different versions?
>
> Were I at the point where the difference mattered, I suspect I'd be having
> speed issues everywhere.  Defer would be the least of my problems (nor the
> solution).

That's my take on it as well.