Marcel Hendrix schrieb am Samstag, 11. September 2021 um 09:39:53 UTC+2:
> On Friday, September 10, 2021 at 10:26:57 PM UTC+2, Andy Valencia wrote:
> > Marcel Hendrix <m...@iae.nl> writes:
> > > I find 565 occurences of PICK in a total of 2077 Forth files. Of these,
> > > 280 are for 2 PICK, far less for 3 PICK and almost none for n > 5.
> > I realize that's a lot of occurrences, but when you run your eyes over
> > the code, do any qualities of the surrounding code jump out at you?
> A very difficult question. Maybe 20% of the usage is caused by
> benchmarks and published code, something one does not really
> want to rewrite from scratch.
>
> I looked at some of the others: it would be a major effort to get rid
> of the 2..4 PICK's (replacing them with convoluted stack gymnastics
> is of course always possible). Although I don't believe in THIRD and
> FOURTH, that may be just because there names are odd.
>
> There appear to be many occurences of strings ( c-addr u -- ) and
> doubles ( lo hi -- ). The number of objects may be less than 3,
> but with strings and doubles that grows to 6 stack items.
>
> The PICKs are almost always accompanied by locals. So 2 .. 4 PICK
> is apparently preferred to locals.
>
> When code is reused, it is with the purpose to add features and
> therefore extra parameters need to be introduced. Rewriting is
> obviously not wanted.
>
> When working with others it is not productive to completely
> rewrite your colleague's code just to get rid of a stack item. (I did
> not find many examples of that, probably because the interface
> was fixed or because locals were unavoidable anyway.)
>

I think that nails it down. It is sometimes difficult to avoid PICK when you have
mixed single cell and double cell arguments.

On 11/09/2021 15:37, Paul Rubin wrote:
> Hans Bezemer <the.beez.speaks@gmail.com> writes:
>> I used it once in a Rosetta Code example because I was in a hurry, but
>> I'm still not proud of that. In 4tH, it is an external word, which
>> makes it quite slow as well. But given it's primary there for porting,
>> it's not that important.
>
> I needed to copy the top 3 stack elements (3DUP) to try the Tak
> benchmark. I wrote something like:
>
> : 3dup.1 ( a b c -- a b c a b c ) >r 2dup r@ -rot r> ;
>
> I sent the benchmark to Mattias Trute and he said he had already done
> this test, and sent his version. It used:
>
> : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
>
> That seemed simpler to me. Of course with locals,
>
> : 3dup.3 { a b c } a b c a b c ;
>
> is as simple as it gets.
>

Also the longest and worst performing.

: 3DUP.4 DUP 2OVER ROT ;

On my system the results were as follows:

3dup.2 12/17 time/bytes
3dup.3 25/26
3dup.4 7/11

Marcel Hendrix <mhx@iae.nl> writes:
> > I realize that's a lot of occurrences, but when you run your eyes over
> > the code, do any qualities of the surrounding code jump out at you?
> A very difficult question. Maybe 20% of the usage is caused by
> benchmarks and published code, something one does not really
> want to rewrite from scratch.
> ...

Thank you for that discussion, it was very interesting.

Andy Valencia
Home page: https://www.vsta.org/andy/
To contact me: https://www.vsta.org/contact/andy.html

On Friday, September 10, 2021 at 11:46:54 AM UTC-5, Marcel Hendrix wrote:
> Some call these THIRD and FOURTH, but for me that make no sense.
> (DUP <= ONE, OVER <= SECOND ?)

>
They were in Baden's Toolbelt along with note stating they
should be primitives.
--
me

dxforth <dxforth@gmail.com> writes:
>On 11/09/2021 15:37, Paul Rubin wrote:
>> : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
>: 3DUP.4 DUP 2OVER ROT ;

Gforth-fast AMD64:

see-code 3dup.2 see-code 3dup.4
$7FE5CCD711A8 third $7FE5CCD71278 dup
7FE5CC966626: mov rax,$10[r13] 7FE5CC966714: sub r13,$08
7FE5CC96662A: sub r13,$08 7FE5CC966718: add r14,$08
7FE5CC96662E: add r14,$08 7FE5CC96671C: mov $08[r13],r15
7FE5CC966632: mov $08[r13],r15 $7FE5CCD71280 2over
7FE5CC966636: mov r15,rax 7FE5CC966720: mov rax,$10[r13]
$7FE5CCD711B0 third 7FE5CC966724: mov rdx,$18[r13]
7FE5CC966639: mov rax,$10[r13] 7FE5CC966728: sub r13,$10
7FE5CC96663D: sub r13,$08 7FE5CC96672C: add r14,$08
7FE5CC966641: add r14,$08 7FE5CC966730: mov $10[r13],r15
7FE5CC966645: mov $08[r13],r15 7FE5CC966734: mov $08[r13],rdx
7FE5CC966649: mov r15,rax 7FE5CC966738: mov r15,rax
$7FE5CCD711B8 third $7FE5CCD71288 rot
7FE5CC96664C: mov rax,$10[r13] 7FE5CC96673B: mov rax,$10[r13]
7FE5CC966650: sub r13,$08 7FE5CC96673F: mov rdx,$08[r13]
7FE5CC966654: add r14,$08 7FE5CC966743: add r14,$08
7FE5CC966658: mov $08[r13],r15 7FE5CC966747: mov $08[r13],r15
7FE5CC96665C: mov r15,rax 7FE5CC96674B: mov $10[r13],rdx
$7FE5CCD711C0 ;s ok 7FE5CC96674F: mov r15,rax
$7FE5CCD71290 ;s ok

And of course, lxf produces the same code for 3dup.4 as for all the
other versions:

804FC1D 8B4500 mov eax , [ebp]
804FC20 8945F4 mov [ebp-Ch] , eax
804FC23 8B4504 mov eax , [ebp+4h]
804FC26 8945F8 mov [ebp-8h] , eax
804FC29 895DFC mov [ebp-4h] , ebx
804FC2C 8D6DF4 lea ebp , [ebp-Ch]
804FC2F C3 ret near

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>And of course, lxf produces the same code for 3dup.4 as for all the
>other versions:
>
> 804FC1D 8B4500 mov eax , [ebp]
> 804FC20 8945F4 mov [ebp-Ch] , eax
> 804FC23 8B4504 mov eax , [ebp+4h]
> 804FC26 8945F8 mov [ebp-8h] , eax
> 804FC29 895DFC mov [ebp-4h] , ebx
> 804FC2C 8D6DF4 lea ebp , [ebp-Ch]
> 804FC2F C3 ret near

And the reason is that all four versions consist only of words that
lxf analyses at compile time, so all four versions result in zero
stuff to compile right away, and the same stack state to reconcile
with the calling convention, so the same code is generated (which
consists entirely of the reconciliation with the calling convention).

Likewise, VFX produces the same code for 3dup.2 and 3dup.4; it
produces different (but equivalent) code for the reconciliation
compared to lxf.

VFX produces worse code for 3dup.1 because it is not analytical about
the return stack; so when it encounters a return stack word, it has to
generate code for the return stack word, and that also restricts how
the data stack reconciliation can work.

Likewise, VFX produces worse code for 3dup.3 because it is not
analytical about locals.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

>>> : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
>>: 3DUP.4 DUP 2OVER ROT ;
> Gforth-fast AMD64: ...

Gforth-fast uses big and fast enough hardware that the cost of locals
(if any) with it will almost always be negligible. On a tiny bytecode
implementation, 3dup.4 is very clever and impressive (probably just 3
bytes). It makes me think that if locals are supported for it, then the
compiler should do some work to avoid copying them around, when
possible.

On Saturday, 11 September 2021 at 23:09:03 UTC+2, Anton Ertl wrote:
> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> >And of course, lxf produces the same code for 3dup.4 as for all the
> >other versions:
> >
> > 804FC1D 8B4500 mov eax , [ebp]
> > 804FC20 8945F4 mov [ebp-Ch] , eax
> > 804FC23 8B4504 mov eax , [ebp+4h]
> > 804FC26 8945F8 mov [ebp-8h] , eax
> > 804FC29 895DFC mov [ebp-4h] , ebx
> > 804FC2C 8D6DF4 lea ebp , [ebp-Ch]
> > 804FC2F C3 ret near
> And the reason is that all four versions consist only of words that
> lxf analyses at compile time, so all four versions result in zero
> stuff to compile right away, and the same stack state to reconcile
> with the calling convention, so the same code is generated (which
> consists entirely of the reconciliation with the calling convention).

Yes this is correct. But it does not analyse the code. Stack operations
just move pointers on a virtual stack, no code is generated. When code
needs to be generated the locations of the stack items are taken from
the virtual stacks and appropriate code are generated. In this case as
you point out code is generated by ; as the function needs to leave the
stack in a known state. Both the data and return stacks are treated in the
same way. Locals are placed on the return stack, the code generator
does not handle them in any specific way. In fact it is unaware of them.

As you have seen to generate good code any points that needs to
reconcile the stacks should be avoided. That is why inlining to avoid
calls gets interesting.

In the example of the rectangle where lxf generated less good code
this happened. The code could be written in this way to improve
: preprect {: x1 y1 x2 y2 :} x1 y1 x1 y2 x2 y2 x2 y1 x1 y1 ; ok
: rect4 preprect moveto lineto lineto lineto lineto ;

the preprect will generate 43 bytes of stack shuffling and rect4
30 bytes of calls ( 5 calls and one tail jump).

doing the same for the stack shuffling variant saves even more
: rect5 2over 2>r 2>r over r@ 2r> over r@ 2r>
moveto lineto lineto lineto lineto ;

This will generate 67 bytes, saving one call and one ret.

Of course this is a stupid way of actually writing it. But if you want to
see what can be achived you will do it.

BR
Peter

> Likewise, VFX produces the same code for 3dup.2 and 3dup.4; it
> produces different (but equivalent) code for the reconciliation
> compared to lxf.
>
> VFX produces worse code for 3dup.1 because it is not analytical about
> the return stack; so when it encounters a return stack word, it has to
> generate code for the return stack word, and that also restricts how
> the data stack reconciliation can work.
>
> Likewise, VFX produces worse code for 3dup.3 because it is not
> analytical about locals.
> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2021: https://euro.theforth.net/2021

On Saturday, September 11, 2021 at 8:54:06 AM UTC+2, Anton Ertl wrote:
> Paul Rubin <no.e...@nospam.invalid> writes:
> >I needed to copy the top 3 stack elements (3DUP) to try the Tak
> >benchmark. I wrote something like:
> >
> > : 3dup.1 ( a b c -- a b c a b c ) >r 2dup r@ -rot r> ;
> >
> >I sent the benchmark to Mattias Trute and he said he had already done
> >this test, and sent his version. It used:
> >
> > : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
> >
> >That seemed simpler to me. Of course with locals,
> >
> > : 3dup.3 { a b c } a b c a b c ;
> >
> >is as simple as it gets.
> On lxf all three produce the same code:
>
> 804FC0A 8B4500 mov eax , [ebp]
> 804FC0D 8945F4 mov [ebp-Ch] , eax
> 804FC10 8B4504 mov eax , [ebp+4h]
> 804FC13 8945F8 mov [ebp-8h] , eax
> 804FC16 895DFC mov [ebp-4h] , ebx
> 804FC19 8D6DF4 lea ebp , [ebp-Ch]
> 804FC1C C3 ret near
>
> You are not going to factor this further, so the factoring
> disadvantage of the return stack and the locals version does not play
> a role. So use whatever you like most. On VFX 3DUP.2 produces
> equally good code, the other versions are worse.
>
> I find it funny that those who disapprove of some Forth features do
> not give any concrete reasons for that. Instead, they use evocative
> language, and point people to books, without pointing out any specific
> place the should read. There are not many clearer ways to indicate
> their lack of arguments.
> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2021: https://euro.theforth.net/2021

Philosophy goes deep. It's not something that fits half a page - as the people who do know most certainly do. However, those who have a notion of design know that software unrevels and decays in two particular instances:
a) Maintenance on software without an explicit design and architecture;
b) Maintenance on software which does not respect an explicit design and architecture.

Moore has always been very vocal on what this design and philosophy was. And it has been very well expressed in Thinking Forth and the fire chats. Violating it almost killed Forth in 1983. But people never learn.

I know of a lot of managers who find themselves extra ordinary "pragmatic" and are arrogant enough to think they can violate the rules and get away with it.

Most of the time however, reality wins. Or do you think it's a smart move to do ducktyping by design and then add all kinds of type checking functionality all of a sudden? Especially when you always stated "there is alaways one obvious way to do it"? Does that consitute a clear vision?

Hans Bezemer

On 12/09/2021 05:50, Anton Ertl wrote:
> dxforth <dxforth@gmail.com> writes:
>>On 11/09/2021 15:37, Paul Rubin wrote:
>>> : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
>>: 3DUP.4 DUP 2OVER ROT ;
>
> Gforth-fast AMD64:
>
> see-code 3dup.2 see-code 3dup.4
> $7FE5CCD711A8 third $7FE5CCD71278 dup
> 7FE5CC966626: mov rax,$10[r13] 7FE5CC966714: sub r13,$08
> 7FE5CC96662A: sub r13,$08 7FE5CC966718: add r14,$08
> 7FE5CC96662E: add r14,$08 7FE5CC96671C: mov $08[r13],r15
> 7FE5CC966632: mov $08[r13],r15 $7FE5CCD71280 2over
> 7FE5CC966636: mov r15,rax 7FE5CC966720: mov rax,$10[r13]
> $7FE5CCD711B0 third 7FE5CC966724: mov rdx,$18[r13]
> 7FE5CC966639: mov rax,$10[r13] 7FE5CC966728: sub r13,$10
> 7FE5CC96663D: sub r13,$08 7FE5CC96672C: add r14,$08
> 7FE5CC966641: add r14,$08 7FE5CC966730: mov $10[r13],r15
> 7FE5CC966645: mov $08[r13],r15 7FE5CC966734: mov $08[r13],rdx
> 7FE5CC966649: mov r15,rax 7FE5CC966738: mov r15,rax
> $7FE5CCD711B8 third $7FE5CCD71288 rot
> 7FE5CC96664C: mov rax,$10[r13] 7FE5CC96673B: mov rax,$10[r13]
> 7FE5CC966650: sub r13,$08 7FE5CC96673F: mov rdx,$08[r13]
> 7FE5CC966654: add r14,$08 7FE5CC966743: add r14,$08
> 7FE5CC966658: mov $08[r13],r15 7FE5CC966747: mov $08[r13],r15
> 7FE5CC96665C: mov r15,rax 7FE5CC96674B: mov $10[r13],rdx
> $7FE5CCD711C0 ;s ok 7FE5CC96674F: mov r15,rax
> $7FE5CCD71290 ;s ok

Few non-optimizing compilers define THIRD presumably as they had PICK
and '2 PICK' doesn't arise that often. The only regular stack word I
have based on PICK or ROLL is 2ROT - again because it's not used that
often despite being 'standard'.

> VFX produces worse code for 3dup.1 because it is not analytical about
> the return stack [...] Likewise [...] locals.

I actually agree with that. A forth programmer expects to get what he
writes - 'close to the metal' and all that. If one has no control over
output one may as well use C.

Hans Bezemer <the.beez.speaks@gmail.com> writes:
> Moore has always been very vocal on what this design and philosophy
> was. And it has been very well expressed in Thinking Forth and the
> fire chats.

I looked for info about this in Thinking Forth (ANS edition) and
couldn't find any. Can you suggest an edition and page number to check?

> Violating it almost killed Forth in 1983. But people never learn.

Forth is fairly dead right now. Maybe the unneeded stack juggling is
a contributing factor.

> Most of the time however, reality wins. Or do you think it's a smart
> move to do ducktyping by design and then add all kinds of type
> checking functionality all of a sudden? Especially when you always
> stated "there is alaways one obvious way to do it"? Does that
> consitute a clear vision?

Are you talking about MyPy? I use it and I think it does some good,
though you are right that it is imperfect as an add-on. There are
similar things for Erlang (the Dialyzer), typed Clojure, typed Racket,
and a general topic called "gradual typing" for such things. In each
case, they seem to help. It seems to me that latent/duck typing works
best for relatively small programs, and static types become worth their
weight in larger programs.

Python started out as a small scripting language that eventually came to
host larger systems, so it started creaking under the load, and type
annotations were added to help with that. Ruby has something similar
now too. There are other problems with Python that also resulted from
"thinking small" at first.

If we're looking for lessons from all this, one lesson might be to think
big from the beginning, like ML did and Haskell sort-of did. Forth has
always been the opposite of that, though that is part of what makes it
interesting.

On 12/09/2021 11:59, Paul Rubin wrote:
> Hans Bezemer <the.beez.speaks@gmail.com> writes:
>> Moore has always been very vocal on what this design and philosophy
>> was. And it has been very well expressed in Thinking Forth and the
>> fire chats.
>
> I looked for info about this in Thinking Forth (ANS edition) and
> couldn't find any. Can you suggest an edition and page number to check?
>
>> Violating it almost killed Forth in 1983. But people never learn.
>
> Forth is fairly dead right now.

So is stamp collecting. It makes no difference to those who are
passionate about it.

P Falth <peter.m.falth@gmail.com> writes:
> Stack operations just move pointers on a virtual stack, no code is
> generated. When code needs to be generated the locations of the stack
> items are taken from the virtual stacks and appropriate code are
> generated.... As you have seen to generate good code any points that
> needs to reconcile the stacks should be avoided. That is why inlining
> to avoid calls gets interesting.

Peter, can you say anything about the subtleties of these operations?
Do you compile every word separately, i.e. starting with with no
knowledge of the stack state other than from the word definition? If
you have to reconcile the stack at the end, do you just put it into the
state that a normal Forth interpreter would leave it in, or do you
possibly make adjustments depending on the definitions of other words?

If you compile FOO and that has some reconciliation at the end, and
later BAR uses FOO's output but might have been better off starting from
an intermediate state of FOO, then maybe the inlined FOO BAR would allow
some reconciliation and rearrangement to "cancel out". Is this useful?

Would you say this virtual stack stuff was complicated to implement? I
know of several compilers that do it, but I haven't examined them. I
think the approach could be useful for something I've gotten interested
in, but I don't know what I'd be getting into if I attempted it. Thanks.

On Sunday, 12 September 2021 at 09:05:56 UTC+2, Paul Rubin wrote:
> P Falth <peter....@gmail.com> writes:
> > Stack operations just move pointers on a virtual stack, no code is
> > generated. When code needs to be generated the locations of the stack
> > items are taken from the virtual stacks and appropriate code are
> > generated.... As you have seen to generate good code any points that
> > needs to reconcile the stacks should be avoided. That is why inlining
> > to avoid calls gets interesting.
> Peter, can you say anything about the subtleties of these operations?
> Do you compile every word separately, i.e. starting with with no
> knowledge of the stack state other than from the word definition? If
> you have to reconcile the stack at the end, do you just put it into the
> state that a normal Forth interpreter would leave it in, or do you
> possibly make adjustments depending on the definitions of other words?

I have to words that control the virtual stacks. v-init ( n --) and v-exit ( n-- )
the argument tells how many stack items to keep in registers.
When stating a new definition with : 1 v-init is issued and at the exit 1 v-exit
This means that every "normal" word that a user compiles stats with
top of stack in ebx and also ends in the same way. When ther is a control flow
break ( if begin then etc) the stack must be set to a know state with 1 v-exit.
The same if a call to a previously compiled word is done.
Early on I tried to be smarter and had a in/out ( n1 n2 -- ) word that
described and stored how many stackitems to keep in registers.
After some testing it was clear that this in the end produced more stack juggling
then just 1 item in a register. more code also meant a bit slower.

For this reason inlining becomes a very important means to improve code
as this avoids reconciliation on calls as they are removed. Best and easiest
is just a simple source inlining. But this of course has a problem of late binding
and possibility of redifinitions and wordlist order. In my 64 bit lxf64 I compile
first to tokenthreaded code to solve this problem

> If you compile FOO and that has some reconciliation at the end, and
> later BAR uses FOO's output but might have been better off starting from
> an intermediate state of FOO, then maybe the inlined FOO BAR would allow
> some reconciliation and rearrangement to "cancel out". Is this useful?
>
> Would you say this virtual stack stuff was complicated to implement? I
> know of several compilers that do it, but I haven't examined them. I
> think the approach could be useful for something I've gotten interested
> in, but I don't know what I'd be getting into if I attempted it. Thanks.

The compiler is about 1500 lines of code 50k bytes so it is not big. I think the
biggest problem is that you need a good integration with an assembler to
generate the final code. in lxf I use a stripped down version that covers the
opcodes I need. in lxf64 I generate intel asm source and send it to an external
assembler dll that returns the compiled blob. Other than that duped stack items
require some consideration. I also gave up on using pop and push instructions
to reconcile the returns stack, I could never get that to work. It was better using
mov instruction and one final adjustment of the stackpointer (esp).
I think Forth is an ideal language to write this in as you can start with a few words
using the code generator and then test and continue with others

BR
Peter

Paul Rubin <no.email@nospam.invalid> writes:
>>>> : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
>>>: 3DUP.4 DUP 2OVER ROT ;
>> Gforth-fast AMD64: ...
>
>Gforth-fast uses big and fast enough hardware that the cost of locals
>(if any) with it will almost always be negligible.

A, say, factor 1.5 slowdown is a factor 1.5 slowdown, whatever the
speed of the hardware. It only does not matter if the program is fast
enough in either case.

A more relevant point is that, if you care for performance, you use a
Forth like lxf, or VFX (and then ask Stephen Pelc when VFX will be
analytical about the return stack and locals:-); if you use Gforth or
dxforth, performance is not the most important thing, so why should
one base one's coding style on the performance on such systems?

>On a tiny bytecode
>implementation, 3dup.4 is very clever and impressive (probably just 3
>bytes).

If your compiler employs Matthias Koch's constant folding (which is
simple to implement), it can easily replace 2 PICK by THIRD, and then
it's also just 3 bytes in a byte-coded implementation. Even if you do
not employ constant folding, you can provide THIRD as separate word,
and get the same result.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

P Falth <peter.m.falth@gmail.com> writes:
>On Saturday, 11 September 2021 at 23:09:03 UTC+2, Anton Ertl wrote:
>> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> >And of course, lxf produces the same code for 3dup.4 as for all the
>> >other versions:
>> >
>> > 804FC1D 8B4500 mov eax , [ebp]
>> > 804FC20 8945F4 mov [ebp-Ch] , eax
>> > 804FC23 8B4504 mov eax , [ebp+4h]
>> > 804FC26 8945F8 mov [ebp-8h] , eax
>> > 804FC29 895DFC mov [ebp-4h] , ebx
>> > 804FC2C 8D6DF4 lea ebp , [ebp-Ch]
>> > 804FC2F C3 ret near
>> And the reason is that all four versions consist only of words that
>> lxf analyses at compile time, so all four versions result in zero
>> stuff to compile right away, and the same stack state to reconcile
>> with the calling convention, so the same code is generated (which
>> consists entirely of the reconciliation with the calling convention).
>
>Yes this is correct. But it does not analyse the code. Stack operations
>just move pointers on a virtual stack, no code is generated.

In the old days we had plain compilers with peephole optimization
(e.g., bigForth with peephole optimization on the machine-code level,
and SwiftForth which AFAIK combines primitives at the Forth level and
generates code for that), and anything beyond that has been called an
analytical compiler.

So if your compiler employs arbitrary stack representations by keeping
track at compile time what stack item resides in what register or what
memory location, that's an analytical compiler, and keeping track of
the stack items is analysis.

Gforth-fast supports few stack representations determined at Gforth
build time, but only represents them as finite states at compile time
and cannot keep track of arbitrary stack representations (e.g.,
already DUP requires at least copying from one register (or memory
location) to a different register), so I would not classify it as
analytical.

>In the example of the rectangle where lxf generated less good code
>this happened. The code could be written in this way to improve
>: preprect {: x1 y1 x2 y2 :} x1 y1 x1 y2 x2 y2 x2 y1 x1 y1 ; ok
>: rect4 preprect moveto lineto lineto lineto lineto ;
>
>the preprect will generate 43 bytes of stack shuffling and rect4
>30 bytes of calls ( 5 calls and one tail jump).

But when I inline PREPRECT manually, the PREPRECT code becomes quite a
bit longer, and I see some stores to the return stack. Strange.

>doing the same for the stack shuffling variant saves even more
>: rect5 2over 2>r 2>r over r@ 2r> over r@ 2r>
>moveto lineto lineto lineto lineto ;
>
>This will generate 67 bytes, saving one call and one ret.
>
>Of course this is a stupid way of actually writing it. But if you want to
>see what can be achived you will do it.

It's only stupid is your goal is to write code that is easy to
understand. If your goal is to produce the shortest code on lxf, it
is smart:-).

Let's try to make a smart version for VFX:

: rect7 ( x1 y1 x2 y2 -- )
3 pick 3 pick over 3 pick 5 pick 2dup 5 pick 5 pick over
moveto lineto lineto lineto lineto ;

( 88 bytes, 25 instructions )

Hmm; it tends not to keep the items that it has already loaded in
registers, and therefore has to reload them. lxf does a little better
at 82 bytes. Admittedly, this is the kind of code that nobody writes,
so it's understandable if VFX and lxf are not optimized for that.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

Hans Bezemer <the.beez.speaks@gmail.com> writes:
>On Saturday, September 11, 2021 at 8:54:06 AM UTC+2, Anton Ertl wrote:
>> I find it funny that those who disapprove of some Forth features do=20
>> not give any concrete reasons for that. Instead, they use evocative=20
>> language, and point people to books, without pointing out any specific=20
>> place the should read. There are not many clearer ways to indicate=20
>> their lack of arguments.
....
>Philosophy goes deep. It's not something that fits half a page - as the peo=
>ple who do know most certainly do.

Tough luck. If you cannot explain the concrete benefits of your
philosophy on a concrete example, the rest of the world cannot
distinguish it from mumbo-jumbo.

>However, those who have a notion of desi=
>gn know that software unrevels and decays in two particular instances:
>a) Maintenance on software without an explicit design and architecture;
>b) Maintenance on software which does not respect an explicit design and ar=
>chitecture.
>
>Moore has always been very vocal on what this design and philosophy was.

It seems to me that Chuck Moore prefers rewriting over maintaining the
software, if new requirements come along. So it's strange that you
mention him in connection with maintenance.

>An=
>d it has been very well expressed in Thinking Forth and the fire chats. Vio=
>lating it almost killed Forth in 1983.

Really. What happened in 1983?

The fact that the Forth-83 standard (which appeared in 1984) made
changes that were incompatible with Forth-79 certainly caused a lot of
damage, but I see no relation to Thinking Forth and Chuck Moore; AFAIK
he has never spoken out against incompatible changes, and introduced a
number of them himself.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 12/09/2021 18:56, Anton Ertl wrote:
> Paul Rubin <no.email@nospam.invalid> writes:
>>>>> : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
>>>>: 3DUP.4 DUP 2OVER ROT ;
>>> Gforth-fast AMD64: ...
>>
>>Gforth-fast uses big and fast enough hardware that the cost of locals
>>(if any) with it will almost always be negligible.
>
> A, say, factor 1.5 slowdown is a factor 1.5 slowdown, whatever the
> speed of the hardware. It only does not matter if the program is fast
> enough in either case.
>
> A more relevant point is that, if you care for performance, you use a
> Forth like lxf, or VFX (and then ask Stephen Pelc when VFX will be
> analytical about the return stack and locals:-); if you use Gforth or
> dxforth, performance is not the most important thing, so why should
> one base one's coding style on the performance on such systems?

If good coding produces good results on any system then surely it is
coding worth learning. If getting good results from locals means
using a specific forth compiler, then one is tied to that system (for
as long or short a time as the author cares to support it).

To my knowledge there is no VFX/LXF/GForth for CP/M systems. OTOH
there does appear to be many RC2014 Z80 systems and a surprising
degree of interest in forth. Their users weren't lured by locals.
They came for the full-on Forth experience.

dxforth <dxforth@gmail.com> writes:
>On 12/09/2021 05:50, Anton Ertl wrote:
>> dxforth <dxforth@gmail.com> writes:
>>>On 11/09/2021 15:37, Paul Rubin wrote:
>>>> : 3dup.2 ( a b c -- a b c a b c ) 2 pick 2 pick 2 pick ;
>>>: 3DUP.4 DUP 2OVER ROT ;
>>
>> Gforth-fast AMD64:
>>
>> see-code 3dup.2 see-code 3dup.4
>> $7FE5CCD711A8 third $7FE5CCD71278 dup
>> 7FE5CC966626: mov rax,$10[r13] 7FE5CC966714: sub r13,$08
>> 7FE5CC96662A: sub r13,$08 7FE5CC966718: add r14,$08
>> 7FE5CC96662E: add r14,$08 7FE5CC96671C: mov $08[r13],r15
>> 7FE5CC966632: mov $08[r13],r15 $7FE5CCD71280 2over
>> 7FE5CC966636: mov r15,rax 7FE5CC966720: mov rax,$10[r13]
>> $7FE5CCD711B0 third 7FE5CC966724: mov rdx,$18[r13]
>> 7FE5CC966639: mov rax,$10[r13] 7FE5CC966728: sub r13,$10
>> 7FE5CC96663D: sub r13,$08 7FE5CC96672C: add r14,$08
>> 7FE5CC966641: add r14,$08 7FE5CC966730: mov $10[r13],r15
>> 7FE5CC966645: mov $08[r13],r15 7FE5CC966734: mov $08[r13],rdx
>> 7FE5CC966649: mov r15,rax 7FE5CC966738: mov r15,rax
>> $7FE5CCD711B8 third $7FE5CCD71288 rot
>> 7FE5CC96664C: mov rax,$10[r13] 7FE5CC96673B: mov rax,$10[r13]
>> 7FE5CC966650: sub r13,$08 7FE5CC96673F: mov rdx,$08[r13]
>> 7FE5CC966654: add r14,$08 7FE5CC966743: add r14,$08
>> 7FE5CC966658: mov $08[r13],r15 7FE5CC966747: mov $08[r13],r15
>> 7FE5CC96665C: mov r15,rax 7FE5CC96674B: mov $10[r13],rdx
>> $7FE5CCD711C0 ;s ok 7FE5CC96674F: mov r15,rax
>> $7FE5CCD71290 ;s ok
>
>Few non-optimizing compilers define THIRD presumably as they had PICK
>and '2 PICK' doesn't arise that often.

In the present case the constant folding built into Gforth replaced 2
PICK with THIRD.

Concerning how often it occurs, here are some numbers from the
Gforth-AMD64 image:

953 dup
275 over
35 third
4 fourth
19 pick (5 pick, 4 pick, and non-constant pick)

For comparison:
335 swap
184 rot
4 roll (all non-constant, all on the same line)

223 2dup
21 2over

35 2swap
1 2rot

482 drop
206 2drop
111 nip
4 2nip

65 tuck
1 2tuck

374 >r
373 r>
157 r@
84 rdrop
48 2>r
45 2r>
13 2r@
11 2rdrop

>> VFX produces worse code for 3dup.1 because it is not analytical about
>> the return stack [...] Likewise [...] locals.
>
>I actually agree with that.

What is there to agree with? It' a statement of fact.

>A forth programmer expects to get what he
>writes - 'close to the metal' and all that. If one has no control over
>output one may as well use C.

I guess that you prefer a compiler that always generates the same
piece of code for a certain word (e.g., machineForth; wait, doesn't
machine code perform tail-call optimization?) to a compiler that
generates faster code in most cases (e.g., VFX, lxf and iForth).

OTOH, I prefer the faster code, and I especially like it if I don't
have to cater to the lack of capability of the compiler by writing
clever stack manipulation code.

I think that VFX has a flag for turning off optimization, which may be
the thing you want (I have not tried it, though).

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

Paul Rubin <no.email@nospam.invalid> writes:
>Hans Bezemer <the.beez.speaks@gmail.com> writes:
>> Most of the time however, reality wins. Or do you think it's a smart
>> move to do ducktyping by design and then add all kinds of type
>> checking functionality all of a sudden? Especially when you always
>> stated "there is alaways one obvious way to do it"? Does that
>> consitute a clear vision?
>
>Are you talking about MyPy? I use it and I think it does some good,
>though you are right that it is imperfect as an add-on. There are
>similar things for Erlang (the Dialyzer), typed Clojure, typed Racket,
>and a general topic called "gradual typing" for such things. In each
>case, they seem to help. It seems to me that latent/duck typing works
>best for relatively small programs, and static types become worth their
>weight in larger programs.

Maybe in large monolithic programs. OTOH, it seems to me that static
type checking is a hindrance for a network of independently developed
packages. There are package repositories like CPAN and PyPI for
dynamic languages, but efforts like this for static languages have not
been similarly successful; e.g., in the case of Java, CJAN ceased
development in 2004 and Apache JJAR is essentially abandoned.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

Paul Rubin <no.email@nospam.invalid> writes:
>Do you compile every word separately, i.e. starting with with no
>knowledge of the stack state other than from the word definition? If
>you have to reconcile the stack at the end, do you just put it into the
>state that a normal Forth interpreter would leave it in, or do you
>possibly make adjustments depending on the definitions of other words?

lxf and VFX only analyze basic blocks (VFX after inlining; lxf does
not inline). iForth seems to analyze the whole definition.

>If you compile FOO and that has some reconciliation at the end, and
>later BAR uses FOO's output but might have been better off starting from
>an intermediate state of FOO, then maybe the inlined FOO BAR would allow
>some reconciliation and rearrangement to "cancel out". Is this useful?

Inlining is generally performed by replaying a name-resolved version
of the source code of the inlined word to the compiler (it also has to
deal with EXIT), and that means that if you have

: foo dup ;
: bar foo drop ;

the inlining compiler will see

: bar dup drop ;

and on VFX this results in:

BAR
( 080C0AB0 C3 ) NEXT,
( 1 bytes, 1 instructions )

>Would you say this virtual stack stuff was complicated to implement? I
>know of several compilers that do it, but I haven't examined them.

IIRC Stephen Pelc wrote that what I called reconciling recently is
very complicated. I think you can make it less complicated by
modeling it as first moving everything to a temporary location, and
then moving everything to the target location, eliminating moves if
liveness permits, and finally emitting the remaining moves. There are
more optimal ways, and I can look them up if you want, but they are
not simpler.

The rest is relatively straightforward.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 12/09/2021 20:53, Anton Ertl wrote:
>
> I guess that you prefer a compiler that always generates the same
> piece of code for a certain word (e.g., machineForth; wait, doesn't
> machine code perform tail-call optimization?) to a compiler that
> generates faster code in most cases (e.g., VFX, lxf and iForth).
> ...
> I think that VFX has a flag for turning off optimization, which may be
> the thing you want (I have not tried it, though).

I've no idea what machineForth does but if it inlines code such that
it breaks a working program (as VFX did) then it's a bug I don't wish
to waste my time running down.

On Sunday, September 12, 2021 at 12:53:13 PM UTC+2, Anton Ertl wrote:
> Tough luck. If you cannot explain the concrete benefits of your
> philosophy on a concrete example, the rest of the world cannot
> distinguish it from mumbo-jumbo.
For "flat earthers" calculations are also mumbo-jumbo. The reason: they can't be bothered by diving into the stuff and do the work. Hence, they write it off as "mumbo-jumbo".
It isn't restricted to "flat earthers", though. Other people - who are far over their head on a specific subject and don't want to do the work - use the term as well. It's closely connected to Dunning-Kruger effect.

> It seems to me that Chuck Moore prefers rewriting over maintaining the
> software, if new requirements come along. So it's strange that you
> mention him in connection with maintenance.
You can't be serous when you try to compare the maintenance of applications written IN a language with the maintenance of the language ITSELF.
> The fact that the Forth-83 standard (which appeared in 1984) made
> changes that were incompatible with Forth-79 certainly caused a lot of
> damage, but I see no relation to Thinking Forth and Chuck Moore; AFAIK
> he has never spoken out against incompatible changes, and introduced a
> number of them himself.
Forth was quite popular in the early 80-ies. However, "pragmatic" changes not only broke a lot of code - they made Forth into an ugly mess. E.g. eliminating "smart" words, setting cellsize to 2 bytes (sic!) and destroying DO...LOOP beyond repair. I didn't like it - and obviously a lot of people with me, since its use fell dramatically. And it still hasn't recovered. Good going.

That you CAN do it, doesn't mean you SHOULD do it. There is a subtle, but relevant difference between the two. You can add wings to a car, but there's a chance it not only flies worse than a plane, but also performs worse as a car.

Hans Bezemer

On Saturday, September 11, 2021 at 8:59:51 PM UTC-5, Paul Rubin wrote:
> Forth is fairly dead right now. Maybe the unneeded stack juggling is
> a contributing factor.

<saga> Lets make Forth great again!
In industry a manager stands in quicksand with alligators biting at
his butt, his arms tightly bound by a boa going for his neck,
and his head in a cloud of mosquitoes and hornets. Along the dry bank
comes whistling and skipping a little programmer. He sees the
manager and shouts "Hey Boss, I got just the thing to fix all your
problems. It's called FORTH!". Something boils up in the manager and
he's able to free one arm from the boa. He grabs the programmer and
throws him down to the gators. With the gators distracted in their
feeding the manager plods his way to the far bank and with his one
free hand takes the boa's head banging it against a tree till its dead.
Taking a head net from his pocket he pulls it over his head for
some relief from the swarming pests and sits down leaning against
the tree. He doesn't sit long. After a few deep breaths, he somehow
bolsters himself up to continue his trek through the jungle knowing
that his next step is likely to land him in the same predicament once
again. Strangely, the thought of the programmer and the word Forth
never enters his consciousness.
</saga>
--
me

On 9/12/21 1:23 PM, S Jack wrote:
> On Saturday, September 11, 2021 at 8:59:51 PM UTC-5, Paul Rubin wrote:
>> Forth is fairly dead right now. Maybe the unneeded stack juggling is
>> a contributing factor.
>
> <saga> Lets make Forth great again!
> In industry a manager stands in quicksand with alligators biting at
> his butt, his arms tightly bound by a boa going for his neck,
> and his head in a cloud of mosquitoes and hornets. Along the dry bank
> comes whistling and skipping a little programmer. He sees the
> manager and shouts "Hey Boss, I got just the thing to fix all your
> problems. It's called FORTH!". Something boils up in the manager and
> he's able to free one arm from the boa. He grabs the programmer and
> throws him down to the gators. With the gators distracted in their
> feeding the manager plods his way to the far bank and with his one
> free hand takes the boa's head banging it against a tree till its dead.
> Taking a head net from his pocket he pulls it over his head for
> some relief from the swarming pests and sits down leaning against
> the tree. He doesn't sit long. After a few deep breaths, he somehow
> bolsters himself up to continue his trek through the jungle knowing
> that his next step is likely to land him in the same predicament once
> again. Strangely, the thought of the programmer and the word Forth
> never enters his consciousness.

Good story about the realities of working in a time and cost competitive
environment.

When the rubber hits the road I would shamelessly use locals in a few
words if that got the job done for the customer deadline and measurement
showed that the code size and execution time were non-issues.

Maybe later clean it up, if time permitted. Maybe.

-Doug