On Sunday, 12 September 2021 at 12:26:25 UTC+2, Anton Ertl wrote:
> P Falth <peter....@gmail.com> writes:
> >On Saturday, 11 September 2021 at 23:09:03 UTC+2, Anton Ertl wrote:
> >> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> >> >And of course, lxf produces the same code for 3dup.4 as for all the
> >> >other versions:
> >> >
> >> > 804FC1D 8B4500 mov eax , [ebp]
> >> > 804FC20 8945F4 mov [ebp-Ch] , eax
> >> > 804FC23 8B4504 mov eax , [ebp+4h]
> >> > 804FC26 8945F8 mov [ebp-8h] , eax
> >> > 804FC29 895DFC mov [ebp-4h] , ebx
> >> > 804FC2C 8D6DF4 lea ebp , [ebp-Ch]
> >> > 804FC2F C3 ret near
> >> And the reason is that all four versions consist only of words that
> >> lxf analyses at compile time, so all four versions result in zero
> >> stuff to compile right away, and the same stack state to reconcile
> >> with the calling convention, so the same code is generated (which
> >> consists entirely of the reconciliation with the calling convention).
> >
> >Yes this is correct. But it does not analyse the code. Stack operations
> >just move pointers on a virtual stack, no code is generated.
> In the old days we had plain compilers with peephole optimization
> (e.g., bigForth with peephole optimization on the machine-code level,
> and SwiftForth which AFAIK combines primitives at the Forth level and
> generates code for that), and anything beyond that has been called an
> analytical compiler.
>
> So if your compiler employs arbitrary stack representations by keeping
> track at compile time what stack item resides in what register or what
> memory location, that's an analytical compiler, and keeping track of
> the stack items is analysis.

Good to know I have an analytical compiler!
> Gforth-fast supports few stack representations determined at Gforth
> build time, but only represents them as finite states at compile time
> and cannot keep track of arbitrary stack representations (e.g.,
> already DUP requires at least copying from one register (or memory
> location) to a different register), so I would not classify it as
> analytical.
> >In the example of the rectangle where lxf generated less good code
> >this happened. The code could be written in this way to improve
> >: preprect {: x1 y1 x2 y2 :} x1 y1 x1 y2 x2 y2 x2 y1 x1 y1 ; ok
> >: rect4 preprect moveto lineto lineto lineto lineto ;
> >
> >the preprect will generate 43 bytes of stack shuffling and rect4
> >30 bytes of calls ( 5 calls and one tail jump).
> But when I inline PREPRECT manually, the PREPRECT code becomes quite a
> bit longer, and I see some stores to the return stack. Strange.
> >doing the same for the stack shuffling variant saves even more
> >: rect5 2over 2>r 2>r over r@ 2r> over r@ 2r>
> >moveto lineto lineto lineto lineto ;
> >
> >This will generate 67 bytes, saving one call and one ret.
> >
> >Of course this is a stupid way of actually writing it. But if you want to
> >see what can be achived you will do it.
> It's only stupid is your goal is to write code that is easy to
> understand. If your goal is to produce the shortest code on lxf, it
> is smart:-).
>
> Let's try to make a smart version for VFX:
>
> : rect7 ( x1 y1 x2 y2 -- )
> 3 pick 3 pick over 3 pick 5 pick 2dup 5 pick 5 pick over
> moveto lineto lineto lineto lineto ;
> ( 88 bytes, 25 instructions )

This version is keeping the original 4 parameters on the stack.
A way without returnstack usage could be

: rect8 ( x1 y1 x2 y2 -- )
3 pick -rot swap 2dup 5 pick 5 pick over
moveto lineto lineto lineto lineto ;

Peter

> Hmm; it tends not to keep the items that it has already loaded in
> registers, and therefore has to reload them. lxf does a little better
> at 82 bytes. Admittedly, this is the kind of code that nobody writes,
> so it's understandable if VFX and lxf are not optimized for that.
> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2021: https://euro.theforth.net/2021

P Falth <peter.m.falth@gmail.com> writes:
>On Sunday, 12 September 2021 at 12:26:25 UTC+2, Anton Ertl wrote:
>> Let's try to make a smart version for VFX:
>>
>> : rect7 ( x1 y1 x2 y2 -- )
>> 3 pick 3 pick over 3 pick 5 pick 2dup 5 pick 5 pick over
>> moveto lineto lineto lineto lineto ;
>> ( 88 bytes, 25 instructions )
>
>This version is keeping the original 4 parameters on the stack.
>A way without returnstack usage could be
>
>: rect8 ( x1 y1 x2 y2 -- )
>3 pick -rot swap 2dup 5 pick 5 pick over
>moveto lineto lineto lineto lineto ;

Thanks for debugging it. This takes 73bytes on VFX, 67 on lxf.

It should be doable with 4 loads, 7 stores, one lea, 4 calls and one
jmp; 4*3+7*3+1*3+4*5+1*5=61 bytes.

lea ebp,[ebp-18h]
mov eax,[ebp+18h] #x2 #a
mov edx,[ebp+14h] #y2
mov [ebp],ebx #x1 #b
mov [ebp+18h],ebx #c
mov ebx,[ebp+1ch] #y1 #d
mov [ebp+10h],eax
mov [ebp+0ch],edx
mov [ebp+08h],eax
mov [ebp+04h],ebx
call MOVETO
call LINETO
call LINETO
call LINETO
jmp LINETO

You cannot just do the loads upfront, you have to do load #d after
stores #b and #c. You also have to do load #a before the store #c.

If you do all the loads upfront, you eventually need a
register-to-register move to get y1 into ebx.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>It should be doable with 4 loads, 7 stores, one lea, 4 calls and one
>jmp; 4*3+7*3+1*3+4*5+1*5=61 bytes.
>
>lea ebp,[ebp-18h]
>mov eax,[ebp+18h] #x2 #a
>mov edx,[ebp+14h] #y2
>mov [ebp],ebx #x1 #b
>mov [ebp+18h],ebx #c
>mov ebx,[ebp+1ch] #y1 #d
>mov [ebp+10h],eax
>mov [ebp+0ch],edx
>mov [ebp+08h],eax
>mov [ebp+04h],ebx
>call MOVETO
>call LINETO
>call LINETO
>call LINETO
>jmp LINETO

So actually you need only 3 loads (x1 is already in a register at the
start), and only 6 stores (you don't need to store the TOS into
memory), resulting in 55 bytes for the whole thing.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

dxforth <dxforth@gmail.com> writes:
> If good coding produces good results on any system then surely it is
> coding worth learning.

If by good results you mean the fastest possible execution, that has
always required painful hacks and sometimes very carefully written
assembly code, usually tuned to the specific processor architecture.
Doing that is sometimes necessary and sometimes fun, but it is a niche
skill that most programmers today don't have. "Good coding" at least to
me means writing clear and maintainable code whose performance is
reasonable for that development approach, and which suffices for the
application.

> To my knowledge there is no VFX/LXF/GForth for CP/M systems. OTOH
> there does appear to be many RC2014 Z80 systems and a surprising
> degree of interest in forth. Their users weren't lured by locals.
> They came for the full-on Forth experience.

What is RC2014? Today's equivalent of CP/M systems is microcontroller
systems (say the Raspberry Pi RP2040, and the equivalent of Forth is
MicroPython. A MicroPython program is likely to be 10x slower than the
equivalent Forth program on the same hardware, but since the hardware is
100x faster than a Z80, the MicroPython RP2040 user can easily solve the
same problems as the Z80 Forth user.

Forth after all was historically a straightforwardly implemented,
interpreted language, that was faster than other interpreted languages
like BASIC. If your program had a speed bottleneck you'd handle that by
writing CODE words. Fancy Forth compilers whose output competed with
handwritten assembly code came much later.

The CPU I'm currently interested in (ATTiny) is more like a Z80 (though
still much faster), and it is treated as somewhat obsolescent in the MCU
world.

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> Tough luck. If you cannot explain the concrete benefits of your
> philosophy on a concrete example, the rest of the world cannot
> distinguish it from mumbo-jumbo.

I would say it is often valuable to read an entire book and come away
from it with a philosophical lesson you can use later, that could not be
crystallized into a concrete example. The trouble is that multiple
readers will have different interpretations and won't always come away
with the same lesson. There is a whole industry of literary criticism
which is about comparing different interpretations of the same book.
I'm sure that literary critics have written lots of entire books that
try to explain the philosophical lessons of Shakespeare's "Hamlet".

Here, Hans Bezemer has advised me to read Thinking Forth in order to
receive a philosophical lesson about the evils of locals. I'm open to
having such an experience, but I have read the book before and looked at
it again last night, and that particular point didn't come across to me
either time. Maybe we need some literary critics to analyze Thinking
Forth and explain what its lessons are. A good starting point might be
identifying a page number where Thinking Forth says anything at all
about locals.

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> There are package repositories like CPAN and PyPI for dynamic
> languages, but efforts like this for static languages have not been
> similarly successful; e.g., in the case of Java, CJAN ceased
> development in 2004 and Apache JJAR is essentially abandoned.

I think that says more about Java culture than static types. Take a
look at hackage.haskell.org as a package repo for a much less widely
used language than Java. There is a ton of stuff there. The types
help, they really do, at least most of the time. It otherwise becomes
too cumbersome to remember what is what.

Rust also has a package library that has lots of stuff, though I haven't
used it or tried Rust yet.

On 13/09/2021 07:04, Paul Rubin wrote:
> dxforth <dxforth@gmail.com> writes:
>> If good coding produces good results on any system then surely it is
>> coding worth learning.
>
> If by good results you mean the fastest possible execution, that has
> always required painful hacks and sometimes very carefully written
> assembly code, usually tuned to the specific processor architecture.
> Doing that is sometimes necessary and sometimes fun, but it is a niche
> skill that most programmers today don't have. "Good coding" at least to
> me means writing clear and maintainable code whose performance is
> reasonable for that development approach, and which suffices for the
> application.

Clear and maintainable to whom - someone who is familiar with C et al.
Moore saw no distinction between learning Forth and learning other
languages - they all require effort. As to the barriers of learning
the former, he said:

"It is very hard to interest an experienced programmer in Forth. That’s
because he has invested in learning the tools for his language/operating
system and has built a library appropriate for his applications. Telling
him that Forth would be smaller, faster, and easier is not persuasive
compared to having to recode everything. A novice programmer, or an
engineer needing to write code, doesn’t face that obstacle and is much
more receptive - as might be the experienced programmer starting a new
project with new constraints"

>
>> To my knowledge there is no VFX/LXF/GForth for CP/M systems. OTOH
>> there does appear to be many RC2014 Z80 systems and a surprising
>> degree of interest in forth. Their users weren't lured by locals.
>> They came for the full-on Forth experience.
>
> What is RC2014? Today's equivalent of CP/M systems is microcontroller
> systems (say the Raspberry Pi RP2040, and the equivalent of Forth is
> MicroPython. A MicroPython program is likely to be 10x slower than the
> equivalent Forth program on the same hardware, but since the hardware is
> 100x faster than a Z80, the MicroPython RP2040 user can easily solve the
> same problems as the Z80 Forth user.

There it is in your own words. You have no reason to learn Forth.

dxforth <dxforth@gmail.com> writes:
> Moore saw no distinction between learning Forth and learning other
> languages - they all require effort. As to the barriers of learning
> the former, he said:

I don't find that. Some are harder than others. Haskell is relatively
hard. Forth is simple, but using it effectively without locals is a big
pain. It seems to take a lot of practice, like playing chess, to train
yourself in the stack juggling methods. If you train yourself at chess
enough, you can get to win chess games, which is nice. What do you get
by training at stack juggling, when you can use locals?

>> the MicroPython RP2040 user can easily solve the same problems as the
>> Z80 Forth user.
> There it is in your own words. You have no reason to learn Forth.

If my application was on an RP2040 I'd certainly do this extension part
in MicroPython or something similar. The RP2040 is a 32-bit cpu with
256k of ram on the chip. I'm instead on an 8-bit AVR with 2k of ram, so
that means Forth.

On 13/09/2021 13:45, Paul Rubin wrote:
> dxforth <dxforth@gmail.com> writes:
>> Moore saw no distinction between learning Forth and learning other
>> languages - they all require effort. As to the barriers of learning
>> the former, he said:
>
> I don't find that. Some are harder than others. Haskell is relatively
> hard. Forth is simple, but using it effectively without locals is a big
> pain. It seems to take a lot of practice, like playing chess, to train
> yourself in the stack juggling methods. If you train yourself at chess
> enough, you can get to win chess games, which is nice. What do you get
> by training at stack juggling, when you can use locals?
>
>>> the MicroPython RP2040 user can easily solve the same problems as the
>>> Z80 Forth user.
>> There it is in your own words. You have no reason to learn Forth.
>
> If my application was on an RP2040 I'd certainly do this extension part
> in MicroPython or something similar. The RP2040 is a 32-bit cpu with
> 256k of ram on the chip. I'm instead on an 8-bit AVR with 2k of ram, so
> that means Forth.
>

Room enough it appears for big dreams about using locals. Good luck.

P Falth <peter.m.falth@gmail.com> writes:
> I have to words that control the virtual stacks. v-init ( n --) and
> v-exit ( n-- )

Thanks for explaining. Do you ever now use n > 1? Or does inlining
eliminate enough of the places where it could do any good?

> When there is a control flow break ( if begin then etc) the stack must
> be set to a know state with 1 v-exit.

For something like "xxx IF foo ELSE bar THEN" is there an obstacle to
copying the v-stack state into both branches, so that you can try
inlining foo and bar and see how well those fit the current v-stack?

In general, does the compiler try different combinations of inlining to
see what works best?

I guess the virtual stacks connect the virtual stack slots to real
locations, which can be slots in the normal D and R stacks (maybe not
with the same slot numbers), plus machine registers. Do you also spill
to memory or maybe some other place, if you run out of registers? Do
you need a register allocator of the sort that fancy compilers have?

> [source inlining] has a problem of late binding and possibility of
> redifinitions and wordlist order. In my 64 bit lxf64 I compile first
> to tokenthreaded code to solve this problem

I think I understand: the token threaded code reflects the source code,
but it gets updated as source words are redefined. And you make a token
for each dictionary word. Let me know if I got that wrong.

> The compiler is about 1500 lines of code 50k bytes so it is not big.

That's impressively small for a compiler, but quite big compared to
anything I've done in Forth so far. I only use a very primitive level
of Forth for the most part (no OOP or anything like that).

Dare I ask if it uses locals? ;-)

> I think the biggest problem is that you need a good integration with
> an assembler to generate the final code... in lxf64 I generate intel
> asm source and send it to an external assembler dll that returns the
> compiled blob.

I guess that must be because of the enormously complex x64 instruction
set. On simpler processors I'm hoping to use a traditional FIG-style
assembler written in Forth.

> Other than that duped stack items require some consideration.

It seems to me also, some forward thinking is required to have stack
items early in a word land in their desired final destinations so they
don't have to be moved. Was that an issue?

> I think Forth is an ideal language to write this in as you can start
> with a few words using the code generator and then test and continue
> with others

Nice! I'll keep that in mind.

On Sunday, September 12, 2021 at 9:17:10 PM UTC+2, Anton Ertl wrote:
> P Falth <peter....@gmail.com> writes:
> >On Sunday, 12 September 2021 at 12:26:25 UTC+2, Anton Ertl wrote:
> >> Let's try to make a smart version for VFX:
> >>
> >> : rect7 ( x1 y1 x2 y2 -- )
> >> 3 pick 3 pick over 3 pick 5 pick 2dup 5 pick 5 pick over
> >> moveto lineto lineto lineto lineto ;
> >> ( 88 bytes, 25 instructions )
> >
> >This version is keeping the original 4 parameters on the stack.
> >A way without returnstack usage could be
> >
> >: rect8 ( x1 y1 x2 y2 -- )
> >3 pick -rot swap 2dup 5 pick 5 pick over
> >moveto lineto lineto lineto lineto ;
> Thanks for debugging it. This takes 73bytes on VFX, 67 on lxf.
>
> It should be doable with 4 loads, 7 stores, one lea, 4 calls and one
> jmp; 4*3+7*3+1*3+4*5+1*5=61 bytes.
>
> lea ebp,[ebp-18h]
> mov eax,[ebp+18h] #x2 #a
> mov edx,[ebp+14h] #y2
> mov [ebp],ebx #x1 #b
> mov [ebp+18h],ebx #c
> mov ebx,[ebp+1ch] #y1 #d
> mov [ebp+10h],eax
> mov [ebp+0ch],edx
> mov [ebp+08h],eax
> mov [ebp+04h],ebx
> call MOVETO
> call LINETO
> call LINETO
> call LINETO
> jmp LINETO

RECT8 (without the four calls and the jump) can be done in 25 bytes:

: rect8 3 PICK -ROT SWAP 2DUP 5 PICK 5 PICK OVER ; ok
FORTH> ' rect8 idis
$0133CC00 : rect8
$0133CC0A mov rbx, [rsp #24 +] qword
$0133CC0F pop rdi
$0133CC10 pop rax
$0133CC11 push rbx
$0133CC12 push rdi
$0133CC13 push rax
$0133CC14 push rdi
$0133CC15 push rax
$0133CC16 mov rbx, [rsp #40 +] qword
$0133CC1B mov rcx, rbx
$0133CC1E mov rbx, [rsp #32 +] qword
$0133CC23 push rcx
$0133CC24 push rbx
$0133CC25 push rcx
$0133CC26 ;
$0133CC30 cmp rbx, rbx
FORTH> $0133CC26 $0133CC0A - . 28 ok

-marcel

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> There are more optimal ways, and I can look them up if you want, but
> they are not simpler.

At the moment I think I need "simple", but thanks ;-).

I found an interesting article by Doneil Hoekman about a "Universal
Stack Word", in Forth Dimensions v07n5 (1986), p. 25,
http://forth.org/fd/FD-V07N5.pdf . He wrote a parsing word STACK that
reads a before-and-after stack picture and rearranges the stack
according to the picture. He gives examples of how to redefine some
standard Forth words:

: DROP STACK A| ;
: DUP STACK A|AA ;
: SWAP STACK AB|BA ;
: ROT STACK ABC|BCA ;
: 2SWAP STACK ABCD|CDAB ;

I didn't examine the "motion planning" for this, but I can see that it
is sort of related.

Paul Rubin <no.email@nospam.invalid> writes:
>P Falth <peter.m.falth@gmail.com> writes:
>> [source inlining] has a problem of late binding and possibility of
>> redifinitions and wordlist order. In my 64 bit lxf64 I compile first
>> to tokenthreaded code to solve this problem
>
>I think I understand: the token threaded code reflects the source code,
>but it gets updated as source words are redefined. And you make a token
>for each dictionary word. Let me know if I got that wrong.

Consider:

: foo ... ;
: bar ... foo ... ;
: foo ... ;
: baz ... bar ... ;

The FOO in BAR refers to the first definition of FOO. When compiling
BAZ, which inlines BAR, source inlining would compile (or inline) a
call to the second definition of FOO. You can also have examples
where different search orders are involved. The way to fix this is to
resolve the names when they are originally compiled, e.g., by
compiling the word into token-threaded code which is then used for
inlining. So the point is *not* to update this code for BAR when
another FOO is defined.

>I guess that must be because of the enormously complex x64 instruction
>set.

Here are numbers from Gforth:

[~/gforth:124764] wc -l arch/amd64/{asm.fs,disasm.fs}
732 arch/amd64/asm.fs
610 arch/amd64/disasm.fs
1342 total

This includes SSE2; not sure how much more of the SIMD stuff is
supported.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

Marcel Hendrix <mhx@iae.nl> writes:
>On Sunday, September 12, 2021 at 9:17:10 PM UTC+2, Anton Ertl wrote:
>> P Falth <peter....@gmail.com> writes:
>> >On Sunday, 12 September 2021 at 12:26:25 UTC+2, Anton Ertl wrote:
>> >> Let's try to make a smart version for VFX:
>> >>
>> >> : rect7 ( x1 y1 x2 y2 -- )
>> >> 3 pick 3 pick over 3 pick 5 pick 2dup 5 pick 5 pick over
>> >> moveto lineto lineto lineto lineto ;
>> >> ( 88 bytes, 25 instructions )
>> >
>> >This version is keeping the original 4 parameters on the stack.
>> >A way without returnstack usage could be
>> >
>> >: rect8 ( x1 y1 x2 y2 -- )
>> >3 pick -rot swap 2dup 5 pick 5 pick over
>> >moveto lineto lineto lineto lineto ;
>> Thanks for debugging it. This takes 73bytes on VFX, 67 on lxf.
>>
>> It should be doable with 4 loads, 7 stores, one lea, 4 calls and one
>> jmp; 4*3+7*3+1*3+4*5+1*5=61 bytes.
>>
>> lea ebp,[ebp-18h]
>> mov eax,[ebp+18h] #x2 #a
>> mov edx,[ebp+14h] #y2
>> mov [ebp],ebx #x1 #b
>> mov [ebp+18h],ebx #c
>> mov ebx,[ebp+1ch] #y1 #d
>> mov [ebp+10h],eax
>> mov [ebp+0ch],edx
>> mov [ebp+08h],eax
>> mov [ebp+04h],ebx
>> call MOVETO
>> call LINETO
>> call LINETO
>> call LINETO
>> jmp LINETO
>
>RECT8 (without the four calls and the jump) can be done in 25 bytes:
>
>: rect8 3 PICK -ROT SWAP 2DUP 5 PICK 5 PICK OVER ; ok
>FORTH> ' rect8 idis
>$0133CC00 : rect8
>$0133CC0A mov rbx, [rsp #24 +] qword
>$0133CC0F pop rdi
>$0133CC10 pop rax
>$0133CC11 push rbx
>$0133CC12 push rdi
>$0133CC13 push rax
>$0133CC14 push rdi
>$0133CC15 push rax
>$0133CC16 mov rbx, [rsp #40 +] qword
>$0133CC1B mov rcx, rbx
>$0133CC1E mov rbx, [rsp #32 +] qword
>$0133CC23 push rcx
>$0133CC24 push rbx
>$0133CC25 push rcx
>$0133CC26 ;
>$0133CC30 cmp rbx, rbx
>FORTH> $0133CC26 $0133CC0A - . 28 ok

Or do three pops, 9 pushes, and one register-register move (15 bytes).
However, code size is not everything. Most CPUs can only do one store
per cycle, which means that the stores in my version above take 6
cycles, the stores in the iForth-generated version take 8 cycles, and
the 9-push version takes 9 cycles. Out-of-order execution can push
outstanding store to later, which can help in some cases. But in this
case, I don't think that this is such a case, because I expect that
the MOVETO will load some of the store results right away. In the
version I posted above it would probably be better to order the stores
such that the items closer to the top are stored early.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 13/09/2021 03:23, S Jack wrote:
>
> <saga> Lets make Forth great again!

Forth lost?

> In industry a manager stands in quicksand with alligators biting at
> his butt, his arms tightly bound by a boa going for his neck,
> and his head in a cloud of mosquitoes and hornets. Along the dry bank
> comes whistling and skipping a little programmer. He sees the
> manager and shouts "Hey Boss, I got just the thing to fix all your
> problems. It's called FORTH!".

That would be the Forth industrial complex talking. Moore has always
been more circumspect as to who might benefit from Forth. Let's just
say his opinion of the average programmer isn't great. Luckily forthers
are all above average.

On Monday, 13 September 2021 at 07:43:02 UTC+2, Paul Rubin wrote:
> P Falth <peter....@gmail.com> writes:
> > I have to words that control the virtual stacks. v-init ( n --) and
> > v-exit ( n-- )
> Thanks for explaining. Do you ever now use n > 1? Or does inlining
> eliminate enough of the places where it could do any good?

Yes in some places n>1 is used. One example is in compiling a constant
There it is used as it creates smaller code.

I should also explain that there is not "one" code generator. Instead each
word is its own code generator. There are about 400 words that have special
code generators. User defined words generate only a call to its compiled
code. There is a possibility to do user defined macros that source inline.

> > When there is a control flow break ( if begin then etc) the stack must
> > be set to a know state with 1 v-exit.
> For something like "xxx IF foo ELSE bar THEN" is there an obstacle to
> copying the v-stack state into both branches, so that you can try
> inlining foo and bar and see how well those fit the current v-stack?

The code generation does not stop after a v-exit it continues but from the new
known state. For sure foo and bar will be able to generate code.
I had from the beginning the idea to save and restore virtual stacks at
branches but it got to complicated.

> In general, does the compiler try different combinations of inlining to
> see what works best?

No nothing like that. That is up to the programmer to discover what
source generates the best code. SEE is a very useful tool for that
> I guess the virtual stacks connect the virtual stack slots to real
> locations, which can be slots in the normal D and R stacks (maybe not
> with the same slot numbers), plus machine registers. Do you also spill
> to memory or maybe some other place, if you run out of registers? Do
> you need a register allocator of the sort that fancy compilers have?

Yes and there are also immediate values (known at compile time)
that are stored directly in the virtual stacks. If for ex. an add operation
sees that the 2 top stack items are immediate values they are added
and the result stored in the virtual stack.

If I run out of registers they are stored to memory. The register allocator
is not sophisticated. It just picks the first register in a list of free registers.
When not needed anymore they are returned to the free list.
> > [source inlining] has a problem of late binding and possibility of
> > redifinitions and wordlist order. In my 64 bit lxf64 I compile first
> > to tokenthreaded code to solve this problem
> I think I understand: the token threaded code reflects the source code,
> but it gets updated as source words are redefined. And you make a token
> for each dictionary word. Let me know if I got that wrong.

Yes you got it wrong. The tokens are like opcodes in a CPU. In the VM
of lxf64 there are currently 155 opcodes. From the forth point of view
it is running natively on a CPU ( the VM). There is a call opcode to
just like on a normal cpu. In fact lxf64 runs perfectly well on the VM.

> > The compiler is about 1500 lines of code 50k bytes so it is not big.
> That's impressively small for a compiler, but quite big compared to
> anything I've done in Forth so far. I only use a very primitive level
> of Forth for the most part (no OOP or anything like that).
>
> Dare I ask if it uses locals? ;-)

For sure. It does not. I started using FORTH before locals was introduced
and have never seen a need to use them.

As a side-note. I also stopped using VALUES when TO got overloaded.
I have never missed them.

> > I think the biggest problem is that you need a good integration with
> > an assembler to generate the final code... in lxf64 I generate intel
> > asm source and send it to an external assembler dll that returns the
> > compiled blob.
> I guess that must be because of the enormously complex x64 instruction
> set. On simpler processors I'm hoping to use a traditional FIG-style
> assembler written in Forth.

Yes that is exactly what i use in lxf. For lxf64 I tried loading the assembler
from gforth. It doubled the size of my forth imagefile!. lxf64 compiles
a complete forth2012 compliant system in an image file of 46k bytes.
But now instead i have 2.5MB dll for assembler and 1.5 MB for disassembler

Peter

> > Other than that duped stack items require some consideration.
> It seems to me also, some forward thinking is required to have stack
> items early in a word land in their desired final destinations so they
> don't have to be moved. Was that an issue?
> > I think Forth is an ideal language to write this in as you can start
> > with a few words using the code generator and then test and continue
> > with others
> Nice! I'll keep that in mind.

On Sunday, September 12, 2021 at 11:15:13 PM UTC+2, Paul Rubin wrote:
> Here, Hans Bezemer has advised me to read Thinking Forth in order to
> receive a philosophical lesson about the evils of locals. I'm open to
> having such an experience, but I have read the book before and looked at
> it again last night, and that particular point didn't come across to me
> either time. Maybe we need some literary critics to analyze Thinking
> Forth and explain what its lessons are. A good starting point might be
> identifying a page number where Thinking Forth says anything at all
> about locals.
Try chapter 7, "The Stylish Stack"?

There is a reason you won't find anything on locals, because it violates its basic philosophy ("stamp out the variables").

I've taken the liberty to pick some quotes from it. Of course, this opens up attacks like "cherry picking", but if you think so, go cherry picking yourself.

"Forth is a language,an operating system, a set of tools, and a philosophy.
It is an ideal means for thinking because it corresponds to the way our minds
work. Thinking Forth is thinking simple, thinking elegant, thinking flexible.

The mechanism that produces this effect is
Forth’s data stack. Forth automatically pushes numbers onto the stack; words
that require numbers as input automatically pop them off the stack; words
that produce numbers as output automatically push them onto the stack.

Because Forth uses a stack for passing data, words can nest within words.
Any word can put numbers on the stack and take them off without upsetting
the f1ow of data between words at a higher level (provided, of course, that the
word doesn’t consume or leave any unexpected values). Thus the stack supports
structured, modular programming while providing a simple mechanism
for passing local arguments.

If Forth code is really well-written, there should be nothing ambiguous
about it. This means that:
• supporting lexicons have a well-designed syntax
• stack inputs and outputs are commented
• the purpose is commented (if it’s not clear from the name or stack comment)
• definitions are not too long
• not too many arguments are passed to a single definition via the stack.

One important result of this approach: Arguments are unnamed. They reside
on the stack, not in named variables. This effect is one of the reasons for Forth’s
elegance. At the same time it’s one of the reasons badly written Forth code
can be unreadable.

Simplify code by using the stack. But don’t stack too deeply within any single
definition. Redesign, or, as a last resort, use a named variable.

Some folks like the words PICK and ROLL. They use these words to access
elements from any level on the stack. We don’t recommend them. For one
thing, PICK and ROLL encourage the programmer to think of the stack as an
array, which it is not. If you have so many elements on the stack that you need
PICK and ROLL, those elements should be in an array instead.
Second, they encourage the programmer to refer to arguments that have
been left on the stack by higher-level, calling definitions without being explicitly
On PICK and ROLL passed as arguments. This makes the definition dependent
on other definitions. That’s unstructured—and dangerous."

Hans Bezemer

On 13/09/2021 17:34, Hans Bezemer wrote:
>
> There is a reason you won't find anything on locals, because it violates its
> basic philosophy ("stamp out the variables").
>
> I've taken the liberty to pick some quotes from it. Of course, this opens up
> attacks like "cherry picking", but if you think so, go cherry picking yourself.

I rather like this one which is an appeal to use the mechanism Forth provides:

"Programmers unaccustomed to a language in which data can be passed implicitly
don’t always utilize the stack as fully as they should."

Sadly the Hayes' syntax has normalized locals use in Forth. After all, one
can't get simpler than a stack comment. An illusion which worked so well
that when ANS announced their syntax, objectors screamed the locals were in
reverse order! Never mind this was Forth which uses a LIFO stack. These
were but subtleties locals could - and should - abstract away.

Paul Rubin <no.email@nospam.invalid> writes:
>anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> There are more optimal ways, and I can look them up if you want, but
>> they are not simpler.
>
>At the moment I think I need "simple", but thanks ;-).

You can find some more discussion of the simple approach in section
4.4.1 of

@InProceedings{ertl&pirker97,
author = {M. Anton Ertl and Christian Pirker},
title = {The Structure of a {Forth} Native Code Compiler},
booktitle = {EuroForth '97 Conference Proceedings},
year = {1997},
address = {Oxford},
pages = {107--116},
url = {http://www.complang.tuwien.ac.at/papers/ertl%26pirker97.ps.gz},
abstract = {Writing a sophisticated Forth native code compiler
poses some tasks that are not discussed in compiler
text books. Some of these tasks arise from specific
language features, others from the requirement for
very fast compilation to maintain interactivity. In
this paper we describe some of the more interesting
data structures and algorithms used in the RAFTS
prototype.}
}

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

P Falth <peter.m.falth@gmail.com> writes:
>For lxf64 I tried loading the assembler
>from gforth. It doubled the size of my forth imagefile!. lxf64 compiles
>a complete forth2012 compliant system in an image file of 46k bytes.
>But now instead i have 2.5MB dll for assembler and 1.5 MB for disassembler

:-) For comparison, in gforth-fast AMD64 we see the following number
of bytes:

data+
threaded native
58720 34524 asm.fs
98616 38552 disasm.fs

And I am sure it is smaller on lxf.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On Sunday, September 12, 2021 at 4:15:13 PM UTC-5, Paul Rubin wrote:
> crystallized into a concrete example. The trouble is that multiple
> readers will have different interpretations and won't always come away
> with the same lesson. There is a whole industry of literary criticism

In Stephen Donaldson's Lord Foul's Bane two characters are conversing.
The dialog is shown so what is being said and what is heard is seen
but also shown in the mind of each character the dialog is interpreted
differently. The two think they are on the same page and the dialog
appears to corroborate it but in actuality the two characters are 180
out. Thus is the nature of the spoken or written language.
--
me

On Sunday, September 12, 2021 at 4:04:05 PM UTC-5, Paul Rubin wrote:
> Forth after all was historically a straightforwardly implemented,
> interpreted language, that was faster than other interpreted languages
> like BASIC. If your program had a speed bottleneck you'd handle that by
> writing CODE words.

That's why I started and the simplicity still appeals to me.
As long as I can beat Basic I'm happy.
--
me

S Jack schrieb am Montag, 13. September 2021 um 17:23:44 UTC+2:
> On Sunday, September 12, 2021 at 4:15:13 PM UTC-5, Paul Rubin wrote:
> > crystallized into a concrete example. The trouble is that multiple
> > readers will have different interpretations and won't always come away
> > with the same lesson. There is a whole industry of literary criticism
> In Stephen Donaldson's Lord Foul's Bane two characters are conversing.
> The dialog is shown so what is being said and what is heard is seen
> but also shown in the mind of each character the dialog is interpreted
> differently. The two think they are on the same page and the dialog
> appears to corroborate it but in actuality the two characters are 180
> out. Thus is the nature of the spoken or written language.

It's just another example for Plato's allegory of the cave.

But computer languages have well-defined syntax and should have well-
defined semantics, and perhaps should work only in Turing-complete systems.
Alas in reality UB is everywhere....

On 13/09/2021 07:15, Paul Rubin wrote:
>
> Here, Hans Bezemer has advised me to read Thinking Forth in order to
> receive a philosophical lesson about the evils of locals. I'm open to
> having such an experience, but I have read the book before and looked at
> it again last night, and that particular point didn't come across to me
> either time.

When Brodie wrote his book in 1984, locals were barely known, appearing
alongside off-beat topics in FD. To say there's no mention of locals in
Thinking Forth was a reflection of the times. What was evident then, was
folks writing code with lots of temporary variables just as they might
write it in C - and this was discussed in the book.
Locals came into public consciousness with ANS' endorsement. ANS was
establishment and, apart from Moore, nobody challenged establishment.
It's only with the crumbling of ANS and its subsequent hi-jacking that
questions are once again being asked.

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> You can find some more discussion of the simple approach in section
> 4.4.1 of
> author = {M. Anton Ertl and Christian Pirker},
> title = {The Structure of a {Forth} Native Code Compiler},

Thanks, this is a good paper and I will try to digest it. One remaining
issue is the difference between a native code compiler, and one that
targets a bytecode VM for purposes of maximizing code density. I
remember reading that on 32-bit targets, native code turned out to be
denser than traditional threaded code, so the transition from 16 bit to
32 bit hardware brought with it a transition to native code, e.g. in
SwiftForth. Bytecode didn't come into consideration because of the
resulting slowdown, and the relative abundance of memory on those
targets.

On an 8-bit target with 16-bit Forth cells, I think I really have to use
bytecode, since otherwise even simple arithmetic or stack operations
take multiple instructions. Traditional bytecode VM's for Forth are
simple stack machines with primitives for common Forth words like DUP,
SWAP, etc. But if the VM is a compiler target, it may be better to
design it in conjunction with the compiler. For example, maybe it
should have registers that the compiler can use.

I'm surprised this type of question has not gotten more attention in the
Forth world, given the long association between Forth and memory starved
targets.