On Wed, 5 May 2021 21:08:28 +0300, Ruvim <ruvim.pinka@gmail.com>
wrote:

>> NDCS = non default compilation semantics, e.g. IF
>>
>> And that's probably what triggered my NDCS esploration. The base
>> problem comes from the standard's definitions of words. There are
>> three cases:
>> 1) normal words
>> 2) immediate words
>> 3) NDCS words
>
>It's unclear do you mean the glossary entries, or the implementations in
>particular Forth system?
>
>In which group does the EXIT word fall?

Let's restate the groups:
a) normal words,
b) NDCS words,
c) Immediate words are a subset of NDCS words.

In most Forth systems, groups b) and c) are the same because the
system only has IMMEDIATE as a way to express NDCS behaviour.

In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
of COMPILE, for such words is NDCS,

: ndcs, \ ??? xt -- ???
\ *G Perform the compilation action of an NDCS word. This may have
\ ** a stack effect or parse the input stream.

: compile-word \ i*x xt -- j*x
\ *G Process an XT for compilation. May parse or have stack effect.
dup ndcs? \ NDCS or normal
if ndcs, else compile, then
;

In VFX, most uses of COMPILE, should use COMPILE-WORD instead.
The alternative is to redefine COMPILE, to take note of NDCS words
but ... politics, politic, politics.

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

On 2021-05-06 14:38, P Falth wrote:
> On Thursday, 6 May 2021 at 10:52:00 UTC+2, dxforth wrote:
>> On 6/05/2021 16:19, Ruvim wrote:
>>> On 2021-05-06 04:49, dxforth wrote:
>>>> On 6/05/2021 00:37, Anton Ertl wrote:
>>>>> dxforth <dxf...@gmail.com> writes:
>>>>>> Only a TC can declare what is standard - so ask them.
>>>>>
>>>>> Standardization committees declare what is standard through the
>>>>> standard document. There may be cases where the standard document is
>>>>> unclear or contradictory, in which case clarification may be needed,
>>>>> but the stack effect or COMPILE, is not one of these cases.
>>>>
>>>> The behaviour of test-if was ambiguous before it got to COMPILE, .
>>>
>>> Here it is:
>>>
>>> : test-if
>>> c" if" find dup 0= if ." unfound" 2drop exit then
>>> 1 = if ." executed" execute else ." compiled" compile, then
>>> ; immediate
>>>
>>>
>>> In what place is it ambiguous? In "find"?
>> Using FIND to invoke IF is ambiguous. I've a good idea what will
>> happen on single-xt systems but that's all. Same for POSTPONE S" .
>> Dual-xt systems have muddied the waters. Today users have no way
>> to know what is portable - only what works on their system.
>
> Yes I agree to this. My LXF is dual-xt. FIND on IF will return the interpretation
> xt. When executed this will print an error message and abort.

It's OK for interpretation state, but not OK for compilation state.

In dual-xt systems FIND should depend on STATE and return the different
results depending on STATE for some words.

My clarification for FIND (the next version)
https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682

>
> In my opinion FIND was left in pre F94 state and not modified to align with
> the execution, interpretation and compilation semantics definitions.

It's specification is too lean. But the Rationale part shows the
intention clearly. And using this rationale we can make the
specification better.

>
> Of course you can get my IF to work with FIND by doing
> : IF postpone IF ; immediate
>
> As an alternative to FIND I can suggest
> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )

If you can implement this, nothing prevent you to implement STATE
dependent FIND

But what for do you need FIND-POSTPONE ?

> The result if found just needs to be executed to perform the respective actions
> It also hides the implementation details ( single-xt, dual-xt, dual headers etc)

It doesn't cover sing-xt systems. In these systems the corresponding
action should be performed only in the corresponding STATE.

--
Ruvim

On 2021-05-06 17:07, Ruvim wrote:
> On 2021-05-06 14:38, P Falth wrote:
[...]
>> As an alternative to FIND I can suggest
>> FIND-INTERPRET  ( addr u -- addr u 0 | xt2 xt1 -1 )
>> FIND-COMPILE     ( addr u -- addr u 0 | xt2 xt1 -1 )
>> FIND-POSTPONE  ( addr u -- addr u 0 | xt2 xt1 -1 )

They are quite close to FIND-NAME NAME>COMPILE

>
> If you can implement this, nothing prevent you to implement STATE
> dependent FIND

But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
either "EXECUTE" or "COMPILE," and nothing else.

Otherwise FIND cannot be implemented via these words.

>
> But what for do you need FIND-POSTPONE ?
>
>
>> The result if found just needs to be executed to perform the
>> respective actions
>> It also hides the implementation details ( single-xt, dual-xt, dual
>> headers etc)
>
> It doesn't cover sing-xt systems. In these systems the corresponding
> action should be performed only in the corresponding STATE.

I missed that they return two xt-s. Then yes, they cover single-xt
systems too.

But then what is a way to detect whether a word is an ordinary word?

And what is a way to get xt of an ordinary word, that identifies the
execution semantics for this word?

--
Ruvim

On Thursday, 6 May 2021 at 16:07:26 UTC+2, Ruvim wrote:
> On 2021-05-06 14:38, P Falth wrote:
> > On Thursday, 6 May 2021 at 10:52:00 UTC+2, dxforth wrote:
> >> On 6/05/2021 16:19, Ruvim wrote:
> >>> On 2021-05-06 04:49, dxforth wrote:
> >>>> On 6/05/2021 00:37, Anton Ertl wrote:
> >>>>> dxforth <dxf...@gmail.com> writes:
> >>>>>> Only a TC can declare what is standard - so ask them.
> >>>>>
> >>>>> Standardization committees declare what is standard through the
> >>>>> standard document. There may be cases where the standard document is
> >>>>> unclear or contradictory, in which case clarification may be needed,
> >>>>> but the stack effect or COMPILE, is not one of these cases.
> >>>>
> >>>> The behaviour of test-if was ambiguous before it got to COMPILE, .
> >>>
> >>> Here it is:
> >>>
> >>> : test-if
> >>> c" if" find dup 0= if ." unfound" 2drop exit then
> >>> 1 = if ." executed" execute else ." compiled" compile, then
> >>> ; immediate
> >>>
> >>>
> >>> In what place is it ambiguous? In "find"?
> >> Using FIND to invoke IF is ambiguous. I've a good idea what will
> >> happen on single-xt systems but that's all. Same for POSTPONE S" .
> >> Dual-xt systems have muddied the waters. Today users have no way
> >> to know what is portable - only what works on their system.
> >
> > Yes I agree to this. My LXF is dual-xt. FIND on IF will return the interpretation
> > xt. When executed this will print an error message and abort.
> It's OK for interpretation state, but not OK for compilation state.
>
> In dual-xt systems FIND should depend on STATE and return the different
> results depending on STATE for some words.

The standard says that FIND may return different XTs depending on state.
It is not obliged to do so

> My clarification for FIND (the next version)
> https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682

I have seen your efforts.
In my opinion it is better to start with a clean sheet and implement something
that can substitute FIND and make it obsolete.

Peter

> >
> > In my opinion FIND was left in pre F94 state and not modified to align with
> > the execution, interpretation and compilation semantics definitions.
> It's specification is too lean. But the Rationale part shows the
> intention clearly. And using this rationale we can make the
> specification better.
> >
> > Of course you can get my IF to work with FIND by doing
> > : IF postpone IF ; immediate
> >
> > As an alternative to FIND I can suggest
> > FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> > FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> > FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
> If you can implement this, nothing prevent you to implement STATE
> dependent FIND
>
> But what for do you need FIND-POSTPONE ?
> > The result if found just needs to be executed to perform the respective actions
> > It also hides the implementation details ( single-xt, dual-xt, dual headers etc)
> It doesn't cover sing-xt systems. In these systems the corresponding
> action should be performed only in the corresponding STATE.
>
>
> --
> Ruvim

On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> On 2021-05-06 17:07, Ruvim wrote:
> > On 2021-05-06 14:38, P Falth wrote:
> [...]
> >> As an alternative to FIND I can suggest
> >> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> >> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
> They are quite close to FIND-NAME NAME>COMPILE

Yes they are. But they also hide the NT and can thus accomodate
also a system with dual NTs. Someone (Mark Williams?) described
such a system that was simple and clean.

> > If you can implement this, nothing prevent you to implement STATE
> > dependent FIND
> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
> either "EXECUTE" or "COMPILE," and nothing else.

To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
limit the possibilities of the system. Take for example a constant.
XT2 could be the actual value of the constant and XT1 LITERAL for compiling.

My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
to a JIT compiler that compiles the tokens to native code and then executes it

> Otherwise FIND cannot be implemented via these words.

There is no requirements to implement FIND with these words.

> >
> > But what for do you need FIND-POSTPONE ?
> >
> >
> >> The result if found just needs to be executed to perform the
> >> respective actions
> >> It also hides the implementation details ( single-xt, dual-xt, dual
> >> headers etc)
> >
> > It doesn't cover sing-xt systems. In these systems the corresponding
> > action should be performed only in the corresponding STATE.
> I missed that they return two xt-s. Then yes, they cover single-xt
> systems too.
>
> But then what is a way to detect whether a word is an ordinary word?

That is not needed!

>
> And what is a way to get xt of an ordinary word, that identifies the
> execution semantics for this word?
' and ['] are still available for this.

BR
Peter
>
>
> --
> Ruvim

P Falth <peter.m.falth@gmail.com> writes:
>On Thursday, 6 May 2021 at 16:07:26 UTC+2, Ruvim wrote:
[...]
>The standard says that FIND may return different XTs depending on state.
>It is not obliged to do so

True. The intention behind that is probably that classic
single-xt+immediate-flag systems return the same xt in either state
and cmForth returns, for some words, one xt in one state, and a
different xt in the other state.

Sure, someone who implements a cmForth-like implementation could say
that there is no requirement in the specification to do it that way,
and produce a find that only returns one of the xts, but that would
mean that user-defined text interpreters that use FIND do not work on
that system.

Likewise for a more modern dual-xt system like yours.

>> My clarification for FIND (the next version)
>> https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682

Or another one:

https://forth-standard.org/proposals/clarify-find?hideDiff#reply-165

>I have seen your efforts.
>In my opinion it is better to start with a clean sheet and implement something
>that can substitute FIND and make it obsolete.

That has already been accepted:

https://forth-standard.org/proposals/find-name?hideDiff#reply-174

But FIND will continue to be in the standard for at least one more
release (or maybe more, it has not been declared obsolescent, and
nobody has proposed making it obsolescent, not even those who claim we
should not fix FIND, because it will be replaced).

Moreover, there are people who make an outrageous claims based on the
current specification of FIND (in particular, IIRC Stephen Pelc made
such a claim based on the usage of "immediate" in the current
specification that is at odds with the usage of "immediate" in the
rest of the document). So while FIND is still in the standard, it
will continue to be a sore point. That can be fixed by clarifying it.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

P Falth <peter.m.falth@gmail.com> writes:
>To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
>limit the possibilities of the system. Take for example a constant.
>XT2 could be the actual value of the constant and XT1 LITERAL for compiling.

Yes, I also played with that idea. But it does not scale to doubles
and floats.

>My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
>to a JIT compiler that compiles the tokens to native code and then executes it

If you implement EXECUTE, why go there?

Another idea I played with: for the compilation token xt1 xt2, let xt2
be a specialized compiler for xt1. I think you can find this idea in
my early writings about the compilation token. But we also have
COMPILE, and need to implement that. So the better way is to dispatch
the specialized compilers from "COMPILE,", and that's what we
implement now:

NAME>COMPILE always produces ' EXECUTE or ' COMPILE, as xt2. This
makes it straightforward to define FIND such that user-defined text
interpreters work. One can use SET-OPTIMIZER to tell the system what
COMPILE, should do for a certain word.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

On Thursday, 6 May 2021 at 19:30:49 UTC+2, Anton Ertl wrote:
> P Falth <peter....@gmail.com> writes:
> >To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
> >limit the possibilities of the system. Take for example a constant.
> >XT2 could be the actual value of the constant and XT1 LITERAL for compiling.
> Yes, I also played with that idea. But it does not scale to doubles
> and floats.

In fact my actual implementation is more compilated, XT2 is an address where
the constant, double or float is stored. XT1 is the appropriate fetch followed
by the appropriate literal

> >My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
> >to a JIT compiler that compiles the tokens to native code and then executes it
> If you implement EXECUTE, why go there?

This is one area I have not decided yet. It could be that I will just execute
the tokenized code when interpreting. At least this would work well when the
tokenized code will be inlined while compiling. No need to waste space
compiling native code for it.

On the other hand words that for different reasons can not be inlined will need
to be compiled to native code so they later can be called directly from other
native code. In this case there is no need to keep the tokencode.

> Another idea I played with: for the compilation token xt1 xt2, let xt2
> be a specialized compiler for xt1. I think you can find this idea in
> my early writings about the compilation token. But we also have
> COMPILE, and need to implement that. So the better way is to dispatch
> the specialized compilers from "COMPILE,", and that's what we
> implement now:

My COMPILE, in both lxf and lxf64 are dumb. They will just compile a call
to the XT

> NAME>COMPILE always produces ' EXECUTE or ' COMPILE, as xt2. This
> makes it straightforward to define FIND such that user-defined text
> interpreters work. One can use SET-OPTIMIZER to tell the system what
> COMPILE, should do for a certain word.

None of my systems produces ' EXECUTE or ' COMPILE, as xt2.
I can see no requirement for limiting NAME>COMPILE in that way.

BR
Peter

> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2020: https://euro.theforth.net/2020

On Thu, 06 May 2021 16:26:01 GMT, anton@mips.complang.tuwien.ac.at
(Anton Ertl) wrote:

>Moreover, there are people who make an outrageous claims based on the
>current specification of FIND (in particular, IIRC Stephen Pelc made
>such a claim based on ...

I am proud to be thought to make outrageous claims, especially
when they are correct.

In particular, until we (standard committee and other interested
parties) come to terms with the impact of NDCS, arguing about
FIND and COMPILE, is just another silly discussion about how
many angels can dance on the head of a pin.

The easy thing to do is to declare non-immediate NDCS words
as non-compliant. Alternatively, you open the box and enable
new standards-compliant notations such as interpretive
BEGIN ... UNTIL and DO ... LOOP.

You have to start by understanding that the standards from
ANS-94 onwards have bugs. This is not to put the authors down,
but to realise that we are all imperfect.

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

On 2021-05-06 16:46, Stephen Pelc wrote:
> On Wed, 5 May 2021 21:08:28 +0300, Ruvim <ruvim.pinka@gmail.com>
> wrote:
>
>>> NDCS = non default compilation semantics, e.g. IF
>>>
>>> And that's probably what triggered my NDCS esploration. The base
>>> problem comes from the standard's definitions of words. There are
>>> three cases:
>>> 1) normal words
>>> 2) immediate words
>>> 3) NDCS words
>>
>> It's unclear do you mean the glossary entries, or the implementations in
>> particular Forth system?
>>
>> In which group does the EXIT word fall?
>
> Let's restate the groups:
> a) normal words,
> b) NDCS words,
> c) Immediate words are a subset of NDCS words.
>
> In most Forth systems, groups b) and c) are the same because the
> system only has IMMEDIATE as a way to express NDCS behaviour.
>
> In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
> of COMPILE, for such words is NDCS,
>
> : ndcs, \ ??? xt -- ???
> \ *G Perform the compilation action of an NDCS word. This may have
> \ ** a stack effect or parse the input stream.
>
> : compile-word \ i*x xt -- j*x
> \ *G Process an XT for compilation. May parse or have stack effect.
> dup ndcs? \ NDCS or normal
> if ndcs, else compile, then
> ;
>
> In VFX, most uses of COMPILE, should use COMPILE-WORD instead.
> The alternative is to redefine COMPILE, to take note of NDCS words
> but ... politics, politic, politics.

Taking into account that "ndcs," just performs another xt:

: (ndcs,) \ i*x xt -- j*x
>code-len @ execute \ execute NDCS/IMMEDIATE word
;

Why not return this xt by "find" ?

: find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
find dup 0= if exit then drop ( xt1 )
dup ndcs? if >code-len @ 1 else
dup immediate? if 1 else -1 then then ( xt1 xt2 n )
state @ if rot else swap then drop
;

After this fix all my tests work as expected.

To be on the safe side, I would redefine/fix some other words too, to
exclude situations when "compile," produces unexpected stack effect when
it's applied to some xt.

Nothing standard word should return such xt for which "compile,"
produces unexpected stack effect.

--
Ruvim

On 2021-05-07 14:12, Ruvim wrote:
> On 2021-05-06 16:46, Stephen Pelc wrote:
>> On Wed, 5 May 2021 21:08:28 +0300, Ruvim <ruvim.pinka@gmail.com>
>> wrote:
>>
>>>> NDCS = non default compilation semantics, e.g. IF
>>>>
>>>> And that's probably what triggered my NDCS esploration. The base
>>>> problem comes from the standard's definitions of words. There are
>>>> three cases:
>>>> 1) normal words
>>>> 2) immediate words
>>>> 3) NDCS words
>>>
>>> It's unclear do you mean the glossary entries, or the implementations in
>>> particular Forth system?
>>>
>>> In which group does the EXIT word fall?
>>
>> Let's restate the groups:
>> a) normal words,
>> b) NDCS words,
>> c) Immediate words are a subset of NDCS words.
>>
>> In most Forth systems, groups b) and c) are the same because the
>> system only has IMMEDIATE as a way to express NDCS behaviour.
>>
>> In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
>> of COMPILE, for such words is NDCS,
>>
>> : ndcs,        \ ??? xt -- ???
>> \ *G Perform the compilation action of an NDCS word. This may have
>> \ ** a stack effect or parse the input stream.
>>
>> : compile-word    \ i*x xt -- j*x
>> \ *G Process an XT for compilation. May parse or have stack effect.
>>    dup ndcs?                \ NDCS or normal
>>    if  ndcs,  else  compile,  then
>> ;
>>
>> In VFX, most uses of COMPILE, should use COMPILE-WORD instead.
>> The alternative is to redefine COMPILE, to take note of NDCS words
>> but ... politics, politic, politics.
>
>
> Taking into account that "ndcs," just performs another xt:
>
>   : (ndcs,)    \ i*x xt -- j*x
>     >code-len @ execute            \ execute NDCS/IMMEDIATE word
>   ;
>
> Why not return this xt by "find" ?
>
>
>   : find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
>     find dup 0= if exit then drop ( xt1 )
>     dup ndcs? if >code-len @ 1 else
>     dup immediate? if 1 else -1 then then ( xt1 xt2 n )
>     state @ if rot else swap then drop
>   ;

A typo correction:

: find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
find dup 0= if exit then drop ( xt1 ) dup
dup ndcs? if >code-len @ 1 else
dup immediate? if 1 else -1 then then ( xt1 xt2 n )
state @ if rot else swap then drop
;

>
>
> After this fix all my tests work as expected.
>
> To be on the safe side, I would redefine/fix some other words too, to
> exclude situations when "compile," produces unexpected stack effect when
> it's applied to some xt.
>
> Nothing standard word should return such xt for which "compile,"
> produces unexpected stack effect.
>
>
>
> --
> Ruvim

On 2021-05-06 18:09, P Falth wrote:
> On Thursday, 6 May 2021 at 16:07:26 UTC+2, Ruvim wrote:
>> On 2021-05-06 14:38, P Falth wrote:
>>> On Thursday, 6 May 2021 at 10:52:00 UTC+2, dxforth wrote:
>>>> On 6/05/2021 16:19, Ruvim wrote:
>>>>> On 2021-05-06 04:49, dxforth wrote:
>>>>>>
>>>>>> The behaviour of test-if was ambiguous before it got to COMPILE, .
>>>>>
>>>>> Here it is:
>>>>>
>>>>> : test-if
>>>>> c" if" find dup 0= if ." unfound" 2drop exit then
>>>>> 1 = if ." executed" execute else ." compiled" compile, then
>>>>> ; immediate
>>>>>
>>>>>
>>>>> In what place is it ambiguous? In "find"?
>>>> Using FIND to invoke IF is ambiguous. I've a good idea what will
>>>> happen on single-xt systems but that's all. Same for POSTPONE S" .
>>>> Dual-xt systems have muddied the waters. Today users have no way
>>>> to know what is portable - only what works on their system.
>>>
>>> Yes I agree to this. My LXF is dual-xt. FIND on IF will return the interpretation
>>> xt. When executed this will print an error message and abort.
>> It's OK for interpretation state, but not OK for compilation state.
>>
>> In dual-xt systems FIND should depend on STATE and return the different
>> results depending on STATE for some words.
>
> The standard says that FIND may return different XTs depending on state.
> It is not obliged to do so

Yes, certainly. I mean, to make a user-defined text interpreter
implementable via FIND (as in "test-if" above), FIND should depend on
STATE in a dual-xt system.

It should be noted, among dual-xt and single-xt systems, only some
dual-xt systems provide FIND that is not capable to implement
user-defined text interpreter.

>> My clarification for FIND (the next version)
>> https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682
>
> I have seen your efforts.
> In my opinion it is better to start with a clean sheet and implement something
> that can substitute FIND and make it obsolete.

Even when a consensus concerning this replacement will be reached, FIND
will remain a method of fallback in many years.

So FIND should be clarified regardless replacements.

--
Ruvim

stephen@mpeforth.com (Stephen Pelc) writes:
>On Wed, 05 May 2021 11:46:26 GMT, anton@mips.complang.tuwien.ac.at
>(Anton Ertl) wrote:
[...]
>And that's probably what triggered my NDCS esploration. The base
>problem comes from the standard's definitions of words. There are
>three cases:
>1) normal words
>2) immediate words
>3) NDCS words
>
>The standard only says how to handle cases 1) and 2).

Really? When I look at the standard, I find, e.g.

|3.4 The Forth text interpreter
|...
| |b. Search the dictionary name space (see 3.4.2). If a definition name
|matching the string is found:
| | 1. if interpreting, perform the interpretation semantics of the
| definition (see 3.4.3.2), and continue at a).
| | 2. if compiling, perform the compilation semantics of the definition
| (see 3.4.3.3), and continue at a).

So here the standard does not talk about normal words, immediate
words, or NDCS words, but instead about the interpretation semantics
and compilation semantics of all words.

>The standard deliberately does not mandate implementation techniques
>and VFX uses a technique for NDCS words that is useful, but breaks
>careless tests.

The standard indeed does not mandate implementation techniques, but
instead specifies behaviour. Standard systems have to comply with
this specification; they can choose various implementation techniques
to achieve this, but if an implementation does not comply with the
specification, it is non-standard, and you can wave the "does not
mandate implementation techniques" card as much as you want, it won't
change that.

If a test shows that your system is non-standard, it's not the fault
of the test, but the fault of your system.

>The EuroForth papers are there to read, and nobody has
>said that my analysis is wrong.

I told you in <597E1567.25877.99B5533@stephen.mpeforth.com> what was
wrong in the draft version of your 2017 paper. Looking at 4 places
where I pointed out that what you wrote is wrong, and comparing that
to the final paper, I see that in three places you kept the wrong
statements. You ammended the fourth place, but the result is still
wrong.

So: I wrote to you that your analysis is wrong before you published
the paper, and I maintain that it is still wrong.

As for complexity, we used to use an approach that uses IFs for
separating various cases (aliases, interpret/compile:, intelligent
"compile,", words that you can TO to); these exceptions were easy to
add step-by-step, but they resulted in a complex mess (in a way,
similar to a text interpreter with five (or was it seven)
special-purpose hooks, one for each extension that had become
necessary over the years), and they are inflexible. So Bernd Paysan
redesigned the Gforth header (with some input from me), and this
resulted in a much more flexible header design that we described in
detail in [paysan19].

@InProceedings{paysan19,
author = {Bernd Paysan and M. Anton Ertl},
title = {The new {Gforth} Header},
crossref = {euroforth19},
pages = {5--20},
url = {http://www.euroforth.org/ef19/papers/paysan.pdf},
url-slides = {http://www.euroforth.org/ef19/papers/paysan-slides.pdf},
video = {https://wiki.forth-ev.de/doku.php/events:ef2019:header},
OPTnote = {refereed},
abstract = {The new Gforth header is designed to directly
implement the requirements of Forth-94 and
Forth-2012. Every header is an object with a fixed
set of fields (code, parameter, count, name, link)
and methods (\texttt{execute}, \texttt{compile,},
\texttt{(to)}, \texttt{defer@}, \texttt{does},
\texttt{name>interpret}, \texttt{name>compile},
\texttt{name>string}, \texttt{name>link}). The
implementation of each method can be changed
per-word (prototype-based object-oriented
programming). We demonstrate how to use these
features to implement optimization of constants,
\texttt{fvalue}, \texttt{defer}, \texttt{immediate},
\texttt{to} and other dual-semantics words, and
\texttt{synonym}.}
}

>My conclusion from all this is that the standard has a bug in it.
>Unfortunately, fixing the bug will be a real pain.

I think the standard is mostly fine in this area. There are proposals
for specifying FIND in a way that would make user-defined text
interpreters standard
<https://forth-standard.org/proposals/clarify-find?hideDiff#reply-165>
<https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682>
and for making the definition of the term "execution token" match it's
usage in FIND, ', ['], NAME>INTERPRET, NAME>COMPILE
<https://forth-standard.org/proposals/reword-the-term-execution-token-?hideDiff#reply-486>.

Fixing these issues is therefore as painless as accepting one of the
Clarify FIND proposals, and accepting the rewording of execution
token.

There is also a case for disallowing ticking and POSTPONEing S" and
S\" (to allow STATE-smart implementations of these words), but I leave
it to those who are interested in such implementations to make such
proposals.

>> I had the
>>impression that you would fix it in VFX 5, but given your reaction,
>>this is now questionable; I work at a university, so I cannot test VFX
>>5 myself.
>
>It is fixed.

Good! I have now tested the test case I sent you in 2015 and VFX 5.11 passed.

Let's see about Ruvim's test from <s6qmbs$ki6$1@dont-email.me>:

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

] test-if [ .s

This fails on VFX 5.11. My guess is that you did not fix your broken
COMPILE, but instead rewrote your text interpreter such that my 2015
test does not invoke COMPILE,. So let's combine my scaffolded
COMPILE, from the 2015 test with Ruvim's test:

: defers
\ "defers X" produces a call to the current content of the deferred word X
' defer@ compile, ; immediate

:noname ( xt -- )
>in @ >r
depth 1- >r
defers compile,
depth r> <> abort" compile, stack effect wrong"
>in @ r> <> abort" compile, input stream effect wrong" ;
is compile,

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

] test-if [ .s

The last line outputs:

|compiledcompile, stack effect wrong

(why would it say "compiled" and not execute the .s after the [ ?)

Let's try a simple user-defined text interpreter based on the one
shown in
<https://forth-standard.org/standard/rationale#rat:core:COMPILE,>

: int-line ( ... "rest of line" -- ... )
cr source type cr
begin
bl word dup c@ while
FIND ?DUP IF ( xt +-1 )
STATE @ IF ( xt +-1 )
0> IF EXECUTE ELSE COMPILE, THEN ( ??? )
ELSE ( xt +-1 )
DROP EXECUTE ( ??? )
THEN
ELSE ( c-addr )
-13 throw \ numbers are left as an exercise
THEN
repeat
drop ;

[upper case stuff from
<https://forth-standard.org/standard/rationale#rat:core:COMPILE,>]

Note that Forth-94/2012 does not guarantee that this text-interprets,
e.g., IF (not even in compile state), because FINDing IF is ambiguous.
Anyway, let's see how various systems fare, on the following input:

int-line state @ .
int-line : foo if dup . then ;
int-line : bar s" bla" ;
int-line state @ true foo drop bar type

The outputs are, for various Forth systems:

SwiftForth 3.11.0, Gforth 0.7.9_20201015, iforth, and vfxlin-4.71 print:
int-line state @ .
0
int-line : foo if dup . then ;

int-line : bar s" bla" ;

int-line state @ true foo bar type
0 bla

vfx64 5.11 prints:
int-line state @ .
0
int-line : foo if dup . then ;

Err# -22 ERR: Control structure mismatch.
Source: "forth/text-interpreter.4th" on line 36
-> int-line : foo if dup . then ;
^sh: geany: command not found

ERROR on command line

The VFX 5.11 result was unexpected. VFX 4.71 works here because this
text interpreter does not keep any text-interpreter data on the data
stack during the call to COMPILE, (except of course the xt itself).
But let's change the user-defined text interpreter in a way that
should not change it's behaviour:

On 2021-05-08 13:34, Anton Ertl wrote:
> stephen@mpeforth.com (Stephen Pelc) writes:
>> On Wed, 05 May 2021 11:46:26 GMT, anton@mips.complang.tuwien.ac.at
>> (Anton Ertl) wrote:
> [...]
>> And that's probably what triggered my NDCS esploration. The base
>> problem comes from the standard's definitions of words. There are
>> three cases:
>> 1) normal words
>> 2) immediate words
>> 3) NDCS words
>>
>> The standard only says how to handle cases 1) and 2).
>
> Really? When I look at the standard, I find, e.g.
>
> |3.4 The Forth text interpreter
> |...
> |
> |b. Search the dictionary name space (see 3.4.2). If a definition name
> |matching the string is found:
> |
> | 1. if interpreting, perform the interpretation semantics of the
> | definition (see 3.4.3.2), and continue at a).
> |
> | 2. if compiling, perform the compilation semantics of the definition
> | (see 3.4.3.3), and continue at a).
>
> So here the standard does not talk about normal words, immediate
> words, or NDCS words, but instead about the interpretation semantics
> and compilation semantics of all words.

As I understand, the Stephen's point was that the standard describes how
to perform default compilation semantics (that's to append ES), and how
to perform compilation semantics for immediate words (that's to perform
ES), but doesn't describe how to perform compilation semantics for other
words.

Actually, it's a fictional problem. The standard just considers and
describes behavior of a system on some level of abstraction, and doesn't
consider by what means this behavior is accomplished, and how the system
behaves on the underlying levels.

I.g., it also doesn't say how compilation semantics are appended to the
current definition.

If the standard doesn't say how to accomplish some results or behavior,
then it's up to an implementer how to to it, and the implementer is free
to use any suitable means.

--
Ruvim

On Saturday, May 8, 2021 at 3:10:49 PM UTC+2, Anton Ertl wrote:
[..]
> So let's combine my scaffolded
> COMPILE, from the 2015 test with Ruvim's test:

I have to dig through a lot of sneaky text (test of a test in 2015?)
to even approach what you have done here.
Maybe just insert it explicitly?

> : defers
> \ "defers X" produces a call to the current content of the deferred word X
> ' defer@ compile, ; immediate

I am not fond of these discussions but must point out that DEFER@
only works for words created by DEFER , a word in the CORE-EXT set.
It is therefore unlikely that DEFER@ is going to work on non-user-defined
words like COMPILE,. Or is there a guarantee now that existence
of DEFER means that all deferred words are created by DEFER ?

> :noname ( xt -- )
> >in @ >r
> depth 1- >r
> defers compile,

Here you assume COMPILE, is a deferred word. It would really help
to show the complete example.

> depth r> <> abort" compile, stack effect wrong"
> >in @ r> <> abort" compile, input stream effect wrong" ;
> is compile,

Again assuming COMPILE, is DEFERred, consisted with the above.

I couldn't (but wanted to) read past the above line.

-marcel

On 2021-05-08 22:17, Ruvim wrote:
> On 2021-05-08 13:34, Anton Ertl wrote:
>> stephen@mpeforth.com (Stephen Pelc) writes:
>>> On Wed, 05 May 2021 11:46:26 GMT, anton@mips.complang.tuwien.ac.at
>>> (Anton Ertl) wrote:
>> [...]
>>> And that's probably what triggered my NDCS esploration. The base
>>> problem comes from the standard's definitions of words. There are
>>> three cases:
>>> 1) normal words
>>> 2) immediate words
>>> 3) NDCS words
>>>
>>> The standard only says how to handle cases 1) and 2).
>>
>> Really?  When I look at the standard, I find, e.g.
>>
>> |3.4 The Forth text interpreter
>> |...
>> |
>> |b. Search the dictionary name space (see 3.4.2). If a definition name
>> |matching the string is found:
>> |
>> |  1. if interpreting, perform the interpretation semantics of the
>> |     definition (see 3.4.3.2), and continue at a).
>> |
>> |  2. if compiling, perform the compilation semantics of the definition
>> |     (see 3.4.3.3), and continue at a).
>>
>> So here the standard does not talk about normal words, immediate
>> words, or NDCS words, but instead about the interpretation semantics
>> and compilation semantics of all words.
>
>
> As I understand, the Stephen's point was that the standard describes how
> to perform default compilation semantics (that's to append ES), and how
> to perform compilation semantics for immediate words (that's to perform
> ES), but doesn't describe how to perform compilation semantics for other
> words.

Nevertheless, we should admit that the standard better describes
single-xt systems than dual-nt systems (or dual-xt systems) in this regard.

A single-xt system doesn't provide "other words" at all, so a
program/user knows how to perform CS for any word. But due to FIND is
insufficiently specified, a user formally doesn't know how to perform CS
for other words in a dual-nt system (or dual-xt system). Certainly, the
system knows that, but a user doesn't know.

One sound expectation is that a user-defined text interpreter from a
single-xt system should work without changes in a dual-nt system. And
from this assumption a user can conclude how to perform CS for other
words in a dual-nt system (but conclusions concerning immediacy, if any,
will be wrong).

A problem is that this expectation is based not on the normative part,
but on the rationales. The purpose of clarification for FIND is to make
the normative part a formal basis for this expectation.

> Actually, it's a fictional problem. The standard just considers and
> describes behavior of a system on some level of abstraction, and doesn't
> consider by what means this behavior is accomplished, and how the system
> behaves on the underlying levels.
>
> I.g., it also doesn't say how compilation semantics are appended to the
> current definition.
>
> If the standard doesn't say how to accomplish some results or behavior,
> then it's up to an implementer how to to it, and the implementer is free
> to use any suitable means.

--
Ruvim

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2021-05-08 13:34, Anton Ertl wrote:
>> stephen@mpeforth.com (Stephen Pelc) writes:
>>> On Wed, 05 May 2021 11:46:26 GMT, anton@mips.complang.tuwien.ac.at
>>> (Anton Ertl) wrote:
>> [...]
>>> And that's probably what triggered my NDCS esploration. The base
>>> problem comes from the standard's definitions of words. There are
>>> three cases:
>>> 1) normal words
>>> 2) immediate words
>>> 3) NDCS words
>>>
>>> The standard only says how to handle cases 1) and 2).
>>
>> Really? When I look at the standard, I find, e.g.
>>
>> |3.4 The Forth text interpreter
>> |...
>> |
>> |b. Search the dictionary name space (see 3.4.2). If a definition name
>> |matching the string is found:
>> |
>> | 1. if interpreting, perform the interpretation semantics of the
>> | definition (see 3.4.3.2), and continue at a).
>> |
>> | 2. if compiling, perform the compilation semantics of the definition
>> | (see 3.4.3.3), and continue at a).
>>
>> So here the standard does not talk about normal words, immediate
>> words, or NDCS words, but instead about the interpretation semantics
>> and compilation semantics of all words.
>
>
>As I understand, the Stephen's point was that the standard describes how
>to perform default compilation semantics (that's to append ES), and how
>to perform compilation semantics for immediate words (that's to perform
>ES), but doesn't describe how to perform compilation semantics for other
>words.

I don't know if this was his point, but anyway, the standard certainly
specifies the compilation semantics of all words:

|3.4.3.3 Compilation semantics
| |Unless otherwise specified in a "Compilation:" section of the glossary
|entry, the compilation semantics of a Forth definition shall be to
|append its execution semantics to the execution semantics of the
|current definition.

So either a word specification has a "Compilation:" section, and that
gives its compilation semantics (then this is an "other word"), or its
compilation semantics is the default compilation semantics specified
in the last half-sentence of 3.4.3.3 (for both standard-specified and
user-defined words), or (for user-defined immediate words) its
compilation semantics is the execution semantics as per 2.1:

|immediate word:
| |A Forth word whose compilation semantics are to perform its execution
|semantics.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

dxforth <dxforth@gmail.com> writes:
>I get certain
>advantages continuing to use a POSTPONE that's based on COMPILE and
>thus far none using POSTPONE in immediate mode - which has always
>been counter-intuitive.

The way you use COMPILE in POSTPONE is compatible with POSTPONEing in
interpret state:

: POSTPONE
?comp bl word find ?dup if
0< if compile compile then
else number then compile, ; immediate

You do not rely on the text interpreter to "," the xt of the postponed
word; instead POSTPONE ","s the xt with COMPILE,. Of course, your
STATE-smart NUMBER remains a problem, but
<2021Apr29.080904@mips.complang.tuwien.ac.at> showed how to fix that.

Your use of COMPILE, raises the question of whether this will work in,
say, a native-code system. It will not, but if we assume that the
system puts code and data in the same section, a slight variation
could work; I'll start with the variant from
<2021Apr29.080904@mips.complang.tuwien.ac.at>:

: POSTPONE
bl word find ?dup if
0< if compile compile then
else number dup >r execute r> then compile, ; immediate

First, we have to define COMPILE and we define it in the almost
classic way:

: compile ( -- ) r> dup @ compile, cell+ >r ;

[inlining must be prevented for this word]

Note that this COMPILE expects the xt (not native code) of the word to
be compiled in the next cell, but then compiles native code with
COMPILE,. So instead of COMPILE COMPILE we have to write something
different:

: POSTPONE
bl word find ?dup if
0< if compile [ ' compile , ] , exit then
else number dup >r execute r> then compile, ; immediate

So in the COMPILE case we have to use ",", while in the other cases we
have to use COMPILE,.

The version above tries to stay close to the COMPILE COMPILE idea, but
it's simpler to write it as:

: POSTPONE
bl word find ?dup if
0< if [ ' compile compile, ] , exit then
else number dup >r execute r> then compile, ; immediate

>> Of course they can. Moreover, it's possible to redefine POSTPONE to
>> handle literals.
>
>Not easily I presume?

If we assume a NUMBER like as used above, and sticking close to your
retro style (because your NUMBER is designed for it):

: postpone ( "name" -- )
>in @ bl word find if
drop >in ! postpone postpone exit then
nip number dup >r execute r> compile, ;

Alternatively, if you have recognizers:

: POSTPONE ( "name" -- )
PARSE-NAME FORTH-RECOGNIZER RECOGNIZE DUP >R
RECTYPE>POST EXECUTE R> RECTYPE>COMP COMPILE, ;

[source: <https://forth-standard.org/proposals/recognizer>]

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

On 10/05/2021 01:21, Anton Ertl wrote:
> dxforth <dxforth@gmail.com> writes:
>>I get certain
>>advantages continuing to use a POSTPONE that's based on COMPILE and
>>thus far none using POSTPONE in immediate mode - which has always
>>been counter-intuitive.
>
> The way you use COMPILE in POSTPONE is compatible with POSTPONEing in
> interpret state:
>
> : POSTPONE
> ?comp bl word find ?dup if
> 0< if compile compile then
> else number then compile, ; immediate
>
> You do not rely on the text interpreter to "," the xt of the postponed
> word; instead POSTPONE ","s the xt with COMPILE,. Of course, your
> STATE-smart NUMBER remains a problem, but
> <2021Apr29.080904@mips.complang.tuwien.ac.at> showed how to fix that.

I've since shifted the state-smartness from NUMBER to INTERPRET where
it belongs. What hasn't changed is the classic behaviour that executing
COMPILE when STATE=0 will crash the interpreter due to IP having been
advanced. COMPILE isn't in ANS for the reason they state, however it
remains useful for threaded-code systems where memory is a premium.

On Sat, 8 May 2021 22:17:52 +0300, Ruvim <ruvim.pinka@gmail.com>
wrote:

>As I understand, the Stephen's point was that the standard describes how
>to perform default compilation semantics (that's to append ES), and how
>to perform compilation semantics for immediate words (that's to perform
>ES), but doesn't describe how to perform compilation semantics for other
>words.

Thanks.

>Actually, it's a fictional problem. The standard just considers and
>describes behavior of a system on some level of abstraction, and doesn't
>consider by what means this behavior is accomplished, and how the system
>behaves on the underlying levels.

But it's not a fictional problem in the presence of testing.
If we do not have standard ways (words) to handle NDCS words, we
cannot have standard tests. Now that VFX Forth handles IF and other
words as NDCS words, we need to add NDCS handling words to the
standard.

>I.g., it also doesn't say how compilation semantics are appended to the
>current definition.
>
>If the standard doesn't say how to accomplish some results or behavior,
>then it's up to an implementer how to to it, and the implementer is free
>to use any suitable means.

Yes. Now the Forth words in the standard need to catch up with the
text. As the phrase warns us "Be careful what you wish for", we first
need to decide what we want in the standard.

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

On 2021-05-10 16:58, Stephen Pelc wrote:
> On Sat, 8 May 2021 22:17:52 +0300, Ruvim <ruvim.pinka@gmail.com>
> wrote:
>
>> As I understand, the Stephen's point was that the standard describes how
>> to perform default compilation semantics (that's to append ES), and how
>> to perform compilation semantics for immediate words (that's to perform
>> ES), but doesn't describe how to perform compilation semantics for other
>> words.
>
> Thanks.
>
>> Actually, it's a fictional problem. The standard just considers and
>> describes behavior of a system on some level of abstraction, and doesn't
>> consider by what means this behavior is accomplished, and how the system
>> behaves on the underlying levels.
>
> But it's not a fictional problem in the presence of testing.

True. So, it isn't a problem for system, but it's a problem for a program.

Actually, a program has some indirect methods:

- perform EVALUATE in compilation state (it's unhygienic, it's
difficult to use for some dual-semantics parsing words, and cannot be
used for other).

- create a wrapper by means of POSTPONE and perform these wrapper in
compilation state (cannot be used during compilation).

But I'm convinced that FIND with EXECUTE should be capable to perform
compilation semantics for any NDCS word (NDCS by implementation),
regardless is it an immediate word or not.

This point is not specified in the normative parts of the standard at
the moment, but the rationales show that it was intended, without any doubt.

And I don't see any reason why this point should not be included in the
specification for FIND. (see my proposal re FIND clarification).

> If we do not have standard ways (words) to handle NDCS words, we
> cannot have standard tests.

The tests should not depend on how NDCS words are implemented, and
whether they are implemented via mechanism of immediacy, or not.

> Now that VFX Forth handles IF and other
> words as NDCS words, we need to add NDCS handling words to the
> standard.

I shown how FIND can be implemented in VFX without any additional words
for handling NDCS words.
See "Fix VFX FIND" at news:s73ea4$nt9$1@dont-email.me

Why not?

>> I.g., it also doesn't say how compilation semantics are appended to the
>> current definition.
>>
>> If the standard doesn't say how to accomplish some results or behavior,
>> then it's up to an implementer how to to it, and the implementer is free
>> to use any suitable means.
>
> Yes. Now the Forth words in the standard need to catch up with the
> text. As the phrase warns us "Be careful what you wish for", we first
> need to decide what we want in the standard.
>

--
Ruvim

dxforth <dxforth@gmail.com> writes:
>I've since shifted the state-smartness from NUMBER to INTERPRET where
>it belongs.

Good.

>What hasn't changed is the classic behaviour that executing
>COMPILE when STATE=0 will crash the interpreter due to IP having been
>advanced.

This has nothing to do with STATE; you identify the dependency
correctly: IP. Here's an example how COMPILE can run when STATE=0:

\ First, define COMPILE, and show STATE when it runs:
: compile state ? r> dup cell+ >r @ , ;

: foo compile . ;
: bar [ foo ] ; \ prints "0" (output of COMPILE)
1 bar \ prints "1"

Tested with gforth-itc 0.7.9_20210415. As you can see, running
COMPILE when STATE=0 works.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

On 2021-05-09 10:35, Marcel Hendrix wrote:
> On Saturday, May 8, 2021 at 3:10:49 PM UTC+2, Anton Ertl wrote:
> [..]
>> So let's combine my scaffolded
>> COMPILE, from the 2015 test with Ruvim's test:
>
> I have to dig through a lot of sneaky text (test of a test in 2015?)
> to even approach what you have done here.
> Maybe just insert it explicitly?
>
>> : defers
>> \ "defers X" produces a call to the current content of the deferred word X
>> ' defer@ compile, ; immediate
>
> I am not fond of these discussions but must point out that DEFER@
> only works for words created by DEFER , a word in the CORE-EXT set.
> It is therefore unlikely that DEFER@ is going to work on non-user-defined
> words like COMPILE,.

True.

These tests are not standard, they use carnal knowledge of the
particular Forth systems. And these tests were created to test these
particular systems.

--
Ruvim

On 2021-05-06 18:23, P Falth wrote:
> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
>> On 2021-05-06 17:07, Ruvim wrote:
>>> On 2021-05-06 14:38, P Falth wrote:
>> [...]
>>>> As an alternative to FIND I can suggest
>>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
>> They are quite close to FIND-NAME NAME>COMPILE
>
> Yes they are. But they also hide the NT and can thus accomodate
> also a system with dual NTs. Someone (Mark Williams?) described
> such a system that was simple and clean.

But why they are better than "FIND"? (except (c-addr u) string).

In my recent iteration I introduced the word FIND-SEM

FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )

It is closer to classic "FIND", with only difference that it accepts a
string as the pair (c-addr u), and returns more convenient flag-special,
rather than -1|1 code.

And why do you need "FIND-POSTPONE"?

>>> If you can implement this, nothing prevent you to implement STATE
>>> dependent FIND
>> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
>> either "EXECUTE" or "COMPILE," and nothing else.
>
> To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
> limit the possibilities of the system. Take for example a constant.
> XT2 could be the actual value of the constant and XT1 LITERAL for compiling.

It looks like you reinvent recognizers under the different name.

> My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
> to a JIT compiler that compiles the tokens to native code and then executes it

Why cannot EXECUTE be who calls a JIT compiler (if a better code is not
generated yet) and then executes this better code?

>> Otherwise FIND cannot be implemented via these words.
>
> There is no requirements to implement FIND with these words.

For back-compatibility, the system should still provide FIND for a
while. And it should be clear how to implement FIND via new methods in
the case when a system doesn't provide FIND (to use programs/libraries
that rely on FIND).

A one shortcoming of NAME>* methods is that they don't guarantee that
FIND can be implementable via them. Moreover, FIND is not implementable
via these methods in a single-xt system, if this system allows to
perform compilation semantics in interpretation state.

>>>
>>> But what for do you need FIND-POSTPONE ?
>>>
>>>
>>>> The result if found just needs to be executed to perform the
>>>> respective actions
>>>> It also hides the implementation details ( single-xt, dual-xt, dual
>>>> headers etc)
>>>
>>> It doesn't cover sing-xt systems. In these systems the corresponding
>>> action should be performed only in the corresponding STATE.
>> I missed that they return two xt-s. Then yes, they cover single-xt
>> systems too.
>>
>> But then what is a way to detect whether a word is an ordinary word?
>
> That is not needed!

I need that.

See for exampe: https://forth-standard.org/standard/tools#reply-445

Another example. A kind of portable SYNONYM should be able to detected
whether a word is an ordinary word, or not.

>> And what is a way to get xt of an ordinary word, that identifies the
>> execution semantics for this word?
>
> ' and ['] are still available for this.

They cannot be applied to any word. They cannot be used to words of
unknown kind.

--
Ruvim

On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
> On 2021-05-06 18:23, P Falth wrote:
> > On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> >> On 2021-05-06 17:07, Ruvim wrote:
> >>> On 2021-05-06 14:38, P Falth wrote:
> >> [...]
> >>>> As an alternative to FIND I can suggest
> >>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> >>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >> They are quite close to FIND-NAME NAME>COMPILE
> >
> > Yes they are. But they also hide the NT and can thus accomodate
> > also a system with dual NTs. Someone (Mark Williams?) described
> > such a system that was simple and clean.
> But why they are better than "FIND"? (except (c-addr u) string).

When I developed the code generator for my forth (about 20 years ago)
I needed a way to execute the generator for a word when it was compiled
into a definition. I started with the CMFORTH style dual headers in different
wordlists. This worked fine until I also needed wordlists for its intended
purpose. That is when I added a compilation xt to the header. For some
words ( normal words, created words) at compiletime was also needed
the interpretation xt. FIND could not give me this. From this was born
FIND-INTERPRET and FIND-COMPILE. They do what I needed!
> In my recent iteration I introduced the word FIND-SEM
>
> FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )
>
> It is closer to classic "FIND", with only difference that it accepts a
> string as the pair (c-addr u), and returns more convenient flag-special,
> rather than -1|1 code.

But that would not let me find both xts at the same time

> And why do you need "FIND-POSTPONE"?

I do not at the moment and it is not implemented. But might be
useful for recognizers. It might serve to do words with non default
postpone semantics (NDPS words!)

That was a joke!

> >>> If you can implement this, nothing prevent you to implement STATE
> >>> dependent FIND
> >> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
> >> either "EXECUTE" or "COMPILE," and nothing else.
> >
> > To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
> > limit the possibilities of the system. Take for example a constant.
> > XT2 could be the actual value of the constant and XT1 LITERAL for compiling.
> It looks like you reinvent recognizers under the different name.

Yes, just 20 years ago!

> > My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
> > to a JIT compiler that compiles the tokens to native code and then executes it
> Why cannot EXECUTE be who calls a JIT compiler (if a better code is not
> generated yet) and then executes this better code?

Yes that is how it can work. ON the stack will be

NT JIT-compiler, coming from find-interpret

I am not there yet. Now I can call native code from the token code. But I also want to
do the reverse.

> >> Otherwise FIND cannot be implemented via these words.
> >
> > There is no requirements to implement FIND with these words.
> For back-compatibility, the system should still provide FIND for a
> while. And it should be clear how to implement FIND via new methods in
> the case when a system doesn't provide FIND (to use programs/libraries
> that rely on FIND).

I do provide a FIND in my systems. In lxf it is not possible to do a user defined
interpreter with it, for 2 reasons. It is not state-smart. It only returns the
interpretation XT. Words like IF with no defined interpretation semantics
return an xt that print an error message and aborts. If needed i could change this
but so far has not needed it.
> A one shortcoming of NAME>* methods is that they don't guarantee that
> FIND can be implementable via them. Moreover, FIND is not implementable
> via these methods in a single-xt system, if this system allows to
> perform compilation semantics in interpretation state.

My lxf64 is still single-xt, FIND works just fine there. There is no requirement
that find must be implemented by NAME>* methods

> >>>
> >>> But what for do you need FIND-POSTPONE ?
> >>>
> >>>
> >>>> The result if found just needs to be executed to perform the
> >>>> respective actions
> >>>> It also hides the implementation details ( single-xt, dual-xt, dual
> >>>> headers etc)
> >>>
> >>> It doesn't cover sing-xt systems. In these systems the corresponding
> >>> action should be performed only in the corresponding STATE.
> >> I missed that they return two xt-s. Then yes, they cover single-xt
> >> systems too.
> >>
> >> But then what is a way to detect whether a word is an ordinary word?
> >
> > That is not needed!
> I need that.
>
> See for exampe: https://forth-standard.org/standard/tools#reply-445
>
> Another example. A kind of portable SYNONYM should be able to detected
> whether a word is an ordinary word, or not.

I provide a completely compliant SYNONYM. Different NTs but same XTs
There is no need to load another synonym.

> >> And what is a way to get xt of an ordinary word, that identifies the
> >> execution semantics for this word?
> >
> > ' and ['] are still available for this.
> They cannot be applied to any word. They cannot be used to words of
> unknown kind.

I do not understand this. Any word that has an XT could be ticked, or?

BR
Peter

> --
> Ruvim