On Tue, 11 May 2021 20:21:57 +0300, Ruvim <ruvim.pinka@gmail.com>
wrote:

>> Personally, and for MPE, I love the original idea of an xt that it
>> be the single point of contact between the user and the word. An
>> xt can be used for many purposes other than compilation and
>> interpretation, e.g. LOCATE XREF and so on.
>
>I don't see how can a user use an xt with LOCATE or XREF

: locate ' (locate) ;

>> Use of xt in this way requires that all words have a primary or
>> interpretation action, which the xt represents.
>>
>> Your fix breaks this and I suppose makes it non-standard.
>
>How does my fix break this?

It does not distinguish between NDCS and IMMEDIATE. It just
ignores the problem.

>> Regardless of nt or xt, I see no way of dealing with NDCS words
>> without upgrading the standard.
>
>Could you please clarify, what cannot a standard program do without
>special words to handle NDCS words?

A standard system cannot provide facilities for NDCS words
without introducing state-smart words. NDCS words allows us
to provide standards compliant code for things like interpretive
DO ... LOOP and so on.

The great strength of NDCS words is that they permit easy
implementation of notations that are at present difficult.

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

P Falth <peter.m.falth@gmail.com> writes:
>There is another problem waiting for a user defined text interpreter and th=
>at
>is locals. They will not be found by FIND ( or find-name, search-wordlist e=
>tc)
>and they do not have an xt. They are stored in a specific table that is set=
>up
>and destroyed at : and ; =20

We have discussed this in
<https://forth-standard.org/standard/locals#contribution-137>; one
thing I found was:

|In VFX and Gforth, FIND finds [a local]; in SwiftForth, iForth, and
|lxf, FIND does not.
| |Apparently this has not been an issue for 26 years, so apparently
|nobody has used a user-defined text interpreter to process a program
|using locals on SwiftForth, iForth, or lxf

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

On Wednesday, 12 May 2021 at 18:53:24 UTC+2, Anton Ertl wrote:
> P Falth <peter....@gmail.com> writes:
> >There is another problem waiting for a user defined text interpreter and th=
> >at
> >is locals. They will not be found by FIND ( or find-name, search-wordlist e=
> >tc)
> >and they do not have an xt. They are stored in a specific table that is set=
> >up
> >and destroyed at : and ; =20
>
> We have discussed this in
> <https://forth-standard.org/standard/locals#contribution-137>; one
> thing I found was:
>
> |In VFX and Gforth, FIND finds [a local]; in SwiftForth, iForth, and
> |lxf, FIND does not.
> |
> |Apparently this has not been an issue for 26 years, so apparently
> |nobody has used a user-defined text interpreter to process a program
> |using locals on SwiftForth, iForth, or lxf
> - anton

Yes it is obvious that nobody has done this!

Also looking at the 2012 standard section 13.3.3.2 Syntax restrictions
I see point e) with the following text

"Words that return execution tokens, such as ’ (tick), [’], or FIND,
shall not be used with local names;"

BR
Peter

> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2020: https://euro.theforth.net/2020

On Tuesday, May 11, 2021 at 4:50:48 PM UTC+1, Stephen Pelc wrote:

> The fix is the sensible one, which is to throw away IMMEDIATE and
> let FIND indicate NDCS and normal. However, on some systems this
> will be a code breaker.

I tend to think that the obvious fix is at the other end!

All implementations of NDCS have the same sort of logic:
- find an xt for a name
- if immediate, execute it
- if NDCS execute its compilation semantics
- if neither, append its execution semantics.

Exactly how this logic is distributed between FIND, the text interpreter,
COMPILE, and possibly NAME>COMPILE is implementation dependent. It depends
at least on what words can be found, on the dictionary structure and on
what characteristics of a word (such as immediacy) you can derive from the
execution token.

And that's fine by me. One of the things I most like about the standard is
that it is forgiving of different implementations.

When it comes to implementing NDCS words, though, it does mean that it
is very unlikely, maybe actually impossible, to define "NDCS," or
whetever we call it in a way that will satisfy all the different flavours
of Forth.

That means we will have to rely on defining the standard in such a way
that different implementations of NDCS words can work, leaving the
underlying details to the implementation.

FIND is already defined very permissively (to the extent that I have
real trouble sometimes understanding some of the discussions about it)
and so far as I can tell is adequate to the job, even if those of us with
single-xt systems find it a bit bizarre.

COMPILE, on the other hand is very prescriptive and unforgiving. I
would propose changing the specification of COMPILE, to "perform the
compilation semantics of xt" and leave it to implementers to decide
what xts it can safely be fed, which will vary depending on the
type of Forth that we have.

I'd expect, for example, that a VFX-type Forth would have NDCS, as
a component of COMPILE, and POSTPONE, and that a Gforth-type Forth
would have something similar as a component of FIND and POSTPONE
and as a return value of NAME>COMPILE.

(It is completely possible that everything I have said is rubbish,
as I am pretty new to Forth and have only been doing it for about
two years.)

> Personally, and for MPE, I love the original idea of an xt that it
> be the single point of contact between the user and the word. An
> xt can be used for many purposes other than compilation and
> interpretation, e.g. LOCATE XREF and so on.

Indeed. I find it really hard to get my head around any other
approach, particularly as the xt is at the root of my HELP
system and it would be downright silly to not allow a user to
get help on something anyplace anytime.
> Use of xt in this way requires that all words have a primary or
> interpretation action, which the xt represents. Your fix breaks
> this and I suppose makes it non-standard.

It does mean that all words have a primary action, it doesn't
necessarily follow that the primary action is the interpretation
action (for example, control-flow words). (I think!)

On Wednesday, 12 May 2021 at 18:53:24 UTC+2, Anton Ertl wrote:
> P Falth <peter....@gmail.com> writes:
> >There is another problem waiting for a user defined text interpreter and th=
> >at
> >is locals. They will not be found by FIND ( or find-name, search-wordlist e=
> >tc)
> >and they do not have an xt. They are stored in a specific table that is set=
> >up
> >and destroyed at : and ; =20
>
> We have discussed this in
> <https://forth-standard.org/standard/locals#contribution-137>; one
> thing I found was:

The test you use in the discussion:

: x c" fluffystunk" ; : foo locals| fluffystunk | [ x find cr .s 2drop ] ;

Will fail on lxf for 2 reasons
- FIND will not search locals as they are not in a wordlist
- Locals can anyway only be found when state is compiling in lxf

Peter
> |In VFX and Gforth, FIND finds [a local]; in SwiftForth, iForth, and
> |lxf, FIND does not.
> |
> |Apparently this has not been an issue for 26 years, so apparently
> |nobody has used a user-defined text interpreter to process a program
> |using locals on SwiftForth, iForth, or lxf
> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2020: https://euro.theforth.net/2020

On 13/05/2021 00:27, Stephen Pelc wrote:
> On Thu, 13 May 2021 00:08:37 +1000, dxforth <dxforth@gmail.com> wrote:
>
>>Probably those within Standard Forth didn't need it either, because if
>>FIND wasn't fit for purpose it would surely have been fixed by now.
>
> Forth has changed since the 1990s and flwas in the 1990s documents
> are being revealed.

What ANS reveals is Forth's Curse - willingness to chase the latest
and greatest idea at the expense of a stable environment in which
application development can take root.

On 2021-05-12 17:08, dxforth wrote:
> On 11/05/2021 20:49, Ruvim wrote:
>> On 2021-05-11 07:42, dxforth wrote:
>>> On 10/05/2021 23:58, Stephen Pelc wrote:
>> [...]
>>>> If we do not have standard ways (words) to handle NDCS words, we
>>>> cannot have standard tests. Now that VFX Forth handles IF and other
>>>> words as NDCS words, we need to add NDCS handling words to the
>>>> standard.
>>>
>>> Standard tests for what purpose?  First one must define the
>>> standard need i.e. why does FIND used in situation X need to
>>> produce the same result on every system?  ANS' premise in
>>> permitting implementers to do as they pleased was that not
>>> every situation required standardizing.
>>
>> Yes. But every standard word should be enough specified to be useful.
>>
>>
>>> Were they wrong?
>>
>> The specification for the FIND word is not sufficient.
>>
>> A.6.2.0945 says:
>>     The intention is that COMPILE, can be used as follows
>>     to write the classic interpreter/compiler loop:
>>
>>     ...                                               ( c-addr )
>>     FIND ?DUP IF                                        ( xt +-1 )
>>       STATE @ IF                                        ( xt +-1 )
>>         0> IF EXECUTE ELSE COMPILE, THEN      ( ??? )
>>       ELSE                                        ( xt +-1 )
>>         DROP EXECUTE                        ( ??? )
>>       THEN
>>     ELSE                                              ( c-addr )
>>       ( whatever you do for an undefined word )
>>     THEN
>>     ...
>>
>> Some systems implement FIND in such a way that this loop doesn't work as
>> expected for the words with non default interpretation semantics.
>>
>>> I couldn't find a single app I'd written that used FIND :)
>>
>> Probably you just didn't ever need a user-defined text interpreter for
>> some advanced DSL (domain specific language), or some advanced extension
>> of a Forth system.
>
> Probably those within Standard Forth didn't need it either, because if
> FIND wasn't fit for purpose it would surely have been fixed by now.

Your conclusion could be right if in the most systems FIND was incapable
to implement user-defined text interpreter. But it seems, such systems
comprise the minority.

"FIND" was standardized in Forth-83, and it just always worked as
expected (in all standard Forth systems that provide this word). But
only in 200x (please correct if I'm wrong) some dual-xt systems appeared
that provides FIND that incapable to implement user-defined text
interpreter.

So those who need the full-fledged "FIND" just uses systems which
provide such "FIND".

--
Ruvim

In article <609aa2c8.1456671@news.eternal-september.org>,
Stephen Pelc <stephen@mpeforth.com> wrote:
<SNIP>
>Personally, and for MPE, I love the original idea of an xt that it
>be the single point of contact between the user and the word. An
>xt can be used for many purposes other than compilation and
>interpretation, e.g. LOCATE XREF and so on.

There is the major design flaw. If we look at lisp the central concept
is the symbol and even uncorrelated behaviours are associated with it.
Looking at Forth the central concept is the xt, a behaviour, which is
equally flawed.
Both are wrong in my opinion. The central concept should
be neither an action, nor a name, but more abstract and allowing for
properties such as actions or names or DMCF (whatever that is.)
Such an "object" can remain without name in some cases, or its name
is changed, it can be relinked into an other vocabulary, etc.
These objects could have several behaviour, or one, and some
behaviours could make no sense in some states, which is ok.
Important is that those properties belong together.
All kind of implementation defined properties can hang off such
an object. In tforth (1993) I proposed that only those tokens
could be passed between words, not exposing the internal fields
like in parameter field, data field, code field unnecessarily.
An enormous roadblock for this is the obsession of Forthers with
efficiency and speed.

Words say a lot. Neither execution taken, nor name token are
acceptable names for such objects, and will forever engender
misunderstandings. You know how I call those.
<SNIP>

>Because of SYNONYM and friends, there is an argument that the
>unique token for a word is the name token.
We are getting there. Now coin a proper name for this.

<SNIP>
>Stephen
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On 2021-05-12 15:12, P Falth wrote:
> On Wednesday, 12 May 2021 at 13:42:46 UTC+2, Ruvim wrote:
>> On 2021-05-12 13:58, P Falth wrote:
[...]
>>> I have looked thru all my sources. FIND is never used. in other peoples
>>> sources I have I see it used some time to define [DEFINED] . For
>>> this purpose my state dump find will work just fine.
>>> This also means that I can test a state-smart find without breaking my system!
>>>
>>> But what will I actually gain from it?
>>>
>> As an implementer — nothing direct gain.
>>
>> The users of your system can use some portable libraries and programs
>> that rely on FIND. Or the Forth users in general can choose your system
>> in more cases.
>>
>> For example, at the moment I cannot use your system, and one of the
>> issues is that a user-defined text interpreter doesn't work.
>
> There is another problem waiting for a user defined text interpreter and that
> is locals. They will not be found by FIND ( or find-name, search-wordlist etc)
> and they do not have an xt. They are stored in a specific table that is setup
> and destroyed at : and ;

As Anton pointed out, it was discussed.

My conclusion is that FIND is not obligated to find locals. And I'm not
convinced that FIND is allowed to traverse anything beyond the search order.

So the problem with locals in a user-defined text interpreter is not
about FIND, and this problem can be addressed to the Locals word set.
Also, Recognizer API should provide a recognizer for locals.

As a workaround, locals can be implemented in a portable way, but of
course not so efficient.

I personally barely use locals.

--
Ruvim

On 2021-05-12 17:44, Stephen Pelc wrote:
> On Tue, 11 May 2021 20:21:57 +0300, Ruvim <ruvim.pinka@gmail.com>
> wrote:
>
>>> Personally, and for MPE, I love the original idea of an xt that it
>>> be the single point of contact between the user and the word. An
>>> xt can be used for many purposes other than compilation and
>>> interpretation, e.g. LOCATE XREF and so on.
>>
>> I don't see how can a user use an xt with LOCATE or XREF
>
> : locate ' (locate) ;

It's a system-specific (VFX-specific) use. Changes in VFX FIND don't
affect such use.

Even the following phrase:

' if >code-len @ (locate)

either works as expected in FVX, or doesn't work, regardless of FIND.

>>> Use of xt in this way requires that all words have a primary or
>>> interpretation action, which the xt represents.
>>>
>>> Your fix breaks this and I suppose makes it non-standard.

Well, at the first, my fix doesn't break this property in VFX: all words
in VFX still have a primary action and the corresponding xt.

At the second, the standard doesn't require any word to have
"interpretation action" which the xt represents.

For example, "interpretation action" for "IF" may be to throw "not
found" exception, and system might not provide any xt that represents
this action for "IF" (since FIND returns 0 for that case).

>> How does my fix break this?
>
> It does not distinguish between NDCS and IMMEDIATE. It just
> ignores the problem.

And what? Could you provide some standard-compliant code examples, that
shows this problem? It's not clear what particular problem do you mean.

>>> Regardless of nt or xt, I see no way of dealing with NDCS words
>>> without upgrading the standard.
>>
>> Could you please clarify, what cannot a standard program do without
>> special words to handle NDCS words?
>
> A standard system cannot provide facilities for NDCS words
> without introducing state-smart words. NDCS words allows us
> to provide standards compliant code for things like interpretive
> DO ... LOOP and so on.

I cannot agree. Actually, a standard system is not limited by the
standard word sets, and it may provide any additional facilities.

For a standard program two approaches (at the moment) are available to
implement interpretive DO LOOP:

- rely on immediacy and STATE-dependent definitions;
- implement a user-defined text interpreter.

And the main problem of implementing interpretive DO LOOP doesn't depend
on what of these approaches (or their alternatives) is chosen.

> The great strength of NDCS words is that they permit easy
> implementation of notations that are at present difficult.

Could you please provide an example: the *implementation* via NDCS words
of some particular notation that is at present *difficult*?

Your past example with DO is not suitable:

: DO NoInterp ; ndcs: ( -- ) s_do, 3 ;

| If you want an interpreted version of DO replace NOINTERP
| by the required code.

Since it's *not difficult* at present, and it can be achieved as following:

: do state @ if s_do, 3 exit then NoInterp ;

Or may be:

: dual ( xt-interp xt-compil "name" -- )
2r> : ]] state @ if [[ r> compile, ]] exit then [[
r> compile, ]] ; [[
;

' NoInterp [: s_do, 3 ;] dual do

If you want, replace NoInterp by the required code. But the problem of
interpretive DO LOOP in this "required code". Not in NDCS.

--
Ruvim

On 2021-05-12 22:01, Paul Liles wrote:
[...]

> COMPILE, on the other hand is very prescriptive and unforgiving.

This issue is addressed by the Recognizer API

The most recently discussed variant can be found at:
https://forth-standard.org/proposals/minimalistic-core-api-for-recognizers?hideDiff#reply-515

> I would propose changing the specification of COMPILE, to "perform the
> compilation semantics of xt" and leave it to implementers to decide
> what xts it can safely be fed, which will vary depending on the
> type of Forth that we have.

"compilation semantics of xt" is a nonsense.

Compilation semantics make sense for only named definitions (words).
Ditto interpretation semantics.

Please refer to 2.1 Definitions of terms
https://forth-standard.org/standard/notation#notation:terms

| compilation semantics:
| The behavior of a Forth definition when its name is
| encountered by the text interpreter in compilation state.

| interpretation semantics:
| The behavior of a Forth definition when its name is
| encountered by the text interpreter in interpretation state.

--
Ruvim

On Wed, 12 May 2021 12:01:36 -0700 (PDT), Paul Liles
<pauldliles@btopenworld.com> wrote:

>All implementations of NDCS have the same sort of logic:
>- find an xt for a name
>- if immediate, execute it
>- if NDCS execute its compilation semantics
>- if neither, append its execution semantics.
>
>Exactly how this logic is distributed between FIND, the text interpreter,

>COMPILE, and possibly NAME>COMPILE is implementation dependent. It depends
>at least on what words can be found, on the dictionary structure and on
>what characteristics of a word (such as immediacy) you can derive from the
>execution token.

>And that's fine by me. One of the things I most like about the standard is
>that it is forgiving of different implementations.

And there is no portable way to determine whether a word is immediate,
let alone is an NDCS word.

>When it comes to implementing NDCS words, though, it does mean that it
>is very unlikely, maybe actually impossible, to define "NDCS," or
>whetever we call it in a way that will satisfy all the different flavours
>of Forth.

What's wrong with your prescription above?
"if NDCS execute its compilation semantics"

>That means we will have to rely on defining the standard in such a way
>that different implementations of NDCS words can work, leaving the
>underlying details to the implementation.

I fully expect NDCS, to need carnal knowledge of the underlying
Forth system.

>I'd expect, for example, that a VFX-type Forth would have NDCS, as
>a component of COMPILE, and POSTPONE, and that a Gforth-type Forth
>would have something similar as a component of FIND and POSTPONE
>and as a return value of NAME>COMPILE.

COMPILE, may not parse source text or have additional stack effects.
Thus it cannot be used for control flow words, S" and so on.

Thank you for your comments.

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

On 11/05/2021 20:51, Anton Ertl wrote:
> dxforth <dxforth@gmail.com> writes:
>>On 11/05/2021 02:09, Anton Ertl wrote:
>>> dxforth <dxforth@gmail.com> writes:
>>>>What hasn't changed is the classic behaviour that executing
>>>>COMPILE when STATE=0 will crash the interpreter due to IP having been
>>>>advanced.
>>>
>>> This has nothing to do with STATE; you identify the dependency
>>> correctly: IP. Here's an example how COMPILE can run when STATE=0:
>>>
>>> \ First, define COMPILE, and show STATE when it runs:
>>> : compile state ? r> dup cell+ >r @ , ;
>>>
>>> : foo compile . ;
>>> : bar [ foo ] ; \ prints "0" (output of COMPILE)
>>> 1 bar \ prints "1"
>>>
>>> Tested with gforth-itc 0.7.9_20210415. As you can see, running
>>> COMPILE when STATE=0 works.
>>
>>AFAIK Forth-83 didn't require that example to work. Same for
>>ANS and POSTPONE.
>
> I guess that changing the goalposts is an admission that the statement
> "executing COMPILE when STATE=0 will crash the interpreter due to IP
> having been advanced." is wrong.
>
> Let's consider the new goalposts:
>
> 1) Return-address manipulation is non-standard (I think also in
> Forth-83). But let's assume the system-defined COMPILE rather than
> the COMPILE defined above.
>
> 2) Forth-83
>
> COMPILE is marked with the C attribute in Forth-83. In Section 11.4,
> Forth-83 states:
>
> | C The word may only be used during compilation of a colon
> | definition.
>
> It's not clear what "use" means here. Does it mean the compilation of
> COMPILE during the definition of FOO? Does it mean the running of
> COMPILE during the definition of BAR? Does it mean the result of the
> COMPILE when running BAR in the final line of the example? I guess
> that it's not the latter, because then COMPILE would be pretty
> unusable.

Go to the definitions:

interpretation semantics:
The behavior of a Forth definition when its name is encountered
by the text interpreter in interpretation state.

6.1.2250 STATE
... STATE is true when in compilation state, false otherwise. ...

So when ANS says 'interpretation semantics undefined' in POSTPONE it
meant STATE=0 - irrespective of whether POSTPONE is being executed
in a definition.

This restriction almost certainly comes from 79/83-COMPILE and the
most popular forth of the day Fig-Forth:

SCR # 41
0 ( COMPILE, SMUDGE, HEX, DECIMAL WFR-79APR20 )
1
2 : COMPILE ( COMPILE THE EXECUTION ADDRESS FOLLOWING *)
3 ?COMP R> DUP 2+ >R @ , ;

The ?COMP prevented COMPILE being executed outside a definition which
would otherwise result in a system crash, but also had the side-effect
of preventing execution when STATE=0. If the goalposts have since
moved, it wasn't the standards' doing.

>
> The other unclear thing is what "during compilation of a colon
> definition" means. My take is that FOO is run during the compilation
> of BAR, so the definition of BAR is Forth-83-conforming in my book.
> Fans of ?COMP will probably take a different position, claiming that
> "during compilation" means STATE=-1, and ignoring the "of a colon
> definition" part.
>
> 3) We have discussed POSTPONE repeatedly over the last
> quarter-century, and I have established repeatedly that according to
> the text of Forth-94 and Forth-2012
>
> : foo postpone . ;
> : bar [ foo ] ;
> 1 bar \ prints "1"
>
> conforms to these standards.

Not by my interpretation above.

>
>>Is there a demonstrated need that it should
>>work that would justify amending or clarifying ANS' ruling?
>
> There is no need to amend or clarify. The text of the standard is
> clear. If anyone wants to change it in the direction of disallowing
> to append something to the current definition when STATE=0, they have
> to propose it and should give a good reason for such a change. I
> demonstrated above that basing POSTPONE on COMPILE is not such a
> reason.
>
>>Change in my case would mean removing ?COMP from COMPILE. The
>>former hasn't proven an issue thus far. I'd like to know what
>>I was getting in return for removing compiler security.
>
> You get at least two bytes. And ?COMP is not compiler security.

Sure it is. Removing ?COMP from COMPILE would mean explicitly
inserting it into IF AHEAD UNTIL AGAIN etc which costs more than
two bytes. COMPILE is left unprotected and it's cost me memory.
If that's all I'm being offered, Fig has the better deal.

On Thursday, 13 May 2021 at 11:58:59 UTC+2, Ruvim wrote:
> On 2021-05-12 15:12, P Falth wrote:
> > On Wednesday, 12 May 2021 at 13:42:46 UTC+2, Ruvim wrote:
> >> On 2021-05-12 13:58, P Falth wrote:
> [...]
> >>> I have looked thru all my sources. FIND is never used. in other peoples
> >>> sources I have I see it used some time to define [DEFINED] . For
> >>> this purpose my state dump find will work just fine.
> >>> This also means that I can test a state-smart find without breaking my system!
> >>>
> >>> But what will I actually gain from it?
> >>>
> >> As an implementer — nothing direct gain.
> >>
> >> The users of your system can use some portable libraries and programs
> >> that rely on FIND. Or the Forth users in general can choose your system
> >> in more cases.
> >>
> >> For example, at the moment I cannot use your system, and one of the
> >> issues is that a user-defined text interpreter doesn't work.
> >
> > There is another problem waiting for a user defined text interpreter and that
> > is locals. They will not be found by FIND ( or find-name, search-wordlist etc)
> > and they do not have an xt. They are stored in a specific table that is setup
> > and destroyed at : and ;
> As Anton pointed out, it was discussed.
>
> My conclusion is that FIND is not obligated to find locals. And I'm not
> convinced that FIND is allowed to traverse anything beyond the search order.
>
> So the problem with locals in a user-defined text interpreter is not
> about FIND, and this problem can be addressed to the Locals word set.
> Also, Recognizer API should provide a recognizer for locals.
>
> As a workaround, locals can be implemented in a portable way, but of
> course not so efficient.
>
> I personally barely use locals.

So on this we agree!

Neither I am a great user of locals. But when I implement something
I try to do it in a good way.

What I have also is a local buffer, working like BUFFER: but assigning
memory from the local stack (return stack). This is accessed with
the name and @/!. Very useful when doing interfaces to Windows or
Linux systemcalls. Individual fields can be accessed via a structure

Based on our discussions here I have now modified FIND to be state-smart.
It was not to complicated. Here is the code:

\ state smart version of find for lxf
\ word types in lxf and flag values
\ flag meaning
\ 0 normal user defined word
\ 1 compiler word, IF etc
\ 2 word with a code generator
\ 4 created word, constant, variable etc
\ 8 immediate word
\ 16 macro, source string to be evaluated

\ ht>xtf ( ht -- xtr xtc flag ) \ from the NT (ht in lxf) get xts and flag
\ xtr = xt for intrepretation action
\ xtc = xt for compilation action

\ : findheader ( c-addr u -- c-addr u 0 | ht )

: f-int ( xtr xtc flag -- xt -1|1 ) \ get the xt and flag while state is 0
9 and if nip 1 else drop -1 then ;
: f-comp ( xtr xtc flag -- xt -1|1 ) \ get the xt and flag while state is -1
27 and if nip 1 else drop -1 then ;
: find ( caddr -- caddr 0 | xt f )
count findheader dup 0= if 2drop 1- 0 exit then
ht>xtf
state @ if f-comp exit then
f-int ;

I also changed all compile-only words to be immediate and
allow ticking instead of printing an error message in interpretation state

BR
Peter

> --
> Ruvim

On 2021-05-13 16:07, dxforth wrote:

> Go to the definitions:
>
>  interpretation semantics:
>    The behavior of a Forth definition when its name is encountered
>    by the text interpreter in interpretation state.
>
>  6.1.2250 STATE
>    ... STATE is true when in compilation state, false otherwise. ...
>
> So when ANS says 'interpretation semantics undefined' in POSTPONE it
> meant STATE=0 - irrespective of whether POSTPONE is being executed
> in a definition.

It looks like you miss the part "name is encountered by the text
interpreter"

Interpretation semantics are not about "a name is being execute", but
about "a name is being encountered".

By your argument, the following definition of "within" [1]:

: within ( test low high -- flag ) over - >r - r> u< ;

cannot work in interpretation state:

1 1 5 within . \ prints -1

since interpretation semantics for ">r" are undefined.

But that's wrong (since this "within" should work in interpretation
state). Then your argument is also wrong.

[1] This example at forth-standard.org
https://forth-standard.org/standard/core/WITHIN#contribution-127

--
Ruvim

P Falth <peter.m.falth@gmail.com> writes:
>Based on our discussions here I have now modified FIND to be state-smart.

It will be interesting to test this.

>I also changed all compile-only words to be immediate and
>allow ticking instead of printing an error message in interpretation state

We also made this change in the development version of Gforth several
years ago. You still get a warning when you text-interpret it in
interpret state or when you tick it.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

On Thursday, 13 May 2021 at 18:18:50 UTC+2, Anton Ertl wrote:
> P Falth <peter....@gmail.com> writes:
> >Based on our discussions here I have now modified FIND to be state-smart.
> It will be interesting to test this.

I will zip it up with some comments and send it to you.
Maybe you can put it on your site for distribution as with the old.
I have a mail address to you from about 16 years ago, can that still be valid?

> >I also changed all compile-only words to be immediate and
> >allow ticking instead of printing an error message in interpretation state
> We also made this change in the development version of Gforth several
> years ago. You still get a warning when you text-interpret it in
> interpret state or when you tick it.

Yes I have seen this while testing gforth and found the warnings a bit irritating.
I also didn't like the blue status line in the bottom row. Can it be turned of?
Some times it bleeds to the lines above when it scrolls

BR
Peter

> - anton
> --
> M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
> comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2020: https://euro.theforth.net/2020

P Falth <peter.m.falth@gmail.com> writes:
>On Thursday, 13 May 2021 at 18:18:50 UTC+2, Anton Ertl wrote:
>> P Falth <peter....@gmail.com> writes:
>> >Based on our discussions here I have now modified FIND to be state-smart.
>> It will be interesting to test this.
>
>I will zip it up with some comments and send it to you.
>Maybe you can put it on your site for distribution as with the old.

I'll be happy to.

>I have a mail address to you from about 16 years ago, can that still be valid?

Sure, and the mail address in this posting is also valid.

>> >I also changed all compile-only words to be immediate and
>> >allow ticking instead of printing an error message in interpretation state
>> We also made this change in the development version of Gforth several
>> years ago. You still get a warning when you text-interpret it in
>> interpret state or when you tick it.
>
>Yes I have seen this while testing gforth and found the warnings a bit irritating.

You can turn (all) warnings off with

warnings off

>I also didn't like the blue status line in the bottom row. Can it be turned of?

-status

>Some times it bleeds to the lines above when it scrolls

You could try sending us a bug report with more details.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

On 5/12/21 2:14 AM, Ruvim wrote:
> On 2021-05-11 13:51, Anton Ertl wrote:
> [...]
>> 3) We have discussed POSTPONE repeatedly over the last
>> quarter-century, and I have established repeatedly that according to
>> the text of Forth-94 and Forth-2012
>>
>> : foo postpone . ;
>> : bar [ foo ] ;
>> 1 bar           \ prints "1"
>>
>> conforms to these standards.
>
> According to the only wording in the specification for POSTPONE this
> code conforms the standard, and it should print "1".
> ...

Is there a *standard* way of detecting an incomplete definition when in
interpretation state?

Krishna

On 14/05/2021 02:15, Ruvim wrote:
> On 2021-05-13 16:07, dxforth wrote:
>
>> Go to the definitions:
>>
>>  interpretation semantics:
>>    The behavior of a Forth definition when its name is encountered
>>    by the text interpreter in interpretation state.
>>
>>  6.1.2250 STATE
>>    ... STATE is true when in compilation state, false otherwise. ...
>>
>> So when ANS says 'interpretation semantics undefined' in POSTPONE it
>> meant STATE=0 - irrespective of whether POSTPONE is being executed
>> in a definition.
>
> It looks like you miss the part "name is encountered by the text
> interpreter"
>
> Interpretation semantics are not about "a name is being execute", but
> about "a name is being encountered".
>
> By your argument, the following definition of "within" [1]:
>
> : within ( test low high -- flag ) over - >r - r> u< ;
>
> cannot work in interpretation state:
>
> 1 1 5 within . \ prints -1
>
> since interpretation semantics for ">r" are undefined.

I can't ignore history and neither did ANS. By ANS' own commentary
in A.6.1.2033, POSTPONE is little more than a combination of COMPILE
and [COMPILE]. Any restrictions on the standard use of POSTPONE is
a consequence of the use of those historic words. To use the
wording of ANS to suggest WITHIN is illegal, or alternatively, to
extrapolate behaviours in ways that break historical precedence,
can only result in fragmentation and chaos. OTOH if it can be
demonstrated change to an existing ruling benefits everyone, that's
a different story and one that even I may be interested in if it
proves to be true.

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>On 5/12/21 2:14 AM, Ruvim wrote:
>> On 2021-05-11 13:51, Anton Ertl wrote:
>> [...]
>>> 3) We have discussed POSTPONE repeatedly over the last
>>> quarter-century, and I have established repeatedly that according to
>>> the text of Forth-94 and Forth-2012
>>>
>>> : foo postpone . ;
>>> : bar [ foo ] ;
>>> 1 bar           \ prints "1"
>>>
>>> conforms to these standards.
>>
>> According to the only wording in the specification for POSTPONE this
>> code conforms the standard, and it should print "1".
>> ...
>
>Is there a *standard* way of detecting an incomplete definition when in
>interpretation state?

A standard program cannot detect whether a definition is incomplete or
can be continued, irrespective of state.

] . [ \ non-standard
: foo [ 1 , ] . ; \ non-standard
: comp-. postpone . ; immediate \ standard
: bar1 comp-. ; \ standard
: bar2 [ comp-. ] ; \ standard

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

dxforth <dxforth@gmail.com> writes:
>On 11/05/2021 20:51, Anton Ertl wrote:
>> dxforth <dxforth@gmail.com> writes:
>>>On 11/05/2021 02:09, Anton Ertl wrote:
>>>> dxforth <dxforth@gmail.com> writes:
>>>>>What hasn't changed is the classic behaviour that executing
>>>>>COMPILE when STATE=0 will crash the interpreter due to IP having been
>>>>>advanced.
>>>>
>>>> This has nothing to do with STATE; you identify the dependency
>>>> correctly: IP. Here's an example how COMPILE can run when STATE=0:
>>>>
>>>> \ First, define COMPILE, and show STATE when it runs:
>>>> : compile state ? r> dup cell+ >r @ , ;
>>>>
>>>> : foo compile . ;
>>>> : bar [ foo ] ; \ prints "0" (output of COMPILE)
>>>> 1 bar \ prints "1"
>>>>
>>>> Tested with gforth-itc 0.7.9_20210415. As you can see, running
>>>> COMPILE when STATE=0 works.
>>>
>>>AFAIK Forth-83 didn't require that example to work. Same for
>>>ANS and POSTPONE.
>>
>> I guess that changing the goalposts is an admission that the statement
>> "executing COMPILE when STATE=0 will crash the interpreter due to IP
>> having been advanced." is wrong.
>>
>> Let's consider the new goalposts:
>>
>> 1) Return-address manipulation is non-standard (I think also in
>> Forth-83). But let's assume the system-defined COMPILE rather than
>> the COMPILE defined above.
>>
>> 2) Forth-83
>>
>> COMPILE is marked with the C attribute in Forth-83. In Section 11.4,
>> Forth-83 states:
>>
>> | C The word may only be used during compilation of a colon
>> | definition.
>>
>> It's not clear what "use" means here. Does it mean the compilation of
>> COMPILE during the definition of FOO? Does it mean the running of
>> COMPILE during the definition of BAR? Does it mean the result of the
>> COMPILE when running BAR in the final line of the example? I guess
>> that it's not the latter, because then COMPILE would be pretty
>> unusable.
>
>Go to the definitions:
>
> interpretation semantics:
> The behavior of a Forth definition when its name is encountered
> by the text interpreter in interpretation state.
>
> 6.1.2250 STATE
> ... STATE is true when in compilation state, false otherwise. ...
>
>So when ANS says 'interpretation semantics undefined' in POSTPONE it
>meant STATE=0

It's unclear to me why you discuss this Forth-94 stuff as answer to my
discussion of Forth-83, but anyway, here we go:

No, it means that text-interpreting POSTPONE in interpret state is
undefined. I.e.:

: foo [ postpone . ] ;

is non-standard, nothing more.

>This restriction almost certainly comes from 79/83-COMPILE and the
>most popular forth of the day Fig-Forth:

Actually the "Implementation semantics is undefined" specification
reflects the practice of cmForth to put words like IF into the
COMPILER vocabulary, but not in the FORTH wordlist; so IF is only
found by the text interpreter in interpret state (but it can still be
run in interpret state).

Interestingly, cmForth puts COMPILE in the FORTH wordlist, not in the
compiler wordlist, even though text-interpreting COMPILE in interpret
state is guaranteed to have unintended results. The Forth-94
committee apparently thought that it is a good idea to undefine the
interpretation semantics of POSTPONE, even though POSTPONE does not
have this problem of COMPILE.

> SCR # 41
> 0 ( COMPILE, SMUDGE, HEX, DECIMAL WFR-79APR20 )
> 1
> 2 : COMPILE ( COMPILE THE EXECUTION ADDRESS FOLLOWING *)
> 3 ?COMP R> DUP 2+ >R @ , ;
>
>The ?COMP prevented COMPILE being executed outside a definition

No, it does not, because in fig-Forth ?COMP only tests STATE. You can
write

] COMPILE + [ \ without enclosing colon definition

and the ?COMP will not prevent it.

>> You get at least two bytes. And ?COMP is not compiler security.
>
>Sure it is. Removing ?COMP from COMPILE would mean explicitly
>inserting it into IF AHEAD UNTIL AGAIN etc which costs more than
>two bytes.

?COMP in IF AHEAD UNTIL AGAIN is not compiler security, either. If
you want to prevent these words from being executed outside colon
definitions, you need a way to identify whether you are inside a colon
definition; STATE is not that way, and has not been ever since [ and ]
were introduced.

But is such a check practically useful? It seems that in Forth,
Inc. experience it's not:

SwiftForth i386-Linux 3.11.0 23-Feb-2021
if ok
..s
134761494 <-Top ok
then ok
..s
<-Top ok

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

In article <2021May14.124013@mips.complang.tuwien.ac.at>,
Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
<SNIP>
>
>?COMP in IF AHEAD UNTIL AGAIN is not compiler security, either. If
>you want to prevent these words from being executed outside colon
>definitions, you need a way to identify whether you are inside a colon
>definition; STATE is not that way, and has not been ever since [ and ]
>were introduced.
>
>But is such a check practically useful? It seems that in Forth,
>Inc. experience it's not:
>
>SwiftForth i386-Linux 3.11.0 23-Feb-2021
>if ok
>.s
>134761494 <-Top ok
>then ok
>.s
><-Top ok

Or in ciforth
Note that you have to load the electives, otherwise
this feature is not available. (-e option).

"
lina64 -e
.....
You should have received a copy of the GNU General Public
....
TASK : ISN'T UNIQUE

TYPE "WANT AUTOLOAD" for automatic loading.
S[ 1 ] OK 12 IF

S[ 12 37766592 0 37766600 2 ] OK "aap" TYPE

S[ 12 37766592 0 37766600 2 ] OK ELSE

S[ 12 37766592 0 37766680 2 ] OK "NOOT " TYPE

S[ 12 37766592 0 37766680 2 ] OK THEN
aap
S[ ] OK
"
It serves no purpose to put roadblocks in the way of a useful
extension.

>- anton

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On 5/14/21 5:33 AM, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> On 5/12/21 2:14 AM, Ruvim wrote:
>>> On 2021-05-11 13:51, Anton Ertl wrote:
>>> [...]
>>>> 3) We have discussed POSTPONE repeatedly over the last
>>>> quarter-century, and I have established repeatedly that according to
>>>> the text of Forth-94 and Forth-2012
>>>>
>>>> : foo postpone . ;
>>>> : bar [ foo ] ;
>>>> 1 bar           \ prints "1"
>>>>
>>>> conforms to these standards.
>>>
>>> According to the only wording in the specification for POSTPONE this
>>> code conforms the standard, and it should print "1".
>>> ...
>>
>> Is there a *standard* way of detecting an incomplete definition when in
>> interpretation state?
>
> A standard program cannot detect whether a definition is incomplete or
> can be continued, irrespective of state.
>
> ] . [ \ non-standard
> : foo [ 1 , ] . ; \ non-standard
> : comp-. postpone . ; immediate \ standard
> : bar1 comp-. ; \ standard
> : bar2 [ comp-. ] ; \ standard
>

If the original example is standard, namely,

: foo postpone . ;
: bar [ foo ] ;

with the behavior, "1 bar" prints "1", then perhaps it is possible to
write a word in standard code which can tell whether or not a definition
is in progress, and may be continued.

FOO is aware of an incomplete definition since FOO is able to append the
execution semantics of "." to the incomplete definition, even when FOO
is executed in interpretation state.

There are two other ways, in Gforth, to obtain the same behavior for BAR.

1.

: foo postpone . ; immediate
: bar foo ;

2.

: foo . ;
: bar [ ' foo compile, ] ;

Both will give behavior that "1 bar" prints "1". I believe there is no
dispute that method 1 is consistent with most people's understanding of
the Forth standard.

For both "POSTPONE" and "COMPILE," the standard says,

"interpretation semantics for this word are undefined"

How do we reconcile the above statement with using POSTPONE and COMPILE,
in interpretation state? Does the term, "interpretation semantics",
then, not apply when one opens a definition and then enters
interpretation state?

I note that in Gforth (and kForth as well), it is possible to write,

' foo compile,

when no definition is in progress. But, it has no useful property within
the standard.

Krishna

albert@cherry.(none) (albert) writes:
>In article <2021May14.124013@mips.complang.tuwien.ac.at>,
>Anton Ertl <anton@mips.complang.tuwien.ac.at> wrote:
><SNIP>
>>
>>?COMP in IF AHEAD UNTIL AGAIN is not compiler security, either. If
>>you want to prevent these words from being executed outside colon
>>definitions, you need a way to identify whether you are inside a colon
>>definition; STATE is not that way, and has not been ever since [ and ]
>>were introduced.
>>
>>But is such a check practically useful? It seems that in Forth,
>>Inc. experience it's not:
>>
>>SwiftForth i386-Linux 3.11.0 23-Feb-2021
>>if ok
>>.s
>>134761494 <-Top ok
>>then ok
>>.s
>><-Top ok
>
>Or in ciforth
>Note that you have to load the electives, otherwise
>this feature is not available. (-e option).

[example of an [if]-like if]

Note that SwiftForth does not tack any particular feature on
interpreted IF and THEN. As far as I can tell, it just performs the
compilation semantics when interpreting IF and THEN:

SwiftForth i386-Linux 3.11.0 23-Feb-2021
cr 1 if 2 . then .s
2
1 <-Top ok
drop ok
cr 0 if 2 . then .s
2
0 <-Top ok

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020