P Falth schrieb am Montag, 10. Mai 2021 um 23:16:26 UTC+2:
> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
> > On 2021-05-06 18:23, P Falth wrote:
> > > On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> > >> On 2021-05-06 17:07, Ruvim wrote:
> > >>> On 2021-05-06 14:38, P Falth wrote:
> > >> [...]
> > >>>> As an alternative to FIND I can suggest
> > >>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> > >>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> > >>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
> > >> They are quite close to FIND-NAME NAME>COMPILE
> > >
> > > Yes they are. But they also hide the NT and can thus accomodate
> > > also a system with dual NTs. Someone (Mark Williams?) described
> > > such a system that was simple and clean.
> > But why they are better than "FIND"? (except (c-addr u) string).
> When I developed the code generator for my forth (about 20 years ago)
> I needed a way to execute the generator for a word when it was compiled
> into a definition. I started with the CMFORTH style dual headers in different
> wordlists. This worked fine until I also needed wordlists for its intended
> purpose. That is when I added a compilation xt to the header. For some
> words ( normal words, created words) at compiletime was also needed
> the interpretation xt. FIND could not give me this. From this was born
> FIND-INTERPRET and FIND-COMPILE. They do what I needed!
> > In my recent iteration I introduced the word FIND-SEM
> >
> > FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )
> >
> > It is closer to classic "FIND", with only difference that it accepts a
> > string as the pair (c-addr u), and returns more convenient flag-special,
> > rather than -1|1 code.
> But that would not let me find both xts at the same time
> > And why do you need "FIND-POSTPONE"?
> I do not at the moment and it is not implemented. But might be
> useful for recognizers. It might serve to do words with non default
> postpone semantics (NDPS words!)
>
> That was a joke!

You can't be serious.

NDPS can be deadly when you POSTPONE TO when in interpretation mode
between square brackets, and while EVALUATEing that thing !!

It is absolutely necessary that someone rewrites the standard !!

On 10/05/2021 23:58, Stephen Pelc wrote:
> On Sat, 8 May 2021 22:17:52 +0300, Ruvim <ruvim.pinka@gmail.com>
> wrote:
>
>>As I understand, the Stephen's point was that the standard describes how
>>to perform default compilation semantics (that's to append ES), and how
>>to perform compilation semantics for immediate words (that's to perform
>>ES), but doesn't describe how to perform compilation semantics for other
>>words.
>
> Thanks.
>
>>Actually, it's a fictional problem. The standard just considers and
>>describes behavior of a system on some level of abstraction, and doesn't
>>consider by what means this behavior is accomplished, and how the system
>>behaves on the underlying levels.
>
> But it's not a fictional problem in the presence of testing.
> If we do not have standard ways (words) to handle NDCS words, we
> cannot have standard tests. Now that VFX Forth handles IF and other
> words as NDCS words, we need to add NDCS handling words to the
> standard.

Standard tests for what purpose? First one must define the
standard need i.e. why does FIND used in situation X need to
produce the same result on every system? ANS' premise in
permitting implementers to do as they pleased was that not
every situation required standardizing. Were they wrong?
I couldn't find a single app I'd written that used FIND :)

On 11/05/2021 02:09, Anton Ertl wrote:
> dxforth <dxforth@gmail.com> writes:
>>I've since shifted the state-smartness from NUMBER to INTERPRET where
>>it belongs.
>
> Good.
>
>>What hasn't changed is the classic behaviour that executing
>>COMPILE when STATE=0 will crash the interpreter due to IP having been
>>advanced.
>
> This has nothing to do with STATE; you identify the dependency
> correctly: IP. Here's an example how COMPILE can run when STATE=0:
>
> \ First, define COMPILE, and show STATE when it runs:
> : compile state ? r> dup cell+ >r @ , ;
>
> : foo compile . ;
> : bar [ foo ] ; \ prints "0" (output of COMPILE)
> 1 bar \ prints "1"
>
> Tested with gforth-itc 0.7.9_20210415. As you can see, running
> COMPILE when STATE=0 works.

AFAIK Forth-83 didn't require that example to work. Same for
ANS and POSTPONE. Is there a demonstrated need that it should
work that would justify amending or clarifying ANS' ruling?

Change in my case would mean removing ?COMP from COMPILE. The
former hasn't proven an issue thus far. I'd like to know what
I was getting in return for removing compiler security.

Marcel Hendrix <mhx@iae.nl> writes:
>On Saturday, May 8, 2021 at 3:10:49 PM UTC+2, Anton Ertl wrote:
[...]
>I am not fond of these discussions but must point out that DEFER@
>only works for words created by DEFER , a word in the CORE-EXT set.
>It is therefore unlikely that DEFER@ is going to work on non-user-defined
>words like COMPILE,.

As it happens, COMPILE, is defined with DEFER in VFX 4.71 and VFX 5.11:

VFX Forth for Linux IA32 Version: 4.71 [build 0537]
see compile,
COMPILE, is DEFERred. Action is COMPILE-REF
COMPILE-REF
....

VFX Forth 64 5.11 RC2 [build 0112] 2021-05-02 for Linux x64
© MicroProcessor Engineering Ltd, 1998-2021

see compile,
COMPILE, is DEFERred. Action is COMPILE-REF
COMPILE-REF
( 0046FC80 483B1D6659FAFF ) CMP RBX, FFFA5966 [RIP] @004155ED
....

COMPILE, is also defined with DEFER in Gforth, SwiftForth and lxf.

This is useful for my original test, because it allows to test the
COMPILE, embedded in the text interpreter of the system rather than
having to write a new text interpreter. For Ruvim's test that I
combined my test with, it would be good enough to just redefine
"COMPILE,", but the version with IS also shows what happens in the
rest of the system.

>Or is there a guarantee now that existence
>of DEFER means that all deferred words are created by DEFER ?

If your question is whether all non-user-defined words are created by
DEFER: No. IIRC such a guarantee exists for no standard-specified
word. In this case I just happened to know that
COMPILE, is deferred on a few systems.

>> :noname ( xt -- )
>> >in @ >r
>> depth 1- >r
>> defers compile,
>
>Here you assume COMPILE, is a deferred word. It would really help
>to show the complete example.

The test is complete. Paste it in VFX, Gforth, SwiftForth, or lxf and
see what happens. But just to have the test and results here to
discuss, I present them all:

The test:
: defers
\ "defers X" produces a call to the current content of the deferred word X
' defer@ compile, ; immediate

:noname ( xt -- )
>in @ >r
depth 1- >r
defers compile,
depth r> <> abort" compile, stack effect wrong"
>in @ r> <> abort" compile, input stream effect wrong" ;
is compile,

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

] test-if [ .s

The results:
[~:121903] gforth /tmp/xxx.fs
redefined Defers with defers executed<3> 0 140166832538920 2

[~:121904] sf "include /tmp/xxx.fs"
executed
134761828 <-Top ok

VFX Forth for Linux IA32 Version: 4.71 [build 0537]
Including /tmp/xxx.fs
CS=0023 DS=002B ES=002B SS=002B FS=0000 GS=0063
EAX=0000:0000 EBX=FFFF:FFFF ECX=090A:0E25 EDX=0000:0000
ESI=090B:2000 EDI=0804:A000 EBP=090B:0FC0 ESP=090A:0F1C
EIP=080C:091A EFLAGS=0001:0297
--- RS top ---
$FFFF:FFFF
$0000:0013
$0805:9C8B DEFER
$080C:0906 DEFERS
$0805:9C8B DEFER
$0805:97F3 (INTERPRET)
$0805:9C8B DEFER
$0805:EE90 ((INCLUDE-FILE))
Signal number SIGSEGV
at address 080C:091A, probably in DEFERS

VFX Forth for Linux IA32 Version: 4.72 [build 0555]
Including /tmp/xxx.fscompile, input stream effect wrong
Source: "/tmp/xxx.fs" on line 14
-> c" if" find dup 0= if ." unfound" 2drop exit then
^
ERROR on command line

VFX Forth 64 5.11 RC2 [build 0112] 2021-05-02 for Linux x64
© MicroProcessor Engineering Ltd, 1998-2021

Including /tmp/xxx.fscompiledcompile, stack effect wrong
Source: "/tmp/xxx.fs" on line 18
-> ] test-if [ .s
^
ERROR on command line

Linux/FORTH (C) 2005 Peter Fälth Version 1.6-982-823 Compiled on 2017-12-03
Running on Linux 4.5.0-0.bpo.2-amd64 on x86_64
Current directory is /home/anton no block file
include /tmp/xxx.fs compiled

Gforth and SwiftForth behave as expected: FIND reports 1 for IF, so
TEST-IF EXECUTEs the xt, and this pushes an orig on the stack.

vfx 4.71 fails due to a bug in DEFER@. This bug was fixed in 4.72,
and there scaffolded COMPILE, shows that VFX's COMPILE, has a
non-standard effect on the input stream when text-interpreting C".
VFX does not show that, but we see a non-standard stack effect of
COMPILE, in Ruvim's test. My guess is that VFX 5.11 defines the text
interpreter differently so the problem with COMPILE, does not show up
there, but it still shows up in a user-defined text interpreter.

lxf has a working COMPILE, but the FIND does not support user-defined
text interpreters.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

In article <s7d1v6$jgs$1@gioia.aioe.org>, dxforth <dxforth@gmail.com> wrote:
<SNIP>
>Standard tests for what purpose? First one must define the
>standard need i.e. why does FIND used in situation X need to
>produce the same result on every system? ANS' premise in
>permitting implementers to do as they pleased was that not
>every situation required standardizing. Were they wrong?
>I couldn't find a single app I'd written that used FIND :)
>
I once had the sophisticated ciasdis assembler to use a wordlist
to contain the objects associated with each assembler mnemonic.
Use of the wordlist came on top of the normal use of the assembler,
they are just words after all.
All other uses I now consider a design fault, and I made the assembler
mnemonic objects a linked list, for that.
A case in point, the objects are usable in my optimiser.
I need a few object of that type in my optimiser that would not
trigger in normal assembly use. That would have been been
a terrible pain, without the separate link chain.

>
Groetjes Albert
>
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On 2021-05-11 07:42, dxforth wrote:
> On 10/05/2021 23:58, Stephen Pelc wrote:
[...]
>> If we do not have standard ways (words) to handle NDCS words, we
>> cannot have standard tests. Now that VFX Forth handles IF and other
>> words as NDCS words, we need to add NDCS handling words to the
>> standard.
>
> Standard tests for what purpose?  First one must define the
> standard need i.e. why does FIND used in situation X need to
> produce the same result on every system?  ANS' premise in
> permitting implementers to do as they pleased was that not
> every situation required standardizing.

Yes. But every standard word should be enough specified to be useful.

> Were they wrong?

The specification for the FIND word is not sufficient.

A.6.2.0945 says:
The intention is that COMPILE, can be used as follows
to write the classic interpreter/compiler loop:

... ( c-addr )
FIND ?DUP IF ( xt +-1 )
STATE @ IF ( xt +-1 )
0> IF EXECUTE ELSE COMPILE, THEN ( ??? )
ELSE ( xt +-1 )
DROP EXECUTE ( ??? )
THEN
ELSE ( c-addr )
( whatever you do for an undefined word )
THEN
...

Some systems implement FIND in such a way that this loop doesn't work as
expected for the words with non default interpretation semantics.

> I couldn't find a single app I'd written that used FIND :)

Probably you just didn't ever need a user-defined text interpreter for
some advanced DSL (domain specific language), or some advanced extension
of a Forth system.

--
Ruvim

dxforth <dxforth@gmail.com> writes:
>On 11/05/2021 02:09, Anton Ertl wrote:
>> dxforth <dxforth@gmail.com> writes:
>>>What hasn't changed is the classic behaviour that executing
>>>COMPILE when STATE=0 will crash the interpreter due to IP having been
>>>advanced.
>>
>> This has nothing to do with STATE; you identify the dependency
>> correctly: IP. Here's an example how COMPILE can run when STATE=0:
>>
>> \ First, define COMPILE, and show STATE when it runs:
>> : compile state ? r> dup cell+ >r @ , ;
>>
>> : foo compile . ;
>> : bar [ foo ] ; \ prints "0" (output of COMPILE)
>> 1 bar \ prints "1"
>>
>> Tested with gforth-itc 0.7.9_20210415. As you can see, running
>> COMPILE when STATE=0 works.
>
>AFAIK Forth-83 didn't require that example to work. Same for
>ANS and POSTPONE.

I guess that changing the goalposts is an admission that the statement
"executing COMPILE when STATE=0 will crash the interpreter due to IP
having been advanced." is wrong.

Let's consider the new goalposts:

1) Return-address manipulation is non-standard (I think also in
Forth-83). But let's assume the system-defined COMPILE rather than
the COMPILE defined above.

2) Forth-83

COMPILE is marked with the C attribute in Forth-83. In Section 11.4,
Forth-83 states:

| C The word may only be used during compilation of a colon
| definition.

It's not clear what "use" means here. Does it mean the compilation of
COMPILE during the definition of FOO? Does it mean the running of
COMPILE during the definition of BAR? Does it mean the result of the
COMPILE when running BAR in the final line of the example? I guess
that it's not the latter, because then COMPILE would be pretty
unusable.

The other unclear thing is what "during compilation of a colon
definition" means. My take is that FOO is run during the compilation
of BAR, so the definition of BAR is Forth-83-conforming in my book.
Fans of ?COMP will probably take a different position, claiming that
"during compilation" means STATE=-1, and ignoring the "of a colon
definition" part.

3) We have discussed POSTPONE repeatedly over the last
quarter-century, and I have established repeatedly that according to
the text of Forth-94 and Forth-2012

: foo postpone . ;
: bar [ foo ] ;
1 bar \ prints "1"

conforms to these standards.

>Is there a demonstrated need that it should
>work that would justify amending or clarifying ANS' ruling?

There is no need to amend or clarify. The text of the standard is
clear. If anyone wants to change it in the direction of disallowing
to append something to the current definition when STATE=0, they have
to propose it and should give a good reason for such a change. I
demonstrated above that basing POSTPONE on COMPILE is not such a
reason.

>Change in my case would mean removing ?COMP from COMPILE. The
>former hasn't proven an issue thus far. I'd like to know what
>I was getting in return for removing compiler security.

You get at least two bytes. And ?COMP is not compiler security.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

In article <s7dnf0$fjm$1@dont-email.me>, Ruvim <ruvim.pinka@gmail.com> wrote:
>On 2021-05-11 07:42, dxforth wrote:
>> I couldn't find a single app I'd written that used FIND :)
>
>Probably you just didn't ever need a user-defined text interpreter for
>some advanced DSL (domain specific language), or some advanced extension
>of a Forth system.

tmanx2 (my interpreter for music, that plays music on the metallophones,
and maybe midi) has elements corresponding to notes, notes-duration, scores,
parts, bars, chords, articulation, transposition, tempo, note-binding.
It certainly can count as an example of an advanced DSL.
That is all handled by the Forth text interpreter itself.
Then there is my syntax checker for Pascal.
IMO DSL and advanced extensions can normally avoid using FIND explicitly,
probably resulting in better portability.
The only example I can think of is if you want to emulate the weird syntax
of lisp related to their idea of macro's, that arbitrarily defines part
of the input evaluated immediately. I tried it, and it made me conclude
that lisp is badly designed.

>--
>Ruvim

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On 2021-05-11 00:16, P Falth wrote:
> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
>> On 2021-05-06 18:23, P Falth wrote:
>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
>>>> On 2021-05-06 17:07, Ruvim wrote:
>>>>> On 2021-05-06 14:38, P Falth wrote:
>>>> [...]
>>>>>> As an alternative to FIND I can suggest
>>>>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>> They are quite close to FIND-NAME NAME>COMPILE
>>>
>>> Yes they are. But they also hide the NT and can thus accomodate
>>> also a system with dual NTs. Someone (Mark Williams?) described
>>> such a system that was simple and clean.
>> But why they are better than "FIND"? (except (c-addr u) string).
>
> When I developed the code generator for my forth (about 20 years ago)
> I needed a way to execute the generator for a word when it was compiled
> into a definition. I started with the CMFORTH style dual headers in different
> wordlists. This worked fine until I also needed wordlists for its intended
> purpose. That is when I added a compilation xt to the header.

Another option is to introduce different flags into the header and a
functionality to find words with given flags only. But if you need both
semantics in the same time, FIND should be performed twice in this approach.

> For some
> words ( normal words, created words) at compiletime was also needed
> the interpretation xt. FIND could not give me this. From this was born
> FIND-INTERPRET and FIND-COMPILE. They do what I needed!

I see. It's OK for internal or system-specific purposes.

But then for compatibility you could make xt=nt (and then it will be a
sing-xt system from the user's point of view).

Or at least FIND should return something like xt of
[: true abort" not implemented" ;]
and n value 1, or just n value 0, when FIND is applied to the "IF"
string in compilation state, and ['] IF with n value 1 — in
interpretation state.

>> In my recent iteration I introduced the word FIND-SEM
>>
>> FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )
>>
>> It is closer to classic "FIND", with only difference that it accepts a
>> string as the pair (c-addr u), and returns more convenient flag-special,
>> rather than -1|1 code.
>
> But that would not let me find both xts at the same time

Do you talk about your system or a program?

In your system you are free to use system-specific methods.

A standard program have to call FIND-SEM twice (in two different states)
for that — the same as with FIND.

>> And why do you need "FIND-POSTPONE"?
>
> I do not at the moment and it is not implemented. But might be
> useful for recognizers. It might serve to do words with non default
> postpone semantics (NDPS words!)
>
> That was a joke!

Oh, I see. After all, NDPS is non standard.

>>>>> If you can implement this, nothing prevent you to implement STATE
>>>>> dependent FIND
>>>> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
>>>> either "EXECUTE" or "COMPILE," and nothing else.
>>>
>>> To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
>>> limit the possibilities of the system. Take for example a constant.
>>> XT2 could be the actual value of the constant and XT1 LITERAL for compiling.
>> It looks like you reinvent recognizers under the different name.
>
> Yes, just 20 years ago!

Then why don't use recognizers for such things?

As Anton pointed out, above approach is possible for single-cell numbers
only, when recognizers are suitable for any literals.

>>> My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
>>> to a JIT compiler that compiles the tokens to native code and then executes it
>> Why cannot EXECUTE be who calls a JIT compiler (if a better code is not
>> generated yet) and then executes this better code?
>
> Yes that is how it can work. ON the stack will be
>
> NT JIT-compiler, coming from find-interpret

I mean that EXECUTE behaves in the following manner:

: execute ( i*x xt -- j*x )
xt>oxt if ( oxt ) oexecute exit then ( xt )
ocompiled ( oxt ) oexecute
;

> I am not there yet. Now I can call native code from the token code. But I also want to
> do the reverse.

>
>>>> Otherwise FIND cannot be implemented via these words.
>>>
>>> There is no requirements to implement FIND with these words.
>> For back-compatibility, the system should still provide FIND for a
>> while. And it should be clear how to implement FIND via new methods in
>> the case when a system doesn't provide FIND (to use programs/libraries
>> that rely on FIND).
>
> I do provide a FIND in my systems.

When you propose an alternative to FIND, it's not about your system.
It's about the standard and the Forth ecosystem.

> In lxf it is not possible to do a user defined
> interpreter with it, for 2 reasons. It is not state-smart. It only returns the
> interpretation XT. Words like IF with no defined interpretation semantics
> return an xt that print an error message and aborts. If needed i could change this
> but so far has not needed it.

As I understand, the second reason is that lxf cannot provide an xt that
identifies the following execution semantics:

[: postpone if ;]

>> A one shortcoming of NAME>* methods is that they don't guarantee that
>> FIND can be implementable via them. Moreover, FIND is not implementable
>> via these methods in a single-xt system, if this system allows to
>> perform compilation semantics in interpretation state.
>
> My lxf64 is still single-xt, FIND works just fine there. There is no requirement
> that find must be implemented by NAME>* methods
>
>>>>>
>>>>> But what for do you need FIND-POSTPONE ?
>>>>>
>>>>>
>>>>>> The result if found just needs to be executed to perform the
>>>>>> respective actions
>>>>>> It also hides the implementation details ( single-xt, dual-xt, dual
>>>>>> headers etc)
>>>>>
>>>>> It doesn't cover sing-xt systems. In these systems the corresponding
>>>>> action should be performed only in the corresponding STATE.
>>>> I missed that they return two xt-s. Then yes, they cover single-xt
>>>> systems too.
>>>>
>>>> But then what is a way to detect whether a word is an ordinary word?
>>>
>>> That is not needed!
>> I need that.
>>
>> See for exampe: https://forth-standard.org/standard/tools#reply-445
>>
>> Another example. A kind of portable SYNONYM should be able to detected
>> whether a word is an ordinary word, or not.
>
> I provide a completely compliant SYNONYM. Different NTs but same XTs
> There is no need to load another synonym.

Again, you talk about your system. But I talk about Forth ecosystem, and
broad variety of Forth systems.

If a standard system doesn't provide SYNONYM but provide a full-fledged
FIND, then SYNONYM (that preserves execution semantics as well as IS and
CS) can be implemented.

>>>> And what is a way to get xt of an ordinary word, that identifies the
>>>> execution semantics for this word?
>>>
>>> ' and ['] are still available for this.
>> They cannot be applied to any word. They cannot be used to words of
>> unknown kind.
>
> I do not understand this. Any word that has an XT could be ticked, or?

Tick returns xt that identifies execution semantics (ES) for the name in
argument, but ES are not specified for many standard words.

Therefore, 4.1.2 says that an ambiguous condition exists if a program
attempts to obtain the execution token of a definition with undefined
interpretation semantics (for all such cases ES are either unspecified,
or cannot be provided in many implementations).

Concerning other words with unspecified ES, it says that applying Tick
to the word is ambiguous, individually for each such word except FILE S"
and FILE S\".

My proposal is to remove individual cases and declare the single rule:
an ambiguous condition exists if a program attempts to obtaining the
execution token of a definition with non default interpretation semantics.

NB: my proposal of FIND clarification explicitly specifies in what case
the returned xt identifies ES for the name/word in argument.

On 2021-05-11 17:00, albert wrote:
> In article <s7dnf0$fjm$1@dont-email.me>, Ruvim <ruvim.pinka@gmail.com> wrote:
>> On 2021-05-11 07:42, dxforth wrote:
>>> I couldn't find a single app I'd written that used FIND :)
>>
>> Probably you just didn't ever need a user-defined text interpreter for
>> some advanced DSL (domain specific language), or some advanced extension
>> of a Forth system.
>
> tmanx2 (my interpreter for music, that plays music on the metallophones,
> and maybe midi) has elements corresponding to notes, notes-duration, scores,
> parts, bars, chords, articulation, transposition, tempo, note-binding.
> It certainly can count as an example of an advanced DSL.
> That is all handled by the Forth text interpreter itself.
> Then there is my syntax checker for Pascal.
> IMO DSL and advanced extensions can normally avoid using FIND explicitly,
> probably resulting in better portability.

We have two options to implement control structures (or other lexical
blocks in source code):
- non parsing words (like "IF")
- parsing words (like "[IF]")

If we want to make it work in interpretation state, only parsing variant
is suitable. Also, parsing variant is easier in implementation since
control parameters can be kept on the r-stack.

But then you have to use a user-defined text interpreter, and rely on
FIND (or it's alternatives).

--
Ruvim

On Fri, 7 May 2021 14:12:24 +0300, Ruvim <ruvim.pinka@gmail.com>
wrote:

>> In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
>> of COMPILE, for such words is NDCS,
>>
>> : ndcs, \ ??? xt -- ???
>> \ *G Perform the compilation action of an NDCS word. This may have
>> \ ** a stack effect or parse the input stream.
>>
>> : compile-word \ i*x xt -- j*x
>> \ *G Process an XT for compilation. May parse or have stack effect.
>> dup ndcs? \ NDCS or normal
>> if ndcs, else compile, then
>> ;
>>
>> In VFX, most uses of COMPILE, should use COMPILE-WORD instead.
>> The alternative is to redefine COMPILE, to take note of NDCS words
>> but ... politics, politic, politics.
>
>Taking into account that "ndcs," just performs another xt:
>
> : (ndcs,) \ i*x xt -- j*x
> >code-len @ execute \ execute NDCS/IMMEDIATE word
> ;
>
>Why not return this xt by "find" ?
>
> : find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
> find dup 0= if exit then drop ( xt1 )
> dup ndcs? if >code-len @ 1 else
> dup immediate? if 1 else -1 then then ( xt1 xt2 n )
> state @ if rot else swap then drop
> ;
>
>After this fix all my tests work as expected.

The fix is the sensible one, which is to throw away IMMEDIATE and
let FIND indicate NDCS and normal. However, on some systems this
will be a code breaker.

Personally, and for MPE, I love the original idea of an xt that it
be the single point of contact between the user and the word. An
xt can be used for many purposes other than compilation and
interpretation, e.g. LOCATE XREF and so on.

Use of xt in this way requires that all words have a primary or
interpretation action, which the xt represents. Your fix breaks
this and I suppose makes it non-standard.

Because of SYNONYM and friends, there is an argument that the
unique token for a word is the name token.

Regardless of nt or xt, I see no way of dealing with NDCS words
without upgrading the standard.

>
>To be on the safe side, I would redefine/fix some other words too, to
>exclude situations when "compile," produces unexpected stack effect when
>it's applied to some xt.
>
>Nothing standard word should return such xt for which "compile,"
>produces unexpected stack effect.

This cannot be done without tools to handle NDCS words.

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

On Mon, 10 May 2021 14:16:25 -0700 (PDT), P Falth
<peter.m.falth@gmail.com> wrote:

>I do provide a FIND in my systems. In lxf it is not possible to do a user defined
>interpreter with it, for 2 reasons. It is not state-smart. It only returns the
>interpretation XT. Words like IF with no defined interpretation semantics
>return an xt that print an error message and aborts. If needed i could change this
>but so far has not needed it.

VFX works the same way.

dis if
IF
( 0001A0D0 E803F2FFFF ) CALL 000192D8 NOINTERP
( 0001A0D5 C3 ) RET/NEXT
( 6 bytes, 2 instructions )
ok

>My lxf64 is still single-xt, FIND works just fine there. There is no requirement
>that find must be implemented by NAME>* methods

Same for VFX.

>I do not understand this. Any word that has an XT could be ticked, or?

For VFX, all words have xts which are accessible by the user.

I am coming round to replacing FIND with a word such as:
: Search-Context \ c-addr len -- 0 | xt 1 | xt -1
\ *G Perform the *\fo{SEARCH-WORDLIST} operation on all wordlists
\ ** within the search order. Returns -1 for a "normal" word,
\ ** +1 for an *\fo{IMMEDIATE} word and +2 for an NDCS word.

If the world changes yet again we could modify SEARCH-CONTEXT to
: Search-Context \ c-addr len -- 0 | xt class

where a word's class is
0 unknown
-1 normal
+1 IMMEDIATE
+2 NDCS

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

On Tuesday, 11 May 2021 at 16:05:10 UTC+2, Ruvim wrote:
> On 2021-05-11 00:16, P Falth wrote:
> > On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
> >> On 2021-05-06 18:23, P Falth wrote:
> >>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> >>>> On 2021-05-06 17:07, Ruvim wrote:
> >>>>> On 2021-05-06 14:38, P Falth wrote:
> >>>> [...]
> >>>>>> As an alternative to FIND I can suggest
> >>>>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> >>>>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >>>>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >>>> They are quite close to FIND-NAME NAME>COMPILE
> >>>
> >>> Yes they are. But they also hide the NT and can thus accomodate
> >>> also a system with dual NTs. Someone (Mark Williams?) described
> >>> such a system that was simple and clean.
> >> But why they are better than "FIND"? (except (c-addr u) string).
> >
> > When I developed the code generator for my forth (about 20 years ago)
> > I needed a way to execute the generator for a word when it was compiled
> > into a definition. I started with the CMFORTH style dual headers in different
> > wordlists. This worked fine until I also needed wordlists for its intended
> > purpose. That is when I added a compilation xt to the header.
> Another option is to introduce different flags into the header and a
> functionality to find words with given flags only. But if you need both
> semantics in the same time, FIND should be performed twice in this approach.
> > For some
> > words ( normal words, created words) at compiletime was also needed
> > the interpretation xt. FIND could not give me this. From this was born
> > FIND-INTERPRET and FIND-COMPILE. They do what I needed!
> I see. It's OK for internal or system-specific purposes.
>
> But then for compatibility you could make xt=nt (and then it will be a
> sing-xt system from the user's point of view).

in my system code, data and header lives in different memory regions.
For this reason nt will always be different from xt.

For me it is also important that execute just makes a call to the address
represented by the xt. I want to be able to execute code in my forth image as
well as in a loaded library or directly in the kernel.

> Or at least FIND should return something like xt of
> [: true abort" not implemented" ;]
> and n value 1, or just n value 0, when FIND is applied to the "IF"
> string in compilation state, and ['] IF with n value 1 — in
> interpretation state.

Today find "IF" will return the the xt of [: true abort" compile only" ;]
in both interpret and compile state.

I could of course make IF reverse immediate, that is have as execution
semantics its compilation semantics and return the immediate flag.
That would make at least IF and friends work.

If I chose to implement a state-smart FIND as you suggest can it also return
different flags in interpretation and compilation state?

To efficiently compile a constant I need the xt of the constant and the xt of literal
Could I return in compile state for a constant a temporary xt that points
to code that calls the constant and then calls literal?

> >> In my recent iteration I introduced the word FIND-SEM
> >>
> >> FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )
> >>
> >> It is closer to classic "FIND", with only difference that it accepts a
> >> string as the pair (c-addr u), and returns more convenient flag-special,
> >> rather than -1|1 code.
> >
> > But that would not let me find both xts at the same time
> Do you talk about your system or a program?
>
> In your system you are free to use system-specific methods.
>
> A standard program have to call FIND-SEM twice (in two different states)
> for that — the same as with FIND.
> >> And why do you need "FIND-POSTPONE"?
> >
> > I do not at the moment and it is not implemented. But might be
> > useful for recognizers. It might serve to do words with non default
> > postpone semantics (NDPS words!)
> >
> > That was a joke!
> Oh, I see. After all, NDPS is non standard.
> >>>>> If you can implement this, nothing prevent you to implement STATE
> >>>>> dependent FIND
> >>>> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
> >>>> either "EXECUTE" or "COMPILE," and nothing else.
> >>>
> >>> To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
> >>> limit the possibilities of the system. Take for example a constant.
> >>> XT2 could be the actual value of the constant and XT1 LITERAL for compiling.
> >> It looks like you reinvent recognizers under the different name.
> >
> > Yes, just 20 years ago!
> Then why don't use recognizers for such things?
>
> As Anton pointed out, above approach is possible for single-cell numbers
> only, when recognizers are suitable for any literals.
> >>> My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
> >>> to a JIT compiler that compiles the tokens to native code and then executes it
> >> Why cannot EXECUTE be who calls a JIT compiler (if a better code is not
> >> generated yet) and then executes this better code?
> >
> > Yes that is how it can work. ON the stack will be
> >
> > NT JIT-compiler, coming from find-interpret
> I mean that EXECUTE behaves in the following manner:
>
> : execute ( i*x xt -- j*x )
> xt>oxt if ( oxt ) oexecute exit then ( xt )
> ocompiled ( oxt ) oexecute
> ;

AS mentioned above I value an execute that can be applied to any address of code

> > I am not there yet. Now I can call native code from the token code. But I also want to
> > do the reverse.
>
>
>
> >
> >>>> Otherwise FIND cannot be implemented via these words.
> >>>
> >>> There is no requirements to implement FIND with these words.
> >> For back-compatibility, the system should still provide FIND for a
> >> while. And it should be clear how to implement FIND via new methods in
> >> the case when a system doesn't provide FIND (to use programs/libraries
> >> that rely on FIND).
> >
> > I do provide a FIND in my systems.
> When you propose an alternative to FIND, it's not about your system.
> It's about the standard and the Forth ecosystem.
> > In lxf it is not possible to do a user defined
> > interpreter with it, for 2 reasons. It is not state-smart. It only returns the
> > interpretation XT. Words like IF with no defined interpretation semantics
> > return an xt that print an error message and aborts. If needed i could change this
> > but so far has not needed it.
> As I understand, the second reason is that lxf cannot provide an xt that
> identifies the following execution semantics:
> [: postpone if ;]

That is right via FIND it can not do that. As explained above I can change IF to
provide that xt from FIND

> >> A one shortcoming of NAME>* methods is that they don't guarantee that
> >> FIND can be implementable via them. Moreover, FIND is not implementable
> >> via these methods in a single-xt system, if this system allows to
> >> perform compilation semantics in interpretation state.
> >
> > My lxf64 is still single-xt, FIND works just fine there. There is no requirement
> > that find must be implemented by NAME>* methods
> >
> >>>>>
> >>>>> But what for do you need FIND-POSTPONE ?
> >>>>>
> >>>>>
> >>>>>> The result if found just needs to be executed to perform the
> >>>>>> respective actions
> >>>>>> It also hides the implementation details ( single-xt, dual-xt, dual
> >>>>>> headers etc)
> >>>>>
> >>>>> It doesn't cover sing-xt systems. In these systems the corresponding
> >>>>> action should be performed only in the corresponding STATE.
> >>>> I missed that they return two xt-s. Then yes, they cover single-xt
> >>>> systems too.
> >>>>
> >>>> But then what is a way to detect whether a word is an ordinary word?
> >>>
> >>> That is not needed!
> >> I need that.
> >>
> >> See for exampe: https://forth-standard.org/standard/tools#reply-445
> >>
> >> Another example. A kind of portable SYNONYM should be able to detected
> >> whether a word is an ordinary word, or not.
> >
> > I provide a completely compliant SYNONYM. Different NTs but same XTs
> > There is no need to load another synonym.
> Again, you talk about your system. But I talk about Forth ecosystem, and
> broad variety of Forth systems.
>
> If a standard system doesn't provide SYNONYM but provide a full-fledged
> FIND, then SYNONYM (that preserves execution semantics as well as IS and
> CS) can be implemented.
> >>>> And what is a way to get xt of an ordinary word, that identifies the
> >>>> execution semantics for this word?
> >>>
> >>> ' and ['] are still available for this.
> >> They cannot be applied to any word. They cannot be used to words of
> >> unknown kind.
> >
> > I do not understand this. Any word that has an XT could be ticked, or?
> Tick returns xt that identifies execution semantics (ES) for the name in
> argument, but ES are not specified for many standard words.
>
> Therefore, 4.1.2 says that an ambiguous condition exists if a program
> attempts to obtain the execution token of a definition with undefined
> interpretation semantics (for all such cases ES are either unspecified,
> or cannot be provided in many implementations).
>
> Concerning other words with unspecified ES, it says that applying Tick
> to the word is ambiguous, individually for each such word except FILE S"
> and FILE S\".
>
> My proposal is to remove individual cases and declare the single rule:
> an ambiguous condition exists if a program attempts to obtaining the
> execution token of a definition with non default interpretation semantics..
>
> NB: my proposal of FIND clarification explicitly specifies in what case
> the returned xt identifies ES for the name/word in argument.

On 2021-05-11 18:50, Stephen Pelc wrote:
> On Fri, 7 May 2021 14:12:24 +0300, Ruvim <ruvim.pinka@gmail.com>
> wrote:
>
>>> In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
>>> of COMPILE, for such words is NDCS,
>>>
[...]
>> Taking into account that "ndcs," just performs another xt:
>>
>> : (ndcs,) \ i*x xt -- j*x
>> >code-len @ execute \ execute NDCS/IMMEDIATE word
>> ;
>>
>> Why not return this xt by "find" ?
>>
>> : find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
>> find dup 0= if exit then drop ( xt1 ) dup
>> dup ndcs? if >code-len @ 1 else
>> dup immediate? if 1 else -1 then then ( xt1 xt2 n )
>> state @ if rot else swap then drop
>> ;
>>
>> After this fix all my tests work as expected.
>
> The fix is the sensible one, which is to throw away IMMEDIATE and
> let FIND indicate NDCS and normal.

Yes. It just doesn't distinguish immediate and other NDCS words.
But a program can still distinguish them in some cases.

Perhaps a better variant for VFX

: find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
find dup 0= if exit then drop ( xt1 )
dup immediate? if 1 exit then
dup ndcs? 0= if -1 exit then
state @ if >code-len @ then 1
;

(to explicitly prefer primary xt in the case of an immediate word)

> However, on some systems this will be a code breaker.

A system is free to use any system-specific things.

If you mean that some programs can be incompatible with such FIND —
probably such a program is not standard since it doesn't take into
account that FIND may return the different xt in the different STATE.

> Personally, and for MPE, I love the original idea of an xt that it
> be the single point of contact between the user and the word. An
> xt can be used for many purposes other than compilation and
> interpretation, e.g. LOCATE XREF and so on.

I don't see how can a user use an xt with LOCATE or XREF

Nevertheless, it looks like in my edition above FIND still returns usual
system xts.

And the following program

: ]execute[ ] execute postpone [ ;

bl word if ' find ]execute[ . ( xt-if-comp )

also returns a usual xt.

You can say that ">name" does not work for this xt. But it isn't
obligated. ">name" also doesn't work for anonymous definitions, e.g.:

:noname 123 . ; >name .name

> Use of xt in this way requires that all words have a primary or
> interpretation action, which the xt represents.
>
> Your fix breaks this and I suppose makes it non-standard.

How does my fix break this?

All words in VFX still have a primary action that an xt represents.

> Because of SYNONYM and friends, there is an argument that the
> unique token for a word is the name token.

nt=xt is also a possible approach.

> Regardless of nt or xt, I see no way of dealing with NDCS words
> without upgrading the standard.

Could you please clarify, what cannot a standard program do without
special words to handle NDCS words?

--
Ruvim

On 5/11/21 12:42 AM, dxforth wrote:

> I couldn't find a single app I'd written that used FIND

I have an OOP extension that relies on FIND for knowing if a message
(name) is already defined. The user of the extension needs this
functionality as he/she writes new classes with their message names.

-Doug

On Tuesday, May 11, 2021 at 9:16:23 AM UTC-7, Stephen Pelc wrote:
> For VFX, all words have xts which are accessible by the user.

This is not compatible with gForth that has words such as semicolon that
do not have XTs and for tick will abort with a bizarre error message
(FIND also fails, IIRC).

Also, your FIND is not compatible with SwiftForth because it says that words
such as IF are non-immediate and SwiftForth says that these words are immediate.

All of this ambiguity in ANS-Forth is fixed by the disambiguifiers.
Why don't you just tell the truth and admit this?
You lied about this previously:
https://groups.google.com/g/comp.lang.forth/c/T-yYkpVwYew/m/C6uvd8djAgAJ
Do you intend to continue to lie about this forever?
Eventually the truth catches up to you!

On 2021-05-11 13:51, Anton Ertl wrote:
[...]
> 3) We have discussed POSTPONE repeatedly over the last
> quarter-century, and I have established repeatedly that according to
> the text of Forth-94 and Forth-2012
>
> : foo postpone . ;
> : bar [ foo ] ;
> 1 bar \ prints "1"
>
> conforms to these standards.

According to the only wording in the specification for POSTPONE this
code conforms the standard, and it should print "1".

But according to the official TC reply to RFI Q99-027, it's not
guaranteed that performing "foo" in interpretation state performs
compilation semantics for "." (dot), and then this code isn't compliant.

>> Is there a demonstrated need that it should
>> work that would justify amending or clarifying ANS' ruling?
>
> There is no need to amend or clarify.
> The text of the standard is clear.

But RFI Q0009 and RFI Q99-027 demonstrate that clarifications were
needed in regard to POSTPONE. It means that the text of the standard is
not enough clear.

--
Ruvim

On 2021-05-11 19:46, P Falth wrote:
> On Tuesday, 11 May 2021 at 16:05:10 UTC+2, Ruvim wrote:
>> On 2021-05-11 00:16, P Falth wrote:
>>> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
>>>> On 2021-05-06 18:23, P Falth wrote:
>>>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
[...]

>>> For some
>>> words ( normal words, created words) at compiletime was also needed
>>> the interpretation xt. FIND could not give me this. From this was born
>>> FIND-INTERPRET and FIND-COMPILE. They do what I needed!
>>
>> I see. It's OK for internal or system-specific purposes.
>>
>> But then for compatibility you could make xt=nt (and then it will be a
>> sing-xt system from the user's point of view).
>
> in my system code, data and header lives in different memory regions.
> For this reason nt will always be different from xt.

I mean, from the user's point of view.

In this approach a system itself uses raw xts (machine code references)
under the hood. But "FIND", "TRAVERSE-WORDLIST", "FIND-NAME", Tick —
any of them returns the same value for given word (i.e., the xt from
Tick is equal to the nt from FIND-NAME for any given word). Then for
anonymous definitions a header should be created too.

> For me it is also important that execute just makes a call to the address
> represented by the xt. I want to be able to execute code in my forth image as
> well as in a loaded library or directly in the kernel.

Since this code is not portable anyway, it could use special set of
words, i.e., special CALL-RAW (or alike), etc.

>> Or at least FIND should return something like xt of
>> [: true abort" not implemented" ;]
>> and n value 1, or just n value 0, when FIND is applied to the "IF"
>> string in compilation state, and ['] IF with n value 1 — in
>> interpretation state.
>
> Today find "IF" will return the the xt of [: true abort" compile only" ;]
> in both interpret and compile state.

Well, this message is just quite confusing when "IF" is actually
processed in compilation state.

> I could of course make IF reverse immediate, that is have as execution
> semantics its compilation semantics and return the immediate flag.

"FIND" is not obligated to return in compilation state an xt that
identifies execution semantics for the word in argument.

But if you do that, then let (1)
' IF EXECUTE
be equivalent to
IF
in interpretation state, to meet the common expectations.

And then, either IS for "IF" will be to perform CS for "IF", or the ES
will be STATE-dependent — any variant is possible.

(1) This code is not standard since ticking "IF" is ambiguous, and an
implementer is free to provide any behavior. But it's better if it meets
the common pattern.

> That would make at least IF and friends work.

Yes, it makes the system compliant with A.6.2.0945 and possibly updated
specification for FIND.

> If I chose to implement a state-smart FIND as you suggest can it also return
> different flags in interpretation and compilation state?

Yes, it may.

For example, for "DUP" it may return -1 in interpretation state (since
the returned xt is the execution token for "DUP"), and 1 in compilation
state (since the returned xt is for a definition that compiles optimized
code for "DUP").

For "EXIT" it may return 1 in interpretation state (to report an error),
and -1 in compilation state (since it's compiled by "COMPILE,").

For "IF" it may return 0 in interpretation state (not found), and 1 in
compilation state (since the returned xt is for a definition that
performs compilation semantics for "IF").

> To efficiently compile a constant I need the xt of the constant and the xt of literal
> Could I return in compile state for a constant a temporary xt that points
> to code that calls the constant and then calls literal?

In all the cases, when some data objects are transient, the standard
explicitly declares this fact, and specifies lifetime duration.

For example, see FILE 'S"', sections 11.3.4, 3.3.3.6.

So, if the lifetime for an object is not specified, then it's not less
than the program (or system) running. Or at least, a program should not
be able to detect that an object is invalid (i.e., a system may use
garbage collection under the hood).

An object code for a constant (regardless whether it's a machine code or
a threaded code) is very simple, and usually it can be easily detected.
I.e., having an xt, you can detect whether it's a constant, and if yes,
get it's value (a number). Having the value, you can efficiently compile it.

Actually, it's a peephole optimization that "COMPILE," can perform.

So, I see three options for your case that are compliant to the standard:

A. Introduce peephole optimization inside "COMPILE,".

B. Make a high-level xt be equal to an nt (then "COMPILE," will know
from the header that it's a constant).

C. Generate quotations (also known as "thunk") on the fly (with or
without garbage collection).

[...]
>>>>>> And what is a way to get xt of an ordinary word, that identifies the
>>>>>> execution semantics for this word?
>>>>>
>>>>> ' and ['] are still available for this.
>>>> They cannot be applied to any word. They cannot be used to words of
>>>> unknown kind.
>>>
>>> I do not understand this. Any word that has an XT could be ticked, or?
>>>
>> Tick returns xt that identifies execution semantics (ES) for the name in
>> argument, but ES are not specified for many standard words.
>>
>> Therefore, 4.1.2 says that an ambiguous condition exists if a program
>> attempts to obtain the execution token of a definition with undefined
>> interpretation semantics (for all such cases ES are either unspecified,
>> or cannot be provided in many implementations).
>>
>> Concerning other words with unspecified ES, it says that applying Tick
>> to the word is ambiguous, individually for each such word except FILE S"
>> and FILE S\".
>>
>> My proposal is to remove individual cases and declare the single rule:
>> an ambiguous condition exists if a program attempts to obtaining the
>> execution token of a definition with non default interpretation semantics.
>>
>> NB: my proposal of FIND clarification explicitly specifies in what case
>> the returned xt identifies ES for the name/word in argument.

>
> Another proposal could be to specify the ES for these words to be the CS

I can guess, it unnecessarily constrains implementations.

Also, taking into account that a system is allowed to provide IS for
such words (see A.3.4.3.2),

" ' X EXECUTE " will *not* be equivalent to " X " in interpretation
state for such word X, that is quite unexpected.

--
Ruvim

On 2021-05-12 12:10, Ruvim wrote:
> On 2021-05-11 19:46, P Falth wrote:
[...]
>> To efficiently compile a constant I need the xt of the constant and
>> the xt of literal
>> Could I return in compile state for a constant a temporary xt that points
>> to code that calls the constant and then calls literal?
>
> In all the cases, when some data objects are transient, the standard
> explicitly declares this fact, and specifies lifetime duration.
>
> For example, see FILE 'S"', sections 11.3.4, 3.3.3.6.
>
> So, if the lifetime for an object is not specified, then it's not less
> than the program (or system) running. Or at least, a program should not
> be able to detect that an object is invalid (i.e., a system may use
> garbage collection under the hood).
>
>
>
> An object code for a constant (regardless whether it's a machine code or
> a threaded code) is very simple, and usually it can be easily detected.
> I.e., having an xt, you can detect whether it's a constant, and if yes,
> get it's value (a number). Having the value, you can efficiently compile
> it.
>
> Actually, it's a peephole optimization that "COMPILE," can perform.
>
>
>
> So, I see three options for your case that are compliant to the standard:
>
>   A. Introduce peephole optimization inside "COMPILE,".
>
>   B. Make a high-level xt be equal to an nt (then "COMPILE," will know
> from the header that it's a constant).
>
>   C. Generate quotations (also known as "thunk") on the fly (with or
> without garbage collection).

D. Generate the static quotation when a constant is created.

--
Ruvim

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2021-05-11 13:51, Anton Ertl wrote:
>[...]
>> 3) We have discussed POSTPONE repeatedly over the last
>> quarter-century, and I have established repeatedly that according to
>> the text of Forth-94 and Forth-2012
>>
>> : foo postpone . ;
>> : bar [ foo ] ;
>> 1 bar \ prints "1"
>>
>> conforms to these standards.
>
>According to the only wording in the specification for POSTPONE this
>code conforms the standard, and it should print "1".
>
>But according to the official TC reply to RFI Q99-027, it's not
>guaranteed that performing "foo" in interpretation state performs
>compilation semantics for "." (dot), and then this code isn't compliant.

>>> Is there a demonstrated need that it should
>>> work that would justify amending or clarifying ANS' ruling?
>>
>> There is no need to amend or clarify.
>> The text of the standard is clear.
>
>But RFI Q0009 and RFI Q99-027 demonstrate that clarifications were
>needed in regard to POSTPONE. It means that the text of the standard is
>not enough clear.

Actually neither of these RFIs states that the text of the standard is
unclear. On the contrary, Q0009 states that it is clear:

<http://forth.sourceforge.net/std/dpans/q0009.htm>:
|Nothing in the standard prevents you executing words with compilation
|semantics from POSTPONE in interpretation state (it does not say that
|it is ambiguous, so it looks like it is defined through the standard).

The reason for these RFIs is that enough people made claims that did
not agree with the text of the standard that the people who asked the
question wanted an official statement by the TC:

<http://forth.sourceforge.net/std/dpans/q0009.htm>:
|IMHO the confusion about this topic has reached a point where I need
|an official statement.
[...]
|Opinions: Elizabeth Rather says that compilation semantics requires
|to be in compilation state.

<http://forth.sourceforge.net/std/dpans/a99-027.txt>:
|Recent discussions on comp.lang.forth indicate there is a body of opinion
|which holds that one or both the following are implied by the ANSI Forth
|Standard:
| | 1. It is an ambiguous condition for a program to perform compilation
|semantics in the interpretation state.
| | 2. It is an ambiguous condition for a program to append semantics to the
|current definition in the interpretation state.

The TC never answered Q0009. The TC answered Q99-027 without
referring to the document at all, only to the TC's intent; instead of
referring to the document, they stated that they have "plans to modify
its terminology in an effort to avoid future misinterpretations of
this sort [i.e., of TC intent]." They never did. Nobody has proposed
such a modification for Forth-200x (not before the release of
Forth-2012 and not since). With every year of inaction the relevance
of a99-027 became less, and with the release of Forth-2012 its
relevance vanished completely.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

On Wednesday, 12 May 2021 at 11:10:40 UTC+2, Ruvim wrote:
> On 2021-05-11 19:46, P Falth wrote:
> > On Tuesday, 11 May 2021 at 16:05:10 UTC+2, Ruvim wrote:
> >> On 2021-05-11 00:16, P Falth wrote:
> >>> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
> >>>> On 2021-05-06 18:23, P Falth wrote:
> >>>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> [...]
> >>> For some
> >>> words ( normal words, created words) at compiletime was also needed
> >>> the interpretation xt. FIND could not give me this. From this was born
> >>> FIND-INTERPRET and FIND-COMPILE. They do what I needed!
> >>
> >> I see. It's OK for internal or system-specific purposes.
> >>
> >> But then for compatibility you could make xt=nt (and then it will be a
> >> sing-xt system from the user's point of view).
> >
> > in my system code, data and header lives in different memory regions.
> > For this reason nt will always be different from xt.
> I mean, from the user's point of view.
>
> In this approach a system itself uses raw xts (machine code references)
> under the hood. But "FIND", "TRAVERSE-WORDLIST", "FIND-NAME", Tick —
> any of them returns the same value for given word (i.e., the xt from
> Tick is equal to the nt from FIND-NAME for any given word). Then for
> anonymous definitions a header should be created too.

I know that approach and I do not like it. A header for a noname definition
just sound like the wrong way to go

> > For me it is also important that execute just makes a call to the address
> > represented by the xt. I want to be able to execute code in my forth image as
> > well as in a loaded library or directly in the kernel.
> Since this code is not portable anyway, it could use special set of
> words, i.e., special CALL-RAW (or alike), etc.
> >> Or at least FIND should return something like xt of
> >> [: true abort" not implemented" ;]
> >> and n value 1, or just n value 0, when FIND is applied to the "IF"
> >> string in compilation state, and ['] IF with n value 1 — in
> >> interpretation state.
> >
> > Today find "IF" will return the the xt of [: true abort" compile only" ;]
> > in both interpret and compile state.
> Well, this message is just quite confusing when "IF" is actually
> processed in compilation state.
> > I could of course make IF reverse immediate, that is have as execution
> > semantics its compilation semantics and return the immediate flag.
> "FIND" is not obligated to return in compilation state an xt that
> identifies execution semantics for the word in argument.
>
> But if you do that, then let (1)
> ' IF EXECUTE
> be equivalent to
> IF
> in interpretation state, to meet the common expectations.

> And then, either IS for "IF" will be to perform CS for "IF", or the ES
> will be STATE-dependent — any variant is possible.
>
>
> (1) This code is not standard since ticking "IF" is ambiguous, and an
> implementer is free to provide any behavior. But it's better if it meets
> the common pattern.
> > That would make at least IF and friends work.
> Yes, it makes the system compliant with A.6.2.0945 and possibly updated
> specification for FIND.
> > If I chose to implement a state-smart FIND as you suggest can it also return
> > different flags in interpretation and compilation state?
> Yes, it may.
>
> For example, for "DUP" it may return -1 in interpretation state (since
> the returned xt is the execution token for "DUP"), and 1 in compilation
> state (since the returned xt is for a definition that compiles optimized
> code for "DUP").
>
> For "EXIT" it may return 1 in interpretation state (to report an error),
> and -1 in compilation state (since it's compiled by "COMPILE,").
>
> For "IF" it may return 0 in interpretation state (not found), and 1 in
> compilation state (since the returned xt is for a definition that
> performs compilation semantics for "IF").
> > To efficiently compile a constant I need the xt of the constant and the xt of literal
> > Could I return in compile state for a constant a temporary xt that points
> > to code that calls the constant and then calls literal?
> In all the cases, when some data objects are transient, the standard
> explicitly declares this fact, and specifies lifetime duration.
>
> For example, see FILE 'S"', sections 11.3.4, 3.3.3.6.
>
> So, if the lifetime for an object is not specified, then it's not less
> than the program (or system) running. Or at least, a program should not
> be able to detect that an object is invalid (i.e., a system may use
> garbage collection under the hood).
>
>
>
> An object code for a constant (regardless whether it's a machine code or
> a threaded code) is very simple, and usually it can be easily detected.
> I.e., having an xt, you can detect whether it's a constant, and if yes,
> get it's value (a number). Having the value, you can efficiently compile it.
>
> Actually, it's a peephole optimization that "COMPILE," can perform.
>
>
>
> So, I see three options for your case that are compliant to the standard:
>
> A. Introduce peephole optimization inside "COMPILE,".
>
> B. Make a high-level xt be equal to an nt (then "COMPILE," will know
> from the header that it's a constant).
>
> C. Generate quotations (also known as "thunk") on the fly (with or
> without garbage collection).

There is also the option to just compile a call to the constant.
Find provides the xt and -1 this xt can then be compiled with compile,

The code generated will be less efficient but it will work.

I have looked thru all my sources. FIND is never used. in other peoples
sources I have I see it used some time to define [DEFINED] . For
this purpose my state dump find will work just fine.
This also means that I can test a state-smart find without breaking my system!

But what will I actually gain from it?

BR
Peter

>
> [...]
> >>>>>> And what is a way to get xt of an ordinary word, that identifies the
> >>>>>> execution semantics for this word?
> >>>>>
> >>>>> ' and ['] are still available for this.
> >>>> They cannot be applied to any word. They cannot be used to words of
> >>>> unknown kind.
> >>>
> >>> I do not understand this. Any word that has an XT could be ticked, or?
> >>>
> >> Tick returns xt that identifies execution semantics (ES) for the name in
> >> argument, but ES are not specified for many standard words.
> >>
> >> Therefore, 4.1.2 says that an ambiguous condition exists if a program
> >> attempts to obtain the execution token of a definition with undefined
> >> interpretation semantics (for all such cases ES are either unspecified,
> >> or cannot be provided in many implementations).
> >>
> >> Concerning other words with unspecified ES, it says that applying Tick
> >> to the word is ambiguous, individually for each such word except FILE S"
> >> and FILE S\".
> >>
> >> My proposal is to remove individual cases and declare the single rule:
> >> an ambiguous condition exists if a program attempts to obtaining the
> >> execution token of a definition with non default interpretation semantics.
> >>
> >> NB: my proposal of FIND clarification explicitly specifies in what case
> >> the returned xt identifies ES for the name/word in argument.
>
> >
> > Another proposal could be to specify the ES for these words to be the CS
> I can guess, it unnecessarily constrains implementations.
>
> Also, taking into account that a system is allowed to provide IS for
> such words (see A.3.4.3.2),
>
> " ' X EXECUTE " will *not* be equivalent to " X " in interpretation
> state for such word X, that is quite unexpected.
>
>
> --
> Ruvim

On 2021-05-12 13:58, P Falth wrote:
> On Wednesday, 12 May 2021 at 11:10:40 UTC+2, Ruvim wrote:
>> On 2021-05-11 19:46, P Falth wrote:
>>> On Tuesday, 11 May 2021 at 16:05:10 UTC+2, Ruvim wrote:
>>>> On 2021-05-11 00:16, P Falth wrote:
>>>>> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
>>>>>> On 2021-05-06 18:23, P Falth wrote:
>>>>>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
>> [...]
>>>>> For some
>>>>> words ( normal words, created words) at compiletime was also needed
>>>>> the interpretation xt. FIND could not give me this. From this was born
>>>>> FIND-INTERPRET and FIND-COMPILE. They do what I needed!
>>>>
>>>> I see. It's OK for internal or system-specific purposes.
>>>>
>>>> But then for compatibility you could make xt=nt (and then it will be a
>>>> sing-xt system from the user's point of view).
>>>
>>> in my system code, data and header lives in different memory regions.
>>> For this reason nt will always be different from xt.
>>>
>> I mean, from the user's point of view.
>>
>> In this approach a system itself uses raw xts (machine code references)
>> under the hood. But "FIND", "TRAVERSE-WORDLIST", "FIND-NAME", Tick —
>> any of them returns the same value for given word (i.e., the xt from
>> Tick is equal to the nt from FIND-NAME for any given word). Then for
>> anonymous definitions a header should be created too.
>
> I know that approach and I do not like it. A header for a noname definition
> just sound like the wrong way to go

I agree with you.
But this approach is standard-compliant too, so it should be mentioned.

[...]

>> So, I see three options for your case that are compliant to the standard:
>>
>> A. Introduce peephole optimization inside "COMPILE,".
>>
>> B. Make a high-level xt be equal to an nt (then "COMPILE," will know
>> from the header that it's a constant).
>>
>> C. Generate quotations (also known as "thunk") on the fly (with or
>> without garbage collection).
>
> There is also the option to just compile a call to the constant.

(I meant the only options that support optimization)

> Find provides the xt and -1 this xt can then be compiled with compile,
>
> The code generated will be less efficient but it will work.

Yes, it's also the simplest variant.

Something similar is described in A.6.2.0945 in length, and it also says:

| "COMPILE," might not generate code that is
| as efficient as normally compiled code.

> I have looked thru all my sources. FIND is never used. in other peoples
> sources I have I see it used some time to define [DEFINED] . For
> this purpose my state dump find will work just fine.
> This also means that I can test a state-smart find without breaking my system!
>
> But what will I actually gain from it?

As an implementer — nothing direct gain.

The users of your system can use some portable libraries and programs
that rely on FIND. Or the Forth users in general can choose your system
in more cases.

For example, at the moment I cannot use your system, and one of the
issues is that a user-defined text interpreter doesn't work.

--
Ruvim

On Wednesday, 12 May 2021 at 13:42:46 UTC+2, Ruvim wrote:
> On 2021-05-12 13:58, P Falth wrote:
> > On Wednesday, 12 May 2021 at 11:10:40 UTC+2, Ruvim wrote:
> >> On 2021-05-11 19:46, P Falth wrote:
> >>> On Tuesday, 11 May 2021 at 16:05:10 UTC+2, Ruvim wrote:
> >>>> On 2021-05-11 00:16, P Falth wrote:
> >>>>> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
> >>>>>> On 2021-05-06 18:23, P Falth wrote:
> >>>>>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> >> [...]
> >>>>> For some
> >>>>> words ( normal words, created words) at compiletime was also needed
> >>>>> the interpretation xt. FIND could not give me this. From this was born
> >>>>> FIND-INTERPRET and FIND-COMPILE. They do what I needed!
> >>>>
> >>>> I see. It's OK for internal or system-specific purposes.
> >>>>
> >>>> But then for compatibility you could make xt=nt (and then it will be a
> >>>> sing-xt system from the user's point of view).
> >>>
> >>> in my system code, data and header lives in different memory regions.
> >>> For this reason nt will always be different from xt.
> >>>
> >> I mean, from the user's point of view.
> >>
> >> In this approach a system itself uses raw xts (machine code references)
> >> under the hood. But "FIND", "TRAVERSE-WORDLIST", "FIND-NAME", Tick —
> >> any of them returns the same value for given word (i.e., the xt from
> >> Tick is equal to the nt from FIND-NAME for any given word). Then for
> >> anonymous definitions a header should be created too.
> >
> > I know that approach and I do not like it. A header for a noname definition
> > just sound like the wrong way to go
> I agree with you.
> But this approach is standard-compliant too, so it should be mentioned.
>
>
>
>
>
> [...]
> >> So, I see three options for your case that are compliant to the standard:
> >>
> >> A. Introduce peephole optimization inside "COMPILE,".
> >>
> >> B. Make a high-level xt be equal to an nt (then "COMPILE," will know
> >> from the header that it's a constant).
> >>
> >> C. Generate quotations (also known as "thunk") on the fly (with or
> >> without garbage collection).
> >
> > There is also the option to just compile a call to the constant.
> (I meant the only options that support optimization)
> > Find provides the xt and -1 this xt can then be compiled with compile,
> >
> > The code generated will be less efficient but it will work.
> Yes, it's also the simplest variant.
>
> Something similar is described in A.6.2.0945 in length, and it also says:
>
> | "COMPILE," might not generate code that is
> | as efficient as normally compiled code.
> > I have looked thru all my sources. FIND is never used. in other peoples
> > sources I have I see it used some time to define [DEFINED] . For
> > this purpose my state dump find will work just fine.
> > This also means that I can test a state-smart find without breaking my system!
> >
> > But what will I actually gain from it?
> As an implementer — nothing direct gain.
>
> The users of your system can use some portable libraries and programs
> that rely on FIND. Or the Forth users in general can choose your system
> in more cases.
>
> For example, at the moment I cannot use your system, and one of the
> issues is that a user-defined text interpreter doesn't work.

There is another problem waiting for a user defined text interpreter and that
is locals. They will not be found by FIND ( or find-name, search-wordlist etc)
and they do not have an xt. They are stored in a specific table that is setup
and destroyed at : and ;

BR
Peter

>
> --
> Ruvim

On 11/05/2021 20:49, Ruvim wrote:
> On 2021-05-11 07:42, dxforth wrote:
>> On 10/05/2021 23:58, Stephen Pelc wrote:
> [...]
>>> If we do not have standard ways (words) to handle NDCS words, we
>>> cannot have standard tests. Now that VFX Forth handles IF and other
>>> words as NDCS words, we need to add NDCS handling words to the
>>> standard.
>>
>> Standard tests for what purpose?  First one must define the
>> standard need i.e. why does FIND used in situation X need to
>> produce the same result on every system?  ANS' premise in
>> permitting implementers to do as they pleased was that not
>> every situation required standardizing.
>
> Yes. But every standard word should be enough specified to be useful.
>
>
>> Were they wrong?
>
> The specification for the FIND word is not sufficient.
>
> A.6.2.0945 says:
> The intention is that COMPILE, can be used as follows
> to write the classic interpreter/compiler loop:
>
> ... ( c-addr )
> FIND ?DUP IF ( xt +-1 )
> STATE @ IF ( xt +-1 )
> 0> IF EXECUTE ELSE COMPILE, THEN ( ??? )
> ELSE ( xt +-1 )
> DROP EXECUTE ( ??? )
> THEN
> ELSE ( c-addr )
> ( whatever you do for an undefined word )
> THEN
> ...
>
> Some systems implement FIND in such a way that this loop doesn't work as
> expected for the words with non default interpretation semantics.
>
>
>
>> I couldn't find a single app I'd written that used FIND :)
>
> Probably you just didn't ever need a user-defined text interpreter for
> some advanced DSL (domain specific language), or some advanced extension
> of a Forth system.

Probably those within Standard Forth didn't need it either, because if
FIND wasn't fit for purpose it would surely have been fixed by now.

On Thu, 13 May 2021 00:08:37 +1000, dxforth <dxforth@gmail.com> wrote:

>Probably those within Standard Forth didn't need it either, because if
>FIND wasn't fit for purpose it would surely have been fixed by now.

Forth has changed since the 1990s and flwas in the 1990s documents
are being revealed.

Stephen

--
Stephen Pelc, stephen@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads