I've written Forth code (below) to demonstrate keeping track of word
dependencies using a Forth system capable of defining dual-semantics
words. Here I use Gforth for the example. I believe the following may
also be doable on VFX, using its NDCS: word. The other requirements of
the system are that the name token (nt) of the last word definition is
accessible, and that there is a mechanism to define a closure.

A word definition may be immediately followed by the word TRACK-DEPS
which will set its compilation semantics to not only compile its
interpretation semantics but also to add an entry into a dependency tree
to keep track of the words into which it is compiled. For example,

: FOO ." foo" ; TRACK-DEPS

: T1 foo ;

The word SHOW-DEPS prints the names of all words in which FOO is compiled.

s" foo" SHOW-DEPS \ prints T1

To continue,

: BAR ." bar" ; TRACK-DEPS

: T2 foo bar ;

s" foo" SHOW-DEPS \ prints the list: T2 T1

s" bar" SHOW-DEPS \ prints the list: T2

With the other definitions in the example code, it should be possible to
follow the entire dependency tree branch which terminates on a word for
which TRACK-DEPS has been applied (this assumes all of the nodes leading
to the word are also being tracked).

For simplicity, the dependency tree is implemented as a fixed size
table. This is not memory efficient and flexible enough for real use,
but is suitable for a demonstration. A tree structure should be used for
actual applications. The main point is to show that being able to set
the compilation semantics for a word independently allows for
capabilities which are likely not possible (or considerably more
difficult) in Forth source on a single-xt + immediate flag system only.

--
Krishna Myneni

dependencies.4th
-----------------
\ dependencies.4th
\ \ Demonstrate tracking first-level word dependencies
\ in a dual-xt system.
\ \ K. Myneni, 2022-10-10
\ \ This program is written for Gforth. It uses the following
\ non-standard words
\ \ LATESTNT ( -- nt )
\ SET-COMPSEM ( xt -- )
\ [N:D ( x -- )

require set-compsem.fs
synonym a@ @

100 constant MAX-TRACKS
64 constant MAX-DEPS
MAX-TRACKS MAX-DEPS * cells constant MAX-DTABLE-SIZE

create DEPENDENCY-TABLE MAX-DTABLE-SIZE allot
DEPENDENCY-TABLE MAX-DTABLE-SIZE erase

: th-track ( u -- a )
DEPENDENCY-TABLE swap MAX-DEPS * cells + ;

0 value curr-track

: get-track ( nt -- n ) \ return -1 if not found
curr-track 0 ?DO
I th-track a@ \ nt nt-track
over = IF drop I UNLOOP EXIT THEN
LOOP
drop -1 ;

: track-dependency ( nt -- )
curr-track th-track !
1 curr-track + to curr-track ;

: add-dependency ( nt-track nt-newdep -- )
swap get-track dup 0< IF
." Not tracking." cr abort
ELSE
th-track \ -- nt-dep a
BEGIN cell+ dup a@ 0= UNTIL !
THEN ;

\ Track first-level dependencies for last defined word
: track-deps ( -- )
latestnt dup track-dependency
[n:d dup name>interpret compile, latestnt add-dependency ;]
set-compsem ;

0 value ndeps

: what-requires? ( nt-track -- nt1 nt2 ... u | 0 )
0 to ndeps
get-track dup 0< IF
drop 0
ELSE
th-track cell+ \ -- a
BEGIN
dup a@ dup \ -- a nt|0 nt|0
WHILE
swap
ndeps 1+ to ndeps
cell+
REPEAT
2drop ndeps
THEN ;

: show-deps ( caddr u -- )
find-name ?dup IF
what-requires?
0 ?DO cr name>string type LOOP cr
then ;

0 [IF] \ Demonstration

\ Define two words for which first-level deps will be tracked
: foo ." foo" ; track-deps
: bar ." bar" ; track-deps

foo cr
bar cr
s" foo" show-deps \ list of words dependent on FOO is empty
s" bar" show-deps \ list of words dependent on BAR is empty
: t1 foo ;
: t2 foo bar ;
s" foo" show-deps \ shows T1 and T2 are dependent on FOO
s" bar" show-deps \ shows T2 is dependent on BAR

[THEN]

Krishna Myneni schrieb am Montag, 10. Oktober 2022 um 17:44:24 UTC+2:
> I've written Forth code (below) to demonstrate keeping track of word
> dependencies using a Forth system capable of defining dual-semantics
> words. Here I use Gforth for the example. I believe the following may
> also be doable on VFX, using its NDCS: word. The other requirements of
> the system are that the name token (nt) of the last word definition is
> accessible, and that there is a mechanism to define a closure.
>
> A word definition may be immediately followed by the word TRACK-DEPS
> which will set its compilation semantics to not only compile its
> interpretation semantics but also to add an entry into a dependency tree
> to keep track of the words into which it is compiled. For example,
>
> : FOO ." foo" ; TRACK-DEPS
>
> : T1 foo ;
>
> The word SHOW-DEPS prints the names of all words in which FOO is compiled.
>
> s" foo" SHOW-DEPS \ prints T1
>
> To continue,
>
> : BAR ." bar" ; TRACK-DEPS
>
> : T2 foo bar ;
>
> s" foo" SHOW-DEPS \ prints the list: T2 T1
>
> s" bar" SHOW-DEPS \ prints the list: T2
>
> With the other definitions in the example code, it should be possible to
> follow the entire dependency tree branch which terminates on a word for
> which TRACK-DEPS has been applied (this assumes all of the nodes leading
> to the word are also being tracked).
>
> For simplicity, the dependency tree is implemented as a fixed size
> table. This is not memory efficient and flexible enough for real use,
> but is suitable for a demonstration. A tree structure should be used for
> actual applications. The main point is to show that being able to set
> the compilation semantics for a word independently allows for
> capabilities which are likely not possible (or considerably more
> difficult) in Forth source on a single-xt + immediate flag system only.
>
> --
> Krishna Myneni
>
>
> dependencies.4th
> -----------------
> \ dependencies.4th
> \
> \ Demonstrate tracking first-level word dependencies
> \ in a dual-xt system.
> \
> \ K. Myneni, 2022-10-10
> \
> \ This program is written for Gforth. It uses the following
> \ non-standard words
> \
> \ LATESTNT ( -- nt )
> \ SET-COMPSEM ( xt -- )
> \ [N:D ( x -- )
>
> require set-compsem.fs
> synonym a@ @
>
> 100 constant MAX-TRACKS
> 64 constant MAX-DEPS
> MAX-TRACKS MAX-DEPS * cells constant MAX-DTABLE-SIZE
>
> create DEPENDENCY-TABLE MAX-DTABLE-SIZE allot
> DEPENDENCY-TABLE MAX-DTABLE-SIZE erase
>
> : th-track ( u -- a )
> DEPENDENCY-TABLE swap MAX-DEPS * cells + ;
>
> 0 value curr-track
>
> : get-track ( nt -- n ) \ return -1 if not found
> curr-track 0 ?DO
> I th-track a@ \ nt nt-track
> over = IF drop I UNLOOP EXIT THEN
> LOOP
> drop -1 ;
>
> : track-dependency ( nt -- )
> curr-track th-track !
> 1 curr-track + to curr-track ;
>
> : add-dependency ( nt-track nt-newdep -- )
> swap get-track dup 0< IF
> ." Not tracking." cr abort
> ELSE
> th-track \ -- nt-dep a
> BEGIN cell+ dup a@ 0= UNTIL !
> THEN ;
>
> \ Track first-level dependencies for last defined word
> : track-deps ( -- )
> latestnt dup track-dependency
> [n:d dup name>interpret compile, latestnt add-dependency ;]
> set-compsem ;
>
> 0 value ndeps
>
> : what-requires? ( nt-track -- nt1 nt2 ... u | 0 )
> 0 to ndeps
> get-track dup 0< IF
> drop 0
> ELSE
> th-track cell+ \ -- a
> BEGIN
> dup a@ dup \ -- a nt|0 nt|0
> WHILE
> swap
> ndeps 1+ to ndeps
> cell+
> REPEAT
> 2drop ndeps
> THEN ;
>
> : show-deps ( caddr u -- )
> find-name ?dup IF
> what-requires?
> 0 ?DO cr name>string type LOOP cr
> then ;
>
>
> 0 [IF] \ Demonstration
>
> \ Define two words for which first-level deps will be tracked
> : foo ." foo" ; track-deps
> : bar ." bar" ; track-deps
>
> foo cr
> bar cr
> s" foo" show-deps \ list of words dependent on FOO is empty
> s" bar" show-deps \ list of words dependent on BAR is empty
> : t1 foo ;
> : t2 foo bar ;
> s" foo" show-deps \ shows T1 and T2 are dependent on FOO
> s" bar" show-deps \ shows T2 is dependent on BAR
>
> [THEN]

FWIW I use a dependency table for recursive elimination of
dead words (ie those not needed for an application).

Dual semantic words, f.ex. here defined in 'my private Idaho syntax':
: MYWORD ." interpreting myword" ;
: [MYWORD] ." compiling myword" ; ' [MYWORD] COMPILES MYWORD
\ only for clarification, [MYWORD] is usually a :NONAME

Compilation of these 2 words sets dependency flags 'on the fly'
in dependency table columns for
MYWORD <- ." [MYWORD]
[MYWORD] <- ." MYWORD
\ ." brings its own dependencies of course

This is ultra-easy for 'static' code. However parsing words or words
involving dictionary search are unpredictable.

Keeping track would mean to maintain the dependency table
as integral Forth compiler database. And it would grow over time.
To keep life easy, dead word elimination stops then and the table is
eliminated from memory.

But back to your proposed dependency tracker:
a) a.m. situation for dual-semantic parsing words applies as well?
b) what is your benefit or use case for dependency introspection?

On 10/10/22 12:26, minf...@arcor.de wrote:
> Krishna Myneni schrieb am Montag, 10. Oktober 2022 um 17:44:24 UTC+2:
>> I've written Forth code (below) to demonstrate keeping track of word
>> dependencies using a Forth system capable of defining dual-semantics
>> words.
...
> FWIW I use a dependency table for recursive elimination of
> dead words (ie those not needed for an application).
>
...
> But back to your proposed dependency tracker:
> a) a.m. situation for dual-semantic parsing words applies as well?

I don't understand the term "a.m. situation" -- please clarify.
Regarding parsing words, the following parsing word appears to work ok.

include dependencies.4th
: foo ' ; track-deps
: bar ." bar" ; track-deps
foo bar \ returns the xt-interp for bar
drop
: test foo ;
test foo \ returns the xt-interp for foo
s" foo" show-deps \ prints test

> b) what is your benefit or use case for dependency introspection?

For now, this is just a demonstration of being able to store, retrieve,
and output dependency information. It shows how to write an
introspection tool in Forth source on a dual-xt system. Actual benefits
are that if we wanted to know which words referenced a given word by
name, we can do so from within the Forth system. Note that it does not
involve searching the source code but uses directly the internal
dictionary information.

--
Krishna

On 10/10/22 12:26, minf...@arcor.de wrote:
> Krishna Myneni schrieb am Montag, 10. Oktober 2022 um 17:44:24 UTC+2:
>> I've written Forth code (below) to demonstrate keeping track of word
>> dependencies using a Forth system capable of defining dual-semantics
>> words. Here I use Gforth for the example.
...
>> \ Track first-level dependencies for last defined word
>> : track-deps ( -- )
>> latestnt dup track-dependency
>> [n:d dup name>interpret compile, latestnt add-dependency ;]
>> set-compsem ;
>>
..
> But back to your proposed dependency tracker:
> a) a.m. situation for dual-semantic parsing words applies as well?
> b) what is your benefit or use case for dependency introspection?

I'm guessing you mean immediate words. Indeed, in the version of
TRACK-DEPS shown above, the code does not work for immediate words. I
found I had to revise TRACK-DEPS as follows to make it work in Gforth
for both precedence cases:

\ Track first-level dependencies for last defined word
: track-deps ( -- )
latestnt dup track-dependency ( -- nt-track)
dup name>compile nip
['] compile, = if
[n:d dup name>interpret compile, latestnt add-dependency ;]
else
[n:d dup name>interpret execute latestnt add-dependency ;]
then
set-compsem ;

Demonstrated as follows:

include dependencies.4th ok
: foo ." foo" ; track-deps ok
: t1 foo ; ok
s" foo" show-deps
t1
ok
: bar ." bar" ; immediate track-deps ok
: t2 foo bar ; bar ok
s" bar" show-deps
t2
ok
s" foo" show-deps
t2
t1
ok

In the above FOO is an "ordinary word", and BAR is an immediate word.
The above shows that TRACK-DEPS and SHOW-DEPS work for both types of words.

I don't understand why the following simpler definition of TRACK-DEPS
does not work under Gforth (0.7.9_20220120):

: track-deps ( -- )
latestnt dup track-dependency ( -- nt-track)
[n:d dup name>compile execute latestnt add-dependency ;]
set-compsem ;

The above definition, which (I thought) should be equivalent to the
working definition shown above leads to some sort of infinite recursion
resulting in a return stack overflow in Gforth v. 0.7.9_20220120.

: foo ." foo" ; track-deps ok
: t1 foo ;
*the terminal*:3:6: error: Return stack overflow
: t1 >>>foo<<< ;

--
Krishna

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>Indeed, in the version of
>TRACK-DEPS shown above, the code does not work for immediate words. I
>found I had to revise TRACK-DEPS as follows to make it work in Gforth
>for both precedence cases:
>
>\ Track first-level dependencies for last defined word
>: track-deps ( -- )
> latestnt dup track-dependency ( -- nt-track)
> dup name>compile nip
> ['] compile, = if
> [n:d dup name>interpret compile, latestnt add-dependency ;]
> else
> [n:d dup name>interpret execute latestnt add-dependency ;]
> then
> set-compsem ;

The NAME>INTERPRETs here should be left away: You pass an xt to the
closure, not an nt. Fortunately (or unfortunately), in Gforth an xt
can be treated as an nt, so this mistake does not result in unintended
behaviour, at least for most words.

>I don't understand why the following simpler definition of TRACK-DEPS
>does not work under Gforth (0.7.9_20220120):
>
>: track-deps ( -- )
> latestnt dup track-dependency ( -- nt-track)
> [n:d dup name>compile execute latestnt add-dependency ;]
> set-compsem ;
>
>The above definition, which (I thought) should be equivalent to the
>working definition shown above leads to some sort of infinite recursion
>resulting in a return stack overflow in Gforth v. 0.7.9_20220120.
>
>: foo ." foo" ; track-deps ok
>: t1 foo ;
>*the terminal*:3:6: error: Return stack overflow
>: t1 >>>foo<<< ;

SET-COMPSEM changes what NAME>COMPILE does for the word, and the end
result is that the NAME>COMPILE EXECUTE inside the closure calls the
closure again, which calls NAME>COMPILE EXECUTE again until the return
stack overflows.

One way to avoid the problem is to do the NAME>COMPILE outside
(untested):

: track-deps ( -- )
latestnt dup track-dependency name>compile ( xt1 xt2 )
[d:d over >r execute r> latestnt add-dependency ;] set-compsem ;

Another alternative is to change NAME>COMPILE directly (with
SET->COMP) instead of using SET-COMPSEM, and call the old NAME>COMPILE
from there (untested):

: track-deps ( -- )
latestnt dup track-dependency ( nt-track )
dup >namehm @ >hm>comp @ ( nt-track old-name>comp )
[d:d swap >r execute r> latestnt add-dependency ;] set->comp ;

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: https://euro.theforth.net

On 10/12/22 11:55, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> Indeed, in the version of
>> TRACK-DEPS shown above, the code does not work for immediate words. I
>> found I had to revise TRACK-DEPS as follows to make it work in Gforth
>> for both precedence cases:
>>
>> \ Track first-level dependencies for last defined word
>> : track-deps ( -- )
>> latestnt dup track-dependency ( -- nt-track)
>> dup name>compile nip
>> ['] compile, = if
>> [n:d dup name>interpret compile, latestnt add-dependency ;]
>> else
>> [n:d dup name>interpret execute latestnt add-dependency ;]
>> then
>> set-compsem ;
>
> The NAME>INTERPRETs here should be left away: You pass an xt to the
> closure, not an nt. Fortunately (or unfortunately), in Gforth an xt
> can be treated as an nt, so this mistake does not result in unintended
> behaviour, at least for most words.
>

An nt is being passed into the closure, so the NAME>INTERPRET is needed
inside the closure. Note the DUP before NAME>COMPILE and the NIP afterwards.

>> I don't understand why the following simpler definition of TRACK-DEPS
>> does not work under Gforth (0.7.9_20220120):
>>
>> : track-deps ( -- )
>> latestnt dup track-dependency ( -- nt-track)
>> [n:d dup name>compile execute latestnt add-dependency ;]
>> set-compsem ;
>>
>> The above definition, which (I thought) should be equivalent to the
>> working definition shown above leads to some sort of infinite recursion
>> resulting in a return stack overflow in Gforth v. 0.7.9_20220120.
>>
>> : foo ." foo" ; track-deps ok
>> : t1 foo ;
>> *the terminal*:3:6: error: Return stack overflow
>> : t1 >>>foo<<< ;
>
> SET-COMPSEM changes what NAME>COMPILE does for the word, and the end
> result is that the NAME>COMPILE EXECUTE inside the closure calls the
> closure again, which calls NAME>COMPILE EXECUTE again until the return
> stack overflows.
>

It's too bad that SET-COMPSEM doesn't simply create a new code sequence
for the compilation semantics and store the corresponding xt. This is
similar to the problem with SET-OPTIMIZER. These words should not change
the behavior of "COMPILE," or of "NAME>COMPILE". Doing so makes these
standard words non-transparent to the programmer, resulting in
unexpected problems like this.

--
Krishna

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>On 10/12/22 11:55, Anton Ertl wrote:
>> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>>> Indeed, in the version of
>>> TRACK-DEPS shown above, the code does not work for immediate words. I
>>> found I had to revise TRACK-DEPS as follows to make it work in Gforth
>>> for both precedence cases:
>>>
>>> \ Track first-level dependencies for last defined word
>>> : track-deps ( -- )
>>> latestnt dup track-dependency ( -- nt-track)
>>> dup name>compile nip
>>> ['] compile, = if
>>> [n:d dup name>interpret compile, latestnt add-dependency ;]
>>> else
>>> [n:d dup name>interpret execute latestnt add-dependency ;]
>>> then
>>> set-compsem ;
>>
>> The NAME>INTERPRETs here should be left away: You pass an xt to the
>> closure, not an nt. Fortunately (or unfortunately), in Gforth an xt
>> can be treated as an nt, so this mistake does not result in unintended
>> behaviour, at least for most words.
>>
>
>An nt is being passed into the closure, so the NAME>INTERPRET is needed
>inside the closure. Note the DUP before NAME>COMPILE and the NIP afterwards.

NAME>COMPILE produces xt1 xt2; you consume xt2 in "['] COMPILE, =",
leaving xt1, which you then pass into a closure. You may expect an nt
and treat it like an nt (and it happens to work to some extent), but
conceptually it's an xt; if we changed Gforth so that an xt would not
be a valid xt, this code would break very obviously.

As it is, you may see some lossage in connection with, e.g., synonyms.
E.g., try

: a ." a" ;
synonym b a track-deps
: c b ;
s" b" show-deps

Does it show that c depends on b?

>>> I don't understand why the following simpler definition of TRACK-DEPS
>>> does not work under Gforth (0.7.9_20220120):
>>>
>>> : track-deps ( -- )
>>> latestnt dup track-dependency ( -- nt-track)
>>> [n:d dup name>compile execute latestnt add-dependency ;]
>>> set-compsem ;
>>>
>>> The above definition, which (I thought) should be equivalent to the
>>> working definition shown above leads to some sort of infinite recursion
>>> resulting in a return stack overflow in Gforth v. 0.7.9_20220120.
>>>
>>> : foo ." foo" ; track-deps ok
>>> : t1 foo ;
>>> *the terminal*:3:6: error: Return stack overflow
>>> : t1 >>>foo<<< ;
>>
>> SET-COMPSEM changes what NAME>COMPILE does for the word, and the end
>> result is that the NAME>COMPILE EXECUTE inside the closure calls the
>> closure again, which calls NAME>COMPILE EXECUTE again until the return
>> stack overflows.
>>
>
>It's too bad that SET-COMPSEM doesn't simply create a new code sequence
>for the compilation semantics and store the corresponding xt. This is
>similar to the problem with SET-OPTIMIZER. These words should not change
>the behavior of "COMPILE," or of "NAME>COMPILE". Doing so makes these
>standard words non-transparent to the programmer, resulting in
>unexpected problems like this.

If SET-OPTIMIZER would not change COMPILE,, it would not work (it
would fail to set the optimizer). For Gforth's current
implementation, if SET-COMPSEM did not change NAME>COMPILE, it would
not work (it would fail to change the compilation semantics).

But these problems are just like any other case where you
inadvertently introduce endless recursion; e.g., such endless
recursions happen easily if you try to reduce the number of primitives
by replacing them with colon definitions in a cross-compiler that
allows forward references. You just have to break the cycle, in the
present case by explicitly performing the old action of NAME>COMPILE
instead of just calling NAME>COMPILE.

Concerning SET-COMPSEM, one could imagine an extended header structure
that also contains a COMPSEM field. SET-COMPSEM then would set that
field and would change NAME>COMPILE to perform:

: compsem-name>comp ( nt -- xt1 xt2 )
>namehm @ >hmcompsem @ ['] execute ;

But that still would not help with the code above, because
NAME>COMPILE still would give you the new compilation semantics, which
would again result in endless recursion.

Another idea is to have a NAME>COMPILE-ORIGINAL and a
NAME>COMPILE-USER. SET-COMPSEM would change NAME>COMPILE-USER, but
not NAME>COMPILE-ORIGINAL. You would call NAME>COMPILE-ORIGINAL in
the code above, while NAME>COMPILE-USER would be called by system
words like the text interpreter and POSTPONE.

This would solve your immediate problem, but it would mean that your
extension would not compose with other extensions that use
SET-COMPSEM: E.g., if I have an extension that counts the number of
times the compilation semantics of this word is invoked, and then
apply your TRACK-DEPS extension to the word, the counting extension is
disabled. If the counting extension also uses NAME>COMPILE-ORIGINAL
instead of the previous NAME>COMPILE-USER action, then applying it
after TRACK-DEPS would disable TRACK-DEPS.

Therefore I think that just invoking the old NAME>COMPILE result
explicitly is the way to go. It's a little more complicated, but
composable.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: https://euro.theforth.net

On 10/13/22 10:30, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> On 10/12/22 11:55, Anton Ertl wrote:
>>> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>>>> Indeed, in the version of
>>>> TRACK-DEPS shown above, the code does not work for immediate words. I
>>>> found I had to revise TRACK-DEPS as follows to make it work in Gforth
>>>> for both precedence cases:
>>>>
>>>> \ Track first-level dependencies for last defined word
>>>> : track-deps ( -- )
>>>> latestnt dup track-dependency ( -- nt-track)
>>>> dup name>compile nip
>>>> ['] compile, = if
>>>> [n:d dup name>interpret compile, latestnt add-dependency ;]
>>>> else
>>>> [n:d dup name>interpret execute latestnt add-dependency ;]
>>>> then
>>>> set-compsem ;
>>>
>>> The NAME>INTERPRETs here should be left away: You pass an xt to the
>>> closure, not an nt. Fortunately (or unfortunately), in Gforth an xt
>>> can be treated as an nt, so this mistake does not result in unintended
>>> behaviour, at least for most words.
>>>
>>
>> An nt is being passed into the closure, so the NAME>INTERPRET is needed
>> inside the closure. Note the DUP before NAME>COMPILE and the NIP afterwards.
>
> NAME>COMPILE produces xt1 xt2; you consume xt2 in "['] COMPILE, =",
> leaving xt1, which you then pass into a closure. You may expect an nt
> and treat it like an nt (and it happens to work to some extent), but
> conceptually it's an xt; if we changed Gforth so that an xt would not
> be a valid xt, this code would break very obviously.
>
> As it is, you may see some lossage in connection with, e.g., synonyms.
> E.g., try
>
> : a ." a" ;
> synonym b a track-deps
> : c b ;
> s" b" show-deps
>
> Does it show that c depends on b?
>
>>>> I don't understand why the following simpler definition of TRACK-DEPS
>>>> does not work under Gforth (0.7.9_20220120):
>>>>
>>>> : track-deps ( -- )
>>>> latestnt dup track-dependency ( -- nt-track)
>>>> [n:d dup name>compile execute latestnt add-dependency ;]
>>>> set-compsem ;
>>>>
>>>> The above definition, which (I thought) should be equivalent to the
>>>> working definition shown above leads to some sort of infinite recursion
>>>> resulting in a return stack overflow in Gforth v. 0.7.9_20220120.
>>>>
>>>> : foo ." foo" ; track-deps ok
>>>> : t1 foo ;
>>>> *the terminal*:3:6: error: Return stack overflow
>>>> : t1 >>>foo<<< ;
>>>
>>> SET-COMPSEM changes what NAME>COMPILE does for the word, and the end
>>> result is that the NAME>COMPILE EXECUTE inside the closure calls the
>>> closure again, which calls NAME>COMPILE EXECUTE again until the return
>>> stack overflows.
>>>
>>
>> It's too bad that SET-COMPSEM doesn't simply create a new code sequence
>> for the compilation semantics and store the corresponding xt. This is
>> similar to the problem with SET-OPTIMIZER. These words should not change
>> the behavior of "COMPILE," or of "NAME>COMPILE". Doing so makes these
>> standard words non-transparent to the programmer, resulting in
>> unexpected problems like this.
>
> If SET-OPTIMIZER would not change COMPILE,, it would not work (it
> would fail to set the optimizer). For Gforth's current
> implementation, if SET-COMPSEM did not change NAME>COMPILE, it would
> not work (it would fail to change the compilation semantics).
>

In the case of standard "COMPILE," the problem is that on a dual-xt
system, "COMPILE," operating on an xt is a fundamentally incorrect way
of compiling a named definition (word). It may still be used for
compiling the xt's obtained from :NONAME definitions, quotations, or
closures; however, compilation of all word definitions should involve
the name token (nt) and NAME>COMPILE followed by EXECUTE. I understand
your "hack" in Gforth is for the purpose of maintaining backwards
compatibility.

> But these problems are just like any other case where you
> inadvertently introduce endless recursion; e.g., such endless
> recursions happen easily if you try to reduce the number of primitives
> by replacing them with colon definitions in a cross-compiler that
> allows forward references. You just have to break the cycle, in the
> present case by explicitly performing the old action of NAME>COMPILE
> instead of just calling NAME>COMPILE.
>

Does SET-COMPSEM erase all memory of the original compilation semantics,
or is the old xt-comp still available? If not, we should be able to fix
this.

> Concerning SET-COMPSEM, one could imagine an extended header structure
> that also contains a COMPSEM field. SET-COMPSEM then would set that
> field and would change NAME>COMPILE to perform:
>
> : compsem-name>comp ( nt -- xt1 xt2 )
> >namehm @ >hmcompsem @ ['] execute ;
>
> But that still would not help with the code above, because
> NAME>COMPILE still would give you the new compilation semantics, which
> would again result in endless recursion.
>

...
>
> This would solve your immediate problem, but it would mean that your
> extension would not compose with other extensions that use
> SET-COMPSEM: E.g., if I have an extension that counts the number of
> times the compilation semantics of this word is invoked, and then
> apply your TRACK-DEPS extension to the word, the counting extension is
> disabled. If the counting extension also uses NAME>COMPILE-ORIGINAL
> instead of the previous NAME>COMPILE-USER action, then applying it
> after TRACK-DEPS would disable TRACK-DEPS.
>
> Therefore I think that just invoking the old NAME>COMPILE result
> explicitly is the way to go. It's a little more complicated, but
> composable.
>

Ok, I see that there is a problem with using NAME>COMPILE within the
the closure's xt, passed to SET-COMPSEM.

NAME>COMPILE should be used outside the closure to obtain the
compilation semantics at the time that TRACK-DEPS was invoked. Thus a
permanent data structure, 3 cells long, may be passed into the closure.
The data structure contains the nt of the word, and the original
compilation semantics, which consists of two cells of data: x xt. Then
the closure can invoke both the original compilation semantics and
append the extra actions needed to add information to the dependency table.

Here's my new version of TRACK-DEPS. It invokes NAME>COMPILE outside of
the closure to obtain and store the current compilation semantics for
the word before SET-COMPSEM is invoked. It should satisfy all cases,
including preserving any extensions to the compilation semantics prior
to invoking TRACK-DEPS. I've tested it on the single-xt+immediate flag
cases. So it appears that the previous xt-comp is still valid.

: track-deps ( -- )
latestnt dup track-dependency ( -- nt-track)
dup name>compile ( -- nt-track x xt )
3 cells allocate drop \ reserve memory for old compsem
dup >r 2! r@ 2 cells + ! r>
[n:d dup 2@ execute 2 cells + a@ latestnt add-dependency ;]
set-compsem ;

--
Krishna

On 10/13/22 14:40, Krishna Myneni wrote:
> On 10/13/22 10:30, Anton Ertl wrote:

>> As it is, you may see some lossage in connection with, e.g., synonyms.
>> E.g., try
>>
>> : a ." a" ;
>> synonym b a track-deps
>> : c b ;
>> s" b" show-deps
>>
>> Does it show that c depends on b?

Yes, the revised version of TRACK-DEPS gives the following, showing that
c depends on b.

: a ." a" ; ok
synonym b a track-deps
*terminal*:59:11: warning: redefined b
locate1.fs:207:3: warning: original location ok
: c b ; ok
s" b" show-deps
c ok

>
> : track-deps ( -- )
>     latestnt dup track-dependency  ( -- nt-track)
>     dup name>compile               ( -- nt-track x xt )
>     3 cells allocate drop          \ reserve memory for old compsem
>     dup >r 2! r@ 2 cells + ! r>
>     [n:d dup 2@ execute 2 cells + a@ latestnt add-dependency ;]
>     set-compsem ;
>
>

Using the revised version of TRACK-DEPS I decided to try it out on a
real application, Ian Osgood's chess game, fcp.f (ver 1.3). I marked
most word definitions, with the exception of a few output words, using
TRACK-DEPS.

Below are some interesting results:

curr-track . 204 ok \ constant MAX-TRACKS set to 250

The number of words being tracked is 204.

s" rank" show-deps
evalDR
evalLR
evalDP
evalLP
taxicab
taxicab
evalSetupSq
evalSetupSq
sqCastleMask
sqCastleMask
updatePawnFile
updatePawnFile
genDP
genDP
genLP
genLP
genCapsDP
genCapsLP
genDPCaptures
genLPCaptures
epdSq
cRank
ok

The word RANK is referenced in the above list of words. At first I was
confused by multiple appearances of the same word, but it is simply
because the dependent word references RANK multiple times. Hence, we can
count the number of references as well for each dependent word.

Words defined by the macro :INLINE are also able to be tracked.

s" rotate" show-deps
evalDB
evalDN
evalDP
?rotate
ok

Above, we see that there are four words which reference the word ROTATE.

s" tolower" show-deps
epd
char>piece
str>sq
keyHitQ?
piece
ok

Some of the words which reference the character case lowering word
TOLOWER. This is probably not a complete list since I didn't mark every
output word with TRACK-DEPS.

s" forEveryRow" show-deps
forEverySq
forEverySq
forEverySq
forEverySq
forEverySq
forEverySq
forEverySq
forEverySq

The word forEverySq appears to be the only word to reference
forEveryRow, and it does so eight times.

s" switchPieces" show-deps
epd
undoNullMove
makeNullMove
makeMove
takeBack
ok

s" mvFrom" show-deps
reps
makeMove
makeMove
takeBack
takeBack
takeBack
updatePawnFiles
genPushCapture
mvHistory
move
epCapSq
epSq
ok

s" .move" show-deps
ex
go
bkNode
curMove
pv
makeMove
moveList
ok

s" makeMove" show-deps
redo
ex
(mv)
go
line:
noLegalMoves?
_search
quiesce
ok

s" quiesce" show-deps
_search
_search
ok

s" _search" show-deps
thinker
failLowSearch
failHighSearch
ok

etc.

Obviously we may gain considerable insight into the structure of a
program by viewing the word dependencies in this manner. It is also
possible to recurse through the dependencies to follow the branches. For
example, a section of a branch may be gleaned from the results above:

mvFrom -> makeMove -> quiesce -> _search -> thinker

mvFrom is a primitive word in the program:

:inline mvFrom $FF AND ; ( mv -- sqFrom )

and THINKER is a high-level word in the program.

This shows the utility of the dependency tracking words, which
themselves are written in Forth source on a dual-xt system.

Below, I give the full listing of the current version of dependencies.4th.

--
Krishna

dependencies.4th
----
\ dependencies.4th
\ \ Demonstrate tracking first-level word dependencies
\ in a dual-xt system.
\ \ K. Myneni, 2022-10-10
\ Revised: 2022-10-13
\ \ This program is written for Gforth. It uses the following
\ non-standard words
\ \ LATESTNT ( -- nt )
\ SET-COMPSEM ( xt -- )
\ [N:D ( x -- )

require set-compsem.fs
synonym a@ @

250 constant MAX-TRACKS
64 constant MAX-DEPS
MAX-TRACKS MAX-DEPS * cells constant MAX-DTABLE-SIZE

create DEPENDENCY-TABLE MAX-DTABLE-SIZE allot
DEPENDENCY-TABLE MAX-DTABLE-SIZE erase

: th-track ( u -- a )
DEPENDENCY-TABLE swap MAX-DEPS * cells + ;

0 value curr-track

: get-track ( nt -- n ) \ return -1 if not found
curr-track 0 ?DO
I th-track a@ \ nt nt-track
over = IF drop I UNLOOP EXIT THEN
LOOP
drop -1 ;

: track-dependency ( nt -- )
curr-track th-track !
1 curr-track + to curr-track ;

: add-dependency ( nt-track nt-newdep -- )
swap get-track dup 0< IF
." Not tracking." cr abort
ELSE
th-track \ -- nt-dep a
BEGIN cell+ dup a@ 0= UNTIL !
THEN ;

\ Track first-level dependencies for last defined word
: track-deps ( -- )
latestnt dup track-dependency ( -- nt-track)
dup name>compile ( -- nt-track x xt )
3 cells allocate drop \ reserve memory for old compsem
dup >r 2! r@ 2 cells + ! r>
[n:d dup 2@ execute 2 cells + a@ latestnt add-dependency ;]
set-compsem ;

0 value ndeps

: what-requires? ( nt-track -- nt1 nt2 ... u | 0 )
0 to ndeps
get-track dup 0< IF
drop 0
ELSE
th-track cell+ \ -- a
BEGIN
dup a@ dup \ -- a nt|0 nt|0
WHILE
swap
ndeps 1+ to ndeps
cell+
REPEAT
2drop ndeps
THEN ;

: show-deps ( caddr u -- )
find-name ?dup IF
what-requires?
0 ?DO cr name>string type LOOP cr
then ;

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>On 10/13/22 10:30, Anton Ertl wrote:
>> If SET-OPTIMIZER would not change COMPILE,, it would not work (it
>> would fail to set the optimizer). For Gforth's current
>> implementation, if SET-COMPSEM did not change NAME>COMPILE, it would
>> not work (it would fail to change the compilation semantics).
>>
>
>In the case of standard "COMPILE," the problem is that on a dual-xt
>system, "COMPILE," operating on an xt is a fundamentally incorrect way
>of compiling a named definition (word).

If you mean that, e.g.,

: [compile,] compile, ; immediate
: foo [ ' s" ] [compile,] bla" ;

is not (and must not be) equivalent to

: foo s" bla" ;

i.e., performing the compilation semantics of S", you are correct.

It's also incorrect for performing the compilation semantics of

: bar ." bar" ; immediate

But

: foo1 [ ' bar ] [compile,] ;

appends the execution=interpretation=compilation semantics to foo1, so
it is equivalent to

: foo1 postpone bar ;

Ticking named words is unambiguous for most named words, and
COMPILE,ing the resulting xt is just as standard as EXECUTEing it.

>I understand
>your "hack" in Gforth is for the purpose of maintaining backwards
>compatibility.

I don't know what you mean with '"hack"'. Forth has COMPILE, for
appending semantics to the current definition, and NAME>COMPILE for
getting the compilation semantics of a named word. These are two
different jobs, and Gforth provides SET-OPTIMIZER and SET->COMP,
respectively, to provide implementations for these words for the
current definition. And that's the right way to do it.

Because one often follows the other, there have been attempts to use
one mechanism for both purposes:

* It is incorrect to use SET-OPTIMIZER for changing the compilation
semantics.

* If you use SET->COMP for changing the code generation, you don't
cover the cases that COMPILE, xts directly instead of going through
NAME>COMPILE, and you have to provide a fallback mechanism for
COMPILE,ing that; and if you implement [COMPILE], you also have to
tell it which uses of SET->COMP change the default compilation
semantics and which don't. You may think that you are saving
complexity by eliminating SET-OPTIMIZER, but the complexity pops up
elsewhere.

I think that providing both mechanisms is a good solution, because it
also provides a nice separation of concerns: If you want to change the
generated code without changing the semantics (i.e., if you want to
optimize), use SET-OPTIMIZER; if you want to change the compilation
semantics, use SET->COMP (or, by extension, SET-COMPSEM).

It also gives different results for your dependency-tracking (which is
outside semantics, so you can use either mechanism). Consider:

: bar ." bar" ; track-deps
: foo postpone bar postpone bar ; immediate
: bla foo foo ;
: blub [ ' bar compile, ] ;

If you track dependencies throug NAME>COMPILE as mechanism, you will
see that BAR is used twice in FOO. If you track dependencies using
"COMPILE,", you will see that BAR is used four times in BLA and one
time in BLUB.

>> You just have to break the cycle, in the
>> present case by explicitly performing the old action of NAME>COMPILE
>> instead of just calling NAME>COMPILE.
>>
>
>Does SET-COMPSEM erase all memory of the original compilation semantics,
>or is the old xt-comp still available? If not, we should be able to fix
>this.

The compilation semantics is specified by the >HM>COMP field (which
contains the xt performing NAME>COMPILE for the word), but may access
other parts of the word. SET-COMPSEM changes the >HM>COMP field, but
not the other parts of the word. Since you do not change the other
parts of the word, you could just remember the old contents of
>HM>COMP and execute that; that's what the untested version I posted
earlier does:

: track-deps ( -- )
latestnt dup track-dependency ( nt-track )
dup >namehm @ >hm>comp @ ( nt-track old-name>comp )
[d:d swap >r execute r> latestnt add-dependency ;] set->comp ;

Here I directly called SET->COMP instead of going through SET-COMPSEM.
Benefits: No additional closure from SET-COMPSEM (saving memory), and
it works correctly with [COMPILE].

>Ok, I see that there is a problem with using NAME>COMPILE within the
>the closure's xt, passed to SET-COMPSEM.
>
>NAME>COMPILE should be used outside the closure to obtain the
>compilation semantics at the time that TRACK-DEPS was invoked. Thus a
>permanent data structure, 3 cells long, may be passed into the closure.
>The data structure contains the nt of the word, and the original
>compilation semantics, which consists of two cells of data: x xt. Then
>the closure can invoke both the original compilation semantics and
>append the extra actions needed to add information to the dependency table.
>
>Here's my new version of TRACK-DEPS. It invokes NAME>COMPILE outside of
>the closure to obtain and store the current compilation semantics for
>the word before SET-COMPSEM is invoked. It should satisfy all cases,
>including preserving any extensions to the compilation semantics prior
>to invoking TRACK-DEPS. I've tested it on the single-xt+immediate flag
>cases. So it appears that the previous xt-comp is still valid.
>
>: track-deps ( -- )
> latestnt dup track-dependency ( -- nt-track)
> dup name>compile ( -- nt-track x xt )
> 3 cells allocate drop \ reserve memory for old compsem
> dup >r 2! r@ 2 cells + ! r>
> [n:d dup 2@ execute 2 cells + a@ latestnt add-dependency ;]
> set-compsem ;

Another way to do that is to simply pass the three values to the
closure. Gforth does not provide pure-stack closure construction
words for that, but unless you are a locals-hater, you may prefer the
following (untested):

: track-deps ( -- )
latestnt dup track-dependency ( nt-track)
dup name>compile ( nt-track x xt )
[{: nt-track x xt :}d x xt execute nt-track latestnt add-dependency ;]
set-compsem ;

or alternatively, turning xt into a defer-flavoured local:

: track-deps ( -- )
latestnt dup track-dependency ( nt-track)
dup name>compile ( nt-track x xt )
[{: nt-track x xt: xt :}d x xt nt-track latestnt add-dependency ;]
set-compsem ;

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: https://euro.theforth.net

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>It also gives different results for your dependency-tracking (which is
>outside semantics, so you can use either mechanism). Consider:
>
>: bar ." bar" ; track-deps
>: foo postpone bar postpone bar ; immediate
>: bla foo foo ;
>: blub [ ' bar compile, ] ;
>
>If you track dependencies throug NAME>COMPILE as mechanism, you will
>see that BAR is used twice in FOO. If you track dependencies using
>"COMPILE,", you will see that BAR is used four times in BLA and one
>time in BLUB.

Correction: How often you see it with the NAME>COMPILE approach
depends on the implementation of POSTPONE, in particular when it
performs NAME>COMPILE. The current implementation of POSTPONE in
Gforth performs NAME>COMPILE four times in this example, when
compiling BLA.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: https://euro.theforth.net

On 10/14/22 02:21, Anton Ertl wrote:
> Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>> On 10/13/22 10:30, Anton Ertl wrote:
>>> If SET-OPTIMIZER would not change COMPILE,, it would not work (it
>>> would fail to set the optimizer). For Gforth's current
>>> implementation, if SET-COMPSEM did not change NAME>COMPILE, it would
>>> not work (it would fail to change the compilation semantics).
>>>
>>
>> In the case of standard "COMPILE," the problem is that on a dual-xt
>> system, "COMPILE," operating on an xt is a fundamentally incorrect way
>> of compiling a named definition (word).
>
> If you mean that, e.g.,
>
> : [compile,] compile, ; immediate
> : foo [ ' s" ] [compile,] bla" ;
>
> is not (and must not be) equivalent to
>
> : foo s" bla" ;
>
> i.e., performing the compilation semantics of S", you are correct.
>
> It's also incorrect for performing the compilation semantics of
>
> : bar ." bar" ; immediate
>
> But
>
> : foo1 [ ' bar ] [compile,] ;
>
> appends the execution=interpretation=compilation semantics to foo1, so
> it is equivalent to
>
> : foo1 postpone bar ;
>
> Ticking named words is unambiguous for most named words, and
> COMPILE,ing the resulting xt is just as standard as EXECUTEing it.
>

The above is only true when you are not talking about dual-semantics
words, where the compilation semantics is not simply that of "ordinary
words" or of immediate words. Dual-semantics systems permit extended
compilation semantics. I think your term for such words is "combined
words". "COMPILE," is inadequate for combined words.

>> I understand
>> your "hack" in Gforth is for the purpose of maintaining backwards
>> compatibility.
>
> I don't know what you mean with '"hack"'. Forth has COMPILE, for
> appending semantics to the current definition, and NAME>COMPILE for
> getting the compilation semantics of a named word. These are two
> different jobs, and Gforth provides SET-OPTIMIZER and SET->COMP,
> respectively, to provide implementations for these words for the
> current definition. And that's the right way to do it.
>

I haven't implemented a dual-semantics system, so I won't strongly opine
on this other than to say, at the present time it does not seem to me
the right way to do it. "COMPILE," when used with NAME>INTERPRET, or
equivalently with '(tick) is fine when it is desired to append the
interpretation semantics, but for appending the compilation semantics of
a "combined word", one should use NAME>COMPILE and EXECUTE. Both of
these procedures must start with the name token (nt). After I implement
dual-semantics, I may be in a better position to understand your rationale.

> Because one often follows the other, there have been attempts to use
> one mechanism for both purposes:
>
> * It is incorrect to use SET-OPTIMIZER for changing the compilation
> semantics.
>
> * If you use SET->COMP for changing the code generation, you don't
> cover the cases that COMPILE, xts directly instead of going through
> NAME>COMPILE, and you have to provide a fallback mechanism for
> COMPILE,ing that; and if you implement [COMPILE], you also have to
> tell it which uses of SET->COMP change the default compilation
> semantics and which don't. You may think that you are saving
> complexity by eliminating SET-OPTIMIZER, but the complexity pops up
> elsewhere.
>

I think what I'm advocating is that, in a dual-semantics system, the
programmer should be aware of when to use "COMPILE," and when not to use
it -- namely use it only for those definitions for which a name token is
meaningless, and for words (named definitions) always use a compilation
chain which starts with the name token. When I use "COMPILE,"
deliberately I don't want the Forth system to alter the xt I give to
"COMPILE," behind the scenes due to the presence of arbitrary
compilation semantics.

You're entirely right that the complexity pops up elsewhere if you
eliminate SET-OPTIMIZER -- it rests on the programmer to use the
appropriate compilation sequence.

> I think that providing both mechanisms is a good solution, because it
> also provides a nice separation of concerns: If you want to change the
> generated code without changing the semantics (i.e., if you want to
> optimize), use SET-OPTIMIZER; if you want to change the compilation
> semantics, use SET->COMP (or, by extension, SET-COMPSEM).
>

Optimization is simply one rationale for changing the compilation
semantics. I don't feel strongly about SET-OPTIMIZER as a substitute for
SET-COMPSEM. What concerns me is that it modifies the behavior of
"COMPILE,". That seems to me to be unnecessary.

> It also gives different results for your dependency-tracking (which is
> outside semantics, so you can use either mechanism). Consider:
>
> : bar ." bar" ; track-deps
> : foo postpone bar postpone bar ; immediate
> : bla foo foo ;
> : blub [ ' bar compile, ] ;
>
> If you track dependencies throug NAME>COMPILE as mechanism, you will
> see that BAR is used twice in FOO. If you track dependencies using
> "COMPILE,", you will see that BAR is used four times in BLA and one
> time in BLUB.
>

Trying this gives the following (with the current version of
dependencies.4th):

include dependencies.4th ok
: bar ." bar" ; track-deps ok
: foo postpone bar postpone bar ; immediate track-deps ok
: bla foo foo ; ok
: blub [ ' bar compile, ] ; ok
s" foo" show-deps
bla
bla
ok
s" bar" show-deps
bla
bla
bla
bla
ok

This shows me that FOO is used twice in BLA and that BAR is used four
times in BLA. POSTPONE works nicely with dependency tracking.

Ticking BAR and compiling it with "COMPILE," in BLUB does not allow for
tracking the dependency, but this is ok because the proper way to
compile BAR in a dual semantics system is not with "COMPILE,". If
instead we use

: blub [ s" bar" find-name name>compile execute ] ;

Then, we get the desired list of words dependent on BAR.

s" bar" show-deps
blub
bla
bla
bla
bla
ok

A word such as COMPILE-NAME as a shortcut for NAME>COMPILE EXECUTE helps
shorten this. Also, we can be specific about which wordlist the name is
compiled from using FIND-NAME-IN. In short, '(tick) and "COMPILE," have
their purpose in a dual-semantics system but it is not a sequence which
should be applied to compiling a word.

>>> You just have to break the cycle, in the
>>> present case by explicitly performing the old action of NAME>COMPILE
>>> instead of just calling NAME>COMPILE.
>>>
>>
>> Does SET-COMPSEM erase all memory of the original compilation semantics,
>> or is the old xt-comp still available? If not, we should be able to fix
>> this.
>
...

>> Ok, I see that there is a problem with using NAME>COMPILE within the
>> the closure's xt, passed to SET-COMPSEM.
>>
>> NAME>COMPILE should be used outside the closure to obtain the
>> compilation semantics at the time that TRACK-DEPS was invoked. Thus a
>> permanent data structure, 3 cells long, may be passed into the closure.
>> The data structure contains the nt of the word, and the original
>> compilation semantics, which consists of two cells of data: x xt. Then
>> the closure can invoke both the original compilation semantics and
>> append the extra actions needed to add information to the dependency table.
>>
>> Here's my new version of TRACK-DEPS. It invokes NAME>COMPILE outside of
>> the closure to obtain and store the current compilation semantics for
>> the word before SET-COMPSEM is invoked. It should satisfy all cases,
>> including preserving any extensions to the compilation semantics prior
>> to invoking TRACK-DEPS. I've tested it on the single-xt+immediate flag
>> cases. So it appears that the previous xt-comp is still valid.
>>
>> : track-deps ( -- )
>> latestnt dup track-dependency ( -- nt-track)
>> dup name>compile ( -- nt-track x xt )
>> 3 cells allocate drop \ reserve memory for old compsem
>> dup >r 2! r@ 2 cells + ! r>
>> [n:d dup 2@ execute 2 cells + a@ latestnt add-dependency ;]
>> set-compsem ;
>
> Another way to do that is to simply pass the three values to the
> closure. Gforth does not provide pure-stack closure construction
> words for that, but unless you are a locals-hater, you may prefer the
> following (untested):
>
> : track-deps ( -- )
> latestnt dup track-dependency ( nt-track)
> dup name>compile ( nt-track x xt )
> [{: nt-track x xt :}d x xt execute nt-track latestnt add-dependency ;]
> set-compsem ;
>
...

On 10/14/22 12:16, Krishna Myneni wrote:
> On 10/14/22 02:21, Anton Ertl wrote:

>>
>> I don't know what you mean with '"hack"'.  Forth has COMPILE, for
>> appending semantics to the current definition, and NAME>COMPILE for
>> getting the compilation semantics of a named word.  These are two
>> different jobs, and Gforth provides SET-OPTIMIZER and SET->COMP,
>> respectively, to provide implementations for these words for the
>> current definition.  And that's the right way to do it.
>>

> ... "COMPILE," when used with NAME>INTERPRET, or
> equivalently with '(tick) is fine when it is desired to append the
> interpretation semantics, but for appending the compilation semantics of
> a "combined word", one should use NAME>COMPILE and EXECUTE. ...

My statement above is nonsense. What I should have said is that
compilation of a word should not be done with '(tick) and "COMPILE," but
start with the name token, e.g. S" <name>" FIND-NAME and then use
NAME>COMPILE and EXECUTE to append the appropriate semantics.

--
Krishna

On 10/14/22 12:56, Krishna Myneni wrote:

>
> ... What I should have said is that
> compilation of a word should not be done with '(tick) and "COMPILE," but
> start with the name token, e.g. S" <name>" FIND-NAME and then use
> NAME>COMPILE and EXECUTE to append the appropriate semantics.
>

A concrete illustration in Gforth is useful for understanding the above
statement.

require set-compsem.fs

I. Compilation semantics for an "ordinary word".

: foo ." foo" ;

s" foo" find-name name>compile .s
\ prints: <2> 140188477601736 140188477079032 ok

' compile, . \ prints: 140188477079032 ok
' foo . \ prints: 140188477601736 ok

Thus, we see that from FOO's nt, NAME>COMPILE returns on the stack

nt -- xt-foo xt-compile,

These two xt's describe the compilation semantics for FOO (or what has
been called "default compilation semantics"). The interpretation
semantics of FOO will be compiled when EXECUTE is performed with the two
xt's on the stack.

II. Compilation semantics for an IMMEDIATE word

: foo ." foo" ; IMMEDIATE

s" foo" find-name name>compile .s
\ prints: <2> 140188477601840 140188477026696 ok

' execute . \ prints: 140188477026696 ok
' foo . \ prints: 140188477601840 ok

From FOO's nt, NAME>COMPILE returns on the stack

nt -- xt-foo xt-execute

The compilation semantics for this version of FOO are the same as its
interpretation semantics. The interpretation semantics of FOO will be
executed when EXECUTE is performed with these two xt's on the stack.

III. Compilation semantics for a dual-semantics word

: foo ." foo" ; \ define word with default compilation semantics
compsem: 'f' emit 'o' emit 'o' emit ;

This example is neither case I nor case II.

s" foo" find-name name>compile .s
\ prints: <2> 140188477602024 140188477026696

' execute . \ prints: 140188477026696 ok
' foo . \ prints: 140188477601944 ok

From FOO's nt, NAME>COMPILE returns on the stack

nt -- xt-? xt-execute

xt-? represents the compilation semantics of FOO and does not involve
xt-foo. xt-? will be performed when EXECUTE is performed with these two
xt's on the stack. The output will look the same as for case II, because
of the way we defined COMPSEM: ... ; to give the same output behavior,
but could also have done

: foo ." foo" ;
compsem: ." bar" ;

Then,

foo foo ok
: test foo ; bar \ bar is printed when FOO is compiled into TEST

The above three cases demonstrate that the sequence

s" <name>" FIND-NAME NAME>COMPILE EXECUTE

will perform the specified compilation semantics for all cases. Cases I
and II apply to single-xt + immediate flag Forth systems, while Cases I
-- III apply to dual-xt systems.

--
Krishna

On 10/14/22 17:00, Krishna Myneni wrote:
...
> A concrete illustration in Gforth ...
>
> require set-compsem.fs
>
> I. Compilation semantics for an "ordinary word".
>
> : foo ." foo" ;
>
> s" foo" find-name name>compile .s
...
> Thus, we see that from FOO's nt, NAME>COMPILE returns on the stack
>
> nt -- xt-foo xt-compile,
>
...
>
> II. Compilation semantics for an IMMEDIATE word
>
> : foo ." foo" ; IMMEDIATE
>
> s" foo" find-name name>compile .s
...
> From FOO's nt, NAME>COMPILE returns on the stack
>
> nt -- xt-foo xt-execute
>
...
> III. Compilation semantics for a dual-semantics word
>
> : foo ." foo" ;  \ define word with default compilation semantics
> compsem: 'f' emit 'o' emit 'o' emit ;
>
> This example is neither case I nor case II.
>
> s" foo" find-name name>compile .s
...
> From FOO's nt, NAME>COMPILE returns on the stack
>
> nt -- xt-? xt-execute
>
> xt-? represents the compilation semantics of FOO and does not involve
> xt-foo. ...
>
> s" <name>" FIND-NAME NAME>COMPILE EXECUTE
>
> will perform the specified compilation semantics for all cases. Cases I
> and II apply to single-xt + immediate flag Forth systems, while Cases I
> -- III apply to dual-xt systems.
>

It is further worth noting that on an *ideal* dual-xt system, as opposed
to a system which allows for dual-semantics, NAME>COMPILE would simply
have the following stack diagram for all three cases:

nt -- xt-comp xt-execute \ xt-comp represents the compilation semantics

Then the xt-execute may be dispensed with and NAME>INTERPRET and some
differently named NAME>COMPILE both would simply return a single xt. In
the *ideal* dual-xt system, we should be able to define

: NAME>COMPSEM ( nt -- xt-comp ) NAME>COMPILE DROP ;

EXECUTE(ing) xt-comp will perform the compilation semantics. This is not
how Gforth works at present (see case I). But, in my opinion, it would
be a cleaner and more consistent design, and easier to understand for
the programmer.

For NAME>INTERPRET, xt is simply xt-interp which is the same value as
returned by '(tick) in all three cases, and this is the behavior in Gforth:

case I:

: foo ." foo" ; ok
s" foo" find-name name>interpret ' foo = . -1 ok
ok

case II:
: foo ." foo" ; IMMEDIATE
s" foo" find-name name>interpret ' foo = . -1 ok
ok

case III:
: foo ." foo" ;
compsem: 'f' emit 'o' emit 'o' emit ; ok
s" foo" find-name name>interpret ' foo = . -1 ok

The issue of Gforth's non-standard word SET-OPTIMIZER altering the
behavior of standard "COMPILE," is a separate issue also, but one on
which our we have already had sufficient discussion.

--
Krishna

On 2022-10-10 15:44, Krishna Myneni wrote:
>
> A word definition may be immediately followed by the word TRACK-DEPS
> which will set its compilation semantics to not only compile its
> interpretation semantics

By this wording you just skew the language of the standard.

By default, not the interpretation semantics, but the execution
semantics of a word are appended (compiled):

| 3.4.3.3 Compilation semantics
| Unless otherwise specified in a "Compilation:" section
| of the glossary entry, the compilation semantics of a Forth
| definition shall be to append its *execution semantics*
| to the execution semantics of the current definition.

(emphasized by me)

There is a number of cases when interpretation semantics for a word are
undefined by the standard, but execution semantics are defined.

But when interpretation semantics are defined via execution semantics,
it's still incorrect to say that they are equivalent in the general case.

Otherwise you introduce such a beast as interpretation semantics in
compilation state — i.e., a behavior that takes place when the Forth
text interpreter encounters a word name in interpretation state, if the
Forth text interpreter in compilation state.

It's absurd. It's like to ask what is your behavior when it rains, if
there isn't raining?

[...]

--
Ruvim

On 10/19/22 04:16, Ruvim wrote:
> On 2022-10-10 15:44, Krishna Myneni wrote:
>>
>> A word definition may be immediately followed by the word TRACK-DEPS
>> which will set its compilation semantics to not only compile its
>> interpretation semantics
>
> By this wording you just skew the language of the standard.
>

The original wording of the statement is flawed. A more accurate
statement follows:

A word definition may be immediately followed by the word TRACK-DEPS
which will set its new compilation semantics to append the original
compilation semantics and additional semantics to keep track of the
words into which it is compiled.

The above statement explicitly indicates that IMMEDIATE may be used
between the end of the definition ";" and TRACK-DEPS

> By default, not the interpretation semantics, but the execution
> semantics of a word are appended (compiled):
>
> | 3.4.3.3 Compilation semantics
> | Unless otherwise specified in a "Compilation:" section
> | of the glossary entry, the compilation semantics of a Forth
> | definition shall be to append its *execution semantics*
> | to the execution semantics of the current definition.
>
> (emphasized by me)
>
>

The wording of the standard reflects a strict adherence to the single-xt
model, while we are discussing here a dual-xt model, in which the term
"execution semantics" can be omitted. In a single-xt system the
compilation semantics and the interpretation semantics are strongly
connected, while in a dual-xt system they are free to be specified
separately.

In the ideal implementation of a dual-xt system, every word provides
execution tokens for both interpretation semantics and compilation
semantics. When the text interpreter parses the word name it will
determine whether to perform the interpretation semantics or the
compilation semantics based on STATE. There is no need to talk of
"execution semantics". For :NONAME definitions, quotations, and other
compositions which result in an execution token, we may simply speak of
the semantics resulting from applying EXECUTE to xt.

With the existence of dual-semantics Forth systems which are able to
properly run code written for single-xt systems, I would argue that the
standard's definition of "compilation semantics", given in 3.4.3.3 is
becoming obsolete.

> There is a number of cases when interpretation semantics for a word are
> undefined by the standard, but execution semantics are defined.
>

In a dual-semantics system, let's say you want to declare a word which
is not allowed to be used in interpretation state. We may do so, as
follows (in Gforth):

: NAME -14 throw ;
\ -14 is throw code for interpreting a compile-only word
COMPSEM: ( i*x -- j*x ) ... ;

Obviously this word has both interpretation semantics (throw an error)
and compilation semantics.

> But when interpretation semantics are defined via execution semantics,
> it's still incorrect to say that they are equivalent in the general case.
>

The "execution semantics" of a word is STATE-dependent when parsed and
executed by the text interpreter. In a dual-xt system we have only
"interpretation semantics" *and* "compilation semantics". Then, the
correct way to compile a word is through the name token, from which
xt-comp would be obtained via NAME>COMPILE and then executed.

> Otherwise you introduce such a beast as interpretation semantics in
> compilation state — i.e., a behavior that takes place when the Forth
> text interpreter encounters a word name in interpretation state, if the
> Forth text interpreter in compilation state.
>

"Interpretation semantics" and "compilation semantics" only refer to
what the text interpreter does with the name of a word.

compilation semantics: The behavior of a Forth definition when its name
is encountered by the *text interpreter* in compilation state.

interpretation semantics: The behavior of a Forth definition when its
name is encountered by the *text interpreter* in interpretation state.

The text interpreter's behavior, as dictated by the standard, is very
clear when it encounters a name in a given state -- perform the
interpretation semantics *or* the compilation semnatics. As far as the
text interpreter is concerned, there is no such beast as you are claiming.

If you want to perform the compilation semantics in interpretation
state, you can do that even on a (Forth 200x) standard single-xt system:

s" <name>" FIND-NAME NAME>COMPILE EXECUTE

It will likely result in inconsistent behavior on different Forth
systems. This is not the point of a dual-xt system though. The point is
to have less constraint on the compilation semantics of a word. We have
illustrated the usefulness for tasks like optimizing the implementation
of the compilation semantics, and for appending additional semantics to
the original compilation semantics (e.g. for instrumentation or for
dependency tracking).

--
Krishna

On 2022-10-19 22:33, Krishna Myneni wrote:
> On 10/19/22 04:16, Ruvim wrote:

[...]
>> By default, not the interpretation semantics, but the execution
>> semantics of a word are appended (compiled):
>>
>> | 3.4.3.3 Compilation semantics
>> | Unless otherwise specified in a "Compilation:" section
>> | of the glossary entry, the compilation semantics of a Forth
>> | definition shall be to append its *execution semantics*
>> | to the execution semantics of the current definition.
>>
>> (emphasized by me)
>>
>>
>
> The wording of the standard reflects a strict adherence to the single-xt
> model,

Probably it does, in few places (like "find"), but not in this place.

> while we are discussing here a dual-xt model, in which the term
> "execution semantics" can be omitted. In a single-xt system the
> compilation semantics and the interpretation semantics are strongly
> connected,
> while in a dual-xt system they are free to be specified separately.

You are wrong: in a single-xt system they can be specified separately too.

: compilation ( -- flag ) state @ 0<> ;

: dual` ( xt.int xt.comp "name" -- )
2>r : 2r> 2lit,
[: ( i*x xt.int xt.comp -- j*x )
compilation if nip else drop then execute
;] compile, postpone ; immediate
;

:noname ( -- ) ." foo" ;
:noname ( -- ) ." bar" ;
dual` foo

Where xt.int and xt.comp are execution tokens for definitions that are
*used* to perform the interpretation semantics and compilation semantics
for /name/.

They don't identify the corresponding behaviors in the general case.
(it's true for dual-xt systems too).

> In the ideal implementation of a dual-xt system, every word provides
> execution tokens for both interpretation semantics and compilation
> semantics. When the text interpreter parses the word name it will
> determine whether to perform the interpretation semantics or the
> compilation semantics based on STATE. There is no need to talk of
> "execution semantics".

I totally disagree.

> For :NONAME definitions, quotations, and other
> compositions which result in an execution token, we may simply speak of
> the semantics resulting from applying EXECUTE to xt.

These semantics are included into the notion of "execution semantics".
There is no any profit to use the phrase:
"the semantics resulting from applying EXECUTE to xt of a definition"
instead of the phrase:
"the execution semantics of a definition"

> With the existence of dual-semantics Forth systems which are able to
> properly run code written for single-xt systems, I would argue that the
> standard's definition of "compilation semantics", given in 3.4.3.3 is
> becoming obsolete.

It makes standard systems non standard.

>> There is a number of cases when interpretation semantics for a word
>> are undefined by the standard, but execution semantics are defined.
>>
>
> In a dual-semantics system, let's say you want to declare a word which
> is not allowed to be used in interpretation state. We may do so, as
> follows (in Gforth):
>
> : NAME  -14 throw ;
> \ -14 is throw code for interpreting a compile-only word
> COMPSEM: ( i*x -- j*x ) ... ;
>
> Obviously this word has both interpretation semantics (throw an error)
> and compilation semantics.

Your example is not a case when interpretation semantics for a word are
undefined by the standard, and execution semantics are defined.

[...]

--
Ruvim

In article <tiptve$4kuu$2@dont-email.me>,
Krishna Myneni <krishna.myneni@ccreweb.org> wrote:
<SNIP>
>In a dual-semantics system, let's say you want to declare a word which
>is not allowed to be used in interpretation state. We may do so, as
>follows (in Gforth):
>
>: NAME -14 throw ;
>\ -14 is throw code for interpreting a compile-only word
>COMPSEM: ( i*x -- j*x ) ... ;
>
>Obviously this word has both interpretation semantics (throw an error)
>and compilation semantics.

You contradict yourself. You stated that the word is not allowed to be
used in interpretation state.
So it cannot be even used to throw an error. 1)
(Language precision, important in this context, "this word cannot
be used in a valid ISO94 program." is correct.)
<SNIP>

1) Anton Ertl would say that "pigs can fly out your nose", useful
excuse to compile something insensible to spite you.

>--
>Krishna

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On 10/20/22 03:18, Ruvim wrote:
> On 2022-10-19 22:33, Krishna Myneni wrote:
>> On 10/19/22 04:16, Ruvim wrote:
>
> [...]

>
>
>> while we are discussing here a dual-xt model, in which the term
>> "execution semantics" can be omitted. In a single-xt system the
>> compilation semantics and the interpretation semantics are strongly
>> connected, while in a dual-xt system they are free to be specified
>> separately.
>
> You are wrong: in a single-xt system they can be specified separately too.
>
>   : compilation ( -- flag ) state @ 0<> ;
>
>   : dual` ( xt.int xt.comp "name" -- )
>     2>r :   2r> 2lit,
>     [: ( i*x xt.int xt.comp -- j*x )
>       compilation if nip else drop then execute
>     ;] compile, postpone ; immediate
>   ;
>
>   :noname ( -- ) ." foo" ;
>   :noname ( -- ) ." bar" ;
>   dual` foo
>
> Where xt.int and xt.comp are execution tokens for definitions that are
> *used* to perform the interpretation semantics and compilation semantics
> for /name/.
>

It's clever, but its state dependency means that FOO is no longer a
"first-class" word in Forth. For example, if we want to execute xt.comp
from the interpreter (the action of which is not ambiguous),

s" foo" find-name name>compile execute \ prints "foo"

which gives an incorrect result with your definition (it prints "foo").
FIND-NAME and NAME>COMPILE are standard words. You have created a
state-smart word which no longer behaves as expected for standard
operations. In a true dual-semantics system, the above sequence should
print "bar", e.g. in Gforth,

: foo ." foo" ; ok
compsem: ." bar" ; ok
s" foo" find-name name>compile execute \ prints "bar"

> They don't identify the corresponding behaviors in the general case.
> (it's true for dual-xt systems too).
>

I don't know what you mean by the above statement.

>
>
>> In the ideal implementation of a dual-xt system, every word provides
>> execution tokens for both interpretation semantics and compilation
>> semantics. When the text interpreter parses the word name it will
>> determine whether to perform the interpretation semantics or the
>> compilation semantics based on STATE. There is no need to talk of
>> "execution semantics".
>
> I totally disagree.
>

I would ask you to elaborate, but it's clear from your prior posts that
your disagreement rests on the basis of the term, "execution semantics,"
being used in the current standard (which appears to be circular logic
to me).

>> For :NONAME definitions, quotations, and other compositions which
>> result in an execution token, we may simply speak of the semantics
>> resulting from applying EXECUTE to xt.
>
> These semantics are included into the notion of "execution semantics".
> There is no any profit to use the phrase:
>   "the semantics resulting from applying EXECUTE to xt of a definition"
> instead of the phrase:
>   "the execution semantics of a definition"
>

A word definition has two types of semantics in a dual-semantics system,
by definition.

>
>
>
>> With the existence of dual-semantics Forth systems which are able to
>> properly run code written for single-xt systems, I would argue that the
>> standard's definition of "compilation semantics", given in 3.4.3.3 is
>> becoming obsolete.
>
> It makes standard systems non standard.
>

A standard fixed in stone is the sign-post for a dead language.
Standards for languages which are still used evolve to take into account
common practice and new functionality desired by programmers and users.
One type of new functionality is the ability to define "first-class"
words (see above) with flexible specification of compilation behavior.
This has led to the development of new Forth systems which adopt
dual-semantics, either with full dual-xt implementation or with mixed
single-xt and dual-xt implementation.

>
>>> There is a number of cases when interpretation semantics for a word
>>> are undefined by the standard, but execution semantics are defined.
>>>
>>
>> In a dual-semantics system, let's say you want to declare a word which
>> is not allowed to be used in interpretation state. We may do so, as
>> follows (in Gforth):
>>
>> : NAME  -14 throw ;
>> \ -14 is throw code for interpreting a compile-only word
>> COMPSEM: ( i*x -- j*x ) ... ;
>>
>> Obviously this word has both interpretation semantics (throw an error)
>> and compilation semantics.
>
> Your example is not a case when interpretation semantics for a word are
> undefined by the standard, and execution semantics are defined.
>

A system still has to do something when you try to invoke interpretation
semantics in the case they are undefined. The semicolon ";" operator has
undefined interpretation semantics. Try executing it in the interpreter
in your system. The above example is no different, except it gives the
programmer full control of the system response when a word designed to
be used only in compilation state is executed in interpretation state.
In contrast, single-xt systems give the programmer no control unless a
state-smart word is used, resulting in second-class words.

--
Krishna

On 10/20/22 06:25, albert wrote:
> In article <tiptve$4kuu$2@dont-email.me>,
> Krishna Myneni <krishna.myneni@ccreweb.org> wrote:
> <SNIP>
>> In a dual-semantics system, let's say you want to declare a word which
>> is not allowed to be used in interpretation state. We may do so, as
>> follows (in Gforth):
>>
>> : NAME -14 throw ;
>> \ -14 is throw code for interpreting a compile-only word
>> COMPSEM: ( i*x -- j*x ) ... ;
>>
>> Obviously this word has both interpretation semantics (throw an error)
>> and compilation semantics.
>
> You contradict yourself. You stated that the word is not allowed to be
> used in interpretation state.
> So it cannot be even used to throw an error. 1)
> (Language precision, important in this context, "this word cannot
> be used in a valid ISO94 program." is correct.)
> <SNIP>

My intended meaning by "not allowed to be used in interpretation state"
is that the word takes no action other than indicating such usage is an
error. If the word doesn't notify the user, the system may or may not.
See my response to ruvim.

--
Krishna

On 2022-10-20 11:41, Krishna Myneni wrote:
> On 10/20/22 03:18, Ruvim wrote:
>> On 2022-10-19 22:33, Krishna Myneni wrote:

[...]
>>> while in a dual-xt system they are free to be specified
>>> separately.
>>
>> You are wrong: in a single-xt system they can be specified separately
>> too.
>>
>>    : compilation ( -- flag ) state @ 0<> ;
>>
>>    : dual` ( xt.int xt.comp "name" -- )
>>      2>r :   2r> 2lit,
>>      [: ( i*x xt.int xt.comp -- j*x )
>>        compilation if nip else drop then execute
>>      ;] compile, postpone ; immediate
>>    ;
>>
>>    :noname ( -- ) ." foo" ;
>>    :noname ( -- ) ." bar" ;
>>    dual` foo
>>
>> Where xt.int and xt.comp are execution tokens for definitions that are
>> *used* to perform the interpretation semantics and compilation
>> semantics for /name/.
>>
>
> It's clever, but its state dependency means that FOO is no longer a
> "first-class" word in Forth.

What is a first-class word?

> For example, if we want to execute xt.comp
> from the interpreter (the action of which is not ambiguous),
>
> s" foo" find-name name>compile execute   \ prints "foo"
>
> which gives an incorrect result with your definition (it prints "foo").

How do you know?

If a single-xt system provides a correct "name>compile", as:

: name>compile ( nt -- x xt )
name> ['] execute-compilatively
;

your test prints "bar".

> FIND-NAME and NAME>COMPILE are standard words. You have created a
> state-smart word which no longer behaves as expected for standard
> operations. In a true dual-semantics system, the above sequence should
> print "bar", e.g. in Gforth,
>
> : foo ." foo" ;  ok
> compsem: ." bar" ;  ok
> s" foo" find-name name>compile execute  \ prints "bar"

So, it is the same behavior that the system demonstrates when it
encounters the word name "foo" in compilation state.

And it's an expected behavior for your test, regardless how a word "foo"
is defined. Isn't it?

[...]

--
Ruvim

On 10/20/22 15:25, Ruvim wrote:
> On 2022-10-20 11:41, Krishna Myneni wrote:
>> On 10/20/22 03:18, Ruvim wrote:
>>> On 2022-10-19 22:33, Krishna Myneni wrote:
>
> [...]
>>>> while in a dual-xt system they are free to be specified separately.
>>>
>>> You are wrong: in a single-xt system they can be specified separately
>>> too.
>>>
>>>    : compilation ( -- flag ) state @ 0<> ;
>>>
>>>    : dual` ( xt.int xt.comp "name" -- )
>>>      2>r :   2r> 2lit,
>>>      [: ( i*x xt.int xt.comp -- j*x )
>>>        compilation if nip else drop then execute
>>>      ;] compile, postpone ; immediate
>>>    ;
>>>
>>>    :noname ( -- ) ." foo" ;
>>>    :noname ( -- ) ." bar" ;
>>>    dual` foo
>>>
>>> Where xt.int and xt.comp are execution tokens for definitions that
>>> are *used* to perform the interpretation semantics and compilation
>>> semantics for /name/.
>>>
>>
>> It's clever, but its state dependency means that FOO is no longer a
>> "first-class" word in Forth.
>
> What is a first-class word?
>

For the purpose of this discussion, let us agree that a "first class
word" is one which behaves as expected when the following standard words
are used in connection with the word:

-- '(tick)
-- [']
-- postpone
-- name>interpret
-- name>compile
-- compile,
-- [compile] (if your system provides it)

>
>
>> For example, if we want to execute xt.comp from the interpreter (the
>> action of which is not ambiguous),
>>
>> s" foo" find-name name>compile execute   \ prints "foo"
>>
>> which gives an incorrect result with your definition (it prints "foo").
>
> How do you know?
>

Output from Gforth:

: compilation state @ 0<> ; ok
: dual` 2>r : 2r> postpone 2literal [: compilation if nip else drop then
execute ;] compile, postpone ; immediate ; ok
:noname ." foo" ; ok 1
:noname ." bar" ; ok 2
dual` foo ok
s" foo" find-name name>compile execute foo ok

Output from kForth (single-xt system):

: compilation state @ 0<> ;
ok
: dual` 2>r : 2r> postpone 2literal [: compilation if nip else drop then
execute ;] compile, postpone ; immediate ;
ok
:noname ." foo" ;
ok
:noname ." bar" ;
ok
dual` foo
ok
s" foo" find-name name>compile execute
foo ok

> If a single-xt system provides a correct "name>compile", as:
>
>   : name>compile ( nt -- x xt )
>     name> ['] execute-compilatively
>   ;
>
> your test prints "bar".
>
>

I don't know what your NAME> and EXECUTE-COMPILATIVELY are supposed to
do -- they are not standard words. NAME>COMPILE in both Gforth and
kForth follow the behavior prescribed by the Forth 200x standard:

15.6.2.1909.10 NAME>COMPILE
“name-to-compile”
TOOLS EXT
( nt – – x xt ) x xt represents the compilation semantics of the word
nt. The returned xt has the stack effect ( i * x x – – j * x ) .
Executing xt consumes x and performs the compilation semantics of the
word represented by nt.

NAME>COMPILE returns the compilation semantics, which when executed
should print "bar". Your implementation of dual-semantics words using
DUAL` does not work with NAME>COMPILE under Gforth or kForth, and I
suspect it will not work on most other Forth systems.

>
>
>> FIND-NAME and NAME>COMPILE are standard words. You have created a
>> state-smart word which no longer behaves as expected for standard
>> operations. In a true dual-semantics system, the above sequence should
>> print "bar", e.g. in Gforth,
>>
>> : foo ." foo" ;  ok
>> compsem: ." bar" ;  ok
>> s" foo" find-name name>compile execute  \ prints "bar"
>
>
> So, it is the same behavior that the system demonstrates when it
> encounters the word name "foo" in compilation state.
>

Printing "bar" is the correct behavior for my test -- Gforth provides
the correct behavior when the compilation semantics are specified by its
COMPSEM: word. I've demonstrated that your DUAL` does not work with
NAME>COMPILE on a standard system.

--
Krishna

Krishna

On 2022-10-20 22:01, Krishna Myneni wrote:
> On 10/20/22 15:25, Ruvim wrote:
[...]
> I've demonstrated that your DUAL` does not work with
> NAME>COMPILE on a standard system.

I cannot agree. There should be admitted either an ambiguous condition,
or an incompletely standard system.

But, before I try to convince you, could you please answer to my
question that you missed:

>> And it's an expected behavior for your test, regardless how a word "foo"
>> is defined. Isn't it?

Regardless how the word "foo" is defined, should your test demonstrate
the same behavior that the system demonstrates when the Forth text
interpreter encounters the word name "foo" in compilation state?

--
Ruvim

On 10/21/22 06:23, Ruvim wrote:
> On 2022-10-20 22:01, Krishna Myneni wrote:
>> On 10/20/22 15:25, Ruvim wrote:
> [...]
>> I've demonstrated that your DUAL` does not work with NAME>COMPILE on a
>> standard system.
>
> I cannot agree. There should be admitted either an ambiguous condition,
> or an incompletely standard system.
>
> But, before I try to convince you, could you please answer to my
> question that you missed:
>
>
>>> And it's an expected behavior for your test, regardless how a word "foo"
>>> is defined. Isn't it?
>
> Regardless how the word "foo" is defined, should your test demonstrate
> the same behavior that the system demonstrates when the Forth text
> interpreter encounters the word name "foo" in compilation state?
>

Perhaps you can pose your question in the form of a test, using only
standard words, so that there is no possibility of misunderstanding.

Also, you may want to apply my test of your word DUAL` in other standard
systems to see what they do.

--
Krishna