In various discussions about the standard (what it means and how it
should be improved) we have used various requirements as justification
for the positions we take, but we actually have not discussed these
requirements much.

Many of the justifications are quite foggy in their actual meaning,
such as the appeal to simplicity (or denouncement of complexity). I
am not particularly interested in those, because they don't give any
real guidance. E.g., wrt simplicity, the simplest Forth system is one
that can do nothing; obviously there are cases where we consider other
things more important than simplicity, but the appeal to simplicity
does not give us any guidance when other things are more important.

But there are also hard requirements from which we can derive test
cases. I want to dicuss these and maybe find consensus on which of
these we want to pursue, or alternatively find out that there is no
consensus on some requirements.

Some requirements that come to my mind (in no particular order) are:

1) It must be possible to write a user-defined text interpreter in
standard Forth.

A.6.2.0945 COMPILE, suggests this. Maybe this can also be found
elsewhere in the standard. When we got rid of STATE-smartness in
Gforth in 1996, I used this requirement as guidance on what to do
about FIND (the specification in the document gives no guidance).
However, when testing whether FIND finds locals, three out of five
systems did not FIND locals, so the user-defined text interpreter does
not work for code with locals. Moreover, on VFX 5.11 some other case
fails, so the only system that does satisfy the requirement is Gforth.
And in comminicating with the maintainers of the other systems, they
apparently do not consider the user-defined text interpreter as a
requirement. And given that we do not hear about programs that break
on these systems, it's not a requirement that programs rely on, either.

So, can we just say that we don't want this requirement, or is there
something behind it that we want?

2) We must be able to copy code between interpreted and
compiled code.

This requirement is the reason for dual-semantics words like FILE S"
and TO, it is satisfied by the text interpreter for normal words and
numbers, and it seduces some people to implement STATE-smart immediate
words. OTOH, other people don't want STATE-smart words, which has led
to word pairs like ' and ['], CHAR and [CHAR]. Another option is to
implement dual-semantics words properly, but this practice is not
common enough to make it a requirement on systems.

In any case, we see this requirement at work as soon as we have a new
word with special compilation semantics, such as [:. Andrew Haley may
wish that this was not a requirement, but that does not make it go
away.

3) It must be able to define a word that is equivalent to any other
word (and maybe add some scaffolding, such as profiling or logging).

This is the requirement behind adding SYNONYM and the proposed
tightening of SYNONYM
<https://forth-standard.org/proposals/tighten-the-specification-of-synonym-version-1-#contribution-60>.
It is also a requirement mentioned by Ruvim as a justification for
some of his positions.

4) All words are equal. There is no fundamental difference between
system-defined and user-defined words. Words that deal with words
(e.g., ' FIND EXECUTE COMPILE,) work on all of them.

This was definitely the case in early Forth; you could even tick and
then EXECUTE and COMPILE, >R and EXIT. But cmForth broke with it: IF
is not always FINDable and not tickable, and there is no standard way
to add a word like S" to cmForth (while there is a system-specific
way). And in some modern systems ticking and then EXECUTEing >R and
EXIT does not work. And some positions say that this is ok, and say
that this has to take a back seat to requirement 3. Wil Baden held
the position that you must not tick words you did not define yourself.
And even in early Forth ['] IF EXECUTE is not equivalent to IF.

This list is not exhaustive. Please contribute additional hard
requirements.

The other direction in which this list can be extended is to go
deeper, e.g., to add test cases resulting from the requirements.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On Wednesday, June 9, 2021 at 1:41:00 PM UTC+2, Anton Ertl wrote:
[..]
> Some requirements that come to my mind (in no particular order) are:

OK, let's assume I lost the source to iForth and must rebuild from
scratch, what kind of Forth would I use for the reboot?

- Introspection:
Type 1: It should be possible to look at the source for any word (the
ultimate documentation).
Type 2: It should be possible for *words* to find out the
characteristics of any other word (documentation does
not count). I frequently want to know (in an application)
the xt and nt of a word (I use ' and head' for that).
Concrete questions are: is this system word a LOCAL, a variable,
a USER, a code definition, where is its data (relative to xt
or nt), how can I go from body to code and back, etc..

> 1) It must be possible to write a user-defined text interpreter in
> standard Forth.
[..]
Given introspection, I can always do that in a system-specific way,
which would suit me fine.

> 2) We must be able to copy code between interpreted and
> compiled code.
[..]
Not interesting. When the source of IF is available I'll copy that
instead.

> This requirement is the reason for dual-semantics words like FILE S"
> and TO, it is satisfied by the text interpreter for normal words and
> numbers, and it seduces some people to implement STATE-smart immediate
> words. OTOH, other people don't want STATE-smart words, which has led
> to word pairs like ' and ['], CHAR and [CHAR]. Another option is to
> implement dual-semantics words properly, but this practice is not
> common enough to make it a requirement on systems.
[..]
If all this goes away by admitting that there is a difference between
interpreted and compiled code that we must know about, that would be
a huge win.

> 3) It must be able to define a word that is equivalent to any other
> word (and maybe add some scaffolding, such as profiling or logging).
[..]
With the source, it is difficult to see where one would get stuck.

At this point it becomes clear that there may be a hidden requirement
in your list: that all these features are portable (i.e you can write
them *without* introspection and without *vendor support*.) I don't
care about portability.

> 4) All words are equal. There is no fundamental difference between
> system-defined and user-defined words. Words that deal with words
> (e.g., ' FIND EXECUTE COMPILE,) work on all of them.

They should be equal in performance. If a mix of methods must be
used for some words, I'm fine with that.

> This was definitely the case in early Forth; you could even tick
> and then EXECUTE and COMPILE, >R and EXIT.
[..]
Definitely no desire to go back to that era if that is the only
reason.

-marcel

On 6/9/21 3:52 AM, Anton Ertl wrote:
> In various discussions about the standard (what it means and how it
> should be improved) we have used various requirements as justification
> for the positions we take, but we actually have not discussed these
> requirements much.
>
> Many of the justifications are quite foggy in their actual meaning,
> such as the appeal to simplicity (or denouncement of complexity). I
> am not particularly interested in those, because they don't give any
> real guidance. E.g., wrt simplicity, the simplest Forth system is one
> that can do nothing; obviously there are cases where we consider other
> things more important than simplicity, but the appeal to simplicity
> does not give us any guidance when other things are more important.
>
> But there are also hard requirements from which we can derive test
> cases. I want to dicuss these and maybe find consensus on which of
> these we want to pursue, or alternatively find out that there is no
> consensus on some requirements.
>
> Some requirements that come to my mind (in no particular order) are:
>
'''
>
> 2) We must be able to copy code between interpreted and
> compiled code.
>

Where do you draw the line on this? When I designed kForth, I wanted to
be able to "converse" with the interpreter and enter statements like,

>file piecewise.dat -6e0 BEGIN fdup f. fdup f 2 spaces f. cr 1e-3 f+
fdup 6e0 f> until console

(here, f is a definition for a floating point function). I can in fact
execute the above statement from the interpreter, which saves me a lot
of time when I need to generate data to an output file. However, not
everything which can be compiled can be executed from the interpreter.

> This requirement is the reason for dual-semantics words like FILE S"
> and TO, it is satisfied by the text interpreter for normal words and
> numbers, and it seduces some people to implement STATE-smart immediate
> words. OTOH, other people don't want STATE-smart words, which has led
> to word pairs like ' and ['], CHAR and [CHAR]. Another option is to
> implement dual-semantics words properly, but this practice is not
> common enough to make it a requirement on systems.
>
> In any case, we see this requirement at work as soon as we have a new
> word with special compilation semantics, such as [:. Andrew Haley may
> wish that this was not a requirement, but that does not make it go
> away.

There's nothing special about "[:" in some implementations of Forth. It
can be defined in standard source in such systems. However, I like the
idea of dual-semantics words. Forth code for such systems can be written
to be more predictable in what words will do when used in different
ways, compared to single-xt + immediate flag systems.

>
> 3) It must be able to define a word that is equivalent to any other
> word (and maybe add some scaffolding, such as profiling or logging).
>
> This is the requirement behind adding SYNONYM and the proposed
> tightening of SYNONYM
> <https://forth-standard.org/proposals/tighten-the-specification-of-synonym-version-1-#contribution-60>.
> It is also a requirement mentioned by Ruvim as a justification for
> some of his positions.
>
> 4) All words are equal. There is no fundamental difference between
> system-defined and user-defined words. Words that deal with words
> (e.g., ' FIND EXECUTE COMPILE,) work on all of them.
>

Ticking a word and being able to execute the xt, in a portable way, are
still two different things. I do care about portability. I don't believe
Forth source code should be tied to a single system. My published code
strives to be portable.

Krishna

On 2021-06-09 11:52, Anton Ertl wrote:
> In various discussions about the standard (what it means and how it
> should be improved) we have used various requirements as justification
> for the positions we take, but we actually have not discussed these
> requirements much.
[...]

> Some requirements that come to my mind (in no particular order) are:
>
> 1) It must be possible to write a user-defined text interpreter in
> standard Forth.
[...]

Yes.

Concerning locals — my position is that a special word to handle locals
should be provided in the Locals word set. Perhaps, as a recognizer at once.

> 2) We must be able to copy code between interpreted and
> compiled code.

Yes, at least partially. Since we should not require interpretive
control-flow structures while they are not implemented in a portable way.

But we should move in this direction, I think.

> This requirement is the reason for dual-semantics words like FILE S"
> and TO, it is satisfied by the text interpreter for normal words and
> numbers, and it seduces some people to implement STATE-smart immediate
> words.

> OTOH, other people don't want STATE-smart words, which has led
> to word pairs like ' and ['], CHAR and [CHAR].

Do they really ready to use "S(", "[TO]", "[IS]", "[ACTION-OF]" ?
Why don't we see such a code?

> Another option is to
> implement dual-semantics words properly, but this practice is not
> common enough to make it a requirement on systems.

You know my position — dual semantics words can be properly implemented
as immediate STATE-dependent words, if "POSTPONE" is properly
implemented. But it's another discussion.

> In any case, we see this requirement at work as soon as we have a new
> word with special compilation semantics, such as [:. Andrew Haley may
> wish that this was not a requirement, but that does not make it go
> away.

Have you meant a new word with special interpretation semantics?

Since a word with undefined IS may be implemented as a single-semantics
word.

> 3) It must be able to define a word that is equivalent to any other
> word (and maybe add some scaffolding, such as profiling or logging).

Yes. But by this I would not mean that the old word and the new word
both can be used (in the source code level) by a program. In some cases
the old words after redefinition should not be available for a program.

> This is the requirement behind adding SYNONYM and the proposed
> tightening of SYNONYM
> <https://forth-standard.org/proposals/tighten-the-specification-of-synonym-version-1-#contribution-60>.
> It is also a requirement mentioned by Ruvim as a justification for
> some of his positions.

Wil Baden redefined dual semantics words [1] in the following way:

: TO STATE @ IF POSTPONE TO ELSE ['] TO EXECUTE THEN ; IMMEDIATE
: S" STATE @ IF POSTPONE S" ELSE ['] S" EXECUTE THEN ; IMMEDIATE

[1] ONLY STANDARD DEFINITIONS -- https://git.io/JZ4FL

It's a quite portable approach.

A more standard way would be to use "FIND" instead of "[']" and
"POSTPONE". But it seems, this way is less portable at the moment.

Yet another options is to provide the standard basic factors for all
dual-semantics words (or, better, that may be implemented as
dual-semantics words).

> 4) All words are equal. There is no fundamental difference between
> system-defined and user-defined words. Words that deal with words
> (e.g., ' FIND EXECUTE COMPILE,) work on all of them.

In this regard, even the standard words are not equal to each other.

Usually, a user-defined word can be always ticked since the execution
semantics are always defined for it.

But via a Recognizer API, a user can define a "word" that can be handled
by the Forth text interpreter, but that cannot be ticked.

[...]

>
> This list is not exhaustive. Please contribute additional hard
> requirements.

>
> The other direction in which this list can be extended is to go
> deeper, e.g., to add test cases resulting from the requirements.

Yes, it should be the next stage.

--
Ruvim

Should the standard possibly promote standard libraries, much like C's "stdio.h" or C++'s <vector>, to be used with the "REQUIRED" keyword (perhaps involving a search path)? Forth users seem to consistently reinvent the wheel in terms of array types, linked-list searches, maybe even wordlist-style named elements (all of which are likely in use internally to a Forth system, but not exposed as tools for the user).

I feel like a robust collection of standard libraries (we could probably drum up the necessary source implementations, to be improved upon later) would do a lot towards promoting the use of Forth in standards-compliant systems. Since the standard could also provide the source code for such libraries, it doesn't seem like it would be too onerous for system developers except maybe on embedded systems with no disk access.

Of course, namespaces would be nice, or at least name prefixes, like "array..index", "list.find", etc. The default space is getting quite cluttered.

--Jim

Marcel Hendrix <mhx@iae.nl> writes:
>On Wednesday, June 9, 2021 at 1:41:00 PM UTC+2, Anton Ertl wrote:
>[..]
>> Some requirements that come to my mind (in no particular order) are:
>
>OK, let's assume I lost the source to iForth and must rebuild from
>scratch, what kind of Forth would I use for the reboot?
>
>- Introspection:
>Type 1: It should be possible to look at the source for any word (the
> ultimate documentation).

Yes, having LOCATE might be another requirement. Stephen Pelc has
argued for that, and Andrew Haley has even argued that recognized
things should be LOCATEable.

And yet nobody has proposed LOCATE for standardization.

>Type 2: It should be possible for *words* to find out the
> characteristics of any other word (documentation does
> not count). I frequently want to know (in an application)
> the xt and nt of a word (I use ' and head' for that).

FIND-NAME and NAME>INTERPRET have been standardized (FIND-NAME only in
the next standard).

> Concrete questions are: is this system word a LOCAL, a variable,
> a USER, a code definition, where is its data (relative to xt
> or nt), how can I go from body to code and back, etc..

I think that Forth has fallen back in introspection, an area where it
once was at the forefront. So yes, standardizing words for these
purposes is a good idea IMO. But you would also have to convince
others; it could be helpful if you can present use cases.

>> 1) It must be possible to write a user-defined text interpreter in
>> standard Forth.
>[..]
>Given introspection, I can always do that in a system-specific way,
>which would suit me fine.

You can always do that in a system-specific way, because it is done in
the system. But the issue is whether a standard program can do it,
and whether it can do it with FIND.

With newfangled stuff, we can do it as follows:

parse-name 2dup find-name dup if
nip nip state @ if name>compile else name>interpret then execute
else
... deal with numbers etc.
then

Or you can implement it with recognizers.

>> 2) We must be able to copy code between interpreted and
>> compiled code.
>[..]
>Not interesting. When the source of IF is available I'll copy that
>instead.

Maybe I did not make myself clear. When I have interpretively tested,
say

' fluffystunk catch

Let's say I want to copy and paste it into a colon definition

: foo
... some other code ...
' fluffystunk catch
... some other code ... ;

Oops, did not work.

Or I have an colon definition

: foo
create ...
[: ... ;] set-does>
[: ... ;] set-optimizer ;

I want to do it step-by-step:

create bar ...
\ works until now
[: ... ;] set-does>
.... \ test BAR
[: ... ;] set-optimizer
.... \ test compiled BAR

[: ... ;] must have the expected interpretation semntics for this to
work.

I don't know how the source code of IF would help in these examples.

>> This requirement is the reason for dual-semantics words like FILE S"
>> and TO, it is satisfied by the text interpreter for normal words and
>> numbers, and it seduces some people to implement STATE-smart immediate
>> words. OTOH, other people don't want STATE-smart words, which has led
>> to word pairs like ' and ['], CHAR and [CHAR]. Another option is to
>> implement dual-semantics words properly, but this practice is not
>> common enough to make it a requirement on systems.
>[..]
>If all this goes away by admitting that there is a difference between
>interpreted and compiled code that we must know about, that would be
>a huge win.

It's not about admitting. Users know there is a difference,
nevertheless they want to copy and paste between interpreted and
compiled code, and system implementors are more or less accomodating
to this wish (up to STATE-smart IF for interpretive uses of IF,
implemented AFAIK in Open Firmware).

So no, this does not go away by admitting that there is a difference.

>> 3) It must be able to define a word that is equivalent to any other
>> word (and maybe add some scaffolding, such as profiling or logging).
>[..]
>With the source, it is difficult to see where one would get stuck.

Even if you have the source, it is not always practical.

>At this point it becomes clear that there may be a hidden requirement
>in your list: that all these features are portable (i.e you can write
>them *without* introspection and without *vendor support*.) I don't
>care about portability.

It's about requirements on the Forth standard (hidden in the title of
my posting:-), so yes, saying that you can do it on every system in a
system-specific way does not count.

As for introspection: If we standardize words for that, then they can
be used in for satisfying the requirements.

>> 4) All words are equal. There is no fundamental difference between
>> system-defined and user-defined words. Words that deal with words
>> (e.g., ' FIND EXECUTE COMPILE,) work on all of them.
>
>They should be equal in performance.

Performance guarantees have not been standardized, and I think they
are hard to standardize.

>> This was definitely the case in early Forth; you could even tick
>> and then EXECUTE and COMPILE, >R and EXIT.
>[..]
>Definitely no desire to go back to that era if that is the only
>reason.

Even so, one can work on satisfying this requirement. E.g., in iForth
Krishna Myneni's test

: test ['] I 10 0 do dup execute . loop drop cr ;

works.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

Anton Ertl schrieb am Mittwoch, 9. Juni 2021 um 13:41:00 UTC+2:
> In various discussions about the standard (what it means and how it
> should be improved) we have used various requirements as justification
> for the positions we take, but we actually have not discussed these
> requirements much.
>
> Many of the justifications are quite foggy in their actual meaning,
> such as the appeal to simplicity (or denouncement of complexity). I
> am not particularly interested in those, because they don't give any
> real guidance. E.g., wrt simplicity, the simplest Forth system is one
> that can do nothing; obviously there are cases where we consider other
> things more important than simplicity, but the appeal to simplicity
> does not give us any guidance when other things are more important.
>
> But there are also hard requirements from which we can derive test
> cases. I want to dicuss these and maybe find consensus on which of
> these we want to pursue, or alternatively find out that there is no
> consensus on some requirements.
>
> Some requirements that come to my mind (in no particular order) are:

It is not clear whether you are referring to the standard, or want to renovate
Forth.

As to the first it is a hopeless case, because based on the current situation
there will be discussions over the implications of state-dependency and
different semantics even in twenty years. **gag**

As to the the second, start with getting rid of STATE and BASE and make
strings a first-class citizen.

On 2021-06-10 02:08, Marcel Hendrix wrote:
> On Wednesday, June 9, 2021 at 1:41:00 PM UTC+2, Anton Ertl wrote:
> [..]
>> Some requirements that come to my mind (in no particular order) are:

>
> - Introspection:
> Type 1: It should be possible to look at the source for any word (the
> ultimate documentation).

Some systems don't provide the source codes for all words. So, such a
requirement in the standard makes these systems non-standard.

> Type 2: It should be possible for *words* to find out the
> characteristics of any other word (documentation does
> not count). I frequently want to know (in an application)
> the xt and nt of a word (I use ' and head' for that).
> Concrete questions are: is this system word a LOCAL, a variable,
> a USER, a code definition, where is its data (relative to xt
> or nt), how can I go from body to code and back, etc..

What if variables, user-variables, constants, — are implemented as
ordinary words?

For example:

: lit, postpone literal ;
: ;, postpone ; ;

: constant ( x "name" -- ) >r : r> lit, ;, ;
: variable ( "name" -- ) align here 0 , constant ;

: zero-colon 0 ;
0 constant zero-constant

The words "zero-colon" and "zero-constant" are indistinguishable from
each other in regards of their kind in such a system.

OTOH, you can redefine some defining words to associate execution tokens
with the corresponding defining words (if any).

--
Ruvim

In article <b91e9bd3-64ff-4dc0-a4b8-04f76eba5c52n@googlegroups.com>,
Jim Peterson <elkniwt@gmail.com> wrote:
>Should the standard possibly promote standard libraries, much like C's
>"stdio.h" or C++'s <vector>, to be used with the "REQUIRED" keyword
>(perhaps involving a search path)? Forth users seem to consistently
>reinvent the wheel in terms of array types, linked-list searches, maybe
>even wordlist-style named elements (all of which are likely in use
>internally to a Forth system, but not exposed as tools for the user).
>
>I feel like a robust collection of standard libraries (we could probably
>drum up the necessary source implementations, to be improved upon later)
>would do a lot towards promoting the use of Forth in standards-compliant
>systems. Since the standard could also provide the source code for such
>libraries, it doesn't seem like it would be too onerous for system
>developers except maybe on embedded systems with no disk access.
>
>Of course, namespaces would be nice, or at least name prefixes, like
>"array.index", "list.find", etc. The default space is getting quite
>cluttered.

Look at the floating point wordset in the standard.
There is no difference between a well defined wordset and a
library.

So all fat Forth systems like iforth swiftforth mpeforth gforth
would add every standardised wordset in their kernels,
as they do now with the fp wordset.
If anything that is what makes libraries so tricky, they are
hiding in plain sight. Hoever if all libraries are loaded all of the
time what is the use of libraries?

So my answer is:
yes we have standard libraries *already*
yes we have namespaces already; a wordlist defines an anonymous namespace.

(A named wordlist in ciforth is created by the word NAMESPACE.)

So Forth is powerful enough. Roll up your sleeves and get
programming already.

>
>--Jim

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>On 6/9/21 3:52 AM, Anton Ertl wrote:
>> 2) We must be able to copy code between interpreted and
>> compiled code.
>>
>
>Where do you draw the line on this?

The question is where we all draw the line, and if we can find a
consensus on that. There are people like Andrew Haley who advocate
Forth without STATE and without user-definable recognizers, with word
pairs like ' ['] whenever necessary. And then there are people like
Mitch Bradley, which want to copy-and-paste code containing control
structures from compiled to interpreted code, and even finds the
pitfalls of STATE-smartness acceptable to achieve this.

Gforth goes relatively far with that, with return-stack words, [: and
DOES> obeying the copy-paste requirement (none of which we did because
of my personal preferences), but does not make control structures and
anything involving locals obey the copy-paste requirement.

I think that this principle is so important to many system
implementors and users that we should not pretend that we can ignore
it when making decisions in the standard. E.g. one advantage of
recognizers (both the old-fashioned integer and FP recognizers, and
the newfangled user-defined recognizers) is that they obey the
copy-paste requirement. Some people have advocated using parsing
words instead, which leads back to the ' ['] vs. dual-semantics
vs. STATE-smart discussions.

>> 4) All words are equal. There is no fundamental difference between
>> system-defined and user-defined words. Words that deal with words
>> (e.g., ' FIND EXECUTE COMPILE,) work on all of them.
>>
>
>Ticking a word and being able to execute the xt, in a portable way, are
>still two different things.

Sure, and in the context of standardization, the result should also be
portable. Of course, for a number of words systems differ in what
they do when ticking and executing a word, but if we follow this
requirement, we should agree on a standard behaviour in that case.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 6/11/21 3:39 AM, minf...@arcor.de wrote:
> Anton Ertl schrieb am Mittwoch, 9. Juni 2021 um 13:41:00 UTC+2:
>> In various discussions about the standard (what it means and how it
>> should be improved) we have used various requirements as justification
>> for the positions we take, but we actually have not discussed these
>> requirements much.
>>
....
> As to the the second, start with getting rid of STATE and BASE and make
> strings a first-class citizen.
>

Getting rid of STATE won't be easy. I'm not sure why you want to get rid
of BASE. But standardized string operations is long overdue in Forth.

Krishna

On 6/10/21 9:24 AM, Jim Peterson wrote:
....
> Of course, namespaces would be nice, or at least name prefixes, like "array.index", "list.find", etc. The default space is getting quite cluttered.
>

As Albert has mentioned, standard Forth has support for namespaces
through wordlists and search order control. A modular programming
framework which supports name reuse, built on top of standard Forth, is
described at the link below. We have used this framework successfully to
write and use Forth source libraries (widely used throughout the kForth
examples) for different applications.

Krishna

https://github.com/mynenik/kForth-64/blob/master/doc/modular-forth.pdf

In article <s9vij8$5fl$1@dont-email.me>,
Krishna Myneni <krishna.myneni@ccreweb.org> wrote:
>On 6/11/21 3:39 AM, minf...@arcor.de wrote:
>> Anton Ertl schrieb am Mittwoch, 9. Juni 2021 um 13:41:00 UTC+2:
>>> In various discussions about the standard (what it means and how it
>>> should be improved) we have used various requirements as justification
>>> for the positions we take, but we actually have not discussed these
>>> requirements much.
>>>
>...
>> As to the the second, start with getting rid of STATE and BASE and make
>> strings a first-class citizen.
>>
>
>Getting rid of STATE won't be easy. I'm not sure why you want to get rid
>of BASE. But standardized string operations is long overdue in Forth.

I have demonstrated that getting rid of STATE is relatively easy.
INTERPRET in compilation mode just has to check whether an immediate word
has placed something on the stack. Then it runs LITERAL as needed.
This presupposes that literals are handled by the regular dictionary
lookup mechanism, i.e. recognizers.

The main problem with strings is that they may be transient.
One can solve that by always ALLOT ting them, but that may be
non-standard. Anyway, that is what I do, it solves more problems
than it creates. I'm pretty sure that ALLOCATE-ing them is okay.

Marcel Hendrix always uses # $ % instead of BASE. Having BASE in iforth
doesn't hurt for the occasional code he wants to import from elsewhere.
So I'm all for getting rid of unused words, BASE doesn't seem one
of them.

>
>Krishna
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On Friday, June 11, 2021 at 7:10:02 AM UTC-4, none albert wrote:
> In article <b91e9bd3-64ff-4dc0...@googlegroups.com>,
> Jim Peterson <elk...@gmail.com> wrote:
> >Should the standard possibly promote standard libraries, much like C's
> Look at the floating point wordset in the standard.
> There is no difference between a well defined wordset and a
> library.

Is this true? I don't see any way, in source code, to specify the loading and unloading of wordsets. I know that wordlists have that capability, and if the standard specified some wordlists the programmer could call upon for some utilities, that would definitely be a good start, but my main concern is running out of words to use. Many very generic words have already been used to specify rather specific functions. That's why I have to define "list.find" with a "list." prefix, and the trend to forego prefixes in the wordsets (except in the floating point wordset you mention, where the prefix seems to be one of "F", "SF", or "DF", for the most part) makes me worry about the clutter that would occur when what I'm proposing could maybe double or triple the number of words standardized.

> So my answer is:
> yes we have standard libraries *already*
> yes we have namespaces already; a wordlist defines an anonymous namespace..

Maybe we have the capability, but the standard isn't leveraging it to expand the utility of the language.

> So Forth is powerful enough. Roll up your sleeves and get
> programming already.

I've *been* programming. I have a vector library like C++'s <vector>. I have a qsort algorithm, some linked-list code, string "classes" and memory-based stacks... all things that others have already written and I could probably find a good version to use *somewhere*, but why?!? That's my point. Why should I have to write/google/download such things? Are they not something the standard would want to provide (as an option, for those concerned with code space), so that someone starting out in Forth could get actual work done rather than having to define words for arrays and array indexing? (ok, maybe that last one is simple enough that I'm overstating the issue... but still...)

Honestly, I worry that the major appeal of Forth is implementing such basic things and feeling proud of one's reinvention of the wheel... that it is not a language one comes to in order to get work done, but a playground from which to derive a false sense of accomplishment. If the standard were to provide many powerful libraries covering a majority of what was needed, would there be any appeal left in trying to remember the order of items on the data stack so as to get a program working compared to just moving on to something like Python or C++?

The one niche I've seen where Forth excels is from what seems to have been Chuck Moore's original impetus: once a base level of functionality exists on an otherwise new, unprogrammable system, the remainder of Forth can be implemented swiftly, bringing the system up to a level of usability that is not as easily achieved with other programming languages that have more complicated grammars. This may be a key aspect to leverage, possibly taking advatange of (and expanding upon) the forth-on-forth work I've seen in Ruvim's github repo. With embedded systems being all the rage for the past decade or more, being able to bring a new variant up to a level of usability quickly sounds like a job Forth is as suited for today as it was back in the era when it was invented.

--Jim

Jim Peterson <elkniwt@gmail.com> writes:
> The one niche I've seen where Forth excels...: once a base level of
> functionality exists on an otherwise new, unprogrammable system, the
> remainder of Forth can be implemented swiftly, bringing the system up
> to a level of usability that is not as easily achieved with other
> programming languages that have more complicated grammars.

These days you'd bring up a new target using a cross compiler on a
separate computer, so the target system would only need a simple
debugging stub, e.g. something like Frank Sergeant's 3-instruction
Forth. The complexity of the application language wouldn't matter since
the host computer already has a working compiler. I'd also say that the
Forths used by professionals are way too complicated for the approach
you're describing.

On 6/11/21 9:23 AM, Jim Peterson wrote:

> but my main concern is running out of words to use. Many very generic
> words have already been used to specify rather specific functions.

This is true.

> That's why I have to define "list.find" with a "list." prefix,

You don't *have* to. Most of my libraries: arrays, strings, etc. are
written as objects with polymorphism. Words like get, put, insert, and
so on can be used on any data type you want.

-Doug

albert@cherry.(none) (albert) writes:
> The main problem with strings is that they may be transient.
> One can solve that by always ALLOT ting them, but that may be
> non-standard. Anyway, that is what I do, it solves more problems
> than it creates. I'm pretty sure that ALLOCATE-ing them is okay.

If strings are first-class, do you mean there is a separate string stack
like the FP stack? If you say DUP on the string stack (or the data
stack, if the string stack is not separate), does that allocate a new
copy of the string? If it's the data stack, how do you know that a
stack element is a string, so that you can allocate it?

If you use ALLOCATE, do you refcount the strings, or simply copy them
willy-nilly? What do you do about VARIABLEs with the addresses of
strings?

I think first-class strings is easiest if you have garbage collection.

Also, a separate string stack (like the separate FP stack) seems like a
weird antipattern caused by Forth's lack of a type system plus the
intricacy of accessing values inside a stack. So if you're trying to
re-imagine Forth from the ground up, that might be a better place to
start.

On 2021-06-11 11:39, minf...@arcor.de wrote:
>
> As to the the second, start with getting rid of STATE

Do you mean getting rid of dual-semantics words?

Since otherwise it doesn't make any sense.

If you can define a dual-semantics word via separate definitions for
interpretation and compilation semantics, then you can define
"IMMEDIATE" and a kind of "STATE". And vise versa: you can create a
mechanism to separately define interpretation and compilation semantics
for a dual-semantics word using "IMMEDIATE" and "STATE".

Let me know if you need a PoC.

--
Ruvim

On 2021-06-10 17:24, Jim Peterson wrote:
> Should the standard possibly promote standard libraries, much like C's "stdio.h" or C++'s <vector>, to be used with the "REQUIRED" keyword (perhaps involving a search path)? Forth users seem to consistently reinvent the wheel in terms of array types, linked-list searches, maybe even wordlist-style named elements (all of which are likely in use internally to a Forth system, but not exposed as tools for the user).
>
> I feel like a robust collection of standard libraries (we could probably drum up the necessary source implementations, to be improved upon later) would do a lot towards promoting the use of Forth in standards-compliant systems. Since the standard could also provide the source code for such libraries, it doesn't seem like it would be too onerous for system developers except maybe on embedded systems with no disk access.

There are too many question, decisions, and work.

But have a look at some existent suites of libraries:

- Forth Scientific Library (FSL)
https://www.taygeta.com/fsl/sciforth.html

- Forth foundation library (FFL)
https://irdvo.nl/FFL/

- the Forth Net - Package manager and repository for Forth
https://theforth.net/packages

The standard should provide a minimal set of the basic factors to
support libraries managing.
One needed thing is the pathname of the file being currently loaded
(it's required to support relative paths)
Another needed thing is Recognizer API (it's required to support
qualified names and lexical blocks)

>
> Of course, namespaces would be nice, or at least name prefixes, like "array.index", "list.find", etc. The default space is getting quite cluttered.

I use names that are qualified by namespaces in the following form:

ns-a::ns-ab::ns-abc::wordname

Each namespace is represented by a word list.
Usually, I define a short alias (with limited scope).

E.g.
ns-a::ns-ab::ns-abc constant x
And then use
x::wordname

--
Ruvim

On Friday, June 11, 2021 at 6:23:21 AM UTC-7, elk...@gmail.com wrote:
> On Friday, June 11, 2021 at 7:10:02 AM UTC-4, none albert wrote:
> > So Forth is powerful enough. Roll up your sleeves and get
> > programming already.

The Forth-200x committee are opposed to my rquotations. They hate general-purpose data-structures
because Elizabeth Rather hates general-purpose data-structures and they have to appease her.

The primary feature of the rquotations is to access the local variables in the parent function (the function
that called the HOF and where the rquotation was defined), despite the fact that the HOF has local variables
of its own. Typically the HOF will iterate through a data structure and execute the rquotation for every node.
The rquotation will communicate information back to the parent function via the parent function's locals.

> I've *been* programming. I have a vector library like C++'s <vector>. I have a qsort algorithm, some linked-list code,
> string "classes" and memory-based stacks... all things that others have already written and I could probably find
> a good version to use *somewhere*, but why?!? That's my point. Why should I have to write/google/download such things?
> Are they not something the standard would want to provide (as an option, for those concerned with code space),
> so that someone starting out in Forth could get actual work done rather than having to define words for arrays
> and array indexing? (ok, maybe that last one is simple enough that I'm overstating the issue... but still...)
>
> Honestly, I worry that the major appeal of Forth is implementing such basic things and feeling proud of one's reinvention
> of the wheel... that it is not a language one comes to in order to get work done, but a playground from which
> to derive a false sense of accomplishment. If the standard were to provide many powerful libraries covering
> a majority of what was needed, would there be any appeal left in trying to remember the order of items on the data stack
> so as to get a program working compared to just moving on to something like Python or C++?

Please truncate the lines in your post to less than 72 characters. You see all of those right-pointy chars above?
I typed them in myself.

Anyway --- please show us your linked-list code --- implementing a linked-list was attempted previously
by Peter Knaggs at EuroForth, but he failed badly. I doubt that there is anybody on the Forth-200x committee
who knows how to implement a linked list.

I agree with you that code libraries featuring general-purpose data-structures are needed to get any programming done.
This is why I wrote the novice-package. The Forth-200x committee absolutely hates this!

I don't think it is a good idea for the Forth-200x Standard to provide code-libraries as part of the Standard,
primarily because the Forth-200x committee members are idiots not capable of writing code that works.
Also, Elizabeth Rather absolutely hates the concept of code-libraries, and doesn't know what code-libraries are,
so the Forth-200x committee have to appease her in order to stay on the committee.

On Thursday, June 10, 2021 at 3:46:40 PM UTC+2, Ruvim wrote:
> On 2021-06-09 11:52, Anton Ertl wrote:
> > In various discussions about the standard (what it means and how it
> > should be improved) we have used various requirements as justification
> > for the positions we take, but we actually have not discussed these
> > requirements much.
> [...]

> > OTOH, other people don't want STATE-smart words, which has led
> > to word pairs like ' and ['], CHAR and [CHAR].
> Do they really ready to use "S(", "[TO]", "[IS]", "[ACTION-OF]" ?
> Why don't we see such a code?

I have [IS] in iForth because I need it. It always causes problems when I
check out other people's code but I consider that a 'pedantic check' that
I don't mind having. I don't have [TO], but I realize now it is a good idea.
I need it a lot for my current iSPICE project (which hinges run-time code
generation). I think S( would be toooo pedantic for me, and [ACTION-OF]
is too long, I use IS? and [IS?] .

I suppose such useful practical tips should be pointed out more often.

-marcel

On Friday, June 11, 2021 at 12:42:13 PM UTC+2, Ruvim wrote:
> On 2021-06-10 02:08, Marcel Hendrix wrote:
> > On Wednesday, June 9, 2021 at 1:41:00 PM UTC+2, Anton Ertl wrote:
> > [..]
> >> Some requirements that come to my mind (in no particular order) are:
>
> >
> > - Introspection:
> > Type 1: It should be possible to look at the source for any word (the
> > ultimate documentation).
> Some systems don't provide the source codes for all words. So, such a
> requirement in the standard makes these systems non-standard.

Then I won't use them.

> > Type 2: It should be possible for *words* to find out the
> > characteristics of any other word (documentation does
> > not count). I frequently want to know (in an application)
> > the xt and nt of a word (I use ' and head' for that).
> > Concrete questions are: is this system word a LOCAL, a variable,
> > a USER, a code definition, where is its data (relative to xt
> > or nt), how can I go from body to code and back, etc..
> What if variables, user-variables, constants, — are implemented as
> ordinary words?

No problem, they will/should report as a colon-definition or code.
In that case I will simply redefine them so that I actually can modify
their storage locations (that is what I normally need).
It is a bit of a nuisance but it will even work for a USER variable:
: BASE! ( +n -- ) BASE ! ;
: BASE@ ( -- +n ) BASE @ ; etc.

It will NOT work for a LOCAL which might force using globals.
: aLocal! ( u -- ) S" TO aLocal!" EVALUATE ; IMMEDIATE COMPILE-ONLY

> : zero-colon 0 ;
> 0 constant zero-constant
>
> The words "zero-colon" and "zero-constant" are indistinguishable from
> each other in regards of their kind in such a system.

Yes, therefore I'd require words like CONSTANT? taking a nt or xt.
> OTOH, you can redefine some defining words to associate execution tokens
> with the corresponding defining words (if any).

Indeed, that is exactly what I do in iForth's introspection library :-)

-marcel

On Thursday, June 10, 2021 at 3:46:40 PM UTC+2, Ruvim wrote:
> On 2021-06-09 11:52, Anton Ertl wrote:
> > In various discussions about the standard (what it means and how it
> > should be improved) we have used various requirements as justification
> > for the positions we take, but we actually have not discussed these
> > requirements much.
> [...]

> > OTOH, other people don't want STATE-smart words, which has led
> > to word pairs like ' and ['], CHAR and [CHAR].
> Do they really ready to use "S(", "[TO]", "[IS]", "[ACTION-OF]" ?
> Why don't we see such a code?

I have [IS] in iForth because I need it. It always causes problems when I
check out other people's code but I consider that a 'pedantic check' that
I don't mind having. I don't have [TO], but I realize now it is a good idea.
I need it a lot for my current iSPICE project (which hinges on run-time code
generation). I think S( would be toooo pedantic for me, and [ACTION-OF]
is toooo long ( I use IS? and [IS?] ).

I suppose such useful practical tips should be pointed out more often.

-marcel

On Thursday, June 10, 2021 at 7:42:11 PM UTC+2, Anton Ertl wrote:
> Marcel Hendrix <m...@iae.nl> writes:
> >On Wednesday, June 9, 2021 at 1:41:00 PM UTC+2, Anton Ertl wrote:
[..]
> Yes, having LOCATE might be another requirement. Stephen Pelc has
> argued for that, and Andrew Haley has even argued that recognized
> things should be LOCATEable.

I use LOCATE all of the time for my own high-level projects, not only
for kernel words. I admit that having to use INSPECT (to jump into the
kernel source) for un-indexed words is a nuisance, but who wants WORDS
to list all the tiny system factors?

I don't know what I would do for recognizers, I honestly can't find a
convincing use case for them yet.

> And yet nobody has proposed LOCATE for standardization.
> >Type 2: It should be possible for *words* to find out the
> > characteristics of any other word (documentation does
> > not count). I frequently want to know (in an application)
> > the xt and nt of a word (I use ' and head' for that).
> FIND-NAME and NAME>INTERPRET have been standardized (FIND-NAME only in
> the next standard).
> > Concrete questions are: is this system word a LOCAL, a variable,
> > a USER, a code definition, where is its data (relative to xt
> > or nt), how can I go from body to code and back, etc..

They might not help with my listed objective.

> I think that Forth has fallen back in introspection, an area where it
> once was at the forefront. So yes, standardizing words for these
> purposes is a good idea IMO. But you would also have to convince
> others; it could be helpful if you can present use cases.

Apart from being the main reason I like Forth, I use introspection
for run-time code generation, which is a big thing for specialized
simulation and mathematical software (i.e. code dominated by
companies that keep the sources proprietary and feel no need
to advance the state of the art). Introspection is surprisingly
powerful in a simulation tool once one is past the novice stage.
However, I doubt my examples will convince a Forth user.

> >> 1) It must be possible to write a user-defined text interpreter in
> >> standard Forth.
> >[..]
> >Given introspection, I can always do that in a system-specific way,
> >which would suit me fine.
> You can always do that in a system-specific way, because it is done in
> the system. But the issue is whether a standard program can do it,
> and whether it can do it with FIND.
>
> With newfangled stuff, we can do it as follows:
[..]
It would be important if vendors were scheming to screw us, but the
current crop seems to be pretty reasonable. Planning for the next
Forth millenium would indeed make it less of a theoretical concern.

[..]
> >Not interesting. When the source of IF is available I'll copy that
> >instead.
> Maybe I did not make myself clear. When I have interpretively tested,
> say
>
> ' fluffystunk catch
[..]
> I don't know how the source code of IF would help in these examples.

No, but surely all the other source code would?

> >[..]
> >If all this goes away by admitting that there is a difference between
> >interpreted and compiled code that we must know about, that would be
> >a huge win.
> It's not about admitting. Users know there is a difference,
> nevertheless they want to copy and paste between interpreted and
> compiled code, and system implementors are more or less accomodating
> to this wish (up to STATE-smart IF for interpretive uses of IF,
> implemented AFAIK in Open Firmware).

"Want to" ? I can only speak for iForth, and we have not seen any request
for this in the past two decades.

> >[..]
> >With the source, it is difficult to see where one would get stuck.
> Even if you have the source, it is not always practical.

That is true. Maybe quality of implementation would help, but when
at the end of the list of suitable vendors, I might wish for a stronger
Standard.

> >At this point it becomes clear that there may be a hidden requirement
> >in your list: that all these features are portable (i.e you can write
> >them *without* introspection and without *vendor support*.) I don't
> >care about portability.
> It's about requirements on the Forth standard (hidden in the title of
> my posting:-), so yes, saying that you can do it on every system in a
> system-specific way does not count.

I didn't want to spoil my fun by reading that too carefully.
> >> 4) All words are equal. There is no fundamental difference between
> >> system-defined and user-defined words. Words that deal with words
> >> (e.g., ' FIND EXECUTE COMPILE,) work on all of them.
> >
> >They should be equal in performance.
> Performance guarantees have not been standardized, and I think they
> are hard to standardize.

Indeed, a slow system with my wishlist would let me recover
the lost iForth sources just fine.

> >> This was definitely the case in early Forth; you could even tick
> >> and then EXECUTE and COMPILE, >R and EXIT.
> >[..]
> >Definitely no desire to go back to that era if that is the only
> >reason.
> Even so, one can work on satisfying this requirement. E.g., in iForth
> Krishna Myneni's test
>
> : test ['] I 10 0 do dup execute . loop drop cr ;
>
> works.
> - anton

There is no guarantee that it works for everything a user could come
up with, and there is no guarantee that what works now will work in a
next release.

-marcel

In article <87zgvw5dkm.fsf@nightsong.com>,
Paul Rubin <no.email@nospam.invalid> wrote:
>albert@cherry.(none) (albert) writes:
>> The main problem with strings is that they may be transient.
>> One can solve that by always ALLOT ting them, but that may be
>> non-standard. Anyway, that is what I do, it solves more problems
>> than it creates. I'm pretty sure that ALLOCATE-ing them is okay.
>
>If strings are first-class, do you mean there is a separate string stack
>like the FP stack? If you say DUP on the string stack (or the data
>stack, if the string stack is not separate), does that allocate a new
>copy of the string? If it's the data stack, how do you know that a
>stack element is a string, so that you can allocate it?
>
>If you use ALLOCATE, do you refcount the strings, or simply copy them
>willy-nilly? What do you do about VARIABLEs with the addresses of
>strings?
>
>I think first-class strings is easiest if you have garbage collection.
>
>Also, a separate string stack (like the separate FP stack) seems like a
>weird antipattern caused by Forth's lack of a type system plus the
>intricacy of accessing values inside a stack. So if you're trying to
>re-imagine Forth from the ground up, that might be a better place to
>start.

I have 8 Gbyte internal space. Default 1/5 is used for ALLOCATE.
The few strings that one types in oneself present a negligable memory leak.
Memory leaks are only a problem if they are a problem, not because
the Creator has an 53th commandment, communicated by self appointed
high priests.
So if I went for ALLOCATEd strings, that would not imply strings
stacks,garbage collection or worrying about pointers to those
permanent strings. If I say "AAP" and the string gets ALLOCATEd
that does say "first-class strings" as if we were talking about
lisp instead of Forth. It just says that you do not need to
worry that the string vanishes under a host of "system defined"
circumstances.

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst