Gerry Jackson <do-not-use@swldwa.uk> writes:
>Why does Gforth need a header for a nameless word?

It needs at least the code field in order to be able to EXECUTE the xt
of the word. It also needs the vt address and vt (which is typically
shared with other words) in order to be able to COMPILE, the xt of the
word. And that's what a nameless word has in terms of header fields.

Conceptually, there is also the idea that all headers are equal. In
an earlier version of the new Gforth header, this meant that headers
of nameless words still had count and link fields; but I think that in
the current version, this is abstracted into a NAME>STRING method that
return the address and length of the name (0 0 for nameless words),
and a NAME>LINK method that returns the link to the previous word, so
this conceptual equality does not cost much RAM.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 8/9/21 3:36 PM, Anton Ertl wrote:
> Gerry Jackson <do-not-use@swldwa.uk> writes:
>> Why does Gforth need a header for a nameless word?
>
> It needs at least the code field in order to be able to EXECUTE the xt
> of the word. It also needs the vt address and vt (which is typically
> shared with other words) in order to be able to COMPILE, the xt of the
> word. And that's what a nameless word has in terms of header fields.
>
> Conceptually, there is also the idea that all headers are equal. In
> an earlier version of the new Gforth header, this meant that headers
> of nameless words still had count and link fields; but I think that in
> the current version, this is abstracted into a NAME>STRING method that
> return the address and length of the name (0 0 for nameless words),
> and a NAME>LINK method that returns the link to the previous word, so
> this conceptual equality does not cost much RAM.
>

What should the following code do?

variable xtFoo
marker m1
: foo ." foo" ;
' foo xtFoo !
:noname xtFoo @ execute ;
m1
execute \ execute the anonymous definition

Presumably the header for FOO is removed from the dictionary when M1 is
executed. In Gforth, I notice that the last EXECUTE in the above code
still works despite FOO no longer being in the dictionary.

In kForth, when the header for a word is removed from the dictionary,
e.g. using FORGET, the xt for the word is no longer valid. In contrast,
:NONAME definitions and quotations, which are derived from :NONAME
definitions, have no headers, and may always be executed provided the
named words which their definitions reference still exist in the
dictionary. So, for example, we can't do the following in kForth,

: foo ." foo" ;
:noname foo ;
forget foo
execute \ will cause error because the word FOO no longer exists.

However, the following will work just fine in kForth because there is no
header to remove for the :NONAME definition,

: bar ." bar" ;
: foo ." foo" ;
:noname bar ;
forget foo
execute

The xt associated with a :NONAME definition may be stored and used
without regard to MARKER or FORGET. Perhaps this is true in Gforth as well.

--
Krishna

On 8/9/21 6:08 PM, Krishna Myneni wrote:
> On 8/9/21 3:36 PM, Anton Ertl wrote:
>> Gerry Jackson <do-not-use@swldwa.uk> writes:
>>> Why does Gforth need a header for a nameless word?
>>
>> It needs at least the code field in order to be able to EXECUTE the xt
>> of the word.  It also needs the vt address and vt (which is typically
>> shared with other words) in order to be able to COMPILE, the xt of the
>> word.  And that's what a nameless word has in terms of header fields.
>>
>> Conceptually, there is also the idea that all headers are equal.
....
> The xt associated with a :NONAME definition may be stored and used
> without regard to MARKER or FORGET. Perhaps this is true in Gforth as well.
>

Ok. I just answered my question with regards to Gforth's behavior. The
following does not work in Gforth:

: foo ." foo" ;
marker m1
:noname foo ;
m1
create arr 1000 allot
arr 1000 erase
execute

causes an Invalid memory address error. Similarly, the following also
does not work in Gforth:

marker m2
:noname 2 3 + . ;
m2
create arr2 1000 allot
arr2 1000 erase
execute

produces an invalid memory error.

In contrast, the :NONAME definitions persist in kForth, owing to their
lack of headers.

: foo ." foo" ;
: m1 ;
:noname foo ;
forget m1
create arr 1000 allot
arr 1000 erase
execute \ types "foo"

: m2 ;
:noname 2 3 + . ;
forget m2
create arr2 1000 allot
arr2 1000 erase
execute \ prints 5

It's worth noting that there can be a difference in the behavior of
Forth systems which provide headers for :NONAME definitions and those
which don't when MARKER and FORGET are involved.

--
Krishna

On 10/08/2021 01:49, Hans Bezemer wrote:
> On Monday, August 9, 2021 at 2:46:27 PM UTC+2, Gerry Jackson wrote:
>> Why does Gforth need a header for a nameless word?
>
> I wouldn't know. Just branch over the shebang.
>
> Addr| Opcode Operand
>
> 0| branch 10
> 1| dup 0
> 2| literal 4
> 3| branch 7
> 4| swap 0
> 5| - 0
> 6| exit 0
> 7| swap 0
> 8| ! 0
> 9| exit 0
>
> You guys do have AHEAD, don't you?
>

Avoid quotations and the whole shebang goes away.

Krishna Myneni <krishna.myneni@ccreweb.org> writes:
>On 8/9/21 3:36 PM, Anton Ertl wrote:
>> Gerry Jackson <do-not-use@swldwa.uk> writes:
>>> Why does Gforth need a header for a nameless word?
>>
>> It needs at least the code field in order to be able to EXECUTE the xt
>> of the word. It also needs the vt address and vt (which is typically
>> shared with other words) in order to be able to COMPILE, the xt of the
>> word. And that's what a nameless word has in terms of header fields.
>>
>> Conceptually, there is also the idea that all headers are equal. In
>> an earlier version of the new Gforth header, this meant that headers
>> of nameless words still had count and link fields; but I think that in
>> the current version, this is abstracted into a NAME>STRING method that
>> return the address and length of the name (0 0 for nameless words),
>> and a NAME>LINK method that returns the link to the previous word, so
>> this conceptual equality does not cost much RAM.
>>
>
>What should the following code do?
>
>variable xtFoo
>marker m1
>: foo ." foo" ;
>' foo xtFoo !
>:noname xtFoo @ execute ;
>m1
>execute \ execute the anonymous definition
>
>Presumably the header for FOO is removed from the dictionary when M1 is
>executed. In Gforth, I notice that the last EXECUTE in the above code
>still works despite FOO no longer being in the dictionary.

As you found out, markers don't overwrite the memory that they
reclaim, so until something else overwrites some of this memory, words
in that memory still work.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On Friday, August 6, 2021 at 11:56:05 AM UTC+3, Anton Ertl wrote:
> We have added multiple sections [ertl-sections16] to Gforth. Sections
> are the same concept as in assembly language and binary formats
> (e.g. ELF): Classical Forth has one section=dictionary: A stretch of
> memory from the start to HERE, with the address space behind available
> for future allocation. Each section works like the dictionary: It has
> a stretch of allocated memory, and behind that there is memory
> available for future allocation.
>
> Gforth offers the following interface for sections: There is a current
> section (by default the ordinary dictionary), and HERE and UNUSED
> produce the results for the current section, and ALLOT etc. work on
> that section. You can temporarily change the section.
>
In FlashForth I have three sections RAM, FLASH, EEPROM.
It was a design decision I made when I started writing it in 2005.
There is a current section pointer which allows ALLOT , HERE CREATE UNUSED to address the current section.
Code and headers go to FLASH, data goes to the current section.
MARKER stores and restores the section pointers and dictionary pointer and the turnkey.
FORGET works only with the FLASH section.

On 2021-08-06 10:50, Anton Ertl wrote:
> We have added multiple sections [ertl-sections16] to Gforth. Sections
> are the same concept as in assembly language and binary formats
> (e.g. ELF): Classical Forth has one section=dictionary: A stretch of
> memory from the start to HERE, with the address space behind available
> for future allocation.

If data space and code space are separated in a Forth system, do you
consider them as one section, or two sections?

The question arises since a section (in binary formats) usually consists
of a single address range in memory.

> Each section works like the dictionary: It has
> a stretch of allocated memory, and behind that there is memory
> available for future allocation.

If a section works like the dictionary, it looks attractive to call it
"dictionary".

One problem is that "dictionary", "data space", "code space" are
singletons in the standard terminology, and the dictionary *contains*
data space and code space. So, if we introduce multiple dictionaries,
then data space and code space are not singletons anymore too. So the
terminology should be adjusted somehow, i.e., a dictionary (as a data
structure) does not contain data space and code space any more, but vice
versa, it's contained by them.

> Gforth offers the following interface for sections: There is a current
> section (by default the ordinary dictionary), and HERE and UNUSED
> produce the results for the current section, and ALLOT etc. work on
> that section. You can temporarily change the section.

In my very old extension for SP-Forth/4 the objects similar to your
sections can be created and destroyed (freed) by the user.

One problem that can arise is spreading a word list among several
sections, when some of them can be destroyed.

So, to create words in a temporary section, the user should create a
word list per each temporary section and retain the association of the
word list and the section, and change both the current section and the
compilation word list at the same time.

To make things simpler, this works automatically in my implementation,
i.e. the associated word list is created for new section, and the
compilation word list is changed automatically according to change of
the current section. When the current section is restored, the
compilation word list is restored too (what it was before).

In this regard, the questions are as following:

1. Can the compilation word list be local for the current section?

2. Can we allow a Forth system to prevent spreading a word list among
several sections?

--
Ruvim

Ruvim schrieb am Montag, 27. September 2021 um 10:46:49 UTC+2:
> On 2021-08-06 10:50, Anton Ertl wrote:
> > We have added multiple sections [ertl-sections16] to Gforth. Sections
> > are the same concept as in assembly language and binary formats
> > (e.g. ELF): Classical Forth has one section=dictionary: A stretch of
> > memory from the start to HERE, with the address space behind available
> > for future allocation.
> If data space and code space are separated in a Forth system, do you
> consider them as one section, or two sections?
>
> The question arises since a section (in binary formats) usually consists
> of a single address range in memory.
> > Each section works like the dictionary: It has
> > a stretch of allocated memory, and behind that there is memory
> > available for future allocation.
> If a section works like the dictionary, it looks attractive to call it
> "dictionary".
>
> One problem is that "dictionary", "data space", "code space" are
> singletons in the standard terminology, and the dictionary *contains*
> data space and code space. So, if we introduce multiple dictionaries,
> then data space and code space are not singletons anymore too. So the
> terminology should be adjusted somehow, i.e., a dictionary (as a data
> structure) does not contain data space and code space any more, but vice
> versa, it's contained by them.
> > Gforth offers the following interface for sections: There is a current
> > section (by default the ordinary dictionary), and HERE and UNUSED
> > produce the results for the current section, and ALLOT etc. work on
> > that section. You can temporarily change the section.
> In my very old extension for SP-Forth/4 the objects similar to your
> sections can be created and destroyed (freed) by the user.
>
> One problem that can arise is spreading a word list among several
> sections, when some of them can be destroyed.
>
> So, to create words in a temporary section, the user should create a
> word list per each temporary section and retain the association of the
> word list and the section, and change both the current section and the
> compilation word list at the same time.
>
> To make things simpler, this works automatically in my implementation,
> i.e. the associated word list is created for new section, and the
> compilation word list is changed automatically according to change of
> the current section. When the current section is restored, the
> compilation word list is restored too (what it was before).
>
> In this regard, the questions are as following:
>
> 1. Can the compilation word list be local for the current section?
>
> 2. Can we allow a Forth system to prevent spreading a word list among
> several sections?
>

Adding to that:
The more sections you have, the more unused section space you have.
And each section can run out of space individually.
This could become an issue in memory-constrained systems.

On 2021-09-27 12:03, minf...@arcor.de wrote:
> Ruvim schrieb am Montag, 27. September 2021 um 10:46:49 UTC+2:
>> On 2021-08-06 10:50, Anton Ertl wrote:
>>> We have added multiple sections [ertl-sections16] to Gforth. Sections
>>> are the same concept as in assembly language and binary formats
>>> (e.g. ELF): Classical Forth has one section=dictionary: A stretch of
>>> memory from the start to HERE, with the address space behind available
>>> for future allocation.
>> If data space and code space are separated in a Forth system, do you
>> consider them as one section, or two sections?
>>
>> The question arises since a section (in binary formats) usually consists
>> of a single address range in memory.
>>> Each section works like the dictionary: It has
>>> a stretch of allocated memory, and behind that there is memory
>>> available for future allocation.
>> If a section works like the dictionary, it looks attractive to call it
>> "dictionary".
>>
>> One problem is that "dictionary", "data space", "code space" are
>> singletons in the standard terminology, and the dictionary *contains*
>> data space and code space. So, if we introduce multiple dictionaries,
>> then data space and code space are not singletons anymore too. So the
>> terminology should be adjusted somehow, i.e., a dictionary (as a data
>> structure) does not contain data space and code space any more, but vice
>> versa, it's contained by them.
>>> Gforth offers the following interface for sections: There is a current
>>> section (by default the ordinary dictionary), and HERE and UNUSED
>>> produce the results for the current section, and ALLOT etc. work on
>>> that section. You can temporarily change the section.
>> In my very old extension for SP-Forth/4 the objects similar to your
>> sections can be created and destroyed (freed) by the user.
>>
>> One problem that can arise is spreading a word list among several
>> sections, when some of them can be destroyed.
>>
>> So, to create words in a temporary section, the user should create a
>> word list per each temporary section and retain the association of the
>> word list and the section, and change both the current section and the
>> compilation word list at the same time.
>>
>> To make things simpler, this works automatically in my implementation,
>> i.e. the associated word list is created for new section, and the
>> compilation word list is changed automatically according to change of
>> the current section. When the current section is restored, the
>> compilation word list is restored too (what it was before).
>>
>> In this regard, the questions are as following:
>>
>> 1. Can the compilation word list be local for the current section?
>>
>> 2. Can we allow a Forth system to prevent spreading a word list among
>> several sections?
>>
>
> Adding to that:
> The more sections you have, the more unused section space you have.
> And each section can run out of space individually.
> This could become an issue in memory-constrained systems.
>

Yes, of course.
But a memory-constrained system might not provide this feature at all.

Concerning the API.

If we associate a section (a dictionary?) with a word list (as it was
implemented in SP-Forth/4), then changing the compilation word list will
change the current dictionary too. And there is no need for other words
to control the current dictionary at all.

The only required new words are to create a word list with its own
dictionary, and to free such a word list (with its dictionary).
Something as follows:

allocate-wordlist ( u-datasize u-codesize -- wid ior )
free-wordlist ( wid -- ior )

The actually allocated memory can be greater than the given sizes, since
the system can reserve additional space to its own needs.

Management of dictionaries remains under the hood.

If we want to create a dictionary that is saved into the image, we can
have something as:

allocate-wordlist-persistent ( u-datasize u-codesize -- wid ior )

A question is: does somebody need the ability to spread one word list
among several dictionaries? I.e., that some words from the word list sit
in one dictionary/section, and other words in another dictionary/section.

--
Ruvim

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2021-08-06 10:50, Anton Ertl wrote:
>> We have added multiple sections [ertl-sections16] to Gforth. Sections
>> are the same concept as in assembly language and binary formats
>> (e.g. ELF): Classical Forth has one section=dictionary: A stretch of
>> memory from the start to HERE, with the address space behind available
>> for future allocation.
>
>If data space and code space are separated in a Forth system, do you
>consider them as one section, or two sections?

Two.

>> Each section works like the dictionary: It has
>> a stretch of allocated memory, and behind that there is memory
>> available for future allocation.
>
>
>If a section works like the dictionary, it looks attractive to call it
>"dictionary".

More precisely, I should have written: Each section works like the
dictionary in classical Forth (which has no separate code space, data
space, name space etc.).

A multi-section Forth system still has only has one dictionary,
distributed across several sections.

>So, if we introduce multiple dictionaries,

We don't. Gforth has multiple sections.

>One problem that can arise is spreading a word list among several
>sections, when some of them can be destroyed.

Then don't.

>So, to create words in a temporary section, the user should create a
>word list per each temporary section and retain the association of the
>word list and the section, and change both the current section and the
>compilation word list at the same time.

No temporary sections in Gforth, and the idea is not to have
per-section wordlists (or per-wordlist sections), but to use sections
to avoid branching around strings and quotations, and for storing
locals names.

>1. Can the compilation word list be local for the current section?
>
>2. Can we allow a Forth system to prevent spreading a word list among
>several sections?

Given that sections are a non-standard feature, a system can do
whatever it wants. However, in the interest of communication clarity
we should use different names for different things.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

"minf...@arcor.de" <minforth@arcor.de> writes:
>The more sections you have, the more unused section space you have.
>And each section can run out of space individually.
>This could become an issue in memory-constrained systems.

If a section runs out of memory, you configure more memory for it on
the next startup. You may need to configure less memory for some
other section (which can then overrun, and so on until you have found
the right sizes for all the sections). That's inconvenient, so you
won't implement sections unless the benefits are worth this cost.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> If a section runs out of memory, you configure more memory for it on
> the next startup.

This issue was traditionally handled by linkers that would relocate the
addresses after the assembler had figured out the sizes of each section,
so they would end up contiguous in memory. Is that terribly
unreasonable for Forth, which has always wanted to do everything in a
single pass?

On 28/09/2021 06:42, Paul Rubin wrote:
> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> If a section runs out of memory, you configure more memory for it on
>> the next startup.
>
> This issue was traditionally handled by linkers that would relocate the
> addresses after the assembler had figured out the sizes of each section,
> so they would end up contiguous in memory. Is that terribly
> unreasonable for Forth, which has always wanted to do everything in a
> single pass?
>

LMI had run-time relocatable binary modules:

The Binary Overlay manager consists of separate modules which create
and load Forth binary overlay files. BSAVE creates a binary
overlay file. BGET loads a binary overlay file, relocates it and
links it into the dictionary. BRUN loads an overlay file using
BGET and then executes a specified definition within the overlay.

Paul Rubin <no.email@nospam.invalid> writes:
>anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> If a section runs out of memory, you configure more memory for it on
>> the next startup.
>
>This issue was traditionally handled by linkers that would relocate the
>addresses after the assembler had figured out the sizes of each section,
>so they would end up contiguous in memory. Is that terribly
>unreasonable for Forth, which has always wanted to do everything in a
>single pass?

The problem (at least for the kind of section usage in Gforth) is that
on creating an image, you get addresses that point to somewhere in a
section, and can use them anytime, so resizing (with relocation) the
section at run-time is not possible (whereas assemblers and linkers
probably can).

In Gforth we then save the image, and by comparing images, create a
relocatable image, so on loading that image you can set the sizes of
the sections (but there is no user facility for doing that yet).

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2021-09-27 22:32, Anton Ertl wrote:
> Ruvim <ruvim.pinka@gmail.com> writes:
>> On 2021-08-06 10:50, Anton Ertl wrote:
>>> We have added multiple sections [ertl-sections16] to Gforth. Sections
>>> are the same concept as in assembly language and binary formats
>>> (e.g. ELF): Classical Forth has one section=dictionary: A stretch of
>>> memory from the start to HERE, with the address space behind available
>>> for future allocation.
>>
>> If data space and code space are separated in a Forth system, do you
>> consider them as one section, or two sections?
>
> Two.
>
>>> Each section works like the dictionary: It has
>>> a stretch of allocated memory, and behind that there is memory
>>> available for future allocation.
>>
>>
>> If a section works like the dictionary, it looks attractive to call it
>> "dictionary".
>
> More precisely, I should have written: Each section works like the
> dictionary in classical Forth (which has no separate code space, data
> space, name space etc.).
>
> A multi-section Forth system still has only has one dictionary,
> distributed across several sections.

[...]

>> So, to create words in a temporary section, the user should create a
>> word list per each temporary section and retain the association of the
>> word list and the section, and change both the current section and the
>> compilation word list at the same time.
>
> No temporary sections in Gforth, and the idea is not to have
> per-section wordlists (or per-wordlist sections), but to use sections
> to avoid branching around strings and quotations, and for storing
> locals names.

Don't you free local names after using?

>
>> 1. Can the compilation word list be local for the current section?
>>
>> 2. Can we allow a Forth system to prevent spreading a word list among
>> several sections?
>
> Given that sections are a non-standard feature, a system can do
> whatever it wants.

Of course. I just think concerning possible compatibility, since these
general idea already implemented in several Forth systems, and it can be
useful for applications.

Having such a feature, an application has a portable way:
- to have a dictionary of required size;
- to hot reload some parts from the sources;
- to have some components (in object code) temporary in memory.

I use hot reloading in a web application server.

> However, in the interest of communication clarity
> we should use different names for different things.

Yes.

So, a section having a single address range of memory, cannot play the
role of the dictionary in any Forth system.

It seems, a portable API cannot be established for sections. What do you
think?

--
Ruvim

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2021-09-27 22:32, Anton Ertl wrote:
>> More precisely, I should have written: Each section works like the
>> dictionary in classical Forth (which has no separate code space, data
>> space, name space etc.).
>>
>> A multi-section Forth system still has only has one dictionary,
>> distributed across several sections.
....
>> No temporary sections in Gforth, and the idea is not to have
>> per-section wordlists (or per-wordlist sections), but to use sections
>> to avoid branching around strings and quotations, and for storing
>> locals names.
>
>Don't you free local names after using?

Yes, Gforth frees the local names at the end of the colon definition.
That's easy given that they are in a separate section. Actually
that's pretty close to your idea of independently manageable
wordlist-sections, whereas other sections in Gforth are much more
intertwined (and therefore not independently manageable).

Why use a section for locals rather than ALLOCATEd memory (which is
what Gforth used earlier)? A section is part of the dictionary, and
therefore ALLOT works as a matter of course, including checking for
dictionary overflow. And creating a name calls ALLOT. With ALLOCATE
we needed quite a bit of extra code for implementing ALLOT in the
ALLOCATEd memory. Also, freeing the ALLOCATEd memory is more
complicated, because we used one ALLOCATE per name.

>Of course. I just think concerning possible compatibility, since these
>general idea already implemented in several Forth systems, and it can be
>useful for applications.

Well, the interface that Gforth has is relatively simple: You switch
to a section, and all ALLOTs then go into that section.

>So, a section having a single address range of memory, cannot play the
>role of the dictionary in any Forth system.

In mant Forth systems, the dictionary has only one section (of course
we don't speak about sections in that case), so I wonder why you write
that.

>It seems, a portable API cannot be established for sections. What do you
>think?

I see no reason why not. Of course, it may turn out that the systems
do something else with multiple address ranges, and their implementors
may not see the need for an API like Gforth offers. This could be
addressed by using different names for the different APIs. Of course,
the question remains if any system beyond Gforth will implement the
Gforth section API.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 01/10/2021 17:11, Anton Ertl wrote:
> Well, the interface that Gforth has is relatively simple: You switch
> to a section, and all ALLOTs then go into that section.
>

In my implementation of sections the interface includes HERE ALLOT ALIGN
C, , UNUSED (plus a few other words) that all operate on the current
section and not just dataspace - perhaps you do the same.

--
Gerry

Gerry Jackson <do-not-use@swldwa.uk> writes:
>On 01/10/2021 17:11, Anton Ertl wrote:
>> Well, the interface that Gforth has is relatively simple: You switch
>> to a section, and all ALLOTs then go into that section.
>>
>
>In my implementation of sections the interface includes HERE ALLOT ALIGN
>C, , UNUSED (plus a few other words) that all operate on the current
>section and not just dataspace - perhaps you do the same.

Yes, Gforth does the same:

unused . \ 7821090 ok
next-section unused . \ 2077250 ok
previous-section unused . \ 7821090 ok
' unused locals-headers . \ 15290 ok
unused . \ 7821090 ok

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2021-10-01 19:11, Anton Ertl wrote:
> Ruvim <ruvim.pinka@gmail.com> writes:
>> On 2021-09-27 22:32, Anton Ertl wrote:
>>> More precisely, I should have written: Each section works like the
>>> dictionary in classical Forth (which has no separate code space, data
>>> space, name space etc.).
>>>
>>> A multi-section Forth system still has only has one dictionary,
>>> distributed across several sections.
> ...
>>> No temporary sections in Gforth, and the idea is not to have
>>> per-section wordlists (or per-wordlist sections), but to use sections
>>> to avoid branching around strings and quotations, and for storing
>>> locals names.
>>
>> Don't you free local names after using?
>
> Yes, Gforth frees the local names at the end of the colon definition.
> That's easy given that they are in a separate section. Actually
> that's pretty close to your idea of independently manageable
> wordlist-sections, whereas other sections in Gforth are much more
> intertwined (and therefore not independently manageable).
>
> Why use a section for locals rather than ALLOCATEd memory (which is
> what Gforth used earlier)? A section is part of the dictionary, and
> therefore ALLOT works as a matter of course, including checking for
> dictionary overflow. And creating a name calls ALLOT. With ALLOCATE
> we needed quite a bit of extra code for implementing ALLOT in the
> ALLOCATEd memory. Also, freeing the ALLOCATEd memory is more
> complicated, because we used one ALLOCATE per name.
>
>> Of course. I just think concerning possible compatibility, since these
>> general idea already implemented in several Forth systems, and it can be
>> useful for applications.
>
> Well, the interface that Gforth has is relatively simple: You switch
> to a section, and all ALLOTs then go into that section.

In Gforth, code generation also goes into that section.

But if a system separates code and data, code generation will not go
into that section, but remains in the original section.

I just don't see a common basis on this abstraction level.

>> So, a section having a single address range of memory, cannot play the
>> role of the dictionary in any Forth system.
>
> In mant Forth systems, the dictionary has only one section (of course
> we don't speak about sections in that case), so I wonder why you write
> that.

I wanted to say, the abstraction on the sections level seems too tight
to be compatible with all possible standard systems.

>
>> It seems, a portable API cannot be established for sections. What do you
>> think?
>
> I see no reason why not. Of course, it may turn out that the systems
> do something else with multiple address ranges, and their implementors
> may not see the need for an API like Gforth offers. This could be
> addressed by using different names for the different APIs. Of course,
> the question remains if any system beyond Gforth will implement the
> Gforth section API.

The sections that are *automatically* used for string literals,
quotations or other things, are an internal business of a Forth system.
A portable program should not directly manage these section, since the
program doesn't depend on whether the system uses them or not. In the
same way a program doesn't depend whether the system uses peephole
optimization, or not. Usually, a configuration for such features (if
any) is implementation-defined.

In principle, a portable configuration is possible too. Probably, it's
just not too much in demand.

One essential requirement for a portable configuration is that this
configuration should not depend on whether a feature being configured is
actually used by the system or not. Hence, the corresponding mechanism
should be able to skip unknown options.

Unlike configurations, when I talk about an API, I imagine a feature
that can be actually used by a program. I.e., if a program relies on the
feature, it cannot work without this feature.

Also, a portable API should be quite abstract to be compatible with any
allowed implementation of a standard system. E.g., if the standard
allows disjoint data space and code space, the API should not be
relevant to only the systems having joint data space and code space.

Concerning sections, a more appealing feature for programs is not just
to have separate parts in data space, but separate parts of dictionary
that can be used also to compile definitions. That is, creating a word
in one part doesn't change anything in other parts, and performing a
marker (or "forget") in one part doesn't change anything in other parts.

In Gforth this feature can be easy implemented via sections. But such a
part of the dictionary cannot be called "section", since in general case
it may consist of many sections internally.

So it seems this feature cannot be formalized only in terms of sections
in a portable manner.

--
Ruvim

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2021-10-01 19:11, Anton Ertl wrote:
>> Well, the interface that Gforth has is relatively simple: You switch
>> to a section, and all ALLOTs then go into that section.
>
>In Gforth, code generation also goes into that section.

The threaded code goes there, yes.

>But if a system separates code and data, code generation will not go
>into that section, but remains in the original section.

Why should it?

In Gforth the native code is in something that's not quite a section:
It consist of a sequence of fixed-size blocks and is therefore only
limited by total memory size. This works because we can insert a jump
to the next block if we run out of space in one block. And it's
somewhat complicated. A simpler solution would be to put the native
code for a section in another section, but then you could run of of
space in the native-code section.

For dealing with the native code of quotations there are many
different options, depending on other implementation details of the
system.

* E.g., a system could produce native code directly while compiling.
To separate code and data, the code could be generated in the next
section (from the header). Strings could be in a special strings
section, or back in the data section (no need to jump around the
string in the native code). To avoid jumping around quotations,
first-level quotations could be in the next section up from the first
native-code section (and so on).

* OTOH, a system that compiles to intermediate code that is only
compiled to native code later (e.g., to facilitate inlining) might
use sections for quotations at the intermediate level and strings,
and it might put the native code all in one section, because it can
compile the quotation before or after the word or quotation that
calls it, avoiding the need to jump around. But it might also keep
the intermediate code in one section and encode the jumps around
strings and quotations in a way that is optimized away when
generating the native code.

* Gforth has only one native code "section" (as described above, it's
not quite a section). But in the threaded code there is no jump
around a quotation (because the quotation is in a different section
on the threaded-code level), so Gforth doesn't compile a
threaded-code jump into the native code. Instead, it just compiles
a threaded-code dispatch, which means that the native code
corresponding to the next threaded-code primitive is executed, and
that native code is after the native code of the quotation. So on
the native-code level there is a jump, it's just not as expensive as
a jump on the threaded-code level.

>The sections that are *automatically* used for string literals,
>quotations or other things, are an internal business of a Forth system.

Yes. For now, Gforth sections are very much oriented towards out
internal needs, and are somewhat limited for other uses (e.g., better
don't try to compile colon definitions in named sections).

>Concerning sections, a more appealing feature for programs is not just
>to have separate parts in data space, but separate parts of dictionary
>that can be used also to compile definitions. That is, creating a word
>in one part doesn't change anything in other parts

Well, we are not far from that.

>and performing a
>marker (or "forget") in one part doesn't change anything in other parts.

Resetting to a specific point in time has always been a characteristic
of marker and forget, for better or for worse. These days, they are
just unneeded baggage IMO.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 5/10/2021 03:32, Anton Ertl wrote:
> Ruvim <ruvim.pinka@gmail.com> writes:
> ...
>>and performing a
>>marker (or "forget") in one part doesn't change anything in other parts.
>
> Resetting to a specific point in time has always been a characteristic
> of marker and forget, for better or for worse. These days, they are
> just unneeded baggage IMO.

EMPTY (a special case of FORGET/MARKER) precedes every application
I've written. In conjunction with a built-in editor it provides
for a fast compile/edit cycle.

On 2021-10-04 19:32, Anton Ertl wrote:
> Ruvim <ruvim.pinka@gmail.com> writes:
>> On 2021-10-01 19:11, Anton Ertl wrote:
>>> Well, the interface that Gforth has is relatively simple: You switch
>>> to a section, and all ALLOTs then go into that section.
>>
>> In Gforth, code generation also goes into that section.
>
> The threaded code goes there, yes.
>
>> But if a system separates code and data, code generation will not go
>> into that section, but remains in the original section.
>
> Why should it?

Just to make sense for programs.

If code generation goes into the old section, then this feature does not
provide any substantially new capabilities to a *program*.

If both data and code generation go into the single new section, then
this way isn't compatible with some implementations, since it unites
data and code.

Actually, a capability to create a data section with a set of methods
like ",", "C,", "ALLOT", etc, can be useful for a program. Such a
section has the following advantages:
- it's completely separated from code generation;
- it provides a convenient way to build compound data objects;
- it allows to avoid allocating many small blocks in the heap;
- it allows to easy clear recently created objects in LIFO order;
- it allows to easy clear all objects (a whole section) at once;
- multiple sections can be used independently in the same time.
Thus, it's just a technique of memory management. I used this approach
in some applications.

One nuance is that this technique can be already implemented in a
portable way, as a reusable component.

So a more attractive option is to manage parts of dictionary that
include both code and data (and their mutual relation should remain
unspecified for compatibility with any implementation approach).

> In Gforth the native code is in something that's not quite a section:
> It consist of a sequence of fixed-size blocks and is therefore only
> limited by total memory size. This works because we can insert a jump
> to the next block if we run out of space in one block. And it's
> somewhat complicated. A simpler solution would be to put the native
> code for a section in another section, but then you could run of of
> space in the native-code section.

The problem of preset limited size is not so critical if a program/user
can configure initial size or create new sections of required size.

>
> For dealing with the native code of quotations there are many
> different options, depending on other implementation details of the
> system.

Yes, but it's another problem. It's a problem of internal
implementation. Internal things don't need to be formalized and
available for programs.

>
> * E.g., a system could produce native code directly while compiling.
> To separate code and data, the code could be generated in the next
> section (from the header). Strings could be in a special strings
> section, or back in the data section (no need to jump around the
> string in the native code). To avoid jumping around quotations,
> first-level quotations could be in the next section up from the first
> native-code section (and so on).
>
> * OTOH, a system that compiles to intermediate code that is only
> compiled to native code later (e.g., to facilitate inlining) might
> use sections for quotations at the intermediate level and strings,
> and it might put the native code all in one section, because it can
> compile the quotation before or after the word or quotation that
> calls it, avoiding the need to jump around. But it might also keep
> the intermediate code in one section and encode the jumps around
> strings and quotations in a way that is optimized away when
> generating the native code.
>
> * Gforth has only one native code "section" (as described above, it's
> not quite a section). But in the threaded code there is no jump
> around a quotation (because the quotation is in a different section
> on the threaded-code level), so Gforth doesn't compile a
> threaded-code jump into the native code. Instead, it just compiles
> a threaded-code dispatch, which means that the native code
> corresponding to the next threaded-code primitive is executed, and
> that native code is after the native code of the quotation. So on
> the native-code level there is a jump, it's just not as expensive as
> a jump on the threaded-code level.
>

>> The sections that are *automatically* used for string literals,
>> quotations or other things, are an internal business of a Forth system.
>
> Yes. For now, Gforth sections are very much oriented towards out
> internal needs, and are somewhat limited for other uses (e.g., better
> don't try to compile colon definitions in named sections).
>
>> Concerning sections, a more appealing feature for programs is not just
>> to have separate parts in data space, but separate parts of dictionary
>> that can be used also to compile definitions. That is, creating a word
>> in one part doesn't change anything in other parts
>
> Well, we are not far from that.
>
>> and performing a
>> marker (or "forget") in one part doesn't change anything in other parts.
>
> Resetting to a specific point in time has always been a characteristic
> of marker and forget, for better or for worse. These days, they are
> just unneeded baggage IMO.

I mention "marker" and "forget" as methods to clear/reset a part of the
dictionary. So if you don't have a method to free/destroy such a part,
you can at least clear the part by the standard means to reuse it. Of
course, a dedicated method to just clear a whole part can be introduced
as well as a method to free it.

--
Ruvim

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2021-10-04 19:32, Anton Ertl wrote:
>> Ruvim <ruvim.pinka@gmail.com> writes:
>>> On 2021-10-01 19:11, Anton Ertl wrote:
>>>> Well, the interface that Gforth has is relatively simple: You switch
>>>> to a section, and all ALLOTs then go into that section.
>>>
>>> In Gforth, code generation also goes into that section.
>>
>> The threaded code goes there, yes.
>>
>>> But if a system separates code and data, code generation will not go
>>> into that section, but remains in the original section.
>>
>> Why should it?
>
>Just to make sense for programs.

"Not go into that section" makes sense, but "remains in the original
section" does not.

>So a more attractive option is to manage parts of dictionary that
>include both code and data (and their mutual relation should remain
>unspecified for compatibility with any implementation approach).

Their relation is partially specified, resulting in restrictions on
programs: No ALLOT (and friends) while compiling a colon defition,
creating a definition ends a contiguous region.

The relevance of a standardized sections wordset (aka API) would be to
get around such restrictions.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2021-10-07 13:17, Anton Ertl wrote:
> Ruvim <ruvim.pinka@gmail.com> writes:
>> On 2021-10-04 19:32, Anton Ertl wrote:
>>> Ruvim <ruvim.pinka@gmail.com> writes:
>>>> On 2021-10-01 19:11, Anton Ertl wrote:
>>>>> Well, the interface that Gforth has is relatively simple: You switch
>>>>> to a section, and all ALLOTs then go into that section.
>>>>
>>>> In Gforth, code generation also goes into that section.
>>>
>>> The threaded code goes there, yes.
>>>
>>>> But if a system separates code and data, code generation will not go
>>>> into that section, but remains in the original section.
>>>
>>> Why should it?
>>
>> Just to make sense for programs.
>
> "Not go into that section" makes sense,
> but "remains in the original section" does not.

Yes, I meant that too.

>> So a more attractive option is to manage parts of dictionary that
>> include both code and data (and their mutual relation should remain
>> unspecified for compatibility with any implementation approach).
>
> Their relation is partially specified, resulting in restrictions on
> programs: No ALLOT (and friends) while compiling a colon defition,
> creating a definition ends a contiguous region.

Well, the *possible* options are specified. But what particular option
is used — it is unknown. Therefore, it restricts programs, and relaxes
implementations.

>
> The relevance of a standardized sections wordset (aka API) would be to
> get around such restrictions.

It's not easy, if possible at all. A library or extension can provide
defining words (or redefine some standard words) that uses data space. E.g.:

: variable ( "name" -- ) align here >r 0 , : r> lit, postpone ; ;

and now data space (the current data space section) is used when a
variable is created.

If you want to guarantee that the current section of data space is not
touched when a word is created (or a file is included), it would be too
restrictive for programs and libraries.

When a definition is created, both the current section of data space and
the current section of code space can be always used. It's their primary
function. No sense to avoid it.

--
Ruvim

Ruvim <ruvim.pinka@gmail.com> writes:
>On 2021-10-07 13:17, Anton Ertl wrote:
>> The relevance of a standardized sections wordset (aka API) would be to
>> get around such restrictions.
>
>It's not easy, if possible at all. A library or extension can provide
>defining words (or redefine some standard words) that uses data space. E.g.:
>
> : variable ( "name" -- ) align here >r 0 , : r> lit, postpone ; ;
>
>and now data space (the current data space section) is used when a
>variable is created.

Yes, but point is that it is in the current section. Other sections
are unaffected.

Therefore I can do

create foo 5 ,
next-section variable bar previous-section bar ,
6 ,

and get a contiguous region starting at the body of FOO, containing 5,
BAR, and 6.

I can also define your VARIABLE above a little differently:

: variable : next-section here 0 , previous-section lit, postpone ; ;

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021