So if you wanted a "descendent of Forth" which fit the hand nicely in the
current compute environment, how about strings? I thought of the toy problem
of counting the number of words in the diction with an "a" in them:

>>> f = open("/usr/share/dict/words", "r")
>>> na = 0
>>> for l in f:
.... if "a" in l:
.... na += 1
....
>>> print na
52849
>>>

(Python, obviously, and it took .13 CPU seconds on an arm64
device, the RockPRO64. The file is 102,401 lines.)

I typed it in just on the fly, no IDE or any dev references needed, and got
it right on the first go--not too surprising.

It could go shorter:

>>> print len([l for l in f if ("a" in l)])
52849
>>>

And--I was surprised--using the same amount of CPU.

Forth as hand written assembly code for a stack machine will never touch
this. I've tried many times to find the language which brings higher level
of expressiveness. Failed every time so far.

Andy Valencia
Home page: https://www.vsta.org/andy/
To contact me: https://www.vsta.org/contact/andy.html

Andy Valencia <vandys@vsta.org> writes:
> Forth as hand written assembly code for a stack machine will never touch
> this. I've tried many times to find the language which brings higher level
> of expressiveness. Failed every time so far.

SQL: SELECT COUNT(word) FROM words WHERE word LIKE '%a%';

I used to disdain SQL since lots of times for large datasets, I could
write a much more practical and performant program by using old
fashioned tools like the Un*x sorting utility. Lately I've become a
convert.

Not long ago I wrote a Python script to match up a bunch of urls (that
contained numeric identifiers) with retrieved filenams (that also had
those identifiers). So I had a dictionary mapping urls to id's, and
another dictionary mapping id's to filenames, and IIRC there was another
dictionary that was a reverse mapping. All very standard stuff in
Python. It was a page or so of code, complex enough that I had to fix a
few bugs during testing, but nothing awful. There were a few hundred
filenames involved, not enough for performance or memory consumption to
matter on my laptop.

Then I realized that what I had actually written was a relational JOIN,
so I ripped out my stuff with dictionaries, imported the sqlite3 module,
made an in-memory SQL table, then populated and extracted it with SQL
queries.

That cut the program size in half, made its functionality much more
obvious, and made it simpler to modify (just add ORDER BY to change the
output sorting, etc). But it did that by pulling in a full-blown SQL
implementation. I think that amount of bloat and dependency is counter
to the Forth spirit. I still have to say that it was convenient.

On 2021-12-24, Andy Valencia <vandys@vsta.org> wrote:
> Forth as hand written assembly code for a stack machine will never touch
> this. I've tried many times to find the language which brings higher level
> of expressiveness. Failed every time so far.

Is the question about whether Forth can be expressive enough to implement string
processing? Is the question about whether a fluent Forth programmer can easily
implement something like the one-line Python example? I'm not clear what this
thread is about.

- meff

meff <email@example.com> writes:
> Is the question about whether Forth can be expressive enough to
> implement string processing? Is the question about whether a fluent
> Forth programmer can easily implement something like the one-line
> Python example?

I think it postulates a hypothetical Forth-descended language that lets
users implement the Python example with comparable ease, and asks what
that Forth-descended language would look like. Forth as we know it
today is much more cumbersome.

We have Oforth, 8th, Factor, and maybe some others as Forth-descended
languages that might fit the bill. These all have their aficionados.
I'm personally not that enthused about any of them, since the Forth
features they preserve (RPN and visible stacks) are imho not that
helpful once the language implementation is that complicated. But
that's just me.

For an article on a distantly comparable situation with Lisp, see:

http://www.cs.kent.ac.uk/people/staff/dat/miranda/wadler87.pdf

Paul Rubin <no.email@nospam.invalid> writes:
> (solve w. SQL)
> That cut the program size in half, made its functionality much more
> obvious, and made it simpler to modify (just add ORDER BY to change the
> output sorting, etc). But it did that by pulling in a full-blown SQL
> implementation. I think that amount of bloat and dependency is counter
> to the Forth spirit. I still have to say that it was convenient.

And I have to say, I've used sqlite3 many, many times. It's not svelte, but
in bang/buck for database functionality, it's the best bargain going.

Andy Valencia
Home page: https://www.vsta.org/andy/
To contact me: https://www.vsta.org/contact/andy.html

On Saturday, December 25, 2021 at 12:01:50 AM UTC+3, Andy Valencia wrote:
> So if you wanted a "descendent of Forth" which fit the hand nicely in the
> current compute environment, how about strings? I thought of the toy problem
> of counting the number of words in the diction with an "a" in them:
>
> >>> f = open("/usr/share/dict/words", "r")
> >>> na = 0
> >>> for l in f:
> ... if "a" in l:
> ... na += 1
> ...
> >>> print na
> 52849
> >>>
>
> (Python, obviously, and it took .13 CPU seconds on an arm64
> device, the RockPRO64. The file is 102,401 lines.)
>
> I typed it in just on the fly, no IDE or any dev references needed, and got
> it right on the first go--not too surprising.
>
> It could go shorter:
>
> >>> print len([l for l in f if ("a" in l)])
> 52849
> >>>
>
> And--I was surprised--using the same amount of CPU.
>
> Forth as hand written assembly code for a stack machine will never touch
> this. I've tried many times to find the language which brings higher level
> of expressiveness. Failed every time so far.
>

Factor variant:

0 "/usr/share/dict/words" utf8 file-lines [ CHAR: a swap index [ 1 + ] [ ] if ] each

Probably, this can be written shorter, but I'm not familiar with Factor enough.

Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 05:49:12 UTC+1:
> On Saturday, December 25, 2021 at 12:01:50 AM UTC+3, Andy Valencia wrote:
> > So if you wanted a "descendent of Forth" which fit the hand nicely in the
> > current compute environment, how about strings? I thought of the toy problem
> > of counting the number of words in the diction with an "a" in them:
> >
> > >>> f = open("/usr/share/dict/words", "r")
> > >>> na = 0
> > >>> for l in f:
> > ... if "a" in l:
> > ... na += 1
> > ...
> > >>> print na
> > 52849
> > >>>
> >
> > (Python, obviously, and it took .13 CPU seconds on an arm64
> > device, the RockPRO64. The file is 102,401 lines.)
> >
> > I typed it in just on the fly, no IDE or any dev references needed, and got
> > it right on the first go--not too surprising.
> >
> > It could go shorter:
> >
> > >>> print len([l for l in f if ("a" in l)])
> > 52849
> > >>>
> >
> > And--I was surprised--using the same amount of CPU.
> >
> > Forth as hand written assembly code for a stack machine will never touch
> > this. I've tried many times to find the language which brings higher level
> > of expressiveness. Failed every time so far.
> >
> Factor variant:
>
> 0 "/usr/share/dict/words" utf8 file-lines [ CHAR: a swap index [ 1 + ] [ ] if ] each
>
> Probably, this can be written shorter, but I'm not familiar with Factor enough.

That's what had me always kept away from Factor: you have to read it forward
and backwards. Your eyes always have to 'go zig-zag' to read Factor code.
I find its syntax annoying and tiring.

On Saturday, December 25, 2021 at 1:28:10 PM UTC+3, minf...@arcor.de wrote:
> Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 05:49:12 UTC+1:
> > On Saturday, December 25, 2021 at 12:01:50 AM UTC+3, Andy Valencia wrote:
> > > So if you wanted a "descendent of Forth" which fit the hand nicely in the
> > > current compute environment, how about strings? I thought of the toy problem
> > > of counting the number of words in the diction with an "a" in them:
> > >
> > > >>> f = open("/usr/share/dict/words", "r")
> > > >>> na = 0
> > > >>> for l in f:
> > > ... if "a" in l:
> > > ... na += 1
> > > ...
> > > >>> print na
> > > 52849
> > > >>>
> > >
> > > (Python, obviously, and it took .13 CPU seconds on an arm64
> > > device, the RockPRO64. The file is 102,401 lines.)
> > >
> > > I typed it in just on the fly, no IDE or any dev references needed, and got
> > > it right on the first go--not too surprising.
> > >
> > > It could go shorter:
> > >
> > > >>> print len([l for l in f if ("a" in l)])
> > > 52849
> > > >>>
> > >
> > > And--I was surprised--using the same amount of CPU.
> > >
> > > Forth as hand written assembly code for a stack machine will never touch
> > > this. I've tried many times to find the language which brings higher level
> > > of expressiveness. Failed every time so far.
> > >
> > Factor variant:
> >
> > 0 "/usr/share/dict/words" utf8 file-lines [ CHAR: a swap index [ 1 + ] [ ] if ] each
> >
> > Probably, this can be written shorter, but I'm not familiar with Factor enough.
> That's what had me always kept away from Factor: you have to read it forward
> and backwards. Your eyes always have to 'go zig-zag' to read Factor code.
> I find its syntax annoying and tiring.

Yes, I've also noticed that. Maybe it is possible to get used to it after some time,
just like with Forth RPN.

But the very concept of combinators is noteworthy and should be borrowed.
Sometimes they allow you to avoid complicated manipulations with the stack.

Andy Valencia <vandys@vsta.org> writes:
>So if you wanted a "descendent of Forth" which fit the hand nicely in the
>current compute environment, how about strings? I thought of the toy problem
>of counting the number of words in the diction with an "a" in them:
>
>>>> f = open("/usr/share/dict/words", "r")
>>>> na = 0
>>>> for l in f:
>... if "a" in l:
>... na += 1
>...
>>>> print na
>52849
>>>>
>
>(Python, obviously, and it took .13 CPU seconds on an arm64
>device, the RockPRO64. The file is 102,401 lines.)

We don't have /usr/share/dict/words on our Rockpro 64 (nor on the two
other systems I checked). Looking around for it, I did not find it,
but found a pointer to
<http://downloads.sourceforge.net/project/wordlist/SCOWL/2020.12.07/scowl-2020.12.07.tar.gz>

I don't see anything with 102401 lines there, so instead I used
final/english-words.80, which has 139209 lines.

Let's see how this works out (I have disabled the slow cores to get
more reliable timings):

[rockpro64:/tmp/scowl-2020.12.07:76145] time python
Python 2.7.13 (default, Sep 26 2018, 18:42:22)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open("final/english-words.80","r")
>>> print len([l for l in f if ("a" in l)])
78886
>>>

real 0m23.269s
user 0m0.136s
sys 0m0.024s

Ok, but Python 2 is no longer supported, so let's try Python 3:

[rockpro64:/tmp/scowl-2020.12.07:76146] time python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open("final/english-words.80","r")
>>> print len([l for l in f if ("a" in l)])
File "<stdin>", line 1
print len([l for l in f if ("a" in l)])
^
SyntaxError: invalid syntax

Ok, so fall back to your first variant:

>>> na = 0
>>> for l in f:
.... if "a" in l:
.... na += 1
....
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2797: invalid continuation byte

Great going, Python!

What would I use for this task?

[rockpro64:/tmp/scowl-2020.12.07:76147] time sh -c "grep a final/english-words.80|wc -l"
78683

real 0m0.092s
user 0m0.048s
sys 0m0.032s

Quite a bit shorter and faster than your short Python 2 version, but
with a different result. Investigation reveals that GNU grep 2.27
apparently silently does not output lines that contain Latin-1
characters when LANG=C.UTF-8, which is even worse than Python 3's
behaviour. This can be fixed with

[rockpro64:/tmp/scowl-2020.12.07:76148] time env LANG=C sh -c "grep a final/english-words.80|wc -l"
78886

real 0m0.077s
user 0m0.064s
sys 0m0.008s

but I guess that the same fix may help Python 3.

Anyway, let's see about Forth:

[rockpro64:/tmp/scowl-2020.12.07:76149] time ~/nfstmp/gforth-arm64/gforth -e "s\" final/english-words.80\" slurp-file :noname 0 >r begin 'a' scan dup while r> 1+ >r #lf scan dup 0= until then 2drop r> ; execute . bye"
78886
real 0m0.105s
user 0m0.044s
sys 0m0.032s

On my first try I forgot the second DUP, but the bug and its fix was
obvious once I saw the stack underflow error message and it worked on
the second try. For this code, gforth is faster than gforth-fast,
because gforth-fast has more startup overhead (especially on Aarch64),
and much of the time is spent in SCAN (which does not benefit much
from the differences between gforth and gforth-fast).

The layout of the program is not so nice in this command-line form, so
here it's with nicer layout:

s" final/english-words.80" slurp-file
:noname
0 >r begin
'a' scan dup while
r> 1+ >r
#lf scan dup 0= until then
2drop r> ;
execute .

Longer than the two python variants, but not by much. Looking at it
again, I would now code it as:

: counts
0 >r begin
'a' scan dup while
r> 1+ >r
#lf scan dup 0= until then
2drop r> ;
s" final/english-words.80" slurp-file counts .

>Forth as hand written assembly code for a stack machine will never touch
>this.

It's not there, but it's close. And concerning compatibility: 'a' did
not work on Gforth 0.6, but 'a did and still does, so by changing the
'a' into 'a, you are compatible back from 0.6 (2003) to the current
development version. SLURP-FILE did not work on Gforth 0.5 (2000), so
to be compatible with that more work would be needed.

>I've tried many times to find the language which brings higher level
>of expressiveness. Failed every time so far.

The Unix shell combined with the shell tools are the language of
choice for this kind of task, as demonstrated here again.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 11:45:19 UTC+1:
> On Saturday, December 25, 2021 at 1:28:10 PM UTC+3, minf...@arcor.de wrote:
> > Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 05:49:12 UTC+1:
> > > On Saturday, December 25, 2021 at 12:01:50 AM UTC+3, Andy Valencia wrote:
> > > > So if you wanted a "descendent of Forth" which fit the hand nicely in the
> > > > current compute environment, how about strings? I thought of the toy problem
> > > > of counting the number of words in the diction with an "a" in them:
> > > >
> > > > >>> f = open("/usr/share/dict/words", "r")
> > > > >>> na = 0
> > > > >>> for l in f:
> > > > ... if "a" in l:
> > > > ... na += 1
> > > > ...
> > > > >>> print na
> > > > 52849
> > > > >>>
> > > >
> > > > (Python, obviously, and it took .13 CPU seconds on an arm64
> > > > device, the RockPRO64. The file is 102,401 lines.)
> > > >
> > > > I typed it in just on the fly, no IDE or any dev references needed, and got
> > > > it right on the first go--not too surprising.
> > > >
> > > > It could go shorter:
> > > >
> > > > >>> print len([l for l in f if ("a" in l)])
> > > > 52849
> > > > >>>
> > > >
> > > > And--I was surprised--using the same amount of CPU.
> > > >
> > > > Forth as hand written assembly code for a stack machine will never touch
> > > > this. I've tried many times to find the language which brings higher level
> > > > of expressiveness. Failed every time so far.
> > > >
> > > Factor variant:
> > >
> > > 0 "/usr/share/dict/words" utf8 file-lines [ CHAR: a swap index [ 1 + ] [ ] if ] each
> > >
> > > Probably, this can be written shorter, but I'm not familiar with Factor enough.
> > That's what had me always kept away from Factor: you have to read it forward
> > and backwards. Your eyes always have to 'go zig-zag' to read Factor code.
> > I find its syntax annoying and tiring.
> Yes, I've also noticed that. Maybe it is possible to get used to it after some time,
> just like with Forth RPN.
>
> But the very concept of combinators is noteworthy and should be borrowed.
> Sometimes they allow you to avoid complicated manipulations with the stack.

Once you have quotations in your Forth, eg
https://www.complang.tuwien.ac.at/forth/ansforth-cvs/quotations.txt
combinators shouldn't be too difficult to implement.

On Saturday, December 25, 2021 at 2:06:38 PM UTC+3, Anton Ertl wrote:
> Andy Valencia <van...@vsta.org> writes:
> >So if you wanted a "descendent of Forth" which fit the hand nicely in the
> >current compute environment, how about strings? I thought of the toy problem
> >of counting the number of words in the diction with an "a" in them:
> >
> >>>> f = open("/usr/share/dict/words", "r")
> >>>> na = 0
> >>>> for l in f:
> >... if "a" in l:
> >... na += 1
> >...
> >>>> print na
> >52849
> >>>>
> >
> >(Python, obviously, and it took .13 CPU seconds on an arm64
> >device, the RockPRO64. The file is 102,401 lines.)
> We don't have /usr/share/dict/words on our Rockpro 64 (nor on the two
> other systems I checked). Looking around for it, I did not find it,
> but found a pointer to
> <http://downloads.sourceforge.net/project/wordlist/SCOWL/2020.12.07/scowl-2020.12.07.tar.gz>
>
> I don't see anything with 102401 lines there, so instead I used
> final/english-words.80, which has 139209 lines.
>
> Let's see how this works out (I have disabled the slow cores to get
> more reliable timings):
>
> [rockpro64:/tmp/scowl-2020.12.07:76145] time python
> Python 2.7.13 (default, Sep 26 2018, 18:42:22)
> [GCC 6.3.0 20170516] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> f = open("final/english-words.80","r")
> >>> print len([l for l in f if ("a" in l)])
> 78886
> >>>
>
> real 0m23.269s
> user 0m0.136s
> sys 0m0.024s
>
> Ok, but Python 2 is no longer supported, so let's try Python 3:
>
> [rockpro64:/tmp/scowl-2020.12.07:76146] time python3
> Python 3.5.3 (default, Sep 27 2018, 17:25:39)
> [GCC 6.3.0 20170516] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> f = open("final/english-words.80","r")
> >>> print len([l for l in f if ("a" in l)])
> File "<stdin>", line 1
> print len([l for l in f if ("a" in l)])
> ^
> SyntaxError: invalid syntax
>
> Ok, so fall back to your first variant:
> >>> na = 0
> >>> for l in f:
> ... if "a" in l:
> ... na += 1
> ...
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/lib/python3.5/codecs.py", line 321, in decode
> (result, consumed) = self._buffer_decode(data, self.errors, final)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2797: invalid continuation byte
>
> Great going, Python!
>
> What would I use for this task?
>
> [rockpro64:/tmp/scowl-2020.12.07:76147] time sh -c "grep a final/english-words.80|wc -l"
> 78683
>
> real 0m0.092s
> user 0m0.048s
> sys 0m0.032s
>
> Quite a bit shorter and faster than your short Python 2 version, but
> with a different result. Investigation reveals that GNU grep 2.27
> apparently silently does not output lines that contain Latin-1
> characters when LANG=C.UTF-8, which is even worse than Python 3's
> behaviour. This can be fixed with
>
> [rockpro64:/tmp/scowl-2020.12.07:76148] time env LANG=C sh -c "grep a final/english-words.80|wc -l"
> 78886
>
> real 0m0.077s
> user 0m0.064s
> sys 0m0.008s
>
> but I guess that the same fix may help Python 3.
>
> Anyway, let's see about Forth:
>
> [rockpro64:/tmp/scowl-2020.12.07:76149] time ~/nfstmp/gforth-arm64/gforth -e "s\" final/english-words.80\" slurp-file :noname 0 >r begin 'a' scan dup while r> 1+ >r #lf scan dup 0= until then 2drop r> ; execute . bye"
> 78886
> real 0m0.105s
> user 0m0.044s
> sys 0m0.032s
>
> On my first try I forgot the second DUP, but the bug and its fix was
> obvious once I saw the stack underflow error message and it worked on
> the second try. For this code, gforth is faster than gforth-fast,
> because gforth-fast has more startup overhead (especially on Aarch64),
> and much of the time is spent in SCAN (which does not benefit much
> from the differences between gforth and gforth-fast).
>
> The layout of the program is not so nice in this command-line form, so
> here it's with nicer layout:
>
> s" final/english-words.80" slurp-file
> :noname
> 0 >r begin
> 'a' scan dup while
> r> 1+ >r
> #lf scan dup 0= until then
> 2drop r> ;
> execute .
>
> Longer than the two python variants, but not by much. Looking at it
> again, I would now code it as:
>
> : counts
> 0 >r begin
> 'a' scan dup while
> r> 1+ >r
> #lf scan dup 0= until then
> 2drop r> ;
> s" final/english-words.80" slurp-file counts .
> >Forth as hand written assembly code for a stack machine will never touch
> >this.
> It's not there, but it's close. And concerning compatibility: 'a' did
> not work on Gforth 0.6, but 'a did and still does, so by changing the
> 'a' into 'a, you are compatible back from 0.6 (2003) to the current
> development version. SLURP-FILE did not work on Gforth 0.5 (2000), so
> to be compatible with that more work would be needed.

What a great example.

So, i've tried to run it on various forths:

: counts
0 >r begin
'a' scan dup while
r> 1+ >r
#lf scan dup 0 until then
2drop r>
;

: main-ertl s" /usr/share/dict/words" slurp-file counts . ;

$ lxf include ertl.fth

evaluate: include ertl.fth col: 16
ertl.fth line: 7 col: 8
undefined word #lf
no block file

No luck with LXF. Probably it is not standard conformant enough.

$ sf ertl.fth
/some/path/ertl.fth
8: #lf scan dup 0= >>> #lf ?
Exit? (Yes/No/Edit)

No luck with SwiftForth.

$ vfxlin32 include ertl.fth
VFX Forth 5.20 [build 0749] 2021-05-27 for Linux x86
© MicroProcessor Engineering Ltd, 1998-2021

Including ertl.fth
Err# -13 ERR: Undefined word.
Source: "ertl.fth" on line 8
-> #lf scan dup 0 ^
ERROR on command line

Vfx also refuses to compile this code.

$ i4 include /some/path/ertl.fth
....
Thrown out of `/some/path/ertl.fth'
Error -13 in line #8 at nesting level 1
#lf scan dup 0-------^
#lf ?

iForth also refuses to build it.

Well, we also have Forth2012 compatible IKForth

$ ikforth -f ertl.fth
....
Undefined word #lf
at '(TRACKED-INCLUDED)' (H# 2000D89C)
LINE# H# 00000008 #lf scan dup 0
No, luck.

But this should be smth simple. Probably #lf is just 10. Let's define it and
check.

[undefined] #lf [if] 10 CONSTANT #lf [then]

But it still doesn't compile on any of them...

It is fun to see how core "Standards Team Member" uses THREE non-standard words
in a simple example.

On Saturday, December 25, 2021 at 2:25:22 PM UTC+3, minf...@arcor.de wrote:
> Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 11:45:19 UTC+1:
> > On Saturday, December 25, 2021 at 1:28:10 PM UTC+3, minf...@arcor.de wrote:
> > > Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 05:49:12 UTC+1:
> > > > On Saturday, December 25, 2021 at 12:01:50 AM UTC+3, Andy Valencia wrote:
> > > > > So if you wanted a "descendent of Forth" which fit the hand nicely in the
> > > > > current compute environment, how about strings? I thought of the toy problem
> > > > > of counting the number of words in the diction with an "a" in them:
> > > > >
> > > > > >>> f = open("/usr/share/dict/words", "r")
> > > > > >>> na = 0
> > > > > >>> for l in f:
> > > > > ... if "a" in l:
> > > > > ... na += 1
> > > > > ...
> > > > > >>> print na
> > > > > 52849
> > > > > >>>
> > > > >
> > > > > (Python, obviously, and it took .13 CPU seconds on an arm64
> > > > > device, the RockPRO64. The file is 102,401 lines.)
> > > > >
> > > > > I typed it in just on the fly, no IDE or any dev references needed, and got
> > > > > it right on the first go--not too surprising.
> > > > >
> > > > > It could go shorter:
> > > > >
> > > > > >>> print len([l for l in f if ("a" in l)])
> > > > > 52849
> > > > > >>>
> > > > >
> > > > > And--I was surprised--using the same amount of CPU.
> > > > >
> > > > > Forth as hand written assembly code for a stack machine will never touch
> > > > > this. I've tried many times to find the language which brings higher level
> > > > > of expressiveness. Failed every time so far.
> > > > >
> > > > Factor variant:
> > > >
> > > > 0 "/usr/share/dict/words" utf8 file-lines [ CHAR: a swap index [ 1 + ] [ ] if ] each
> > > >
> > > > Probably, this can be written shorter, but I'm not familiar with Factor enough.
> > > That's what had me always kept away from Factor: you have to read it forward
> > > and backwards. Your eyes always have to 'go zig-zag' to read Factor code.
> > > I find its syntax annoying and tiring.
> > Yes, I've also noticed that. Maybe it is possible to get used to it after some time,
> > just like with Forth RPN.
> >
> > But the very concept of combinators is noteworthy and should be borrowed.
> > Sometimes they allow you to avoid complicated manipulations with the stack.
> Once you have quotations in your Forth, eg
> https://www.complang.tuwien.ac.at/forth/ansforth-cvs/quotations.txt
> combinators shouldn't be too difficult to implement.

This is a bit more complex. In Forth quotations are just :NONAME inside colon
definition. I.e. they are compiled and there is no way except MARKER to remove
them. In Factor (AFAIK) they are garbage collected.

This prevents us from using RTCG in Forth. Eventually we will run out of
memory. Consider this example.

: GEN [: POSTPONE LIT ;] ; \ simple demo
: TTT 0 0 1000000 DO I GEN EXECUTE + LOOP ;
TTT

Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 13:35:53 UTC+1:
> On Saturday, December 25, 2021 at 2:25:22 PM UTC+3, minf...@arcor.de wrote:
> > Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 11:45:19 UTC+1:
> > > On Saturday, December 25, 2021 at 1:28:10 PM UTC+3, minf...@arcor.de wrote:
> > > > Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 05:49:12 UTC+1:
> > > > > On Saturday, December 25, 2021 at 12:01:50 AM UTC+3, Andy Valencia wrote:
> > > > > > So if you wanted a "descendent of Forth" which fit the hand nicely in the
> > > > > > current compute environment, how about strings? I thought of the toy problem
> > > > > > of counting the number of words in the diction with an "a" in them:
> > > > > >
> > > > > > >>> f = open("/usr/share/dict/words", "r")
> > > > > > >>> na = 0
> > > > > > >>> for l in f:
> > > > > > ... if "a" in l:
> > > > > > ... na += 1
> > > > > > ...
> > > > > > >>> print na
> > > > > > 52849
> > > > > > >>>
> > > > > >
> > > > > > (Python, obviously, and it took .13 CPU seconds on an arm64
> > > > > > device, the RockPRO64. The file is 102,401 lines.)
> > > > > >
> > > > > > I typed it in just on the fly, no IDE or any dev references needed, and got
> > > > > > it right on the first go--not too surprising.
> > > > > >
> > > > > > It could go shorter:
> > > > > >
> > > > > > >>> print len([l for l in f if ("a" in l)])
> > > > > > 52849
> > > > > > >>>
> > > > > >
> > > > > > And--I was surprised--using the same amount of CPU.
> > > > > >
> > > > > > Forth as hand written assembly code for a stack machine will never touch
> > > > > > this. I've tried many times to find the language which brings higher level
> > > > > > of expressiveness. Failed every time so far.
> > > > > >
> > > > > Factor variant:
> > > > >
> > > > > 0 "/usr/share/dict/words" utf8 file-lines [ CHAR: a swap index [ 1 + ] [ ] if ] each
> > > > >
> > > > > Probably, this can be written shorter, but I'm not familiar with Factor enough.
> > > > That's what had me always kept away from Factor: you have to read it forward
> > > > and backwards. Your eyes always have to 'go zig-zag' to read Factor code.
> > > > I find its syntax annoying and tiring.
> > > Yes, I've also noticed that. Maybe it is possible to get used to it after some time,
> > > just like with Forth RPN.
> > >
> > > But the very concept of combinators is noteworthy and should be borrowed.
> > > Sometimes they allow you to avoid complicated manipulations with the stack.
> > Once you have quotations in your Forth, eg
> > https://www.complang.tuwien.ac.at/forth/ansforth-cvs/quotations.txt
> > combinators shouldn't be too difficult to implement.
> This is a bit more complex. In Forth quotations are just :NONAME inside colon
> definition. I.e. they are compiled and there is no way except MARKER to remove
> them. In Factor (AFAIK) they are garbage collected.
>
> This prevents us from using RTCG in Forth. Eventually we will run out of
> memory. Consider this example.
>
> : GEN [: POSTPONE LIT ;] ; \ simple demo
> : TTT 0 0 1000000 DO I GEN EXECUTE + LOOP ;
> TTT

I think I get your meaning, although IMHO your demo just does what it had been told
namely compile an excessive number of literals.

I don't know Factor well enough, but contrary to Forth all data (including numbers)
reside in the heap. The stack just contains pointers to data items, which then can be
all kinds of data including code lists = quotations. At least that's how I understand
it (which might be wrong).

Consequently DROP in addition to decrement the stack point has to free assigned data.
ABORT/THROW et cetera have to walk the stack to free all assigned data when needed.
So it's a stack-based garbage collection. MARKER would have to free other data of course,
like VALUES and VARIABLEs (don't know if they exist in Factor).

BTW I use a similar mechanism in my own Linear Algebra package in Forth.

On Sat, 25 Dec 2021 09:50:35 GMT
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> [rockpro64:/tmp/scowl-2020.12.07:76146] time python3
> Python 3.5.3 (default, Sep 27 2018, 17:25:39)
> [GCC 6.3.0 20170516] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> f = open("final/english-words.80","r")
> >>> print len([l for l in f if ("a" in l)])
> File "<stdin>", line 1
> print len([l for l in f if ("a" in l)])
> ^
> SyntaxError: invalid syntax

Andy's original and short versions run and produce same result using python 2 or 3 if print statement parens are added:
x86_64:$ time python
Python 2.7.16 (default, Oct 10 2019, 22:02:15)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open("/usr/share/dict/words", "r")
>>> print( len([l for l in f if ("a" in l)]) )
52849

real 0m6.197s
user 0m0.039s
sys 0m0.000s

x86_64:$ time python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open("/usr/share/dict/words", "r")
>>> print( len([l for l in f if ("a" in l)]) )
52849

real 0m4.048s
user 0m0.040s
sys 0m0.011s

For a quick result I would use the more explicit version, which I could also write and modify faster.
Timings were similar:
user 0m0.029s Python 2
user 0m0.050s Python 3

Jan Coombs

On Saturday, December 25, 2021 at 12:51:37 PM UTC+1, Nickolay Kolchin wrote:
[..]
> What a great example.

> iForth also refuses to build it.

Really?

FORTH> : counts ( c-addr u -- n )
<3>[FORTH>] 0 >r
<3>[FORTH>] begin
[3]<3>[FORTH>] 'a' scan dup
[3]<3>[FORTH>] while
[6]<3>[FORTH>] r> 1+ >r
[6]<3>[FORTH>] #10 scan dup 0=
[6]<3>[FORTH>] until then
<3>[FORTH>] 2drop r> ; ok
FORTH> S" Hallo, aarde." counts . 1 ok
FORTH> S\" Hallo, aarde.\nMaan hier." counts . 2 ok
FORTH> S\" Hallo, aarde.\nMa\na\nn hier." counts . 3 ok

-marcel

On Saturday, December 25, 2021 at 4:53:52 PM UTC+3, Marcel Hendrix wrote:
> On Saturday, December 25, 2021 at 12:51:37 PM UTC+1, Nickolay Kolchin wrote:
> [..]
> > What a great example.
> > iForth also refuses to build it.
> Really?
>
> FORTH> : counts ( c-addr u -- n )
> <3>[FORTH>] 0 >r
> <3>[FORTH>] begin
> [3]<3>[FORTH>] 'a' scan dup
> [3]<3>[FORTH>] while
> [6]<3>[FORTH>] r> 1+ >r
> [6]<3>[FORTH>] #10 scan dup 0=
> [6]<3>[FORTH>] until then
> <3>[FORTH>] 2drop r> ; ok
> FORTH> S" Hallo, aarde." counts . 1 ok
> FORTH> S\" Hallo, aarde.\nMaan hier." counts . 2 ok
> FORTH> S\" Hallo, aarde.\nMa\na\nn hier." counts . 3 ok
>

That's unfair. You haven't compiled 'main-ertl'.

P.S. But I'm on iForth4, so who knows.

On Friday, December 24, 2021 at 3:01:50 PM UTC-6, Andy Valencia wrote:
> So if you wanted a "descendent of Forth" which fit the hand nicely in the
> current compute environment, how about strings? I thought of the toy problem
> of counting the number of words in the diction with an "a" in them:
>
> >>> f = open("/usr/share/dict/words", "r")
> >>> na = 0
> >>> for l in f:
> ... if "a" in l:
> ... na += 1
> ...
> >>> print na
> 52849
> >>>
>
> (Python, obviously, and it took .13 CPU seconds on an arm64
> device, the RockPRO64. The file is 102,401 lines.)
>
> I typed it in just on the fly, no IDE or any dev references needed, and got
> it right on the first go--not too surprising.
>
> It could go shorter:
>
> >>> print len([l for l in f if ("a" in l)])
> 52849
> >>>
>
> And--I was surprised--using the same amount of CPU.
>
> Forth as hand written assembly code for a stack machine will never touch
> this. I've tried many times to find the language which brings higher level
> of expressiveness. Failed every time so far.
>
> Andy Valencia
> Home page: https://www.vsta.org/andy/
> To contact me: https://www.vsta.org/contact/andy.html

I thought PostScript might be good at this, and it's broadly construed
as a Forth variant (though its actually history suggests a common ancestor
in the Burrough's Large System architecture). It has a string type, at least.

I had to add a few helper functions to my toolbag.

each-line {src proc}{
src type /stringtype eq { /src src (r) file store } if
{
src 512 string readline {proc}{pop exit} ifelse
} loop
} @func
contains { search { pop pop pop true }{ pop false } ifelse }

And there's a bit of boilerplate to use it from the command line.

$ gsnd -q -dNOSAFER struct2.ps -c "[(struct2.ps){}each-line]{(a)contains{1}{0}ifelse}map{add}reduce = quit"
60
$ gsnd -q -dNOSAFER struct2.ps -c "[(struct2.ps){(a)contains{1}{0}ifelse}each-line]{add}reduce = quit"
60

It can probably be shorter using Anton's idea to suppress the zeros and
take the length of the array. (I also don't have /usr/share/dict/word but
I doubt ghostscript is going to win any speed contests.)

On Saturday, December 25, 2021 at 4:25:22 AM UTC-7, minf...@arcor.de wrote:
> Nickolay Kolchin schrieb am Samstag, 25. Dezember 2021 um 11:45:19 UTC+1:
> > On Saturday, December 25, 2021 at 1:28:10 PM UTC+3, minf...@arcor.de wrote:
> > But the very concept of combinators is noteworthy and should be borrowed.
> > Sometimes they allow you to avoid complicated manipulations with the stack.
> Once you have quotations in your Forth, eg
> https://www.complang.tuwien.ac.at/forth/ansforth-cvs/quotations.txt
> combinators shouldn't be too difficult to implement.

Factor's combinators were the inspiration for my rquotations.
It is necessary for the rquotation to have access to the parent function's local variables
despite the HOF (higher-order-function called by the parent function that executes
the rquotation) having local variables of its own.

The idea that the Paysan-faked quotations, that lack this feature, are a step toward
combinators is total baloney! This is just Forth-200x fantasy, not based on reality at all.

My rquotations only work one level deep, so they are not like Factor combinators that can
stay in scope at any level deep from the parent function and can even stay in scope after the
parent function has exited. Factor combinators are like closures in Lisp, and Common Lisp
is in fact Slava's background. This isn't necessary for my purposes. I am just supporting
general-purpose data-structures. The idea is that the HOF traverses a data-structure and
executes the rquotation that it is given for every node in the data-structure. The programmer
who writes the HOF has to know how the data-structure is implemented internally, but does
not need to know what they payload is. The application programmer who writes the parent
function and the rquotation has to know what the payload is (this is specific to the application)
but does not have to know how the data-structure or HOF are implemented internally.
This is called information hiding!

In regard to your string pattern-matching and replacement program, that would be trivial
given the novice-package that has STRING-STACK.4TH, rquotations, ASSOCIATION.4TH etc..
The novice--package really makes programming easy!

On Saturday, December 25, 2021 at 11:54:37 PM UTC+3, Hugh Aguilar wrote:
> Factor's combinators were the inspiration for my rquotations.

Can you share a link to your factor vs forth comparison?

lauantai 25. joulukuuta 2021 klo 0.49.18 UTC+2 Paul Rubin kirjoitti:
> meff <em...@example.com> writes:
> We have Oforth, 8th, Factor, and maybe some others as Forth-descended
> languages that might fit the bill. These all have their aficionados.
> I'm personally not that enthused about any of them, since the Forth
> features they preserve (RPN and visible stacks) are imho not that
> helpful once the language implementation is that complicated. But
> that's just me.

8th fits the bill nicely as there are high level words for string processing. I would probably write something like:

0 "/usr/share/dict/words" f:slurp
( "a" search null? nip nip if n:1+ then ) s:eachline . cr

or a little slower:

0 "/usr/share/dict/words" f:open-ro
( "a" search null? nip nip if n:1+ then ) f:eachline drop . cr

My dictionary is a lot bigger: 654299 lines. First code runs in: 0m0.341s on my Raspberry PI 4B.

On 26/12/2021 12:20, Jali Heinonen wrote:
> lauantai 25. joulukuuta 2021 klo 0.49.18 UTC+2 Paul Rubin kirjoitti:
>> meff <em...@example.com> writes:
>> We have Oforth, 8th, Factor, and maybe some others as Forth-descended
>> languages that might fit the bill. These all have their aficionados.
>> I'm personally not that enthused about any of them, since the Forth
>> features they preserve (RPN and visible stacks) are imho not that
>> helpful once the language implementation is that complicated. But
>> that's just me.
>
> 8th fits the bill nicely as there are high level words for string processing. I would probably write something like:
>
> 0 "/usr/share/dict/words" f:slurp
> ( "a" search null? nip nip if n:1+ then ) s:eachline . cr
>
> or a little slower:
>
> 0 "/usr/share/dict/words" f:open-ro
> ( "a" search null? nip nip if n:1+ then ) f:eachline drop . cr
>
> My dictionary is a lot bigger: 654299 lines. First code runs in: 0m0.341s on my Raspberry PI 4B.

I also took a look, my version:

0 >r "/usr/share/dict/words" f:open-ro
( "a" s:search number? if 1 r+ then 2drop ) f:eachline drop
r> . cr bye

On Sunday, December 26, 2021 at 2:43:41 AM UTC-7, Nickolay Kolchin wrote:
> On Saturday, December 25, 2021 at 11:54:37 PM UTC+3, Hugh Aguilar wrote:
> > Factor's combinators were the inspiration for my rquotations.
> Can you share a link to your factor vs forth comparison?

I haven't written any Factor vs Forth comparison.
In regard to how combinators inspired rquotations, that is obvious.
Here are a couple of HOFs that use rquotations similar to Factor:
------------------------------------------------------------------------------------------------------------------
: |iterate { cnt rq -- }
begin cnt while
rq rex
-1 +to cnt repeat ;

: |until { cnt rq | original -- completed }
cnt to original
begin cnt while
rq rex
-1 +to cnt
if original cnt - exit then
repeat
original ;
------------------------------------------------------------------------------------------------------------------

Here are some functions that use |ITERATE internally.
------------------------------------------------------------------------------------------------------------------

: old-up. \ limit n -- \ assumes LIMIT is above N
?do I . loop ;

: up. ( limit n -- ) \ assumes LIMIT is above N
tuck - r[ dup . 1+ ]r |iterate drop ;

: old-down. \ limit n -- \ assumes LIMIT is below N
swap 1+ swap ?do I . -1 +loop ;

: down. ( limit n -- ) \ assumes LIMIT is below N
tuck - negate r[ dup . 1- ]r |iterate drop ;

: design ( val -- |val| sign )
dup 0< if negate -1 exit then
1 ;

: through. ( limit n -- )
tuck - \ -- n count
design -rot \ -- step n |count|
r[ dup . over + ]r |iterate 2drop ;

: through. { limit n | step -- }
limit n - \ -- count
design to step \ -- |count|
r[ n . step +to n ]r |iterate ;
------------------------------------------------------------------------------------------------------------------

For the most part though, I don't think it is a good idea to use rquotations
for iteration. The rquotations are primarily for supporting general-purpose
data-structures.

On Monday, December 27, 2021 at 6:12:27 AM UTC+3, Hugh Aguilar wrote:
> On Sunday, December 26, 2021 at 2:43:41 AM UTC-7, Nickolay Kolchin wrote:
> > On Saturday, December 25, 2021 at 11:54:37 PM UTC+3, Hugh Aguilar wrote:
> > > Factor's combinators were the inspiration for my rquotations.
> > Can you share a link to your factor vs forth comparison?
> I haven't written any Factor vs Forth comparison.

I thought that was your text: http://www.rosycrew.org/FactorVsForth.pdf

- https://twitter.com/shishini/status/1060677855946915841

Nickolay Kolchin <nbkolchin@gmail.com> writes:
> I thought that was your text: http://www.rosycrew.org/FactorVsForth.pdf
> - https://twitter.com/shishini/status/1060677855946915841

The rosycrew url won't connect and the weibo one linked in the tweet
goes to weibo's main page in Chinese. Factorcode.org is also down.

I haven't looked closely at Factor but I think of it (maybe incorrectly)
as Lisp-like but with postfix syntax.

On Monday, December 27, 2021 at 11:09:26 AM UTC+3, Paul Rubin wrote:
> Nickolay Kolchin <nbko...@gmail.com> writes:
> > I thought that was your text: http://www.rosycrew.org/FactorVsForth.pdf
> > - https://twitter.com/shishini/status/1060677855946915841
> The rosycrew url won't connect and the weibo one linked in the tweet
> goes to weibo's main page in Chinese. Factorcode.org is also down.

Yes, that's why I asked Hugh for pdf. It was written ages ago when Factor
was different, but still can be interesting as a "die-hard" forther view on
a different stack-based language.

>
> I haven't looked closely at Factor but I think of it (maybe incorrectly)
> as Lisp-like but with postfix syntax.

And I thought about APL with RPN notation. :)