On Monday, October 18, 2021 at 5:17:03 AM UTC-5, dxforth wrote:
> I used this in a recent app and figure others may find it useful.
>

For the rare occasion:
:) cat foo
cr .( Hello )
cr .( World )
cr bye
:) xxd foo
00000000: 6372 202e 2820 4865 6c6c 6f20 290d 0a63 cr .( Hello )..c
00000010: 7220 2e28 2057 6f72 6c64 2029 0d0a 6372 r .( World )..cr
00000020: 2062 7965 0d0a bye..
:) sed -e 's/\x0d/\n/' -e 's/\x0a$//' foo| xxd
00000000: 6372 202e 2820 4865 6c6c 6f20 290a 6372 cr .( Hello ).cr
00000010: 202e 2820 576f 726c 6420 290a 6372 2062 .( World ).cr b
00000020: 7965 0a ye.
:) sed -e 's/\x0d/\n/' -e 's/\x0a$//' foo > bar ; frogd '"bar" fload'

Hello
World
:)

--
me

Nickolay Kolchin ezt írta (2021. október 20., szerda, 14:53:46 UTC+2):
> Only 15 times slower than C.
>
> : test
> "test.txt" f:open
> repeat
> f:getline drop
> f:eof? not
> while!
> drop
> ;
>

Hi,

In 8th you can mmap the file, it is faster.

: process-line drop ;

: test
"test.txt" true f:mmap >s
' process-line s:eachline
;

pahihu

On Wednesday, October 20, 2021 at 4:53:21 PM UTC+3, pahihu wrote:
> Nickolay Kolchin ezt írta (2021. október 20., szerda, 14:53:46 UTC+2):
> > Only 15 times slower than C.
> >
> > : test
> > "test.txt" f:open
> > repeat
> > f:getline drop
> > f:eof? not
> > while!
> > drop
> > ;
> >
> Hi,
>
> In 8th you can mmap the file, it is faster.
>
> : process-line drop ;
>
> : test
> "test.txt" true f:mmap >s
> ' process-line s:eachline
> ;
>

Yes. A bit faster -- 2,982s.

On 20/10/2021 15:53, Nickolay Kolchin wrote:
> On Wednesday, October 20, 2021 at 2:58:24 PM UTC+3, Ron AARON wrote:
>> On 20/10/2021 14:47, Nickolay Kolchin wrote:
>>> Nevertheless, it should not be worth the 15-45 times performance regression.
>> 8th's "f:getline" also handles all the CRLF,CR,LF variants (mixed in the
>> same file or not). Precisely because we can't control what format data
>> arrives in.
>>
>> The performance penalty is minuscule, because the only time a check is
>> performed is if the character read was a CR or LF.
>>
>> In my experience, the time spent in the entire rest of your program is
>> almost always greater than the time spent parsing lines of text from a
>> text file.
>
> Only 15 times slower than C.
>
> : test
> "test.txt" f:open
> repeat
> f:getline drop
> f:eof? not
> while!
> drop
> ;
>
> test bye
>
> - 4.344s

Yes, it is slower than C. But C doesn't handle arbitrary EOL
terminators, nor does it allocate a dynamic string to pass on. So not
exactly a head-to-head comparison.

Furthermore, my point is that the time spent in reading a line is most
often much less than whatever time you spend processing that line.
You're just dropping it, which is not a particularly useful test case.

In any event, while I do spend some effort in making primitives faster
in 8th, I don't think I've actually every tried to make f:getline any
faster. Nobody has complained so far... and there's lots of other stuff
to do.

On 20/10/2021 16:53, pahihu wrote:
> Nickolay Kolchin ezt írta (2021. október 20., szerda, 14:53:46 UTC+2):
>> Only 15 times slower than C.
>>
>> : test
>> "test.txt" f:open
>> repeat
>> f:getline drop
>> f:eof? not
>> while!
>> drop
>> ;
>>
>
> Hi,
>
> In 8th you can mmap the file, it is faster.
>
> : process-line drop ;
>
> : test
> "test.txt" true f:mmap >s
> ' process-line s:eachline
> ;

True, that is faster (depending on the system configuration as well).

I still maintain that in normal use, what you do with the line is going
to take more time than just snipping it from the file.

The f:getline is built on top of C library fgetc().

I just did a 'perf' analysis, and 86% of the time of 'getline' is
actually spent in getc(). So probably a refactor would be a good idea...

On Wednesday, October 20, 2021 at 7:00:00 PM UTC+3, Ron AARON wrote:
> On 20/10/2021 15:53, Nickolay Kolchin wrote:
> > On Wednesday, October 20, 2021 at 2:58:24 PM UTC+3, Ron AARON wrote:
> >> On 20/10/2021 14:47, Nickolay Kolchin wrote:
> >>> Nevertheless, it should not be worth the 15-45 times performance regression.
> >> 8th's "f:getline" also handles all the CRLF,CR,LF variants (mixed in the
> >> same file or not). Precisely because we can't control what format data
> >> arrives in.
> >>
> >> The performance penalty is minuscule, because the only time a check is
> >> performed is if the character read was a CR or LF.
> >>
> >> In my experience, the time spent in the entire rest of your program is
> >> almost always greater than the time spent parsing lines of text from a
> >> text file.
> >
> > Only 15 times slower than C.
> >
> > : test
> > "test.txt" f:open
> > repeat
> > f:getline drop
> > f:eof? not
> > while!
> > drop
> > ;
> >
> > test bye
> >
> > - 4.344s
> Yes, it is slower than C. But C doesn't handle arbitrary EOL
> terminators, nor does it allocate a dynamic string to pass on. So not
> exactly a head-to-head comparison.
>
> Furthermore, my point is that the time spent in reading a line is most
> often much less than whatever time you spend processing that line.
> You're just dropping it, which is not a particularly useful test case.
>

oforth will be probably a fair comparison:

: test
"test.txt" File new dup open(File.READ)
begin
dup readLine drop dup end?
until
;

test bye

Result -- 1,757s

> In any event, while I do spend some effort in making primitives faster
> in 8th, I don't think I've actually every tried to make f:getline any
> faster. Nobody has complained so far... and there's lots of other stuff
> to do.

That's true. 8th is definitely not a performance workhorse.

On 20/10/2021 19:36, Nickolay Kolchin wrote:
> On Wednesday, October 20, 2021 at 7:00:00 PM UTC+3, Ron AARON wrote:
>> On 20/10/2021 15:53, Nickolay Kolchin wrote:
>>> On Wednesday, October 20, 2021 at 2:58:24 PM UTC+3, Ron AARON wrote:
>>>> On 20/10/2021 14:47, Nickolay Kolchin wrote:
>>>>> Nevertheless, it should not be worth the 15-45 times performance regression.
>>>> 8th's "f:getline" also handles all the CRLF,CR,LF variants (mixed in the
>>>> same file or not). Precisely because we can't control what format data
>>>> arrives in.
>>>>
>>>> The performance penalty is minuscule, because the only time a check is
>>>> performed is if the character read was a CR or LF.
>>>>
>>>> In my experience, the time spent in the entire rest of your program is
>>>> almost always greater than the time spent parsing lines of text from a
>>>> text file.
>>>
>>> Only 15 times slower than C.
>>>
>>> : test
>>> "test.txt" f:open
>>> repeat
>>> f:getline drop
>>> f:eof? not
>>> while!
>>> drop
>>> ;
>>>
>>> test bye
>>>
>>> - 4.344s
>> Yes, it is slower than C. But C doesn't handle arbitrary EOL
>> terminators, nor does it allocate a dynamic string to pass on. So not
>> exactly a head-to-head comparison.
>>
>> Furthermore, my point is that the time spent in reading a line is most
>> often much less than whatever time you spend processing that line.
>> You're just dropping it, which is not a particularly useful test case.
>>
>
> oforth will be probably a fair comparison:
>
> : test
> "test.txt" File new dup open(File.READ)
> begin
> dup readLine drop dup end?
> until
> ;
>
> test bye
>
> Result -- 1,757s
>
>> In any event, while I do spend some effort in making primitives faster
>> in 8th, I don't think I've actually every tried to make f:getline any
>> faster. Nobody has complained so far... and there's lots of other stuff
>> to do.
>
> That's true. 8th is definitely not a performance workhorse.

No arguments from me: performance per se is not one of its goals.
*Adequate* performance for the tasks it's used for is.

Nickolay Kolchin <nbkolchin@gmail.com> writes:
>- gforth-0.7.3: 3.730s
>- gforth-fast-0.7.3: 3.296s
>- gforth 0.7.9_20211007: 5.228s
>- gforth-fast 0.7.9_20211007: 5.037s
....
>P.S. Please treat this as an official performance regression report for
>gforth.

Thank you. Mostly Fixed by commit
1d7f2373f67acea44d8a2e084b29f1ac2e375022.

For my smaller benchmark:

LC_NUMERIC=en_US.utf8 perf stat -e cycles -e cycles:u -e cycles:k -e instructions:u gforth-fast -e 's" ../forth/count-unique.in" r/o open-file throw constant f create buf 256 allot : foo 0 begin buf 256 f read-line throw nip while 1+ repeat ; foo . cr bye'
cycles:u
802M 0.7.9 before patch
503,446,908 0.7.9 with patch
485,725,830 0.7.3

It's possible to be a lot faster by implementing buffering for
READ-FILE and READ-LINE ourselves (one can then use SIMD instructions
for scanning for CR and LF), but for now it does not appear to be that
important to be faster to go to these lengths.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

Nickolay Kolchin <nbkolchin@gmail.com> writes:
>The real question -- why gforth doesn't use fgets()?

That's a good question. I looked at gforth-0.3.0 which only deals
with one kind of newline, and it does not use fgets(), either; so it's
not just that fgets() does not work with our requirements for
newlines, apparently we found a mismatch between READ-LINE
requirements and what fgets() provides already earlier. I don't know
what that mismatch was, though.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 2021-10-18, dxforth <dxforth@gmail.com> wrote:
> I used this in a recent app and figure others may find it useful.
>
> My READ-LINE automatically handles LF or CRLF end-of-line (as well as
> CP/M end-of-file character). Rarely do I process files with CR line
> endings but should it happen, here's a substitute routine I've used
> for that. It automatically handles CR, LF or CRLF endings.
>
What are supported line endings when standard line endings are NOT?

--

7-77-777
Evil Sinner!
with software, you repeat same experiment, expecting different results...

On 21/10/2021 02:59, Ron AARON wrote:
>
> Furthermore, my point is that the time spent in reading a line is most
> often much less than whatever time you spend processing that line.
> You're just dropping it, which is not a particularly useful test case.

Everyone loves a benchmark comparison - never mind how useful it is :)

Back in the day how fast your forth compiled source was seen as a
measure of its quality. Despite the hard drives and fast desktops of
today old habits die hard. SwiftForth's multi-thread dictionary and
reading in whole source files before compiling being an example. The
argument will be that memory is cheap so why not.

On 21/10/2021 12:00, Branimir Maksimovic wrote:
> On 2021-10-18, dxforth <dxforth@gmail.com> wrote:
>> I used this in a recent app and figure others may find it useful.
>>
>> My READ-LINE automatically handles LF or CRLF end-of-line (as well as
>> CP/M end-of-file character). Rarely do I process files with CR line
>> endings but should it happen, here's a substitute routine I've used
>> for that. It automatically handles CR, LF or CRLF endings.
>>
> What are supported line endings when standard line endings are NOT?
>

The line endings supported by READ-LINE is implementation-defined.

On 21/10/2021 00:28, S Jack wrote:
> On Monday, October 18, 2021 at 5:17:03 AM UTC-5, dxforth wrote:
>> I used this in a recent app and figure others may find it useful.
>>
>
> For the rare occasion:
> :) cat foo
> cr .( Hello )
> cr .( World )
> cr bye
> :) xxd foo
> 00000000: 6372 202e 2820 4865 6c6c 6f20 290d 0a63 cr .( Hello )..c
> 00000010: 7220 2e28 2057 6f72 6c64 2029 0d0a 6372 r .( World )..cr
> 00000020: 2062 7965 0d0a bye..
> :) sed -e 's/\x0d/\n/' -e 's/\x0a$//' foo| xxd
> 00000000: 6372 202e 2820 4865 6c6c 6f20 290a 6372 cr .( Hello ).cr
> 00000010: 202e 2820 576f 726c 6420 290a 6372 2062 .( World ).cr b
> 00000020: 7965 0a ye.
> :) sed -e 's/\x0d/\n/' -e 's/\x0a$//' foo > bar ; frogd '"bar" fload'
>
> Hello
> World
> :)

This is equivalent to using an external utility to convert a text file
to the desired line endings before passing it to the application?

That's what I did for the first version of the app. However because
conversion would almost certainly be necessary, I decided the next
version of the app should handle the conversion internally and not
burden the user.

On Thursday, October 21, 2021 at 1:27:23 AM UTC+3, Anton Ertl wrote:
> Nickolay Kolchin <nbko...@gmail.com> writes:
> >- gforth-0.7.3: 3.730s
> >- gforth-fast-0.7.3: 3.296s
> >- gforth 0.7.9_20211007: 5.228s
> >- gforth-fast 0.7.9_20211007: 5.037s
> ...
> >P.S. Please treat this as an official performance regression report for
> >gforth.
> Thank you. Mostly Fixed by commit
> 1d7f2373f67acea44d8a2e084b29f1ac2e375022.
>

Confirmed.

- gforth-0.7.9_20211014: 2.783s
- gforth-fast-0.7.9_20211014: 2.686s

Re-written for a tighter main loop:

\ Scan for CR LF CRLF return len & offset to next line
: /EOL ( a u -- u' offs )
over swap begin dup while
over c@ dup $0D - while $0A - while 1 /string
repeat \ got LF
drop swap - dup 1+ exit
then \ got CR
2drop tuck swap - swap 1+ c@ $0A <> over + 2+ exit
then \ neither
drop swap - dup ;

Heinrich Hohl <hheinrich.hohl@gmail.com> writes:
>On Tuesday, October 19, 2021 at 10:40:49 AM UTC+2, Anton Ertl wrote:
>> Which files would READ-LINE open and close and why? Gforth's
>> READ-FILE certainly does not do that.
>
>Sorry, my mistake. You are right. Files are opened only once, then many
>READ-LINE operations are executed, and finally the file is closed again.
>But the file position must be updated after reading each line.

Yes, that's what SwiftForth is doing.

>Actually, speed was not that bad when I read the postscript file from a local hard disk.
>I assume that the hard disk cache made sure that performance does not suffer too much.

On a competent OS the OS cache does that, but it still costs system
call overhead (and the overhead of reading more than needed).

The alternative is to use user-level buffered I/O. Gforth uses C's
buffered I/O for that. Unfortunately the C interface is not so great
for implementing READ-LINE, but still better than using raw file
access.

>However, the file converter was extremely slow when I tried to read the
>huge postscript file from a NAS via the network. Not usable at all.
>Maybe file repositioning is slow if a network is involved.

What network file system did you use? NFS tends to cache the read
file for some seconds, so with NFS I would not expect to see such
behaviour.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

dxforth <dxforth@gmail.com> writes:
>On 20/10/2021 12:45, dxforth wrote:
>> On 20/10/2021 06:26, Nickolay Kolchin wrote:
>>>
>>> 1024 CONSTANT maxline
>>> maxline BUFFER: buf
>>
>> Shouldn't that be:
>>
>> maxline 2 + BUFFER: buf
>
>I notice ANS says:
>
>"The line buffer provided by c-addr should be at least u1+2 characters long."
>
>"should" equates to "recommend". Why not "shall"? AFAIR a standard program
>has no knowledge of READ-LINE's workings and must assume worst case.

Yes. That's pretty clear from the text, so the use of "should"
apparently has not caused confusion (I don't remember anybody being
confused by it). It's an error-prone interface, though.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 22/10/2021 03:09, Anton Ertl wrote:
> dxforth <dxforth@gmail.com> writes:
>>On 20/10/2021 12:45, dxforth wrote:
>>> On 20/10/2021 06:26, Nickolay Kolchin wrote:
>>>>
>>>> 1024 CONSTANT maxline
>>>> maxline BUFFER: buf
>>>
>>> Shouldn't that be:
>>>
>>> maxline 2 + BUFFER: buf
>>
>>I notice ANS says:
>>
>>"The line buffer provided by c-addr should be at least u1+2 characters long."
>>
>>"should" equates to "recommend". Why not "shall"? AFAIR a standard program
>>has no knowledge of READ-LINE's workings and must assume worst case.
>
> Yes. That's pretty clear from the text, so the use of "should"
> apparently has not caused confusion (I don't remember anybody being
> confused by it).

Because they weren't bitten by it? Interesting to know how many
implementations actually crash if the extra buffer space is not
provided.

> It's an error-prone interface, though.

Remembering to add 2 to the buffer size? Agree with that but
what's the alternative if one insists on handling any line ending?

On 22/10/2021 11:43, dxforth wrote:
> On 22/10/2021 03:09, Anton Ertl wrote:
>> dxforth <dxforth@gmail.com> writes:
>>>On 20/10/2021 12:45, dxforth wrote:
>>>> On 20/10/2021 06:26, Nickolay Kolchin wrote:
>>>>>
>>>>> 1024 CONSTANT maxline
>>>>> maxline BUFFER: buf
>>>>
>>>> Shouldn't that be:
>>>>
>>>> maxline 2 + BUFFER: buf
>>>
>>>I notice ANS says:
>>>
>>>"The line buffer provided by c-addr should be at least u1+2 characters long."
>>>
>>>"should" equates to "recommend". Why not "shall"? AFAIR a standard program
>>>has no knowledge of READ-LINE's workings and must assume worst case.
>>
>> Yes. That's pretty clear from the text, so the use of "should"
>> apparently has not caused confusion (I don't remember anybody being
>> confused by it).
>
> Because they weren't bitten by it? Interesting to know how many
> implementations actually crash if the extra buffer space is not
> provided.
>
>> It's an error-prone interface, though.
>
> Remembering to add 2 to the buffer size? Agree with that but
> what's the alternative if one insists on handling any line ending?

I suppose the spec could have been written:

"Read the next line from the file specified by fileid into memory given
by address /c-addr u1/. Up to two implementation-defined line-terminating
characters may be read into memory at the end of the line, but are not
included in the count u2. The line buffer provided by c-addr /shall/ be
at least 2 characters long."

On Tuesday, October 19, 2021 at 9:26:56 PM UTC+2, Nickolay Kolchin wrote:
[..]
> test.txt: 1016666745 bytes (970Mb). The file was generated from FASTA
> "language shootout benchmark" with argument 100000000.

Can you provide a simpler test file than text.txt? I don't want to implement
the fasta program just for this purpose.

-marcel

On Friday, October 22, 2021 at 10:10:48 AM UTC+3, Marcel Hendrix wrote:
> On Tuesday, October 19, 2021 at 9:26:56 PM UTC+2, Nickolay Kolchin wrote:
> [..]
> > test.txt: 1016666745 bytes (970Mb). The file was generated from FASTA
> > "language shootout benchmark" with argument 100000000.
>
> Can you provide a simpler test file than text.txt? I don't want to implement
> the fasta program just for this purpose.
>
> -marcel

Here is the link to python program that generates required file:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/fasta-python3-2.html

execute with:

python3 app.py 100000000

And on next link you can find programs in other languages that can generate required output.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/fasta.html

Here is link to pregenerated file: https://nbkolchin.com/forth/test.txt.gz

dxforth <dxforth@gmail.com> writes:
>Because they weren't bitten by it? Interesting to know how many
>implementations actually crash if the extra buffer space is not
>provided.

Typically the Forth system may overwrite these two characters, so if
that memory is accessible (the usual case), the result is not a crash
of READ-LINE, but memory corruption.

>> It's an error-prone interface, though.
>
>Remembering to add 2 to the buffer size? Agree with that but
>what's the alternative if one insists on handling any line ending?

1) If we stick with the read-line stack effect, the ideal is to just
write the characters into the buffer without the line ending. That
requires either an interface that gives you one character in a
register (not something like Unix' read()), or you need to read stuff
into a buffer elsewhere (not so nice for very small systems).

2) The given length is the total buffer size, u2 is the size excluding
any newline characters, and whether the line end has yet to be reached
is indicated in some other way than by checking whether u1=u2; maybe
with a specific flag value.

If we let our minds wander further: READ-LINE's nastyness comes from
designing an interface that deals with arbitrarily long lines, but a
pre-allocated buffer that may be too short for the line. If we allow
to dynamically allocate the buffer, we can have a much nicer
interface, as discussed in
<2018Jun12.105031@mips.complang.tuwien.ac.at>:

|GET-LINE ( fid -- c-addr u flag ior )
| |flag is false at the end of file

But of course, that's not an interface for tiny embedded systems.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

dxforth <dxforth@gmail.com> writes:
>On 22/10/2021 11:43, dxforth wrote:
>> On 22/10/2021 03:09, Anton Ertl wrote:
>>> It's an error-prone interface, though.
>>
>> Remembering to add 2 to the buffer size? Agree with that but
>> what's the alternative if one insists on handling any line ending?
>
>I suppose the spec could have been written:
>
> "Read the next line from the file specified by fileid into memory given
> by address /c-addr u1/. Up to two implementation-defined line-terminating
> characters may be read into memory at the end of the line, but are not
> included in the count u2. The line buffer provided by c-addr /shall/ be
> at least 2 characters long."

Yes, that's my option 2), but you also need to specify how to
recognize whether the line end has been reached or not.

Anyway, it's water down the river.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On 22/10/2021 19:30, Anton Ertl wrote:
> dxforth <dxforth@gmail.com> writes:
>>Because they weren't bitten by it? Interesting to know how many
>>implementations actually crash if the extra buffer space is not
>>provided.
>
> Typically the Forth system may overwrite these two characters, so if
> that memory is accessible (the usual case), the result is not a crash
> of READ-LINE, but memory corruption.

I'm thinking of the /EOL algorithm which relies on a preceding file
read 1 character greater than the user requested. If the user
assigned a buffer of u characters and asked READ-LINE for u characters
then the buffer will be overwritten. Does fgets have this problem
of overwrite - or does it conform to the buffer size it has been given?

>
>>> It's an error-prone interface, though.
>>
>>Remembering to add 2 to the buffer size? Agree with that but
>>what's the alternative if one insists on handling any line ending?
>
> 1) If we stick with the read-line stack effect, the ideal is to just
> write the characters into the buffer without the line ending. That
> requires either an interface that gives you one character in a
> register (not something like Unix' read()), or you need to read stuff
> into a buffer elsewhere (not so nice for very small systems).
>
> 2) The given length is the total buffer size, u2 is the size excluding
> any newline characters, and whether the line end has yet to be reached
> is indicated in some other way than by checking whether u1=u2; maybe
> with a specific flag value.
>
> If we let our minds wander further: READ-LINE's nastyness comes from
> designing an interface that deals with arbitrarily long lines, but a
> pre-allocated buffer that may be too short for the line. If we allow
> to dynamically allocate the buffer, we can have a much nicer
> interface, as discussed in
> <2018Jun12.105031@mips.complang.tuwien.ac.at>:
>
> |GET-LINE ( fid -- c-addr u flag ior )
> |
> |flag is false at the end of file
>
> But of course, that's not an interface for tiny embedded systems.

That's something I'd have in an app or library. Implementation still
requires a block read function and a buffer.

Marcel Hendrix <mhx@iae.nl> writes:
>On Tuesday, October 19, 2021 at 9:26:56 PM UTC+2, Nickolay Kolchin wrote:
>[..]
>> test.txt: 1016666745 bytes (970Mb). The file was generated from FASTA
>> "language shootout benchmark" with argument 100000000.
>
>Can you provide a simpler test file than text.txt? I don't want to implement
>the fasta program just for this purpose.

I ran iforth on the ten bibles from Ben Hoyt's count-unique task:

LC_NUMERIC=en_US.utf8 perf stat -e cycles -e cycles:u -e cycles:k -e instructions:u iforth 's" /home/anton/forth/count-unique.in" r/o open-file throw constant f create buf 256 allot : foo 0 begin buf 256 f read-line throw nip while 1+ repeat ; foo . cr bye'
998170

Performance counter stats for 'iforth s" /home/anton/forth/count-unique.in" r/o open-file throw constant f create buf 256 allot : foo 0 begin buf 256 f read-line throw nip while 1+ repeat ; foo . cr bye':

971,836,455 cycles
788,403,506 cycles:u
183,080,326 cycles:k
1,827,909,289 instructions:u # 2.32 insn per cycle

0.250143388 seconds time elapsed

0.208843000 seconds user
0.036146000 seconds sys

For comparison:

LC_NUMERIC=en_US.utf8 perf stat -e cycles -e cycles:u -e cycles:k -e instructions:u gforth-fast -e 's" /home/anton/forth/count-unique.in" r/o open-file throw constant f create buf 256 allot : foo 0 begin buf 256 f read-line throw nip while 1+ repeat ; foo . cr bye'
998170

Performance counter stats for 'gforth-fast -e s" /home/anton/forth/count-unique.in" r/o open-file throw constant f create buf 256 allot : foo 0 begin buf 256 f read-line throw nip while 1+ repeat ; foo . cr bye':

549,667,728 cycles
506,120,832 cycles:u
43,316,660 cycles:k
1,606,393,446 instructions:u # 3.17 insn per cycle

0.144901109 seconds time elapsed

0.139777000 seconds user
0.003993000 seconds sys

So slightly worse than Gforth, quite a bit better than other Forth
systems. Quite a lot of the speed difference is due to iForth's
bigger startup overhead (346M cycles=166M cycles:u+180M cycles:k).

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021