On 2021-11-02 03:53, dxforth wrote:
> On 2/11/2021 01:16, Anton Ertl wrote:
>> Ruvim <ruvim.pinka@gmail.com> writes:
>>> It's obvious that if u2=0 then the line is empty (when flag is true).
>>
>> What makes you think so?
>>
>> The standard says:
>>
>> |When u1 = u2 the line terminator has yet to be reached.
>>
>> That also holds if u1=u2=0.
>>
>>> So u2=u1 (regardless of u1) should always mean that a line terminator
>>> is not reached.
>>
>> It does.  So if the programmer calls READ-LINE with u1=0, READ-LINE
>> should not consume any characters from the file.  Conversely, if the
>> program should make any progress in the file, it should call READ-LINE
>> with u1>0.
>>
>>> Agreed. But some other mistakes are still present in this glossary
>>> entry anyway (e.g. in "before u1").
>>
>> Apart from the misleading "0 <= u2 <= u1" (where "0 <= u2 <u1" would
>> be better), anything else?
>
> So the ANS spec isn't really dead - it has "misleading" statements (you);
> or implementers should rewind the file in the case of u1=u2 and line
> terminator received (Ruvim).

It depends on a number of factors whether an implementer should rewind
the file.

In some combinations of line terminators it's possible to implement
READ-LINE without rewind (reposition back), by reading character by
character.

But if you need to look ahead, or use something like "SEARCH", you have
to rewind the file in some cases (not only u2=u1). And if the underlying
stream cannot be rewound, you have to implement buffering.

OTOH, if you implement buffering, reading character by character becomes
not so expensive.

--
Ruvim

On 2/11/2021 21:23, Ruvim wrote:
> On 2021-11-02 03:53, dxforth wrote:
>> On 2/11/2021 01:16, Anton Ertl wrote:
>>> Ruvim <ruvim.pinka@gmail.com> writes:
>>>> It's obvious that if u2=0 then the line is empty (when flag is true).
>>>
>>> What makes you think so?
>>>
>>> The standard says:
>>>
>>> |When u1 = u2 the line terminator has yet to be reached.
>>>
>>> That also holds if u1=u2=0.
>>>
>>>> So u2=u1 (regardless of u1) should always mean that a line terminator
>>>> is not reached.
>>>
>>> It does.  So if the programmer calls READ-LINE with u1=0, READ-LINE
>>> should not consume any characters from the file.  Conversely, if the
>>> program should make any progress in the file, it should call READ-LINE
>>> with u1>0.
>>>
>>>> Agreed. But some other mistakes are still present in this glossary
>>>> entry anyway (e.g. in "before u1").
>>>
>>> Apart from the misleading "0 <= u2 <= u1" (where "0 <= u2 <u1" would
>>> be better), anything else?
>>
>> So the ANS spec isn't really dead - it has "misleading" statements (you);
>> or implementers should rewind the file in the case of u1=u2 and line
>> terminator received (Ruvim).
>
> It depends on a number of factors whether an implementer should rewind
> the file.
>
> In some combinations of line terminators it's possible to implement
> READ-LINE without rewind (reposition back), by reading character by
> character.
>
> But if you need to look ahead, or use something like "SEARCH", you have
> to rewind the file in some cases (not only u2=u1). And if the underlying
> stream cannot be rewound, you have to implement buffering.
>
> OTOH, if you implement buffering, reading character by character becomes
> not so expensive.

These are band-aids solutions for a problematic specification. The question
is whether to keep the spec or replace it. ISTM there's little stomach for
the latter, not least because it would undermine ANS.

On 2021-11-03 08:51, dxforth wrote:
> On 2/11/2021 21:23, Ruvim wrote:
>> On 2021-11-02 03:53, dxforth wrote:
>>> On 2/11/2021 01:16, Anton Ertl wrote:
>>>> Ruvim <ruvim.pinka@gmail.com> writes:
[...]
>>>>> But some other mistakes are still present in this glossary
>>>>> entry anyway (e.g. in "before u1").
>>>>
>>>> Apart from the misleading "0 <= u2 <= u1" (where "0 <= u2 <u1" would
>>>> be better), anything else?
>>>
>>> So the ANS spec isn't really dead - it has "misleading" statements
>>> (you);
>>> or implementers should rewind the file in the case of u1=u2 and line
>>> terminator received (Ruvim).
>>
>> It depends on a number of factors whether an implementer should rewind
>> the file.
>>
>> In some combinations of line terminators it's possible to implement
>> READ-LINE without rewind (reposition back), by reading character by
>> character.
>>
>> But if you need to look ahead, or use something like "SEARCH", you have
>> to rewind the file in some cases (not only u2=u1). And if the underlying
>> stream cannot be rewound, you have to implement buffering.
>>
>> OTOH, if you implement buffering, reading character by character becomes
>> not so expensive.
>
> These are band-aids solutions for a problematic specification.

What make you think the specification for READ-LINE is problematic?

BTW, reservation of 2 characters makes nothing problem for implementations.

> The question is whether to keep the spec or replace it.

Do you mean an incompatible replacement?

> ISTM there's little stomach for the latter, not least because
> it would undermine ANS.

There's little ground for discussion without your draft for a new
specification ;-)

--
Ruvim

On 3/11/2021 18:48, Ruvim wrote:
> On 2021-11-03 08:51, dxforth wrote:
>> On 2/11/2021 21:23, Ruvim wrote:
>>> On 2021-11-02 03:53, dxforth wrote:
>>>> On 2/11/2021 01:16, Anton Ertl wrote:
>>>>> Ruvim <ruvim.pinka@gmail.com> writes:
> [...]
>>>>>> But some other mistakes are still present in this glossary
>>>>>> entry anyway (e.g. in "before u1").
>>>>>
>>>>> Apart from the misleading "0 <= u2 <= u1" (where "0 <= u2 <u1" would
>>>>> be better), anything else?
>>>>
>>>> So the ANS spec isn't really dead - it has "misleading" statements
>>>> (you);
>>>> or implementers should rewind the file in the case of u1=u2 and line
>>>> terminator received (Ruvim).
>>>
>>> It depends on a number of factors whether an implementer should rewind
>>> the file.
>>>
>>> In some combinations of line terminators it's possible to implement
>>> READ-LINE without rewind (reposition back), by reading character by
>>> character.
>>>
>>> But if you need to look ahead, or use something like "SEARCH", you have
>>> to rewind the file in some cases (not only u2=u1). And if the underlying
>>> stream cannot be rewound, you have to implement buffering.
>>>
>>> OTOH, if you implement buffering, reading character by character becomes
>>> not so expensive.
>>
>> These are band-aids solutions for a problematic specification.
>
> What make you think the specification for READ-LINE is problematic?

It's open to multiple interpretations e.g.

"When u1 = u2, the line terminator has yet to be reached."

>
> BTW, reservation of 2 characters makes nothing problem for implementations.

Can you imagine an fgets with such a requirement :)

>
>> The question is whether to keep the spec or replace it.
>
> Do you mean an incompatible replacement?

Yes

>
>> ISTM there's little stomach for the latter, not least because
>> it would undermine ANS.
>
>
> There's little ground for discussion without your draft for a new
> specification ;-)

Let me know what needs fixing and I'll consider it :)

On 3/11/2021 22:33, dxforth wrote:
> On 3/11/2021 18:48, Ruvim wrote:
>> On 2021-11-03 08:51, dxforth wrote:
>> ...
>>> The question is whether to keep the spec or replace it.
>>
>> Do you mean an incompatible replacement?
>
> Yes

That was in reference to the need for an additional flag to indicate
EOL received. Thinking about it, there's no reason this can't be
combined with the existing 'success' flag.

Summary of changes:

- Previously 'flag' was true|false. In the replacement it may take
the values -1|0|1 so we rename 'flag' to 'n3'.

- Previously 'u1' was the maximum chars to receive. Now it refers
to the buffer size.

\ Sample implementation

\ Scan for CR LF CRLF return len & offset to next line
: /EOL ( a u -- u' offs )
over swap begin dup while
over c@ dup $0D - while $0A - while 1 /string
repeat \ got LF
drop swap - dup 1+ exit
then \ got CR
2drop tuck swap - swap 1+ c@ $0A <> over + 2 + exit
then \ neither
drop swap - dup ;

\ DTC 8086 version
\ Scan for CR LF CRLF return len & offset to next line
code /EOL ( a u -- u' offs )
cx pop bx pop bx di mov dx dx sub $0A0D # ax mov
1 $: 4 $ jcxz al 0 [bx] cmp 2 $ jz ah 0 [bx] cmp 3 $ jz
bx inc cx dec 1 $ ju 2 $: dx inc ah 1 [bx] cmp 4 $ jnz
3 $: dx inc 4 $: di bx sub bx push dx bx add bx push
next
end-code

\ buffer size at least u chars and a minimum of one
\ n3: 0 = EOF, -1 = EOL received, 1 = EOL not received
: READ-LINE ( a u1 fid -- u2 n3 ior )
>r 2dup r@ read-file dup if r> drop nip exit then \ ior
over 0= if r> drop 2swap 2drop 0 exit then \ EOF
drop >r tuck 1- r@ min /eol r> - s>d
r@ file-position drop d+ r> reposition-file drop
tuck 1+ = 2* 1+ 0 ;

The above checked ok on SwiftForth, VFX, Gforth, LX|NT/Forth, Win32Forth;
but not on Minforth 3.4.7 (I didn't look into why).
The test used:

https://pastebin.com/5vnsvehx

On Friday, November 5, 2021 at 3:04:22 PM UTC+3, dxforth wrote:
> On 3/11/2021 22:33, dxforth wrote:

here 2 c, 13 c, 10 c, count 2constant

Is this generally portable? I.e. implementation can use CELL as size for
counted strings.

P.S. Back in "good old days" we distinguish between "counted" (CELL
sized) and "packed" (byte size) strings.

dxforth schrieb am Freitag, 5. November 2021 um 13:04:22 UTC+1:
> On 3/11/2021 22:33, dxforth wrote:
> > On 3/11/2021 18:48, Ruvim wrote:
> >> On 2021-11-03 08:51, dxforth wrote:
> >> ...
> >>> The question is whether to keep the spec or replace it.
> >>
> >> Do you mean an incompatible replacement?
> >
> > Yes
> That was in reference to the need for an additional flag to indicate
> EOL received. Thinking about it, there's no reason this can't be
> combined with the existing 'success' flag.
>
> Summary of changes:
>
> - Previously 'flag' was true|false. In the replacement it may take
> the values -1|0|1 so we rename 'flag' to 'n3'.
>
> - Previously 'u1' was the maximum chars to receive. Now it refers
> to the buffer size.
>
> \ Sample implementation
>
> \ Scan for CR LF CRLF return len & offset to next line
> : /EOL ( a u -- u' offs )
> over swap begin dup while
> over c@ dup $0D - while $0A - while 1 /string
> repeat \ got LF
> drop swap - dup 1+ exit
> then \ got CR
> 2drop tuck swap - swap 1+ c@ $0A <> over + 2 + exit
> then \ neither
> drop swap - dup ;
>
> \ DTC 8086 version
> \ Scan for CR LF CRLF return len & offset to next line
> code /EOL ( a u -- u' offs )
> cx pop bx pop bx di mov dx dx sub $0A0D # ax mov
> 1 $: 4 $ jcxz al 0 [bx] cmp 2 $ jz ah 0 [bx] cmp 3 $ jz
> bx inc cx dec 1 $ ju 2 $: dx inc ah 1 [bx] cmp 4 $ jnz
> 3 $: dx inc 4 $: di bx sub bx push dx bx add bx push
> next
> end-code
>
> \ buffer size at least u chars and a minimum of one
> \ n3: 0 = EOF, -1 = EOL received, 1 = EOL not received
> : READ-LINE ( a u1 fid -- u2 n3 ior )
> >r 2dup r@ read-file dup if r> drop nip exit then \ ior
> over 0= if r> drop 2swap 2drop 0 exit then \ EOF
> drop >r tuck 1- r@ min /eol r> - s>d
> r@ file-position drop d+ r> reposition-file drop
> tuck 1+ = 2* 1+ 0 ;
>
> The above checked ok on SwiftForth, VFX, Gforth, LX|NT/Forth, Win32Forth;
> but not on Minforth 3.4.7 (I didn't look into why).
> The test used:
>
> https://pastebin.com/5vnsvehx

MF347 had an unnecessary buffer size restriction (apart from not
considering Mac line endings). Thanks for pointing this out.

I just checked with actual version MF348 and it seems to run ok now.

https://sourceforge.net/projects/minforth/files/

dxforth schrieb am Freitag, 5. November 2021 um 13:04:22 UTC+1:
> On 3/11/2021 22:33, dxforth wrote:
> > On 3/11/2021 18:48, Ruvim wrote:
> >> On 2021-11-03 08:51, dxforth wrote:
> >> ...
> >>> The question is whether to keep the spec or replace it.
> >>
> >> Do you mean an incompatible replacement?
> >
> > Yes
> That was in reference to the need for an additional flag to indicate
> EOL received. Thinking about it, there's no reason this can't be
> combined with the existing 'success' flag.
>
> Summary of changes:
>
> - Previously 'flag' was true|false. In the replacement it may take
> the values -1|0|1 so we rename 'flag' to 'n3'.
>
> - Previously 'u1' was the maximum chars to receive. Now it refers
> to the buffer size.
>
> \ Sample implementation
>
> \ Scan for CR LF CRLF return len & offset to next line
> : /EOL ( a u -- u' offs )
> over swap begin dup while
> over c@ dup $0D - while $0A - while 1 /string
> repeat \ got LF
> drop swap - dup 1+ exit
> then \ got CR
> 2drop tuck swap - swap 1+ c@ $0A <> over + 2 + exit
> then \ neither
> drop swap - dup ;
>
> \ DTC 8086 version
> \ Scan for CR LF CRLF return len & offset to next line
> code /EOL ( a u -- u' offs )
> cx pop bx pop bx di mov dx dx sub $0A0D # ax mov
> 1 $: 4 $ jcxz al 0 [bx] cmp 2 $ jz ah 0 [bx] cmp 3 $ jz
> bx inc cx dec 1 $ ju 2 $: dx inc ah 1 [bx] cmp 4 $ jnz
> 3 $: dx inc 4 $: di bx sub bx push dx bx add bx push
> next
> end-code
>
> \ buffer size at least u chars and a minimum of one
> \ n3: 0 = EOF, -1 = EOL received, 1 = EOL not received
> : READ-LINE ( a u1 fid -- u2 n3 ior )
> >r 2dup r@ read-file dup if r> drop nip exit then \ ior
> over 0= if r> drop 2swap 2drop 0 exit then \ EOF
> drop >r tuck 1- r@ min /eol r> - s>d
> r@ file-position drop d+ r> reposition-file drop
> tuck 1+ = 2* 1+ 0 ;
>
> The above checked ok on SwiftForth, VFX, Gforth, LX|NT/Forth, Win32Forth;
> but not on Minforth 3.4.7 (I didn't look into why).
> The test used:
>
> https://pastebin.com/5vnsvehx

MF347 had an unnecessary buffer size restriction (apart from not
considering Mac line endings). Thanks for pointing this out.
The actual version MF348 seems to run ok:

https://sourceforge.net/projects/minforth/files/

BTW IMHO you should open/create the test files in BIN mode.
C-library file functions in Windows auto-convert single LFs to CRLFs
in text mode, so one might be in for little surprises...

On 6/11/2021 09:00, minf...@arcor.de wrote:
> dxforth schrieb am Freitag, 5. November 2021 um 13:04:22 UTC+1:
>> On 3/11/2021 22:33, dxforth wrote:
>> ...
>> The above checked ok on SwiftForth, VFX, Gforth, LX|NT/Forth, Win32Forth;
>> but not on Minforth 3.4.7 (I didn't look into why).
>> The test used:
>>
>> https://pastebin.com/5vnsvehx
>
> MF347 had an unnecessary buffer size restriction (apart from not
> considering Mac line endings). Thanks for pointing this out.
> The actual version MF348 seems to run ok:
>
> https://sourceforge.net/projects/minforth/files/
>
> BTW IMHO you should open/create the test files in BIN mode.
> C-library file functions in Windows auto-convert single LFs to CRLFs
> in text mode, so one might be in for little surprises...

Thanks for the Minforth update. Added 'BIN' opening to the test as
suggested (I didn't know about).

*** CORRECTION ***

READ-LINE as implemented above didn't match the description (return
codes were reversed). Revised implementation and test is here:

https://pastebin.com/BT4UQ1Zu

On 6/11/2021 05:05, Nickolay Kolchin wrote:
> On Friday, November 5, 2021 at 3:04:22 PM UTC+3, dxforth wrote:
>> On 3/11/2021 22:33, dxforth wrote:
>
> here 2 c, 13 c, 10 c, count 2constant
>
> Is this generally portable? I.e. implementation can use CELL as size for
> counted strings.
>
> P.S. Back in "good old days" we distinguish between "counted" (CELL
> sized) and "packed" (byte size) strings.
>

'cell-counted' byte strings may have existed in some systems however
ANS defines a 'counted string' as:

"A data structure consisting of one character containing a length
followed by zero or more contiguous data characters. Normally,
counted strings contain text."

On Saturday, November 6, 2021 at 5:23:24 AM UTC+3, dxforth wrote:
> On 6/11/2021 05:05, Nickolay Kolchin wrote:
> > On Friday, November 5, 2021 at 3:04:22 PM UTC+3, dxforth wrote:
> >> On 3/11/2021 22:33, dxforth wrote:
> >
> > here 2 c, 13 c, 10 c, count 2constant
> >
> > Is this generally portable? I.e. implementation can use CELL as size for
> > counted strings.
> >
> > P.S. Back in "good old days" we distinguish between "counted" (CELL
> > sized) and "packed" (byte size) strings.
> >
> 'cell-counted' byte strings may have existed in some systems however
> ANS defines a 'counted string' as:
>
> "A data structure consisting of one character containing a length
> followed by zero or more contiguous data characters. Normally,
> counted strings contain text."

Well, this is another "dumb thing" from ANS. :(

On 2021-11-05 21:05, Nickolay Kolchin wrote:
> On Friday, November 5, 2021 at 3:04:22 PM UTC+3, dxforth wrote:
>> On 3/11/2021 22:33, dxforth wrote:
>
> here 2 c, 13 c, 10 c, count 2constant

A name is missed after "2constant".

Also, a counted string seems to be redundant in this case.
The same can be defined as:

here 13 c, 10 c, 2 2constant crlf

>
> Is this generally portable? I.e. implementation can use CELL as size for
> counted strings.

Yes, it's portable. A standard Forth system may use only a character for
the length in a counted string.

See 3.1.3.4 Counted strings, 6.1.0980 COUNT
https://forth-standard.org/standard/core/COUNT

> P.S. Back in "good old days" we distinguish between "counted" (CELL
> sized) and "packed" (byte size) strings.

In the Forth-83 and Forth-79 standards the length in a counted string
takes exactly one *byte* (8 bits). And the glossary entry for COUNT
said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"

SP-Forth provided a word "XCOUNT ( addr1 -- addr2 u )" for counted-like
strings having one cell for the length:

: xcount ( addr1 -- addr2 u ) dup cell+ swap @ ;

--
Ruvim

On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
> On 2021-11-05 21:05, Nickolay Kolchin wrote:
> > On Friday, November 5, 2021 at 3:04:22 PM UTC+3, dxforth wrote:
> >> On 3/11/2021 22:33, dxforth wrote:
> >
> > here 2 c, 13 c, 10 c, count 2constant
> A name is missed after "2constant".
>
> Also, a counted string seems to be redundant in this case.
> The same can be defined as:
>
> here 13 c, 10 c, 2 2constant crlf
> >
> > Is this generally portable? I.e. implementation can use CELL as size for
> > counted strings.
> Yes, it's portable. A standard Forth system may use only a character for
> the length in a counted string.
>
> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
> https://forth-standard.org/standard/core/COUNT
> > P.S. Back in "good old days" we distinguish between "counted" (CELL
> > sized) and "packed" (byte size) strings.
> In the Forth-83 and Forth-79 standards the length in a counted string
> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
>

Actually F79 used correct naming "packed" and F83 started using
misleading term "counted".

IMHO, 255 bytes limitation is "blast from past" and should be lifted
nowadays...

On 2021-11-06 13:16, Nickolay Kolchin wrote:
> On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
>> On 2021-11-05 21:05, Nickolay Kolchin wrote:

>> A standard Forth system may use only a character for
>> the length in a counted string.
>>
>> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
>> https://forth-standard.org/standard/core/COUNT
>>
>>> P.S. Back in "good old days" we distinguish between "counted" (CELL
>>> sized) and "packed" (byte size) strings.
>>
>> In the Forth-83 and Forth-79 standards the length in a counted string
>> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
>> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
>>
>
> Actually F79 used correct naming "packed" and F83 started using
> misleading term "counted".

As I can see, the term "packed" was used in only one glossary entry,
namely for the "WORD" word. It looks like just an omission.

It's not obvious to me why the "packed string" term is better than
"counted string".

> IMHO, 255 bytes limitation is "blast from past" and should be lifted
> nowadays...

Forth-94 doesn't limit it by 255 bytes, it requires that this limit
shall be *not less* than 255 bytes (i.e., the character size cannot be
less than 8 bits).

The actual limit depends on the character size in the system.

The "counted string" data type is already almost outdated, there only
two standard words use them: "WORD" and "FIND".

The "character string" data type takes two cells on a stack ( c-addr u )
and it limits the string length by the cell size only.

--
Ruvim

On Saturday, November 6, 2021 at 1:49:42 PM UTC+3, Ruvim wrote:
> On 2021-11-06 13:16, Nickolay Kolchin wrote:
> > On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
> >> On 2021-11-05 21:05, Nickolay Kolchin wrote:
>
> >> A standard Forth system may use only a character for
> >> the length in a counted string.
> >>
> >> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
> >> https://forth-standard.org/standard/core/COUNT
> >>
> >>> P.S. Back in "good old days" we distinguish between "counted" (CELL
> >>> sized) and "packed" (byte size) strings.
> >>
> >> In the Forth-83 and Forth-79 standards the length in a counted string
> >> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
> >> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
> >>
> >
> > Actually F79 used correct naming "packed" and F83 started using
> > misleading term "counted".
> As I can see, the term "packed" was used in only one glossary entry,
> namely for the "WORD" word. It looks like just an omission.
>
> It's not obvious to me why the "packed string" term is better than
> "counted string".
> > IMHO, 255 bytes limitation is "blast from past" and should be lifted
> > nowadays...
> Forth-94 doesn't limit it by 255 bytes, it requires that this limit
> shall be *not less* than 255 bytes (i.e., the character size cannot be
> less than 8 bits).
>
> The actual limit depends on the character size in the system.
>
> The "counted string" data type is already almost outdated, there only
> two standard words use them: "WORD" and "FIND".
>
> The "character string" data type takes two cells on a stack ( c-addr u )
> and it limits the string length by the cell size only.
>

We just need to add `PLACE` and say that actual counted string
presentation is system dependent.

There is even a proposal for that, which clearly states:

"Since then, in 2016 the Forth-200x committee in favour of eliminating ambiguous conditions has decided to require “1 CHARS = 1” thus making systems that have other character sizes than on not compliant to future Forth-200x standards [2][3]. Requesting standard systems to have byte size characters limit counted strings to the known maximal length of 255 characters."

On Saturday, November 6, 2021 at 2:10:59 PM UTC+3, Nickolay Kolchin wrote:
> On Saturday, November 6, 2021 at 1:49:42 PM UTC+3, Ruvim wrote:
> > On 2021-11-06 13:16, Nickolay Kolchin wrote:
> > > On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
> > >> On 2021-11-05 21:05, Nickolay Kolchin wrote:
> >
> > >> A standard Forth system may use only a character for
> > >> the length in a counted string.
> > >>
> > >> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
> > >> https://forth-standard.org/standard/core/COUNT
> > >>
> > >>> P.S. Back in "good old days" we distinguish between "counted" (CELL
> > >>> sized) and "packed" (byte size) strings.
> > >>
> > >> In the Forth-83 and Forth-79 standards the length in a counted string
> > >> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
> > >> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
> > >>
> > >
> > > Actually F79 used correct naming "packed" and F83 started using
> > > misleading term "counted".
> > As I can see, the term "packed" was used in only one glossary entry,
> > namely for the "WORD" word. It looks like just an omission.
> >
> > It's not obvious to me why the "packed string" term is better than
> > "counted string".
> > > IMHO, 255 bytes limitation is "blast from past" and should be lifted
> > > nowadays...
> > Forth-94 doesn't limit it by 255 bytes, it requires that this limit
> > shall be *not less* than 255 bytes (i.e., the character size cannot be
> > less than 8 bits).
> >
> > The actual limit depends on the character size in the system.
> >
> > The "counted string" data type is already almost outdated, there only
> > two standard words use them: "WORD" and "FIND".
> >
> > The "character string" data type takes two cells on a stack ( c-addr u )
> > and it limits the string length by the cell size only.
> >
> We just need to add `PLACE` and say that actual counted string
> presentation is system dependent.
>
> There is even a proposal for that, which clearly states:
>
> "Since then, in 2016 the Forth-200x committee in favour of eliminating ambiguous conditions has decided to require “1 CHARS = 1” thus making systems that have other character sizes than on not compliant to future Forth-200x standards [2][3]. Requesting standard systems to have byte size characters limit counted strings to the known maximal length of 255 characters."

I.e. these two words must give identical results:

: test-s S" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" NIP . ;

: test-c C" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" COUNT NIP . ;

test-s
test-c

Currently they cause hell among Forth implementations.

On 6/11/2021 22:29, Nickolay Kolchin wrote:
> On Saturday, November 6, 2021 at 2:10:59 PM UTC+3, Nickolay Kolchin wrote:
>> On Saturday, November 6, 2021 at 1:49:42 PM UTC+3, Ruvim wrote:
>> > On 2021-11-06 13:16, Nickolay Kolchin wrote:
>> > > On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
>> > >> On 2021-11-05 21:05, Nickolay Kolchin wrote:
>> >
>> > >> A standard Forth system may use only a character for
>> > >> the length in a counted string.
>> > >>
>> > >> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
>> > >> https://forth-standard.org/standard/core/COUNT
>> > >>
>> > >>> P.S. Back in "good old days" we distinguish between "counted" (CELL
>> > >>> sized) and "packed" (byte size) strings.
>> > >>
>> > >> In the Forth-83 and Forth-79 standards the length in a counted string
>> > >> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
>> > >> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
>> > >>
>> > >
>> > > Actually F79 used correct naming "packed" and F83 started using
>> > > misleading term "counted".
>> > As I can see, the term "packed" was used in only one glossary entry,
>> > namely for the "WORD" word. It looks like just an omission.
>> >
>> > It's not obvious to me why the "packed string" term is better than
>> > "counted string".
>> > > IMHO, 255 bytes limitation is "blast from past" and should be lifted
>> > > nowadays...
>> > Forth-94 doesn't limit it by 255 bytes, it requires that this limit
>> > shall be *not less* than 255 bytes (i.e., the character size cannot be
>> > less than 8 bits).
>> >
>> > The actual limit depends on the character size in the system.
>> >
>> > The "counted string" data type is already almost outdated, there only
>> > two standard words use them: "WORD" and "FIND".
>> >
>> > The "character string" data type takes two cells on a stack ( c-addr u )
>> > and it limits the string length by the cell size only.
>> >
>> We just need to add `PLACE` and say that actual counted string
>> presentation is system dependent.
>>
>> There is even a proposal for that, which clearly states:
>>
>> "Since then, in 2016 the Forth-200x committee in favour of eliminating ambiguous conditions has decided to require “1 CHARS = 1” thus making systems that have other character sizes than on not compliant to future Forth-200x standards [2][3]. Requesting standard systems to have byte size characters limit counted strings to the known maximal length of 255 characters."
>
> I.e. these two words must give identical results:
>
> : test-s S" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" NIP . ;
>
> : test-c C" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" COUNT NIP . ;
>
> test-s
> test-c
>
> Currently they cause hell among Forth implementations.

For all manner of 'implementation-defined' reasons e.g.

11.3.6 Parsing

Lines of at least 128 characters shall be supported. A program that requires
lines of more than 128 characters has an environmental dependency.

On Saturday, November 6, 2021 at 3:52:52 PM UTC+3, dxforth wrote:
> On 6/11/2021 22:29, Nickolay Kolchin wrote:
> > On Saturday, November 6, 2021 at 2:10:59 PM UTC+3, Nickolay Kolchin wrote:
> >> On Saturday, November 6, 2021 at 1:49:42 PM UTC+3, Ruvim wrote:
> >> > On 2021-11-06 13:16, Nickolay Kolchin wrote:
> >> > > On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
> >> > >> On 2021-11-05 21:05, Nickolay Kolchin wrote:
> >> >
> >> > >> A standard Forth system may use only a character for
> >> > >> the length in a counted string.
> >> > >>
> >> > >> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
> >> > >> https://forth-standard.org/standard/core/COUNT
> >> > >>
> >> > >>> P.S. Back in "good old days" we distinguish between "counted" (CELL
> >> > >>> sized) and "packed" (byte size) strings.
> >> > >>
> >> > >> In the Forth-83 and Forth-79 standards the length in a counted string
> >> > >> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
> >> > >> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
> >> > >>
> >> > >
> >> > > Actually F79 used correct naming "packed" and F83 started using
> >> > > misleading term "counted".
> >> > As I can see, the term "packed" was used in only one glossary entry,
> >> > namely for the "WORD" word. It looks like just an omission.
> >> >
> >> > It's not obvious to me why the "packed string" term is better than
> >> > "counted string".
> >> > > IMHO, 255 bytes limitation is "blast from past" and should be lifted
> >> > > nowadays...
> >> > Forth-94 doesn't limit it by 255 bytes, it requires that this limit
> >> > shall be *not less* than 255 bytes (i.e., the character size cannot be
> >> > less than 8 bits).
> >> >
> >> > The actual limit depends on the character size in the system.
> >> >
> >> > The "counted string" data type is already almost outdated, there only
> >> > two standard words use them: "WORD" and "FIND".
> >> >
> >> > The "character string" data type takes two cells on a stack ( c-addr u )
> >> > and it limits the string length by the cell size only.
> >> >
> >> We just need to add `PLACE` and say that actual counted string
> >> presentation is system dependent.
> >>
> >> There is even a proposal for that, which clearly states:
> >>
> >> "Since then, in 2016 the Forth-200x committee in favour of eliminating ambiguous conditions has decided to require “1 CHARS = 1” thus making systems that have other character sizes than on not compliant to future Forth-200x standards [2][3]. Requesting standard systems to have byte size characters limit counted strings to the known maximal length of 255 characters."
> >
> > I.e. these two words must give identical results:
> >
> > : test-s S" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" NIP . ;
> >
> > : test-c C" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" COUNT NIP . ;
> >
> > test-s
> > test-c
> >
> > Currently they cause hell among Forth implementations.
> For all manner of 'implementation-defined' reasons e.g.
>
> 11.3.6 Parsing
>
> Lines of at least 128 characters shall be supported. A program that requires
> lines of more than 128 characters has an environmental dependency.

1. You are citing that like holy Bible, which it is not.

2. There is no practical reason for line length limitation when we are reading
source from file on modern computers. This limitation makes perfect sense for
memory constrained environments or interactive input, but not for machine with
gigabytes of memory...

Now about "hell":

- lxf and gforth accept the source but silently truncate length. They should throw
exception if line is too big for `C"`.
- SwiftForth coredump, which probably tells a lot about it general quality.
- VFX also segfaults, but is able to continue running.
- kforth does nothing. No errors, no parsing...
- ikforth coredumps when executing `test-c`, but not `test-s`.
- SPF4 prints some cryptic message (probably cp1251), but probably handles
this case right.

On 7/11/2021 00:19, Nickolay Kolchin wrote:
> On Saturday, November 6, 2021 at 3:52:52 PM UTC+3, dxforth wrote:
>> On 6/11/2021 22:29, Nickolay Kolchin wrote:
>> > On Saturday, November 6, 2021 at 2:10:59 PM UTC+3, Nickolay Kolchin wrote:
>> >> On Saturday, November 6, 2021 at 1:49:42 PM UTC+3, Ruvim wrote:
>> >> > On 2021-11-06 13:16, Nickolay Kolchin wrote:
>> >> > > On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
>> >> > >> On 2021-11-05 21:05, Nickolay Kolchin wrote:
>> >> >
>> >> > >> A standard Forth system may use only a character for
>> >> > >> the length in a counted string.
>> >> > >>
>> >> > >> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
>> >> > >> https://forth-standard.org/standard/core/COUNT
>> >> > >>
>> >> > >>> P.S. Back in "good old days" we distinguish between "counted" (CELL
>> >> > >>> sized) and "packed" (byte size) strings.
>> >> > >>
>> >> > >> In the Forth-83 and Forth-79 standards the length in a counted string
>> >> > >> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
>> >> > >> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
>> >> > >>
>> >> > >
>> >> > > Actually F79 used correct naming "packed" and F83 started using
>> >> > > misleading term "counted".
>> >> > As I can see, the term "packed" was used in only one glossary entry,
>> >> > namely for the "WORD" word. It looks like just an omission.
>> >> >
>> >> > It's not obvious to me why the "packed string" term is better than
>> >> > "counted string".
>> >> > > IMHO, 255 bytes limitation is "blast from past" and should be lifted
>> >> > > nowadays...
>> >> > Forth-94 doesn't limit it by 255 bytes, it requires that this limit
>> >> > shall be *not less* than 255 bytes (i.e., the character size cannot be
>> >> > less than 8 bits).
>> >> >
>> >> > The actual limit depends on the character size in the system.
>> >> >
>> >> > The "counted string" data type is already almost outdated, there only
>> >> > two standard words use them: "WORD" and "FIND".
>> >> >
>> >> > The "character string" data type takes two cells on a stack ( c-addr u )
>> >> > and it limits the string length by the cell size only.
>> >> >
>> >> We just need to add `PLACE` and say that actual counted string
>> >> presentation is system dependent.
>> >>
>> >> There is even a proposal for that, which clearly states:
>> >>
>> >> "Since then, in 2016 the Forth-200x committee in favour of eliminating ambiguous conditions has decided to require “1 CHARS = 1” thus making systems that have other character sizes than on not compliant to future Forth-200x standards [2][3]. Requesting standard systems to have byte size characters limit counted strings to the known maximal length of 255 characters."
>> >
>> > I.e. these two words must give identical results:
>> >
>> > : test-s S" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" NIP . ;
>> >
>> > : test-c C" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" COUNT NIP . ;
>> >
>> > test-s
>> > test-c
>> >
>> > Currently they cause hell among Forth implementations.
>> For all manner of 'implementation-defined' reasons e.g.
>>
>> 11.3.6 Parsing
>>
>> Lines of at least 128 characters shall be supported. A program that requires
>> lines of more than 128 characters has an environmental dependency.
>
> 1. You are citing that like holy Bible, which it is not.
>
> 2. There is no practical reason for line length limitation when we are reading
> source from file on modern computers. This limitation makes perfect sense for
> memory constrained environments or interactive input, but not for machine with
> gigabytes of memory...
>
> Now about "hell":
>
> - lxf and gforth accept the source but silently truncate length. They should throw
> exception if line is too big for `C"`.
> - SwiftForth coredump, which probably tells a lot about it general quality.
> - VFX also segfaults, but is able to continue running.
> - kforth does nothing. No errors, no parsing...
> - ikforth coredumps when executing `test-c`, but not `test-s`.
> - SPF4 prints some cryptic message (probably cp1251), but probably handles
> this case right.
>

ANS/200x represents the limits of what those systems are required to do.
That not all implementers feel the need to fool-proof their systems is
their prerogative. Early Forth Inc systems had little to no compiler
security. It didn't stop programmers doing great work with them.

On Saturday, November 6, 2021 at 8:24:31 PM UTC+3, dxforth wrote:
> On 7/11/2021 00:19, Nickolay Kolchin wrote:
> > On Saturday, November 6, 2021 at 3:52:52 PM UTC+3, dxforth wrote:
> >> On 6/11/2021 22:29, Nickolay Kolchin wrote:
> >> > On Saturday, November 6, 2021 at 2:10:59 PM UTC+3, Nickolay Kolchin wrote:
> >> >> On Saturday, November 6, 2021 at 1:49:42 PM UTC+3, Ruvim wrote:
> >> >> > On 2021-11-06 13:16, Nickolay Kolchin wrote:
> >> >> > > On Saturday, November 6, 2021 at 12:53:33 PM UTC+3, Ruvim wrote:
> >> >> > >> On 2021-11-05 21:05, Nickolay Kolchin wrote:
> >> >> >
> >> >> > >> A standard Forth system may use only a character for
> >> >> > >> the length in a counted string.
> >> >> > >>
> >> >> > >> See 3.1.3.4 Counted strings, 6.1.0980 COUNT
> >> >> > >> https://forth-standard.org/standard/core/COUNT
> >> >> > >>
> >> >> > >>> P.S. Back in "good old days" we distinguish between "counted" (CELL
> >> >> > >>> sized) and "packed" (byte size) strings.
> >> >> > >>
> >> >> > >> In the Forth-83 and Forth-79 standards the length in a counted string
> >> >> > >> takes exactly one *byte* (8 bits). And the glossary entry for COUNT
> >> >> > >> said: "addr2 is addr1+1", where "COUNT ( addr1 -- addr2 +n )"
> >> >> > >>
> >> >> > >
> >> >> > > Actually F79 used correct naming "packed" and F83 started using
> >> >> > > misleading term "counted".
> >> >> > As I can see, the term "packed" was used in only one glossary entry,
> >> >> > namely for the "WORD" word. It looks like just an omission.
> >> >> >
> >> >> > It's not obvious to me why the "packed string" term is better than
> >> >> > "counted string".
> >> >> > > IMHO, 255 bytes limitation is "blast from past" and should be lifted
> >> >> > > nowadays...
> >> >> > Forth-94 doesn't limit it by 255 bytes, it requires that this limit
> >> >> > shall be *not less* than 255 bytes (i.e., the character size cannot be
> >> >> > less than 8 bits).
> >> >> >
> >> >> > The actual limit depends on the character size in the system.
> >> >> >
> >> >> > The "counted string" data type is already almost outdated, there only
> >> >> > two standard words use them: "WORD" and "FIND".
> >> >> >
> >> >> > The "character string" data type takes two cells on a stack ( c-addr u )
> >> >> > and it limits the string length by the cell size only.
> >> >> >
> >> >> We just need to add `PLACE` and say that actual counted string
> >> >> presentation is system dependent.
> >> >>
> >> >> There is even a proposal for that, which clearly states:
> >> >>
> >> >> "Since then, in 2016 the Forth-200x committee in favour of eliminating ambiguous conditions has decided to require “1 CHARS = 1” thus making systems that have other character sizes than on not compliant to future Forth-200x standards [2][3]. Requesting standard systems to have byte size characters limit counted strings to the known maximal length of 255 characters."
> >> >
> >> > I.e. these two words must give identical results:
> >> >
> >> > : test-s S" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" NIP . ;
> >> >
> >> > : test-c C" 0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" COUNT NIP . ;
> >> >
> >> > test-s
> >> > test-c
> >> >
> >> > Currently they cause hell among Forth implementations.
> >> For all manner of 'implementation-defined' reasons e.g.
> >>
> >> 11.3.6 Parsing
> >>
> >> Lines of at least 128 characters shall be supported. A program that requires
> >> lines of more than 128 characters has an environmental dependency.
> >
> > 1. You are citing that like holy Bible, which it is not.
> >
> > 2. There is no practical reason for line length limitation when we are reading
> > source from file on modern computers. This limitation makes perfect sense for
> > memory constrained environments or interactive input, but not for machine with
> > gigabytes of memory...
> >
> > Now about "hell":
> >
> > - lxf and gforth accept the source but silently truncate length. They should throw
> > exception if line is too big for `C"`.
> > - SwiftForth coredump, which probably tells a lot about it general quality.
> > - VFX also segfaults, but is able to continue running.
> > - kforth does nothing. No errors, no parsing...
> > - ikforth coredumps when executing `test-c`, but not `test-s`.
> > - SPF4 prints some cryptic message (probably cp1251), but probably handles
> > this case right.
> >
> ANS/200x represents the limits of what those systems are required to do.
> That not all implementers feel the need to fool-proof their systems is
> their prerogative. Early Forth Inc systems had little to no compiler
> security. It didn't stop programmers doing great work with them.

Nobody questions this. But we are living in a bit different world now.

On Saturday, November 6, 2021 at 2:19:12 PM UTC+1, Nickolay Kolchin wrote:
[..]
> - ikforth coredumps when executing `test-c`, but not `test-s`.

A new kid on the block!

-marcel

On 7/11/2021 04:59, Nickolay Kolchin wrote:
> On Saturday, November 6, 2021 at 8:24:31 PM UTC+3, dxforth wrote:
>> ...
>> ANS/200x represents the limits of what those systems are required to do.
>> That not all implementers feel the need to fool-proof their systems is
>> their prerogative. Early Forth Inc systems had little to no compiler
>> security. It didn't stop programmers doing great work with them.
>
> Nobody questions this. But we are living in a bit different world now.

Humans still only have two eyes and limited visual comprehension.
The less information presented them at once, the better. A computer
needs to read text only to the extent humans need to read it. When
a 255 char quoted string is already beyond human comprehension,
suggesting a computer could handle more becomes rather academic.

On Sunday, November 7, 2021 at 7:00:11 AM UTC+3, dxforth wrote:
> On 7/11/2021 04:59, Nickolay Kolchin wrote:
> > On Saturday, November 6, 2021 at 8:24:31 PM UTC+3, dxforth wrote:
> >> ...
> >> ANS/200x represents the limits of what those systems are required to do.
> >> That not all implementers feel the need to fool-proof their systems is
> >> their prerogative. Early Forth Inc systems had little to no compiler
> >> security. It didn't stop programmers doing great work with them.
> >
> > Nobody questions this. But we are living in a bit different world now.
> Humans still only have two eyes and limited visual comprehension.
> The less information presented them at once, the better. A computer
> needs to read text only to the extent humans need to read it. When
> a 255 char quoted string is already beyond human comprehension,
> suggesting a computer could handle more becomes rather academic.

Nah! You are just defending bad programming. Application should not
segfault on such input.

On 7/11/2021 15:48, Nickolay Kolchin wrote:
> On Sunday, November 7, 2021 at 7:00:11 AM UTC+3, dxforth wrote:
>> On 7/11/2021 04:59, Nickolay Kolchin wrote:
>> > On Saturday, November 6, 2021 at 8:24:31 PM UTC+3, dxforth wrote:
>> >> ...
>> >> ANS/200x represents the limits of what those systems are required to do.
>> >> That not all implementers feel the need to fool-proof their systems is
>> >> their prerogative. Early Forth Inc systems had little to no compiler
>> >> security. It didn't stop programmers doing great work with them.
>> >
>> > Nobody questions this. But we are living in a bit different world now.
>> Humans still only have two eyes and limited visual comprehension.
>> The less information presented them at once, the better. A computer
>> needs to read text only to the extent humans need to read it. When
>> a 255 char quoted string is already beyond human comprehension,
>> suggesting a computer could handle more becomes rather academic.
>
> Nah! You are just defending bad programming. Application should not
> segfault on such input.

When all else fails read the manual :)

On Sunday, November 7, 2021 at 5:00:11 AM UTC+1, dxforth wrote:
[..]
> Humans still only have two eyes and limited visual comprehension.
> The less information presented them at once, the better. A computer
> needs to read text only to the extent humans need to read it. When
> a 255 char quoted string is already beyond human comprehension,
> suggesting a computer could handle more becomes rather academic.

More (than 255 chars) is needed when cutting and pasting between
programs.

-marcel