On 5/04/2022 03:36, Hans Bezemer wrote:
> On Monday, April 4, 2022 at 5:14:52 PM UTC+2, Hans Bezemer wrote:
> I thought, "let's be nice". It took me a few minutes to port. Seems to run ok, but your mileage may vary:
>
> ---8<---
> \ 4tH library - SSCANF - Copyright 2022 J.L. Bezemer

Perhaps not everyone knows C well enough (am I the only one?) for a scanf to be
attractive. Nevertheless your factors would be familiar to anyone who has been
down the 'scan a numeric' path. For the latter audience

(putback) looks suspiciously like '-1 /string' ... which brings me to

(sign) ( a1 n1 -- f a2 n2)

I'm increasingly seeing forths which include this function variously called
SIGN? /SIGN etc. with a stack effect of ( a1 n1 -- a2 n2 f )

Your app may not need it but a generic /SIGN should probably include an empty
string check. From there it's trivial to define another handy function:

: /NUMBER ( addr u -- addr2 u2 d|ud )
/sign >r 0 0 2swap >number 2swap r> if dnegate then ;

which is a std function in my forth (along with /FLOAT )

A /SIGN that didn't check empty string could come unstuck with:

: >pad ( a u -- a' u ) tuck pad swap cmove pad swap ;

s" -1234" >pad 2drop
s" " >pad /sign .s 4749724 -1 -1

On Tuesday, April 5, 2022 at 5:15:23 AM UTC+2, dxforth wrote:
> Perhaps not everyone knows C well enough (am I the only one?) for a scanf to be
> attractive. Nevertheless your factors would be familiar to anyone who has been
> down the 'scan a numeric' path.
Being intimately familiar with your numeric functions (they are part of 4tH's FP suites)
I can concur here. And my thanks for that. BTW, C/STRING should also be familiar to you ;-)
> (putback) looks suspiciously like '-1 /string' ... which brings me to
It is. But I pulled this trick from 4tH's uBasic/4tH interpreter - which also features this
"quick peek & rollback" function.

> Your app may not need it but a generic /SIGN should probably include an empty
> string check. From there it's trivial to define another handy function:
In 4tH practically every string is "guarded" - which is a consequence of using ASCIIZ strings.
If you pull this trick you simply get the NULL byte back. In vanilla Forth this might be a problem.
Do you mean I should "wrap" the (SIGN) function into:

DUP IF <previous sign> ELSE FALSE THEN

It's not rocket science, but I want to know if that's what you mean. It's quite easy to do in %d
and would add an extra check.

> : /NUMBER ( addr u -- addr2 u2 d|ud )
> /sign >r 0 0 2swap >number 2swap r> if dnegate then ;
> which is a std function in my forth (along with /FLOAT )
That's basically what's done in the handling of "%d". Others, like %u, %o, %x do unsigned numbers.

> A /SIGN that didn't check empty string could come unstuck with:
>
> : >pad ( a u -- a' u ) tuck pad swap cmove pad swap ;
>
> s" -1234" >pad 2drop
> s" " >pad /sign .s 4749724 -1 -1

I don't quite know what you mean by there, but I lifted the routine from FPIN and this is what I got:
- You FIRST do the "sign flag";
- Then you determine whether a 1 /STRING should be executed.

So I think this will do the job:

[char] d of \ 'd' requires a sign
dup if
(sign) ['] (dec) (number) 2>r swap if negate then (assign)
else 2>r true then endof

This will even signal it as an error (which basically, it is). You could even argue that ANY attempt
(apart from %s) to extract anything from an empty string is futile - and hence: an error.

I'm quite interested what you think of this line of thought.

Hans Bezemer

On Tuesday, April 5, 2022 at 5:15:23 AM UTC+2, dxforth wrote:
I'm thinking quite slow - so sometimes some thought come afterwards.

Of course, I've done quite some testing. There are basically two ways SSCANF stops:
- Either all there has to be parsed, has been parsed;
- Or there is some error along the way.

However - at the very end BOTH these conditions may occur. Now, the basic question is:
Do you want a "permissive" or "strict" SSCANF. There is something to be said for both.

Now let's apply it to the real world, where data is flawed and work has got to be done. That
would speak for a more permissive approach. You don't want this thing to stop two dozen times
for a trivial error when doing a run with thousands of records.

On the other hand.. ;-)

Hans Bezemer

On Tuesday, April 5, 2022 at 5:15:23 AM UTC+2, dxforth wrote:
Obviously - I wrote that one too ;-)

IMHO, it has a lot more sane behavior:

%15c%d
012345678901234

Unparsed format:
Number of data read: 1
Stack bleed: 1

Contents
b = -.554405041/23002/0/
c$ = 012345678901234

%15c%d
01234567890

Unparsed format:
Number of data read: 1
Stack bleed: 1

Contents
b = -.554405041/23002/0/
c$ = 01234567890

%d%d
01234567890

Unparsed format:
Number of data read: 1
Stack bleed: 1

Contents
a = -.554405041/23002/0/
b = 1234567890

%d,%d
01234567890

Unparsed format: %d
Number of data read: 1
Stack bleed: 1

Hans Bezemer

On Tuesday, April 5, 2022 at 5:15:23 AM UTC+2, dxforth wrote:
Updated Forth version - if some of the words elude you: https://sourceforge.net/p/forth-4th/code/HEAD/tree/trunk/4th.src/lib/easy.4th

BTW, I've already converted the test program as well. Lemme know if there's any interest. It's basic, but it works.

---8<---
\ 4tH library - SSCANF - Copyright 2022 J.L. Bezemer
\ You can redistribute this file and/or modify it under
\ the terms of the GNU General Public License

\ Like sscanf(), but with a few differences
\ - Whitespace in both buffer and format string is largely ignored;
\ - %c with a width REQUIRES a string, %c without REQUIRES a variable;
\ - %s will stop on ANY defined delimiter, not just whitespace;
\ - Returns the unscanned part of the format string;
\ - Variable #SCANF returns the number of assignments made;
\ - Stack diagram of variables is inverted: d c b a s" %a %b %c %d";
\ - On failure, unused variables WILL remain on the stack.

\ Typical use:
\ a b c$ d$ e f g
\ s" %c%c %d%%%s, %4c%c%c" s" ab -12345%This is the end, 543210" sscanf

s" MAX-N" environment? \ query environment
[IF] \ if successful
constant max-n \ create constant MAX-N
[ELSE]
..( Warning: MAX-N undefined) cr
[THEN]

variable (delim) \ delimiter of string
variable (width) \ width of string
variable #scanf \ number of assignments
\ ANS Forth interface
: char- 1 chars - ; ( a -- a-1)
: >zero dup xor ; ( n -- 0)
: ;then postpone exit postpone then ; immediate
: unless postpone 0= postpone if ; immediate
: c/string over >r 1- swap char+ swap r> c@ ; ( a n -- a n-1 c)
\ a few execution tokens
: (dec) decimal ; : (hex) hex ; : (oct) 8 base ! ;
: (putback) 1+ swap char- swap ; ( a n -- a-1 n+1)
: (unumber) 0. 2swap >number 2>r d>s 2r> ; ( a1 n1 -- n3 a2 n2)
: (number) base @ >r execute (unumber) r> base ! ;
: (width!) ['] (dec) (number) rot dup unless max-n + then (width) ! ;
: (delimiter!) dup if c/string >r (putback) r> else dup then (delim) ! ;
: (delimiter?) over [char] ! - 0< over bl = and >r = r> or ;
: (assigned) 1 #scanf +! false ; ( -- f)
: (assign) swap ! (assigned) ; ( x n -- f)
: (place) rot place (assigned) ; ( a1 a2 n2 -- f)
\ get sign flag
: (sign) ( a1 n1 -- f a2 n2)
c/string dup [char] - = dup >r \ is it a minus sign?
if drop else [char] + <> if (putback) then then r> -rot
; \ drop plus sign
\ skip white space
: (skipwhite) ( a1 n1 -- a2 n2)
begin dup while c/string bl > dup >r if (putback) then r> until then
; \ parse buffer string
: (getstr) ( a1 n1 -- a2 n2 f)
over >r begin \ save starting address
dup \ any string left?
while \ if so, still within width?
over r@ - (width) @ < \ did we hit the delimiter?
while \ if so, put back the character
c/string (delim) @ (delimiter?) dup if >r (putback) r> then
until then then over r@ - r> swap \ calculate string dimensions
2swap 2>r (place) 2r> rot \ place string and get flag
;

: (getitem) ( a1 n1 c -- a2 n2 f)
case \ select type specifier
[char] d of \ 'd' requires a sign
(sign) ['] (dec) (number) 2>r swap if negate then (assign) endof
[char] u of ['] (dec) (number) 2>r (assign) endof
[char] x of ['] (hex) (number) 2>r (assign) endof
[char] o of ['] (oct) (number) 2>r (assign) endof
[char] % of c/string [char] % <> -rot 2>r endof
[char] c of \ if default width specified
(width) @ max-n = if \ just parse a single character
c/string -rot 2>r (assign) \ if a width has been specified
else \ take the entire length specified
2dup (width) @ /string 0 max 2>r (width) @ min (place)
then endof \ and put it in a string
endcase 2r> rot
; \ handle type specifiers
: (gettype) ( a1 n1 a2 n2 -- f)
2>r (width!) c/string >r (delimiter!) r> -rot
2r> 2swap 2>r (skipwhite) rot dup [char] s =
if drop (getstr) else over if (getitem) else 0<> then then
2r> rot >r 2swap r> \ restore stack and pull up flag
; \ select character on format string
: (select) ( a1 n1 c a2 n2 -- a3 n3 a4 n4 f)
rot dup >r \ save current character
[char] % = if rdrop (gettype) ;then \ act on type specifier
r@ bl = if (skipwhite) 2swap (skipwhite) 2swap r> >zero ;then
c/string r> <> \ does this character match
;

: sscanf ( xn .. x0 a1 n1 a2 n2 -- a3 n3)
0 #scanf ! 2>r begin dup while c/string 2r> (select) -rot 2>r until then 2rdrop
; ---8<---

Hans Bezemer

On 5/04/2022 20:05, Hans Bezemer wrote:
> On Tuesday, April 5, 2022 at 5:15:23 AM UTC+2, dxforth wrote:
>
>> A /SIGN that didn't check empty string could come unstuck with:
>>
>> : >pad ( a u -- a' u ) tuck pad swap cmove pad swap ;
>>
>> s" -1234" >pad 2drop
>> s" " >pad /sign .s 4749724 -1 -1
>
> I don't quite know what you mean by there, but I lifted the routine from FPIN and this
> is what I got:
> - You FIRST do the "sign flag";
> - Then you determine whether a 1 /STRING should be executed.

I had to look at it twice to see that it handled the empty string case.
In retrospect I wouldn't write it that way today given it potentially
accesses a character that's out of bounds. A more robust definition
would be:

: /SIGN ( a u -- a' u' f )
dup if
over c@ dup [char] + =
swap [char] - = dup >r or
negate /string r> exit
then 0 ;

>
> So I think this will do the job:
>
> [char] d of \ 'd' requires a sign
> dup if
> (sign) ['] (dec) (number) 2>r swap if negate then (assign)
> else 2>r true then endof
>
> This will even signal it as an error (which basically, it is). You could even argue that ANY attempt
> (apart from %s) to extract anything from an empty string is futile - and hence: an error.
>
> I'm quite interested what you think of this line of thought.

Can't speak for sscanf but the default for /NUMBER is to return numeric 0
for an empty string. I would prefer the app decide whether it's acceptable
or not, rather than the routine.

On Wednesday, April 6, 2022 at 2:39:34 AM UTC+2, dxforth wrote:
> I had to look at it twice to see that it handled the empty string case.
Frankly, I had to put it into an editor and write out the stack diagram for every few words
to interpret what was going on. But sometimes I have to do that for my own programs
as well, so I didn't see any point to it mentioning that at first. ;-)

> Can't speak for sscanf but the default for /NUMBER is to return numeric 0
> for an empty string. I would prefer the app decide whether it's acceptable
> or not, rather than the routine.
Well, as usual I delved into the standard to see what was written there.

"3.2.1.2 Digit conversion
<blabla>
The value in BASE is the radix for number conversion. A digit has a value ranging from zero to one
less than the contents of BASE. The digit with the value zero corresponds to the character 0.
This representation of digits proceeds through the character set to the decimal value nine
corresponding to the character 9. For digits beginning with the decimal value ten the graphic
characters beginning with the character A are used. This correspondence continues up to and
including the digit with the decimal value thirty-five which is represented by the character Z".

=> The conversion of digits outside this range is implementation defined. <=
I don't quite know what to make of this - but it probably concerns stuff outside radix 2 to 36.

However - NOWHERE is mentioned what to do with either an empty string - or a string of blanks
(>FLOAT does, however - but that's outside this issue). On the other hand, the behavior of an empty
string to >NUMBER in Gforth IMPLIES that such conversion is valid. Since there is NO way to
check that such a case is rejected (you'd also expect an empty string when a single S" 0" was fed
to it). Spaces are clearly rejected though.

I'm not gonna quote the standard on >NUMBER again - it's simply ignored there. Overflow is mentioned
as a possible issue, but that's it.

So the standard doesn't respond to the question whether empty strings are valid numbers. I'm afraid
that puts us in a stalemate.

Personally, if a string is empty I don't like the idea of "inventing" numbers when they're obviously not
there. That doesn't mean that there are no "use cases" where this could apply. I've seen enough corporate
data to agree with you on that issue.

That having said - my 4tH compilers "NUMBER" fails on an empty string as well. Which means I've been quite
consistent in the last 30 years.

Of course, feel free to plug your /SIGN into any SSCANF version and use it as you like. But given the
alternative, I tend to like the consistent behavior of this more recent version of SSCANF - and I thank you
for the suggestions you made to make me reconsider - although it might not have worked out in the
way that I verbatim accepted and implemented them.

Hans Bezemer

Hans Bezemer <the.beez.speaks@gmail.com> writes:

>However - NOWHERE is mentioned what to do with either an empty string
>- or a string of blanks (>FLOAT does, however - but that's outside
>this issue). On the other hand, the behavior of an empty string to
>>NUMBER in Gforth IMPLIES that such conversion is valid.

The word

>NUMBER ( ud1 c-addr1 u1 -- ud2 c-addr2 u2 )

is pretty low-level. It just converts as many digits as it finds and
then stops. How to use that is up to you. E.g., if you only want to
accept a string that only contains digits, you check if u2=0; if you
don't want to accept an empty string, you check if u1=0; if you want
to process a non-zero number of digits that may be followed by
something else, you check if u1>u2.

>I'm not gonna quote the standard on >NUMBER again - it's simply
>ignored there. Overflow is mentioned as a possible issue, but that's
>it.
>
>So the standard doesn't respond to the question whether empty strings
>are valid numbers.

As I see it, it clearly does. If u1=0, ud2=ud1. If the first
character of c-addr1 u1 is a non-digit, ud2=ud1.

If you want something more high-level than >NUMBER, Gforth currently
offers REC-NUM:

s" -123" rec-num ... clearstack \ <2> #-123 `recognized-num ok
s" $123." rec-num ... clearstack \ <3> #291 #0 `recognized-dnum ok
s" $123x" rec-num ... clearstack \ <1> `notfound ok

These examples demonstrate a number of things that >NUMBER (by itself)
lacks:

* support for negative numbers

* support for number prefixes

* support for doubles

I reformatted your overly long lines.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2021: https://euro.theforth.net/2021

On Wednesday, April 6, 2022 at 2:05:57 PM UTC+2, Anton Ertl wrote:
> >NUMBER ( ud1 c-addr1 u1 -- ud2 c-addr2 u2 )
>
> is pretty low-level. It just converts as many digits as it finds and
> then stops. How to use that is up to you. E.g., if you only want to
> accept a string that only contains digits, you check if u2=0; if you
> don't want to accept an empty string, you check if u1=0; if you want
> to process a non-zero number of digits that may be followed by
> something else, you check if u1>u2.

I see where you're coming from - and I might even agree with you.
That having said, I would expect such an explanation in the Rationale -
if only to eradicate any doubt - since the reference to "digit conversion"
doesn't cut it. At least not completely - it leaves room for interpretation
IMHO.

BTW, it was not intended as any criticism towards Gforth. I just happens
to be that Gforth was positioned as a "reference implementation" and -
as I've stated numerous times - I tend to use it as such to test conforming
behavior. As such was the case here.

As for your reference to REC-NUM - it's not part of the ANS-94 standard.
I won't doubt its usefulness - 4tH's got a variant called "NUMBER" which
roughly equals the same basic function - but IMHO it can't be regarded
as a reference point in this discussion. And given that criteria - neither
can "NUMBER" (which clearly rejects an empty string).

Hans Bezemer

On Tuesday, April 5, 2022 at 7:39:34 PM UTC-5, dxforth wrote:

> Can't speak for sscanf but the default for /NUMBER is to return numeric 0
> for an empty string. I would prefer the app decide whether it's acceptable
> or not, rather than the routine.

Just throw. If the user needs a value for the empty case, he can catch and supply it.
Isn't that the purpose of catch/throw?
--
me

On Wednesday, April 6, 2022 at 2:41:15 PM UTC+2, S Jack wrote:
> On Tuesday, April 5, 2022 at 7:39:34 PM UTC-5, dxforth wrote:
>
> > Can't speak for sscanf but the default for /NUMBER is to return numeric 0
> > for an empty string. I would prefer the app decide whether it's acceptable
> > or not, rather than the routine.
> Just throw. If the user needs a value for the empty case, he can catch and supply it.
> Isn't that the purpose of catch/throw?
In my routine you wouldn't even have to CATCH|THROW. Like sscanf(), you can extract the number
of values that would have been assigned and consume the left over variables yourself - that's especially
easy when they're all numerical values:

#expected #sscanf - 0 ?do 0 swap ! loop

HB

S Jack schrieb am Mittwoch, 6. April 2022 um 14:41:15 UTC+2:
> On Tuesday, April 5, 2022 at 7:39:34 PM UTC-5, dxforth wrote:
>
> > Can't speak for sscanf but the default for /NUMBER is to return numeric 0
> > for an empty string. I would prefer the app decide whether it's acceptable
> > or not, rather than the routine.
> Just throw. If the user needs a value for the empty case, he can catch and supply it.
> Isn't that the purpose of catch/throw?

The only issue I ever had with >NUMBER was overflow of the ud accumulator.
It had been a pita to debug. Now my >NUMBER throws an exception on overflow.

This is one example where the standard wrap-around behavior of integer math
in Forth is poor.

On 6/04/2022 21:23, Hans Bezemer wrote:
> ...
> I'm not gonna quote the standard on >NUMBER again - it's simply ignored there.

But >NUMBER isn't a complete str-to-int routine - only a parser. I think the
spec is clear what happens when passed an empty string - it's a NOP.

> So the standard doesn't respond to the question whether empty strings are valid numbers.
> I'm afraid that puts us in a stalemate.

Perhaps because unlike >FLOAT, ANS never provided an equivalent routine for ints -
only a primitive for making one.

> Personally, if a string is empty I don't like the idea of "inventing" numbers when they're obviously not
> there. That doesn't mean that there are no "use cases" where this could apply. I've seen enough corporate
> data to agree with you on that issue.
>
> That having said - my 4tH compilers "NUMBER" fails on an empty string as well. Which means I've been quite
> consistent in the last 30 years.

IIRC C provides several such routines each having their own behaviour.

On 6/04/2022 23:43, minf...@arcor.de wrote:
> S Jack schrieb am Mittwoch, 6. April 2022 um 14:41:15 UTC+2:
>> On Tuesday, April 5, 2022 at 7:39:34 PM UTC-5, dxforth wrote:
>>
>> > Can't speak for sscanf but the default for /NUMBER is to return numeric 0
>> > for an empty string. I would prefer the app decide whether it's acceptable
>> > or not, rather than the routine.
>> Just throw. If the user needs a value for the empty case, he can catch and supply it.
>> Isn't that the purpose of catch/throw?
>
> The only issue I ever had with >NUMBER was overflow of the ud accumulator.
> It had been a pita to debug. Now my >NUMBER throws an exception on overflow.
>
> This is one example where the standard wrap-around behavior of integer math
> in Forth is poor.

Just like the [Turbo] C compiler :)

On Wednesday, April 6, 2022 at 4:14:31 PM UTC+2, dxforth wrote:
> > This is one example where the standard wrap-around behavior of integer math
> > in Forth is poor.
> Just like the [Turbo] C compiler :)

I don't mind "wrap around" behavior. Sometimes that just is what the doctor ordered.
Of all the "bit tricks" that I pulled I wouldn't want to see the behavior of VBA, where
every overflow results in "BEEP - I halt your program right there".

BTW, the standard says "An ambiguous condition exists if ud2 overflows during the conversion".
So, you may pretty much do as you see fit.

Yes, >NUMBER is seriously lacking in some respects - and as usual the required functionality
is lacking from the standard - leaving it to the likes of us to come up with our highly different
solutions to the problem. Oh, I really love portability.

Hans Bezemer

On 6/04/2022 22:41, S Jack wrote:
> On Tuesday, April 5, 2022 at 7:39:34 PM UTC-5, dxforth wrote:
>
>> Can't speak for sscanf but the default for /NUMBER is to return numeric 0
>> for an empty string. I would prefer the app decide whether it's acceptable
>> or not, rather than the routine.
>
> Just throw. If the user needs a value for the empty case, he can catch and supply it.
> Isn't that the purpose of catch/throw?

catch/throw is a solution of last resort

On 7/04/2022 00:42, Hans Bezemer wrote:
> On Wednesday, April 6, 2022 at 4:14:31 PM UTC+2, dxforth wrote:
>> > This is one example where the standard wrap-around behavior of integer math
>> > in Forth is poor.
>> Just like the [Turbo] C compiler :)
>
> I don't mind "wrap around" behavior. Sometimes that just is what the doctor ordered.
> Of all the "bit tricks" that I pulled I wouldn't want to see the behavior of VBA, where
> every overflow results in "BEEP - I halt your program right there".
>
> BTW, the standard says "An ambiguous condition exists if ud2 overflows during the conversion".
> So, you may pretty much do as you see fit.
>
> Yes, >NUMBER is seriously lacking in some respects - and as usual the required functionality
> is lacking from the standard - leaving it to the likes of us to come up with our highly different
> solutions to the problem. Oh, I really love portability.

Standard >NUMBER works with doubles; so it takes some effort on the part of
the user to overflow even on a 16-bit system. For all but safety-critical apps
it probably suffices and 'garbage in - garbage out' applies.

What is problematic is each forth's 'NUMBER' routine as each implementer seems
to have their own idea what constitutes a 'valid' integer. Nor do they
necessarily document these rules. What's for sure is the more strict 'NUMBER'
is, the less flexible and harder to apply it becomes. When it I wanted to
implement comma-separated values for a command-line app I balked at the prospect
of having to create yet another parser and more workarounds just so I could use
my glorious and fool-proof NUMBER? That's when I added /NUMBER which is as
simple as it gets. And the funny thing is, I could probably use it where I
previously used NUMBER? and the user would scarcely know.

That said, I'm not here to promote /NUMBER. What I do think every forth
should have is /SIGN - not least because they already do - in one form or
another. It simply hasn't been standardized - which is a pity given it's
just as fundamental and useful as >NUMBER .