Hi colorForthers,

Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.

Today I wrote the code to do this, as a demonstration, and it works!!!

You can see the demonstration here :
http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf

I've been trying to do this for about 15 years, so I am very excited
that the idea just came to me yesterday.

In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
colour in the bottom four bits, and 28 bits of up to 7 letters.
For example, a white lower case comment "rrrrrrr" is
$11111119 , where '1' is a four bit code for the letter 'r', and '9' is
a white colour token. A sequence of Shannon-Fano letters can be
terminated by a 4-bit Shannon-Fano NULL, which displays as a space
character.

The simple idea is that if the first Shannon-Fano encoded letter is a
NULL ( all four bits are 0 ) the token does not display anything (except
a space), so that in this case the remaining 24 bits can be interpreted
as three 8 bit ASCII/UTF-8 characters.

Now all I need to do is add ASCII/UTF-8 input, and support for
international QWERTZ, AZERTY and QWERTY etc. keyboards...

Watch this space :-)

Cheers
Howerd (very excited)

On Thursday, April 21, 2022 at 6:15:58 AM UTC+2, Howerd wrote:
> Hi colorForthers,
>
> Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
>
> Today I wrote the code to do this, as a demonstration, and it works!!!
>
> You can see the demonstration here :
> http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
>
> I've been trying to do this for about 15 years, so I am very excited
> that the idea just came to me yesterday.
>
> In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
> colour in the bottom four bits, and 28 bits of up to 7 letters.
> For example, a white lower case comment "rrrrrrr" is
> $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
> a white colour token. A sequence of Shannon-Fano letters can be
> terminated by a 4-bit Shannon-Fano NULL, which displays as a space
> character.
>
> The simple idea is that if the first Shannon-Fano encoded letter is a
> NULL ( all four bits are 0 ) the token does not display anything (except
> a space), so that in this case the remaining 24 bits can be interpreted
> as three 8 bit ASCII/UTF-8 characters.
>
> Now all I need to do is add ASCII/UTF-8 input, and support for
> international QWERTZ, AZERTY and QWERTY etc. keyboards...
>
> Watch this space :-)
>
> Cheers
> Howerd (very excited)

The link does not work.

-marcel

On Thursday, 21 April 2022 at 17:58:22 UTC+1, Marcel Hendrix wrote:
> On Thursday, April 21, 2022 at 6:15:58 AM UTC+2, Howerd wrote:
> > Hi colorForthers,
> >
> > Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
> >
> > Today I wrote the code to do this, as a demonstration, and it works!!!
> >
> > You can see the demonstration here :
> > http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
> >
> > I've been trying to do this for about 15 years, so I am very excited
> > that the idea just came to me yesterday.
> >
> > In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
> > colour in the bottom four bits, and 28 bits of up to 7 letters.
> > For example, a white lower case comment "rrrrrrr" is
> > $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
> > a white colour token. A sequence of Shannon-Fano letters can be
> > terminated by a 4-bit Shannon-Fano NULL, which displays as a space
> > character.
> >
> > The simple idea is that if the first Shannon-Fano encoded letter is a
> > NULL ( all four bits are 0 ) the token does not display anything (except
> > a space), so that in this case the remaining 24 bits can be interpreted
> > as three 8 bit ASCII/UTF-8 characters.
> >
> > Now all I need to do is add ASCII/UTF-8 input, and support for
> > international QWERTZ, AZERTY and QWERTY etc. keyboards...
> >
> > Watch this space :-)
> >
> > Cheers
> > Howerd (very excited)
> The link does not work.
>
> -marcel

The link definitely works in the UK. I just looked at it.
For now I downloaded it and put it in a dropbox
https://www.dropbox.com/sh/3t57jpblkpnirr3/AABOuY7bLbnzSQa_RQHr5Ixja?dl=0

On Thursday, 21 April 2022 at 18:11:35 UTC+1, Jurgen Pitaske wrote:
> On Thursday, 21 April 2022 at 17:58:22 UTC+1, Marcel Hendrix wrote:
> > On Thursday, April 21, 2022 at 6:15:58 AM UTC+2, Howerd wrote:
> > > Hi colorForthers,
> > >
> > > Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
> > >
> > > Today I wrote the code to do this, as a demonstration, and it works!!!
> > >
> > > You can see the demonstration here :
> > > http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
> > >
> > > I've been trying to do this for about 15 years, so I am very excited
> > > that the idea just came to me yesterday.
> > >
> > > In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
> > > colour in the bottom four bits, and 28 bits of up to 7 letters.
> > > For example, a white lower case comment "rrrrrrr" is
> > > $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
> > > a white colour token. A sequence of Shannon-Fano letters can be
> > > terminated by a 4-bit Shannon-Fano NULL, which displays as a space
> > > character.
> > >
> > > The simple idea is that if the first Shannon-Fano encoded letter is a
> > > NULL ( all four bits are 0 ) the token does not display anything (except
> > > a space), so that in this case the remaining 24 bits can be interpreted
> > > as three 8 bit ASCII/UTF-8 characters.
> > >
> > > Now all I need to do is add ASCII/UTF-8 input, and support for
> > > international QWERTZ, AZERTY and QWERTY etc. keyboards...
> > >
> > > Watch this space :-)
> > >
> > > Cheers
> > > Howerd (very excited)
> > The link does not work.
> >
> > -marcel
> The link definitely works in the UK. I just looked at it.
> For now I downloaded it and put it in a dropbox
> https://www.dropbox.com/sh/3t57jpblkpnirr3/AABOuY7bLbnzSQa_RQHr5Ixja?dl=0

Howerd's website says not secure - this might be the issue ...

Am 21/04/2022 um 18:58 schrieb Marcel Hendrix:
> On Thursday, April 21, 2022 at 6:15:58 AM UTC+2, Howerd wrote:
>> Hi colorForthers,
>>
>> Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
>>
>> Today I wrote the code to do this, as a demonstration, and it works!!!
>>
>> You can see the demonstration here :
>> http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
>>
>> I've been trying to do this for about 15 years, so I am very excited
>> that the idea just came to me yesterday.
>>
>> In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
>> colour in the bottom four bits, and 28 bits of up to 7 letters.
>> For example, a white lower case comment "rrrrrrr" is
>> $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
>> a white colour token. A sequence of Shannon-Fano letters can be
>> terminated by a 4-bit Shannon-Fano NULL, which displays as a space
>> character.
>>
>> The simple idea is that if the first Shannon-Fano encoded letter is a
>> NULL ( all four bits are 0 ) the token does not display anything (except
>> a space), so that in this case the remaining 24 bits can be interpreted
>> as three 8 bit ASCII/UTF-8 characters.
>>
>> Now all I need to do is add ASCII/UTF-8 input, and support for
>> international QWERTZ, AZERTY and QWERTY etc. keyboards...
>>
>> Watch this space :-)
>>
>> Cheers
>> Howerd (very excited)
>
> The link does not work.
>
> -marcel
Hi Marcel,

It could be that it needs https:
https://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf

Or maybe funny line endings?

Cheers,
Howerd

Am 21/04/2022 um 20:24 schrieb Jurgen Pitaske:
> On Thursday, 21 April 2022 at 18:11:35 UTC+1, Jurgen Pitaske wrote:
>> On Thursday, 21 April 2022 at 17:58:22 UTC+1, Marcel Hendrix wrote:
>>> On Thursday, April 21, 2022 at 6:15:58 AM UTC+2, Howerd wrote:
>>>> Hi colorForthers,
>>>>
>>>> Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
>>>>
>>>> Today I wrote the code to do this, as a demonstration, and it works!!!
>>>>
>>>> You can see the demonstration here :
>>>> http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
>>>>
>>>> I've been trying to do this for about 15 years, so I am very excited
>>>> that the idea just came to me yesterday.
>>>>
>>>> In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
>>>> colour in the bottom four bits, and 28 bits of up to 7 letters.
>>>> For example, a white lower case comment "rrrrrrr" is
>>>> $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
>>>> a white colour token. A sequence of Shannon-Fano letters can be
>>>> terminated by a 4-bit Shannon-Fano NULL, which displays as a space
>>>> character.
>>>>
>>>> The simple idea is that if the first Shannon-Fano encoded letter is a
>>>> NULL ( all four bits are 0 ) the token does not display anything (except
>>>> a space), so that in this case the remaining 24 bits can be interpreted
>>>> as three 8 bit ASCII/UTF-8 characters.
>>>>
>>>> Now all I need to do is add ASCII/UTF-8 input, and support for
>>>> international QWERTZ, AZERTY and QWERTY etc. keyboards...
>>>>
>>>> Watch this space :-)
>>>>
>>>> Cheers
>>>> Howerd (very excited)
>>> The link does not work.
>>>
>>> -marcel
>> The link definitely works in the UK. I just looked at it.
>> For now I downloaded it and put it in a dropbox
>> https://www.dropbox.com/sh/3t57jpblkpnirr3/AABOuY7bLbnzSQa_RQHr5Ixja?dl=0
>
> Howerd's website says not secure - this might be the issue ...
Hi Jurgen,

Yes, the link says http , I think it needs to be https :

https://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf

Your Dropbox links for me - thanks :-)

Cheers,
Howerd

On Thursday, April 21, 2022 at 10:33:38 PM UTC+2, Howerd wrote:
> Am 21/04/2022 um 18:58 schrieb Marcel Hendrix:
> > On Thursday, April 21, 2022 at 6:15:58 AM UTC+2, Howerd wrote:
> >> Hi colorForthers,
> >>
> >> Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
> >>
> >> Today I wrote the code to do this, as a demonstration, and it works!!!
> >>
> >> You can see the demonstration here :
> >> http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
> >>
> >> I've been trying to do this for about 15 years, so I am very excited
> >> that the idea just came to me yesterday.
> >>
> >> In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
> >> colour in the bottom four bits, and 28 bits of up to 7 letters.
> >> For example, a white lower case comment "rrrrrrr" is
> >> $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
> >> a white colour token. A sequence of Shannon-Fano letters can be
> >> terminated by a 4-bit Shannon-Fano NULL, which displays as a space
> >> character.
> >>
> >> The simple idea is that if the first Shannon-Fano encoded letter is a
> >> NULL ( all four bits are 0 ) the token does not display anything (except
> >> a space), so that in this case the remaining 24 bits can be interpreted
> >> as three 8 bit ASCII/UTF-8 characters.
> >>
> >> Now all I need to do is add ASCII/UTF-8 input, and support for
> >> international QWERTZ, AZERTY and QWERTY etc. keyboards...
> >>
> >> Watch this space :-)
> >>
> >> Cheers
> >> Howerd (very excited)
> >
> > The link does not work.
> >
> > -marcel
> Hi Marcel,
>
> It could be that it needs https:
> https://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
>
> Or maybe funny line endings?
>
> Cheers,
> Howerd

The https. It works now.

-marcel

On Thu, 21 Apr 2022 06:15:54 +0200
Howerd Oakford <howerd@inventio.co.uk> wrote:

> Hi colorForthers,
>
> Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
>
> Today I wrote the code to do this, as a demonstration, and it works!!!
>
> You can see the demonstration here :
> http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
>
On that web page (which worked fine for my old insecure computer) you post some nasm code:

<Excerpt>

lowercase: ; display a white text word in normal lower-case letters
call white
showSF_EDI_: ; ( -- ) \ display a Shanon-Fano encoded token pointed to by edi in the current colour
_DUP_
mov _TOS_, [ ( edi * 4 ) - 0x04 ] ; fetch the next token – drops through to showShannonFano
showShannonFano: ; ( token -- ) \ display the Shannon-Fano encoded token on TOS
; ASCII / UTF8 support. If the first Shannon-Fano encoded letter is a 4 bit NULL,
; display the next 24 bits as three ASCII characters.
mov _SCRATCH_, _TOS_ ; save the token value
and _SCRATCH_, 0xF0000000
;;;;;; cmp _SCRATCH_, 0x00000000
jnz .forward
; display as three ASCII characters

</Excerpt>

I've commented out the 'cmp' test - the ZR flag is set/unset by the preceding 'and'.
HTH!

> I've been trying to do this for about 15 years, so I am very excited
> that the idea just came to me yesterday.
>
> In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
> colour in the bottom four bits, and 28 bits of up to 7 letters.
> For example, a white lower case comment "rrrrrrr" is
> $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
> a white colour token. A sequence of Shannon-Fano letters can be
> terminated by a 4-bit Shannon-Fano NULL, which displays as a space
> character.
>
> The simple idea is that if the first Shannon-Fano encoded letter is a
> NULL ( all four bits are 0 ) the token does not display anything (except
> a space), so that in this case the remaining 24 bits can be interpreted
> as three 8 bit ASCII/UTF-8 characters.
>
> Now all I need to do is add ASCII/UTF-8 input, and support for
> international QWERTZ, AZERTY and QWERTY etc. keyboards...
>
> Watch this space :-)
>

--
Bah, and indeed Humbug.

Am 22/04/2022 um 11:01 schrieb Kerr-Mudd, John:
> On Thu, 21 Apr 2022 06:15:54 +0200
> Howerd Oakford <howerd@inventio.co.uk> wrote:
>
>> Hi colorForthers,
>>
>> Yesterday I had an idea about how to add ASCII/UTF-8 support to colorForth.
>>
>> Today I wrote the code to do this, as a demonstration, and it works!!!
>>
>> You can see the demonstration here :
>> http://www.inventio.co.uk/cf2022/Encoding%20ASCII%20into%20cf2022%20colorForth%20tokens.pdf
>>
> On that web page (which worked fine for my old insecure computer) you post some nasm code:
>
> <Excerpt>
>
> lowercase: ; display a white text word in normal lower-case letters
> call white
> showSF_EDI_: ; ( -- ) \ display a Shanon-Fano encoded token pointed to by edi in the current colour
> _DUP_
> mov _TOS_, [ ( edi * 4 ) - 0x04 ] ; fetch the next token – drops through to showShannonFano
> showShannonFano: ; ( token -- ) \ display the Shannon-Fano encoded token on TOS
> ; ASCII / UTF8 support. If the first Shannon-Fano encoded letter is a 4 bit NULL,
> ; display the next 24 bits as three ASCII characters.
> mov _SCRATCH_, _TOS_ ; save the token value
> and _SCRATCH_, 0xF0000000
> ;;;;;; cmp _SCRATCH_, 0x00000000
> jnz .forward
> ; display as three ASCII characters
>
> </Excerpt>
>
>
> I've commented out the 'cmp' test - the ZR flag is set/unset by the preceding 'and'.
> HTH!
>
>
>> I've been trying to do this for about 15 years, so I am very excited
>> that the idea just came to me yesterday.
>>
>> In cf2022 colorForth, a Shannon-Fano encoded token consists of a 4 bit
>> colour in the bottom four bits, and 28 bits of up to 7 letters.
>> For example, a white lower case comment "rrrrrrr" is
>> $11111119 , where '1' is a four bit code for the letter 'r', and '9' is
>> a white colour token. A sequence of Shannon-Fano letters can be
>> terminated by a 4-bit Shannon-Fano NULL, which displays as a space
>> character.
>>
>> The simple idea is that if the first Shannon-Fano encoded letter is a
>> NULL ( all four bits are 0 ) the token does not display anything (except
>> a space), so that in this case the remaining 24 bits can be interpreted
>> as three 8 bit ASCII/UTF-8 characters.
>>
>> Now all I need to do is add ASCII/UTF-8 input, and support for
>> international QWERTZ, AZERTY and QWERTY etc. keyboards...
>>
>> Watch this space :-)
>>
>
>
Hi John(?),

Thanks - you are right!

I will update the code accordingly.

BTW Did you know that your name, placed in the reverse order, with a
comma, sounds like curmudgeon? I'm guessing yes.
I love some of the inventive names here on clf ;-)

Cheers,
Howerd

On Fri, 22 Apr 2022 21:22:07 +0200
Howerd Oakford <howerd@inventio.co.uk> wrote:

> Am 22/04/2022 um 11:01 schrieb Kerr-Mudd, John:
> > On Thu, 21 Apr 2022 06:15:54 +0200
[]
> > and _SCRATCH_, 0xF0000000
> > ;;;;;; cmp _SCRATCH_, 0x00000000
> > jnz .forward
[]
> >
> >
> > I've commented out the 'cmp' test - the ZR flag is set/unset by the preceding 'and'.
> > HTH!
> >
[]
> >
> Hi John(?),
Fine.
>
> Thanks - you are right!
>
> I will update the code accordingly.
>
It's not a big thing, but I thought I'd mention it, I dabble in (n)asm a bit.

> BTW Did you know that your name, placed in the reverse order, with a
> comma, sounds like curmudgeon? I'm guessing yes.

It was a silly "make up a nym for usenet" I did years ago and I've just stuck with it. But I do get more curmudgeonly as I age (don't we all?)

--
Bah, and indeed Humbug.