I need to compare utf-8 strings ignoring the case on windows and linux.

I tried _stricmp on windows and strcasecmp on linux
changing the locale to utf8. No success.

Has anyone tried this? Have this problem?

I'm giving up trying to make this work using native SO functions..and maybe I will use a utf8-lib that has some implementation.

On Wednesday, 13 April 2022 at 12:49:33 UTC+1, Thiago Adams wrote:
> I need to compare utf-8 strings ignoring the case on windows and linux.
>
> I tried _stricmp on windows and strcasecmp on linux
> changing the locale to utf8. No success.
>
> Has anyone tried this? Have this problem?
>
> I'm giving up trying to make this work using native SO functions..and maybe I will use a utf8-lib that has some implementation.
>
Case-insensitivity doesn't translate easily to non-English languages. Even in French, e-acute is capitalised as a simple "E"
without an accent, so the reversibility principle is broken. In Hebrew, you don't have a concept of capitals at all, but
you do have other variation in the basic 22 character set.

On Wednesday, April 13, 2022 at 9:34:00 AM UTC-3, Malcolm McLean wrote:
> On Wednesday, 13 April 2022 at 12:49:33 UTC+1, Thiago Adams wrote:
> > I need to compare utf-8 strings ignoring the case on windows and linux.
> >
> > I tried _stricmp on windows and strcasecmp on linux
> > changing the locale to utf8. No success.
> >
> > Has anyone tried this? Have this problem?
> >
> > I'm giving up trying to make this work using native SO functions..and maybe I will use a utf8-lib that has some implementation.
> >
> Case-insensitivity doesn't translate easily to non-English languages. Even in French, e-acute is capitalised as a simple "E"
> without an accent, so the reversibility principle is broken.

é becomes E in French?

The lib I am looking at this moment
https://github.com/sheredom/utf8.h/blob/master/utf8.h
(see utf8casecmp.)
Compares if lowercase matches or if uppercase matches.

So comparing "é" with "E" in French
if é == e or E == E returns true.

On Wednesday, 13 April 2022 at 13:53:49 UTC+1, Thiago Adams wrote:
> On Wednesday, April 13, 2022 at 9:34:00 AM UTC-3, Malcolm McLean wrote:
> > On Wednesday, 13 April 2022 at 12:49:33 UTC+1, Thiago Adams wrote:
> > > I need to compare utf-8 strings ignoring the case on windows and linux.
> > >
> > > I tried _stricmp on windows and strcasecmp on linux
> > > changing the locale to utf8. No success.
> > >
> > > Has anyone tried this? Have this problem?
> > >
> > > I'm giving up trying to make this work using native SO functions..and maybe I will use a utf8-lib that has some implementation.
> > >
> > Case-insensitivity doesn't translate easily to non-English languages. Even in French, e-acute is capitalised as a simple "E"
> > without an accent, so the reversibility principle is broken.
> é becomes E in French?
>
> The lib I am looking at this moment
> https://github.com/sheredom/utf8.h/blob/master/utf8.h
> (see utf8casecmp.)
> Compares if lowercase matches or if uppercase matches.
>
> So comparing "é" with "E" in French
> if é == e or E == E returns true.
>
Yes. You don't write the accents for uppercase letters in French.
So E matches either e or e-accented. So the logic
if (tolower(char_a) == tolower(char_b))

which is acceptable for English, would break in French. If you used
toupper() it would work, but possibly break for other languages.

It seems the casecmp() functions are defined to work if either toupper()
or tolower() give a match, which might work sufficiently well to be
usable.

Thiago Adams wrote:
>
> I need to compare utf-8 strings ignoring the case on windows and linux.

This is a nontrivial task with Unicode:
<https://www.unicode.org/reports/tr21/tr21-5.html#Caseless_Matching>

> [...]
> and maybe I will use a utf8-lib that has some implementation.

One such library is ICU: <https://icu.unicode.org/>
Common GNU/Linux distributions should ship a package for it.
According to Wikipedia, it is shipped with Windows 10 since version
1703:
<https://en.wikipedia.org/wiki/International_Components_for_Unicode>

The function u_strcasecmp() maybe is suitable for what you want to do:
<https://unicode-org.github.io/icu-docs/apidoc/released/icu4c/ustring_8h.html#a418fdda6c0b3ffd4da6fa8b4f46fafec>

2022-04-13 at 06:03 -0700, Malcolm McLean wrote:
> Yes. You don't write the accents for uppercase letters in French.

That is very much incorrect. Missing accents on French uppercase
letters is only a sign of laziness or ancient technical limitations.

Mateusz

On 4/13/2022 3:17 PM, Mateusz Viste wrote:
> 2022-04-13 at 06:03 -0700, Malcolm McLean wrote:
>> Yes. You don't write the accents for uppercase letters in French.
>
> That is very much incorrect. Missing accents on French uppercase
> letters is only a sign of laziness or ancient technical limitations.
>
> Mateusz
>

I believe you are right:

é É

á Á

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> On Wednesday, 13 April 2022 at 13:53:49 UTC+1, Thiago Adams wrote:
>> On Wednesday, April 13, 2022 at 9:34:00 AM UTC-3, Malcolm McLean wrote:

>> > Case-insensitivity doesn't translate easily to non-English
>> > languages. Even in French, e-acute is capitalised as a simple "E"
>> > without an accent, so the reversibility principle is broken.
>>
>> é becomes E in French?
<cut>
> Yes. You don't write the accents for uppercase letters in French.

Je vous propose:

https://www.academie-francaise.fr/questions-de-langue

--
Ben.

On Wednesday, 13 April 2022 at 15:50:50 UTC+1, Ben wrote:
> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>
> > On Wednesday, 13 April 2022 at 13:53:49 UTC+1, Thiago Adams wrote:
> >> On Wednesday, April 13, 2022 at 9:34:00 AM UTC-3, Malcolm McLean wrote:
>
> >> > Case-insensitivity doesn't translate easily to non-English
> >> > languages. Even in French, e-acute is capitalised as a simple "E"
> >> > without an accent, so the reversibility principle is broken.
> >>
> >> é becomes E in French?
> <cut>
> > Yes. You don't write the accents for uppercase letters in French.
> Je vous propose:
>
> https://www.academie-francaise.fr/questions-de-langue
>
Ah, the academy disagrees.
I was taught not to put accents on capitals by my French teacher and
that's the practice followed by many printers. But it is wrong.

Which creates an even more difficult situation for programmers.

On Wed, 2022-04-13, Malcolm McLean wrote:
> On Wednesday, 13 April 2022 at 12:49:33 UTC+1, Thiago Adams wrote:
>> I need to compare utf-8 strings ignoring the case on windows and linux.
>>
>> I tried _stricmp on windows and strcasecmp on linux
>> changing the locale to utf8. No success.
>>
>> Has anyone tried this? Have this problem?
>>
>> I'm giving up trying to make this work using native SO
>> functions..and maybe I will use a utf8-lib that has some
>> implementation.
>>
> Case-insensitivity doesn't translate easily to non-English languages.

But it /does/ translate.

[snip examples]

If we go back to string comparisons in general, there are other
challenges, too. The right ordering depends on culture, time and
context. If you're unlucky it also depends on the semantics of the
text. The swedish rule for the ae ligature is[1] to sort it as
a-umlaut if the word comes from Iceland, but as ae if it comes from
Latin.

I don't know what Gnu libc does for sv_SE in that case. But I know
it has plenty of code for collation. Table-driven, I think, and
the strings to compare get "compiled" to something which can simply
be strcmp()ed in the "C" locale in the end.

The main conclusion I have drawn about this stuff is: if at all
possible, avoid doing anything "humanistic" to text in your programs.
This includes uppercasing, lowercasing, and sorting in a way which
makes sense to non-programmers. Life becomes much easier this way.
Like the Unix file system -- it doesn't have to know anything about
character sets or languages, because file names are just arrays of
char.

/Jorgen

[1] Or was. Sometimes the rules get dumbed down so they can be
automated. Sometimes they get dumbed down because everyone is
already using software which ignores the rules, and there's no
point having a rule which only a handful of humanists care about.

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

In article <slrnt5e4i3.2ude.grahn+nntp@frailea.sa.invalid>,
Jorgen Grahn <grahn+nntp@snipabacken.se> wrote:

> If we go back to string comparisons in general, there are other
> challenges, too. The right ordering depends on culture, time and

If your text is in unicode, like UTF8, you can define 'caseless'
with unicode upper, lower, and title case. Another old technique
is to use collation tables. Such a table gives a collation order
to each character, so you compare the collation order of
characters rather than the character themselves.

One complication is that a character like e-acute can regarded as
equivalent to a character sequence like e + acute-mark. Unicode
has defined procedures to create canonical encodings for all
strings.

--
:-<> Siri Seal of Disavowal #000-001. Disavowed. Denied. Deleted. @
'I desire mercy, not sacrifice.' /|\
Discordia: not just a religion but also a parody. This post / \
I am an Andrea Doria sockpuppet. insults Islam. Mohammed

On 13/04/2022 14:53, Thiago Adams wrote:
> On Wednesday, April 13, 2022 at 9:34:00 AM UTC-3, Malcolm McLean wrote:
>> On Wednesday, 13 April 2022 at 12:49:33 UTC+1, Thiago Adams wrote:
>>> I need to compare utf-8 strings ignoring the case on windows and linux.
>>>
>>> I tried _stricmp on windows and strcasecmp on linux
>>> changing the locale to utf8. No success.
>>>
>>> Has anyone tried this? Have this problem?
>>>
>>> I'm giving up trying to make this work using native SO functions..and maybe I will use a utf8-lib that has some implementation.
>>>
>> Case-insensitivity doesn't translate easily to non-English languages. Even in French, e-acute is capitalised as a simple "E"
>> without an accent, so the reversibility principle is broken.
>
> é becomes E in French?
>
> The lib I am looking at this moment
> https://github.com/sheredom/utf8.h/blob/master/utf8.h
> (see utf8casecmp.)
> Compares if lowercase matches or if uppercase matches.
>
> So comparing "é" with "E" in French
> if é == e or E == E returns true.
>

I don't know enough about French to comment. But Malcolm's point about
the problem being massively complicated in other languages is true.

In Turkish, the letter i capitalises to İ (with a dot above it), while
the letter ı (with no dot) capitalises to I. These are entirely
different letters.

In German, ß is used in some circumstances for two small s letters, but
the capital is written SS. Greek has two different small letter sigmas
but only one capital. Arabic has, if I remember correctly (I don't know
the language) five different versions for each written letter rather
than just two as we have in the Latin alphabet.

At the very least, any kind of case-insensitive comparison is going to
be dependent on the language.

Then you can have the joys of transliteration of non-English letters to
English, even disregarding capitalisation. The Norwegian letter "å" is
sometimes transliterated as "aa", sometimes just "a". So the name "Åse"
might sometimes be written "Ase" or "AAse" but be considered the same.
(And "å" is sorted at the end of the alphabet - so sometimes "aa" will
sort at the start, sometimes at the end of a sorting.)

David Brown <david.brown@hesbynett.no> writes:
[...]
> In German, ß is used in some circumstances for two small s letters,
> but the capital is written SS.

The character ß is called "LATIN SMALL LETTER SHARP S" in Unicode.
There's also a character ẞ "LATIN CAPITAL LETTER SHARP S". (The capital
letter is smaller than the small letter in the font I'm using.) I
hanve't checked the German capitalization rules, but they could in
principle use *either* SS or ẞ.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips
void Void(void) { Void(); } /* The recursive call of the void */

On 4/13/2022 6:49 AM, Thiago Adams wrote:
> I need to compare utf-8 strings ignoring the case on windows and linux.
>
> I tried _stricmp on windows and strcasecmp on linux
> changing the locale to utf8. No success.
>
> Has anyone tried this? Have this problem?
>
> I'm giving up trying to make this work using native SO functions..and maybe I will use a utf8-lib that has some implementation.

Looks like strcmpi has the same problem.
https://www.ibm.com/docs/en/i/7.1?topic=functions-strcmpi-compare-strings-without-case-sensitivity

Also boost::iequals
https://stackoverflow.com/questions/11635/case-insensitive-string-comparison-in-c

Lynn

Le 13/04/2022 à 13:49, Thiago Adams a écrit :
> I need to compare utf-8 strings ignoring the case on windows and linux.
>
> I tried _stricmp on windows and strcasecmp on linux
> changing the locale to utf8. No success.
>
> Has anyone tried this? Have this problem?

There's no easy and standard way of doing this AFAIK. Capitalization
with alphabets other than roman is a complex task.

That's why file systems, such as exFAT (and I think NTFS) which require
case-insensitivity embed "up-case tables", so the file system must be
formatted with the locale supposed to be used for it, and this table is
what should be used to perform the up-casing.

You can probably find libraries embedding such tables for a variety of
locales, but I don't really know of a standard one for this. If anyone
does...

Keith Thompson wrote:
> David Brown <david.brown@hesbynett.no> writes:
> >
> [...]
> > In German, ß is used in some circumstances for two small s letters,
> > but the capital is written SS.
>
> The character ß is called "LATIN SMALL LETTER SHARP S" in Unicode.
> There's also a character ẞ "LATIN CAPITAL LETTER SHARP S". (The capital
> letter is smaller than the small letter in the font I'm using.) I
> hanve't checked the German capitalization rules, but they could in
> principle use *either* SS or ẞ.

The ẞ "LATIN CAPITAL LETTER SHARP S" has not existed in the past.
The reason was (AFAIK) the ligature nature of ß, that was never used
at the beginning of a word.

If uppercase was needed, e.g. in headlines, SS was used instead.
This was problematic because there are some german words that have
different meaning if they are written with ss instead of ß, e.g.
Maße (measures) vs. Masse (mass). Eventually a capital sharp s was
invented.

Unicode full case folding [1] maps:

S => s
ß => ss
ẞ => ss

This means in a case-insensitive string compare "ß", "ss", "ẞ" and "SS"
should match each other. A case-insensitive search for any of the
following four words should therefore match all of them:

Masse MASSE
Maße MAẞE

As noted above, this is rather wrong in german, because of the different
meaning. But in the past MASSE was the official capitalization for both
words (at least for some time).

__________
[1] <https://www.unicode.org/Public/UCD/latest/ucd/CaseFolding.txt>

On Wednesday, April 13, 2022 at 6:18:05 AM UTC-7, Mateusz Viste wrote:
> 2022-04-13 at 06:03 -0700, Malcolm McLean wrote:
> > Yes. You don't write the accents for uppercase letters in French.
> That is very much incorrect. Missing accents on French uppercase
> letters is only a sign of laziness or ancient technical limitations.
>
> Mateusz

No accents on uppercase "are these days tolerated", uppercase letter without accent is French grammar pedantic.
Thank you. Jeez.

no name <0mmw00@gmail.com> writes:

> On Wednesday, April 13, 2022 at 6:18:05 AM UTC-7, Mateusz Viste wrote:
>> 2022-04-13 at 06:03 -0700, Malcolm McLean wrote:
>> > Yes. You don't write the accents for uppercase letters in French.
>> That is very much incorrect. Missing accents on French uppercase
>> letters is only a sign of laziness or ancient technical limitations.
>
> No accents on uppercase "are these days tolerated", uppercase letter
> without accent is French grammar pedantic.

https://www.academie-francaise.fr/questions-de-langue#5_strong-em-accentuation-des-majuscules-em-strong

--
Ben.

no name <0mmw00@gmail.com> writes:
> On Wednesday, April 13, 2022 at 6:18:05 AM UTC-7, Mateusz Viste wrote:
>> 2022-04-13 at 06:03 -0700, Malcolm McLean wrote:
>> > Yes. You don't write the accents for uppercase letters in French.
>> That is very much incorrect. Missing accents on French uppercase
>> letters is only a sign of laziness or ancient technical limitations.
>>
>> Mateusz
>
> No accents on uppercase "are these days tolerated", uppercase letter without accent is French grammar pedantic.
> Thank you. Jeez.

Can you rephrase that? I know you're talking about accents on uppercase
letters, but I honestly don't know what you're saying about them.

(Anticipating the obvious responses, speculation from others about what
"no name" meant is not useful.)

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */