Examples of using the right-to-left mark and left-to-right mark
(Unicode) to solve direction problems with occasional Arabic or
Yiddish embedded in left-to-right text.

https://rudhar.com/sfreview/rtolmark/en.htm

--
Ruud Harmsen, http://rudhar.com

On Friday, March 31, 2023 at 3:04:45 PM UTC-4, Ruud Harmsen wrote:
> Examples of using the right-to-left mark and left-to-right mark
> (Unicode) to solve direction problems with occasional Arabic or
> Yiddish embedded in left-to-right text.
>
> https://rudhar.com/sfreview/rtolmark/en.htm

MSWord handles that with no problems at all.

InDesign, OTOH, which is what the vast majority of typesetters use,
does not.

Fri, 31 Mar 2023 12:27:30 -0700 (PDT): "Peter T. Daniels"
<grammatim@verizon.net> scribeva:

>On Friday, March 31, 2023 at 3:04:45?PM UTC-4, Ruud Harmsen wrote:
>> Examples of using the right-to-left mark and left-to-right mark
>> (Unicode) to solve direction problems with occasional Arabic or
>> Yiddish embedded in left-to-right text.
>>
>> https://rudhar.com/sfreview/rtolmark/en.htm
>
>MSWord handles that with no problems at all.

1) I read a report in Facebook group "Learn Yiddish" that a copy-paste
from web to Word still goes wrong, reversed all the letters. Don’t
know which version that was.
https://www.facebook.com/groups/learnyiddish/posts/10159039924571056/

2) How can Word distinguish between a question mark that is part of an
English sentence with some embedded Hebrew or Yiddish, and the same
where the question mark belongs to the right-to-left part? As in the
examples I gave. The correct decision is only possible by
understanding the text.

In other words, in your "MSWord handles that with no problems at all",
what exactly do you mean by "that"?

>InDesign, OTOH, which is what the vast majority of typesetters use,
>does not.

That is bad.
--
Ruud Harmsen, http://rudhar.com

On Fri, 31 Mar 2023, Ruud Harmsen wrote:

> 2) How can Word distinguish between a question mark that is part of an
> English sentence with some embedded Hebrew or Yiddish, and the same
> where the question mark belongs to the right-to-left part? As in the
> examples I gave. The correct decision is only possible by
> understanding the text.

The Unicode Bidi algorithm does not understand but just handles the most
frequent cases correctly. IIRC the next character with unique writing
direction after the question mark decides on the assumed writing direction of
the question mark. The following examples do not contain any LRM or RLM:

Are the first letters in the Hebrew alphabet א, ב, ג? Yes.

Are the first letters in the Hebrew alphabet א, ב, ג? כן.

A more logical sentence would be for the first:
Are the first letters in the Hebrew alphabet א‎, ב‎, ג? Yes.

It requires left-to-right marks before the two commas but none before the
question mark, as this is determined by the following Y.

If the second sentence is meant as an English question with a Hebrew answer,
it reqires an additional left-to-right marks before the question mark and a
right-to-left mark after the final period.

Are the first letters in the Hebrew alphabet א‎, ב‎, ג‎? כן.‏

I don’t know why it does not work with RTL before the period as in the
following line:

Are the first letters in the Hebrew alphabet א‎, ב‎, ג‎? כן‏.

--
Helmut Richter

On Saturday, April 1, 2023 at 10:57:31 AM UTC+2, Helmut Richter wrote:
> Are the first letters in the Hebrew alphabet א‎, ב‎, ג‎? כן.‏
>
> I don’t know why it does not work with RTL before the period as in the
> following line:
>
> Are the first letters in the Hebrew alphabet א‎, ב‎, ג‎? כן‏.

I think the full stop character, although it is neutral in matters of direction, defaults to being LTR, unless it is enclosed between two inherently RTL characters.
In the first of your examples that I quote here, you have:
question mark, space, kaf, nun, stop, RTL-mark, newline.
So the stop or dot character has neighbours on both sides that are clearly RTL.

In the second example:
question mark, space, kaf, nun, RTL-mark, stop, newline.
Now the stop character has an RTL neighbour and a neutral neighbour (the CR or LF, or similar control character dependeding on platform). Apparently RTL-neutral causes the interpretation LTR. But that is strange, because in longer running Hebrew text, you could have the sequence comma-space, so the the comma is between an RTL Hebrew character and the neutral space, and should be interpreted as RTL, not LTR.

Sat, 1 Apr 2023 03:33:10 -0700 (PDT): "Ruud Harmsen via Google Groups
<google@rudhar.com>" <google@rudhar.com> scribeva:
>I think the full stop character, although it is neutral in matters of direction, defaults to being LTR, unless it is enclosed between two inherently RTL characters.
>In the first of your examples that I quote here, you have:
>question mark, space, kaf, nun, stop, RTL-mark, newline.
>So the stop or dot character has neighbours on both sides that are clearly RTL.
>
>In the second example:
>question mark, space, kaf, nun, RTL-mark, stop, newline.
>Now the stop character has an RTL neighbour and a neutral neighbour (the CR or LF, or similar control character dependeding on platform). Apparently RTL-neutral causes the interpretation LTR. But that is strange, because in longer running Hebrew text, you could have the sequence comma-space, so the the comma is between an RTL Hebrew character and the neutral space, and should be interpreted as RTL, not LTR.

Which of course is exactly your point.
--
Ruud Harmsen, http://rudhar.com

On Saturday, April 1, 2023 at 12:34:57 PM UTC+2, Ruud Harmsen wrote:
>>Now the stop character has an RTL neighbour and a neutral neighbour (the CR or LF, or similar control character dependeding on platform). Apparently RTL-neutral causes the interpretation LTR. But that is strange, because in longer running Hebrew text, you could have the sequence comma-space, so the the comma is between an RTL Hebrew character and the neutral space, and should be interpreted as RTL, not LTR.
> Which of course is exactly your point.

It’s more complicated than that, that’s why:
https://unicode.org/reports/tr9/#Bidirectional_Character_Types
Comma and full stop are weak, a space is neutral.

On Friday, March 31, 2023 at 5:23:27 PM UTC-4, Ruud Harmsen wrote:
> Fri, 31 Mar 2023 12:27:30 -0700 (PDT): "Peter T. Daniels"
> <gram...@verizon.net> scribeva:
> >On Friday, March 31, 2023 at 3:04:45?PM UTC-4, Ruud Harmsen wrote:
> >> Examples of using the right-to-left mark and left-to-right mark
> >> (Unicode) to solve direction problems with occasional Arabic or
> >> Yiddish embedded in left-to-right text.
> >>
> >> https://rudhar.com/sfreview/rtolmark/en.htm
> >
> >MSWord handles that with no problems at all.
> 1) I read a report in Facebook group "Learn Yiddish" that a copy-paste
> from web to Word still goes wrong, reversed all the letters. Don’t
> know which version that was.

Irrelevant. If the web site had screwed it up, that's not MSWord's
problem. You feed a perfect Word file into InDesign, it screws jt up.
I'm about to have THIRD proofs of a tiny article in *Journal of Semitic
Studies* because it couldn't handle THREE WORDS in Hebrew.

> https://www.facebook.com/groups/learnyiddish/posts/10159039924571056/
>
> 2) How can Word distinguish between a question mark that is part of an
> English sentence with some embedded Hebrew or Yiddish, and the same
> where the question mark belongs to the right-to-left part? As in the
> examples I gave. The correct decision is only possible by
> understanding the text.

Because the writer switches between scripts at the appropriate point.
Word has an option to "detect language" and change keyboards when
you change the font to a language-specific one. For Roman alphabet
when you switch to German or French, it automatically does the quotation
marks right, for instance. Presumably it does the same for Cyrillic languages
and hopefully for Arabic-script ones.

> In other words, in your "MSWord handles that with no problems at all",
> what exactly do you mean by "that"?

EXACTLY WHAT YOU SAID. R-to-l and l-to-r marks.

> >InDesign, OTOH, which is what the vast majority of typesetters use,
> >does not.
>
> That is bad.

On Saturday, April 1, 2023 at 3:23:05 PM UTC+2, Peter T. Daniels wrote:
> On Friday, March 31, 2023 at 5:23:27 PM UTC-4, Ruud Harmsen wrote:
> > Fri, 31 Mar 2023 12:27:30 -0700 (PDT): "Peter T. Daniels"
> > <gram...@verizon.net> scribeva:
> > >On Friday, March 31, 2023 at 3:04:45?PM UTC-4, Ruud Harmsen wrote:
> > >> Examples of using the right-to-left mark and left-to-right mark
> > >> (Unicode) to solve direction problems with occasional Arabic or
> > >> Yiddish embedded in left-to-right text.
> > >>
> > >> https://rudhar.com/sfreview/rtolmark/en.htm
> > >
> > >MSWord handles that with no problems at all.
> > 1) I read a report in Facebook group "Learn Yiddish" that a copy-paste
> > from web to Word still goes wrong, reversed all the letters. Don’t
> > know which version that was.
> Irrelevant. If the web site had screwed it up, that's not MSWord's
> problem.

Website do not screw up, because there are standards, HTML is simple, and all major browsers now adhere to standards. And there are largely only two engines: Gecko and Chromium. Even Edge now use Chromium.

https://forward.com/yiddish/ does not screw up.

So it anything goes wrong, it is Word’s and Microsoft’s fault. But I expect newer Words to do it right too.

>You feed a perfect Word file into InDesign, it screws jt up.
> I'm about to have THIRD proofs of a tiny article in *Journal of Semitic
> Studies* because it couldn't handle THREE WORDS in Hebrew.

So why still use such defective software?

> > https://www.facebook.com/groups/learnyiddish/posts/10159039924571056/
> >
> > 2) How can Word distinguish between a question mark that is part of an
> > English sentence with some embedded Hebrew or Yiddish, and the same
> > where the question mark belongs to the right-to-left part? As in the
> > examples I gave. The correct decision is only possible by
> > understanding the text.
> Because the writer switches between scripts at the appropriate point.
> Word has an option to "detect language" and change keyboards when
> you change the font to a language-specific one. For Roman alphabet
> when you switch to German or French, it automatically does the quotation
> marks right, for instance. Presumably it does the same for Cyrillic languages
> and hopefully for Arabic-script ones.

When I set the language to English at the appropriate point in my last example, it didn’t help.

> > In other words, in your "MSWord handles that with no problems at all",
> > what exactly do you mean by "that"?
> EXACTLY WHAT YOU SAID. R-to-l and l-to-r marks.

OK. So we agree. Not the first time.

> > >InDesign, OTOH, which is what the vast majority of typesetters use,
> > >does not.
> >
> > That is bad.

On Saturday, April 1, 2023 at 4:37:16 PM UTC-4, Ruud Harmsen via Google Groups <google@rudhar.com> wrote:
> On Saturday, April 1, 2023 at 3:23:05 PM UTC+2, Peter T. Daniels wrote:

> Website do not screw up, because there are standards,

That should appear, with attribution, in someone's .sig

> >You feed a perfect Word file into InDesign, it screws jt up.
> > I'm about to have THIRD proofs of a tiny article in *Journal of Semitic
> > Studies* because it couldn't handle THREE WORDS in Hebrew.
>
> So why still use such defective software?

Do you ever pay attention?

Authors in journals have no say over what those contracted by the
publisher4 use.

Sun, 2 Apr 2023 06:34:44 -0700 (PDT): "Peter T. Daniels"
<grammatim@verizon.net> scribeva:

>On Saturday, April 1, 2023 at 4:37:16?PM UTC-4, Ruud Harmsen via Google Groups <google@rudhar.com> wrote:
>> On Saturday, April 1, 2023 at 3:23:05?PM UTC+2, Peter T. Daniels wrote:
>
>> Website do not screw up, because there are standards,
>
>That should appear, with attribution, in someone's .sig

???

>> >You feed a perfect Word file into InDesign, it screws jt up.
>> > I'm about to have THIRD proofs of a tiny article in *Journal of Semitic
>> > Studies* because it couldn't handle THREE WORDS in Hebrew.
>>
>> So why still use such defective software?
>
>Do you ever pay attention?

Always AFAIK.

>Authors in journals have no say over what those contracted by the
>publisher4 use.

Then self-publish. Neem de teugels in eigen hand.
--
Ruud Harmsen, http://rudhar.com