Kaz Kylheku <864-117-4973@kylheku.com> writes:
>On 2023-11-17, Scott Lurndal <scott@slp53.sl.home> wrote:
>>>https://austin-group-l.opengroup.narkive.com/lKcmfoRI/predictable-scheduling-behavior-in-pthread-cond-broadcast
>>
>> When David wrote that, multiprocessor systems were still rare and
>
>The referenced discussion appears to be from early 2010; maybe
>he was reiterating something from an earlier time?

The Posix 1003.4 (.4b - realtime and .4c - threads) work was in the
mid 90's. At that time, SMP systems were expensive and relatively
rare. My 2p octane workstation in 1998 was great, but uncommon even
in SGI engineering.

Bonita Montero wrote:
> Am 15.11.2023 um 03:10 schrieb Pavel:
>> Bonita Montero wrote:
>>> Am 09.11.2023 um 06:48 schrieb Chris M. Thomasson:
>>>> On 11/8/2023 9:45 PM, Bonita Montero wrote:
>>>>> Am 09.11.2023 um 06:44 schrieb Chris M. Thomasson:
>>>>>
>>>>>> Wait morphing is not in the realm of the compiler. It's in the
>>>>>> kernel.
>>>>>
>>>>> Read this:
>>>>> https://stackoverflow.com/questions/45163701/which-os-platforms-implement-wait-morphing-optimization
>>>>>
>>>>
>>>> Wait morphing can be highly beneficial.
>>>
>>> Wait morphing isn't necessary under Win32 since you can wait
>>> for the mutexe's binary semaphore and for the condvar's counting
>>> semaphore in one step with WaitForMultipleObjects. Unfortunately
>>> there's nothing under Linux like that.
>>>
>> Not true. A Linux program can wait for any number of eventfd
>> semaphores that can be used as either binary or a counting one.
>
> An eventfd can be a semaphore, but only one semaphore and not a set
> like SysV semaphores. You can wait for multiple semaphores with poll
> or select, but a change on one semaphore releases the waiting thread
> whereas with Win32 you can wait for all conditions to be met at once.
>
There is no system call in UNIX to wait for many conditions to be met at
once; but I do not see why it would be useful.

E.g., if there are several vars initialized to 0 and it is required to
wake up a consumer thread when they all are made positive (or >5 or
whatever) by some producer thread or threads, a producer thread that
sets the last variable to the needed value can signal a single condvar
and the consumer thread will wake up to all conditions met at once.

On 2023-11-18, Scott Lurndal <scott@slp53.sl.home> wrote:
> Kaz Kylheku <864-117-4973@kylheku.com> writes:
>>On 2023-11-17, Scott Lurndal <scott@slp53.sl.home> wrote:
>>>>https://austin-group-l.opengroup.narkive.com/lKcmfoRI/predictable-scheduling-behavior-in-pthread-cond-broadcast
>>>
>>> When David wrote that, multiprocessor systems were still rare and
>>
>>The referenced discussion appears to be from early 2010; maybe
>>he was reiterating something from an earlier time?
>
> The Posix 1003.4 (.4b - realtime and .4c - threads) work was in the
> mid 90's. At that time, SMP systems were expensive and relatively
> rare. My 2p octane workstation in 1998 was great, but uncommon even
> in SGI engineering.

In 1995, I had a PC with a consumer-grade dual Pentium 100 ASUS motherboard,
running Linux 1.3.x, FWIW.

Two parallel gcc compilations were a whopping 30% faster than one!

:)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Am 18.11.2023 um 03:16 schrieb Pavel:

> There is no system call in UNIX to wait for many conditions to be met at
> once; but I do not see why it would be useful.

poll() and wait() wait for at least one condition to be met.

Bonita Montero wrote:
> Am 18.11.2023 um 03:16 schrieb Pavel:
>
>> There is no system call in UNIX to wait for many conditions to be met
>> at once; but I do not see why it would be useful.
>
> poll() and wait() wait for at least one condition to be met.
what does it have to do with what I wrote?

Again, what a system call that waits for many conditions to be met at
once can be useful for?

Am 18.11.2023 um 18:53 schrieb Pavel:
> Bonita Montero wrote:
>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>
>>> There is no system call in UNIX to wait for many conditions to be met
>>> at once; but I do not see why it would be useful.
>>
>> poll() and wait() wait for at least one condition to be met.
> what does it have to do with what I wrote?
>
> Again, what a system call that waits for many conditions to be met at
> once can be useful for?

If you want to substitute a semaphore set with two eventfd()s you'd
should at best wait for both signalled together to avoid the discussed
double kernel call. As you this isn't possible with poll() or wait()
you can stick with normal posix semaphores, which do the same. The
best would be if you signal the waiting thread with a Posix signal,
then you can awake the thread upon an arbitrary state. I gues that's
what the glibc does, or I don't know another way to have only one
kernel voluntary context switch with pthread_cond_wait.

Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> writes:
>Bonita Montero wrote:
>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>
>>> There is no system call in UNIX to wait for many conditions to be met
>>> at once; but I do not see why it would be useful.
>>
>> poll() and wait() wait for at least one condition to be met.
>what does it have to do with what I wrote?
>
>Again, what a system call that waits for many conditions to be met at
>once can be useful for?

Rendezvous.

e.g. pthread_barrier_wait.

On 2023-11-18, Scott Lurndal <scott@slp53.sl.home> wrote:
> Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> writes:
>>Bonita Montero wrote:
>>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>>
>>>> There is no system call in UNIX to wait for many conditions to be met
>>>> at once; but I do not see why it would be useful.
>>>
>>> poll() and wait() wait for at least one condition to be met.
>>what does it have to do with what I wrote?
>>
>>Again, what a system call that waits for many conditions to be met at
>>once can be useful for?
>
> Rendezvous.
>
> e.g. pthread_barrier_wait.

Also this:

while (!this_condition() && !that_condition() && !other_condition())
pthread_cond_wait(&cond, &mutex);

One condition variable can be used to wait for any subset of the
actual conditions over the shared variables protected by the mutex.

When I switched to a Win32 development job in the mid 1990's, it amused
me that Win32 had all these stupid things like WaitForMultipleObjects
(with a weird 64 limit), instead of just providing condition variables.

Today, Microsoft provides condition variables.

Modules of a large program should use callbacks to notify that something
is ready. That's the most flexible API. The caller can build proper
asynchronous control flows around that, or bind the callback to some
signaling object according to how they see fit.

Say I have a module that works with three other modules that can
spontaneously make some condition ready. If they all provide a callback,
I can set a flag in the callback and broadcast or signal a condition
variable. Then I can have threads wait for all three things to be
ready, or any one of the three things.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 11/18/2023 9:53 AM, Pavel wrote:
> Bonita Montero wrote:
>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>
>>> There is no system call in UNIX to wait for many conditions to be met
>>> at once; but I do not see why it would be useful.
>>
>> poll() and wait() wait for at least one condition to be met.
> what does it have to do with what I wrote?
>
> Again, what a system call that waits for many conditions to be met at
> once can be useful for?

select?

Kaz Kylheku <864-117-4973@kylheku.com> writes:
>On 2023-11-18, Scott Lurndal <scott@slp53.sl.home> wrote:
>> Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> writes:
>>>Bonita Montero wrote:
>>>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>>>
>>>>> There is no system call in UNIX to wait for many conditions to be met
>>>>> at once; but I do not see why it would be useful.
>>>>
>>>> poll() and wait() wait for at least one condition to be met.
>>>what does it have to do with what I wrote?
>>>
>>>Again, what a system call that waits for many conditions to be met at
>>>once can be useful for?
>>
>> Rendezvous.
>>
>> e.g. pthread_barrier_wait.
>
>Also this:
>
> while (!this_condition() && !that_condition() && !other_condition())
> pthread_cond_wait(&cond, &mutex);
>
>One condition variable can be used to wait for any subset of the
>actual conditions over the shared variables protected by the mutex.
>
>When I switched to a Win32 development job in the mid 1990's, it amused
>me that Win32 had all these stupid things like WaitForMultipleObjects
>(with a weird 64 limit), instead of just providing condition variables.

IIRC, that's a concept imported from VMS into NT by DC.

The Burroughs MCP had a 'complex wait' system call which could wait
for the occurance of any of a number of conditions (card reader ready,
tape unit ready, input ready from datacomm station, core-to-core
communication from cooperating process, inter-processor persistent
storage queue item availability, time delay, disk seek complete,
network data present, etc.

But it was a wait for any, not wait for all.

"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>On 11/18/2023 9:53 AM, Pavel wrote:
>> Bonita Montero wrote:
>>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>>
>>>> There is no system call in UNIX to wait for many conditions to be met
>>>> at once; but I do not see why it would be useful.
>>>
>>> poll() and wait() wait for at least one condition to be met.
>> what does it have to do with what I wrote?
>>
>> Again, what a system call that waits for many conditions to be met at
>> once can be useful for?
>
>select?

I took Pavel to mean 'wait for all conditions to be true' rather than
'wait for one or more conditions to be true'.

On 11/18/2023 1:47 PM, Scott Lurndal wrote:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>> On 11/18/2023 9:53 AM, Pavel wrote:
>>> Bonita Montero wrote:
>>>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>>>
>>>>> There is no system call in UNIX to wait for many conditions to be met
>>>>> at once; but I do not see why it would be useful.
>>>>
>>>> poll() and wait() wait for at least one condition to be met.
>>> what does it have to do with what I wrote?
>>>
>>> Again, what a system call that waits for many conditions to be met at
>>> once can be useful for?
>>
>> select?
>
> I took Pavel to mean 'wait for all conditions to be true' rather than
> 'wait for one or more conditions to be true'.
>

Ahhh. I missed that. Sorry everybody! ;^o

Scott Lurndal wrote:
> Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> writes:
>> Bonita Montero wrote:
>>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>>
>>>> There is no system call in UNIX to wait for many conditions to be met
>>>> at once; but I do not see why it would be useful.
>>>
>>> poll() and wait() wait for at least one condition to be met.
>> what does it have to do with what I wrote?
>>
>> Again, what a system call that waits for many conditions to be met at
>> once can be useful for?
>
> Rendezvous.
Is it even useful? I got familiar with it first in late 80s while
learning Ada but never needed it since, even as a concept.
> e.g. pthread_barrier_wait.

It seems to work well as it is, my feeling is that implementing it in
user space on top of some "wait_for_all_xxxs" (which is on itself too
high-level for my taste) would not be efficient.

After giving it some thought, I think I could write a reasonable
implementation with 1 mutex, 1 condvar and 2 ints (unless some
"predictable scheduling policy" is required -- but the implementation
with "wait_for_all_xxxs" should have issue with it as well).

Kaz Kylheku wrote:
> On 2023-11-18, Scott Lurndal <scott@slp53.sl.home> wrote:
>> Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> writes:
>>> Bonita Montero wrote:
>>>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>>>
>>>>> There is no system call in UNIX to wait for many conditions to be met
>>>>> at once; but I do not see why it would be useful.
>>>>
>>>> poll() and wait() wait for at least one condition to be met.
>>> what does it have to do with what I wrote?
>>>
>>> Again, what a system call that waits for many conditions to be met at
>>> once can be useful for?
>>
>> Rendezvous.
>>
>> e.g. pthread_barrier_wait.
>
> Also this:
>
> while (!this_condition() && !that_condition() && !other_condition())
> pthread_cond_wait(&cond, &mutex);

I had to solve similar tasks once in a while, would usually just have a
count of conditions left or a mask of condition wanted as bits and the
thread satisfying a condition would only signal once count is zero or
the mask covers all conditions, respectively to
- avoid unnecessary wake-ups;
- let waiters check that count or mask instead of doing multiple checks

>
> One condition variable can be used to wait for any subset of the
> actual conditions over the shared variables protected by the mutex.
>
> When I switched to a Win32 development job in the mid 1990's, it amused
> me that Win32 had all these stupid things like WaitForMultipleObjects
> (with a weird 64 limit), instead of just providing condition variables.
>
> Today, Microsoft provides condition variables.
>
> Modules of a large program should use callbacks to notify that something
> is ready. That's the most flexible API. The caller can build proper
> asynchronous control flows around that, or bind the callback to some
> signaling object according to how they see fit.
>
> Say I have a module that works with three other modules that can
> spontaneously make some condition ready. If they all provide a callback,
> I can set a flag in the callback and broadcast or signal a condition
> variable. Then I can have threads wait for all three things to be
> ready, or any one of the three things.
>

Chris M. Thomasson wrote:
> On 11/18/2023 1:47 PM, Scott Lurndal wrote:
>> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>>> On 11/18/2023 9:53 AM, Pavel wrote:
>>>> Bonita Montero wrote:
>>>>> Am 18.11.2023 um 03:16 schrieb Pavel:
>>>>>
>>>>>> There is no system call in UNIX to wait for many conditions to be met
>>>>>> at once; but I do not see why it would be useful.
>>>>>
>>>>> poll() and wait() wait for at least one condition to be met.
>>>> what does it have to do with what I wrote?
>>>>
>>>> Again, what a system call that waits for many conditions to be met at
>>>> once can be useful for?
>>>
>>> select?
>>
>> I took Pavel to mean 'wait for all conditions to be true' rather than
>> 'wait for one or more conditions to be true'.
>>
>
> Ahhh. I missed that. Sorry everybody! ;^o
NP, Scott is correct, this was about "wait for many conditions to be met
at once" that Bonita missed on Linux for some reason I still cannot get
or relate to.

Am 18.11.2023 um 20:24 schrieb Scott Lurndal:

> e.g. pthread_barrier_wait.

pthread_barrier_wait is for multiple threads joining on one event,
not one thread waiting for multiple events as discussed.

On 2023-11-19, Bonita Montero <Bonita.Montero@gmail.com> wrote:
> Am 18.11.2023 um 20:24 schrieb Scott Lurndal:
>
>> e.g. pthread_barrier_wait.
>
> pthread_barrier_wait is for multiple threads joining on one event,
> not one thread waiting for multiple events as discussed.

That is untrue; there are effectively N events in a barrier:

- thread 1 has arrived at the barrier
- thread 2 has arrived at the barrier
....
- thread N has arrived at the barrier

When all these events have occurred, then all threads at the barrier are
released (and an indication is given to one of them that it is the
special "serial thread").

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Am 19.11.2023 um 06:02 schrieb Kaz Kylheku:
> On 2023-11-19, Bonita Montero <Bonita.Montero@gmail.com> wrote:
>> Am 18.11.2023 um 20:24 schrieb Scott Lurndal:
>>
>>> e.g. pthread_barrier_wait.
>>
>> pthread_barrier_wait is for multiple threads joining on one event,
>> not one thread waiting for multiple events as discussed.
>
> That is untrue; there are effectively N events in a barrier:

No, a barrier internally consists of an atomic which each thread
decrements and if it wasn't the last thread decrementing the atomic
it waits for a semaphore, a.k.a. event, which is signalled by the
last thread which decremented the atomic.

On 2023-11-19, Bonita Montero <Bonita.Montero@gmail.com> wrote:
> Am 19.11.2023 um 06:02 schrieb Kaz Kylheku:
>> On 2023-11-19, Bonita Montero <Bonita.Montero@gmail.com> wrote:
>>> Am 18.11.2023 um 20:24 schrieb Scott Lurndal:
>>>
>>>> e.g. pthread_barrier_wait.
>>>
>>> pthread_barrier_wait is for multiple threads joining on one event,
>>> not one thread waiting for multiple events as discussed.
>>
>> That is untrue; there are effectively N events in a barrier:
>
> No, a barrier internally consists of an atomic which each thread
> decrements

That's an implementation detail. Because the events are one-shots we can
just use a counter to represent the state that we need for the barrier
to be able to conclude that all events have occurred.

Each "thread P is waiting for barrier" event is a one shot, because
once a thread is waiting, that fact stays latched.

If the waiting threads could get nervous and leave the barrier
before it activates, then a counter could not be used; there would
have to be a set representation like a bitmask.

Rest unread.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Am 19.11.2023 um 07:08 schrieb Kaz Kylheku:

> That's an implementation detail. ...

That's how it works.
So from the scheduling / kernel perspective there's only one event.

On 11/19/2023 3:05 AM, Bonita Montero wrote:
> Am 19.11.2023 um 07:08 schrieb Kaz Kylheku:
>
>> That's an implementation detail. ...
>
> That's how it works.
> So from the scheduling / kernel perspective there's only one event.
>
>

I suspect that you are unfamiliar with how WaitForMultipleObjects
works... I remember a test for 50,000 concurrent connections, that was
mentioned by Microsoft. It was testing events vs IOCP. Did you know that
they recommended altering the indexes of the events waited for by
WaitForMultipleObjects? I remember it, is was a long time ago, 2001 ish,
iirc.

Iirc, it was recommended to basically randomize, or shift the indexes.
God damn, I need to find that old post on msdn. Again, iirc, if we did
not do this, some events could starve in the server experiment.

The funny part is that the event based model performed pretty good, not
as scalable as IOCP, however, IOCP did not completely blow it out of the
water wrt performance and throughput.

On 11/19/2023 12:08 PM, Chris M. Thomasson wrote:
> On 11/19/2023 3:05 AM, Bonita Montero wrote:
>> Am 19.11.2023 um 07:08 schrieb Kaz Kylheku:
>>
>>> That's an implementation detail. ...
>>
>> That's how it works.
>> So from the scheduling / kernel perspective there's only one event.
>>
>>
>
> I suspect that you are unfamiliar with how WaitForMultipleObjects
> works... I remember a test for 50,000 concurrent connections, that was
> mentioned by Microsoft. It was testing events vs IOCP. Did you know that
> they recommended altering the indexes of the events waited for by
> WaitForMultipleObjects? I remember it, is was a long time ago, 2001 ish,
> iirc.
>
> Iirc, it was recommended to basically randomize, or shift the indexes.
> God damn, I need to find that old post on msdn. Again, iirc, if we did
> not do this, some events could starve in the server experiment.
>
> The funny part is that the event based model performed pretty good, not
> as scalable as IOCP, however, IOCP did not completely blow it out of the
> water wrt performance and throughput.

After I read that msdn post, I basically created a proxy server. One
using events, and another using IOCP. IOCP beats it, but did not
slaughter it at that time. I found that to be interesting.

On 11/19/2023 12:11 PM, Chris M. Thomasson wrote:
> On 11/19/2023 12:08 PM, Chris M. Thomasson wrote:
>> On 11/19/2023 3:05 AM, Bonita Montero wrote:
>>> Am 19.11.2023 um 07:08 schrieb Kaz Kylheku:
>>>
>>>> That's an implementation detail. ...
>>>
>>> That's how it works.
>>> So from the scheduling / kernel perspective there's only one event.
>>>
>>>
>>
>> I suspect that you are unfamiliar with how WaitForMultipleObjects
>> works... I remember a test for 50,000 concurrent connections, that was
>> mentioned by Microsoft. It was testing events vs IOCP. Did you know
>> that they recommended altering the indexes of the events waited for by
>> WaitForMultipleObjects? I remember it, is was a long time ago, 2001
>> ish, iirc.
>>
>> Iirc, it was recommended to basically randomize, or shift the indexes.
>> God damn, I need to find that old post on msdn. Again, iirc, if we did
>> not do this, some events could starve in the server experiment.
>>
>> The funny part is that the event based model performed pretty good,
>> not as scalable as IOCP, however, IOCP did not completely blow it out
>> of the water wrt performance and throughput.
>
> After I read that msdn post, I basically created a proxy server. One
> using events, and another using IOCP. IOCP beats it, but did not
> slaughter it at that time. I found that to be interesting.

That's way back when I remember using AIO for the HTTP proxy server over
on Linux and compared it to IOCP. Time flies!

Am 19.11.2023 um 21:08 schrieb Chris M. Thomasson:

> I suspect that you are unfamiliar with how WaitForMultipleObjects
> works... ...

Absolutely not, but with WFMO you'd need a lot of semaphores or events
whereas my solution needs one semaphore and you'd be limited to MAXIMUM
_WAIT_OBJECTS == 64 threads which can join the barrier.
And why do you think I didn't know that ? I've shown that I used that
with my monitor and you even didn't have to look at the source since
I've explained that I'm using WFMO multiple times.

On 11/20/2023 4:26 AM, Bonita Montero wrote:
> Am 19.11.2023 um 21:08 schrieb Chris M. Thomasson:
>
>> I suspect that you are unfamiliar with how WaitForMultipleObjects
>> works... ...
>
> Absolutely not, but with WFMO you'd need a lot of semaphores or events
> whereas my solution needs one semaphore and you'd be limited to MAXIMUM
> _WAIT_OBJECTS == 64 threads which can join the barrier.
> And why do you think I didn't know that ? I've shown that I used that
> with my monitor and you even didn't have to look at the source since
> I've explained that I'm using WFMO multiple times.
>

I was just thinking about the case that did not have to wait on all the
events your are passing into WFMO to be in a signaled state. That old
50,000 connection test (event vs. iocp) required us to shift their
positions in the array in order to get around some starvation issues...

Did you read that article on MSDN as well? Iirc, its around 20 years old.