On 11/14/2023 8:57 PM, Bonita Montero wrote:
> Am 14.11.2023 um 21:25 schrieb Chris M. Thomasson:
>
>> Well, we have cmpxchg8b on 32-bit Intel...
>
> That's what I also thought, i.e. you can make a 64 bit store with that.
> But that's for sure much slower than the trick with the x87-FPU I've
> shown.
>
>

Are you sure its 100% atomic for use in multi-thread sync algorithms?

On 11/14/2023 5:26 PM, Pavel wrote:
> Kaz Kylheku wrote:
>> On 2023-11-14, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo>
>> wrote:
>>> Kaz Kylheku wrote:
>>>> Because of the situation that the thread which was signaled is
>>>> trying to acquire the mutex, which, stupidly, the signaling thread
>>>> is still holding. So, oops, back it goes into suspended state, and we
>>>> have to context switch to the mutex holder which has to release the
>>>> mutex and then switch to that signaled thread again.
>>>>
>>> Please see my response to your other post. "as-if" calling of
>>> pthread_mutex_lock does not prescribe actually calling it. Nothing in
>>> the standard prevents pthread_cond_signal from doing all the job of
>>> stuffing the unblocked threads to the mutex waiting list in accordance
>>> with some scheduling policy.
>>
>> Have you noticed how no mutex appears in the pthread_cond_signal API?
>>
> Of course but so what? Nothing prevents the implementation from listing
> the temporarily released mutices of the waiting pthread_cond_wait
> callers in the cond var where pthread_cond_signal can access them.
>
> Or, possibly even better, keeping a pointer to them with the threads
> waiting  for the condvar (there can only be at most one such temporarily
> released mutex per the waiting thread). And the threads waiting for the
> condvar are accessible for the pthread_cond_signal according to the spec.

Just as long as the implementation works wrt calling
pthread_cond_signal/broadcast outside of the mutex. If not, it is broken.

On 11/14/2023 5:47 PM, Pavel wrote:
> Chris M. Thomasson wrote:
>> On 11/13/2023 9:28 PM, Chris M. Thomasson wrote:
>>> On 11/13/2023 7:27 PM, Kaz Kylheku wrote:
>>>> On 2023-11-14, Pavel
>>>> <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
>>>>> "as-if" and "according to the scheduling policy (if applicable)" here
>>>>> are important. The standard intent may be to permit only the threads
>>>>> that are "unblocked" contend for the mutex -- as opposed to all
>>>>> non-blocked threads.
>>>>
>>>> It's possible that only one thread is unblocked by a
>>>> pthread_cond_signal. It takes at least two parties to contend for
>>>> something.
>>>>
>>>
>>> A signalling thread does its thing while holding the mutex... Well,
>>> it hears the following playing in the background:
>>>
>>> https://youtu.be/7YvAYIJSSZY
>>
>> A condvar that cannot signal/broadcast from outside of a held mutex is
>> broken by default.
> It can, it's just that the desired scheduling policy cannot be
> *effectively* applied if a program signal outside of that mutex.
>
> E.g. in some scenario a higher-prio thread could stay indefinitely
> starved by the lower-prio thread, regardless of the thread library
> implementation.

Afaict, it is an implementation detail. Say, SCHED_FIFO, the impl shall
strive to honor this. If it has to use a bakery algorithm to do it, so
be it.

On 11/15/2023 12:18 PM, Chris M. Thomasson wrote:
> On 11/14/2023 5:47 PM, Pavel wrote:
>> Chris M. Thomasson wrote:
>>> On 11/13/2023 9:28 PM, Chris M. Thomasson wrote:
>>>> On 11/13/2023 7:27 PM, Kaz Kylheku wrote:
>>>>> On 2023-11-14, Pavel
>>>>> <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
>>>>>> "as-if" and "according to the scheduling policy (if applicable)" here
>>>>>> are important. The standard intent may be to permit only the threads
>>>>>> that are "unblocked" contend for the mutex -- as opposed to all
>>>>>> non-blocked threads.
>>>>>
>>>>> It's possible that only one thread is unblocked by a
>>>>> pthread_cond_signal. It takes at least two parties to contend for
>>>>> something.
>>>>>
>>>>
>>>> A signalling thread does its thing while holding the mutex... Well,
>>>> it hears the following playing in the background:
>>>>
>>>> https://youtu.be/7YvAYIJSSZY
>>>
>>> A condvar that cannot signal/broadcast from outside of a held mutex
>>> is broken by default.
>> It can, it's just that the desired scheduling policy cannot be
>> *effectively* applied if a program signal outside of that mutex.
>>
>> E.g. in some scenario a higher-prio thread could stay indefinitely
>> starved by the lower-prio thread, regardless of the thread library
>> implementation.
>
> Afaict, it is an implementation detail. Say, SCHED_FIFO, the impl shall
> strive to honor this. If it has to use a bakery algorithm to do it, so
> be it.

Also, you as the programmer can choose to signal within a locked region.
No problem.

On 11/14/2023 8:54 PM, Bonita Montero wrote:
> Am 14.11.2023 um 21:27 schrieb Chris M. Thomasson:
>> On 11/14/2023 2:23 AM, Bonita Montero wrote:
>>> Am 14.11.2023 um 11:08 schrieb Chris M. Thomasson:
>>>
>>>> So, you are finished with any future on the fly corrections, right?
>>>
>>> I'm currently not missing anything with the code.
>>
>> Okay, can you please make a new post that contains the finished code?
>
> I already posted the finished code in two parts with this little
> correction (missing "!"). The xhandle.h is still the same.
>
>

Correction? Just post the 100% finished code in a brand new thread.

On 2023-11-15, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
> Chris M. Thomasson wrote:
>> On 11/13/2023 9:28 PM, Chris M. Thomasson wrote:
>>> On 11/13/2023 7:27 PM, Kaz Kylheku wrote:
>>>> On 2023-11-14, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo>
>>>> wrote:
>>>>> "as-if" and "according to the scheduling policy (if applicable)" here
>>>>> are important. The standard intent may be to permit only the threads
>>>>> that are "unblocked" contend for the mutex -- as opposed to all
>>>>> non-blocked threads.
>>>>
>>>> It's possible that only one thread is unblocked by a
>>>> pthread_cond_signal. It takes at least two parties to contend for
>>>> something.
>>>>
>>>
>>> A signalling thread does its thing while holding the mutex... Well, it
>>> hears the following playing in the background:
>>>
>>> https://youtu.be/7YvAYIJSSZY
>>
>> A condvar that cannot signal/broadcast from outside of a held mutex is
>> broken by default.
> It can, it's just that the desired scheduling policy cannot be
> *effectively* applied if a program signal outside of that mutex.
>
> E.g. in some scenario a higher-prio thread could stay indefinitely
> starved by the lower-prio thread, regardless of the thread library
> implementation.

I don't see what difference it can actually make, other than in
situations when a thread receives a priority boost while holding a
mutex.

Only one of these is true: (1) either the pthread_cond_signal and
pthread_cond_broadcast functions transfer threads from waiting on the
condition to waiting on the mutex, or (2) they don't: threads are woken,
and have to execute an operation equivalent to pthread_mutex_lock.

(Though there is no mutex argument in the API, the thread is waiting
with respect to a particular mutex. Each thread waiting on the same
condition could nominate a different mutex. So for each thread waiting
on a condition, we know which mutex it released in doing so and can
transfer it to wait on the mutex.)

If the pthread_cond_signal function transfers a thread from the
condition to the mutex according to (1), then everything is cool with
regard to scheduling.

Whether we have this:

pthread_mutex_unlock(&m);
pthread_cond_signal(&c);

...

// re-acquire
pthread_mutex_lock(&m);

or this:

pthread_cond_signal(&c);

pthread_mutex_unlock(&m);

...
// re-acquire
pthread_mutex_lock(&m);

the most important thing is that the waiting threads are already
transferred to waiting on the mutex, before the signaling thread call
tries to re-acquire the mutex.

I.e. the woken threads always reach the mutex wait first, regardless of
these two orders of operations on the part of the signaling thread.

On the other hand, if the pthread_cond_signal operation just wakes up
the threads, such that they themselves have to call the equivalent of
pthread_mutex_lock to reacquire the mutex, we do not gain any special
guarantees in one order or the other. Under either order of the
statements, the signaling thread can snatch the mutex away from the
signaled threads, when reaching the pthread_mutex_lock call,
even if it is a low priority thread and the others are high.

Now with real-time priorities and mutexes that prevent priority
inversion, we could have the following situation:

While a low-priority thread holds a mutex, it is temporarily boosted to
the highest priority from among the set of threads waiting for that
mutex.

There we have a difference between signaling inside or outside, the
reason being that when the thread leaves the mutex to signal outside, it
loses the temporary priority boost. That could lead to an undue delay in
signaling the condition.

The statement in the POSIX standard about predictable scheduling
behavior is like recommending sleep calls to for obtaining more
predictable behavior in a program riddled with race conditions.

If I'm an implementor reading that statement, I cannot infer any
concrete requirements about what it is that I'm supposed to do in
pthread_cond_signal to bring about more predictable scheduling.

Specifications should give specific requirements, not vague opinions.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Am 15.11.2023 um 21:08 schrieb Chris M. Thomasson:

> Are you sure its 100% atomic for use in multi-thread sync algorithms?

If MSVC generates that code it's atomic.

Am 15.11.2023 um 21:21 schrieb Chris M. Thomasson:

> Correction? Just post the 100% finished code in a brand new thread.

I've postet everything that is necessary, you only have to add a "!".

On 11/15/2023 7:00 PM, Bonita Montero wrote:
> Am 15.11.2023 um 21:21 schrieb Chris M. Thomasson:
>
>> Correction? Just post the 100% finished code in a brand new thread.
>
> I've postet everything that is necessary, you only have to add a "!".
>

Wow... Anyway, where do I add this "!"?

On 2023-11-16, Chris M. Thomasson <chris.m.thomasson.1@gmail.com> wrote:
> On 11/15/2023 7:00 PM, Bonita Montero wrote:
>> Am 15.11.2023 um 21:21 schrieb Chris M. Thomasson:
>>
>>> Correction? Just post the 100% finished code in a brand new thread.
>>
>> I've postet everything that is necessary, you only have to add a "!".
>>
>
> Wow... Anyway, where do I add this "!"?

I would guess, in front of "necessary".

Chris M. Thomasson wrote:
> On 11/14/2023 5:26 PM, Pavel wrote:
>> Kaz Kylheku wrote:
>>> On 2023-11-14, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo>
>>> wrote:
>>>> Kaz Kylheku wrote:
>>>>> Because of the situation that the thread which was signaled is
>>>>> trying to acquire the mutex, which, stupidly, the signaling thread
>>>>> is still holding. So, oops, back it goes into suspended state, and we
>>>>> have to context switch to the mutex holder which has to release the
>>>>> mutex and then switch to that signaled thread again.
>>>>>
>>>> Please see my response to your other post. "as-if" calling of
>>>> pthread_mutex_lock does not prescribe actually calling it. Nothing in
>>>> the standard prevents pthread_cond_signal from doing all the job of
>>>> stuffing the unblocked threads to the mutex waiting list in accordance
>>>> with some scheduling policy.
>>>
>>> Have you noticed how no mutex appears in the pthread_cond_signal API?
>>>
>> Of course but so what? Nothing prevents the implementation from
>> listing the temporarily released mutices of the waiting
>> pthread_cond_wait callers in the cond var where pthread_cond_signal
>> can access them.
>>
>> Or, possibly even better, keeping a pointer to them with the threads
>> waiting  for the condvar (there can only be at most one such
>> temporarily released mutex per the waiting thread). And the threads
>> waiting for the condvar are accessible for the pthread_cond_signal
>> according to the spec.
>
> Just as long as the implementation works wrt calling
> pthread_cond_signal/broadcast outside of the mutex. If not, it is broken.

I am unsure where from that discussion of implementation came. The
*spec* permits this (signalling without holding the mutex), just warns
that a predictable scheduling behavior cannot be achieved then.

What would you consider an example of a "non-working" implementation?

Does my example that illustrates how a higher-prio thread may never make
progress is of a broken implementation in your opinion?

Kaz Kylheku wrote:
> On 2023-11-15, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
>> Chris M. Thomasson wrote:
>>> On 11/13/2023 9:28 PM, Chris M. Thomasson wrote:
>>>> On 11/13/2023 7:27 PM, Kaz Kylheku wrote:
>>>>> On 2023-11-14, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo>
>>>>> wrote:
>>>>>> "as-if" and "according to the scheduling policy (if applicable)" here
>>>>>> are important. The standard intent may be to permit only the threads
>>>>>> that are "unblocked" contend for the mutex -- as opposed to all
>>>>>> non-blocked threads.
>>>>>
>>>>> It's possible that only one thread is unblocked by a
>>>>> pthread_cond_signal. It takes at least two parties to contend for
>>>>> something.
>>>>>
>>>>
>>>> A signalling thread does its thing while holding the mutex... Well, it
>>>> hears the following playing in the background:
>>>>
>>>> https://youtu.be/7YvAYIJSSZY
>>>
>>> A condvar that cannot signal/broadcast from outside of a held mutex is
>>> broken by default.
>> It can, it's just that the desired scheduling policy cannot be
>> *effectively* applied if a program signal outside of that mutex.
>>
>> E.g. in some scenario a higher-prio thread could stay indefinitely
>> starved by the lower-prio thread, regardless of the thread library
>> implementation.
>
> I don't see what difference it can actually make, other than in
> situations when a thread receives a priority boost while holding a
> mutex.
>
> Only one of these is true: (1) either the pthread_cond_signal and
> pthread_cond_broadcast functions transfer threads from waiting on the
> condition to waiting on the mutex, or (2) they don't: threads are woken,
> and have to execute an operation equivalent to pthread_mutex_lock.
>
> (Though there is no mutex argument in the API, the thread is waiting
> with respect to a particular mutex. Each thread waiting on the same
> condition could nominate a different mutex. So for each thread waiting
> on a condition, we know which mutex it released in doing so and can
> transfer it to wait on the mutex.)
>
> If the pthread_cond_signal function transfers a thread from the
> condition to the mutex according to (1), then everything is cool with
> regard to scheduling.
>
> Whether we have this:
>
> pthread_mutex_unlock(&m);
> pthread_cond_signal(&c);
>
> ...
>
> // re-acquire
> pthread_mutex_lock(&m);
>
> or this:
>
> pthread_cond_signal(&c);
>
> pthread_mutex_unlock(&m);
>
> ...
> // re-acquire
> pthread_mutex_lock(&m);
>
> the most important thing is that the waiting threads are already
> transferred to waiting on the mutex, before the signaling thread call
> tries to re-acquire the mutex.
>
> I.e. the woken threads always reach the mutex wait first, regardless of
> these two orders of operations on the part of the signaling thread.
>
> On the other hand, if the pthread_cond_signal operation just wakes up
> the threads, such that they themselves have to call the equivalent of
> pthread_mutex_lock to reacquire the mutex, we do not gain any special
> guarantees in one order or the other. Under either order of the
> statements, the signaling thread can snatch the mutex away from the
> signaled threads, when reaching the pthread_mutex_lock call,
> even if it is a low priority thread and the others are high.
>
> Now with real-time priorities and mutexes that prevent priority
> inversion, we could have the following situation:
>
> While a low-priority thread holds a mutex, it is temporarily boosted to
> the highest priority from among the set of threads waiting for that
> mutex.
>
> There we have a difference between signaling inside or outside, the
> reason being that when the thread leaves the mutex to signal outside, it
> loses the temporary priority boost. That could lead to an undue delay in
> signaling the condition.
>
> The statement in the POSIX standard about predictable scheduling
> behavior is like recommending sleep calls to for obtaining more
> predictable behavior in a program riddled with race conditions.
>
> If I'm an implementor reading that statement, I cannot infer any
> concrete requirements about what it is that I'm supposed to do in
> pthread_cond_signal to bring about more predictable scheduling.
>
> Specifications should give specific requirements, not vague opinions.

POSIX *specifies* exactly what scheduling policies a compliant
implementation shall support (SCHED_OTHER, SCHED_FIFO, SCHED_RR, and
SCHED_SPORADIC) and how exactly each of these shall behave.

In addition to specifying the behavior, the spec *explains* the purpose
of some policies so programmers would know when they were meant to be
used. Explanations are there to be helpful, they are not specs. In C++
Standard, they would be formatted as a non-normative Notes.

For example, SCHED_RR, under some conditions, has an effect "to ensure..
that one of them [the threads] does not monopolize the processor". To
me, this is clearly an example of "predictable scheduling behavior".

The spec for pthread_cond_signal and broadast further simply adds one
more condition required to achieve such behavior, namely "the signalling
thread owns the mutex that the waiter threads used to start waiting". If
your doesn't need a predictable scheduling behavior, your app doesn't
have to do that.

For an implementer, this statement is actually very useful: it permits
them to not ensure "predictable scheduling behavior" when the cond
waiter is signaled from a thread not owning the mutex.

Could this statement be thrown away? -- Yes. Would this make
implementor's life easier? -- No, it would make it much harder. The
implementor could be bugged by the user complaints like "Your
implementation must be broken, else why my higher-prio thread is
starving?". With the current spec, they could answer: "This is because
the spec allows my implementation to starve your higher-prio thread to
death -- or exhibit any other unpredictable scheduling behavior -- if
your condvar-signalling thread does not own the mutex that the
higher-prio thread used to start waiting for the condvar".

Am 16.11.2023 um 04:10 schrieb Chris M. Thomasson:

> Wow... Anyway, where do I add this "!"?

You don't have to change it because the Windows code was correct
anyway.

On 2023-11-16, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
> I am unsure where from that discussion of implementation came. The
> *spec* permits this (signalling without holding the mutex), just warns
> that a predictable scheduling behavior cannot be achieved then.

The spec is supposed to tell implementors what to make.

Loose wording like this is dangerous. An implementor is free to
interpret it like this: whenever a program calls pthread_cond_signal,
and it so happens that the calling threads owns no mutex whatsoever, the
implementation is free to set a flag somewhere, based on which it will
willfully damage the integrity of the scheduling implementation somehow,
so that it behaves like crap.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Am 13.11.2023 um 05:50 schrieb Kaz Kylheku:

> Because of the situation that the thread which was signaled is
> trying to acquire the mutex, which, stupidly, the signaling thread
> is still holding. So, oops, back it goes into suspended state, and
> we have to context switch to the mutex holder which has to release
> the mutex and then switch to that signaled thread again.

As I've shown with the comparison of my monitor against a C++ mutex
along with a condition variable there are no additional context swit-
ches with that. The number of voluntary context switches is about the
same as with my monitor, which comprehensibly waits for the mutex and
the condition variable in one step.
As the mutex part and the condition variable part are separate there's
no opportunity to implement that the way I did with a SysV semaphore
set. I guess the backing of the mutex and condition variable on Linux
is the glibc-implementation and it uses signals for that as when you
use a pure mutex; with that you can have an aribitrary state combina-
tion to wake up a thread even if the states are managed separately as
with a mutex and a condition variable.

I simulated that with the follwing code:

#include <iostream>
#include <thread>
#include <vector>
#include <pthread.h>
#include <sys/resource.h>

using namespace std;

int main()
{ constexpr size_t ROUNDS = 1'000;
struct state
{
pthread_mutex_t mtx;
pthread_cond_t cv;
} a, b;
auto init = []( state &s )
{
if( pthread_mutex_init( &s.mtx, nullptr )
|| pthread_cond_init( &s.cv, nullptr ) )
terminate();
};
init( a );
init( b );
auto switches = []()
{
rusage ru;
if( getrusage( RUSAGE_THREAD, &ru ) )
terminate();
return ru.ru_nvcsw;

};
atomic_uint64_t sumSwitches( 0 );
auto thr = [&]( state &my, state &yours )
{
uint64_t before = switches();
for( size_t r = ROUNDS; r--; )
if( pthread_mutex_lock( &my.mtx )
|| pthread_cond_wait( &my.cv, &my.mtx )
|| pthread_mutex_unlock( &my.mtx )
|| pthread_mutex_lock( &yours.mtx )
|| pthread_cond_signal( &yours.cv )
|| pthread_mutex_unlock( &yours.mtx ) )
terminate();
sumSwitches.fetch_add( switches() - before, memory_order_relaxed );
};
vector<jthread> threads;
threads.emplace_back( thr, ref( a ), ref( b ) );
threads.emplace_back( thr, ref( b ), ref( a ) );
if( pthread_cond_signal( &a.cv ) )
terminate();
threads.resize( 0 );
cout << sumSwitches << endl;
}

The result is about 2'000 voluntary context switches.
So no need to signal from outside.

Bonita Montero <Bonita.Montero@gmail.com> writes:
>Am 15.11.2023 um 21:08 schrieb Chris M. Thomasson:
>
>> Are you sure its 100% atomic for use in multi-thread sync algorithms?
>
>If MSVC generates that code it's atomic.

No, if Intel architecture documentation specifies that it is
single-copy atomic, then it is single-copy atomic.

MSVC is just a poor-to-middling C compiler.

Am 16.11.2023 um 16:02 schrieb Scott Lurndal:
> Bonita Montero <Bonita.Montero@gmail.com> writes:
>> Am 15.11.2023 um 21:08 schrieb Chris M. Thomasson:
>>
>>> Are you sure its 100% atomic for use in multi-thread sync algorithms?
>>
>> If MSVC generates that code it's atomic.
>
> No, if Intel architecture documentation specifies that it is
> single-copy atomic, then it is single-copy atomic.

MSVC relies for sure on the right behaviour.

> MSVC is just a poor-to-middling C compiler.

MSVC has the strongest C++20-conformance but the weakest optimizer.

Bonita Montero <Bonita.Montero@gmail.com> writes:
>Am 16.11.2023 um 16:02 schrieb Scott Lurndal:
>> Bonita Montero <Bonita.Montero@gmail.com> writes:
>>> Am 15.11.2023 um 21:08 schrieb Chris M. Thomasson:
>>>
>>>> Are you sure its 100% atomic for use in multi-thread sync algorithms?
>>>
>>> If MSVC generates that code it's atomic.
>>
>> No, if Intel architecture documentation specifies that it is
>> single-copy atomic, then it is single-copy atomic.
>
>MSVC relies for sure on the right behaviour.

Right. Keep thinking that.

>
>> MSVC is just a poor-to-middling C compiler.
>
>MSVC has the strongest C++20-conformance but the weakest optimizer.

Faint praise, indeed. Nobody other than Herb cares about C++20 conformance.

Most real world C++ code relies on C++11 for portability. Most
of our systems and our customers systems don't even support C++17.

On 11/15/2023 9:36 PM, Bonita Montero wrote:
> Am 16.11.2023 um 04:10 schrieb Chris M. Thomasson:
>
>> Wow... Anyway, where do I add this "!"?
>
> You don't have to change it because the Windows code was correct
> anyway.
>

Huh? Do I have to correct your code with that "!" or not?

On 11/12/2023 4:44 PM, Chris M. Thomasson wrote:
> On 11/12/2023 4:29 PM, Pavel wrote:
>> Kaz Kylheku wrote:
>>> On 2023-11-12, Bonita Montero <Bonita.Montero@gmail.com> wrote:
>>>> No, Java and .net keep holding the mutex while doing a notify().
>>>> That's called a Mesa monitor.
>>>
>>> I don't suspect that is part of Mesa semantics. (It's definitely part of
>>> Hoare semantics.) Do you have a reference?
>> She doesn't. See my citation in response to her post for the reference
>> to the contrary.
>>
>>>
>>> Since under Mesa semantics, threads re-acquire the mutex with fewer
>>> guarantees and must re-test the desired condition, Mesa semantics
>>> supports the more efficient protocol of signaling outside of the
>>> monitor.
>>>
>>> If you're using POSIX mutexes and conditions, you should call
>>> pthread_cond_signal and pthread_cond_broadcast outside of the mutex,
>>> whenever possible.
>> This is recommended against by the standard for the same reason why
>> Java implements Hoare monitor behavior. Citation:
>>
>> "
>> The pthread_cond_broadcast() or pthread_cond_signal() functions may be
>> called by a thread whether or  not  it  currently  owns  the  mutex
>> that threads calling pthread_cond_wait() or pthread_cond_timedwait()
>> have associated with the condition variable during their waits;
>> however, if predictable  scheduling  behavior  is required, then that
>> mutex shall be locked by the thread calling pthread_cond_broadcast()
>> or pthread_cond_signal(). > "
>>
>>>
>>> (In a nutshell, if you're going to be telling some thread(s) to go ahead
>>> and grab a mutex, get the hell out of their way first).
>>>
>>
>
> Basically, if you signal while locked then another waiting thread is
> going to try to acquire the mutex that is already locked by the
> signalling thread, instant contention. However, wait morphing techniques
> can be used to handle this since a mutex and a condvar are very intimate
> with each other anyway. Usually, signalling outside of the mutex is ideal.

Actually, for some damn reason this is making me think about the
pass-the-buck algorithm, iirc, its from SUN.

A signal to a thread means the signaled thread already owns the mutex.

Kaz Kylheku wrote:
> On 2023-11-16, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
>> I am unsure where from that discussion of implementation came. The
>> *spec* permits this (signalling without holding the mutex), just warns
>> that a predictable scheduling behavior cannot be achieved then.
>
> The spec is supposed to tell implementors what to make.
>
> Loose wording like this is dangerous. An implementor is free to
> interpret it like this: whenever a program calls pthread_cond_signal,
> and it so happens that the calling threads owns no mutex whatsoever, the
> implementation is free to set a flag somewhere, based on which it will
> willfully damage the integrity of the scheduling implementation somehow,
> so that it behaves like crap.
>
If your complaint is about "predictable scheduling behavior" not being
defined in the standard, I understand and sympathize. It could be better
defined. When POSIX threads were developed, this was kinda obvious
because RT guys required that and they know what they mean by that
(meaning -- more or less agree). This is not a good excuse for not
spelling it in the standard of course.

See if this old posting of David Butenhof, one of the spec authors may
clarify the concept:

https://austin-group-l.opengroup.narkive.com/lKcmfoRI/predictable-scheduling-behavior-in-pthread-cond-broadcast

Am 16.11.2023 um 22:06 schrieb Chris M. Thomasson:

> Huh? Do I have to correct your code with that "!" or not?

Yes, but that was in the #if defined(__unix__) branch.

Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> writes:
>Kaz Kylheku wrote:
>> On 2023-11-16, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
>>> I am unsure where from that discussion of implementation came. The
>>> *spec* permits this (signalling without holding the mutex), just warns
>>> that a predictable scheduling behavior cannot be achieved then.
>>
>> The spec is supposed to tell implementors what to make.
>>
>> Loose wording like this is dangerous. An implementor is free to
>> interpret it like this: whenever a program calls pthread_cond_signal,
>> and it so happens that the calling threads owns no mutex whatsoever, the
>> implementation is free to set a flag somewhere, based on which it will
>> willfully damage the integrity of the scheduling implementation somehow,
>> so that it behaves like crap.
>>
>If your complaint is about "predictable scheduling behavior" not being
>defined in the standard, I understand and sympathize. It could be better
>defined. When POSIX threads were developed, this was kinda obvious
>because RT guys required that and they know what they mean by that
>(meaning -- more or less agree). This is not a good excuse for not
>spelling it in the standard of course.
>
>See if this old posting of David Butenhof, one of the spec authors may
>clarify the concept:
>
>https://austin-group-l.opengroup.narkive.com/lKcmfoRI/predictable-scheduling-behavior-in-pthread-cond-broadcast

When David wrote that, multiprocessor systems were still rare and
limited to very expensive systems (e.g. Sequent) (and for the most
part, weren't running real-time code). I was on the X/Open working group for
a few years during that time period, and one of my collegues at
SGI was part of the 1003.4 working group.

thirty years later, pthread_cond_t is heavily used by non-realtime
code, where such priority inversions (usually very transient) don't
matter.

For the number of cases where pthread_cond_t is used in hard
real-time code, the appropriate techniques (i.e. holding the mutex
while signaling the condition) are available.

On 11/17/2023 8:29 AM, Scott Lurndal wrote:
> Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> writes:
>> Kaz Kylheku wrote:
>>> On 2023-11-16, Pavel <pauldontspamtolk@removeyourself.dontspam.yahoo> wrote:
>>>> I am unsure where from that discussion of implementation came. The
>>>> *spec* permits this (signalling without holding the mutex), just warns
>>>> that a predictable scheduling behavior cannot be achieved then.
>>>
>>> The spec is supposed to tell implementors what to make.
>>>
>>> Loose wording like this is dangerous. An implementor is free to
>>> interpret it like this: whenever a program calls pthread_cond_signal,
>>> and it so happens that the calling threads owns no mutex whatsoever, the
>>> implementation is free to set a flag somewhere, based on which it will
>>> willfully damage the integrity of the scheduling implementation somehow,
>>> so that it behaves like crap.
>>>
>> If your complaint is about "predictable scheduling behavior" not being
>> defined in the standard, I understand and sympathize. It could be better
>> defined. When POSIX threads were developed, this was kinda obvious
>> because RT guys required that and they know what they mean by that
>> (meaning -- more or less agree). This is not a good excuse for not
>> spelling it in the standard of course.
>>
>> See if this old posting of David Butenhof, one of the spec authors may
>> clarify the concept:
>>
>> https://austin-group-l.opengroup.narkive.com/lKcmfoRI/predictable-scheduling-behavior-in-pthread-cond-broadcast
>
> When David wrote that, multiprocessor systems were still rare and
> limited to very expensive systems (e.g. Sequent) (and for the most
> part, weren't running real-time code). I was on the X/Open working group for
> a few years during that time period, and one of my collegues at
> SGI was part of the 1003.4 working group.
>
> thirty years later, pthread_cond_t is heavily used by non-realtime
> code, where such priority inversions (usually very transient) don't
> matter.
>
> For the number of cases where pthread_cond_t is used in hard
> real-time code, the appropriate techniques (i.e. holding the mutex
> while signaling the condition) are available.
>

Exactly. It depends on what the programmer is trying to accomplish.
Signaling inside of a mutex locked region will work regardless of
priority issues, your code will work.

However, _if_ your goal is not real time and/or enforcing some specific
scheduling polity... Then, I would refrain from that as signalling
outside of the mutex is more efficient in this case.

On 2023-11-17, Scott Lurndal <scott@slp53.sl.home> wrote:
>>https://austin-group-l.opengroup.narkive.com/lKcmfoRI/predictable-scheduling-behavior-in-pthread-cond-broadcast
>
> When David wrote that, multiprocessor systems were still rare and

The referenced discussion appears to be from early 2010; maybe
he was reiterating something from an earlier time?

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.