Am 21.09.2023 um 09:58 schrieb Bonita Montero:

>> They are only set to zero if they are not initialized explicitly.

> I'm not talking about what the standad says but how it's implemented
> with all operating systems that support virtual memory.

This shows "all zero" on all modern operating systems:

#include <iostream>
#include <algorithm>

using namespace std;

int main()
{ static char mem[0x1000000];
bool zero = find_if( mem, end( mem ), []( char c ) -> bool { return c;
} ) == end( mem );
cout << (zero ? "all zero" : "has non-zero") << endl;
}

On Thursday, September 21, 2023 at 7:13:45 AM UTC+3, Chris M. Thomasson wrote:
> On 9/20/2023 12:36 PM, Michael S wrote:
> > On Wednesday, September 20, 2023 at 9:55:09 PM UTC+3, Chris M. Thomasson wrote:
> >> On 9/20/2023 9:48 AM, Michael S wrote:
> >>> On Wednesday, September 20, 2023 at 4:18:42 PM UTC+3, Bonita Montero wrote:
> >>>> Am 20.09.2023 um 13:54 schrieb Richard Damon:
> [...]
> > It's unlikely to be unreliable in this particular circumstances.
> > But apart from it, my code has bug (race condition) :(
> > Just shows that this sort of crap is not easy.
> [...]
> Implement your algorithm in Relacy Race Detector. It can help. I know
> the guy who created it. A good guy.

I like my last version, the one that wakes up waiting threads in chained mannner
by means of QueueUserAPC(), better.
It's only disadvantage is higher memory footage - 6 bytes per guarded object instead of 1.
Can be reduced to 5 bytes, but in practice compiler will allocate 8 bytes anyway, so let it
be 6.

On Thursday, September 21, 2023 at 7:46:33 AM UTC+3, Kaz Kylheku wrote:
> On 2023-09-21, Chris M. Thomasson <chris.m.t...@gmail.com> wrote:
> > On 9/20/2023 12:36 PM, Michael S wrote:
> > PulseEvent never worked right.
>
> PulseEvent might have worked under cooperative multitasking (i.e.
> up until Windows 3).
>

Did they have PulseEvent() before Win32 ?
I never studied Win16 API (preferred DOS up until Nt 4) and by now
I can't find Win16 API documentation.

> Under cooperative multitasking, effectively, the running thread holds a
> de-facto global mutex: one which prevents all other threads from
> running. Yielding is like giving up that mutex.
>
> Under cooperative multitasking, certain synchronization objects like
> events effectively behave like condition variables due to the implicit
> mutex.
>
> You know that the thread which calls PulseEvent is the only one running,
> and so after the operation, you know no new threads are waiting on the
> event; no thread gets to run until that thread yields.

It seems to me, co-operative multitasking by itself is not a significant
simplification. What simplifies things dramatically is co-operative multitasking
on single processor.
Single processor part appear more important than co-operative multitasking
part. Implementing PulseEvent() on single processor OS with preemptive
multitasking in user space, but non-preemptive kernel should be quite easy.

> --
> TXR Programming Language: http://nongnu.org/txr
> Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
> Mastodon: @Kazi...@mstdn.ca
> NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On Thursday, September 21, 2023 at 10:58:51 AM UTC+3, Bonita Montero wrote:
> Am 21.09.2023 um 02:22 schrieb Pavel:
>
> > You did not even show that they use mutex at all ...
>
> This doesn't matter, because it's clear that static initialization
> can't work any other way than with a mutex. Yielding is inacceptable
> and spinlocking is inacceptable for reasons I've describet multiple
> times.

Actually, considering that contention is very rare in practice, spinning
with Sleep(1) in the loop body will work adequately, if not ideally.
The rest can be implemented with InterlockedCompareExchange().

So even use of Futex | Critical Section | SRW lock, although it make
a lot of sense in practice, is not necessary from theoretical P.O.V.

> > This is irrelevant to the discussion, but still is wrong:
> There's a non-initialized BSS segment with every executable and every
> shared library. This isn't physically part of the executable file but
> there's only a declared size to reserve. This block is initially com-
> pletely zero. This helps the C++ implementation since every static
> local object has initially an initialized-flag which is zero. This
> is stored along with the object itself.
> > They are only set to zero if they are not initialized explicitly.
> I'm not talking about what the standad says but how it's implemented
> with all operating systems that support virtual memory.

Am 21.09.2023 um 10:38 schrieb Michael S:

> Actually, considering that contention is very rare in practice, ...

The initialization might last an arbitrary time, so spinning
is inacceptable.

> So even use of Futex | Critical Section | SRW lock, although it make
> a lot of sense in practice, is not necessary from theoretical P.O.V.

Why should a SRW-lock make sense here ? If the object isn't initialized
iz can't be read meanwhile. And a futex isn't a replacement for a mutex
but just a faster slow path for a mutex.

On Thursday, September 21, 2023 at 12:03:01 PM UTC+3, Bonita Montero wrote:
> Am 21.09.2023 um 10:38 schrieb Michael S:
>
> > Actually, considering that contention is very rare in practice, ...
>
> The initialization might last an arbitrary time, so spinning
> is inacceptable.

Why is it unacceptable? The impact of needless wake up of the waiting thread
once per tick is negligible in terms of system performance and not too
horrible in terms power consumption. The impact of needless sleep till the
next tick is also insignificant under assumption that initialization took long time.

I'd call it non-ideal when better solutions available, but perfectly acceptable otherwise.

> > So even use of Futex | Critical Section | SRW lock, although it make
> > a lot of sense in practice, is not necessary from theoretical P.O.V.
> Why should a SRW-lock make sense here ?

Because SRW lock in exclusive mode is better critical section than critical
section. On newer versions of Windows critical sections exist purely for
backward compatibility.
That's what I was told unofficially by knowledgeable Microsoft guy.

> If the object isn't initialized
> iz can't be read meanwhile. And a futex isn't a replacement for a mutex
> but just a faster slow path for a mutex.

Are you sure?
My impression was that fast path (uncontended case) is also faster due
to absence of syscall overhead.

On 21/09/2023 11:59, Michael S wrote:
> On Thursday, September 21, 2023 at 12:03:01 PM UTC+3, Bonita Montero wrote:
>> Am 21.09.2023 um 10:38 schrieb Michael S:
>>
>>> Actually, considering that contention is very rare in practice, ...
>>
>> The initialization might last an arbitrary time, so spinning
>> is inacceptable.
>
> Why is it unacceptable? The impact of needless wake up of the waiting thread
> once per tick is negligible in terms of system performance and not too
> horrible in terms power consumption. The impact of needless sleep till the
> next tick is also insignificant under assumption that initialization took long time.
>
> I'd call it non-ideal when better solutions available, but perfectly acceptable otherwise.
>

I don't know why Bonita dislikes spinning and sleeping, but I know why
/I/ don't like it as a solution. Sleeping does not affect thread
priority in the way mutexes do. That means if one low priority thread
starts the initialisation and takes the lock, then gets pre-empted by a
higher priority thread that tries to take the lock, your whole system
can deadlock. (The high priority thread may be sleeping, but there may
be a mid-priority thread that can run and blocks the low-priority thread
from running.)

This is going to be a more likely failure situation if you have fewer
(or only one) cores. But the same compilers and toolchains are used for
a wide variety of systems - you can't pick a solution that might fail on
some systems. And note that these kinds of failures never turn up in
testing - they are rare, and only happen at the worst possible moment
after deployment.

This is particularly a risk for real-time systems and embedded systems,
where you are more likely to have fewer cores (perhaps only one), or
maybe some of your cores are dedicated to real-time tasks and
unavailable for other threads. You will also likely have high priority
and real-time priority threads, which will block the low-priority
threads. And on such systems there might be very serious costs or
safety risks involved in system hangs - you want to be sure that they
are not possible, to the best of practical abilities, rather than
designing in a solution that has a definite non-zero chance of failure.

Using mutexes avoids this problem, because if a high priority thread is
waiting on a mutex held by a low priority thread, the holder's priority
gets boosted to that of the high priority thread until it releases the lock.

Spinning should only be used when you have full control of the
priorities of threads that may take the spin lock, and can be sure it is
safe.

On 9/20/23 11:42 PM, Bonita Montero wrote:
> Am 21.09.2023 um 00:46 schrieb Richard Damon:
>
>> It likely just needs to "Yield" (like call SwitchToThread) to let
>> other threads get a chance to run, and then check the initialization
>> status again.
>
> A yield yould be to inefficient.

Why?

You can't do anything until the object you are waiting to be initialized
gets initialized, so your "efficiency" isn't important.

Remember, the ONLY reason you need to wait is because you came to an
object that is actively in the process of being initialized, so a fairly
rare condition.

If it hasn't been initialized yet, you don't wait, you just claim the
initialization and go, or if the objects initialization is complete, you
just use it.

Also, Yield can't be less efficient than blocking on a Mutex, as the
implementation of the mutex blocking needs to yield too.

As has elsewhere been mentioned, you typically wait a little bit in a
spin wait to see if it will be quick or slow, so "Yield" is only done on
the "Slow" path, and it must be ok to be slow on the slow path, as it is
the slow path.

On 9/21/23 4:03 AM, Bonita Montero wrote:
> Am 21.09.2023 um 09:58 schrieb Bonita Montero:
>
>>> They are only set to zero if they are not initialized explicitly.
>
>> I'm not talking about what the standad says but how it's implemented
>> with all operating systems that support virtual memory.
>
> This shows "all zero" on all modern operating systems:
>
> #include <iostream>
> #include <algorithm>
>
> using namespace std;
>
> int main()
> {
>     static char mem[0x1000000];
>     bool zero = find_if( mem, end( mem ), []( char c ) -> bool {
> return c; } ) == end( mem );
>     cout << (zero ? "all zero" : "has non-zero") << endl;
> }

As it must by the standard.

Note, depending on the OS and implementation, that zeroing will either
be done by having the loaded program image just contain a great big
block of zeros, or (to save image size at the cost of CPU time) the
system might just put "static" objects that need to be zero initialized
into one common segment and zero-fill as an extension of the loading
process, or in some cases the OS can just promise that new segments for
the program are just always zero-filled.

On Thursday, September 21, 2023 at 2:19:59 PM UTC+3, David Brown wrote:
> On 21/09/2023 11:59, Michael S wrote:
> > On Thursday, September 21, 2023 at 12:03:01 PM UTC+3, Bonita Montero wrote:
> >> Am 21.09.2023 um 10:38 schrieb Michael S:
> >>
> >>> Actually, considering that contention is very rare in practice, ...
> >>
> >> The initialization might last an arbitrary time, so spinning
> >> is inacceptable.
> >
> > Why is it unacceptable? The impact of needless wake up of the waiting thread
> > once per tick is negligible in terms of system performance and not too
> > horrible in terms power consumption. The impact of needless sleep till the
> > next tick is also insignificant under assumption that initialization took long time.
> >
> > I'd call it non-ideal when better solutions available, but perfectly acceptable otherwise.
> >
> I don't know why Bonita dislikes spinning and sleeping, but I know why
> /I/ don't like it as a solution. Sleeping does not affect thread
> priority in the way mutexes do. That means if one low priority thread
> starts the initialisation and takes the lock, then gets pre-empted by a
> higher priority thread that tries to take the lock, your whole system
> can deadlock. (The high priority thread may be sleeping, but there may
> be a mid-priority thread that can run and blocks the low-priority thread
> from running.)
>
> This is going to be a more likely failure situation if you have fewer
> (or only one) cores. But the same compilers and toolchains are used for
> a wide variety of systems - you can't pick a solution that might fail on
> some systems. And note that these kinds of failures never turn up in
> testing - they are rare, and only happen at the worst possible moment
> after deployment.
>
> This is particularly a risk for real-time systems and embedded systems,
> where you are more likely to have fewer cores (perhaps only one), or
> maybe some of your cores are dedicated to real-time tasks and
> unavailable for other threads. You will also likely have high priority
> and real-time priority threads, which will block the low-priority
> threads. And on such systems there might be very serious costs or
> safety risks involved in system hangs - you want to be sure that they
> are not possible, to the best of practical abilities, rather than
> designing in a solution that has a definite non-zero chance of failure.
>
> Using mutexes avoids this problem, because if a high priority thread is
> waiting on a mutex held by a low priority thread, the holder's priority
> gets boosted to that of the high priority thread until it releases the lock.
>
> Spinning should only be used when you have full control of the
> priorities of threads that may take the spin lock, and can be sure it is
> safe.

The problem you mention does not apply to general-purpose systems
like Windows, Linux, BSD-linage Unixes, Solaris e.t.c. They all have
built-in avoidance of deadlocks caused by priority-inversion. Most
commonly it's done by applying random priority boosts.

Also I don't see how mutex semantics can possibly help. The common
problem scenario is that low-priority thread that is doing initialization is
preempted when it *does not* hold the mutex. The mutex is held for
two very short durations of flags update. The preemption is far more
likely to happen during the middle phase - constructor itself.

On real-time system with 1 or 2 processors you probably want
completely different design in which a mutex is held during all duration
of initialization, but you can't expect such solution to be included
in g++/clang++/MSVC compiler support libraries.

Which again brings us to the point made by myself and few others:
don't do it! Don't use function-local static objects with constructors.
Or, at least don't use them in multitasking scenarios. Compiler's one
time initialization infrastructure can be super-robust, but being generic
it's unlikely to be optimal answer in any concrete scenario.

Am 21.09.2023 um 11:59 schrieb Michael S:

> Why is it unacceptable? ...

Because the initializing thread might be scheduled away, thereby
keeping other threads spinning.

> I'd call it non-ideal when better solutions available, but perfectly acceptable otherwise.

It's inacceptable.

> Because SRW lock in exclusive mode is better critical section than critical
> section. ...

No, if you only lock exclusively there's no difference.

> Are you sure?

The fast path of mutexes is fast anyway, there's no need for further
speed ups. The futex accellerates the slow path and is a replacement
for the binary semaphore attached to a mutex.

Am 21.09.2023 um 13:19 schrieb David Brown:

> I don't know why Bonita dislikes spinning and sleeping, ...

I only dislike spinning in userspace except when there's a
limit to go for the slow path. The thread holding a spinlock
can be scheduled away an arbitrary time, that's unacceptable.

Am 21.09.2023 um 13:36 schrieb Richard Damon:

> Why?

Because the initialization might only take a microsecond or less
and a yield is a full timeslice, on Linux and Windows usually one
millisecond.

On 21/09/2023 14:01, Michael S wrote:
> On Thursday, September 21, 2023 at 2:19:59 PM UTC+3, David Brown wrote:
>> On 21/09/2023 11:59, Michael S wrote:
>>> On Thursday, September 21, 2023 at 12:03:01 PM UTC+3, Bonita Montero wrote:
>>>> Am 21.09.2023 um 10:38 schrieb Michael S:
>>>>
>>>>> Actually, considering that contention is very rare in practice, ...
>>>>
>>>> The initialization might last an arbitrary time, so spinning
>>>> is inacceptable.
>>>
>>> Why is it unacceptable? The impact of needless wake up of the waiting thread
>>> once per tick is negligible in terms of system performance and not too
>>> horrible in terms power consumption. The impact of needless sleep till the
>>> next tick is also insignificant under assumption that initialization took long time.
>>>
>>> I'd call it non-ideal when better solutions available, but perfectly acceptable otherwise.
>>>
>> I don't know why Bonita dislikes spinning and sleeping, but I know why
>> /I/ don't like it as a solution. Sleeping does not affect thread
>> priority in the way mutexes do. That means if one low priority thread
>> starts the initialisation and takes the lock, then gets pre-empted by a
>> higher priority thread that tries to take the lock, your whole system
>> can deadlock. (The high priority thread may be sleeping, but there may
>> be a mid-priority thread that can run and blocks the low-priority thread
>> from running.)
>>
>> This is going to be a more likely failure situation if you have fewer
>> (or only one) cores. But the same compilers and toolchains are used for
>> a wide variety of systems - you can't pick a solution that might fail on
>> some systems. And note that these kinds of failures never turn up in
>> testing - they are rare, and only happen at the worst possible moment
>> after deployment.
>>
>> This is particularly a risk for real-time systems and embedded systems,
>> where you are more likely to have fewer cores (perhaps only one), or
>> maybe some of your cores are dedicated to real-time tasks and
>> unavailable for other threads. You will also likely have high priority
>> and real-time priority threads, which will block the low-priority
>> threads. And on such systems there might be very serious costs or
>> safety risks involved in system hangs - you want to be sure that they
>> are not possible, to the best of practical abilities, rather than
>> designing in a solution that has a definite non-zero chance of failure.
>>
>> Using mutexes avoids this problem, because if a high priority thread is
>> waiting on a mutex held by a low priority thread, the holder's priority
>> gets boosted to that of the high priority thread until it releases the lock.
>>
>> Spinning should only be used when you have full control of the
>> priorities of threads that may take the spin lock, and can be sure it is
>> safe.
>
> The problem you mention does not apply to general-purpose systems
> like Windows, Linux, BSD-linage Unixes, Solaris e.t.c. They all have
> built-in avoidance of deadlocks caused by priority-inversion. Most
> commonly it's done by applying random priority boosts.

As far as I know, such random priority boosts will not boost a
non-realtime priority thread to real-time priority on Linux (and
presumably not on other systems). That would negate the whole concept
of realtime priority threads.

>
> Also I don't see how mutex semantics can possibly help. The common
> problem scenario is that low-priority thread that is doing initialization is
> preempted when it *does not* hold the mutex. The mutex is held for
> two very short durations of flags update. The preemption is far more
> likely to happen during the middle phase - constructor itself.
>
> On real-time system with 1 or 2 processors you probably want
> completely different design in which a mutex is held during all duration
> of initialization, but you can't expect such solution to be included
> in g++/clang++/MSVC compiler support libraries.
>

Yes, you would need the mutex to be held during critical stages of the
initialisation.

I can understand that this would be expensive compared to spinlocks that
would work fine for "normal" systems (especially with the random boosts
you described). And I understand that toolchains have to be optimised
for normal situations, not things that are extremely rare even in
particular niche situations.

But getting threading, locking and synchronisation details right is very
difficult - most programmers don't get it right. But you can't tell
that you have a problem by testing the code, as failures typically
require extreme bad luck in timing. So it always worries me when I see
something that is hidden, which most programmers will assume "just
works, by some compiler magic", but which can be a problem in some
circumstances.

(As another example, gcc's "libatomic" uses spinlocks - if you use
std::atomic types on a microcontroller in situations where atomics would
be useful, they will hang your system any time there is a coincidental
access.)

> Which again brings us to the point made by myself and few others:
> don't do it! Don't use function-local static objects with constructors.
> Or, at least don't use them in multitasking scenarios. Compiler's one
> time initialization infrastructure can be super-robust, but being generic
> it's unlikely to be optimal answer in any concrete scenario.
>

Yes, that is one option. You can also use "constinit" statics safely.
And gcc has a "-fno-threadsafe-statics" option (also useable as a
pragma, but unfortunately not as a neater __attribute__) that puts the
responsibility of ensuring things are threadsafe onto the user.

On 21/09/2023 14:40, Bonita Montero wrote:
> Am 21.09.2023 um 13:36 schrieb Richard Damon:
>
>> Why?
>
> Because the initialization might only take a microsecond or less
> and a yield is a full timeslice, on Linux and Windows usually one
> millisecond.

"Yield" won't take a timeslice unless there is something else of the
same priority, waiting to run on the same core. And if that's the case,
then fine - that thread is clearly equally important.

Am 21.09.2023 um 15:05 schrieb David Brown:

> "Yield" won't take a timeslice unless there is something else of the
> same priority, waiting to run on the same core. ...

Ok, you're right, but the code would be still unacceptable under load.

On Thursday, September 21, 2023 at 4:03:46 PM UTC+3, David Brown wrote:
> On 21/09/2023 14:01, Michael S wrote:
> > On Thursday, September 21, 2023 at 2:19:59 PM UTC+3, David Brown wrote:
> >> On 21/09/2023 11:59, Michael S wrote:
> >>> On Thursday, September 21, 2023 at 12:03:01 PM UTC+3, Bonita Montero wrote:
> >>>> Am 21.09.2023 um 10:38 schrieb Michael S:
> >>>>
> >>>>> Actually, considering that contention is very rare in practice, ...
> >>>>
> >>>> The initialization might last an arbitrary time, so spinning
> >>>> is inacceptable.
> >>>
> >>> Why is it unacceptable? The impact of needless wake up of the waiting thread
> >>> once per tick is negligible in terms of system performance and not too
> >>> horrible in terms power consumption. The impact of needless sleep till the
> >>> next tick is also insignificant under assumption that initialization took long time.
> >>>
> >>> I'd call it non-ideal when better solutions available, but perfectly acceptable otherwise.
> >>>
> >> I don't know why Bonita dislikes spinning and sleeping, but I know why
> >> /I/ don't like it as a solution. Sleeping does not affect thread
> >> priority in the way mutexes do. That means if one low priority thread
> >> starts the initialisation and takes the lock, then gets pre-empted by a
> >> higher priority thread that tries to take the lock, your whole system
> >> can deadlock. (The high priority thread may be sleeping, but there may
> >> be a mid-priority thread that can run and blocks the low-priority thread
> >> from running.)
> >>
> >> This is going to be a more likely failure situation if you have fewer
> >> (or only one) cores. But the same compilers and toolchains are used for
> >> a wide variety of systems - you can't pick a solution that might fail on
> >> some systems. And note that these kinds of failures never turn up in
> >> testing - they are rare, and only happen at the worst possible moment
> >> after deployment.
> >>
> >> This is particularly a risk for real-time systems and embedded systems,
> >> where you are more likely to have fewer cores (perhaps only one), or
> >> maybe some of your cores are dedicated to real-time tasks and
> >> unavailable for other threads. You will also likely have high priority
> >> and real-time priority threads, which will block the low-priority
> >> threads. And on such systems there might be very serious costs or
> >> safety risks involved in system hangs - you want to be sure that they
> >> are not possible, to the best of practical abilities, rather than
> >> designing in a solution that has a definite non-zero chance of failure..
> >>
> >> Using mutexes avoids this problem, because if a high priority thread is
> >> waiting on a mutex held by a low priority thread, the holder's priority
> >> gets boosted to that of the high priority thread until it releases the lock.
> >>
> >> Spinning should only be used when you have full control of the
> >> priorities of threads that may take the spin lock, and can be sure it is
> >> safe.
> >
> > The problem you mention does not apply to general-purpose systems
> > like Windows, Linux, BSD-linage Unixes, Solaris e.t.c. They all have
> > built-in avoidance of deadlocks caused by priority-inversion. Most
> > commonly it's done by applying random priority boosts.
> As far as I know, such random priority boosts will not boost a
> non-realtime priority thread to real-time priority on Linux (and
> presumably not on other systems). That would negate the whole concept
> of realtime priority threads.

If you have enough of ready real time threads to occupy all CPUs for
unacceptably long time then, indeed, you have a problem.
But to me it looks unlikely that deadlock due to priority inversion is your
main problem in such scenario.

> >
> > Also I don't see how mutex semantics can possibly help. The common
> > problem scenario is that low-priority thread that is doing initialization is
> > preempted when it *does not* hold the mutex. The mutex is held for
> > two very short durations of flags update. The preemption is far more
> > likely to happen during the middle phase - constructor itself.
> >
> > On real-time system with 1 or 2 processors you probably want
> > completely different design in which a mutex is held during all duration
> > of initialization, but you can't expect such solution to be included
> > in g++/clang++/MSVC compiler support libraries.
> >
> Yes, you would need the mutex to be held during critical stages of the
> initialisation.
>
> I can understand that this would be expensive compared to spinlocks that
> would work fine for "normal" systems (especially with the random boosts
> you described).

What's advocated here by most posters and what's seems to be implemented
by major toolchains is not a choice between speenlocks and one or pair of mutexes
or equivalents. The spinlocks is mere something that we mention to argue against
Bonita's suggestion that the C++ Standard in its current form can be impossible
to implement.
Bonita aside, the real choice is between 4 options:
(A) Mutex per object. Held for all duration of the constructor.

(B) Serializing all initialization of function-local static objects with one global mutex.
Again, the mutex held throughout all duration of the constructor.

(C) Few flags and possible a little more of auxiliary states (like thread Id in my
proposed solution) per objects. Access to flags is guarded by global mutex,
but initialization itself is not guarded. The are several possible solution for
wake-up of contending threads of waiting for finish of initialization. Among such
solutions, spinlocks, either classical or amended by sleep() call are possible, but
least likely on full-featured OSes. The most natural solution [on f-f OS] is wait on
per-object conditional variable with global mutex.

(D) Creative combinations of A and B, e.g. small pool of mutexs distributed
dynamically between objects. Like A and B, mutex held throughout all duration
of the constructor.

All major toolchains appear to use variants of (C)

> And I understand that toolchains have to be optimised
> for normal situations, not things that are extremely rare even in
> particular niche situations.
>
> But getting threading, locking and synchronisation details right is very
> difficult - most programmers don't get it right. But you can't tell
> that you have a problem by testing the code, as failures typically
> require extreme bad luck in timing. So it always worries me when I see
> something that is hidden, which most programmers will assume "just
> works, by some compiler magic", but which can be a problem in some
> circumstances.
>
> (As another example, gcc's "libatomic" uses spinlocks - if you use
> std::atomic types on a microcontroller in situations where atomics would
> be useful, they will hang your system any time there is a coincidental
> access.)
> > Which again brings us to the point made by myself and few others:
> > don't do it! Don't use function-local static objects with constructors.
> > Or, at least don't use them in multitasking scenarios. Compiler's one
> > time initialization infrastructure can be super-robust, but being generic
> > it's unlikely to be optimal answer in any concrete scenario.
> >
> Yes, that is one option. You can also use "constinit" statics safely.

I am not aware of constinit. Is it something new, like C++14 or later?

> And gcc has a "-fno-threadsafe-statics" option (also useable as a
> pragma, but unfortunately not as a neater __attribute__) that puts the
> responsibility of ensuring things are threadsafe onto the user.

On Thursday, September 21, 2023 at 3:40:37 PM UTC+3, Bonita Montero wrote:
> Am 21.09.2023 um 13:36 schrieb Richard Damon:
>
> > Why?
>
> Because the initialization might only take a microsecond or less
> and a yield is a full timeslice, on Linux and Windows usually one
> millisecond.

If initialization is short then contention is very unlikely. Wasting a
a full timeslice once in a blue moon is o.k.
If initialization is long then contention is likely, however wasting a
a full timeslice after long initialization is o.k.
Not an excellent solution, but acceptable.

Richard Damon <Richard@Damon-Family.org> writes:
>On 9/21/23 4:03 AM, Bonita Montero wrote:
>> Am 21.09.2023 um 09:58 schrieb Bonita Montero:
>>
>>>> They are only set to zero if they are not initialized explicitly.
>>
>>> I'm not talking about what the standad says but how it's implemented
>>> with all operating systems that support virtual memory.

Nonsense. As Pavel pointed out in the part this thread you
snipped, both pthread_once and pthread_mutex can be statically
initialized (and the value may or may not be zero).

>>
>> This shows "all zero" on all modern operating systems:

You're changing the topic. BSS has been guaranteed to
be initialized to zero since long before windows existed.

>Note, depending on the OS and implementation, that zeroing will either
>be done by having the loaded program image just contain a great big
>block of zeros, or (to save image size at the cost of CPU time) the
>system might just put "static" objects that need to be zero initialized
>into one common segment and zero-fill as an extension of the loading
>process, or in some cases the OS can just promise that new segments for
>the program are just always zero-filled.

If the bss section is page-aligned, demand paging will ensure
that the contents are zeroed before being accessed.

Bonita Montero <Bonita.Montero@gmail.com> writes:
>Am 21.09.2023 um 10:38 schrieb Michael S:
>
>> Actually, considering that contention is very rare in practice, ...
>
>The initialization might last an arbitrary time, so spinning
>is inacceptable.

It's rare in practice because good programmers very seldom[*] use
non-global static variables in threaded applications.

[*] Effectively never.

Am 21.09.2023 um 16:48 schrieb Michael S:

> If initialization is short then contention is very unlikely.

Maybe, but no one would cose this solutions if mutexes are available.
Yielding in this situation is crap.

Am 21.09.2023 um 16:56 schrieb Scott Lurndal:

> It's rare in practice because good programmers very seldom[*]
> use non-global static variables in threaded applications.

It's just your taste that this should be avoided.

On 21/09/2023 16:30, Michael S wrote:
> On Thursday, September 21, 2023 at 4:03:46 PM UTC+3, David Brown wrote:
>> On 21/09/2023 14:01, Michael S wrote:
>>> On Thursday, September 21, 2023 at 2:19:59 PM UTC+3, David Brown wrote:
>>>> On 21/09/2023 11:59, Michael S wrote:
>>>>> On Thursday, September 21, 2023 at 12:03:01 PM UTC+3, Bonita Montero wrote:
>>>>>> Am 21.09.2023 um 10:38 schrieb Michael S:
>>>>>>
>>>>>>> Actually, considering that contention is very rare in practice, ...
>>>>>>
>>>>>> The initialization might last an arbitrary time, so spinning
>>>>>> is inacceptable.
>>>>>
>>>>> Why is it unacceptable? The impact of needless wake up of the waiting thread
>>>>> once per tick is negligible in terms of system performance and not too
>>>>> horrible in terms power consumption. The impact of needless sleep till the
>>>>> next tick is also insignificant under assumption that initialization took long time.
>>>>>
>>>>> I'd call it non-ideal when better solutions available, but perfectly acceptable otherwise.
>>>>>
>>>> I don't know why Bonita dislikes spinning and sleeping, but I know why
>>>> /I/ don't like it as a solution. Sleeping does not affect thread
>>>> priority in the way mutexes do. That means if one low priority thread
>>>> starts the initialisation and takes the lock, then gets pre-empted by a
>>>> higher priority thread that tries to take the lock, your whole system
>>>> can deadlock. (The high priority thread may be sleeping, but there may
>>>> be a mid-priority thread that can run and blocks the low-priority thread
>>>> from running.)
>>>>
>>>> This is going to be a more likely failure situation if you have fewer
>>>> (or only one) cores. But the same compilers and toolchains are used for
>>>> a wide variety of systems - you can't pick a solution that might fail on
>>>> some systems. And note that these kinds of failures never turn up in
>>>> testing - they are rare, and only happen at the worst possible moment
>>>> after deployment.
>>>>
>>>> This is particularly a risk for real-time systems and embedded systems,
>>>> where you are more likely to have fewer cores (perhaps only one), or
>>>> maybe some of your cores are dedicated to real-time tasks and
>>>> unavailable for other threads. You will also likely have high priority
>>>> and real-time priority threads, which will block the low-priority
>>>> threads. And on such systems there might be very serious costs or
>>>> safety risks involved in system hangs - you want to be sure that they
>>>> are not possible, to the best of practical abilities, rather than
>>>> designing in a solution that has a definite non-zero chance of failure.
>>>>
>>>> Using mutexes avoids this problem, because if a high priority thread is
>>>> waiting on a mutex held by a low priority thread, the holder's priority
>>>> gets boosted to that of the high priority thread until it releases the lock.
>>>>
>>>> Spinning should only be used when you have full control of the
>>>> priorities of threads that may take the spin lock, and can be sure it is
>>>> safe.
>>>
>>> The problem you mention does not apply to general-purpose systems
>>> like Windows, Linux, BSD-linage Unixes, Solaris e.t.c. They all have
>>> built-in avoidance of deadlocks caused by priority-inversion. Most
>>> commonly it's done by applying random priority boosts.
>> As far as I know, such random priority boosts will not boost a
>> non-realtime priority thread to real-time priority on Linux (and
>> presumably not on other systems). That would negate the whole concept
>> of realtime priority threads.
>
> If you have enough of ready real time threads to occupy all CPUs for
> unacceptably long time then, indeed, you have a problem.
> But to me it looks unlikely that deadlock due to priority inversion is your
> main problem in such scenario.
>

I always want to think through possible issues when it comes to
multi-threading - no matter how unlikely.

It is certainly not uncommon to have more ready real-time threads than
cpus in real-time systems - though perhaps not on real-time Linux
systems. In microcontroller systems, your OS (if you have one) is
usually an RTOS, and most of the threads are real-time (of different
priorities). And you usually only have one, occasionally two, cpu.

But I think for that kind of system, your recommended solution - don't
have function-local statics that need such initialisation
synchronisation - is far and away the most common solution. (It's
certainly the one I use myself.) Function-local statics with dynamic
initialisation have quite a bit of overhead, both for initialisation and
in use.

>>>
>>> Also I don't see how mutex semantics can possibly help. The common
>>> problem scenario is that low-priority thread that is doing initialization is
>>> preempted when it *does not* hold the mutex. The mutex is held for
>>> two very short durations of flags update. The preemption is far more
>>> likely to happen during the middle phase - constructor itself.
>>>
>>> On real-time system with 1 or 2 processors you probably want
>>> completely different design in which a mutex is held during all duration
>>> of initialization, but you can't expect such solution to be included
>>> in g++/clang++/MSVC compiler support libraries.
>>>
>> Yes, you would need the mutex to be held during critical stages of the
>> initialisation.
>>
>> I can understand that this would be expensive compared to spinlocks that
>> would work fine for "normal" systems (especially with the random boosts
>> you described).
>
> What's advocated here by most posters and what's seems to be implemented
> by major toolchains is not a choice between speenlocks and one or pair of mutexes
> or equivalents. The spinlocks is mere something that we mention to argue against
> Bonita's suggestion that the C++ Standard in its current form can be impossible
> to implement.
> Bonita aside, the real choice is between 4 options:
> (A) Mutex per object. Held for all duration of the constructor.
>
> (B) Serializing all initialization of function-local static objects with one global mutex.
> Again, the mutex held throughout all duration of the constructor.
>
> (C) Few flags and possible a little more of auxiliary states (like thread Id in my
> proposed solution) per objects. Access to flags is guarded by global mutex,
> but initialization itself is not guarded. The are several possible solution for
> wake-up of contending threads of waiting for finish of initialization. Among such
> solutions, spinlocks, either classical or amended by sleep() call are possible, but
> least likely on full-featured OSes. The most natural solution [on f-f OS] is wait on
> per-object conditional variable with global mutex.
>
> (D) Creative combinations of A and B, e.g. small pool of mutexs distributed
> dynamically between objects. Like A and B, mutex held throughout all duration
> of the constructor.
>
> All major toolchains appear to use variants of (C)

I must have a look at the language support library provided with the
embedded (32-bit arm-none-eabi-gcc) I use. (D) seems the natural choice
there, with less risk of conflict than (B) while avoiding the space
costs of (A). But my guess is that is actually done like (D) but with
simple spinlocks, since the compiler support library is independent of
the RTOS and therefore does not have access to mutexes.

Looking at generated code (using godbolt.org) for Linux, it seems there
is just a single byte boolean guard per object - there is no thread id
or anything else stored per object. I don't know what is going on
inside "__cxa_guard_acquire" and other such functions, however.

>
>> And I understand that toolchains have to be optimised
>> for normal situations, not things that are extremely rare even in
>> particular niche situations.
>>
>> But getting threading, locking and synchronisation details right is very
>> difficult - most programmers don't get it right. But you can't tell
>> that you have a problem by testing the code, as failures typically
>> require extreme bad luck in timing. So it always worries me when I see
>> something that is hidden, which most programmers will assume "just
>> works, by some compiler magic", but which can be a problem in some
>> circumstances.
>>
>> (As another example, gcc's "libatomic" uses spinlocks - if you use
>> std::atomic types on a microcontroller in situations where atomics would
>> be useful, they will hang your system any time there is a coincidental
>> access.)
>>> Which again brings us to the point made by myself and few others:
>>> don't do it! Don't use function-local static objects with constructors.
>>> Or, at least don't use them in multitasking scenarios. Compiler's one
>>> time initialization infrastructure can be super-robust, but being generic
>>> it's unlikely to be optimal answer in any concrete scenario.
>>>
>> Yes, that is one option. You can also use "constinit" statics safely.
>
> I am not aware of constinit. Is it something new, like C++14 or later?

On 9/21/2023 1:04 AM, Michael S wrote:
> On Thursday, September 21, 2023 at 7:13:45 AM UTC+3, Chris M. Thomasson wrote:
>> On 9/20/2023 12:36 PM, Michael S wrote:
>>> On Wednesday, September 20, 2023 at 9:55:09 PM UTC+3, Chris M. Thomasson wrote:
>>>> On 9/20/2023 9:48 AM, Michael S wrote:
>>>>> On Wednesday, September 20, 2023 at 4:18:42 PM UTC+3, Bonita Montero wrote:
>>>>>> Am 20.09.2023 um 13:54 schrieb Richard Damon:
>> [...]
>>> It's unlikely to be unreliable in this particular circumstances.
>>> But apart from it, my code has bug (race condition) :(
>>> Just shows that this sort of crap is not easy.
>> [...]
>> Implement your algorithm in Relacy Race Detector. It can help. I know
>> the guy who created it. A good guy.
>
> I like my last version, the one that wakes up waiting threads in chained mannner
> by means of QueueUserAPC(), better.
> It's only disadvantage is higher memory footage - 6 bytes per guarded object instead of 1.
> Can be reduced to 5 bytes, but in practice compiler will allocate 8 bytes anyway, so let it
> be 6.

Here is Relacy, a pretty nice way to model sync algorithms:

https://www.1024cores.net/home/relacy-race-detector

On 9/21/2023 12:47 AM, Bonita Montero wrote:
> Am 21.09.2023 um 06:24 schrieb Chris M. Thomasson:
>
>> On 9/20/2023 8:42 PM, Bonita Montero wrote:
>
>>> A yield yould be to inefficient.
>
>> Have you heard about an adaptive backoff, different than a
>> "traditional" adaptive mutex? ...
>
> We discussed yield with periodic polling.
>

An adaptive backoff tries to automatically set its spin limit based on
some statistics it gathers dynamically during runtime, aka before it
waits in the kernel. One way to do it, a very interesting way imvho, is
instead of spinning and yielding, it tries to do some other unrelated
work instead. Pretty nice, actually. Something akin to:

while (! try_lock_or_whatever())
{ if (! try_to_do_some_other_unrelated_work())
{
// wait in the kernel, really slow path...
}
}

{
// We are locked! :^D
}

unlock();

If your program can handle something like this, it actually works pretty
good...