On 9/17/23 3:29 PM, Bonita Montero wrote:
> Am 17.09.2023 um 21:20 schrieb Kaz Kylheku:
>
>> Locking a page is just a state change, not a resource allocation.
>
> That's a completely different discussion. The standard should not
> be based on individual implementations, but should allow for the
> possibility that the failure of synchronization or the synchroni-
> zation itself fails. The standard also provides for this with a
> std::mutex, why not for the initialization of statically local
> objects ?
>
> Rest unread.
>

Why does it need to take into account that an BAD implementation might fail.

Since it HAS been shown that you can do the operation correctly without
failure, then allowing failure is just pandering to bad implementations.

The Standard is written so it is useful for programmers, and achievable
by implementations.

On 2023-09-17, Bonita Montero <Bonita.Montero@gmail.com> wrote:
> Am 17.09.2023 um 21:20 schrieb Kaz Kylheku:
>
>> Locking a page is just a state change, not a resource allocation.
>
> That's a completely different discussion.

It's one that you started where you made false claims that futex
operations have to acquire resources, such as wait queues, which can
fail.

I spent about two minutes of looking at kernel source to trivially
discover that it's false.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 9/17/23 2:51 PM, Bonita Montero wrote:
> Am 17.09.2023 um 20:49 schrieb Kaz Kylheku:
>
>> When mutexes fail, it is due to bugs in the application.  In that case,
>> if that happens at global init time, the program can just abort.
>
> C++ mutexes always have a delayed initialitation since the constructor
> is noexcept. So the semaphore behind the mutex which backs contention
> is created on synchonzation, which may fail.
>
>> In POSIX, ..
>
> It doesn't matter what a specific operating system says about that,
> the standard should honour a failure in this case, even more if you
> have deferred initialization with C++ mutexes.
>
> Rest unread.
>

So, you say that it doesn't matter what a specific operating system
says, except then you think it should.

Make up your mind.

On 9/17/2023 12:12 PM, Bonita Montero wrote:
> Am 17.09.2023 um 21:01 schrieb Chris M. Thomasson:
>
>> A slow-path in a synchronization algorithm can cause it to choose to
>> wait in the kernel, ...
>
> Futexes are a substitute for a semaphore, which in turn is a part of
> a mutex.

Yawn.

> There must be a way to get a thread sleeping when there's
> contention. So there must be kernel-support of a futex.
> If you say that everything on that futex happens in userspace this
> does just mean that there's no explicit kernel call from userspace,
> but the kernel-transition is done through a MMU-trap, wich is usually
> faster on most CPUs than a explicit kernel call.
>
>> No. Bonita. You need to study up.
>
> I've read the paper, you didn't understand or you didn't read it.
>

I still don't think you know how they work.

Bonita Montero wrote:
> Am 17.09.2023 um 19:37 schrieb Pavel:
>
>> Bonita Montero wrote:
>>> As I've shown with my program MSVC, libstdc++ (g++) and libc++ (clang++)
>>> have per object mutexes.
>
>> No your program does not show this. ...
>
> Of course I show this. I've got a generic lambda and each thread
> gets its own instantiation of the calling operator of that lambda,
> thereby having _individual_ static objects per calling operator.
>
Of course you don't show that the synchronization primitives that are
used for static initialization are mutexes.

And of course you don't show that the synchronization primitives are
used in a way that they may fail.

On 9/17/2023 12:20 PM, Kaz Kylheku wrote:
> On 2023-09-17, Bonita Montero <Bonita.Montero@gmail.com> wrote:
>> Am 17.09.2023 um 20:25 schrieb Chris M. Thomasson:
>>
>>> Getting WAIT_FAILED means something really bad has hit the fan.
>>> WAIT_ABANDONED is fine and can be handled, akin to EOWNERDEAD.
>>> This is for inter-process robust sync.
>>
>> It's unspecified why this happens.
>>
>>> A futex does not need to create new kernel resources on demand.
>>> They are already there.
>>
>> Read the Linux futex paper. The page which holds the futex is locked
>> and there's a wating queue for contenders inside the kernel.
>
> Locking a page is just a state change, not a resource allocation.
>
> Waiting on wait queues doesn't require external resources; Linux wait
> queue nodes are allocated and initialized on the stack. The enqueueing
> and dequeueing is just pointer operations.
>
> Wait queues themselves have to be allocated somewhere. The Linux futex
> subsystem pre-allocated a big hash table containing queues. A futex
> is hashed and then arbitrarily mapped to a queue which already exists.
> The operation does not fail; there is no error checking or handling in
> that code path at all. The futex hash allows unrelated futexes to map to
> the same wait queue. That means that when a futex is woken up, tasks
> waiting for a different futex may be woken as a side effect.
>
> A page which holds a futex might be initially not present and have
> to be swapped in. That's a resource allocation that can fail due to OOM.
> However, that threat exists for any not-present page in an overcommitted
> VM system; it's not a futex problem. You can malloc a block of memory,
> get a non-null pointer, only to have it fail on access later.
>

Afaict, and iirc, you are right on par here. Thanks. :^)

Iirc, a futex tends to broadcast to get around hash collisions. Thus the
user algorithm must be based on a predicate that it can loop around and
check _before_ it decides to wait again in the kernel via futex.

Bonita Montero wrote:
> Am 17.09.2023 um 20:37 schrieb Richard Damon:
>
>> You haven't actually looked at HOW the code actually does what it
>> does, so you haven't shown anything (except your own limitations in
>> understanding code).
>
> I don't need to check that. I've proven that each static local
> object has its own initialization and not a shared mutex because
> the code runs exactly one second.
so far so good

> So if you have an indivivual
> mutex
how do your result show that this was a mutex?

> per static local object this synchronization may fail.
Even if it were a mutex (which you did not prove) if it is a statically
constant-initialized individual mutex, why would it fail?

> A pre-allocated global mutex for all static local initialization
> may not fail,
And in exactly same way, an individual per-static variable mutex does
not have to have code path to fail.

> but the standard should honour platforms where
> this actually can happen.
The standard does not have to honor any platform. Many platforms cannot
fully implement C++ standard; it is not the standard's fault (and often
not the platform's fault if the platform does not need the full
implementation or it would come at unreasonable cost).

On 9/17/2023 11:59 AM, Bonita Montero wrote:
> Am 17.09.2023 um 20:53 schrieb Chris M. Thomasson:
>> On 9/17/2023 11:51 AM, Bonita Montero wrote:
>>> Am 17.09.2023 um 20:49 schrieb Kaz Kylheku:
>>>
>>>> When mutexes fail, it is due to bugs in the application.  In that case,
>>>> if that happens at global init time, the program can just abort.
>>>
>>> C++ mutexes always have a delayed initialitation since the constructor
>>> is noexcept. So the semaphore behind the mutex which backs contention
>>> is created on synchonzation, which may fail.
>>
>> Wtf are you rambling on about!
>
> You alleged me to not understand futexes and you don't understand
> that yourself.

I only wrote that I think you are totally misunderstanding (or have a
rather _limited_ experience with them, Bonita) futexes because I read
what _you_ wrote here in this thread. See?

For some damn reason, you seem to think that A must be implemented in
the way _you_ think it should be. Then _you_ proclaim that as the "only
one way", so to speak. Strange to me. Anyway... You need to learn more
about futexes, the difference between a fast-path and a slow-path,
basically, synchronization in general...

You code attacking a straw man is cruel to all scarecrows. ;^o

> Ask yourself how a mutex handles contention so that
> a thread sleeps while another is processing.

I personally consider contention to be a so-called slow-path. A mutex
can try to spin a couple times on its atomic predicate to see if it can
successfully lock _before_ actually waiting in the kernel... If it spins
a couples times and acquires the mutex, well, it just avoided a kernel
call! Good. Adaptive mutexes are a tradeoff wrt their spin count
thresholds, another story in and of itself... Anyway, if it (a contended
mutex) failed enough times, it can finally decide to wait on a futex, a
binary semaphore, ect...

I have implemented many different ways that a mutex can be created. A
fun one uses a bakery algorithm and is 100% loop free in its logic.

Bonita, you need to study up.

On 9/11/2023 8:58 PM, Pavel wrote:
> Chris M. Thomasson wrote:
>> On 9/10/2023 3:50 PM, Pavel wrote:
>>> Richard Damon wrote:
>>>> On 9/10/23 1:57 AM, Bonita Montero wrote:
>>>>> Am 09.09.2023 um 22:37 schrieb Richard Damon:
>>>>>
>>>>>> The Standard defines the final results that must happen
>>>>>> (or possibilities that can happen in some cases).
>>>>>
>>>>> The standards also should consider side effects as in this example
>>>>> when a static initialization would fail because of a synchronization
>>>>> error.
>>>>>
>>>> You don't seem to understand that sincd it CAN be done in a way that
>>>> always works, any method that doesn't always work is just WRONG and
>>>> non-conforming.
>>>>
>>>> I guess you think that 1 + 1 must be allowed to be 1 in case of
>>>> "synchronization errors".
>>>>
>>>> Static Initialization has DEFINED behavior, and that behavior must
>>>> be honored, and the implementation do what is need to make that happen.
>>>>
>>>> Methods have been shown to do it (perhaps less performant than this
>>>> is some conditions),
>>> Actually, I think there should be always a better-performing method
>>> to implement C++ static initialization than using windows or pthread
>>> mutex -- e.g. using a specialized control like those used for
>>> call_once implementation -- at least on UNIX or Windows.
>>>
>>> This is because IMHO mutex API is a textbook case of feature bloat on
>>> both platforms.
>>>
>>> E.g. in standard UNIX (former POSIX), you can have recursive / robust
>>> / shared / errorcheck mutex (not all mutually exclusive) and who
>>> knows what else. pthread_mutex_lock shall be able to deal with either
>>> combination and the feature-check code makes (at least the current)
>>> implementation quite messy, with multiple dispatches on the same
>>> condition.
>>>
>>> On Windows, the number of options is less; but, to compensate for
>>> that :-), a mutex is always recursive plus wait functions have to
>>> know how to deal with inter-process mutexes because any handle passed
>>> to them can turn out to be of an inter-process mutex (or even not a
>>> mutex at all). Therefore, I do not think their use for static
>>> initialization (that does not require either recursive or
>>> inter-process lock) even could be optimal.
>>
>> I might be missing your point here. Humm... Even though POSIX has all
>> of these different types of mutexs, we can use them by organization.
> The point is that, even assuming that all types of mutices are very well
> organized in the implementation, some dispatch of control still has to
> occur when one calls "pthread_mutex_lock" because it is a single entry
> point for locking a mutex with any combination of features. Given that
> locking of a non-contended mutex is supposed to be (and in fact is, on
> current Linux) very cheap (but not necessarily same-cheap if the mutex
> is recursive :-) ), the relative cost of this dispatch is not guaranteed
> to be negligible (even if that dispatch is implemented via a single
> extra level of indirection).
>
> Further, I am unsure if these features actually *can* be very well
> organized because not all features of mutices are mutually exclusive (no
> pun intended) so a single extra level of indirection might not be enugh.
> As an example, mutexattr type can be one of
>
>  PTHREAD_MUTEX_NORMAL PTHREAD_MUTEX_ERRORCHECK PTHREAD_MUTEX_RECURSIVE
> PTHREAD_MUTEX_DEFAULT

Yeah. It's up to us programmers to make sure everything is right on our
end. What about:

intra_process_mutex_recursive m_mutex_recurive;

I find that a more descriptive name can help wrt reading code created by
somebody else. Humm... Actaly, I do not like to use recursive mutexes
very much (if at all) because I have had to debug some other peoples
"nightmare" code that recursed, and caused a deadlock around four locks
deep.... It technically was not the fault of the recursion ability, but
it was an error on the programmers side. The name of the lock was
something like (object.m_lock), it was vague and I did not even know it
recursed until I took a deeper look at the damn "nightmare" code. So,
iirc, I said out loud, name it something else... How about:

object.m_lock_recusive__never__use__

and got a laugh from the room. :^)

>
> but each of these can be additionally either robust or non-robust (and,
> pthread_mutex_lock is supposed to behave differently for some
> combinations of type and robustness). Unless of course we dispatch to a
> cartesian product of options (which is not impossible, but see below).
>
> And, lastly, the above are all hypotheticals and the actual code (see
> e.g. https://codebrowser.dev/glibc/glibc/nptl/pthread_mutex_lock.c.html
> ) is slightly messier than the best organized implementation possible.
> From practical perspective, I would expect that someone implementing a
> thread-safe static initialization in a C++ compiler has their mouth full
> with the tasks other than improving the existing pthread_mutex (or
> pthread_once) implementation as a distraction; at most (I would
> speculate), they would take a quick (~5 min) look into the API and the
> available code of the existing established synchronization primitives
> and decide which one is likely to better fit their spec and be faster
> as-is to use to implement that pesky initialization spec. In this
> scenario, I would guess that pthread_once_t would win the beauty contest
> and get selected.
>
> And, sure enough, (likely due to no feature bloat) the *existing*
> pthread_once implementation code is much cleaner (see
> https://codebrowser.dev/glibc/glibc/nptl/pthread_once.c.html) and at
> first glance should be measurably faster.
>
> I guess, instead of guessing, (no pun untended again), we could just
> disassemble some static initialization and see how it's done in, say,
> gcc. Maybe later...
>
>> So, wrt windows, an intra-process mutex can be a CRITICAL_SECTION. Or
>> an inter-process as a mutex HANDLE.
>>
>> https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-createmutexexa
>>
>> Wrt the inter-process route we may need to deal with WAIT_ABANDONED,
>> or EOWNERDEAD wrt POSIX. So we create different types:
>>
>>
>> struct intra_process_mutex
>> {
>>      CRITICAL_SECTION m_os_lock;
>> };
>>
>>
>> struct inter_process_mutex
>> {
>>      HANDLE m_os_lock;
>> };
>>
>> struct inter_process_mutex_robust
>> {
>>      HANDLE m_os_lock;
>> };
>>
>>
>>
>> These simple types can be rather easy to reason about. Wrt POSIX, the
>> different types would be:
>>
>>
>> struct intra_process_mutex
>> {
>>      pthread_mutex_t m_os_lock;
>> };
>>
>>
>> struct inter_process_mutex
>> {
>>      pthread_mutex_t m_os_lock;
>> };
>>
>> struct inter_process_mutex_robust
>> {
>>      pthread_mutex_t m_os_lock;
>> };
>>
>>
>> The point is trying to organize things a bit here.
>>
>> Sound somewhat decent?
>>
>>
>>>
>>>> so it is possible, so you can't say the Standard is asking for
>>>> impossible behavior, thus any "optimizations" that don't meet that
>>>> defined behavior are just WRONG.
>>>>
>>>> Saying something wrong must be made right is just WRONG, so you are
>>>> WRONG in your claim.
>>>>
>>>> PERIOD.
>>>
>>
>

17.09.2023 21:43 Bonita Montero kirjutas:

[...]

> The calling operator of the theads's function object is generic
> and instantited once per T. So the static is object is individual
> per each instantiation of the calling operator. The above code
> takes about one second with MSVC, g++ (libstdc++) and clan++
> (libc++), so there's an individual mutex per calling operator
> of the above lambda.

Studying the actual code shows MSVC is using a critical section and
libstc++ is using a futex. These are both single-process only facilities
which cannot fail in a way an inter-process fully-kernel mutex could
fail. Moreover, as the C++ standard does not allow the static
initialization to fail for synchronization reasons, a conforming
implementation could not use a general mutex anyway because it could fail.

I guess that's a big part of why the implementations are using the
mechanisms they are using, and these are not general mutexes.

Am 17.09.2023 um 21:57 schrieb Chris M. Thomasson:

> I still don't think you know how they work.

You didn't read the paper, not me.

Am 17.09.2023 um 21:36 schrieb Kaz Kylheku:

> No it shouldn't. Synchronization that fails for reasons other than
> a bug in the program (deadlock, unlocking un-owned mutex, recursively
> acquiring non-recursive mutex, ...) is a wrongheaded idea, ...

And why could a mutex throw a system_error on synchonization.
According to what you say everything could be noexcept.

Am 17.09.2023 um 21:47 schrieb Richard Damon:

> Why does it need to take into account that an BAD implementation might
> fail.

With the deferred initialization of a C++ mutex the mutex
could fail on synchronization even without being bad.

Am 17.09.2023 um 22:23 schrieb Chris M. Thomasson:

> I personally consider contention to be a so-called slow-path. ...

The futex part is to make the slow path faster.

Am 17.09.2023 um 21:49 schrieb Kaz Kylheku:

> It's one that you started ...

No, I didn't start it, Chris did. And at this point it was
no longer a question of whether synchronization could fail.

Rest unread.

Am 17.09.2023 um 23:29 schrieb Paavo Helde:

> Studying the actual code shows MSVC is using a critical section and
> libstc++ is using a futex. These are both single-process only facilities
> which cannot fail in ...

Specific implementations don't matter because the point is that
C++ should honor implementations that can theoretically fail.

Am 17.09.2023 um 21:43 schrieb Richard Damon:

> Nope, you shown that it can't be a single mutex held for the entire time
> of the constructor.

That's what I said.

Am 17.09.2023 um 22:01 schrieb Pavel:

> Of course you don't show that the synchronization primitives that are
> used for static initialization are mutexes.

Call it mutex or whatever: the standard requires that contenders
sleep in place, and that's only possible with something that works
like a mutex.

On 9/18/23 1:18 AM, Bonita Montero wrote:
> Am 17.09.2023 um 21:43 schrieb Richard Damon:
>
>> Nope, you shown that it can't be a single mutex held for the entire
>> time of the constructor.
>
> That's what I said.

No, you said it was a Mutex per object, which is different than not a
single Mutex.

You are assuming is uses the object the OS is calling a "Mutex", which
is an object that works inter-process, but we don't need anything that
powerful.

So, the alternative to using a Windows Mutex is using something else.

Something which can't fail.

On 9/18/23 1:18 AM, Bonita Montero wrote:
> Am 17.09.2023 um 23:29 schrieb Paavo Helde:
>
>> Studying the actual code shows MSVC is using a critical section and
>> libstc++ is using a futex. These are both single-process only
>> facilities which cannot fail in ...
>
> Specific implementations don't matter because the point is that
> C++ should honor implementations that can theoretically fail.
>

Why?

Since it can be done without ever failing, why should the Standard allow
for failure.

I guess you think all programs must have Undefined Behavior, or that it
is ok for x = 1 + 1; to set x to 1.

You just are showing you are too ignorant of how to do things that your
mind is stuck on things that don't actually work.

Am 18.09.2023 um 14:13 schrieb Richard Damon:

> Why?

Because a mutex does this also with an appropriate exception. And there
wouldn't be more problems if such an excepton would be thrown, but less.

> Since it can be done without ever failing, ...

I've shown that libc++, libstdc++ and MSVC handle this with per-object
mutexes, and if you create an arbitrary number of mutexes this might
fail.

Am 18.09.2023 um 14:11 schrieb Richard Damon:

> You are assuming is uses the object the OS is calling a "Mutex", which
> is an object that works inter-process, but we don't need anything that
> powerful.

When I talk about a mutex in the context of C++, especially when
it comes to static initialization, it should be clear that I don't
mean mutexes that are "inter-process".

> So, the alternative to using a Windows Mutex is using something else.
> Something which can't fail.

At least the creation of the kernel resource for the slow path of
the mutex can fail. And under Windows even a WaitForSingleObject(),
which usually is used for the slow path, can fail for not entirely
specified reasons.

I had the same discussion on Stack Overflow and had the same objections,
but lastly the people with these objections saw that what I was saying
made sense. Here, however, there are only idiots.

I don't know what the fuss is about: if waiting for a std::mutex
can fail, the people who developed the standard figured that the
operating system part could fail. Accordingly, there is an exception
for std::mutex if you lock it. If you look at this consistently,
static initialization should also be able to throw an exception
for the same reason because the same kernel mechanisms are behind
it.

Bonita Montero <Bonita.Montero@gmail.com> writes:
>Am 17.09.2023 um 23:29 schrieb Paavo Helde:
>
>> Studying the actual code shows MSVC is using a critical section and
>> libstc++ is using a futex. These are both single-process only facilities
>> which cannot fail in ...
>
>Specific implementations don't matter because the point is that
>C++ should honor implementations that can theoretically fail.

Feel free to submit a proposal to the C++ standard committee.

Since everyone particpating in this discussion on comp.lang.c++
seems to take the opposite view that you do, I don't believe you'll
have much success in your proposal.

Am 18.09.2023 um 16:59 schrieb Scott Lurndal:

> Feel free to submit a proposal to the C++ standard committee.

That would make sense for me, but it's not that extremely important
for me since I terminate on synchronization failures anyway for the
code I write so far.

> Since everyone particpating in this discussion on comp.lang.c++
> seems to take the opposite view that you do, I don't believe you'll
> have much success in your proposal.

The people on Stack Overflow agreed to me at last.
But here are only n00bs.

On 2023-09-18, Bonita Montero <Bonita.Montero@gmail.com> wrote:
> Am 17.09.2023 um 21:36 schrieb Kaz Kylheku:
>
>> No it shouldn't. Synchronization that fails for reasons other than
>> a bug in the program (deadlock, unlocking un-owned mutex, recursively
>> acquiring non-recursive mutex, ...) is a wrongheaded idea, ...
>
> And why could a mutex throw a system_error on synchonization.
> According to what you say everything could be noexcept.

It could be, but a C++ API that does not throw exceptions or return
error indications wouldn't be able to express support for mutex
debugging, and would not be extensible to situations like process-shared
robust mutexes.

Just because the API says that it can throw exceptions doesn't mean that
we expect them in a correct program that has not misused the mutex in
any way or caused a deadlock, such that we have to put try blocks
everywhere and handle something.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.