Am 23.09.2023 um 08:35 schrieb Bonita Montero:
> Am 23.09.2023 um 06:41 schrieb Bonita Montero:
>
>> Ok, then Posix threads also can do delayed initialization of the
>> kernel  part for the slow path. And I see that InitializeCriticalSection
>> doesn't fail but EnterCriticalSection can, i.e. there's also delayed
>> initialization.
>
> This is how a mutex whose constructor is noexcept works.
> The code demonstrates the delayed initialization of the slow path:
>
> #if defined(_WIN32)
>     #include <Windows.h>
> #elif defined(__unix__)
>     #include <fcntl.h>
>     #include <semaphore.h>
>  #endif
> #include <iostream>
> #include <atomic>
> #include <cassert>
> #include <system_error>
> #include <memory>
>
> using namespace std;
>
> struct DelayedMutex
> {
>     DelayedMutex() noexcept;
>     ~DelayedMutex();
>     void lock();
>     void unlock();
> private:
> #if defined(_WIN32)
>     HANDLE getSem();
> #elif defined(__unix__)
>     sem_t *getSem();
> #endif
>     atomic<unsigned> m_lockCounter;
> #if defined(_WIN32)
>     static_assert(atomic<HANDLE>::is_always_lock_free, "atomic<HANDLE>
> must be lock-free");
>     atomic<HANDLE> m_hSem;
> #elif defined(__unix__)
>     atomic<sem_t *> m_sem;
> #endif
> };
>
> DelayedMutex::DelayedMutex() noexcept :
>     m_lockCounter( 0 ),
> #if defined(_WIN32)
>     m_hSem( NULL )
> #elif defined(__unix__)
>     m_sem( nullptr )
> #endif
> {
> }
>
> DelayedMutex::~DelayedMutex()
> {
>     assert(!m_lockCounter.load( memory_order_relaxed ));
> #if defined(_WIN32)
>     HANDLE hSem = m_hSem.load( memory_order_relaxed );
>     if( hSem )
>         CloseHandle( hSem );
> #elif defined(__unix__)
>     sem_t *sem = m_sem.load( memory_order_relaxed );
>     if( sem )
>         sem_destroy( sem ),
>         delete sem;
> #endif
> }
>
> void DelayedMutex::lock()
> {
>     unsigned before = m_lockCounter.fetch_add( 1, memory_order_acquire );
>     assert(before != -1);
>     if( !before ) [[likely]]
>         return;
> #if defined(_WIN32)
>     if( WaitForSingleObject( getSem(), INFINITE ) != WAIT_OBJECT_0 )
> [[unlikely]]
> #elif defined(__unix__)
>     if( sem_wait( getSem() ) ) [[unlikely]]
> #endif
>         terminate();
> }
>
> void DelayedMutex::unlock()
> {
>     unsigned before = m_lockCounter.fetch_sub( 1, memory_order_acquire );

release

>     assert(before >= 1);
>     if( before == 1 ) [[likely]]
>         return;
> #if defined(_WIN32)
>     if( !SetEvent( getSem() ) )[[unlikely]]
> #elif defined(__unix__)
>     if( sem_post( getSem() ) ) [[unlikely]]
> #endif
>         terminate();
> }
>
> #if defined(_WIN32)
> HANDLE DelayedMutex::getSem()
> {
>     HANDLE hSem = m_hSem.load( memory_order_relaxed );
>     if( hSem ) [[likely]]
>         return hSem;
>     hSem = CreateEventA( nullptr, FALSE, FALSE, nullptr );
>     if( !hSem ) [[unlikely]]
>         throw system_error( GetLastError(), system_category(), "kernel
> semaphore creation failed" );
>     HANDLE hSemExpected = NULL;
>     if( !m_hSem.compare_exchange_weak( hSemExpected, hSem,
> memory_order_relaxed, memory_order_relaxed ) ) [[unlikely]]
>         CloseHandle( hSem ),
>         hSem = hSemExpected;
>     return hSem;
> }
> #elif defined(__unix__)
> sem_t *DelayedMutex::getSem()
> {
>     sem_t *sem = m_sem.load( memory_order_relaxed );
>     if( sem ) [[likely]]
>         return sem;
>     unique_ptr<sem_t> newSem( make_unique<sem_t>() );
>     if( sem_init( &*newSem, 0, 0 ) ) [[unlikely]]
>         throw system_error( errno, system_category(), "kernel semaphore
> creation failed" );
>     sem_t *semExpected = nullptr;
>     if( m_sem.compare_exchange_weak( semExpected, &*newSem,
> memory_order_relaxed, memory_order_relaxed ) ) [[likely]]
>         sem = &*newSem,
>         newSem.release();
>     else
>         sem_destroy( &*newSem ),
>         sem = semExpected;
>     return sem;
> }
> #endif
>
> int main()
> {
> }

On 2023-09-23, Bonita Montero <Bonita.Montero@gmail.com> wrote:
> Am 22.09.2023 um 20:38 schrieb Kaz Kylheku:
>
>> POSIX provides a PTHREAD_MUTEX_INITIALIZER which lets you do just this:
>> pthread_mutex_t mtx = PTHREAD_MUTEX_INITIALIZER;
>
> Ok, then Posix threads also can do delayed initialization of the
> kernel part for the slow path.

Not in any way that can report a resource error, while remaining
POSIX compliant.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

Am 23.09.2023 um 13:20 schrieb Kaz Kylheku:

> Not in any way that can report a resource error, while remaining
> POSIX compliant.

If there's no kernel call while initializing the mutex the slow path
kernel part has to be initialized on contention. So if the initiali-
zation is noexcept, the synchronization needs to be capable to throw
a system_error at least for the delayed inittialization of this mutex
part.
And waiting on the slow path may also fail if there's no space for
en entry in the wait queue of the semaphore. This may not apply for
all operating systems, but this is a theoretical possibility the
standard shout honor.

On 9/23/23 12:38 AM, Bonita Montero wrote:
> Am 23.09.2023 um 00:31 schrieb Richard Damon:
>
>> The fact that you are too stupid to understand how these things
>> actually work and give performant results doesn't mean they don't do
>> it. The fact that they do shows your ignorance.
>
> Yielding at this place and spin-locking in userspace is unprofessional.
>

As is saying you should be able to fail at something that actually can
be done without failing.

Bonita Montero <Bonita.Montero@gmail.com> writes:
>Am 23.09.2023 um 00:22 schrieb Scott Lurndal:
>
>> That is beside the point. The mutex can be statically initialized
>> ...
>
>This depends on the program which uses the mutex.

Nice topic shift. Not.

We're talking specifically about the implementation mechanism
used to serialize static initialization of a function-scope
static object.

Am 23.09.2023 um 17:00 schrieb Scott Lurndal:

> We're talking specifically about the implementation mechanism
> used to serialize static initialization of a function-scope
> static object.

I've noticed that a Pthread mutex can be statically initialized and
Win32's CRITICAL_SECTION never fails on initialization either. But
this mean that the semaphore for the slow path on contention has to
be initialized delayed. So if there's contention the sychronization
can still fail.
And even if the semaphore creation doesn't fail the synchronization
on it may fail, depending on the implementation inside the kernel,
f.e. if there's no memory for an additional entry in the wait-queue
on the semaphore.
I think it's ridiculous to say "look at operating system X, it would
never fail" since that's not the way the standard should be designed.
As I've shown this may be true for the synchronization part only but
for sure not for the creation of the binary semaphore.
If std::mutex may fail on synchronization and implementations have
to create the kernel part of the "mutexes" for static initialization
on demand, static initialization should be definedto throw a
system_error.
The probability of this happening is very low, simply because the
probability of contention at this point is usually low and creating
a semaphore may fail under rare conditions, but what happens then
needs to be clearly defined for me. std::mutex has defined means
to handle that, static initialization not.

On 9/23/2023 5:23 AM, Bonita Montero wrote:
> Am 23.09.2023 um 13:20 schrieb Kaz Kylheku:
>
>> Not in any way that can report a resource error, while remaining
>> POSIX compliant.
>
> If there's no kernel call while initializing the mutex the slow path
> kernel part has to be initialized on contention. So if the initiali-
> zation is noexcept, the synchronization needs to be capable to throw
> a system_error at least for the delayed inittialization of this mutex
> part.
> And waiting on the slow path may also fail if there's no space for
> en entry in the wait queue of the semaphore. This may not apply for
> all operating systems, but this is a theoretical possibility the
> standard shout honor.
>

Sigh.

On 9/23/2023 6:40 AM, Richard Damon wrote:
> On 9/23/23 12:38 AM, Bonita Montero wrote:
>> Am 23.09.2023 um 00:31 schrieb Richard Damon:
>>
>>> The fact that you are too stupid to understand how these things
>>> actually work and give performant results doesn't mean they don't do
>>> it. The fact that they do shows your ignorance.
>>
>> Yielding at this place and spin-locking in userspace is unprofessional.
>>
>
> As is saying you should be able to fail at something that actually can
> be done without failing.

Basically. Yikes!

On 9/23/2023 5:23 AM, Bonita Montero wrote:
> Am 23.09.2023 um 13:20 schrieb Kaz Kylheku:
>
[...]
> And waiting on the slow path may also fail if there's no space for
> en entry in the wait queue of the semaphore.

Huh? I don't think you know how a wait queue works. A node can be in
per-thread memory, even on its stack. Study up Buddy!

On 9/7/2023 2:13 PM, Scott Lurndal wrote:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>> On 9/7/2023 12:52 PM, Richard Damon wrote:
>>> On 9/7/23 12:20 PM, Chris M. Thomasson wrote:
>>>> On 9/7/2023 11:06 AM, Richard Damon wrote:
>>>>> On 9/7/23 10:27 AM, Scott Lurndal wrote:
>>>>>> Bonita Montero <Bonita.Montero@gmail.com> writes:
>>>>>>> Am 07.09.2023 um 16:44 schrieb Pavel:
>>>>>>>
>>>>>>>> How so? Waiters will wait for the time of initialization.
>>>>>>>> The initializing thread will be yielded ...
>>>>>>>
>>>>>>> Initializing is usually much faster than a whole timeslice,
>>>>>>> so yielding would be incacceptabel. That's just a stupid idea.
>>>>>>
>>>>>> So tell us, on which operating systems will there be more than
>>>>>> one thread running when application static objects are
>>>>>> initialized (which happens generally before the application
>>>>>> 'main' function is called, and thus before the application
>>>>>> has a chance to create any threads)?
>>>>>
>>>>> While GLOBAL static objects get initialized before main starts,
>>>>> function local static objects don't get initialized until the first
>>>>> call of the function. These will need some synchronization if the
>>>>> function is called from multiple threads at "the same time".
>>>>
>>>> A simple global hash lock scheme is where we can hash addresses
>>>> directly into a static locking table. The lock table is created
>>>> _before_ any program logic is executed.
>>>>
>>>> https://groups.google.com/g/comp.lang.c++/c/sV4WC_cBb9Q/m/wwYQCG2hAwAJ
>>>>
>>>>
>>>>> Also, some global object could start up a thread in its constructor.
>>>>
>>>> YIKES! Shit. I have had to debug other peoples code that did this.
>>>> Many points of errors... One was a rather common peach of a bug. The
>>>> constructor would create a thread that would in turn call into a
>>>> virtual function and start using the object before its constructor was
>>>> completed. A massive race condition, nasty ones!
>>>>
>>>
>>> Yes, such a thread needs to understand that the system isn't fully
>>> configured. And yes, a base class creating a thread needs that thread to
>>> understand that the object isn't fully created yet and do something to
>>> handle that issue (which likely requires some help from the most derived
>>> class).
>>
>> I actually got to a point where I said no creating threads in
>> constructors! I have seen to many problems. Create the object _first_,
>> then expose it to a thread, or create a thread that works with the
>> _fully_ constructed object. The types of race conditions I have had to
>> deal with wrt objects creating threads in constructors have tended to be
>> rather nasty in nature...
>
> The race condition is pretty easy to resolve. Turn thread initialization
> into a two-phase operation.
>
> When we have an object that requires its own thread, it will inherit
> from a base class called c_thread. The constructor for c_thread will
> create the thread which eventually executes the objects 'run' function. The
> pthread_create call starts the thread executing at the static function
> c_thread::run(void *obj), where the argument is a pointer to the object
> itself.
>
> That static function sets the process name (prctl(PR_SET_NAME)) and
> then waits on a pthread_cond_t (thread_initialized) declared as a data member of
> c_thread.
>
> Meanwhile, the derived object constructor runs in the original thread after the c_thread
> constructor returns and initializes the object. Once the object is fully
> initialized (which may happen in the constructor, or may happen
> at some later point in time during program startup), the c_thread::run
> method condition variable (thread_initialized) is signaled and the
> static ::run function transfers control to the derived class virtual ::run
> function.
>
> /**
> * Static pthread start function. Invokes thread specific run virtual
> * function.
> *
> * @param arg The class object pointer passed in from the pthread_create.
> */
> void *
> c_thread::run(void *arg)
> {
> c_thread *tp = (c_thread *)arg;
> int diag;
>
> pthread_mutex_lock(&tp->t_threadlock);
> while (!tp->t_thread_initialized) {
> pthread_cond_wait(&tp->t_ready, &tp->t_threadlock);
> }
> pthread_mutex_unlock(&tp->t_threadlock);
>
> diag = prctl(PR_SET_NAME, tp->t_thread_name, NULL, NULL, NULL, NULL);
> if (diag == -1) {
> tp->t_logger->log("Unable to set thread name: %s\n", strerror(errno));
> }
>
> tp->t_running = true;
> tp->run();
> tp->t_running = false;
>
> return NULL;
> }

Actually, I like to ensure that the "object" is 100% initialized before
any thread is able to acquire a pointer to it. A little proxy object in
C++ tends to work fine for me:

template<typename T>
struct thread_proxy
{ T m_object;
std::thread m_thread;
};

We can launch the thread in the constructor of thread_proxy<T>.

thread_proxy<T>::m_object is 100% fully initialized before the thread is
created. Simple and works for me. Fair enough?

Bonita Montero wrote:
> Am 23.09.2023 um 04:50 schrieb Pavel:
>
>> for some mutex it may fail, for some not. C++ standard never says that
>> std::mutex can fail. ...
>
> std::mutex::lock() can throw a system_error.
Wrong. The standard says that lock can throw while discussing general
requirements for all types of mutices (mutex, recursive_mutex,
timed_mutex, recursive_timed_mutex, shared_mutex, and
shared_timed_mutex). Then, when discussing specific mutices, the
standard says that recursive_mutex, recursive_timed_mutex and any of the
shared mutices can throw system_error). About std::mutex the standard
says no such thing.

The only mentioning of an error for std::mutex is in a non-normative
note that explains: "If the
implementation can detect the deadlock, a resource_deadlock_would_occur
error condition might be observed". We knew that already (using
deadlock-detecting mutex is asking for error) -- as well as that no C++
implementer would use a more complex error-throwing deadlock-detecting
mutex given a simpler alternative for static initialization.

In particular, on UNIX, an implementer will use
PTHREAD_MUTEX_INITIALIZER that initializes a non-ERRORCHECK mutex.

On Windows, critical section is essentially a recursive
deadlock-detecting (timed) mutex so it is relatively expensive and can fail.

All it means that, on Windows, to implement static initialization, a C++
implementor has to use other primitives, most conveniently --
InitOnceExecuteOnce -- that cannot fail unless the callback it executes
fails.

>
>> "delayed" or "deferred" initialization of synchronization primitives
>> that can be used to initialize static C++ object is entirely your
>> fantasy. It cannot be used for this purpose, period.
>
> The mutex' constructor doesn't throw, so there must be delayed
> initialization.
Wrong. a mutex can be created during static initialization entirely in
user space (like POSIX mutex on linux is created)

> But as we've seen recently pthread Mutexes also
> allow delayed initialization
"Allow" does not mean C++ static initialization implementer has to use
it. On Linux, mutex_lock tries light-weight user-space initialization
internally so no need to reinvent that wheel for implementers. But of
course on Linux any reasonable implementer will use pthread_once (which
also does not have any delayed iniitialization), rather than a mutex.

> of the slow path and Win32 always
> does the same.
Yes, on Win32 C++ static initialization does not have to use mutex
either. INIT_ONCE can be initialized statically to INIT_ONCE_STATIC_INIT
-- which is all one needs for static initialization (coincidentally, on
Win32 specifically InitOnceInitialize does not fail either but this is
irrelevant to the issue of static C++ initialization).

On 9/23/2023 2:58 PM, Pavel wrote:
> Bonita Montero wrote:
>> Am 23.09.2023 um 04:50 schrieb Pavel:
>>
>>> for some mutex it may fail, for some not. C++ standard never says
>>> that std::mutex can fail. ...
>>
>> std::mutex::lock() can throw a system_error.
> Wrong. The standard says that lock can throw while discussing general
> requirements for all types of mutices (mutex, recursive_mutex,
> timed_mutex, recursive_timed_mutex, shared_mutex, and
> shared_timed_mutex). Then, when discussing specific mutices, the
> standard says that recursive_mutex, recursive_timed_mutex and any of the
> shared mutices can throw system_error). About std::mutex the standard
> says no such thing.
>
> The only mentioning of an error for std::mutex is in a non-normative
> note that explains: "If the
> implementation can detect the deadlock, a resource_deadlock_would_occur
> error condition might be observed". We knew that already (using
> deadlock-detecting mutex is asking for error) -- as well as that no C++
> implementer would use a more complex error-throwing deadlock-detecting
> mutex given a simpler alternative for static initialization.
>
> In particular, on UNIX, an implementer will use
> PTHREAD_MUTEX_INITIALIZER that initializes a non-ERRORCHECK mutex.
>
> On Windows, critical section is essentially a recursive
> deadlock-detecting (timed) mutex so it is relatively expensive and can
> fail.
[...]

I don't think there is a timed wait for a windows critical section. I
know it has a try, but I cannot remember a timed wait...

On 9/23/2023 2:58 PM, Pavel wrote:
> Bonita Montero wrote:
>> Am 23.09.2023 um 04:50 schrieb Pavel:
>>
>>> for some mutex it may fail, for some not. C++ standard never says
>>> that std::mutex can fail. ...
>>
>> std::mutex::lock() can throw a system_error.
> Wrong. The standard says that lock can throw while discussing general
> requirements for all types of mutices (mutex, recursive_mutex,
> timed_mutex, recursive_timed_mutex, shared_mutex, and
> shared_timed_mutex). Then, when discussing specific mutices, the
> standard says that recursive_mutex, recursive_timed_mutex and any of the
> shared mutices can throw system_error). About std::mutex the standard
> says no such thing.
>
> The only mentioning of an error for std::mutex is in a non-normative
> note that explains: "If the
> implementation can detect the deadlock, a resource_deadlock_would_occur
> error condition might be observed". We knew that already (using
> deadlock-detecting mutex is asking for error) -- as well as that no C++
> implementer would use a more complex error-throwing deadlock-detecting
> mutex given a simpler alternative for static initialization.
>
> In particular, on UNIX, an implementer will use
> PTHREAD_MUTEX_INITIALIZER that initializes a non-ERRORCHECK mutex.
>
> On Windows, critical section is essentially a recursive
> deadlock-detecting (timed) mutex so it is relatively expensive and can
> fail.
>
> All it means that, on Windows, to implement static initialization, a C++
> implementor has to use other primitives, most conveniently --
> InitOnceExecuteOnce -- that cannot fail unless the callback it executes
> fails.
>
>>
>>> "delayed" or "deferred" initialization of synchronization primitives
>>> that can be used to initialize static C++ object is entirely your
>>> fantasy. It cannot be used for this purpose, period.
>>
>> The mutex' constructor doesn't throw, so there must be delayed
>> initialization.
> Wrong. a mutex can be created during static initialization entirely in
> user space (like POSIX mutex on linux is created)
>
>> But as we've seen recently pthread Mutexes also
>> allow delayed initialization
> "Allow" does not mean C++ static initialization implementer has to use
> it. On Linux, mutex_lock tries light-weight user-space initialization
> internally so no need to reinvent that wheel for implementers. But of
> course on Linux any reasonable implementer will use pthread_once (which
> also does not have any delayed iniitialization), rather than a mutex.
>
>>  of the slow path and Win32 always
>> does the same.
> Yes, on Win32 C++ static initialization does not have to use mutex
> either. INIT_ONCE can be initialized statically to INIT_ONCE_STATIC_INIT
> -- which is all one needs for static initialization (coincidentally, on
> Win32 specifically InitOnceInitialize does not fail either but this is
> irrelevant to the issue of static C++ initialization).

Iirc, I think a windows critical section might raise an error if its
been waiting for 30 days or something, deadlocked.

Am 23.09.2023 um 23:58 schrieb Pavel:

>> std::mutex::lock() can throw a system_error.

> Wrong. The standard says that lock can throw while discussing general
> requirements for all types of mutices (mutex, recursive_mutex,
> timed_mutex, recursive_timed_mutex, shared_mutex, and
> shared_timed_mutex). Then, when discussing specific mutices, the
> standard says that recursive_mutex, recursive_timed_mutex and any of the
> shared mutices can throw system_error). About std::mutex the standard
> says no such thing.

constexpr mutex() noexcept;

https://en.cppreference.com/w/cpp/thread/mutex/mutex

> Wrong. a mutex can be created during static initialization entirely
> in user space (like POSIX mutex on linux is created)

If the mutex' constructor is noexcept the kernel-part for the slow
path has to be initialized delayed while locking.

Rest unread.

On 9/23/2023 7:10 PM, Bonita Montero wrote:
> Am 23.09.2023 um 23:58 schrieb Pavel:
>
>>> std::mutex::lock() can throw a system_error.
>
>> Wrong. The standard says that lock can throw while discussing general
>> requirements for all types of mutices (mutex, recursive_mutex,
>> timed_mutex, recursive_timed_mutex, shared_mutex, and
>> shared_timed_mutex). Then, when discussing specific mutices, the
>> standard says that recursive_mutex, recursive_timed_mutex and any of
>> the shared mutices can throw system_error). About std::mutex the
>> standard says no such thing.
>
> constexpr mutex() noexcept;
>
> https://en.cppreference.com/w/cpp/thread/mutex/mutex
>
>> Wrong. a mutex can be created during static initialization entirely
>> in user space (like POSIX mutex on linux is created)
>
> If the mutex' constructor is noexcept the kernel-part for the slow
> path has to be initialized delayed while locking.

Huh?

>
> Rest unread.
>

Figured.

On 2023-09-23, Bonita Montero <Bonita.Montero@gmail.com> wrote:
> Am 23.09.2023 um 13:20 schrieb Kaz Kylheku:
>
>> Not in any way that can report a resource error, while remaining
>> POSIX compliant.
>
> If there's no kernel call while initializing the mutex the slow path
> kernel part has to be initialized on contention. So if the initiali-
> zation is noexcept, the synchronization needs to be capable to throw
> a system_error at least for the delayed inittialization of this mutex
> part.
> And waiting on the slow path may also fail if there's no space for
> en entry in the wait queue of the semaphore. This may not apply for
> all operating systems, but this is a theoretical possibility the
> standard shout honor.

The thing is, mutexes don't require a kernel object per.
A possible design is that each *thread* has a wait semaphore. If a
thread is created, it has a semaphore;the resource check can be done at
thread creation time. If we cannot create that important semaphore, the
thread doesn't get created; thread creation fails.

When a thread waits on mutex, it puts itself into the mutex's wait
queue, and then waits on its own semaphore. It uses that semaphore
for all mutexes.

When the mutex is unlocked, one (or maybe more) threads waiting on it
are dequeued and woken by signaling their individual semaphore.

These semaphores can be hidden under operations suspend() and
resume(thread). suspend() finds the thread's self structure, and
blocks on self->suspend_sem. resume(thread) signals the
thread->suspend_sem.

Roughly speaking.

You need a lower level locking mechanism to do the queuing.

Inside a kernel, you'd never bother with a semaphore per lock,
because you just put the thread into a wait queue, change it to
a blocked state, and call the scheduler.

Those mechanisms could be used to *build* a semaphore. Or barrier,

(The semaphore-per-thread in user space simulates that: using suspend()
and resume(thread) is like scheduling. suspend() deschedules this
thread; resume(thread) chooses a thread to be runnable.)
condition variable, etc.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 9/23/2023 7:22 PM, Kaz Kylheku wrote:
> On 2023-09-23, Bonita Montero <Bonita.Montero@gmail.com> wrote:
>> Am 23.09.2023 um 13:20 schrieb Kaz Kylheku:
>>
>>> Not in any way that can report a resource error, while remaining
>>> POSIX compliant.
>>
>> If there's no kernel call while initializing the mutex the slow path
>> kernel part has to be initialized on contention. So if the initiali-
>> zation is noexcept, the synchronization needs to be capable to throw
>> a system_error at least for the delayed inittialization of this mutex
>> part.
>> And waiting on the slow path may also fail if there's no space for
>> en entry in the wait queue of the semaphore. This may not apply for
>> all operating systems, but this is a theoretical possibility the
>> standard shout honor.
>
> The thing is, mutexes don't require a kernel object per.
>
> A possible design is that each *thread* has a wait semaphore. If a
> thread is created, it has a semaphore;the resource check can be done at
> thread creation time. If we cannot create that important semaphore, the
> thread doesn't get created; thread creation fails.
>
> When a thread waits on mutex, it puts itself into the mutex's wait
> queue, and then waits on its own semaphore. It uses that semaphore
> for all mutexes.
>
> When the mutex is unlocked, one (or maybe more) threads waiting on it
> are dequeued and woken by signaling their individual semaphore.
>
> These semaphores can be hidden under operations suspend() and
> resume(thread). suspend() finds the thread's self structure, and
> blocks on self->suspend_sem. resume(thread) signals the
> thread->suspend_sem.
>
> Roughly speaking.
>
> You need a lower level locking mechanism to do the queuing.
>
> Inside a kernel, you'd never bother with a semaphore per lock,
> because you just put the thread into a wait queue, change it to
> a blocked state, and call the scheduler.
>
> Those mechanisms could be used to *build* a semaphore. Or barrier,
>
> (The semaphore-per-thread in user space simulates that: using suspend()
> and resume(thread) is like scheduling. suspend() deschedules this
> thread; resume(thread) chooses a thread to be runnable.)
> condition variable, etc.
>

That's about it. The only problem wrt emulating it in user space is that
it basically takes the scheduling ability away from the OS.

struct per_thread
{ per_thread* m_next;
os_semaphore m_sema;
};

So, to wake a thread on the unlock of a mutex or something, wrt the wait
queue it would just pop a node and post to its semaphore. It did not
"necessarily" have to search itself for a specific thread... We can use
some indirection:

struct per_thread_node
{ per_thread_wait_node* m_next;
per_thread* m_thread;
};

And use per_thread_node instead of using per_thread directly to link
them into a wait queue.

On 9/23/2023 8:23 AM, Bonita Montero wrote:
> Am 23.09.2023 um 17:00 schrieb Scott Lurndal:
>
>> We're talking specifically about the implementation mechanism
>> used to serialize static initialization of a function-scope
>> static object.
>
> I've noticed that a Pthread mutex can be statically initialized and
> Win32's CRITICAL_SECTION never fails on initialization either. But
> this mean that the semaphore for the slow path on contention has to
> be initialized delayed. So if there's contention the sychronization
> can still fail.
> And even if the semaphore creation doesn't fail the synchronization
> on it may fail, depending on the implementation inside the kernel,
> f.e. if there's no memory for an additional entry in the wait-queue
> on the semaphore.
> I think it's ridiculous to say "look at operating system X, it would
> never fail" since that's not the way the standard should be designed.
> As I've shown this may be true for the synchronization part only but
> for sure not for the creation of the binary semaphore.
> If std::mutex may fail on synchronization and implementations have
> to create the kernel part of the "mutexes" for static initialization
> on demand, static initialization should be definedto throw a
> system_error.
> The probability of this happening is very low, simply because the
> probability of contention at this point is usually low and creating
> a semaphore may fail under rare conditions, but what happens then
> needs to be clearly defined for me. std::mutex has defined means
> to handle that, static initialization not.
>
>

Why do you think that static initialization must use a std::mutex
anyway? Why do you think that std::mutex must use some delayed kernel
resource initialization? Strange.

On 9/23/2023 7:59 PM, Chris M. Thomasson wrote:
> On 9/23/2023 7:22 PM, Kaz Kylheku wrote:
>> On 2023-09-23, Bonita Montero <Bonita.Montero@gmail.com> wrote:
>>> Am 23.09.2023 um 13:20 schrieb Kaz Kylheku:
>>>
>>>> Not in any way that can report a resource error, while remaining
>>>> POSIX compliant.
>>>
>>> If there's no kernel call while initializing the mutex the slow path
>>> kernel part has to be initialized on contention. So if the initiali-
>>> zation is noexcept, the synchronization needs to be capable to throw
>>> a system_error at least for the delayed inittialization of this mutex
>>> part.
>>> And waiting on the slow path may also fail if there's no space for
>>> en entry in the wait queue of the semaphore. This may not apply for
>>> all operating systems, but this is a theoretical possibility the
>>> standard shout honor.
>>
>> The thing is, mutexes don't require a kernel object per.
>> A possible design is that each *thread* has a wait semaphore. If a
>> thread is created, it has a semaphore;the resource check can be done at
>> thread creation time. If we cannot create that important semaphore, the
>> thread doesn't get created; thread creation fails.
>>
>> When a thread waits on mutex, it puts itself into the mutex's wait
>> queue, and then waits on its own semaphore. It uses that semaphore
>> for all mutexes.
>>
>> When the mutex is unlocked, one (or maybe more) threads waiting on it
>> are dequeued and woken by signaling their individual semaphore.
>>
>> These semaphores can be hidden under operations suspend() and
>> resume(thread). suspend() finds the thread's self structure, and
>> blocks on self->suspend_sem.  resume(thread) signals the
>> thread->suspend_sem.
>>
>> Roughly speaking.
>>
>> You need a lower level locking mechanism to do the queuing.
>>
>> Inside a kernel, you'd never bother with a semaphore per lock,
>> because you just put the thread into a wait queue, change it to
>> a blocked state, and call the scheduler.
>>
>> Those mechanisms could be used to *build* a semaphore. Or barrier,
>>
>> (The semaphore-per-thread in user space simulates that: using suspend()
>> and resume(thread) is like scheduling. suspend() deschedules this
>> thread; resume(thread) chooses a thread to be runnable.)
>> condition variable, etc.
>>
>
> That's about it. The only problem wrt emulating it in user space is that
> it basically takes the scheduling ability away from the OS.
>
> struct per_thread
> {
>     per_thread* m_next;
>     os_semaphore m_sema;
> };
>
> So, to wake a thread on the unlock of a mutex or something, wrt the wait
> queue it would just pop a node and post to its semaphore. It did not
> "necessarily" have to search itself for a specific thread... We can use
> some indirection:
>
> struct per_thread_node
> {
>     per_thread_wait_node* m_next;
^^^^^^^^^^^^^^^^^^^^

That should be:

per_thread_node* m_next;

>     per_thread* m_thread;
> };

Damn typos!

>
>
> And use per_thread_node instead of using per_thread directly to link
> them into a wait queue.

Am 24.09.2023 um 04:22 schrieb Kaz Kylheku:

> A possible design is that each *thread* has a wait semaphore.

You'd have to register the binary semaphore and register as a waiter in
one atomic step. You could do that with a DCAS'd ring, but you coudn't
atomically update the link pointers of the other nodes at the same
time you update the roots link pointers. And you would have the deallo-
cation poblem, i.e. the memory of your object may never be deallocated.
Show me your implementation !
It's ridiculous because mutex are a cheap resource. And if such imple-
mentation would be possible the standard shouldn't mandate it since
not every platform has a DCAS.

Am 24.09.2023 um 05:05 schrieb Chris M. Thomasson:

> Why do you think that static initialization must use a std::mutex
> anyway? ...

Call it std::mutex or whatever. It is mandated that contenders
are sleeping - that's not possible without sth. else than a mutex.

On Sunday, September 24, 2023 at 11:09:31 AM UTC+3, Bonita Montero wrote:
> Am 24.09.2023 um 05:05 schrieb Chris M. Thomasson:
>
> > Why do you think that static initialization must use a std::mutex
> > anyway? ...
>
> Call it std::mutex or whatever. It is mandated that contenders
> are sleeping - that's not possible without sth. else than a mutex.

Few dozens posts above I demonstrated an implementation
for pre-Vista Windows where contenders are sleeping just fine and
waking just fine and initialization of different objects proceeds
simultaneously just fine. And it does not use mutex for that.
It uses SleepEx() for wait and QueueUserApc() for wakeup.
It uses one global mutex (in form of critical section) for a short
while for metadata updates, but that's done for simplicity and
practicality rather than out of necessity. I am pretty sure that
with enough effort I can device a variant that achieves the same
with CaS.

Am 24.09.2023 um 11:50 schrieb Michael S:

> ...And it does not use mutex for that.
> It uses SleepEx() for wait and QueueUserApc() for wakeup.

That's also a kind of semaphore.

On 9/24/23 4:08 AM, Bonita Montero wrote:
> Am 24.09.2023 um 04:22 schrieb Kaz Kylheku:
>
>> A possible design is that each *thread* has a wait semaphore.
>
> You'd have to register the binary semaphore and register as a waiter in
> one atomic step. You could do that with a DCAS'd ring, but you coudn't
> atomically update the link pointers of the other nodes at the same
> time you update the roots link pointers. And you would have the deallo-
> cation poblem, i.e. the memory of your object may never be deallocated.
> Show me your implementation !
> It's ridiculous because mutex are a cheap resource. And if such imple-
> mentation would be possible the standard shouldn't mandate it since
> not every platform has a DCAS.
>

So, you still think that the Standard should allow a very common
operation to fail in ways that the program can not handle it when
another method exists to do the operation exists that has no opportunity
for failure.

You just don't understand programming.

Something that is marginally faster, but introduces chance of failure,
is NOT an improvement or valid optimization.

The fact that YOU don't understand how to do a standard operation,
doesn't mean it isn't a good way to do it.

On 9/24/23 4:09 AM, Bonita Montero wrote:
> Am 24.09.2023 um 05:05 schrieb Chris M. Thomasson:
>
>> Why do you think that static initialization must use a std::mutex
>> anyway? ...
>
> Call it std::mutex or whatever. It is mandated that contenders
> are sleeping - that's not possible without sth. else than a mutex.
>
>

Where is it mandated that they "Sleep"?

What is mandated is that initialization occur just once and no code can
access the object (if properly accessed) before its initialization.

I think you don't understand the meaning of the Standard.