Pass __wait_args_base by const reference instead of const pointer. I
don't see a reason it needs to be passed by pointer to the internals.
We can also avoid constructing a __wait_args from __wait_args_base in
some places, instaad just using the latter directly.
The code using the __wait_flags bitmask type is broken, because the
__spin_only constant includes the __do_spin element. This means that
testing (__args & __wait_flags::__spin_only) will be inadvertently true
when only __do_spin is set. This causes the __wait_until_impl function
to never actually wait on the futex (or condition variable), turning all
uses of that function into expensive busy spins. Change __spin_only to
be a single bit (i.e. a bitmask element) and adjust the places where
that bit is set so that they also use the __do_spin element.
Update the __args._M_old value when looping in __atomic_wait_address, so
that the next wait doesn't fail spuriously.
With the new __atomic_wait_address logic, the value function needs to
return the correct type, not just a bool. Without that change, the
boolean value returned by the value function is used as the value
passed to the futex wait, but that mean we're comparing (_M_a == 0) to
_M_a and so can block on the futex when we shouldn't, and then never
wake up.
libstdc++-v3/ChangeLog:
* include/bits/atomic_timed_wait.h (__cond_wait_impl): Add
missing inline keyword.
(__spin_until_impl): Change parameter from pointer to reference.
Replace make_pair with list-initialization. Initialize variable
for return value.
(__wait_until_impl): Likewise. Remove some preprocessor
conditional logic. Use _S_track for contention tracking.
Avoid unnecessary const_cast.
(__wait_until): Change parameter from pointer to reference.
Replace make_pair with list-initialization.
(__wait_for): Change parameter from pointer to reference. Add
__do_spin flag to args.
* include/bits/atomic_wait.h (__waiter_pool_impl::_S_track): New
function returning an RAII object for contention tracking.
(__wait_flags): Do not set the __do_spin flag in the __spin_only
enumerator. Comment out the unused __abi_version_mask
enumerator. Define operator| and operator|= overloads.
(__wait_args_base::operator&): Define.
(__wait_args::operator&, __wait_args::_S_default_flags): Remove.
(__wait_args::operator|, __wait_args::operator|=): Remove.
(__spin_impl): Change parameter from pointer to reference.
Replace make_pair call with list-initialization.
(__wait_impl): Likewise. Remove some preprocessor conditional
logic. Always store old value in __args._M_old. Avoid
unnecessary const_cast. Use _S_track.
(__notify_impl): Change parameter to reference. Remove some
preprocessor conditional logic.
(__atomic_wait_address): Add comment. Update __args._M_old on
each iteration.
(__atomic_wait_address_v): Add comment.
* include/std/latch (latch::wait): Adjust predicates for new
logic.
* testsuite/29_atomics/atomic_integral/wait_notify.cc: Improve
test.
219bb905a6