We are currently generating a loop which has more comparisons than you'd
typically need as the probablities on the small size loop are such that it
assumes the likely case is that an element is not found.
This again generates a pattern that's harder for branch predictors to follow,
but also just generates more instructions for the what one could say is the
typical case: That your hashtable contains the entry you are looking for.
This patch adds a __builtin_expect in _M_find_before_node where at the moment
the loop is optimized for the case where we don't do any iterations.
A simple testcase is (compiled with -fno-split-path to simulate the loop
in libstdc++):
#include <stdbool.h>
bool foo (int **a, int n, int val, int *tkn)
{
for (int i = 0; i < n; i++)
{
if (!a[i] || a[i]==tkn)
return false;
if (*a[i] == val)
return true;
}
}
which generataes:
foo:
cmp w1, 0
ble .L1
add x1, x0, w1, uxtw 3
b .L4
.L9:
ldr w4, [x4]
cmp w4, w2
beq .L6
cmp x0, x1
beq .L1
.L4:
ldr x4, [x0]
add x0, x0, 8
cmp x4, 0
ccmp x4, x3, 4, ne
bne .L9
mov w0, 0
.L1:
ret
.L6:
mov w0, 1
ret
i.e. BB rotation makes is generate an unconditional branch to a conditional
branch. However this method is only called when the size is above a certain
threshold, and so it's likely that we have to do that first iteration.
Adding:
#include <stdbool.h>
bool foo (int **a, int n, int val, int *tkn)
{
for (int i = 0; i < n; i++)
{
if (__builtin_expect(!a[i] || a[i]==tkn, 0))
return false;
if (*a[i] == val)
return true;
}
}
to indicate that we will likely do an iteration more generates:
foo:
cmp w1, 0
ble .L1
add x1, x0, w1, uxtw 3
.L4:
ldr x4, [x0]
add x0, x0, 8
cmp x4, 0
ccmp x4, x3, 4, ne
beq .L5
ldr w4, [x4]
cmp w4, w2
beq .L6
cmp x0, x1
bne .L4
.L1:
ret
.L5:
mov w0, 0
ret
.L6:
mov w0, 1
ret
which results in ~0-10% extra on top of the previous patch.
In table form:
+-------------+---------------+-------+--------------------+-------------------+-----------------+
| benchmark | Type | Size | Inline vs baseline | final vs baseline | final vs inline |
+-------------+---------------+-------+--------------------+-------------------+-----------------+
| find many | uint64_t | 11253 | -15.67% | -22.96% | -8.65% |
| find many | uint64_t | 11253 | -16.74% | -23.37% | -7.96% |
| find single | uint64_t | 345 | -5.88% | -11.54% | -6.02% |
| find many | string | 11253 | -4.50% | -9.56% | -5.29% |
| find single | uint64_t | 345 | -4.38% | -9.41% | -5.26% |
| find single | shared string | 11253 | -6.67% | -11.00% | -4.64% |
| find single | shared string | 11253 | -4.63% | -9.03% | -4.61% |
| find single | shared string | 345 | -10.41% | -14.44% | -4.50% |
| find many | string | 11253 | -3.41% | -7.51% | -4.24% |
| find many | shared string | 11253 | -2.30% | -5.72% | -3.50% |
| find many | string | 13 | 2.86% | -0.30% | -3.07% |
| find single | string | 11253 | 4.47% | 1.34% | -3.00% |
| find many | custom string | 11253 | 0.25% | -2.75% | -2.99% |
| find single | uint64_t | 345 | 2.99% | 0.01% | -2.90% |
| find single | shared string | 345 | -11.53% | -13.67% | -2.41% |
| find single | uint64_t | 11253 | 0.49% | -1.59% | -2.07% |
+-------------+---------------+-------+--------------------+-------------------+-----------------+
libstdc++-v3/ChangeLog:
* include/bits/hashtable.h
(_M_find_before_node): Make it likely that the map has at least one
entry and so we do at least one iteration.
This directory contains the GNU Compiler Collection (GCC).
The GNU Compiler Collection is free software. See the files whose
names start with COPYING for copying permission. The manuals, and
some of the runtime libraries, are under different terms; see the
individual source files for details.
The directory INSTALL contains copies of the installation information
as HTML and plain text. The source of this information is
gcc/doc/install.texi. The installation information includes details
of what is included in the GCC sources and what files GCC installs.
See the file gcc/doc/gcc.texi (together with other files that it
includes) for usage and porting information. An online readable
version of the manual is in the files gcc/doc/gcc.info*.
See http://gcc.gnu.org/bugs/ for how to report bugs usefully.
Copyright years on GCC source files may be listed using range
notation, e.g., 1987-2012, indicating that every year in the range,
inclusive, is a copyrightable year that could otherwise be listed
individually.
f8f686a129