procyberian/gcc
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add make autoprofiledbootstrap

Browse source 
Add support for profiledbootstrap with autofdo. Will be useful
to get better testing coverage of autofdo.

This requires Linux perf and autofdo to be installed, only
really for x86_64 linux on Intel so far.

Profile the whole build process with perf, then convert the file,
and pass it back to the compiler in the feedback stage.

The conversion has to be done per language, as only that knows
the name of the binary. Currently we only do it for C and C++,
as the other languages don't have enough coverage during a normal
bootstrap.

For lto1 it is also disabled, because it would only be useful during a LTO
bootstrap, but right now autofdo and LTO are not working together due to
PR66229

For common backend files always the profile output of the C++ compiler
is used. In theory multiple inputs could be merged here, but so far
that is not implemented.

The method is not friendly to partial rebuilds, as only the profile
information from the current rebuild is used. So if an error
occurs it is best to clean and restart, otherwise the code
quality may be worse.

This patch is fairly large, but most of it is auto generated from
autogen in Makefile.in. for the new stage targets.

Passes profiledbootstrap and normal bootstrap on x86_64-linux.
autoprofiledbootstrap is currently not working due to
PR70427 (but it finishes with that worked around)

The autofdo'ed compiler is ~7% faster on insn-recog.i (vs ~11% for
profiledfeedback), and ~4% faster for tramp3d-v4 (vs 10% for
profiledfeedback) on a Sandy Bridge system.

gcc/lto/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Make-lang.in: Add support for autofdo (disabled for now)

gcc/cp/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Make-lang.in: Add support for autofdo.

gcc/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Makefile.in: Regenerate.
	* doc/install.texi: Document autoprofiledbootstrap.

/:
2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Makefile.def: Add autoprofiledbootstrap.
	* Makefile.tpl: Dito.
	* Makefile.in: Regenerate.

gcc/c/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Make-lang.in: Add support for autofdo.

From-SVN: r237733
This commit is contained in:
Andi Kleen 2016-06-23 14:34:16 +00:00 • committed by Andi Kleen
parent 7f586d904d
commit 277d7ee0c9
13 changed files with 5713 additions and 175 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
ChangeLog
View file

@ -1,3 +1,9 @@
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Makefile.def: Add autoprofiledbootstrap.
* Makefile.tpl: Dito.
* Makefile.in: Regenerate.
2016-06-23 Jakub Sejdak <jakub.sejdak@phoesys.com>
* configure.ac: Disable libgcj and libgloss for Phoenix-RTOS targets.

7
Makefile.def
View file

@ -639,3 +639,10 @@ bootstrap_stage = {
bootstrap_stage = {
id=feedback ; prev=profile ;
bootstrap_target=profiledbootstrap ; };
bootstrap_stage = {
id=autoprofile ; prev=1 ;
autoprofile="$$s/gcc/config/i386/$(AUTO_PROFILE)" ; };
bootstrap_stage = {
id=autofeedback ; prev=autoprofile ;
bootstrap_target=autoprofiledbootstrap ;
profile_data="PERF_DATA=perf.data" ; };

5780
Makefile.in
View file

File diff suppressed because it is too large Load diff

16
Makefile.tpl
View file

@ -382,6 +382,8 @@ MAKEINFO = @MAKEINFO@
EXPECT = @EXPECT@
RUNTEST = @RUNTEST@
AUTO_PROFILE = gcc-auto-profile -c 1000000
# This just becomes part of the MAKEINFO definition passed down to
# sub-makes. It lets flags be given on the command line while still
# using the makeinfo from the object tree.
@ -418,6 +420,8 @@ CXXFLAGS = @CXXFLAGS@
LIBCXXFLAGS = $(CXXFLAGS) -fno-implicit-templates
GOCFLAGS = $(CFLAGS)
CREATE_GCOV = create_gcov
TFLAGS =
# Defaults for all stages; some are overridden below.
@ -462,6 +466,12 @@ STAGEprofile_TFLAGS = $(STAGE2_TFLAGS)
STAGEfeedback_CFLAGS = $(STAGE3_CFLAGS) -fprofile-use
STAGEfeedback_TFLAGS = $(STAGE3_TFLAGS)
STAGEautoprofile_CFLAGS = $(STAGE2_CFLAGS) -g
STAGEautoprofile_TFLAGS = $(STAGE2_TFLAGS)
STAGEautofeedback_CFLAGS = $(STAGE3_CFLAGS)
STAGEautofeedback_TFLAGS = $(STAGE3_TFLAGS)
do-compare = @do_compare@
do-compare3 = $(do-compare)
@ -617,7 +627,8 @@ EXTRA_HOST_FLAGS = \
'READELF=$(READELF)' \
'STRIP=$(STRIP)' \
'WINDRES=$(WINDRES)' \
'WINDMC=$(WINDMC)'
'WINDMC=$(WINDMC)' \
'CREATE_GCOV=$(CREATE_GCOV)'
FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS)
@ -1147,6 +1158,7 @@ all-stage[+id+]-[+prefix+][+module+]: configure-stage[+id+]-[+prefix+][+module+]
[+exports+][+ IF prev +] \
[+poststage1_exports+][+ ENDIF prev +] [+extra_exports+] \
cd [+subdir+]/[+module+] && \
[+autoprofile+] \
$(MAKE) $(BASE_FLAGS_TO_PASS)[+ IF prefix +] \
CFLAGS="$(CFLAGS_FOR_TARGET)" \
CXXFLAGS="$(CXXFLAGS_FOR_TARGET)" \
@ -1160,7 +1172,7 @@ all-stage[+id+]-[+prefix+][+module+]: configure-stage[+id+]-[+prefix+][+module+]
LIBCFLAGS_FOR_TARGET="$(LIBCFLAGS_FOR_TARGET)" \
[+args+] [+IF prev +][+poststage1_args+][+ ELSE prev +] \
[+stage1_args+][+ ENDIF prev +] [+extra_make_flags+] \
TFLAGS="$(STAGE[+id+]_TFLAGS)" \
TFLAGS="$(STAGE[+id+]_TFLAGS)" [+profile_data+] \
$(TARGET-stage[+id+]-[+prefix+][+module+])
maybe-clean-stage[+id+]-[+prefix+][+module+]: clean-stage[+id+]-[+prefix+][+module+]

5
gcc/ChangeLog
View file

@ -1,3 +1,8 @@
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Makefile.in: Regenerate.
* doc/install.texi: Document autoprofiledbootstrap.
2016-06-23 Andi Kleen <ak@linux.intel.com>
* config/i386/gcc-auto-profile: New file.

9
gcc/Makefile.in
View file

@ -1567,6 +1567,13 @@ ALL_HOST_BACKEND_OBJS = $(GCC_OBJS) $(OBJS) $(OBJS-libcommon) \
$(GCOV_TOOL_OBJS) $(GENGTYPE_OBJS) gcc-ar.o gcc-nm.o gcc-ranlib.o \
lto-wrapper.o collect-utils.o
# for anything that is shared use the cc1plus profile data, as that
# is likely the most exercised during the build
ifeq ($(shell cat ../stage_current),stageautofeedback)
$(ALL_HOST_BACKEND_OBJS): ALL_COMPILERFLAGS += -fauto-profile=cc1plus.fda
$(ALL_HOST_BACKEND_OBJS): cc1plus.fda
endif
# This lists all host object files, whether they are included in this
# compilation or not.
ALL_HOST_OBJS = $(ALL_HOST_FRONTEND_OBJS) $(ALL_HOST_BACKEND_OBJS)
@ -1596,7 +1603,7 @@ MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
gcov-iov$(build_exeext) gcov$(exeext) gcov-dump$(exeext) \
gcov-tool$(exeect) \
gengtype$(exeext) *.[0-9][0-9].* *.[si] *-checksum.c libbackend.a \
libcommon-target.a libcommon.a libgcc.mk
libcommon-target.a libcommon.a libgcc.mk perf.data
# This symlink makes the full installation name of the driver be available
# from within the *build* directory, for use when running the JIT library

4
gcc/c/ChangeLog
View file

@ -1,3 +1,7 @@
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Make-lang.in: Add support for autofdo.
2016-06-22 David Malcolm <dmalcolm@redhat.com>
PR c/70339

14
gcc/c/Make-lang.in
View file

@ -60,6 +60,11 @@ c_OBJS = $(C_OBJS) cc1-checksum.o c/gccspec.o
# Use strict warnings for this front end.
c-warn = $(STRICT_WARN)
ifeq ($(shell cat ../stage_current),stageautofeedback)
$(C_OBJS): ALL_COMPILERFLAGS += -fauto-profile=cc1.fda
$(C_OBJS): cc1.fda
endif
# compute checksum over all object files and the options
# re-use the checksum from the prev-final stage so it passes
# the bootstrap comparison and allows comparing of the cc1 binary
@ -77,6 +82,10 @@ cc1-checksum.c : build/genchecksum$(build_exeext) checksum-options \
cc1$(exeext): $(C_OBJS) cc1-checksum.o $(BACKEND) $(LIBDEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ $(C_OBJS) \
cc1-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
cc1.fda: ../stage1-gcc/cc1$(exeext) ../prev-gcc/$(PERF_DATA)
$(CREATE_GCOV) -binary ../stage1-gcc/cc1$(exeext) -gcov cc1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
#
# Build hooks:
@ -126,6 +135,7 @@ c.mostlyclean:
-rm -f cc1$(exeext)
-rm -f c/*$(objext)
-rm -f c/*$(coverageexts)
-rm -f cc1.fda
c.clean:
c.distclean:
-rm -f c/config.status c/Makefile
@ -146,3 +156,7 @@ c.stageprofile: stageprofile-start
-mv c/*$(objext) stageprofile/c
c.stagefeedback: stagefeedback-start
-mv c/*$(objext) stagefeedback/c
c.autostageprofile: autostageprofile-start
-mv c/*$(objext) autostageprofile/c
c.autostagefeedback: autostagefeedback-start
-mv c/*$(objext) autostagefeedback/c

4
gcc/cp/ChangeLog
View file

@ -1,3 +1,7 @@
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Make-lang.in: Add support for autofdo.
2016-06-21 Jason Merrill <jason@redhat.com>
* constraint.cc (constraints_satisfied_p): Keep as many levels of

14
gcc/cp/Make-lang.in
View file

@ -81,6 +81,11 @@ CXX_AND_OBJCXX_OBJS = cp/call.o cp/decl.o cp/expr.o cp/pt.o cp/typeck2.o \
cp/vtable-class-hierarchy.o cp/constexpr.o cp/cp-ubsan.o \
cp/constraint.o cp/logic.o $(CXX_C_OBJS)
ifeq ($(shell cat ../stage_current),stageautofeedback)
$(CXX_AND_OBJCXX_OBJS): CFLAGS += -fauto-profile=cc1plus.fda
$(CXX_AND_OBJCXX_OBJS): cc1plus.fda
endif
# Language-specific object files for C++.
CXX_OBJS = cp/cp-lang.o c-family/stub-objc.o $(CXX_AND_OBJCXX_OBJS)
@ -122,6 +127,9 @@ endif
gperf -o -C -E -k '1-6,$$' -j1 -D -N 'libc_name_p' -L C++ \
$(srcdir)/cp/cfns.gperf --output-file $(srcdir)/cp/cfns.h
cc1plus.fda: ../stage1-gcc/cc1plus$(exeext) ../prev-gcc/$(PERF_DATA)
$(CREATE_GCOV) -binary ../stage1-gcc/cc1plus$(exeext) -gcov cc1plus.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
#
# Build hooks:
@ -235,7 +243,7 @@ c++.mostlyclean:
-rm -f doc/g++.1
-rm -f cp/*$(objext)
-rm -f cp/*$(coverageexts)
-rm -f xg++$(exeext) g++-cross$(exeext) cc1plus$(exeext)
-rm -f xg++$(exeext) g++-cross$(exeext) cc1plus$(exeext) cc1plus.fda
c++.clean:
c++.distclean:
-rm -f cp/config.status cp/Makefile
@ -257,3 +265,7 @@ c++.stageprofile: stageprofile-start
-mv cp/*$(objext) stageprofile/cp
c++.stagefeedback: stagefeedback-start
-mv cp/*$(objext) stagefeedback/cp
c++.autostageprofile: stageprofile-start
-mv cp/*$(objext) autostageprofile/cp
c++.autostagefeedback: stagefeedback-start
-mv cp/*$(objext) autostagefeedback/cp

11
gcc/doc/install.texi
View file

@ -2776,6 +2776,17 @@ Unlike standard bootstrap, several additional restrictions apply. The
compiler used to build @code{stage1} needs to support a 64-bit integral type.
It is recommended to only use GCC for this.
On Linux/x86_64 hosts with some restrictions (no virtualization) it is
also possible to do autofdo build with @samp{make
autoprofiledback}. This uses Linux perf to sample branches in the
binary and then rebuild it with feedback derived from the profile.
Linux perf and the @code{autofdo} toolkit needs to be installed for
this.
Only the profile from the current build is used, so when an error
occurs it is recommended to clean before restarting. Otherwise
the code quality may be much worse.
@html
<hr />
<p>

4
gcc/lto/ChangeLog
View file

@ -1,3 +1,7 @@
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Make-lang.in: Add support for autofdo (disabled for now)
2016-06-10 Martin Sebor <msebor@redhat.com>
PR c/71392

14
gcc/lto/Make-lang.in
View file

@ -25,6 +25,15 @@ LTO_EXE = lto1$(exeext)
LTO_OBJS = lto/lto-lang.o lto/lto.o lto/lto-object.o attribs.o lto/lto-partition.o lto/lto-symtab.o
lto_OBJS = $(LTO_OBJS)
# this is only useful in a LTO bootstrap, but this does not work right
# now. Should reenable after this is fixed, but only when LTO bootstrap
# is enabled.
#ifeq ($(shell cat ../stage_current),stageautofeedback)
#$(LTO_OBJS): CFLAGS += -fauto-profile=lto1.fda
#$(LTO_OBJS): lto1.fda
#endif
# Rules
# These hooks are used by the main GCC Makefile. Consult that
@ -50,7 +59,7 @@ lto.srcinfo:
lto.install-plugin:
lto.mostlyclean:
rm -f $(LTO_OBJS) $(LTO_EXE)
rm -f $(LTO_OBJS) $(LTO_EXE) lto1.fda
lto.clean:
lto.distclean:
@ -71,5 +80,8 @@ $(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
lto1.fda: ../prev-gcc/lto1$(exeext) ../prev-gcc/$(PERF_DATA)
$(CREATE_GCOV) -binary ../prev-gcc/lto1$(exeext) -gcov lto1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
# LTO testing is done as part of C/C++/Fortran etc. testing.
check-lto: