procyberian/gcc
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Makefile.in (omp-low.o): Depend on $(TARGET_H).

Browse source 
* Makefile.in (omp-low.o): Depend on $(TARGET_H).
	* cfgloop.h (struct loop): Add safelen, force_vect, simduid.
	* function.h (struct function): Add has_force_vect_loops and
	has_simduid_loops.
	* gimple-pretty-print.c (dump_gimple_omp_for): Handle
	GF_OMP_FOR_KIND*.
	* gimple.c (gimple_build_omp_critical): Add KIND argument and
	handle it.
	* gimple.def: Update CLAUSES comments.
	* gimple.h (enum gf_mask): Add GF_OMP_FOR_KIND_{FOR,SIMD}.
	(gimple_build_omp_for): Add argument to prototype.
	(gimple_omp_for_kind): New.
	(gimple_omp_for_set_kind): New.
	* gimplify.c (enum gimplify_omp_var_data): Add GOVD_LINEAR to
	GOVD_DATA_SHARE_CLASS.
	(enum omp_region_type): Add ORT_SIMD.
	(gimple_add_tmp_var): Handle ORT_SIMD.
	(gimplify_var_or_parm_decl): Same.
	(is_gimple_stmt): Same.
	(omp_firstprivatize_variable): Same.
	(omp_add_variable): Only use splay_tree_insert if lookup failed.
	(omp_notice_variable): Handle ORT_SIMD.
	(omp_is_private): Add SIMD argument and handle it as well as
	ORT_SIMD.
	(omp_check_private): Handle ORT_SIMD.
	(gimplify_scan_omp_clauses): Handle OMP_CLAUSE_LINEAR and
	OMP_CLAUSE_SAFELEN.
	(gimplify_adjust_omp_clauses_1): Handle GOVD_LINEAR.
	Handle OMP_CLAUSE_LASTPRIVATE.
	(gimplify_adjust_omp_clauses): Handle OMP_CLAUSE_LINEAR and
	OMP_CLAUSE_SAFELEN.
	(gimplify_omp_for): Handle OMP_SIMD and OMP_CLAUSE_LINEAR.
	(gimplify_expr): Handle OMP_SIMD.
	* internal-fn.c (expand_GOMP_SIMD_LANE): New.
	(expand_GOMP_SIMD_VF): New.
	(expand_GOMP_SIMD_LAST_LANE): New.
	* internal-fn.def (GOMP_SIMD_LANE): New.
	(GOMP_SIMD_VF): New.
	(GOMP_SIMD_LAST_LANE): New.
	* omp-low.c: Include target.h.
	(extract_omp_for_data): Handle OMP_SIMD, OMP_CLAUSE_LINEAR,
	OMP_CLAUSE_SAFELEN.
	(check_omp_nesting_restrictions): Same.
	(omp_max_vf): New.
	(lower_rec_simd_input_clauses): New.
	(lower_rec_input_clauses): Handle OMP_SIMD, GF_OMP_FOR_KIND_SIMD,
	OMP_CLAUSE_LINEAR.
	(lower_lastprivate_clauses): Handle OMP_CLAUSE_LINEAR,
	GF_OMP_FOR_KIND_SIMD, OMP_SIMD.
	(expand_omp_build_assign): New.
	(expand_omp_for_init_counts): New.
	(expand_omp_for_init_vars): New.
	(extract_omp_for_update_vars): New.
	(expand_omp_for_generic): Use expand_omp_for_{init,update}_vars
	and rewrite accordingly.
	(expand_omp_simd): New.
	(expand_omp_for): Use expand_omp_simd.
	(lower_omp_for_lastprivate): Unshare vinit when appropriate.
	(lower_omp_for): Do not lower the body.
	* tree-data-ref (get_references_in_stmt): Allow IFN_GOMP_SIMD_LANE
	in their own loops.
	* tree-flow.h (find_omp_clause): Remove prototype.
	* tree-if-conv.c (main_tree_if_conversion): Run if doing if
	conversion, forcing vectorization of the loop, or if
	flag_tree_vectorize.
	(gate_tree_if_conversion): Similarly.
	* tree-inline.c (remap_gimple_stmt): Pass for kind argument to
	gimple_build_omp_for.
	(copy_cfg_body): set has_force_vect_loops and has_simduid_loops.
	* tree-parloops (create_parallel_loop): Pass kind argument to
	gimple_build_omp_for.
	* tree-pretty-print.c (dump_omp_clause): Add cases for
	OMP_CLAUSE_UNIFORM, OMP_CLAUSE_LINEAR, OMP_CLAUSE_SAFELEN,
	OMP_CLAUSE__SIMDUID_.
	(dump_generic_node): Handle OMP_SIMD.
	* tree-ssa-ccp.c (likely_value): Handle IFN_GOMP_SIMD*.
	* tree-ssa-loop-ivcanon.c (tree_unroll_loops_completely_1): Do not
	unroll OMP_SIMD loops here.
	* tree-ssa-loop.c (gate_tree_vectorize): Run if
	has_force_vect_loops.
	* tree-vect-data-refs.c (vect_analyze_data_ref_dependence): Handle
	loop->safelen
	(vect_analyze_data_refs): Handle simd loops.
	* tree-vect-loop.c (vectorizable_live_operation): Handle
	IFN_GOMP_SIMD*.
	* tree-vect-stmts.c (vectorizable_call): Handle
	IFN_GOMP_SIMD_LANE.
	(vectorizable_store): Handle STMT_VINFO_SIMD_LANE_ACCESS_P.
	(vectorizable_load): Same.
	* tree-vectorizer.c: Include hash-table.h and
	tree-ssa-propagate.h.
	(struct simduid_to_vf): New.
	(simduid_to_vf::hash): New.
	(simduid_to-vf::equal): New.
	(struct simd_array_to_simduid): New.
	(simd_array_to_simduid::hash): New.
	(simd_array_to_simduid::equal): New.
	(adjust_simduid_builtins): New.
	(struct note_simd_array_uses_struct): New.
	(note_simd_array_uses_cb): New.
	(note_simd_array_uses): New.
	(vectorize_loops): Handle simd hints and adjust simd builtins
	accordingly.
	* tree-vectorizer.h (struct _stmt_vec_info): Add
	simd_lane_access_p field.
	(STMT_VINFO_SIMD_LANE_ACCESS_P): New macro.
	* tree.c (omp_clause_num_ops): Add entries for OMP_CLAUSE_LINEAR,
	OMP_CLAUSE_SAFELEN, OMP_CLAUSE__SIMDUID_, OMP_CLAUSE_UNIFORM.
	(omp_clause_code_name): Same.
	(walk_tree_1): Handle OMP_CLAUSE_UNIFORM, OMP_CLAUSE_SAFELEN,
	OMP_CLAUSE__SIMDUID_, OMP_CLAUSE_LINEAR.
	* tree.def (OMP_SIMD): New entry.
	* tree.h (enum omp_clause_code): Add entries for
	OMP_CLAUSE_LINEAR, OMP_CLAUSE_UNIFORM, OMP_CLAUSE_SAFELEN,
	OMP_CLAUSE__SIMDUID_.
	(OMP_CLAUSE_DECL): Adjust range for new clauses.
	(OMP_CLAUSE_LINEAR_NO_COPYIN): New.
	(OMP_CLAUSE_LINEAR_NO_COPYOUT): New.
	(OMP_CLAUSE_LINEAR_STEP): New.
	(OMP_CLAUSE_SAFELEN_EXPR): New.
	(OMP_CLAUSE__SIMDUID__DECL): New.
	(find_omp_clause): New prototype.
cp/
	* cp-tree.h (CP_OMP_CLAUSE_INFO): Adjust range for new clauses.

Co-Authored-By: Aldy Hernandez <aldyh@redhat.com>

From-SVN: r202029
This commit is contained in:
Jakub Jelinek 2013-08-27 20:38:00 +02:00 • committed by Aldy Hernandez
parent d0cf4e841d
commit 74bf76ed78
31 changed files with 2153 additions and 335 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

126
gcc/ChangeLog
View file

@ -1,3 +1,129 @@
2013-08-27 Jakub Jelinek <jakub@redhat.com>
Aldy Hernandez <aldyh@redhat.com>
* Makefile.in (omp-low.o): Depend on $(TARGET_H).
* cfgloop.h (struct loop): Add safelen, force_vect, simduid.
* function.h (struct function): Add has_force_vect_loops and
has_simduid_loops.
* gimple-pretty-print.c (dump_gimple_omp_for): Handle
GF_OMP_FOR_KIND*.
* gimple.c (gimple_build_omp_critical): Add KIND argument and
handle it.
* gimple.def: Update CLAUSES comments.
* gimple.h (enum gf_mask): Add GF_OMP_FOR_KIND_{FOR,SIMD}.
(gimple_build_omp_for): Add argument to prototype.
(gimple_omp_for_kind): New.
(gimple_omp_for_set_kind): New.
* gimplify.c (enum gimplify_omp_var_data): Add GOVD_LINEAR to
GOVD_DATA_SHARE_CLASS.
(enum omp_region_type): Add ORT_SIMD.
(gimple_add_tmp_var): Handle ORT_SIMD.
(gimplify_var_or_parm_decl): Same.
(is_gimple_stmt): Same.
(omp_firstprivatize_variable): Same.
(omp_add_variable): Only use splay_tree_insert if lookup failed.
(omp_notice_variable): Handle ORT_SIMD.
(omp_is_private): Add SIMD argument and handle it as well as
ORT_SIMD.
(omp_check_private): Handle ORT_SIMD.
(gimplify_scan_omp_clauses): Handle OMP_CLAUSE_LINEAR and
OMP_CLAUSE_SAFELEN.
(gimplify_adjust_omp_clauses_1): Handle GOVD_LINEAR.
Handle OMP_CLAUSE_LASTPRIVATE.
(gimplify_adjust_omp_clauses): Handle OMP_CLAUSE_LINEAR and
OMP_CLAUSE_SAFELEN.
(gimplify_omp_for): Handle OMP_SIMD and OMP_CLAUSE_LINEAR.
(gimplify_expr): Handle OMP_SIMD.
* internal-fn.c (expand_GOMP_SIMD_LANE): New.
(expand_GOMP_SIMD_VF): New.
(expand_GOMP_SIMD_LAST_LANE): New.
* internal-fn.def (GOMP_SIMD_LANE): New.
(GOMP_SIMD_VF): New.
(GOMP_SIMD_LAST_LANE): New.
* omp-low.c: Include target.h.
(extract_omp_for_data): Handle OMP_SIMD, OMP_CLAUSE_LINEAR,
OMP_CLAUSE_SAFELEN.
(check_omp_nesting_restrictions): Same.
(omp_max_vf): New.
(lower_rec_simd_input_clauses): New.
(lower_rec_input_clauses): Handle OMP_SIMD, GF_OMP_FOR_KIND_SIMD,
OMP_CLAUSE_LINEAR.
(lower_lastprivate_clauses): Handle OMP_CLAUSE_LINEAR,
GF_OMP_FOR_KIND_SIMD, OMP_SIMD.
(expand_omp_build_assign): New.
(expand_omp_for_init_counts): New.
(expand_omp_for_init_vars): New.
(extract_omp_for_update_vars): New.
(expand_omp_for_generic): Use expand_omp_for_{init,update}_vars
and rewrite accordingly.
(expand_omp_simd): New.
(expand_omp_for): Use expand_omp_simd.
(lower_omp_for_lastprivate): Unshare vinit when appropriate.
(lower_omp_for): Do not lower the body.
* tree-data-ref (get_references_in_stmt): Allow IFN_GOMP_SIMD_LANE
in their own loops.
* tree-flow.h (find_omp_clause): Remove prototype.
* tree-if-conv.c (main_tree_if_conversion): Run if doing if
conversion, forcing vectorization of the loop, or if
flag_tree_vectorize.
(gate_tree_if_conversion): Similarly.
* tree-inline.c (remap_gimple_stmt): Pass for kind argument to
gimple_build_omp_for.
(copy_cfg_body): set has_force_vect_loops and has_simduid_loops.
* tree-parloops (create_parallel_loop): Pass kind argument to
gimple_build_omp_for.
* tree-pretty-print.c (dump_omp_clause): Add cases for
OMP_CLAUSE_UNIFORM, OMP_CLAUSE_LINEAR, OMP_CLAUSE_SAFELEN,
OMP_CLAUSE__SIMDUID_.
(dump_generic_node): Handle OMP_SIMD.
* tree-ssa-ccp.c (likely_value): Handle IFN_GOMP_SIMD*.
* tree-ssa-loop-ivcanon.c (tree_unroll_loops_completely_1): Do not
unroll OMP_SIMD loops here.
* tree-ssa-loop.c (gate_tree_vectorize): Run if
has_force_vect_loops.
* tree-vect-data-refs.c (vect_analyze_data_ref_dependence): Handle
loop->safelen
(vect_analyze_data_refs): Handle simd loops.
* tree-vect-loop.c (vectorizable_live_operation): Handle
IFN_GOMP_SIMD*.
* tree-vect-stmts.c (vectorizable_call): Handle
IFN_GOMP_SIMD_LANE.
(vectorizable_store): Handle STMT_VINFO_SIMD_LANE_ACCESS_P.
(vectorizable_load): Same.
* tree-vectorizer.c: Include hash-table.h and
tree-ssa-propagate.h.
(struct simduid_to_vf): New.
(simduid_to_vf::hash): New.
(simduid_to-vf::equal): New.
(struct simd_array_to_simduid): New.
(simd_array_to_simduid::hash): New.
(simd_array_to_simduid::equal): New.
(adjust_simduid_builtins): New.
(struct note_simd_array_uses_struct): New.
(note_simd_array_uses_cb): New.
(note_simd_array_uses): New.
(vectorize_loops): Handle simd hints and adjust simd builtins
accordingly.
* tree-vectorizer.h (struct _stmt_vec_info): Add
simd_lane_access_p field.
(STMT_VINFO_SIMD_LANE_ACCESS_P): New macro.
* tree.c (omp_clause_num_ops): Add entries for OMP_CLAUSE_LINEAR,
OMP_CLAUSE_SAFELEN, OMP_CLAUSE__SIMDUID_, OMP_CLAUSE_UNIFORM.
(omp_clause_code_name): Same.
(walk_tree_1): Handle OMP_CLAUSE_UNIFORM, OMP_CLAUSE_SAFELEN,
OMP_CLAUSE__SIMDUID_, OMP_CLAUSE_LINEAR.
* tree.def (OMP_SIMD): New entry.
* tree.h (enum omp_clause_code): Add entries for
OMP_CLAUSE_LINEAR, OMP_CLAUSE_UNIFORM, OMP_CLAUSE_SAFELEN,
OMP_CLAUSE__SIMDUID_.
(OMP_CLAUSE_DECL): Adjust range for new clauses.
(OMP_CLAUSE_LINEAR_NO_COPYIN): New.
(OMP_CLAUSE_LINEAR_NO_COPYOUT): New.
(OMP_CLAUSE_LINEAR_STEP): New.
(OMP_CLAUSE_SAFELEN_EXPR): New.
(OMP_CLAUSE__SIMDUID__DECL): New.
(find_omp_clause): New prototype.
2013-08-27 H.J. Lu <hongjiu.lu@intel.com>
* config/i386/driver-i386.c (host_detect_local_cpu): Update

2
gcc/Makefile.in
View file

@ -2565,7 +2565,7 @@ omp-low.o : omp-low.c $(CONFIG_H) $(SYSTEM_H) coretypes.h $(TM_H) $(TREE_H) \
$(RTL_H) $(GIMPLE_H) $(TREE_INLINE_H) langhooks.h $(DIAGNOSTIC_CORE_H) \
$(TREE_FLOW_H) $(FLAGS_H) $(EXPR_H) $(DIAGNOSTIC_CORE_H) \
$(TREE_PASS_H) $(GGC_H) $(EXCEPT_H) $(SPLAY_TREE_H) $(OPTABS_H) \
$(CFGLOOP_H) tree-iterator.h gt-omp-low.h
$(CFGLOOP_H) tree-iterator.h $(TARGET_H) gt-omp-low.h
tree-browser.o : tree-browser.c tree-browser.def $(CONFIG_H) $(SYSTEM_H) \
coretypes.h $(HASH_TABLE_H) $(TREE_H) $(TREE_PRETTY_PRINT_H)
omega.o : omega.c $(OMEGA_H) $(CONFIG_H) $(SYSTEM_H) coretypes.h $(DUMPFILE_H) \

14
gcc/cfgloop.h
View file

@ -168,6 +168,20 @@ struct GTY ((chain_next ("%h.next"))) loop {
describes what is the state of the estimation. */
enum loop_estimation estimate_state;
/* If > 0, an integer, where the user asserted that for any
I in [ 0, nb_iterations ) and for any J in
[ I, min ( I + safelen, nb_iterations ) ), the Ith and Jth iterations
of the loop can be safely evaluated concurrently. */
int safelen;
/* True if we should try harder to vectorize this loop. */
bool force_vect;
/* For SIMD loops, this is a unique identifier of the loop, referenced
by IFN_GOMP_SIMD_VF, IFN_GOMP_SIMD_LANE and IFN_GOMP_SIMD_LAST_LANE
builtins. */
tree simduid;
/* Upper bound on number of iterations of a loop. */
struct nb_iter_bound *bounds;

5
gcc/cp/ChangeLog
View file

@ -1,3 +1,8 @@
2013-08-27 Jakub Jelinek <jakub@redhat.com>
Aldy Hernandez <aldyh@redhat.com>
* cp-tree.h (CP_OMP_CLAUSE_INFO): Adjust range for new clauses.
2013-08-27 Paolo Carlini <paolo.carlini@oracle.com>
* decl.c (grokfndecl): Remove old bison hack.

2
gcc/cp/cp-tree.h
View file

@ -4020,7 +4020,7 @@ more_aggr_init_expr_args_p (const aggr_init_expr_arg_iterator *iter)
See semantics.c for details. */
#define CP_OMP_CLAUSE_INFO(NODE) \
TREE_TYPE (OMP_CLAUSE_RANGE_CHECK (NODE, OMP_CLAUSE_PRIVATE, \
OMP_CLAUSE_COPYPRIVATE))
OMP_CLAUSE_LINEAR))
/* Nonzero if this transaction expression's body contains statements. */
#define TRANSACTION_EXPR_IS_STMT(NODE) \

8
gcc/function.h
View file

@ -650,6 +650,14 @@ struct GTY(()) function {
adjusts one of its arguments and forwards to another
function. */
unsigned int is_thunk : 1;
/* Nonzero if the current function contains any loops with
loop->force_vect set. */
unsigned int has_force_vect_loops : 1;
/* Nonzero if the current function contains any loops with
nonzero value in loop->simduid. */
unsigned int has_simduid_loops : 1;
};
/* Add the decl D to the local_decls list of FUN. */

28
gcc/gimple-pretty-print.c
View file

@ -1088,8 +1088,20 @@ dump_gimple_omp_for (pretty_printer *buffer, gimple gs, int spc, int flags)
if (flags & TDF_RAW)
{
dump_gimple_fmt (buffer, spc, flags, "%G <%+BODY <%S>%nCLAUSES <", gs,
gimple_omp_body (gs));
const char *kind;
switch (gimple_omp_for_kind (gs))
{
case GF_OMP_FOR_KIND_FOR:
kind = "";
break;
case GF_OMP_FOR_KIND_SIMD:
kind = " simd";
break;
default:
gcc_unreachable ();
}
dump_gimple_fmt (buffer, spc, flags, "%G%s <%+BODY <%S>%nCLAUSES <", gs,
kind, gimple_omp_body (gs));
dump_omp_clauses (buffer, gimple_omp_for_clauses (gs), spc, flags);
dump_gimple_fmt (buffer, spc, flags, " >,");
for (i = 0; i < gimple_omp_for_collapse (gs); i++)
@ -1105,7 +1117,17 @@ dump_gimple_omp_for (pretty_printer *buffer, gimple gs, int spc, int flags)
}
else
{
pp_string (buffer, "#pragma omp for");
switch (gimple_omp_for_kind (gs))
{
case GF_OMP_FOR_KIND_FOR:
pp_string (buffer, "#pragma omp for");
break;
case GF_OMP_FOR_KIND_SIMD:
pp_string (buffer, "#pragma omp simd");
break;
default:
gcc_unreachable ();
}
dump_omp_clauses (buffer, gimple_omp_for_clauses (gs), spc, flags);
for (i = 0; i < gimple_omp_for_collapse (gs); i++)
{

4
gcc/gimple.c
View file

@ -902,19 +902,21 @@ gimple_build_omp_critical (gimple_seq body, tree name)
/* Build a GIMPLE_OMP_FOR statement.
BODY is sequence of statements inside the for loop.
KIND is the `for' variant.
CLAUSES, are any of the OMP loop construct's clauses: private, firstprivate,
lastprivate, reductions, ordered, schedule, and nowait.
COLLAPSE is the collapse count.
PRE_BODY is the sequence of statements that are loop invariant. */
gimple
gimple_build_omp_for (gimple_seq body, tree clauses, size_t collapse,
gimple_build_omp_for (gimple_seq body, int kind, tree clauses, size_t collapse,
gimple_seq pre_body)
{
gimple p = gimple_alloc (GIMPLE_OMP_FOR, 0);
if (body)
gimple_omp_set_body (p, body);
gimple_omp_for_set_clauses (p, clauses);
gimple_omp_for_set_kind (p, kind);
p->gimple_omp_for.collapse = collapse;
p->gimple_omp_for.iter
= ggc_alloc_cleared_vec_gimple_omp_for_iter (collapse);

8
gcc/gimple.def
View file

@ -287,7 +287,7 @@ DEFGSCODE(GIMPLE_OMP_ORDERED, "gimple_omp_ordered", GSS_OMP)
BODY is a the sequence of statements to be executed by all threads.
CLAUSES is a TREE_LIST node with all the clauses.
CLAUSES is an OMP_CLAUSE chain with all the clauses.
CHILD_FN is set when outlining the body of the parallel region.
All the statements in BODY are moved into this newly created
@ -306,7 +306,7 @@ DEFGSCODE(GIMPLE_OMP_PARALLEL, "gimple_omp_parallel", GSS_OMP_PARALLEL)
BODY is a the sequence of statements to be executed by all threads.
CLAUSES is a TREE_LIST node with all the clauses.
CLAUSES is an OMP_CLAUSE chain with all the clauses.
CHILD_FN is set when outlining the body of the explicit task region.
All the statements in BODY are moved into this newly created
@ -334,7 +334,7 @@ DEFGSCODE(GIMPLE_OMP_SECTION, "gimple_omp_section", GSS_OMP)
/* OMP_SECTIONS <BODY, CLAUSES, CONTROL> represents #pragma omp sections.
BODY is the sequence of statements in the sections body.
CLAUSES is a TREE_LIST node holding the list of associated clauses.
CLAUSES is an OMP_CLAUSE chain holding the list of associated clauses.
CONTROL is a VAR_DECL used for deciding which of the sections
to execute. */
DEFGSCODE(GIMPLE_OMP_SECTIONS, "gimple_omp_sections", GSS_OMP_SECTIONS)
@ -346,7 +346,7 @@ DEFGSCODE(GIMPLE_OMP_SECTIONS_SWITCH, "gimple_omp_sections_switch", GSS_BASE)
/* GIMPLE_OMP_SINGLE <BODY, CLAUSES> represents #pragma omp single
BODY is the sequence of statements inside the single section.
CLAUSES is a TREE_LIST node holding the associated clauses. */
CLAUSES is an OMP_CLAUSE chain holding the associated clauses. */
DEFGSCODE(GIMPLE_OMP_SINGLE, "gimple_omp_single", GSS_OMP_SINGLE)
/* GIMPLE_PREDICT <PREDICT, OUTCOME> specifies a hint for branch prediction.

26
gcc/gimple.h
View file

@ -110,6 +110,9 @@ enum gf_mask {
GF_CALL_ALLOCA_FOR_VAR = 1 << 5,
GF_CALL_INTERNAL = 1 << 6,
GF_OMP_PARALLEL_COMBINED = 1 << 0,
GF_OMP_FOR_KIND_MASK = 3 << 0,
GF_OMP_FOR_KIND_FOR = 0 << 0,
GF_OMP_FOR_KIND_SIMD = 1 << 0,
/* True on an GIMPLE_OMP_RETURN statement if the return does not require
a thread synchronization via some sort of barrier. The exact barrier
@ -799,7 +802,7 @@ gimple gimple_build_switch_nlabels (unsigned, tree, tree);
gimple gimple_build_switch (tree, tree, vec<tree> );
gimple gimple_build_omp_parallel (gimple_seq, tree, tree, tree);
gimple gimple_build_omp_task (gimple_seq, tree, tree, tree, tree, tree, tree);
gimple gimple_build_omp_for (gimple_seq, tree, size_t, gimple_seq);
gimple gimple_build_omp_for (gimple_seq, int, tree, size_t, gimple_seq);
gimple gimple_build_omp_critical (gimple_seq, tree);
gimple gimple_build_omp_section (gimple_seq);
gimple gimple_build_omp_continue (tree, tree);
@ -3948,6 +3951,27 @@ gimple_omp_critical_set_name (gimple gs, tree name)
}
/* Return the kind of OMP for statemement. */
static inline int
gimple_omp_for_kind (const_gimple g)
{
GIMPLE_CHECK (g, GIMPLE_OMP_FOR);
return (gimple_omp_subcode (g) & GF_OMP_FOR_KIND_MASK);
}
/* Set the OMP for kind. */
static inline void
gimple_omp_for_set_kind (gimple g, int kind)
{
GIMPLE_CHECK (g, GIMPLE_OMP_FOR);
g->gsbase.subcode = (g->gsbase.subcode & ~GF_OMP_FOR_KIND_MASK)
| (kind & GF_OMP_FOR_KIND_MASK);
}
/* Return the clauses associated with OMP_FOR GS. */
static inline tree

200
gcc/gimplify.c
View file

@ -58,14 +58,17 @@ enum gimplify_omp_var_data
GOVD_LOCAL = 128,
GOVD_DEBUG_PRIVATE = 256,
GOVD_PRIVATE_OUTER_REF = 512,
GOVD_LINEAR = 2048,
GOVD_DATA_SHARE_CLASS = (GOVD_SHARED | GOVD_PRIVATE | GOVD_FIRSTPRIVATE
| GOVD_LASTPRIVATE | GOVD_REDUCTION | GOVD_LOCAL)
| GOVD_LASTPRIVATE | GOVD_REDUCTION | GOVD_LINEAR
| GOVD_LOCAL)
};
enum omp_region_type
{
ORT_WORKSHARE = 0,
ORT_SIMD = 1,
ORT_PARALLEL = 2,
ORT_COMBINED_PARALLEL = 3,
ORT_TASK = 4,
@ -710,7 +713,9 @@ gimple_add_tmp_var (tree tmp)
if (gimplify_omp_ctxp)
{
struct gimplify_omp_ctx *ctx = gimplify_omp_ctxp;
while (ctx && ctx->region_type == ORT_WORKSHARE)
while (ctx
&& (ctx->region_type == ORT_WORKSHARE
|| ctx->region_type == ORT_SIMD))
ctx = ctx->outer_context;
if (ctx)
omp_add_variable (ctx, tmp, GOVD_LOCAL | GOVD_SEEN);
@ -2061,7 +2066,9 @@ gimplify_var_or_parm_decl (tree *expr_p)
&& decl_function_context (decl) != current_function_decl)
{
struct gimplify_omp_ctx *ctx = gimplify_omp_ctxp;
while (ctx && ctx->region_type == ORT_WORKSHARE)
while (ctx
&& (ctx->region_type == ORT_WORKSHARE
|| ctx->region_type == ORT_SIMD))
ctx = ctx->outer_context;
if (!ctx && !pointer_set_insert (nonlocal_vlas, decl))
{
@ -4702,6 +4709,7 @@ is_gimple_stmt (tree t)
case STATEMENT_LIST:
case OMP_PARALLEL:
case OMP_FOR:
case OMP_SIMD:
case OMP_SECTIONS:
case OMP_SECTION:
case OMP_SINGLE:
@ -5714,7 +5722,8 @@ omp_firstprivatize_variable (struct gimplify_omp_ctx *ctx, tree decl)
else
return;
}
else if (ctx->region_type != ORT_WORKSHARE)
else if (ctx->region_type != ORT_WORKSHARE
&& ctx->region_type != ORT_SIMD)
omp_add_variable (ctx, decl, GOVD_FIRSTPRIVATE);
ctx = ctx->outer_context;
@ -5806,7 +5815,8 @@ omp_add_variable (struct gimplify_omp_ctx *ctx, tree decl, unsigned int flags)
FIRSTPRIVATE and LASTPRIVATE. */
nflags = n->value | flags;
gcc_assert ((nflags & GOVD_DATA_SHARE_CLASS)
== (GOVD_FIRSTPRIVATE | GOVD_LASTPRIVATE));
== (GOVD_FIRSTPRIVATE | GOVD_LASTPRIVATE)
|| (flags & GOVD_DATA_SHARE_CLASS) == 0);
n->value = nflags;
return;
}
@ -5870,7 +5880,10 @@ omp_add_variable (struct gimplify_omp_ctx *ctx, tree decl, unsigned int flags)
}
}
splay_tree_insert (ctx->variables, (splay_tree_key)decl, flags);
if (n != NULL)
n->value |= flags;
else
splay_tree_insert (ctx->variables, (splay_tree_key)decl, flags);
}
/* Notice a threadprivate variable DECL used in OpenMP context CTX.
@ -5936,7 +5949,8 @@ omp_notice_variable (struct gimplify_omp_ctx *ctx, tree decl, bool in_code)
enum omp_clause_default_kind default_kind, kind;
struct gimplify_omp_ctx *octx;
if (ctx->region_type == ORT_WORKSHARE)
if (ctx->region_type == ORT_WORKSHARE
|| ctx->region_type == ORT_SIMD)
goto do_outer;
/* ??? Some compiler-generated variables (like SAVE_EXPRs) could be
@ -6049,7 +6063,7 @@ omp_notice_variable (struct gimplify_omp_ctx *ctx, tree decl, bool in_code)
to the contrary in the innermost scope, generate an error. */
static bool
omp_is_private (struct gimplify_omp_ctx *ctx, tree decl)
omp_is_private (struct gimplify_omp_ctx *ctx, tree decl, bool simd)
{
splay_tree_node n;
@ -6060,8 +6074,12 @@ omp_is_private (struct gimplify_omp_ctx *ctx, tree decl)
{
if (ctx == gimplify_omp_ctxp)
{
error ("iteration variable %qE should be private",
DECL_NAME (decl));
if (simd)
error ("iteration variable %qE is predetermined linear",
DECL_NAME (decl));
else
error ("iteration variable %qE should be private",
DECL_NAME (decl));
n->value = GOVD_PRIVATE;
return true;
}
@ -6079,16 +6097,26 @@ omp_is_private (struct gimplify_omp_ctx *ctx, tree decl)
else if ((n->value & GOVD_REDUCTION) != 0)
error ("iteration variable %qE should not be reduction",
DECL_NAME (decl));
else if (simd && (n->value & GOVD_LASTPRIVATE) != 0)
error ("iteration variable %qE should not be lastprivate",
DECL_NAME (decl));
else if (simd && (n->value & GOVD_PRIVATE) != 0)
error ("iteration variable %qE should not be private",
DECL_NAME (decl));
else if (simd && (n->value & GOVD_LINEAR) != 0)
error ("iteration variable %qE is predetermined linear",
DECL_NAME (decl));
}
return (ctx == gimplify_omp_ctxp
|| (ctx->region_type == ORT_COMBINED_PARALLEL
&& gimplify_omp_ctxp->outer_context == ctx));
}
if (ctx->region_type != ORT_WORKSHARE)
if (ctx->region_type != ORT_WORKSHARE
&& ctx->region_type != ORT_SIMD)
return false;
else if (ctx->outer_context)
return omp_is_private (ctx->outer_context, decl);
return omp_is_private (ctx->outer_context, decl, simd);
return false;
}
@ -6113,7 +6141,8 @@ omp_check_private (struct gimplify_omp_ctx *ctx, tree decl)
if (n != NULL)
return (n->value & GOVD_SHARED) == 0;
}
while (ctx->region_type == ORT_WORKSHARE);
while (ctx->region_type == ORT_WORKSHARE
|| ctx->region_type == ORT_SIMD);
return false;
}
@ -6166,6 +6195,15 @@ gimplify_scan_omp_clauses (tree *list_p, gimple_seq *pre_p,
flags = GOVD_REDUCTION | GOVD_SEEN | GOVD_EXPLICIT;
check_non_private = "reduction";
goto do_add;
case OMP_CLAUSE_LINEAR:
if (gimplify_expr (&OMP_CLAUSE_LINEAR_STEP (c), pre_p, NULL,
is_gimple_val, fb_rvalue) == GS_ERROR)
{
remove = true;
break;
}
flags = GOVD_LINEAR | GOVD_EXPLICIT;
goto do_add;
do_add:
decl = OMP_CLAUSE_DECL (c);
@ -6264,6 +6302,7 @@ gimplify_scan_omp_clauses (tree *list_p, gimple_seq *pre_p,
case OMP_CLAUSE_UNTIED:
case OMP_CLAUSE_COLLAPSE:
case OMP_CLAUSE_MERGEABLE:
case OMP_CLAUSE_SAFELEN:
break;
case OMP_CLAUSE_DEFAULT:
@ -6321,7 +6360,8 @@ gimplify_adjust_omp_clauses_1 (splay_tree_node n, void *data)
splay_tree_node on
= splay_tree_lookup (ctx->variables, (splay_tree_key) decl);
if (on && (on->value & (GOVD_FIRSTPRIVATE | GOVD_LASTPRIVATE
| GOVD_PRIVATE | GOVD_REDUCTION)) != 0)
| GOVD_PRIVATE | GOVD_REDUCTION
| GOVD_LINEAR)) != 0)
break;
ctx = ctx->outer_context;
}
@ -6334,6 +6374,8 @@ gimplify_adjust_omp_clauses_1 (splay_tree_node n, void *data)
code = OMP_CLAUSE_PRIVATE;
else if (flags & GOVD_FIRSTPRIVATE)
code = OMP_CLAUSE_FIRSTPRIVATE;
else if (flags & GOVD_LASTPRIVATE)
code = OMP_CLAUSE_LASTPRIVATE;
else
gcc_unreachable ();
@ -6366,6 +6408,7 @@ gimplify_adjust_omp_clauses (tree *list_p)
case OMP_CLAUSE_PRIVATE:
case OMP_CLAUSE_SHARED:
case OMP_CLAUSE_FIRSTPRIVATE:
case OMP_CLAUSE_LINEAR:
decl = OMP_CLAUSE_DECL (c);
n = splay_tree_lookup (ctx->variables, (splay_tree_key) decl);
remove = !(n->value & GOVD_SEEN);
@ -6381,6 +6424,31 @@ gimplify_adjust_omp_clauses (tree *list_p)
OMP_CLAUSE_SET_CODE (c, OMP_CLAUSE_PRIVATE);
OMP_CLAUSE_PRIVATE_DEBUG (c) = 1;
}
if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_LINEAR
&& ctx->outer_context
&& !(OMP_CLAUSE_LINEAR_NO_COPYIN (c)
&& OMP_CLAUSE_LINEAR_NO_COPYOUT (c))
&& !is_global_var (decl))
{
if (ctx->outer_context->region_type == ORT_COMBINED_PARALLEL)
{
n = splay_tree_lookup (ctx->outer_context->variables,
(splay_tree_key) decl);
if (n == NULL
|| (n->value & GOVD_DATA_SHARE_CLASS) == 0)
{
int flags = OMP_CLAUSE_LINEAR_NO_COPYIN (c)
? GOVD_LASTPRIVATE : GOVD_SHARED;
if (n == NULL)
omp_add_variable (ctx->outer_context, decl,
flags | GOVD_SEEN);
else
n->value |= flags | GOVD_SEEN;
}
}
else
omp_notice_variable (ctx->outer_context, decl, true);
}
}
break;
@ -6406,6 +6474,7 @@ gimplify_adjust_omp_clauses (tree *list_p)
case OMP_CLAUSE_COLLAPSE:
case OMP_CLAUSE_FINAL:
case OMP_CLAUSE_MERGEABLE:
case OMP_CLAUSE_SAFELEN:
break;
default:
@ -6509,14 +6578,40 @@ gimplify_omp_for (tree *expr_p, gimple_seq *pre_p)
gimple gfor;
gimple_seq for_body, for_pre_body;
int i;
bool simd;
bitmap has_decl_expr = NULL;
for_stmt = *expr_p;
simd = TREE_CODE (for_stmt) == OMP_SIMD;
gimplify_scan_omp_clauses (&OMP_FOR_CLAUSES (for_stmt), pre_p,
ORT_WORKSHARE);
simd ? ORT_SIMD : ORT_WORKSHARE);
/* Handle OMP_FOR_INIT. */
for_pre_body = NULL;
if (simd && OMP_FOR_PRE_BODY (for_stmt))
{
has_decl_expr = BITMAP_ALLOC (NULL);
if (TREE_CODE (OMP_FOR_PRE_BODY (for_stmt)) == DECL_EXPR
&& TREE_CODE (DECL_EXPR_DECL (OMP_FOR_PRE_BODY (for_stmt)))
== VAR_DECL)
{
t = OMP_FOR_PRE_BODY (for_stmt);
bitmap_set_bit (has_decl_expr, DECL_UID (DECL_EXPR_DECL (t)));
}
else if (TREE_CODE (OMP_FOR_PRE_BODY (for_stmt)) == STATEMENT_LIST)
{
tree_stmt_iterator si;
for (si = tsi_start (OMP_FOR_PRE_BODY (for_stmt)); !tsi_end_p (si);
tsi_next (&si))
{
t = tsi_stmt (si);
if (TREE_CODE (t) == DECL_EXPR
&& TREE_CODE (DECL_EXPR_DECL (t)) == VAR_DECL)
bitmap_set_bit (has_decl_expr, DECL_UID (DECL_EXPR_DECL (t)));
}
}
}
gimplify_and_add (OMP_FOR_PRE_BODY (for_stmt), &for_pre_body);
OMP_FOR_PRE_BODY (for_stmt) = NULL_TREE;
@ -6535,7 +6630,44 @@ gimplify_omp_for (tree *expr_p, gimple_seq *pre_p)
|| POINTER_TYPE_P (TREE_TYPE (decl)));
/* Make sure the iteration variable is private. */
if (omp_is_private (gimplify_omp_ctxp, decl))
tree c = NULL_TREE;
if (simd)
{
splay_tree_node n = splay_tree_lookup (gimplify_omp_ctxp->variables,
(splay_tree_key)decl);
omp_is_private (gimplify_omp_ctxp, decl, simd);
if (n != NULL && (n->value & GOVD_DATA_SHARE_CLASS) != 0)
omp_notice_variable (gimplify_omp_ctxp, decl, true);
else if (TREE_VEC_LENGTH (OMP_FOR_INIT (for_stmt)) == 1)
{
c = build_omp_clause (input_location, OMP_CLAUSE_LINEAR);
OMP_CLAUSE_LINEAR_NO_COPYIN (c) = 1;
if (has_decl_expr
&& bitmap_bit_p (has_decl_expr, DECL_UID (decl)))
OMP_CLAUSE_LINEAR_NO_COPYOUT (c) = 1;
OMP_CLAUSE_DECL (c) = decl;
OMP_CLAUSE_CHAIN (c) = OMP_FOR_CLAUSES (for_stmt);
OMP_FOR_CLAUSES (for_stmt) = c;
omp_add_variable (gimplify_omp_ctxp, decl,
GOVD_LINEAR | GOVD_EXPLICIT | GOVD_SEEN);
}
else
{
bool lastprivate
= (!has_decl_expr
|| !bitmap_bit_p (has_decl_expr, DECL_UID (decl)));
c = build_omp_clause (input_location,
lastprivate ? OMP_CLAUSE_LASTPRIVATE
: OMP_CLAUSE_PRIVATE);
OMP_CLAUSE_DECL (c) = decl;
OMP_CLAUSE_CHAIN (c) = OMP_FOR_CLAUSES (for_stmt);
omp_add_variable (gimplify_omp_ctxp, decl,
(lastprivate ? GOVD_LASTPRIVATE : GOVD_PRIVATE)
| GOVD_SEEN);
c = NULL_TREE;
}
}
else if (omp_is_private (gimplify_omp_ctxp, decl, simd))
omp_notice_variable (gimplify_omp_ctxp, decl, true);
else
omp_add_variable (gimplify_omp_ctxp, decl, GOVD_PRIVATE | GOVD_SEEN);
@ -6577,6 +6709,8 @@ gimplify_omp_for (tree *expr_p, gimple_seq *pre_p)
case PREINCREMENT_EXPR:
case POSTINCREMENT_EXPR:
t = build_int_cst (TREE_TYPE (decl), 1);
if (c)
OMP_CLAUSE_LINEAR_STEP (c) = t;
t = build2 (PLUS_EXPR, TREE_TYPE (decl), var, t);
t = build2 (MODIFY_EXPR, TREE_TYPE (var), var, t);
TREE_VEC_ELT (OMP_FOR_INCR (for_stmt), i) = t;
@ -6585,6 +6719,8 @@ gimplify_omp_for (tree *expr_p, gimple_seq *pre_p)
case PREDECREMENT_EXPR:
case POSTDECREMENT_EXPR:
t = build_int_cst (TREE_TYPE (decl), -1);
if (c)
OMP_CLAUSE_LINEAR_STEP (c) = t;
t = build2 (PLUS_EXPR, TREE_TYPE (decl), var, t);
t = build2 (MODIFY_EXPR, TREE_TYPE (var), var, t);
TREE_VEC_ELT (OMP_FOR_INCR (for_stmt), i) = t;
@ -6618,6 +6754,20 @@ gimplify_omp_for (tree *expr_p, gimple_seq *pre_p)
tret = gimplify_expr (&TREE_OPERAND (t, 1), &for_pre_body, NULL,
is_gimple_val, fb_rvalue);
ret = MIN (ret, tret);
if (c)
{
OMP_CLAUSE_LINEAR_STEP (c) = TREE_OPERAND (t, 1);
if (TREE_CODE (t) == MINUS_EXPR)
{
t = TREE_OPERAND (t, 1);
OMP_CLAUSE_LINEAR_STEP (c)
= fold_build1 (NEGATE_EXPR, TREE_TYPE (t), t);
tret = gimplify_expr (&OMP_CLAUSE_LINEAR_STEP (c),
&for_pre_body, NULL,
is_gimple_val, fb_rvalue);
ret = MIN (ret, tret);
}
}
break;
default:
@ -6648,11 +6798,21 @@ gimplify_omp_for (tree *expr_p, gimple_seq *pre_p)
}
}
BITMAP_FREE (has_decl_expr);
gimplify_and_add (OMP_FOR_BODY (for_stmt), &for_body);
gimplify_adjust_omp_clauses (&OMP_FOR_CLAUSES (for_stmt));
gfor = gimple_build_omp_for (for_body, OMP_FOR_CLAUSES (for_stmt),
int kind;
switch (TREE_CODE (for_stmt))
{
case OMP_FOR: kind = GF_OMP_FOR_KIND_FOR; break;
case OMP_SIMD: kind = GF_OMP_FOR_KIND_SIMD; break;
default:
gcc_unreachable ();
}
gfor = gimple_build_omp_for (for_body, kind, OMP_FOR_CLAUSES (for_stmt),
TREE_VEC_LENGTH (OMP_FOR_INIT (for_stmt)),
for_pre_body);
@ -6669,7 +6829,10 @@ gimplify_omp_for (tree *expr_p, gimple_seq *pre_p)
}
gimplify_seq_add_stmt (pre_p, gfor);
return ret == GS_ALL_DONE ? GS_ALL_DONE : GS_ERROR;
if (ret != GS_ALL_DONE)
return GS_ERROR;
*expr_p = NULL_TREE;
return GS_ALL_DONE;
}
/* Gimplify the gross structure of other OpenMP worksharing constructs.
@ -7587,6 +7750,7 @@ gimplify_expr (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p,
break;
case OMP_FOR:
case OMP_SIMD:
ret = gimplify_omp_for (expr_p, pre_p);
break;

24
gcc/internal-fn.c
View file

@ -109,6 +109,30 @@ expand_STORE_LANES (gimple stmt)
expand_insn (get_multi_vector_move (type, vec_store_lanes_optab), 2, ops);
}
/* This should get expanded in adjust_simduid_builtins. */
static void
expand_GOMP_SIMD_LANE (gimple stmt ATTRIBUTE_UNUSED)
{
gcc_unreachable ();
}
/* This should get expanded in adjust_simduid_builtins. */
static void
expand_GOMP_SIMD_VF (gimple stmt ATTRIBUTE_UNUSED)
{
gcc_unreachable ();
}
/* This should get expanded in adjust_simduid_builtins. */
static void
expand_GOMP_SIMD_LAST_LANE (gimple stmt ATTRIBUTE_UNUSED)
{
gcc_unreachable ();
}
/* Routines to expand each internal function, indexed by function number.
Each routine has the prototype:

3
gcc/internal-fn.def
View file

@ -40,3 +40,6 @@ along with GCC; see the file COPYING3. If not see
DEF_INTERNAL_FN (LOAD_LANES, ECF_CONST | ECF_LEAF)
DEF_INTERNAL_FN (STORE_LANES, ECF_CONST | ECF_LEAF)
DEF_INTERNAL_FN (GOMP_SIMD_LANE, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW)
DEF_INTERNAL_FN (GOMP_SIMD_VF, ECF_CONST | ECF_LEAF | ECF_NOTHROW)
DEF_INTERNAL_FN (GOMP_SIMD_LAST_LANE, ECF_CONST | ECF_LEAF | ECF_NOTHROW)

1317
gcc/omp-low.c
View file

File diff suppressed because it is too large Load diff

23
gcc/tree-data-ref.c
View file

@ -4331,10 +4331,25 @@ get_references_in_stmt (gimple stmt, vec<data_ref_loc, va_stack> *references)
/* ASM_EXPR and CALL_EXPR may embed arbitrary side effects.
As we cannot model data-references to not spelled out
accesses give up if they may occur. */
if ((stmt_code == GIMPLE_CALL
&& !(gimple_call_flags (stmt) & ECF_CONST))
|| (stmt_code == GIMPLE_ASM
&& (gimple_asm_volatile_p (stmt) || gimple_vuse (stmt))))
if (stmt_code == GIMPLE_CALL
&& !(gimple_call_flags (stmt) & ECF_CONST))
{
/* Allow IFN_GOMP_SIMD_LANE in their own loops. */
if (gimple_call_internal_p (stmt)
&& gimple_call_internal_fn (stmt) == IFN_GOMP_SIMD_LANE)
{
struct loop *loop = gimple_bb (stmt)->loop_father;
tree uid = gimple_call_arg (stmt, 0);
gcc_assert (TREE_CODE (uid) == SSA_NAME);
if (loop == NULL
|| loop->simduid != SSA_NAME_VAR (uid))
clobbers_memory = true;
}
else
clobbers_memory = true;
}
else if (stmt_code == GIMPLE_ASM
&& (gimple_asm_volatile_p (stmt) || gimple_vuse (stmt)))
clobbers_memory = true;
if (!gimple_vuse (stmt))

1
gcc/tree-flow.h
View file

@ -344,7 +344,6 @@ extern struct omp_region *new_omp_region (basic_block, enum gimple_code,
struct omp_region *);
extern void free_omp_regions (void);
void omp_expand_local (basic_block);
extern tree find_omp_clause (tree, enum omp_clause_code);
tree copy_var_decl (tree, tree, tree);
/*---------------------------------------------------------------------------

7
gcc/tree-if-conv.c
View file

@ -1787,6 +1787,10 @@ main_tree_if_conversion (void)
return 0;
FOR_EACH_LOOP (li, loop, 0)
if (flag_tree_loop_if_convert == 1
|| flag_tree_loop_if_convert_stores == 1
|| flag_tree_vectorize
|| loop->force_vect)
changed |= tree_if_conversion (loop);
if (changed)
@ -1811,7 +1815,8 @@ main_tree_if_conversion (void)
static bool
gate_tree_if_conversion (void)
{
return ((flag_tree_vectorize && flag_tree_loop_if_convert != 0)
return (((flag_tree_vectorize || cfun->has_force_vect_loops)
&& flag_tree_loop_if_convert != 0)
|| flag_tree_loop_if_convert == 1
|| flag_tree_loop_if_convert_stores == 1);
}

5
gcc/tree-inline.c
View file

@ -1298,7 +1298,8 @@ remap_gimple_stmt (gimple stmt, copy_body_data *id)
case GIMPLE_OMP_FOR:
s1 = remap_gimple_seq (gimple_omp_body (stmt), id);
s2 = remap_gimple_seq (gimple_omp_for_pre_body (stmt), id);
copy = gimple_build_omp_for (s1, gimple_omp_for_clauses (stmt),
copy = gimple_build_omp_for (s1, gimple_omp_for_kind (stmt),
gimple_omp_for_clauses (stmt),
gimple_omp_for_collapse (stmt), s2);
{
size_t i;
@ -2399,6 +2400,8 @@ copy_cfg_body (copy_body_data * id, gcov_type count, int frequency_scale,
get_loop (src_cfun, 0));
/* Defer to cfgcleanup to update loop-father fields of basic-blocks. */
loops_state_set (LOOPS_NEED_FIXUP);
cfun->has_force_vect_loops |= src_cfun->has_force_vect_loops;
cfun->has_simduid_loops |= src_cfun->has_simduid_loops;
}
/* If the loop tree in the source function needed fixup, mark the

2
gcc/tree-parloops.c
View file

@ -1695,7 +1695,7 @@ create_parallel_loop (struct loop *loop, tree loop_fn, tree data,
t = build_omp_clause (loc, OMP_CLAUSE_SCHEDULE);
OMP_CLAUSE_SCHEDULE_KIND (t) = OMP_CLAUSE_SCHEDULE_STATIC;
for_stmt = gimple_build_omp_for (NULL, t, 1, NULL);
for_stmt = gimple_build_omp_for (NULL, GF_OMP_FOR_KIND_FOR, t, 1, NULL);
gimple_set_location (for_stmt, loc);
gimple_omp_for_set_index (for_stmt, 0, initvar);
gimple_omp_for_set_initial (for_stmt, 0, cvar_init);

36
gcc/tree-pretty-print.c
View file

@ -316,11 +316,14 @@ dump_omp_clause (pretty_printer *buffer, tree clause, int spc, int flags)
case OMP_CLAUSE_COPYPRIVATE:
name = "copyprivate";
goto print_remap;
case OMP_CLAUSE_UNIFORM:
name = "uniform";
goto print_remap;
print_remap:
pp_string (buffer, name);
pp_left_paren (buffer);
dump_generic_node (buffer, OMP_CLAUSE_DECL (clause),
spc, flags, false);
spc, flags, false);
pp_right_paren (buffer);
break;
@ -433,6 +436,30 @@ dump_omp_clause (pretty_printer *buffer, tree clause, int spc, int flags)
pp_string (buffer, "mergeable");
break;
case OMP_CLAUSE_LINEAR:
pp_string (buffer, "linear(");
dump_generic_node (buffer, OMP_CLAUSE_DECL (clause),
spc, flags, false);
pp_character (buffer, ':');
dump_generic_node (buffer, OMP_CLAUSE_LINEAR_STEP (clause),
spc, flags, false);
pp_character (buffer, ')');
break;
case OMP_CLAUSE_SAFELEN:
pp_string (buffer, "safelen(");
dump_generic_node (buffer, OMP_CLAUSE_SAFELEN_EXPR (clause),
spc, flags, false);
pp_character (buffer, ')');
break;
case OMP_CLAUSE__SIMDUID_:
pp_string (buffer, "_simduid_(");
dump_generic_node (buffer, OMP_CLAUSE__SIMDUID__DECL (clause),
spc, flags, false);
pp_character (buffer, ')');
break;
default:
/* Should never happen. */
dump_generic_node (buffer, clause, spc, flags, false);
@ -2179,6 +2206,13 @@ dump_generic_node (pretty_printer *buffer, tree node, int spc, int flags,
case OMP_FOR:
pp_string (buffer, "#pragma omp for");
goto dump_omp_loop;
case OMP_SIMD:
pp_string (buffer, "#pragma omp simd");
goto dump_omp_loop;
dump_omp_loop:
dump_omp_clauses (buffer, OMP_FOR_CLAUSES (node), spc, flags);
if (!(flags & TDF_SLIM))

16
gcc/tree-ssa-ccp.c
View file

@ -631,6 +631,22 @@ likely_value (gimple stmt)
if (has_constant_operand)
all_undefined_operands = false;
if (has_undefined_operand
&& code == GIMPLE_CALL
&& gimple_call_internal_p (stmt))
switch (gimple_call_internal_fn (stmt))
{
/* These 3 builtins use the first argument just as a magic
way how to find out a decl uid. */
case IFN_GOMP_SIMD_LANE:
case IFN_GOMP_SIMD_VF:
case IFN_GOMP_SIMD_LAST_LANE:
has_undefined_operand = false;
break;
default:
break;
}
/* If the operation combines operands like COMPLEX_EXPR make sure to
not mark the result UNDEFINED if only one part of the result is
undefined. */

5
gcc/tree-ssa-loop-ivcanon.c
View file

@ -1125,6 +1125,11 @@ tree_unroll_loops_completely_1 (bool may_increase_size, bool unroll_outer,
if (changed)
return true;
/* Don't unroll #pragma omp simd loops until the vectorizer
attempts to vectorize those. */
if (loop->force_vect)
return false;
/* Try to unroll this loop. */
loop_father = loop_outer (loop);
if (!loop_father)

2
gcc/tree-ssa-loop.c
View file

@ -314,7 +314,7 @@ tree_vectorize (void)
static bool
gate_tree_vectorize (void)
{
return flag_tree_vectorize;
return flag_tree_vectorize || cfun->has_force_vect_loops;
}
namespace {

104
gcc/tree-vect-data-refs.c
View file

@ -255,6 +255,15 @@ vect_analyze_data_ref_dependence (struct data_dependence_relation *ddr,
/* Unknown data dependence. */
if (DDR_ARE_DEPENDENT (ddr) == chrec_dont_know)
{
/* If user asserted safelen consecutive iterations can be
executed concurrently, assume independence. */
if (loop->safelen >= 2)
{
if (loop->safelen < *max_vf)
*max_vf = loop->safelen;
return false;
}
if (STMT_VINFO_GATHER_P (stmtinfo_a)
|| STMT_VINFO_GATHER_P (stmtinfo_b))
{
@ -291,6 +300,15 @@ vect_analyze_data_ref_dependence (struct data_dependence_relation *ddr,
/* Known data dependence. */
if (DDR_NUM_DIST_VECTS (ddr) == 0)
{
/* If user asserted safelen consecutive iterations can be
executed concurrently, assume independence. */
if (loop->safelen >= 2)
{
if (loop->safelen < *max_vf)
*max_vf = loop->safelen;
return false;
}
if (STMT_VINFO_GATHER_P (stmtinfo_a)
|| STMT_VINFO_GATHER_P (stmtinfo_b))
{
@ -2930,6 +2948,7 @@ vect_analyze_data_refs (loop_vec_info loop_vinfo,
stmt_vec_info stmt_info;
tree base, offset, init;
bool gather = false;
bool simd_lane_access = false;
int vf;
again:
@ -2961,12 +2980,17 @@ again:
if (!DR_BASE_ADDRESS (dr) || !DR_OFFSET (dr) || !DR_INIT (dr)
|| !DR_STEP (dr))
{
/* If target supports vector gather loads, see if they can't
be used. */
if (loop_vinfo
&& DR_IS_READ (dr)
bool maybe_gather
= DR_IS_READ (dr)
&& !TREE_THIS_VOLATILE (DR_REF (dr))
&& targetm.vectorize.builtin_gather != NULL
&& targetm.vectorize.builtin_gather != NULL;
bool maybe_simd_lane_access
= loop_vinfo && loop->simduid;
/* If target supports vector gather loads, or if this might be
a SIMD lane access, see if they can't be used. */
if (loop_vinfo
&& (maybe_gather || maybe_simd_lane_access)
&& !nested_in_vect_loop_p (loop, stmt))
{
struct data_reference *newdr
@ -2979,14 +3003,59 @@ again:
&& DR_STEP (newdr)
&& integer_zerop (DR_STEP (newdr)))
{
dr = newdr;
gather = true;
if (maybe_simd_lane_access)
{
tree off = DR_OFFSET (newdr);
STRIP_NOPS (off);
if (TREE_CODE (DR_INIT (newdr)) == INTEGER_CST
&& TREE_CODE (off) == MULT_EXPR
&& host_integerp (TREE_OPERAND (off, 1), 1))
{
tree step = TREE_OPERAND (off, 1);
off = TREE_OPERAND (off, 0);
STRIP_NOPS (off);
if (CONVERT_EXPR_P (off)
&& TYPE_PRECISION (TREE_TYPE (TREE_OPERAND (off,
0)))
< TYPE_PRECISION (TREE_TYPE (off)))
off = TREE_OPERAND (off, 0);
if (TREE_CODE (off) == SSA_NAME)
{
gimple def = SSA_NAME_DEF_STMT (off);
tree reft = TREE_TYPE (DR_REF (newdr));
if (gimple_call_internal_p (def)
&& gimple_call_internal_fn (def)
== IFN_GOMP_SIMD_LANE)
{
tree arg = gimple_call_arg (def, 0);
gcc_assert (TREE_CODE (arg) == SSA_NAME);
arg = SSA_NAME_VAR (arg);
if (arg == loop->simduid
/* For now. */
&& tree_int_cst_equal
(TYPE_SIZE_UNIT (reft),
step))
{
DR_OFFSET (newdr) = ssize_int (0);
DR_STEP (newdr) = step;
dr = newdr;
simd_lane_access = true;
}
}
}
}
}
if (!simd_lane_access && maybe_gather)
{
dr = newdr;
gather = true;
}
}
else
if (!gather && !simd_lane_access)
free_data_ref (newdr);
}
if (!gather)
if (!gather && !simd_lane_access)
{
if (dump_enabled_p ())
{
@ -3013,7 +3082,7 @@ again:
if (bb_vinfo)
break;
if (gather)
if (gather || simd_lane_access)
free_data_ref (dr);
return false;
}
@ -3046,7 +3115,7 @@ again:
if (bb_vinfo)
break;
if (gather)
if (gather || simd_lane_access)
free_data_ref (dr);
return false;
}
@ -3065,7 +3134,7 @@ again:
if (bb_vinfo)
break;
if (gather)
if (gather || simd_lane_access)
free_data_ref (dr);
return false;
}
@ -3086,7 +3155,7 @@ again:
if (bb_vinfo)
break;
if (gather)
if (gather || simd_lane_access)
free_data_ref (dr);
return false;
}
@ -3221,12 +3290,17 @@ again:
if (bb_vinfo)
break;
if (gather)
if (gather || simd_lane_access)
free_data_ref (dr);
return false;
}
STMT_VINFO_DATA_REF (stmt_info) = dr;
if (simd_lane_access)
{
STMT_VINFO_SIMD_LANE_ACCESS_P (stmt_info) = true;
datarefs[i] = dr;
}
/* Set vectype for STMT. */
scalar_type = TREE_TYPE (DR_REF (dr));
@ -3247,7 +3321,7 @@ again:
if (bb_vinfo)
break;
if (gather)
if (gather || simd_lane_access)
{
STMT_VINFO_DATA_REF (stmt_info) = NULL;
free_data_ref (dr);

38
gcc/tree-vect-loop.c
View file

@ -5378,7 +5378,7 @@ vectorizable_induction (gimple phi, gimple_stmt_iterator *gsi ATTRIBUTE_UNUSED,
bool
vectorizable_live_operation (gimple stmt,
gimple_stmt_iterator *gsi ATTRIBUTE_UNUSED,
gimple *vec_stmt ATTRIBUTE_UNUSED)
gimple *vec_stmt)
{
stmt_vec_info stmt_info = vinfo_for_stmt (stmt);
loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info);
@ -5398,7 +5398,41 @@ vectorizable_live_operation (gimple stmt,
return false;
if (!is_gimple_assign (stmt))
return false;
{
if (gimple_call_internal_p (stmt)
&& gimple_call_internal_fn (stmt) == IFN_GOMP_SIMD_LANE
&& gimple_call_lhs (stmt)
&& loop->simduid
&& TREE_CODE (gimple_call_arg (stmt, 0)) == SSA_NAME
&& loop->simduid
== SSA_NAME_VAR (gimple_call_arg (stmt, 0)))
{
edge e = single_exit (loop);
basic_block merge_bb = e->dest;
imm_use_iterator imm_iter;
use_operand_p use_p;
tree lhs = gimple_call_lhs (stmt);
FOR_EACH_IMM_USE_FAST (use_p, imm_iter, lhs)
{
gimple use_stmt = USE_STMT (use_p);
if (gimple_code (use_stmt) == GIMPLE_PHI
|| gimple_bb (use_stmt) == merge_bb)
{
if (vec_stmt)
{
tree vfm1
= build_int_cst (unsigned_type_node,
loop_vinfo->vectorization_factor - 1);
SET_PHI_ARG_DEF (use_stmt, e->dest_idx, vfm1);
}
return true;
}
}
}
return false;
}
if (TREE_CODE (gimple_assign_lhs (stmt)) != SSA_NAME)
return false;

145
gcc/tree-vect-stmts.c
View file

@ -1755,6 +1755,14 @@ vectorizable_call (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
if (nargs == 0 || nargs > 3)
return false;
/* Ignore the argument of IFN_GOMP_SIMD_LANE, it is magic. */
if (gimple_call_internal_p (stmt)
&& gimple_call_internal_fn (stmt) == IFN_GOMP_SIMD_LANE)
{
nargs = 0;
rhs_type = unsigned_type_node;
}
for (i = 0; i < nargs; i++)
{
tree opvectype;
@ -1830,11 +1838,26 @@ vectorizable_call (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
fndecl = vectorizable_function (stmt, vectype_out, vectype_in);
if (fndecl == NULL_TREE)
{
if (dump_enabled_p ())
dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
"function is not vectorizable.");
return false;
if (gimple_call_internal_p (stmt)
&& gimple_call_internal_fn (stmt) == IFN_GOMP_SIMD_LANE
&& !slp_node
&& loop_vinfo
&& LOOP_VINFO_LOOP (loop_vinfo)->simduid
&& TREE_CODE (gimple_call_arg (stmt, 0)) == SSA_NAME
&& LOOP_VINFO_LOOP (loop_vinfo)->simduid
== SSA_NAME_VAR (gimple_call_arg (stmt, 0)))
{
/* We can handle IFN_GOMP_SIMD_LANE by returning a
{ 0, 1, 2, ... vf - 1 } vector. */
gcc_assert (nargs == 0);
}
else
{
if (dump_enabled_p ())
dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
"function is not vectorizable.");
return false;
}
}
gcc_assert (!gimple_vuse (stmt));
@ -1932,9 +1955,30 @@ vectorizable_call (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
vargs.quick_push (vec_oprnd0);
}
new_stmt = gimple_build_call_vec (fndecl, vargs);
new_temp = make_ssa_name (vec_dest, new_stmt);
gimple_call_set_lhs (new_stmt, new_temp);
if (gimple_call_internal_p (stmt)
&& gimple_call_internal_fn (stmt) == IFN_GOMP_SIMD_LANE)
{
tree *v = XALLOCAVEC (tree, nunits_out);
int k;
for (k = 0; k < nunits_out; ++k)
v[k] = build_int_cst (unsigned_type_node, j * nunits_out + k);
tree cst = build_vector (vectype_out, v);
tree new_var
= vect_get_new_vect_var (vectype_out, vect_simple_var, "cst_");
gimple init_stmt = gimple_build_assign (new_var, cst);
new_temp = make_ssa_name (new_var, init_stmt);
gimple_assign_set_lhs (init_stmt, new_temp);
vect_init_vector_1 (stmt, init_stmt, NULL);
new_temp = make_ssa_name (vec_dest, NULL);
new_stmt = gimple_build_assign (new_temp,
gimple_assign_lhs (init_stmt));
}
else
{
new_stmt = gimple_build_call_vec (fndecl, vargs);
new_temp = make_ssa_name (vec_dest, new_stmt);
gimple_call_set_lhs (new_stmt, new_temp);
}
vect_finish_stmt_generation (stmt, new_stmt, gsi);
if (j == 0)
@ -3796,6 +3840,7 @@ vectorizable_store (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
enum vect_def_type dt;
stmt_vec_info prev_stmt_info = NULL;
tree dataref_ptr = NULL_TREE;
tree dataref_offset = NULL_TREE;
gimple ptr_incr = NULL;
int nunits = TYPE_VECTOR_SUBPARTS (vectype);
int ncopies;
@ -4085,9 +4130,26 @@ vectorizable_store (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
/* We should have catched mismatched types earlier. */
gcc_assert (useless_type_conversion_p (vectype,
TREE_TYPE (vec_oprnd)));
dataref_ptr = vect_create_data_ref_ptr (first_stmt, aggr_type, NULL,
NULL_TREE, &dummy, gsi,
&ptr_incr, false, &inv_p);
bool simd_lane_access_p
= STMT_VINFO_SIMD_LANE_ACCESS_P (stmt_info);
if (simd_lane_access_p
&& TREE_CODE (DR_BASE_ADDRESS (first_dr)) == ADDR_EXPR
&& VAR_P (TREE_OPERAND (DR_BASE_ADDRESS (first_dr), 0))
&& integer_zerop (DR_OFFSET (first_dr))
&& integer_zerop (DR_INIT (first_dr))
&& alias_sets_conflict_p (get_alias_set (aggr_type),
get_alias_set (DR_REF (first_dr))))
{
dataref_ptr = unshare_expr (DR_BASE_ADDRESS (first_dr));
dataref_offset = build_int_cst (reference_alias_ptr_type
(DR_REF (first_dr)), 0);
}
else
dataref_ptr
= vect_create_data_ref_ptr (first_stmt, aggr_type,
simd_lane_access_p ? loop : NULL,
NULL_TREE, &dummy, gsi, &ptr_incr,
simd_lane_access_p, &inv_p);
gcc_assert (bb_vinfo || !inv_p);
}
else
@ -4108,8 +4170,13 @@ vectorizable_store (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
dr_chain[i] = vec_oprnd;
oprnds[i] = vec_oprnd;
}
dataref_ptr = bump_vector_ptr (dataref_ptr, ptr_incr, gsi, stmt,
TYPE_SIZE_UNIT (aggr_type));
if (dataref_offset)
dataref_offset
= int_const_binop (PLUS_EXPR, dataref_offset,
TYPE_SIZE_UNIT (aggr_type));
else
dataref_ptr = bump_vector_ptr (dataref_ptr, ptr_incr, gsi, stmt,
TYPE_SIZE_UNIT (aggr_type));
}
if (store_lanes_p)
@ -4161,8 +4228,10 @@ vectorizable_store (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
vec_oprnd = result_chain[i];
data_ref = build2 (MEM_REF, TREE_TYPE (vec_oprnd), dataref_ptr,
build_int_cst (reference_alias_ptr_type
(DR_REF (first_dr)), 0));
dataref_offset
? dataref_offset
: build_int_cst (reference_alias_ptr_type
(DR_REF (first_dr)), 0));
align = TYPE_ALIGN_UNIT (vectype);
if (aligned_access_p (first_dr))
misalign = 0;
@ -4181,8 +4250,9 @@ vectorizable_store (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
TYPE_ALIGN (elem_type));
misalign = DR_MISALIGNMENT (first_dr);
}
set_ptr_info_alignment (get_ptr_info (dataref_ptr), align,
misalign);
if (dataref_offset == NULL_TREE)
set_ptr_info_alignment (get_ptr_info (dataref_ptr), align,
misalign);
/* Arguments are ready. Create the new vector stmt. */
new_stmt = gimple_build_assign (data_ref, vec_oprnd);
@ -4314,6 +4384,7 @@ vectorizable_load (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
tree dummy;
enum dr_alignment_support alignment_support_scheme;
tree dataref_ptr = NULL_TREE;
tree dataref_offset = NULL_TREE;
gimple ptr_incr = NULL;
int nunits = TYPE_VECTOR_SUBPARTS (vectype);
int ncopies;
@ -4947,9 +5018,32 @@ vectorizable_load (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
{
/* 1. Create the vector or array pointer update chain. */
if (j == 0)
dataref_ptr = vect_create_data_ref_ptr (first_stmt, aggr_type, at_loop,
offset, &dummy, gsi,
&ptr_incr, false, &inv_p);
{
bool simd_lane_access_p
= STMT_VINFO_SIMD_LANE_ACCESS_P (stmt_info);
if (simd_lane_access_p
&& TREE_CODE (DR_BASE_ADDRESS (first_dr)) == ADDR_EXPR
&& VAR_P (TREE_OPERAND (DR_BASE_ADDRESS (first_dr), 0))
&& integer_zerop (DR_OFFSET (first_dr))
&& integer_zerop (DR_INIT (first_dr))
&& alias_sets_conflict_p (get_alias_set (aggr_type),
get_alias_set (DR_REF (first_dr)))
&& (alignment_support_scheme == dr_aligned
|| alignment_support_scheme == dr_unaligned_supported))
{
dataref_ptr = unshare_expr (DR_BASE_ADDRESS (first_dr));
dataref_offset = build_int_cst (reference_alias_ptr_type
(DR_REF (first_dr)), 0);
}
else
dataref_ptr
= vect_create_data_ref_ptr (first_stmt, aggr_type, at_loop,
offset, &dummy, gsi, &ptr_incr,
simd_lane_access_p, &inv_p);
}
else if (dataref_offset)
dataref_offset = int_const_binop (PLUS_EXPR, dataref_offset,
TYPE_SIZE_UNIT (aggr_type));
else
dataref_ptr = bump_vector_ptr (dataref_ptr, ptr_incr, gsi, stmt,
TYPE_SIZE_UNIT (aggr_type));
@ -4999,8 +5093,10 @@ vectorizable_load (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
data_ref
= build2 (MEM_REF, vectype, dataref_ptr,
build_int_cst (reference_alias_ptr_type
(DR_REF (first_dr)), 0));
dataref_offset
? dataref_offset
: build_int_cst (reference_alias_ptr_type
(DR_REF (first_dr)), 0));
align = TYPE_ALIGN_UNIT (vectype);
if (alignment_support_scheme == dr_aligned)
{
@ -5022,8 +5118,9 @@ vectorizable_load (gimple stmt, gimple_stmt_iterator *gsi, gimple *vec_stmt,
TYPE_ALIGN (elem_type));
misalign = DR_MISALIGNMENT (first_dr);
}
set_ptr_info_alignment (get_ptr_info (dataref_ptr),
align, misalign);
if (dataref_offset == NULL_TREE)
set_ptr_info_alignment (get_ptr_info (dataref_ptr),
align, misalign);
break;
}
case dr_explicit_realign:

267
gcc/tree-vectorizer.c
View file

@ -66,13 +66,218 @@ along with GCC; see the file COPYING3. If not see
#include "cfgloop.h"
#include "tree-vectorizer.h"
#include "tree-pass.h"
#include "hash-table.h"
#include "tree-ssa-propagate.h"
/* Loop or bb location. */
LOC vect_location;
/* Vector mapping GIMPLE stmt to stmt_vec_info. */
vec<vec_void_p> stmt_vec_info_vec;
/* For mapping simduid to vectorization factor. */
struct simduid_to_vf : typed_free_remove<simduid_to_vf>
{
unsigned int simduid;
int vf;
/* hash_table support. */
typedef simduid_to_vf value_type;
typedef simduid_to_vf compare_type;
static inline hashval_t hash (const value_type *);
static inline int equal (const value_type *, const compare_type *);
};
inline hashval_t
simduid_to_vf::hash (const value_type *p)
{
return p->simduid;
}
inline int
simduid_to_vf::equal (const value_type *p1, const value_type *p2)
{
return p1->simduid == p2->simduid;
}
/* This hash maps the OMP simd array to the corresponding simduid used
to index into it. Like thus,
_7 = GOMP_SIMD_LANE (simduid.0)
...
...
D.1737[_7] = stuff;
This hash maps from the simduid.0 to OMP simd array (D.1737[]). */
struct simd_array_to_simduid : typed_free_remove<simd_array_to_simduid>
{
tree decl;
unsigned int simduid;
/* hash_table support. */
typedef simd_array_to_simduid value_type;
typedef simd_array_to_simduid compare_type;
static inline hashval_t hash (const value_type *);
static inline int equal (const value_type *, const compare_type *);
};
inline hashval_t
simd_array_to_simduid::hash (const value_type *p)
{
return DECL_UID (p->decl);
}
inline int
simd_array_to_simduid::equal (const value_type *p1, const value_type *p2)
{
return p1->decl == p2->decl;
}
/* Fold IFN_GOMP_SIMD_LANE, IFN_GOMP_SIMD_VF and IFN_GOMP_SIMD_LAST_LANE
into their corresponding constants. */
static void
adjust_simduid_builtins (hash_table <simduid_to_vf> &htab)
{
basic_block bb;
FOR_EACH_BB (bb)
{
gimple_stmt_iterator i;
for (i = gsi_start_bb (bb); !gsi_end_p (i); gsi_next (&i))
{
unsigned int vf = 1;
enum internal_fn ifn;
gimple stmt = gsi_stmt (i);
tree t;
if (!is_gimple_call (stmt)
|| !gimple_call_internal_p (stmt))
continue;
ifn = gimple_call_internal_fn (stmt);
switch (ifn)
{
case IFN_GOMP_SIMD_LANE:
case IFN_GOMP_SIMD_VF:
case IFN_GOMP_SIMD_LAST_LANE:
break;
default:
continue;
}
tree arg = gimple_call_arg (stmt, 0);
gcc_assert (arg != NULL_TREE);
gcc_assert (TREE_CODE (arg) == SSA_NAME);
simduid_to_vf *p = NULL, data;
data.simduid = DECL_UID (SSA_NAME_VAR (arg));
if (htab.is_created ())
p = htab.find (&data);
if (p)
vf = p->vf;
switch (ifn)
{
case IFN_GOMP_SIMD_VF:
t = build_int_cst (unsigned_type_node, vf);
break;
case IFN_GOMP_SIMD_LANE:
t = build_int_cst (unsigned_type_node, 0);
break;
case IFN_GOMP_SIMD_LAST_LANE:
t = gimple_call_arg (stmt, 1);
break;
default:
gcc_unreachable ();
}
update_call_from_tree (&i, t);
}
}
}
/* Helper structure for note_simd_array_uses. */
struct note_simd_array_uses_struct
{
hash_table <simd_array_to_simduid> *htab;
unsigned int simduid;
};
/* Callback for note_simd_array_uses, called through walk_gimple_op. */
static tree
note_simd_array_uses_cb (tree *tp, int *walk_subtrees, void *data)
{
struct walk_stmt_info *wi = (struct walk_stmt_info *) data;
struct note_simd_array_uses_struct *ns
= (struct note_simd_array_uses_struct *) wi->info;
if (TYPE_P (*tp))
*walk_subtrees = 0;
else if (VAR_P (*tp)
&& lookup_attribute ("omp simd array", DECL_ATTRIBUTES (*tp))
&& DECL_CONTEXT (*tp) == current_function_decl)
{
simd_array_to_simduid data;
if (!ns->htab->is_created ())
ns->htab->create (15);
data.decl = *tp;
data.simduid = ns->simduid;
simd_array_to_simduid **slot = ns->htab->find_slot (&data, INSERT);
if (*slot == NULL)
{
simd_array_to_simduid *p = XNEW (simd_array_to_simduid);
*p = data;
*slot = p;
}
else if ((*slot)->simduid != ns->simduid)
(*slot)->simduid = -1U;
*walk_subtrees = 0;
}
return NULL_TREE;
}
/* Find "omp simd array" temporaries and map them to corresponding
simduid. */
static void
note_simd_array_uses (hash_table <simd_array_to_simduid> *htab)
{
basic_block bb;
gimple_stmt_iterator gsi;
struct walk_stmt_info wi;
struct note_simd_array_uses_struct ns;
memset (&wi, 0, sizeof (wi));
wi.info = &ns;
ns.htab = htab;
FOR_EACH_BB (bb)
for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi))
{
gimple stmt = gsi_stmt (gsi);
if (!is_gimple_call (stmt) || !gimple_call_internal_p (stmt))
continue;
switch (gimple_call_internal_fn (stmt))
{
case IFN_GOMP_SIMD_LANE:
case IFN_GOMP_SIMD_VF:
case IFN_GOMP_SIMD_LAST_LANE:
break;
default:
continue;
}
tree lhs = gimple_call_lhs (stmt);
if (lhs == NULL_TREE)
continue;
imm_use_iterator use_iter;
gimple use_stmt;
ns.simduid = DECL_UID (SSA_NAME_VAR (gimple_call_arg (stmt, 0)));
FOR_EACH_IMM_USE_STMT (use_stmt, use_iter, lhs)
if (!is_gimple_debug (use_stmt))
walk_gimple_op (use_stmt, note_simd_array_uses_cb, &wi);
}
}
/* Function vectorize_loops.
@ -86,12 +291,21 @@ vectorize_loops (void)
unsigned int vect_loops_num;
loop_iterator li;
struct loop *loop;
hash_table <simduid_to_vf> simduid_to_vf_htab;
hash_table <simd_array_to_simduid> simd_array_to_simduid_htab;
vect_loops_num = number_of_loops (cfun);
/* Bail out if there are no loops. */
if (vect_loops_num <= 1)
return 0;
{
if (cfun->has_simduid_loops)
adjust_simduid_builtins (simduid_to_vf_htab);
return 0;
}
if (cfun->has_simduid_loops)
note_simd_array_uses (&simd_array_to_simduid_htab);
init_stmt_vec_info_vec ();
@ -101,7 +315,8 @@ vectorize_loops (void)
than all previously defined loops. This fact allows us to run
only over initial loops skipping newly generated ones. */
FOR_EACH_LOOP (li, loop, 0)
if (optimize_loop_nest_for_speed_p (loop))
if ((flag_tree_vectorize && optimize_loop_nest_for_speed_p (loop))
|| loop->force_vect)
{
loop_vec_info loop_vinfo;
vect_location = find_loop_location (loop);
@ -122,6 +337,20 @@ vectorize_loops (void)
"Vectorized loop\n");
vect_transform_loop (loop_vinfo);
num_vectorized_loops++;
/* Now that the loop has been vectorized, allow it to be unrolled
etc. */
loop->force_vect = false;
if (loop->simduid)
{
simduid_to_vf *simduid_to_vf_data = XNEW (simduid_to_vf);
if (!simduid_to_vf_htab.is_created ())
simduid_to_vf_htab.create (15);
simduid_to_vf_data->simduid = DECL_UID (loop->simduid);
simduid_to_vf_data->vf = loop_vinfo->vectorization_factor;
*simduid_to_vf_htab.find_slot (simduid_to_vf_data, INSERT)
= simduid_to_vf_data;
}
}
vect_location = UNKNOWN_LOC;
@ -149,6 +378,40 @@ vectorize_loops (void)
free_stmt_vec_info_vec ();
/* Fold IFN_GOMP_SIMD_{VF,LANE,LAST_LANE} builtins. */
if (cfun->has_simduid_loops)
adjust_simduid_builtins (simduid_to_vf_htab);
/* Shrink any "omp array simd" temporary arrays to the
actual vectorization factors. */
if (simd_array_to_simduid_htab.is_created ())
{
for (hash_table <simd_array_to_simduid>::iterator iter
= simd_array_to_simduid_htab.begin ();
iter != simd_array_to_simduid_htab.end (); ++iter)
if ((*iter).simduid != -1U)
{
tree decl = (*iter).decl;
int vf = 1;
if (simduid_to_vf_htab.is_created ())
{
simduid_to_vf *p = NULL, data;
data.simduid = (*iter).simduid;
p = simduid_to_vf_htab.find (&data);
if (p)
vf = p->vf;
}
tree atype
= build_array_type_nelts (TREE_TYPE (TREE_TYPE (decl)), vf);
TREE_TYPE (decl) = atype;
relayout_decl (decl);
}
simd_array_to_simduid_htab.dispose ();
}
if (simduid_to_vf_htab.is_created ())
simduid_to_vf_htab.dispose ();
if (num_vectorized_loops > 0)
{
/* If we vectorized any loop only virtual SSA form needs to be updated.

4
gcc/tree-vectorizer.h
View file

@ -576,6 +576,9 @@ typedef struct _stmt_vec_info {
/* For loads only, true if this is a gather load. */
bool gather_p;
bool stride_load_p;
/* For both loads and stores. */
bool simd_lane_access_p;
} *stmt_vec_info;
/* Access Functions. */
@ -591,6 +594,7 @@ typedef struct _stmt_vec_info {
#define STMT_VINFO_DATA_REF(S) (S)->data_ref_info
#define STMT_VINFO_GATHER_P(S) (S)->gather_p
#define STMT_VINFO_STRIDE_LOAD_P(S) (S)->stride_load_p
#define STMT_VINFO_SIMD_LANE_ACCESS_P(S) (S)->simd_lane_access_p
#define STMT_VINFO_DR_BASE_ADDRESS(S) (S)->dr_base_address
#define STMT_VINFO_DR_INIT(S) (S)->dr_init

20
gcc/tree.c
View file

@ -236,6 +236,8 @@ unsigned const char omp_clause_num_ops[] =
4, /* OMP_CLAUSE_REDUCTION */
1, /* OMP_CLAUSE_COPYIN */
1, /* OMP_CLAUSE_COPYPRIVATE */
2, /* OMP_CLAUSE_LINEAR */
1, /* OMP_CLAUSE_UNIFORM */
1, /* OMP_CLAUSE_IF */
1, /* OMP_CLAUSE_NUM_THREADS */
1, /* OMP_CLAUSE_SCHEDULE */
@ -245,7 +247,9 @@ unsigned const char omp_clause_num_ops[] =
3, /* OMP_CLAUSE_COLLAPSE */
0, /* OMP_CLAUSE_UNTIED */
1, /* OMP_CLAUSE_FINAL */
0 /* OMP_CLAUSE_MERGEABLE */
0, /* OMP_CLAUSE_MERGEABLE */
1, /* OMP_CLAUSE_SAFELEN */
1, /* OMP_CLAUSE__SIMDUID_ */
};
const char * const omp_clause_code_name[] =
@ -258,6 +262,8 @@ const char * const omp_clause_code_name[] =
"reduction",
"copyin",
"copyprivate",
"linear",
"uniform",
"if",
"num_threads",
"schedule",
@ -267,7 +273,9 @@ const char * const omp_clause_code_name[] =
"collapse",
"untied",
"final",
"mergeable"
"mergeable",
"safelen",
"_simduid_"
};
@ -11064,6 +11072,9 @@ walk_tree_1 (tree *tp, walk_tree_fn func, void *data,
case OMP_CLAUSE_IF:
case OMP_CLAUSE_NUM_THREADS:
case OMP_CLAUSE_SCHEDULE:
case OMP_CLAUSE_UNIFORM:
case OMP_CLAUSE_SAFELEN:
case OMP_CLAUSE__SIMDUID_:
WALK_SUBTREE (OMP_CLAUSE_OPERAND (*tp, 0));
/* FALLTHRU */
@ -11087,6 +11098,11 @@ walk_tree_1 (tree *tp, walk_tree_fn func, void *data,
WALK_SUBTREE_TAIL (OMP_CLAUSE_CHAIN (*tp));
}
case OMP_CLAUSE_LINEAR:
WALK_SUBTREE (OMP_CLAUSE_DECL (*tp));
WALK_SUBTREE (OMP_CLAUSE_OPERAND (*tp, 1));
WALK_SUBTREE_TAIL (OMP_CLAUSE_CHAIN (*tp));
case OMP_CLAUSE_REDUCTION:
{
int i;

4
gcc/tree.def
View file

@ -1030,6 +1030,10 @@ DEFTREECODE (OMP_TASK, "omp_task", tcc_statement, 2)
unspecified by the standard. */
DEFTREECODE (OMP_FOR, "omp_for", tcc_statement, 6)
/* OpenMP - #pragma omp simd [clause1 ... clauseN]
Operands like for OMP_FOR. */
DEFTREECODE (OMP_SIMD, "omp_simd", tcc_statement, 6)
/* OpenMP - #pragma omp sections [clause1 ... clauseN]
Operand 0: OMP_SECTIONS_BODY: Sections body.
Operand 1: OMP_SECTIONS_CLAUSES: List of clauses. */

42
gcc/tree.h
View file

@ -365,6 +365,12 @@ enum omp_clause_code
/* OpenMP clause: copyprivate (variable_list). */
OMP_CLAUSE_COPYPRIVATE,
/* OpenMP clause: linear (variable-list[:linear-step]). */
OMP_CLAUSE_LINEAR,
/* OpenMP clause: uniform (argument-list). */
OMP_CLAUSE_UNIFORM,
/* OpenMP clause: if (scalar-expression). */
OMP_CLAUSE_IF,
@ -393,7 +399,13 @@ enum omp_clause_code
OMP_CLAUSE_FINAL,
/* OpenMP clause: mergeable. */
OMP_CLAUSE_MERGEABLE
OMP_CLAUSE_MERGEABLE,
/* OpenMP clause: safelen (constant-integer-expression). */
OMP_CLAUSE_SAFELEN,
/* Internally used only clause, holding SIMD uid. */
OMP_CLAUSE__SIMDUID_
};
/* The definition of tree nodes fills the next several pages. */
@ -560,6 +572,9 @@ struct GTY(()) tree_base {
OMP_CLAUSE_PRIVATE_DEBUG in
OMP_CLAUSE_PRIVATE
OMP_CLAUSE_LINEAR_NO_COPYIN in
OMP_CLAUSE_LINEAR
TRANSACTION_EXPR_RELAXED in
TRANSACTION_EXPR
@ -580,6 +595,9 @@ struct GTY(()) tree_base {
OMP_CLAUSE_PRIVATE_OUTER_REF in
OMP_CLAUSE_PRIVATE
OMP_CLAUSE_LINEAR_NO_COPYOUT in
OMP_CLAUSE_LINEAR
TYPE_REF_IS_RVALUE in
REFERENCE_TYPE
@ -1803,7 +1821,7 @@ extern void protected_set_expr_location (tree, location_t);
#define OMP_CLAUSE_DECL(NODE) \
OMP_CLAUSE_OPERAND (OMP_CLAUSE_RANGE_CHECK (OMP_CLAUSE_CHECK (NODE), \
OMP_CLAUSE_PRIVATE, \
OMP_CLAUSE_COPYPRIVATE), 0)
OMP_CLAUSE_UNIFORM), 0)
#define OMP_CLAUSE_HAS_LOCATION(NODE) \
(LOCATION_LOCUS ((OMP_CLAUSE_CHECK (NODE))->omp_clause.locus) \
!= UNKNOWN_LOCATION)
@ -1870,6 +1888,25 @@ extern void protected_set_expr_location (tree, location_t);
#define OMP_CLAUSE_REDUCTION_PLACEHOLDER(NODE) \
OMP_CLAUSE_OPERAND (OMP_CLAUSE_SUBCODE_CHECK (NODE, OMP_CLAUSE_REDUCTION), 3)
/* True if a LINEAR clause doesn't need copy in. True for iterator vars which
are always initialized inside of the loop construct, false otherwise. */
#define OMP_CLAUSE_LINEAR_NO_COPYIN(NODE) \
(OMP_CLAUSE_SUBCODE_CHECK (NODE, OMP_CLAUSE_LINEAR)->base.public_flag)
/* True if a LINEAR clause doesn't need copy out. True for iterator vars which
are declared inside of the simd construct. */
#define OMP_CLAUSE_LINEAR_NO_COPYOUT(NODE) \
TREE_PRIVATE (OMP_CLAUSE_SUBCODE_CHECK (NODE, OMP_CLAUSE_LINEAR))
#define OMP_CLAUSE_LINEAR_STEP(NODE) \
OMP_CLAUSE_OPERAND (OMP_CLAUSE_SUBCODE_CHECK (NODE, OMP_CLAUSE_LINEAR), 1)
#define OMP_CLAUSE_SAFELEN_EXPR(NODE) \
OMP_CLAUSE_OPERAND (OMP_CLAUSE_SUBCODE_CHECK (NODE, OMP_CLAUSE_SAFELEN), 0)
#define OMP_CLAUSE__SIMDUID__DECL(NODE) \
OMP_CLAUSE_OPERAND (OMP_CLAUSE_SUBCODE_CHECK (NODE, OMP_CLAUSE__SIMDUID_), 0)
enum omp_clause_schedule_kind
{
OMP_CLAUSE_SCHEDULE_STATIC,
@ -4813,6 +4850,7 @@ extern tree build_translation_unit_decl (tree);
extern tree build_block (tree, tree, tree, tree);
extern tree build_empty_stmt (location_t);
extern tree build_omp_clause (location_t, enum omp_clause_code);
extern tree find_omp_clause (tree, enum omp_clause_code);
extern tree build_vl_exp_stat (enum tree_code, int MEM_STAT_DECL);
#define build_vl_exp(c,n) build_vl_exp_stat (c,n MEM_STAT_INFO)