procyberian/gcc
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

c-tree.texi: Document new tree codes.

Browse source 
* doc/c-tree.texi: Document new tree codes.
        * doc/md.texi: Document new optabs.
        * tree-pretty-print.c (dump_generic_node): Handle print of new tree
        codes.
        * optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
        * optabs.h (optab_index): Add new.
        (vec_extract_even_optab, vec_extract_odd_optab,
        vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
        * genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
        vec_interleave_high_optab, vec_interleave_low_optab): Initialize
        new optabs.
        * expr.c (expand_expr_real_1): Add implementation for new tree codes.
        * tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
        * tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
        along with macros for their access.
        * tree-data-ref.h (first_location_in_loop, data_reference): Update
        comment.
        * tree-vect-analyze.c (toplev.h): Include.
        (vect_determine_vectorization_factor): Fix indentation.
        (vect_insert_into_interleaving_chain,
        vect_update_interleaving_chain, vect_equal_offsets): New functions.
        (vect_analyze_data_ref_dependence): Add argument for interleaving
        check. Check for interleaving if it's true.
        (vect_check_dependences): New function.
        (vect_analyze_data_ref_dependences): Call vect_check_dependences for
        every ddr. Call vect_analyze_data_ref_dependence with new argument.
        (vect_update_misalignment_for_peel): Update for interleaving.
        (vect_verify_datarefs_alignment): Check only first data-ref for
        interleaving.
        (vect_enhance_data_refs_alignment): Update for interleaving. Check
        only first data-ref for interleaving.
        (vect_analyze_data_ref_access): Check interleaving, update
        interleaving data.
        (vect_analyze_data_refs): Call compute_data_dependences_for_loop
        with different parameters.
        * tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
        VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
        * tree-inline.c (estimate_num_insns_1): Add cases for new codes.
        * tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
        Update step in case of interleaving.
        (vect_strided_store_supported, vect_permute_store_chain): New
        functions.
        (vectorizable_store): Handle strided stores.
        (vect_strided_load_supported, vect_permute_load_chain,
        vect_transform_strided_load): New functions.
        (vectorizable_load): Handle strided loads.
        (vect_transform_stmt): Add argument. Handle strided stores. Check
        that vectorized stmt exists for patterns.
        (vect_gen_niters_for_prolog_loop): Update calculation for
        interleaving.
        (vect_transform_loop): Remove stmt_vec_info for strided stores after
        whole chain vectorization.
        * config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
        UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
        (vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
        vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
        vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.

From-SVN: r119088
This commit is contained in:
Ira Rosen 2006-11-22 08:46:03 +00:00 • committed by Ira Rosen
parent b0c6db58ad
commit 98b44b0eea
43 changed files with 3768 additions and 200 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

60
gcc/ChangeLog
View file

@ -1,3 +1,63 @@
2006-11-22 Ira Rosen <irar@il.ibm.com>
* doc/c-tree.texi: Document new tree codes.
* doc/md.texi: Document new optabs.
* tree-pretty-print.c (dump_generic_node): Handle print of new tree
codes.
* optabs.c (optab_for_tree_code, init_optabs): Handle new optabs.
* optabs.h (optab_index): Add new.
(vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): New optabs.
* genopinit.c (vec_extract_even_optab, vec_extract_odd_optab,
vec_interleave_high_optab, vec_interleave_low_optab): Initialize
new optabs.
* expr.c (expand_expr_real_1): Add implementation for new tree codes.
* tree-vectorizer.c (new_stmt_vec_info): Initialize new fields.
* tree-vectorizer.h (stmt_vec_info): Add new fields for interleaving
along with macros for their access.
* tree-data-ref.h (first_location_in_loop, data_reference): Update
comment.
* tree-vect-analyze.c (toplev.h): Include.
(vect_determine_vectorization_factor): Fix indentation.
(vect_insert_into_interleaving_chain,
vect_update_interleaving_chain, vect_equal_offsets): New functions.
(vect_analyze_data_ref_dependence): Add argument for interleaving
check. Check for interleaving if it's true.
(vect_check_dependences): New function.
(vect_analyze_data_ref_dependences): Call vect_check_dependences for
every ddr. Call vect_analyze_data_ref_dependence with new argument.
(vect_update_misalignment_for_peel): Update for interleaving.
(vect_verify_datarefs_alignment): Check only first data-ref for
interleaving.
(vect_enhance_data_refs_alignment): Update for interleaving. Check
only first data-ref for interleaving.
(vect_analyze_data_ref_access): Check interleaving, update
interleaving data.
(vect_analyze_data_refs): Call compute_data_dependences_for_loop
with different parameters.
* tree.def (VEC_EXTRACT_EVEN_EXPR, VEC_EXTRACT_ODD_EXPR,
VEC_INTERLEAVE_HIGH_EXPR, VEC_INTERLEAVE_LOW_EXPR): New tree codes.
* tree-inline.c (estimate_num_insns_1): Add cases for new codes.
* tree-vect-transform.c (vect_create_addr_base_for_vector_ref):
Update step in case of interleaving.
(vect_strided_store_supported, vect_permute_store_chain): New
functions.
(vectorizable_store): Handle strided stores.
(vect_strided_load_supported, vect_permute_load_chain,
vect_transform_strided_load): New functions.
(vectorizable_load): Handle strided loads.
(vect_transform_stmt): Add argument. Handle strided stores. Check
that vectorized stmt exists for patterns.
(vect_gen_niters_for_prolog_loop): Update calculation for
interleaving.
(vect_transform_loop): Remove stmt_vec_info for strided stores after
whole chain vectorization.
* config/rs6000/altivec.md (UNSPEC_EXTEVEN, UNSPEC_EXTODD,
UNSPEC_INTERHI, UNSPEC_INTERLO): New constants.
(vpkuhum_nomode, vpkuwum_nomode, vec_extract_even<mode>,
vec_extract_odd<mode>, altivec_vmrghsf, altivec_vmrglsf,
vec_interleave_high<mode>, vec_interleave_low<mode>): Implement.
2006-11-22 Steven Bosscher <steven@gcc.gnu.org>
* cse.c (enum taken): Remove PATH_AROUND.

341
gcc/config/rs6000/altivec.md
View file

@ -115,6 +115,22 @@
(UNSPEC_REALIGN_LOAD 215)
(UNSPEC_REDUC_PLUS 217)
(UNSPEC_VECSH 219)
(UNSPEC_EXTEVEN_V4SI 220)
(UNSPEC_EXTEVEN_V8HI 221)
(UNSPEC_EXTEVEN_V16QI 222)
(UNSPEC_EXTEVEN_V4SF 223)
(UNSPEC_EXTODD_V4SI 224)
(UNSPEC_EXTODD_V8HI 225)
(UNSPEC_EXTODD_V16QI 226)
(UNSPEC_EXTODD_V4SF 227)
(UNSPEC_INTERHI_V4SI 228)
(UNSPEC_INTERHI_V8HI 229)
(UNSPEC_INTERHI_V16QI 230)
(UNSPEC_INTERHI_V4SF 231)
(UNSPEC_INTERLO_V4SI 232)
(UNSPEC_INTERLO_V8HI 233)
(UNSPEC_INTERLO_V16QI 234)
(UNSPEC_INTERLO_V4SF 235)
(UNSPEC_VCOND_V4SI 301)
(UNSPEC_VCOND_V4SF 302)
(UNSPEC_VCOND_V8HI 303)
@ -136,7 +152,9 @@
(UNSPEC_VUPKLUH 319)
(UNSPEC_VPERMSI 320)
(UNSPEC_VPERMHI 321)
])
(UNSPEC_INTERHI 322)
(UNSPEC_INTERLO 323)
])
(define_constants
[(UNSPECV_SET_VRSAVE 30)
@ -855,6 +873,23 @@
"vmrghw %0,%1,%2"
[(set_attr "type" "vecperm")])
(define_insn "altivec_vmrghsf"
[(set (match_operand:V4SF 0 "register_operand" "=v")
(vec_merge:V4SF (vec_select:V4SF (match_operand:V4SF 1 "register_operand" "v")
(parallel [(const_int 0)
(const_int 2)
(const_int 1)
(const_int 3)]))
(vec_select:V4SF (match_operand:V4SF 2 "register_operand" "v")
(parallel [(const_int 2)
(const_int 0)
(const_int 3)
(const_int 1)]))
(const_int 5)))]
"TARGET_ALTIVEC"
"vmrghw %0,%1,%2"
[(set_attr "type" "vecperm")])
(define_insn "altivec_vmrglb"
[(set (match_operand:V16QI 0 "register_operand" "=v")
(vec_merge:V16QI (vec_select:V16QI (match_operand:V16QI 1 "register_operand" "v")
@ -938,6 +973,23 @@
"vmrglw %0,%1,%2"
[(set_attr "type" "vecperm")])
(define_insn "altivec_vmrglsf"
[(set (match_operand:V4SF 0 "register_operand" "=v")
(vec_merge:V4SF (vec_select:V4SF (match_operand:V4SF 1 "register_operand" "v")
(parallel [(const_int 2)
(const_int 0)
(const_int 3)
(const_int 1)]))
(vec_select:V4SF (match_operand:V4SF 2 "register_operand" "v")
(parallel [(const_int 0)
(const_int 2)
(const_int 1)
(const_int 3)]))
(const_int 5)))]
"TARGET_ALTIVEC"
"vmrglw %0,%1,%2"
[(set_attr "type" "vecperm")])
(define_insn "altivec_vmuleub"
[(set (match_operand:V8HI 0 "register_operand" "=v")
(unspec:V8HI [(match_operand:V16QI 1 "register_operand" "v")
@ -2601,3 +2653,290 @@
DONE;
}")
(define_expand "vec_extract_evenv4si"
[(set (match_operand:V4SI 0 "register_operand" "")
(unspec:V8HI [(match_operand:V4SI 1 "register_operand" "")
(match_operand:V4SI 2 "register_operand" "")]
UNSPEC_EXTEVEN_V4SI))]
"TARGET_ALTIVEC"
"
{
rtx mask = gen_reg_rtx (V16QImode);
rtvec v = rtvec_alloc (16);
RTVEC_ELT (v, 0) = gen_rtx_CONST_INT (QImode, 0);
RTVEC_ELT (v, 1) = gen_rtx_CONST_INT (QImode, 1);
RTVEC_ELT (v, 2) = gen_rtx_CONST_INT (QImode, 2);
RTVEC_ELT (v, 3) = gen_rtx_CONST_INT (QImode, 3);
RTVEC_ELT (v, 4) = gen_rtx_CONST_INT (QImode, 8);
RTVEC_ELT (v, 5) = gen_rtx_CONST_INT (QImode, 9);
RTVEC_ELT (v, 6) = gen_rtx_CONST_INT (QImode, 10);
RTVEC_ELT (v, 7) = gen_rtx_CONST_INT (QImode, 11);
RTVEC_ELT (v, 8) = gen_rtx_CONST_INT (QImode, 16);
RTVEC_ELT (v, 9) = gen_rtx_CONST_INT (QImode, 17);
RTVEC_ELT (v, 10) = gen_rtx_CONST_INT (QImode, 18);
RTVEC_ELT (v, 11) = gen_rtx_CONST_INT (QImode, 19);
RTVEC_ELT (v, 12) = gen_rtx_CONST_INT (QImode, 24);
RTVEC_ELT (v, 13) = gen_rtx_CONST_INT (QImode, 25);
RTVEC_ELT (v, 14) = gen_rtx_CONST_INT (QImode, 26);
RTVEC_ELT (v, 15) = gen_rtx_CONST_INT (QImode, 27);
emit_insn (gen_vec_initv16qi (mask, gen_rtx_PARALLEL (V16QImode, v)));
emit_insn (gen_altivec_vperm_v4si (operands[0], operands[1], operands[2], mask));
DONE;
}")
(define_expand "vec_extract_evenv4sf"
[(set (match_operand:V4SF 0 "register_operand" "")
(unspec:V8HI [(match_operand:V4SF 1 "register_operand" "")
(match_operand:V4SF 2 "register_operand" "")]
UNSPEC_EXTEVEN_V4SF))]
"TARGET_ALTIVEC"
"
{
rtx mask = gen_reg_rtx (V16QImode);
rtvec v = rtvec_alloc (16);
RTVEC_ELT (v, 0) = gen_rtx_CONST_INT (QImode, 0);
RTVEC_ELT (v, 1) = gen_rtx_CONST_INT (QImode, 1);
RTVEC_ELT (v, 2) = gen_rtx_CONST_INT (QImode, 2);
RTVEC_ELT (v, 3) = gen_rtx_CONST_INT (QImode, 3);
RTVEC_ELT (v, 4) = gen_rtx_CONST_INT (QImode, 8);
RTVEC_ELT (v, 5) = gen_rtx_CONST_INT (QImode, 9);
RTVEC_ELT (v, 6) = gen_rtx_CONST_INT (QImode, 10);
RTVEC_ELT (v, 7) = gen_rtx_CONST_INT (QImode, 11);
RTVEC_ELT (v, 8) = gen_rtx_CONST_INT (QImode, 16);
RTVEC_ELT (v, 9) = gen_rtx_CONST_INT (QImode, 17);
RTVEC_ELT (v, 10) = gen_rtx_CONST_INT (QImode, 18);
RTVEC_ELT (v, 11) = gen_rtx_CONST_INT (QImode, 19);
RTVEC_ELT (v, 12) = gen_rtx_CONST_INT (QImode, 24);
RTVEC_ELT (v, 13) = gen_rtx_CONST_INT (QImode, 25);
RTVEC_ELT (v, 14) = gen_rtx_CONST_INT (QImode, 26);
RTVEC_ELT (v, 15) = gen_rtx_CONST_INT (QImode, 27);
emit_insn (gen_vec_initv16qi (mask, gen_rtx_PARALLEL (V16QImode, v)));
emit_insn (gen_altivec_vperm_v4sf (operands[0], operands[1], operands[2], mask));
DONE;
}")
(define_expand "vec_extract_evenv8hi"
[(set (match_operand:V4SI 0 "register_operand" "")
(unspec:V8HI [(match_operand:V8HI 1 "register_operand" "")
(match_operand:V8HI 2 "register_operand" "")]
UNSPEC_EXTEVEN_V8HI))]
"TARGET_ALTIVEC"
"
{
rtx mask = gen_reg_rtx (V16QImode);
rtvec v = rtvec_alloc (16);
RTVEC_ELT (v, 0) = gen_rtx_CONST_INT (QImode, 0);
RTVEC_ELT (v, 1) = gen_rtx_CONST_INT (QImode, 1);
RTVEC_ELT (v, 2) = gen_rtx_CONST_INT (QImode, 4);
RTVEC_ELT (v, 3) = gen_rtx_CONST_INT (QImode, 5);
RTVEC_ELT (v, 4) = gen_rtx_CONST_INT (QImode, 8);
RTVEC_ELT (v, 5) = gen_rtx_CONST_INT (QImode, 9);
RTVEC_ELT (v, 6) = gen_rtx_CONST_INT (QImode, 12);
RTVEC_ELT (v, 7) = gen_rtx_CONST_INT (QImode, 13);
RTVEC_ELT (v, 8) = gen_rtx_CONST_INT (QImode, 16);
RTVEC_ELT (v, 9) = gen_rtx_CONST_INT (QImode, 17);
RTVEC_ELT (v, 10) = gen_rtx_CONST_INT (QImode, 20);
RTVEC_ELT (v, 11) = gen_rtx_CONST_INT (QImode, 21);
RTVEC_ELT (v, 12) = gen_rtx_CONST_INT (QImode, 24);
RTVEC_ELT (v, 13) = gen_rtx_CONST_INT (QImode, 25);
RTVEC_ELT (v, 14) = gen_rtx_CONST_INT (QImode, 28);
RTVEC_ELT (v, 15) = gen_rtx_CONST_INT (QImode, 29);
emit_insn (gen_vec_initv16qi (mask, gen_rtx_PARALLEL (V16QImode, v)));
emit_insn (gen_altivec_vperm_v8hi (operands[0], operands[1], operands[2], mask));
DONE;
}")
(define_expand "vec_extract_evenv16qi"
[(set (match_operand:V4SI 0 "register_operand" "")
(unspec:V8HI [(match_operand:V16QI 1 "register_operand" "")
(match_operand:V16QI 2 "register_operand" "")]
UNSPEC_EXTEVEN_V16QI))]
"TARGET_ALTIVEC"
"
{
rtx mask = gen_reg_rtx (V16QImode);
rtvec v = rtvec_alloc (16);
RTVEC_ELT (v, 0) = gen_rtx_CONST_INT (QImode, 0);
RTVEC_ELT (v, 1) = gen_rtx_CONST_INT (QImode, 2);
RTVEC_ELT (v, 2) = gen_rtx_CONST_INT (QImode, 4);
RTVEC_ELT (v, 3) = gen_rtx_CONST_INT (QImode, 6);
RTVEC_ELT (v, 4) = gen_rtx_CONST_INT (QImode, 8);
RTVEC_ELT (v, 5) = gen_rtx_CONST_INT (QImode, 10);
RTVEC_ELT (v, 6) = gen_rtx_CONST_INT (QImode, 12);
RTVEC_ELT (v, 7) = gen_rtx_CONST_INT (QImode, 14);
RTVEC_ELT (v, 8) = gen_rtx_CONST_INT (QImode, 16);
RTVEC_ELT (v, 9) = gen_rtx_CONST_INT (QImode, 18);
RTVEC_ELT (v, 10) = gen_rtx_CONST_INT (QImode, 20);
RTVEC_ELT (v, 11) = gen_rtx_CONST_INT (QImode, 22);
RTVEC_ELT (v, 12) = gen_rtx_CONST_INT (QImode, 24);
RTVEC_ELT (v, 13) = gen_rtx_CONST_INT (QImode, 26);
RTVEC_ELT (v, 14) = gen_rtx_CONST_INT (QImode, 28);
RTVEC_ELT (v, 15) = gen_rtx_CONST_INT (QImode, 30);
emit_insn (gen_vec_initv16qi (mask, gen_rtx_PARALLEL (V16QImode, v)));
emit_insn (gen_altivec_vperm_v16qi (operands[0], operands[1], operands[2], mask));
DONE;
}")
(define_expand "vec_extract_oddv4si"
[(set (match_operand:V4SI 0 "register_operand" "")
(unspec:V8HI [(match_operand:V4SI 1 "register_operand" "")
(match_operand:V4SI 2 "register_operand" "")]
UNSPEC_EXTODD_V4SI))]
"TARGET_ALTIVEC"
"
{
rtx mask = gen_reg_rtx (V16QImode);
rtvec v = rtvec_alloc (16);
RTVEC_ELT (v, 0) = gen_rtx_CONST_INT (QImode, 4);
RTVEC_ELT (v, 1) = gen_rtx_CONST_INT (QImode, 5);
RTVEC_ELT (v, 2) = gen_rtx_CONST_INT (QImode, 6);
RTVEC_ELT (v, 3) = gen_rtx_CONST_INT (QImode, 7);
RTVEC_ELT (v, 4) = gen_rtx_CONST_INT (QImode, 12);
RTVEC_ELT (v, 5) = gen_rtx_CONST_INT (QImode, 13);
RTVEC_ELT (v, 6) = gen_rtx_CONST_INT (QImode, 14);
RTVEC_ELT (v, 7) = gen_rtx_CONST_INT (QImode, 15);
RTVEC_ELT (v, 8) = gen_rtx_CONST_INT (QImode, 20);
RTVEC_ELT (v, 9) = gen_rtx_CONST_INT (QImode, 21);
RTVEC_ELT (v, 10) = gen_rtx_CONST_INT (QImode, 22);
RTVEC_ELT (v, 11) = gen_rtx_CONST_INT (QImode, 23);
RTVEC_ELT (v, 12) = gen_rtx_CONST_INT (QImode, 28);
RTVEC_ELT (v, 13) = gen_rtx_CONST_INT (QImode, 29);
RTVEC_ELT (v, 14) = gen_rtx_CONST_INT (QImode, 30);
RTVEC_ELT (v, 15) = gen_rtx_CONST_INT (QImode, 31);
emit_insn (gen_vec_initv16qi (mask, gen_rtx_PARALLEL (V16QImode, v)));
emit_insn (gen_altivec_vperm_v4si (operands[0], operands[1], operands[2], mask));
DONE;
}")
(define_expand "vec_extract_oddv4sf"
[(set (match_operand:V4SF 0 "register_operand" "")
(unspec:V8HI [(match_operand:V4SF 1 "register_operand" "")
(match_operand:V4SF 2 "register_operand" "")]
UNSPEC_EXTODD_V4SF))]
"TARGET_ALTIVEC"
"
{
rtx mask = gen_reg_rtx (V16QImode);
rtvec v = rtvec_alloc (16);
RTVEC_ELT (v, 0) = gen_rtx_CONST_INT (QImode, 4);
RTVEC_ELT (v, 1) = gen_rtx_CONST_INT (QImode, 5);
RTVEC_ELT (v, 2) = gen_rtx_CONST_INT (QImode, 6);
RTVEC_ELT (v, 3) = gen_rtx_CONST_INT (QImode, 7);
RTVEC_ELT (v, 4) = gen_rtx_CONST_INT (QImode, 12);
RTVEC_ELT (v, 5) = gen_rtx_CONST_INT (QImode, 13);
RTVEC_ELT (v, 6) = gen_rtx_CONST_INT (QImode, 14);
RTVEC_ELT (v, 7) = gen_rtx_CONST_INT (QImode, 15);
RTVEC_ELT (v, 8) = gen_rtx_CONST_INT (QImode, 20);
RTVEC_ELT (v, 9) = gen_rtx_CONST_INT (QImode, 21);
RTVEC_ELT (v, 10) = gen_rtx_CONST_INT (QImode, 22);
RTVEC_ELT (v, 11) = gen_rtx_CONST_INT (QImode, 23);
RTVEC_ELT (v, 12) = gen_rtx_CONST_INT (QImode, 28);
RTVEC_ELT (v, 13) = gen_rtx_CONST_INT (QImode, 29);
RTVEC_ELT (v, 14) = gen_rtx_CONST_INT (QImode, 30);
RTVEC_ELT (v, 15) = gen_rtx_CONST_INT (QImode, 31);
emit_insn (gen_vec_initv16qi (mask, gen_rtx_PARALLEL (V16QImode, v)));
emit_insn (gen_altivec_vperm_v4sf (operands[0], operands[1], operands[2], mask));
DONE;
}")
(define_insn "vpkuhum_nomode"
[(set (match_operand:V16QI 0 "register_operand" "=v")
(unspec:V16QI [(match_operand 1 "register_operand" "v")
(match_operand 2 "register_operand" "v")]
UNSPEC_VPKUHUM))]
"TARGET_ALTIVEC"
"vpkuhum %0,%1,%2"
[(set_attr "type" "vecperm")])
(define_insn "vpkuwum_nomode"
[(set (match_operand:V8HI 0 "register_operand" "=v")
(unspec:V8HI [(match_operand 1 "register_operand" "v")
(match_operand 2 "register_operand" "v")]
UNSPEC_VPKUWUM))]
"TARGET_ALTIVEC"
"vpkuwum %0,%1,%2"
[(set_attr "type" "vecperm")])
(define_expand "vec_extract_oddv8hi"
[(set (match_operand:V8HI 0 "register_operand" "")
(unspec:V8HI [(match_operand:V8HI 1 "register_operand" "")
(match_operand:V8HI 2 "register_operand" "")]
UNSPEC_EXTODD_V8HI))]
"TARGET_ALTIVEC"
"
{
emit_insn (gen_vpkuwum_nomode (operands[0], operands[1], operands[2]));
DONE;
}")
(define_expand "vec_extract_oddv16qi"
[(set (match_operand:V16QI 0 "register_operand" "")
(unspec:V16QI [(match_operand:V16QI 1 "register_operand" "")
(match_operand:V16QI 2 "register_operand" "")]
UNSPEC_EXTODD_V16QI))]
"TARGET_ALTIVEC"
"
{
emit_insn (gen_vpkuhum_nomode (operands[0], operands[1], operands[2]));
DONE;
}")
(define_expand "vec_interleave_highv4sf"
[(set (match_operand:V4SF 0 "register_operand" "")
(unspec:V4SF [(match_operand:V4SF 1 "register_operand" "")
(match_operand:V4SF 2 "register_operand" "")]
UNSPEC_INTERHI_V4SF))]
"TARGET_ALTIVEC"
"
{
emit_insn (gen_altivec_vmrghsf (operands[0], operands[1], operands[2]));
DONE;
}")
(define_expand "vec_interleave_lowv4sf"
[(set (match_operand:V4SF 0 "register_operand" "")
(unspec:V4SF [(match_operand:V4SF 1 "register_operand" "")
(match_operand:V4SF 2 "register_operand" "")]
UNSPEC_INTERLO_V4SF))]
"TARGET_ALTIVEC"
"
{
emit_insn (gen_altivec_vmrglsf (operands[0], operands[1], operands[2]));
DONE;
}")
(define_expand "vec_interleave_high<mode>"
[(set (match_operand:VI 0 "register_operand" "")
(unspec:VI [(match_operand:VI 1 "register_operand" "")
(match_operand:VI 2 "register_operand" "")]
UNSPEC_INTERHI))]
"TARGET_ALTIVEC"
"
{
emit_insn (gen_altivec_vmrgh<VI_char> (operands[0], operands[1], operands[2]));
DONE;
}")
(define_expand "vec_interleave_low<mode>"
[(set (match_operand:VI 0 "register_operand" "")
(unspec:VI [(match_operand:VI 1 "register_operand" "")
(match_operand:VI 2 "register_operand" "")]
UNSPEC_INTERLO))]
"TARGET_ALTIVEC"
"
{
emit_insn (gen_altivec_vmrgl<VI_char> (operands[0], operands[1], operands[2]));
DONE;
}")

21
gcc/doc/c-tree.texi
View file

@ -1936,6 +1936,10 @@ This macro returns the attributes on the type @var{type}.
@tindex VEC_UNPACK_LO_EXPR
@tindex VEC_PACK_MOD_EXPR
@tindex VEC_PACK_SAT_EXPR
@tindex VEC_EXTRACT_EVEN_EXPR
@tindex VEC_EXTRACT_ODD_EXPR
@tindex VEC_INTERLEAVE_HIGH_EXPR
@tindex VEC_INTERLEAVE_LOW_EXPR
The internal representation for expressions is for the most part quite
straightforward. However, there are a few facts that one must bear in
@ -2783,4 +2787,21 @@ elements, of an integral type whose size is half as wide. In both cases
the elements of the two vectors are demoted and merged (concatenated) to form
the output vector.
@item VEC_EXTRACT_EVEN_EXPR
@item VEC_EXTRACT_ODD_EXPR
These nodes represent extracting of the even/odd elements of the two input
vectors, respectively. Their operands and result are vectors that contain the
same number of elements of the same type.
@item VEC_INTERLEAVE_HIGH_EXPR
@item VEC_INTERLEAVE_LOW_EXPR
These nodes represent merging and interleaving of the high/low elements of the
two input vectors, respectively. The operands and the result are vectors that
contain the same number of elements (@code{N}) of the same type.
In the case of @code{VEC_INTERLEAVE_HIGH_EXPR}, the high @code{N/2} elements of
the first input vector are interleaved with the high @code{N/2} elements of the
second input vector. In the case of @code{VEC_INTERLEAVE_LOW_EXPR}, the low
@code{N/2} elements of the first input vector are interleaved with the low
@code{N/2} elements of the second input vector.
@end table

28
gcc/doc/md.texi
View file

@ -3465,6 +3465,34 @@ operand 1 is new value of field and operand 2 specify the field index.
Extract given field from the vector value. Operand 1 is the vector, operand 2
specify field index and operand 0 place to store value into.
@cindex @code{vec_extract_even@var{m}} instruction pattern
@item @samp{vec_extract_even@var{m}}
Extract even elements from the input vectors (operand 1 and operand 2).
The even elements of operand 2 are concatenated to the even elements of operand
1 in their original order. The result is stored in operand 0.
The output and input vectors should have the same modes.
@cindex @code{vec_extract_odd@var{m}} instruction pattern
@item @samp{vec_extract_odd@var{m}}
Extract odd elements from the input vectors (operand 1 and operand 2).
The odd elements of operand 2 are concatenated to the odd elements of operand
1 in their original order. The result is stored in operand 0.
The output and input vectors should have the same modes.
@cindex @code{vec_interleave_high@var{m}} instruction pattern
@item @samp{vec_interleave_high@var{m}}
Merge high elements of the two input vectors into the output vector. The output
and input vectors should have the same modes (@code{N} elements). The high
@code{N/2} elements of the first input vector are interleaved with the high
@code{N/2} elements of the second input vector.
@cindex @code{vec_interleave_low@var{m}} instruction pattern
@item @samp{vec_interleave_low@var{m}}
Merge low elements of the two input vectors into the output vector. The output
and input vectors should have the same modes (@code{N} elements). The low
@code{N/2} elements of the first input vector are interleaved with the low
@code{N/2} elements of the second input vector.
@cindex @code{vec_init@var{m}} instruction pattern
@item @samp{vec_init@var{m}}
Initialize the vector to given values. Operand 0 is the vector to initialize

24
gcc/expr.c
View file

@ -8745,6 +8745,30 @@ expand_expr_real_1 (tree exp, rtx target, enum machine_mode tmode,
return temp;
}
case VEC_EXTRACT_EVEN_EXPR:
case VEC_EXTRACT_ODD_EXPR:
{
expand_operands (TREE_OPERAND (exp, 0), TREE_OPERAND (exp, 1),
NULL_RTX, &op0, &op1, 0);
this_optab = optab_for_tree_code (code, type);
temp = expand_binop (mode, this_optab, op0, op1, target, unsignedp,
OPTAB_WIDEN);
gcc_assert (temp);
return temp;
}
case VEC_INTERLEAVE_HIGH_EXPR:
case VEC_INTERLEAVE_LOW_EXPR:
{
expand_operands (TREE_OPERAND (exp, 0), TREE_OPERAND (exp, 1),
NULL_RTX, &op0, &op1, 0);
this_optab = optab_for_tree_code (code, type);
temp = expand_binop (mode, this_optab, op0, op1, target, unsignedp,
OPTAB_WIDEN);
gcc_assert (temp);
return temp;
}
case VEC_LSHIFT_EXPR:
case VEC_RSHIFT_EXPR:
{

4
gcc/genopinit.c
View file

@ -199,6 +199,10 @@ static const char * const optabs[] =
"sync_lock_release[$A] = CODE_FOR_$(sync_lock_release$I$a$)",
"vec_set_optab->handlers[$A].insn_code = CODE_FOR_$(vec_set$a$)",
"vec_extract_optab->handlers[$A].insn_code = CODE_FOR_$(vec_extract$a$)",
"vec_extract_even_optab->handlers[$A].insn_code = CODE_FOR_$(vec_extract_even$a$)",
"vec_extract_odd_optab->handlers[$A].insn_code = CODE_FOR_$(vec_extract_odd$a$)",
"vec_interleave_high_optab->handlers[$A].insn_code = CODE_FOR_$(vec_interleave_high$a$)",
"vec_interleave_low_optab->handlers[$A].insn_code = CODE_FOR_$(vec_interleave_low$a$)",
"vec_init_optab->handlers[$A].insn_code = CODE_FOR_$(vec_init$a$)",
"vec_shl_optab->handlers[$A].insn_code = CODE_FOR_$(vec_shl_$a$)",
"vec_shr_optab->handlers[$A].insn_code = CODE_FOR_$(vec_shr_$a$)",

16
gcc/optabs.c
View file

@ -359,6 +359,18 @@ optab_for_tree_code (enum tree_code code, tree type)
case ABS_EXPR:
return trapv ? absv_optab : abs_optab;
case VEC_EXTRACT_EVEN_EXPR:
return vec_extract_even_optab;
case VEC_EXTRACT_ODD_EXPR:
return vec_extract_odd_optab;
case VEC_INTERLEAVE_HIGH_EXPR:
return vec_interleave_high_optab;
case VEC_INTERLEAVE_LOW_EXPR:
return vec_interleave_low_optab;
default:
return NULL;
}
@ -5384,6 +5396,10 @@ init_optabs (void)
udot_prod_optab = init_optab (UNKNOWN);
vec_extract_optab = init_optab (UNKNOWN);
vec_extract_even_optab = init_optab (UNKNOWN);
vec_extract_odd_optab = init_optab (UNKNOWN);
vec_interleave_high_optab = init_optab (UNKNOWN);
vec_interleave_low_optab = init_optab (UNKNOWN);
vec_set_optab = init_optab (UNKNOWN);
vec_init_optab = init_optab (UNKNOWN);
vec_shl_optab = init_optab (UNKNOWN);

11
gcc/optabs.h
View file

@ -27,6 +27,7 @@ Boston, MA 02110-1301, USA. */
/* Optabs are tables saying how to generate insn bodies
for various machine modes and numbers of operands.
Each optab applies to one operation.
For example, add_optab applies to addition.
The insn_code slot is the enum insn_code that says how to
@ -253,6 +254,12 @@ enum optab_index
OTI_vec_set,
/* Extract specified field of vector operand. */
OTI_vec_extract,
/* Extract even/odd fields of vector operands. */
OTI_vec_extract_even,
OTI_vec_extract_odd,
/* Interleave fields of vector operands. */
OTI_vec_interleave_high,
OTI_vec_interleave_low,
/* Initialize vector operand. */
OTI_vec_init,
/* Whole vector shift. The shift amount is in bits. */
@ -397,6 +404,10 @@ extern GTY(()) optab optab_table[OTI_MAX];
#define vec_set_optab (optab_table[OTI_vec_set])
#define vec_extract_optab (optab_table[OTI_vec_extract])
#define vec_extract_even_optab (optab_table[OTI_vec_extract_even])
#define vec_extract_odd_optab (optab_table[OTI_vec_extract_odd])
#define vec_interleave_high_optab (optab_table[OTI_vec_interleave_high])
#define vec_interleave_low_optab (optab_table[OTI_vec_interleave_low])
#define vec_init_optab (optab_table[OTI_vec_init])
#define vec_shl_optab (optab_table[OTI_vec_shl])
#define vec_shr_optab (optab_table[OTI_vec_shr])

26
gcc/testsuite/ChangeLog
View file

@ -1,3 +1,29 @@
2006-11-22 Ira Rosen <irar@il.ibm.com>
* gcc.dg/vect/vect-1.c: Additional loop is now vectorizable on
platforms that have interleaving support.
* gcc.dg/vect/vect-107.c, gcc.dg/vect/vect-98.c: Likewise.
* gcc.dg/vect/vect-strided-a-u16-i2.c,
gcc.dg/vect/vect-strided-a-u16-i4.c, gcc.dg/vect/vect-strided-u16-i2.c,
gcc.dg/vect/vect-strided-u16-i4.c, gcc.dg/vect/vect-strided-u32-i4.c,
gcc.dg/vect/vect-strided-u32-i8.c, gcc.dg/vect/vect-strided-u8-i2.c,
gcc.dg/vect/vect-strided-u8-i2-gap.c,
gcc.dg/vect/vect-strided-u8-i8.c,
gcc.dg/vect/vect-strided-u8-i8-gap2.c,
gcc.dg/vect/vect-strided-u8-i8-gap4.c,
gcc.dg/vect/vect-strided-u8-i8-gap7.c,
gcc.dg/vect/vect-strided-float.c,
gcc.dg/vect/vect-strided-a-mult.c,
gcc.dg/vect/vect-strided-mult-char-ls.c,
gcc.dg/vect/vect-strided-a-u16-mult.c,
gcc.dg/vect/vect-strided-a-u32-mult.c,
gcc.dg/vect/vect-strided-a-u8-i2-gap.c,
gcc.dg/vect/vect-strided-a-u8-i8-gap2.c,
gcc.dg/vect/vect-strided-a-u8-i8-gap7.c,
gcc.dg/vect/vect-strided-mult.c,
gcc.dg/vect/vect-strided-u32-mult.c: New testcases.
* lib/target-supports.exp (vect_extract_even_odd, vect_interleave): New.
2006-11-22 Paul Thomas <pault@gcc.gnu.org>
PR fortran/25087

7
gcc/testsuite/gcc.dg/vect/vect-1.c
View file

@ -59,7 +59,8 @@ foo (int n)
fbar (a);
/* Not vectorizable yet (access pattern). */
/* Strided access. Vectorizable on platforms that support load of strided
accesses (extract of even/odd vector elements). */
for (i = 0; i < N/2; i++){
a[i] = b[2*i+1] * c[2*i+1] - b[2*i] * c[2*i];
d[i] = b[2*i] * c[2*i+1] + b[2*i+1] * c[2*i];
@ -85,6 +86,6 @@ foo (int n)
fbar (a);
}
/* { dg-final { scan-tree-dump-times "vectorized 3 loops" 1 "vect" } } */
/* { dg-final { scan-tree-dump-times "Vectorizing an unaligned access" 0 "vect" } } */
/* { dg-final { scan-tree-dump-times "vectorized 4 loops" 1 "vect" { target vect_extract_even_odd } } } */
/* { dg-final { scan-tree-dump-times "vectorized 3 loops" 1 "vect" { xfail vect_extract_even_odd } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

6
gcc/testsuite/gcc.dg/vect/vect-107.c
View file

@ -14,7 +14,8 @@ main1 (void)
float c[N] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
float d[N] = {0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30};
/* Strided access pattern. */
/* Strided access. Vectorizable on platforms that support load of strided
accesses (extract of even/odd vector elements). */
for (i = 0; i < N/2; i++)
{
a[i] = b[2*i+1] * c[2*i+1] - b[2*i] * c[2*i];
@ -38,5 +39,6 @@ int main (void)
return main1 ();
}
/* { dg-final { scan-tree-dump-times "vectorized 0 loops" 1 "vect" } } */
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target vect_extract_even_odd } } } */
/* { dg-final { scan-tree-dump-times "vectorized 0 loops" 1 "vect" { xfail vect_extract_even_odd } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

5
gcc/testsuite/gcc.dg/vect/vect-98.c
View file

@ -36,6 +36,7 @@ int main (void)
return main1 (ia);
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { xfail *-*-* } } } */
/* { dg-final { scan-tree-dump-times "not vectorized: complicated access pattern" 1 "vect" } } */
/* Needs interleaving support. */
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 0 "vect" { xfail { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

76
gcc/testsuite/gcc.dg/vect/vect-strided-a-mult.c Normal file
View file

@ -0,0 +1,76 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned short a;
unsigned short b;
} s;
typedef struct {
unsigned int a;
unsigned int b;
} ii;
int
main1 ()
{
s arr[N];
s *ptr = arr;
ii iarr[N];
ii *iptr = iarr;
s res[N];
ii ires[N];
int i;
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
iarr[i].a = i;
iarr[i].b = i * 3;
if (arr[i].a == 178)
abort();
}
for (i = 0; i < N; i++)
{
ires[i].a = iptr->b - iptr->a;
ires[i].b = iptr->b + iptr->a;
res[i].b = ptr->b - ptr->a;
res[i].a = ptr->b + ptr->a;
iptr++;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].b != arr[i].b - arr[i].a
|| ires[i].a != iarr[i].b - iarr[i].a
|| res[i].a != arr[i].b + arr[i].a
|| ires[i].b != iarr[i].b + iarr[i].a
)
abort ();
}
return 0;
}
int main (void)
{
int i;
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

60
gcc/testsuite/gcc.dg/vect/vect-strided-a-u16-i2.c Normal file
View file

@ -0,0 +1,60 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned short a;
unsigned short b;
} s;
int
main1 ()
{
s arr[N];
s *ptr = arr;
s res[N];
int i;
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
if (arr[i].a == 178)
abort();
}
for (i = 0; i < N; i++)
{
res[i].a = ptr->b - ptr->a;
res[i].b = ptr->b + ptr->a;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].a != arr[i].b - arr[i].a
|| res[i].b != arr[i].a + arr[i].b)
abort ();
}
return 0;
}
int main (void)
{
int i;
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

73
gcc/testsuite/gcc.dg/vect/vect-strided-a-u16-i4.c Normal file
View file

@ -0,0 +1,73 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned short a;
unsigned short b;
unsigned short c;
unsigned short d;
} s;
int
main1 ()
{
s arr[N];
s *ptr = arr;
s res[N];
int i;
unsigned short x, y, z, w;
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
if (arr[i].a == 178)
abort();
}
for (i = 0; i < N; i++)
{
x = ptr->b - ptr->a;
y = ptr->d - ptr->c;
res[i].c = x + y;
z = ptr->a + ptr->c;
w = ptr->b + ptr->d;
res[i].a = z + w;
res[i].d = x + y;
res[i].b = x + y;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].a != arr[i].a + arr[i].c + arr[i].b + arr[i].d
|| res[i].d != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].b != arr[i].b - arr[i].a + arr[i].d - arr[i].c)
abort ();
}
return 0;
}
int main (void)
{
int i;
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

67
gcc/testsuite/gcc.dg/vect/vect-strided-a-u16-mult.c Normal file
View file

@ -0,0 +1,67 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned short a;
unsigned short b;
} s;
int
main1 ()
{
s arr[N];
s *ptr = arr;
unsigned int iarr[N];
unsigned int *iptr = iarr;
s res[N];
unsigned int ires[N];
int i;
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
iarr[i] = i * 3;
if (arr[i].a == 178)
abort();
}
for (i = 0; i < N; i++)
{
ires[i] = *iptr;
res[i].b = ptr->b - ptr->a;
res[i].a = ptr->b + ptr->a;
iptr++;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].b != arr[i].b - arr[i].a
|| ires[i] != iarr[i]
|| res[i].a != arr[i].b + arr[i].a)
abort ();
}
return 0;
}
int main (void)
{
int i;
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

67
gcc/testsuite/gcc.dg/vect/vect-strided-a-u32-mult.c Normal file
View file

@ -0,0 +1,67 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned int a;
unsigned int b;
} ii;
int
main1 ()
{
unsigned short arr[N];
unsigned short *ptr = arr;
ii iarr[N];
ii *iptr = iarr;
unsigned short res[N];
ii ires[N];
int i;
for (i = 0; i < N; i++)
{
arr[i] = i;
iarr[i].a = i;
iarr[i].b = i * 3;
if (arr[i] == 178)
abort();
}
for (i = 0; i < N; i++)
{
ires[i].a = iptr->b - iptr->a;
ires[i].b = iptr->b + iptr->a;
res[i] = *ptr;
iptr++;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i] != arr[i]
|| ires[i].a != iarr[i].b - iarr[i].a
|| ires[i].b != iarr[i].b + iarr[i].a)
abort ();
}
return 0;
}
int main (void)
{
int i;
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

74
gcc/testsuite/gcc.dg/vect/vect-strided-a-u8-i2-gap.c Normal file
View file

@ -0,0 +1,74 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 64
typedef struct {
unsigned char a;
unsigned char b;
} s;
int
main1 ()
{
s arr[N];
s *ptr = arr;
s res[N];
int i;
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
if (arr[i].a == 178)
abort();
}
for (i = 0; i < N; i++)
{
res[i].a = ptr->a;
res[i].b = ptr->a;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].a != arr[i].a
|| res[i].b != arr[i].a)
abort ();
}
ptr = arr;
/* Not vectorizable: gap in store. */
for (i = 0; i < N; i++)
{
res[i].a = ptr->b;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].a != arr[i].b)
abort ();
}
return 0;
}
int main (void)
{
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

82
gcc/testsuite/gcc.dg/vect/vect-strided-a-u8-i8-gap2.c Normal file
View file

@ -0,0 +1,82 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include <stdio.h>
#include "tree-vect.h"
#define N 16
typedef struct {
unsigned char a;
unsigned char b;
unsigned char c;
unsigned char d;
unsigned char e;
unsigned char f;
unsigned char g;
unsigned char h;
} s;
int
main1 ()
{
int i;
s arr[N];
s *ptr = arr;
s res[N];
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
arr[i].e = i + 5;
arr[i].f = i * 2 + 2;
arr[i].g = i - 3;
arr[i].h = 56;
if (arr[i].a == 178)
abort();
}
for (i = 0; i < N; i++)
{
res[i].c = ptr->a;
res[i].a = ptr->f + ptr->a;
res[i].d = ptr->f - ptr->a;
res[i].b = ptr->f;
res[i].f = ptr->a;
res[i].e = ptr->f - ptr->a;
res[i].h = ptr->f;
res[i].g = ptr->f - ptr->a;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].a
|| res[i].a != arr[i].f + arr[i].a
|| res[i].d != arr[i].f - arr[i].a
|| res[i].b != arr[i].f
|| res[i].f != arr[i].a
|| res[i].e != arr[i].f - arr[i].a
|| res[i].h != arr[i].f
|| res[i].g != arr[i].f - arr[i].a)
abort();
}
}
int main (void)
{
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

86
gcc/testsuite/gcc.dg/vect/vect-strided-a-u8-i8-gap7.c Normal file
View file

@ -0,0 +1,86 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 16
typedef struct {
unsigned char a;
unsigned char b;
unsigned char c;
unsigned char d;
unsigned char e;
unsigned char f;
unsigned char g;
unsigned char h;
} s;
int
main1 ()
{
int i;
s arr[N];
s *ptr = arr;
s res[N];
unsigned char u, t, s, x, y, z, w;
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
arr[i].e = i * 3 + 5;
arr[i].f = i * 5;
arr[i].g = i - 3;
arr[i].h = 67;
if (arr[i].a == 178)
abort();
}
for (i = 0; i < N; i++)
{
u = ptr->b - ptr->a;
t = ptr->d - ptr->c;
res[i].c = u + t;
x = ptr->b + ptr->d;
res[i].a = ptr->a + x;
res[i].d = u + t;
s = ptr->h - ptr->a;
res[i].b = s + t;
res[i].f = ptr->f + ptr->h;
res[i].e = ptr->b + ptr->e;
res[i].h = ptr->d;
res[i].g = u + t;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].a != arr[i].a + arr[i].b + arr[i].d
|| res[i].d != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].b != arr[i].h - arr[i].a + arr[i].d - arr[i].c
|| res[i].f != arr[i].f + arr[i].h
|| res[i].e != arr[i].b + arr[i].e
|| res[i].h != arr[i].d
|| res[i].g != arr[i].b - arr[i].a + arr[i].d - arr[i].c)
abort();
}
}
int main (void)
{
check_vect ();
main1 ();
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

44
gcc/testsuite/gcc.dg/vect/vect-strided-float.c Normal file
View file

@ -0,0 +1,44 @@
/* { dg-require-effective-target vect_float } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 16
int
main1 (void)
{
int i;
float a[N*2];
float b[N*2] = {0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57,60,63,66,69,72,75,78,81,84,87,90,93};
float c[N*2] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31};
/* Strided access pattern. */
for (i = 0; i < N/2; i++)
{
a[i*2] = b[2*i+1] * c[2*i+1] - b[2*i] * c[2*i];
a[i*2+1] = b[2*i+8] * c[2*i+9] + b[2*i+9] * c[2*i+8];
}
/* Check results. */
for (i = 0; i < N/2; i++)
{
if (a[i*2] != b[2*i+1] * c[2*i+1] - b[2*i] * c[2*i]
|| a[i*2+1] != b[2*i+8] * c[2*i+9] + b[2*i+9] * c[2*i+8])
abort();
}
return 0;
}
int main (void)
{
check_vect ();
return main1 ();
}
/* Needs interleaving support. */
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { scan-tree-dump-times "vectorized 0 loops" 1 "vect" { xfail { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

76
gcc/testsuite/gcc.dg/vect/vect-strided-mult-char-ls.c Normal file
View file

@ -0,0 +1,76 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 32
typedef struct {
unsigned char a;
unsigned char b;
} s;
typedef struct {
unsigned int a;
unsigned int b;
} ii;
int
main1 (s *arr, ii *iarr)
{
s *ptr = arr;
ii *iptr = iarr;
s res[N];
ii ires[N];
int i;
for (i = 0; i < N; i++)
{
ires[i].a = iptr->b;
ires[i].b = iptr->a;
res[i].b = ptr->b - ptr->a;
res[i].a = ptr->b + ptr->a;
iptr++;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].b != arr[i].b - arr[i].a
|| ires[i].a != iarr[i].b
|| res[i].a != arr[i].b + arr[i].a
|| ires[i].b != iarr[i].a
)
abort ();
}
return 0;
}
int main (void)
{
int i;
s arr[N];
ii iarr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
iarr[i].a = i;
iarr[i].b = i * 3;
if (arr[i].a == 178)
abort();
}
main1 (arr, iarr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

76
gcc/testsuite/gcc.dg/vect/vect-strided-mult.c Normal file
View file

@ -0,0 +1,76 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned short a;
unsigned short b;
} s;
typedef struct {
unsigned int a;
unsigned int b;
} ii;
int
main1 (s *arr, ii *iarr)
{
s *ptr = arr;
ii *iptr = iarr;
s res[N];
ii ires[N];
int i;
for (i = 0; i < N; i++)
{
ires[i].a = iptr->b - iptr->a;
ires[i].b = iptr->b + iptr->a;
res[i].b = ptr->b - ptr->a;
res[i].a = ptr->b + ptr->a;
iptr++;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].b != arr[i].b - arr[i].a
|| ires[i].a != iarr[i].b - iarr[i].a
|| res[i].a != arr[i].b + arr[i].a
|| ires[i].b != iarr[i].b + iarr[i].a
)
abort ();
}
return 0;
}
int main (void)
{
int i;
s arr[N];
ii iarr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
iarr[i].a = i;
iarr[i].b = i * 3;
if (arr[i].a == 178)
abort();
}
main1 (arr, iarr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

60
gcc/testsuite/gcc.dg/vect/vect-strided-u16-i2.c Normal file
View file

@ -0,0 +1,60 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned short a;
unsigned short b;
} s;
int
main1 (s *arr)
{
s *ptr = arr;
s res[N];
int i;
for (i = 0; i < N; i++)
{
res[i].a = ptr->b - ptr->a;
res[i].b = ptr->b + ptr->a;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].a != arr[i].b - arr[i].a
|| res[i].b != arr[i].a + arr[i].b)
abort ();
}
return 0;
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

73
gcc/testsuite/gcc.dg/vect/vect-strided-u16-i4.c Normal file
View file

@ -0,0 +1,73 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned short a;
unsigned short b;
unsigned short c;
unsigned short d;
} s;
int
main1 (s *arr)
{
int i;
s *ptr = arr;
s res[N];
unsigned short x, y, z, w;
for (i = 0; i < N; i++)
{
x = ptr->b - ptr->a;
y = ptr->d - ptr->c;
res[i].c = x + y;
z = ptr->a + ptr->c;
w = ptr->b + ptr->d;
res[i].a = z + w;
res[i].d = x + y;
res[i].b = x + y;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].a != arr[i].a + arr[i].c + arr[i].b + arr[i].d
|| res[i].d != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].b != arr[i].b - arr[i].a + arr[i].d - arr[i].c)
abort ();
}
return 0;
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

68
gcc/testsuite/gcc.dg/vect/vect-strided-u32-i4.c Normal file
View file

@ -0,0 +1,68 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
int a;
int b;
int c;
int d;
} s;
int
main1 (s *arr)
{
int i;
s *ptr = arr;
s res[N];
for (i = 0; i < N; i++)
{
res[i].c = ptr->b - ptr->a + ptr->d - ptr->c;
res[i].a = ptr->a + ptr->c + ptr->b + ptr->d;
res[i].d = ptr->b - ptr->a + ptr->d - ptr->c;
res[i].b = ptr->b - ptr->a + ptr->d - ptr->c;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].a != arr[i].a + arr[i].c + arr[i].b + arr[i].d
|| res[i].d != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].b != arr[i].b - arr[i].a + arr[i].d - arr[i].c)
abort ();
}
return 0;
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

82
gcc/testsuite/gcc.dg/vect/vect-strided-u32-i8.c Normal file
View file

@ -0,0 +1,82 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
int a;
int b;
int c;
int d;
int e;
int f;
int g;
int h;
} s;
int
main1 (s *arr)
{
int i;
s *ptr = arr;
s res[N];
for (i = 0; i < N; i++)
{
res[i].c = ptr->b - ptr->a + ptr->d - ptr->c;
res[i].a = ptr->a + ptr->g + ptr->b + ptr->d;
res[i].d = ptr->b - ptr->a + ptr->d - ptr->c;
res[i].b = ptr->h - ptr->a + ptr->d - ptr->c;
res[i].f = ptr->f + ptr->h;
res[i].e = ptr->b - ptr->e;
res[i].h = ptr->d - ptr->g;
res[i].g = ptr->b - ptr->a + ptr->d - ptr->c;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].a != arr[i].a + arr[i].g + arr[i].b + arr[i].d
|| res[i].d != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].b != arr[i].h - arr[i].a + arr[i].d - arr[i].c
|| res[i].f != arr[i].f + arr[i].h
|| res[i].e != arr[i].b - arr[i].e
|| res[i].h != arr[i].d - arr[i].g
|| res[i].g != arr[i].b - arr[i].a + arr[i].d - arr[i].c)
abort();
}
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
arr[i].e = i * 3 + 5;
arr[i].f = i * 5;
arr[i].g = i - 3;
arr[i].h = 56;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

66
gcc/testsuite/gcc.dg/vect/vect-strided-u32-mult.c Normal file
View file

@ -0,0 +1,66 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 128
typedef struct {
unsigned int a;
unsigned int b;
} ii;
int
main1 (unsigned short *arr, ii *iarr)
{
unsigned short *ptr = arr;
ii *iptr = iarr;
unsigned short res[N];
ii ires[N];
int i;
for (i = 0; i < N; i++)
{
ires[i].a = iptr->b - iptr->a;
ires[i].b = iptr->b + iptr->a;
res[i] = *ptr;
iptr++;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i] != arr[i]
|| ires[i].a != iarr[i].b - iarr[i].a
|| ires[i].b != iarr[i].b + iarr[i].a)
abort ();
}
return 0;
}
int main (void)
{
int i;
unsigned short arr[N];
ii iarr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i] = i;
iarr[i].a = i;
iarr[i].b = i * 3;
if (arr[i] == 178)
abort();
}
main1 (arr, iarr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

76
gcc/testsuite/gcc.dg/vect/vect-strided-u8-i2-gap.c Normal file
View file

@ -0,0 +1,76 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 64
typedef struct {
unsigned char a;
unsigned char b;
} s;
int
main1 (s *arr)
{
s *ptr = arr;
s res[N];
int i;
for (i = 0; i < N; i++)
{
res[i].a = ptr->b;
res[i].b = ptr->b;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].a != arr[i].b
|| res[i].b != arr[i].b)
abort ();
}
ptr = arr;
/* Not vectorizable: gap in store. */
for (i = 0; i < N; i++)
{
res[i].a = ptr->b;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].a != arr[i].b)
abort ();
}
return 0;
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

59
gcc/testsuite/gcc.dg/vect/vect-strided-u8-i2.c Normal file
View file

@ -0,0 +1,59 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 64
typedef struct {
unsigned char a;
unsigned char b;
} s;
int
main1 (s *arr)
{
s *ptr = arr;
s res[N];
int i;
for (i = 0; i < N; i++)
{
res[i].a = ptr->b - ptr->a;
res[i].b = ptr->b + ptr->a;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].a != arr[i].b - arr[i].a
|| res[i].b != arr[i].a + arr[i].b)
abort ();
}
return 0;
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

84
gcc/testsuite/gcc.dg/vect/vect-strided-u8-i8-gap2.c Normal file
View file

@ -0,0 +1,84 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include <stdio.h>
#include "tree-vect.h"
#define N 16
typedef struct {
unsigned char a;
unsigned char b;
unsigned char c;
unsigned char d;
unsigned char e;
unsigned char f;
unsigned char g;
unsigned char h;
} s;
int
main1 (s *arr)
{
int i;
s *ptr = arr;
s res[N];
for (i = 0; i < N; i++)
{
res[i].c = ptr->b;
res[i].a = ptr->f + ptr->b;
res[i].d = ptr->f - ptr->b;
res[i].b = ptr->f;
res[i].f = ptr->b;
res[i].e = ptr->f - ptr->b;
res[i].h = ptr->f;
res[i].g = ptr->f - ptr->b;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b
|| res[i].a != arr[i].f + arr[i].b
|| res[i].d != arr[i].f - arr[i].b
|| res[i].b != arr[i].f
|| res[i].f != arr[i].b
|| res[i].e != arr[i].f - arr[i].b
|| res[i].h != arr[i].f
|| res[i].g != arr[i].f - arr[i].b)
abort();
}
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
arr[i].e = i + 5;
arr[i].f = i * 2 + 2;
arr[i].g = i - 3;
arr[i].h = 56;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

85
gcc/testsuite/gcc.dg/vect/vect-strided-u8-i8-gap4.c Normal file
View file

@ -0,0 +1,85 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 16
typedef struct {
unsigned char a;
unsigned char b;
unsigned char c;
unsigned char d;
unsigned char e;
unsigned char f;
unsigned char g;
unsigned char h;
} s;
int
main1 (s *arr)
{
int i;
s *ptr = arr;
s res[N];
unsigned char x;
for (i = 0; i < N; i++)
{
res[i].c = ptr->b + ptr->c;
x = ptr->c + ptr->f;
res[i].a = x + ptr->b;
res[i].d = ptr->b + ptr->c;
res[i].b = ptr->c;
res[i].f = ptr->f + ptr->e;
res[i].e = ptr->b + ptr->e;
res[i].h = ptr->c;
res[i].g = ptr->b + ptr->c;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b + arr[i].c
|| res[i].a != arr[i].c + arr[i].f + arr[i].b
|| res[i].d != arr[i].b + arr[i].c
|| res[i].b != arr[i].c
|| res[i].f != arr[i].f + arr[i].e
|| res[i].e != arr[i].b + arr[i].e
|| res[i].h != arr[i].c
|| res[i].g != arr[i].b + arr[i].c)
abort();
}
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
arr[i].e = i * 3 + 5;
arr[i].f = i * 5;
arr[i].g = i - 3;
arr[i].h = 56;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

88
gcc/testsuite/gcc.dg/vect/vect-strided-u8-i8-gap7.c Normal file
View file

@ -0,0 +1,88 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include "tree-vect.h"
#define N 16
typedef struct {
unsigned char a;
unsigned char b;
unsigned char c;
unsigned char d;
unsigned char e;
unsigned char f;
unsigned char g;
unsigned char h;
} s;
int
main1 (s *arr)
{
int i;
s *ptr = arr;
s res[N];
unsigned char u, t, s, x, y, z, w;
for (i = 0; i < N; i++)
{
u = ptr->b - ptr->a;
t = ptr->d - ptr->c;
res[i].c = u + t;
x = ptr->b + ptr->d;
res[i].a = ptr->a + x;
res[i].d = u + t;
s = ptr->h - ptr->a;
res[i].b = s + t;
res[i].f = ptr->f + ptr->h;
res[i].e = ptr->b + ptr->e;
res[i].h = ptr->d;
res[i].g = u + t;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].a != arr[i].a + arr[i].b + arr[i].d
|| res[i].d != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].b != arr[i].h - arr[i].a + arr[i].d - arr[i].c
|| res[i].f != arr[i].f + arr[i].h
|| res[i].e != arr[i].b + arr[i].e
|| res[i].h != arr[i].d
|| res[i].g != arr[i].b - arr[i].a + arr[i].d - arr[i].c)
abort();
}
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
arr[i].e = i * 3 + 5;
arr[i].f = i * 5;
arr[i].g = i - 3;
arr[i].h = 67;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

91
gcc/testsuite/gcc.dg/vect/vect-strided-u8-i8.c Normal file
View file

@ -0,0 +1,91 @@
/* { dg-require-effective-target vect_int } */
#include <stdarg.h>
#include <stdio.h>
#include "tree-vect.h"
#define N 32
typedef struct {
unsigned char a;
unsigned char b;
unsigned char c;
unsigned char d;
unsigned char e;
unsigned char f;
unsigned char g;
unsigned char h;
} s;
int
main1 (s *arr)
{
int i;
s *ptr = arr;
s res[N];
unsigned char u, t, s, x, y, z, w;
for (i = 0; i < N; i++)
{
u = ptr->b - ptr->a;
t = ptr->d - ptr->c;
res[i].c = u + t;
s = ptr->a + ptr->g;
x = ptr->b + ptr->d;
res[i].a = s + x;
res[i].d = u + t;
s = ptr->h - ptr->a;
x = ptr->d - ptr->c;
res[i].b = s + x;
res[i].f = ptr->f + ptr->h;
res[i].e = ptr->b + ptr->e;
res[i].h = ptr->d - ptr->g;
res[i].g = u + t;
ptr++;
}
/* check results: */
for (i = 0; i < N; i++)
{
if (res[i].c != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].a != arr[i].a + arr[i].g + arr[i].b + arr[i].d
|| res[i].d != arr[i].b - arr[i].a + arr[i].d - arr[i].c
|| res[i].b != arr[i].h - arr[i].a + arr[i].d - arr[i].c
|| res[i].f != arr[i].f + arr[i].h
|| res[i].e != arr[i].b + arr[i].e
|| res[i].h != arr[i].d - arr[i].g
|| res[i].g != arr[i].b - arr[i].a + arr[i].d - arr[i].c
)
abort();
}
}
int main (void)
{
int i;
s arr[N];
check_vect ();
for (i = 0; i < N; i++)
{
arr[i].a = i;
arr[i].b = i * 2;
arr[i].c = 17;
arr[i].d = i+34;
arr[i].e = i;
arr[i].f = i + 5;
arr[i].g = i + 3;
arr[i].h = 67;
if (arr[i].a == 178)
abort();
}
main1 (arr);
return 0;
}
/* { dg-final { scan-tree-dump-times "vectorized 1 loops" 1 "vect" { target { vect_interleave && vect_extract_even_odd } } } } */
/* { dg-final { cleanup-tree-dump "vect" } } */

38
gcc/testsuite/lib/target-supports.exp
View file

@ -1841,6 +1841,44 @@ proc check_effective_target_vect_int_mult { } {
return $et_vect_int_mult_saved
}
# Return 1 if the target supports vector even/odd elements extraction, 0 otherwise.
proc check_effective_target_vect_extract_even_odd { } {
global et_vect_extract_even_odd_saved
if [info exists et_vect_extract_even_odd_saved] {
verbose "check_effective_target_vect_extract_even_odd: using cached result" 2
} else {
set et_vect_extract_even_odd_saved 0
if { [istarget powerpc*-*-*] } {
set et_vect_extract_even_odd_saved 1
}
}
verbose "check_effective_target_vect_extract_even_odd: returning $et_vect_extract_even_odd_saved" 2
return $et_vect_extract_even_odd_saved
}
# Return 1 if the target supports vector interleaving, 0 otherwise.
proc check_effective_target_vect_interleave { } {
global et_vect_interleave_saved
if [info exists et_vect_interleave_saved] {
verbose "check_effective_target_vect_interleave: using cached result" 2
} else {
set et_vect_interleave_saved 0
if { [istarget powerpc*-*-*]
|| [istarget i?86-*-*]
|| [istarget x86_64-*-*] } {
set et_vect_interleave_saved 1
}
}
verbose "check_effective_target_vect_interleave: returning $et_vect_interleave_saved" 2
return $et_vect_interleave_saved
}
# Return 1 if the target supports section-anchors
proc check_effective_target_section_anchors { } {

56
gcc/tree-data-ref.h
View file

@ -24,25 +24,27 @@ Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
#include "lambda.h"
/** {base_address + offset + init} is the first location accessed by data-ref
in the loop, and step is the stride of data-ref in the loop in bytes;
e.g.:
/*
The first location accessed by data-ref in the loop is the address of data-ref's
base (BASE_ADDRESS) plus the initial offset from the base. We divide the initial offset
into two parts: loop invariant offset (OFFSET) and constant offset (INIT).
STEP is the stride of data-ref in the loop in bytes.
Example 1 Example 2
data-ref a[j].b[i][j] a + x + 16B (a is int*)
First location info:
First location info:
base_address &a a
offset j_0*D_j + i_0*D_i + C_a x
init C_b 16
offset j_0*D_j + i_0*D_i x
init C_b + C_a 16
step D_j 4
access_fn NULL {16, +, 1}
Base object info:
Base object info:
base_object a NULL
access_fn <access_fns of indexes of b> NULL
**/
*/
struct first_location_in_loop
{
tree base_address;
@ -51,7 +53,6 @@ struct first_location_in_loop
tree step;
/* Access function related to first location in the loop. */
VEC(tree,heap) *access_fns;
};
struct base_object_info
@ -97,10 +98,39 @@ struct data_reference
struct ptr_info_def *ptr_info;
subvar_t subvars;
/* Alignment information. */
/* The offset of the data-reference from its base in bytes. */
/* Alignment information.
MISALIGNMENT is the offset of the data-reference from its base in bytes.
ALIGNED_TO is the maximum data-ref's alignment.
Example 1,
for i
for (j = 3; j < N; j++)
a[j].b[i][j] = 0;
For a[j].b[i][j], the offset from base (calculated in get_inner_reference()
will be 'i * C_i + j * C_j + C'.
We try to substitute the variables of the offset expression
with initial_condition of the corresponding access_fn in the loop.
'i' cannot be substituted, since its access_fn in the inner loop is i. 'j'
will be substituted with 3.
Example 2
for (j = 3; j < N; j++)
a[j].b[5][j] = 0;
Here the offset expression (j * C_j + C) will not contain variables after
subsitution of j=3 (3*C_j + C).
Misalignment can be calculated only if all the variables can be
substituted with constants, otherwise, we record maximum possible alignment
in ALIGNED_TO. In Example 1, since 'i' cannot be substituted,
MISALIGNMENT will be NULL_TREE, and the biggest divider of C_i (a power of
2) will be recorded in ALIGNED_TO.
In Example 2, MISALIGNMENT will be the value of 3*C_j + C in bytes, and
ALIGNED_TO will be NULL_TREE.
*/
tree misalignment;
/* The maximum data-ref's alignment. */
tree aligned_to;
/* The type of the data-ref. */

5
gcc/tree-inline.c
View file

@ -1771,6 +1771,11 @@ estimate_num_insns_1 (tree *tp, int *walk_subtrees, void *data)
case WIDEN_MULT_EXPR:
case VEC_EXTRACT_EVEN_EXPR:
case VEC_EXTRACT_ODD_EXPR:
case VEC_INTERLEAVE_HIGH_EXPR:
case VEC_INTERLEAVE_LOW_EXPR:
case RESX_EXPR:
*count += 1;
break;

32
gcc/tree-pretty-print.c
View file

@ -1957,6 +1957,38 @@ dump_generic_node (pretty_printer *buffer, tree node, int spc, int flags,
}
break;
case VEC_EXTRACT_EVEN_EXPR:
pp_string (buffer, " VEC_EXTRACT_EVEN_EXPR < ");
dump_generic_node (buffer, TREE_OPERAND (node, 0), spc, flags, false);
pp_string (buffer, ", ");
dump_generic_node (buffer, TREE_OPERAND (node, 1), spc, flags, false);
pp_string (buffer, " > ");
break;
case VEC_EXTRACT_ODD_EXPR:
pp_string (buffer, " VEC_EXTRACT_ODD_EXPR < ");
dump_generic_node (buffer, TREE_OPERAND (node, 0), spc, flags, false);
pp_string (buffer, ", ");
dump_generic_node (buffer, TREE_OPERAND (node, 1), spc, flags, false);
pp_string (buffer, " > ");
break;
case VEC_INTERLEAVE_HIGH_EXPR:
pp_string (buffer, " VEC_INTERLEAVE_HIGH_EXPR < ");
dump_generic_node (buffer, TREE_OPERAND (node, 0), spc, flags, false);
pp_string (buffer, ", ");
dump_generic_node (buffer, TREE_OPERAND (node, 1), spc, flags, false);
pp_string (buffer, " > ");
break;
case VEC_INTERLEAVE_LOW_EXPR:
pp_string (buffer, " VEC_INTERLEAVE_LOW_EXPR < ");
dump_generic_node (buffer, TREE_OPERAND (node, 0), spc, flags, false);
pp_string (buffer, ", ");
dump_generic_node (buffer, TREE_OPERAND (node, 1), spc, flags, false);
pp_string (buffer, " > ");
break;
default:
NIY;
}

609
gcc/tree-vect-analyze.c
View file

@ -38,6 +38,7 @@ Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
#include "tree-data-ref.h"
#include "tree-scalar-evolution.h"
#include "tree-vectorizer.h"
#include "toplev.h"
/* Main analysis functions. */
static loop_vec_info vect_analyze_loop_form (struct loop *);
@ -56,13 +57,12 @@ static bool vect_determine_vectorization_factor (loop_vec_info);
static bool exist_non_indexing_operands_for_use_p (tree, tree);
static tree vect_get_loop_niters (struct loop *, tree *);
static bool vect_analyze_data_ref_dependence
(struct data_dependence_relation *, loop_vec_info);
(struct data_dependence_relation *, loop_vec_info, bool);
static bool vect_compute_data_ref_alignment (struct data_reference *);
static bool vect_analyze_data_ref_access (struct data_reference *);
static bool vect_can_advance_ivs_p (loop_vec_info);
static void vect_update_misalignment_for_peel
(struct data_reference *, struct data_reference *, int npeel);
/* Function vect_determine_vectorization_factor
@ -185,9 +185,10 @@ vect_determine_vectorization_factor (loop_vec_info loop_vinfo)
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "nunits = %d", nunits);
if (!vectorization_factor
|| (nunits > vectorization_factor))
vectorization_factor = nunits;
if (!vectorization_factor
|| (nunits > vectorization_factor))
vectorization_factor = nunits;
}
}
@ -559,6 +560,295 @@ vect_analyze_scalar_cycles (loop_vec_info loop_vinfo)
}
/* Function vect_insert_into_interleaving_chain.
Insert DRA into the interleaving chain of DRB according to DRA's INIT. */
static void
vect_insert_into_interleaving_chain (struct data_reference *dra,
struct data_reference *drb)
{
tree prev, next, next_init;
stmt_vec_info stmtinfo_a = vinfo_for_stmt (DR_STMT (dra));
stmt_vec_info stmtinfo_b = vinfo_for_stmt (DR_STMT (drb));
prev = DR_GROUP_FIRST_DR (stmtinfo_b);
next = DR_GROUP_NEXT_DR (vinfo_for_stmt (prev));
while (next)
{
next_init = DR_INIT (STMT_VINFO_DATA_REF (vinfo_for_stmt (next)));
if (tree_int_cst_compare (next_init, DR_INIT (dra)) > 0)
{
/* Insert here. */
DR_GROUP_NEXT_DR (vinfo_for_stmt (prev)) = DR_STMT (dra);
DR_GROUP_NEXT_DR (stmtinfo_a) = next;
return;
}
prev = next;
next = DR_GROUP_NEXT_DR (vinfo_for_stmt (prev));
}
/* We got to the end of the list. Insert here. */
DR_GROUP_NEXT_DR (vinfo_for_stmt (prev)) = DR_STMT (dra);
DR_GROUP_NEXT_DR (stmtinfo_a) = NULL_TREE;
}
/* Function vect_update_interleaving_chain.
For two data-refs DRA and DRB that are a part of a chain interleaved data
accesses, update the interleaving chain. DRB's INIT is smaller than DRA's.
There are four possible cases:
1. New stmts - both DRA and DRB are not a part of any chain:
FIRST_DR = DRB
NEXT_DR (DRB) = DRA
2. DRB is a part of a chain and DRA is not:
no need to update FIRST_DR
no need to insert DRB
insert DRA according to init
3. DRA is a part of a chain and DRB is not:
if (init of FIRST_DR > init of DRB)
FIRST_DR = DRB
NEXT(FIRST_DR) = previous FIRST_DR
else
insert DRB according to its init
4. both DRA and DRB are in some interleaving chains:
choose the chain with the smallest init of FIRST_DR
insert the nodes of the second chain into the first one. */
static void
vect_update_interleaving_chain (struct data_reference *drb,
struct data_reference *dra)
{
stmt_vec_info stmtinfo_a = vinfo_for_stmt (DR_STMT (dra));
stmt_vec_info stmtinfo_b = vinfo_for_stmt (DR_STMT (drb));
tree next_init, init_dra_chain, init_drb_chain, first_a, first_b;
tree node, prev, next, node_init, first_stmt;
/* 1. New stmts - both DRA and DRB are not a part of any chain. */
if (!DR_GROUP_FIRST_DR (stmtinfo_a) && !DR_GROUP_FIRST_DR (stmtinfo_b))
{
DR_GROUP_FIRST_DR (stmtinfo_a) = DR_STMT (drb);
DR_GROUP_FIRST_DR (stmtinfo_b) = DR_STMT (drb);
DR_GROUP_NEXT_DR (stmtinfo_b) = DR_STMT (dra);
return;
}
/* 2. DRB is a part of a chain and DRA is not. */
if (!DR_GROUP_FIRST_DR (stmtinfo_a) && DR_GROUP_FIRST_DR (stmtinfo_b))
{
DR_GROUP_FIRST_DR (stmtinfo_a) = DR_GROUP_FIRST_DR (stmtinfo_b);
/* Insert DRA into the chain of DRB. */
vect_insert_into_interleaving_chain (dra, drb);
return;
}
/* 3. DRA is a part of a chain and DRB is not. */
if (DR_GROUP_FIRST_DR (stmtinfo_a) && !DR_GROUP_FIRST_DR (stmtinfo_b))
{
tree old_first_stmt = DR_GROUP_FIRST_DR (stmtinfo_a);
tree init_old = DR_INIT (STMT_VINFO_DATA_REF (vinfo_for_stmt (
old_first_stmt)));
tree tmp;
if (tree_int_cst_compare (init_old, DR_INIT (drb)) > 0)
{
/* DRB's init is smaller than the init of the stmt previously marked
as the first stmt of the interleaving chain of DRA. Therefore, we
update FIRST_STMT and put DRB in the head of the list. */
DR_GROUP_FIRST_DR (stmtinfo_b) = DR_STMT (drb);
DR_GROUP_NEXT_DR (stmtinfo_b) = old_first_stmt;
/* Update all the stmts in the list to point to the new FIRST_STMT. */
tmp = old_first_stmt;
while (tmp)
{
DR_GROUP_FIRST_DR (vinfo_for_stmt (tmp)) = DR_STMT (drb);
tmp = DR_GROUP_NEXT_DR (vinfo_for_stmt (tmp));
}
}
else
{
/* Insert DRB in the list of DRA. */
vect_insert_into_interleaving_chain (drb, dra);
DR_GROUP_FIRST_DR (stmtinfo_b) = DR_GROUP_FIRST_DR (stmtinfo_a);
}
return;
}
/* 4. both DRA and DRB are in some interleaving chains. */
first_a = DR_GROUP_FIRST_DR (stmtinfo_a);
first_b = DR_GROUP_FIRST_DR (stmtinfo_b);
if (first_a == first_b)
return;
init_dra_chain = DR_INIT (STMT_VINFO_DATA_REF (vinfo_for_stmt (first_a)));
init_drb_chain = DR_INIT (STMT_VINFO_DATA_REF (vinfo_for_stmt (first_b)));
if (tree_int_cst_compare (init_dra_chain, init_drb_chain) > 0)
{
/* Insert the nodes of DRA chain into the DRB chain.
After inserting a node, continue from this node of the DRB chain (don't
start from the beginning. */
node = DR_GROUP_FIRST_DR (stmtinfo_a);
prev = DR_GROUP_FIRST_DR (stmtinfo_b);
first_stmt = first_b;
}
else
{
/* Insert the nodes of DRB chain into the DRA chain.
After inserting a node, continue from this node of the DRA chain (don't
start from the beginning. */
node = DR_GROUP_FIRST_DR (stmtinfo_b);
prev = DR_GROUP_FIRST_DR (stmtinfo_a);
first_stmt = first_a;
}
while (node)
{
node_init = DR_INIT (STMT_VINFO_DATA_REF (vinfo_for_stmt (node)));
next = DR_GROUP_NEXT_DR (vinfo_for_stmt (prev));
while (next)
{
next_init = DR_INIT (STMT_VINFO_DATA_REF (vinfo_for_stmt (next)));
if (tree_int_cst_compare (next_init, node_init) > 0)
{
/* Insert here. */
DR_GROUP_NEXT_DR (vinfo_for_stmt (prev)) = node;
DR_GROUP_NEXT_DR (vinfo_for_stmt (node)) = next;
prev = node;
break;
}
prev = next;
next = DR_GROUP_NEXT_DR (vinfo_for_stmt (prev));
}
if (!next)
{
/* We got to the end of the list. Insert here. */
DR_GROUP_NEXT_DR (vinfo_for_stmt (prev)) = node;
DR_GROUP_NEXT_DR (vinfo_for_stmt (node)) = NULL_TREE;
prev = node;
}
DR_GROUP_FIRST_DR (vinfo_for_stmt (node)) = first_stmt;
node = DR_GROUP_NEXT_DR (vinfo_for_stmt (node));
}
}
/* Function vect_equal_offsets.
Check if OFFSET1 and OFFSET2 are identical expressions. */
static bool
vect_equal_offsets (tree offset1, tree offset2)
{
bool res0, res1;
STRIP_NOPS (offset1);
STRIP_NOPS (offset2);
if (offset1 == offset2)
return true;
if (TREE_CODE (offset1) != TREE_CODE (offset2)
|| !BINARY_CLASS_P (offset1)
|| !BINARY_CLASS_P (offset2))
return false;
res0 = vect_equal_offsets (TREE_OPERAND (offset1, 0),
TREE_OPERAND (offset2, 0));
res1 = vect_equal_offsets (TREE_OPERAND (offset1, 1),
TREE_OPERAND (offset2, 1));
return (res0 && res1);
}
/* Function vect_check_interleaving.
Check if DRA and DRB are a part of interleaving. In case they are, insert
DRA and DRB in an interleaving chain. */
static void
vect_check_interleaving (struct data_reference *dra,
struct data_reference *drb)
{
HOST_WIDE_INT type_size_a, type_size_b, diff_mod_size, step, init_a, init_b;
/* Check that the data-refs have same first location (except init) and they
are both either store or load (not load and store). */
if ((DR_BASE_ADDRESS (dra) != DR_BASE_ADDRESS (drb)
&& (TREE_CODE (DR_BASE_ADDRESS (dra)) != ADDR_EXPR
|| TREE_CODE (DR_BASE_ADDRESS (drb)) != ADDR_EXPR
|| TREE_OPERAND (DR_BASE_ADDRESS (dra), 0)
!= TREE_OPERAND (DR_BASE_ADDRESS (drb),0)))
|| !vect_equal_offsets (DR_OFFSET (dra), DR_OFFSET (drb))
|| !tree_int_cst_compare (DR_INIT (dra), DR_INIT (drb))
|| DR_IS_READ (dra) != DR_IS_READ (drb))
return;
/* Check:
1. data-refs are of the same type
2. their steps are equal
3. the step is greater than the difference between data-refs' inits */
type_size_a = TREE_INT_CST_LOW (TYPE_SIZE_UNIT (TREE_TYPE (DR_REF (dra))));
type_size_b = TREE_INT_CST_LOW (TYPE_SIZE_UNIT (TREE_TYPE (DR_REF (drb))));
if (type_size_a != type_size_b
|| tree_int_cst_compare (DR_STEP (dra), DR_STEP (drb)))
return;
init_a = TREE_INT_CST_LOW (DR_INIT (dra));
init_b = TREE_INT_CST_LOW (DR_INIT (drb));
step = TREE_INT_CST_LOW (DR_STEP (dra));
if (init_a > init_b)
{
/* If init_a == init_b + the size of the type * k, we have an interleaving,
and DRB is accessed before DRA. */
diff_mod_size = (init_a - init_b) % type_size_a;
if ((init_a - init_b) > step)
return;
if (diff_mod_size == 0)
{
vect_update_interleaving_chain (drb, dra);
if (vect_print_dump_info (REPORT_DR_DETAILS))
{
fprintf (vect_dump, "Detected interleaving ");
print_generic_expr (vect_dump, DR_REF (dra), TDF_SLIM);
fprintf (vect_dump, " and ");
print_generic_expr (vect_dump, DR_REF (drb), TDF_SLIM);
}
return;
}
}
else
{
/* If init_b == init_a + the size of the type * k, we have an
interleaving, and DRA is accessed before DRB. */
diff_mod_size = (init_b - init_a) % type_size_a;
if ((init_b - init_a) > step)
return;
if (diff_mod_size == 0)
{
vect_update_interleaving_chain (dra, drb);
if (vect_print_dump_info (REPORT_DR_DETAILS))
{
fprintf (vect_dump, "Detected interleaving ");
print_generic_expr (vect_dump, DR_REF (dra), TDF_SLIM);
fprintf (vect_dump, " and ");
print_generic_expr (vect_dump, DR_REF (drb), TDF_SLIM);
}
return;
}
}
}
/* Function vect_analyze_data_ref_dependence.
Return TRUE if there (might) exist a dependence between a memory-reference
@ -566,7 +856,8 @@ vect_analyze_scalar_cycles (loop_vec_info loop_vinfo)
static bool
vect_analyze_data_ref_dependence (struct data_dependence_relation *ddr,
loop_vec_info loop_vinfo)
loop_vec_info loop_vinfo,
bool check_interleaving)
{
unsigned int i;
struct loop *loop = LOOP_VINFO_LOOP (loop_vinfo);
@ -581,6 +872,14 @@ vect_analyze_data_ref_dependence (struct data_dependence_relation *ddr,
unsigned int loop_depth;
if (DDR_ARE_DEPENDENT (ddr) == chrec_known)
{
/* Independent data accesses. */
if (check_interleaving)
vect_check_interleaving (dra, drb);
return false;
}
if ((DR_IS_READ (dra) && DR_IS_READ (drb)) || dra == drb)
return false;
if (DDR_ARE_DEPENDENT (ddr) == chrec_dont_know)
@ -659,6 +958,36 @@ vect_analyze_data_ref_dependence (struct data_dependence_relation *ddr,
}
/* Function vect_check_dependences.
Return TRUE if there is a store-store or load-store dependence between
data-refs in DDR, otherwise return FALSE. */
static bool
vect_check_dependences (struct data_dependence_relation *ddr)
{
struct data_reference *dra = DDR_A (ddr);
struct data_reference *drb = DDR_B (ddr);
if (DDR_ARE_DEPENDENT (ddr) == chrec_known || dra == drb)
/* Independent or same data accesses. */
return false;
if (DR_IS_READ (dra) == DR_IS_READ (drb) && DR_IS_READ (dra))
/* Two loads. */
return false;
if (vect_print_dump_info (REPORT_DR_DETAILS))
{
fprintf (vect_dump, "possible store or store/load dependence between ");
print_generic_expr (vect_dump, DR_REF (dra), TDF_SLIM);
fprintf (vect_dump, " and ");
print_generic_expr (vect_dump, DR_REF (drb), TDF_SLIM);
}
return true;
}
/* Function vect_analyze_data_ref_dependences.
Examine all the data references in the loop, and make sure there do not
@ -670,12 +999,24 @@ vect_analyze_data_ref_dependences (loop_vec_info loop_vinfo)
unsigned int i;
VEC (ddr_p, heap) *ddrs = LOOP_VINFO_DDRS (loop_vinfo);
struct data_dependence_relation *ddr;
bool check_interleaving = true;
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "=== vect_analyze_dependences ===");
/* We allow interleaving only if there are no store-store and load-store
dependencies in the loop. */
for (i = 0; VEC_iterate (ddr_p, ddrs, i, ddr); i++)
if (vect_analyze_data_ref_dependence (ddr, loop_vinfo))
{
if (vect_check_dependences (ddr))
{
check_interleaving = false;
break;
}
}
for (i = 0; VEC_iterate (ddr_p, ddrs, i, ddr); i++)
if (vect_analyze_data_ref_dependence (ddr, loop_vinfo, check_interleaving))
return false;
return true;
@ -830,11 +1171,20 @@ vect_update_misalignment_for_peel (struct data_reference *dr,
struct data_reference *current_dr;
int dr_size = GET_MODE_SIZE (TYPE_MODE (TREE_TYPE (DR_REF (dr))));
int dr_peel_size = GET_MODE_SIZE (TYPE_MODE (TREE_TYPE (DR_REF (dr_peel))));
stmt_vec_info stmt_info = vinfo_for_stmt (DR_STMT (dr));
stmt_vec_info peel_stmt_info = vinfo_for_stmt (DR_STMT (dr_peel));
/* For interleaved data accesses the step in the loop must be multiplied by
the size of the interleaving group. */
if (DR_GROUP_FIRST_DR (stmt_info))
dr_size *= DR_GROUP_SIZE (vinfo_for_stmt (DR_GROUP_FIRST_DR (stmt_info)));
if (DR_GROUP_FIRST_DR (peel_stmt_info))
dr_peel_size *= DR_GROUP_SIZE (peel_stmt_info);
if (known_alignment_for_access_p (dr)
&& known_alignment_for_access_p (dr_peel)
&& (DR_MISALIGNMENT (dr)/dr_size ==
DR_MISALIGNMENT (dr_peel)/dr_peel_size))
&& (DR_MISALIGNMENT (dr) / dr_size ==
DR_MISALIGNMENT (dr_peel) / dr_peel_size))
{
DR_MISALIGNMENT (dr) = 0;
return;
@ -848,15 +1198,15 @@ vect_update_misalignment_for_peel (struct data_reference *dr,
{
if (current_dr != dr)
continue;
gcc_assert (DR_MISALIGNMENT (dr)/dr_size ==
DR_MISALIGNMENT (dr_peel)/dr_peel_size);
gcc_assert (DR_MISALIGNMENT (dr) / dr_size ==
DR_MISALIGNMENT (dr_peel) / dr_peel_size);
DR_MISALIGNMENT (dr) = 0;
return;
}
if (known_alignment_for_access_p (dr)
&& known_alignment_for_access_p (dr_peel))
{
{
DR_MISALIGNMENT (dr) += npeel * dr_size;
DR_MISALIGNMENT (dr) %= UNITS_PER_SIMD_WORD;
return;
@ -883,6 +1233,14 @@ vect_verify_datarefs_alignment (loop_vec_info loop_vinfo)
for (i = 0; VEC_iterate (data_reference_p, datarefs, i, dr); i++)
{
tree stmt = DR_STMT (dr);
stmt_vec_info stmt_info = vinfo_for_stmt (stmt);
/* For interleaving, only the alignment of the first access matters. */
if (DR_GROUP_FIRST_DR (stmt_info)
&& DR_GROUP_FIRST_DR (stmt_info) != stmt)
continue;
supportable_dr_alignment = vect_supportable_dr_alignment (dr);
if (!supportable_dr_alignment)
{
@ -1007,6 +1365,8 @@ vect_enhance_data_refs_alignment (loop_vec_info loop_vinfo)
bool do_peeling = false;
bool do_versioning = false;
bool stat;
tree stmt;
stmt_vec_info stmt_info;
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "=== vect_enhance_data_refs_alignment ===");
@ -1051,12 +1411,47 @@ vect_enhance_data_refs_alignment (loop_vec_info loop_vinfo)
TODO: Use a cost model. */
for (i = 0; VEC_iterate (data_reference_p, datarefs, i, dr); i++)
if (!DR_IS_READ (dr) && !aligned_access_p (dr))
{
dr0 = dr;
do_peeling = true;
break;
}
{
stmt = DR_STMT (dr);
stmt_info = vinfo_for_stmt (stmt);
/* For interleaving, only the alignment of the first access
matters. */
if (DR_GROUP_FIRST_DR (stmt_info)
&& DR_GROUP_FIRST_DR (stmt_info) != stmt)
continue;
if (!DR_IS_READ (dr) && !aligned_access_p (dr))
{
if (DR_GROUP_FIRST_DR (stmt_info))
{
/* For interleaved access we peel only if number of iterations in
the prolog loop ({VF - misalignment}), is a multiple of the
number of the interelaved accesses. */
int elem_size, mis_in_elements;
int vf = LOOP_VINFO_VECT_FACTOR (loop_vinfo);
/* FORNOW: handle only known alignment. */
if (!known_alignment_for_access_p (dr))
{
do_peeling = false;
break;
}
elem_size = UNITS_PER_SIMD_WORD / vf;
mis_in_elements = DR_MISALIGNMENT (dr) / elem_size;
if ((vf - mis_in_elements) % DR_GROUP_SIZE (stmt_info))
{
do_peeling = false;
break;
}
}
dr0 = dr;
do_peeling = true;
break;
}
}
/* Often peeling for alignment will require peeling for loop-bound, which in
turn requires that we know how to adjust the loop ivs after the loop. */
@ -1077,8 +1472,16 @@ vect_enhance_data_refs_alignment (loop_vec_info loop_vinfo)
mis = DR_MISALIGNMENT (dr0);
mis /= GET_MODE_SIZE (TYPE_MODE (TREE_TYPE (DR_REF (dr0))));
npeel = LOOP_VINFO_VECT_FACTOR (loop_vinfo) - mis;
/* For interleaved data access every iteration accesses all the
members of the group, therefore we divide the number of iterations
by the group size. */
stmt_info = vinfo_for_stmt (DR_STMT (dr0));
if (DR_GROUP_FIRST_DR (stmt_info))
npeel /= DR_GROUP_SIZE (stmt_info);
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "Try peeling by %d",npeel);
fprintf (vect_dump, "Try peeling by %d", npeel);
}
/* Ensure that all data refs can be vectorized after the peel. */
@ -1089,6 +1492,14 @@ vect_enhance_data_refs_alignment (loop_vec_info loop_vinfo)
if (dr == dr0)
continue;
stmt = DR_STMT (dr);
stmt_info = vinfo_for_stmt (stmt);
/* For interleaving, only the alignment of the first access
matters. */
if (DR_GROUP_FIRST_DR (stmt_info)
&& DR_GROUP_FIRST_DR (stmt_info) != stmt)
continue;
save_misalignment = DR_MISALIGNMENT (dr);
vect_update_misalignment_for_peel (dr, dr0, npeel);
supportable_dr_alignment = vect_supportable_dr_alignment (dr);
@ -1146,10 +1557,17 @@ vect_enhance_data_refs_alignment (loop_vec_info loop_vinfo)
{
for (i = 0; VEC_iterate (data_reference_p, datarefs, i, dr); i++)
{
if (aligned_access_p (dr))
continue;
stmt = DR_STMT (dr);
stmt_info = vinfo_for_stmt (stmt);
supportable_dr_alignment = vect_supportable_dr_alignment (dr);
/* For interleaving, only the alignment of the first access
matters. */
if (aligned_access_p (dr)
|| (DR_GROUP_FIRST_DR (stmt_info)
&& DR_GROUP_FIRST_DR (stmt_info) != stmt))
continue;
supportable_dr_alignment = vect_supportable_dr_alignment (dr);
if (!supportable_dr_alignment)
{
@ -1266,14 +1684,157 @@ static bool
vect_analyze_data_ref_access (struct data_reference *dr)
{
tree step = DR_STEP (dr);
HOST_WIDE_INT dr_step = TREE_INT_CST_LOW (step);
tree scalar_type = TREE_TYPE (DR_REF (dr));
HOST_WIDE_INT type_size = TREE_INT_CST_LOW (TYPE_SIZE_UNIT (scalar_type));
tree stmt = DR_STMT (dr);
/* For interleaving, STRIDE is STEP counted in elements, i.e., the size of the
interleaving group (including gaps). */
HOST_WIDE_INT stride = dr_step / type_size;
if (!step || tree_int_cst_compare (step, TYPE_SIZE_UNIT (scalar_type)))
if (!step)
{
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "bad data-ref access");
return false;
}
/* Consecutive? */
if (!tree_int_cst_compare (step, TYPE_SIZE_UNIT (scalar_type)))
{
/* Mark that it is not interleaving. */
DR_GROUP_FIRST_DR (vinfo_for_stmt (stmt)) = NULL_TREE;
return true;
}
/* Not consecutive access is possible only if it is a part of interleaving. */
if (!DR_GROUP_FIRST_DR (vinfo_for_stmt (stmt)))
{
/* Check if it this DR is a part of interleaving, and is a single
element of the group that is accessed in the loop. */
/* Gaps are supported only for loads. STEP must be a multiple of the type
size. The size of the group must be a power of 2. */
if (DR_IS_READ (dr)
&& (dr_step % type_size) == 0
&& stride > 0
&& exact_log2 (stride) != -1)
{
DR_GROUP_FIRST_DR (vinfo_for_stmt (stmt)) = stmt;
DR_GROUP_SIZE (vinfo_for_stmt (stmt)) = stride;
if (vect_print_dump_info (REPORT_DR_DETAILS))
{
fprintf (vect_dump, "Detected single element interleaving %d ",
DR_GROUP_SIZE (vinfo_for_stmt (stmt)));
print_generic_expr (vect_dump, DR_REF (dr), TDF_SLIM);
fprintf (vect_dump, " step ");
print_generic_expr (vect_dump, step, TDF_SLIM);
}
return true;
}
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "not consecutive access");
return false;
}
if (DR_GROUP_FIRST_DR (vinfo_for_stmt (stmt)) == stmt)
{
/* First stmt in the interleaving chain. Check the chain. */
tree next = DR_GROUP_NEXT_DR (vinfo_for_stmt (stmt));
struct data_reference *data_ref = dr;
unsigned int count = 1;
tree next_step;
tree prev_init = DR_INIT (data_ref);
tree prev = stmt;
HOST_WIDE_INT diff, count_in_bytes;
while (next)
{
/* Skip same data-refs. In case that two or more stmts share data-ref
(supported only for loads), we vectorize only the first stmt, and
the rest get their vectorized loads from the the first one. */
if (!tree_int_cst_compare (DR_INIT (data_ref),
DR_INIT (STMT_VINFO_DATA_REF (
vinfo_for_stmt (next)))))
{
/* For load use the same data-ref load. (We check in
vect_check_dependences() that there are no two stores to the
same location). */
DR_GROUP_SAME_DR_STMT (vinfo_for_stmt (next)) = prev;
prev = next;
next = DR_GROUP_NEXT_DR (vinfo_for_stmt (next));
continue;
}
prev = next;
/* Check that all the accesses have the same STEP. */
next_step = DR_STEP (STMT_VINFO_DATA_REF (vinfo_for_stmt (next)));
if (tree_int_cst_compare (step, next_step))
{
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "not consecutive access in interleaving");
return false;
}
data_ref = STMT_VINFO_DATA_REF (vinfo_for_stmt (next));
/* Check that the distance between two accesses is equal to the type
size. Otherwise, we have gaps. */
diff = (TREE_INT_CST_LOW (DR_INIT (data_ref))
- TREE_INT_CST_LOW (prev_init)) / type_size;
if (!DR_IS_READ (data_ref) && diff != 1)
{
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "interleaved store with gaps");
return false;
}
/* Store the gap from the previous member of the group. If there is no
gap in the access, DR_GROUP_GAP is always 1. */
DR_GROUP_GAP (vinfo_for_stmt (next)) = diff;
prev_init = DR_INIT (data_ref);
next = DR_GROUP_NEXT_DR (vinfo_for_stmt (next));
/* Count the number of data-refs in the chain. */
count++;
}
/* COUNT is the number of accesses found, we multiply it by the size of
the type to get COUNT_IN_BYTES. */
count_in_bytes = type_size * count;
/* Check the size of the interleaving is not greater than STEP. */
if (dr_step < count_in_bytes)
{
if (vect_print_dump_info (REPORT_DETAILS))
{
fprintf (vect_dump, "interleaving size is greater than step for ");
print_generic_expr (vect_dump, DR_REF (dr), TDF_SLIM);
}
return false;
}
/* Check that STEP is a multiple of type size. */
if ((dr_step % type_size) != 0)
{
if (vect_print_dump_info (REPORT_DETAILS))
{
fprintf (vect_dump, "step is not a multiple of type size: step ");
print_generic_expr (vect_dump, step, TDF_SLIM);
fprintf (vect_dump, " size ");
print_generic_expr (vect_dump, TYPE_SIZE_UNIT (scalar_type),
TDF_SLIM);
}
return false;
}
/* FORNOW: we handle only interleaving that is a power of 2. */
if (exact_log2 (stride) == -1)
{
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "interleaving is not a power of 2");
return false;
}
DR_GROUP_SIZE (vinfo_for_stmt (stmt)) = stride;
}
return true;
}
@ -1335,7 +1896,7 @@ vect_analyze_data_refs (loop_vec_info loop_vinfo)
if (vect_print_dump_info (REPORT_DETAILS))
fprintf (vect_dump, "=== vect_analyze_data_refs ===");
compute_data_dependences_for_loop (loop, false,
compute_data_dependences_for_loop (loop, true,
&LOOP_VINFO_DATAREFS (loop_vinfo),
&LOOP_VINFO_DDRS (loop_vinfo));

999
gcc/tree-vect-transform.c
View file

File diff suppressed because it is too large Load diff

6
gcc/tree-vectorizer.c
View file

@ -1370,6 +1370,12 @@ new_stmt_vec_info (tree stmt, loop_vec_info loop_vinfo)
else
STMT_VINFO_DEF_TYPE (res) = vect_loop_def;
STMT_VINFO_SAME_ALIGN_REFS (res) = VEC_alloc (dr_p, heap, 5);
DR_GROUP_FIRST_DR (res) = NULL_TREE;
DR_GROUP_NEXT_DR (res) = NULL_TREE;
DR_GROUP_SIZE (res) = 0;
DR_GROUP_STORE_COUNT (res) = 0;
DR_GROUP_GAP (res) = 0;
DR_GROUP_SAME_DR_STMT (res) = NULL_TREE;
return res;
}

53
gcc/tree-vectorizer.h
View file

@ -235,21 +235,50 @@ typedef struct _stmt_vec_info {
/* Classify the def of this stmt. */
enum vect_def_type def_type;
/* Interleaving info. */
/* First data-ref in the interleaving group. */
tree first_dr;
/* Pointer to the next data-ref in the group. */
tree next_dr;
/* The size of the interleaving group. */
unsigned int size;
/* For stores, number of stores from this group seen. We vectorize the last
one. */
unsigned int store_count;
/* For loads only, the gap from the previous load. For consecutive loads, GAP
is 1. */
unsigned int gap;
/* In case that two or more stmts share data-ref, this is the pointer to the
previously detected stmt with the same dr. */
tree same_dr_stmt;
} *stmt_vec_info;
/* Access Functions. */
#define STMT_VINFO_TYPE(S) (S)->type
#define STMT_VINFO_STMT(S) (S)->stmt
#define STMT_VINFO_LOOP_VINFO(S) (S)->loop_vinfo
#define STMT_VINFO_RELEVANT(S) (S)->relevant
#define STMT_VINFO_LIVE_P(S) (S)->live
#define STMT_VINFO_VECTYPE(S) (S)->vectype
#define STMT_VINFO_VEC_STMT(S) (S)->vectorized_stmt
#define STMT_VINFO_DATA_REF(S) (S)->data_ref_info
#define STMT_VINFO_IN_PATTERN_P(S) (S)->in_pattern_p
#define STMT_VINFO_RELATED_STMT(S) (S)->related_stmt
#define STMT_VINFO_SAME_ALIGN_REFS(S) (S)->same_align_refs
#define STMT_VINFO_DEF_TYPE(S) (S)->def_type
#define STMT_VINFO_TYPE(S) (S)->type
#define STMT_VINFO_STMT(S) (S)->stmt
#define STMT_VINFO_LOOP_VINFO(S) (S)->loop_vinfo
#define STMT_VINFO_RELEVANT(S) (S)->relevant
#define STMT_VINFO_LIVE_P(S) (S)->live
#define STMT_VINFO_VECTYPE(S) (S)->vectype
#define STMT_VINFO_VEC_STMT(S) (S)->vectorized_stmt
#define STMT_VINFO_DATA_REF(S) (S)->data_ref_info
#define STMT_VINFO_IN_PATTERN_P(S) (S)->in_pattern_p
#define STMT_VINFO_RELATED_STMT(S) (S)->related_stmt
#define STMT_VINFO_SAME_ALIGN_REFS(S) (S)->same_align_refs
#define STMT_VINFO_DEF_TYPE(S) (S)->def_type
#define STMT_VINFO_DR_GROUP_FIRST_DR(S) (S)->first_dr
#define STMT_VINFO_DR_GROUP_NEXT_DR(S) (S)->next_dr
#define STMT_VINFO_DR_GROUP_SIZE(S) (S)->size
#define STMT_VINFO_DR_GROUP_STORE_COUNT(S) (S)->store_count
#define STMT_VINFO_DR_GROUP_GAP(S) (S)->gap
#define STMT_VINFO_DR_GROUP_SAME_DR_STMT(S)(S)->same_dr_stmt
#define DR_GROUP_FIRST_DR(S) (S)->first_dr
#define DR_GROUP_NEXT_DR(S) (S)->next_dr
#define DR_GROUP_SIZE(S) (S)->size
#define DR_GROUP_STORE_COUNT(S) (S)->store_count
#define DR_GROUP_GAP(S) (S)->gap
#define DR_GROUP_SAME_DR_STMT(S) (S)->same_dr_stmt
#define STMT_VINFO_RELEVANT_P(S) ((S)->relevant != vect_unused_in_loop)

8
gcc/tree.def
View file

@ -1088,6 +1088,14 @@ DEFTREECODE (VEC_UNPACK_LO_EXPR, "vec_unpack_lo_expr", tcc_unary, 1)
DEFTREECODE (VEC_PACK_MOD_EXPR, "vec_pack_mod_expr", tcc_binary, 2)
DEFTREECODE (VEC_PACK_SAT_EXPR, "vec_pack_sat_expr", tcc_binary, 2)
/* Extract even/odd fields from vectors. */
DEFTREECODE (VEC_EXTRACT_EVEN_EXPR, "vec_extracteven_expr", tcc_binary, 2)
DEFTREECODE (VEC_EXTRACT_ODD_EXPR, "vec_extractodd_expr", tcc_binary, 2)
/* Merge input vectors interleaving their fields. */
DEFTREECODE (VEC_INTERLEAVE_HIGH_EXPR, "vec_interleavehigh_expr", tcc_binary, 2)
DEFTREECODE (VEC_INTERLEAVE_LOW_EXPR, "vec_interleavelow_expr", tcc_binary, 2)
/*
Local variables:
mode:c