Functions compiled when inlined (thus from inside the optimiser)
mustn't retain the lexical environment of the caller or there will be
tears. See discussion at
https://lists.gnu.org/archive/html/emacs-devel/2021-05/msg01227.html .
Bug found by Stefan Monnier.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand):
Bind byte-optimize--lexvars to nil when re-entering the compiler
recursively.
* test/lisp/emacs-lisp/bytecomp-resources/bc-test-alpha.el:
* test/lisp/emacs-lisp/bytecomp-resources/bc-test-beta.el: New files.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-defsubst): New test.
(byte-compile-inline-expand): When inlining code from another file,
always inline the byte-code version of the function.
(byte-optimize--pcase): Simplify edebug spec.
It's good practice to compare integers using 'eql' because two bignum
objects representing the same integer might not be 'eq'. However,
'eql' is slower and doesn't have its own byte code. Therefore,
replace it with 'eq' if one argument is guaranteed to be a fixnum on
all platforms.
* lisp/emacs-lisp/byte-opt.el (byte-optimize--fixnump): New helper
function.
(byte-optimize-equal, byte-optimize-member, byte-optimize-assoc): Use
it to optimize 'eql' etc. to 'eq' if it will always compare fixnums.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker): Don't
perform incorrect optimisations when a condition-case variable shadows
another lexical variable.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
New test case.
* lisp/emacs-lisp/byte-opt.el (pure-fns): Treat string>,
string-greaterp, string-empty-p, string-blank-p, string-prefix-p and
string-suffix-p as pure functions in the compiler.
* doc/lispref/functions.texi (Calling Functions): Document it.
* lisp/subr.el (always): New function.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns): Mark it as
side effect free.
There is no point in traversing conditional branches that are
statically known never to be executed. This saves some optimisation
effort, but more importantly prevents variable assignments and
references in those branches from blocking effective constant
propagation.
Also attempt to traverse as much as possible in an unconditional
context, which enables constant-propagation through (linear)
assignments.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form):
Rewrite the (tail) recursion into an explicit loop. Normalise a
return value of (quote nil) to nil, for easier subsequent
optimisations.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker): Don't
traverse dead `if` branches. Use unconditional traversion context
when possible.
This bug was introduced by the lexical variable constant propagation
mechanism. It was discovered by Michael Heerdegen.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-let-form)
(byte-optimize-body): Let the effects of a local defvar declaration be
scoped by let and let*, not any arbitrary Lisp expression body (such
as progn).
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--get-vars)
(bytecomp-local-defvar): New test.
The recent change in macroexp triggered a cyclic dependency error
during eager macroexpansion when neither `bytecomp` nor `byte-opt` had
been byte-compiled yet. This fixes it by moving the offending
function to macroexp.el.
* lisp/emacs-lisp/macroexp.el (macroexp--unfold-lambda): Move from
byte-opt.el and rename.
(macroexp--expand-all): Use it.
* lisp/emacs-lisp/byte-opt.el (byte-compile-unfold-lambda): Move to
macroexp.el.
(byte-compile-inline-expand, byte-optimize-form-code-walker):
* lisp/emacs-lisp/bytecomp.el (byte-compile-form):
Use `macroexp--unfold-lambda` instead.
* lisp/emacs-lisp/byte-opt.el (byte-optimize--lexvars)
(byte-optimize--vars-outside-condition)
(byte-optimize-form-code-walker, byte-optimize-let-form):
Clarify various aspects in the variable constant-propagation code,
as kindly pointed out by Stefan Monnier.
Lexical variables bound to a constant value (symbol, number or string)
are substituted at their point of use and the variable then eliminated
if possible. Example:
(let ((x (+ 2 3))) (f x)) => (f 5)
This reduces code size, eliminates stack operations, and enables
further optimisations. The implementation is conservative, and is
strongly curtailed by the presence of variable mutation, conditions
and loops.
* lisp/emacs-lisp/byte-opt.el
(byte-optimize-enable-variable-constprop)
(byte-optimize-warn-eliminated-variable): New constants.
(byte-optimize--lexvars, byte-optimize--vars-outside-condition)
(byte-optimize--vars-outside-loop, byte-optimize--dynamic-vars):
New dynamic variables.
(byte-optimize--substitutable-p, byte-optimize-let-form):
New functions.
(byte-optimize-form-code-walker): Adapt clauses for variable
constprop, and add clauses for 'setq' and 'defvar'.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-test-var)
(bytecomp-test-get-var, bytecomp-test-identity)
(byte-opt-testsuite-arith-data): Add test cases.
Before this patch doing:
rm lisp/calendar/calendar.elc
make lisp/calendar/cal-hebrew.elc
would spew out lots of spurious such warnings about a `date` argument,
pointing to code which has no `date` argument in sight. This was
because that code had calls to inlinable functions (taking a `date`
argument) defined in `calendar.el`, and while `date` is a normal
lexical var at the site of those functions' definitions, it was
declared as dynbound at the call site.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand):
Don't impose our local context onto the inlined function.
* test/lisp/emacs-lisp/bytecomp-tests.el: Add matching test.
This introduces two new optimizations. They're designed for code like
(while
(let (...)
(if ... (progn blabla t) (progn blabla nil)))
...)
and they allow the elimination of the test internal to `while` since
we can immediately know when we return `t` or `nil` what the result
of the test will be.
`cl-labels` tends to generate this kind of code when it applies the
tail-call optimization.
This moves two optimizations from the final pass to the main loop.
Both may enable further optimizations (and the second can be applied
repeatedly but "from the end", so the loop in the final pass only gets
to apply it once).
* doc/lispref/sequences.texi (Sequence Functions): Document them.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns): Mark them as
side-effect-free.
* lisp/emacs-lisp/shortdoc.el (list): Mention them.
* src/fns.c (Flength): Mention them in the doc string.
(length_internal): New function.
(Flength_less, Flength_greater, Flength_equal): New defuns.
(syms_of_fns): Sym them.
Any function that is pure is also side-effect-free and some are also
error-free. Right now these have to be declared separately.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns): Add
bool-vector-count-consecutive, bool-vector-count-population,
bool-vector-subsetp, copysign, isnan, lax-plist-get, ldexp, memql,
regexp-opt and string-to-syntax.
(side-effect-and-error-free-fns): Add type-of.
* lisp/subr.el (kbd, string-replace): Declare side-effect-free.
Since a supplied test function can do anything, assoc is not
side-effect-free (bug#44018). However, with only two arguments it is
pure and should be optimised accordingly.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns): Remove 'assoc'.
(byte-optimize-assoc): Constant-propagate through 2-arg assoc calls.
* test/lisp/emacs-lisp/bytecomp-tests.el
(byte-opt-testsuite-arith-data): Add test cases.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-associative-math)
(byte-optimize-min-max): Transform 3-arg min/max call into two 2-arg
calls, which is faster.
* lisp/emacs-lisp/bytecomp.el (byte-compile-associative): Rename to...
(byte-compile-variadic-numeric): ...this function and simplify,
fixing incorrect comments. The 3-arg strength reduction is now
always done in the optimisers and is no longer needed here.
(byte-compile-min-max): New function.
(byte-compile-minus): Simplify, remove incorrect comment, and use
byte-compile-variadic-numeric.
(byte-compile-quo): Simplify and fix comment.
(byte-optimize-form-code-walker): Use `byte-optimize-form` after
inlining, so optimizations are also applied to the top level call.
Simplify the code for `pure` functions using `byte-optimize-constant-args`.
(byte-optimize-all-constp): Remove, not used any more.
(byte-optimize-1+, byte-optimize-1-): Remove, they are redundant
with the `pure` annotation.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
Remove clause for 'with-output-to-temp-buffer', since it is a
macro and will have been expanded before reaching this point.
Move clauses for 'lambda' and 'closure' to avoid splitting
a cond jump table.
Turn (+ a b c) into (+ (+ a b) c), and do the same for - and *.
The 2-arg operations have their own bytecode which results in a 1.5×
speed-up. Furthermore, the transform enables other optimisations; for
example, (+ a 1 b) -> (+ (1+ a) b).
* lisp/emacs-lisp/byte-opt.el (byte-optimize-plus, byte-optimize-minus)
(byte-optimize-multiply): Transform (OP a b c) into (OP (OP a b) c).
This is the same transformation made for member to memq.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-assoc): New function.
(assoc, rassoc): Set the byte-optimizer property.
Most pure functions need no explicit optimisation; we can do away with
almost all uses of byte-optimize-predicate (now renamed to
byte-optimize-constant-args, since it is not just for predicates).
Also remove some superfluous arity warnings.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-identity, byte-optimize-memq)
(byte-optimize-nth, byte-optimize-nthcdr):
Remove arity warnings and simplify.
* lisp/emacs-lisp/byte-opt.el (<, >, <=, >=, not, null, consp, listp)
(symbolp, stringp, string<, string-lessp, proper-list-p, logand)
(logior, logxor, lognot, car, cdr, car-safe, cdr-safe):
Remove superfluous byte-optimizer property.
(byte-optimize-predicate): Rename to byte-optimize-constant-args.
All uses changed.
Extend the list of 'pure' functions to many predicates and numerical
functions that we are reasonably confident will give portable results.
Also include various list and array accessors, because our use of purity
in the byte compiler isn't affected by the mutability of arguments.
* lisp/emacs-lisp/byte-opt.el: Update example in comment.
(pure-fns): Add many functions.
(byte-optimize-form-code-walker) Don't signal errors during evaluation
of calls to pure functions with constant arguments at compile time,
since such calls are not necessarily reachable.
With bignums, the set of representable integers is no longer
platform-dependent, and since we use nothing but IEEE754 64-bit
floats, all numbers are now portable. Take advantage of this fact
to simplify constant-folding in the byte compiler, allowing it to
be applied more widely.
* lisp/emacs-lisp/byte-opt.el (byte-opt--portable-max)
(byte-opt--portable-min, byte-opt--portable-numberp): Remove.
(byte-opt--arith-reduce, byte-optimize-minus, byte-optimize-1+)
(byte-optimize-1-): Simplify: any number will do, and if N is a
number, then so are -N, N+1 and N-1.
`make-byte-code' wraps `vector' doing some sanity check on the input
arguments. `vector' is in side-effect-and-error-free-fns so add
`make-byte-code' to side-effect-free-fns.
From a patch privately suggested by Mattias Engdegård on 2020-05-11
in a followup to Bug#40671.
* admin/charsets/cp51932.awk:
* admin/charsets/eucjp-ms.awk:
Generate code that does not modify constant conses.
* doc/misc/emacs-mime.texi (Encoding Customization):
* lisp/emacs-lisp/byte-opt.el (byte-compile-side-effect-free-ops):
* lisp/frameset.el (frameset-persistent-filter-alist):
* lisp/gnus/gnus-sum.el (gnus-article-mode-line-format-alist):
Use append instead of nconc.
* lisp/language/japanese.el (japanese-ucs-cp932-to-jis-map)
(jisx0213-to-unicode):
Use mapcar instead of mapc.
* lisp/language/lao-util.el (lao-transcription-consonant-alist)
(lao-transcription-vowel-alist):
* lisp/language/tibetan.el (tibetan-subjoined-transcription-alist):
Use copy-sequence.
* test/src/fns-tests.el (fns-tests-nreverse):
(fns-tests-sort, fns-tests-collate-sort)
(fns-tests-string-version-lessp, fns-tests-mapcan):
Use copy-sequence, vector, and list.
The 'substring' byte op was not emitted, apparently by mistake. Fix.
Suggested by Mark Oteiza <mvoteiza@udel.edu>.
* lisp/emacs-lisp/bytecomp.el (byte-defop-compiler): Add '1-3' clause.
(byte-compile-one-to-three-args): New.
* lisp/emacs-lisp/byte-opt.el (byte-compile-side-effect-free-ops):
Add 'byte-substring'.
* test/lisp/emacs-lisp/bytecomp-tests.el
(byte-opt-testsuite-arith-data): Test 'substring'.
