Previously, variables bound outside `while` loops were not substituted
inside even in the absense of mutation. Add the necessary mutation
checking inside loops to allow propagation of values and aliased
variables.
* lisp/emacs-lisp/byte-opt.el
(byte-optimize--inhibit-outside-loop-constprop): New variable.
(byte-optimize-form-code-walker): First traverse each loop without
substitution to discover mutation, then without restrictions.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-test-loop): New.
(bytecomp-tests--test-cases): Add test cases.
Replace uses of a variable aliasing another variable with that aliased
variable, to allow for variable removal when possible. This also
enables opportunities for other optimisations. Example:
(let ((y x)) (f y)) => (f x)
The optimisation is only performed if both aliased and aliasing
variables are lexically bound. Shadowing bindings are α-renamed when
necessary for correctness. Example:
(let* ((b a) (a EXPR)) (f a b))
=> (let* ((a{new} EXPR)) (f a{new} a))
* lisp/emacs-lisp/byte-opt.el (byte-optimize--aliased-vars): New.
(byte-optimize-form-code-walker): Cancel aliasing upon mutation.
(byte-optimize--rename-var-body, byte-optimize--rename-var): New.
(byte-optimize-let-form): Add the optimisation.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add relevant test cases.
* lisp/emacs-lisp/byte-opt.el
(byte-optimize-assq): New.
(byte-optimize-member, byte-optimize-assoc, byte-optimize-memq):
When the list argument is constant nil, the result is always nil.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test cases.
The current method of propagating constants through setq was unsound
because it relied on each setq form only being traversed at most once
during optimisation, which isn't necessarily true in general; it could
be made to miscompile code in rare cases.
Since it was only used in limited circumstances, disabling this
optimisation doesn't cost us much.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
Don't update the known value when traversing `setq`.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test case.
Simplify (let ((X1 E1) ... (Xn En)) Xn)
=> (progn E1 ... En)
and (let* ((X1 E1) ... (Xn En)) Xn)
=> (let* ((X1 E1) ... (Xn-1 En-1)) En)
and similarly the case where the body is a constant, extending a
previous optimisation that only applied to the constant nil.
This reduces the number of bound variables, shortens the code, and
enables further optimisations.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-letX): Rewrite using
`pcase` and add the aforementioned transformations.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test cases.
Reported by Gregor Zattler.
* lisp/emacs-lisp/bytecomp.el (byte-compile--cond-switch-prefix):
Be more careful in the selection of equality.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test case.
Wrong number of arguments in inlining function calls (to `defsubst` or
explicitly using `inline`) did not result in warnings, or in very
cryptic ones.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand): Add calls
to `byte-compile--check-arity-bytecode`.
* lisp/emacs-lisp/bytecomp.el (byte-compile-emit-callargs-warn)
(byte-compile--check-arity-bytecode): New functions.
(byte-compile-callargs-warn): Use factored-out function.
* test/lisp/emacs-lisp/bytecomp-resources/warn-callargs-defsubst.el:
* test/lisp/emacs-lisp/bytecomp-tests.el ("warn-callargs-defsubst.el"):
New test case.
Functions compiled when inlined (thus from inside the optimiser)
mustn't retain the lexical environment of the caller or there will be
tears. See discussion at
https://lists.gnu.org/archive/html/emacs-devel/2021-05/msg01227.html .
Bug found by Stefan Monnier.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand):
Bind byte-optimize--lexvars to nil when re-entering the compiler
recursively.
* test/lisp/emacs-lisp/bytecomp-resources/bc-test-alpha.el:
* test/lisp/emacs-lisp/bytecomp-resources/bc-test-beta.el: New files.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-defsubst): New test.
Allow a condition-case handler on the form (:success BODY) to be
specified as the success continuation of the protected form, with
the specified variable bound to its result.
* src/eval.c (Fcondition_case): Update the doc string.
(internal_lisp_condition_case): Implement in interpreter.
(syms_of_eval): Defsym :success.
* lisp/emacs-lisp/bytecomp.el (byte-compile-condition-case):
Implement in byte-compiler.
* lisp/emacs-lisp/cl-macs.el (cl--self-tco): Allow self-TCO
from success handler.
* doc/lispref/control.texi (Handling Errors): Update manual.
* etc/NEWS: Announce.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases)
(bytecomp-condition-case-success):
* test/lisp/emacs-lisp/cl-macs-tests.el (cl-macs--labels):
Add test cases.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker): Don't
perform incorrect optimisations when a condition-case variable shadows
another lexical variable.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
New test case.
These changes allow all bytecomp-tests to be run interactively.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp--with-warning-test)
(bytecomp--define-warning-file-test): Interpret any space in the
pattern as arbitrary whitespace to tolerate line breaks.
Don't abuse the expected-failure mechanism when checking
for the expected absense of a warning.
(bytecomp/*.el): Rewrite patterns to work with line breaks
in the middle.
Now all test cases are run with both lexical and dynamic binding
where applicable, comparing interpreted against compiled results.
Previously, almost all tests were only run with dynamic binding
which was definitely not intended.
* test/lisp/emacs-lisp/bytecomp-tests.el
(byte-opt-testsuite-arith-data): Rename to bytecomp-tests--test-cases.
(bytecomp-check-1, bytecomp-explain-1, bytecomp-tests)
(bytecomp-lexbind-tests, bytecomp-lexbind-check-1)
(bytecomp-lexbind-explain-1): Remove.
(bytecomp-tests--eval-interpreted, bytecomp-tests--eval-compiled)
(bytecomp-tests-lexbind, bytecomp-tests-dynbind)
(bytecomp-tests--test-cases-lexbind-only): New.
(cconv--convert-funcbody): Check there's something after a docstring.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-string-vs-docstring):
New corresponding test.
* lisp/emacs-lisp/bytecomp.el (byte-compile--reify-function): Don't
move let bindings into the lambda. Don't reverse list of
bindings. (byte-compile): Evaluate the return value if it was
previously reified.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-reify-function):
Add tests.
This bug was introduced by the lexical variable constant propagation
mechanism. It was discovered by Michael Heerdegen.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-let-form)
(byte-optimize-body): Let the effects of a local defvar declaration be
scoped by let and let*, not any arbitrary Lisp expression body (such
as progn).
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--get-vars)
(bytecomp-local-defvar): New test.
Lexical variables bound to a constant value (symbol, number or string)
are substituted at their point of use and the variable then eliminated
if possible. Example:
(let ((x (+ 2 3))) (f x)) => (f 5)
This reduces code size, eliminates stack operations, and enables
further optimisations. The implementation is conservative, and is
strongly curtailed by the presence of variable mutation, conditions
and loops.
* lisp/emacs-lisp/byte-opt.el
(byte-optimize-enable-variable-constprop)
(byte-optimize-warn-eliminated-variable): New constants.
(byte-optimize--lexvars, byte-optimize--vars-outside-condition)
(byte-optimize--vars-outside-loop, byte-optimize--dynamic-vars):
New dynamic variables.
(byte-optimize--substitutable-p, byte-optimize-let-form):
New functions.
(byte-optimize-form-code-walker): Adapt clauses for variable
constprop, and add clauses for 'setq' and 'defvar'.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-test-var)
(bytecomp-test-get-var, bytecomp-test-identity)
(byte-opt-testsuite-arith-data): Add test cases.
Before this patch doing:
rm lisp/calendar/calendar.elc
make lisp/calendar/cal-hebrew.elc
would spew out lots of spurious such warnings about a `date` argument,
pointing to code which has no `date` argument in sight. This was
because that code had calls to inlinable functions (taking a `date`
argument) defined in `calendar.el`, and while `date` is a normal
lexical var at the site of those functions' definitions, it was
declared as dynbound at the call site.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand):
Don't impose our local context onto the inlined function.
* test/lisp/emacs-lisp/bytecomp-tests.el: Add matching test.
* test/lisp/progmodes/elisp-mode-tests.el (xref-elisp-test-run):
Make sure file names can be compared as strings, by running them
through 'file-truename'. Reported by Vin Shelton
<acs@alumni.princeton.edu>.
* test/lisp/emacs-lisp/bytecomp-tests.el ("warn-obsolete-hook.el")
("warn-obsolete-variable.el"): Use [^z-a] to match a newline as
well. Reported by Vin Shelton <acs@alumni.princeton.edu>.
* lisp/emacs-lisp/eieio-core.el (eieio-defclass-autoload): Try to make
the wording of the warning about the obsoleted variable less confusing.
* lisp/emacs-lisp/bytecomp.el (byte-compile-check-variable): Don't
warn for lexical variables (Bug#39169). Fix spurious `or'.
* test/lisp/emacs-lisp/bytecomp-tests.el
(bytecomp/warn-obsolete-variable-bound\.el): New test.
* test/lisp/emacs-lisp/bytecomp-resources/warn-obsolete-variable-bound.el:
New file.
* lisp/emacs-lisp/bytecomp.el (byte-compile-file): Don’t fail if
target filename doesn’t contain a directory name.
* test/lisp/emacs-lisp/bytecomp-tests.el
(bytecomp-tests--target-file-no-directory): New unit test.
See Bug#44631. While testing for a readonly output directory has
slightly different semantics, in practice they should cover cases
where Emacs is sandboxed and can only write to the destination file,
not its directory.
* lisp/emacs-lisp/bytecomp.el (byte-compile-file): Handle the case
where the output directory is not writable.
* test/lisp/emacs-lisp/bytecomp-tests.el
(bytecomp-tests--not-writable-directory)
(bytecomp-tests--dest-mountpoint): New unit tests.
* test/lisp/emacs-lisp/bytecomp-tests.el
(bytecomp--with-warning-test): New macro.
(bytecomp-warn-wrong-args, bytecomp-warn-wrong-args-subr):
Use above new macro.
(bytecomp-warn-variable-lacks-prefix): New test.
Since a supplied test function can do anything, assoc is not
side-effect-free (bug#44018). However, with only two arguments it is
pure and should be optimised accordingly.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns): Remove 'assoc'.
(byte-optimize-assoc): Constant-propagate through 2-arg assoc calls.
* test/lisp/emacs-lisp/bytecomp-tests.el
(byte-opt-testsuite-arith-data): Add test cases.
This bug affected compilation of
(cond ((member '(some list) variable) ...) ...)
While equal is symmetric, member is not; in the latter case the
arguments must be a variable and a constant list, in that order.
Reported by Ikumi Keita.
* lisp/emacs-lisp/bytecomp.el (byte-compile--cond-switch-prefix):
Don't treat equality and member predicates in the same way; only
the former are symmetric in their arguments.
* test/lisp/emacs-lisp/bytecomp-tests.el
(byte-opt-testsuite-arith-data): Add test cases.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-check-1)
(test-byte-opt-arithmetic, bytecomp-lexbind-check-1)
(bytecomp-lexbind-explain-1):
When comparing interpreted with compiled results, don't consider all
errors to be equal; take the error type into account. (The error
arguments may differ, but there may be good reasons for that.)
* test/lisp/emacs-lisp/bytecomp-tests.el
(test-byte-comp-macro-expand-lexical-override): Remove functions
before testing so that the test can be run twice without failing.
* lisp/emacs-lisp/bytecomp.el (byte-compile-associative):
Translate numerical identity expressions, such as (+ x) and (* x),
into (* x 1) since the previous translation (+ x 0) gets it wrong
for x = -0.0.
* test/lisp/emacs-lisp/bytecomp-tests.el
(byte-opt-testsuite-arith-data): Add test cases.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-check-1)
(bytecomp-explain-1, test-byte-opt-arithmetic, bytecomp-lexbind-check-1)
(bytecomp-lexbind-explain-1):
If an expression raises an error when evaluated, don't treat it as if
it had succeeded with the value nil; use 'bytecomp-check-error' as the
result instead.
The 'substring' byte op was not emitted, apparently by mistake. Fix.
Suggested by Mark Oteiza <mvoteiza@udel.edu>.
* lisp/emacs-lisp/bytecomp.el (byte-defop-compiler): Add '1-3' clause.
(byte-compile-one-to-three-args): New.
* lisp/emacs-lisp/byte-opt.el (byte-compile-side-effect-free-ops):
Add 'byte-substring'.
* test/lisp/emacs-lisp/bytecomp-tests.el
(byte-opt-testsuite-arith-data): Test 'substring'.
A single `cond' form can how be compiled to any number of switch ops,
optionally interspersed with non-switch conditions.
Previously, switch ops would only be used for whole `cond' forms
containing no other tests.
* lisp/emacs-lisp/bytecomp.el (byte-compile--cond-vars):
Rename from `byte-compile-cond-vars'.
(byte-compile--default-val): Remove.
(byte-compile--cond-switch-prefix):
Replace `byte-compile-cond-jump-table-info'; now also returns
trailing non-switch clauses.
(byte-compile-cond-jump-table): New arguments; no longer compiles
the default case.
(byte-compile-cond): Look for and compile switches at any place in the
list of clauses.
* test/lisp/emacs-lisp/bytecomp-tests.el (byte-opt-testsuite-arith-data):
Add test expression.
Allow any mixture of `eq', `eql' and `equal', `memq', `memql' and
`member' in a switch-like `cond' to be compiled into a single switch.
* lisp/emacs-lisp/bytecomp.el (byte-compile--common-test): New.
(byte-compile-cond-jump-table-info): Use most specific common test.
* test/lisp/emacs-lisp/bytecomp-tests.el (byte-opt-testsuite-arith-data):
Add test cases for multi-value clause cond forms.
