These are symbols with position from source code, which should not be replaced
by bare symbols in, e.g., optimization functions.
* lisp/Makefile.in: (BYTE_COMPILE_FLAGS, compile-first case): Set
max-specpdl-size to 5000 for the benefit of lisp/emacs-lisp/comp.el.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker)
(byte-optimize--rename-var, byte-optimize-if, byte-optimize-letX)
* lisp/emacs-lisp/bytecomp.el (byte-compile-recurse-toplevel)
(byte-compile-lambda)
* lisp/emacs-lisp/cconv.el (cconv-convert)
* lisp/emacs-lisp/macroexp.el (macroexp--expand-all): Preserve, e.g., (car
form) in the byte compiler, when this form's car is a symbol with position of
a special form, rather than replacing the symbol with a bare symbol, e.g.
'cond.
Since string-lessp has its own byte-op, using it is much faster than
calling string-greaterp even with the need to bind a temporary
variable.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-string-greaterp): New.
(string-greaterp, string>): Set byte-optimizer.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns, pure-fns):
Mark base64-decode-string, base64-encode-string and
base64url-encode-string as pure and side-effect-free.
* lisp/emacs-lisp/byte-opt.el (byte-optimize--substitutable-p):
Treat (internal-get-closed-var N) as constants for propagation
purposes, because that is effectively what such forms will be compiled
to. This allows for the elimination of some lexical variables.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test case.
This branch is intended to generate correct position information in warning
and error messages from the byte compiler, and is intended thereby to fix bugs
It introduces a new mechanism, the symbol with position. This is taken over
from the previous git branch scratch/accurate-warning-pos which was abandoned
for being too slow. The main difference in the current branch is that the
symbol `nil' is never given a position, thus speeding up NILP markedly.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand)
(byte-optimize-form-code-walker, byte-optimize-let-form, byte-optimize-while)
(byte-optimize-apply): Use byte-compile-warn-x in place of byte-compile-warn.
* lisp/emacs-lisp/bytecomp.el (byte-compile--form-stack): New variable.
(byte-compile-strip-s-p-1, byte-compile-strip-symbol-positions): New
functions.
(byte-compile-recurse-toplevel, byte-compile-initial-macro-environment)
(byte-compile-preprocess, byte-compile-macroexpand-declare-function): Bind
print-symbols-bare to non-nil.
(byte-compile--first-symbol, byte-compile--warning-source-offset): New
functions.
(byte-compile-warning-prefix): Modify to output two sets of position
information, the old (incorrect) set and the new set.
(byte-compile-warn): Strip positions from symbols before outputting.
(byte-compile-warn-x): New function which outputs a correct position supplied
in an argument.
(byte-compile-warn-obsolete, byte-compile-emit-callargs-warn)
(byte-compile-format-warn, byte-compile-nogroup-warn)
(byte-compile-arglist-warn, byte-compile-docstring-length-warn)
(byte-compile-warn-about-unresolved-functions, byte-compile-file)
(byte-compile--check-prefixed-var, byte-compile--declare-var)
(byte-compile-file-form-defvar-function, byte-compile-file-form-defmumble)
(byte-compile-check-lambda-list, byte-compile--warn-lexical-dynamic)
(byte-compile-lambda, byte-compile-form, byte-compile-normal-call)
(byte-compile-check-variable, byte-compile-free-vars-warn)
(byte-compile-subr-wrong-args, byte-compile-fset, byte-compile-set-default)
(byte-compile-condition-case, byte-compile-save-excursion)
(byte-compile-defvar, byte-compile-autoload)
(byte-compile-make-variable-buffer-local, byte-compile-define-symbol-prop)
(byte-compile-define-keymap): Replace byte-compile-warn with
byte-compile-warn-x.
(byte-compile-file, compile-defun): Bind symbols-with-pos-enabled to non-nil.
(compile-defun, byte-compile-from-buffer): Use `read-positioning-symbols'
rather than plain `read'.
(byte-compile-toplevel-file-form, byte-compile-form): Dynamically bind
byte-compile--form-stack.
(byte-compile-file-form-autoload, byte-compile-file-form-defvar)
(byte-compile-file-form-make-obsolete, byte-compile-lambda)
(byte-compile-push-constant, byte-compile-cond-jump-table)
(byte-compile-define-keymap, byte-compile-annotate-call-tree):
Strip positions from symbols where they are unwanted.
(byte-compile-file-form-defvar): Strip positions from symbols using
`bare-symbol'.
(byte-compile-file-form-defmumble): New variable bare-name, a version of name
without its position.
(byte-compile-lambda): Similarly, new variable bare-arglist.
(byte-compile-free-vars-warn): New argument arg supplying position information
to byte-compile-warn-x.
(byte-compile-push-constant): Manipulation of symbol positions.
(display-call-tree): Strip positions from symbols.
* lisp/emacs-lisp/cconv.el (cconv-convert, cconv--analyze-use)
(cconv--analyze-function, cconv-analyze-form): Replace use of
byte-compile-warn with byte-compile-warn-x.
* lisp/emacs-lisp/cl-generic.el (cl-defmethod): New variable org-name which
will supply position information to a new macroexp-warn-and-return.
* lisp/emacs-lisp/cl-macs.el (cl-macs--strip-s-p-1)
(cl-macs--strip-symbol-positions): New functions to strip positions from
symbols in an expression. These duplicaate similarly named functions in
bytecomp.el.
* lisp/emacs-lisp/macroexpand.el (macroexp--warn-wrap): Calls
byte-compile-warn-x in place of byte-compile-warn.
(macroexp-warn-and-return): Commented out new position parameter _arg.
* src/.gdbinit: Add in code to handle symbols with position.
* src/alloc.c (XPNTR, set_symbol_name, valid_lisp_object_p, purecopy)
(mark_char_table, mark_object, survives_gc_p, symbol_uses_obj): Use
BARE_SYMBOL_P and XBARE_SYMBOL in place of the former SYMBOLP and XSYMBOL.
(build_symbol_with_pos): New function.
(Fgarbage_collect): Bind Qsymbols_with_pos_enabled to nil around the call to
garbage_collect.
* src/data.c (Ftype_of): Add case for PVEC_SYMBOL_WITH_POS.
(Fbare_symbol_p, Fsymbol_with_pos_p, Fbare_symbol, Fsymbol_with_pos_pos)
(Fposition_symbol): New functions.
(symbols_with_pos_enabled): New boolean variable.
* src/fns.c (internal_equal, hash_lookup): Handle symbols with position.
* src/keyboard.c (recursive_edit_1): Bind Qsymbols_with_pos_enabled and
Qprint_symbols_bare to nil.
* src/lisp.h (lisp_h_PSEUDOVECTORP): New macro.
(lisp_h_BASE_EQ): New name for the former lisp_h_EQ.
(lisp_h_EQ): Extended to handle symbols with position.
(lisp_h_NILP): Now uses BASE_EQ rather than EQ.
(lisp_h_SYMBOL_WITH_POS_P, lisp_h_BARE_SYMBOL_P): New macros.
(lisp_h_SYMBOLP): Redefined to handle symbols with position.
(BARE_SYMBOL_P, BASE_EQ): New macros.
(SYMBOLP (macro)): Removed.
(SYMBOLP (function), XSYMBOL, make_lisp_symbol, builtin_lisp_symbol)
(c_symbol_p): Moved to later in file.
(struct Lisp_Symbol_With_Pos): New data type.
(pvec_type): PVEC_SYMBOL_WITH_POS: New type code.
(PSEUDOVECTORP): Redefined to use the lisp_h_PSEUDOVECTORP.
(BARE_SYMBOL_P, SYMBOL_WITH_POS_P, SYMBOLP, XSYMBOL_WITH_POS, XBARE_SYMBOL)
(XSYMBOL, make_lisp_symbol, builtin_lisp_symbol, c_symbol_p, CHECK_SYMBOL)
(BASE_EQ): New functions, or functions moved from earlier in the file.
(SYMBOL_WITH_POS_SYM, SYMBOL_WITH_POS_POS): New INLINE functions.
* src/lread.c (read0, read1, read_list, read_vector, read_internal_start)
(list2): Add a new bool parameter locate_syms.
(Fread_positioning_symbols): New function.
(Fread_from_string, read_internal_start, read0, read1, read_list): Pass around
suitable values for locate_syms.
(read1): Build symbols with position when locate_syms is true.
* src/print.c (print_vectorlike): Add handling for PVEC_SYMBOL_WITH_POS.
(print_object): Replace EQ with BASE_EQ.
(print_symbols_bare): New boolean variable.
* lisp/keymap.el: New file with all the new keymap-* functions.
* lisp/loadup.el ("keymap"): Load.
* lisp/subr.el (kbd): Refactor out all the code to key-parse.
(define-key-after, keyboard-translate, global-set-key)
(local-set-key, global-unset-key, local-unset-key)
(local-key-binding, global-key-binding)
(substitute-key-definition): Note in doc strings that these are
legacy functions.
(define-keymap--define): Use keymap-set.
* lisp/emacs-lisp/byte-opt.el: Remove the optimizations for
defvar-keymap and define-keymap since the macros now only
understand the kbd syntax.
* lisp/emacs-lisp/bytecomp.el (byte-compile-define-keymap)
(byte-compile-define-keymap--define): Warn about invalid key
definitions in all keymap-* functions.
* lisp/emacs-lisp/shortdoc.el (keymaps): Add shortdocs form
keymap* functions.
* src/keymap.c (possibly_translate_key_sequence): Adjust callers
to key-valid-p and key-parse.
(syms_of_keymap): Adjust defs.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-cond):
Optimise clause-free `cond`, which can arise from earlier
transformations. This enables further optimisations.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test cases.
This should really be taken care of by a syntax normalisation step in
the frontend, but there is no such step for non-lexbind code yet.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-letX): Tolerate bindingsa
without initialising expressions.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test cases.
Previously, variables bound outside `while` loops were not substituted
inside even in the absense of mutation. Add the necessary mutation
checking inside loops to allow propagation of values and aliased
variables.
* lisp/emacs-lisp/byte-opt.el
(byte-optimize--inhibit-outside-loop-constprop): New variable.
(byte-optimize-form-code-walker): First traverse each loop without
substitution to discover mutation, then without restrictions.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-test-loop): New.
(bytecomp-tests--test-cases): Add test cases.
Replace uses of a variable aliasing another variable with that aliased
variable, to allow for variable removal when possible. This also
enables opportunities for other optimisations. Example:
(let ((y x)) (f y)) => (f x)
The optimisation is only performed if both aliased and aliasing
variables are lexically bound. Shadowing bindings are α-renamed when
necessary for correctness. Example:
(let* ((b a) (a EXPR)) (f a b))
=> (let* ((a{new} EXPR)) (f a{new} a))
* lisp/emacs-lisp/byte-opt.el (byte-optimize--aliased-vars): New.
(byte-optimize-form-code-walker): Cancel aliasing upon mutation.
(byte-optimize--rename-var-body, byte-optimize--rename-var): New.
(byte-optimize-let-form): Add the optimisation.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add relevant test cases.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-body): Flatten body.
This simplifies the source tree and reduces the number of different
cases that other optimisations need to take into account.
Treat `ignore` as any other function during source-level optimisation,
to avoid having its warning-suppression effects cancelled by repeated
passes. Instead, define a custom code generation function.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
Don't treat `ignore' specially here.
(side-effect-free-fns): Don't mark `ignore` as side-effect-free
or error-free (although it is), since that would allow the optimiser
to elide calls.
* lisp/emacs-lisp/bytecomp.el (ignore, byte-compile-ignore):
Define and register a code-gen function.
* lisp/emacs-lisp/byte-opt.el
(byte-optimize-assq): New.
(byte-optimize-member, byte-optimize-assoc, byte-optimize-memq):
When the list argument is constant nil, the result is always nil.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test cases.
The current method of propagating constants through setq was unsound
because it relied on each setq form only being traversed at most once
during optimisation, which isn't necessarily true in general; it could
be made to miscompile code in rare cases.
Since it was only used in limited circumstances, disabling this
optimisation doesn't cost us much.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
Don't update the known value when traversing `setq`.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test case.
Ensure in cconv that let-bindings have the normal form (VAR EXPR)
where VAR is a valid variable name, so that we don't need to keep
re-checking this all the time in the optimiser.
* lisp/emacs-lisp/byte-opt.el
(byte-optimize-enable-variable-constprop)
(byte-optimize-warn-eliminated-variable): Remove; these were mainly
used for debugging.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-let-form):
Assume normalised let-bindings (with lexical-binding).
Stop using the variables removed above.
* lisp/emacs-lisp/cconv.el (cconv-convert): Ensure normalised
let-bindings. Malformed bindings are dropped after warning.
remove byte-optimize-warn-eliminated-variable
Simplify (let ((X1 E1) ... (Xn En)) Xn)
=> (progn E1 ... En)
and (let* ((X1 E1) ... (Xn En)) Xn)
=> (let* ((X1 E1) ... (Xn-1 En-1)) En)
and similarly the case where the body is a constant, extending a
previous optimisation that only applied to the constant nil.
This reduces the number of bound variables, shortens the code, and
enables further optimisations.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-letX): Rewrite using
`pcase` and add the aforementioned transformations.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test cases.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-let-form): Move warnings...
* lisp/emacs-lisp/cconv.el (cconv-convert): ...here, which is an
overall better place (closer to the front-end).
Rewrite (prog1 CONST FORMS...) => (progn FORMS... CONST)
where CONST is a compile-time constant, because putting the value last
allows the lapcode peephole pass to do important improvements like
branch elimination. Also use progn instead of prog1 for `ignore`.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
New `prog1` and `ignore` transforms.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
Remove for-effect uses of lexical variables. We previously relied on
this being done by the lapcode peephole optimiser but at source level
it enables more optimisation opportunities.
Keywords are elided for the same reason.
Make the optimiser aware of lexical arguments. Otherwise we cannot
know for sure whether a variable is lexical or dynamic during
traversal.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-one-form): New optimiser
entry point, replacing the recursive byte-optimize-form.
* lisp/emacs-lisp/bytecomp.el (byte-optimize-one-form): Autoload.
(byte-compile-keep-pending, byte-compile-top-level):
Use byte-optimize-one-form.
Wrong number of arguments in inlining function calls (to `defsubst` or
explicitly using `inline`) did not result in warnings, or in very
cryptic ones.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand): Add calls
to `byte-compile--check-arity-bytecode`.
* lisp/emacs-lisp/bytecomp.el (byte-compile-emit-callargs-warn)
(byte-compile--check-arity-bytecode): New functions.
(byte-compile-callargs-warn): Use factored-out function.
* test/lisp/emacs-lisp/bytecomp-resources/warn-callargs-defsubst.el:
* test/lisp/emacs-lisp/bytecomp-tests.el ("warn-callargs-defsubst.el"):
New test case.
Found by Pip Cet.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-quote): Fix mistake that
made this optimiser ineffective at removing quoting of nil, t, and
keywords. The only obvious consequence is that we no longer need...
(byte-optimize-form): ...a 'nil => nil normalising step here; remove.
(byte-optimize-form-code-walker): Make the compiler warn about (quote).
Functions compiled when inlined (thus from inside the optimiser)
mustn't retain the lexical environment of the caller or there will be
tears. See discussion at
https://lists.gnu.org/archive/html/emacs-devel/2021-05/msg01227.html .
Bug found by Stefan Monnier.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand):
Bind byte-optimize--lexvars to nil when re-entering the compiler
recursively.
* test/lisp/emacs-lisp/bytecomp-resources/bc-test-alpha.el:
* test/lisp/emacs-lisp/bytecomp-resources/bc-test-beta.el: New files.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-defsubst): New test.
(byte-compile-inline-expand): When inlining code from another file,
always inline the byte-code version of the function.
(byte-optimize--pcase): Simplify edebug spec.
It's good practice to compare integers using 'eql' because two bignum
objects representing the same integer might not be 'eq'. However,
'eql' is slower and doesn't have its own byte code. Therefore,
replace it with 'eq' if one argument is guaranteed to be a fixnum on
all platforms.
* lisp/emacs-lisp/byte-opt.el (byte-optimize--fixnump): New helper
function.
(byte-optimize-equal, byte-optimize-member, byte-optimize-assoc): Use
it to optimize 'eql' etc. to 'eq' if it will always compare fixnums.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker): Don't
perform incorrect optimisations when a condition-case variable shadows
another lexical variable.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
New test case.
