Lambda-lifted variables (ones passed explicitly to lambda-lifted
functions) that are also captured in an outer closure and shadowed
were renamed incorrectly (bug#51982).
Reported by Paul Pogonyshev.
* lisp/emacs-lisp/cconv.el (cconv--lifted-arg): New.
(cconv-convert): Provide correct definiens for the closed-over
variable.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
* test/lisp/emacs-lisp/cconv-tests.el (cconv-tests--intern-all)
(cconv-closure-convert-remap-var): Add tests.
(cherry picked from commit 45252ad8f9)
Fix longstanding bug due to unexpected interference via side-effect.
* lisp/emacs-lisp/cl-generic.el (cl--generic-get-dispatcher):
Copy the `dispatch` arg before storing it into the hash-table.
1. Check the type (symbol with position) of the argument given to the native
compiled version of SYMBOL_WITH_POS_SYM.
2. Handle infinite recursion caused by circular lists, etc., in
macroexp-strip-symbol-positions by using hash tables.
3. Read byte compiled functions without giving symbols positions.
* lisp/emacs-lisp/comp.el (comp-finalize-relocs): Add symbol-with-pos-p into
the list of relocated symbols.
* lisp/emacs-lisp/macroexp.el (macroexp--ssp-conses-seen)
(macroexp--ssp-vectors-seen, macroexp--ssp-records-seen): Renamed, and
animated as hash tables.
(macroexp--strip-s-p-2): Optionally tests for the presence of an argument in
one of the above hash tables, so as to handle otherwise infinite recursion.
(byte-compile-strip-s-p-1): Add a condition-case to handle infinite recursion
caused by circular lists etc., using the above hash tables as required.
* src/comp.c (comp_t): New element symbol_with_pos_sym.
(emit_SYMBOL_WITH_POS_SYM): Amend just to call the new SYMBOL_WITH_POS_SYM.
(emit_CHECK_SYMBOL_WITH_POS, define_SYMBOL_WITH_POS_SYM): New functions.
(Fcomp__init_ctxt): Register an emitter for Qsymbol_with_pos_p.
(Fcomp__compile_ctxt_to_file): Call define_SYMBOL_WITH_POS_SYM.
(syms_of_comp): Define Qsymbol_with_pos_p.
* src/data.c (syms_of_data): Define a new error symbol Qrecursion_error, an
error category for the new error symbols Qexcessive_variable_binding and
Qexcessive_lisp_nesting.
* src/eval.c (grow_specpdl): Change the signal_error call to an xsignal0 call
using the new error symbol Qexcessive_variable_binding.
(eval_sub, Ffuncall): Change the `error' calls to xsignal using the new error
symbol Qexcessive_lisp_nesting.
* src/lread.c (read1): When reading a compiled function, read the components
of the vector without giving its symbols a position.
Since the location of those files changes between build time and
installation time, this requires to tweak the file name used in those
(#$ . NNN) references during the dump so they don't hardcode the build
directory. We do it in the same way as was already done for those
same file names in `load-history`, except we convert them back to
absolute file names more lazily (i.e. when fetching the actual
docstring rather than at startup), which requires remembering the
`lisp-dir` computed at startup in the new `lisp-directory` variable.
* src/Makefile.in ($(etc)/DOC): Don't scan Lisp files any more.
* src/lread.c (Fload): Use relative file names for `load-file-name`
when preloading for the dump, like we already did for `current-load-list`.
(read_list): Don't zero-out dynamic docstring references during the
preload since they won't be filled later by Snarf-documentation any more.
(read1): Remove the hash-hack for doc references that were zeroed.
* lisp/startup.el (lisp-directory): New variable.
(command-line): Set it.
* src/doc.c (get_doc_string): Use `lisp-directory` for dynamic
docstring references using relative file names.
(syms_of_doc): Add `Qlisp_directory`.
* lib-src/make-docfile.c (scan_file): Don't handle `.el` or `.elc`
files any more.
(IS_SLASH): Remove macro, not used any more.
(skip_white, read_lisp_symbol, search_lisp_doc_at_eol)
(scan_lisp_file): Remove functions, not used any more.
* doc/lispref/loading.texi (Library Search): Mention `lisp-directory`.
Assertions should only be used to check internal consistency within a
package, not to check arguments passed by callers. Instead, define
and use a new error symbol.
* lisp/emacs-lisp/ert.el (ert-test-unbound): New error symbol.
(ert-select-tests): Use it.
* test/lisp/emacs-lisp/ert-tests.el (ert-test-select-undefined): New
unit test.
* etc/NEWS: Document new behavior.
This version of the software should bootstrap Emacs successfully with native
compilation enabled.
* lisp/emacs-lisp/bytecomp.el (byte-compile-strip-s-p-1)
(byte-compile-strip-symbol-positions): Rename and move to macroexp.el. Rename
calls to these functions throughout the file.
(byte-compile-initial-macro-environment): In the code sections for
eval-when-compile and eval-and-compile, call macroexp-strip-symbol-positions
before evaluating code.
(byte-compile-file, byte-compile-output-file-form)
(byte-compile-file-form-defmumble, byte-compile, batch-byte-compile): Call
macroexp-strip-symbol-positions from code being passed to the native compiler.
* lisp/emacs-lisp/cl-macs.el (cl-macs--strip-s-p-1)
(cl-macs--strip-symbol-positions): Remove, replacing them with the renamed
functions in macroexp.el.
(cl-define-compiler-macro): Apply macroexp-strip-symbol-positions to ARGS and
BODY.
* lisp/emacs-lisp/comp.el (comp-limplify-lap-inst): Use `null' to compile
byte-not rather than a compilation of `eq'.
(comp--native-compile): bind symbols-with-pos-enabled to t.
* lisp/emacs-lisp/macroexp.el (byte-compile--ssp-conses-seen)
(byte-compile--ssp-vectors-seen, byte-compile--ssp-records-seen): Provisional
auxiliary variables to support the following functions.
(macroexp--strip-s-p-2, byte-compile-strip-s-p-1)
(macroexp-strip-symbol-positions): Functions moved from bytecomp.el, renamed,
and further developed.
(macroexp--compiler-macro): Bind symbol-with-pos-enabled to t around the call
to `handler'.
(internal-macroexpand-for-load): Strip symbol positions from the form being
eagerly expanded for macros.
* src/comp.c (F_SYMBOLS_WITH_POS_ENABLED_RELOC_SYM): New macro for a
relocation symbol.
(comp_t): New elements bool_ptr_type, f_symbols_with_pos_enabled_ref,
lisp_symbol_with_position, lisp_symbol_with_position_header,
lisp_symbol_with_position_sym, lisp_symbol_with_position_pos,
lisp_symbol_with_position_type, lisp_symbol_with_position_ptr_type,
get_symbol_with_position.
(helper_GET_SYMBOL_WITH_POSITION): New function.
(emit_BASE_EQ): Function rename from emit_EQ.
(emit_AND, emit_OR, emit_BARE_SYMBOL_P, emit_SYMBOL_WITH_POS_P)
(emit_SYMBOL_WITH_POS_SYM): New functions.
(emit_EQ): New function which handles symbols with position correctly.
(emit_NILP): Use emit_BASE_EQ rather than emit_EQ.
(emit_limple_insn): When emitting a conditional branch, check each operand for
being a literal Qnil, and if one of them is, use emit_BASE_EQ rather than
emit_EQ.
(declare_runtime_imported_funcs): Declare helper_GET_SYMBOL_WITH_POSITION.
(emit_ctxt_code): Export the global F_SYMBOLS_WITH_POS_ENABLED_RELOC_SYM.
(define_lisp_symbol_with_position, define_GET_SYMBOL_WITH_POSITION): New
functions.
(Fcomp__init_ctxt): Initialise comp.bool_ptr_type, call the two new
define_.... functions.
(load_comp_unit): Initialise **f_symbols_with_pos_enabled_reloc.
* src/fns.c (Fput): Strip positions from symbols in PROPNAME and VALUE.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Improve
formatting of key missing in the description. Use face help-key-name
on terminals that can't display underline.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Improve display
of the keys TAB, RET, SPC, DEL, and ESC. This fixes a bug where " "
was highlighted in the description in a confusing way.
* test/lisp/emacs-lisp/rmc-tests.el
(test-rmc--add-key-description)
(test-rmc--add-key-description/with-attributes): Update tests for the
above change.
* lisp/emacs-lisp/rmc.el (rmc--show-help): Factor out new function
from read-multiple-choice.
(read-multiple-choice): Add new optional argument show-help.
* doc/lispref/commands.texi (Reading One Event): Document above new
optional argument.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Add face
property also when key is not in the name string.
* test/lisp/emacs-lisp/rmc-tests.el
(test-rmc--add-key-description/with-attributes)
(test-rmc--add-key-description/non-graphical-display): Update tests.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Factor out
new function from...
(read-multiple-choice): ...here.
* test/lisp/emacs-lisp/rmc-tests.el (test-rmc--add-key-description)
(test-rmc--add-key-description/with-attributes)
(test-rmc--add-key-description/non-graphical-display): New tests.
* test/lisp/emacs-lisp/cconv-tests.el: Remove `:expected-result :failed`
from the bug#28557 tests.
(cconv-tests-cl-function-:documentation): Account for the presence of
the arglist (aka "usage") in the docstring.
* lisp/emacs-lisp/cl-macs.el (cl--transform-lambda):
* lisp/emacs-lisp/cl-generic.el (cl-defgeneric):
Handle non-constant `:documentation`.
* lisp/emacs-lisp/generator.el (iter-lambda):
* lisp/emacs-lisp/cconv.el (cconv--convert-funcbody):
Use `macroexp-parse-body`.
This fixes a known bug in `named-let`.
* lisp/emacs-lisp/cl-macs.el (cl--self-tco): Prevent TCO from inside
dynamic variable bindings.
* test/lisp/emacs-lisp/cl-macs-tests.el (cl-macs--labels): Add test.
This predicate can be used for discriminating between lexically and
dynamically bound variables during macro-expansion (only).
It is restricted to internal use for the time being.
* lisp/emacs-lisp/bytecomp.el (byte-compile-initial-macro-environment):
Use macroexpand--all-toplevel.
* lisp/emacs-lisp/macroexp.el (macroexp-dynamic-variable-p): New.
(macroexp--expand-all): Maintain macroexp--dynvars.
(macroexpand-all): Rebind macroexp--dynvars.
(macroexpand--all-toplevel): New.
(internal-macroexpand-for-load): Use macroexpand--all-toplevel.
* src/eval.c (eval_sub): Transfer defvar declarations from
Vinternal_interpreter_environment into macroexp--dynvars during
lazy macro-expansion.
* src/lread.c (readevalloop): Rebind macroexp--dynvars around
read-and-evaluate operations.
(syms_of_lread): Define macroexp--dynvars.
* test/lisp/emacs-lisp/macroexp-resources/vk.el: New file.
* test/lisp/emacs-lisp/macroexp-tests.el (macroexp-tests--run-emacs)
(macroexp-tests--eval-in-subprocess)
(macroexp-tests--byte-compile-in-subprocess)
(macroexp--tests-dynamic-variable-p): Add tests.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns, pure-fns):
Mark base64-decode-string, base64-encode-string and
base64url-encode-string as pure and side-effect-free.
This optimisation is of very limited utility and miscompiles top-level
code having the form (eval 'CODE t) by replacing it with CODE which
will then, as things currently stand, be evaluated with dynamic binding.
* lisp/emacs-lisp/bytecomp.el (byte-compile-file-form-eval): Remove.
* lisp/emacs-lisp/ert.el (ert-load-file-name): New defvar.
(ert-write-junit-test-report): Use it. Make detection of selector
more robust. Protect calls of `xml-escape-string' with 'noerror.
* test/infra/Makefile.in (subdir_template): Use "make -k ...".
* test/infra/gitlab-ci.yml (test-filenotify-gio):
Adapt make_params in order to get a JUnit test report.
(test-native-comp-speed0): Use "make -k ...".
* test/infra/test-jobs.yml: Regenerate.
* test/lisp/progmodes/perl-mode-tests.el (top):
Set `ert-load-file-name'.
Avoid the use of the old (eql VAL) specializer, preferring the new
(eql EXP) form.
* lisp/emacs-lisp/multisession.el (multisession-backend-value)
multisession--backend-set-value, multisession--backend-values)
multisession--backend-delete, multisession-backend-value)
multisession--backend-set-value, multisession--backend-values)
multisession--backend-delete): Quote the symbol passed to `eql` specializer.
