No longer strip positions from symbols before each use of a form, instead
relying on the low level C routines to do the right thing. Instead strip them
from miscellaneous places where this is needed. Stip them alson in
`function-put'.
Push forms onto byte-compile-form-stack and pop them "by hand" rather than by
binding the variable at each pushing, so that it will still have its data
after an error has been thrown and caught by a condition case. This gives an
source position to the ensuing error message.
* lisp/emacs-lisp/byte-run.el (byte-run--ssp-seen, byte-run--circular-list-p)
(byte-run--strip-s-p-1, byte-run-strip-symbol-positions): New functions and
variables, which together implement stripping of symbol positions. The latest
(?final) version modifies the argument in place rather than making a copy.
(function-put): Strip symbol positions from all of the arguments before doing
the `put'.
* lisp/emacs-lisp/bytecomp.el (byte-compile--form-stack): has been renamed to
byte-compile-form-stack and moved to macroexp.el.
(byte-compile-initial-macro-environment (eval-and-compile)): Replace
macroexpand-all-toplevel with macroexpand--all-toplevel.
(displaying-byte-compile-warnings): bind byte-compile-form-stack here.
(byte-compile-toplevel-file-form, byte-compile-form): Push the top level form
onto byte-compile-form-stack (whereas formally the variable was bound at each
pushing). Manually pop this from of the variable at the end of the function.
* lisp/emacs-lisp/cl-macs.el (cl-define-compiler-macro): Remove the symbol
stripping.
* lisp/emacs-lisp/comp.el (comp--native-compile): Set max-specpdl-size to at
least 5000 (previously it was 2500). Bind print-symbols-bare to t.
* lisp/emacs-lisp/macroexp.el (byte-compile-form-stack): Definition move here
from bytecomp.el for easier compilation.
(byte-compile-strip-symbol-positions and associated functions): Removed.
(macro--expand-all): push argument FORM onto byte-compile-form-stack at the
start of this function, and pop it off at the end.
(internal-macroexpand-for-load): No longer strip symbol positions. Bind
symbols-with-pos-enabled and print-symbols-bare to t.
* lisp/help.el (help--make-usage): Strip any position from argument ARG.
* src/fns.c (Fput): No longer strip symbol positions from any of the
arguments.
Fix longstanding bug due to unexpected interference via side-effect.
* lisp/emacs-lisp/cl-generic.el (cl--generic-get-dispatcher):
Copy the `dispatch` arg before storing it into the hash-table.
1. Check the type (symbol with position) of the argument given to the native
compiled version of SYMBOL_WITH_POS_SYM.
2. Handle infinite recursion caused by circular lists, etc., in
macroexp-strip-symbol-positions by using hash tables.
3. Read byte compiled functions without giving symbols positions.
* lisp/emacs-lisp/comp.el (comp-finalize-relocs): Add symbol-with-pos-p into
the list of relocated symbols.
* lisp/emacs-lisp/macroexp.el (macroexp--ssp-conses-seen)
(macroexp--ssp-vectors-seen, macroexp--ssp-records-seen): Renamed, and
animated as hash tables.
(macroexp--strip-s-p-2): Optionally tests for the presence of an argument in
one of the above hash tables, so as to handle otherwise infinite recursion.
(byte-compile-strip-s-p-1): Add a condition-case to handle infinite recursion
caused by circular lists etc., using the above hash tables as required.
* src/comp.c (comp_t): New element symbol_with_pos_sym.
(emit_SYMBOL_WITH_POS_SYM): Amend just to call the new SYMBOL_WITH_POS_SYM.
(emit_CHECK_SYMBOL_WITH_POS, define_SYMBOL_WITH_POS_SYM): New functions.
(Fcomp__init_ctxt): Register an emitter for Qsymbol_with_pos_p.
(Fcomp__compile_ctxt_to_file): Call define_SYMBOL_WITH_POS_SYM.
(syms_of_comp): Define Qsymbol_with_pos_p.
* src/data.c (syms_of_data): Define a new error symbol Qrecursion_error, an
error category for the new error symbols Qexcessive_variable_binding and
Qexcessive_lisp_nesting.
* src/eval.c (grow_specpdl): Change the signal_error call to an xsignal0 call
using the new error symbol Qexcessive_variable_binding.
(eval_sub, Ffuncall): Change the `error' calls to xsignal using the new error
symbol Qexcessive_lisp_nesting.
* src/lread.c (read1): When reading a compiled function, read the components
of the vector without giving its symbols a position.
Since the location of those files changes between build time and
installation time, this requires to tweak the file name used in those
(#$ . NNN) references during the dump so they don't hardcode the build
directory. We do it in the same way as was already done for those
same file names in `load-history`, except we convert them back to
absolute file names more lazily (i.e. when fetching the actual
docstring rather than at startup), which requires remembering the
`lisp-dir` computed at startup in the new `lisp-directory` variable.
* src/Makefile.in ($(etc)/DOC): Don't scan Lisp files any more.
* src/lread.c (Fload): Use relative file names for `load-file-name`
when preloading for the dump, like we already did for `current-load-list`.
(read_list): Don't zero-out dynamic docstring references during the
preload since they won't be filled later by Snarf-documentation any more.
(read1): Remove the hash-hack for doc references that were zeroed.
* lisp/startup.el (lisp-directory): New variable.
(command-line): Set it.
* src/doc.c (get_doc_string): Use `lisp-directory` for dynamic
docstring references using relative file names.
(syms_of_doc): Add `Qlisp_directory`.
* lib-src/make-docfile.c (scan_file): Don't handle `.el` or `.elc`
files any more.
(IS_SLASH): Remove macro, not used any more.
(skip_white, read_lisp_symbol, search_lisp_doc_at_eol)
(scan_lisp_file): Remove functions, not used any more.
* doc/lispref/loading.texi (Library Search): Mention `lisp-directory`.
Assertions should only be used to check internal consistency within a
package, not to check arguments passed by callers. Instead, define
and use a new error symbol.
* lisp/emacs-lisp/ert.el (ert-test-unbound): New error symbol.
(ert-select-tests): Use it.
* test/lisp/emacs-lisp/ert-tests.el (ert-test-select-undefined): New
unit test.
* etc/NEWS: Document new behavior.
This version of the software should bootstrap Emacs successfully with native
compilation enabled.
* lisp/emacs-lisp/bytecomp.el (byte-compile-strip-s-p-1)
(byte-compile-strip-symbol-positions): Rename and move to macroexp.el. Rename
calls to these functions throughout the file.
(byte-compile-initial-macro-environment): In the code sections for
eval-when-compile and eval-and-compile, call macroexp-strip-symbol-positions
before evaluating code.
(byte-compile-file, byte-compile-output-file-form)
(byte-compile-file-form-defmumble, byte-compile, batch-byte-compile): Call
macroexp-strip-symbol-positions from code being passed to the native compiler.
* lisp/emacs-lisp/cl-macs.el (cl-macs--strip-s-p-1)
(cl-macs--strip-symbol-positions): Remove, replacing them with the renamed
functions in macroexp.el.
(cl-define-compiler-macro): Apply macroexp-strip-symbol-positions to ARGS and
BODY.
* lisp/emacs-lisp/comp.el (comp-limplify-lap-inst): Use `null' to compile
byte-not rather than a compilation of `eq'.
(comp--native-compile): bind symbols-with-pos-enabled to t.
* lisp/emacs-lisp/macroexp.el (byte-compile--ssp-conses-seen)
(byte-compile--ssp-vectors-seen, byte-compile--ssp-records-seen): Provisional
auxiliary variables to support the following functions.
(macroexp--strip-s-p-2, byte-compile-strip-s-p-1)
(macroexp-strip-symbol-positions): Functions moved from bytecomp.el, renamed,
and further developed.
(macroexp--compiler-macro): Bind symbol-with-pos-enabled to t around the call
to `handler'.
(internal-macroexpand-for-load): Strip symbol positions from the form being
eagerly expanded for macros.
* src/comp.c (F_SYMBOLS_WITH_POS_ENABLED_RELOC_SYM): New macro for a
relocation symbol.
(comp_t): New elements bool_ptr_type, f_symbols_with_pos_enabled_ref,
lisp_symbol_with_position, lisp_symbol_with_position_header,
lisp_symbol_with_position_sym, lisp_symbol_with_position_pos,
lisp_symbol_with_position_type, lisp_symbol_with_position_ptr_type,
get_symbol_with_position.
(helper_GET_SYMBOL_WITH_POSITION): New function.
(emit_BASE_EQ): Function rename from emit_EQ.
(emit_AND, emit_OR, emit_BARE_SYMBOL_P, emit_SYMBOL_WITH_POS_P)
(emit_SYMBOL_WITH_POS_SYM): New functions.
(emit_EQ): New function which handles symbols with position correctly.
(emit_NILP): Use emit_BASE_EQ rather than emit_EQ.
(emit_limple_insn): When emitting a conditional branch, check each operand for
being a literal Qnil, and if one of them is, use emit_BASE_EQ rather than
emit_EQ.
(declare_runtime_imported_funcs): Declare helper_GET_SYMBOL_WITH_POSITION.
(emit_ctxt_code): Export the global F_SYMBOLS_WITH_POS_ENABLED_RELOC_SYM.
(define_lisp_symbol_with_position, define_GET_SYMBOL_WITH_POSITION): New
functions.
(Fcomp__init_ctxt): Initialise comp.bool_ptr_type, call the two new
define_.... functions.
(load_comp_unit): Initialise **f_symbols_with_pos_enabled_reloc.
* src/fns.c (Fput): Strip positions from symbols in PROPNAME and VALUE.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Improve
formatting of key missing in the description. Use face help-key-name
on terminals that can't display underline.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Improve display
of the keys TAB, RET, SPC, DEL, and ESC. This fixes a bug where " "
was highlighted in the description in a confusing way.
* test/lisp/emacs-lisp/rmc-tests.el
(test-rmc--add-key-description)
(test-rmc--add-key-description/with-attributes): Update tests for the
above change.
* lisp/emacs-lisp/rmc.el (rmc--show-help): Factor out new function
from read-multiple-choice.
(read-multiple-choice): Add new optional argument show-help.
* doc/lispref/commands.texi (Reading One Event): Document above new
optional argument.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Add face
property also when key is not in the name string.
* test/lisp/emacs-lisp/rmc-tests.el
(test-rmc--add-key-description/with-attributes)
(test-rmc--add-key-description/non-graphical-display): Update tests.
* lisp/emacs-lisp/rmc.el (rmc--add-key-description): Factor out
new function from...
(read-multiple-choice): ...here.
* test/lisp/emacs-lisp/rmc-tests.el (test-rmc--add-key-description)
(test-rmc--add-key-description/with-attributes)
(test-rmc--add-key-description/non-graphical-display): New tests.
* test/lisp/emacs-lisp/cconv-tests.el: Remove `:expected-result :failed`
from the bug#28557 tests.
(cconv-tests-cl-function-:documentation): Account for the presence of
the arglist (aka "usage") in the docstring.
* lisp/emacs-lisp/cl-macs.el (cl--transform-lambda):
* lisp/emacs-lisp/cl-generic.el (cl-defgeneric):
Handle non-constant `:documentation`.
* lisp/emacs-lisp/generator.el (iter-lambda):
* lisp/emacs-lisp/cconv.el (cconv--convert-funcbody):
Use `macroexp-parse-body`.
This fixes a known bug in `named-let`.
* lisp/emacs-lisp/cl-macs.el (cl--self-tco): Prevent TCO from inside
dynamic variable bindings.
* test/lisp/emacs-lisp/cl-macs-tests.el (cl-macs--labels): Add test.
This predicate can be used for discriminating between lexically and
dynamically bound variables during macro-expansion (only).
It is restricted to internal use for the time being.
* lisp/emacs-lisp/bytecomp.el (byte-compile-initial-macro-environment):
Use macroexpand--all-toplevel.
* lisp/emacs-lisp/macroexp.el (macroexp-dynamic-variable-p): New.
(macroexp--expand-all): Maintain macroexp--dynvars.
(macroexpand-all): Rebind macroexp--dynvars.
(macroexpand--all-toplevel): New.
(internal-macroexpand-for-load): Use macroexpand--all-toplevel.
* src/eval.c (eval_sub): Transfer defvar declarations from
Vinternal_interpreter_environment into macroexp--dynvars during
lazy macro-expansion.
* src/lread.c (readevalloop): Rebind macroexp--dynvars around
read-and-evaluate operations.
(syms_of_lread): Define macroexp--dynvars.
* test/lisp/emacs-lisp/macroexp-resources/vk.el: New file.
* test/lisp/emacs-lisp/macroexp-tests.el (macroexp-tests--run-emacs)
(macroexp-tests--eval-in-subprocess)
(macroexp-tests--byte-compile-in-subprocess)
(macroexp--tests-dynamic-variable-p): Add tests.
* lisp/emacs-lisp/byte-opt.el (side-effect-free-fns, pure-fns):
Mark base64-decode-string, base64-encode-string and
base64url-encode-string as pure and side-effect-free.
This optimisation is of very limited utility and miscompiles top-level
code having the form (eval 'CODE t) by replacing it with CODE which
will then, as things currently stand, be evaluated with dynamic binding.
* lisp/emacs-lisp/bytecomp.el (byte-compile-file-form-eval): Remove.
* lisp/emacs-lisp/ert.el (ert-load-file-name): New defvar.
(ert-write-junit-test-report): Use it. Make detection of selector
more robust. Protect calls of `xml-escape-string' with 'noerror.
* test/infra/Makefile.in (subdir_template): Use "make -k ...".
* test/infra/gitlab-ci.yml (test-filenotify-gio):
Adapt make_params in order to get a JUnit test report.
(test-native-comp-speed0): Use "make -k ...".
* test/infra/test-jobs.yml: Regenerate.
* test/lisp/progmodes/perl-mode-tests.el (top):
Set `ert-load-file-name'.
Avoid the use of the old (eql VAL) specializer, preferring the new
(eql EXP) form.
* lisp/emacs-lisp/multisession.el (multisession-backend-value)
multisession--backend-set-value, multisession--backend-values)
multisession--backend-delete, multisession-backend-value)
multisession--backend-set-value, multisession--backend-values)
multisession--backend-delete): Quote the symbol passed to `eql` specializer.
