Since the location of those files changes between build time and
installation time, this requires to tweak the file name used in those
(#$ . NNN) references during the dump so they don't hardcode the build
directory. We do it in the same way as was already done for those
same file names in `load-history`, except we convert them back to
absolute file names more lazily (i.e. when fetching the actual
docstring rather than at startup), which requires remembering the
`lisp-dir` computed at startup in the new `lisp-directory` variable.
* src/Makefile.in ($(etc)/DOC): Don't scan Lisp files any more.
* src/lread.c (Fload): Use relative file names for `load-file-name`
when preloading for the dump, like we already did for `current-load-list`.
(read_list): Don't zero-out dynamic docstring references during the
preload since they won't be filled later by Snarf-documentation any more.
(read1): Remove the hash-hack for doc references that were zeroed.
* lisp/startup.el (lisp-directory): New variable.
(command-line): Set it.
* src/doc.c (get_doc_string): Use `lisp-directory` for dynamic
docstring references using relative file names.
(syms_of_doc): Add `Qlisp_directory`.
* lib-src/make-docfile.c (scan_file): Don't handle `.el` or `.elc`
files any more.
(IS_SLASH): Remove macro, not used any more.
(skip_white, read_lisp_symbol, search_lisp_doc_at_eol)
(scan_lisp_file): Remove functions, not used any more.
* doc/lispref/loading.texi (Library Search): Mention `lisp-directory`.
This version of the software should bootstrap Emacs successfully with native
compilation enabled.
* lisp/emacs-lisp/bytecomp.el (byte-compile-strip-s-p-1)
(byte-compile-strip-symbol-positions): Rename and move to macroexp.el. Rename
calls to these functions throughout the file.
(byte-compile-initial-macro-environment): In the code sections for
eval-when-compile and eval-and-compile, call macroexp-strip-symbol-positions
before evaluating code.
(byte-compile-file, byte-compile-output-file-form)
(byte-compile-file-form-defmumble, byte-compile, batch-byte-compile): Call
macroexp-strip-symbol-positions from code being passed to the native compiler.
* lisp/emacs-lisp/cl-macs.el (cl-macs--strip-s-p-1)
(cl-macs--strip-symbol-positions): Remove, replacing them with the renamed
functions in macroexp.el.
(cl-define-compiler-macro): Apply macroexp-strip-symbol-positions to ARGS and
BODY.
* lisp/emacs-lisp/comp.el (comp-limplify-lap-inst): Use `null' to compile
byte-not rather than a compilation of `eq'.
(comp--native-compile): bind symbols-with-pos-enabled to t.
* lisp/emacs-lisp/macroexp.el (byte-compile--ssp-conses-seen)
(byte-compile--ssp-vectors-seen, byte-compile--ssp-records-seen): Provisional
auxiliary variables to support the following functions.
(macroexp--strip-s-p-2, byte-compile-strip-s-p-1)
(macroexp-strip-symbol-positions): Functions moved from bytecomp.el, renamed,
and further developed.
(macroexp--compiler-macro): Bind symbol-with-pos-enabled to t around the call
to `handler'.
(internal-macroexpand-for-load): Strip symbol positions from the form being
eagerly expanded for macros.
* src/comp.c (F_SYMBOLS_WITH_POS_ENABLED_RELOC_SYM): New macro for a
relocation symbol.
(comp_t): New elements bool_ptr_type, f_symbols_with_pos_enabled_ref,
lisp_symbol_with_position, lisp_symbol_with_position_header,
lisp_symbol_with_position_sym, lisp_symbol_with_position_pos,
lisp_symbol_with_position_type, lisp_symbol_with_position_ptr_type,
get_symbol_with_position.
(helper_GET_SYMBOL_WITH_POSITION): New function.
(emit_BASE_EQ): Function rename from emit_EQ.
(emit_AND, emit_OR, emit_BARE_SYMBOL_P, emit_SYMBOL_WITH_POS_P)
(emit_SYMBOL_WITH_POS_SYM): New functions.
(emit_EQ): New function which handles symbols with position correctly.
(emit_NILP): Use emit_BASE_EQ rather than emit_EQ.
(emit_limple_insn): When emitting a conditional branch, check each operand for
being a literal Qnil, and if one of them is, use emit_BASE_EQ rather than
emit_EQ.
(declare_runtime_imported_funcs): Declare helper_GET_SYMBOL_WITH_POSITION.
(emit_ctxt_code): Export the global F_SYMBOLS_WITH_POS_ENABLED_RELOC_SYM.
(define_lisp_symbol_with_position, define_GET_SYMBOL_WITH_POSITION): New
functions.
(Fcomp__init_ctxt): Initialise comp.bool_ptr_type, call the two new
define_.... functions.
(load_comp_unit): Initialise **f_symbols_with_pos_enabled_reloc.
* src/fns.c (Fput): Strip positions from symbols in PROPNAME and VALUE.
This predicate can be used for discriminating between lexically and
dynamically bound variables during macro-expansion (only).
It is restricted to internal use for the time being.
* lisp/emacs-lisp/bytecomp.el (byte-compile-initial-macro-environment):
Use macroexpand--all-toplevel.
* lisp/emacs-lisp/macroexp.el (macroexp-dynamic-variable-p): New.
(macroexp--expand-all): Maintain macroexp--dynvars.
(macroexpand-all): Rebind macroexp--dynvars.
(macroexpand--all-toplevel): New.
(internal-macroexpand-for-load): Use macroexpand--all-toplevel.
* src/eval.c (eval_sub): Transfer defvar declarations from
Vinternal_interpreter_environment into macroexp--dynvars during
lazy macro-expansion.
* src/lread.c (readevalloop): Rebind macroexp--dynvars around
read-and-evaluate operations.
(syms_of_lread): Define macroexp--dynvars.
* test/lisp/emacs-lisp/macroexp-resources/vk.el: New file.
* test/lisp/emacs-lisp/macroexp-tests.el (macroexp-tests--run-emacs)
(macroexp-tests--eval-in-subprocess)
(macroexp-tests--byte-compile-in-subprocess)
(macroexp--tests-dynamic-variable-p): Add tests.
This optimisation is of very limited utility and miscompiles top-level
code having the form (eval 'CODE t) by replacing it with CODE which
will then, as things currently stand, be evaluated with dynamic binding.
* lisp/emacs-lisp/bytecomp.el (byte-compile-file-form-eval): Remove.
The position return by read-positioning-symbols is now the position in the
buffer, rather than the offset from the start of a form, enabling warning
positions in other parts of the buffer to be output.
* src/lisp.h (lisp_h_EQ): Add XLI casts so that it compiles cleanly.
* src/data.c (Fremove_pos_from_symbol): New DEFUN.
* src/lread.c (readchar_count): renamed to readchar_offset.
(read_internal_start) Initialize readchar_offset to the buffer's point when
STREAM is a buffer.
* lisp/emacs-lisp/bytecomp.el (byte-compile-warning-prefix): Amend to use
OFFSET as a buffer position, not an offset from the start of a form.
(byte-compile-warn): Remove symbol positions from any shape of ARGS, not just
a symbol with position.
* lisp/emacs-lisp/cconv.c (cconv-convert): In the :unused case, position the
new IGNORE symbol with the VAR it has replaced.
* lisp/emacs-lisp/macroexp.el (macroexp--warn-wrap, macroexp-warn-and-return):
Add an extra position parameter to each.
* lisp/emacs-lisp/bindat.el (bindat-type), lisp/emacs-lisp/byte-run.el
(defmacro, defun), lisp/emacs-lisp/cconv.el (cconv--convert-func-body)
(cconv-convert), lisp/emacs-lisp/cl-generic.el (cl-defmethod),
lisp/emacs-lisp/cl-macs.el (cl-symbol-macrolet, cl-defstruct),
lisp/emacs-lisp/easy-mmode.el (define-minor-mode),
lisp/emacs-lisp/eieio-core.el (eieio-oref, eieio-oref-default)
(eieio-oset-default), lisp/emacs-lisp/eieio.el (defclass),
lisp/emacs-lisp/gv.el (gv-ref), lisp/emacs-lisp/macroexp.el
(macroexp-macroexpand, macroexp--unfold-lambda, macroexp--expand-all),
lisp/emacs-lisp/pcase.el (pcase-compile-patterns, pcase--u1): Add an extra
position argument to each call of macroexp-warn-and-return.
This branch is intended to generate correct position information in warning
and error messages from the byte compiler, and is intended thereby to fix bugs
It introduces a new mechanism, the symbol with position. This is taken over
from the previous git branch scratch/accurate-warning-pos which was abandoned
for being too slow. The main difference in the current branch is that the
symbol `nil' is never given a position, thus speeding up NILP markedly.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand)
(byte-optimize-form-code-walker, byte-optimize-let-form, byte-optimize-while)
(byte-optimize-apply): Use byte-compile-warn-x in place of byte-compile-warn.
* lisp/emacs-lisp/bytecomp.el (byte-compile--form-stack): New variable.
(byte-compile-strip-s-p-1, byte-compile-strip-symbol-positions): New
functions.
(byte-compile-recurse-toplevel, byte-compile-initial-macro-environment)
(byte-compile-preprocess, byte-compile-macroexpand-declare-function): Bind
print-symbols-bare to non-nil.
(byte-compile--first-symbol, byte-compile--warning-source-offset): New
functions.
(byte-compile-warning-prefix): Modify to output two sets of position
information, the old (incorrect) set and the new set.
(byte-compile-warn): Strip positions from symbols before outputting.
(byte-compile-warn-x): New function which outputs a correct position supplied
in an argument.
(byte-compile-warn-obsolete, byte-compile-emit-callargs-warn)
(byte-compile-format-warn, byte-compile-nogroup-warn)
(byte-compile-arglist-warn, byte-compile-docstring-length-warn)
(byte-compile-warn-about-unresolved-functions, byte-compile-file)
(byte-compile--check-prefixed-var, byte-compile--declare-var)
(byte-compile-file-form-defvar-function, byte-compile-file-form-defmumble)
(byte-compile-check-lambda-list, byte-compile--warn-lexical-dynamic)
(byte-compile-lambda, byte-compile-form, byte-compile-normal-call)
(byte-compile-check-variable, byte-compile-free-vars-warn)
(byte-compile-subr-wrong-args, byte-compile-fset, byte-compile-set-default)
(byte-compile-condition-case, byte-compile-save-excursion)
(byte-compile-defvar, byte-compile-autoload)
(byte-compile-make-variable-buffer-local, byte-compile-define-symbol-prop)
(byte-compile-define-keymap): Replace byte-compile-warn with
byte-compile-warn-x.
(byte-compile-file, compile-defun): Bind symbols-with-pos-enabled to non-nil.
(compile-defun, byte-compile-from-buffer): Use `read-positioning-symbols'
rather than plain `read'.
(byte-compile-toplevel-file-form, byte-compile-form): Dynamically bind
byte-compile--form-stack.
(byte-compile-file-form-autoload, byte-compile-file-form-defvar)
(byte-compile-file-form-make-obsolete, byte-compile-lambda)
(byte-compile-push-constant, byte-compile-cond-jump-table)
(byte-compile-define-keymap, byte-compile-annotate-call-tree):
Strip positions from symbols where they are unwanted.
(byte-compile-file-form-defvar): Strip positions from symbols using
`bare-symbol'.
(byte-compile-file-form-defmumble): New variable bare-name, a version of name
without its position.
(byte-compile-lambda): Similarly, new variable bare-arglist.
(byte-compile-free-vars-warn): New argument arg supplying position information
to byte-compile-warn-x.
(byte-compile-push-constant): Manipulation of symbol positions.
(display-call-tree): Strip positions from symbols.
* lisp/emacs-lisp/cconv.el (cconv-convert, cconv--analyze-use)
(cconv--analyze-function, cconv-analyze-form): Replace use of
byte-compile-warn with byte-compile-warn-x.
* lisp/emacs-lisp/cl-generic.el (cl-defmethod): New variable org-name which
will supply position information to a new macroexp-warn-and-return.
* lisp/emacs-lisp/cl-macs.el (cl-macs--strip-s-p-1)
(cl-macs--strip-symbol-positions): New functions to strip positions from
symbols in an expression. These duplicaate similarly named functions in
bytecomp.el.
* lisp/emacs-lisp/macroexpand.el (macroexp--warn-wrap): Calls
byte-compile-warn-x in place of byte-compile-warn.
(macroexp-warn-and-return): Commented out new position parameter _arg.
* src/.gdbinit: Add in code to handle symbols with position.
* src/alloc.c (XPNTR, set_symbol_name, valid_lisp_object_p, purecopy)
(mark_char_table, mark_object, survives_gc_p, symbol_uses_obj): Use
BARE_SYMBOL_P and XBARE_SYMBOL in place of the former SYMBOLP and XSYMBOL.
(build_symbol_with_pos): New function.
(Fgarbage_collect): Bind Qsymbols_with_pos_enabled to nil around the call to
garbage_collect.
* src/data.c (Ftype_of): Add case for PVEC_SYMBOL_WITH_POS.
(Fbare_symbol_p, Fsymbol_with_pos_p, Fbare_symbol, Fsymbol_with_pos_pos)
(Fposition_symbol): New functions.
(symbols_with_pos_enabled): New boolean variable.
* src/fns.c (internal_equal, hash_lookup): Handle symbols with position.
* src/keyboard.c (recursive_edit_1): Bind Qsymbols_with_pos_enabled and
Qprint_symbols_bare to nil.
* src/lisp.h (lisp_h_PSEUDOVECTORP): New macro.
(lisp_h_BASE_EQ): New name for the former lisp_h_EQ.
(lisp_h_EQ): Extended to handle symbols with position.
(lisp_h_NILP): Now uses BASE_EQ rather than EQ.
(lisp_h_SYMBOL_WITH_POS_P, lisp_h_BARE_SYMBOL_P): New macros.
(lisp_h_SYMBOLP): Redefined to handle symbols with position.
(BARE_SYMBOL_P, BASE_EQ): New macros.
(SYMBOLP (macro)): Removed.
(SYMBOLP (function), XSYMBOL, make_lisp_symbol, builtin_lisp_symbol)
(c_symbol_p): Moved to later in file.
(struct Lisp_Symbol_With_Pos): New data type.
(pvec_type): PVEC_SYMBOL_WITH_POS: New type code.
(PSEUDOVECTORP): Redefined to use the lisp_h_PSEUDOVECTORP.
(BARE_SYMBOL_P, SYMBOL_WITH_POS_P, SYMBOLP, XSYMBOL_WITH_POS, XBARE_SYMBOL)
(XSYMBOL, make_lisp_symbol, builtin_lisp_symbol, c_symbol_p, CHECK_SYMBOL)
(BASE_EQ): New functions, or functions moved from earlier in the file.
(SYMBOL_WITH_POS_SYM, SYMBOL_WITH_POS_POS): New INLINE functions.
* src/lread.c (read0, read1, read_list, read_vector, read_internal_start)
(list2): Add a new bool parameter locate_syms.
(Fread_positioning_symbols): New function.
(Fread_from_string, read_internal_start, read0, read1, read_list): Pass around
suitable values for locate_syms.
(read1): Build symbols with position when locate_syms is true.
* src/print.c (print_vectorlike): Add handling for PVEC_SYMBOL_WITH_POS.
(print_object): Replace EQ with BASE_EQ.
(print_symbols_bare): New boolean variable.
* lisp/keymap.el (keymap-substitute): Make the keymap the first
parameter for symmetry with the other functions.
* lisp/emacs-lisp/shortdoc.el (keymaps):
* lisp/emacs-lisp/bytecomp.el (lambda): Adjust.
* lisp/keymap.el: New file with all the new keymap-* functions.
* lisp/loadup.el ("keymap"): Load.
* lisp/subr.el (kbd): Refactor out all the code to key-parse.
(define-key-after, keyboard-translate, global-set-key)
(local-set-key, global-unset-key, local-unset-key)
(local-key-binding, global-key-binding)
(substitute-key-definition): Note in doc strings that these are
legacy functions.
(define-keymap--define): Use keymap-set.
* lisp/emacs-lisp/byte-opt.el: Remove the optimizations for
defvar-keymap and define-keymap since the macros now only
understand the kbd syntax.
* lisp/emacs-lisp/bytecomp.el (byte-compile-define-keymap)
(byte-compile-define-keymap--define): Warn about invalid key
definitions in all keymap-* functions.
* lisp/emacs-lisp/shortdoc.el (keymaps): Add shortdocs form
keymap* functions.
* src/keymap.c (possibly_translate_key_sequence): Adjust callers
to key-valid-p and key-parse.
(syms_of_keymap): Adjust defs.
* lisp/emacs-lisp/bytecomp.el (byte-compile-warning-types): Allow
inhibiting the `not-unused' warning (bug#31641). (There has been
some discussion about removing the `not-unused' warning, but it's
still in there, so making it possible to inhibit it seems like the
right thing to do.)
* lisp/emacs-lisp/cconv.el (cconv--analyze-use): Don't warn about
`not-unused'.
* lisp/emacs-lisp/bytecomp.el
(byte-compile-docstring-length-warn): Warn about overly long
docstring in lambda. (Bug#44858)
(byte-compile--wide-docstring-p): Improve comment.
* test/lisp/emacs-lisp/bytecomp-tests.el
("warn-wide-docstring-defun.el"): Update to test for the above new
warning.
* lisp/emacs-lisp/bytecomp.el (byte-compile--wide-docstring-p):
Ignore more function argument lists.
* test/lisp/emacs-lisp/bytecomp-tests.el
(bytecomp-tests-byte-compile--wide-docstring-p): New test.
Allowing &rest without a variable name following turned out not to be
very useful, and it never worked properly. Disallow it.
* lisp/emacs-lisp/bytecomp.el (byte-compile-check-lambda-list):
* src/eval.c (funcall_lambda):
Signal error for &rest without variable name.
* doc/lispref/functions.texi (Argument List): Adjust manual.
* etc/NEWS (file): Announce.
* test/src/eval-tests.el (eval-tests--bugs-24912-and-24913):
Extend test, also checking with and without lexical binding.
(eval-tests-accept-empty-optional-rest): Reduce to...
(eval-tests-accept-empty-optional): ...this, again checking
with and without lexical binding.
* lisp/emacs-lisp/generator.el (iter-defun):
* lisp/emacs-lisp/comp.el (comp-clean-up-stale-eln):
* lisp/emacs-lisp/bytecomp.el (byte-compile-lapcode):
* lisp/emacs-lisp/autoload.el (autoload-insert-section-header):
Include description of arguments in the doc string's first line.
* lisp/emacs-lisp/comp.el ('native-comp-compiler-options): New option.
* lisp/emacs-lisp/bytecomp.el (byte-compile-from-buffer): Add support
for new 'native-comp-compiler-options'.
* src/comp.c (Fcomp_native_compiler_options_effective_p): New function.
(add_compiler_options): New function.
(Fcomp__compile_ctxt_to_file): Call 'add_compiler_options'.
Treat `ignore` as any other function during source-level optimisation,
to avoid having its warning-suppression effects cancelled by repeated
passes. Instead, define a custom code generation function.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-form-code-walker):
Don't treat `ignore' specially here.
(side-effect-free-fns): Don't mark `ignore` as side-effect-free
or error-free (although it is), since that would allow the optimiser
to elide calls.
* lisp/emacs-lisp/bytecomp.el (ignore, byte-compile-ignore):
Define and register a code-gen function.
Make the optimiser aware of lexical arguments. Otherwise we cannot
know for sure whether a variable is lexical or dynamic during
traversal.
* lisp/emacs-lisp/byte-opt.el (byte-optimize-one-form): New optimiser
entry point, replacing the recursive byte-optimize-form.
* lisp/emacs-lisp/bytecomp.el (byte-optimize-one-form): Autoload.
(byte-compile-keep-pending, byte-compile-top-level):
Use byte-optimize-one-form.
Reported by Gregor Zattler.
* lisp/emacs-lisp/bytecomp.el (byte-compile--cond-switch-prefix):
Be more careful in the selection of equality.
* test/lisp/emacs-lisp/bytecomp-tests.el (bytecomp-tests--test-cases):
Add test case.
Wrong number of arguments in inlining function calls (to `defsubst` or
explicitly using `inline`) did not result in warnings, or in very
cryptic ones.
* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand): Add calls
to `byte-compile--check-arity-bytecode`.
* lisp/emacs-lisp/bytecomp.el (byte-compile-emit-callargs-warn)
(byte-compile--check-arity-bytecode): New functions.
(byte-compile-callargs-warn): Use factored-out function.
* test/lisp/emacs-lisp/bytecomp-resources/warn-callargs-defsubst.el:
* test/lisp/emacs-lisp/bytecomp-tests.el ("warn-callargs-defsubst.el"):
New test case.
* lisp/emacs-lisp/bytecomp.el (byte-compile--cond-switch-prefix):
Treat (not VAR) and (null VAR) as (eq VAR nil) when computing the
extent of switch ops.
* lisp/emacs-lisp/bytecomp.el (byte-compile--wide-docstring-p):
Avoid constructing an invalid regexp during byte-compilation by
limiting the number of columns to the current RE_DUP_MAX of 65535.
This protects against pathological values of fill-column, for
example (bug#49426).
* lisp/emacs-lisp/bytecomp.el (byte-compile--wide-docstring-p):
Don't consider the function signature when determining whether the
doc string is too wide (bug#49007). (The signature is folded
later when displaying help.)
* lisp/emacs-lisp/bytecomp.el (byte-compile-file): Produce .elc
temporary files only when non native compiling or when native
compiling but `byte+native-compile' is non nil.
As the Info node `(elisp) Specification List' explains, it is not
correct to use `body' or t for a piece of code that the macro wraps in
a `lambda' form. These should use `def-body' instead.
* lisp/info-xref.el (info-xref-with-file):
* lisp/subr.el (subr--with-wrapper-hook-no-warnings, track-mouse)
(combine-change-calls, with-eval-after-load):
* lisp/emacs-lisp/bytecomp.el (displaying-byte-compile-warnings):
* lisp/emacs-lisp/cl-macs.el (cl-do-symbols, cl-progv):
* lisp/emacs-lisp/ert-x.el (ert-with-test-buffer):
* lisp/emacs-lisp/gv.el (gv-letplace):
* lisp/emacs-lisp/nadvice.el (define-advice):
* lisp/emacs-lisp/thunk.el (thunk-delay):
* lisp/vc/vc-dispatcher.el (vc-run-delayed): Use 'def-body' instead of
t or 'body' where applicable.
* lisp/emacs-lisp/package.el (package--with-response-buffer): Remove
evaluation of the body altogether. I have no idea how to write it
correctly in this case.
