procyberian/emacs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Eliminate lazy bytecode loading

Browse source 
The obsolete lazy-loaded bytecode feature, enabled by
`byte-compile-dynamic`, slows down Lisp execution even when not in use
because every call to a bytecode function has to check that function
for laziness.

This change forces up-front loading of all lazy bytecode so that we
can remove all those checks.  (Dynamically loaded doc strings are not
affected.)

There is no point in generating lazy bytecode any more so we stop
doing that; this simplifies the compiler.  `byte-compile-dynamic` now
has no effect.

This is a fully compatible change; the few remaining users of
`byte-compile-dynamic` should not notice any difference.

* src/lread.c (bytecode_from_rev_list): Force eager loading of
lazy bytecode.
* src/bytecode.c (exec_byte_code): Remove lazy bytecode checks.
* src/eval.c (fetch_and_exec_byte_code, Ffetch_bytecode): Remove.
(funcall_lambda): Call exec_byte_code directly, avoiding checks.
* lisp/subr.el (fetch-bytecode): New definition, obsolete no-op.
* lisp/emacs-lisp/disass.el (disassemble-1):
* lisp/emacs-lisp/bytecomp.el (byte-compile-unfold-bcf):
Remove calls to fetch-bytecode.
(byte-compile-dynamic): Update doc string.
(byte-compile-close-variables, byte-compile-from-buffer)
(byte-compile-insert-header, byte-compile-output-file-form)
(byte-compile--output-docform-recurse, byte-compile-output-docform)
(byte-compile-file-form-defmumble):
Remove effects of byte-compile-dynamic.
* doc/lispref/compile.texi (Dynamic Loading): Remove node now that
the entire `byte-compile-dynamic` facility has been rendered inert.
* etc/NEWS: Announce changes.
This commit is contained in:
Mattias Engdegård 2024-01-30 17:55:19 +01:00
parent 7e85311a91
commit 9bcc9690a8
9 changed files with 58 additions and 221 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

66
doc/lispref/compile.texi
View file

@ -35,7 +35,6 @@ variable binding for @code{no-byte-compile} into it, like this:
* Speed of Byte-Code:: An example of speedup from byte compilation.
* Compilation Functions:: Byte compilation functions.
* Docs and Compilation:: Dynamic loading of documentation strings.
* Dynamic Loading:: Dynamic loading of individual functions.
* Eval During Compile:: Code to be evaluated when you compile.
* Compiler Errors:: Handling compiler error messages.
* Byte-Code Objects:: The data type used for byte-compiled functions.
@ -289,71 +288,6 @@ stands for the name of this file, as a string. Do not use these
constructs in Lisp source files; they are not designed to be clear to
humans reading the file.
@node Dynamic Loading
@section Dynamic Loading of Individual Functions
@cindex dynamic loading of functions
@cindex lazy loading
When you compile a file, you can optionally enable the @dfn{dynamic
function loading} feature (also known as @dfn{lazy loading}). With
dynamic function loading, loading the file doesn't fully read the
function definitions in the file. Instead, each function definition
contains a place-holder which refers to the file. The first time each
function is called, it reads the full definition from the file, to
replace the place-holder.
The advantage of dynamic function loading is that loading the file
should become faster. This is a good thing for a file which contains
many separate user-callable functions, if using one of them does not
imply you will probably also use the rest. A specialized mode which
provides many keyboard commands often has that usage pattern: a user may
invoke the mode, but use only a few of the commands it provides.
The dynamic loading feature has certain disadvantages:
@itemize @bullet
@item
If you delete or move the compiled file after loading it, Emacs can no
longer load the remaining function definitions not already loaded.
@item
If you alter the compiled file (such as by compiling a new version),
then trying to load any function not already loaded will usually yield
nonsense results.
@end itemize
These problems will never happen in normal circumstances with
installed Emacs files. But they are quite likely to happen with Lisp
files that you are changing. The easiest way to prevent these problems
is to reload the new compiled file immediately after each recompilation.
@emph{Experience shows that using dynamic function loading provides
benefits that are hardly measurable, so this feature is deprecated
since Emacs 27.1.}
The byte compiler uses the dynamic function loading feature if the
variable @code{byte-compile-dynamic} is non-@code{nil} at compilation
time. Do not set this variable globally, since dynamic loading is
desirable only for certain files. Instead, enable the feature for
specific source files with file-local variable bindings. For example,
you could do it by writing this text in the source file's first line:
@example
-*-byte-compile-dynamic: t;-*-
@end example
@defvar byte-compile-dynamic
If this is non-@code{nil}, the byte compiler generates compiled files
that are set up for dynamic function loading.
@end defvar
@defun fetch-bytecode function
If @var{function} is a byte-code function object, this immediately
finishes loading the byte code of @var{function} from its
byte-compiled file, if it is not fully loaded already. Otherwise,
it does nothing. It always returns @var{function}.
@end defun
@node Eval During Compile
@section Evaluation During Compilation
@cindex eval during compilation

1
doc/lispref/elisp.texi
View file

@ -653,7 +653,6 @@ Byte Compilation
* Speed of Byte-Code:: An example of speedup from byte compilation.
* Compilation Functions:: Byte compilation functions.
* Docs and Compilation:: Dynamic loading of documentation strings.
* Dynamic Loading:: Dynamic loading of individual functions.
* Eval During Compile:: Code to be evaluated when you compile.
* Compiler Errors:: Handling compiler error messages.
* Byte-Code Objects:: The data type used for byte-compiled functions.

7
etc/NEWS
View file

@ -1846,6 +1846,13 @@ The declaration '(important-return-value t)' sets the
'important-return-value' property which indicates that the function
return value should probably not be thrown away implicitly.
** Bytecode is now always loaded eagerly.
Bytecode compiled with older Emacs versions for lazy loading using
'byte-compile-dynamic' is now loaded all at once.
As a consequence, 'fetch-bytecode' has no use, does nothing, and is
now obsolete. The variable 'byte-compile-dynamic' has no effect any
more; compilation will always yield bytecode for eager loading.
+++
** New functions 'file-user-uid' and 'file-group-gid'.
These functions are like 'user-uid' and 'group-gid', respectively, but

66
lisp/emacs-lisp/bytecomp.el
View file

@ -231,17 +231,8 @@ This includes variable references and calls to functions such as `car'."
:type 'boolean)
(defvar byte-compile-dynamic nil
"If non-nil, compile function bodies so they load lazily.
They are hidden in comments in the compiled file,
and each one is brought into core when the
function is called.
To enable this option, make it a file-local variable
in the source file you want it to apply to.
For example, add -*-byte-compile-dynamic: t;-*- on the first line.
When this option is true, if you load the compiled file and then move it,
the functions you loaded will not be able to run.")
"Formerly used to compile function bodies so they load lazily.
This variable no longer has any effect.")
(make-obsolete-variable 'byte-compile-dynamic "not worthwhile any more." "27.1")
;;;###autoload(put 'byte-compile-dynamic 'safe-local-variable 'booleanp)
@ -1858,7 +1849,6 @@ It is too wide if it has any lines longer than the largest of
;;
(byte-compile-verbose byte-compile-verbose)
(byte-optimize byte-optimize)
(byte-compile-dynamic byte-compile-dynamic)
(byte-compile-dynamic-docstrings
byte-compile-dynamic-docstrings)
(byte-compile-warnings byte-compile-warnings)
@ -2428,8 +2418,7 @@ With argument ARG, insert value in current buffer after the form."
(defun byte-compile-insert-header (_filename outbuffer)
"Insert a header at the start of OUTBUFFER.
Call from the source buffer."
(let ((dynamic byte-compile-dynamic)
(optimize byte-optimize))
(let ((optimize byte-optimize))
(with-current-buffer outbuffer
(goto-char (point-min))
;; The magic number of .elc files is ";ELC", or 0x3B454C43. After
@ -2463,10 +2452,7 @@ Call from the source buffer."
((eq optimize 'byte) " byte-level optimization only")
(optimize " all optimizations")
(t "out optimization"))
".\n"
(if dynamic ";;; Function definitions are lazy-loaded.\n"
"")
"\n\n"))))
".\n\n\n"))))
(defun byte-compile-output-file-form (form)
;; Write the given form to the output buffer, being careful of docstrings
@ -2487,7 +2473,7 @@ Call from the source buffer."
(print-circle t)) ; Handle circular data structures.
(if (memq (car-safe form) '(defvar defvaralias defconst
autoload custom-declare-variable))
(byte-compile-output-docform nil nil nil '("\n(" ")") form nil 3 nil
(byte-compile-output-docform nil nil nil '("\n(" ")") form nil 3
(memq (car form)
'(defvaralias autoload
custom-declare-variable)))
@ -2498,15 +2484,11 @@ Call from the source buffer."
(defvar byte-compile--for-effect)
(defun byte-compile--output-docform-recurse
(info position form cvecindex docindex specindex quoted)
(info position form cvecindex docindex quoted)
"Print a form with a doc string. INFO is (prefix postfix).
POSITION is where the next doc string is to be inserted.
CVECINDEX is the index in the FORM of the constant vector, or nil.
DOCINDEX is the index of the doc string (or nil) in the FORM.
If SPECINDEX is non-nil, it is the index in FORM
of the function bytecode string. In that case,
we output that argument and the following argument
\(the constants vector) together, for lazy loading.
QUOTED says that we have to put a quote before the
list that represents a doc string reference.
`defvaralias', `autoload' and `custom-declare-variable' need that.
@ -2529,29 +2511,7 @@ Return the position after any inserted docstrings as comments."
(while (setq form (cdr form))
(setq index (1+ index))
(insert " ")
(cond ((and (numberp specindex) (= index specindex)
;; Don't handle the definition dynamically
;; if it refers (or might refer)
;; to objects already output
;; (for instance, gensyms in the arg list).
(let (non-nil)
(when (hash-table-p print-number-table)
(maphash (lambda (_k v) (if v (setq non-nil t)))
print-number-table))
(not non-nil)))
;; Output the byte code and constants specially
;; for lazy dynamic loading.
(goto-char position)
(let ((lazy-position (byte-compile-output-as-comment
(cons (car form) (nth 1 form))
t)))
(setq position (point))
(goto-char (point-max))
(princ (format "(#$ . %d) nil" lazy-position)
byte-compile--outbuffer)
(setq form (cdr form))
(setq index (1+ index))))
((eq index cvecindex)
(cond ((eq index cvecindex)
(let* ((cvec (car form))
(len (length cvec))
(index2 0)
@ -2564,7 +2524,7 @@ Return the position after any inserted docstrings as comments."
(byte-compile--output-docform-recurse
'("#[" "]") position
(append elt nil) ; Convert the vector to a list.
2 4 specindex nil))
2 4 nil))
(prin1 elt byte-compile--outbuffer))
(setq index2 (1+ index2))
(unless (eq index2 len)
@ -2590,16 +2550,12 @@ Return the position after any inserted docstrings as comments."
(defun byte-compile-output-docform (preface tailpiece name info form
cvecindex docindex
specindex quoted)
quoted)
"Print a form with a doc string. INFO is (prefix postfix).
If PREFACE, NAME, and TAILPIECE are non-nil, print them too,
before/after INFO and the FORM but after the doc string itself.
CVECINDEX is the index in the FORM of the constant vector, or nil.
DOCINDEX is the index of the doc string (or nil) in the FORM.
If SPECINDEX is non-nil, it is the index in FORM
of the function bytecode string. In that case,
we output that argument and the following argument
\(the constants vector) together, for lazy loading.
QUOTED says that we have to put a quote before the
list that represents a doc string reference.
`defvaralias', `autoload' and `custom-declare-variable' need that."
@ -2627,7 +2583,7 @@ list that represents a doc string reference.
(insert preface)
(prin1 name byte-compile--outbuffer))
(byte-compile--output-docform-recurse
info position form cvecindex docindex specindex quoted)
info position form cvecindex docindex quoted)
(when tailpiece
(insert tailpiece))))))
@ -2971,7 +2927,6 @@ not to take responsibility for the actual compilation of the code."
(if macro '(" '(macro . #[" "])") '(" #[" "]"))
(append code nil) ; Turn byte-code-function-p into list.
2 4
(and (atom code) byte-compile-dynamic 1)
nil)
t)))))
@ -3810,7 +3765,6 @@ lambda-expression."
(alen (length (cdr form)))
(dynbinds ())
lap)
(fetch-bytecode fun)
(setq lap (byte-decompile-bytecode-1 (aref fun 1) (aref fun 2) t))
;; optimized switch bytecode makes it impossible to guess the correct
;; `byte-compile-depth', which can result in incorrect inlined code.

2
lisp/emacs-lisp/disass.el
View file

@ -191,8 +191,6 @@ OBJ should be a call to BYTE-CODE generated by the byte compiler."
(if (consp obj)
(setq bytes (car (cdr obj)) ;the byte code
constvec (car (cdr (cdr obj)))) ;constant vector
;; If it is lazy-loaded, load it now
(fetch-bytecode obj)
(setq bytes (aref obj 1)
constvec (aref obj 2)))
(cl-assert (not (multibyte-string-p bytes)))

2
lisp/subr.el
View file

@ -2023,6 +2023,8 @@ instead; it will indirectly limit the specpdl stack size as well.")
(defvaralias 'native-comp-deferred-compilation 'native-comp-jit-compilation)
(define-obsolete-function-alias 'fetch-bytecode #'ignore "30.1")
;;;; Alternate names for functions - these are not being phased out.

27
src/bytecode.c
View file

@ -792,22 +792,19 @@ exec_byte_code (Lisp_Object fun, ptrdiff_t args_template,
Lisp_Object original_fun = call_fun;
if (SYMBOLP (call_fun))
call_fun = XSYMBOL (call_fun)->u.s.function;
Lisp_Object template;
Lisp_Object bytecode;
if (COMPILEDP (call_fun)
/* Lexical binding only. */
&& (template = AREF (call_fun, COMPILED_ARGLIST),
FIXNUMP (template))
/* No autoloads. */
&& (bytecode = AREF (call_fun, COMPILED_BYTECODE),
!CONSP (bytecode)))
if (COMPILEDP (call_fun))
{
fun = call_fun;
bytestr = bytecode;
args_template = XFIXNUM (template);
nargs = call_nargs;
args = call_args;
goto setup_frame;
Lisp_Object template = AREF (call_fun, COMPILED_ARGLIST);
if (FIXNUMP (template))
{
/* Fast path for lexbound functions. */
fun = call_fun;
bytestr = AREF (call_fun, COMPILED_BYTECODE),
args_template = XFIXNUM (template);
nargs = call_nargs;
args = call_args;
goto setup_frame;
}
}
Lisp_Object val;

59
src/eval.c
View file

@ -3122,19 +3122,6 @@ funcall_subr (struct Lisp_Subr *subr, ptrdiff_t numargs, Lisp_Object *args)
xsignal2 (Qwrong_number_of_arguments, fun, make_fixnum (numargs));
}
/* Call the compiled Lisp function FUN. If we have not yet read FUN's
bytecode string and constants vector, fetch them from the file first. */
static Lisp_Object
fetch_and_exec_byte_code (Lisp_Object fun, ptrdiff_t args_template,
ptrdiff_t nargs, Lisp_Object *args)
{
if (CONSP (AREF (fun, COMPILED_BYTECODE)))
Ffetch_bytecode (fun);
return exec_byte_code (fun, args_template, nargs, args);
}
static Lisp_Object
apply_lambda (Lisp_Object fun, Lisp_Object args, specpdl_ref count)
{
@ -3204,8 +3191,7 @@ funcall_lambda (Lisp_Object fun, ptrdiff_t nargs,
ARGLIST slot value: pass the arguments to the byte-code
engine directly. */
if (FIXNUMP (syms_left))
return fetch_and_exec_byte_code (fun, XFIXNUM (syms_left),
nargs, arg_vector);
return exec_byte_code (fun, XFIXNUM (syms_left), nargs, arg_vector);
/* Otherwise the bytecode object uses dynamic binding and the
ARGLIST slot contains a standard formal argument list whose
variables are bound dynamically below. */
@ -3293,7 +3279,7 @@ funcall_lambda (Lisp_Object fun, ptrdiff_t nargs,
val = XSUBR (fun)->function.a0 ();
}
else
val = fetch_and_exec_byte_code (fun, 0, 0, NULL);
val = exec_byte_code (fun, 0, 0, NULL);
return unbind_to (count, val);
}
@ -3411,46 +3397,6 @@ lambda_arity (Lisp_Object fun)
return Fcons (make_fixnum (minargs), make_fixnum (maxargs));
}
DEFUN ("fetch-bytecode", Ffetch_bytecode, Sfetch_bytecode,
1, 1, 0,
doc: /* If byte-compiled OBJECT is lazy-loaded, fetch it now. */)
(Lisp_Object object)
{
Lisp_Object tem;
if (COMPILEDP (object))
{
if (CONSP (AREF (object, COMPILED_BYTECODE)))
{
tem = read_doc_string (AREF (object, COMPILED_BYTECODE));
if (! (CONSP (tem) && STRINGP (XCAR (tem))
&& VECTORP (XCDR (tem))))
{
tem = AREF (object, COMPILED_BYTECODE);
if (CONSP (tem) && STRINGP (XCAR (tem)))
error ("Invalid byte code in %s", SDATA (XCAR (tem)));
else
error ("Invalid byte code");
}
Lisp_Object bytecode = XCAR (tem);
if (STRING_MULTIBYTE (bytecode))
{
/* BYTECODE must have been produced by Emacs 20.2 or earlier
because it produced a raw 8-bit string for byte-code and now
such a byte-code string is loaded as multibyte with raw 8-bit
characters converted to multibyte form. Convert them back to
the original unibyte form. */
bytecode = Fstring_as_unibyte (bytecode);
}
pin_string (bytecode);
ASET (object, COMPILED_BYTECODE, bytecode);
ASET (object, COMPILED_CONSTANTS, XCDR (tem));
}
}
return object;
}
/* Return true if SYMBOL's default currently has a let-binding
which was made in the buffer that is now current. */
@ -4512,7 +4458,6 @@ alist of active lexical bindings. */);
defsubr (&Srun_hook_with_args_until_success);
defsubr (&Srun_hook_with_args_until_failure);
defsubr (&Srun_hook_wrapped);
defsubr (&Sfetch_bytecode);
defsubr (&Sbacktrace_debug);
DEFSYM (QCdebug_on_exit, ":debug-on-exit");
defsubr (&Smapbacktrace);

49
src/lread.c
View file

@ -3481,6 +3481,8 @@ vector_from_rev_list (Lisp_Object elems)
return obj;
}
static Lisp_Object get_lazy_string (Lisp_Object val);
static Lisp_Object
bytecode_from_rev_list (Lisp_Object elems, Lisp_Object readcharfun)
{
@ -3495,14 +3497,18 @@ bytecode_from_rev_list (Lisp_Object elems, Lisp_Object readcharfun)
&& FIXNATP (vec[COMPILED_STACK_DEPTH])))
invalid_syntax ("Invalid byte-code object", readcharfun);
if (load_force_doc_strings
&& NILP (vec[COMPILED_CONSTANTS])
&& STRINGP (vec[COMPILED_BYTECODE]))
/* Always read 'lazily-loaded' bytecode (generated by the
`byte-compile-dynamic' feature prior to Emacs 30) eagerly, to
avoid code in the fast path during execution. */
if (CONSP (vec[COMPILED_BYTECODE]))
vec[COMPILED_BYTECODE] = get_lazy_string (vec[COMPILED_BYTECODE]);
/* Lazily-loaded bytecode is represented by the constant slot being nil
and the bytecode slot a (lazily loaded) string containing the
print representation of (BYTECODE . CONSTANTS). Unpack the
pieces by coerceing the string to unibyte and reading the result. */
if (NILP (vec[COMPILED_CONSTANTS]))
{
/* Lazily-loaded bytecode is represented by the constant slot being nil
and the bytecode slot a (lazily loaded) string containing the
print representation of (BYTECODE . CONSTANTS). Unpack the
pieces by coerceing the string to unibyte and reading the result. */
Lisp_Object enc = vec[COMPILED_BYTECODE];
Lisp_Object pair = Fread (Fcons (enc, readcharfun));
if (!CONSP (pair))
@ -3512,25 +3518,20 @@ bytecode_from_rev_list (Lisp_Object elems, Lisp_Object readcharfun)
vec[COMPILED_CONSTANTS] = XCDR (pair);
}
if (!((STRINGP (vec[COMPILED_BYTECODE])
&& VECTORP (vec[COMPILED_CONSTANTS]))
|| CONSP (vec[COMPILED_BYTECODE])))
if (!(STRINGP (vec[COMPILED_BYTECODE])
&& VECTORP (vec[COMPILED_CONSTANTS])))
invalid_syntax ("Invalid byte-code object", readcharfun);
if (STRINGP (vec[COMPILED_BYTECODE]))
{
if (STRING_MULTIBYTE (vec[COMPILED_BYTECODE]))
{
/* BYTESTR must have been produced by Emacs 20.2 or earlier
because it produced a raw 8-bit string for byte-code and
now such a byte-code string is loaded as multibyte with
raw 8-bit characters converted to multibyte form.
Convert them back to the original unibyte form. */
vec[COMPILED_BYTECODE] = Fstring_as_unibyte (vec[COMPILED_BYTECODE]);
}
/* Bytecode must be immovable. */
pin_string (vec[COMPILED_BYTECODE]);
}
if (STRING_MULTIBYTE (vec[COMPILED_BYTECODE]))
/* BYTESTR must have been produced by Emacs 20.2 or earlier
because it produced a raw 8-bit string for byte-code and
now such a byte-code string is loaded as multibyte with
raw 8-bit characters converted to multibyte form.
Convert them back to the original unibyte form. */
vec[COMPILED_BYTECODE] = Fstring_as_unibyte (vec[COMPILED_BYTECODE]);
/* Bytecode must be immovable. */
pin_string (vec[COMPILED_BYTECODE]);
XSETPVECTYPE (XVECTOR (obj), PVEC_COMPILED);
return obj;