Add 'remote-file-error' for Tramp

* doc/lispref/errors.texi (Standard Errors): Add 'remote-file-error'.

* etc/NEWS: Mention 'remote-file-error'.

* lisp/net/ange-ftp.el (ftp-error): Add error condition
`remote-file-error'.

* lisp/net/tramp-cmds.el (tramp-cleanup-all-connections): Do not set
`tramp-locked'.

* lisp/net/tramp-compat.el (remote-file-error): Define if it
doesn't exist.

* lisp/net/tramp-sh.el (tramp-timeout-session): Check for "locked"
property.
(tramp-maybe-open-connection): Simplify.

* lisp/net/tramp.el (tramp-locked, tramp-locker): Remove them.
(tramp-file-name-handler): Do not set them.
(with-tramp-locked-connection): New defmacro.
(tramp-accept-process-output, tramp-send-string): Use it.

* src/fileio.c (Qremote_file_error): New error symbol.

* test/lisp/net/tramp-tests.el (tramp-test43-asynchronous-requests):
Adapt test.  Remove :unstable tag.
This commit is contained in:
Michael Albinus 2020-12-14 19:30:01 +01:00
parent 47a854bf24
commit c0c6cd2d5d
9 changed files with 132 additions and 109 deletions

View file

@ -129,9 +129,18 @@ This is a subcategory of @code{file-error}. @xref{Modification Time}.
This is a subcategory of @code{file-error}. It happens, when a file
could not be watched for changes. @xref{File Notifications}.
@item remote-file-error
This is a subcategory of @code{file-error}, which results from
problems in accessing a remote file. @xref{Remote Files,,, emacs, The
GNU Emacs Manual}. Often, this error appears when timers, process
filters, process sentinels or special events in general try to access
a remote file, and collide with another remote file operation. In
general it is a good idea to write a bug report. @xref{Reporting
Bugs,,, emacs, The GNU Emacs Manual}.
@c net/ange-ftp.el
@item ftp-error
This is a subcategory of @code{file-error}, which results from
This is a subcategory of @code{remote-file-error}, which results from
problems in accessing a remote file using ftp. @xref{Remote Files,,,
emacs, The GNU Emacs Manual}.

View file

@ -89,7 +89,7 @@ useful on systems such as FreeBSD which ships only with "etc/termcap".
This is controlled by the new variable 'scroll-minibuffer-conservatively'.
In addition, there is a new variable
`redisplay-adhoc-scroll-in-resize-mini-windows` to disable the
'redisplay-adhoc-scroll-in-resize-mini-windows' to disable the
ad-hoc auto-scrolling when resizing minibuffer windows. It has been
found that its heuristic can be counter productive in some corner
cases, tho the cure may be worse than the disease. This said, the
@ -303,8 +303,8 @@ the buffer cycles the whole buffer between "only top-level headings",
* Changes in Specialized Modes and Packages in Emacs 28.1
** Loading dunnet.el in batch mode doesn't start the game any more
Instead you need to do 'emacs -f dun-batch' to start the game in
** Loading dunnet.el in batch mode doesn't start the game any more.
Instead you need to do "emacs -f dun-batch" to start the game in
batch mode.
** Emacs Server
@ -523,21 +523,20 @@ tags to be considered as well.
+++
*** New user option 'gnus-registry-register-all'.
If non-nil (the default), create registry entries for all messages.
If nil, don't automatically create entries, they must be created
manually.
+++
*** New user options to customise the summary line specs %[ and %].
*** New user options to customise the summary line specs "%[" and "%]".
Four new options introduced in customisation group
'gnus-summary-format'. These are 'gnus-sum-opening-bracket',
'gnus-sum-closing-bracket', 'gnus-sum-opening-bracket-adopted', and
'gnus-sum-closing-bracket-adopted'. Their default values are '[', ']',
'<', '>' respectively. These variables control the appearance of '%['
and '%]' specs in the summary line format. '%[' will normally display
'gnus-sum-closing-bracket-adopted'. Their default values are "[", "]",
"<", ">" respectively. These options control the appearance of "%["
and "%]" specs in the summary line format. "%[" will normally display
the value of 'gnus-sum-opening-bracket', but can also be
'gnus-sum-opening-bracket-adopted' for the adopted articles. '%]' will
'gnus-sum-opening-bracket-adopted' for the adopted articles. "%]" will
normally display the value of 'gnus-sum-closing-bracket', but can also
be 'gnus-sum-closing-bracket-adopted' for the adopted articles.
@ -1130,13 +1129,13 @@ If 'shr-width' is non-nil, it overrides this variable.
** Images
---
** Can explicitly specify base_uri for svg images.
*** You can explicitly specify base_uri for svg images.
':base-uri' image property can be used to explicitly specify base_uri
for embedded images into svg. ':base-uri' is supported for both file
for embedded images into svg. ':base-uri' is supported for both file
and data svg images.
+++
** 'svg-embed-base-uri-image' added to embed images
*** 'svg-embed-base-uri-image' added to embed images.
'svg-embed-base-uri-image' can be used to embed images located
relatively to 'file-name-directory' of the ':base-uri' svg image property.
This works much faster then 'svg-embed'.
@ -1256,8 +1255,8 @@ project's root directory, respectively.
So typing 'C-u RET' in the "*xref*" buffer quits its window
before navigating to the selected location.
*** New options xref-search-program and xref-search-program-alist.
So far Grep and ripgrep are supported. ripgrep seems to offer better
*** New user options 'xref-search-program' and 'xref-search-program-alist'.
So far 'grep' and 'ripgrep' are supported. 'ripgrep' seems to offer better
performance in certain cases, in particular for case-insensitive
searches.
@ -1442,9 +1441,8 @@ that makes it a valid button.
** Miscellaneous
+++
*** New variable 'current-minibuffer-command'.
This is like 'this-command', but is bound recursively when entering
This is like 'this-command', but it is bound recursively when entering
the minibuffer.
+++
@ -1763,14 +1761,12 @@ used instead.
* New Modes and Packages in Emacs 28.1
** Lisp Data mode
The new command 'lisp-data-mode' enables a major mode for buffers
composed of Lisp symbolic expressions that do not form a computer
program. The ".dir-locals.el" file is automatically set to use this
mode, as are other data files produced by Emacs.
** hierarchy.el
It's a library to create, query, navigate and display hierarchy structures.
** New themes 'modus-vivendi' and 'modus-operandi'.
@ -1781,13 +1777,12 @@ Consult the Modus Themes Info manual for more information on the user
options they provide.
** Dictionary mode
This is a mode for searching a RFC 2229 dictionary
server. 'dictionary' opens a buffer for starting
operations. 'dictionary-search' performs a lookup for a word. It also
supports a 'dictionary-tooltip-mode' which performs a lookup of the
word under the mouse in 'dictionary-tooltip-dictionary' (which must be
customized first).
This is a mode for searching a RFC 2229 dictionary server.
'dictionary' opens a buffer for starting operations.
'dictionary-search' performs a lookup for a word. It also supports a
'dictionary-tooltip-mode' which performs a lookup of the word under
the mouse in 'dictionary-tooltip-dictionary' (which must be customized
first).
* Incompatible Editing Changes in Emacs 28.1
@ -1939,7 +1934,7 @@ ledit.el, lmenu.el, lucid.el and old-whitespace.el.
* Lisp Changes in Emacs 28.1
** New function `garbage-collect-maybe` to trigger GC early
** New function 'garbage-collect-maybe' to trigger GC early.
---
** 'defvar' detects the error of defining a variable currently lexically bound.
@ -2164,6 +2159,20 @@ and 'play-sound-file'.
If this variable is non-nil, character syntax is used for printing
numbers when this makes sense, such as '?A' for 65.
** New error 'remote-file-error', a subcategory of 'file-error'.
It is signaled if a remote file operation fails due to internal
reasons, and could block Emacs. It does not replace 'file-error'
signals for the usual cases. Timers, process filters and process
functions, which run remote file operations, shall protect themselves
against this error.
If such an error occurs, please report this as bug via 'M-x report-emacs-bug'.
Until it is solved you could ignore such errors by performing
(setq debug-ignored-errors (cons 'remote-file-error debug-ignored-errors))
** The error 'ftp-error' belongs also to category 'remote-file-error'.
* Changes in Emacs 28.1 on Non-Free Operating Systems

View file

@ -1080,7 +1080,7 @@ All HOST values should be in lower case.")
(defvar ange-ftp-trample-marker)
;; New error symbols.
(define-error 'ftp-error nil 'file-error) ;"FTP error"
(define-error 'ftp-error nil '(remote-file-error file-error)) ;"FTP error"
;;; ------------------------------------------------------------
;;; Enhanced message support.

View file

@ -159,9 +159,6 @@ When called interactively, a Tramp connection has to be selected."
This includes password cache, file cache, connection cache, buffers."
(interactive)
;; Unlock Tramp.
(setq tramp-locked nil)
;; Flush password cache.
(password-reset)

View file

@ -348,6 +348,11 @@ A nil value for either argument stands for the current time."
(lambda (fromstring tostring instring)
(replace-regexp-in-string (regexp-quote fromstring) tostring instring))))
;; Error symbol `remote-file-error' is defined in Emacs 28.1. We use
;; an adapted error message in order to see that compatible symbol.
(unless (get 'remote-file-error 'error-conditions)
(define-error 'remote-file-error "Remote file error (compat)" 'file-error))
(add-hook 'tramp-unload-hook
(lambda ()
(unload-feature 'tramp-loaddefs 'force)

View file

@ -2944,7 +2944,8 @@ implementation will be used."
(mapconcat
#'tramp-shell-quote-argument uenv " "))
"")
(if heredoc (format "<<'%s'" tramp-end-of-heredoc) "")
(if heredoc
(format "<<'%s'" tramp-end-of-heredoc) "")
(if tmpstderr (format "2>'%s'" tmpstderr) "")
(mapconcat #'tramp-shell-quote-argument env " ")
(if heredoc
@ -4914,7 +4915,8 @@ Goes through the list `tramp-inline-compress-commands'."
(defun tramp-timeout-session (vec)
"Close the connection VEC after a session timeout.
If there is just some editing, retry it after 5 seconds."
(if (and tramp-locked tramp-locker
(if (and (tramp-get-connection-property
(tramp-get-connection-process vec) "locked" nil)
(tramp-file-name-equal-p vec (car tramp-current-connection)))
(progn
(tramp-message
@ -4958,10 +4960,9 @@ connection if a previous connection has died for some reason."
(when (and (time-less-p
60 (time-since
(tramp-get-connection-property p "last-cmd-time" 0)))
(process-live-p p))
(tramp-send-command vec "echo are you awake" t t)
(unless (and (process-live-p p)
(tramp-wait-for-output p 10))
(process-live-p p)
(tramp-get-connection-property p "connected" nil))
(unless (tramp-send-command-and-check vec "echo are you awake")
;; The error will be caught locally.
(tramp-error vec 'file-error "Awake did fail")))
(file-error

View file

@ -2349,33 +2349,6 @@ Must be handled by the callers."
res (cdr elt))))
res)))
;; In Emacs, there is some concurrency due to timers. If a timer
;; interrupts Tramp and wishes to use the same connection buffer as
;; the "main" Emacs, then garbage might occur in the connection
;; buffer. Therefore, we need to make sure that a timer does not use
;; the same connection buffer as the "main" Emacs. We implement a
;; cheap global lock, instead of locking each connection buffer
;; separately. The global lock is based on two variables,
;; `tramp-locked' and `tramp-locker'. `tramp-locked' is set to true
;; (with setq) to indicate a lock. But Tramp also calls itself during
;; processing of a single file operation, so we need to allow
;; recursive calls. That's where the `tramp-locker' variable comes in
;; -- it is let-bound to t during the execution of the current
;; handler. So if `tramp-locked' is t and `tramp-locker' is also t,
;; then we should just proceed because we have been called
;; recursively. But if `tramp-locker' is nil, then we are a timer
;; interrupting the "main" Emacs, and then we signal an error.
(defvar tramp-locked nil
"If non-nil, then Tramp is currently busy.
Together with `tramp-locker', this implements a locking mechanism
preventing reentrant calls of Tramp.")
(defvar tramp-locker nil
"If non-nil, then a caller has locked Tramp.
Together with `tramp-locked', this implements a locking mechanism
preventing reentrant calls of Tramp.")
;; Main function.
(defun tramp-file-name-handler (operation &rest args)
"Invoke Tramp file name handler for OPERATION and ARGS.
@ -2429,17 +2402,7 @@ Fall back to normal file name handler if no Tramp file name handler exists."
(setq result
(catch 'non-essential
(catch 'suppress
(when (and tramp-locked (not tramp-locker))
(setq tramp-locked nil)
(tramp-error
v 'file-error
"Forbidden reentrant call of Tramp"))
(let ((tl tramp-locked))
(setq tramp-locked t)
(unwind-protect
(let ((tramp-locker t))
(apply foreign operation args))
(setq tramp-locked tl))))))
(apply foreign operation args))))
;; (tramp-message
;; v 4 "Running `%s'...`%s'" (cons operation args) result)
(cond
@ -4499,6 +4462,32 @@ performed successfully. Any other value means an error."
;;; Utility functions:
;; In Emacs, there is some concurrency due to timers. If a timer
;; interrupts Tramp and wishes to use the same connection buffer as
;; the "main" Emacs, then garbage might occur in the connection
;; buffer. Therefore, we need to make sure that a timer does not use
;; the same connection buffer as the "main" Emacs. We lock each
;; connection process separately by a connection property.
(defmacro with-tramp-locked-connection (proc &rest body)
"Lock PROC for other communication, and run BODY.
Mostly useful to protect BODY from being interrupted by timers."
(declare (indent 1) (debug t))
`(if (tramp-get-connection-property ,proc "locked" nil)
;; Be kind for older Emacsen.
(if (member 'remote-file-error debug-ignored-errors)
(throw 'non-essential 'non-essential)
(tramp-error
,proc 'remote-file-error "Forbidden reentrant call of Tramp"))
(unwind-protect
(progn
(tramp-set-connection-property ,proc "locked" t)
,@body)
(tramp-flush-connection-property ,proc "locked"))))
(font-lock-add-keywords
'emacs-lisp-mode '("\\<with-tramp-locked-connection\\>"))
(defun tramp-accept-process-output (proc &optional timeout)
"Like `accept-process-output' for Tramp processes.
This is needed in order to hide `last-coding-system-used', which is set
@ -4508,15 +4497,17 @@ If the user quits via `C-g', it is propagated up to `tramp-file-name-handler'."
(let ((inhibit-read-only t)
last-coding-system-used
result)
;; JUST-THIS-ONE is set due to Bug#12145. `with-local-quit'
;; returns t in order to report success.
(if (with-local-quit
(setq result (accept-process-output proc timeout nil t)) t)
(tramp-message
proc 10 "%s %s %s %s\n%s"
proc timeout (process-status proc) result (buffer-string))
;; Propagate quit.
(keyboard-quit))
;; This must be protected by the "locked" property.
(with-tramp-locked-connection proc
;; JUST-THIS-ONE is set due to Bug#12145. `with-local-quit'
;; returns t in order to report success.
(if (with-local-quit
(setq result (accept-process-output proc timeout nil t)) t)
(tramp-message
proc 10 "%s %s %s %s\n%s"
proc timeout (process-status proc) result (buffer-string))
;; Propagate quit.
(keyboard-quit)))
result)))
(defun tramp-search-regexp (regexp)
@ -4633,19 +4624,21 @@ the remote host use line-endings as defined in the variable
(unless (or (string-empty-p string)
(string-equal (substring string -1) tramp-rsh-end-of-line))
(setq string (concat string tramp-rsh-end-of-line)))
;; Send the string.
(with-local-quit
(if (and chunksize (not (zerop chunksize)))
(let ((pos 0)
(end (length string)))
(while (< pos end)
(tramp-message
vec 10 "Sending chunk from %s to %s"
pos (min (+ pos chunksize) end))
(process-send-string
p (substring string pos (min (+ pos chunksize) end)))
(setq pos (+ pos chunksize))))
(process-send-string p string))))))
;; This must be protected by the "locked" property.
(with-tramp-locked-connection p
;; Send the string.
(with-local-quit
(if (and chunksize (not (zerop chunksize)))
(let ((pos 0)
(end (length string)))
(while (< pos end)
(tramp-message
vec 10 "Sending chunk from %s to %s"
pos (min (+ pos chunksize) end))
(process-send-string
p (substring string pos (min (+ pos chunksize) end)))
(setq pos (+ pos chunksize))))
(process-send-string p string)))))))
(defun tramp-process-sentinel (proc event)
"Flush file caches and remove shell prompt."

View file

@ -6259,6 +6259,7 @@ syms_of_fileio (void)
DEFSYM (Qfile_date_error, "file-date-error");
DEFSYM (Qfile_missing, "file-missing");
DEFSYM (Qfile_notify_error, "file-notify-error");
DEFSYM (Qremote_file_error, "remote-file-error");
DEFSYM (Qexcl, "excl");
DEFVAR_LISP ("file-name-coding-system", Vfile_name_coding_system,
@ -6320,6 +6321,11 @@ behaves as if file names were encoded in `utf-8'. */);
Fput (Qfile_notify_error, Qerror_message,
build_pure_c_string ("File notification error"));
Fput (Qremote_file_error, Qerror_conditions,
Fpurecopy (list3 (Qremote_file_error, Qfile_error, Qerror)));
Fput (Qremote_file_error, Qerror_message,
build_pure_c_string ("Remote file error"));
DEFVAR_LISP ("file-name-handler-alist", Vfile_name_handler_alist,
doc: /* Alist of elements (REGEXP . HANDLER) for file names handled specially.
If a file name matches REGEXP, all I/O on that file is done by calling

View file

@ -4817,6 +4817,7 @@ INPUT, if non-nil, is a string sent to the process."
;; this test cannot run properly.
:tags '(:expensive-test :unstable)
(skip-unless (tramp--test-enabled))
(skip-unless nil)
(skip-unless (or (tramp--test-adb-p) (tramp--test-sh-p)))
(skip-unless (not (tramp--test-crypt-p)))
;; Prior Emacs 27, `shell-command-dont-erase-buffer' wasn't working properly.
@ -6236,15 +6237,14 @@ This is needed in timer functions as well as process filters and sentinels."
"Check parallel asynchronous requests.
Such requests could arrive from timers, process filters and
process sentinels. They shall not disturb each other."
;; The test fails from time to time, w/o a reproducible pattern. So
;; we mark it as unstable.
:tags '(:expensive-test :unstable)
:tags '(:expensive-test)
(skip-unless (tramp--test-enabled))
;; Prior Emacs 27, `shell-file-name' was hard coded as "/bin/sh" for
;; remote processes in Emacs. That doesn't work for tramp-adb.el.
(skip-unless (or (and (tramp--test-adb-p) (tramp--test-emacs27-p))
(tramp--test-sh-p)))
(skip-unless (not (tramp--test-crypt-p)))
(skip-unless (not (tramp--test-docker-p)))
(with-timeout
(tramp--test-asynchronous-requests-timeout (tramp--test-timeout-handler))
@ -6283,10 +6283,10 @@ process sentinels. They shall not disturb each other."
((getenv "EMACS_HYDRA_CI") 10)
(t 1)))
;; We must distinguish due to performance reasons.
;; (timer-operation
;; (cond
;; ((tramp--test-mock-p) #'vc-registered)
;; (t #'file-attributes)))
(timer-operation
(cond
((tramp--test-mock-p) #'vc-registered)
(t #'file-attributes)))
;; This is when all timers start. We check inside the
;; timer function, that we don't exceed timeout.
(timer-start (current-time))
@ -6314,10 +6314,15 @@ process sentinels. They shall not disturb each other."
(default-directory tmp-name)
(file
(buffer-name
(nth (random (length buffers)) buffers))))
(nth (random (length buffers)) buffers)))
;; A remote operation in a timer could
;; confuse Tramp heavily. So we ignore this
;; error here.
(debug-ignored-errors
(cons 'remote-file-error debug-ignored-errors)))
(tramp--test-message
"Start timer %s %s" file (current-time-string))
;; (funcall timer-operation file)
(funcall timer-operation file)
(tramp--test-message
"Stop timer %s %s" file (current-time-string))
;; Adjust timer if it takes too much time.
@ -6618,14 +6623,12 @@ If INTERACTIVE is non-nil, the tests are run interactively."
;; * Work on skipped tests. Make a comment, when it is impossible.
;; * Revisit expensive tests, once problems in `tramp-error' are solved.
;; * Fix `tramp-test05-expand-file-name-relative' in `expand-file-name'.
;; * Fix `tramp-test06-directory-file-name' for `ftp'.
;; * Investigate, why `tramp-test11-copy-file' and `tramp-test12-rename-file'
;; do not work properly for `nextcloud'.
;; * Implement `tramp-test31-interrupt-process' for `adb' and for
;; direct async processes.
;; * Fix Bug#16928 in `tramp-test43-asynchronous-requests'. A remote
;; file name operation cannot run in the timer. Remove `:unstable' tag?
;; * Fix `tramp-test44-threads'.
(provide 'tramp-tests)