This ensures that output targets in Eshell are only closed when Eshell
is actually done with them. In particular, this means that
"{ echo foo; echo bar } | rev" prints "raboof" as expected
(bug#59545).
* lisp/eshell/esh-io.el (eshell-create-handles): Structure the handles
differently so the targets and their ref-count can be shared.
(eshell-duplicate-handles): Reimplement this to share targets between
the original and new handle sets. Add STEAL-P argument.
(eshell-protect-handles, eshell-copy-output-handle)
(eshell-interactive-output-p, eshell-output-object): Account for
changes to the handle structure.
(eshell-close-handle): New function...
(eshell-close-handles, eshell-set-output-handle): ... use it.
(eshell-get-targets): Remove. This only existed to make the previous
implementation of 'eshell-duplicate-handles' work.
* lisp/eshell/esh-cmd.el (eshell-with-copied-handles): New argument
STEAL-P.
(eshell-do-pipelines): Use STEAL-P for the last item in the pipeline.
(eshell-parse-command): Don't copy handles for the last command in the
list; explain why we can't use STEAL-P here.
(eshell-eval-command): When queuing input, set 'eshell-command-body'
and 'eshell-test-body' for the 'if' conditional (see
'eshell-do-eval').
* test/lisp/eshell/esh-io-tests.el (esh-io-test/redirect-pipe): Split
into...
(esh-io-test/pipeline/default, esh-io-test/pipeline/all): ... these.
(esh-io-test/pipeline/subcommands): New test.
* test/lisp/eshell/esh-cmd-tests.el (esh-cmd-test/for-loop-pipe)
(esh-cmd-test/while-loop-pipe, esh-cmd-test/if-statement-pipe)
esh-cmd-test/if-else-statement-pipe): New tests.
(esh-cmd-test/while-loop): Use 'pop' to simplify the test a bit.
* test/lisp/eshell/eshell-test-helpers.el
(eshell-test--max-subprocess-time): Rename to...
(eshell-test--max-wait-time): ... this.
(eshell-wait-for): New function...
(eshell-wait-for-subprocess): ... use it.
* test/lisp/eshell/eshell-tests.el (eshell-test/queue-input): Fix this
test. Previously, it didn't correctly verify that the original
command completed.
* test/lisp/eshell/em-tramp-tests.el
(em-tramp-test/should-replace-command): New macro...
(em-tramp-test/su-default, em-tramp-test/su-user)
(em-tramp-test/su-login, em-tramp-test/sudo-shell)
(em-tramp-test/sudo-user-shell, em-tramp-test/doas-shell)
(em-tramp-test/doas-user-shell): ... use it.
Previously, the output handles in nested forms would be reset to the
default, leading to wrong behavior for commands like
{echo a; echo b} > file
"b" would be written to "file" as expected, but "a" would go to
standard output (bug#59545).
* lisp/eshell/esh-cmd.el (eshell-parse-command): Use
'eshell-with-copied-handles' for each statement within the whole
Eshell command.
* test/lisp/eshell/esh-io-tests.el (esh-io-test/redirect-subcommands)
(esh-io-test/redirect-subcommands/override)
(esh-io-test/redirect-subcommands/interpolated): New tests.
* test/lisp/eshell/em-script-tests.el
(em-script-test/source-script/redirect)
(em-script-test/source-script/redirect/dev-null): New tests.
(em-script-test/source-script, em-script-test/source-script/arg-vars)
(em-script-test/source-script/all-args-var): Tweak names/docstrings.
* test/lisp/eshell/em-extpipe-tests.el (em-extpipe-tests--deftest):
Skip over the newly-added 'eshell-with-copied-handles' form when
checking the parse results.
* test/lisp/eshell/em-tramp-tests.el (em-tramp-test/su-default)
(em-tramp-test/su-user, em-tramp-test/su-login)
(em-tramp-test/sudo-shell, em-tramp-test/sudo-user-shell)
(em-tramp-test/doas-shell, em-tramp-test/doas-user-shell): Update
expected command forms.
* lisp/eshell/esh-io.el (eshell-create-handles): Support creating with
multiple targets for stdout and/or stderr. Make the targets for a
handle always be a list, and store whether the targets are the default
in a separate 'default' field.
(eshell-protect-handles, eshell-close-handles)
(eshell-copy-output-handle, eshell-interactive-output-p)
(eshell-output-object): Update for changes in 'eshell-create-handles'.
(eshell-duplicate-handles, eshell-get-targets): New functions.
* lisp/eshell/esh-cmd.el (eshell-copy-handles): Rename and alias to...
(eshell-with-copied-handles): ... this function, and use
'eshell-duplicate-handles'.
(eshell-execute-pipeline): Use 'eshell-duplicate-handles'.
This allows splicing lists in-place in argument lists, which is
particularly important when defining aliases using the '$*' special
variable (bug#59960).
* lisp/eshell/esh-var.el (eshell-parse-variable): Add support for the
splice operator.
(eshell-interpolate-variable): Let 'eshell-parse-variable' handle
adding 'eshell-escape-arg'.
(eshell-complete-variable-reference): Handle the splice operator.
* lisp/eshell/esh-arg.el (eshell-concat-groups)
(eshell-prepare-splice): New functions...
(eshell-resolve-current-argument): ... use them.
(eshell-splice-args): New function.
* lisp/eshell/esh-cmd.el (eshell-rewrite-named-command): Handle
'eshell-splice-args'.
* lisp/eshell/esh-util.el (eshell-list-to-string): New function...
(eshell-flatten-and-stringify): ... use it.
* lisp/eshell/em-cmpl.el (eshell-complete-parse-arguments): Remove
'eshell-splice-args' sigils in Eshell command forms so that we can
perform completion on splice-expansions.
* lisp/eshell/em-unix.el (eshell-complete-host-reference): Don't try
to complete arguments containing "$@".
* test/lisp/eshell/esh-var-tets.el (esh-var-test/interp-list-var)
(esh-var-test/interp-list-var-concat, esh-var-test/interp-var-splice)
(esh-var-test/interp-var-splice-concat)
(esh-var-test/quoted-interp-list-var)
(esh-var-test/quoted-interp-list-var-concat)
(esh-var-test/quoted-interp-var-splice)
(esh-var-test/quoted-interp-var-splice-concat): New tests.
* test/lisp/eshell/em-alias-tests.el
(em-alias-test/alias-all-args-var-splice): New test.
* doc/misc/eshell.texi (Dollars Expansion): Explain the splice
operator.
(Aliases): Expand documentation and use '$@*'.
(Built-ins, Bugs and Ideas): Use '$@*' where appropriate.
* etc/NEWS: Announce this change.
This makes commands like "COLUMNS=40 some-command" work as expected.
* lisp/eshell/esh-cmd.el (eshell-subcommand-bindings): Remove
'process-environment' from here...
* lisp/eshell/esh-var.el (eshell-var-initialize): ... and add to here,
along with 'eshell-variable-aliases-list'.
(eshell-inside-emacs): Convert to a 'defvar-local' to make it settable
in a particular Eshell buffer.
(eshell-variable-aliases-list): Make $?, $$, and $* read-only and
update docstring.
(eshell-set-variable): New function...
(eshell-handle-local-variables, eshell/export, eshell/unset): ... use
it.
(eshell/set, pcomplete/eshell-mode/set): New functions.
(eshell-get-variable): Get the variable alias's getter function when
appropriate and use a safer method for checking function arity.
* test/lisp/eshell/esh-var-tests.el (esh-var-test/set/env-var)
(esh-var-test/set/symbol, esh-var-test/unset/env-var)
(esh-var-test/unset/symbol, esh-var-test/setq, esh-var-test/export)
(esh-var-test/local-variables, esh-var-test/alias/function)
(esh-var-test/alias/function-pair, esh-var-test/alias/string)
(esh-var-test/alias/string/prefer-lisp, esh-var-test/alias/symbol)
(esh-var-test/alias/symbol-pair, esh-var-test/alias/export)
(esh-var-test/alias/local-variables): New tests.
* doc/misc/eshell.texi (Built-ins): Add 'set' and update 'unset'
documentation.
(Variables): Expand documentation of how to get/set variables.
* lisp/eshell/esh-arg.el: Require cl-lib.
(eshell-finish-arg): Allow passing multiple ARGUMENTS.
(eshell-quote-argument): Handle the case when 'eshell-finish-arg' was
passed multiple arguments.
* lisp/eshell/esh-cmd.el (eshell-do-pipelines)
(eshell-do-pipelines-synchronously): Only set stdout output handle.
* lisp/eshell/esh-io.el (eshell-redirection-operators-alist): New
constant.
(eshell-io-initialize): Prefer sharp quotes for functions.
(eshell-parse-redirection, eshell-strip-redirections): Add support for
more redirection forms.
(eshell-copy-output-handle, eshell-set-all-output-handles): New
functions.
* test/lisp/eshell/esh-io-tests.el
(esh-io-test/redirect-all/overwrite, esh-io-test/redirect-all/append)
(esh-io-test/redirect-all/insert, esh-io-test/redirect-copy)
(esh-io-test/redirect-copy-first, esh-io-test/redirect-pipe): New
tests.
* doc/misc/eshell.texi (Redirection): Document new redirection syntax.
(Pipelines): Document '|&' syntax.
(Bugs and ideas): Update item about redirection syntax.
* etc/NEWS: Announce this change.
* lisp/eshell/esh-cmd.el (eshell-exec-lisp): Handle
'eshell-pipe-broken'.
* lisp/eshell/esh-io.el (eshell-output-object-to-target): Only signal
'eshell-pipe-broken' if the process being written to has finished.
* test/lisp/eshell/esh-proc-tests.el
(esh-proc-test/pipeline-connection-type/middle)
(esh-proc-test/pipeline-connection-type/last): Remove ':unstable'.
Ref: https://lists.gnu.org/archive/html/emacs-devel/2022-08/msg00524.html
* lisp/eshell/esh-cmd.el (eshell-rewrite-if-command): Fix misplaced
't' symbol; it should be passed to 'eshell-invokify-arg'.
(eshell-do-eval): Pass 'synchronous-p' to recursive calls in some
missing cases.
* test/lisp/eshell/esh-cmd-tests.el (esh-cmd-test/if-statement,
(esh-cmd-test/if-else-statement)
(esh-cmd-test/if-else-statement-lisp-form)
(esh-cmd-test/if-else-statement-lisp-form-2)
(esh-cmd-test/if-else-statement-ext-cmd)
(esh-cmd-test/unless-statement)
(esh-cmd-test/unless-else-statement)
(esh-cmd-test/unless-else-statement-lisp-form)
(esh-cmd-test/unless-else-statement-ext-cmd): Use
'eshell-command-result-equal'.
Previously, '$?' (last exit code) was only useful for external
commands, and '$$' (last result) was only useful for Lisp commands.
* lisp/eshell/esh-cmd.el (eshell-lisp-form-nil-is-failure): New
option.
(eshell-lisp-command): Set last exit code to 1 when the command
signals an error, and 2 if it returns nil (for Lisp forms only).
* lisp/eshell/esh-proc.el (eshell-sentinel): Set last result to t if
the command succeeded.
* test/lisp/eshell/esh-cmd-tests.el
(esh-cmd-test/while-loop-lisp-form, esh-cmd-test/until-loop-lisp-form)
(esh-cmd-test/if-else-statement-lisp-form)
(esh-cmd-test/if-else-statement-lisp-form-2)
(esh-cmd-test/unless-else-statement-lisp-form): New tests.
* test/lisp/eshell/esh-var-tests.el
(esh-var-test/last-status-var-lisp-command)
(esh-var-test/last-status-var-lisp-form)
(esh-var-test/last-status-var-lisp-form-2)
(esh-var-test/last-status-var-ext-cmd)
(esh-var-test/last-status-var-ext-cmd): New tests.
(esh-var-test/last-result-var2): Rename from this...
( esh-var-test/last-result-var-twice): ... to this.
* doc/misc/eshell.texi (Variables): Update documentation about '$?'
and '$$'.
(Control Flow): Mention that '(lisp forms)' can be used as
conditionals.
* etc/NEWS: Announce this change (bug#57129).
* lisp/eshell/esh-cmd.el (eshell-structure-basic-command): Forms
beginning with 'eshell-escape-arg' are "data-wise".
* test/lisp/eshell/esh-cmd-tests.el (esh-cmd-test/while-loop)
(esh-cmd-test/until-loop, esh-cmd-test/if-statement)
(esh-cmd-test/if-else-statement, esh-cmd-test/unless-statement)
(esh-cmd-test/unless-else-statement): Use variable interpolation.
(esh-cmd-test/while-loop-ext-cmd, esh-cmd-test/until-loop-ext-cmd)
(esh-cmd-test/if-else-statement-ext-cmd)
(esh-cmd-test/unless-else-statement-ext-cmd): New tests, adapted from
the existing ones.
* doc/misc/eshell.texi (Control Flow): Update documentation for
conditionals (bug#57129).
This simplifies usage of 'eshell-close-handles' in several places and
makes it work more like the docstring indicated it would.
* lisp/eshell/esh-io.el (eshell-close-handles): Only store EXIT-CODE
and RESULT if they're non-nil. Also, use 'dotimes' and 'dolist' to
simplify the implementation.
* lisp/eshell/em-alias.el (eshell-write-aliases-list):
* lisp/eshell/esh-cmd.el (eshell-rewrite-for-command)
(eshell-structure-basic-command): Adapt calls to
'eshell-close-handles'.
* test/lisp/eshell/eshell-tests.el (eshell-test/simple-command-result)
(eshell-test/lisp-command, eshell-test/lisp-command-with-quote)
(eshell-test/for-loop, eshell-test/for-name-loop)
(eshell-test/for-name-shadow-loop, eshell-test/lisp-command-args)
(eshell-test/subcommand, eshell-test/subcommand-args)
(eshell-test/subcommand-lisp): Move from here...
* test/lisp/eshell/esh-cmd-tests.el
(esh-cmd-test/simple-command-result, esh-cmd-test/lisp-command)
(esh-cmd-test/lisp-command-with-quote, esh-cmd-test/for-loop)
(esh-cmd-test/for-name-loop, esh-cmd-test/for-name-shadow-loop)
(esh-cmd-test/lisp-command-args, esh-cmd-test/subcommand)
(esh-cmd-test/subcommand-args, esh-cmd-test/subcommand-lisp): ... to
here.
(esh-cmd-test/and-operator, esh-cmd-test/or-operator)
(esh-cmd-test/for-loop-list, esh-cmd-test/for-loop-multiple-args)
(esh-cmd-test/while-loop, esh-cmd-test/until-loop)
(esh-cmd-test/if-statement, esh-cmd-test/if-else-statement)
(esh-cmd-test/unless-statement, esh-cmd-test/unless-else-statement):
New tests.
* doc/misc/eshell.texi (Invocation): Explain '&&' and '||'.
(for loop): Move from here...
(Control Flow): ... to here, and add documentation for other control
flow forms.
* lisp/eshell/esh-cmd.el (eshell-subcommand-bindings)
(eshell-command-to-value): Set 'eshell-in-pipeline-p' to nil.
* test/lisp/eshell/eshell-tests.el
(eshell-test/subcommand-reset-in-pipeline)
(eshell-test/lisp-reset-in-pipeline): New tests (bug#55620).
* lisp/eshell/esh-cmd.el (eshell-execute-pipeline): Use 'make-symbol'
for headproc and tailproc.
(eshell-do-pipelines, eshell-do-pipelines-synchronously): Adapt to the
above.
* test/lisp/eshell/eshell-tests.el (eshell-test/pipe-subcommand)
(eshell-test/pipe-subcommand-with-pipe): New test.
* doc/misc/eshell.texi (Bugs and ideas): Remove item about piping to
process from loop; this commit fixes it (bug#55590).
* lisp/eshell/esh-cmd.el (eshell-eval-argument): New function.
* lisp/eshell/esh-util.el (eshell-file-attributes): Pass original
value of FILE to 'file-attributes'.
* lisp/eshell/em-pred.el (eshell-predicate-alist): Change socket char
to '=', since 's' conflicts with setuid.
(eshell-modifier-alist): Fix 'E' (eval) modifier by using
'eshell-eval-argument'. Also improve performance of 'O' (reversed
sort) modifier.
(eshell-modifier-help-string): Fix documentation of global
substitution modifier.
(eshell-pred-substitute): Fix infinite loop in some global
substitutions.
(eshell-join-members): Fix joining with implicit " " delimiter.
(Bug#54470)
* test/lisp/eshell/em-pred-tests.el: New file.
* doc/misc/eshell.texi (Argument Predication): New section.
This abstracts out the somewhat-unusual "insert&delete" logic in
'eshell-parse-command' so that it can be used elsewhere, and also
ensures that the deletion occurs even if an an error occurs.
* lisp/eshell/esh-cmd.el (eshell-with-temp-command): New macro.
(eshell-parse-command): Use it.
Previously, if a non-process was piped to a process, this could end up
being nil, which isn't correct. 'eshell-last-async-procs' should just
ignore non-process commands in a pipeline.
* lisp/eshell/esh-cmd.el (eshell-do-pipelines): Set 'headproc'
correctly.
* test/lisp/eshell/eshell-tests.el (eshell-test/pipe-headproc): New test.
Previously, input was sent to the last process in the pipeline,
resulting in unexpected behavior when running commands like
'tr a-z A-Z | rev'.
* lisp/eshell/esh-util.el (eshell-process-pair-p)
(eshell-make-process-pair): New functions.
* lisp/eshell/esh-cmd.el (eshell-last-async-proc): Rename to...
(eshell-last-async-procs): ... this, and store a pair of processes.
(eshell-interactive-process): Replace with...
(eshell-interactive-process-p, eshell-head-process)
(eshell-tail-process): ... these.
(eshell-cmd-initialize): Set 'eshell-last-async-procs'.
(eshell-do-pipelines): Set 'headproc'.
(eshell-execute-pipeline): Return 'headproc' and 'tailproc'.
(eshell-resume-eval): Use 'eshell-last-async-procs'.
(eshell-do-eval): Make sure we work with a pair of processes.
* lisp/eshell/esh-proc.el (eshell-send-eof-to-process): Move from
here...
* lisp/eshell/esh-mode.el (eshell-send-eof-to-process): ... to here,
and only send EOF to the head process.
* lisp/eshell/em-cmpl.el (eshell-complete-parse-arguments)
* lisp/eshell/esh-mode.el (eshell-intercept-commands)
(eshell-watch-for-password-prompt):
Use 'eshell-interactive-process-p'.
* lisp/eshell/em-rebind.el (eshell-delchar-or-maybe-eof)
* lisp/eshell/em-term.el (eshell-term-send-raw-string)
* lisp/eshell/esh-mode.el (eshell-self-insert-command)
(eshell-send-input, eshell-send-invisible):
Use 'eshell-head-process'.
* lisp/eshell/esh-cmd.el (eshell-as-subcommand):
Use 'eshell-tail-process'.
* lisp/eshell/eshell.el (eshell-command):
* test/lisp/eshell/eshell-tests-helpers.el
(eshell-wait-for-subprocess):
Use 'eshell-interactive-process-p' and 'eshell-tail-process'.
* test/lisp/eshell/eshell-tests.el (eshell-test/pipe-headproc-stdin):
New test.
In particular, this used to fail for pipelines where the last process
in the pipeline came from the first element of the pipeline. This
could happen when a process was piped to an ordinary Lisp function,
like in '*echo hi | echo bye'.
* lisp/eshell/esh-cmd.el (eshell-do-pipelines): Set the tailproc even
for the first process in the pipeline.
* test/lisp/eshell/eshell-tests.el (eshell-test/pipe-tailproc): New
test.
This covers the case when a subcommand is to be invoked in more places
than before, for example when a subcommand is concatenated in an
argument.
* lisp/eshell/esh-cmd.el (eshell--find-subcommands): New fuction.
(eshell--invoke-command-directly): Use 'eshell-find-subcommands'.
* test/lisp/eshell/eshell-tests.el
(eshell-test/interp-cmd-external-concat): New test (bug#30725).
When an Eshell command contains an asynchronous subcommand (such as
calling an external process), it must be evaluated iteratively. See
bug#30725.
* lisp/eshell/esh-cmd.el (eshell-invoke-command): Move most of the
logic from here...
(eshell--invoke-command-directly): ... to here. Also add checks for
subcommands.
* test/lisp/eshell/eshell-tests.el (eshell-test--max-subprocess-time):
New variable.
(eshell-wait-for-subprocess): New function.
(eshell-command-result-p): Use 'eshell-wait-for-subprocess'.
(eshell-test/interp-cmd-external): New test (bug#30725).
* lisp/eshell/esh-util.el (eshell-user-name): Redefine as obsolete
function alias for 'user-login-name'.
(eshell-copy-tree): Redefine as obsolete function alias for
'copy-tree'.
* lisp/eshell/esh-cmd.el (eshell-do-eval): Don't use above
obsolete alias.
* lisp/eshell/esh-cmd.el (eshell-invoke-directly): Fix problem
with (require 'em-tramp) making password prompting from "sudo
bash" no longer work (bug#43772).
* lisp/eshell/esh-cmd.el (pcomplete--here): Declare function to
avoid byte compilation warning, which is what erc-notify.el also
does in a similar situation.
* lisp/eshell/esh-arg.el: Move defsubst and vars before first use.
Don't require `esh-mode but esh-util instead.
* lisp/eshell/esh-cmd.el: Require esh-module and esh-io.
* lisp/eshell/esh-ext.el: Don't require esh-proc nor esh-cmd.
(eshell-external-command): Require esh-proc for
eshell-gather-process-output.
* lisp/eshell/esh-mode.el: Don't require esh-io nor esh-var, but
require esh-arg.
(eshell-directory-name): Move from eshell.el.
* lisp/eshell/esh-module.el: Don't require eshell.
* lisp/eshell/esh-opt.el: Don't require esh-ext at top-level.
(eshell--do-opts, eshell-show-usage): Require it here instead.
* lisp/eshell/esh-proc.el: Don't require esh-cmd, but require esh-io.
(eshell-reset-after-proc, eshell-record-process-object)
(eshell-gather-process-output, eshell-send-eof-to-process):
Require esh-mode and esh-var here.
* lisp/eshell/esh-var.el: Require esh-module, esh-arg, and esh-io.
* lisp/eshell/eshell.el: Require esh-module, esh-proc, esh-io, and esh-cmd.
But don't require esh-mode.
(eshell-directory-name): Move to esh-mode.
(eshell-return-exits-minibuffer): Don't bind 'return' and 'M-return'
since we already bind RET and M-RET.
* lisp/eshell/em-dirs.el (eshell-dirs-initialize): Reveal the lambdas.
(eshell-expand-multiple-dots): Fix ^+$ => \`+\' in the regexp.
* lisp/eshell/esh-cmd.el (eshell-this-command-hook): Declare as
dynamically scoped.
(eshell-trap-errors): Use `mapc funcall` since this can't have
global/local settings like a true hook.
(eshell-do-eval): Split the `let` case from the rest so we can use
`cl-progv` rather than `eval` for it.
(eshell/which): Use `fboundp` test instead of ugly
gymnastics to try and hide the function call from the compiler.
* lisp/eshell/esh-var.el (eshell-variable-aliases-list): Reveal the lambdas.
(eshell-parse-variable-ref): Use backquotes.
Most of this change is to boilerplate commentary such as license URLs.
This change was prompted by ftp://ftp.gnu.org's going-away party,
planned for November. Change these FTP URLs to https://ftp.gnu.org
instead. Make similar changes for URLs to other organizations moving
away from FTP. Also, change HTTP to HTTPS for URLs to gnu.org and
fsf.org when this works, as this will further help defend against
man-in-the-middle attacks (for this part I omitted the MS-DOS and
MS-Windows sources and the test tarballs to keep the workload down).
HTTPS is not fully working to lists.gnu.org so I left those URLs alone
for now.
* lisp/help-fns.el (help-fns--analyse-function)
(help-fns-function-description-header): New functions, extracted from
describe-function-1.
(describe-function-1): Use them.
* lisp/eshell/esh-cmd.el (eshell/which): Use
`help-fns-function-description-header' instead of
`describe-function-1'.
