This fixes bug#59495.
Before this change, python tries to skip nested function
definition. Now we don't skip any nested defun.
* lisp/progmodes/python.el (python-treesit-beginning-of-defun)
(python-treesit-end-of-defun): Remove functions.
* lisp/progmodes/python.el (python-ts-mode): Use tree-sitter's default
navigation function.
Generating imenu index doesn't require going down to the bottom of the
tree (defun's are usually top-level). Add limit so we don't go too
far down on very large buffers.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--imenu)
* lisp/progmodes/java-ts-mode.el (java-ts-mode--imenu)
* lisp/progmodes/js.el (js--treesit-imenu)
* lisp/progmodes/json-ts-mode.el (json-ts-mode--imenu)
* lisp/progmodes/python.el (python-imenu-treesit-create-index)
* lisp/textmodes/css-mode.el (css--treesit-imenu): Add limit to
treesit-induce-sparse-tree.
To keep consistency with other tree-sitter major modes.
* lisp/progmodes/python.el (python--treesit-settings)
(python-ts-mode): Merge function-name and class-name features into
definition.
* lisp/progmodes/python.el (python-base-mode): New virtual mode that
contains most of the setup.
(python-mode): Change to inherit from python-base-mode.
(python-ts-mode): New mode that sets up tree-sitter.
Forgot to update python-mode's code when I added the tree-sitter
notifier facility. Now it doesn't need any special treatment anymore.
Leaving it as is causes some incorrect fontification.
* lisp/progmodes/python.el (python--treesit-fontify-string): Accept
the string node rather than the quote node.
(python--treesit-settings): Capture string node.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--font-lock-settings): Use new
escape, number, property, operator, bracket, and delimiter font-lock
faces.
(c-ts-mode--base-mode): Add them to the feature list.
(c-ts-mode): Fix typo.
* lisp/progmodes/python.el (python--treesit-settings): Use new number,
property, operator, bracket, and delimiter font-lock faces.
(python-mode): Add them to the feature list.
Now nodes that are affected and changed during re-parse will be
correctly refontified. Nodes can change even if it corresponding text
wasn't edited: additional text can complete the parse tree and resolve
error nodes, for example.
* lisp/progmodes/python.el (python-mode): Create Python parser before
calling treesit-major-mode-setup.
* lisp/treesit.el (treesit--font-lock-notifier): New function.
(treesit-major-mode-setup): Register fontifier with every existing
parser.
We now have a better facility that can replace the contextual
hack. The C part is in the previous commit, and the Lisp part work
will be in the next commit.
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--treesit-font-lock-settings)
* lisp/progmodes/python.el (python--treesit-settings)
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Stop marking
contextual nodes.
* lisp/treesit.el (treesit-font-lock-contextual-post-process): Remove
function.
(treesit-font-lock-fontify-region): Remove code processing contextual
nodes.
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Update
function to only fontify within the region.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Pass START and END to
fontification functions.
* lisp/progmodes/python.el: Remove function.
(python--treesit-settings): Capture contextual node.
* lisp/treesit.el (treesit--set-nonsticky):
(treesit-font-lock-contextual-post-process): New functions.
(treesit-font-lock-fontify-region): Change local variable START and
END to NODE-START and NODE-END, handle special capture name
"contextual".
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
Change from
(START END NODE OVERRIDE &rest _)
to
(NODE OVERRIDE &rest _)
START and END aren't used frequently enough to justify. If a
fontification function needs them, it can get them from NODE.
* doc/lispref/modes.texi (Parser-based Font Lock): Update manual.
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string)
(python--treesit-fontify-string-end): Change signature.
* lisp/treesit.el (treesit-font-lock-rules): Update docstring.
(treesit-font-lock-fontify-region): Remove START and END arguments.
Now when user inserts a ending quote, the whole string is ganranteed
to be refontified and redisplayed.
* lisp/progmodes/python.el (python--treesit-fontify-string): Change
docstring, now it's called on the leading quote.
(python--treesit-fontify-string-end): New function.
(python--treesit-settings): Capture both leading and ending quote in a
string.
* lisp/progmodes/python.el (python--treesit-fontify-string): Handle
not only f-strings, but also docstrings, and NODE is now the last
quote rather than the whole string.
(python--treesit-settings): Use python--treesit-fontify-string for
every occasion.
This version: central variable, everything controlled by
treesit-settings. Major mode sets up tree-sitter/non-tree-sitter
in a conditional branch, based on the setting.
* lisp/treesit.el (treesit-settings): New option.
(treesit-defun-type-regexp): Change docstring.
(treesit-mode-supported)
(treesit-required-languages)
(treesit--local-variable-backup): Remove variables.
(treesit--backup-local-variable)
(treesit-mode)
(global-treesit-mode--turn-on)
(global-treesit-mode): Remove functions.
(treesit--setting-for-mode): New function.
(treesit-ready-p): New argument MODE, changed REPORT to QUIET, and
LANGUAGEs to LANGUAGE (now it can be a single symbol or a list of
them).
(treesit-major-mode-setup): New function. Mostly comes from
treesit-mode.
* test/src/treesit-tests.el (treesit-misc): New test.
* lisp/progmodes/python.el (python-mode): Move some setup code into
the conditional branch at the end.
* lisp/progmodes/js.el (js-json-mode)
(js-mode): Move some setup code into the conditional branch at the
end.
* lisp/progmodes/ts-mode.el: Move tree-sitter setup into the
conditional branch.
Now instead of a toggle function (major-mode-backend-function), we let
major mode set local variables like treesit-font-lock-settings,
treesit-imenu-function, then treesit-mode takes care of activating
those things (clearing font-lock-keywords, setting
imenu-create-index-function to treesit-imenu-function, etc).
js.el and python.el: I've returned js-mode and python-mode to exactly
what they were before tree-sitter change, plus lines at the end
setting up tree-sitter variables. Sorry about all these fuss :-D
* lisp/treesit.el (treesit-mode-inhibit-message): Remove option.
(major-mode-backend-function)
(treesit-remapped-major-mode-alist): Remove variables.
(treesit-mode): Move down to the end of buffer. Do more things.
(global-treesit-mode): Move down to the end of buffer. Don't handle
major-mode-remap-alist anymore.
(global-treesit-mode--turn-on): Move down to the end of buffer.
(treesit-ready-p): Move down to the end of buffer. Changed signature.
(treesit-font-lock-enable): Remove function.
(treesit-defun-type-regexp): New variable.
(treesit-beginning-of-defun)
(treesit-end-of-defun): New function.
(treesit-imenu-function): New variable.
(treesit-mode-supported)
(treesit-required-languages)
(treesit--local-variable-backup): New variables.
(treesit--backup-local-variable): New function
* lisp/progmodes/js.el (js-use-tree-sitter): Remove option.
(js--treesit-defun-type-regexp): Remove variable. (Now set inline in
js-mode.)
(js--treesit-beginning-of-defun)
(js--treesit-end-of-defun): Remove functions. (Now use
treesit-beginning/end-of-defun.)
(js--backend-toggle)
(js--json-backend-toggle): Remove function.
(js-mode)
(js-json-mode): Restore back to before tree-sitter changes. Add
tree-sitter setup at the end.
* lisp/progmodes/python.el (python--backend-toggle): Remove function.
(python-mode): Restore back to before tree-sitter changes. Add
tree-sitter setup at the end.
* lisp/progmodes/ts-mode.el (ts-mode--font-lock-settings): Use
js--fontify-template-string.
(ts-mode--fontify-template-string): Remove function (because we can
just use js--fontify-template-string).
(ts-mode--defun-type-regexp): Remove variable (now set inline in
ts-mode).
(ts-mode--beginning-of-defun)
(ts-mode--end-of-defun): Remove functions (now using
treesit-beginning/end-of-defun).
(ts-mode): Setup tree-sitter variables and then turn on treesit-mode
or move to js-mode.
Basically we now have treesit-mode and global-treesit-mode. Major
modes set major-mode-backend-function which treesit-mode calls to
activate/deactivate tree-sitter.
js.el needs a bit explanation: I'm so sorry for messing up the
history, but basically now js-mode and js-json-mode should be exactly
the same as before any tree-sitter change was introduced, sans some
initialization code that are moved to js(-json)--backend-toggle.
js-mode and js-json-mode now just sets major-mode-backend-function and
initialize with js(-json)--backend-toggle.
* lisp/treesit.el (treesit-mode-inhibit-message): New option.
(treesit-can-enable-p): Remove function.
(major-mode-backend-function)
(treesit-remapped-major-mode-alist): New variables.
(treesit-mode)
(global-treesit-mode): New minor modes.
(global-treesit-mode--turn-on)
(treesit-ready-p): New functions.
* lisp/progmodes/python.el: Remove option.
(python--backend-toggle): New function.
(python-mode): Remove the if-form, all the initialization code are
moved to python--backend-toggle, python-mode now just sets
major-mode-backend-function and initialize with
python--backend-toggle.
* lisp/progmodes/js.el (js--treesit-can-enable-p)
(js--json-treesit-can-enable-p)
(js--treesit-enable)
(js--json-treesit-enable): Remove functions.
(js--backend-toggle)
(js-json--backend-toggle): New function.
* lisp/progmodes/ts-mode.el (ts-mode): Use treesit-ready-p.
* lisp/progmodes/python.el (python-shell-completion-at-point): Add
check if point is before line-start.
* test/lisp/progmodes/python-tests.el (python-shell-completion-shell-buffer-1)
(python-shell-completion-shell-buffer-native-1): New tests
(bug#58548).
* lisp/progmodes/python.el (python-shell-completion-at-point): Disable
completion in Python buffer when PDB is active in Python Shell buffer.
* test/lisp/progmodes/python-tests.el (python-shell-completion-pdb-1):
New test (bug#58562).
Remove :toggle and :level, add :feature.
* lisp/progmodes/js.el (js--treesit-settings): Add :feature.
(js--treesit-enable): Set treesit-font-lock-feature-list.
* lisp/progmodes/python.el: Replace :level with :feature.
(python-mode): Set treesit-font-lock-feature-list.
* lisp/treesit.el (treesit-font-lock-feature-list): New variable.
(treesit-font-lock-settings): Change format.
(treesit-font-lock-rules): Remove :toggle and :level, add :feature.
(treesit-font-lock-recompute-features): New function.
(treesit-font-lock-fontify-region): Change to work with the new
format.
(treesit-font-lock-enable): Add call to
treesit-font-lock-recompute-features. And improve the font-lock-mode
workaround.
* lisp/progmodes/python.el (python-shell-send-string-no-output):
Save and restore `comint-last-prompt-overlay' or `comint-last-prompt'.
* test/lisp/progmodes/python-tests.el (python-tests-shell-wait-for-prompt):
New helper function.
(python-tests-with-temp-buffer-with-shell): New helper macro.
(python-shell-completion-1, python-shell-completion-native-1)
(python-shell-completion-native-with-ffap-1)
(python-shell-completion-native-with-eldoc-1): New tests (bug#58389).
* lisp/progmodes/python.el (python--treesit-settings): Add docstring.
(python--imenu-treesit-create-index-1): Rewrite with
treesit-induce-sparse-tree.
(python-imenu-treesit-create-index): Move main body to
python--imenu-treesit-create-index-1.
(python-imenu-treesit-create-flat-index): Fix typo.
(python-mode): Enable treesit-imenu. Also fix indentation for
which-func code.
* lisp/progmodes/python.el
(python-info-looking-at-beginning-of-defun): Check syntax after moving
to the beginning of line.
* test/lisp/progmodes/python-tests.el (python-nav-beginning-of-defun-6)
(python-end-of-defun-1, python-info-looking-at-beginning-of-defun-3):
New tests (bug#58023).
