Commit graph

429 commits

Author SHA1 Message Date
Yuan Fu
ce7b7e5af3
Remove comment-start-skip preset in tree-sitter indentation engine
Comment indentation should use a adaptive-fill-based indent, rather
than comment-start-skip.

Also remove manual description of removed variables in treesit.el and
add documentation for n-p-gp upon request.

* doc/lispref/modes.texi (Parser-based Indentation)
* lisp/treesit.el (treesit-simple-indent-presets): Add n-p-gp, remove
treesit-comment-start/end, remove comment-start-skip.

* doc/lispref/parsing.texi (Tree-sitter major modes): Remove
treesit-comment-start/end.
2022-12-18 14:48:38 -08:00
Theodor Thornhill
69f2c71135
Fix treesit-query-validate problem with view-mode
* lisp/treesit.el (treesit-query-validate): Move the (view-mode) into
the correct scope.
2022-12-17 15:33:55 -08:00
Yuan Fu
67bc96c9d9
Remove treesit-defun-prefer-top-level and friends
Because they are not used anymore, after the new defun navigation is
installed.

* lisp/treesit.el (treesit-defun-prefer-top-level)
(treesit--defun-maybe-top-level): Remove.
2022-12-16 15:04:32 -08:00
Yuan Fu
3d348c46e7
; Handle nil ARG in treesit-beginning/end-of-defun
* lisp/treesit.el (treesit-beginning-of-defun)
(treesit-end-of-defun): Handle nil ARG.
2022-12-16 14:53:45 -08:00
Stefan Kangas
03892d4f7c Merge from origin/emacs-29
033071692c ; Fix typos
f4a513344d Add lambda_expression-rule to java-ts-mode (bug#60091)
546aed3543 eglot: Add support for new language server csharp-ls
cb761eb7ac Use the new tree-sitter commands
037407ad95 Add "function" feature to python-ts-mode (bug#59977)
fee2efe1b0 Add go-ts-mode and go-mod-ts-mode (Bug#60025)
e8f7ab67ad Add basic support for hideshow in python-ts-mode (bug#60044)
cac070b23e Add "this" keyword to java-ts-mode (bug#60086)
c8d75046a2 When completing relative project file names, use relative...
3b618d0e3e Avoid segfaults due to invalid selected-window's buffer

# Conflicts:
#	lisp/progmodes/sh-script.el
2022-12-16 08:45:06 +01:00
Stefan Kangas
033071692c ; Fix typos 2022-12-16 08:29:28 +01:00
Yuan Fu
cb761eb7ac
Use the new tree-sitter commands
* lisp/progmodes/c-ts-mode.el (c-ts-mode--defun-valid-p)
(c-ts-mode--defun-skipper): New functions.
(c-ts-base-mode): Setup defun navigation.
* lisp/progmodes/sh-script.el (bash-ts-mode): Setup defun navigation.
* lisp/treesit.el (treesit-beginning-of-defun)
(treesit-end-of-defun): Change to new implementation, which is
intended to be used as commands.
(treesit-major-mode-setup): Setup remap for beginning/end-of-defun
commands.
2022-12-15 17:44:07 -08:00
Stefan Kangas
fa70076974 Merge from origin/emacs-29
fafcf02c85 Fix syntax tables of tree-sitter modes (bug#59807)
91b8d9b7db ; * admin/notes/tree-sitter/build-module/build.sh: Minor ...
1b0e282a7f ; Comment and stylistic change in treesit.el
489b02d03c * doc/misc/Makefile.in (need_emacsver): Add use-package.
2022-12-15 21:13:13 +01:00
Yuan Fu
1b0e282a7f
; Comment and stylistic change in treesit.el
* lisp/treesit.el (treesit-defun-skipper): Docstring change.
(treesit--navigate-defun): Comment and stylistic change.
2022-12-13 18:04:32 -08:00
Stefan Kangas
8036739c1b Merge from origin/emacs-29
cd5856e403 Fix bug when calling `rgrep` non-interactively
ba4bdd6a25 Adapt Tramp specific tests in eglot-tests.el
1d5c35c8e4 * lisp/textmodes/texinfo.el (texinfo-flymake): Improve do...
a99d0e7e6c Support a function in the BUFFER-LIST arg of list-buffers...
def51dd645 ; Fix typos
4980ed7a6d Don't allow lazy highlight from recursive minibuffers
4ef8b9f544 Improve resetting face attributes when looking for suitab...
c4b8bc90a8 ; Fix typos in doc strings
c45eb13845 ; * lisp/bs.el (bs-attributes-list): Doc fix
d6adaf487d Add lexical-binding to example package header
03ad1a92a2 Add improved tree-sitter navigation
a5272e2a7c ; * test/src/treesit-tests.el: Add outline headers.
489bcacc7c Add cross-reference to flush-lines
0f9e6532b1 Use font-lock-number-face for numeric values in csharp-mode
4bccb7b211 Make treesit-query-validate create a read-only buffer
c0fe6c72ce Improve dockerfile-ts-mode imenu generation (Bug#59979)
631908f701 Add "->" to python--treesit-operators (bug#59968)
5d4274d9b6 ; * admin/notes/tree-sitter/build-module/build.sh: Add -f...
d264b75669 Align C++ access specifiers to their enclosing class/stru...
ca67d988d8 Add cmake-ts-mode
8ec923775d Tweak various ts-mode's indent and fontification (bug#59931)
647b6a8099 Add expression for generic_name in csharp-ts-mode (bug#59...
5b178efd85 ; Adjust eglot test to recent autopep8/pycodestyle
58b8ed8b55 ; Avoid compilation warning on MS-Windows
40c23c11e8 * lisp/outline.el: Fix the value 'insert' of outline-mino...
527eb11de2 * lisp/minibuffer.el (completions-group-separator): Rever...
42d740fb2c ; Skip two eglot tests when typescript is missing
19ef86f775 ; Remove outdated text describing overlays
081bf58300 Skip Eglot rust-analyzer tests if 'cargo' isn't available

# Conflicts:
#	lisp/progmodes/typescript-ts-mode.el
#	lisp/treesit.el
2022-12-14 00:06:29 +01:00
Stefan Kangas
def51dd645 ; Fix typos 2022-12-13 18:44:41 +01:00
Eli Zaretskii
c4b8bc90a8 ; Fix typos in doc strings
* lisp/treesit.el (treesit-defun-tactic)
(treesit-default-defun-skipper): Fix typos in doc strings.
2022-12-13 16:01:13 +02:00
Yuan Fu
03ad1a92a2
Add improved tree-sitter navigation
This new set of functions (and tests) should eliminate
defun-navigation bugs and limitations we currently have.  This commit
doesn't change any existing bahavior: treesit-beginning/end-of-defun
and friends are unchanged.  The plan is to later switch gear and
replace the current functions with the new ones introduced in this
change.

This is a relatively big change, but I've setup a comprehensive test,
and it should fix current bugs, so I think it's ok to put it on the
release branch.

The gist of the new navigation is to use treesit--defuns-around to
find the previous sibling defun, next sibling defun, and the parent
defun, then use this information to move to previous/next
beginning/end of defun in treesit--navigate-defun.

I also added comprehensive testing that tests all four possible
operations (prev-beg, next-beg, prev-end, next-end) starting at all
possible positions (between two sibling defuns, inside a sibling
defun, etc).

* lisp/treesit.el (treesit-defun-type-regexp): Expand definition to
allow (REGEXP . FILTER).  Old functions don't support this, but it
should be fine since we are soon replacing them.

(treesit-defun-tactic)
(treesit-defun-skipper): New variables.

(treesit-default-defun-skipper)
(treesit--defuns-around)
(treesit--top-level-defun)
(treesit--navigate-defun): New functions.

* test/src/treesit-tests.el (treesit--ert-insert-and-parse-marker)
(treesit--ert-collect-positions)
(treesit--ert-test-defun-navigation): New helper functions.

(treesit--ert-defun-navigation-python-program)
(treesit--ert-defun-navigation-js-program)
(treesit--ert-defun-navigation-bash-program)
(treesit--ert-defun-navigation-nested-master): New variables.

(treesit-defun-navigation-nested-1)
(treesit-defun-navigation-nested-2)
(treesit-defun-navigation-nested-3)
(treesit-defun-navigation-top-level): New tests.
2022-12-12 21:17:40 -08:00
Jostein Kjønigsen
4bccb7b211
Make treesit-query-validate create a read-only buffer
* lisp/treesit.el: use view-mode for generated validation buffer.
2022-12-12 15:08:55 -08:00
Stefan Kangas
fcd0b377e0 Merge from origin/emacs-29
06ef030f93 use-package.texi: New section "Manual installation"
f4ce6fa7d3 Revert "Revert "Improve last change to xfaces.c" (05ece1e...
b8d2ec920f Revert "Improve last change to xfaces.c" (05ece1eb8b)
24c8c28ae6 Do not pare arguments unnecessarily.
9c0d7bb73b Add automated tests for Eglot
d3669cfe15 Eglot: allow skipping compile-time warnings about LSP int...
04b7e01885 ; project.el: Bump version.
f2876014ad Add customizale faces for tree-sitter explorer
3e349ee119 Fix error message when installing non-existent package
733cdeabfb Don't use diff-mode buffer as a patch when it's visiting ...
87475f4af2 Fix pcase rx patterns using rx-let bindings (bug#59814)
4893a15631 Fix use-package-defaults defcustom type (bug#59941)
074b7e6f4d ; * lisp/use-package/bind-key.el: Remove ineffective back...
864ed9dfa1 ; * lisp/progmodes/dockerfile-ts-mode.el: use \' instead ...
9f7e5584a4 * lisp/language/indian.el: Improve Brahmi composition rul...
78ad33bb05 ; Minor cleanup of last change in xfaces.c.
2024ade271 ; Improve docs of relaxing face-font attribute match (bug...
2022-12-12 09:05:53 +01:00
Yuan Fu
f2876014ad
Add customizale faces for tree-sitter explorer
* lisp/treesit.el (treesit-explorer-anonymous-node)
(treesit-explorer-field-name): New face.
(treesit--explorer-draw-node): Use the new faces.
(treesit-explore-mode): Change playground to explorer.
2022-12-11 14:51:33 -08:00
Theodor Thornhill
b889eced44 Add prog-fill-reindent-defun (bug#59664)
Introduce a new command that aims to reindent code in a defun, or fill
a paragraph of text.  The command uses treesit.el when available,
otherwise falls back to using syntax-ppss and regexps.  Treesit.el
needs a new variable that is intended to be set by the major modes so
that this and other future functions can know what kind of node we are
looking at.

* doc/emacs/programs.texi: Mention the new command.
* etc/NEWS: Mention the new command.
* lisp/progmodes/c-ts-mode.el (c++-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/csharp-mode.el (csharp-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/java-ts-mode.el (java-ts-mode): Add regexp for the
new variable.
* lisp/progmodes/js.el (js-ts-mode): Add regexp for the new variable.
* list/progmodes/prog-mode.el (prog-mode-map): Bind the new command by
default.
(prog-fill-reindent-defun): New command.
* lisp/progmodes/sh-script.el (bash-ts-mode): Add regexp for the new
variable.
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-base-mode): Add
regexp for the new variable.
* lisp/treesit.el (treesit-text-type-regexp): New variable.
2022-12-11 22:31:31 +02:00
Eli Zaretskii
d3494f1bde Resurrect changes omitted by a recent merge from emacs-29 (bug#59921)
This includes the changes for the following commits:

  670daa8b
  b429e524
  c83c95634e
  6479691cf0
  b710ca62c0
  d31a253983
  a669d5fae5
  f7262b8f81
  fef1755736
  bf81df86e5
  bfc00f1c12
  d2411615e8
  dcf69a1d
2022-12-10 13:22:48 +02:00
Yuan Fu
670daa8b62
Add recursion limit to treesit--children-covering-range-recurse
* lisp/treesit.el (treesit--children-covering-range-recurse): Add
limit.
2022-12-07 16:36:07 -08:00
Yuan Fu
b429e52428
Improve tree-sitter fontification in edge-cases (bug#59738)
* lisp/treesit.el (treesit-font-lock-fontify-region): Use
treesit--children-covering-range-recurse.
2022-12-07 16:04:17 -08:00
Yuan Fu
c26fe45cb8
Fix treesit-query-capture
Before this change Ftreesit_query_capture doesn't convert character
position to byte position for BEG and END parameters.  I observed
fontification issue in css files but couldn't figure out why, now I
know :-)

I decide to keep treesit--font-lock-query-expand-range, since it might
provide a escape hatch for problems we discover in the future, and it
should be very cheap so no downside of keeping it.

* lisp/textmodes/css-mode.el (css-ts-mode): Stop setting
treesit--font-lock-query-expand-range.
* lisp/treesit.el (treesit--font-lock-query-expand-range): Update
docstring.
* src/treesit.c (Ftreesit_query_capture): Convert BEG and END to byte
position.  Also added parentheses wround "beg_byte - visible_beg" in
the call to ts_query_cursor_set_byte_range (i.e., style change).
2022-12-05 19:56:47 -08:00
Yuan Fu
ec00d292ec
Improve treesit-fontify-with-override
This also fixes fontification problem with c-ts-mode--fontify-defun.

Now treesit-fontify-with-override clips the fontification region for
the user, so no need for (max start node-start) shenanigans anymore.
More importantly it doesn't fontify unless the region between
node-start and node-end intersects with the region between start and
end, which fixes the problem with c-ts-mode--fontify-defun.

* lisp/treesit.el (treesit-fontify-with-override): Add optional
parameter BOUND-START and BOUND-END.  Wrap the function body in a
when-form.
* lisp/progmodes/c-ts-mode.el (c-ts-mode--fontify-declarator)
(c-ts-mode--fontify-variable)
(c-ts-mode--fontify-defun)
(c-ts-fontify-error)
* lisp/progmodes/js.el (js--fontify-template-string)
* lisp/progmodes/python.el (python--treesit-fontify-string): Use the
new signature.
2022-12-04 20:03:28 -08:00
Yuan Fu
520a4e12f8
; * lisp/treesit.el (treesit-end-of-defun): Guard against nil value. 2022-12-03 14:49:37 -08:00
Eli Zaretskii
a86ccb5f9d ; Fix recently added treesit documentation
* lisp/treesit.el (treesit-defun-prefer-top-level):
* doc/lispref/positions.texi (List Motion): Fix wording.
2022-12-01 17:25:28 +02:00
Yuan Fu
01e7d4b2a1
Make treesit-defun-prefer-top-level more flexible
* doc/lispref/positions.texi (List Motion): Update manual.
* lisp/treesit.el (treesit-defun-prefer-top-level): Update docstring.
(treesit--defun-maybe-top-level): Change to accept new format.
2022-11-30 16:57:08 -08:00
Yuan Fu
d5dc1dbf7c
Remove treesit-comment-start/end and use comment-start/end-skip
treesit-comment-start/end is unnecessary because of
comment-start/end-skip, so they should be removed.

Cleanup and set comment-start/end-skip for tree-sitter C-like major
modes.

I replaced the [ \t]* part in comment-start-skip with (syntax
whitespace), which is what comment-end-skip uses.  I also added
grouping in comment-start-skip to match that of comment-end-skip.

* lisp/progmodes/c-ts-mode.el (c-ts-mode)
(c++-ts-mode)
* lisp/progmodes/csharp-mode.el (csharp-ts-mode)
* lisp/progmodes/java-ts-mode.el (java-ts-mode)
* lisp/progmodes/js.el (js-ts-mode)
* lisp/progmodes/typescript-ts-mode.el (typescript-ts-mode): Setup
comment-start/end-skip.

* lisp/treesit.el (treesit-comment-start)
(treesit-comment-end): Remove variables.
(treesit-simple-indent-presets): Use comment-start/end-skip instead.
2022-11-27 14:20:56 -08:00
Eli Zaretskii
80dcd78ff1 ; Fix recent documentation changes in treesit.el
* lisp/treesit.el (treesit-font-lock-level)
(treesit-font-lock-recompute-features): Doc fixes.
2022-11-27 10:06:02 +02:00
Yuan Fu
447b9d48d9
Add treesit-font-lock-level
This replaces font-lock-maximum-decoration and allows us to disable
the busiest fontification level by default.

* lisp/treesit.el (treesit-font-lock-level): New variable.
(treesit-font-lock-feature-list)
(treesit-font-lock-settings): Change docstring.
(treesit-font-lock-recompute-features): Use the new variable.
2022-11-26 15:40:24 -08:00
Yuan Fu
73c94d5a9f
; Fix comment-end in treesit-simple-indent-presets
* lisp/treesit.el (treesit-simple-indent-presets): Fix comment-end.
2022-11-25 19:00:22 -08:00
Stefan Kangas
4ee5618c6b ; Remove unused variable treesit-imenu-function
* lisp/treesit.el (treesit-imenu-function): Remove unused
variable.  (Bug#59475)
2022-11-25 08:58:57 +01:00
Yuan Fu
183c66be97
; Relayout comments in treesit-font-lock-fontify-region
; * lisp/treesit.el (treesit-font-lock-fontify-region): Relayout
comments.
2022-11-23 19:48:28 -08:00
Yuan Fu
00e7da76a7
Don't always filter out captured tree-sitter nodes outside of region
* lisp/treesit.el (treesit-font-lock-fontify-region): If the capture
name is a function, don't filter.
2022-11-23 19:48:28 -08:00
Yuan Fu
6785273a82
More flexible tree-sitter defun navigation
Before this change, treesit-beginning-of-defun skips nested defuns.
Now user can decide whether to skip nested defuns.

* lisp/treesit.el (treesit-search-forward-goto): Improve docstring.
(treesit-defun-prefer-top-level): New variable.
(treesit--defun-maybe-top-level): New function.
(treesit-beginning-of-defun)
(treesit-end-of-defun): Use treesit--defun-maybe-top-level.
2022-11-23 13:27:56 -08:00
Stefan Kangas
5e8c62ffca ; Fix typos 2022-11-23 05:49:22 +01:00
Yuan Fu
17ed39cc8d
; Remove obsolete use of inhibit-point-motion-hooks
* lisp/treesit.el (treesit-font-lock-fontify-region): Remove it.
2022-11-22 19:57:44 -08:00
Yuan Fu
c69858b3f0
; * lisp/treesit.el (treesit-ready-p): Guard against empty buffers. 2022-11-22 17:29:26 -08:00
Yuan Fu
057901f55a
Fix treesit-update-ranges
* lisp/treesit.el (treesit--clip-ranges): New function.
(treesit-update-ranges): Now clips the range within (point-min)
and (point-max), so the new range we use are not out-of-range.
2022-11-22 12:38:20 -08:00
Yuan Fu
2fa6ccab01
; Fix treesit--children-covering-range
* lisp/treesit.el (treesit--children-covering-range): Handle the case
when NODE doesn't have children, and when CHILD is nil.
2022-11-22 12:38:14 -08:00
Eli Zaretskii
368d2531be Fix 'treesit-max-buffer-size' and its use
* lisp/treesit.el (treesit-max-buffer-size): Avoid overflow in
computing buffer-size limit.  Account for 32-but systems built
"--with-wide-int".  Extend doc string.
(treesit-ready-p): Compare the limit with the size of the buffer
in bytes, not in characters.

* src/treesit.c (treesit_check_buffer_size): Measure buffer size
in bytes.
2022-11-22 20:22:41 +02:00
Yuan Fu
6e07838051
treesit-font-lock-recompute-features now has two modes of operation
1. Set activation of each feature (changes every feature)
2. Add/remove features (only change those explicitly configured by
ADD-LIST and REMOVE-LIST)

This is useful for enabling/disabling certain features for all
modes (without resetting others) by calling this function in
prog-mode-hook

* lisp/treesit.el (treesit-font-lock-recompute-features): See above
description.
2022-11-22 01:06:51 -08:00
Yuan Fu
c7202e7f4a
; Fix docstrings in treesit.el
* lisp/treesit.el (treesit--explorer--nodes-to-highlight)
(treesit--explorer-draw-node): Fix docstrings.
2022-11-22 01:06:51 -08:00
Yuan Fu
74892eefc8
* lisp/treesit.el (treesit-max-buffer-size): Enlarge value. 2022-11-22 01:06:51 -08:00
Juanma Barranquero
ea73fd69f0 ; Fix typos in tree-sitter files
* admin/notes/tree-sitter/starter-guide (Font-lock)
(Debugging queries, Indent, Navigation, Which-func)
(More features?):
* lisp/treesit.el (treesit--merge-ranges)
(treesit-font-lock-feature-list, treesit-font-lock-rules)
(treesit-font-lock-fontify-region, treesit--font-lock-notifier)
(treesit-simple-indent-presets, treesit--font-lock-fast-mode)
(treesit--indent-region-batch-size)
(treesit--indent-rules-optimize, treesit-ready-p): Fix typos.
2022-11-22 04:40:49 +01:00
Stefan Kangas
40539c7587 ; Fix typos 2022-11-22 02:24:20 +01:00
Yuan Fu
6cf8bf5042
Allow major modes to tweak tree-sitter fontification
treesit--font-lock-query-expand-range allows a major mode to fix
fontification problems temporarily before the parser can be fixed.

* lisp/treesit.el (treesit--font-lock-query-expand-range): New variable.
(treesit-font-lock-fontify-region): Use the new variable.
* lisp/textmodes/css-mode.el (css-ts-mode): Use the new variable.
2022-11-21 11:26:46 -08:00
Yuan Fu
0cbb7d3bd0
Further tweak tree-sitter fontification heuristics
So it turns out the slowness observed in bug#59415 is not due to the
size, but the strangely tall tree.  Adjust the heuristic to DTRT:
don't enable the heuristic by default or when buffer is large, enable
when query is abnormally slow.  We could do some clever thing that
calibrates a base reading for the query time instead of using a
hard-coded value, but it doesn't seem necessary.

* lisp/treesit.el (treesit--font-lock-fast-mode): New variable.
(treesit-font-lock-fontify-region): Don't activate heuristic by
default (reasons in comments).  Measure the query time and activate
the fast mode if query time is long.
2022-11-21 10:52:55 -08:00
Eli Zaretskii
27e89ff116 ; * lisp/treesit.el (treesit-parser-add-notifier): Add declaration. 2022-11-21 17:24:42 +02:00
Yuan Fu
938e68bb28
Fix tree-sitter fontification heuristic
Previously applied heuristic 2 sometimes invalidates heuristic 1, add
a guard so it doesn't.

The new function is just for clearity of the code and has nothing to
do with the change itself.

* lisp/treesit.el (treesit--node-length): New function
(treesit-font-lock-fontify-region): Use the new function.  Only do
heuristic 2 when the node is large.
2022-11-21 01:29:31 -08:00
Yuan Fu
76eb88df02
Add another heuristic to speed up tree-sitter fontification
* lisp/treesit.el (treesit-font-lock-fontify-region): Use
treesit--children-covering-range to reduce the node size.
2022-11-20 17:15:42 -08:00
Yuan Fu
625ea08652
Improve tree-sitter fontification on large buffers
* lisp/treesit.el (treesit--children-covering-range)
(treesit--children-covering-range-recurse): New functions.  They are
not currently used but could be useful in the future, so I left them
in place.
(treesit-font-lock-fontify-region):
* lisp/treesit.el (treesit-font-lock-fontify-region): Use the result
of treesit-node-on instead of the root node.
2022-11-20 17:04:58 -08:00