* lisp/org-element.el (org-element--cache-diagnostics-modifications):
(org-element--cache-silent-modification-check): Remove variables that
were introduced to deal with false positives. The new check is more
reliable.
(org-element--cache-last-buffer-size): New variable storing the last
known buffer size registered before/after handled buffer edits.
(org-element--cache-sync): Use `buffer-size' instead of
`buffer-chars-modified-tick' heuristics. `buffer-size' may not catch
all the modifications, but it does not cause numerous
false-positives triggered by quail. See Emacs Bug#51766 for the
detailed discussion.
(org-element--cache-before-change):
(org-element-cache-reset): Record buffer size before/after changes.
* lisp/org-macs.el (org-unique-local-variables): Do not carry over
buffer-local `org-element--cache-last-buffer-size' when cloning
buffer.
Emacs now advises using "website" (instead of "homepage") and
consequently to use the "URL" comment header instead of "Homepage".
* README:
* README_ELPA:
* doc/org-guide.org (Creating Footnotes):
* doc/org-manual.org (Creating Footnotes):
* etc/styles/README (URL): Prefer "website" to "homepage".
* lisp/*.el:
* mk/org-fixup.el: Replace the "Homepage" header comment with "URL".
* lisp/org-element.el (org-element--cache-setup-change-functions): New
function setting up `org-element--cache-before-change' and
`org-element--cache-after-change'.
(org-element-cache-reset): Setup before/after-change-functions inside
indirect buffers.
* lisp/org-element.el (org-element--cache-key-less-p): Clarify that
comparison between buffer points and cache keys is not reliable.
(org-element--cache-find): Do not compare POS with cache keys.
(org-element-at-point): Use cache BEG field to determine the cached
buffer portion.
* lisp/org-element.el (org-element--cache-for-removal): Directly check
for headline at point instead of running full
`org-element--current-element' that is matching a large number of
regexps.
* lisp/org-element.el (org-element--cache-hash-size):
(org-element--cache-hash-statistics):
(org-element--cache-hash-nocache):
(org-element--cache-hash-size):
(org-element--cache-hash-left):
(org-element--cache-hash-right): Implement recent search cache for
`org-element--cache-find'. The cache stores recent cache tree queries
as a vector with O(1) access time.
(org-element--cache-find): Make use of `org-element--cache-hash-left'
and `org-element--cache-hash-right' when cached query element starts
at POS and SIDE is nil. Record statistics.
(org-element-cache-reset): Reset search cache on init.
(org-element-cache-hash-show-statistics): Provide a command reporting
the new caching efficiency. Can be used for debugging/survey
purposes.
* lisp/org-macs.el (org-knuth-hash): Implement multiplicative hash
function.
Preliminary testing reveals that this simple strategy can reduce query
time from O(Log N) down to O(1) for ~30%-50% cache queries.
* lisp/org-element.el (org-element--current-element): Use
`org-element-dynamic-block-open-re' to match blocks.
* testing/lisp/test-org-element.el (test-org-element/paragraph-parser):
Fix test relying on previous incorrect behaviour. Empty #+BEGIN:
should be parsed as a keyword.
Reported in https://list.orgmode.org/Ycay4s3iAdEGSwgt@tuxteam.de/T/#t
* lisp/org-element.el (org-element--parse-to):
(org-element--cache-verify-element): Add check asserting that all the
parsed elements are assigned with non-nil parent (except org-data).
* lisp/org-element.el (org-element--cache-before-change): Fix
optimisation when edits are involving element :end position. Checking
that END is at BOL fails when we insert at the beginning of a
headline.
* lisp/org-element.el (org-element--cache-before-change):
(org-element--cache-after-change): Do not prevent trigerring if the
current indirect buffer is not in Org mode, but its base buffer is in
Org mode.
* lisp/org-element.el (org-element-context): Use
`org-element-at-point', which makes use of cache.
* testing/lisp/test-org-element.el (test-org-element/lineage): Expect
full lineage up to org-data from `org-element-context'.
* lisp/org-element.el (org-element--parse-to): Manually parse org-data
and set appropriate parser mode when POS is before first headline and
cache is disabled.
(org-element-org-data-parser--recurse): New variable flagging if
`org-element-data-parser' is called recursively.
(org-element-org-data-parser): Handle recursive calls from inside `org-element-at-point-no-context'.
This also fixes false-positives in `org-element--cache-verify-element'.
* lisp/org-element.el (org-element--cache-for-removal): Consider
preceding keywords to be updated unconditionally.
(org-element-cache-map): Fix infinite loop revealed by the new test.
* testing/lisp/test-org-element.el (test-org-element/cache-affiliated):
New test.
* lisp/org-element.el (org-element--cache-silent-modification-check):
New variable allowing the user to suppress modification checks completely.
* lisp/org-element.el (org-element--cache-sync): Support
`org-element--cache-silent-modification-check'.
The existing heuristics for built-in non-modifying commands that
change `buffer-chars-modified-tick' is not sufficient. At least
`capitalize-word' changes the tick regardless whether the buffer was
actually modified or not. Yet, `capitalize-word' does not call
`after-change-functions' if the buffer is not really modified.
Reported in https://list.orgmode.org/2022-01-06T12-13-17@devnull.Karl-Voit.at/T/#mb3771758f81b31721ba2f420878a4d16081dc483
* lisp/org-element.el (org-element-cache-map-continue-from): New
variable forcing `org-element-cache-map' to continue from a custom
point in buffer.
(org-element-cache-map): Add support for
`org-element-cache-map-continue-from'. Update docstring accordingly.
Also, make sure that mapping terminates correctly when FUNC deletes
all elements in buffer.
* testing/lisp/test-org.el (test-org/map-entries): Add test.
Fixes https://orgmode.org/list/CADywB5KOJ1p0NpvA=iX-ybHsO=huGA8qL3xMpUTETmS2qp7_ng@mail.gmail.com
* lisp/org-element.el (org-element--parse-to): Disable cache in
`org-element--current-element'. When calling it here, we know for
sure that element at point is not yet in cache.
* lisp/org-element.el (org-element--cache-process-request): New OFFSET
argument used to correct newly added parents during Phase 1. The
`org-element--parse-to' call inside Phase 1 may add new elements to
cache that intersect with future edits. Boundaries of these elements
may be shifted twice, so we have to offset the future shift.
(org-element--cache-sync): New OFFSET argument providing future change
info to `org-element--cache-process-request'.
(org-element--cache-submit-request): Provide offset value in
`org-elemnt--cache-sync' call.
(org-element--cache-submit-request):
(org-element--cache-process-request):
(org-element--cache-sync): Never use %d format for region boundaries.
It may be a marker and cause error. Use %S instead.
(org-element--cache-process-request): Use unique symbols for
catch-throw.
Fixes https://list.orgmode.org/CAFyQvY3Qv5xn-ET83L6Rzg-V1zOVu4y1gt+-_CpfaWNAdt87xA@mail.gmail.com/T/#t
* lisp/org-element.el (org-element--cache-after-change): Always extend
changed region to bol.
* testing/lisp/test-org-element.el (test-org-element/cache): Add test
checking the new fix. Amend some tests around making sure that cache
is active during testing.
* lisp/org-element.el (org-element--cache-sync): Set
`org-element--cache-warning' to nil at the end of synchronisation.
We do not need to consider that next request might be merged with
existing request in such scenario.
(org-element--cache-before-change): Add comment explaining the use of `org-element--cache-warning'.
* lisp/org-element.el (org-element--cache-after-change): Extend
changed region to bol when we are editing near beginning of an element
within or right after indentation. Such edits potentially change
:post-blank value of the previous element.
* testing/lisp/test-org-element.el (test-org-element/cache): Add test
checking one of such cases.
* lisp/org-element.el (org-element--cache-for-removal): Drawers are
generally robust elements (they cannot be changed if a non-sensitive
change is made inside their contents). The only exception is
PROPERTIES drawer that may switch back and forth between ordinary
drawer and properties drawer depending on its contents. The old code
treated all possible drawer as non-robust for this reason, degrading
performance on large LOGBOOK drawers that are now processed much
faster since they do not need to be removed and re-parsed on every
single change.
* lisp/org-element.el (org-element--cache-sensitive-re): Remove
#+begin_ and \begin{ lines from sensitive regexp. Such changes can
be handled without a need to re-parse from earlier position. If a
change introduces a new element, the new element will be recognised
and the unupdated elements not intersecting with the new element will
be handled during Phase 1.
* lisp/org-element.el (org-element--cache-before-change): Do not match
`org-element--cache-sensitive-re' against an element starting at END
when END is at bol. Such changes never break an element structure (as
opposed to inserting non-newline right at :begin of an element).
* lisp/org-element.el (org-element--cache-for-removal): Only force
re-parsing headlines when the changed region falls within the headline
before property drawer end position.
* lisp/org-element.el (org-element--cache-submit-request): Extend
calculation of affected parent elements to the full changed region.
The previous behaviour could miss some outer elements to be removed.
* lisp/org-element.el (org-element--cache-sensitive-re): Do not treat
properties inside property drawers as global sensitive change. It can
cause cache drop on large logbook drawers where a new clock entry is
inserted.
(org-element--cache-for-removal): Use the same technique used for
detecting property drawer <-> ordinary drawer changes under headlines
for org-data.
* lisp/org-element.el (org-element-cache-map): Move all possible
let-bindings outside the loop to avoid remory re-allocation on every
iteration. Track statistics for `re-search-forward' calls.
* lisp/org-element.el (org-element--cache-sensitive-re): Make proprety
drawer lines sensitive.
(org-element--cache-for-removal): Make sensitive edits inside cache
gap right after org-data re-parse the org-data. Make sensitive
top-section edits re-parse org-data.
The specific error has been reported in
https://github.com/yantar92/org/issues/40
The recepy involves loading org-contrib:
1. emacs -Q -L ~/.emacs.d/.local/straight/repos/org/lisp -L ~/.emacs.d/.local/straight/repos/org-contrib/lisp -l org
-l org-eldoc
2. Create a test.org file with the following content:
:PROPERTIES:
:ID: test
:END:
Test.
3.
(goto-char (point-min))
(org-entry-put nil "ID" "test")
(save-buffer)
(org-entry-get nil "ID")
Return value is nil, while should be "test"
* lisp/org-element.el (org-element--cache-sync): Suppress silent edit
warnings for Emacs <28. In older Emacs, Emacs internal functions can
modify `buffer-chars-modified-tick' and cannot be distinguished from
dangerous edits under `inhibit-modification-hooks'. Some of the
functions even have the same footprint with the edits we want to
avoid.
* lisp/org-element.el (org-element-at-point): Describe warning as
parser error when `org-element--parse-to' throws an error. The error
may be thrown when parser encounters issue not related to cache.
Fixes confusion in https://list.orgmode.org/871r2vpblx.fsf@yandex.com/T/#u
* lisp/org-element.el (org-element--cache-avoid-synchronous-headline-re-parsing):
New internal variable controlling latency of cache
`after-change-functions'.
(org-element--cache-for-removal): Use
`org-element--cache-avoid-synchronous-headline-re-parsing' to decide
if we re-parse changed headlines immidiately.
* lisp/org-archive.el (org-archive-subtree): Let-bind
`org-element--cache-avoid-synchronous-headline-re-parsing' to t while
archiving for better speed.
* lisp/org-element.el (org-element--current-element): Do not call
`org-at-heading-p' to determine if we are at headline element.
`org-at-heading-p' calls `beginning-of-line' and may match
non-headlines as headlines.
* lisp/org-element.el (org-element--cache-for-removal): Consider edits
within blank before first section destructive and make cache delete
the section even though the edits are technically before its
beginning.
* lisp/org-element.el (org-element-parse-buffer): Increase
`gc-cons-threshold' to improve performance. In my tests on large Org
buffers, garbage collection took 50% of the time without increasing
the threshold.
* lisp/org-element.el (org-element--cache-gapless): Move definition
before first use to remove byte-compiler warning about assignment to
free variable.
* lisp/org-element.el (org-element--cache-sync): Add heuristics
skipping silent buffer changes made by quail.el when inserting "S-\"
using computer-russian input method.
Reported in https://list.orgmode.org/smre9o$hn1$1@ciao.gmane.io/
* lisp/org-element.el (org-element--parse-generic-emphasis): New fuction.
(org-element-bold-parser):
(org-element-code-parser):
(org-element-italic-parser):
(org-element-strike-through-parser):
(org-element-underline-parser):
(org-element-verbatim-parser): Use new function.
The consequence of this patch is that the syntax does not inherit from
unrelated limitations from ‘org-emph-re’, i.e., the number of lines
allowed in the markup.
* lisp/org-element.el (org-element--parse-to): Fix incorrect heading
regexps when `org-odd-levels-only' is non-nil.
(org-element--cache-before-change): Use reduced level for
`org-element--cache-change-warning' when headlines are modified.
* lisp/org-element.el (org-element--cache-diagnostics-modifications):
New variable to enable/disable warnings when unregistered
modifications are detected in Org buffers.
* lisp/org-element.el (org-element--cache-sync): Only show
"Unregistered modification" warning when
`org-element--cache-diagnostics-modifications' is non-nil or during
Org tests. Add special handling for unregistered changes by
`replace-match' in Emacs <28.
Fixes https://list.orgmode.org/87pmr6lu1y.fsf@localhost/T/#t
* lisp/org-element.el (org-element--cache-sync): Detect silent changes
in buffer made by quail.el when non-latin input methods are active.
These silent changes do not modify the buffer, but only change
`buffer-chars-modified-tick' and do no affect element cache. Note
that the detection uses heuristics.
Fixes https://list.orgmode.org/87sfw2luhj.fsf@localhost/T/#you
* lisp/org-element.el (org-element-at-point): Do no unconditionally
return first matching cached element when point is within blank lines
after section element and CACHED-ONLY is non-nil.
Fixes https://list.orgmode.org/smbl59$qjm$1@ciao.gmane.io/T/#u
* lisp/org-element.el: Do not require `backtrace'. It is not
available in Emacs 26.
(org-element--cache-sync, org-element-at-point): Do not try to show
backtrace in warning when `backtrace' library is not available.
Reported in https://list.orgmode.org/d20a8753-8d0a-b4bf-c3fc-bce2f0a22392@gmail.com/T/#u
* lisp/org-element.el (org-element-copy): Make sure that element
properties containing secondary objects are also copied.
(org-element--parse-elements): Avoid modifying cached elements.
Fixes https://list.orgmode.org/CAHyO48yS2EAJnhiYoK7syjb1_Fbfxcv2A0fk4t5RFzTLj1hSGA@mail.gmail.com/
In the backtrace provided in the email, org-roam calls to
`org-element-parse-buffer' add uncached `org-data' element as cached
element `:parent' property. Uncached elements in `:parent' property
break the cache code.
* lisp/org-element.el (org-element--cache-active-p): Prevent cache
queries when `org-element--cache-after-change' is not in
`after-change-functions'. `after-change-functions' can be temporalily
set to nil by i.e. `combine-change-calls'. We should not try to get
information from cache in such scenarios because cache may not yet be
up-to-date. The modifications will only be registered upon exiting
the `combine-change-calls' macro.
Fixes https://list.orgmode.org/875ytggcuk.fsf@yandex.com/T/#t
* lisp/org-element.el (org-element--cache-gapless): New variable
tracking when cache does not contain gaps.
(org-element-cache-reset): Initialise `org-element--cache-gapless'.
(org-element-cache-map): Fill the cache gaps before running FUNC
query. When multiple calls to `org-element-cache-map' are done on
unchanged buffer, pre-processing only requires a single regexp search
pass across i.e. headlines. Subsequent `org-element-cache-map' calls
can then be reduced to a simple cache tree walk.
* lisp/org-element.el (org-element-cache-map): Fix reported bug [1]
when some agenda items are missing when `org-element-cache-map' is
called from `org-agenda-get-scheduled'. Byte compilation of FUNC
matcher somehow fails to produce equivalent function in Emacs <28.
[1] https://list.orgmode.org/87tuha62rq.fsf@localhost/T/#t
Ihor Radchenko
Stefan Kangas
Nicolas Goaziou
Kyle Meyer