* lisp/org-element.el (org-element--cache-for-removal): Directly check
for headline at point instead of running full
`org-element--current-element' that is matching a large number of
regexps.
* lisp/org-element.el (org-element--cache-hash-size):
(org-element--cache-hash-statistics):
(org-element--cache-hash-nocache):
(org-element--cache-hash-size):
(org-element--cache-hash-left):
(org-element--cache-hash-right): Implement recent search cache for
`org-element--cache-find'. The cache stores recent cache tree queries
as a vector with O(1) access time.
(org-element--cache-find): Make use of `org-element--cache-hash-left'
and `org-element--cache-hash-right' when cached query element starts
at POS and SIDE is nil. Record statistics.
(org-element-cache-reset): Reset search cache on init.
(org-element-cache-hash-show-statistics): Provide a command reporting
the new caching efficiency. Can be used for debugging/survey
purposes.
* lisp/org-macs.el (org-knuth-hash): Implement multiplicative hash
function.
Preliminary testing reveals that this simple strategy can reduce query
time from O(Log N) down to O(1) for ~30%-50% cache queries.
* lisp/org-element.el (org-element--current-element): Use
`org-element-dynamic-block-open-re' to match blocks.
* testing/lisp/test-org-element.el (test-org-element/paragraph-parser):
Fix test relying on previous incorrect behaviour. Empty #+BEGIN:
should be parsed as a keyword.
Reported in https://list.orgmode.org/Ycay4s3iAdEGSwgt@tuxteam.de/T/#t
* lisp/org-element.el (org-element--parse-to):
(org-element--cache-verify-element): Add check asserting that all the
parsed elements are assigned with non-nil parent (except org-data).
* lisp/org-element.el (org-element--cache-before-change): Fix
optimisation when edits are involving element :end position. Checking
that END is at BOL fails when we insert at the beginning of a
headline.
* lisp/org-element.el (org-element--cache-before-change):
(org-element--cache-after-change): Do not prevent trigerring if the
current indirect buffer is not in Org mode, but its base buffer is in
Org mode.
* lisp/org-element.el (org-element-context): Use
`org-element-at-point', which makes use of cache.
* testing/lisp/test-org-element.el (test-org-element/lineage): Expect
full lineage up to org-data from `org-element-context'.
* lisp/org-element.el (org-element--parse-to): Manually parse org-data
and set appropriate parser mode when POS is before first headline and
cache is disabled.
(org-element-org-data-parser--recurse): New variable flagging if
`org-element-data-parser' is called recursively.
(org-element-org-data-parser): Handle recursive calls from inside `org-element-at-point-no-context'.
This also fixes false-positives in `org-element--cache-verify-element'.
* lisp/org-element.el (org-element--cache-for-removal): Consider
preceding keywords to be updated unconditionally.
(org-element-cache-map): Fix infinite loop revealed by the new test.
* testing/lisp/test-org-element.el (test-org-element/cache-affiliated):
New test.
* lisp/org-element.el (org-element--cache-silent-modification-check):
New variable allowing the user to suppress modification checks completely.
* lisp/org-element.el (org-element--cache-sync): Support
`org-element--cache-silent-modification-check'.
The existing heuristics for built-in non-modifying commands that
change `buffer-chars-modified-tick' is not sufficient. At least
`capitalize-word' changes the tick regardless whether the buffer was
actually modified or not. Yet, `capitalize-word' does not call
`after-change-functions' if the buffer is not really modified.
Reported in https://list.orgmode.org/2022-01-06T12-13-17@devnull.Karl-Voit.at/T/#mb3771758f81b31721ba2f420878a4d16081dc483
* lisp/org-element.el (org-element-cache-map-continue-from): New
variable forcing `org-element-cache-map' to continue from a custom
point in buffer.
(org-element-cache-map): Add support for
`org-element-cache-map-continue-from'. Update docstring accordingly.
Also, make sure that mapping terminates correctly when FUNC deletes
all elements in buffer.
* testing/lisp/test-org.el (test-org/map-entries): Add test.
Fixes https://orgmode.org/list/CADywB5KOJ1p0NpvA=iX-ybHsO=huGA8qL3xMpUTETmS2qp7_ng@mail.gmail.com
* lisp/org-element.el (org-element--parse-to): Disable cache in
`org-element--current-element'. When calling it here, we know for
sure that element at point is not yet in cache.
* lisp/org-element.el (org-element--cache-process-request): New OFFSET
argument used to correct newly added parents during Phase 1. The
`org-element--parse-to' call inside Phase 1 may add new elements to
cache that intersect with future edits. Boundaries of these elements
may be shifted twice, so we have to offset the future shift.
(org-element--cache-sync): New OFFSET argument providing future change
info to `org-element--cache-process-request'.
(org-element--cache-submit-request): Provide offset value in
`org-elemnt--cache-sync' call.
(org-element--cache-submit-request):
(org-element--cache-process-request):
(org-element--cache-sync): Never use %d format for region boundaries.
It may be a marker and cause error. Use %S instead.
(org-element--cache-process-request): Use unique symbols for
catch-throw.
Fixes https://list.orgmode.org/CAFyQvY3Qv5xn-ET83L6Rzg-V1zOVu4y1gt+-_CpfaWNAdt87xA@mail.gmail.com/T/#t
* lisp/org-element.el (org-element--cache-after-change): Always extend
changed region to bol.
* testing/lisp/test-org-element.el (test-org-element/cache): Add test
checking the new fix. Amend some tests around making sure that cache
is active during testing.
* lisp/org-element.el (org-element--cache-sync): Set
`org-element--cache-warning' to nil at the end of synchronisation.
We do not need to consider that next request might be merged with
existing request in such scenario.
(org-element--cache-before-change): Add comment explaining the use of `org-element--cache-warning'.
* lisp/org-element.el (org-element--cache-after-change): Extend
changed region to bol when we are editing near beginning of an element
within or right after indentation. Such edits potentially change
:post-blank value of the previous element.
* testing/lisp/test-org-element.el (test-org-element/cache): Add test
checking one of such cases.
* lisp/org-element.el (org-element--cache-for-removal): Drawers are
generally robust elements (they cannot be changed if a non-sensitive
change is made inside their contents). The only exception is
PROPERTIES drawer that may switch back and forth between ordinary
drawer and properties drawer depending on its contents. The old code
treated all possible drawer as non-robust for this reason, degrading
performance on large LOGBOOK drawers that are now processed much
faster since they do not need to be removed and re-parsed on every
single change.
* lisp/org-element.el (org-element--cache-sensitive-re): Remove
#+begin_ and \begin{ lines from sensitive regexp. Such changes can
be handled without a need to re-parse from earlier position. If a
change introduces a new element, the new element will be recognised
and the unupdated elements not intersecting with the new element will
be handled during Phase 1.
* lisp/org-element.el (org-element--cache-before-change): Do not match
`org-element--cache-sensitive-re' against an element starting at END
when END is at bol. Such changes never break an element structure (as
opposed to inserting non-newline right at :begin of an element).
* lisp/org-element.el (org-element--cache-for-removal): Only force
re-parsing headlines when the changed region falls within the headline
before property drawer end position.
* lisp/org-element.el (org-element--cache-submit-request): Extend
calculation of affected parent elements to the full changed region.
The previous behaviour could miss some outer elements to be removed.
* lisp/org-element.el (org-element--cache-sensitive-re): Do not treat
properties inside property drawers as global sensitive change. It can
cause cache drop on large logbook drawers where a new clock entry is
inserted.
(org-element--cache-for-removal): Use the same technique used for
detecting property drawer <-> ordinary drawer changes under headlines
for org-data.
* lisp/org-element.el (org-element-cache-map): Move all possible
let-bindings outside the loop to avoid remory re-allocation on every
iteration. Track statistics for `re-search-forward' calls.
* lisp/org-element.el (org-element--cache-sensitive-re): Make proprety
drawer lines sensitive.
(org-element--cache-for-removal): Make sensitive edits inside cache
gap right after org-data re-parse the org-data. Make sensitive
top-section edits re-parse org-data.
The specific error has been reported in
https://github.com/yantar92/org/issues/40
The recepy involves loading org-contrib:
1. emacs -Q -L ~/.emacs.d/.local/straight/repos/org/lisp -L ~/.emacs.d/.local/straight/repos/org-contrib/lisp -l org
-l org-eldoc
2. Create a test.org file with the following content:
:PROPERTIES:
:ID: test
:END:
Test.
3.
(goto-char (point-min))
(org-entry-put nil "ID" "test")
(save-buffer)
(org-entry-get nil "ID")
Return value is nil, while should be "test"
* lisp/org-element.el (org-element--cache-sync): Suppress silent edit
warnings for Emacs <28. In older Emacs, Emacs internal functions can
modify `buffer-chars-modified-tick' and cannot be distinguished from
dangerous edits under `inhibit-modification-hooks'. Some of the
functions even have the same footprint with the edits we want to
avoid.
* lisp/org-element.el (org-element-at-point): Describe warning as
parser error when `org-element--parse-to' throws an error. The error
may be thrown when parser encounters issue not related to cache.
Fixes confusion in https://list.orgmode.org/871r2vpblx.fsf@yandex.com/T/#u
* lisp/org-element.el (org-element--cache-avoid-synchronous-headline-re-parsing):
New internal variable controlling latency of cache
`after-change-functions'.
(org-element--cache-for-removal): Use
`org-element--cache-avoid-synchronous-headline-re-parsing' to decide
if we re-parse changed headlines immidiately.
* lisp/org-archive.el (org-archive-subtree): Let-bind
`org-element--cache-avoid-synchronous-headline-re-parsing' to t while
archiving for better speed.
* lisp/org-element.el (org-element--current-element): Do not call
`org-at-heading-p' to determine if we are at headline element.
`org-at-heading-p' calls `beginning-of-line' and may match
non-headlines as headlines.
* lisp/org-element.el (org-element--cache-for-removal): Consider edits
within blank before first section destructive and make cache delete
the section even though the edits are technically before its
beginning.
* lisp/org-element.el (org-element-parse-buffer): Increase
`gc-cons-threshold' to improve performance. In my tests on large Org
buffers, garbage collection took 50% of the time without increasing
the threshold.
* lisp/org-element.el (org-element--cache-gapless): Move definition
before first use to remove byte-compiler warning about assignment to
free variable.
* lisp/org-element.el (org-element--cache-sync): Add heuristics
skipping silent buffer changes made by quail.el when inserting "S-\"
using computer-russian input method.
Reported in https://list.orgmode.org/smre9o$hn1$1@ciao.gmane.io/
* lisp/org-element.el (org-element--parse-generic-emphasis): New fuction.
(org-element-bold-parser):
(org-element-code-parser):
(org-element-italic-parser):
(org-element-strike-through-parser):
(org-element-underline-parser):
(org-element-verbatim-parser): Use new function.
The consequence of this patch is that the syntax does not inherit from
unrelated limitations from ‘org-emph-re’, i.e., the number of lines
allowed in the markup.
* lisp/org-element.el (org-element--parse-to): Fix incorrect heading
regexps when `org-odd-levels-only' is non-nil.
(org-element--cache-before-change): Use reduced level for
`org-element--cache-change-warning' when headlines are modified.
* lisp/org-element.el (org-element--cache-diagnostics-modifications):
New variable to enable/disable warnings when unregistered
modifications are detected in Org buffers.
* lisp/org-element.el (org-element--cache-sync): Only show
"Unregistered modification" warning when
`org-element--cache-diagnostics-modifications' is non-nil or during
Org tests. Add special handling for unregistered changes by
`replace-match' in Emacs <28.
Fixes https://list.orgmode.org/87pmr6lu1y.fsf@localhost/T/#t
* lisp/org-element.el (org-element--cache-sync): Detect silent changes
in buffer made by quail.el when non-latin input methods are active.
These silent changes do not modify the buffer, but only change
`buffer-chars-modified-tick' and do no affect element cache. Note
that the detection uses heuristics.
Fixes https://list.orgmode.org/87sfw2luhj.fsf@localhost/T/#you
* lisp/org-element.el (org-element-at-point): Do no unconditionally
return first matching cached element when point is within blank lines
after section element and CACHED-ONLY is non-nil.
Fixes https://list.orgmode.org/smbl59$qjm$1@ciao.gmane.io/T/#u
* lisp/org-element.el: Do not require `backtrace'. It is not
available in Emacs 26.
(org-element--cache-sync, org-element-at-point): Do not try to show
backtrace in warning when `backtrace' library is not available.
Reported in https://list.orgmode.org/d20a8753-8d0a-b4bf-c3fc-bce2f0a22392@gmail.com/T/#u
* lisp/org-element.el (org-element-copy): Make sure that element
properties containing secondary objects are also copied.
(org-element--parse-elements): Avoid modifying cached elements.
Fixes https://list.orgmode.org/CAHyO48yS2EAJnhiYoK7syjb1_Fbfxcv2A0fk4t5RFzTLj1hSGA@mail.gmail.com/
In the backtrace provided in the email, org-roam calls to
`org-element-parse-buffer' add uncached `org-data' element as cached
element `:parent' property. Uncached elements in `:parent' property
break the cache code.
* lisp/org-element.el (org-element--cache-active-p): Prevent cache
queries when `org-element--cache-after-change' is not in
`after-change-functions'. `after-change-functions' can be temporalily
set to nil by i.e. `combine-change-calls'. We should not try to get
information from cache in such scenarios because cache may not yet be
up-to-date. The modifications will only be registered upon exiting
the `combine-change-calls' macro.
Fixes https://list.orgmode.org/875ytggcuk.fsf@yandex.com/T/#t
* lisp/org-element.el (org-element--cache-gapless): New variable
tracking when cache does not contain gaps.
(org-element-cache-reset): Initialise `org-element--cache-gapless'.
(org-element-cache-map): Fill the cache gaps before running FUNC
query. When multiple calls to `org-element-cache-map' are done on
unchanged buffer, pre-processing only requires a single regexp search
pass across i.e. headlines. Subsequent `org-element-cache-map' calls
can then be reduced to a simple cache tree walk.
* lisp/org-element.el (org-element-cache-map): Fix reported bug [1]
when some agenda items are missing when `org-element-cache-map' is
called from `org-agenda-get-scheduled'. Byte compilation of FUNC
matcher somehow fails to produce equivalent function in Emacs <28.
[1] https://list.orgmode.org/87tuha62rq.fsf@localhost/T/#t
* lisp/org-element.el (org-element--cache-interrupt-C-g,
org-element--cache-interrupt-C-g-max-count,
org-element--cache-interrupt-C-g-count): Add docstring and change
default value of `org-element--cache-interrupt-C-g' to t.
(org-element--parse-to): Show informative message when user tries to
interrupt in the middle of cache sync.
* lisp/org-element.el (org-element--cache-sync,
org-element-cache-reset): Do not use `buffer-chars-modified-tick' as
`org-element--cache-sync-keys-value'. Instead, make sure that
`org-element--cache-sync-keys-value' is always changed upon completing
cache request sequence. Using `buffer-chars-modified-tick' can
trigger using obsolete sync keys after we add elements to cache via
`org-element--parse-to' after cache sync is completed, but no other
changes were made in the buffer. The newly added elements are then
not comparable with the obsolete sync keys.
(org-element--cache-persist-before-write): Cleanup
`:org-element--cache-sync-key' from elements before saving element
cache to file. If we do not do it, obsolete keys may be reused
inintentionally during next Emacs session.
* lisp/org-element.el (org-element--cache-interrupt-C-g,
org-element--cache-interrupt-C-g-max-count,
org-element--cache-interrupt-C-g-count): New variables controlling
force-quitting cache sync process. If
`org-element--cache-interrupt-C-g' is non-nil, user can force
terminating syncronisation request even in the middle of sync process
by invoking `keyboard-quit'
`org-element--cache-interrupt-C-g-max-count' times.
`org-element--cache-interrupt-C-g-count' stores the number of
invokations during current sync process.
(org-element--cache-sync): Initialise
`org-element--cache-interrupt-C-g-count' before processing sync
request.
(org-element--parse-to): Handle interruption in potentially infinite
loop.
* lisp/org-element.el (org-element--cache-submit-request): Re-enable
FUTURE-CHANGE argument in `org-element--cache-sync' call. Without
this argument, cache can be processed incorrectly because elements
within changed region may be shifted and not deleted in subsequent
phase 0 request.
* lisp/org-compat.el (org-file-name-concat): Do not use
`string-empty-p'.
(combine-change-calls): Create a stub when `combine-change-calls' were
not yet available.
(org-replace-buffer-contents): Add compatibility function for
`replace-buffer-contents'.
* lisp/org-element.el (org-element--current-element): Do not use
`if-let'.
* lisp/org-persist.el (org-persist-gc): Do not use `when-let'.
* lisp/org-plot.el (org-plot/gnuplot): Do not use `if-let'.
* lisp/org-src.el (org-edit-src-save, org-edit-src-exit): Use
`org-replace-buffer-contents'.
* lisp/org.el (org-narrow-to-subtree, org--property-local-values,
org-entry-get-with-inheritance, org-in-commented-heading-p,
org-up-heading-safe, org-goto-first-child): Do not use
`if-let'/`when-let'.
* testing/org-test.el (org-test-at-time): Fallback to old
`decode-time' specification in older Emacs.
* lisp/org-compat.el: Remove `length>' alias
* lisp/org-element.el (org-element-cache-map): Suppress compiler
warning.
* lisp/org-persist.el (org-persist-register): Do not `push' into
`plist-get'. Not supported yet in Emacs 27.
(org-persist-unregister): Do not use `length>'. Use
`org-directory-empty-p'.
(org-persist-gc): Use `org-directory-empty-p'.
* lisp/org-element.el (org-element-with-disabled-cache): New macro.
(org-element-greater-elements): Add new org-data element. It
functions like a virtual headline containing the whole buffer. The
org-data properties are like headlie properties, but according to the
top-level drawer. org-data's category is the buffer's category as
defined by top-level property drawer, #+CATEGORY keyworsd, and the
buffer file name.
(org-element--cache-element-properties, org-element-set-element): New
variable containing properties to be transferred when updating changed
element in cache in `org-element-set-element'.
(org-element--get-node-properties): Allow parsing node propreties in
top-level drawer when new optional argument is passed. Respect
PROPERTY+ syntax.
(org-element--get-global-node-properties): New function. It returns
node properties for top-level property drawer.
(org-element-org-data-parser, org-element-org-data-interpreter):
Implement the new org-data element.
(org-element-headline-parser, org-element-section-parser): Add new
:robust-begin and :robust-end
properties delimiting safe changes that do not modify headline
element.
(org-element--list-struct): Fix cache update when adding a headline
inside list.
(org-element--current-element): Implement cache support. Record
parsing mode (:mode) and parsing granularity (:granularity) in the
element properties.
(org-element-parse-buffer, org-element--next-mode): Support new
org-data element.
(org-element--parse-elements): Record parsing granularity in the
returned tree
(org-element-use-cache): Enable cache by default.
(org-element-cache-persistent): New variable controlling cache
persistance across sessions. Enabled by default.
(org-element--cache-self-verify,
org-element--cache-self-verify-frequency,
org-element--cache-diagnostics, org-element--cache-map-statistics,
org-element--cache-map-statistics-threshold,
org-element--cache-diagnostics-level,
org-element--cache-diagnostics-ring,
org-element--cache-diagnostics-ring-size): New variables controlling
cache diagnostics and self-diagnostics. Greatly simplifies cache
debugging.
(org-element--cache, org-element--cache-sync-requests,
org-element--cache-sync-timer): Make cache buffer-local by default.
(org-element--headline-cache): Implement separate cache storing only
headlines and inlinetasks.
(org-element--cache-size, org-element--headline-cache-size): New
variables containing cache sizes. This is much faster than
`avl-tree-size'.
(org-element--cache-sync-requests): Update docstring explaning the
request list structure.
(org-element--cache-sync-keys-value): New variable replacing
`org-element--cache-sync-keys' hash table. The hash table was not
reliable because it was using elements as keys. Upon any cached
element update/shift, the keys were invalidated making cache ordering
incorrect and breaking the cache badly. Now, the cache keys are
stored as :org-element--cache-sync-key element property and the new
variable stores marker value indicating the current sync request
cycle. See `org-element--cache-key' for more details.
(org-element--cache-change-tic): New variable controlling buffer
modification count that is registered in cache. This variable allows
catching "stealth" edits.
(org-element--cache-non-modifying-commands): New variable listing
commands that will not be slown down if we fill cache on the fly.
(org-element--request-key, org-element--request-beg,
org-element--request-end, org-element--request-offset,
org-element--request-parent, org-element--request-phase): New macros.
They improve code readability (especially when using nameless-mode).
(org-element--format-element, org-element--cache-log-message,
org-element--cache-warn): New macros implementing generic logging
functionality.
(org-element--cache-key): Add section and org-data element support.
Change cache key storage from hash map to :org-element--cache-sync-key
element property + `org-element--cache-sync-keys-value'. We use the
latter to group all the cache keys during a single cache request
sequence. Once sync request is fully complete, the
`org-element--cache-sync-keys-value' is updated making all the old
sync keys obsolete (they will still be store as element properties).
(org-element--headline-cache-root): New function returning headline
cache root.
(org-element--cache-active-p): Prevent cache updates when
`inhibit-modification-hooks' is non-nil, unless non-nil optional
argument is provided.
(org-element--cache-find): Share cache between indirect buffers and
the base buffer. We have to do it because after-change hooks for
indirect buffer are not called in the base buffer and vice versa. Add
support for section and org-data elements.
(org-element--cache-put): Implement new approach for cache key
storage. Add diagnostics. Indicate cached elements using :cached
element property. Support cache size calculation.
(org-element--cache-remove): Invalidate parent contents when removing
element. Support cache size calculation. Detect cache corruption due
to misordered elements.
(org-element--cache-shift-positions): Support :robust-begin and
:robust-end element properties.
(org-element--cache-sync): Add diagnostics. Add detailed comments.
Prevent slowdown when large cache chunks need to be deleted forcing
O(N) complexity cutoff. In phase 2, fix cases when next request
contains deleted cache key. In phase 2, fix scenario when newly
inserted element intersects with existing elements in cache. In phase
2, detect obsolete parents removed from cache.
(org-element--open-end-p): New function checking if an element can
have blank lines right after its :contents-end.
(org-element--parse-to): Do not alter match data. Process complex
parsing mode changes correctly. Support headlines in cache. Support
org-data parsing. Add detailed comments. Add diagnostics.
(org-element--cache-sensitive-re): Make list lines sensitive.
(org-element--cache-change-warning): Update docstring. Now, the
variable can have t, nil, and number values. Numbers are used to
provide more details about changed headlines.
(org-element--cache-before-change, org-element--cache-after-change):
Handle headline hierarchy. Properly handle cache in indirect
buffers.
(org-element--cache-after-change): Update docstring clarifying the
return values. Add special handling for headline and org-data
elements updating them in-place instead of removing together with the
whole contents when possible. Use :robust-begin/:robust-end element
properties to detect robust changes.
(org-element--cache-submit-request): Add detailed comments. Correctly
handle cache in indirect buffers. Delegate element modifications to
`org-element--cache-for-removal'.
(org-element--cache-verify-element): New function for cache
self-verification.
(org-element--cache-persist-before-write,
org-element--cache-persist-before-read,
org-element--cache-persist-after-read): Implement cache persistance.
(org-element-cache-reset): Correctly handle cache in indirect
buffers. Support cache persistance. Support new cache size
calculation and new cache key schema.
(org-element-cache-map): New function analagous to `org-element-map',
but much faster. The function overperforms org-ql written by Adam
Porter aka alphapapa [1] and reuses some ideas from there (namely,
fast element skipping via regexps).
[1] https://github.com/alphapapa/org-ql/
(org-element-at-point): The returned elements are now guaranteed to
have correct parents up to org-data. New optional argument
CACHED-ONLY limits element search to current cache---if element is not
in cache and current command is not in cache
`org-element--cache-non-modifying-commands', the cache is not updated
and the function returns nil. Also, support cache verification.
(org-element-at-point-no-context): New function. It is analogous of
older `org-element-at-point' with no guarantee that :parent properties
are correct beyond direct parent heading. This function does not
update cache and can be useful when cache updates should be avoided
for performance reasons.
* lisp/ob-core.el (org-babel-where-is-src-block-result): Support
section and org-data elements in cache.
* lisp/org-macro.el (org-macro-replace-all,
org-macro--find-keyword-value): Support org-element-cache.
* lisp/org-table.el (orgtbl-to-generic): Support org-element-cache.
* lisp/org.el (org-mode): Add cache persistance.
(org-up-element): Preserve old behaviour when error is returned for
section and org-data element.
*
testing/lisp/test-org-archive.el (test-org-archive/update-status-cookie):
Fix test when cache is active.
* testing/lisp/test-org-colview.el (test-org-colview/columns-update):
Fix test.
* testing/lisp/test-org-element.el (test-org-element/extract-element):
Add suport for new org-data element.
* testing/lisp/test-org-element.el (test-org-element/parent-property):
Fix equality check. Parents returned by cache and `org-element-map'
may not be `eq' now. Just `equal'.
* testing/lisp/test-org-element.el (test-org-element/context): Support
section and headline parents.
* lisp/org-element.el (org-element-headline-parser):
Pretty sure this one should be `skip-chars-backward`, not
`skip-syntax-backward`, since \t isn't a valid syntax code.
org-element: use correct function
58102466e32d4dd9c7d816cdc3f4595a2145f332
Mattias Engdegård
Thu Sep 30 17:16:25 2021 +0200
* lisp/org-element.el (org-element-citation-key-re):
(org-element-citation-prefix-re): New variables.
(org-element--set-regexps): Set `org-element--object-regexp' so it
finds citations.
(org-element-all-objects): Add citation and citation-reference
objects.
(org-element-recursive-objects): Add citation object.
(org-element-object-restrictions): Add citation and citation-reference
to restrictions.
(org-element-secondary-value-alist): citation and citation references
can hold secondary strings.
(org-element-citation-parser):
(org-element-citation-interpreter):
(org-element-citation-reference-parser):
(org-element-citation-reference-interpreter): New functions.
(org-element--object-lex): Parse citations and citations references.
* testing/lisp/test-org-element.el (test-org-element/citation-parser):
(test-org-element/citation-reference-parser):
(test-org-element/citation-interpreter): New tests.
This patch adds support for [cite:@key], [cite:pre @key post]
[cite:global prefix; pre @key1 post; pre @key2 post; global suffix]
objects along with their [cite/style: ...] counterparts.
* lisp/org-element.el (org-element-drawer-parser): A single ":END:"
line should not be recognized as a drawer.
* testing/lisp/test-org-element.el (test-org-element/drawer-parser):
Add test.
Reported-by: Ihor Radchenko <yantar92@gmail.com>
<http://lists.gnu.org/r/emacs-orgmode/2021-05/msg00819.html>
This reverts commit 4056e11ddc.
We shouldn't code escape inline source blocks, as escaping can be
inlined any time, at which point escape character is no longer treated
specially. E.g.,
src_c{a
,* 2}
may become
src_c{a ,* a}
where the comma is no longer an escape character.
* lisp/org-element.el (org-element-inline-src-block-parser): Un-escape
code in inline source blocks.
(org-element-inline-src-block-interpreter): Escape it again if
necessary.
* testing/lisp/test-org-element.el (test-org-element/inline-src-block-parser):
Add test.
While at it, I enabled lexical-binding in the affected files.
* lisp/org-agenda.el (org-let, org-let2): Move from org-macs and
use `declare`.
* lisp/org-macs.el (org-let, org-let2): Move these functions that are
inherently harmful to your karma to the only package that uses them.
(org-scroll): Use `pcase` to avoid `eval` and use more readable syntax
for those integers standing for events.
* lisp/org-element.el (org-element-map): Use `declare`.
Prefer `declare` over a `put` of `list-indent-function`.
8d5dfafab7dc40d4b74dc0b56d1b314fd8cac390
Stefan Monnier
Mon Feb 22 11:54:17 2021 -0500
* lisp/org-element.el (org-element-parse-secondary-string):
Don't set buffer-local variables buffer-file-name and buffer-file-truename
in temporary buffer.
Don't set file name variable in org-element-parse-secondary-string (bug#44524)
79d04ae13ff33a93f631061d912168e9703251dd
Juri Linkov
Tue Nov 10 21:27:37 2020 +0200
* lisp/org-element.el (org-element--next-mode): Reorder two patterns
to sidestep an error on Emacs 24.
The valid pattern introduced in 4a27b67fd (org-element: Fix property
drawers parsing, 2020-04-22) makes entering Org mode in Emacs 24.5
fail with "Eager macro-expansion failure: (wrong-type-argument listp
:pcase--succeed)". This seems to be due to a bug fixed in Emacs 25.
Reported-by: Jens Lechtenboerger <lechten@wi.uni-muenster.de>
Ref: https://orgmode.org/list/87pn6n5u13.fsf@wi.uni-muenster.de
* lisp/org-element.el (org-element-normalize-contents): Update comment
The comment mentions that caching for `org-elements' is enabled by
default, but this isn’t the case anymore since
bbdecd1e64, cf. `org-element-use-cache'.
* lisp/org-element.el
(org-element--collect-affiliated-keywords): Save match data when
setting the main value for keywords.
This fixes a bug about defining short captions. Thanks to Anthony
Cowley for reporting this and suggesting this fix.
* lisp/org-attach.el (org-attach-link-expand): New function for link
element expansion.
* lisp/org-element.el (org-element-link-parser): Remove info about
expanded attachment paths from link elements.
* lisp/ol.el (org-link-open)
* lisp/ox-texinfo.el (org-texinfo-link)
* lisp/ox-odt.el (org-odt-link)
* lisp/ox-md.el (org-md-link)
* lisp/ox-man.el (org-man-link)
* lisp/ox-latex.el (org-latex--inline-image, org-latex-link)
* lisp/ox-html.el (org-html-link)
* lisp/ox-ascii.el (org-ascii-link): Refactor to use new link
expansion function from org-attach.el instead of (now removed)
custom link property from org-element.el.
* lisp/org-element.el (org-element-at-point): Remove :parent from the
list of always available properties. This is not true for,
e.g. headlines.
Reported-by: Ihor Radchenko <yantar92@gmail.com>
* lisp/org-element.el (org-element-link-parser): Add info about
expanded attachment paths to the link parse tree export.
* lisp/org-attach.el Remove org-attach-open-link. Let attachment
links use the built in code that already is developed for file
links.
* lisp/ol.el (org-link-open): Add knowledge about attachment links to
the function opening links, so they can be opened exactly as file
links are opened.
* lisp/ox-texinfo.el (org-texinfo-link)
* lisp/ox-odt.el (org-odt-link)
* lisp/ox-md.el (org-md-link)
* lisp/ox-man.el (org-man-link)
* lisp/ox-latex.el (org-latex--inline-image, org-latex-link)
* lisp/ox-html.el (org-html-link)
* lisp/ox-ascii.el (org-ascii-link): Refactor to use property from
link parser instead of invoking attachment expansion in the
exporter.
* lisp/org-element.el (org-element--parse-elements): Only ignore parts
due to folding, not because of fontification.
* lisp/org-macs.el (org-invisible-p): Add an optional argument.
* testing/lisp/test-org-element.el (test-org-element/parse-buffer-visible):
Add tests.
* lisp/org-element.el (org-element--current-element): Forbid
affiliated keywords for comments.
* testing/lisp/test-org-element.el (test-org-element/comment-parser):
Tiny refactoring.
The motivation is to avoid distinguishing between top-level comments,
which cannot have affiliated keywords, and other comments, which
could. Therefore, we keep a single type of comment.
* lisp/org-element.el (org-element-comment-parser): Top level comments
do not have affiliated keywords.
(org-element--current-element): Properly parse top-level comments and
top-level properties drawer.
(org-element--next-mode): Change signature.
(org-element--parse-elements): Apply signature change.
(org-element--parse-to): Apply signature change. Look for top-level
comments or properties drawer.
* testing/lisp/test-org-element.el (test-org-element/property-drawer-parser):
Add comment.
Initially, `org-element--current-element' would defer to
`org-get-property-block' to validate a property-drawer, which is the
wrong way to look at the low-level Org Element library. Also, it would
allow affiliated keywords, which is not possible.
Add functionality to define property-blocks on document level, in
addition to at headline level.
* doc/org-manual.org:
* etc/ORG-NEWS: Document new functionality.
* lisp/org.el (org-keyword-regexp): Define constant instead of
hardcoding.
(org-file-properties): Renamed, see next line.
(org-keyword-properties): Renamed from above. Due to the fact that
properties can be defined for the whole document using property
drawers this local variable needs a rename to make its name less
ambigous.
(org-refresh-properties, org-refresh-property, org-entry-properties)
(org-refresh-category-properties, org-get-property-block)
(org-entry-get-with-inheritance, org-entry-put)
(org-insert-property-drawer, org-end-of-subtree): Made to work before
first headline.
(org-at-property-block-p): New function to validate if point is at the
start of a property block.
(org-property-global-value): Renamed, see next line.
(org-property-global-or-keyword-value): Renamed from above to match
its functionality better.
(org-back-to-heading-or-point-min): New function to make a document
work as a level 0 node in the outline.
(org-at-keyword-p): Predicate function to answer to if we're currently
at a keyword line or not.
(org-up-heading-or-point-min): New function to make a document work as
a level 0 node in the outline.
* lisp/org-element.el (org-element--current-element): Can now detect
property-blocks before first headline according to it's positional
rules.
* lisp/org-attach.el (org-attach): Make it possible to call the
attachment dispatcher also before the first headline, since document
property drawers make attachments possible for the whole document
now.
* lisp/org-capture.el: Modified only due to rename of function in
org.el.
* lisp/org-compat.el (org-file-properties)
(org-property-global-value): Renamed functions declared obsolete.
* testing/lisp/test-org.el (org/insert-property-drawer)
(org/set-property, org/delete-property, org/delete-property-globally):
Additions of tests to check if they work before first headline.
(org/at-property-p, org/at-property-block-p, org/get-property-block)
(org/entry-get, org/refresh-properties): New tests
* testing/examples/property-inheritance.org: Switch from
property-keywords to a property-drawer in the testfile.
Functionality should be the same, but now using a document drawer
instead of property-keywords.
Reason for switching is that I'd like us to slowly depricate
property-keywords.
* testing/lisp/test-org-element.el:
* contrib/lisp/ox-taskjuggler.el: A comment is modified only due to
rename of function in org.el.
* lisp/org-agenda.el (org-agenda-check-clock-gap):
* lisp/org-clock.el (org-clock-get-clocked-time)
(org-clock-resolve-clock, (org-clock-resolve)
(org-resolve-clocks, org-resolve-clocks-if-idle)
(org-clock-in, org-clock-out, org-clock-sum, org-clocktable-steps):
* lisp/org-element.el (org-element-cache-sync-duration)
(org-element--cache-set-timer, org-element--cache-interrupt-p):
(org-element--cache-sync):
* lisp/org-habit.el (org-habit-insert-consistency-graphs):
* lisp/org-indent.el (org-indent-add-properties):
* lisp/org-timer.el (org-timer-start):
(org-timer-pause-or-continue, org-timer-set-timer):
* lisp/org.el (org-today, org-auto-repeat-maybe): Port time-related
changes from the Emacs repo by using compatibility wrappers.
In the Emacs repo, there has been a lot of changes to Org files
involving time-related code. I've ported some of those changes but
have largely ignored any changes that break compatibility with older
Emacsen that we support. That, however, isn't a good approach because
it will be hard to do a systematic update once we bump our minimum
Emacs requirement. Instead use the recently added compatibility
wrappers where needed, which is ugly but more maintainable.
The main time-related changes this leaves unported are changes that
replace (apply #'encode-time args) calls with (encode-time args).
Until the first form is unsupported, adding a compatibility function
doesn't seem worth the churn.
Relevant Emacs commits include
c75f505dea6a560b825384cf3d277690f86840bf,
57c74793c46c6533b63836f00aecaf3ac2accb6d,
988e37fa0f922b852715671d59a0e3f682373411,
476066e89d6f0bb87220da690b8a476bf9655b80,
89c63b3522b62c0fd725f0b348927a2069238452.
* lisp/org-agenda.el (org-agenda-to-appt):
* lisp/org-clock.el (org-clock-resolve-clock)
(org-clock-resolve, org-resolve-clocks-if-idle):
* lisp/org-colview.el (org-columns-edit-value, org-columns)
(org-agenda-columns):
* lisp/org-element.el (org-element--cache-interrupt-p)
(org-element--cache-sync):
* lisp/org-habit.el (org-habit-get-faces)
(org-habit-insert-consistency-graphs):
* lisp/org-indent.el (org-indent-add-properties):
* lisp/org-timer.el ((org-timer-show-remaining-time):
* lisp/org.el (org-babel-load-file, org-current-time)
(org-today, org-auto-repeat-maybe)
(org-small-year-to-year, org-goto-calendar):
* lisp/ox.el (org-export-insert-default-template):
Use nil instead of (current-time) where either will do, as nil is
a bit more efficient and should have less timing error.
Prefer nil to (current-time) when either works
c75f505dea6a560b825384cf3d277690f86840bf
Paul Eggert
Fri Oct 20 19:42:23 2017 -0700
Note(km): The changes that will reverted in the next commit have been
dropped from the ChangeLog entries above.
* lisp/org-element.el (org-element--current-element): Increase number
of checks to avoid more false positive.
In particular "+-+" is not recognized anymore as a table.el table.
* lisp/org-element.el (org-element-headline-parser): Check for space
after todo-kwd.
* testing/lisp/test-org-element.el (test-org-element/headline-todo-keyword):
Respective tests.
* lisp/org-capture.el (org-capture-member): Make obsolete; the old
definition was identical to ‘org-capture-get’ anyway.
(org-capture-mode-map): Move the calls to ‘define-key’ up to where the
variable is defined.
(org-capture-mode-hook): Small docstring tweak.
(org-capture-mode): Fix typo in mode lighter.
(org-capture-set-target-location, org-capture-place-item):
(org-capture-place-plain-text, org-capture-narrow):
(org-capture-empty-lines-after):
(org-capture-import-remember-templates): ‘if’ without else -> ‘when’
* lisp/org-colview.el (org-columns-edit-value): Change an error to a
user-error.
(org-columns-uncompile-format): Improve docstring.
* lisp/org-compat.el (org-remove-from-invisibility-spec): Make
obsolete, the underlying emacs function exists since 1997, commit 31aa282e.
(org-in-invisibility-spec-p, org-count-lines): ‘if’ without else -> ‘when’.
* lisp/org-element.el (org-element-swap-A-B):
* lisp/org-entities.el (org-entities-create-table):
* lisp/org-list.el (org-insert-item):
* lisp/org-macs.el (org-with-point-at, org-base-buffer):
(org-preserve-local-variables, org-overlay-display):
(org-overlay-before-string): ‘if’ without else -> ‘when’.
* lisp/org-eshell.el (org-eshell-open): Fix docstring typo.
* lisp/org-pcomplete.el (pcomplete/org-mode/file-option/language):
(pcomplete/org-mode/file-option/startup):
(pcomplete/org-mode/file-option/options):
(pcomplete/org-mode/file-option/infojs_opt):
(pcomplete/org-mode/link, pcomplete/org-mode/tex):
(pcomplete/org-mode/todo, pcomplete/org-mode/searchhead):
(pcomplete/org-mode/tag, pcomplete/org-mode/prop): Avoid the formerly
misspelled ‘pcomplete-uniqify-list’ function. It has a defalias in
emacs >= 27; we add our own for older emacsen.
(pcomplete/org-mode/file-option/bind): ‘if’ without else -> ‘when’.
* lisp/org-protocol.el (org-protocol-capture):
(org-protocol-convert-query-to-plist): ‘if’ without else -> ‘when’.
(org-protocol-do-capture): Pacify byte compiler, simplify conditional
logic.
* lisp/org-table.el (org-table-create-with-table.el): Simplify conditional
logic.
(org-table-create, org-table-convert-region, org-table-next-field):
(org-table-beginning-of-field, org-table-end-of-field):
* lisp/org-w3m.el (org-w3m-copy-for-org-mode): ‘if’ without else ->
‘when’.
* lisp/org.el (org-babel-do-load-languages, org-previous-link):
(org-refile): Use ‘(foo ...)’ instead of ‘(funcall 'foo ...)’.
(org-add-log-note): Convert a long cond into a cl-case.
(org-priority): Improve docstring, show a deprecation warning if the
‘show’ argument is passed (which was previously silently ignored).
Also, use ?\s instead of ?\ as a character literal for space.
(org-fast-tag-insert): Fix docstring typo.
(org-fill-element): ‘if’ without else -> ‘when’.
(org-on-target-p): Remove ancient compatibility alias.
* lisp/org-element.el (org-element-macro-parser): Ignore
non-meaningful spaces (e.g., indentation).
(org-element-macro-interpreter): Do not interpret macro from its raw
value. Instead, build it again from its parsed contents.
* lisp/org-element.el (org-element--list-struct): Correctly handle end
of sub-lists and end of last item.
* testing/lisp/test-org-element.el (test-org-element/item-parser):
(test-org-element/plain-list-parser): Add tests.
* lisp/org-element.el (org-element-latex-fragment-parser): Also check
border character right after opening "$" sign.
* testing/lisp/test-org-element.el (test-org-element/latex-fragment-parser):
Add tests.
Reported-by: thomas <thomas@friendlyvillagers.com>
<http://permalink.gmane.org/gmane.emacs.orgmode/114285>
* lisp/org-element.el (org-element-context): Do not parse timestamps
within planning line.
* testing/lisp/test-org-element.el (test-org-element/context): Remove test.
Strictly speaking, timestamps within planning lines are parameters
values that use timestamp syntax, not real timestamps belonging to the
document contents.
* lisp/org-element.el (org-element-latex-fragment-parser):
* lisp/org.el (org-latex-regexps): Allow an apostrophe right after
a fragment.
* testing/lisp/test-org-element.el (test-org-element/latex-fragment-parser):
Add test.
In Text mode, and, as a consequence, in Org mode, "'" is not treated
as punctuation, but as word constituent. The apostrophe isn't caught
by "\s." regexp.
Reported-by: Joe Corneli <holtzermann17@gmail.com>
<http://permalink.gmane.org/gmane.emacs.orgmode/113464>
* lisp/org-element.el (org-element--object-lex): Fix parsing of radio
links within emphasis.
* testing/lisp/test-org-element.el (test-org-element/link-parser): Add
test.
Reported-by: R C <recifx@gmail.com>
<http://permalink.gmane.org/gmane.emacs.orgmode/113063>
* lisp/org-element.el (org-element-link-parser):
Handle drive names in uri like file:///c:/dir/file
* lisp/ox.el (org-export-file-uri):
Handle drive names in uri like file:///c:/dir/file
* testing/lisp/test-ox.el (test-org-export/file-uri):
Generate the right uri to be tested against link exporter.
* lisp/org-element.el (org-element-insert-before): Do not call
`reverse' as contents might use `eq' objects.
* testing/lisp/test-ox.el (test-org-export/uninterpreted): Add test
Reported-by: Justin Kirby <justinkirby@gmail.com>
<http://permalink.gmane.org/gmane.emacs.orgmode/112065>
* lisp/org-element.el (org-element-use-cache): Set default value to
nil.
Using cache introduces Emacs' freezes under not well known
circumstances. Until this is sorted out, this variable should be nil,
at least on "maint" branch.
* lisp/org-element.el (org-element--cache-objects): Remove variable.
(org-element--cache-put):
(org-element--cache-remove):
(org-element--cache-process-request):
(org-element-cache-reset):
(org-element-context): Apply removal.
Caching objects is complicated and doesn't buy much in most cases,
since a change in an element reset the whole objects cache for that
element.
* lisp/org-element.el (org-element-normalize-contents): Fix return
value when any line after the first has no indentation.
* testing/lisp/test-org-element.el (test-org-element/normalize-contents):
Add test.
* lisp/org-element.el (org-element-class): Make it a defsubst.
* lisp/org-table.el (orgtbl-to-generic): Do not use cache when
building Org table. Factor out calls to Org Export functions when
they are not necessary.
(org-table--to-generic-row): Factor out calls to Org Export functions
when they are not necessary.
* lisp/ox.el (org-export-resolve-fuzzy-link):
(org-export-table-has-header-p):
(org-export-table-row-group):
(org-export-table-cell-width):
(org-export-table-cell-alignment): Small refactoring.
(org-export-table-row-number): Add caching.
* testing/lisp/test-org-element.el (test-org-element/class): Remove
test.
* lisp/org.el (org-set-emph-re): Refactor code. Rip "~" and "="
markers off `org-emph-re'.
(org-do-emphasis-faces):
(org-sort-remove-invisible): Handle both `org-emph-re' and
`org-verbatim-re'.
(org-in-verbatim-emphasis): Use `org-verbatim-re' instead of
`org-emph-re'.
* lisp/org-element.el (org-element-code-parser):
(org-element-verbatim-parser): Use `org-verbatim-re' instead of
`org-emph-re'.
* testing/lisp/test-org-element.el (test-org-element/bold-parser):
(test-org-element/code-parser):
(test-org-element/italic-parser):
(test-org-element/strike-through-parser):
(test-org-element/underline-parser):
(test-org-element/verbatim-parser): Update tests, which no longer need
to bind `org-emph-re'.
* lisp/org-element.el (org-element--current-element): Check that
planning info and property drawers are correctly located before
parsing them.
* testing/lisp/test-org-element.el (test-org-element/planning-parser):
(test-org-element/property-drawer-parser): Add tests.
* lisp/org-element.el (org-element--object-lex): Do not re-match past
1-character long radio links.
* testing/lisp/test-org-element.el (test-org-element/link-parser): Add
test.
Reported-by: Daniel Clemente <n142857@gmail.com>
<http://permalink.gmane.org/gmane.emacs.orgmode/109861>
* lisp/org-element.el (org-element-class): New function.
(org-element-map):
(org-element-interpret-data):
* lisp/org-footnote.el (org-footnote--allow-reference-p):
* lisp/org-src.el (org-src--on-datum-p):
* lisp/ox-odt.el (org-odt-footnote-reference):
(org-odt-table-cell):
* lisp/ox.el (org-export-data):
(org-export-expand): Use new function.
* testing/lisp/test-org-element.el (test-org-element/class): New test.
Using generic `org-element-class' allows to handle unknown, i.e.,
pseudo, object or element types. It also reduces code duplication in
`org-element-interpret-data' and `org-export-data', preventing, e.g.,
bugs as the one fixed in c58e1b5.
* lisp/org-element.el (org-element-link-interpreter): % in description
are confused with format string placeholders. Escape them so as to
fix the error.
* testing/lisp/test-org-element.el (test-org-element/link-interpreter):
Add test.
Reported-by: Daniel Clemente <n142857@gmail.com>
<http://permalink.gmane.org/gmane.emacs.orgmode/109878>
* lisp/org-element.el (org-element-interpret-data):
* lisp/ox.el (org-export-data): Correctly guess if a pseudo element or
object should be followed by spaces or blank lines.
* testing/lisp/test-ox.el (test-org-export/pseudo-objects):
(test-org-export/pseudo-elements): New tests.
* lisp/org-element.el (org-element-link-parser): Add :format property.
Tiny refactoring.
* testing/lisp/test-org-element.el (test-org-element/link-interpreter):
Add one test. Update some others.
Since the link format is lost during parsing, the interpreter can hardly
handle nested links. Indeed, in that case, the inner link is
interpreted as a bracket link, thus breaking the outer link.
Another option could be to guess a safe format for the link to be
interpreted. E.g.,
- any inner link could default to angle format unless it contains ">",
it which case it would become a plain link;
- other links would have the bracket format, unless they contain "][" or
"]]".
In any case, defining :format is less error prone and is similar to what
entities and sub/super-scripts have for curly brackets.
Reported-by: Thibault Marin <thibault.marin@gmx.com>
<http://permalink.gmane.org/gmane.emacs.orgmode/109623>
* lisp/org-element.el (org-element-interpret-data): Elements and objects
are compared with `eq' instead of `equal'. Also remove useless symbol
prefix since move to lexical binding.
Ihor Radchenko
Nicolas Goaziou
Kyle Meyer
Mattias Engdegård
Stefan Kangas
Bastien
Stefan Monnier
Juri Linkov
Leo Vivier
Gustav Wikström
Paul Eggert
Marco Wahl
Aaron Ecay
Tim Landscheidt
Kaushal Modi
Fabrice Popineau