Commit Graph

25291 Commits

Author SHA1 Message Date
Ihor Radchenko 46da64c4cf
org-persist.el: Fix `add-hook' priority argument 2021-10-19 20:07:31 +08:00
Ihor Radchenko c130f71036
org-persist.el: Fix typo 2021-10-19 20:07:18 +08:00
Ihor Radchenko bad854d5fa
org-persist.el: Catch write errors
* lisp/org-persist.el (org-persist--get-index): Check index file.
(org-persist--read-index): Warn when index file is missing.
(org-persist-write): Report write access errors.
2021-10-19 20:06:08 +08:00
Ihor Radchenko 3b48224994
org-persist.el: Rename `org-persist-path' to `org-persist-directory' 2021-10-19 20:05:30 +08:00
Marco Wahl 7810804222 Merge branch 'bugfix' into main 2021-10-18 15:13:27 +02:00
Marco Wahl 37791bb4fc org-manual: Fix one kindex entry
Reminder: the kindex can be accessed via key "i" in info.  This fixes
also the pdf generation.
2021-10-18 15:09:58 +02:00
Ihor Radchenko d574186448
Disable cache persistence during build
* mk/default.mk (BATCH): Set `org-element-cache-persistent' to nil
when running make.
2021-10-18 20:59:10 +08:00
Ihor Radchenko 821977611c
org-agenda-get-scheduled: Make sure that 'org-marker is at timestamp
* lisp/org-agenda.el (org-agenda-get-scheduled): Use beginning
position of the timestamp as 'org-marker.

Fixes https://list.orgmode.org/87sfwyrcyj.fsf@gnu.org/T/#meeca619b0abd3c3b30fe5057b51c891e8de705b8
2021-10-18 16:07:40 +08:00
Ihor Radchenko d18beb7c6f
Fix effort calculation in agenda
* lisp/org-agenda.el (org-agenda-get-scheduled): Fix property symbol
in `org-element-property' call.

(org-agenda-get-todos, org-agenda-get-scheduled,
org-agenda-get-timestamps, org-agenda-get-sexps,
org-agenda-get-progress, org-agenda-get-deadlines,
org-agenda-get-blocks, org-agenda-change-all-lines): Pass effort
properties to `org-agenda-format-item'
2021-10-18 13:40:06 +08:00
Ihor Radchenko 0e912bd288
org-persist.el: Make sure that org-persist-path is absolute 2021-10-18 13:37:02 +08:00
Ihor Radchenko 7a04f3b1cd
Fix 'org-hd-marker position changed in 0ef88e2d9
* lisp/org-agenda.el (org-agenda-get-scheduled): Move to correct point
before calculating the marker position.
2021-10-17 18:48:33 +08:00
Ihor Radchenko 0ef88e2d91
org-agenda.el/org-agenda-get-scheduled: Use cache
* lisp/org-agenda.el (org-agenda-get-scheduled): Use
`org-element-cache-map' for faster agenda generation.
2021-10-17 18:29:35 +08:00
Ihor Radchenko f4bcc0cc56
Merge branch 'main' into feature/org-element-cache-new 2021-10-17 14:49:59 +08:00
Ihor Radchenko 004ac14a5b
Fix compatibility with Emacs 26
* lisp/org-compat.el (org-file-name-concat): Do not use
`string-empty-p'.
(combine-change-calls): Create a stub when `combine-change-calls' were
not yet available.
(org-replace-buffer-contents): Add compatibility function for
`replace-buffer-contents'.

* lisp/org-element.el (org-element--current-element): Do not use
`if-let'.
* lisp/org-persist.el (org-persist-gc): Do not use `when-let'.
* lisp/org-plot.el (org-plot/gnuplot): Do not use `if-let'.
* lisp/org-src.el (org-edit-src-save, org-edit-src-exit): Use
`org-replace-buffer-contents'.

* lisp/org.el (org-narrow-to-subtree, org--property-local-values,
org-entry-get-with-inheritance, org-in-commented-heading-p,
org-up-heading-safe, org-goto-first-child): Do not use
`if-let'/`when-let'.

* testing/org-test.el (org-test-at-time): Fallback to old
`decode-time' specification in older Emacs.
2021-10-17 14:34:10 +08:00
Ihor Radchenko 8ceb9e7902
Add NEWS entry for new element cache 2021-10-17 13:57:11 +08:00
Ihor Radchenko eeb04504ef
Use XDG cache dir as default dir for cache persistance
* lisp/org-persist.el (org-persist-path): Change default dir to XDG
cache dir and fall back to `user-emacs-directory' as the last resort.
2021-10-17 12:24:32 +08:00
Ihor Radchenko 85712d6456
Add comments documenting cache diagnostics 2021-10-17 12:23:58 +08:00
Ihor Radchenko 07ca988bb4
Fix compatibility with Emacs 27
* lisp/org-compat.el: Remove `length>' alias
* lisp/org-element.el (org-element-cache-map): Suppress compiler
warning.
* lisp/org-persist.el (org-persist-register): Do not `push' into
`plist-get'.  Not supported yet in Emacs 27.
(org-persist-unregister): Do not use `length>'.  Use
`org-directory-empty-p'.
(org-persist-gc): Use `org-directory-empty-p'.
2021-10-17 11:44:33 +08:00
Kyle Meyer 9475993a81 Merge branch 'bugfix' 2021-10-16 13:56:43 -04:00
Kyle Meyer 9bc3a2b98d ORG-NEWS: Mention org-table-blank-field keybinding removal
The keybinding was dropped in the 9.5 release by 0c4e844c8 (Remove
default binding for org-table-blank-field, 2021-04-28), but the change
didn't come with a NEWS entry.  9.5 is already out, but add an entry
anyway because it will hopefully still help some users, including
those that use the Org that ships with Emacs.

Reported-by: Michael Brand <michael.ch.brand@gmail.com>
Link: https://list.orgmode.org/87lf2wcbqs.fsf@gnu.org
2021-10-16 13:55:45 -04:00
Ihor Radchenko abe7222ed8
Add declares to suppress compiler warnings 2021-10-17 00:01:56 +08:00
Ihor Radchenko 885808fd58
Fix incorrectly written test
* testing/lisp/test-org.el (test-org/entry-get): The test was passing
in the past because of bug in org.el.
2021-10-17 00:00:21 +08:00
Ihor Radchenko 3c4290e668
org.el/org-scan-tags: Make use of fast `org-element-cache-map' 2021-10-17 00:00:01 +08:00
Ihor Radchenko 85e0a69567
Avoid frequent cache updates in some functions
* lisp/org.el (org-promote-subtree, org-demote-subtree,
org-paste-subtree, org--align-node-property): Group buffer changes
together and call after-change-functions once to avoid performance
degradation during cache updates.
2021-10-16 23:58:40 +08:00
Ihor Radchenko 5aeeb4f739
Use `org-element-at-point-no-context' in performance-critical places
* lisp/org-compat.el (org-mode-flyspell-verify, org-indent-line,
org-indent-region, org-indent-drawer, org-indent-block): Use
`org-element-at-point-no-context' in place of `org-element-at-point'
when we are doing changes.
* lisp/org.el (org-collect-keywords): Use direct parsing in place of
`org-elemnt-at-point'
2021-10-16 23:56:29 +08:00
Ihor Radchenko e70a8aac59
Use org-element-cache in place of text property cache in agenda
* lisp/org-agenda.el (org-agenda-skip): Use
`org-in-archived-heading-p' and `org-in-commented-heading-p' in place
of text property cache.

(org-agenda-get-todos, org-agenda-get-timestamps,
org-agenda-get-sexps, org-agenda-get-progress,
org-agenda-get-deadlines, org-agenda-get-scheduled,
org-agenda-get-blocks): Do not use text property cache in favour of
Org API functions.  The API functions use cache now.

* lisp/org-clock.el (org-element--cache-active-p): Declare function to
suppress compiler warning.

(org-clock-in): Do not use text property cache when element cache is
active.

* lisp/org-duration.el (org-duration-to-minutes): Do not change match
data.  It is needed to not break agenda---agenda relies on match data
not being altered.

* lisp/org.el (org-run-like-in-org-mode): Use element cache.
(org-refresh-category-properties): Use element cache.
(org-make-tags-matcher, org-agenda-prepare-buffers): Do not rely on
text property cache.

* testing/lisp/test-org.el (test-org/refresh-category-properties): Do
not use text property cache.
2021-10-16 23:50:21 +08:00
Ihor Radchenko 60c927f8b8
Add new element parser and cache tests 2021-10-16 23:42:30 +08:00
Ihor Radchenko fe6cefdaaf
ox.el: Support cache during export
* lisp/ox.el (org-export-ignored-local-variables): Preserve cache
state in export buffer.
(org-export--generate-copy-script): Fix cache confusion about buffer
change tic.
2021-10-16 23:41:00 +08:00
Ihor Radchenko fede2588e4
org.el/org-in-archived-heading-p: Support cache and passing element arg 2021-10-16 23:40:28 +08:00
Ihor Radchenko 86345df9ab
org.el/org-in-commented-heading-p: Support cache and passing element arg 2021-10-16 23:40:14 +08:00
Ihor Radchenko 399a29c4f4
org.el/org-up-heading-safe: Add cache support 2021-10-16 23:39:43 +08:00
Ihor Radchenko ec737554d0
org.el/org-end-of-subtree: Support cache and passing element arg 2021-10-16 23:39:11 +08:00
Ihor Radchenko d43781707b
org.el/org-goto-first-child: Support cache and passing element arg 2021-10-16 23:38:46 +08:00
Ihor Radchenko 5d162b7bcf
org.el/org-back-to-heading: Handle inlinetasks correctly
* lisp/org.el(org-back-to-heading): Rewrite `outline-back-to-heading'
considering inlinetasks.
2021-10-16 23:37:15 +08:00
Ihor Radchenko 5bf5fdbc28
org.el/org-entry-get-with-inheritance: Support cache and passing element arg
* lisp/org.el (org-entry-get-with-inheritance): Add cache support.
Add new optional argument---an element to get properties from.  Fix
getting top-level properties when a headline is located at BOB.
2021-10-16 23:33:10 +08:00
Ihor Radchenko 7b83168295
org.el/org--property-local-values: Support cache and passing element arg 2021-10-16 23:32:39 +08:00
Ihor Radchenko 78abbcd052
org.el/org-get-tags: Support cache and passing element arg
* lisp/org.el (org-fast-tag-selection): Allowing passing an element or
position as first argument.  Add cache support.
2021-10-16 23:31:15 +08:00
Ihor Radchenko 38b632d2ea
org.el/org--get-local-tags: Add cache support 2021-10-16 23:30:52 +08:00
Ihor Radchenko 7159ec0be0
org.el/org-at-planning-p: Add cache support 2021-10-16 23:30:13 +08:00
Ihor Radchenko 68a44eadac
org.el/org-narrow-to-subtree: Support cache and passing element arg 2021-10-16 23:29:11 +08:00
Ihor Radchenko bc52c4d9ab
Fix compatibility for older Emacs versions
* lisp/org-compat.el (org-length>, org-file-name-concat): Add backward
compatibility for `length>' and `org-file-name-concat'.
2021-10-16 23:26:43 +08:00
Ihor Radchenko fc80d052db
Re-implement org-element-cache and add headline support
* lisp/org-element.el (org-element-with-disabled-cache): New macro.

(org-element-greater-elements): Add new org-data element.  It
functions like a virtual headline containing the whole buffer.  The
org-data properties are like headlie properties, but according to the
top-level drawer.  org-data's category is the buffer's category as
defined by top-level property drawer, #+CATEGORY keyworsd, and the
buffer file name.

(org-element--cache-element-properties, org-element-set-element): New
variable containing properties to be transferred when updating changed
element in cache in `org-element-set-element'.

(org-element--get-node-properties): Allow parsing node propreties in
top-level drawer when new optional argument is passed.  Respect
PROPERTY+ syntax.

(org-element--get-global-node-properties): New function.  It returns
node properties for top-level property drawer.

(org-element-org-data-parser, org-element-org-data-interpreter):
Implement the new org-data element.

(org-element-headline-parser, org-element-section-parser): Add new
:robust-begin and :robust-end
properties delimiting safe changes that do not modify headline
element.

(org-element--list-struct): Fix cache update when adding a headline
inside list.

(org-element--current-element): Implement cache support.  Record
parsing mode (:mode) and parsing granularity (:granularity) in the
element properties.

(org-element-parse-buffer, org-element--next-mode): Support new
org-data element.

(org-element--parse-elements): Record parsing granularity in the
returned tree

(org-element-use-cache): Enable cache by default.

(org-element-cache-persistent): New variable controlling cache
persistance across sessions.  Enabled by default.

(org-element--cache-self-verify,
org-element--cache-self-verify-frequency,
org-element--cache-diagnostics, org-element--cache-map-statistics,
org-element--cache-map-statistics-threshold,
org-element--cache-diagnostics-level,
org-element--cache-diagnostics-ring,
org-element--cache-diagnostics-ring-size): New variables controlling
cache diagnostics and self-diagnostics.  Greatly simplifies cache
debugging.

(org-element--cache, org-element--cache-sync-requests,
org-element--cache-sync-timer): Make cache buffer-local by default.

(org-element--headline-cache): Implement separate cache storing only
headlines and inlinetasks.

(org-element--cache-size, org-element--headline-cache-size): New
variables containing cache sizes.  This is much faster than
`avl-tree-size'.

(org-element--cache-sync-requests): Update docstring explaning the
request list structure.

(org-element--cache-sync-keys-value): New variable replacing
`org-element--cache-sync-keys' hash table.  The hash table was not
reliable because it was using elements as keys.  Upon any cached
element update/shift, the keys were invalidated making cache ordering
incorrect and breaking the cache badly.  Now, the cache keys are
stored as :org-element--cache-sync-key element property and the new
variable stores marker value indicating the current sync request
cycle.  See `org-element--cache-key' for more details.

(org-element--cache-change-tic): New variable controlling buffer
modification count that is registered in cache.  This variable allows
catching "stealth" edits.

(org-element--cache-non-modifying-commands): New variable listing
commands that will not be slown down if we fill cache on the fly.

(org-element--request-key, org-element--request-beg,
org-element--request-end, org-element--request-offset,
org-element--request-parent, org-element--request-phase): New macros.
They improve code readability (especially when using nameless-mode).

(org-element--format-element, org-element--cache-log-message,
org-element--cache-warn): New macros implementing generic logging
functionality.

(org-element--cache-key): Add section and org-data element support.
Change cache key storage from hash map to :org-element--cache-sync-key
element property + `org-element--cache-sync-keys-value'.  We use the
latter to group all the cache keys during a single cache request
sequence.  Once sync request is fully complete, the
`org-element--cache-sync-keys-value' is updated making all the old
sync keys obsolete (they will still be store as element properties).

(org-element--headline-cache-root): New function returning headline
cache root.

(org-element--cache-active-p): Prevent cache updates when
`inhibit-modification-hooks' is non-nil, unless non-nil optional
argument is provided.

(org-element--cache-find): Share cache between indirect buffers and
the base buffer.  We have to do it because after-change hooks for
indirect buffer are not called in the base buffer and vice versa.  Add
support for section and org-data elements.

(org-element--cache-put): Implement new approach for cache key
storage.  Add diagnostics.  Indicate cached elements using :cached
element property.  Support cache size calculation.

(org-element--cache-remove): Invalidate parent contents when removing
element.  Support cache size calculation.  Detect cache corruption due
to misordered elements.

(org-element--cache-shift-positions): Support :robust-begin and
:robust-end element properties.

(org-element--cache-sync): Add diagnostics.  Add detailed comments.
Prevent slowdown when large cache chunks need to be deleted forcing
O(N) complexity cutoff.  In phase 2, fix cases when next request
contains deleted cache key.  In phase 2, fix scenario when newly
inserted element intersects with existing elements in cache.  In phase
2, detect obsolete parents removed from cache.

(org-element--open-end-p): New function checking if an element can
have blank lines right after its :contents-end.

(org-element--parse-to): Do not alter match data.  Process complex
parsing mode changes correctly.  Support headlines in cache.  Support
org-data parsing.  Add detailed comments.  Add diagnostics.

(org-element--cache-sensitive-re): Make list lines sensitive.

(org-element--cache-change-warning): Update docstring.  Now, the
variable can have t, nil, and number values.  Numbers are used to
provide more details about changed headlines.

(org-element--cache-before-change, org-element--cache-after-change):
Handle headline hierarchy.  Properly handle cache in indirect
buffers.

(org-element--cache-after-change): Update docstring clarifying the
return values.  Add special handling for headline and org-data
elements updating them in-place instead of removing together with the
whole contents when possible.  Use :robust-begin/:robust-end element
properties to detect robust changes.

(org-element--cache-submit-request): Add detailed comments.  Correctly
handle cache in indirect buffers.  Delegate element modifications to
`org-element--cache-for-removal'.

(org-element--cache-verify-element): New function for cache
self-verification.

(org-element--cache-persist-before-write,
org-element--cache-persist-before-read,
org-element--cache-persist-after-read): Implement cache persistance.

(org-element-cache-reset): Correctly handle cache in indirect
buffers.  Support cache persistance.  Support new cache size
calculation and new cache key schema.

(org-element-cache-map): New function analagous to `org-element-map',
but much faster.  The function overperforms org-ql written by Adam
Porter aka alphapapa [1] and reuses some ideas from there (namely,
fast element skipping via regexps).

[1] https://github.com/alphapapa/org-ql/

(org-element-at-point): The returned elements are now guaranteed to
have correct parents up to org-data.  New optional argument
CACHED-ONLY limits element search to current cache---if element is not
in cache and current command is not in cache
`org-element--cache-non-modifying-commands', the cache is not updated
and the function returns nil.  Also, support cache verification.

(org-element-at-point-no-context): New function.  It is analogous of
older `org-element-at-point' with no guarantee that :parent properties
are correct beyond direct parent heading.  This function does not
update cache and can be useful when cache updates should be avoided
for performance reasons.

* lisp/ob-core.el (org-babel-where-is-src-block-result): Support
section and org-data elements in cache.

* lisp/org-macro.el (org-macro-replace-all,
org-macro--find-keyword-value): Support org-element-cache.

* lisp/org-table.el (orgtbl-to-generic): Support org-element-cache.

* lisp/org.el (org-mode): Add cache persistance.

(org-up-element): Preserve old behaviour when error is returned for
section and org-data element.

*
testing/lisp/test-org-archive.el (test-org-archive/update-status-cookie):
Fix test when cache is active.

* testing/lisp/test-org-colview.el (test-org-colview/columns-update):
Fix test.

* testing/lisp/test-org-element.el (test-org-element/extract-element):
Add suport for new org-data element.

* testing/lisp/test-org-element.el (test-org-element/parent-property):
Fix equality check.  Parents returned by cache and `org-element-map'
may not be `eq' now.  Just `equal'.

* testing/lisp/test-org-element.el (test-org-element/context): Support
section and headline parents.
2021-10-16 23:22:18 +08:00
Ihor Radchenko 6933c1ad78
lisp/org-persist.el: New library to store data across sessions 2021-10-16 21:16:11 +08:00
Nicolas Goaziou ad224f5b8e Merge branch 'bugfix' 2021-10-15 19:45:44 +02:00
Nicolas Goaziou 745f0913fc oc-basic: Load unconditionally
oc-basic is used for opening links and for fontifying them out of the
box.
2021-10-15 19:41:11 +02:00
Marco Wahl 93132c2390 org-goto: Fix UI
* lisp/org-goto.el (org-goto-location): Drop the wrap by
`org-no-popups'.  This restores the expected functionality of org-goto
with interface type 'outline.

See ml: https://list.orgmode.org/87mtnovv7f.fsf@alphapapa.net/.
2021-10-14 12:07:49 +02:00
Nicolas Goaziou 8799422a32 Merge branch 'bugfix' 2021-10-13 22:23:03 +02:00
Nicolas Goaziou 009900d7fa oc: Add autoload cookie for `org-cite-insert'
* lisp/oc.el (org-cite-insert): Autoload function.
2021-10-13 22:22:22 +02:00
Nicolas Goaziou c83fe001c7 oc: Fix error message
* lisp/oc.el (org-cite-insert): Fix copy-pasta in error message.
2021-10-13 22:20:11 +02:00
Nicolas Goaziou 2b1fc6ba72 Merge branch 'bugfix' 2021-10-11 23:59:00 +02:00