* lisp/org-element.el (org-element--cache-persist-after-read): When
the persistent cache has elements without parent, do not use such
cache and report a warning to user.
This garbage collects many times when calling
`org-get-buffer-tags' (such as during tag completion) in a large file
with many tags. Raise `gc-cons-threshold' as this may affect other
callers in large files.
Note that `org-element-parse-buffer' also raises `gc-cons-threshold'
as a precedent.
This has no effect on most callers and increases performance for
extreme cases.
* lisp/org-element.el (org-element-cache-map): Raise
`gc-cons-threshold'.
* lisp/org-element.el (org-element-item-parser): Do not parse beyond
LIMIT - they may extend :post-blank beyond parent list contents.
(org-element-plain-list-parser): Make sure that plain list always owns
the trailing blank lines.
(org-element-cache-version): Bump cache version.
* testing/lisp/test-org-element.el (test-org-element/item-parser): Add
test.
Reported-by: Tom Alexander <tom@fizz.buzz>
Link: https://orgmode.org/list/1c833eb8-c556-437b-ac5b-be360ebcc869@app.fastmail.com
* lisp/org-element.el (org-element-cache-version): New constant
defining the current cache version.
(org-element-cache-reset): Use the new variable.
* lisp/org.el (org-mode): Assert loading the correct cache version.
Reported-by: Gregor Zattler <telegraph@gmx.net>
Link: https://orgmode.org/list/87350cvm3d.fsf@no.lan
* lisp/org-element.el (org-element--string-cache): Change the value to
hash table. Update the docstring.
(org-element--get-cached-string): Use hash table.
obarrays have a number of rough edges when used for purposes other
than interning symbols. Using obarrays may cause unexpected problems
and hash tables are preferred. Credit: Stefan Monnier.
* lisp/org-element.el (org-element-at-point): Explicitly throw an
error when not in org-mode buffer. The parser expects certain
buffer-local constants and may fail otherwise. It is better to
indicate the problem explicitly instead of dealing with cryptic
generic errors.
(org-element-plain-list-interpreter):
* testing/lisp/test-ox.el (test-org-export/expand-include/links): Make
sure that we never invoke parser in non-Org buffers.
* lisp/org-element-ast.el (org-element--plist-property): Shield
annoying warning spam with `with-no-warnings'. The warnings are not
present in newer Emacs.
* lisp/org-element.el (org-element--interpret-affiliated-keywords):
Suppress warning about two token unused `_' arguments.
* lisp/org-id.el (org-id-find-id-in-file): Suppress warning about
number of arguments.
* lisp/org-element.el (org-element--cache-before-change): Fix edge
case when a heading is created because we insert a newline right
before ****.
* testing/lisp/test-org-element.el (test-org-element/cache-headline):
Add tests.
Reported-by: Rodrigo Morales <moralesrodrigo1100@gmail.com>
Link: https://orgmode.org/list/CAGxMbPbbqc33iaqJ=EceyKrLaf4maJAxaUmJGaPOvG_Rpw+xcQ@mail.gmail.com
* lisp/org-element.el (org-element--parse-to): Remove numerous special
conditions added to satisfy tests in `test-org-element/at-point'.
They create divergence with `org-element-parse-buffer' by sometimes
parsing inner elements up to beyond parent's :contents-end. Instead,
modify org-data, section, and headline parsers to make their contents
include trailing blank lines. This way, `org-element-at-point' will
naturally transcend inside section at the end of a heading.
(org-element-headline-parser):
(org-element-org-data-parser):
(org-element-section-parser): Span :contents-end all the way up to
:end. Set :post-blank to 0 all the time. Adjust :robust-end.
(org-element--open-end-p): Remove no longer used function.
Reported-by: Tom Alexander <tom@fizz.buzz>
Link: https://orgmode.org/list/6dd08eb7-9468-402d-bf88-eefb01442b99@app.fastmail.com
* lisp/org-element.el:
(org-element--cache-for-removal): Explicitly assign :granularity to
not deceive parser when parsing for different granularity than cache.
Cache is always using 'element granularity.
This commit is only needed on bugfix. On main, :granularity is set
correctly.
* lisp/org-element.el (org-element--cache-for-removal): Do not update
heading using object granularity. `org-element-at-point' parses
headings without :title objects and cache updates should not change
this.
* lisp/org-element.el (org-element-normalize-contents): Use
`org-element-set-contents' to set contents instead of assuming
(TYPE PROP CONTENTS) structure.
* lisp/org-src.el (org-src-preserve-indentation-p): New function
checking whether block should preserve indentation. This function
abstracts away the check for block type, indentation flag, and
customized `org-src-preserve-indentation' value.
(org-src--edit-element):
* lisp/ob-core.el (org-babel--normalize-body):
(org-babel-read-element):
(org-babel-update-block-body):
* lisp/ob-exp.el (org-babel-exp-process-buffer):
* lisp/org-element.el (org-element-example-block-interpreter):
(org-element-src-block-interpreter):
* lisp/org.el (org-fixup-indentation):
(org-indent-region):
* lisp/ox.el (org-export-unravel-code): Use the new function instead
of duplicating code.
* lisp/ob-haskell.el (org-babel-haskell-export-to-lhs): Add FIXME. We
do not have access to the block element here and cannot easily check
the flag.
* lisp/ob-tangle.el (org-babel-tangle-single-block):
* lisp/org-src.el (org-src-font-lock-fontify-block):
* lisp/org.el (org-indent-line): Check block flag in addition to `org-src-preserve-indentation'.
This commit unifies logic deciding whether to preserve block
indentation into a single place to avoid confusion.
The new property is needed to reduce ambiguity when interpreting the
parsed data.
* lisp/org-element.el (org-element-timestamp-interpreter): Preserve old
behavior when :range-type is `nil'. Take into account :range-type
value when interpreting ranges. When :range-type is `timerange',
return a timerange (<YYYY-mm-DD HH:MM-HH:MM>). If :range-type is
`daterange' return a daterange (<...>--<...>). When :range-type is
nil, return a daterange (as it was before). When :range-type is
`daterange' or `timerange' and :type is `active'/`inactive', throw an
error.
(org-element-timestamp-parser): Add :range-type property.
* lisp/org.el (org-timestamp-split-range): Make sure that :range-type
is nil for a split timestamp.
* testing/lisp/test-org-element.el
(test-org-element/timestamp-interpreter): Add new tests.
(test-org-element/timestamp-parser): Add tests for :range-type
property.
* etc/ORG-NEWS (Major changes and additions to Org API): Add news about this property.
Link: https://list.orgmode.org/orgmode/87y1ot6dqz.fsf@gmail.com/
* lisp/org-element.el (org-element--headline-archivedp):
(org-element--headline-footnote-section-p):
(org-element--headline-raw-value):
(org-element--headline-parse-title-raw):
(org-element--headline-parse-title-parse):
(org-element--headline-deferred):
(org-element--get-global-node-properties):
(org-element--headline-parent-deferred): New internal constants
holding deferred values used by parser. These values are not a
subject of modification and thus can be re-used instead of
re-allocating a new instance for every new parsed element.
(org-element--headline-parse-title):
(org-element-headline-parser):
(org-element-org-data-parser):
(org-element-inlinetask-parser):
(org-element--parse-to): Use the new constants.
This patch aims for smaller memory footprint of Org AST and cache.
* lisp/org-element.el (org-element--headline-comment-re):
(org-element--timestamp-raw-value-regexp): New internal regexp
constants.
(org-element--headline-parse-title):
(org-element-timestamp-parser): Use the new constants.
* lisp/org-element.el (org-element-dynamic-block-open-re-nogroup):
(org-element--latex-begin-environment-nogroup): New constants for
internal regexps.
(org-element-comment-parser):
(org-element--current-element-re):
(org-element--current-element): Prefer string constants instead of
repeated plain strings that may be considered different by Emacs
regexp compiler. Also, avoid groups in these regexps as creating
groups makes Emacs regexp matches allocate extra memory. Finally,
avoid using \{N\} constructs in regexps as they are slightly more
costly for regexp engine.
* lisp/org-element.el (org-element--current-element): Slight
optimization of checking for property drawer at point. `forward-line'
is significantly faster compared to `beginning-of-line'.
`skip-chars-forward' also provides a bit of improvement compared to
`looking-at-p'.
Reported-by: Mattias Engdegård <mattias.engdegard@gmail.com>
Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63225#80
* lisp/org-element.el (org-element-node-property-parser): Remove
unnecessary `re-search-forward'. It always triggers as long as
`org-property-drawer-re' only allow node properties inside property
drawer. Also, remove unnecessary `case-fold-search' setting that has
no meaning since we do not match against specific letters.
* lisp/org-element.el (org-element-drawer-re-nogroup): New constant
regexp matching drawer line without creating regexp groups.
(org-element--current-element): Use the new constant.
* lisp/org.el (org-drawer-regexp): Ensure that it is the same with
org-element version and mark for removal/alias.
(org-property-drawer-re): Simplify, removing unnecessary matching of
node property structures.
Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63225#62
* lisp/org-element.el (org-element--cache-remove):
(org-element--cache-interrupt-p):
(org-element--cache-sync):
(org-element--cache-process-request):
(org-element--parse-to):
(org-element--cache-submit-request):
(org-element--cache-for-removal): use `org-element-property-1' to
retrieve parent without checking if it is deferred. All the cached
elements have proper parent.
* lisp/org-element.el (org-element--cache-variables): New constant
holding variables involved in cache state.
(org-element-cache-reset): Fix modification hooks when cache is reset
and buffer already have indirect child buffers.
(org-element-with-enabled-cache): New macro, to enable cache around
body.
(org-element-cache-map): Enable cache when executing.
* lisp/org-element.el: Add commentary explaining some regexp-related
optimizations useful for the parser.
Link: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63225
(org-element--headline-deferred):
(org-element-item-parser):
(org-element-item-interpreter):
(org-element--list-struct):
(org-element-plain-list-parser):
(org-element-example-block-parser):
(org-element-fixed-width-interpreter):
(org-element-paragraph-parser):
(org-element-src-block-parser):
(org-element-table-parser):
(org-element--parse-generic-emphasis):
(org-element-export-snippet-interpreter):
(org-element-link-parser):
(org-element--current-element):
(org-element--collect-affiliated-keywords):
(org-element-parse-buffer):
(org-element-normalize-string):
(org-element-normalize-contents):
(org-element--parse-to):
(org-element--cache-before-change):
(org-element--cache-for-removal):
(org-element-context): Avoid storing match data unless strictly
necessary. Explain the necessity in places where we have to use
`save-match-data'. Prefer `looking-at-p' (does not alter match data)
to `looking-at'. Simplify regexps. Update docstrings clearly
indicating when match data might be modified.
* lisp/org.el:
(org-offer-links-in-entry):
* lisp/ob-exp.el (org-babel-exp-process-buffer):
* lisp/org-agenda.el: Fix places where we need to protect match data.
* lisp/org-element.el (org-element--headline-parent-deferred): New
function to calculate :parent of a heading on demand.
(org-element--parse-to): Assign deferred :parent when cache is
inactive.
* lisp/org-element.el (org-element-begin):
(org-element-end):
(org-element-contents-begin):
(org-element-contents-end):
(org-element-post-affiliated):
(org-element-post-blank): New functions for fast access to frequently
used element properties. The functions are `setf'-able.
Update all the relevant code in Org to use the new functions instead
of genetic property queries.
* lisp/org-element.el (org-element-parse-buffer): Do not force-undefer
when new optional argument is provided.
* lisp/org-lint.el (org-lint--generate-reports):
* lisp/org.el (org-unindent-buffer):
* lisp/ox.el (org-export--missing-definitions):
(org-export--annotate-info): Do not undefer when unnecessary.
Adjust all the declare forms.
* lisp/org-element-ast.el (org-element-ast-map):
* lisp/org-element.el (org-element-map): Treat TYPES t as all possible
types. Add new optional parameter to no resolve deferred while
traversing the AST.
* lisp/org-element.el (org-element--substring):
(org-element--unescape-substring): New functions to be used for
deferred buffer substring calculation.
(org-element--headline-raw-value):
(org-element-comment-block-parser):
(org-element-example-block-parser):
(org-element-export-block-parser):
(org-element-latex-environment-parser):
(org-element-src-block-parser):
(org-element-table-parser):
(org-element--parse-generic-emphasis):
(org-element-inline-babel-call-parser):
(org-element-latex-fragment-parser):
(org-element-export-snippet-parser): Defer large buffer substrings.
* lisp/org-element.el (org-element--headline-raw-value): New function
calculating :raw-value property dynamically.
(org-element-headline-parser): Use the new function, deferring
:raw-value to reduce memory footprint of the cache.
* lisp/org-element.el (org-element-map): Use `org-element-ast-map' to
traverse the AST. Update the docstring adding new features from
`org-element-ast-map' - FUN can now throw `:org-element-skip' signal
to skip current node and its child nodes. Clarify the structure of
INFO argument. Reverse the traverse order for dual keywords -
secondary value is now traversed after the main value.
* testing/lisp/test-org-element.el (test-org-element/map): Update the test
reflecting the change in dual keyword traversal order.
* lisp/org-element.el (org-element-parse-buffer): Resolve all the
deferred values in the string. If not, we might leave pointers to
killed buffer.
(org-element--parse-elements): Resolve deferred in objects.
(org-element--object-lex): Store :buffer property.
* lisp/org-macro.el (org-macro--find-date): Do not try to print
:buffer property.
* lisp/org-element.el (org-element--cache-persist-before-write):
(org-element--cache-persist-after-read): Clear and restore
non-printable buffer objects in :buffer property.
* lisp/org-element.el: Do not reverse affiliated keyword order.
(org-element-map): Remove special arrangement to restore keyword
ordering.
(org-element--interpret-affiliated-keywords): Obey the new keyword order.
* lisp/ox.el (org-export-get-caption):
* testing/lisp/test-org-element.el (test-org-element/affiliated-keywords-parser):
* testing/lisp/test-org-element.el (test-org-element/interpret-data):
Do not assume reverse order.
This is one of the undocumented conventions that is inconsistent with
other aspects of the parser.
* lisp/org-element.el (org-element-class):
(org-element-interpret-data): Explicitly consider anonymous syntax
node type.
* lisp/org-element.el (org-element-put-property):
(org-element-set-contents):
(org-element-secondary-p):
(org-element-adopt-elements):
(org-element-extract-element):
(org-element-insert-before):
(org-element-set-element):
(org-element-create):
(org-element-copy):
(org-element-lineage): Remove from org-element.el, using
org-element-ast function versions.
* lisp/org-element.el (org-element-headline-parser):
(org-element-inlinetask-parser):
(org-element-item-parser):
(org-element-citation-parser):
(org-element-citation-reference-parser): Assign :secondary property.
(org-element-parse-buffer):
(org-element-parse-secondary-string): Resolve deferred properties.
(org-element--cache-shift-positions): Use the new AST API.
(org-element--cache-for-removal): Use optional argument for
`org-element-set-element' to keep the needed property values.
(org-element--cache-gapless): Bump element cache version when persisting.
* testing/lisp/test-org-element.el (test-org-element/class): Update
test using the new :secondary property convention.
* lisp/ox-man.el (org-man-paragraph): Use the new AST API.
* lisp/org-element.el (org-element--cache-setup-change-functions): Set
buffer-local cache values to nil in indirect buffers.
Reported-by: Sebastian Wålinder <s.walinder@gmail.com>
Link: https://orgmode.org/list/87v8funovi.fsf@localhost
* lisp/org-element.el (org-element-cache-map): Fix when DATA is
obsolete after buffer modification.
* testing/lisp/test-org.el (test-org/map-entries): Add test.
Reported-by: Victor A. Stoichita <victor@svictor.net>
Link: https://orgmode.org/list/87v8fzxbfo.fsf@svictor.net
* lisp/org-element.el (org-element-headline-parser): Store :true-level
property.
(org-element--parse-to): Call `org-get-limited-outline-regexp'
directly rather than via `org-with-limited-levels'. Avoid moving
point unnecessarily. Prefer `org-headline-re' to generate regexps.
* lisp/org-macs.el (org-headline-re):
(org-skip-whitespace): Inline.
* lisp/org-element.el (org-element--current-element): Do not try to
search cache. This is rarely useful and forces an awkward
`org-element-copy' workaround what parsing buffer.
(org-element--parse-elements): Do not use `org-element-copy' for
element at point - it is now safe to assume that
`org-element--current-element' does not return cached.
(org-element--parse-to): Do not force disabled cache when calling
`org-element--current-element'. It is no longer needed.
* lisp/org-element.el (org-element--current-element): Try hard
spending minimal amount of time in testing which parser to call.
Prefer `look-at-p' that does not modify match data and thus save us
significant amount of time. Do not call `org-with-limited-levels' and
instead use the simplest possible tests with all unnecessary branches
removed. Add commentary explaining the importance of writing fast
core when adding new elements.
(org-inlinetask-min-level): Declare variable.
* lisp/org-list.el (org--item-re-cache):
(org-item-re): Cache results rather than calculating regexp every time
the parser is invoked.
* lisp/org-macs.el:
(org--headline-re-cache):
(org--headline-re-cache-no-bol):
(org--headline-re-cache-bol):
(org-headline-re): Prefer plists to store headline regexp cache - the
number of items tends to be within 10-20 and hence hash table is an
overkill and will be slower.
* lisp/org-macs.el (org-headline-re): New function to retrieve cached
or get a new regexp for headline of level TRUE-LEVEL.
(org-get-limited-outline-regexp): Use `org-headline-re'. Add new
optional argument WITH-BOL.
* lisp/org-element.el (org-element-headline-parser):
(org-element-inlinetask-parser):
(org-element-item-parser):
(org-element-example-block-parser):
(org-element-src-block-parser):
(org-element-link-parser):
(org-element-timestamp-parser): Do not preserve text properties in the
element property values that are strings.
* lisp/org-element.el (org-element-cache-map): Fix when we use
NEXT-RE/FAIL-RE, the initial cached guess is not right, and the
NEXT-RE hits not-yet-cached element.
* lisp/org.el (org-time-stamp-formats):
(org-timestamp-formats):
(org-time-stamp-rounding-minutes):
(org-timestamp-rounding-minutes):
(org-time-stamp-custom-formats):
(org-timestamp-custom-formats):
(org-time-stamp):
(org-timestamp):
(org-time-stamp-inactive):
(org-timestamp-inactive):
(org-insert-time-stamp):
(org-insert-timestamp):
(org-toggle-time-stamp-overlays):
(org-toggle-timestamp-overlays):
(org-time-stamp-to-now):
(org-timestamp-to-now):
* lisp/ox.el (org-export-time-stamp-file):
(org-export-timestamp-file): Rename using "timestamp" term, keeping
the old name as alias.
* doc/org-manual.org: Update all the uses, adding #+findex and
#+vindex entries. Keep the alias names for searchability.
Adjust all the callers.
The following "time-stamp" uses are unchanged:
1. `org-time-stamp-format' where obsolete and _different_ function
`org-timestamp-format' still exists.
2. :time-stamp-file property in export INFO plist. Changing this
would be breaking.
3. ORG-NEWS remains unchanged.
* lisp/org-element.el (org-element--cache-find): Do not collect
hashing statistics. According to
https://orgmode.org/list/87sfffawfe.fsf@localhost, it is quite useful
for a fraction of users, yielding up to 30% recent cache queries being
memoized. Add additional commentary. Also, slight refactoring.
(org-element--cache-hash-nocache):
(org-element--cache-hash-statistics):
(org-element-cache-hash-show-statistics): Remove.
Also, do not use [^\000] as a poor-man's replacement for
(rx (or any newline)).
* lisp/ob-core.el (org-babel-src-block-regexp):
* lisp/ob-haskell.el (org-babel-haskell-export-to-lhs):
* lisp/org-compat.el (org-hide-block-toggle-all):
* lisp/org-element.el:
* lisp/org-feed.el (org-feed-read-previous-status):
(org-feed-parse-rss-feed):
(org-feed-parse-rss-entry):
* lisp/ox-org.el (org-org-publish-to-org):
* testing/lisp/test-ob-tangle.el:
* lisp/org.el (org-block-regexp):
(org-clock-drawer-re): Use \(.\|\n\) regexp instead of [^\000].
(org-latex-regexps): Do not try to match \000 inside latex fragments -
we now use parser for this purpose.
Reported-by: Tommy Kelly <tommy.kelly@verilab.com>
Link: https://orgmode.org/list/875yfk9vlv.fsf@localhost
* doc/org-manual.org (Exporting): Add cindex entry for both "backend"
and "back-end" for better searchability.
All other changes are trivial.
Note that `org-element-export-snippet-parser' will still use :back-end
property. So will ox.el in INFO channel.
* lisp/org-element.el (org-element--cache-self-verify): Switch to nil
by default, disabling costly cache self-verification. This is
relatively safe as no relevant bug reports have been submitted for a
while. Also, the most critical cache failures are still monitored.
* lisp/org-element.el (org-element-headline-parser): Allow empty title
with tags. Do not consider space after COMMENT to be a part of title.
*
testing/lisp/test-org-element.el (test-org-element/headline-todo-keyword):
Add tests.
Reported-by: Leo Butler <Leo.Butler@umanitoba.ca>
Link: https://orgmode.org/list/87zg8t4zgo.fsf@localhost
* lisp/org-element.el (org-element-headline-parser): Allow end of line
instead of space after todo keyword.
*
testing/lisp/test-org-element.el (test-org-element/headline-todo-keyword):
Add new test.
* lisp/org-element.el (org-element-set-contents): Do alter anonymous
elements (el1 el2 ...). Such elements are used, for example, when
parsing keyword values during export, like :title.
* testing/lisp/test-org-element.el (test-org-element/set-contents):
Add test.
The patch fixed bug during export when exporting a subtree with
option stat:nil. The :title during subtree export is taken from the
heading title and parsed. However, the parsed value is stored outside
the parse tree, in :title property of the INFO channel. The parsed
value does get filtered through `org-export--prune-tree', but before
this commit, `org-element-set-contents' did not actually alter the
out-of-AST-tree parent lists of elements.
Reported-by: Leo Butler <Leo.Butler@umanitoba.ca>
Link: https://orgmode.org/list/87mt4w8epo.fsf@t14.reltub.ca
* lisp/ob-octave.el (org-babel-octave-evaluate-session):
* lisp/org-agenda.el (org-agenda-format-item):
* lisp/org-element.el (org-element-cache-map): Do not use `eq' when
matching for equality by contents.
* lisp/org-element.el: Query cache variables from the base buffer.
They are only kept up-to-date there.
* testing/lisp/test-org.el (test-org/map-entries): Add test.
Reported-by: Hanno Perrey <hanno@hoowl.se>
Link: https://orgmode.org/list/87pmau4fi3.fsf@hoowl.se
* lisp/org-element.el (org-element-cache-map): Never map over element
starting before the point manually moved (or set via
`org-element-cache-map-continue-from') by FUNC.
* testing/lisp/test-org.el (test-org/map-entries): Add tests.
Reported-by: Matthieu Caneill <matt@brokenwa.re>
Link: https://orgmode.org/list/Y9MKUzWWbErLFhmq@brokenwa.re
* lisp/org-element.el (org-element--cache-sync): Add new optional
argument FORCE to force cache synchronization even when
`org-element--cache-active-p' returns non-nil. Update commentary
explaining why `org-element--cache-active-p' needs to be checked.
(org-element--cache-submit-request): Force cache synchronization. We
are safe to force it as all the callers of
`org-element--cache-submit-request' do check that cache is active.
This fixes an edge case when Org buffer has transient indirect buffers
with `after-change-functions' not being setup. If a change happens in
the main Org buffer, `org-element--cache-after-change' gets called and
cache request is processed even though (org-element--cache-active-p),
but not (org-element--cache-active-p t), returns nil (due to the
indirect buffer).
Before this commit, `org-element--cache-submit-request' called by
`org-element--cache-after-change' missed cache synchronization, as
`org-element--cache-sync' did nothing. As a result, if the indirect
buffer would get deleted, we would end up with incorrect cache state.
Reported-by: Gregor Zattler <telegraph@gmx.net>
Ihor Radchenko
Allen Li
Ilya Chernyshov
Max Nikulin
Rudolf Adamkovič