Commit Graph

379 Commits

Author SHA1 Message Date
Ihor Radchenko fe23bec607
* lisp/org-element.el (org-element-subscript-parser): Fix edge case
Fix when both subscript and underline object match in
`org-element--object-lex'.
*
testing/lisp/test-org-element.el (test-org-element/underline-parser):
Add new test.

Reported-by: Tom Alexander <tom@fizz.buzz>
Link: https://orgmode.org/list/dad964f5-c764-4dd5-9829-ca38e3fbeb0d@app.fastmail.com
2023-10-10 15:05:38 +03:00
Ihor Radchenko 20162f5a5c
org-element-dynamic-block-open-re-nogroup: Demand block name
* lisp/org-element.el (org-element-dynamic-block-open-re-nogroup): Fix
inconsistency with `org-element-dynamic-block-open-re' - block name is
mandatory.
*
testing/lisp/test-org-element.el (test-org-element/dynamic-block-parser):
Add test.
2023-09-12 11:15:17 +03:00
Ihor Radchenko 53c9d91d3c
org-element: Fix post-blank shared between items
* lisp/org-element.el (org-element-item-parser): Do not parse beyond
LIMIT - they may extend :post-blank beyond parent list contents.
(org-element-plain-list-parser): Make sure that plain list always owns
the trailing blank lines.
(org-element-cache-version): Bump cache version.
* testing/lisp/test-org-element.el (test-org-element/item-parser): Add
test.

Reported-by: Tom Alexander <tom@fizz.buzz>
Link: https://orgmode.org/list/1c833eb8-c556-437b-ac5b-be360ebcc869@app.fastmail.com
2023-08-22 11:25:59 +03:00
Ihor Radchenko e4504853b5
Merge branch 'bugfix' 2023-08-16 11:43:12 +03:00
Ihor Radchenko 27a41d418d
org-element--cache-before-change: Fix edge case creating heading
* lisp/org-element.el (org-element--cache-before-change): Fix edge
case when a heading is created because we insert a newline right
before ****.
* testing/lisp/test-org-element.el (test-org-element/cache-headline):
Add tests.

Reported-by: Rodrigo Morales <moralesrodrigo1100@gmail.com>
Link: https://orgmode.org/list/CAGxMbPbbqc33iaqJ=EceyKrLaf4maJAxaUmJGaPOvG_Rpw+xcQ@mail.gmail.com
2023-08-16 11:41:31 +03:00
Ihor Radchenko e340dde63e
org-element-export-snippet-parser: Fix when no closing @@
* lisp/org-element.el (org-element-export-snippet-parser): Return nil
when there is no closing @@.
* testing/lisp/test-org-element.el (test-org-element/context): Add
test.

Reported-by: Tom Alexander <tom@fizz.buzz>
Link: https://orgmode.org/list/fb61ea28-f004-4c25-adf7-69fc55683ed4@app.fastmail.com
2023-08-15 11:37:37 +03:00
Ilya Chernyshov fe85d61a92
testing: Delete duplicate tests
* testing/lisp/test-ol.el (test-org-link/store-link): Delete a duplicate test.

* testing/lisp/test-org-clock.el (test-org-clock/clocktable/properties): Delete a duplicate test.

* testing/lisp/test-org-element.el (test-org-element/link-parser,
test-org-element/timestamp-parser): Delete duplicate tests.

* testing/lisp/test-org-table.el (test-org-table/get-field): Delete a duplicate test.

* testing/lisp/test-org.el (test-org/auto-fill-function): Delete a duplicate test.
2023-08-08 15:33:06 +03:00
Ilya Chernyshov c74c4ab18d
lisp/org-element.el: Add new timestamp property :range-type
The new property is needed to reduce ambiguity when interpreting the
parsed data.

* lisp/org-element.el (org-element-timestamp-interpreter): Preserve old
behavior when :range-type is `nil'.  Take into account :range-type
value when interpreting ranges.  When :range-type is `timerange',
return a timerange (<YYYY-mm-DD HH:MM-HH:MM>).  If :range-type is
`daterange' return a daterange (<...>--<...>).  When :range-type is
nil, return a daterange (as it was before).  When :range-type is
`daterange' or `timerange' and :type is `active'/`inactive', throw an
error.
(org-element-timestamp-parser): Add :range-type property.

* lisp/org.el (org-timestamp-split-range): Make sure that :range-type
is nil for a split timestamp.

* testing/lisp/test-org-element.el
(test-org-element/timestamp-interpreter): Add new tests.
(test-org-element/timestamp-parser): Add tests for :range-type
property.

* etc/ORG-NEWS (Major changes and additions to Org API): Add news about this property.

Link: https://list.orgmode.org/orgmode/87y1ot6dqz.fsf@gmail.com/
2023-07-12 11:12:46 +03:00
Ihor Radchenko 948c896448
Rename `org-element-property-1' to `org-element-property-raw'
The new name is more readable.

Link: https://orgmode.org/list/87sfail69e.fsf@localhost
2023-07-01 14:36:32 +03:00
Ihor Radchenko fcde1f779d
test-org-element/at-point: Add new test
* testing/lisp/test-org-element.el (test-org-element/at-point): Test
when point is not in current buffer.
2023-07-01 14:36:24 +03:00
Ihor Radchenko b2482d9cc4
* testing/lisp/test-org-element.el: Add new tests
(test-org-element/type):
(test-org-element/type-p):
(test-org-element/org-element-property-1):
(test-org-element/property):
(test-org-element/properties-resolve):
(test-org-element/secondary-p):
(test-org-element/class):
(test-org-element/ast-map):
(test-org-element/properties-mapc):
(test-org-element/properties-map):
(test-org-element/org-element-create):
(test-org-element/put-property):
(test-org-element/set):
(test-org-element/lineage-map): add new and extend tests according to
the new API.
2023-07-01 14:36:09 +03:00
Ihor Radchenko 81a7e8c836
org-element-lineage: Allow TYPES to be a symbol
* lisp/org-element-ast.el (org-element-lineage): Allow TYPES argument
to be a symbol.
* lisp/ox-ascii.el (org-ascii--current-text-width):
(org-ascii-format-inlinetask-default):
(org-ascii-section):
* lisp/ox-beamer.el (org-beamer--format-block):
* lisp/ox-html.el (org-html-section):
(org-html-table-cell):
(org-html-table-row):
* lisp/ox-latex.el (org-latex-table-cell):
(org-latex-table-row):
* lisp/ox-odt.el (org-odt-headline):
(org-odt-table-style-spec):
(org-odt-get-table-cell-styles):
(org-odt-table-cell):
(org-odt-table-row):
* lisp/ox-org.el (org-org-section):
* lisp/ox-publish.el (org-publish-collect-index):
* lisp/ox-texinfo.el (org-texinfo--get-node):
(org-texinfo-section):
(org-texinfo-table-row):
* lisp/ox.el (org-export--skip-p):
(org-export-get-node-property):
(org-export-table-cell-borders):
(org-export-table-row-in-header-p):
(org-export-table-row-number):
(org-export-collect-headlines): Update all the callers.
2023-07-01 14:35:43 +03:00
Ihor Radchenko a7d1dfa171
Use `org-element-type-p'
Use `org-element-type-p' across Org source, except some tests, where `eq' can
produce more meaningful failure explanations.
2023-07-01 14:35:35 +03:00
Ihor Radchenko c22697f472
Use new function names `org-element-extract' and `org-element-set' 2023-07-01 14:35:32 +03:00
Ihor Radchenko 55ba9f05a7
org-element-map: Refactor using `org-element-ast-map'
* lisp/org-element.el (org-element-map): Use `org-element-ast-map' to
traverse the AST.  Update the docstring adding new features from
`org-element-ast-map' - FUN can now throw `:org-element-skip' signal
to skip current node and its child nodes.  Clarify the structure of
INFO argument.  Reverse the traverse order for dual keywords -
secondary value is now traversed after the main value.
* testing/lisp/test-org-element.el (test-org-element/map): Update the test
reflecting the change in dual keyword traversal order.
2023-07-01 14:35:28 +03:00
Ihor Radchenko f4aa3747e1
org-element: Preserve order of multiple affiliated keywords
* lisp/org-element.el: Do not reverse affiliated keyword order.
(org-element-map): Remove special arrangement to restore keyword
ordering.
(org-element--interpret-affiliated-keywords): Obey the new keyword order.
* lisp/ox.el (org-export-get-caption):
* testing/lisp/test-org-element.el (test-org-element/affiliated-keywords-parser):
* testing/lisp/test-org-element.el (test-org-element/interpret-data):
Do not assume reverse order.

This is one of the undocumented conventions that is inconsistent with
other aspects of the parser.
2023-07-01 14:35:25 +03:00
Ihor Radchenko 924a64da39
org-element: Use the new org-element-ast library
* lisp/org-element.el (org-element-class):
(org-element-interpret-data): Explicitly consider anonymous syntax
node type.
* lisp/org-element.el (org-element-put-property):
(org-element-set-contents):
(org-element-secondary-p):
(org-element-adopt-elements):
(org-element-extract-element):
(org-element-insert-before):
(org-element-set-element):
(org-element-create):
(org-element-copy):
(org-element-lineage): Remove from org-element.el, using
org-element-ast function versions.
* lisp/org-element.el (org-element-headline-parser):
(org-element-inlinetask-parser):
(org-element-item-parser):
(org-element-citation-parser):
(org-element-citation-reference-parser): Assign :secondary property.
(org-element-parse-buffer):
(org-element-parse-secondary-string): Resolve deferred properties.
(org-element--cache-shift-positions): Use the new AST API.
(org-element--cache-for-removal): Use optional argument for
`org-element-set-element' to keep the needed property values.
(org-element--cache-gapless): Bump element cache version when persisting.
* testing/lisp/test-org-element.el (test-org-element/class): Update
test using the new :secondary property convention.
* lisp/ox-man.el (org-man-paragraph): Use the new AST API.
2023-07-01 14:35:24 +03:00
Ihor Radchenko f81ba451a7
Prefer "backend" over "back-end"
* doc/org-manual.org (Exporting): Add cindex entry for both "backend"
and "back-end" for better searchability.

All other changes are trivial.

Note that `org-element-export-snippet-parser' will still use :back-end
property.  So will ox.el in INFO channel.
2023-04-20 14:11:19 +02:00
Ihor Radchenko 20b33106cd
org-element-headline-parser: Fix empty headings with tags
* lisp/org-element.el (org-element-headline-parser): Allow empty title
with tags.  Do not consider space after COMMENT to be a part of title.
*
testing/lisp/test-org-element.el (test-org-element/headline-todo-keyword):
Add tests.

Reported-by: Leo Butler <Leo.Butler@umanitoba.ca>
Link: https://orgmode.org/list/87zg8t4zgo.fsf@localhost
2023-03-22 16:08:32 +01:00
Ihor Radchenko 93bf820384
org-element-headline-parser: Treat "* TODO" as empty title + TODO keyword
* lisp/org-element.el (org-element-headline-parser): Allow end of line
instead of space after todo keyword.
*
testing/lisp/test-org-element.el (test-org-element/headline-todo-keyword):
Add new test.
2023-03-22 15:52:24 +01:00
Ihor Radchenko 8589de94d8
Merge branch 'bugfix' 2023-03-07 14:09:48 +01:00
Rudolf Adamkovič eb6379a417
Fix LaTeX spelled as Latex
etc/ORG-NEWS (Non-floating minted listings in LaTeX export):
* lisp/ob-latex.el: (org-babel-execute:latex):
* lisp/org-element.el:
* lisp/ox-ascii.el:
* lisp/ox-html.el:
* lisp/ox-latex.el:
* lisp/ox-md.el:
* lisp/ox-odt.el:
* testing/examples/ob-maxima-test.org (LaTeX output):
* testing/lisp/test-org-element.el (test-org-element/cache): Spell LaTeX
correctly.
2023-03-07 14:08:40 +01:00
Ihor Radchenko f93cc661c6
org-element-set-contents: Do alter the anonymous parents
* lisp/org-element.el (org-element-set-contents): Do alter anonymous
elements (el1 el2 ...).  Such elements are used, for example, when
parsing keyword values during export, like :title.
* testing/lisp/test-org-element.el (test-org-element/set-contents):
Add test.

The patch fixed bug during export when exporting a subtree with
option stat:nil.  The :title during subtree export is taken from the
heading title and parsed.  However, the parsed value is stored outside
the parse tree, in :title property of the INFO channel.  The parsed
value does get filtered through `org-export--prune-tree', but before
this commit, `org-element-set-contents' did not actually alter the
out-of-AST-tree parent lists of elements.

Reported-by: Leo Butler <Leo.Butler@umanitoba.ca>
Link: https://orgmode.org/list/87mt4w8epo.fsf@t14.reltub.ca
2023-03-03 16:31:37 +01:00
Ihor Radchenko 89d13998bd
org-element-fixed-width-parser: Fix when :value is empty
* lisp/org-element.el (org-element-fixed-width-interpreter): Interpret
fixed-width element with :value "" as ":\n", not as empty string.
* testing/lisp/test-org-element.el (test-org-element/fixed-width-interpreter):
Fixed the test.

The previous test, albeit intentionally asserting "" result, breaks
reversibility of parse-interpret sequence.
2023-01-12 12:23:19 +03:00
Ihor Radchenko 9eed8567e8
testing/lisp/test-org-element.el: Uncomment working test
* testing/lisp/test-org-element.el (test-org-element/class):
Uncomment.  Whatever caused the failure in the past is not longer the
problem.  Tests are passing.
2022-11-26 16:05:19 +08:00
Ihor Radchenko 4c40fc3d4d
org: Allow spaces in #+LINK abbreviation definitions
* lisp/org.el (org-set-regexps-and-options): Allow spaces when
defining link abbreviations via #+LINK keyword.
* testing/lisp/test-org-element.el (test-org-element/link-parser): Add
a new test.
* doc/org-manual.org (Link Abbreviations): Add example demonstrating
link abbreviation with spaces.

Reported-by: Rudolf Adamkovič <salutis@me.com>
Link: https://orgmode.org/list/87zgf7zujc.fsf@localhost
2022-11-20 13:15:02 +08:00
Bruno BARBIER 98cae03b7d
lisp/org-element: Add a parameters-line property to special blocks
Add a property `:parameters' to special blocks, to store the
PARAMETERS as a string.

* lisp/org-element.el (org-element-special-block-parser): Parse
PARAMETERS and set the property `:parameters'.

(org-element-special-block-interpreter): Interpret the property
`:parameters'.

*
testing/lisp/test-org-element.el (test-org-element/special-block-parser):
Add a new test for PARAMETERS.

(test-org-element/special-block-interpreter): Add new tests for PARAMETERS.
2022-10-20 17:39:55 +08:00
Bruno BARBIER 7d1e3dc38e
org-element-special-block-interpreter: Fix when no content
* lisp/org-element.el (org-element-special-block-interpreter): Use
empty string when content is nil.

*
testing/lisp/test-org-element.el (test-org-element/special-block-interpreter):
Test the case with no content.
2022-10-20 17:39:54 +08:00
Stefan Monnier 1a5e3f931c
testing: Make all files use `lexical-binding`
Mainly, add the corresponding cookie, but also add various `require`s
so that the compiler knows which vars should be trated as dynbound.
This does not fix all the warnings, but does try to eliminate
all those about "unused" variables.  For the variables truly unused,
the patch usually adds an underscore to their name to silence the warning.

Some of the fixes affect files which already used `lexical-binding`.
Not sure why the test worked before: maybe because the tests were run
without compiling them first (which could cause some of the
missing `require`d packages to be autoloaded before we got to the
problematic code, thus hiding the problem)?

I found some suspicious code, for which I added FIXMEs.

There are also a few changes to the main files.

* lisp/org-protocol.el (org-protocol-check-filename-for-protocol):
Don't call `server-edit` if it's not yet defined.  [ Needed to get
the tests to pass. ]

* lisp/ob-core.el (org-babel-temporary-directory)
(org-babel-temporary-stable-directory): Always define (and use nil
if we don't want to create a directory for it).  Simplify the code based
on the fact that (defvar V E) only evaluates E if V is not yet `boundp`.
(org-babel-temp-file, org-babel-temp-stable-file)
(org-babel-remove-temporary-directory)
(org-babel-remove-temporary-stable-directory): Adjust accordingly.

* lisp/org.el (org-log-beginning): Add FIXME.

* testing/org-test.el: Require `org` and `org-id`.
(org-id-locations-file): Don't `defconst` it.
(org-test-at-id, org-test-in-example-file, org-test-at-marker)
(org-test-with-temp-text, org-test-with-temp-text-in-file): Move edebug
specs into `declare` (and simplify them).
(org-test-with-tramp-remote-dir--worker): Declare dynbound tramp vars.
(org--compile-when): Fix quoting of `exp`.
(org-test-load): Tweak regexps.

* testing/org-batch-test-init.el: Tweak regexp, remove dead code and
add a FIXME about it.

* testing/lisp/test-ox.el: Require `ox` instead of
erroring out if it's not already loaded.  Also require `org-inlinetask`.
(org-test-with-parsed-data): Silence warnings when `info` is not used.
(test-org-export/bind-keyword): Add FIXME.

* testing/lisp/test-ox-publish.el: Require `org-test` and `ox-publish`.
(test-org-publish/resolve-external-link): Expose lambdas to
the compiler.  Remove unused var `ids`.
(test-org-publish/get-project-from-filename): Remove unused var `file`.

* testing/lisp/test-org.el: Require `org-macs`, `org`,
`org-inlinetask`, `org-refile`, and `org-agenda`.
(test-org/org-read-date): Declare `org-time-was-given` as dynbound.
(test-org/set-regexps-and-options): Add FIXME.

* testing/lisp/test-org-timer.el: Require `org-timer`.

* testing/lisp/test-org-table.el: Require `ox`.

* testing/lisp/test-org-protocol.el: Require `org-protocol` instead of
erroring out if it's not already loaded.  Also require `capture`, and
add missing `provide` statement.

* testing/lisp/test-org-pcomplete.el: Require `org`.

* testing/lisp/test-org-list.el: Require `org-list` and `org`.

* testing/lisp/test-org-lint.el: Require `org-footnote` and `org-lint`.

* testing/lisp/test-org-footnote.el: Require `org-footnote`.

* testing/lisp/test-org-element.el: Require `org-element` instead of
erroring out if it's not already loaded.  Also require `org` and
`org-inlinetask`.

* testing/lisp/test-org-duration.el: Require `org-duration`.

* testing/lisp/test-org-datetree.el: Require `org-datetree`.

* testing/lisp/test-org-colview.el: Require `org-colview`,
`org-duration`, and `org-inlinetask`.

* testing/lisp/test-org-clock.el: Require `org-duration` and `org-clock`.

* testing/lisp/test-org-archive.el: Require `org-archive`.

* testing/lisp/test-org-agenda.el
(test-org-agenda/bulk-custom-arg-func): Add FIXME.

* testing/lisp/test-ol.el: Require `ol` and `org-id`.
(test-org-link/store-link): Declare `org-store-link-props` and add FIXME.

* testing/lisp/test-oc.el (test-org-cite/export-capability): Add FIXME.

* testing/lisp/test-ob.el: Require `ob-core`, `org-src`, `ob-ref`,
and `org-table`.
(test-ob/eval-header-argument): Rename `foo` to `test-ob--foo` and
declare it as dynbound.
(test-ob/blocks-with-spaces, test-ob/specific-colnames): Add FIXME.
(test-ob/noweb-expansions-in-cache):
Declare `noweb-expansions-in-cache-var` as dynbound.

* testing/lisp/test-ob-tangle.el: Require `org` and `ob-tangle`.

* testing/lisp/test-ob-shell.el:
* testing/lisp/test-ob-python.el: Require `ob-core`.

* testing/lisp/test-ob-lob.el: Require `ob-lob`.
(temporary-value-for-test): Declare as dynbound.

* testing/lisp/test-ob-plantuml.el: Require `ob-plantuml` instead of
erroring out if it's not already loaded.
* testing/lisp/test-ob-lilypond.el: Require `ob-lilypond` instead of
erroring out if it's not already loaded.  Use `with-current-buffer`.

* testing/lisp/test-ob-julia.el: Require `ob-core`.

* testing/lisp/test-ob-java.el (org-babel-temporary-directory):
Remove dead code now that `org-babel-temporary-directory` is always bound.

* testing/lisp/test-ob-exp.el: Require `ob-exp`, `org-src`, and `org-test`.
(ob-exp/evaluate-all-executables-in-order):
Declare `*evaluation-collector*` as dynbound.

* testing/lisp/test-ob-emacs-lisp.el (ob-emacs-lisp/dynamic-lexical-edit)
(ob-emacs-lisp/dynamic-lexical-execute):
Rename dynbound var to `ob-emacs--x` and declare it as such.

* testing/lisp/test-ob-R.el: Require `ob-core`.
(ess-ask-for-ess-directory, ess-history-file): Declare vars.
2022-09-15 19:17:19 +08:00
Stefan Kangas 41e19f1d7a
; Fix typos 2022-07-01 17:42:52 +08:00
Ihor Radchenko e5e7ae36cf
org-element--current-element: Fix #+BEGIN$ parsed as special block
* lisp/org-element.el (org-element--current-element): Use
`org-element-dynamic-block-open-re' to match blocks.

* testing/lisp/test-org-element.el (test-org-element/paragraph-parser):
Fix test relying on previous incorrect behaviour.  Empty #+BEGIN:
should be parsed as a keyword.

Reported in https://list.orgmode.org/Ycay4s3iAdEGSwgt@tuxteam.de/T/#t
2022-05-14 13:51:56 +08:00
Nicolas Goaziou ef66660cf4 Merge branch 'bugfix' 2022-02-28 15:46:06 +01:00
Nicolas Goaziou bafeaf161c element: Fix parsing of inline footnotes
* lisp/org-element.el (org-element--pair-round-table):
(org-element--pair-square-table):
(org-element--pair-curly-table): De-activate all but one pair of
parenthesis.
* testing/lisp/test-org-element.el (test-org-element/footnote-reference-parser):
Add test.

Reported-by: Greg Minshall <minshall@umich.edu>
<http://lists.gnu.org/r/emacs-orgmode/2022-02/msg00369.html>
2022-02-28 15:45:25 +01:00
Ihor Radchenko 26decec006
test-org-element-at-point: Add tests for point outside contents 2022-01-24 22:54:18 +08:00
Ihor Radchenko 846226a202
test-org-element/cache: Fix typo 2022-01-21 20:53:11 +08:00
Ihor Radchenko 9b58ead467
org-element-context: Use element cache
* lisp/org-element.el (org-element-context): Use
`org-element-at-point', which makes use of cache.
* testing/lisp/test-org-element.el (test-org-element/lineage): Expect
full lineage up to org-data from `org-element-context'.
2022-01-17 20:20:57 +08:00
Ihor Radchenko 515ce56d4e
org-element-cache: Fix transforming keywords to affiliated
* lisp/org-element.el (org-element--cache-for-removal): Consider
preceding keywords to be updated unconditionally.
(org-element-cache-map): Fix infinite loop revealed by the new test.

* testing/lisp/test-org-element.el (test-org-element/cache-affiliated):
New test.
2022-01-07 21:24:43 +08:00
Ihor Radchenko e2a8e95576
org-element-cache: Fix when edit extends previous element
* lisp/org-element.el (org-element--cache-after-change): Always extend
changed region to bol.
* testing/lisp/test-org-element.el (test-org-element/cache): Add test
checking the new fix.  Amend some tests around making sure that cache
is active during testing.
2021-12-17 20:23:50 +08:00
Ihor Radchenko d267486002
org-element-cache: Fix some edits right after indent at :begin
* lisp/org-element.el (org-element--cache-after-change): Extend
changed region to bol when we are editing near beginning of an element
within or right after indentation.  Such edits potentially change
:post-blank value of the previous element.
* testing/lisp/test-org-element.el (test-org-element/cache): Add test
checking one of such cases.
2021-12-16 21:39:25 +08:00
Ihor Radchenko f0c474e659
test-org-element: Add new tests for org-element-cache
* testing/lisp/test-org-element.el (test-org-element/cache): Add tests
for edits near :end of an element.
2021-12-16 12:03:18 +08:00
Ihor Radchenko 4bad221a60
test-org-element: Add new tests
* testing/lisp/test-org-element.el (test-org-element/cache): Add new
tests checking against intersected elements and checking that Phase 2
removes obsolete elements.
2021-12-16 11:53:01 +08:00
Ihor Radchenko 003968f94e
test-org-element: Rename cache-bugs test to cache-table
* testing/lisp/test-org-element.el (test-org-element/cache): The bug
has been fixed.  Old test name does not make sense anymore.
2021-12-16 11:52:22 +08:00
Nicolas Goaziou 26e68816bd element: Integrate some syntax constants
* lisp/org-element.el (org-element-archive-tag):
(org-element-clock-line-re):
(org-element-comment-string):
(org-element-closed-keyword):
(org-element-deadline-keyword):
(org-element-scheduled-keyword):
(org-element-planning-keywords-re):
(org-element-planning-line-re):
(org-element-drawer-re):
(org-element-dynamic-block-open-re):
(org-element-headline-re): New constants.
(org-element-drawer-parser):
(org-element-dynamic-block-parser):
(org-element--footnote-separator):
(org-element--get-node-properties):
(org-element--get-time-properties):
(org-element-headline-parser):
(org-element-headline-interpreter):
(org-element-inlinetask-parser):
(org-element--list-struct):
(org-element-paragraph-parser):
(org-element-planning-parser):
(org-element-planning-interpreter):
(org-element--current-element):
(org-element--cache-for-removal):
(org-element-cache-map):
(org-element-context): Use new constants so as to not use org.el's.
* testing/lisp/test-org-element.el (test-org-element/headline-comment-keyword):
(test-org-element/headline-archive-tag):
* testing/lisp/test-ox.el (test-org-export/handle-options): Fix tests.
2021-11-20 11:37:52 +01:00
Ihor Radchenko 60c927f8b8
Add new element parser and cache tests 2021-10-16 23:42:30 +08:00
Ihor Radchenko fc80d052db
Re-implement org-element-cache and add headline support
* lisp/org-element.el (org-element-with-disabled-cache): New macro.

(org-element-greater-elements): Add new org-data element.  It
functions like a virtual headline containing the whole buffer.  The
org-data properties are like headlie properties, but according to the
top-level drawer.  org-data's category is the buffer's category as
defined by top-level property drawer, #+CATEGORY keyworsd, and the
buffer file name.

(org-element--cache-element-properties, org-element-set-element): New
variable containing properties to be transferred when updating changed
element in cache in `org-element-set-element'.

(org-element--get-node-properties): Allow parsing node propreties in
top-level drawer when new optional argument is passed.  Respect
PROPERTY+ syntax.

(org-element--get-global-node-properties): New function.  It returns
node properties for top-level property drawer.

(org-element-org-data-parser, org-element-org-data-interpreter):
Implement the new org-data element.

(org-element-headline-parser, org-element-section-parser): Add new
:robust-begin and :robust-end
properties delimiting safe changes that do not modify headline
element.

(org-element--list-struct): Fix cache update when adding a headline
inside list.

(org-element--current-element): Implement cache support.  Record
parsing mode (:mode) and parsing granularity (:granularity) in the
element properties.

(org-element-parse-buffer, org-element--next-mode): Support new
org-data element.

(org-element--parse-elements): Record parsing granularity in the
returned tree

(org-element-use-cache): Enable cache by default.

(org-element-cache-persistent): New variable controlling cache
persistance across sessions.  Enabled by default.

(org-element--cache-self-verify,
org-element--cache-self-verify-frequency,
org-element--cache-diagnostics, org-element--cache-map-statistics,
org-element--cache-map-statistics-threshold,
org-element--cache-diagnostics-level,
org-element--cache-diagnostics-ring,
org-element--cache-diagnostics-ring-size): New variables controlling
cache diagnostics and self-diagnostics.  Greatly simplifies cache
debugging.

(org-element--cache, org-element--cache-sync-requests,
org-element--cache-sync-timer): Make cache buffer-local by default.

(org-element--headline-cache): Implement separate cache storing only
headlines and inlinetasks.

(org-element--cache-size, org-element--headline-cache-size): New
variables containing cache sizes.  This is much faster than
`avl-tree-size'.

(org-element--cache-sync-requests): Update docstring explaning the
request list structure.

(org-element--cache-sync-keys-value): New variable replacing
`org-element--cache-sync-keys' hash table.  The hash table was not
reliable because it was using elements as keys.  Upon any cached
element update/shift, the keys were invalidated making cache ordering
incorrect and breaking the cache badly.  Now, the cache keys are
stored as :org-element--cache-sync-key element property and the new
variable stores marker value indicating the current sync request
cycle.  See `org-element--cache-key' for more details.

(org-element--cache-change-tic): New variable controlling buffer
modification count that is registered in cache.  This variable allows
catching "stealth" edits.

(org-element--cache-non-modifying-commands): New variable listing
commands that will not be slown down if we fill cache on the fly.

(org-element--request-key, org-element--request-beg,
org-element--request-end, org-element--request-offset,
org-element--request-parent, org-element--request-phase): New macros.
They improve code readability (especially when using nameless-mode).

(org-element--format-element, org-element--cache-log-message,
org-element--cache-warn): New macros implementing generic logging
functionality.

(org-element--cache-key): Add section and org-data element support.
Change cache key storage from hash map to :org-element--cache-sync-key
element property + `org-element--cache-sync-keys-value'.  We use the
latter to group all the cache keys during a single cache request
sequence.  Once sync request is fully complete, the
`org-element--cache-sync-keys-value' is updated making all the old
sync keys obsolete (they will still be store as element properties).

(org-element--headline-cache-root): New function returning headline
cache root.

(org-element--cache-active-p): Prevent cache updates when
`inhibit-modification-hooks' is non-nil, unless non-nil optional
argument is provided.

(org-element--cache-find): Share cache between indirect buffers and
the base buffer.  We have to do it because after-change hooks for
indirect buffer are not called in the base buffer and vice versa.  Add
support for section and org-data elements.

(org-element--cache-put): Implement new approach for cache key
storage.  Add diagnostics.  Indicate cached elements using :cached
element property.  Support cache size calculation.

(org-element--cache-remove): Invalidate parent contents when removing
element.  Support cache size calculation.  Detect cache corruption due
to misordered elements.

(org-element--cache-shift-positions): Support :robust-begin and
:robust-end element properties.

(org-element--cache-sync): Add diagnostics.  Add detailed comments.
Prevent slowdown when large cache chunks need to be deleted forcing
O(N) complexity cutoff.  In phase 2, fix cases when next request
contains deleted cache key.  In phase 2, fix scenario when newly
inserted element intersects with existing elements in cache.  In phase
2, detect obsolete parents removed from cache.

(org-element--open-end-p): New function checking if an element can
have blank lines right after its :contents-end.

(org-element--parse-to): Do not alter match data.  Process complex
parsing mode changes correctly.  Support headlines in cache.  Support
org-data parsing.  Add detailed comments.  Add diagnostics.

(org-element--cache-sensitive-re): Make list lines sensitive.

(org-element--cache-change-warning): Update docstring.  Now, the
variable can have t, nil, and number values.  Numbers are used to
provide more details about changed headlines.

(org-element--cache-before-change, org-element--cache-after-change):
Handle headline hierarchy.  Properly handle cache in indirect
buffers.

(org-element--cache-after-change): Update docstring clarifying the
return values.  Add special handling for headline and org-data
elements updating them in-place instead of removing together with the
whole contents when possible.  Use :robust-begin/:robust-end element
properties to detect robust changes.

(org-element--cache-submit-request): Add detailed comments.  Correctly
handle cache in indirect buffers.  Delegate element modifications to
`org-element--cache-for-removal'.

(org-element--cache-verify-element): New function for cache
self-verification.

(org-element--cache-persist-before-write,
org-element--cache-persist-before-read,
org-element--cache-persist-after-read): Implement cache persistance.

(org-element-cache-reset): Correctly handle cache in indirect
buffers.  Support cache persistance.  Support new cache size
calculation and new cache key schema.

(org-element-cache-map): New function analagous to `org-element-map',
but much faster.  The function overperforms org-ql written by Adam
Porter aka alphapapa [1] and reuses some ideas from there (namely,
fast element skipping via regexps).

[1] https://github.com/alphapapa/org-ql/

(org-element-at-point): The returned elements are now guaranteed to
have correct parents up to org-data.  New optional argument
CACHED-ONLY limits element search to current cache---if element is not
in cache and current command is not in cache
`org-element--cache-non-modifying-commands', the cache is not updated
and the function returns nil.  Also, support cache verification.

(org-element-at-point-no-context): New function.  It is analogous of
older `org-element-at-point' with no guarantee that :parent properties
are correct beyond direct parent heading.  This function does not
update cache and can be useful when cache updates should be avoided
for performance reasons.

* lisp/ob-core.el (org-babel-where-is-src-block-result): Support
section and org-data elements in cache.

* lisp/org-macro.el (org-macro-replace-all,
org-macro--find-keyword-value): Support org-element-cache.

* lisp/org-table.el (orgtbl-to-generic): Support org-element-cache.

* lisp/org.el (org-mode): Add cache persistance.

(org-up-element): Preserve old behaviour when error is returned for
section and org-data element.

*
testing/lisp/test-org-archive.el (test-org-archive/update-status-cookie):
Fix test when cache is active.

* testing/lisp/test-org-colview.el (test-org-colview/columns-update):
Fix test.

* testing/lisp/test-org-element.el (test-org-element/extract-element):
Add suport for new org-data element.

* testing/lisp/test-org-element.el (test-org-element/parent-property):
Fix equality check.  Parents returned by cache and `org-element-map'
may not be `eq' now.  Just `equal'.

* testing/lisp/test-org-element.el (test-org-element/context): Support
section and headline parents.
2021-10-16 23:22:18 +08:00
Nicolas Goaziou af0fcb6833 element: Allow citations in table cells
* lisp/org-element.el (org-element-object-restrictions): Allow
citations in table cells.
* testing/lisp/test-org-element.el (test-org-element/citation-parser):
Add test.  Remove outdated test.
2021-07-28 10:32:29 +02:00
Nicolas Goaziou fed07be5b8 element: Add citation support
* lisp/org-element.el (org-element-citation-key-re):
(org-element-citation-prefix-re): New variables.
(org-element--set-regexps): Set `org-element--object-regexp' so it
finds citations.
(org-element-all-objects): Add citation and citation-reference
objects.
(org-element-recursive-objects): Add citation object.
(org-element-object-restrictions): Add citation and citation-reference
to restrictions.
(org-element-secondary-value-alist): citation and citation references
can hold secondary strings.
(org-element-citation-parser):
(org-element-citation-interpreter):
(org-element-citation-reference-parser):
(org-element-citation-reference-interpreter): New functions.
(org-element--object-lex): Parse citations and citations references.
* testing/lisp/test-org-element.el (test-org-element/citation-parser):
(test-org-element/citation-reference-parser):
(test-org-element/citation-interpreter): New tests.

This patch adds support for [cite:@key], [cite:pre @key post]
[cite:global prefix; pre @key1 post; pre @key2 post; global suffix]
objects along with their [cite/style: ...] counterparts.
2021-07-09 08:47:14 +02:00
Bastien 4362b93018 Merge branch 'maint' 2021-05-16 10:36:42 +02:00
Nicolas Goaziou 8846ea71fa element: Fix drawer parsing
* lisp/org-element.el (org-element-drawer-parser): A single ":END:"
line should not be recognized as a drawer.
* testing/lisp/test-org-element.el (test-org-element/drawer-parser):
Add test.

Reported-by: Ihor Radchenko <yantar92@gmail.com>
<http://lists.gnu.org/r/emacs-orgmode/2021-05/msg00819.html>
2021-05-16 10:14:54 +02:00
Ihor Radchenko 3f59ea981c Add a test for known bug in org-element-cache
* testing/lisp/test-org-element.el (test-org-element/cache-bugs): A
new test documenting a known bug:

1. Create initial file like:

P0

  P1
  | a | b |
<point>| c | d |

Note that second line of the table is not indented.  Now,
org-element-at-point returns table-row.

2. Modify the file to test org-element-cache

P0

- item

  P1
  | a | b |
<point>| c | d |

Now, the first (indented) row of the table belongs to item.  The
second row should be an individual table and org-element-at-point
returns table.
2021-05-15 16:13:49 +02:00