* lisp/ob-python.el (org-babel-python--eval-ast): Removed.
(org-babel-python-format-session-value): New function that returns
Python code to evaluate for sessions with value results.
(org-babel-python-evaluate-session): Replace
org-babel-python--eval-ast with org-babel-python-format-session-value.
Motivation is that the new function is more flexible than the old
format string. We can pass in the full result-params, which can be
used to add more formatting options in future. We could also add more
optional arguments in the future to extend this functionality.
This function will be particularly helpful for a couple patches after
9.4:
- ob-reticulate
- https://orgmode.org/list/875z98gj4f.fsf@gmail.com/
- Improved formatting
- https://orgmode.org/list/87eenpfe77.fsf@gmail.com/
However, I add this function in now 9.4, so those future patches can
rely on it in 9.4. In particular, ob-reticulate will probably be
packaged separately from org-mode, and I want to provide a stable
interface to it so that it can be released after 9.4.
* lisp/org-faces.el (org-block): Tiny docstring enhancement.
* lisp/org.el (org-fontify-meta-lines-and-blocks-1): Use the
`org-block' face for every true blocks.
Reported-by: Sébastien Miquel <sebastien.miquel@posteo.eu>
https://orgmode.org/list/42749c98-ddfe-bad4-43a5-1119e24972bd@posteo.eu
* lisp/org.el (org-in-archived-heading-p): New function.
* lisp/ob-exp.el (org-babel-exp-process-buffer):
* lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Use
`org-in-archived-heading-p' to skip archived headings when tangling
and exporting.
* testing/lisp/test-org.el (test-org/in-archived-heading-p): Add
test for `org-in-archived-heading-p'.
Reported-by: flare <gabrielxaviersmith@gmail.com>
See https://orgmode.org/list/877dt9ey2c.fsf@gmail.com/
* lisp/ob-python.el (py-send-string-no-output): Remove external function.
(py-shell-send-string): Add external function.
(org-babel-python--eval-ast): Remove blank lines.
(org-babel-python--send-string): New function to send string to Python
process and return output.
(org-babel-python-evaluate-session): Call
org-babel-python--send-string to evaluate code.
New function to send code to Python session. It has the following
desirable properties:
- Wait for evaluation to finish before returning.
- Input can have arbitrary spaces/newlines.
- Avoid echoing input to the session.
- Echo all output to the session.
- Output is also captured and returned separately.
- Avoid adding extra prompts to the session.
- Work well with both Python and IPython repls.
The implementation borrows from `python-shell-send-string-no-output'
and `org-babel-comint-with-output'. This commit is related to
632ceabb1, which tried to implement the same thing, but also
introduced bugs, and had to be partially reverted in b5709ddc9.
* lisp/ob-python.el (org-babel-python-initiate-session-by-key): Remove
workaround for emacs<24.1. Sleep after starting Python, to prevent
startup messages leaking into output results.
* testing/lisp/test-ob-python.el (test-ob-python/session-multiline):
Remove workaround for startup message leaking into results.
* lisp/ob-shell.el (org-babel-execute:shell)
(org-babel-sh-evaluate): Use the exit code as the value and,
consequently, as the result when no :results header is set.
See this thread:
https://orgmode.org/list/CA+A2iZaziAfMeGpBqL6qGrzrWEVvLvC0DUw++T4gCF3NGuW-DQ@mail.gmail.com/
And Tim's useful summary here:
https://orgmode.org/list/87sgiszqc1.fsf@gmail.com/
This commit resolves the issue by deciding not to make an exception
for ob-shell.el:
- the "value" of executing a shell/bash script is the exit code
- when no :results header is set, use the value as the result
- use :results output to use the stdout
* lisp/ob-core.el (org-babel--expand-body): New function.
(org-babel-check-confirm-evaluate):
(org-babel-execute-src-block): Use org-babel--expand-body.
As of 727c6d7fe (lisp/ob-core.el: org-babel-check-confirm-evaluate
strip coderefs, 2020-09-04), org-babel-check-confirm-evaluate and
org-babel-execute-src-block use the same logic to expand the body and
strip coderefs. Move that code to a helper function.
This patch is functionally equivalent to the patch originally proposed
by Tom Gillespie at
https://orgmode.org/list/CA+G3_PNi3uMvBiWgBdKuC3C6VJt1T1j-RKH43LRqYbr+4NS8ZA@mail.gmail.com
* lisp/ob-core.el (org-babel-check-confirm-evaluate): strip coderefs
before passing the body of the block to org-confirm-babel-evaluate
using the same let block as is used in org-babel-execute-src-block.
* lisp/org.el (org-read-date-display-live): Document that live
display is only available when `org-read-date-popup-calendar' is
non-nil.
See <https://orgmod.org/list/20200630180259.zj3krk57x5iqmbsj@orion>
Reported-by: Edmund Christian Herenz <eherenz@eso.org>
* lisp/ob-core.el (org-babel-result-to-file): Use
`buffer-base-buffer' so that `buffer-file-name' returns correct
results in indirect buffers too.
See <https://orgmode.org/list/87eeos3w7v.fsf@localhost>
TINYCHANGE
* lisp/org-capture.el (org-capture-finalize): Update
`org-capture-plist' with local-value before finalizing.
We use the global-variable `org-capture-plist' to populate the
local-variable `org-capture-current-plist' on the init of the
`org-capture' buffer. However, we do not do the opposite (i.e. update
the global-variable with the local-variable) on
`org-capture-finalize'.
This is fine for the majority of `org-capture-finalize', since we’re
using the LOCAL arg of `org-capture-get' to read
`org-capture-current-plist' instead of `org-capture-list', but this
trick does not work for `org-capture-after-finalize', since the hook
is run after the `org-capture-buffer' has been closed.
This causes problem with `:kill-buffer t', and it limits what can be
done with cleanup functions in `org-capture-after-finalize'.
See <https://orgmode.org/list/87h7tv9pkm.fsf@hidden/> for details.
* lisp/org.el (org-make-tags-matcher): Add a parameter to make it
optional to include non-buffer tags.
(org-match-sparse-tree): Use the new parameter.
TINYCHANGE
* lisp/ox-odt.el (org-odt--translate-latex-fragments):
* lisp/ox-html.el (org-html--format-image): Use
`org-preview-latex-image-directory' instead of hardcoding the
directory name.
See <https://orgmode.org/list/87a70v9jan.fsf@ucl.ac.uk>
Reported-by: Eric S Fraga <e.fraga@ucl.ac.uk>
* lisp/org-goto.el (org-goto-location): Call make-indirect-buffer with
a non-nil CLONE to preserve the base buffer's state, avoiding a more
expensive call to org-mode.
Reported-by: Vladimir Nikishkin <lockywolf@gmail.com>
Suggested-by: Ihor Radchenko <yantar92@gmail.com>
* lisp/ob-python.el (py-shell-send-string): Remove unneeded
declare-function.
(org-babel-python-eoe-indicator): Add back this const.
(org-babel-python--eval-ast): Fix spacing so the block can be entered
line by line into the interpreter.
(org-babel-python-evaluate-session): Revert previous changes to
evaluation for value results.
* testing/lisp/test-ob-python.el (test-ob-python/session-value-sleep):
New test for session blocks that don't return immediately.
632ceabb1 introduced a bug where session blocks don't wait for value
results if not returned immediately. This reverts part of that
commit, specifically for evaluation of session blocks with value
results. It leaves other parts of that commit alone, for example the
changes to session output results, which is more robust handling
prompts from ipython interpreter.
A downside of reverting the session value implementation is that the
ugly code from org-babel-python--eval-ast is again printed in the
console, but this is less bad than the new bug that was introduced by
the change, hence reverted.
* lisp/ol.el (org-store-link): Store link with CUSTOM_ID anchor
in indirect buffer.
(org-insert-link): Do not add file name to search and CUSTOM_ID
links pointed to the same file when called from indirect buffer.
When a subtree was open in an indirect buffer, wrong argument
of `abbreviate-file-name' prevented storing link to a CUSTOM_ID
anchor. Internal link to a header line or to a CUSTOM_ID anchor
was created with file name instead of concise form.
This is a follow up of 784e5f1488.
TINYCHANGE
* lisp/ob-plantuml (org-babel-variable-assignments:plantuml): Support
using plantuml executable instead of jar.
Some systems come with an executable for plantuml instead of a specific
JAR file. This adds support for two different modes:
- jar :: using java together with a JAR (previous behavior)
- plantuml :: using a PlantUML executable
The PlantUML executable can be configured via
`org-plantuml-executable-path` and also the arguments that will be given
via `org-plantuml-executable-args`.
* lisp/ob-J.el (org-babel-execute:J, org-babel-J-eval-string): Add
customizability.
(org-babel-execute:J): Lookup optional parameter `:sit' to allow one
to wait for a specified amount of time before grabbing the output of
the J subprocess. Pass this specified value or the previous default
of .1 to `org-babel-J-eval-string'.
(org-babel-eval-string): Pass new argument `sit-time' to `sit-for'
before grabbing output.
The problem is that we read the contents of the output after 0.1
seconds, which, for expensive computations, results in the mangling of
output. Output from expensive computations gets propagated down to
subsequent code-blocks' outputs, producing a horrible mess.
TINYCHANGE
* lisp/ob-core.el (org-babel-import-elisp-from-file): Don't try to
convert empty file to a table.
* testing/lisp/test-ob.el (test-ob/import-elisp-from-file): Add tests.
If org-babel-import-elisp-from-file is called with an empty file
(which many ob- libraries do when there are no results), feeding that
to org-table-import leads to a beginning-of-buffer error. Before
14878f3f9 (ob-core: Display warning on failure to read results,
2020-05-21), this error was hard to notice because, after catching it,
it was reported as a quickly buried message. After that commit, it is
displayed as a warning, which is not appropriate for the common and
unproblematic case of empty results.
Avoid the warning by only doing the table conversion if the file has
content. Another option would be to do the table conversion but add a
beginning-of-buffer arm to the surrounding condition-case. However,
that risks swallowing other sources of that error.
Reported-by: Colin Baxter <m43cap@yandex.com>
<https://orgmode.org/list/878se3nhbj.fsf@yandex.com>
* lisp/ob-python.el: Require python.el at top-level.
(org-babel-python-eoe-indicator): Remove unused variable.
(org-babel-python-initiate-session-by-key): Rename
python-buffer to avoid obsolete warning.
(org-babel-python-evaluate-external-process): Remove unnecessary
require.
(org-babel-python--exec-tmpfile): Simplify this template.
(org-babel-python--eval-ast): Add printing of results to this
template, requiring additional string escapes.
(org-babel-python-evaluate-session): Simplify to use adjusted
templates, and call out to functionality in python.el or
python-mode.el.
(org-babel-python-evaluate-session): Simplified to use adjusted templates.
This commit refactors and cleans up code related to session
evaluation. python.el is now required at the top-level. Python
templates for wrapping code are simplified. Instead of directly
pasting code to the REPL, functionality from python.el and
python-mode.el are used; this fixes issues with code being echoed to
the REPL, and should be generally more robust. Finally, in the
:results value case, special handling of exceptions is removed, and we
no longer print None when the last statement isn't an expression.
* lisp/ob-python.el (org-babel-python-evaluate-session): Fix
discrepancy between how single-line and multiline code blocks return
output.
For example, before this commit, the 2 blocks behaved differently:
+begin_src python :session :results output
pass
"foo"
+end_src
+RESULTS:
+begin_src python :session :results output
"foo"
+end_src
+RESULTS:
: 'foo'
But now, they both behave as the former.
* lisp/org-num.el (org-num--skip-value): Handle empty headlines.
This happens when org-num-skip-commented is non-nil. When creating
a new headline, often org-num--skip-value is invoked before any
headline text is created. This means that ‘title‘ is nil and the
string-match check breaks. Checking if title is non-nil fixes this.
TINYCHANGE
* lisp/org-clock.el (org-clock-sum-current-item): Do not include
clocked time from the parent when inside inlinetask.
Calling `org-narrow-to-subtree' from inside inlinetask would narrow to
the parent's subtree (correct behaviour). However, it is not what we
want when calculating the clocking sum. Inlinetask case should be
treated specially.
This fixes a bug in set-tags-command excluding a tag that is both set
locally and inherited from the initial minibuffer input by modifying
org-get-tags to prefer keeping the locally set tag over the inherited
tag, as this behavior is more intuitive for org-get-tags anyway.
* lisp/org.el (org-get-tags): Keep local tags over inherited.
* testing/lisp/test-org.el (test-org/set-tags-command): Add test.
* lisp/ob-python.el (org-babel-python-evaluate-session): Process
temporary file name with `org-babel-process-file-name' before
inserting it into code Python code snippets.
Before this change, the entire temporary filename was sent to the
Python session for execution, causing a 'No such file' error when the
filename had a Tramp format such as
/ssh:user@server:/tmp/python-ABCDEF.
TINYCHANGE
* lisp/org.el (org-get-cursor-date): Fix regular expression.
Previous regular expression assumed the time grid string will have two
digits in the hour portion of the time string. However, the time grid
string does not always have two digits. For example:
" 8:00......"
* lisp/org-faces.el (org-block): Set background extension beyond
end-of-line.
* lisp/org-compat.el (org--set-faces-extend): New function to
temporarily (re)set :extend for Emacs≥27.
* lisp/org.el (org-mode): Call it to set the extend attribute of
relevant faces to the correct value.
* ob-core.el (org-babel-execute-src-block): Source code block header
argument `:file-mode' can set file permissions if `:file' argument is
provided.
(org-babel-common-header-args-w-values): Add `:file-mode' to common
header arguments.
TINYCHANGE
* lisp/ob-core.el (org-babel-check-confirm-evaluate): Expand noweb
references in the body passed to org-confirm-babel-evaluate.
* testing/lisp/test-ob.el (test-ob/check-eval-noweb-expanded): New
test.
When noweb references are set to be expanded for evaluation,
org-confirm-babel-evaluate should receive the expanded body so that it
can make its decision based on what actually will be evaluated
(without resorting to calling org-babel-get-src-block-info itself).
Note that this results in up to three calls to
org-babel-expand-noweb-references. Possible ways to avoid this are
discussed at the thread referenced below.
Reported-by: Tom Gillespie <tgbugs@gmail.com>
https://orgmode.org/list/CA+G3_PNi3uMvBiWgBdKuC3C6VJt1T1j-RKH43LRqYbr+4NS8ZA@mail.gmail.com/
* lisp/ob-screen.el (org-babel-screen-session-socketname): Quote
session name before giving it to string-match because the name may
have regexp characters.
While touching the line, make two cosmetic tweaks.
* lisp/ob-screen.el (org-babel-default-header-args:screen): Add
default header argument `:screenrc' and value "/dev/null".
(org-babel-prep-session:screen): Use header argument in variable
`screenrc' and not hard-coded value.
* lisp/ob-screen.el (org-babel-prep-session:screen): Remove concat of
"org-babel-session-" string onto session name when creating session.
(org-babel-screen-session-socketname): Remove concat of
"org-babel-session-" string onto session name searching for existing
screen session.
* lisp/org.el (org-fill-paragraph): Do not fill paragraph before
region.
* testing/lisp/test-org.el (test-org/fill-paragraph): Add test.
Reported-by: Matt Lundin <mdl@imapmail.org>
<http://lists.gnu.org/r/emacs-orgmode/2020-07/msg00164.html>
* lisp/org-agenda.el (org-agenda-filter-by-effort):
(org-agenda-filter-completion-function): Ignore case when querying
effort property key in org-global-properties since property keys are
documented as case-insensitive.
* lisp/org.el (org-buffer-property-keys): Enhance the completion list
with property names from #+PROPERTY keywords, not just property
drawers. Also, for each xxx_ALL property, make sure that the bare xxx
property is added too.
* testing/lisp/test-org.el (test-org/buffer-property-keys): Add test
cases for #+PROPERTY keywords and also for xxx_ALL --> xxx properties.
See https://emacs.stackexchange.com/questions/59448/ for details.
Modified-by: Kyle Meyer <kyle@kyleam.com>
Cosmetic tweaks to _ALL stripping.
* lisp/org.el (org-ctrl-c-tab): Don't move point when acting on
outline level 0 (which is before first heading). This is consistent to
the behavior for level > 0. And refactor some. And correct the
docstring.
* lisp/org.el (org-show-entry): Before first heading show the region
up to the first heading. Remove the now useless ignore-errors.
This behavior allows to reveal invisible text before the first
heading.
* lisp/org-id.el (org-id-update-id-locations): When
`org-id-extra-files' is a symbol, it should be evaluated as a
variable.
In 9865e6bd8be65229be4eac4f459f62e47fab2be737a5020bb,
`org-id-update-id-locations' was rewritten, and the functionality to
allow the variable `org-id-extra-files' to be a symbol that references
another variable was removed. This change restores that
functionality.
* lisp/org-agenda.el (org-agenda-finalize): Call the hooks after the
save-excursion.
This opens the way for hooks to position the cursor after agenda
generation.
* lisp/org-macs.el (org-save-outline-visibility):
(org-invisible-p):
* lisp/org.el (org-show-set-visibility):
(org--forward-paragraph-once):
(org--backward-paragraph-once): Remove reference to `org-hide-drawer',
no longer used to hide drawers.
* lisp/org-src.el (org-src--edit-element): Clear undo information once
the initial contents have been inserted.
* testing/lisp/test-org-src.el (test-org-src/undo): New test.
* lisp/org-table.el (org-table-collapse-header): New function.
* lisp/org-plot.el (org-plot/gnuplot): Use org-table-collapse-header
and trust there will be no more leading `hline' symbols in lisp table.
* testing/lisp/test-org-table.el (test-org-table/to-lisp):
Adding tests to already existing to-lisp function.
(test-org-table/collapse-header): Adding tests to new
collapse-header function.
* testing/lisp/test-ox.el (test-org-export/has-header-p): Testing
exporting table with multi-line header.
* lisp/org.el (org-todo): Use nil instead of
org-loop-over-headlines-in-active-region as the MATCH argument.
org-loop-over-headlines-in-active-region isn't related to the MATCH
argument; it's related to SCOPE and decides the value of the let-bound
cl variable that's passed as SCOPE. Note, though, that
org-loop-over-headlines-in-active-region is let-bound to nil around
the org-map-entries call, so this didn't cause any issues.
* lisp/ox.el (org-export--dispatch-ui): Restore buffer position after
inserting text to avoid auto-scrolling at start.
When filling in the dispatch buffer, point ends up at the end of the
buffer, which can cause the buffer to be re-centered on point when
org-fit-window-to-buffer is called, in particular if scroll-margin is
above zero and there is a vertical split. Avoid this by putting point
where it was before the buffer was refreshed.
Reported-by: Gustavo Barros <gusbrs.2016@gmail.com>
https://orgmode.org/list/87tv3a56vv.fsf@gmail.com
* lisp/org-habit.el (org-habit-build-graph): Check if on the very
first "done" and set the standard color for done.
Recent behavior: Give that very first done the color to mean "done
ahead of schedule".
* lisp/ox-html.el (html): Remove `footnote-definition' translator
since 1) it does not exist and 2) footnote definitions are translated
as part of of the inner template.
* lisp/org-table.el (org-table-align): Try to align the table even
when it only consists of horizontal rules.
* testing/lisp/test-org-table.el (test-org-table/align): Add test.
* lisp/org.el (org-fontify-meta-lines-and-blocks-1): Rewrite the "line
begins with *" check so that it does not assume that match data has
remained intact.
The upstream call to org-src-font-lock-fontify-block overwrites the
match data in some cases. As a result, (match-string 1) signals an
error and #+end_src is not fontified. Wrapping the call to
org-src-font-lock-fontify-block with save-match-data would fix the
reported issue. However, the position at the beginning of the line is
already stored in a variable, so use that information instead to check
whether the line starts with "*".
Reported-by: John Ciolfi <ciolfi@mathworks.com>
<r9r1ulpctz.fsf@ah-ciolfi-lah-ciolfi-l.dhcp.mathworks.com>
* lisp/org-capture.el (org-capture-place-entry): Prevent breaking the
following headline inside the capture buffer. This should match the
behavior from 9.3.
(org-capture-finalize): Reverts cb2774d1a, which solves a similar
problem but only in the finalize stage, so the subtree structure would
still be broken in the middle of editing the capture.
* lisp/org-lint.el (org-lint-deprecated-header-syntax): DIR property
is also meaningful in attachments, so don't warn about header-args
prefix.
Reported-by: Gustavo Barros <gusbrs.2016@gmail.com>
<http://lists.gnu.org/r/emacs-orgmode/2020-06/msg00270.html>
* lisp/org.el (org-startup-numerated): Fix "numeroted" typo in
variable name.
(org-startup-options):
(org-mode):
* doc/org-manual.org (Dynamic Headline Numbering):
(Summary of In-Buffer Settings): Update for rename.
Also fix a few related typos in ORG-NEWS and the manual.
No compatibility alias has been added, as this variable hasn't yet
made it into a release.
* org-capture.el (org-capture-place-template): Allow
`org-capture-current-plist' access during `org-capture-mode-hook'
Ensure consistency between org-capture's hooks.
`org-capture-after-finalize-hook' is now the only hook that cannot
access `org-capture-current-plist' because the capture buffer is
killed when it is run.
* lisp/org-element.el (org-element-normalize-contents): Update comment
The comment mentions that caching for `org-elements' is enabled by
default, but this isn’t the case anymore since
bbdecd1e64, cf. `org-element-use-cache'.
* lisp/ob-python.el (org-babel-python-evaluate-external-process): Use
functions from python.el to indent lines, avoiding multiline strings.
* testing/lisp/test-ob-python.el (test-ob-python/multiline-var): Set
test as expected to succeed.
(test-ob-python/multiline-str): Add test for multiline string in body.
(test-ob-python/header-var-assignment): Test that :var is in correct
scope and can be assigned to.
cf. https://orgmode.org/list/87tv009l9a.fsf@gmail.com/#t
* lisp/ob-tangle.el (org-babel-tangle-single-block): Fix regression
from v9.0.2 that resulted in org-babel-tangle-use-relative-file-links
being ignored.
* testing/lisp/test-ob-tangle.el
(ob-tangle/comment-links-relative-file): Add test.
As of 7b148e2d0 (ob-tangle: Respect buffer local variables,
2016-12-10), the condition in org-babel-tangle-single-block that
checks whether to return a file link with a relative target always
returns nil because the full org-link-types-re match, including the
final colon, is compared with "file" (no colon). Also, were this
condition to succeed, "file:" would be used as the file name.
Adjust the condition and fix the extracted file name.
Reported-by: Jeremias Gonzalez <jgonzalez49@ucmerced.edu>
https://orgmode.org/list/6fb12326-52d3-7177-eff5-62603261b388@ucmerced.edu
Jack Kamm
Bastien
Trevor Murphy
Kyle Meyer
Tom Gillespie
Nicholas Vollmer
Ihor Radchenko
Leo Vivier
E.L.K
nilninull
Anthony Carrico
Dmitrii Korobeinikov
Bastien Guerry
Max Nikulin
Terje Larsen
Joseph Novakovich
Eric S Fraga
Anders Johansson
Allen Li
Christian Vanderwall
Benson Chu
Nicolas Goaziou
Kévin Le Gouguec
John Herrlin
Bassam Saeed
Leo Vivier
Marco Wahl
Kenneth D. Mankoff
Alexandru-Sergiu Marton
Nick Dokos
Albert Krewinkel
Erik Hetzner
mfrasca
Kevin Liu
Lockywolf