diff --git a/doc/images/dirs.png b/doc/images/dirs.png new file mode 100644 index 000000000..8f272423f Binary files /dev/null and b/doc/images/dirs.png differ diff --git a/lisp/org-babel.el b/lisp/org-babel.el index 35c2d0568..63a7f75c7 100644 --- a/lisp/org-babel.el +++ b/lisp/org-babel.el @@ -98,7 +98,7 @@ then run `org-babel-pop-to-session'." (defun org-babel-set-interpreters (var value) (set-default var value) (setq org-babel-src-block-regexp - (concat "#\\+begin_src \\(" + (concat "^[ \t]*#\\+begin_src \\(" (mapconcat 'regexp-quote value "\\|") "\\)[ \t]*" "\\([ \t]+\\([^\n]+\\)\\)?\n" ;; match header arguments @@ -331,9 +331,14 @@ may be specified in the properties of the current outline entry." (defun org-babel-parse-src-block-match () (let* ((lang (org-babel-clean-text-properties (match-string 1))) - (lang-headers (intern (concat "org-babel-default-header-args:" lang)))) + (lang-headers (intern (concat "org-babel-default-header-args:" lang))) + (body (org-babel-clean-text-properties (match-string 4)))) (list lang - (org-babel-strip-protective-commas (org-babel-clean-text-properties (match-string 4))) + (with-temp-buffer ;; get src block body removing properties, protective commas, and indentation + (save-match-data + (insert (org-babel-strip-protective-commas body)) + (org-do-remove-indentation) + (buffer-string))) (org-babel-merge-params org-babel-default-header-args (org-babel-params-from-properties) diff --git a/org-babel-worg.org b/org-babel-worg.org index c8646ad2f..43af3e1ce 100644 --- a/org-babel-worg.org +++ b/org-babel-worg.org @@ -14,16 +14,20 @@ #+end_html * Introduction + :PROPERTIES: + :CUSTOM_ID: introduction + :END: Org-babel provides the following modifications to [[http://orgmode.org/manual/Literal-examples.html][the existing support]] for blocks of source code examples in the org-mode core. 1. source code execution @@ -31,7 +35,15 @@ 3. exportation of source code blocks to files (literate programming) * Getting started - Add the following lines to your .emacs, replacing the path as + :PROPERTIES: + :CUSTOM_ID: getting-started + :END: + Grab the latest code from the git repo at [[http://github.com/eschulte/org-babel/tree/master][github/org-babel]] +#+begin_src sh +git clone git://github.com/eschulte/org-babel.git +#+end_src + + And add the following lines to your .emacs, replacing the path as appropriate. A good place to check that things are up and running would the examples in [[* Basic org-babel functionality][Basic org-babel functionality]]. #+begin_src emacs-lisp @@ -40,36 +52,44 @@ #+end_src * Basic org-babel functionality + :PROPERTIES: + :CUSTOM_ID: basic-functionality + :END: *** Source code execution + :PROPERTIES: + :CUSTOM_ID: source-code-execution + :END: For interpreted languages such as shell, python, R, etc, org-babel allows source blocks to be executed: the code is passed to the interpreter and you have control over what is done with the results of excecution. E.g. place point anywhere in the following block and use C-c C-c to run the code: -#+begin_src python :results output -import time -x = 4 -print("hello\n") -#print time.ctime() -print [5, 10] +[[http://www.ruby-lang.org/][Ruby]] source code +#+begin_src ruby +"This file was last evaluated on #{Date.today}" #+end_src +Results of Ruby evaluation #+resname: -: hello -: 510 +: This file was last evaluated on 2009-08-09 +[[http://www.r-project.org/][R]] source code #+begin_src R :results value x = 4 date() c(5, 10) #+end_src +Results of R evaluation #+resname: | 5 | | 10 | *** What happens to the results? + :PROPERTIES: + :CUSTOM_ID: results + :END: Org-babel provides two fundamentally different modes for capturing the results of code evaluation, specified by the :results header argument: @@ -91,13 +111,167 @@ c(5, 10) **** Additional :results settings *** Arguments to source code blocks + :PROPERTIES: + :CUSTOM_ID: arguments-to-source-code-blocks + :END: In addition to evaluation of code blocks, org-babel allows them to be parameterised (i.e. have arguments). Thus source code blocks now have the status of *functions*. +Inputs for fibonacci-seq + +#+tblname: fibonacci-inputs +| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | +| 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 | + +in the Org-mode buffer this looks like +: #+tblname: fibonacci-inputs +: | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | +: | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 | + +[[http://www.gnu.org/software/emacs/manual/elisp.html][Emacs Lisp]] source code +#+srcname: fibonacci-seq +#+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs + (defun fibonacci (n) + (if (or (= n 0) (= n 1)) + n + (+ (fibonacci (- n 1)) (fibonacci (- n 2))))) + + (mapcar (lambda (row) + (mapcar #'fibonacci row)) fib-inputs) +#+end_src + +in the Org-mode buffer this looks like +: #+srcname: fibonacci-seq +: #+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs +: (defun fibonacci (n) +: (if (or (= n 0) (= n 1)) +: n +: (+ (fibonacci (- n 1)) (fibonacci (- n 2))))) +: +: (mapcar (lambda (row) +: (mapcar #'fibonacci row)) fib-inputs) +: #+end_src + +Results of Emacs Lisp code evaluation +#+resname: +| 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 | +| 1 | 3 | 8 | 21 | 55 | 144 | 377 | 987 | 2584 | 6765 | + * A meta-programming language for org-mode + :PROPERTIES: + :CUSTOM_ID: meta-programming-language + :END: + +Since information can pass freely between source-code blocks and +org-mode tables you can mix and match languages using each language +for those tasks to which it is suited. This makes Org-mode files with +Org-babel into a kind of meta-functional programming language in which +functions from many languages can work together. + +As an example, lets take some system diagnostics in the shell, and +then graph them with R. + +1. Shell source code +#+srcname: directories + #+begin_src bash :results replace + cd ~ && du -sc * |grep -v total + #+end_src +2. Results of the shell source code (on my system, grab this org-mode + files and try running it on your own) +#+resname: directories +| 72 | "Desktop" | +| 12156104 | "Documents" | +| 3482440 | "Downloads" | +| 2901720 | "Library" | +| 57344 | "Movies" | +| 16548024 | "Music" | +| 120 | "News" | +| 7649472 | "Pictures" | +| 0 | "Public" | +| 152224 | "Sites" | +| 8 | "System" | +| 56 | "bin" | +| 3821872 | "mail" | +| 10605392 | "src" | +| 1264 | "tools" | +3. R source code (which calls the previous shell source code) +#+srcname: directory-pie + #+begin_src R :var dirs = directories :session R-pie-example + pie(dirs[,1], labels = dirs[,2]) + #+end_src +4. Results of R code [[file:images/dirs.png]] + * Spreadsheet plugins for org-mode in any language + :PROPERTIES: + :CUSTOM_ID: spreadsheet + :END: + +*NOTE*: Maybe in-addition-to/in-stead-of this example we should do a +more traditional "spreadsheet" example with R [Eric] + +Not only can Org-babel pass entire tables of data to source code +blocks (see [[arguments-to-source-code-blocks]]), Org-babel can also be +used to call source code blocks from *within* tables using the +Org-mode's [[http://orgmode.org/manual/The-spreadsheet.html#The-spreadsheet][existing spreadsheet functionality]]. + +In fact the functional test suite for Org-babel is implemented as a +large Org-mode table. To run the entire test suite you simple +evaluate the table =C-u C-c C-c=, and all of the tests are run +updating the table with pass/fail statistics. + +Here's a sample of our test suite. + +#+TBLNAME: org-babel-tests +| functionality | block | arg | expected | results | pass | +|------------------+--------------+-----+-------------+-------------+------| +| basic evaluation | | | | | pass | +|------------------+--------------+-----+-------------+-------------+------| +| emacs lisp | basic-elisp | 2 | 4 | 4 | pass | +| shell | basic-shell | | 6 | 6 | pass | +| ruby | basic-ruby | | org-babel | org-babel | pass | +| python | basic-python | | hello world | hello world | pass | +| R | basic-R | | 13 | 13 | pass | +#+TBLFM: $5='(if (= (length $3) 1) (progn (message (format "running %S" '(sbe $2 (n $3)))) (sbe $2 (n $3))) (sbe $2))::$6='(if (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5)) +#+TBLFM: $5=""::$6="" + +*** code blocks for tests + +#+srcname: basic-elisp +#+begin_src emacs-lisp :var n=7 +(* 2 n) +#+end_src + +#+srcname: basic-shell +#+begin_src sh :results silent +expr 1 + 5 +#+end_src + +#+srcname: date-simple +#+begin_src sh :results silent +date +#+end_src + +#+srcname: basic-ruby +#+begin_src ruby :results silent +"org-babel" +#+end_src + +#+srcname: basic-python +#+begin_src python :results silent +'hello world' +#+end_src + +#+srcname: basic-R +#+begin_src R :results silent +b <- 9 +b + 4 +#+end_src + * Library of Babel + :PROPERTIES: + :CUSTOM_ID: library-of-babel + :END: What about those source code blocks which are so useful you want to have them available in every org-mode buffer? @@ -112,14 +286,48 @@ c(5, 10) (org-babel-lob-ingest "path/to/file.org") #+end_src -* Reproducible research - - output vs. value mode - - file & graphical output - - controlling export +* Reproducible Research + :PROPERTIES: + :CUSTOM_ID: reproducable-research + :END: +#+begin_quote +An article about computational science in a scientific publication is +not the scholarship itself, it is merely advertising of the +scholarship. The actual scholarship is the complete software +development environment and the complete set of instructions which +generated the figures. -- D. Donoho +#+end_quote + +[[http://reproducibleresearch.net/index.php/Main_Page][Reproducible Research]] (RR) is the practice of distributing along with +an article of research all data, code, and tools required to reproduce +the results discussed in the paper. As such the paper becomes not +only a document describing the research but a complete laboratory +reproducing the research. + +Org-mode already has exceptional support for [[http://orgmode.org/manual/Exporting.html#Exporting][exporting to html and +LaTeX]]. Org-babel makes Org-mode a tool for RR by *activating* the +data and source code embedded into Org-mode documents making the +entire document executable. This makes it not only possible, but +natural to distribute research in a format that encourages readers to +recreate your results, and perform their own analysis. + +Existing RR tools like [[http://en.wikipedia.org/wiki/Sweave][Sweave]] provide for the embedding of [[http://www.r-project.org/][R]] code into +LaTeX documents. While this is very useful, such documents often +still require a large degree of "glue code" in the form of external +shell scripts, python scripts, and Makefiles. To my knowledge +Org-babl is the only RR tool which allows multiple languages and data +to coexist and cooperate inside of a single document. + * Literate programming + :PROPERTIES: + :CUSTOM_ID: literate-programming + :END: - org-babel-tangle - org-babel-load-file * Reference / Documentation + :PROPERTIES: + :CUSTOM_ID: reference-and-documentation + :END: *** Source Code block syntax diff --git a/org-babel.org b/org-babel.org index e080c8e9a..3628db83b 100644 --- a/org-babel.org +++ b/org-babel.org @@ -218,7 +218,13 @@ would then be [[#sandbox][the sandbox]]. #+end_src -* Tasks [41/62] +* Tasks [41/64] +** TODO org-bable-tangle: no default extension if one already exists +** TODO source-name visible in LaTeX and html exports +Maybe this should be done in backend specific manners. + +The listings package may provide for naming a source-code block... + ** PROPOSED new results types (org, html, latex) Thanks to Tom Short for this recommendation. @@ -2776,8 +2782,45 @@ dot("$(2a,0)$",(2,0),N+E); *** DONE sh -* Bugs [31/40] -** TODO export problems when support for a language is missing +* Bugs [32/40] +** PROPOSED require users to explicitly turn on each language +As we continue to add more languages to org-babel, many of which will +require new major-modes we need to re-think how languages are added to +org-babel. + +Currently we are requiring all available languages in the +[[file:lisp/org-babel-init.el][org-babel-init.el]] file. I think we need to change this to a user +setting so that only the language which have their requirements met +(in terms of system executables and emacs major modes) are loaded. It +is one more step for install, but it seems to me to be the only +solution. Thoughts? + +*** proposed + +we add something like the following to the instillation instructions + +#+begin_src emacs-lisp +;; Uncomment each of the following require lines if you want org-babel +;; to support that language. Each language has a comment explaining +;; it's dependencies. See the related files in lisp/langs for more +;; detailed explanations of requirements. +;; +;; (require 'org-babel-ruby) ;; inf-ruby mode, ruby and irb must be installed on your system +;; (require 'org-babel-python) ;; python-mode +;; (require 'org-babel-R) ;; ess-mode +;; (require 'org-babel-gnuplot) ;; gnuplot-mode +;; (require 'org-babel-dot) ;; dot be installed on your system +;; (require 'org-babel-asymptote) ;; asymptote be installed on your system +;; (require 'org-babel-ditaa) ;; ditaa be installed on your system +;; (require 'org-babel-sql) ;; none +;; (require 'org-babel-css) ;; none +#+end_src + +note that =org-babel-sh=, =org-babel-emacs-lisp=, and + are not included in the list as they can safely be +assumed to work on any system. + +*** impetus we should come up with a way to gracefully degrade when support for a specific language is missing @@ -2901,12 +2944,6 @@ even a third" #+begin_src sh size=5 #+end_src - -** TODO Allow source blocks to be recognised when #+ are not first characters on the line - I think Carsten has recently altered the core so that #+ can have - preceding whitespace, at least for literal/code examples. org-babel - should support this. - ** TODO non-orgtbl formatted lists for example @@ -2962,6 +2999,26 @@ the same for the other languages. [Dan] find . \( -path \*/SCCS -o -path \*/RCS -o -path \*/CVS -o -path \*/MCVS -o -path \*/.svn -o -path \*/.git -o -path \*/.hg -o -path \*/.bzr -o -path \*/_MTN -o -path \*/_darcs -o -path \*/\{arch\} \) -prune -o -type f \( -iname \*.el \) -exec grep -i -nH -e org-babel-trim {} \; #+end_src +** DONE Allow source blocks to be recognised when #+ are not first characters on the line + I think Carsten has recently altered the core so that #+ can have + preceding whitespace, at least for literal/code examples. org-babel + should support this. + +#+srcname: testing-indentation + #+begin_src emacs-lisp :results silent + (message "i'm indented") + #+end_src + +#+srcname: testing-non-indentation +#+begin_src emacs-lisp :results silent +(message "I'm not indented") +#+end_src + +#+srcname: i-resolve-references-to-the-indented +#+begin_src emacs-lisp :var speech=testing-indentation :results silent +(message "I said %s" speech) +#+end_src + ** DONE are the org-babel-trim s necessary? at the end of e.g. org-babel-R-evaluate, org-babel-python-evaluate, but not org-babel-ruby-evaluate diff --git a/publish-babel.org b/publish-babel.org index e9fc0a159..2441265d5 100644 --- a/publish-babel.org +++ b/publish-babel.org @@ -17,7 +17,7 @@ publishing. Publish a project with =C-c C-e X=. `("org-babel-documentation" :base-directory ,org-babel-dir :base-extension "org" - :exclude "org-babel.org" + :exclude ,(regexp-opt-group '("org-babel.org" "test-export.org" "test-tangle.org" "test-tangle-load.org")) :publishing-directory ,(expand-file-name "doc" org-babel-dir) :index-filename "org-babel-worg.org" :auto-postamble nil