15 KiB

Raw Blame History

Org-babel

Introduction
Getting started
Basic org-babel functionality
A meta-programming language for org-mode
Spreadsheet plugins for org-mode in any language
Library of Babel
Reproducible Research
Literate programming
Reference / Documentation

<div id="subtitle"> <p>executable source code blocks in org-mode</p> </div> <div id="logo"> <p> <img src="images/tower-of-babel.png" alt="images/tower-of-babel.png" title="And the Lord said, Behold, the people is one, and they have all one language; and this they begin to do; and now nothing will be restrained from them, which they have imagined to do. Genesis 11:1-9"/> <div id="attr"> from <a href="http://www.flickr.com/photos/23379658@N05/" title=""><b>Martijn Streefkerk</b></a> </div> </p> </div>

Introduction

Org-babel provides the following modifications to the existing support for blocks of source code examples in the org-mode core.

source code execution
arguments to source code blocks
exportation of source code blocks to files (literate programming)

Getting started

Grab the latest code from the git repo at github/org-babel

git clone git://github.com/eschulte/org-babel.git

And add the following lines to your .emacs, replacing the path as appropriate. A good place to check that things are up and running would the examples in Basic org-babel functionality.

  (add-to-list 'load-path "/path/to/org-babel/lisp")
  (require 'org-babel-init)

Basic org-babel functionality

Source code execution

For interpreted languages such as shell, python, R, etc, org-babel allows source blocks to be executed: the code is passed to the interpreter and you have control over what is done with the results of excecution. E.g. place point anywhere in the following block and use C-c C-c to run the code:

Ruby source code

"This file was last evaluated on #{Date.today}"

Results of Ruby evaluation

This file was last evaluated on 2009-08-09

R source code

x = 4
date()
c(5, 10)

Results of R evaluation

What happens to the results?

Org-babel provides two fundamentally different modes for capturing the results of code evaluation, specified by the :results header argument:

:results value

This means that the 'result' of code evaluation is defined to be the value of the last statement in the block. Thus with this setting, one can view the code block as a function with a return value. And not only can one view it that way, but you can actually use the return value of one source block as input for another (see later). This setting is the default.

:results output

With this setting, org-babel captures all the text output of the code block and places it in the org buffer. One can think of this as a 'scripting' mode: the code block contains a series of commands, and you get the output of all the commands. Unlike in the 'functional' mode specified by :results value, the code block has no return value. (This mode will be familiar to Sweave users).

Additional :results settings

Arguments to source code blocks

In addition to evaluation of code blocks, org-babel allows them to be parameterised (i.e. have arguments). Thus source code blocks now have the status of functions.

Inputs for fibonacci-seq

1	2	3	4	5	6	7	8	9	10
2	4	6	8	10	12	14	16	18	20

in the Org-mode buffer this looks like

#+tblname: fibonacci-inputs
| 1 | 2 | 3 | 4 |  5 |  6 |  7 |  8 |  9 | 10 |
| 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |

Emacs Lisp source code

  (defun fibonacci (n)
    (if (or (= n 0) (= n 1))
        n
      (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
  
  (mapcar (lambda (row)
            (mapcar #'fibonacci row)) fib-inputs)

in the Org-mode buffer this looks like

#+srcname: fibonacci-seq
#+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs
  (defun fibonacci (n)
    (if (or (= n 0) (= n 1))
        n
      (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
  
  (mapcar (lambda (row)
            (mapcar #'fibonacci row)) fib-inputs)
#+end_src

Results of Emacs Lisp code evaluation

1	1	2	3	5	8	13	21	34	55
1	3	8	21	55	144	377	987	2584	6765

A meta-programming language for org-mode

Since information can pass freely between source-code blocks and org-mode tables you can mix and match languages using each language for those tasks to which it is suited. This makes Org-mode files with Org-babel into a kind of meta-functional programming language in which functions from many languages can work together.

As an example, lets take some system diagnostics in the shell, and then graph them with R.

Shell source code

cd ~ && du -sc * |grep -v total

Results of the shell source code (on my system, grab this org-mode files and try running it on your own)

72	"Desktop"
12156104	"Documents"
3482440	"Downloads"
2901720	"Library"
57344	"Movies"
16548024	"Music"
120	"News"
7649472	"Pictures"
0	"Public"
152224	"Sites"
8	"System"
56	"bin"
3821872	"mail"
10605392	"src"
1264	"tools"

R source code (which calls the previous shell source code)

pie(dirs[,1], labels = dirs[,2])

Results of R code

Spreadsheet plugins for org-mode in any language

NOTE: Maybe in-addition-to/in-stead-of this example we should do a more traditional "spreadsheet" example with R [Eric]

Not only can Org-babel pass entire tables of data to source code blocks (see /ndwarshuis/org-mode/src/commit/99ba71d35e284c77314255277468a06576af93e8/arguments-to-source-code-blocks), Org-babel can also be used to call source code blocks from within tables using the Org-mode's existing spreadsheet functionality.

In fact the functional test suite for Org-babel is implemented as a large Org-mode table. To run the entire test suite you simple evaluate the table C-u C-c C-c, and all of the tests are run updating the table with pass/fail statistics.

Here's a sample of our test suite.

functionality	block	arg	expected	results	pass
basic evaluation					pass
emacs lisp	basic-elisp	2	4	4	pass
shell	basic-shell		6	6	pass
ruby	basic-ruby		org-babel	org-babel	pass
python	basic-python		hello world	hello world	pass
R	basic-R		13	13	pass

code blocks for tests

(* 2 n)

expr 1 + 5

date

"org-babel"

'hello world'

b <- 9
b + 4

Library of Babel

What about those source code blocks which are so useful you want to have them available in every org-mode buffer?

The Library of Babel is an extensible collection of ready-made and easily-shortcut-callable source-code blocks for handling common tasks. Org-babel comes pre-populated with the source-code blocks located in the library-of-babel.org file. It is possible to add source-code blocks from any org-mode file to the library by calling

(org-babel-lob-ingest "path/to/file.org")

Reproducible Research

An article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures. – D. Donoho

Reproducible Research (RR) is the practice of distributing along with an article of research all data, code, and tools required to reproduce the results discussed in the paper. As such the paper becomes not only a document describing the research but a complete laboratory reproducing the research.

Org-mode already has exceptional support for exporting to html and LaTeX. Org-babel makes Org-mode a tool for RR by activating the data and source code embedded into Org-mode documents making the entire document executable. This makes it not only possible, but natural to distribute research in a format that encourages readers to recreate your results, and perform their own analysis.

Existing RR tools like Sweave provide for the embedding of R code into LaTeX documents. While this is very useful, such documents often still require a large degree of "glue code" in the form of external shell scripts, python scripts, and Makefiles. To my knowledge Org-babl is the only RR tool which allows multiple languages and data to coexist and cooperate inside of a single document.

Literate programming

org-babel-tangle
org-babel-load-file

Reference / Documentation

Source Code block syntax

The basic syntax of source-code blocks is as follows:

#+srcname: name
#+begin_src language header-arguments
body
#+end_src

name: This name is associated with the source-code block. This is similar to the #+TBLNAME lines which can be used to name tables in org-mode files. By referencing the srcname of a source-code block it is possible to evaluate the block for other places, files, or from inside tables.
language: The language of the code in the source-code block, valid values must be members of `org-babel-interpreters'.
header-arguments: Header arguments control many facets of the input to, evaluation of, and output of source-code blocks. See the Header Arguments section for a complete review of available header arguments.
body: The actual source code which will be evaluated. This can be edited with `org-edit-special'.

Header Arguments

results

results arguments specify what should be done with the output of source-code blocks

The following options are mutually exclusive, and specify how the results should be collected from the source-code block

value

output
The following options are mutually exclusive and specify what type of results the code block will return

vector

specifies that the results should be interpreted as a multidimensional vector (even if the vector is trivial), and will be inserted into the org-mode file as a table

scalar

specifies that the results should be interpreted as a scalar value, and will be inserted into the org-mode file as quoted text

file

specifies that the results should be interpreted as the path to a file, and will be inserted into the org-mode file as a link
The following options specify how the results should be inserted into the org-mode file

replace

the current results replace any previously inserted results from the code block

silent

rather than being inserted into the org-mode file the results are echoed into the message bar

exports

exports arguments specify what should be included in html or latex exports of the org-mode file

code: the body of code is included into the exported file
results: the results of evaluating the code is included in the exported file
both: both the code and results are included in the exported file
none: nothing is included in the exported file

tangle

tangle arguments specify whether or not the source-code block should be included in tangled extraction of source-code files

yes: the source-code block is exported to a source-code file named after the basename (name w/o extension) of the org-mode file
no: (default) the source-code block is not exported to a source-code file
other: any other string passed to the tangle header argument is interpreted as a file basename to which the block will be exported

15 KiB Raw Blame History Unescape Escape