I've done a fairly destructive edit of this file. The main goal was
to enforce a structure on the document that we can use moving forward, so that any future objective changes are all made to the main objective list. I apologize for removing sections written by other people. I did this when they were redundant or it was not clear how to fit them into this structure. Rest assured if the previous text wasn't persisted in git I would have been much more cautious about removing it. I hope that this outline structure should be able to remain stable through the process of fleshing out objectives, and cashing those objectives out into tasks. That said, please feel free to make any changes that you see fit. Changes include... - tearing stuff out, and imposing structure - updated .gitignore to ignore exported files
This commit is contained in:
parent
9eeff6504b
commit
d26a98d71a
|
@ -1,3 +1,4 @@
|
||||||
*~
|
*~
|
||||||
|
*.html
|
||||||
.Rhistory
|
.Rhistory
|
||||||
.Rdata
|
.Rdata
|
||||||
|
|
767
rorg.html
767
rorg.html
|
@ -1,767 +0,0 @@
|
||||||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
|
|
||||||
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
||||||
<html xmlns="http://www.w3.org/1999/xhtml"
|
|
||||||
lang="en" xml:lang="en">
|
|
||||||
<head>
|
|
||||||
<title>rorg — Code evaluation in org-mode, with an emphasis on R</title>
|
|
||||||
<meta http-equiv="Content-Type" content="text/html;charset=iso-8859-1"/>
|
|
||||||
<meta name="generator" content="Org-mode"/>
|
|
||||||
<meta name="generated" content="2009-02-08 14:56:13 EST"/>
|
|
||||||
<meta name="author" content="Dan"/>
|
|
||||||
<style type="text/css">
|
|
||||||
<!--/*--><![CDATA[/*><!--*/
|
|
||||||
html { font-family: Times, serif; font-size: 12pt; }
|
|
||||||
.title { text-align: center; }
|
|
||||||
.todo { color: red; }
|
|
||||||
.done { color: green; }
|
|
||||||
.tag { background-color: #add8e6; font-weight:normal }
|
|
||||||
.target { }
|
|
||||||
.timestamp { color: #bebebe; }
|
|
||||||
.timestamp-kwd { color: #5f9ea0; }
|
|
||||||
p.verse { margin-left: 3% }
|
|
||||||
pre {
|
|
||||||
border: 1pt solid #AEBDCC;
|
|
||||||
background-color: #F3F5F7;
|
|
||||||
padding: 5pt;
|
|
||||||
font-family: courier, monospace;
|
|
||||||
font-size: 90%;
|
|
||||||
overflow:auto;
|
|
||||||
}
|
|
||||||
table { border-collapse: collapse; }
|
|
||||||
td, th { vertical-align: top; }
|
|
||||||
dt { font-weight: bold; }
|
|
||||||
div.figure { padding: 0.5em; }
|
|
||||||
div.figure p { text-align: center; }
|
|
||||||
.linenr { font-size:smaller }
|
|
||||||
.code-highlighted {background-color:#ffff00;}
|
|
||||||
.org-info-js_info-navigation { border-style:none; }
|
|
||||||
#org-info-js_console-label { font-size:10px; font-weight:bold;
|
|
||||||
white-space:nowrap; }
|
|
||||||
.org-info-js_search-highlight {background-color:#ffff00; color:#000000;
|
|
||||||
font-weight:bold; }
|
|
||||||
/*]]>*/-->
|
|
||||||
</style>
|
|
||||||
<script type="text/javascript">
|
|
||||||
<!--/*--><![CDATA[/*><!--*/
|
|
||||||
function CodeHighlightOn(elem, id)
|
|
||||||
{
|
|
||||||
var target = document.getElementById(id);
|
|
||||||
if(null != target) {
|
|
||||||
elem.cacheClassElem = elem.className;
|
|
||||||
elem.cacheClassTarget = target.className;
|
|
||||||
target.className = "code-highlighted";
|
|
||||||
elem.className = "code-highlighted";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
function CodeHighlightOff(elem, id)
|
|
||||||
{
|
|
||||||
var target = document.getElementById(id);
|
|
||||||
if(elem.cacheClassElem)
|
|
||||||
elem.className = elem.cacheClassElem;
|
|
||||||
if(elem.cacheClassTarget)
|
|
||||||
target.className = elem.cacheClassTarget;
|
|
||||||
}
|
|
||||||
/*]]>*/-->
|
|
||||||
</script>
|
|
||||||
</head><body>
|
|
||||||
<h1 class="title">rorg — Code evaluation in org-mode, with an emphasis on R</h1>
|
|
||||||
|
|
||||||
|
|
||||||
<div id="table-of-contents">
|
|
||||||
<h2>Table of Contents</h2>
|
|
||||||
<div id="text-table-of-contents">
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-1">Overview (Dan) </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-1.1">Project objectives </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-1.1.1">It produces text/numeric output </a></li>
|
|
||||||
<li><a href="#sec-1.1.2">It produces graphical output </a></li>
|
|
||||||
<li><a href="#sec-1.1.3">It creates some non-graphics files </a></li>
|
|
||||||
<li><a href="#sec-1.1.4">It alters the environment by side effect in some other way </a></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-1.2">Implementation questions </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-1.2.1">How is the code placed in the org file? </a></li>
|
|
||||||
<li><a href="#sec-1.2.2">When is the code evaluated? </a></li>
|
|
||||||
<li><a href="#sec-1.2.3">What is the result of code evaluation? </a></li>
|
|
||||||
</ul></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-2">Commentary </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-2.1">Eric </a></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-3">Objectives </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-3.1">Send data to R from org </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-3.1.1">Implementations </a></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-3.2">Evaluate R code from org and deal with output appropriately </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-3.2.1">vector output </a></li>
|
|
||||||
<li><a href="#sec-3.2.2">graphical output </a></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-3.3">Evaluate R code on export </a></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-4">Notes </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-4.1">Special editing and evaluation of source code in R blocks </a>
|
|
||||||
<ul>
|
|
||||||
<li><a href="#sec-4.1.1">R-block proposal </a></li>
|
|
||||||
<li><a href="#sec-4.1.2">Source code blocks </a></li>
|
|
||||||
<li><a href="#sec-4.1.3">dblocks </a></li>
|
|
||||||
<li><a href="#sec-4.1.4">R blocks </a></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-4.2">Interaction with the R process </a></li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li><a href="#sec-5">Tasks </a></li>
|
|
||||||
<li><a href="#sec-6">buffer dictionary </a></li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1" class="outline-2">
|
|
||||||
<h2 id="sec-1">Overview (Dan <span class="timestamp">2009-02-08 Sun</span>) </h2>
|
|
||||||
<div id="text-1">
|
|
||||||
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.1" class="outline-3">
|
|
||||||
<h3 id="sec-1.1">Project objectives </h3>
|
|
||||||
<div id="text-1.1">
|
|
||||||
|
|
||||||
<p>This project is basically about putting source code into org
|
|
||||||
files. This isn't just code to look pretty as a source code example,
|
|
||||||
but code to be evaluated. Org files have 3 main export targets: org,
|
|
||||||
html and latex. Thus the aim of this project is to produce files in
|
|
||||||
those formats that have benefitted in some way from the evaluation of
|
|
||||||
source code that is present in the source org file. We have a current
|
|
||||||
focus on R code, but we are regarding that more as a working example
|
|
||||||
than as a defining feature of the project.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
Code evaluation can have three relevant consequences. Our aim is to
|
|
||||||
deal with these consequences as follows:
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.1.1" class="outline-4">
|
|
||||||
<h4 id="sec-1.1.1">It produces text/numeric output </h4>
|
|
||||||
<div id="text-1.1.1">
|
|
||||||
|
|
||||||
<p>We (optionally) incorporate the text output as text in the target
|
|
||||||
document
|
|
||||||
</p></div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.1.2" class="outline-4">
|
|
||||||
<h4 id="sec-1.1.2">It produces graphical output </h4>
|
|
||||||
<div id="text-1.1.2">
|
|
||||||
|
|
||||||
<p>We either link to the graphics or (html/latex) include them inline.
|
|
||||||
</p></div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.1.3" class="outline-4">
|
|
||||||
<h4 id="sec-1.1.3">It creates some non-graphics files </h4>
|
|
||||||
<div id="text-1.1.3">
|
|
||||||
|
|
||||||
<p>? We link to other file output
|
|
||||||
</p></div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.1.4" class="outline-4">
|
|
||||||
<h4 id="sec-1.1.4">It alters the environment by side effect in some other way </h4>
|
|
||||||
<div id="text-1.1.4">
|
|
||||||
|
|
||||||
<p>We bear this in mind
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.2" class="outline-3">
|
|
||||||
<h3 id="sec-1.2">Implementation questions </h3>
|
|
||||||
<div id="text-1.2">
|
|
||||||
|
|
||||||
<p>These objectives raise three questions:
|
|
||||||
</p>
|
|
||||||
<ol>
|
|
||||||
<li>
|
|
||||||
How is the code placed in the org file?
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
When is the code evaluated?
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
What is the result of code evaluation?
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ol>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.2.1" class="outline-4">
|
|
||||||
<h4 id="sec-1.2.1">How is the code placed in the org file? </h4>
|
|
||||||
<div id="text-1.2.1">
|
|
||||||
|
|
||||||
<p>Using some version of the code block ideas that Eric and Austin
|
|
||||||
have worked on. (In addition, an aim of org-R was to allow Org
|
|
||||||
users who are not R users to specify R code implicitly, using
|
|
||||||
native org syntax. I'd like to maintain that, but it's not central
|
|
||||||
to this project.)
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.2.2" class="outline-4">
|
|
||||||
<h4 id="sec-1.2.2">When is the code evaluated? </h4>
|
|
||||||
<div id="text-1.2.2">
|
|
||||||
|
|
||||||
<p>Let's use an asterisk to indicate content which includes the
|
|
||||||
<b>result</b> of code evaluation, rather than the code itself. Clearly
|
|
||||||
we have a requirement for the following transformation:
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
org → org*
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
Let's say this transformation is effected by a function
|
|
||||||
`org-eval-buffer'. This transformation is necessary when the
|
|
||||||
target format is org (say you want to update the values in an org
|
|
||||||
table, or generate a plot and create an org link to it), and it
|
|
||||||
can also be used as the first step by which to reach html and
|
|
||||||
latex:
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
org → org* → html
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
org → org* → latex
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
Thus in principle we can reach our 3 target formats with
|
|
||||||
`org-eval-buffer', `org-export-as-latex' and `org-export-as-html'.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
An extra transformation that we might want is
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
org → latex
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
I.e. export to latex without evaluation of code, in such a way that R
|
|
||||||
code can subsequently be evaluated using
|
|
||||||
<code>Sweave(driver=RweaveLatex)</code>, which is what the R community is
|
|
||||||
used to. This would provide a `bail out' avenue where users can
|
|
||||||
escape org mode and enter a workflow in which the latex/noweb file
|
|
||||||
is treated as source.
|
|
||||||
</p>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-1.2.2.1">How do we implement `org-eval-buffer'? <br/>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
AIUI The following can all be viewed as implementations of
|
|
||||||
org-eval-buffer for R code:
|
|
||||||
</p>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-1.2.2.1.1">org-eval-light <br/>
|
|
||||||
This is the beginnings of a general evaluation mechanism, that
|
|
||||||
could evaluate python, ruby, shell, perl, in addition to R.
|
|
||||||
The header says it's based on org-eval, what is org-eval??
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li id="sec-1.2.2.1.2">org-R <br/>
|
|
||||||
This accomplishes org → org* in elisp by visiting code blocks
|
|
||||||
and evaluating code using ESS.
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li id="sec-1.2.2.1.3">RweaveOrg <br/>
|
|
||||||
This accomplishes org → org* using R via
|
|
||||||
|
|
||||||
<pre class="example">
|
|
||||||
Sweave("file-with-unevaluated-code.org", driver=RweaveOrg, syntax=SweaveSyntaxOrg)
|
|
||||||
</pre>
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li id="sec-1.2.2.1.4">org-exp-blocks.el <br/>
|
|
||||||
Like org-R, this achieves org → org* in elisp by visiting code
|
|
||||||
blocks and using ESS to evaluate R code.
|
|
||||||
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-1.2.3" class="outline-4">
|
|
||||||
<h4 id="sec-1.2.3">What is the result of code evaluation? </h4>
|
|
||||||
<div id="text-1.2.3">
|
|
||||||
|
|
||||||
<p>Here we have to consider text/numeric output, and graphical
|
|
||||||
output. And also the stage at which evaluation occurs
|
|
||||||
</p><ul>
|
|
||||||
<li id="sec-1.2.3.1">org → org* <br/>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-1.2.3.1.1">Text / numerical output <br/>
|
|
||||||
In the case of org → org*, I would argue that, where
|
|
||||||
appropriate, it should be stored in org tables. Thus an advantage
|
|
||||||
our project would have over Sweave is that tabular output is
|
|
||||||
automatically conveqrted to native tables on export to HTML and
|
|
||||||
latex.
|
|
||||||
</li>
|
|
||||||
<li id="sec-1.2.3.1.2">Graphical output <br/>
|
|
||||||
We place an org link to the file. This is done already by
|
|
||||||
org-R-apply, and by RweaveOrg.
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
<li id="sec-1.2.3.2">latex → latex* <br/>
|
|
||||||
This is done by Sweave(driver=RweaveLatex) and so is out of our hands
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-2" class="outline-2">
|
|
||||||
<h2 id="sec-2">Commentary </h2>
|
|
||||||
<div id="text-2">
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-2.1" class="outline-3">
|
|
||||||
<h3 id="sec-2.1">Eric <span class="timestamp">2009-02-06 Fri 15:41</span> </h3>
|
|
||||||
<div id="text-2.1">
|
|
||||||
|
|
||||||
<p>I think we're getting close to a comprehensive set of objectives
|
|
||||||
(although since you two are the real R user's I leave that decision up
|
|
||||||
to you). Once we've agreed on a set of objectives and agreed on at
|
|
||||||
least to broad strokes of implementation, I think we should start
|
|
||||||
listing out and assigning tasks.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-3" class="outline-2">
|
|
||||||
<h2 id="sec-3">Objectives </h2>
|
|
||||||
<div id="text-3">
|
|
||||||
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-3.1" class="outline-3">
|
|
||||||
<h3 id="sec-3.1">Send data to R from org </h3>
|
|
||||||
<div id="text-3.1">
|
|
||||||
|
|
||||||
<p>Org-mode includes orgtbl-mode, an extremely convenient way of using
|
|
||||||
tabular data in a plain text file. Currently, spreadsheet
|
|
||||||
functionality is available in org tables using the emacs package
|
|
||||||
calc. It would be a boon both to org users and R users to allow
|
|
||||||
org tables to be manipulated with the R programming language. Org
|
|
||||||
tables give R users an easy way to enter and display data; R gives
|
|
||||||
org users a powerful way to perform vector operations, statistical
|
|
||||||
tests, and visualization on their tables.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-3.1.1" class="outline-4">
|
|
||||||
<h4 id="sec-3.1.1">Implementations </h4>
|
|
||||||
<div id="text-3.1.1">
|
|
||||||
|
|
||||||
<ul>
|
|
||||||
<li id="sec-3.1.1.1">naive <br/>
|
|
||||||
Naive implementation would be to use <code>(org-export-table "tmp.csv")</code>
|
|
||||||
and <code>(ess-execute "read.csv('tmp.csv')")</code>.
|
|
||||||
</li>
|
|
||||||
<li id="sec-3.1.1.2">org-R <br/>
|
|
||||||
org-R passes data to R from two sources: org tables, or csv
|
|
||||||
files. Org tables are first exported to a temporary csv file
|
|
||||||
using <a href="existing_tools/org-R.el">org-R-export-to-csv</a>.
|
|
||||||
</li>
|
|
||||||
<li id="sec-3.1.1.3">org-exp-blocks <br/>
|
|
||||||
org-exp-blocks uses <a href="#sec-3.1.1.3">org-interblock-R-command-to-string</a> to send
|
|
||||||
commands to an R process running in a comint buffer through ESS.
|
|
||||||
org-exp-blocks has no support for dumping table data to R process, or
|
|
||||||
vice versa.
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li id="sec-3.1.1.4">RweaveOrg <br/>
|
|
||||||
NA
|
|
||||||
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-3.2" class="outline-3">
|
|
||||||
<h3 id="sec-3.2">Evaluate R code from org and deal with output appropriately </h3>
|
|
||||||
<div id="text-3.2">
|
|
||||||
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-3.2.1" class="outline-4">
|
|
||||||
<h4 id="sec-3.2.1">vector output </h4>
|
|
||||||
<div id="text-3.2.1">
|
|
||||||
|
|
||||||
<p>When R code evaluation generates vectors and 2-dimensional arrays,
|
|
||||||
this should be formatted appropriately in org buffers (orgtbl-mode) as well
|
|
||||||
as in export targets (html, latex)
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
Agreed, if we can convert the vector data to lists then we can use
|
|
||||||
the many orgtbl-to-* functions to convert the list to whatever
|
|
||||||
output format we desire. See `orgtbl-to-orgtbl, `orgtbl-to-latex',
|
|
||||||
`orgtbl-to-html', `orgtbl-to-csv', etc…
|
|
||||||
</p>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-3.2.1.1">Implementations <br/>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-3.2.1.1.1">org-R <br/>
|
|
||||||
org-R converts R output (vectors, or matrices / 2d-arrays) to an
|
|
||||||
org table and stores it in the org buffer, or in a separate org
|
|
||||||
file (csv output would also be perfectly possible).
|
|
||||||
</li>
|
|
||||||
<li id="sec-3.2.1.1.2">org-exp-blocks <br/>
|
|
||||||
</li>
|
|
||||||
<li id="sec-3.2.1.1.3">RweaveOrg <br/>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-3.2.2" class="outline-4">
|
|
||||||
<h4 id="sec-3.2.2">graphical output </h4>
|
|
||||||
<div id="text-3.2.2">
|
|
||||||
|
|
||||||
<p>R can generate graphical output on a screen graphics device
|
|
||||||
(e.g. X11, quartz), and in various standard image file formats
|
|
||||||
(png, jpg, ps, pdf, etc). When graphical output is generated by
|
|
||||||
evaluation of R code in Org, at least the following two things are desirable:
|
|
||||||
</p><ol>
|
|
||||||
<li>
|
|
||||||
output to screen for immediate viewing is possible
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
graphical output to file is linked to appropriately from the
|
|
||||||
org file This should have the automatic consequence that it is
|
|
||||||
included appropriately in subsequent export targets (html,
|
|
||||||
latex).
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ol>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-3.2.2.1">Implementations <br/>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-3.2.2.1.1">org-R <br/>
|
|
||||||
org-R does (1) if no output file is specified and (2) otherwise
|
|
||||||
</li>
|
|
||||||
<li id="sec-3.2.2.1.2">org-exp-blocks <br/>
|
|
||||||
org-exp-blocks tries to do 2, but I don't think that part was
|
|
||||||
every really working
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li id="sec-3.2.2.1.3">RweaveOrg <br/>
|
|
||||||
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-3.3" class="outline-3">
|
|
||||||
<h3 id="sec-3.3">Evaluate R code on export </h3>
|
|
||||||
<div id="text-3.3">
|
|
||||||
|
|
||||||
<p>At first I was leaning towards leaving the exporting to Sweave, but it
|
|
||||||
seems that once we have evaluation or R working, it will not be
|
|
||||||
difficult to implement full evaluation of R blocks, one-liners, and
|
|
||||||
creation of R graphics on export directly in elisp.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
I think that this would be worth the effort since it would eliminate
|
|
||||||
the need for using Sweave, and would allow for exportation to any
|
|
||||||
target format supported by org-mode.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-4" class="outline-2">
|
|
||||||
<h2 id="sec-4">Notes </h2>
|
|
||||||
<div id="text-4">
|
|
||||||
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-4.1" class="outline-3">
|
|
||||||
<h3 id="sec-4.1">Special editing and evaluation of source code in R blocks </h3>
|
|
||||||
<div id="text-4.1">
|
|
||||||
|
|
||||||
<p>Unfortunately org-mode how two different block types, both useful.
|
|
||||||
In developing RweaveOrg, a third was introduced.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
Eric is leaning towards using the <code>#+begin_src</code> blocks, as that is
|
|
||||||
really what these blocks contain: source code. Austin believes
|
|
||||||
that specifying export options at the beginning of a block is
|
|
||||||
useful functionality, to be preserved if possible.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
Note that upper and lower case are not relevant in block headings.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-4.1.1" class="outline-4">
|
|
||||||
<h4 id="sec-4.1.1"><span class="todo">PROPOSED</span> R-block proposal </h4>
|
|
||||||
<div id="text-4.1.1">
|
|
||||||
|
|
||||||
<p>I (Eric) propose that we use the syntax of source code blocks as they
|
|
||||||
currently exist in org-mode with the addition of <b>evaluation</b>,
|
|
||||||
<b>header-arguments</b>, <b>exportation</b>, <b>single-line-blocks</b>, and
|
|
||||||
<b>references-to-table-data</b>.
|
|
||||||
</p>
|
|
||||||
<ol>
|
|
||||||
<li>
|
|
||||||
<b>evaluation</b>: These blocks can be evaluated through <code>\C-c\C-c</code> with
|
|
||||||
a slight addition to the code already present and working in
|
|
||||||
<a href="existing_tools/org-eval-light.el">org-eval-light.el</a>. All we should need to add for R support would
|
|
||||||
be an appropriate entry in <a href="#sec-4.1.1">org-eval-light-interpreters</a> with a
|
|
||||||
corresponding evaluation function. For an example usinga
|
|
||||||
org-eval-light see <a href="#sec-4.1.1.1">* src block evaluation w/org-eval-light</a>.
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<b>header-arguments</b>: These can be implemented along the lines of
|
|
||||||
Austin's header arguments in <a href="existing_tools/RweaveOrg/org-sweave.el">org-sweave.el</a>.
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<b>exportation</b>: Should be as similar as possible to that done by
|
|
||||||
Sweave, and hopefully can re-use some of the code currently present
|
|
||||||
in <a href="existing_tools/exp-blocks/org-exp-blocks.el ">org-exp-blocks.el</a>.
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<b>single-line-blocks</b>: It seems that it is useful to be able to
|
|
||||||
place a single line of R code on a line by itself. Should we add
|
|
||||||
syntax for this similar to Dan's <code>#+R:</code> lines? I would lean
|
|
||||||
towards something here that can be re-used for any type of source
|
|
||||||
code in the same manner as the <code>#+begin_src R</code> blocks, maybe
|
|
||||||
<code>#+src_R</code>?
|
|
||||||
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<b>references-to-table-data</b>: I get this impression that this is
|
|
||||||
vital to the efficient use of R code in an org file, so we should
|
|
||||||
come up with a way to reference table data from a single-line-block
|
|
||||||
or from an R source-code block. It looks like Dan has already done
|
|
||||||
this in <a href="existing_tools/org-R.el">org-R.el</a>.
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ol>
|
|
||||||
|
|
||||||
<p>What do you think? Does this accomplish everything we want to be able
|
|
||||||
to do with embedded R source code blocks?
|
|
||||||
</p>
|
|
||||||
<ul>
|
|
||||||
<li id="sec-4.1.1.1">src block evaluation w/org-eval-light <br/>
|
|
||||||
here's an example using org-eval-light.el
|
|
||||||
|
|
||||||
<p>
|
|
||||||
first load the org-eval-light.el file
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
<i><elisp:(load (expand-file-name "org-eval-light.el" (expand-file-name "existing_tools" (file-name-directory buffer-file-name))))></i>
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
then press <code>\C-c\C-c</code> inside of the following src code snippet. The
|
|
||||||
results should appear in a comment immediately following the source
|
|
||||||
code block. It shouldn't be too hard to add R support to this
|
|
||||||
function through the `org-eval-light-interpreters' variable.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
(Dan: The following causes error on export to HTML hence spaces inserted at bol)
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
#+begin<sub>src</sub> shell
|
|
||||||
date
|
|
||||||
#+end<sub>src</sub>
|
|
||||||
</p>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-4.1.2" class="outline-4">
|
|
||||||
<h4 id="sec-4.1.2">Source code blocks </h4>
|
|
||||||
<div id="text-4.1.2">
|
|
||||||
|
|
||||||
<p>Org has an extremely useful method of editing source code and
|
|
||||||
examples in their native modes. In the case of R code, we want to
|
|
||||||
be able to use the full functionality of ESS mode, including
|
|
||||||
interactive evaluation of code.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
Source code blocks look like the following and allow for the
|
|
||||||
special editing of code inside of the block through
|
|
||||||
`org-edit-special'.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
<pre class="src src-r">
|
|
||||||
|
|
||||||
,<span style="color: #add8e6;">## </span><span style="color: #add8e6;">hit C-c ' within this block to enter a temporary buffer in r-mode.
|
|
||||||
</span>
|
|
||||||
,<span style="color: #add8e6;">## </span><span style="color: #add8e6;">while in the temporary buffer, hit C-c C-c on this comment to
|
|
||||||
</span>,<span style="color: #add8e6;">## </span><span style="color: #add8e6;">evaluate this block
|
|
||||||
</span>a <span style="color: #98fb98;"><-</span> 3
|
|
||||||
a
|
|
||||||
|
|
||||||
,<span style="color: #add8e6;">## </span><span style="color: #add8e6;">hit C-c ' to exit the temporary buffer
|
|
||||||
</span></pre>
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-4.1.3" class="outline-4">
|
|
||||||
<h4 id="sec-4.1.3">dblocks </h4>
|
|
||||||
<div id="text-4.1.3">
|
|
||||||
|
|
||||||
<p>dblocks are useful because org-mode will automatically call
|
|
||||||
`org-dblock-write:dblock-type' where dblock-type is the string
|
|
||||||
following the <code>#+BEGIN:</code> portion of the line.
|
|
||||||
</p>
|
|
||||||
<p>
|
|
||||||
dblocks look like the following and allow for evaluation of the
|
|
||||||
code inside of the block by calling <code>\C-c\C-c</code> on the header of
|
|
||||||
the block.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-4.1.4" class="outline-4">
|
|
||||||
<h4 id="sec-4.1.4">R blocks </h4>
|
|
||||||
<div id="text-4.1.4">
|
|
||||||
|
|
||||||
<p>In developing RweaveOrg, Austin created <a href="existing_tools/RweaveOrg/org-sweave.el">org-sweave.el</a>. This
|
|
||||||
allows for the kind of blocks shown in <a href="existing_tools/RweaveOrg/testing.Rorg">testing.Rorg</a>. These blocks
|
|
||||||
have the advantage of accepting options to the Sweave preprocessor
|
|
||||||
following the #+BEGIN<sub>R</sub> declaration.
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-4.2" class="outline-3">
|
|
||||||
<h3 id="sec-4.2">Interaction with the R process </h3>
|
|
||||||
<div id="text-4.2">
|
|
||||||
|
|
||||||
|
|
||||||
<p>
|
|
||||||
We should take care to implement this in such a way that all of the
|
|
||||||
different components which have to interactive with R including:
|
|
||||||
</p><ul>
|
|
||||||
<li>
|
|
||||||
evaluation of source code blocks
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
automatic evaluation on export
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
evaluation of \R{} snippets
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
evaluation of single source code lines
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
sending/receiving vector data
|
|
||||||
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
|
|
||||||
<p>I think we currently have two implementations of interaction with R
|
|
||||||
processes; <a href="existing_tools/org-R.el">org-R.el</a> and <a href="existing_tools/exp-blocks/org-exp-blocks.el ">org-exp-blocks.el</a>. We should be sure to take
|
|
||||||
the best of each of these approaches.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-5" class="outline-2">
|
|
||||||
<h2 id="sec-5">Tasks </h2>
|
|
||||||
<div id="text-5">
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="outline-container-6" class="outline-2">
|
|
||||||
<h2 id="sec-6">buffer dictionary </h2>
|
|
||||||
<div id="text-6">
|
|
||||||
|
|
||||||
<p>LocalWords: DBlocks dblocks
|
|
||||||
</p></div>
|
|
||||||
</div>
|
|
||||||
<div id="postamble"><p class="author"> Author: Dan
|
|
||||||
<a href="mailto:dan@Tichodroma"><dan@Tichodroma></a>
|
|
||||||
</p>
|
|
||||||
<p class="date"> Date: 2009-02-08 14:56:13 EST</p>
|
|
||||||
<p>HTML generated by org-mode 6.21trans in emacs 22</p>
|
|
||||||
</div></body>
|
|
||||||
</html>
|
|
287
rorg.org
287
rorg.org
|
@ -3,45 +3,54 @@
|
||||||
#+SEQ_TODO: TODO PROPOSED | DONE DROPPED MAYBE
|
#+SEQ_TODO: TODO PROPOSED | DONE DROPPED MAYBE
|
||||||
#+STARTUP: oddeven
|
#+STARTUP: oddeven
|
||||||
|
|
||||||
* Overview (Dan [2009-02-08 Sun])
|
* Overview
|
||||||
** Project objectives
|
|
||||||
This project is basically about putting source code into org
|
This project is basically about putting source code into org
|
||||||
files. This isn't just code to look pretty as a source code example,
|
files. This isn't just code to look pretty as a source code example,
|
||||||
but code to be evaluated. Org files have 3 main export targets: org,
|
but code to be evaluated. Org files have 3 main export targets: org,
|
||||||
html and latex. Thus the aim of this project is to produce files in
|
html and latex. Once we have implemented a smooth bi-directional flow
|
||||||
those formats that have benefitted in some way from the evaluation of
|
of data between org-mode formats (including tables, and maybe lists
|
||||||
source code that is present in the source org file. We have a current
|
and property values) and source-code blocks, we will be able to use
|
||||||
|
org-mode's built in export to publish this data in any org-supported
|
||||||
|
format using org-mode as an intermediate format. We have a current
|
||||||
focus on R code, but we are regarding that more as a working example
|
focus on R code, but we are regarding that more as a working example
|
||||||
than as a defining feature of the project.
|
than as a defining feature of the project.
|
||||||
|
|
||||||
Code evaluation can have three relevant consequences. Our aim is to
|
The main objectives of this project are...
|
||||||
deal with these consequences as follows:
|
|
||||||
|
|
||||||
*** It produces text/numeric output
|
# Lets start with this list and make changes as appropriate. Please
|
||||||
We (optionally) incorporate the text output as text in the target
|
# try to make changes to this list, rather than starting any new
|
||||||
document
|
# lists.
|
||||||
*** It produces graphical output
|
|
||||||
We either link to the graphics or (html/latex) include them inline.
|
|
||||||
*** It creates some non-graphics files
|
|
||||||
? We link to other file output
|
|
||||||
*** It alters the environment by side effect in some other way
|
|
||||||
We bear this in mind
|
|
||||||
|
|
||||||
** Implementation questions
|
- [[* evaluation of embedded source code][evaluation of embedded source code]]
|
||||||
These objectives raise three questions:
|
- [[* execution on demand and on export][execution on demand and on export]]
|
||||||
|
- [[* evaluation of source blocks][evaluation of source blocks]]
|
||||||
|
- [[* inline source evaluation][inline source evaluation]]
|
||||||
|
- [[* interaction with the source-code's process][interaction with the source-code's process]]
|
||||||
|
- [[* output of code evaluation][output of code evaluation]]
|
||||||
|
- [[* textual/numeric output][textual/numeric output]]
|
||||||
|
- [[* graphical output][graphical output]]
|
||||||
|
- [[* file creation][non-graphics file creation]]
|
||||||
|
- [[* side effects][side effects]]
|
||||||
|
- [[* reference to data and evaluation results][reference to data and evaluation results]]: This could happen in many
|
||||||
|
directions
|
||||||
|
- [[* reference format][reference format]]
|
||||||
|
- [[* source-target pairs][source-target pairs]]
|
||||||
|
- [[* source block output from org tables][source block output from org tables]]
|
||||||
|
- [[* source block outpt from other source block][source block outpt from other source block]]
|
||||||
|
- [[* source block output from org list][source block output from org list]] ?? maybe
|
||||||
|
- [[* org table from source block][org table from source block]]
|
||||||
|
- [[* org table from org table][org table from org table]]
|
||||||
|
- [[* org properties from source block][org properties from source block]]
|
||||||
|
- [[* org properties from org table][org properties from org table]]
|
||||||
|
- [[* caching of evaluation][caching of evaluation]]
|
||||||
|
- [[* export][export]]
|
||||||
|
|
||||||
1. How is the code placed in the org file?
|
|
||||||
2. When is the code evaluated?
|
|
||||||
3. What is the result of code evaluation?
|
|
||||||
|
|
||||||
*** How is the code placed in the org file?
|
* Objectives
|
||||||
Using some version of the code block ideas that Eric and Austin
|
|
||||||
have worked on. (In addition, an aim of org-R was to allow Org
|
|
||||||
users who are not R users to specify R code implicitly, using
|
|
||||||
native org syntax. I'd like to maintain that, but it's not central
|
|
||||||
to this project.)
|
|
||||||
|
|
||||||
*** When is the code evaluated?
|
** evaluation of embedded source code
|
||||||
|
|
||||||
|
*** execution on demand and on export
|
||||||
Let's use an asterisk to indicate content which includes the
|
Let's use an asterisk to indicate content which includes the
|
||||||
*result* of code evaluation, rather than the code itself. Clearly
|
*result* of code evaluation, rather than the code itself. Clearly
|
||||||
we have a requirement for the following transformation:
|
we have a requirement for the following transformation:
|
||||||
|
@ -81,7 +90,16 @@ deal with these consequences as follows:
|
||||||
***** org-eval-light
|
***** org-eval-light
|
||||||
This is the beginnings of a general evaluation mechanism, that
|
This is the beginnings of a general evaluation mechanism, that
|
||||||
could evaluate python, ruby, shell, perl, in addition to R.
|
could evaluate python, ruby, shell, perl, in addition to R.
|
||||||
The header says it's based on org-eval, what is org-eval??
|
The header says it's based on org-eval
|
||||||
|
|
||||||
|
what is org-eval??
|
||||||
|
|
||||||
|
org-eval was written by Carsten. It lives in the
|
||||||
|
org/contrib/lisp directory because it is too dangerous to
|
||||||
|
include in the base. Unlike org-eval-light org-eval evaluates
|
||||||
|
all source blocks in an org-file when the file is first opened,
|
||||||
|
which could be a security nightmare for example if someone
|
||||||
|
emailed you a pernicious file.
|
||||||
|
|
||||||
***** org-R
|
***** org-R
|
||||||
This accomplishes org \to org* in elisp by visiting code blocks
|
This accomplishes org \to org* in elisp by visiting code blocks
|
||||||
|
@ -96,43 +114,56 @@ deal with these consequences as follows:
|
||||||
Like org-R, this achieves org \to org* in elisp by visiting code
|
Like org-R, this achieves org \to org* in elisp by visiting code
|
||||||
blocks and using ESS to evaluate R code.
|
blocks and using ESS to evaluate R code.
|
||||||
|
|
||||||
|
*** evaluation of source blocks
|
||||||
|
(see [[* Special editing and evaluation of source code][Special editing and evaluation of source code]])
|
||||||
|
|
||||||
*** What is the result of code evaluation?
|
*** inline source evaluation
|
||||||
Here we have to consider text/numeric output, and graphical
|
|
||||||
output. And also the stage at which evaluation occurs
|
|
||||||
***** org \to org*
|
|
||||||
****** Text / numerical output
|
|
||||||
In the case of org \to org*, I would argue that, where
|
|
||||||
appropriate, it should be stored in org tables. Thus an advantage
|
|
||||||
our project would have over Sweave is that tabular output is
|
|
||||||
automatically conveqrted to native tables on export to HTML and
|
|
||||||
latex.
|
|
||||||
****** Graphical output
|
|
||||||
We place an org link to the file. This is done already by
|
|
||||||
org-R-apply, and by RweaveOrg.
|
|
||||||
***** latex \to latex*
|
|
||||||
This is done by Sweave(driver=RweaveLatex) and so is out of our hands
|
|
||||||
|
|
||||||
* Commentary
|
** interaction with the source-code's process
|
||||||
|
We should settle on a uniform API for sending code and receiving
|
||||||
|
output from a source process. Then to add a new language all we need
|
||||||
|
to do is implement this API.
|
||||||
|
|
||||||
** Eric <2009-02-06 Fri 15:41>
|
for related notes see ([[* Interaction with the R process][Interaction with the R process]])
|
||||||
I think we're getting close to a comprehensive set of objectives
|
|
||||||
(although since you two are the real R user's I leave that decision up
|
|
||||||
to you). Once we've agreed on a set of objectives and agreed on at
|
|
||||||
least to broad strokes of implementation, I think we should start
|
|
||||||
listing out and assigning tasks.
|
|
||||||
|
|
||||||
|
** output of code evaluation
|
||||||
|
*** textual/numeric output
|
||||||
|
We (optionally) incorporate the text output as text in the target
|
||||||
|
document
|
||||||
|
*** graphical output
|
||||||
|
We either link to the graphics or (html/latex) include them
|
||||||
|
inline.
|
||||||
|
|
||||||
|
I would say, if the block is being evaluated interactively then
|
||||||
|
lets pop up the image in a new window, and if it is being exported
|
||||||
|
then we can just include a link to the file which will be exported
|
||||||
|
appropriately by org-mode.
|
||||||
|
|
||||||
|
*** non-graphics files
|
||||||
|
? We link to other file output
|
||||||
|
*** side effects
|
||||||
|
If we are using a continuous process in (for example an R process
|
||||||
|
handled by ESS) then any side effects of the process (for example
|
||||||
|
setting values of R variables) will be handled automatically
|
||||||
|
|
||||||
* Objectives
|
Are there side-effects which need to be considered aside from those
|
||||||
** Send data to R from org
|
internal to the source-code evaluation process?
|
||||||
Org-mode includes orgtbl-mode, an extremely convenient way of using
|
|
||||||
tabular data in a plain text file. Currently, spreadsheet
|
** reference to data and evaluation results
|
||||||
functionality is available in org tables using the emacs package
|
I think this will be very important. I would suggest that since we
|
||||||
calc. It would be a boon both to org users and R users to allow
|
are using lisp we use lists as our medium of exchange. Then all we
|
||||||
org tables to be manipulated with the R programming language. Org
|
need are functions going converting all of our target formats to and
|
||||||
tables give R users an easy way to enter and display data; R gives
|
from lists. These functions are already provided by for org tables.
|
||||||
org users a powerful way to perform vector operations, statistical
|
|
||||||
tests, and visualization on their tables.
|
It would be a boon both to org users and R users to allow org tables
|
||||||
|
to be manipulated with the R programming language. Org tables give R
|
||||||
|
users an easy way to enter and display data; R gives org users a
|
||||||
|
powerful way to perform vector operations, statistical tests, and
|
||||||
|
visualization on their tables.
|
||||||
|
|
||||||
|
This means that we will need to consider unique id's for source
|
||||||
|
blocks, as well as for org tables, and for any other data source or
|
||||||
|
target.
|
||||||
|
|
||||||
*** Implementations
|
*** Implementations
|
||||||
**** naive
|
**** naive
|
||||||
|
@ -151,59 +182,44 @@ vice versa.
|
||||||
**** RweaveOrg
|
**** RweaveOrg
|
||||||
NA
|
NA
|
||||||
|
|
||||||
|
*** reference format
|
||||||
|
This will be tricky, Dan has already come up with a solution for R, I
|
||||||
|
need to look more closely at that and we should try to come up with a
|
||||||
|
formats for referencing data from source-code in such a way that it
|
||||||
|
will be as source-code-language independent as possible.
|
||||||
|
|
||||||
** Evaluate R code from org and deal with output appropriately
|
*** source-target pairs
|
||||||
*** vector output
|
|
||||||
When R code evaluation generates vectors and 2-dimensional arrays,
|
|
||||||
this should be formatted appropriately in org buffers (orgtbl-mode) as well
|
|
||||||
as in export targets (html, latex)
|
|
||||||
|
|
||||||
Agreed, if we can convert the vector data to lists then we can use
|
|
||||||
the many orgtbl-to-* functions to convert the list to whatever
|
|
||||||
output format we desire. See `orgtbl-to-orgtbl, `orgtbl-to-latex',
|
|
||||||
`orgtbl-to-html', `orgtbl-to-csv', etc...
|
|
||||||
|
|
||||||
**** Implementations
|
|
||||||
***** org-R
|
|
||||||
org-R converts R output (vectors, or matrices / 2d-arrays) to an
|
|
||||||
org table and stores it in the org buffer, or in a separate org
|
|
||||||
file (csv output would also be perfectly possible).
|
|
||||||
***** org-exp-blocks
|
|
||||||
***** RweaveOrg
|
|
||||||
*** graphical output
|
|
||||||
R can generate graphical output on a screen graphics device
|
|
||||||
(e.g. X11, quartz), and in various standard image file formats
|
|
||||||
(png, jpg, ps, pdf, etc). When graphical output is generated by
|
|
||||||
evaluation of R code in Org, at least the following two things are desirable:
|
|
||||||
1. output to screen for immediate viewing is possible
|
|
||||||
2. graphical output to file is linked to appropriately from the
|
|
||||||
org file This should have the automatic consequence that it is
|
|
||||||
included appropriately in subsequent export targets (html,
|
|
||||||
latex).
|
|
||||||
|
|
||||||
**** Implementations
|
The following can be used for special considerations based on
|
||||||
***** org-R
|
source-target pairs
|
||||||
org-R does (1) if no output file is specified and (2) otherwise
|
|
||||||
***** org-exp-blocks
|
|
||||||
org-exp-blocks tries to do 2, but I don't think that part was
|
|
||||||
every really working
|
|
||||||
|
|
||||||
***** RweaveOrg
|
**** source block output from org tables
|
||||||
|
**** source block outpt from other source block
|
||||||
|
**** source block output from org list
|
||||||
|
**** org table from source block
|
||||||
|
**** org table from org table
|
||||||
|
**** org properties from source block
|
||||||
|
**** org properties from org table
|
||||||
|
|
||||||
|
** caching of evaluation
|
||||||
|
|
||||||
** Evaluate R code on export
|
I'm personally not clear on how this would be implemented, but it does
|
||||||
At first I was leaning towards leaving the exporting to Sweave, but it
|
seem to be important. I'd be interested to hear how Sweave
|
||||||
seems that once we have evaluation or R working, it will not be
|
accomplished this. Should it be based on tracking changes in source
|
||||||
difficult to implement full evaluation of R blocks, one-liners, and
|
blocks.
|
||||||
creation of R graphics on export directly in elisp.
|
|
||||||
|
|
||||||
I think that this would be worth the effort since it would eliminate
|
** export
|
||||||
the need for using Sweave, and would allow for exportation to any
|
once the previous objectives are met export should be fairly simple.
|
||||||
target format supported by org-mode.
|
Basically it will consist of triggering the evaluation of source code
|
||||||
|
blocks with the org-export-preprocess-hook.
|
||||||
|
|
||||||
|
This block export evaluation will be aware of the target format
|
||||||
|
through the htmlp and latexp variables, and can then create quoted
|
||||||
|
=#+begin_html= and =#+begin_latex= blocks appropriately.
|
||||||
|
|
||||||
|
|
||||||
* Notes
|
* Notes
|
||||||
** Special editing and evaluation of source code in R blocks
|
** Special editing and evaluation of source code
|
||||||
Unfortunately org-mode how two different block types, both useful.
|
Unfortunately org-mode how two different block types, both useful.
|
||||||
In developing RweaveOrg, a third was introduced.
|
In developing RweaveOrg, a third was introduced.
|
||||||
|
|
||||||
|
@ -214,7 +230,20 @@ target format supported by org-mode.
|
||||||
|
|
||||||
Note that upper and lower case are not relevant in block headings.
|
Note that upper and lower case are not relevant in block headings.
|
||||||
|
|
||||||
*** PROPOSED R-block proposal
|
*** block headers/parameters
|
||||||
|
regardless of the syntax/format chosen for the source blocks, we will
|
||||||
|
need to be able to pass a list of parameters to these blocks. These
|
||||||
|
should include (but should certainly not be limited to)
|
||||||
|
- label of the block
|
||||||
|
- names of file to which graphical/textual/numerical/tabular output
|
||||||
|
should be written
|
||||||
|
- flags for when/if the block should be evaluated (on export etc...)
|
||||||
|
- flags for how the results of the export should be displayed/included
|
||||||
|
- flags specific to the language of the source block
|
||||||
|
- etc...
|
||||||
|
|
||||||
|
*** block format
|
||||||
|
**** PROPOSED R-block proposal
|
||||||
I (Eric) propose that we use the syntax of source code blocks as they
|
I (Eric) propose that we use the syntax of source code blocks as they
|
||||||
currently exist in org-mode with the addition of *evaluation*,
|
currently exist in org-mode with the addition of *evaluation*,
|
||||||
*header-arguments*, *exportation*, *single-line-blocks*, and
|
*header-arguments*, *exportation*, *single-line-blocks*, and
|
||||||
|
@ -250,7 +279,7 @@ currently exist in org-mode with the addition of *evaluation*,
|
||||||
What do you think? Does this accomplish everything we want to be able
|
What do you think? Does this accomplish everything we want to be able
|
||||||
to do with embedded R source code blocks?
|
to do with embedded R source code blocks?
|
||||||
|
|
||||||
**** src block evaluation w/org-eval-light
|
***** src block evaluation w/org-eval-light
|
||||||
here's an example using org-eval-light.el
|
here's an example using org-eval-light.el
|
||||||
|
|
||||||
first load the org-eval-light.el file
|
first load the org-eval-light.el file
|
||||||
|
@ -268,7 +297,7 @@ function through the `org-eval-light-interpreters' variable.
|
||||||
date
|
date
|
||||||
#+end_src
|
#+end_src
|
||||||
|
|
||||||
*** Source code blocks
|
**** Source code blocks
|
||||||
Org has an extremely useful method of editing source code and
|
Org has an extremely useful method of editing source code and
|
||||||
examples in their native modes. In the case of R code, we want to
|
examples in their native modes. In the case of R code, we want to
|
||||||
be able to use the full functionality of ESS mode, including
|
be able to use the full functionality of ESS mode, including
|
||||||
|
@ -290,7 +319,7 @@ a
|
||||||
,## hit C-c ' to exit the temporary buffer
|
,## hit C-c ' to exit the temporary buffer
|
||||||
#+END_SRC
|
#+END_SRC
|
||||||
|
|
||||||
*** dblocks
|
**** dblocks
|
||||||
dblocks are useful because org-mode will automatically call
|
dblocks are useful because org-mode will automatically call
|
||||||
`org-dblock-write:dblock-type' where dblock-type is the string
|
`org-dblock-write:dblock-type' where dblock-type is the string
|
||||||
following the =#+BEGIN:= portion of the line.
|
following the =#+BEGIN:= portion of the line.
|
||||||
|
@ -302,11 +331,11 @@ a
|
||||||
#+BEGIN: dblock-type
|
#+BEGIN: dblock-type
|
||||||
#+END:
|
#+END:
|
||||||
|
|
||||||
*** R blocks
|
**** R blocks
|
||||||
In developing RweaveOrg, Austin created [[file:existing_tools/RweaveOrg/org-sweave.el][org-sweave.el]]. This
|
In developing RweaveOrg, Austin created [[file:existing_tools/RweaveOrg/org-sweave.el][org-sweave.el]]. This
|
||||||
allows for the kind of blocks shown in [[file:existing_tools/RweaveOrg/testing.Rorg][testing.Rorg]]. These blocks
|
allows for the kind of blocks shown in [[file:existing_tools/RweaveOrg/testing.Rorg][testing.Rorg]]. These blocks
|
||||||
have the advantage of accepting options to the Sweave preprocessor
|
have the advantage of accepting options to the Sweave preprocessor
|
||||||
following the #+BEGIN_R declaration.
|
following the #+BEGIN_R declaration.
|
||||||
|
|
||||||
** Interaction with the R process
|
** Interaction with the R process
|
||||||
|
|
||||||
|
@ -326,5 +355,33 @@ the best of each of these approaches.
|
||||||
* Tasks
|
* Tasks
|
||||||
|
|
||||||
|
|
||||||
* buffer dictionary
|
* COMMENT Commentary
|
||||||
|
I'm seeing this as like commit notes, and a place for less formal
|
||||||
|
communication of the goals of our changes.
|
||||||
|
|
||||||
|
** Eric <2009-02-06 Fri 15:41>
|
||||||
|
I think we're getting close to a comprehensive set of objectives
|
||||||
|
(although since you two are the real R user's I leave that decision up
|
||||||
|
to you). Once we've agreed on a set of objectives and agreed on at
|
||||||
|
least to broad strokes of implementation, I think we should start
|
||||||
|
listing out and assigning tasks.
|
||||||
|
|
||||||
|
** Eric <2009-02-09 Mon 14:25>
|
||||||
|
I've done a fairly destructive edit of this file. The main goal was
|
||||||
|
to enforce a structure on the document that we can use moving forward,
|
||||||
|
so that any future objective changes are all made to the main
|
||||||
|
objective list.
|
||||||
|
|
||||||
|
I apologize for removing sections written by other people. I did this
|
||||||
|
when they were redundant or it was not clear how to fit them into this
|
||||||
|
structure. Rest assured if the previous text wasn't persisted in git
|
||||||
|
I would have been much more cautious about removing it.
|
||||||
|
|
||||||
|
I hope that this outline structure should be able to remain stable
|
||||||
|
through the process of fleshing out objectives, and cashing those
|
||||||
|
objectives out into tasks. That said, please feel free to make any
|
||||||
|
changes that you see fit.
|
||||||
|
|
||||||
|
|
||||||
|
* Buffer Dictionary
|
||||||
LocalWords: DBlocks dblocks
|
LocalWords: DBlocks dblocks
|
||||||
|
|
Loading…
Reference in New Issue