Notes on org agenda files for batch exporting

Org-agenda-files

In this project, we use org-agenda-batch-csv to export events in our agenda to a csv file. This allows easy manipulations with other tools like Pandas. To enable testing, we created a demo org file. The question is: How do you pass this file to org-agenda-batch-csv?

Setting org-agenda-files parameter

Org-agenda-batch-csv is meant to be launch from emacs batch mode. That is, you call emacs from command line and pass an expression to evaluate.

emacs -batch -l ~/.emacs.d/init.el -eval '(org-batch-agenda-csv "a" org-agenda-span 3000 org-agenda-time-grid nil org-agenda-files (quote ("/Users/username/SET_YOUR_ROUTE/TO/org-timeline-viewer/timestamp-test1.org")))' > tmp_agenda.csv

Trying to get the previous command, we tried several options and discovered the following details.

Unquoted file as parameter

According the org-agenda.el source you can pass parameters unquoted.

;;; Commentary:
;; This file contains the code for creating and using the Agenda for Org-mode.
;;
;; The functions `org-batch-agenda', `org-batch-agenda-csv', and
;; `org-batch-store-agenda-views' are implemented as macros to provide
;; a convenient way for extracting agenda information from the command
;; line.  The Lisp does not evaluate parameters of a macro call; thus
;; it is not necessary to quote the parameters passed to one of those
;; functions.  E.g. you can write:
;;
;;   emacs -batch -l ~/.emacs -eval '(org-batch-agenda "a" org-agenda-span 7)'
;;
;; To export an agenda spanning 7 days.  If `org-batch-agenda' would
;; have been implemented as a regular function you'd have to quote the
;; symbol org-agenda-span.  Moreover: To use a symbol as parameter
;; value you would have to double quote the symbol.
;;
;; This is a hack, but it works even when running Org byte-compiled.

Checking the org-agenda-files variable documentation:

If an entry is a directory, all files in that directory that are matched
by ‘org-agenda-file-regexp’ will be part of the file list.

If the value of the variable is not a list but a single file name, then
the list of agenda files is actually stored and maintained in that file,
one agenda file per line.  In this file paths can be given relative to
‘org-directory’.  Tilde expansion and environment variable substitution
are also made.

org-agenda-files unquoted points to:

• A directory containing org-files
• A file with a list of files (one agenda file per line)

Neither of these options would work for a single org file.

Quoted file as parameter

Org-manual includes an example of quoted file as parameter:

emacs -batch -l ~/.emacs                                      \
   -eval '(org-batch-agenda "a"                               \
           org-agenda-span (quote month)                      \
           org-agenda-include-diary nil                       \
           org-agenda-files (quote ("~/org/project.org")))'   \
   | lpr

In that example, the tilde is expanded as per the documentation. Route might be also org-directory at ~/org/.

Note that only tilde expansion and environmental variables work. Relative routes like ./file.org or ../dir/file.org will not expand. This will result in an empty output. When no route is specified, instead of opening the file in the current directory, emacs appends org-directory to the given filename, provoking another empty output.

Known errors

Error 1

./emacs -batch -eval '(org-batch-agenda-csv "a" org-agenda-span 10 org-agenda-files "/Users/metamaking/dev/org-timeline-viewer/timestamp-test1.org")' > tmp_agenda.csv  

Non-existent agenda file ~/org/* TODO Demo scheluded date with tag :tag1:.  [R]emove from list or [A]bort?

Explanation: org-agenda-files is interpreted as a file that includes a list of agenda files. emacs reads the first line of the file and tries to find a file with that name. Given the first line is a TODO item, the process will fail. It will also fail for all the other lines.

Solution: add a quote block around the full route.

Error 2

./emacs -batch -eval '(org-batch-agenda-csv "a" org-agenda-span 10 org-agenda-files "/Users/username/dev/org-timeline-viewer/agenda-list.txt")' > tmp_agenda_list.csv 

Non-existent agenda file ~/org/timestamp-test1.org.  [R]emove from list or [A]bort?

Content of agenda-list.txt is timestamp-test1.org

Explanation: Given no route is given for timestamp-test1.org emacs assumes it is stored under org-directory. File is under current directory so the process fails producing the agenda. Solution: add full route to agenda items in agenda-list.txt

Error 3

./emacs -batch -eval '(org-batch-agenda-csv "a" org-agenda-span 10 org-agenda-files "./timestamp-test1.org")' > tmp_agenda_list.csv 

Explanation: unquoted files are interpreted as dir, or lists of files under org-directory. Local route is not expanded and emacs fails to open the file. No output file is produced. Solution: use the full route and quote it.