News

Orgmode and Roundup: Bridging public bugtrackers and local tasklists

cwebber, November 10th, 2010

So maybe you’re already familiar with the problem. You’re collaborating with other people, and especially if you’re in a free software environment (but maybe even some install at your work) you have some bugtracker, and that’s where everyone collaborates. But on the other hand, you have a life, your own todo systems, your own notes, etc. Even for the tasks that are on the bugtracker, you might keep your own local copy of that task and notes on that task. Eventually things start to get out of sync. Chaos!

Wouldn’t it be great if you could sync both worlds? Keep the notes that are relevant to being public on the public bugtracker, but keep private notes that would just clutter up the ticket/issue/bugreport private. Mesh the public task system with your private task system. Well, why not?

So this was the very problem I’d run into. I have my work bugtracker for here at CC, our install of roundup, and then I have my own TODO setup, a collection of Org-mode files.

There are a lot of things I like about org-mode. It’s in emacs (though there’s apparently a lean vim port in the works), it’s plaintext (which means I can sync across all my machines with git… which I do!), tasks are nested trees / outlines (I really tend to break down tasks in very granular fashions as I go so I don’t get lost), notes are integrated directly with tasks (I take a lot of notes), and it’s as simple as you need it or as complex as you want to get (I started out very simple, now my usage of org-mode is fairly intricate). It also does a good job of spanning across multiple files while still retraining the ability to pull everything together with its agenda, which is useful since I like to keep things semi-organized.

And of course, the relevant file here is all my Creative Commons stuff, which I keep in a file called ccommons.org. There’s a lot of private data in here, but I’ve uploaded a minimalist version of my ccommons.org file.

So! Syncing things. If you open the file in an emacs version with org-mode installed, you’ll notice 4 sections. Two of these are crucial to my setup, but we won’t be using them today: “Events” holds say, meeting at X time, traveling on certain days; “Various Tasks” contains not roundup-related tasks. Then there’s the other two: “Roundup” will collect all the tasks we need to work on, and “Supporting funcs” has a couple of org-babel blocks in Python and emacs-lisp.

Anyway, enough talk, let’s give it a spin. You’ll need a recent version org-mode and a copy of emacs. Make sure that newer org-mode is on your load-path and then evaluate:

(require 'org)
(require 'org-install)
(require 'ob-python)
(setq org-confirm-babel-evaluate nil)
(setq org-src-fontify-natively t)
(setq org-src-tab-acts-natively t)

Next open up the relevant org-mode file. Move to the “Roundup” line, hit Tab to cycle its visibility, and move to the line that starts with “#+call:”

Now press “Ctrl+c Ctrl+c”. You’ll see it populate with issues from my issues list:

What’s happening here? So we’re executing an org-babel block at point. Org-babel is an org-mode extension that allows you to make blocks of code executable, and even chain from one language to another (it also has some stuff relevant to Donald Knuth’s “literate programming” which is cool but I’m not using here). If we look at the code blocks:

Anyway, there are three code blocks here.

  • ccommons-roundup-parse: uses python to read the CSV file generated by roundup which is relevant to my task list, converts it into a list of two-item lists (task id, task title)
  • ccommons-roundup-insert-func: the function that actually inserts items into our “* Roundup” heading. It checks the ROUNDUPID property to see if that task is already inserted or not. If not, it inserts the task with the appropriate title and ROUNDUPID.
  • ccommons-roundup-insert the actual block we end up invoking. It binds together the data from ccommons-roundup-parse with a function call to the function defined in ccommons-roundup-insert-func.

You can evaluate it multiple times. It’ll only insert new tasks that aren’t on your list currently. Now you can take notes on your tasks, schedule them for various dates, make subtasks, etc. When you’re ready to close out a task close it out both on the ticket and in org-mode. If you want to use a similar setup for org-mode, I think it’s easy enough to borrow these methods and just change the CSV URL to whatever URL is appropriate for your user’s tasks.

Now admittedly this still isn’t even the best setup. It would be good if it told you when some tasks are marked as closed in your org-mode and open in roundup and vice versa. Org-babel still feels a bit hacky… I probably wouldn’t use it on anything other than scripts-I-want-to-embed-in-my-orgmode-files (for now at least). I even had to strip out quotes from the titles because org-babel python doesn’t escape quotations from strings correctly currently (but that’s a bug, one that will hopefully be fixed). Even so, I’ve been trying to close out a lot of roundup tasks lately, and it’s really helped me to bridge both worlds.

Edit: And in case you’re wondering why I didn’t use url.el instead of piping to python, the reason is because of CSV support… there’s none builtin to emacs as far as I know, and splitting on commas doesn’t handle all of the escaping intricacies… and org-babel makes it pretty easy to be lazy and simply use python for what python already handles well.

5 Responses to “Orgmode and Roundup: Bridging public bugtrackers and local tasklists”

  1. Christian Egli says:

    Why didn’t you just use org-feed.el (see http://orgmode.org/worg/org-contrib/org-feed.php)?

  2. cwebber says:

    Hello Christian,

    Org-feed looks cool, I haven’t tried it. But I don’t believe roundup (or at least the version of roundup we have installed) has feed support though, so I couldn’t subscribe to an RSS or an Atom feed, hence parsing a CSV here. For other bugtrackers which do provide feeds though, that’s probably good enough. Anyway, I will look into using org-feed for some other things! :)

  3. Paul Mantz says:

    There is a csv.el that handles the reading of commented entries if you’d like to make your script pure elisp: http://ulf.epplejasper.de/downloads/csv.el

  4. Gergo says:

    Great article, this looks promising, and is the closest match to what I’m looking for: importing data into Redmine from a plain text todo list (can easily convert my files and my self to orgmode).
    It seems the other way around than your solution, but I guess it’s still doable?

    Later, I’d like two way synchorisation, with Redmine&Bugzilla for bulk editing and “refactoring” of the tasks.. Fortunately I could use proper APIS to communicate with those services.

    You mention “babel feels hacky” – but could it still be best (easiest,cheapest) way to start? Thanks for the tips anyway!

  5. cwebber says:

    Paul Mantz: Thanks for pointing me toward csv.el… if I ever do make it pure elisp, I’ll certainly use that!

    Gergo: Sure, I think org-babel is a pretty nice way to spec these things out and embed tools like this into and out of orgmode. It’s a good way to make a hacky first start anyway that you could work into something more refined later anyway. If you did it in pure elisp, I guess there’s no difference between using org-babel and just writing it in a separate file, except for the convenience of “bundling” it with that file. However, if external languages like python/ruby make interfacing with your bugtracker of choice easier babel is kind of nice for piecing things together fast.

    I think that even if you ended up making the bridge pure elisp, it’s kind of nice to have an src block to call the main function from that you can just evaluate with C-c C-c on point.

Leave a Reply