Getting XML Output from Python Unit Tests with unittest2 and pyjunitxml

I use Jenkins for continuous integration. (If you've not heard of Jenkins, but you've heard of Hudson, they're basically the same thing.)  Jenkins an amazingly powerful (yet easy to use) piece of software that you can set up to build and test your code every time you check in. This immediate feedback is really useful so you can detect and fix problems right away rather than waiting for your co-workers (or your customers!) to find them.

Jenkins can be set up to run Python unit tests and track test failures over time. The tricky part is that you need to be able to run your tests in such a way as to produce XML output. nose is an extension to the built-in Python unittest library that provides numerous plugins to handle things like XML output and tracking code coverage.

I've been using the unittest2 module for writing my unit tests. Basically, it's a backport of the new and much-improved unittest library that's available for Python 2.7, making it available for Python 2.4, 2.5 and 2.6. Sadly, nose does not work well with unittest2. There is the beginnings of a nose2 that depends on a new plugin-capable branch of the unittest2 module, but as of now there's nothing ready for "production". So how do people get there unittest2 tests to run with XML output?

I asked about this on the Testing in Python mailing list, and I was pointed at pyjunitxml. This module basically implements a new TestResult class that can be used in place of the default unittest TextTestResult class to write XML output. It still needs an wrapper script to set up the result and run all of your tests, so it's not quite the solution I was looking for, but it was a start.

I took pyjunitxml and added my own command-line script to run named unit tests or hook in to unittest2's test discovery. It works with Python 2.4, 2.5, 2.6, 2.7, 3.1 and 3.2, and it works with or without the unittest2 package installed (although you won't get test discovery without it on older Python versions). (I used Jenkins to test all of those combinations at once!) It's currently available from my branch, but hopefully it'll get merged into the pyjunitxml trunk soon. Feel free to try it out, and please give me any feedback you might have!

Flymake and Pyflakes with TRAMP

When I get everything figured out, I'll post more of my Emacs config, but here's the bit to get flymake and Pyflakes working.

I don't remember exactly why I needed all this magic, but I think the flymake-create-temp-intemp function is needed to get Flymake to work with TRAMP; it makes flymake put the temporary file on the local machine instead of the remote one, where Pyflakes (running locally) can see it. The rest is just standard stuff to tell flymake to use Pyflakes with Python code.

;; pyflakes
;; adapted from http://plope.com/Members/chrism/flymake-mode
;; and http://www.emacswiki.org/emacs/FlymakeRuby

(defun flymake-create-temp-intemp (file-name prefix)
  "Return file name in temporary directory for checking FILE-NAME.
This is a replacement for `flymake-create-temp-inplace'. The
difference is that it gives a file name in
`temporary-file-directory' instead of the same directory as
FILE-NAME.

For the use of PREFIX see that function.

Note that not making the temporary file in another directory
\(like here) will not if the file you are checking depends on
relative paths to other files \(for the type of checks flymake
makes)."
  (unless (stringp file-name)
    (error "Invalid file-name"))
  (or prefix
      (setq prefix "flymake"))
  (let* ((name (concat
                (file-name-nondirectory
                 (file-name-sans-extension file-name))
                "_" prefix))
         (ext  (concat "." (file-name-extension file-name)))
         (temp-name (make-temp-file name nil ext))
         )
    (flymake-log 3 "create-temp-intemp: file=%s temp=%s"
   file-name temp-name)
    temp-name))

(when (load "flymake" t)
  (defun flymake-pyflakes-init ()
    (let* ((temp-file (flymake-init-create-temp-buffer-copy
         'flymake-create-temp-intemp))
    (local-file (file-relative-name
   temp-file
   (file-name-directory buffer-file-name))))
      (flymake-log 3 "flymake-pyflakes-init: dir=%s %s"
     buffer-file-name (file-name-directory temp-file))
      (list "pyflakes" (list local-file)
     (file-name-directory temp-file))))

  (add-to-list 'flymake-allowed-file-name-masks
        '("\\.py\\'" flymake-pyflakes-init)))

(add-hook 'find-file-hook 'flymake-find-file-hook)

Emacs, Tramp and Python auto-completion

At work, I use Aqauamcs Emacs on my OS X laptop for editing code that runs on FreeBSD and Linux servers in our colo. Since the files are remote (as is the environment for running them), I use TRAMP to let me conveniently edit these files from my laptop. Before TRAMP, I used sshfs to achieve the same goal, but I ran into issues where files would get corrupted, and I haven't tried it since.

I have a few useful customizations for Python editing: flymake and Pyflakes support to flag the more egregious errors, and highlight-80+-mode to show me when my lines get too long. But it still leaves quite a bit to be desired.

About once a year, I get all ambitious and try to get auto-completion working for Python code in Emacs. This generally results in me searching the internet endlessly, installing a number of packages that ultimately don't work, contemplating fixing some of them until I realize I don't know Emacs Lisp, and then promising myself I'll learn it. (I never do.) This generally results in at least 2 days of wasted effort, and I still don't have auto-completion.

It seems that pretty much everyone out there uses Pymacs and pycomplete or ropemacs to do this, but unfortunately Pymacs only runs on the local host. (I've patched it to get it to run on a remote host, but it's not a great solution as it can only run on one machine.) All these tools get confused by TRAMP filenames, so all in all its a pretty ugly situation.

This year, I made a little bit of progress. I found auto-complete mode. By default, it doesn't give you anything fancy for Python editing, but it will automatically complete other words you've used in the same buffer. As it turns out that's most of them, making this is a great 90% solution. It'd be nice to get completion that's a little more semantically-aware, but that's not so easy.

I also installed yasnippet, which is a pretty powerful tool that can save you some typing and integrates somewhat with auto-complete. Getting auto-complete and yasnippet to play nicely together was a bit tricky; I ended up having to change yasnippet's key bindings to conflict less with auto-complete.(Otherwise, the behavior I'd get depended on whether I waited long enough for auto-complete's completion window to pop up. Yikes!)

All in all, I'd say I've still got some work to do to make my Emacs setup as efficient as I'd like, but this week's efforts have definitely helped quite a bit!