FWIW, Python has multiprocessing.Pool which theoretically allows you to dole out work to a subprocess pool: http://docs.python.org/2/library/multiprocessing.html#module-multiprocessing.pool

Unfortunately, the implementation in Python 2 (and possibly still Python 3) has some pretty nasty gotchas in that it doesn’t handle signals properly (http://www.bryceboe.com/2012/02/14/python-multiprocessing-pool-and-keyboardinterrupt-revisited/). So, you inevitably need to roll your own version.

For the temporary file, check out tempfile.NamedTemporaryFile. e.g.

with tempfile.NamedTemporaryFile() as tf:
tf.write(…)
tf.flush()

path = tf.name
# Execute your command

# When the context manager (with) goes out of scope, the temp file is unlinked. Yay context managers!

I think this make-based solution abuses make because you aren’t really using make for dependencies. I think a tool like GNU Parallel is much better suited for the task at hand. That being said, the abuse isn’t severe, make is a tool we have readily available in the build system, and, most importantly, it works. Great work!

The jstests.py harness has parallel execution capability. It may be unnecessary given your make-based approach, but it does work. It also deals with timeouts and parallel output.

There’s some ugly work in between here and merging the two test suites, though.