Software Carpentry

Helping scientists make better software since 1997

Archive for September 2009

A Strange Obsession

I just spent a few minutes browsing these slides from a talk given in Karlsruhe in September by Microsoft’s Fabrizio Gagliardi.  The talk’s title was “Cloud Computing for Scientific Research”, and it’s chock-full of big: mega-this, peta-that, and isn’t it all exciting? The only mention of anything at the desktop scale is on slide 19, which mentions a plugin to allow MATLAB to talk to the cloud, and Excel views of Azure data. Once again, I’m puzzled (and a bit disappointed) that the world’s premier desktop software company has decided to ignore what most scientists care about most. I’m equally disappointed that there was nothing at all in these slides about improving scientists’ skills, especially since Microsoft has invested so heavily in improving its own processes. Oh well…


Written by Greg Wilson

2009/09/29 at 15:14

Posted in Opinion

Presentation, Presentation, Presentation

Right now, the Software Carpentry material is basically printed pages on the web. Each lecture is a linear HTML page: bullet point follows bullet point, interrupted only by code snippets, tables, and diagrams. If I’m going to update the content, I’d also like to update the presentation; the question is, “To what?” An audio recording of me talking over the slides would add some value, though I think that typing in what I would say would probably be more useful, since most people can read faster than I can speak, and audio still isn’t googleable.

I’ve also thought about recording screecasts (audio on top of a video recording of my computer desktop). That would allow me to show live coding sessions, which I think many students would find valuable. Flipping that around, I could embed small snippets of video in the HTML pages. Then there are tools like Crunchy that allow you to create tutorials by embedding snippets of Python in web pages. That could help the programming parts of the course, but not with version control, Make (if we stick to Make, which I hope we don’t), or many other parts.

So: what’s the best online tutorial you’ve ever seen? What made it the best? Do you know how much effort it took to build the first time? How much effort it would take to build once the authors were experts in [name of tutorial-building technology goes here]?  Pointers would be very welcome…

Written by Greg Wilson

2009/09/24 at 13:17

Posted in Tooling, Version 4

Grant Proposal

Rebuilding the Software Carpentry course is too big to do in anyone’s spare time, so I’ve submitted a small funding application (included below). If you know any venues that might welcome a similar proposal, I’d be grateful for pointers.

Read the rest of this entry »

Written by Greg Wilson

2009/09/22 at 16:40

Posted in Version 4

Another Reason to Care About Provenance

A vice president at ETH is resigning today because of accusations of fraud.  A key lab notebook is missing, retractions have been retracted, and according to ETH President Ralph Eichler, “there is now no legal way of finding out for sure who was responsible for the falsifications.”

Question: if someone accused you of falsifying results, how well and how easily could you defend yourself? How long would it take you to pull together all the notes, data, and programs you used three years ago to produce the paper being challenged?  And would you career survive?  Even if you “won”, you would probably lose weeks or months of research time.

I think this is one of the strongest arguments in favor of using data provenance systems.  Young scientists’ lives are difficult enough (see for example Peter Lawrence’s recent PLoS Biology article); an accusation of fraud, well-intentioned or otherwise, could effectively destroy someone’s  career even if it was unfounded. Using computers to create an audit trail is just good insurance…

Written by Greg Wilson

2009/09/21 at 13:07

Posted in Noticed, Opinion

Updated Outline for Revised Course

I have updated the description of how I plan/hope to reorganize the course.  My thanks to everyone who commented on the earlier draft; I’d be very grateful for feedback on this one as well—I realize that some of the lectures are still hopelessly ambitious, but I hope it’s at least a target to shoot at.

Written by Greg Wilson

2009/09/18 at 17:36

Posted in Version 4

Partial Outline of New Version of Course

I have put a mostly-complete draft of my ideas for a rewrite of the course online at  This version would start with specific problems, then backfill the tools and skills needed to solve them, but there are some big outstanding issues:

  1. There still isn’t anything on the computational thinking skills that Jon Udell identified.  I don’t know how to tackle these without relying more than I want to on closed-source offerings like Yahoo! Pipes that could disappear at a moment’s notice.
  2. I just can’t see how to fit some topics—particularly object-oriented programming and web programming—into the available space (a maximum of two hours of lecture and four hours of lab for each). What are the minimum useful problem/lecture for each?

There are a few others as well (just search for ‘TODO’), but those are the big ones.  Any help you can give would be very welcome.

Written by Greg Wilson

2009/09/15 at 17:53

Posted in Version 4

Job Opening: MITACS Scientific Coordinator

MITACS (which provided funding for this summer’s offering of the course, and which funds a lot of other mathematically-oriented work in Canada) has an opening for a scientific coordinator.  Details are in the job posting; specific responsibilities include:

  1. Promote MITACS programs within the scientific community and provide a point of contact for feedback and questions
  2. Ensure quality, fairness and timeliness of scientific peer-review processes
  3. Program management tasks such as:
    • Establish program strategy and action plans for new MITACS programs
    • Develop and monitor program budgets
    • Coordinate with other MITACS staff on program implementation and ongoing monitoring of program effectiveness
    • Work closely with the Communications and Government Relations Department on program promotion and feedback
    • Provide regular status updates, reports and program evaluation
  4. Other duties as assigned from time to time

If you’d like to help shape the development of mathematics and science in Canada, this might be the job for you…

Written by Greg Wilson

2009/09/11 at 18:39

Posted in MITACS