Software Carpentry

Helping scientists make better software since 1997

Archive for the ‘Version 3’ Category

Feedback and Boundaries

Thanks to the initiative of Dominique Vuvan (who took Software Carpentry last summer), we ran a semi-formal version of the course from last November through to this past week for grad students in Psychology, Linguistics, and a few other disciplines at the University of Toronto. Weekly tutorials were offered in both Python and MATLAB by graduate teaching assistants from Computer Science, covering roughly half of the existing material.

Read the rest of this entry »


Written by Greg Wilson

2010/04/04 at 01:29

Posted in Toronto, Version 3

How We Got Here, and Where We Are

I gave a talk in Santa Fe early in 1997 describing a set of articles I’d organized for the Summer 1996 and Fall 1996 issues of IEEE Computational Science and Engineering (now Computing in Science & Engineering on the subject, “What should computer scientists teach physical scientists and engineers?” After the talk, John Reynders (then the director of the Advanced Computing Lab at Los Alamos National Laboratory) challenged me to put my money where my mouth was and actually teach basic software development skills to working scientists.

Brent Gorda and I ran the course for the first time in July 1997. We used Perl as a programming language, and covered topics such as CVS, regular expressions, and a little bit of web client programming. Our part of the course was three days long, and was followed by a two-day consulting visit from Steve McConnell (whose books Code Complete and Rapid Development were at the top of the charts). We ran the course in various forms another five or six times in the next three years, during which time we switched to Python and expanded it to five days. All told, about 120 LANL technical staff went through the course, most of them under 35.

In 2004, after I’d taught the course for the Space Telescope Science Institute and the US Navy, the Python Software Foundation gave me a grant to reorganize, update, and expand the material. That version is the core of what’s now online; when I last checked, the site was getting 10-12,000 distinct visitors a month, and the material was being used in whole or in part at Caltech, Indiana, several schools in the UK and Germany, Chile, South Korea, and of course here in Toronto.

Based on follow-ups with alumni, I’d guess that it has no effect at all on 20-25%, who take the course because their manager or supervisor told them they had to, and get little out of it. The rest routinely describe it as game changing: a PhD candidate in Psychology who did the course with us in July 2009 told me a few days ago that what she learned probably saved her six months on her current project, and that without it, a second project would simply not have occurred to her to try. As another data point, one of the other alumni of that offering came to me early in October to say that several of her labmates wanted to take the course, and was I planning to offer it again any time soon? I told her that I wasn’t, but that I could arrange for a CS grad student to teach it. Three weeks later, 65 students from Pysch and Linguistics had signed up to do it as a non-credit course, roughly 45 of whom have stuck with it so far.

While I don’t have data to back this up, I believe very strongly that what most students get out of the course isn’t specific knowledge about relational databases, regular expressions, or object-oriented programming, but rather a mental map of the computing landscape, so that they know what’s supposed to be easy, what else is supposed to be possible, and where to go looking for more information. Another student from the July 2009 offering said that the biggest thing the course did for him was turn “unknown unknowns” into “known unknowns”. I’m supposed to conduct a follow-up survey with those students later this month to see how much they’re using what they learned, and what impact is has had; I hope to have results up on the web by Easter.

And as regular readers will know, I’m presently trying to raise money to update the material: this post explains the background, while this plan incorporates what I’ve learned from students and instructors on four continents about what material, sequence, and presentation will actually “reach” scientists. Sadly, though, funding agencies and companies mostly still seem to think that only HPC-related training is worth funding, which I feel is asking scientists to run before they can walk. This CiSE paper talks about this particular frustration, while our survey results put weight behind the claim that the overwhelming majority of scientists will benefit much more from being helped with development issues than from anything to do with big iron.

Written by Greg Wilson

2010/01/10 at 18:11

Thanks, Jamie

The warmup tutorials for our grassroots Software Carpentry course started this week, and we’d like to send a “thank you” to Jamie Winter at The MathWorks, who has provided students with temporary licenses for MATLAB. It’s all been very last minute, and we’re grateful to Jamie for pulling this off on such short notice.

Written by Greg Wilson

2009/11/28 at 01:31

Tutorials Start This Week

After a lot of hard work from Dominique and Jon, we’re kicking off warmup tutorials for Software Carpentry this week. 65 students from Psychology, Linguistics, Chemical Engineering, and a couple of other departments will get three weeks of review on basic programming, then start the regular material in January. Our thanks to MITACS, the MathWorks, SciNet, and DCS for their support.

Written by Greg Wilson

2009/11/24 at 13:35

Videos from Symposium Are Now Online

I have put video recordings of the guest talks given at our July 29 symposium on Science 2.0 — please click the titles of the talks on the symposium page to view them, or go to the (very simple) index page on my departmental site. (I’ll figure out a better hosting solution after Canadian Thanksgiving is out of the way, but didn’t want to delay posting the talks any longer.) Thanks again to all of our speakers, to the MaRS Centre technicians for the raw recordings, and to Tanya Murdoch for editing.

Written by Greg Wilson

2009/10/08 at 18:57

Posted in Content, Version 3

Guest Speakers’ Slides Now Available

Our guest speakers’ slides are now on the web; video will follow soon.

Written by Greg Wilson

2009/08/03 at 13:19

Posted in Content, Version 3

Next Steps

It’s clear from Friday’s end-of-course review that the course needs shaking up. Before that starts, though, there’s a higher-level question to answer: should the course notes be converted to a wiki to encourage contributions from others? It was always my hope that other people would contribute material, but in four years, only five ever have; perhaps wikification would change that.

Right now, the notes are stored as HTML pages in a Subversion repository and compiled by a little Python script to resolve cross-references, insert code samples, and so on. The advantages of this approach are:

  1. People can work locally and push coordinated changes when ready.
  2. Slide format can be skinned by changing a flag in the Makefile to select different CSS. (For example, I’m still hoping to get S5 or S5R working.)
  3. The build step can also insert code fragments, ensure that bibliography references resolve, etc.

Advantages of a wiki are:

  1. Easier collaboration: people can make small fixes in place without doing an “svn checkout” or running Make.

As a programmer, the first three weigh heavier in my mind than the last one, but again, only five people have contributed material in four years, which isn’t sustainable. What do you think?  Would switching to a wiki make you more likely to add material or not?

Written by Greg Wilson

2009/08/02 at 13:06

Posted in Content, Tooling, Version 3