Goals and Optimism
Blake Winton and I talked yesterday about Basie’s future—more specifically, about whether Basie has a future, given that this will be the fifth (!) summer that students have been working on “an entry-level project portal for undergraduate software engineering classes” (the original pitch from 2004), and we still only have a handful of users outside Toronto. I think that taking part in these kinds of discussions is as important to the education of junior developers as taking part in discussions about design, so I’d welcome feedback from everyone on the team.
The first (and most important) rule is this: never make decisions about the future of a project based on how much you’ve already put into it, since that effort is “sunk” (i.e., there’s no way to recover it). As painful as it may be to walk away from months or years of work, the only sensible way to decide on the future of something is to compare how much investment is still needed to the likelihood of, and payoff for, success.
The second rule is, you have to know what success looks like before you can decide if you’re going to be able to get there. For me, success for Basie means:
- Make project forges common in undergraduate software engineering education. Victory here would be having Basie used as widely as DrJava, i.e., have hundreds of schools adopt it. (This goal inspired the name “DrProject”, by the way.) This is why I keep trying to emphasize ease of installation, and why I think that support for any version control system other than Subversion is premature something-or-other (though as Blake pointed out, given our progress to date, Subversion could well be regarded as a legacy system by the time we release).
- Replace Trac as an entry-level forge for non-academic use. There are at least thousand of Trac installations worldwide; DrProject is already better in some respects (native support for multiple projects and email, for example), but has lagged behind in many others, so it hasn’t been “better enough” to make inroads.
- Give some students hands-on mentoring in real-world software development. We’ve already accomplished this for a handful of students.
- A research platform. My plan when I came to U of T was to have grad students add new and nifty features to DrProject, then study their impact. I think there’s still a need for this, but lack of funding, plus lack of interest on the part of my grad students and other faculty, pretty much nixes it.
In the spirit of complete honesty, I should add a few personal goals:
- Authorship. I also dreamed a couple of years ago (when I was teaching U of T’s software archtitecture course, since discontinued) about using DrProject/Basie as a worked example—sort of what OS/161 is for operating systems courses, or any number of free Pascal compilers were for compiler courses in the 1980s. That goal has long since fallen by the wayside, but I still hope to write an introductory software engineering textbook that’s more practical than the two leaders in the area by combining material from Software Carpentry and my CSC301 class, using Basie as a running example of both how to design and build things, and what tools developers should use. Doing this is essential to #1.
- Self-education. I don’t actually know how to write a modern web application; I keep hoping I’ll be able to find (OK, make) time to get into coding for Basie so that I can learn. #5 is more important, though: I can get students to write code, but not to write books.
- Ego. I’ve written a lot of software in the last 25 years, but none of it has been widely used. As a matter of personal pride, I’d like to have my name on something that thousands of people care about before I retire.
So, what are our chances? Slim. A few hours a week each term from a bunch of students, plus some full-time effort each summer, hasn’t been enough to get DrProject or Basie to the level needed for #1 or #2. There’s no realistic prospect of more effort becoming available (i.e., of funding suddenly materializing), so the only hope is to make Basie attractive enough for other developers to start contributing effort in the traditional open source manner. The revised 2.0 feature list was made up with that in mind: by late July, we have to have something that at least a few current Trac developers and users will like enough to invest in.
As far as the other goals go, #5 is doable without #1, but would be less compelling (the more things instructors have to cobble together in order to use a book, the less likely they are to adopt it). #3 (educating students) can be satisfied at least as well by following in David Humphrey’s footsteps and putting students to work on “real” open source projects, though that would require some retooling effort on the part of instructors.
Finally, I should answer the question I always ask students at the end of the term: if I could send an email back in time, what would I tell my earlier self to do differently? The answer here is, “Don’t fork Trac—join it.” We could have added external authentication, role-base authorization, and email to Trac in the summer of 2005, which would have given us something we could use for senior projects. If we had then moved it from hand-rolled SQL to SQLAlchemy during the Fall 2005 and Winter 2006 terms, we could have had enough credibility in the Trac community to propose native multi-project support in the summer of 2006. Yes, I would have had to do a lot more negotiating, and some or all of the changes needed for academic use might have been refused, but students would have learned at least as much (again using David Humphrey’s work at Seneca with Mozilla as a model), and we’d probably have gotten farther, sooner. I think DrProject’s source is much cleaner than Trac’s these days thanks to my decision to fork, but most potential users don’t care about that: they care about what it can do.
So does this mean I’m pessimistic? Yes. Am I gloomy? No. Most startups and open source projects fail; if you don’t go into them knowing that, you’re in for a world of hurt. Whatever happens, I’m going to have fun, learn a lot, and hopefully make a few friends. And hey, maybe the horse will learn how to whistle after all…
Your thoughts?
Your post has left me sad. We are one of those “handful of users outside Toronto”. We use DrProject to document (wiki) and keep track of tasks (tickets) of half a dozen projects at the library (mostly involving the Computer Center) of my university (Universitat Autònoma de Barcelona). During the last years, we have tested different combinations of Usemod wiki and Roundup, and none of them worked well.
Then I tried Trac, but we had a number of problems with the (lack of) multiproject support (http://trac.edgewall.org/ticket/130), confusing user identification and login, the tickets are too focused on software development, etc. etc.
Until we found DrProject mentioned on that #130 ticket. It works really smoothly for us. The only trouble has been to know which release is the stable one (we use 1.2), and the migration path (there is a 2.0-dev, a 3.0-dev and Basie). I tried (and failed) to install 2.0-dev on Debian Stable, and that’s why we are stuck on 1.2.
Other than that, we have found nothing that is so flexible to accomodate our workflow. Librarians and people from the Computer Center have been using it for more than a year without much trouble. It works fast on Apache, scgi and SQLite.
The Django path seems logical to me, although I cannot comment much more on that, because I don’t know much either about modern web development. What we would really troubles us is not knowing if the project will survive, and how.
Thanks for all those nice work you have done, and please do not let it die. ;-(
Ferran
Ferran Jorba
3 May 09 at 5:54 pm
Ferran,
how would you like to take over maintainership of the project? I bet Greg would be happy to continue hosting it for you, at least for a while…
(I’m only half-joking here. It seems to me that the project is going to need a new maintainer in the next year or two no matter what Greg decides next term. Do any of the people who read this blog feel like stepping up to the plate?)
I’ve reached the point in my life where I’m starting to cut things out which show no signs of going somewhere. As I’m fond of saying, I recently crossed Scheme off of my list of things to get more familiar with, because I just don’t have the time to do everything anymore, and I have to start to concentrate my efforts.
So, given that point of view, I have to ask where Greg should spend his effort? Should it be on a project with a slim chance of achieving its goals, or should it be on something like Software Carpentry which seems to be doing more good for more people? I vote for the latter, if only because it seems like a better use of his time and money.
(Of course, that also begs the question of where I should be spending my time. I enjoy the stuff I’m doing on Basie, and since I have no long-term plans for it, I don’t mind spending time on it for as long as Greg wants to run it.)
Later,
Blake.
bwinton
3 May 09 at 10:27 pm
Hang on a sec — I’m not *quite* ready to throw in the towel just yet
We’re closer to having something that meets my goals than were were in September or January, and with three full-timers this summer (two of whom already know the code base), we’re still in the game. And anyway, I think Blake would be the logical person to take it over… 
Greg Wilson
3 May 09 at 10:41 pm
I didn’t say you were going to throw in the towel, just that I think you should.
We are closer, in terms of a product, but we don’t seem any closer in terms of a community. Apart from me, how many people are volunteering on Basie? How about DrProject? How many patches have we received from third-parties? One of the problems of last term was people not being willing to put half-baked patches up on ReviewBoard. Maybe it’s time to put a half-baked project portal into the public eye. As long as you can show progress, and have _something_, I don’t think it’ll hurt, and having more people look at it might even help. (Sure, more funding isn’t going to be available, but maybe, just maybe, some community resources could be.)
Finally, about me taking over, you might think I was the logical choice, but with two young children, and more other personal projects than I can shake a stick at, I don’t think I’ll be taking this one on as well.
(If I were going to do something like this, I’ld use CouchDB, and JQuery, and Mercurial. Not that I’ve been thinking about it or anything…
)
bwinton
4 May 09 at 12:14 am
Three years ago I had the opportunity to attend a course out of my university at 8pm every Tuesday. Its name was “Course of Programming using Free Software” and it was imparted only that year by members of Cuba’s Free Software Community. That was my first encounter with subvertion, trac and unit test and I had liked to hear about them from my first months in the university. From that moment I took the deployment of some forge in my university as a personal project so that Computer Science students were able to use it (with the hope that it could cause in the future their use in the small companies that develop software in Cuba).
I first tried with Gforge and Savane but I didn’t succeed. In the summer I started to learn about Django developing at home a project to handle multiple trac sites and finally the next course my boyfriend sent me DrProject’s url. At that time DrProject 2.0 was available and I ended modifying it to install it here. A modified version of DrProject 3.0 have been instaled here during more than a year but a regular course like Software Carpentry will be neccesary to surpass the 20 undergraduate projects hosted in it.
I agree with Blake in that Software Carpentry is more important than the development of Basie and can contribute a lot in make project forges common in undergraduate software engineering education. Though even if Greg has to spend lest time in Basie the project will not die so quickly and there is still a change that it can survive. At least I don’t want to throw in the towel yet
I also agree with Blake about the “half-baked project portal”.
zuzelvp
4 May 09 at 11:28 am
(Somehow my previous comment didn’t get published. I’ll write it again.)
@Blake: I’m really sorry, but I just don’t have the skills nor the time to maintain something like DrProject.
But if my opinion counts as maintainer of a local DrProject instance, having and easy installation is a must. I tested Trac just typing `apt-get install trac’, but DrProject was quite more involving, specially because some of the dependencies are not packaged for Debian. Considering that the closest alternative (Redmine) cannot be easily packaged (http://changelog.complete.org/archives/701-at-long-last-softwarecompleteorg-migrated-to-redmine), and that not everybody has the permissions, skills or time to install arbitrary pieces of sorftware, don’t underestimate the easiness of installation to improve DrProject-Basie user base. In the same vein, I wasn’t unable to install 2.0-dev when I tried, due to some dependency I could not resolve easily.
In that respect, I’d say that the Django path seems a good one, given that according to a post by Greg, Django provides most of the things you had to look for elsewhere with DrProject. Django and some Django extensions are already in Debian, so that would make it easier to reach many potential users.
Ferran Jorba
4 May 09 at 5:51 pm
Just out of curiosity, what is done in terms of marketing to achieve #1? Is it only the occasional word of mouth? Looking back to my undergraduate project-based software dev class, I would have loved to have something like DrProject.
Liz Blankenship
4 May 09 at 11:45 pm
[out of topic]
Ferran:
I have a DrProject 3.0 installed in a server with etch (main and contrib). If you want to try again to move to 2.0 or 3.0 maybe I can help you. Write me to zuze a_t uh.cu if you are insterested.
zuzelvp
5 May 09 at 8:22 am
Hola Zuzel,
as a matter of fact, as we are upgrading hardware and software, and migrating from Debian Etch (4.0) to Lenny (5.0), one of the things in my to do list is to update our DrProject, but probably not earlier than in a month, after the main application in that server is already migrated.
Gracias,
Ferran
Ferran Jorba
5 May 09 at 9:06 am
Hola Ferran,
Then let me know in a month
Suerte
zuzelvp
5 May 09 at 11:43 am
@Liz: Yeah, that’s my bad — I kept holding off on announcing DrProject to the world because I wanted a decent installer, better documentation, etc. Should have pushed it at the Trac community in 2007… If the Big Seven are working in July, we’ll launch Basie then (we’ll just call it an alpha
.
Greg Wilson
5 May 09 at 11:43 am
Hi,
We’re running an instance of DrProject 3.0 with our company, and we’re quite pleased with it overall. We initially were using Trac, but needed to easily support more than one project per instance. We’ve got over a dozen projects running with our instance. Granting selective access to the projects is another vital feature provided by DrProject.
The only big missing feature from our point of view is being able to search across all the projects that a user has permission for. Support for other VCSs (especially DVCSs) would be cool, but Subversion is OK for now.
I haven’t read through the blog for all the motivations behind the Django re-write (though I hear that Django is cool).
The time I can spend maintaining the existing DrProject 3.0 code base is minimal at best. But at a last resort, please contact me if it is necessary to shut down the existing site. I can scrape up a server to host it.
James Graves
ansible@xnet.com
James Graves
14 May 09 at 3:35 pm