darcs hacking sprint 3 report

The Darcs Hacking sprint was co-located in Vienna, Austria and Portland, Oregon, USA.
The Vienna team had a unique experience as the sprint happened to coincide with a student protest that resulted in an occupied University building. Great fun was had by all as we shared our hacking space with the Occupiers on Saturday.
The Portland team was graciously hosted by Galois. As a special treat, we got a visit from Thomas Hartman, one of the folks behind Patch-tag.

Overview

Among other things, the upcoming Darcs 2.4 release has some nice performance improvements from Petr Ročkai's hashed-storage work.
One first goal in this sprint was to polish up this work, making our hashed repository support good enough for GHC team to upgrade from the deprecated old-fashioned format. We think we've largely accomplished what we set out to do. Read on for more details!
Our second goal was to provide a space for new Darcs hackers to get started with making contributions to the project. We had 14 new Darcs developers at this sprint! We hope that they will stay on with the project and continue submitting patches.

Issues resolved

Note that some of this work is still pending patch review or amendements.

• issue540 - darcs remove --recursive - Roman Plášil
• issue835 - show files with arguments - Luca Molteni
• issue1122 - get --complete UI - David Markvica
• issue1208 - trackdown --bisect - Matthias Fischmann, Radoslav Dorcik
• issue1224 - darcs convert on darcs-2 repositories - Tomáš Caithaml
• issue1377 - hardcoding of darcs executable name - Benedikt Huber
• issue1392 - use Parsec for authorspellings - Tomáš Caithaml
• issue1394 - show time elapsed after each test - Kim Wallmark
• issue1499 - versioned show files - Thomas Hartman
• issue1500 - misleading darcs progress reports - Roman Plášil
• issue1624 - break up global cache into subdirectories - Luca Molteni
• issue1643 - optimize --upgrade should do optimize - Christian Berrer

Hacking

Studying darcs

Thomas DuBuisson and Jason Dagit studied some hanging merge cases on the GHC repository to see if we could garner some insights into them. While this did not yield much fruit, it did result a few code cleanups.
Jonathan Daugherty and Josh Hoyt took a tour of the Darcs source. We hope they'll be joining us as Darcs hackers in the future :-)
Jason Dagit produced a concrete and low-level description of the Darcs record command.

Code cleanups

As a warm-up exercise, everybody worked on tidying the Darcs and hashed-storage source, eliminating annoying GHC warnings and implementing suggestions from hlint.


Tomáš Caithaml simplified the darcs show authors authorspellings code by rewriting it in Parsec.
Jason Dagit cleaned up the elegant_merge function and removed commutex in favor of commute.
Jonathan Daugherty submitted some improvements to the user manual.

 

 

Performance improvements

Luca Molteni improved Darcs's handling of the global cache, splitting it into buckets for better performance.

User interface improvements

Tomáš Caithaml got darcs convert to abort in case user accidently tries to convert a darcs 2 repository a second time.

Roman Plášil improved the user experience behind Darcs progress reporting replacing misleadingly definitive text with a more accurate description as shown in the following example :

                           OLD TEXT    NEW TEXT
Fetching pristine cache... 4/45        4 done, 41 queued
(sometime later)...        10/166      10 done, 155 queued
(finally)...               297/297     297 done

Christian Berrer extended the darcs optimize --upgrade command to also act as plain old darcs optimize in addition to upgrading the repository format.
David Markvica modified darcs get --complete so that it no longer offers to create a lazy repository.
Benedikt Huber relaxed the assumption that darcs executable would be called "darcs", instead fetching this from the command line.

New features

Matthias Fischmann and Radoslav Dorcik worked on a new trackdown --bisect feature for darcs trackdown which makes trackdown more useful for larger sets sets of patches.
Roman Plášil implemented an oft-requested remove -r feature, making it easier to undo accidental adds of large directories.
Luca Molteni implemented darcs show files with an argument to limit the output to a specific file or directory.
Thomas Hartman implemented matchers support for darcs show files, allowing to get the list of files tracked by Darcs at a given point in the history (eg. darcs show files --tag 2.4).
Petr Ročkai worked on a new more extensible, self-documenting format mechanism as a future alterantive to _darcs/format.

UTF-8 Support

Reinier Lamers continued his work on storing darcs patch metadata in UTF-8. He worked on autodetecting UTF-8 content (in the patch metadata) and found some subtle bugs in the utf8-string library along the way.

 

SVN integration

The Vienna local team, Christian Berrer, Thomas Danecker and David Markvica, worked on SVN integration and studied the libsvn directory in great detail.

 

 

Zero Windows Bugs!

Salvatore Insalaco slayed the 18 Windows bugs that resulted from our recent hashed-storage work. This means that we can can definitely release the new hashed-storage work in Darcs 2.4.

 

 

 

 Darcs Team infrastructure

Joachim Breitner made some improvements to the darcswatch core and user interface. He also added hooks to Darcswatch for integration with Darcs Team patch tracker. Joachim Breitner, Eric Kow, Petr Ročkai also added some polish to patch tracker, in particular to its email gateway interface and interaction with the darcs-users mailing list. The result of these efforts is a more efficient Darcs Team with simpler and smoother patch review process.
Kim Wallmark worked on making the 'cabal test' friendlier by outputting the amount of time elapsed after long tests.

Discussions

Priorities for the Sprint

Salvatore Insalaco, Eric Kow, Reinier Lamers, Luca Molteni and Petr Ročkai discussed the key goals for the sprint and also produced a list of the top ProbablyEasy.

 

 Type witnesses

Jason Dagit gave a presentation about our type-witnesses, how the type hackery in darcs is implemented, and why it is useful.

Patch-tag

Thomas Hartman and the Portland crew discussed patch-tag: how it felt from a user perspective and what the most useful features would be.

SVN integration

Thomas Danecker, Eric Kow, Petr Ročkai, Ganesh Sittampalam (IRC) discussed the SVN integration project: what the roadmap should look like and how this feature might fit into Darcs, whether as a fully integrated feature, a plugin or a standalone application.

Petr Ročkai and Eric Kow also discussed the nature of patch/changeset based revision control (like Darcs) and snapshot based revision control (everything else). Petr observed that changeset based version control can be seen as just snapshot based control with a fixed diffing algorithm. To support things like SVN integration, Darcs will need to freeze its diffing algorithm.

Darcs roadmap

Salvatore Insalaco, Eric Kow, Reinier Lamers and all Vienna hackers developed the roadmap for Darcs 2.4 (January 2010) and Darcs 2.5 (July 2010). Using a chalkboard we listed the features we wanted, how desirable they were and what order we should do them in.

Windows support

Salvatore Insalaco, Eric Kow and Reinier Lamers discussed the future of Windows support in Darcs.
Official binary: Our first priority will be to get an official binary for Windows downloadable from Darcs.net. The upcoming Darcs release (Darcs 2.4 in January 2010) will the first to provide an official binaries. Currently, we have the resources to provide a binary for Windows only, but in the future the list of platforms may grow.
Refined integration: After the Darcs 2.4 release, we will work on developing a friendly Windows installer for Darcs which includes not just the Darcs binary but (optionally), TortoiseSSH for better ssh support and a tool for sending email via darcs send.
Cygwin support: We will not be able to support Cygwin, but we will work on providing more explicit documentation on what works (mostly everything) and what does not (absolute cygwin paths). We think that the improved Windows installer will provide a sufficiently comfortable Darcs experience for most Cygwin users.

Fun and speculation

Joachim Breitner, Reinier Lamers and Petr Ročkai discussed abstracting over patch types to just maps (eg. directories), sets and lists (eg. files).

Better Darcs Hacking Sprints

Every sprint teaches something new for what we hope to be a long tradition of biannual Darcs Hacking events.

One thing which worked out well this year was that we provided a clean list of the best ProbablyEasy bugs for new hackers to work on. This allowed new developers to make highly desirable contributions from the very beginning. This year the list was built with live discussion, a few Darcs Team members huddled in a hostel room over an open bug tracker. Perhaps we can replicate the success by having similar pre-sprint meetings in the future.
Replicating success is one thing; how can we do even better? Mainly we need to improve the experience that new hackers have. We were lucky this year to have many new Haskellers and Darcs developers, but we could have done a better job in helping them to get started.
First, some technical issues: A lot of our new developers lost the better part of Saturday morning setting up the machines to build Darcs and to send patches to the list. In the next sprint, we will be ready with much more precise and detailed setup instructions (developed hastily during this sprint) going all the way to the best default settings for Darcs, configuration files for msmtp (or similar sendmail replacements). It also has been suggested that we provide a virtual machine image for instant Darcs hacking.
Second, our mentoring strategy: We were fairly successful at providing in-depth individual mentoring; however what we could done better was to provide more mass mentoring at the very beginning, to help developers send their first patches. To make this work, we should try starting the next sprint with a group mentoring session with the specific purpose of getting people set up to send their first patches. We should encourage new developers to work in small teams, for example with an extra USB keyboard to facilitate pair programming on a single machine.

Attendees

Vienna

1. Joachim Breitner
2. Benedikt Huber
3. Eric Kow
4. Reinier Lamers
5. David Markvica
6. Petr Rockai
7. Radoslav Dorcik
8. Salvatore Insalaco
9. Matthias Fischmann
10. Thomas Danecker
11. Christian Berrer
12. Luca Molteni
13. Roman Plášil
14. Tomáš Caithaml
15. Pivo

Portland

1. Jason Dagit
2. Josh Hoyt
3. Thomas Hartman
4. Thomas DuBuisson
5. Kim Wallmark
6. Jonathan Daugherty

Thanks!

Many thanks to the local team at TU Vienna and Galois for a wonderful welcome! Thanks to the many generous donors who helped us to subsidise travel to the sprint. Thanks also to Microsoft Austria for sponsoring the sprint with drinks and snacks. Molte grazie to Salvatore Insalaco for a wonderful lunch on Sunday.
Finally, thanks to Petr and Luca for the photos, and more generally to everybody who participated in the sprint! It was great to have you and we hope to see you again either on the mailing list or at future sprints.

See you in March!

The fourth Darcs Hacking Sprint will be taking place in Zürich on 19-21 March 2010 as part of the Haskell Hackathon. Hope to see you there!