Friday, July 15, 2011

Losers assemble, real men compose

There are basically two approaches to software development. Assembling the code by cherry-picking pre-made components or writing nearly everything from scratch. As usually, both have their cons and pros.

Assembling is suitable for undermanned, time-or-otherwise pressed projects or simply for projects with a rather narrow focus which is noticeable even on the base functionality level. By reusing third-party components, these projects avoid the Not-Invented-Here syndrome, which may be a good thing because it makes other people's work worthier and saves the project's own resources. On the other hand, it may be a bad thing too as the extensive use of non-homebrew base components destroys the identity of the accepting project and can introduce bloat in the form of various compatibility and glue layers, as well as the very much undesired burden of maintaining a large foreign codebase and its bugs.

For example, it turns out that our companion multiserver projects, MINIX 3 and GNU Hurd, already have or soon will have some parts assembled from:
  • old Linux networking stack (GNU Hurd)
  • old Linux device drivers (GNU Mach)
  • old Linux file system drivers (GNU Hurd)
  • L4's DDE framework for using contemporary Linux drivers (GNU Hurd and MINIX 3)
  • lwIP networking stack (MINIX 3)
  • NetBSD standard C library and base utilities (MINIX 3)
This bears questions like whether MINIX 3 without its traditional userland will still continue to be MINIX 3 or rather it becomes a NetBSD distro, and gives us some idea about why are some GNU Hurd's and GNU Mach's components stuck in times of Linux 2.2 or even older.

The other extreme is HelenOS, also an undermanned multiserver OS project and one in which I am actually involved. We tend to be affected by NIH and often find ourselves to be reinventing the wheel, but at the end of the day, here we go with our own implementations of the above components (networking stack, filesystems, drivers and device driver framework, and libc). Sure, they may not be perfect today or tomorrow, but they are native and unique to HelenOS and remain under our control and in our custody.

4 comments:

Dominik said...

Though it is nice to stay unique and such, sometimes it is very contra-productive. Just look at device drivers - why would you write essentially the same code again? You can just adapt some other work to your needs. There's no wheel to be reinvented, but simply used.

Jakub Jermar said...

Well, we are also trying to avoid bloat. Some bloat would certainly come in the essence of compatibility and accommodation layers if we were to use some third-party drivers that only work with a third-party device driver framework.

So what you say is true, but only for a tiny piece of each third-party driver, which is device-specific. The rest of the code is OS-specific and would complicate things. In case of some non-portable OSes, the rest may be even platform-specific, which is even more evil.

Dominik said...

I'm with you on bloat, but are you sure there's so much of it or are you just guessing? I mean, nowadays everyone wants to be cool and "in" so everyone adopts object oriented design - I don't see why the driver would be bloated if it just provided interface to its device driver framework?

Truth is, though, I have never written any device driver, so I am guessing... :-)

Jakub Jermar said...

For an interesting insight of a another project, which actually has a real-world experience with using Linux DDE, take a look at a release notes section of Genode 12.05.