Complacency in the computer industry - a rant.

    By Julian Smith. 2006 Oct.

I think the computer industry (both closed and open-source) is
exhibiting an astounding lack of understanding about the serious
problems that plague it. There is a passive acceptance of complexity and
duplication that makes working in this industry deeply unsatisfying.

An example: C++. A while ago, I decided to stop attending the UK C++
panel's meetings, because I couldn't see anything fundamentally new
being done. Instead, adding fairly simple features (such as initialisers
or chained-constructor) to C++ was requiring enormous amounts of time
and energy from equally enormously talented and dedicated people,
because C++ is such an insanely complicated language. And this
despite the best efforts of Stroustrup et al to keep C++ as simple as
possible. The classic example is the issue of name-lookup. In C++, name
lookup is simply ridiculously complicated - I would be very surprised
if even a dozen people in the whole world understand it properly (I am
certainly not one of them).

To help navigate the complexity that today's programming involves, we
turn to books. There are some excellent ones such as Meyers' Effective
C++/STL that stand out amongst the rubbish that is deposited on the
industry, but I wonder whether Meyers and others are actually doing us
a disservice. In effect, they make it possible for us to muddle along
with current languages and development systems when we should really be
throwing them out and doing things properly.

I've thought for a long time that Effective C++ is basically an
admission that C++ is a mess. That C++ is a mess is not a particularly
novel idea of course, it's well known and there are good reasons for why
it is as it is. Meanwhile, Effective XYZ is written to explain how to
avoids the pitfalls in the language and new kludges are invented (I'm
afraid I include Boost's various meta-programming libraries here). But
it's not as though there are any significantly better alternatives
around, and it's not just programming languages that are rubbish. More
generally, there are hundreds of incompatible build systems around,
we're stuck with an antiquated hierarchical filesystem design, and
computers still go wrong far too often. Books spend hundreds of pages
talking about the importance of using abstract interface classes and
making only leaf-classes concrete, despite this involving so much boiler
plate code that your header files double in size. Object orientation
with methods owned by classes was proclaimed for years as the solution
to all programming problems, despite manifestly not working in many
situations; whole books are written explaining how to hack classes
with methods so that they don't bite you too often or too badly. The
existence of the horrific GNU autoconf/automake tools allows Unix
vendors to get away with putting their header files in different places
and offloading the resultant incompatibilities and needless complexities
onto developers.

Every so often, I read something about Smalltalk or Lisp systems that
had development systems that appear to be everything that we want today,
have wanted for years but still don't have, but were around *decades*
ago. Things like incremental compilation, modification of running code,
reflection etc were around and understood then. Meanwhile, this sort
of thing simply cannot be done in C++ because the grammar is almost
impossible to parse in batch fashion, never mind in an incremental
way. Java was a massive opportunity, but appears to simply have made a
different but roughly equal-sized set of mistakes. On the plus side,
languages like Python seem rather well designed and potentially allow
one to spend the majority of one's effort solving real problems, but I
wonder whether they are really much better than the state of the art 20
years ago?

I think what I'd like to see is some admission from the industry (and
that includes us) that the current state of affairs is a massive
embarrassment to us all. I'd like it to be written in something very
large and brightly-coloured that somehow things have gone badly wrong
when writing a database system for a few million tax payers in the
UK costs millions of pounds, and doesn't actually work, despite the
fact that *all* of the fundamental technical problems of concurrency,
transactions etc were *solved* decades ago.

What should happen when a problem is solved (i.e. understood) in
computing, is that the solution should be disseminated, and we should
all move on to new and interesting things. Instead, we see people saying
`wow' when a java applet displays an animation. People had the same
reaction twenty years ago when BBC micros or Spectrums displayed the
same rotating cube. I worked for a company a few of years ago where we
were trying to write a system where a web page writen in HTML contained
some javascript, which talked to a java programme which talked to
some C++ which made a COM call across a network to a server. It was a
nightmare involving thousands of lines of hand-written code, but the
actual problem being solved was mind-numbingly simple. I fear that
thousands of people are engaged in similarly pointless activities, when
they could be doing something new and/or useful.

Consider the rise of GNU/Linux and the BSDs. Many people find that a
Linux server is much easier to maintain than a Windows one; things
like Samba and Apache work fairly well and have allowed Linux/BSD to
be adopted significantly. But have a close look at Linux (or any other
unix). I think it's incredibly crude. After modifying a configuration,
you have to manually restart the relevant process. You have to manually
set up some sort of backup system, none of which ever do exactly what
you want. There is an increasingly complicated package system to avoid
the DLL-hell problem. If you want to maintain previous versions of
files, you have to setup and use CVS or subversion, which is hardly
straightforward. KDE and Gnome look nice and work fairly well, but they
take enormous amounts of resources and fundamentally don't do anything
new compared to Windows or Apple's equivalents. Being sure that a Unix
system is safe against instrusion is distinctly non-trivial.

I can hear people replying with `but you can do that with xyz', `choice
is important', `if you don't know what you're doing, you shouldn't be
doing it'. My answer is that ordinary non-technical people want to
do all of the things mentioned in the previous paragraph, but they
cannot. I want to do them too, but they are a solved problem, so I
should not need to spend time on them - I'd rather spend time trying
to work on something new or different, not being the ten-millionth
person to read the cvs man page and master its idiosyncrocies. Windows,
to its credit, tries to simplify these tasks, but we all know that it
doesn't work particularly well either. Francis Glassborow's recent book
starts from the premise that anyone should be able to write a computer
programme. Similarly, anyone should be able to use computers safely and
efficiently without having a nightmare getting access to old emails when
transferring from one computer to a new one.

The rant is almost over. I shall finish by claiming that what we really
need is to start over. Yes, really start over. Forget about the books
that talk about how to avoid programme crashes by never deriving from
a class that doesn't have a virtual destructor, or having containers
of pointers. Design a language where the compiler figures out what
to do in order to be safe. If you give it a decent grammar, you can
spend all the man years that would be spent on a C++ parser, on code
analysis that generates faster code than the equivalent C++. Forget
about teaching people about the importance of saving regularly under
numbered filenames in case something crashes or you make a mistake;
instead, design a filesystem where everything is automatically version
controlled and backed up. Hey, you could even get rid of the Save
button! Rather than writing yet another build system, try to think more
generally and consider what a dependency tree is and how it could be
integrated with the filesystem/backup system; consider what ccache does,
and think how it could be generalised. Don't write or use yet another
Widget system, try to abstract things a lot more - consider how widets
in a window are grouped, but so are files in a directory, or emails from
a particular person, and try to figure out an abstraction that reflects
these similarities.

I have a few ideas about these issues, some of which I'm developing at
the moment. It's not easy, and there are many things that I don't know
how to do properly, but I think that the important thing is to realise
that things could be so much better, otherwise we are stuck in a world
where eventually nothing new will ever be thought about, let alone done.

[Julian's non-programming interests include violin playing and cycling
in the himalays (http://op59.net/, jules@op59.net). He recently
co-founded Undo Software (http://undo-software.com/) where he works on
the undodb bi-directional debugger.]