Forgotten software development tasks

It allways seems to be the same. Again I've heard about two different software development projects which are more in trouble than needed. The first one has no real requirements engineering and an unorganized process. There's a continous flow of new requirements which also flow unbundled, unpriorized, and continous into the development. As a result the software updates are short planned and testing is difficult. That's a noxious environment for the software quality and for the developers. I don't know very much about this project, but I've once been responsible to cleanup a project in an almost similar situation. The first step has been the establishment of an iterative process with timeboxes. I've choosen 6 weeks, but this depends on the individual project situation. The second step was the installation of my request management system, a simple web application for the management of requirements and issues. Finally I've set up a process of collecting and documenting the requirements and their dependencies, priorized them, and published the features of the next three releases - with decrescent likeliness. The result has been a steady development with predictable results for our customers. Additionally we've been able to do a better resource planning so that a part of the team had the time to dig into the sources and improve them.

The second project I've been told about had two cost estimations for the finalization. One seems to be relative realistic, following the opinion of the team members. It has been done through the current project manager, who has a very good reputation in his team. But the estimation is also high. The other one has been done through the his predecessor. His estimation is much lower. So even if he has been replaced due to his flop before, the management now follows him. The lower costs are much more convenient. I can't believe it, what's happening there? Doing software cost estimation is one of my interests. And I always get relative high numbers using the function point analysis and COCOMO II. The people around me are very often scared about those numbers and so they are sometimes not accepted. But when a project moves towards the end I often see that the estimated effort is almost accurate. So it makes no sense to tell oneself a lie.

Both cases show how software development consist of more than just coding. Capers Jones analyzed a lot of projects and categorized them into six different types: web, MIS, outsourcing, commercial, system, and military. These types have diferent numbers of activities and percentages of work effort per activity. The minimum number is 7 activities, the maximum 25. Take a look at the table inside the referenced document. It show's very good what to keep in mind when realizing a software project. And even if it is unpleasing it is just the truth.

Keep it simple in Scheme

One features of the new Tideland Dynamic Content Processor - and also of the Tideland Train of Thoughts - will be the usage of a simple markup language, which is a kind of XML, but without attributes and less bloated through the usage of a Lisp-like style.

{document
{table-of-contents}
{section Introduction
  {p The new {strong Tideland Dynamic Content Processor} is a
     lightweight content management system developed in Scheme. The data
     is stored in SML files.
  }
}
}

My first approach in Ruby some time ago ended in a DOM-like implementation with classes for the document, for elements, and for text. The first reimplementation in Scheme used the same approach based on the class system of PLT Scheme. But I soon discoverd that this way isn't natural for Scheme. It works, yes, but Scheme OOP isn't as simple and straight as in Smalltalk.

So I switch to a simple structure, a kind of a list with fields for tag, id, parent, and childs, together with a set of procedures. They allow to easily create a document tree, navigate through it, or search nodes. What astonished me most was how small this solution is. I'm doing OO for over 20 years now. This formed my development style to think in frameworks immediately. Scheme now refocusses my mind to always look for possible simpler solutions.

Juggling multiple projects

Sometimes I'm wondering about myself how I'm handling all my different projects. Writing articles and book reviews for computer magazines, writing articles in my wiki (in German) and this blog, developing different software projects in Smalltalk, learning Scheme and realizing a first project with it, maintaining my server, and reading a lot. And all that beside my family and my job. Colleagues and friends also asked me allready.

But now I'm reading "The Myths of Innovation" by Scott Berkun. It's the second of his books I'm reading and even if I'm just at the beginning I can say that it is a very good book. And one part handles the importance of idling during the process of being creative. This pause is needed to allow the subconscious mind to think about the topic and to build the right idea out of the stuff developed before. This pause can be - and typically should be - really lazy. But sometimes switching the concentration to a different topic is also a kind of idling, related to the original topic.

And that's right, I often have the feeling that the work on a new project or with a new technology makes my mind free for the longer running projects. And afterwards I'm more productive. So everything is fine, I can go on, and I don't have to care. *smile*

Beyond the mainstream

During my IT career I've seen very much decissions made due to the fact, that the winning technology or system is mainstream.The simple argumentation is "Everybody is using it, so it has to be good.", sometimes with the small qualification "OK, we know that it isn't the best one, but you can get people with know-how for it very simple.". That's partly right, but not in total.

If a technology or system is mainstream there are really many people which are engaged with it. But there are two limitations: There is also a huge demand for those people and the quality of their know-how is varying extremely. So you may get some staff but it is hard to predict how productive they will be. There's no guaranty. But the main objection is the fact of loosing any unique selling point. Maybe you're in a kind of business where you don't need this, you are forced to use mainstream technology due to customer requirements. But if the right mix of flexibility and speed is needed to compete with your business rivals or to enable your customer to do so you can't just follow the masses.

Today very often applications are developed using JEE or .NET, spiced with tons of XML files, and garnished with huge relational database management systems from Oracle, IBM or Microsoft. I don't want to say that there are no needs for such solutions. But they only cover a special segment. However, due to their market power they are used for a growing number of individual, company-internal, or SaaS software development projects. Unfortunately those projects grow very fast into difficult manageable monsters, hard to finalize, and even harder to maintain. Large portions of the product habe to be developed to just handle technological needs. The focus on the functional requirements gets lost.

So what's the alternative? The first step is to remove blinders. Open your mind for uncommon approaches and take your time to play with them. Even if you don't use a new technology your solutions will get inspired. And when it is time for a decission evaluate carefully. Take a look how different technologies may help you and compare not only technological facts but also the ability to realize complex scenarios easily with a high productivity.

In my case that's why I'm developing in Smalltalk and Scheme - and sometimes Python and Ruby - using an ODBMS, alltegether on a Mac. Like said above I don't want that everybody is using that mix. But for me it is absolutely a high productive and comfortable environment. And I really hope I'll get the chance to use this mix, or parts of it, in external projects.

Fun using Scheme

Since about a month I'm playing with Scheme. I've first learned it during my studies, now I'm using it just for fun. First I've just wanted to know if my remembrance has been right but now I've discovered Scheme as a nice language for my smaller projects, like the successor of the Tideland Dynamic Content Processor (DCP), currently written in Python.

I've decided to use PLT Scheme. It is portable and works really fine on OS X, Linux and Windows. Also the development environment DrScheme has nice features, even if it can't compare to a typical Smalltalk environment. One example is the highlighting of the usage of variables, functions, and returns through arrows when you mark them with your mouse. You can follow them and trace their scopes. Also the code navigation, the very flexible editor, the debugger, and the profiler are pleasing. It suits for my current small projects.

Scheme as a language is also really interesting. Alan Kay told about Lisp, the big brother of Scheme: "Lisp isn't a language, it's a buildings material.". Right now I'm just using Scheme just as a language. But I'm also using the object-oriented class extension, which shows how Scheme can be extended providing an additional paradigm. The macro system of Scheme isn't just a string replacement. It is a full feature that works in the program context and the lexical scope. So powerful procedures can be build to extend the base language and automaticly create even more useful procedures in one step for the given context if needed.

Right now I develop a simple markup language for the DCP. Maybe I can show some nice example of how Scheme supports me here soon.

Silence during the last weeks

Since may I've been very lazy in writing new articles here into my blog. There are several reasons: much work, writing a german online article about software cost estimation, designing and documenting the model of the Tideland Train of Thoughts, (re-) learning Scheme, and a kind of being distracted, surely also through the death of my niece. But now it seems to get better. Last weekend we've worked hard in the garden and cleaned up our cellar. My reward for it has been a bottle of Chianti, together with a plate of cheese, salami, and cantuccini. Very good!

I've allready collected several topics for my next entries here. So I hope I really write them soon.