The next step

After the development of a flexible persistency solution - our Tideland Smalltalk Object Store - the next we've started to construct the next brick of our foundation frameworks for application solutions. It's the Tideland Lightweight Application Server, a small and simple framework for dynamicly managed service components handling synchronous and asynchronous requests. I've allready wrote about my experiments with a service bus implementation. The results have been very impressive so that we decided to combine that with the former tiny components that used the inversion of control pattern.

Both together, the asynchronous service components using the publish/subscribe pattern and the synchronous service components are now managed by the service broker and work in a given context. The ability of a client-dependent component resolution allows to operate flexible SaaS scenarios. So soon you'll read here about our projects.

List, priorize and link

One of the most importand tools in managing projects is the list. Lists are a really simple construct, so everyone understands them and work with them immediately. They are generally accepted. But a simple listing is just a medium, not very helpful. For your work as a responsible person in a software development project you'll need different lists, with different priorizations and links between them. By the way, one very easy to use single user tool for lists in terms of tables is Excel. That's one of the reasons why there's so much Management-by-Excel. *smile* Databases with web-frontends would do a much better job here.

What lists do you need? First of all it's the project vision. It doesn't have to be very detailed and the mission statements should cover clear user scenarios on a business process level. It's just an outline of what the final product will look like viewed as a whole. The contained statements have to be priorized by their importance for the customer. So his input is very importand. The result has to be a list with identifiable statements in a clear order from "must have" down to "nice to have". You know, this list can and will change while the project is in progress. But it builds the foundation for the next steps.

Based on the vision the list of features will be created. Each feature has to be linked to at least vision statement. Even non-functional requirements have to be listed as a feature and linked to a vision statement. If features without a clear vision statement exist that list isn't complete. Each feature inherits the major priority from the highest priorized linked vision statement. But typically there are much more features than vision statements. So they need a minor priorization controlling the order of equal major priorized. Additionally, to make this list useful for the daily work, you'll need a field where you name the responsible person for the realization and a simple status showing if the realization is in progress, finished, tested or if this feature is allready integrated. There's much more useful information, but keep in mind that this list shouln't be too complex. An optimal software could handle this in different views.

A very importand list you've got to maintain during the whole project is the risk management list. It contains individual risks, their possibility of an occurrence, their lost if they occur and counteractive measures. The possibility of occurence has to be defined in percentage, the list in a comperable size - most time this will be money. *smile* Both together multiplied result in the probable loss which controls the priorization. Risks can have multiple links to vision statements, features, or issues. But they also can be without a link, e.g. when the risk is based on the personnel situation.

As allready written above there's another type of lists: the issue list. An issue is typically a problem with the system or a former risk that occured. Each issue has also to be priorized in two steps. This time the first one is based on the quality of the issue as a "show stopper" down to "not so nice". The second priority is this time the priority of the linked vision statement, feature or risk.

Those few lists, maintained in an open accessable tool and used as a base for your daily work and communication will allready help a lot.

Featured blog: Presentation Zen

Today many presentations are produced. Very often PowerPoint is used for presented slides, sometimes Keynote or other tools. But it is very common that by far too much presentations are held without the adequate dramaturgy, the slides are overloaded, doesn't care about human recognition behaviour and use by far too much visual effects instead of real content. Garr Reynold is interested in professional presentation and provides a very fine blog about it: Presentation Zen. For me it is one of the best blogs in the net.

The power and beauty of dynamic typed languages

One major topic of many software systems is the configuration of the system. Typically there are APIs with key/value pairings, maybe with groups. Another way are more or less complex XML based systems accessed by specific classes for a defined part of the application.

My intention has been to provide a generic hierarchical system, easy to read and use, with a seamless integration into Smalltalk and the possibility to read and write XML. Yes, me too. *smile* But XML is not the major access language, it's only an additional feature to read and store external configurations in closed applications. My typical way to use the configuration shall be the Smalltalk API.

Result is the set of the three classes CslConfiguration, CslConfigurationValue and CslConfigurationError. The entry class is just a wrapper with nil as superclass, passing almost all message sends to the wrapped configuration value. But it has the own message withDefault: which expects a block as argument. What is the idea behind it? Imagine an access to a configuration value that doesn't exist. The call would be

foo := cfg foo withDefault: [#bar].

In this case foo would get the value #bar and the configuration value would also be set to #bar. A second call

foo := cfg foo withDefault: [#yadda].

would now still return #bar. To set it to #yadda the statement has to be

cfg foo: #yadda.

A simple get without a default should not surprise you:

foo := cfg foo.

But CslConfiguration has no methods foo: or foo. It just uses doesNotUnderstand: to map the message to a setter or getter of an internal dictionary key. And if the key doesn't exist and a getter is called it returns a new nested CslConfiguration instance. So you can also do things like

cfg database connection poolSize: 10.

even if the configuration is empty at this time. Also CslConfiguration understands isNil, ifNil:, and ifNil:ifNotNil: and behaves like nil if is empty. So it's no problem to perform statements like

cfg foo bar isNil ifTrue: [...].

As a result configuration can be setup very easy, no subclassing or similar is needed and the using classes can also define default values inside the getter statement. Static languages don't even allow the compilation. *laugh*

Book order

I've done it. Playing with the idea allready a longer time I've just ordered the books "Siddhartha" by Herman Hesse, "The Myth of Sisyphus" by Albert Camus, and a bookset containing "Hamlet", "Romeo and Juliet", "A Midsummer Night's Dream", "Macbeth", "King Lear", "Coriolanus", "The Taming of the Shrew", "Much Ado About Nothing", "The Merry Wives of Windsor", "As You Like It", and "All's Well That Ends Well" by the famous William Shakespeare. I'm really excited about it.

Service bus in Smalltalk

As I've told a few days ago I'm trying the explorative modelling like it has been described by SAP and Heeg on the OOP. My business is about SOA, so I've invested some hours - about 8 - to develope a service bus where multiple publishers can send messages to and multiple subscribers can retrieve those they are interested in defined through a filter. The filter is a block that evaluates the message and returns true if it wants to handle it. So ony published message can have multiple subscribers. Also the messages are priorized in the range of the Smalltalk processes. So they are handled and processed asynchronously in the so defined order and with the given priority.

First tests showed a very high performance, I've been impressed. It will now get an extension for processing contexts to handle states between message handlings and a mechanism for an automatic routing of answer messages back to the sender. I'll also integrate our execution time measuring and create a better test scenario. So I'll be able to present you more informations about the performance soon.

SOS improvements

The last days the Smalltalk Object Store has made good improvements towards stability and flexibility. The system now relies on the CslRuntimeEnvironment and the new CslConfiguration. Also the file handling has been refactored. The directory and file structure is now better to manage, the needed directories are created on demand. The background write process is now a one shot process started after a commit and only if it is not allready running. So less resources are consumed.

The next step will be integration of the lock file and some final tests. After that the beta testing can start. So if you're interested just send a mail to mue@tideland.biz.

Don't worry, embrace change

Since I've been born in 1965 so many things have changed. Politically, ecologically, technologically - here in Germany, in Europe, and worldwide. Some changes need a very long time, like the change from the dinosaurs to the mammals, or the ice ages. Other changes only need a few years, compared to a lifetime, like the political reorder in Europe, or a few months, like the death of many dot.com startups.

Also in my daily job as a senior consultant I'm surrounded by changes. Changes of software requirements, changes of development proesses, changes of the organization, changes of the customers I work with. All around almost nothing is constant but much is varying. It seems to be totally natural. So what is allways astonishing to me is how many people have problems with changes. If you talk to them they are scared about there future and what every alteration brings.

Many psychologists research and discuss about it. I'm not really interested in the reasons. But as I can't stop the fact that everything is changing I've learned to make my best out of it. The besting wording I've found for it is embrace change. I've heard it the first time in the context of agile processes. The idea behind it is to take each rearrangement and see it as a chance. You are forced to think about your habbits and routines and lay them down or improve them. OK, sometimes it's really cushy to bank on own experiences and familar knack. But many of them get just older and doesn't fit into new needs.

So I'll continue to embrace change. *smile*

Where are you creative?

Did you allready recognized when or where your most creative time is? Not the time when or the place where you're most productive. But those where you get your good ideas? For me it's the bathroom in the morning. I don't know why, but I'm often telling my wife about new ideas I had while I'm preparing for the day. Really funny. Please tell me about your favorite places.

Zen and the art of ODBMS

I think it isn't really difficult to see: I'm a fan of object-oriented database management systems. But how does it come, why do I appreciate them so much and how do I compare them to relational database management systems and object-relational mappers?

If I remember right my fiirst steps in object-orientation begun in the late 1980s. I've read about Smalltalk, Objective-C and C++. But licenses had been expensive, and powerful machines too. In 1989 Borland released Turbo Pascal 5.5 which had some OO features. This software allowed me to do my first practical steps in object-oriented software development. And I really must say that I've liked it from the first day. The concept of designing and programming construction plans for "living" objects inside the system, that have states and communicate with each other, has been so natural for me that I do OOP until today. So it is no surprise that I also followed the database market and read about ODBMS. I found this idea a logical continuation of the OO paradigm.

In 1999 I've gotten the chance of leading a software development team implementing a content management system based on J2EE. I've tested the usage of ODBMS a short time before using the ObjectStore PSE for Java. So in this project we evaluated some persistency strategies and systems and we decided us for using POET. We had no mass data with a high query load. But our idea has been to store XML in a parsed DOM tree directly inside the database. The elements inside the documents could be regular elements or dynamic elements. Those have been executed for including other documents or querying external databases or data sources before being rendered via XSL-T and then delivered.

Here you allready see one of the major domains for ODBMS. When object structure are complex, maybe hierarchical, and there are no complex queries but an access via identifiers and then through the navigation over the networked objects, then you're right with ODBMS. The entry point is typically a registered root object, often a collection. Dictionaries or b-trees allow the access via a unique key. Also there may be different kinds of entry points, depending on the business case. So customers could be accessed via their customer number but also via a reference inside an invoice retrieved by the invoice number. Tight meshed objects allow a large number of possible navigation paths. This allready shows one disadvantage: all those references have to by maintained by the application. But in relational databases foreign keys have to be maintained, too. So the overhead is tolerable.

Another great plus is that no mapping is needed. Depending on the ODBMS the persistent classes may need to be derived from a special base class, or they need an anotation. Other systems - like our Tideland SOS - handle the persistency totally transparent for the developer. This also applies to the CRUD operations. Depending on new added instances to the network of other persistent objects, navigating between them, changing the values of instance variables or just set them to nil all operations are handled automaticly. The only needed actions are beginning a transaction, navigating starting at a root object, and committing or aborting the transaction. So an ODBMS presents itself as a large persistent extension of the main memory.

OK, ODBMS naturally also have disadvantages. They differ from vendor to vendor. So I just list them.

• The support for queries (e.g. with OQL) is often bad.
• The access through different languages often is difficult.
• Schema evolution sometimes isn't supported very good. OK, that's hard for RDBMS based apps too.

Our Tideland SOS has a tight binding to Smalltalk, queries are only provided by the typical methods like select and schema evolution is currently only supported for additional instance variables and classes. But the concept for more is allready outlined. But it depends on your needs. Our ODBMS is a very lightweight system for small to mid-sized applications.

How about O/R mapper? Hmm, I don't really like them. The mapping, even if it is tool supported, so much additional configuration files, often in XML, not as flexible as ODBMS. Maybe I've just made my experiences with a wrong designed system (about 750 classes managed via TOPlink). But when the usage of an RDBMS is needed - customer constraints, legacy data, application integration, OLAP systems - I prefer patterns like the table data gateways and data transfer objects. Here you also see one of the great advantages I see in RDBMS.: they are query optimized. So if an application system heavily relies on data retrieval relational databases are best. Also if an access through different languages - object-oriented or not - and existing apps is needed they do a really good job.

Does an answer to the question "What is the best persistence strategy?" exist? Hmm, I don't think so. It really depends, see above. But if I'm asked for my favorite, I would answer: ODBMS. *smile*

KISS, YAGNI, DRY

Most of you know these acronyms relating the style of programming. The principles behind them help to build a software that doesn't lose itself in the jungle of feature creep and stays maintainable during the application lifecycle. OK, they are not the only aspects to think about, but they are really important.

So, if they help a programmer in develop a good piece of software, why shouldn't they help a project manager accomplish a good project? Let's take a deeper look into each acronym.

KISS

Keep it simple, stupid - it can't be repeated often enough. There's so much literature or training about project management with many processes, standards, and documentation templates. Many of them are really good and very detailed. They show a broad spectrum of methods, formalities, and artefacts regarding their special subject. This may be the requirements engineering, software cost estimation, risk management, architecture and design, coding, testing, and much more. Each time specialists write about their own area of expertise providing a more than 100% solution for the topic. This is not bad, you get a very deep insight. But this does not mean that you shall use everything in your project.

Depending on your customer, your team, your management and inhouse standards all your partners have different needs. The customer is interested in a good consultation and a fulfillment of his requirements, your management has commercial requirements and needs a reporting, your QM has defined quality standards, and your developers need a productive environment with the right toolset. To achieve all you can use your know-how and build a superset of all demands together with a safety belt out of your books or trainings. But if you would draw this on a piece of paper you could see that this cloud contains by far too much overhead for you and many of the involved people. It is too much to remember everything and does not allow an undisturbed flow of work. The bureaucracy kills the productivity and creativity of your team.

YAGNI

You ain't gonna need it - so you've got to strip it down to the most important intersections and try to negotiate a priorization about each wish that's expressed by only one person or a single group. Try to compare how much each request is really worth and what it does cost to get a good argumentation basis. And also negotiate with yourself about the ideas out of your books and trainings. Even if processes and ideas really sound good and productive you've got to check if this is suitable for you project, your environment, and your situation. As a project manager you're a service provider for the customer, your team and the management. So parts of the work shouldn't be ends in itself.

DRY

Don't repeat yourself - define a clear and simple process, adjust it with your team, communicate it to everyone, publish it at a central place in your intranet with news announcements and a history, and take care that everybody gets informed if and why something changes. So, over the time, you change your environment to live its own simple and aligned process and the project management as an integral part of the software development instead of a disturbing evil ordered by the manager.

The last steps finishing the Tideland Smalltalk Object Store

The refactoring of the file I/O is done. The overhead could be reduced so that the file size is now about 30 to 40% of the old size, depending on the size of the instance variables. The new structure has a better separation of concerns. This improves the maintainability for future extensions. Those are the configurable number of clusters and a packing of the data files.

So for the official beta testing only the unit tests for the queries and enumerations on named roots and a locking mechanism against concurrent access are missing.

Follow the related link for details.

Domain specific languages

In Live UML Hans Beck responded to my report of the Smalltalk evening at the OOP 2007. Here the SAP and the Georg Heeg eK described how they used Smalltalk for a method they called "Explorative Modeling". Hans wrote about a similar idea he developed four years ago. Both share the concept of modeling in Smalltalk to get a static and dynamic model which is discussed and designed together with the customer. And in both cases the main implementation is not in Smalltalk but in the target language. The advantage is ovious, customer and developer share a common understanding of the domain, not only by the classes and their relations but also by the behaviour of those classes. The base is the development of a domain specific language (DSL) in Smalltalk.

An almost similar idea has been implemented by MetaCase. Their tool MetaEdit+ allows to design and execute a graphical DSL. Additionally they allow to generate source code out of the model. So they combine model-driven software development (MDSD) with DSL. It is a very nice tool written in VisualWorks. The developers told me that they are really Smalltalk enthusiats. Developing this tool in an other language would have been much more time-consuming. Even if this isn't really astonishing it is allways good news.

Altogether there seems to be a trend in to compensate the lack of simpleness and dynamic evaluation of the UML with domain specific languages. They try to fill the gap in understanding between the customer and the devloper. Like the natural language is too weak and unformal for a development the UML is too complex and bulky for the customer. The risk of misunderstanding is too high, so approaches like the explorative modeling with domain specific languages are needed. And languages like Smalltalk as a base for that are really powerful. Smalltalk is easy to understand, metaprogrammable, highly interactive, and has a rich set of tools. Even the development of simple UIs - not for design but as an interface for the evaluation - is really simple.

So I'll try to find an evaluation project in my professional environment.