Co-moderator of Software Architecture group at Xing

Since yesterday I'm co-moderator of the Software Architecture group at Xing. You're all welcome to participate to discuss interesting topics like modeling and model-driven engineering, technologies, and patterns.

For those of you who don't know Xing: it's a more business-driven networking platform like Facebook. I'm here since almost 4 years with lots of interesting contacts in in many groups. In march 2005 I've created the Smalltalk group, a still growing community of loyal and new fans.

OOP 2008 - Architecture Day

The second half of this day has been filled up with three tracks of the Architecture Day. First a concept of black box and white box architecture documentation presented by Peter Hruschka, then ways to better quality in software architectures without doing too much useless work presented by Gernot Starke, and at last Eric Evans with modelling strategics for larger systems. The names of the speakers speak for themselves, their tracks have been the best ones. The ending of this years OOP will be a software architecture track by Frank Buschmann tomorrow. 

Lightweight Message Bus

The last days I've worked intensively on the Lightweight Message Bus (CELLMB). After a first approach developing everything in one module it is now splitted up into three modules, one more general and two for the CELLMB. The first one is the Service Process Pool (CELSPP), which starts service processes up to a defined number. The start function returns a process ID which now can be used as if the process is only one single instance process. But instead of handling those messages itself, the received messages will be resent round robin to the service processes. So here and in its lazy automatic process creation it differs from pg2. A normal exit is resent to all service processes to terminate gracefully, also died processes are removed and recreated on demand automaticly. A later extension will support the processes to opt out and terminate after an idle time.

The CELLMB now uses this pool for the internal dispatching of messages with a given context, verb, and noun to subscribed processes. Those can be single and pooled processes using the CELSPP. For a simple development of those processes the module CELLMBSVC provides a generic behaviour. Just the callbacks init, handle and terminate have to be implemented - of course beside possibly needed helper functions.

The next step will be the implementation of the unit tests for the CELLMB and then the first services for my Erlang Business Library (EBL) which I'll use in my different portal projects. Those will handle configuration, authentication, authorization, client management, user management, and addresses based on vCards. They all will use Mnesia as their persistency backend and will be loose-coupled using the CELLMB for communication.

Some thoughts on software architecture

Beside requirements engineering software architecture is the second important foundation of a software system. The architecture heavily relies on the requirements and has to define an abstract view with a clear separation of concerns. It outlines the distribution of tasks to components and declares the interfaces betwen them. The design of the communication flow between those components bases on the use cases, so that there's a direct link from the requirements to the architecture artefacts.

Very often, especially in small starting projects, the software architecture is disregarded. The usage of JEE or .NET as a technological platform is viewed as enough. But it isn't. It's just a rough layout, a blueprint, which mostly defines the vertical layers of an application system. But that's not enough for the functional separation. And also very often systems aren't realizable independently and from the ground up. They have to fit into existing environments, managing interfaces to existing systems. Starting here without taking the time to look ahead will lead to quick results, but also to badly increasing problems when the complexity grows or the system has to be evolved and maintained over the years.

So there's allways a strong need for a software architecture. Clear goals have to be defined and priorized as well as also a clear proceeding is needed. A matrix helps to sort all requirements into functional (vertical) and technological (horizontal) aspects. The result are clear layers (persistency, model, business logic, orchestration, and user interface) and business domains (e.g. financial accounting, supply chain management, customer relationship management, controlling, or human resources). The matrix gives a first impression of possible components and libraries. Now it's possible to define groupings, dependencies, interfaces, and the communication between the components.

It depends on the project how deep this has to be broken down. But what is also needed are the major patterns for the realization. Those are SOA and EAI patterns, software architecture patterns, and the most important ones for the medium and small components. This helps to get a common language, a common understanding of the whole system. Well documented in a small handout (max. 50 to 100 pages) with easy to understand deployment, component, and class diagrams - maybe some activity or sequence diagrams for dynamic aspects - will help all team members right from the start.

Zen and the art of ODBMS

I think it isn't really difficult to see: I'm a fan of object-oriented database management systems. But how does it come, why do I appreciate them so much and how do I compare them to relational database management systems and object-relational mappers?

If I remember right my fiirst steps in object-orientation begun in the late 1980s. I've read about Smalltalk, Objective-C and C++. But licenses had been expensive, and powerful machines too. In 1989 Borland released Turbo Pascal 5.5 which had some OO features. This software allowed me to do my first practical steps in object-oriented software development. And I really must say that I've liked it from the first day. The concept of designing and programming construction plans for "living" objects inside the system, that have states and communicate with each other, has been so natural for me that I do OOP until today. So it is no surprise that I also followed the database market and read about ODBMS. I found this idea a logical continuation of the OO paradigm.

In 1999 I've gotten the chance of leading a software development team implementing a content management system based on J2EE. I've tested the usage of ODBMS a short time before using the ObjectStore PSE for Java. So in this project we evaluated some persistency strategies and systems and we decided us for using POET. We had no mass data with a high query load. But our idea has been to store XML in a parsed DOM tree directly inside the database. The elements inside the documents could be regular elements or dynamic elements. Those have been executed for including other documents or querying external databases or data sources before being rendered via XSL-T and then delivered.

Here you allready see one of the major domains for ODBMS. When object structure are complex, maybe hierarchical, and there are no complex queries but an access via identifiers and then through the navigation over the networked objects, then you're right with ODBMS. The entry point is typically a registered root object, often a collection. Dictionaries or b-trees allow the access via a unique key. Also there may be different kinds of entry points, depending on the business case. So customers could be accessed via their customer number but also via a reference inside an invoice retrieved by the invoice number. Tight meshed objects allow a large number of possible navigation paths. This allready shows one disadvantage: all those references have to by maintained by the application. But in relational databases foreign keys have to be maintained, too. So the overhead is tolerable.

Another great plus is that no mapping is needed. Depending on the ODBMS the persistent classes may need to be derived from a special base class, or they need an anotation. Other systems - like our Tideland SOS - handle the persistency totally transparent for the developer. This also applies to the CRUD operations. Depending on new added instances to the network of other persistent objects, navigating between them, changing the values of instance variables or just set them to nil all operations are handled automaticly. The only needed actions are beginning a transaction, navigating starting at a root object, and committing or aborting the transaction. So an ODBMS presents itself as a large persistent extension of the main memory.

OK, ODBMS naturally also have disadvantages. They differ from vendor to vendor. So I just list them.

• The support for queries (e.g. with OQL) is often bad.
• The access through different languages often is difficult.
• Schema evolution sometimes isn't supported very good. OK, that's hard for RDBMS based apps too.

Our Tideland SOS has a tight binding to Smalltalk, queries are only provided by the typical methods like select and schema evolution is currently only supported for additional instance variables and classes. But the concept for more is allready outlined. But it depends on your needs. Our ODBMS is a very lightweight system for small to mid-sized applications.

How about O/R mapper? Hmm, I don't really like them. The mapping, even if it is tool supported, so much additional configuration files, often in XML, not as flexible as ODBMS. Maybe I've just made my experiences with a wrong designed system (about 750 classes managed via TOPlink). But when the usage of an RDBMS is needed - customer constraints, legacy data, application integration, OLAP systems - I prefer patterns like the table data gateways and data transfer objects. Here you also see one of the great advantages I see in RDBMS.: they are query optimized. So if an application system heavily relies on data retrieval relational databases are best. Also if an access through different languages - object-oriented or not - and existing apps is needed they do a really good job.

Does an answer to the question "What is the best persistence strategy?" exist? Hmm, I don't think so. It really depends, see above. But if I'm asked for my favorite, I would answer: ODBMS. *smile*

KISS, YAGNI, DRY

Most of you know these acronyms relating the style of programming. The principles behind them help to build a software that doesn't lose itself in the jungle of feature creep and stays maintainable during the application lifecycle. OK, they are not the only aspects to think about, but they are really important.

So, if they help a programmer in develop a good piece of software, why shouldn't they help a project manager accomplish a good project? Let's take a deeper look into each acronym.

KISS

Keep it simple, stupid - it can't be repeated often enough. There's so much literature or training about project management with many processes, standards, and documentation templates. Many of them are really good and very detailed. They show a broad spectrum of methods, formalities, and artefacts regarding their special subject. This may be the requirements engineering, software cost estimation, risk management, architecture and design, coding, testing, and much more. Each time specialists write about their own area of expertise providing a more than 100% solution for the topic. This is not bad, you get a very deep insight. But this does not mean that you shall use everything in your project.

Depending on your customer, your team, your management and inhouse standards all your partners have different needs. The customer is interested in a good consultation and a fulfillment of his requirements, your management has commercial requirements and needs a reporting, your QM has defined quality standards, and your developers need a productive environment with the right toolset. To achieve all you can use your know-how and build a superset of all demands together with a safety belt out of your books or trainings. But if you would draw this on a piece of paper you could see that this cloud contains by far too much overhead for you and many of the involved people. It is too much to remember everything and does not allow an undisturbed flow of work. The bureaucracy kills the productivity and creativity of your team.

YAGNI

You ain't gonna need it - so you've got to strip it down to the most important intersections and try to negotiate a priorization about each wish that's expressed by only one person or a single group. Try to compare how much each request is really worth and what it does cost to get a good argumentation basis. And also negotiate with yourself about the ideas out of your books and trainings. Even if processes and ideas really sound good and productive you've got to check if this is suitable for you project, your environment, and your situation. As a project manager you're a service provider for the customer, your team and the management. So parts of the work shouldn't be ends in itself.

DRY

Don't repeat yourself - define a clear and simple process, adjust it with your team, communicate it to everyone, publish it at a central place in your intranet with news announcements and a history, and take care that everybody gets informed if and why something changes. So, over the time, you change your environment to live its own simple and aligned process and the project management as an integral part of the software development instead of a disturbing evil ordered by the manager.

OOP 2007 - Architecture is like decathlon

Peter Hruschka compared architecture to decathlon in his track today. The disciplines are design, decide, simplify, evaluate, verify, document, implement, communicate, balance, and consult. In his opinion it is not enough to hold the world record in one discipline, even beeing very good in several disciplines isn't enough. To be top you have to be real good in every discipline. I like this comparison.

The eBay architecture

In a presentation at the SD Forum 2006 Randy Shoup and Dan Pritchett showed interesting details about the eBay architecture. They demonstrated how the requirements in site stability, feature velocity, performance, and cost changed from the beginning in 1995 to now with about 212 million registered users and 26 billion SQL executions per day. *wow* A really impressing and thrilling paper. You'll find it following the related link below.