Day 6 - Time for something complete different

OK, also today we've done a bit for our renovation. We prepared the first radiators for painting and will continue tomorrow. But the most time we've gone shopping, had a BBQ, and have been visited by my sister and her husband.

But yesterday I've read about the new release 4.0 of PLT Scheme. So I've visited the site and once again I've been fascinated of the Scheme style. Even if I'm currently preferring Erlang for my work, I like the Scheme syntax more. But I also like the keyword arguments of Smalltalk. So my favorite language should be a mix of those three. Concurrency, distribution, pattern-matching, and non-modifiable variables from Erlang, the syntax like Scheme, and arguments in the style of Smalltalk. This could look like

(define (insert customer: Customer into-database: Database)
    (let ((PCustomer (prepare customer: Customer))
          (OpenDatabase (open database: Database)))
          (...)))

Now one question would be, how pattern-matching would look like. And message sending and receiving. Erlang tuples could just also be lists. But I also would need atoms. And at last I would like Smalltalks closures more than funs and lambdas.

(map fun: (|item: Item| (print Item)) on-list: MyList)

I'll never implement such a language, but it's fun to think about it.

Co-moderator of Software Architecture group at Xing

Since yesterday I'm co-moderator of the Software Architecture group at Xing. You're all welcome to participate to discuss interesting topics like modeling and model-driven engineering, technologies, and patterns.

For those of you who don't know Xing: it's a more business-driven networking platform like Facebook. I'm here since almost 4 years with lots of interesting contacts in in many groups. In march 2005 I've created the Smalltalk group, a still growing community of loyal and new fans.

Lightweight Message Bus development and test progress

After testing synchronous requests for the Lightweight Message Bus I've today tested asynchronous requests with unordered response receivings. Everything works fine, simple services, multiple services answering to a single request and the orchestration of services. So I now only have to test some convenience functions, the timeout behaviour when the time exceeds, and massive parallel regression testing. 

My next step after those tests will be a simple web connector for the CELLMB. Post requests with serialized Erlang data structures will be converted into a message an published onto the bus. The respones will be serialized and sent back. URI parts control service, command, correlation id, and timeout. 

The idea behind it is a combination of using Seaside, which still is the best web framework, for the frontend and Erlang as the runtime environment for my backend services using the bus and Mnesia as database. So I'm able to use the right tools for the right job.

OOP 2008 - Smalltalk evening

Today has been the traditional Smalltalk evening by Georg Heeg, Cincom and Gemstone. There've been three parallel tracks about Seaside for Ruby on Rails developers, GLASS and VisualWorks 7.6. After those Lukas Renggli presented Seaside more detailed. This has been a very good presentations of this great framework with many questions out of the audience. Finally we've had the well known smalltalk with snacks and drinks. *smile* So I could talk to Yvonne Schickel and James of Cincom about the roadmap of VisualWorks. Without saying too much - they are doing well and have interesting plans and ideas.

One imression of me is that the Smalltalk community should thank Ruby. It opened minds for intelligent non-mainstream solutions. So I that this is one of the reasons why the interest in Smalltalk is growing (increasing number of Seaside mailing list subscribers, more participiants in the Smalltalk evening than last year).

OOP 2008 - Smalltalk won the Dynamic Language Shootout

As you allready could read on Planet Smalltalk the winner of the dynamic language shootout is a Smalltalk app based on VisualWorks and Seaside. It is a neat app with a great user interface. The task has been to develop a Scrabble application with a number of constraints. I've talked with Georg Heeg and Stefan Tilkov - both members of the jury - and both said that it was really superior to the other solutions. The second place used JRuby, the third Lua. The Lua app was one of the shortest and additionally made the most points in a game. Tomorrow I'll get more infos by Stefan than he told me while we played a real Scrabble - every jury member got one - in the subway. *smile* Additionaly James promised that he will publish his video of the presentation and a screencast.

Progress in SOS migration

The migration of the Smalltalk Object Store, my ODBMS, to Squeak has made a large progress. Almost all unit tests for the in-memory strategy with lazy writing are working, there's only one known error. So the next strategy, the file system strategy, needs only few changes, most are allready made. I hope we can provide the betas for download soon.

During the migration there have been two obstacles I've underestimated. The first one has been selfmade. The classes providing a runtime environment and application configuration have been a bit tricky, using the VisualWorks process environments and dynamic method invocation inside doesNotUnterstand:. It worked, but it has been platform specific and also not very maintainable. I decided to change the mechansm, but that has had a deep impact on the Lightweight Application Server and even more on the Smalltalk Object Store. The positive result are now better handable frameworks.

But the second one that bothered me more are the incompatabilities between the different Smalltalk dialects. OK, it's acceptable that different producer - commercial as free - try to provide unique selling points. But those should focus special high-end frameworks, like container for server components, libraries for distributed computing, or for user interfaces. But I would like a living standard, like actually in Scheme the R5RS and the discussion for the R6RS, for a class library containing all base types with a powerful string handling, the chronological classes, all containers, file handling, and the basic network protocols as server and client. The latest ANSI standard is now almost 10 years old. It would be really nice if the Smalltalk community would demonstrate their vitality through a new standardization process.

Migration goes on

After the import of the sources into Squeak and first rough changes I'm proceeding with the unit tests and needed adjustments. Most ones are simple, but I changed the configuration class to work more simple. The old way has been hard to debug due to the heavy usage of dynamic method calls out of doesNotUnderstand:. However it is now less flexible, but still powerful enough.

The second big change concers the runtime environment. Until now it has been automaticly bound to the process where a lookup is called and where no environment exists and no parent process has a bound environment. But this has been a bit precarious in multi environment systems. So now every runtime environment has an id which has to be used for the lookup. Both has been applied to the Common Smalltalk Library (CSL), the unit tests have also been changed where needed and now all tests succeed. *smile*

The second library is now the Lightweight Application Server (LAS) which relies on the environment. So I've started to change the lookup mechanism like mentioned above. Here the first tests are green, but also some are failing. Those problems should be easy to fix.

The Smalltalk Object Store (SOS) has not yet been tested. There are two parts that may be a bit tricky. We'll see.

Another effect of playing with Scheme

Since I've tested PLT Scheme and started to re-develop the Tideland Dynamic Content Processor with it - the document node tree, navigation on it, and parsing SML text documents are allready working - I've done almost nothing in Smalltalk. So now I've thought a bit about the reason and I think I've got the answer.

I've started programming in Smalltalk with Squeak which is a full featured and innovative environment with a strange look'n'feel *scnr* and a motley community. After a while I switched to VisualWorks which is more professional and straight. I would really like to realize a professional project with VW. But it is also huge (not compared to Eclipse or Visual Studio *smile*), in some parts complex (e.g. Store vs Monticello), and - what counts most - not open source. Don't understand me wrong, I've got nothing against commercial software. I've bought lots of software for my Mac. But when developing my private software I use open source. So I want to give my applications and libraries back to the community. And I want to use them commercially and allow all others to do the same. Beside that I like that all together in a small and handy tool - like Squeak and PLT Scheme.

So I've allready started the porting of my sources back to Squeak. The export from VW is done, the Squeak development environment is set up, and first file-in tests leading to changes directly in the Smalltalk source file have been done. The only thing I've still got to change externally are the line endings. After that the file-in should work without problems and I can start to find needed code changes via the unit tests which cover most of my code. I hope I soon can provide the betas of the Common Smalltalk Library, the Smalltalk Object Store, and the Lightweight Application Server (take a look at the features). Additionally I can re-integrate the RDBMS strategies allowing the SOS to store objects in SQLite and PostgreSQL. Only the handling of large objects has changed and has to be adapted. The future development of the Net Business Framework - which is intended to have a REST interface - depends on some evaluations I'll do after the porting.

Keep it simple in Scheme

One features of the new Tideland Dynamic Content Processor - and also of the Tideland Train of Thoughts - will be the usage of a simple markup language, which is a kind of XML, but without attributes and less bloated through the usage of a Lisp-like style.

{document
{table-of-contents}
{section Introduction
  {p The new {strong Tideland Dynamic Content Processor} is a
     lightweight content management system developed in Scheme. The data
     is stored in SML files.
  }
}
}

My first approach in Ruby some time ago ended in a DOM-like implementation with classes for the document, for elements, and for text. The first reimplementation in Scheme used the same approach based on the class system of PLT Scheme. But I soon discoverd that this way isn't natural for Scheme. It works, yes, but Scheme OOP isn't as simple and straight as in Smalltalk.

So I switch to a simple structure, a kind of a list with fields for tag, id, parent, and childs, together with a set of procedures. They allow to easily create a document tree, navigate through it, or search nodes. What astonished me most was how small this solution is. I'm doing OO for over 20 years now. This formed my development style to think in frameworks immediately. Scheme now refocusses my mind to always look for possible simpler solutions.

New Tideland application

It has been a bit silent here the last days. But I'm still busy. Based on some ideas I'm carrying since a longer time and the need for a demo application for the Tideland Application Stack I've started a new project. It is called Tideland Train of Thoughts and is intended to allow distributed teams to share their ideas. This sounds a bit like a wiki, but linking and tagging work different and allow many powerful views for the users.

Regardless if you want to organize projects with vision, scenarios, requirements, tasks, and problems, or a knowledge base with many cross links, attachments, and categories - the Tideland Train of Thoughts helps you and your team.

Usage of the Lightweight Application Server

After some questions about how the Tideland Lightweight Application Server has to be used I've added a small documentation in our wiki.

For German readers: Article about Squeak/Seaside

The today published iX magazine contains an article about Squeak and Seaside startiing at page 136. It is my third Smalltalk article in this magazine and I'll try to publish it on the ESUG pages again later.

Tideland Application Stack

The Tideland Net Business Framework (NBF) is moving towards the internal alpha tests. So I would like to show you an overview of the Tideland Application Stack here.

It shows how web applications use the NBF for web controllers and rendering, the business logic is developed as stateful and stateless components and asynchronous services in the LAS, and the persistent model is stored using the SOS. All frameworks use, as the future applications may do also, the CSL with helpful classes.

The NBF will contain the Webtop API. It provides an easy to use framework for web-based application. I'll show you some prototype screenshots soon.

Discovering Smalltalk

It's allways the same when Smalltalkers talk to users of other, typical C-style programming languages. As long as there are just the words class, object or method they have a relative common understanding. But when we are talking about sending messages to objects or submitting changes into the living image they don't understand. At this point Smalltalk is a totally different world. This experience has been described here and commented here.

I've also tried to write about this, online and in articles. But it is really not easy to communicate this real great advantage of Smalltalk, this secret of why developing in Smalltalk is so productive. It seems that my next entries about this language and environment have to be focussed on those differences.

First LAS design documentation

The first design documentation of the Tideland Lightweight Application Server can be read here. You can see how the LAS provides an infrastraucture for the dynamic, client-dependend instantiation of components and services. Components work synchronous in their context or shared and services work asynchronous.

The next step

After the development of a flexible persistency solution - our Tideland Smalltalk Object Store - the next we've started to construct the next brick of our foundation frameworks for application solutions. It's the Tideland Lightweight Application Server, a small and simple framework for dynamicly managed service components handling synchronous and asynchronous requests. I've allready wrote about my experiments with a service bus implementation. The results have been very impressive so that we decided to combine that with the former tiny components that used the inversion of control pattern.

Both together, the asynchronous service components using the publish/subscribe pattern and the synchronous service components are now managed by the service broker and work in a given context. The ability of a client-dependent component resolution allows to operate flexible SaaS scenarios. So soon you'll read here about our projects.

The power and beauty of dynamic typed languages

One major topic of many software systems is the configuration of the system. Typically there are APIs with key/value pairings, maybe with groups. Another way are more or less complex XML based systems accessed by specific classes for a defined part of the application.

My intention has been to provide a generic hierarchical system, easy to read and use, with a seamless integration into Smalltalk and the possibility to read and write XML. Yes, me too. *smile* But XML is not the major access language, it's only an additional feature to read and store external configurations in closed applications. My typical way to use the configuration shall be the Smalltalk API.

Result is the set of the three classes CslConfiguration, CslConfigurationValue and CslConfigurationError. The entry class is just a wrapper with nil as superclass, passing almost all message sends to the wrapped configuration value. But it has the own message withDefault: which expects a block as argument. What is the idea behind it? Imagine an access to a configuration value that doesn't exist. The call would be

foo := cfg foo withDefault: [#bar].

In this case foo would get the value #bar and the configuration value would also be set to #bar. A second call

foo := cfg foo withDefault: [#yadda].

would now still return #bar. To set it to #yadda the statement has to be

cfg foo: #yadda.

A simple get without a default should not surprise you:

foo := cfg foo.

But CslConfiguration has no methods foo: or foo. It just uses doesNotUnderstand: to map the message to a setter or getter of an internal dictionary key. And if the key doesn't exist and a getter is called it returns a new nested CslConfiguration instance. So you can also do things like

cfg database connection poolSize: 10.

even if the configuration is empty at this time. Also CslConfiguration understands isNil, ifNil:, and ifNil:ifNotNil: and behaves like nil if is empty. So it's no problem to perform statements like

cfg foo bar isNil ifTrue: [...].

As a result configuration can be setup very easy, no subclassing or similar is needed and the using classes can also define default values inside the getter statement. Static languages don't even allow the compilation. *laugh*

Service bus in Smalltalk

As I've told a few days ago I'm trying the explorative modelling like it has been described by SAP and Heeg on the OOP. My business is about SOA, so I've invested some hours - about 8 - to develope a service bus where multiple publishers can send messages to and multiple subscribers can retrieve those they are interested in defined through a filter. The filter is a block that evaluates the message and returns true if it wants to handle it. So ony published message can have multiple subscribers. Also the messages are priorized in the range of the Smalltalk processes. So they are handled and processed asynchronously in the so defined order and with the given priority.

First tests showed a very high performance, I've been impressed. It will now get an extension for processing contexts to handle states between message handlings and a mechanism for an automatic routing of answer messages back to the sender. I'll also integrate our execution time measuring and create a better test scenario. So I'll be able to present you more informations about the performance soon.

SOS improvements

The last days the Smalltalk Object Store has made good improvements towards stability and flexibility. The system now relies on the CslRuntimeEnvironment and the new CslConfiguration. Also the file handling has been refactored. The directory and file structure is now better to manage, the needed directories are created on demand. The background write process is now a one shot process started after a commit and only if it is not allready running. So less resources are consumed.

The next step will be integration of the lock file and some final tests. After that the beta testing can start. So if you're interested just send a mail to mue@tideland.biz.

Zen and the art of ODBMS

I think it isn't really difficult to see: I'm a fan of object-oriented database management systems. But how does it come, why do I appreciate them so much and how do I compare them to relational database management systems and object-relational mappers?

If I remember right my fiirst steps in object-orientation begun in the late 1980s. I've read about Smalltalk, Objective-C and C++. But licenses had been expensive, and powerful machines too. In 1989 Borland released Turbo Pascal 5.5 which had some OO features. This software allowed me to do my first practical steps in object-oriented software development. And I really must say that I've liked it from the first day. The concept of designing and programming construction plans for "living" objects inside the system, that have states and communicate with each other, has been so natural for me that I do OOP until today. So it is no surprise that I also followed the database market and read about ODBMS. I found this idea a logical continuation of the OO paradigm.

In 1999 I've gotten the chance of leading a software development team implementing a content management system based on J2EE. I've tested the usage of ODBMS a short time before using the ObjectStore PSE for Java. So in this project we evaluated some persistency strategies and systems and we decided us for using POET. We had no mass data with a high query load. But our idea has been to store XML in a parsed DOM tree directly inside the database. The elements inside the documents could be regular elements or dynamic elements. Those have been executed for including other documents or querying external databases or data sources before being rendered via XSL-T and then delivered.

Here you allready see one of the major domains for ODBMS. When object structure are complex, maybe hierarchical, and there are no complex queries but an access via identifiers and then through the navigation over the networked objects, then you're right with ODBMS. The entry point is typically a registered root object, often a collection. Dictionaries or b-trees allow the access via a unique key. Also there may be different kinds of entry points, depending on the business case. So customers could be accessed via their customer number but also via a reference inside an invoice retrieved by the invoice number. Tight meshed objects allow a large number of possible navigation paths. This allready shows one disadvantage: all those references have to by maintained by the application. But in relational databases foreign keys have to be maintained, too. So the overhead is tolerable.

Another great plus is that no mapping is needed. Depending on the ODBMS the persistent classes may need to be derived from a special base class, or they need an anotation. Other systems - like our Tideland SOS - handle the persistency totally transparent for the developer. This also applies to the CRUD operations. Depending on new added instances to the network of other persistent objects, navigating between them, changing the values of instance variables or just set them to nil all operations are handled automaticly. The only needed actions are beginning a transaction, navigating starting at a root object, and committing or aborting the transaction. So an ODBMS presents itself as a large persistent extension of the main memory.

OK, ODBMS naturally also have disadvantages. They differ from vendor to vendor. So I just list them.

• The support for queries (e.g. with OQL) is often bad.
• The access through different languages often is difficult.
• Schema evolution sometimes isn't supported very good. OK, that's hard for RDBMS based apps too.

Our Tideland SOS has a tight binding to Smalltalk, queries are only provided by the typical methods like select and schema evolution is currently only supported for additional instance variables and classes. But the concept for more is allready outlined. But it depends on your needs. Our ODBMS is a very lightweight system for small to mid-sized applications.

How about O/R mapper? Hmm, I don't really like them. The mapping, even if it is tool supported, so much additional configuration files, often in XML, not as flexible as ODBMS. Maybe I've just made my experiences with a wrong designed system (about 750 classes managed via TOPlink). But when the usage of an RDBMS is needed - customer constraints, legacy data, application integration, OLAP systems - I prefer patterns like the table data gateways and data transfer objects. Here you also see one of the great advantages I see in RDBMS.: they are query optimized. So if an application system heavily relies on data retrieval relational databases are best. Also if an access through different languages - object-oriented or not - and existing apps is needed they do a really good job.

Does an answer to the question "What is the best persistence strategy?" exist? Hmm, I don't think so. It really depends, see above. But if I'm asked for my favorite, I would answer: ODBMS. *smile*