Embrace change

Yesterday I've introduced the new architecture of the Tideland EAS - formerly known as "Events and Services", now the "Erlang/OTP Application Server". One of the new concepts is the definition of agents for the control of business processes. Those agents are typical callback modules working like finite state machines. The state change is done by internal code, by the returning from services calls, or by the arguments of an external continuation after reaching a waypoint.
 
So here's one example of how to use the agent API from a view or a connector. First the agent will be started, then the reply will be handled asynchronously. Both, the agent and the reply call have a timeout. The first one defines the timespan in which the agent should complete its work, the second one is just for the reply call. 

start_agent(Args) -> 
  % my_agent is the name of the callback module. 
  easagt:start(my_agent, Args, {5, hours}). 
 process_agent(Aid) -> 
  case easagt:reply(Aid, {5, seconds}) of 
   {waypoint, foo, Info} -> 
     ContinuationArgs = handle_foo(Info), 
     easagt:continue(Aid, ContinuationArgs), 
     process_agent(Aid); 
   {waypoint, bar, Info} -> 
     ContinuationArgs = handle_bar(Info), 
     easagt:continue(Aid, ContinuationArgs), 
     process_agent(Aid); 
   {finished, error, Reason} -> 
     {error, Reason}; 
   {finished, ok, Reply} -> 
     {ok, Reply}; 
   {error, timeout} -> 
     {error, timeout} 
  end. 

This is a very simplified code fragment, but it shows how a running agent can stop working when reaching a waypoint where external action - or interaction - is needed. The last shown error is the timeout of the reply call, the inner one can also be a timeout, in this case caused by the agent. If the timeouts aren't passed as integers representing milliseconds the time library of the CEL calculates them based on the tuples.
 
The control of state changes inside an agent is done through the result of the current processing.

process(foo, Args, StateData) -> 
  % Do something ... 
  {next, bar, NewArgs, NewStateData}. 
 process(bar, Args, StateData) -> 
  % Do something else ... 
  {call, [{svc_a, req_a, ArgsA}, {svc_b, req_b, ArgsB}], baz, NewStateData}. 
 process(baz, ServiceCallResults, StateData) -> 
  % Do something with the service call results ... 
  {waypoint, baz, Info, yadda, NewStateData}. 
 process(yadda, ContinuationArgs, StateData) -> 
  % Do something with the continuation args ... 
  {stop, ok, Result}. 

It's easy to see how one or more services can be called, the processing will continue when all services replied or after an optional timeout, or the control can be passed to an external waypoint. Additionally simple service calls can directly be done inside the callback functions. 

{ok, Reply} = eassvc:call(svc_x, req_x, ArgsX, Timeout) 

This API isn't stable yet, but it shows the direction. I'll write more about it here.

Loading mentions Retweet

It's time for a change, a deeper change. A change based on the experiences made during the development and testing of the EAS like it works today. The original idea behind the EAS has been an event-driven architecture where services subscribe to different types of events - therefore the name EAS, events and services. Additionally the platform contains a generic complex event processing. This system now works fine, only the timeout mechanism for replies and time based event processing is missing. So far so good.

 But there's still one problem. The idea has been that business process are a flow of events. Each step can raise one or more events that will be handled then. It's a simple idea, but it's hard to handle. The managing of the control flow is hard, especially when a change is needed. That's not worth it. So the idea of changing the design came up. There are still services, which handle requests asynchronously. Additionally there are agents, each one is a kind of state machine representing a business process. The third part is the complex event processing, still based on subscriptions.

Due to the experiences with the EAS the prototype for this new design will be available soon. The persistence of requests, replies, and aggregations together with the dispatching mechanism like it is implemented today supports the scalability using of large number of nodes in parallel. So the new approach can fulfill the requirements together with an easier API for the later implementation of services, business processes, and event processings.
 
One additional thing that helped a lot during the redesign has been my first usage of the Fundamental Modeling Concepts (FMC / http://www.fmc-modeling.org/). It really helps to design and document software-intensive systems in a simple, clear, and esthetic way. Beside static views like above dynamic ones are supported too. So I'll test it more.

Loading mentions Retweet