Showing posts with label Design. Show all posts
Showing posts with label Design. Show all posts

Tuesday, May 15, 2012

Achieving Decoupling: Dependency Injection v/s Event driven design

(Update: A more exhaustive sequel post is in progress on this topic so don't be disappointed by lack of details and stay tuned)

Coupling is considered to be a bad sign in a software design, having seen enough of it in practice I agree with this belief. The need for decoupling has resulted in a cottage industry of IoC containers in Java ecosystem, where it still remains popular, so this is not a philosophical topic.

When I was learning object oriented software design as a student, the gist I received was encapsulation and message passing between objects with state. In theory this sounded great, objects with behavior and encapsulated state seemed unbeatable (given that I only knew BASIC and C).  In real life, however, I have found that it is hard to keep objects simple while keeping their dependency fulcrum in control. This may not correlate in real OO programming languages like Smalltalk but I can't speak for that.

Given that I've only worked on a relatively medium to larger enterprise applications (ranging from 10KLOC to 1MLOC+)  this might sound a bit biased: most enterprise applications are not object oriented. This is counter intuitive given that OO approach is considered to excel at modelling large systems by breaking it down in to more manageable pieces, but I digress.

Coming back to the main issue, how can we control coupling?

Dependency Injection
The most popular option so far is to exploit dependency injection to invert the dependencies. The touted benefits of it are: testability, flexibility and abstracted, and hence, switchable implementations.
For example:
interface IHttp{  InputStream open(URL url); }
class FireFox implements IHttp{ 
 InputStream open(HtttpURL url){  doOpen(url).andLeakSomeMemoryAsDesigned(); } 
}

class Chrome implements IHttp{
 InputStream open(HttpURL url){ feedUserInfoTo(googleAI).thenOpenUrl(url).makeItFast(); }
}

class InternetExplorer implements IHttp{
 //Since no one uses it apart from downloading other browser,
 // being efficient, user friendly or useful is just a misplaced goal.
 InputStream open(HttpUrl url){ 
  waitIndefinitelyOrForeverFor(url).attemptToLoadWhenever(url).possiblyReadOrJustGiveUp(); 
 }
}

class MavenHttpClient implements IHttp{
 InputStream open(HttpUrl url) {
   downloadKnownInternetFiveThousandTimesEveryHour().completelyIgnoringWtfHttp()
   .then()
   .inflictDarkSufferingOnUserAndBeReluctantlyOpinianatedAboutHowYouScrewUpBuilds().asAlways(); 
 }
}

class MyHappyEnterpriseApp{
 IHttp http;
 setBrowser(IHttp browser){this.http = http} //Injected dependency
 void myBusinessLogic(){ http.open("http://blog.nirav.name").doSomething() }
}
So by coding to interface and relying on a dependency injection "container" we get all the prescribed  benefits. This is the de-facto standard to achieve decoupling in current enterprise application architecture in Java ecosystem.

Event Driver Design
The second obvious alternative (may be not so obvious to many) is to fall back to good old message passing which can be synchronous or async. Isn't OO approach all about message passing and keeping state encapsulated while reacting to meaningful messages? Turns out, it doesn't jive well enough with statically typed languages such as Java.

Instead of lethally injecting objects with dependencies (abstract or concrete) why not just send a message which will be observed by objects who are really interested in responding? The benefits here are inclusive of the ones offered by dependency injection but are more evolutionary.

For example, The similar implementation of the above design can be represented as events:
interface IEvent{}
interface HTTPEvents{
 interface OpenUrl extends IEvent{
   InputStream open(HtttpURL url);
 }
}
class FireFox implements HTTPEvents.OpenUrl{
 InputStream open(HtttpURL url){  doOpen(url).andLeakSomeMemoryAsDesigned(); } 
}

class Eventing{
  Dictionary events; //initialize implementations
  T  sync( evt){
    return events.lookupImpl(evt.getType()).send(evt);
  }
}
class MyHappyEnterpriseApp{
 void myBusinessLogic(){ doSomething(Eventing.sync(HTTPEvents.OpenUrl, "http://blog.nirav.name")); }
}

While simplistic, this implementation should provide general idea that dependency injection is not the only true way to achieve decoupling.

Even the most complex systems are designed around the idea of message passing (Linux kernel, Win API and so on) where it successfully achieves decoupling from user land API (POSIX) from physical implementations (e.g. CPU interrupt faults and traps). This elucidates a strong merit that message passing can be applied to a general purpose applications (which are comparatively less complex). It is not a novel approach to system design but it appears that reinventing the wheel is in the fashion as far as enterprise application development goes. JMS has been a solid spec geared more or less towards similar principles but it is viewed only as a queueing mechanism to offload the async task.

I guess not every one sees coupling from the same angle and starts to build yet another money making framework to achieve the same goal. I hope that agent driven libraries like Akka and Erlang style message passing gets some more attention from enterprise app community to see how benefits beyond coupling (performance, fault tolerance and scalability) are achieved with message passing.

Tuesday, September 06, 2011

Thoughts on Event sourcing

I read about event sourcing a while back and I couldn't stop thinking about it so it had to explode here as a blog post.

A typical data driven application involves CRUD operations on domain entities which are important to business. Such application typically capture data from user or other system in a centralized database for reporting or for further distribution. Architecture of such application is generally simple and there's a vast ecosystem of platforms, frameworks, tools and libraries to support it.

If we wanted to represent traditional design of such application in terms of a state machine, then we can say that such application capture, distribute and allow reporting of domain in a certain state. The primary objective of such application is to facilitate data manipulation (CRUD) which changes state of the domain. For most applications it is generally a sound design ignoring familiar caveats.

Thinking in terms of state machines, there's an interesting alternative: we can store all the state transitions that led initial domain model to its current state. This second perspective to application design has many interesting repercussions. In this approach, current state of domain is no longer as important as the earlier approach because it can be recreated just by repeating all the transitions. This second approach is named "Event sourcing" where event is just a little familiar name for state transition.

Not every application care about the easy recreatibility aspect of domain, most business applications care only about current state of domain. However many applications, especially the ones with mandated audit trails or domain with significant historical data, can benefit from event sourcing. A common example of such application is a version control system, version control systems capture state transitions (diffs) of domain entities (source files) so you can switch to any state (version) and rebuild it to desired state by successive applications of  state transitions (diffs).

As far as business domains goes, Insurance domain by far seems to be a great area of application for event sourcing given that audit trail is a legal compliance requirement and insurance domain models tend to be really complex. Think of an insurance policy as a state and all the changes to it (endorsements) as transitions. By just tracking the transitions, one can rebuild a policy to its current state and reason about it for underwriting analysis and audit. Compare it, instead, with our initial approach of capturing multiple states (identical records in relational database) with complex logic to diff them and comparing them. This approach has profound positive implication on usability of application as well as testability, with this approach it is easy to visualize and rebuild data of interest to any point in time.

One more interesting application of event sourcing, I think, is in data mining. If data is stored as events it is fairly easy to sample, plot and build historical and predictive models. My limited experience in mining data has always involved custom (expensive) efforts to store historical data, such custom efforts usually involve complex development efforts just to extract marginally meaningful information.

It shouldn't be surprising that event sourcing can have significant influence on application architecture which may not be an easy sell especially in a larger setting. There are many related concepts to event sourcing, specifically CQRS which lead to wild architectures (which I'm not quite fond of yet).

I'm learning this is not new or revolutionary and has been done in past but never caught up for whatever reasons, nonetheless I find it interesting. As far as my technical curiosities go I'm very much inclined to try it out with a pet project to see how far these benefits are viable.

Tuesday, April 21, 2009

How Scala's pattern matching can replace Visitors

The primary motivation of Visitor design pattern is to separate model traversal from operational logic. A visitable model takes the responsibility of model navigation while the behavior is defined by arbitrary visitors. In this post I will try to explain problems associated with Visitors in general and how Scala's pattern matching feature can eliminate such problems cleanly.

Consider a simplified Insurance Policy model as follows (In Java):


public class PolicyElement {
static class Quote extends PolicyElement {
protected final Risk risk;
public Quote(Risk risk) {
this.risk = risk;
}
public void accept(PolicyVisitor visitor){
visitor.visit(this);
visitor.visit(this.risk);
}
}

static class Risk extends PolicyElement {
protected Coverage coverage;
public Risk(Coverage coverage) {
this.coverage = coverage;
}
public void accept(PolicyVisitor visitor){
visitor.visit(coverage);
}
}

static class Coverage extends PolicyElement {
protected final Premium prem;
public Coverage(Premium prem) {
this.prem = prem;
}
public void accept(PolicyVisitor visitor){
visitor.visit(prem);
}
}

static class Premium extends PolicyElement {
protected final double amt;
public Premium(double amt) {
this.amt = amt;
}
public void accept(PolicyVisitor visitor){
visitor.visit(this);
}
}
}

public interface PolicyVisitor {
public void visit(Quote quote);
public void visit(Risk risk);
public void visit(Coverage cvrg);
public void visit(Premium prem);
}
public class PolicyTest {
static class PremiumCalcVisitor implements PolicyVisitor {
private double totalPremium;

@Override
public void visit(Premium prem) {
totalPremium = getTotalPremium() + prem.amt;
}

@Override
public void visit(Coverage cvrg) {
}

@Override
public void visit(Risk risk) {
}

@Override
public void visit(Quote quote) {
}

public double getTotalPremium() {
return totalPremium;
}
};

public static void main(String[] args) {
Quote quote1 = new Quote(new Risk(new Coverage(new Premium(10))));
Quote quote2 = new Quote(new Risk(new Coverage(new Premium(30))));
PremiumCalcVisitor visitor1 = new PremiumCalcVisitor();
PremiumCalcVisitor visitor2 = new PremiumCalcVisitor();
quote1.accept(visitor1);
quote2.accept(visitor2);
assert visitor1.getTotalPremium() + visitor2.getTotalPremium() == 40;
}
}


(Generally, we introduce one more abstract class to omit empty implementations in Visitors but I have left it for brevity.)

Now, not so apparent problem here is that if the object model changes (which is more frequently the case in real life), we have to add one more method to PolicyVisitor interface, all visitor implementations if change is substantial and have new Policy elements implement visitor methods. This invasive nature of Visitor couples it tightly with the model.

With pattern matching and views in Scala, you can have alternative implementation which is precise as well as non-invasive unlike visitors.

class PolicyElement
case class Quote(risks: Risk) extends PolicyElement
case class Risk(cvrg: Coverage) extends PolicyElement
case class Coverage(limit: Premium) extends PolicyElement
case class Premium(amt: Double) extends PolicyElement
object PremCalcTest {
class PremCalculator(pol: PolicyElement){
def calcPrem : Double = calcPrem(pol)

def calcPrem(policy: PolicyElement): Double = policy match{
case Quote(risk) => calcPrem(risk)
case Risk(coverage) => calcPrem(coverage)
case Coverage(premium)=> calcPrem(premium)
case Premium(amt) => amt
}
}

implicit def calPremV(pol: PolicyElement)= new PremCalculator(pol)

def main(string: Array[String]){
val risk1 = Risk(Coverage(Premium(10)))
val risk2 = Risk(Coverage(Premium(30)))
println(Quote(risk1).calcPrem + Quote(risk2).calcPrem)
}
}

This code requires some explanation. What we have done here is we labeled domain classes with a 'case' keyword in Scala. If you tag a class with 'case' it can be used for pattern matching in a switch-case like structure as done in method 'calcPrem'. You don't need to create members or setter/getters for them, they are created by compiler for you. A case class can be instantiated without 'new' keyword; So Risk(Coverage(Premium(0)) is translated as new Risk(new Coverage(new Premium(0D))) in equivalent Java code.

The code in 'calcPrem' function can be assumed to be something similar to instanceOf checks for each possible case in Java, for example:


if(object instanceOf Premium)
return ((Premium)object).amt;


What we also have done silently is added a method 'calcPrem' to PolicyObject class. This is done through implicitly defined function 'calPremV', this will allow us to call 'calcPrem' method on any PolicyObject without actually modifying the domain model code. This type of lexically scoped class extension is known as a View in Scala and is similar to what is available in Ruby as open classes except without scoping.

In case if the model changes in this case, we just need to modify a single function and we are done. These programming language features of Scala frees us from coupling introduced by inheritance.

So it is easy to see that Scala's language features can be elegant and far more powerful than other languages (specifically Java) without sacrificing compiler checks and type safety.

Monday, October 22, 2007

On IClass (anti-)pattern

I'd been re-reading this post by Sidu and the comments on it. I had to do exactly similar to IClass anti-pattern for one of the development effort. And I think having IWhatever completely makes sense for the current state of affairs in my development environment.

While it's unfortunate, most of the time, to have an Interface for just about every class in the code-base, It does have certain uses in some situations (apart from the (IoC) frameworks which mandate you to have them) like:
  • Support parallel development.
I will define parallel development as a situation where there are many components involved in developing a software and your component (or module or whatever..) and it's arbitrary version has to support other components (and their integration tests depending on your environment). For example, for something as important as (so called) service layer in an Enterprise application you need the Interface with single implementation at one time, it might just be a 'placeholder implementation' for a dependent component while the new version being developed parallely. It just makes sure that there is something to hold the place which can be tested easily. This is one more learning from distributed development, you're not alone developing it man!
  • When using Generative development methods
If you use code generators heavily, say, most of your code-base is generated out of models (UML, BPMN, BlahL ...) then depending on the generation pattern you'll find interface for just about every implementation (or other way around for EMF generator - one implementation for each interface). This is necessary as you might want to customize forward engineering by overriding parts of generated artifacts.

Most of the Eclipse code does have this pattern, for every class there's IClass. For Eclipse, In my opinion, it is justifiable (to some extent) to have them considering it is a development platform rather than just an application. However, I would be more than interested to know conceivable reasons to have those IInterfaces which I might be missing.

Saturday, February 03, 2007

Refactoring and Upfront Designs

Low level software design is one of my regular work, most of the time it turns out to be really interesting; so much so that I fail to sense environmental changes around me :). Most of the decisions in such designs, confined to refactoring, are upfront.

There were times when these decisions used to drive me nuts (for example, I used to extend framework at larger granularity, failed to identify framework extensions at right places etc.), with time (and few refactoring book skims) I learned how to simplify it. Some of them are separating the concerns of mammoth classes into few "aware" type of interfaces (Spring style), distribute methods, extract stateless routines, derive common abstract base class etc. These decisions are quite easy to tackle with upfront designs.

Refactoring is wonderful, refactoring to patterns is even more interesting. Recently, Ketan identified design smells in his GUI testing framework for SWT. I did a test drive into the initial framework to understand and tried the same upfront approach to make it smell free, I failed; but I understood the potential pain points. Then, We brainstormed and discussed the design on white board, espied the smells; asking questions, suggesting and rejecting alternatives designs. That cleared few confusing points and we came with a good enough, smell free design (Ketan, all we did was shifting and distributing the bottleneck somewhere else :), as it can't be eliminated - "SWT"- I wonder why SWT is not very cleanly designed, performance?).

The lessons learned? You can't refactor well if you're in confusion (requirements or other), refactoring in this case drags you away, and Upfront designs takes extensive experience on variety of problem domains.

Tuesday, October 17, 2006

Reuse Pain in Eclipse UI Code

Code Reuse is one of the most interesting, if not alluring, part when you are contributing to Eclipse. In eclipse, many a times I'm tempted to use code of other plugins. Eclipse guys like to keep UI-code 'internal' , still, I try and again to reuse, which ends up in "monkey see/monkey copy+paste" with added disarrays.

This temptation is not just because of my indolence to duplicate development cycles but something which is commonly done (it's all about reuse right?), ofcourse at very clear granular leves (called extensions) which, I, somehow find limiting.

It is too frustrating to figure out sheer context information for some code deep in eclipse dungeons, the more you read code, deeper it takes (and ofcourse how can the temptation of what is inside be controlled?). This results in a tempest of hazardous shortcuts which keeps you away from starting afresh or thinking in different way and you end up in violent mess of yellow signs and incomprehensive code !

Recently one of my collegue ran in similar situation, I looked in to it to see how serious it can get, I too quicky drawn in it. The task involved implementation of preference pages for code/comments etc. in our code generation templates, as well as templates for ADDL( home grown action semantics language). We started looking into JDT UI, which has a preference page with similar functionality named "CodeTemplatePreferencePage" which delegates UI to CodeTemplateBlock which in turn owns inner classes to do "real" stuff. There are hardly any hooks where it can be extended "as is". We started with extending it and ended in copy pasting somewhere around 6-7 java files (and contrived satisfaction that it was looking good to end-user).

It just didn't helped! we finally created UI using Eclipse Visual Editor and programmed widgets/viewers and preferences in less than one forth the time we spent in reading and fumbling in JDT-UI code.

Lessons learned after hours of pairing, never try to reuse what is "internal" and don't ignore it no matter how tantalizing it is!

Wednesday, August 16, 2006

Story of a Decoupled Visitor

I find visitor pattern mesmerizing. Master Uncle Bob has published great articles on visitor family of patterns (Acyclic Visitor, Visitor Family of Design Patterns) apart from others.

In his articles he illustrates how visitor can be applied to prevent interface bloating by introducing visitation support in the domain model (The Modem Hierarchy in the article) and how easy is to introduce new visitation based behavior without modifying the hierarchy.

So far so good, this approach works for the developments where you have the luxury of initial design. You may add support for visitation based traversal in the domain model or the layer extending it, or your hierarchy code generator, say antlr or JavaCC, may generate cool visitor based code so that its easy for you to compute metrics or do other stuff on the model (any sophisticated AST is a Model, I'll prefer calling class hierarchies a Model here).

Imagine we have a model wherein visitor is disliked alien (ayee! ugly! visitor is an implementation detail, not needed in my platform independent model!!!), in short, I've a "pure" domain model. But our model composition is so complex that normal iteration isn't going to give us big advantages, and we need extensibility as well.

I have faced similar design problem back when I was developing a intra-model transformation framework, where, the model was predefined without visitor supported api. During transformation, every now and then an element search from model was becoming inevitable, every searcher wanted to do something interesting and unique. For example, Let's say I want to find an element from source model and do a shallow copy of it and add it to target model and so on. I tried to fit visitor for this design problem, I wanted the extensibility without bloating the existing model.

I can't add the visitation API in the given model, but I can always create a utility (or add api to context object) which accepts model and visitor and does a generic traversal (as in loop based or recursive) as shown in the diagram figure.

Context holds the actual model and defines utility api to invoke visitor implementation. Leaving other details, each client (*ModemDiagnostic in this example) can create an implementation of Visitor and visit the hierarchy. Context might end up doing a list based iteration for full scan of the model which can be controlled from the visitor. This variant double dispatches for each model item but is totally decoupled from the domain model (there u go, greetings!!).

I've used this pattern repeatedly (BluePrint Foundry Transformation Framework, FileSytem Utilities) but I can't tell it a "regular visitor" or acyclic visitor, but I'll call it a Decoupled Visitor. Its easy to identify that design solution offered by this approach can be replace by iteration and template methods and inheritance, but then there's trade-off between inheritance and composition ans so is the story.