For the love of beer

Funny ad, see what it takes to make a beer ;), never seen trebuchets being used for throwing beer ingredients (esp. nv girls, animals...). Cheers!

Executable Models

Imagine...

A work place where your regular tasks involve analyzing and modeling the parts of system based on stories (or requirements, whatever) and writing few snippets of instructions. You've a world class IDE with some wonderful perspectives, editors, views and a state of art execution infrastructure. The IDE allows quick execution of your models to test them, allows visual model debugging while encouraging ergonomics to prevent heavy "keying and mousing", IDE supporting finest model refactoring to allow painless design change, while providing real time collaboration for distributed brainstorming and discussions...

You pair model it, you test drive it, you version it at finer granular unit of control, you've a catalogue of blueprints (design + domain) which can be used statically as well as in context of existing system, you have continuous integration of models which compiles, validates (tests) and builds the system. At end of the day, you've a system which can simulate in development environment as well as deployable as production software....

Imagine a execution infrastructure able interpret UML (relax, take unambiguous parts of UML or Mellor subsets) directly, or may be it emits a script which can be plugged in existing JVM, or may be UML VM itself with attach on demand and hot model replacements :) ....

I dream to be in such an environment as well as developing such environment.

Realize....

These imaginations, although will sound like philosophies, are not without basis, existing tools and technologies can be leveraged and bridged to provide this envisioned platform; it's hard, takes lot of resources but not impossible. Ok, it will not give quick returns, will have adoption problems which can be handled.

At first, Executable UML will seem like a buzzword, to me it sounded daunting and it seemed like everyone wants to change the world. That change is for good, how long you want to work with something as ugly as XML or JSP and other complexities which in no way adds to customer requirements? They deserve to be generated or they don't even deserve to be in our codebase (I should say modelbase :) ).

Executability of models is not something entirely new (apart from Virtual Machines, OOP etc.), guys like Steven Mellor and Marc Balcer are working on it for quite sometime now.

If you feel, that, support for such tooling is not enough; Let me tell you, Eclipse has made such feeling "Past"; With Eclipse, We can leverage most of the Eclipse related technologies for achieving it, ECF for collaboration, UML/EMF/GEF/GMF for modeling, Debug Framework and GEF for simulation and model debugging (in this case, we may need to change the abstraction semantics of stack frames etc.) . Of course building VM or runtime is non-trivial, but it's worth trying alternative approaches like integrating with existing runtime.

This is the time for improving the software crafting (or engineering?), no one needs costly software, and current fashions make them expensive; Well its different topic to debate about productivity levels of existing methods but it can surely be improved.

When will 'Software Modeling' catch big time attention?

From last one year, I'm working on a modeling tool featuring support for UML2 and BPMN1.0 specifications. Well, this tool is not envisioned just like "yet another modeling tool" allowing you to draw basic elements (and strange visual manipulations) and generate structural code and documentation (which is not impressively useful!).

It's commonly known that modeling is good for visualizing and documenting a system. Well, I've heavily used UML for documenting and I found it useful for communicating software blueprints to those who don't understand the technology well.

Contemporary tools can do better apart from visualizing, Typically a modeling tool doesn't sponsor full blown generation of an application for target platform, say J2EE; Not that it's insane attempt but it's simply too much of work. They generate structural code (skeletal) from model (say, classes from Class Diagrams) ; this is commonly known as forward engineering. Tools also support round trip engineering for regenerating the code from model while maintaining the manual customizations.

How redundant, Why to model and code at the same time and why maintain both?

The tool I'm working on is supposed to generate "typical application" end-to-end (heh, really?!). Also, Tool has a special language(sort of BASIC dialect) , that can be used to write business logic for the application which might, otherwise, result in tedious micro modeling (boy, you will require incredible mouse capabilities for that level of modeling). This language is customized implementation of OMG(tm) Action Semantics Language (ASL) and Object Constraint Language (OCL); The logic in this language is translated in to platform specific code during transformation and code generation.

Technically, this is enough for generating entire application. Recently, tool vendors are trying to avoid the ASL approach altogether and use supplemental UML models (like activity and state machines) to specify behavioral aspect of system; but this approach has usability problems, although it is an interesting technical challenge (umm... given enough beer and I'll even admit that I wrote a state machine compiler for money to generate behavioral code..).

How bad again, Why I want to learn UML, a dumb ASL language and still struggle with generated code to customize it?

I agree that the tool can generate "student projects" like applications, but still keeps me skeptical about *serious* applications which involve complexities of security, performance and integration. It's not impossible to do that either, however, supporting multiple security/persistence/application frameworks/third party interfaces for multiple platforms, in its entirety, would be a non-trivially huge attempt ever made.

Lately, I'm observing generative methods in regular development; these days, most applications are combination of generated + hand written code(in some reasonable combination), people manage both code as well as model. Generative Technologies are gaining acceptance gradually, especially because support from Eclipse projects like EMF, GEF, GMF and other Modeling subprojects apart from rising community interest, hundreds of modeling tools are available and major players are investing in this technologies.

We have witnessed the transitions in technologies; Transitions from Hardware specific Assembly code to C code, C code to C++/Java/RoR code.., these transisions are fine and gradual, but with modeling being a paradigm shift in the way we see software development (its not just about learning new language, right?), modeling imposes the steep learning curve, and understanding the mammoth specifications. It's always difficult to adopt a different way of doing things.

With the current state of tooling and specifications, it is difficult to offer "Generate everything, Write Nothing" jargon feature, and its pain in a**e to maintain both model and code in sync (tools are improving though). I'm sure of one thing, generative development is going to be the next generation of software development, not sure how many more years...

Escape Analysis in Mustang - II

EDIT: As commented by Brian, these Escape Analysis optimizations were dropped before final release and deferred to Java 7. The debug version of VM can no longer be found at http://download.java.net/download/jdk6/6u1/promoted/b03/binaries/jdk-6u1-ea-bin-b03-windows-i586-debug-19_jan_2007.jar

In my previous post, I mentioned that escape analysis is available in mustang releases and debug flags available for it in HotSpot(tm) VM.

I ran some micro benchmarks yesterday, they are fairly trivial but good enough to identify the performance gains, and here is what I found:

import java.io.ByteArrayOutputStream;
import java.io.OutputStream;
import java.io.PrintStream;

public class TestEA {
   public static class MyPrintStream extends PrintStream {
       public MyPrintStream(OutputStream out) {
           super(out);
       }
      
       @Override
       public void println(int cnt) {/* deep inline candidate */
       }
      
   }
  
   private static final int COUNT = 10000000;
  
   public static void main(String[] args) throws Exception {
       System.setOut(new MyPrintStream(new ByteArrayOutputStream()));
       for (int i = 0; i <= 10; i++)
           test();
      
   }
  
   private static void test() {
       int cnt = 0;
       Object objLock = new Object();
       long start = System.currentTimeMillis();
       for (int i = 0; i < COUNT; i++) {
           synchronized (objLock) {
               cnt++;
           }
           consume(cnt);
       }
       long end = System.currentTimeMillis();
       System.err.println(cnt + ", time=" + (end - start) / 1000.0);
   }
  
   private static void consume(int cnt) {
       System.out.println(cnt);
   }
  
} 

The results
VMArgs: -server

10000000, time=0.907
10000000, time=0.938
10000000, time=0.969
10000000, time=0.906
10000000, time=0.906

With
VMArgs: -server -XX+DoEscapeAnalysis
10000000, time=0.89
10000000, time=0.922
10000000, time=0.016
10000000, time=0.016
10000000, time=0.015
10000000, time=0.016

JVMOut:
28 JavaObject NoEscape [[ 54F]] 28 Allocate ....
40 LocalVar NoEscape [[ 28P]] 40 Proj ....
90 LocalVar NoEscape [[ 28P]] 90 Phi ....
213 JavaObject NoEscape [[]] 213 Allocate ....
225 LocalVar NoEscape [[ 213P]] 225 Proj ....
======== Connection graph for TestEscapeAnalysis::test ....
172 JavaObject NoEscape [[]] 172 Allocate ....
184 LocalVar NoEscape [[ 172P]] 184 Proj ....

I ran the loop in main for few times because server VM does a deep inline and it might optimize the execution considering cnt is not significantly used anywhere.

Well, It can be seen that escape analysis successfully eliminated synchronization on objLock (lock elision) . As I posted earlier, synchronization has significant impact on execution speed, elimination of this heavy operation improved the speed substantially. Consider the effect of it on a highly concurrent web server (JAWS) handling hundreds of simultaneous requests. Of course, it happened because objLock is allocated locally and VM identified that it can't be shared between multiple threads and its safe to remove the sync overhead.

I also tried making objLock a static field of the class and found that escape analysis has no positive impact on execution speed as can be seen here:

VMArgs: -server -XX:+DoEscapeAnalysis (objLock as static field in above program)
10000000, time=0.922
10000000, time=0.922
10000000, time=0.906
10000000, time=0.891
10000000, time=0.921

As expected, objLock being a shared field, VM has no way of identifying whether lock on it can be eliminated or not.

Unfortunately, stack allocation seems to be absent in current builds of mustang (or there were no hints in debug output as when it was done. Also, its hard to decipher the VM output and I found no explanation for it). Here is an excellent presentation on new optimization in HotSpot(tm) VM. And for lock elimination enhancements refer to this.

So, there you go, one more optimization for the managed runtime; for those who are still in the illusion that native static compiler optimized programs are the fastest, think again, your programs are static at runtime can't organize itself, managed runtime is metamorphic, it can adapt and substantially optimize itself at runtime. Man, that's what I call programming.

Escape Analysis in Mustang - I

EDIT: As commented by Brian, these Escape Analysis optimizations were dropped before final release and deferred to Java 7. The debug version of VM can no longer be found at http://download.java.net/download/jdk6/6u1/promoted/b03/binaries/jdk-6u1-ea-bin-b03-windows-i586-debug-19_jan_2007.jar. Production VM here means production release type, as compared to debug, not the final release.

Java 6 brings a lot of improvements in (much ignored) synchronization; like lock elision, adaptive locking etc., under, the awaited, escape analysis hood. I've been reading quite a bit on stack allocation and lock detection techniques to reduce GC overhead. (as stack comes for free; and sometimes, on full speed core).

Escape analysis is one of the interesting techniques in compiler optimization, read more here.

Few days back, I posted an off-topic curiosity on lang.java google group, It's not officially mentioned anywhere that escape analysis is implemented in mustang releases. Brian Goetz wrote an article stating EA is available in mustang builds, that article is pretty old and someone questioned whether it's really available or not...

A web search revealed that escape analysis *is* available in mustang (not well marketed?), and production VM binary (I've b-03) supports it out of the box in server mode; debug binaries supports few more switches to see the analysis. I'm trying to benchmark it with few programs playing around large number of object creation, not a big deal.

Well, if you are interested, try these switches with mustang (in server mode)
-XX:+PrintEscapeAnalysis,
-XX:+PrintOptoAssembly,
-XX:+DoEscapeAnalysis (works on production VM only).

May be this weekend, I'll post the results of the benchmarks and other findings on it.
Java's performance is getting better and better..

Importance of System Metaphor

Kent Beck describes System metaphor in his legendary book Extreme Programming : Embrace Change as A Story that everyone in XP team can tell about how the system works.

So? Why do we need one more buzzword? Why should I care about it?

Consider a team working on a large software application; typically work is componentized by the architects and/or managers. Every member works on specific part of the component or on specific component. Few people (Software Heroes) know what is the real picture of the application and how it works or is supposed to work. (As a metaphor, every member in group is holding/attached to parts of an elephant, some one is holding his pillar leg, some one holding tail and few holding trunk and god-sized ear; while no one knows how he looks like and how to sell it! Nawal, thanks!).

The result? Software heroes become the concentration point (High risk :)); lack of shared vision in team leads to communication gaps (Our product can generate application, end-to-end in three different platforms); Everyone in the team has her own (mis)understanding of the application (we're building a modeling tool, no- it's a part of framework; ok, that framework is persistence framework- may be that framework is generated using our tool); frequent suboptimal technical decisions (e.g. selection of specific framework which may not fit well with other's work- we'll code GEF editor, we will generate GEF Editor using GMF); rework/duplication (existence of several similar utilities in different module);

Having worked in such scenario, I realized that lacking shared vision or system metaphor results in chaos (implementation to deployment), countless bad decisions and loss of interest in regular work. Everyone becomes concerned about his/her own work rather then end-product (customer satisfaction) - there is no more a collective ownership. Work becomes chore and no fun :(. We may even lose out good ideas from team which might be real worth. Team becomes just a mixture of testers, coders, analyst and so on(and you'll not mind being called a Java Resource!).

One more thing I love about XP is, it keeps everyone informed of what is going on, which in turns backs feedback cycles. When everyone knows what is going on, they participate in overall effort rather then mere work assignment.

I understood the importance of system metaphor the harder way..

Refactoring and Upfront Designs

Low level software design is one of my regular work, most of the time it turns out to be really interesting; so much so that I fail to sense environmental changes around me :). Most of the decisions in such designs, confined to refactoring, are upfront.

There were times when these decisions used to drive me nuts (for example, I used to extend framework at larger granularity, failed to identify framework extensions at right places etc.), with time (and few refactoring book skims) I learned how to simplify it. Some of them are separating the concerns of mammoth classes into few "aware" type of interfaces (Spring style), distribute methods, extract stateless routines, derive common abstract base class etc. These decisions are quite easy to tackle with upfront designs.

Refactoring is wonderful, refactoring to patterns is even more interesting. Recently, Ketan identified design smells in his GUI testing framework for SWT. I did a test drive into the initial framework to understand and tried the same upfront approach to make it smell free, I failed; but I understood the potential pain points. Then, We brainstormed and discussed the design on white board, espied the smells; asking questions, suggesting and rejecting alternatives designs. That cleared few confusing points and we came with a good enough, smell free design (Ketan, all we did was shifting and distributing the bottleneck somewhere else :), as it can't be eliminated - "SWT"- I wonder why SWT is not very cleanly designed, performance?).

The lessons learned? You can't refactor well if you're in confusion (requirements or other), refactoring in this case drags you away, and Upfront designs takes extensive experience on variety of problem domains.

The Pune Eclipse Developer Group

Sriram and Ketan initiated The Pune Eclipse Developers' group to promote collaboration between eclipse developers in Pune. I came to know about good strength working on eclipse plug-ins and related technologies in Pune, its good idea to get them together, share ideas-techniques and socialize with like minded people around town.

Recently at GNUnify '07 Ketan and Sriram spoke about Eclipse Rich Client Application Development. Eclipse Enthusiast can find the presentation here and workshop code here. The presentation is interesting, explains Eclipse architecture; frameworks around it; benefits etc.

I wish Pune Eclipse Developers' group existed before few months, I would have found few more good guys for my team :).