Friday, December 05, 2008

Introducing PyConsole!

I have been pretty inactive this year, unlike what I planned (yawn, haven't heard from DLTK since June either) . I started learning Python earlier this year and felt the need of an embedded python shell inside Eclipse. This was something PyDev is missing [Edit: Simone pointed out that PyDev already has this neat feature, I was late to discover it] as compared to other python tools in Linux such as Eric. So I decided to write one.

PyConsole is direct result of my irregular and intermittent Eclipse development (lately), it's very simple plug-in with a basic exploit of Eclipse Console View and manages two way synchronized I/O from the python interpreter process controlled by a background Eclipse job. It provides few basic features anyone might expect from a shell such as syntax highlighting, command history etc.

Here's the code repository if anyone would like to check it out. If you have suggestions or feedback please write to me or create a bug.

Saturday, August 16, 2008

Productive keyboard shortcuts for Eclipse Editors

I've seen very few developers using true power of Eclipse editors, so I thought of listing few very handy shortcuts that can significantly improve your source editing and navigation speed with Eclipse.

1. Ctrl + J / Ctrl + Shift + J : Inline search, meaning you can type on the fly to search text in current editor. You can use Up/Down arrow keys to navigate between matching text fragment in forward/reverse direction. This is one of the most powerful features you will find in advanced editors like Vim or Emacs.

2. Ctrl + K/ Ctrl + Shift + K : Search text in forward or reverse direction.

3. Ctrl + L : Go to line.

4. Ctrl + Alt + Up/Down : Duplicate current line above/below. Very useful in repeatative assignment statements (Java code, eh?).

5. Alt + Up/Down : Move current line up or down. Also, very handy to select current line.

6. Ctrl + . OR , : This is contextual and may be specific to Java editors: if you have error or warnings in current file in editor it will navigate to them.

All of these except for no. 6 are generic text editor features which are inherited by all eclipse text editors. You will realize that you will work much faster than average user if you start using these key bindings.

Wednesday, August 06, 2008

Synchronizing client requests using Ajax

Before few months I was dealing with a problem where I needed to synchronize the server call sequence on browser. The problem was something like this: Client code will open new window navigating to a different page, but that should happen only after post on current page. The link that new window was opening needed to be server synchronized because of database updates etc. on server.

Being ignorant java-script developer I went ahead and put 'window.open' in onsubmit event of submit button, hoping it will be executed before form is submitted; Apparently, it failed. The event handling/dispatching/capturing in java-script is in sad state - it's specific to browser and I must make it work for IE so the pain is obvious. To reveal some more ignorance, I thought of putting 'window.open' in onmousedown event assuming form will not be submitted till the code in onmousedown is executed
fully, that too failed. Then, it was clear that 'window.open' was essentially a thread fork and will execute in parallel so this event capturing is not going to work here.

Since there was no obvious fix to my problem, I thought of using XMLHTTP for asynchronous wait to synchronize the request. I created new page which will open in a new window with javascript that handles form post in parent window, looks like following:
<html>

<body onload="postParentWindow()">
<%
out.print("Loading Print Preview, Please wait...");
%>
</body>
</html>


getFormValues is a function which concatenates all the input's selected values and makes a request string. More interesting things are done in makePOSTRequest method:
var http_request = false;
function makePOSTRequest(url, parameters) {
http_request = false;
try {
http_request = newActiveXObject("Msxml2.XMLHTTP");
} catch (e) {
try {
http_request = new ActiveXObject("Microsoft.XMLHTTP");
} catch (e) {}
}
}
http_request.onreadystatechange = navigateToOtherPage;
http_request.open('POST', url, true);
http_request.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
http_request.setRequestHeader("Content-length", parameters.length);
http_request.setRequestHeader("Connection", "close");
http_request.send(parameters);
}

function navigateToOtherPage() {
if (http_request.readyState == 4) {
if (http_request.status == 200) {
window.navigate('newpage.jsf');
} else {
alert('There was a problem with the request.');

}
}

(Note: This code completely disregards browser compatibility and good error handling etc.) The important thing here is, we can assign a function to XMLHTTP object that will be executed after server response is received so that serves my purpose of synchronizing the request. I find this use of AJAX pretty useful in building stateful web-applications. The code might look very crude but powerful abstractions can be built over it to make a nice utility framework for such use-cases.

Tuesday, August 05, 2008

What's wrong with HP Quality Center?

This entry is rant so you might just want to skip it.

HP Quality Center (formerly Mercury) is a, how so amusingly called, web-based test management tool. You will find a lot of marketing information if you ask Google, most of which is irrelevant and may only have 1-800 number for support; because they know when you are looking for it on web, you are in trouble and no one can help you. Unfortunately, I've to spend a good amount of time on this tool these days and it inspired me a great deal to write my inexpressible ovation for this tool here.

As a programmer, HP Quality center is one of the worst and most annoying tools you may have to work with (some may compare it with Lotus notes though).

This tool is implemented as a giant Active X; Apparently, they didn't want to make it web-based but when sales team threatened their developers, they some how adjusted their extra-thick client to web by (evil) means of Active-X(TM) technology. It only takes one or two more minutes to load than my Eclipse instance does, in Intra-net i.e.. It works only on lame Internet Explorer, in a sense that makes it 70% less web based. To add to the insult of using this tool, you have no option but use god-forsaken IE6 (or 7 if you are slightly lucky).


HP Quality center crashes at will, each crash is designed scientifically so that you waste a few hours to get back to normal work before it crashes again. For example, I add attachment to a defect; next, I go and try to add comment on it and it crashes - right on time. I open new IE window; browse it and wait for it to load, log-in again (this special web based tool doesn't have feature to remember your password in an enterprise-wide SSO environment). After recalling what I was doing last, I go to the defect where it crashed and try to add comment again; only to discover that it still thinks I'm working on it. 'The object is locked by user: [my userid]'. You will praise arcane brilliance imbued in this tool because these problems disappears randomly once it pisses you off - it knows when.

Some more praise on it's resource utilization, so one can't wonder why it is rightfully web-based. HP Quality Center uses ~120mb of your core if you have no defects in your view. This amazing tool can make your wish to see memory usage on crack come true, have 50-40 defects in your view and see the difference.

This is only the tip of the iceberg but gives you bigger picture and idea why companies make more money in support, there's always room for new version, patch, hotfix etc. Why in the world people pay for such crap? there are ton of free alternatives which are hundred times better and are totally free. And the other day someone was complaining about open-source tool usability...

Thursday, April 24, 2008

How to: Always eagerly load objects from database with Hibernate 3 (without changing configuration xmls)

So it was getting annoying to debug my app. with Hibernate 3 for it's 'lazy load everything by default' feature. Every time I wanted to see what's the value of some collection I will see these weird CGLIB proxy members lurking in debug views.

CGLIB$CALLBACK_0    org.hibernate.proxy.pojo.cglib.CGLIBLazyInitializer@9be3aa
class$net$sf$cglib$proxy$InvocationHandler interface net.sf.cglib.proxy.InvocationHandler
class$net$sf$cglib$proxy$NoOp interface net.sf.cglib.proxy.NoOp
class$org$hibernate$proxy$pojo$BasicLazyInitializer null
componentIdType null
constructed true

For every association I would either evaluate the code or create expressions to see what's coming from DB. Lazy loading makes debugging more slower than it already is. What's the big deal?, you will say, Just go ahead and change configuration to lazy-load = "false". Well, I can do it but changing more than five hundred .hbm.xml files will help me more with retiring early than fixing the problem at hand.

So before it becomes nightmare for me I thought of hacking hibernate to make it eagerly load entire object from database. I'm saying hacking because there's no API in Session or SessionFactory interfaces that would allow you to unwrap these un-dead proxies automatically.

Here's how you can hack hibernate to prevent lazy loading:

The basic idea of making hibernate do what you want is to invert it's assumption, i.e. make everything load eager by default, we'll plug ourself to Mappings class and will force eager load implicitely. Follow these steps:

1. First create a class in org.hibernate.cfg package which extends Mappings:

package org.hibernate.cfg;

import java.util.List;
import java.util.Map;
import org.hibernate.MappingException;
import org.hibernate.mapping.PersistentClass;

public class MyMapping extends Mappings {


public MyMapping(Map classes, Map collections, Map tables, Map queries, Map sqlqueries, Map sqlResultSetMappings,
Map imports, List secondPasses, List propertyReferences, NamingStrategy namingStrategy,
Map typeDefs, Map filterDefinitions, Map extendsQueue, List auxiliaryDatabaseObjects,
Map tableNamebinding, Map columnNameBindingPerTable) {

super(classes, collections, tables, queries, sqlqueries, sqlResultSetMappings, imports, secondPasses,
propertyReferences, namingStrategy, typeDefs, filterDefinitions, extendsQueue, auxiliaryDatabaseObjects,
tableNamebinding, columnNameBindingPerTable);

}

@Override
public void addClass(PersistentClass persistentClass) throws MappingException {

persistentClass.setLazy(false); // No Proxies, load everything in one shot
super.addClass(persistentClass);

}

}

The 'addClass' method achieves all we need, just make persistenceClass non-lazy, it would be lazy by default.

2. Now hook this class with Configuration class.
    public class MyConfiguration extends Configuration {
@Override
public Mappings createMappings() {
return new MyMapping(classes, collections, tables, namedQueries, namedSqlQueries, sqlResultSetMappings,
imports, secondPasses, propertyReferences, namingStrategy, typeDefs,
filterDefinitions, extendsQueue, auxiliaryDatabaseObjects, tableNameBinding,
columnNameBindingPerTable);
}
}

There you go, You can now create session factory out of it and you will get fully debug-compatible Java objects loaded from db.

This was way easier to achieve than I could think hooking hairy SessionImpl beast and making org.hibernate.event.LoadEventListener.LoadType INTERNAL_LOAD_EAGER from INTERNAL_LOAD_LAZY without changing SessionImpl itself.

Other than that, I found some talking about lazy loading being problematic with XStream serialization. Threre's a JIRA defect for XStream relating to this as well. I faced the same few weeks before when I thought of removing hibernate from test cases, XStream basically throws up when it finds a confusing CGLIB proxy like this:

com.thoughtworks.xstream.converters.ConversionException: Cannot handle CGLIB enhanced proxies with multiple callbacks
at com.thoughtworks.xstream.converters.reflection.CGLIBEnhancedConverter.marshal(CGLIBEnhancedConverter.java:85)
at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:55)
at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:50)
at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:73)
at com.thoughtworks.xstream.core.ReferenceByXPathMarshallingStrategy.marshal(ReferenceByXPathMarshallingStrategy.java:34)
at com.thoughtworks.xstream.XStream.marshal(XStream.java:765)
at com.thoughtworks.xstream.XStream.marshal(XStream.java:754)
at com.thoughtworks.xstream.XStream.toXML(XStream.java:735)
at com.thoughtworks.xstream.XStream.toXML(XStream.java:725)

I faced this problem before as well and now I don't have to worry about it anymore. You guys can probably just go ahead and load your objects as I described above and your serialization would work fine.

Thursday, March 13, 2008

Unusual tip to find refactoring hot-spot

Disclaimer: This post contains generalization based on my experience, it may or may not be applicable in every state of code-base resembling this situation.

Most of you have probably been through this list of smells to refactoring. In Recent refactoring sessions I discovered a new (at least unusually new) way to identify "hot-spot" needing refactoring.

This might be in turn related to some other smells (like most of them), but here it is:

- A (set of) source file with significantly more revisions compared to it's dependencies demands refactoring.

The obvious question here is why?
- Number of revisions for a source file indicate that it is being modified more frequently than it should. There can be many reasons why a source file is being modified among:
  • Imperative requirements change or
  • A source file that tries to do so much than it should
Now, requirements change is something very common in everyday life. But generally speaking ( opps! ), Not all requirements ask for a significant change in control flow but data. Depending on the domain, you can externalize the portions which keep changing to helper classes or configurations.

A source file trying to do more than it could is always a problem and is generally perceived as 'God module', it's time to slice it down.

So next time you feel like hunting for bad code, look at the revisions and you will have hint where to start. Good hunting.

PS: Note that I've deliberately generalized this to 'source files' and not the configurations.

Tuesday, March 04, 2008

How to find which jar file contains your class at 'runtime'

If you ever wanted to know which jar your class belongs, or wanted to access the meta-information from that jar file (such as certs from Manifest.mf etc.), here's how you can do it.

    public URL getJarURL() {
URL clsUrl = getClass().getResource(getClass().getSimpleName() + ".class");
if (clsUrl != null) {
try {
URLConnection conn = clsUrl.openConnection();
if (conn instanceof JarURLConnection) {
JarURLConnection connection = (JarURLConnection) conn;
return connection.getJarFileURL();
}
}
catch (IOException e) {
throw new RuntimeException(e);
}
}
return null;
}
This would work in most of the enterprisely Java environment (unless your classloader is hacked by some stupid framework in between), not sure if this would be valid under OSGi container.

Why in the name of programming I need to write such trivial hacks? O Java Module system (JSR277), I await you.

Note: This is the simplest way I could write, if you know smarter way, feel free to comment here.

Monday, February 25, 2008

Six reasons why you should know internals of your IDE

May be this is overwhelmingly obvious to geeks, but I am going to describe it nonetheless. This might not significantly impress non-developers or the people and programmers (yes, they are different) who can't describe the word 'tooling'. A developer's productivity is highly dependent on the tools they use, whether it's a fancy big fat IDE platform or an editor as abstemious as VI.

Apparently, the business we work for doesn't really care for the tools we use most of the time - unless of course they are suppose to pay for it. But as simple as it is, If you know how the IDE you use works and how you can make it work the way you want, can make all the difference in your productivity. Here are some unimpressive hows:

1. Many mundane and stupefyingly boring tasks can be automated if your IDE supports macros or extensions. Learn it, it will relieve your back/wrist pain.

2. You may have some fun exploring a totally different software architecture than you may be writing day-to-day, this may turn out to be way too interesting than your day job. Learn it you will know how to design.

3. Sometimes your IDE is just used for a single purpose - coding, but If you know how to extend it you might be motivated to add some cool stuff to it like chat view next to problems view in Eclipse or may be rolling reddit RSS entries in status bar. Learn it, possibilities are never ending.

4. Integrate your IDE to reproduce bugs in your application. This is similar to 1 but different in the way it solves your problem, this may not always possible but worth a thought. I managed to write several plug ins for quickly reproducing bugs related to my application components. Learn it and you can quickly fix them.

5. You can devise custom metrics from your code like how many lines you delete everyday compared to how many you write. Learn it and you will get to know your weird programming habits.

6. You never know when you get in to alternative job market if you know your tools well. Tool developers are far less than application developers. So learn how your IDE works and you might end up earning more money than your own job pays.


All of these, among others I can't really word without beer, are from my own experience. Since Level 1 Human Programmer in 'M$-Land' I've been curious about the tools I use, right from Visual Studio Add-in to Eclipse plug-ins, it has been wonderfully interesting journey so far. Given a chance to speak and I would totally be interested in how Eclipse is architected and how Gel used to support extensions and how painful it used to be to debug Add-ins in Visual Studio 6.0. Boy, it is interesting!

Tuesday, February 12, 2008

State of Eclipse based Python Development Tools

For starters, Python Interpreter console isn't a bad place for trying hands on the language. But if you plan on building larger programs with it you will need a good development environment, There are plenty of integrated development environments for Python today. Most of them are freely available open source projects, some have specialized support for GUI building while some are general purpose.

Since I'm Eclipse platform zealot, the first one I used was PyDev, PyDev is excellent python tooling built on Eclipse platform. It builds on excellent Eclipse UI and non commercial version offers basic refactoring, integrated debugging, Jython integration, Unit testing and several regular features which most Eclipse based IDEs inherit.

When it comes to dynamic languages tooling, DLTK is one potential place to look for in Eclipse Ecosystem. From time to time I'm lurking around DLTK newsgroups as well as keeping an eye on things moving in codebase for Python IDE. As of today, Python support in DLTK isn't too bad but it's still not as good as other IDEs out there, although it's coming up slowly. Eclipse DLTK is one fantastic tooling idea, it is so cool to build IDE for dynamic languages that you might want to try your hands building one, have a look at the guide here.

Out of Eclipse ecosystem, Eric is an IDE which I got chance to try hands on so far. Although It doesn't look as refreshing as PyDev or Eclipse in general, but it's pretty decent IDE nevertheless. It has lot of features, so many of them that you will confuse yourself on where to start, Eric UI has to be the last thing to impress anyone. It primarily features syntax highlighting editor, refactoring tool, debugger, package diagram viewer, regex builder, integrated version control and Python shell among others. While I'm not sure if Netbeans has any rock solid plan to support Python as Eclipse DLTK but I came across JPyDbg which is cross-IDE (and one of it's kind) for Python development. JPyDbg is kind of inspiring idea and I'm wondering if cross-ide-platform plugins would be a good idea? but that deserves separate post.

I haven't mentioned JyDT etc. because they seem to be dying or not actively compatible with later Eclipse versions. I would keep posting on python tooling as I go along working more on it.

PS: DLTK, I'm watching you.

Monday, February 11, 2008

Climbing the Python hill

Finally I have started working on new year's resolution. As a part of it, I'm learning a dynamic language called Python.

There are several reasons why I'm a new Python convert. One of them is not because blogosphere keeps intimidating Java programmers that they are writing crap. And I've honest sympathy for those who hold the assumption that Java is dead and other strangling naive opinions like that.

I am rather going ahead with some mature opinion like programming in a language with different philosophy gives you another way of solving the same problem, And sometimes the other way is better or at least, it will bring out interesting techniques which can be useful in general. Since I have very limited experience writing sizable software using them, I am expecting a lot from dynamically typed languages, Python in particular.

Python has been particularly interesting to me because I am curious about it since academics. I've played with it in past but that was just a stab that didn't bleed my nerve. I have no hard feelings for Ruby (or "hoopla oriented" Rails), because I'm not a huge fan of Web applications - web applications are boring.

Python is actually fun to program, programming in Python is like kidding, no it's not as frightening as it's name implies. For example, Methods in Java you know? they can't be moved around with out puzzlers and pain. But in Python, methods are also treated as types, called function types, which you move around like variables - no wait believe me it's as simple as passing around variables. Look at the closure example below and tell me how difficult it is to understand?
x = lambda: a,b: a**b

x(10,12)

Like you do in a regular mathematical equation, assign an expression to a variable or symbol and bind that symbol with value as shorthand like this.
a = x(10,2) * x(6, 8) / x(10, 2)
Also, any method which is not using 'lambda; keyword is also treated in same way, no partiality. Other obvious thing to like about Python is - it forces correct indentation in source code, that makes Python program much more readable in a consistent style. No need for one thousand formatters and coding standards.

I'm not comparing Java with Python, that's not my Job. I'm just trying to realize that how apt it is to solve the same problem aptly in a different way.

Python community, like Python itself, looks great. It isn't impossible to realize that - hangout on Python google group sometime and you will know. I am glad that they are very accepting and responsive to the n00bs like me.

I have started with SQLAlchemy ORM and db api in general to learn python because I'm curious on how they play with tuples and all. Later on I'll write a bit of parser as I've heard Python has some neat and funky stuff for parsing.

And oh, I'm having little trouble getting out of Java shoes lately - hmm, there is nothing like private fields or methods. But I guess that's better than writing getter and setter for that private variable which does nothing but to add overhead for dynamic compiler of hotspot VM to inline it after 3000 iterations, gosh!! But hey, Python is missing the package convention or let me say it prefers configuration over convention here, __init__.py is not as intuitive, is it?

Wednesday, February 06, 2008

How to Auto Increment version (build id) with Ant

For those who are still stuck with manual ant builds, this tiny little tip will tell you how to auto increment the version number with each Ant build. Follow the steps:

1. Create a file say build.number (or change the name to one which gives you pride)
2. Add the following target to your build.xml file, change entry key to whatever you want the property name to be:
<target name="increment_teh_build_number">
<propertyfile file="build.number"
comment="Build Number for ANT. Edit not!">

<entry key="build.number" type="int"
operation=
"+"
default=
"1" />

</propertyfile>
</target>
3. Make sure this target is executed on successful build for example after 'test' target, Once this target is executed you can use property build.number where-ever you like in build.xml, For example,

<property name="app.version" value="${major.minor}.${build.number}" />

Hope that saves you some.