Introducing PyConsole!

I have been pretty inactive this year, unlike what I planned (yawn, haven't heard from DLTK since June either) . I started learning Python earlier this year and felt the need of an embedded python shell inside Eclipse. This was something PyDev is missing [Edit: Simone pointed out that PyDev already has this neat feature, I was late to discover it] as compared to other python tools in Linux such as Eric. So I decided to write one.

PyConsole is direct result of my irregular and intermittent Eclipse development (lately), it's very simple plug-in with a basic exploit of Eclipse Console View and manages two way synchronized I/O from the python interpreter process controlled by a background Eclipse job. It provides few basic features anyone might expect from a shell such as syntax highlighting, command history etc.

Here's the code repository if anyone would like to check it out. If you have suggestions or feedback please write to me or create a bug.

What I want to do coming year?

I have been bored programming dear old enterprise applications lately, so I am seriously thinking of doing something new and more intellectual than writing data-driven applications.

I'm thinking of doing something significant coming year; while it would be pretty tough to stick to it all year along, I would at least want to build a strong tooling around, barely known, Felix language. Felix, compiler is written in OCaml, seems to be an interesting procedural/functional scripting language. In coming months I want to build an IDE for Felix using Eclipse and DLTK sub-project. There are many strong motivations for me to start this off apart from working on something intellectually stimulating and looking DLTK from inside.

I have just downloaded sources for OCaml, Python and Felix and compiling them to get started. As of now there's no progress and I'm yet to learn python, Objective Caml, Felix and DLTK itself, but I'm sure it would be a lot of fun.

While compiling OCaml on Ubuntu 7.10 I got this error

/usr/bin/ld: crt1.o: No such file: No such file or directory

and thanks to this guy for reminding that I'm not just Ubuntu user but a developer as well and would need libc-dev for just about everything :).

I'm looking forward to creating something interesting.

How would you compare XMLs?

I am supposed to deal with something similar to comparing two gigantic XML documents in wild ways.

I can think of several upfront techniques to achieve it which might involve performance and maintainability trade offs. As you might know, writing code for parsing XML by hand was the the activity of ancient times (hey there are you still writing code for parsing XML?), today we've plethora of tools to parse, bind and persist XML with very less pain. I came across several XML binding libraries like JAXB2.0, XMLBeans, JiBX etc (and given a change why not EMF?). JiBX seams to be interesting but since I'm bounded by not using open source at will, I tried JAXB2.0. The XML schema provided to me was a huge XSD document, the JAXB binding compiler spitted 550 Java classes out of that.

A test driven simple recursive depth-first reflective (opps, too many adjectives) traversal algorithm on the generated object tree sufficed the requirements to identify XML delta information. This was very obvious and pretty fast solution (fast to develop), the downside is, it would require maintenance of 550 generated classes, though they can be regenerated and synchronized with the help of XJC ant task but still the memory foot print and object creation time can be circumscribing for production code.

The other approach I tried was calculating XML diffs using XML processing. I found a nice little utility library XMLUnit among others, which does almost the same what I want. XMLUnit is a tool primarily for unit testing XML-intensive applications, It is very small with clean API and well documented (if you want to read i.e.). There are several utility classes which shields you from looking/writing ugly XML processing code which I used to get the XML diffs. Although I need to poke around on XMLwith XPath still because of the complex requirements.

I would have tried my favourite XStream as well but FAQs suggests me not to, anyway, What would be your strategy to deal with something like this ?

Pain of Sourceless Development for an Open source developer

I'm proudly an open source developer, working on Java, And I just realized I can't live without source code of every framework or library I work with. I know many of you are like me, we know we speak code, we write it, we hear it and we communicate in it. Not having source means we are impaired.

Before it happened to me, Neither could I imagine nor perceive myself working with a piece of software without source code, It was so common to have code before, and hey shouldn't that be included in programmer's bill of rights? But, First time in last few years, luck had it and I am working with a closed propriety framework, its painful for all. If you're in the great world of open-source software, you might have started thinking how painful it can be to program without code.

Not having source code means lot of things, debugging becomes nightmare which not only makes you feel stupid to debug but also it frustrates as you feel back in blind environments like the one in M$'. If there's a bug you have no way to look what's going on in the library or framework, you would wait till their (software vendor's) "support team" identifies or guesses what's the problem and gives a patch, which would take like weeks.

In Java, You can try decompiling the binaries, it's easy with Java unless binaries are obfuscated, this can be helpful to an extent. But most decompilers are not very debug-friendly, it would generate source in arbitrary assumptions, so unless you recompile the source back, you're stuck. But decompiling, recompiling and then looking around the weird identifier and method names and reordered statements will drive you crazy, but for Java developers there is a cool eclipse plug-in which would feel like a medic, it's name is JadeClipse. Though, it's too much for just knowing what's going on inside the third party's code.

While wider acceptance of Open-source in the industry is a definitely good thing, but beware it makes you feel less comfortable working with proprietary closed source softwares once you are used to it. I'm still wondering, how long would closed source software exist in platforms like Java or languages like Ruby and likes?

But please, for the hack's sake, please let us, the software developers on the mother Earth, see the SOURCE CODE.

Layman's review on Modeling etc. features in Netbeans 6.0

I know, I'm not going to sound very good for this post on the planet and expecting very less dialog in form of comments on it, but anyway...

The Java development blogosphere is experiencing a new wave of marketing campaign, yes everyone is talking about Netbeans, but hold on, it's not just viral-marketing, I found reviews where some says it's ruby support is the bomb, some are finding profiler better, someone is just finding it coming-up with better attitude:). Well, I've to tell it, I'd tried Netbeans in various forms in last few weeks, I can tell you it can do pretty much what all IDEs do these days. Netbeans might be looking less refreshing with all that swing (elegant inside!) rendering, but giving Eclipse Key binding, for example, really shows it's eager to compete your favorite IDE. In this post I'm not trying to say Netbeans is better or something like that, my love for Eclipse is unparalleled for various reasons, but yes I am, sure, eager to tell you my modeling experience with Netbeans.

Before I do that I'd just write down my opinions about Netbeans 6. I would say, profiling is really pretty good, I don't need to understand nefariously complex architecture to use it, I don't need to go to help to use it. See you would say, TPTP is complex only for lame users. Are you talking about UI? how many perspectives you have as part of TPTP? which one to use for profiling? and what you've to do to profile remote Java application? can you answer it without looking in documentation? Don't get me wrong, I'm trying to be as unbiased as I can.
As for Ruby support, It is just about as good as DLTK Ruby IDE, I'm no Ruby big shot as yet, but things works pretty OK with DLTK as well, cheers! Netbeans has fantastic UI editor though, VE plain sucks, oh and is it around now? last time I heard it's dead.

Now on modeling, To My fellow Eclipse committer guys, let's face it, Eclipse doesn't give anything in modeling front which an average user can use out of the box. It might have innovative and wonderful framework / UML infrastructure for building tools around it though, which is very well appreciated by 2% eclipse plug-in developers (including me), fine. So for our hacky interests EMF might replace MOF but Eclipse users are not getting any advantage of it. I know, some might say it's no big deal to not have modeling as a part of IDE, but hey that's the whole point, everything from single place is the point. Few would like to use Visio to model class diagram and then write code for it. We've quite a few other reluctantly-opensource projects, in the community, controlled by arbitrary licenses which are not very useful.

For those who don't agree, download and try Netbeans for modeling, You will, for sure, appreciate the efforts put into it, it can do hell lot of stuff (It can't copy and paste model elements in single diagram though :) ), Java reverse engineering, simple modeling, in context design pattern appliance, it's just there and works! you don't need to find something on Internet and read it's license thrice before using it. And the best part of it, It doesn't cost an arm and leg to get these features. OK, it might behave little in-development but it's just there for you, try it!

Eclipse is the great Plaform and I can't even start to compare it or any of it's innovative aspects with something as trivial as Netbeans in that context, but guys, shouldn't Eclipse consider it's "end-users" who expect nice features? Advocacy on plain words don't help, it needs strong backing of solid product which offers something more. Learn Learn..

Fat Jar Plugin for Europa release

There are quite a few bugs on Fat Jar Tracker lately, some of them related to incompatibility with Eclipse 3.3 release (aka. Europa). It's been a long time since I touched the code-base of Fat Jar plug-in, There were several Eclipse 3.3 API incompatibilities including setSorter(ResourceSorter rs) method, now deprecated, and replaced with setSorter(ResourceComparator rc) in org.eclipse.ui.dialogs.ElementTreeSelectionDialog class etc. I'm glad to have no other major, and painful, changes :) (at least, I haven't encounter one so far).

If you are code freak, changes are committed to cvs repository at fjep.cvs.sf.net. For the users, Since I'm just a developer member of the project, I don't have enough rights to upload the build and release the plug in officially, all I can do is to provide you a link for the latest build which works with Europa release. As of this time, I'm not sure when the latest plug in will be available from sf.net, in the mean time you can download it from here.

Enjoy Deployment with Europa :)!

Closed development instance in Eclipse ecosystem

I've been reading Bjorn's posts on open source and open development. I can't disagree with his view on open source being more than just making code available. Getting project participation, in greater capacities than just mailing lists, from community is a good thing, and that should definitely be taken into consideration with proper feedback. The very purpose of being "open" is defeated when a project starts ignoring community contribution without sensible reasons.

I've reason to write this, although I'm not strictly against closely developed open source projects.

I've been working with a modeling toolkit for quite sometime and had customized it to suit my requirements. The contribution was in the core part of the tool (GEF based clipboard, motility and other ergonomic support for better modeling experience). I sent a mail to project admin asking if they would be interested in taking this contribution, after unknown number of days I got the reply asking for contribution review. I sent it as plug-in source code, not as patch, listing changes in the packages and configs. Most of the features offered in the contribution are actually in the project road-map as enhancement list.

But, after more than two months now, I'm still without reply(positive or negative), without comments, without acknowledgment. Now, I've almost lost interest in contributing it.

I hope those required features will be implemented (in anyway project team wishes) in next major release of the project, but I can only hope because the project is not really being developed openly. How sad...

Don't get me wrong. I've nothing against the tool, the tool is excellent and remarkably well designed and implemented; Nor with the team developing it, it's their job. But they should definitely mention somewhere that they are not open to external contributions in development.

HOW TO: Setting up GForge in minutes

GForge is a collaborative development environment, which was forked from SourceForge, read more about it here. GForge is being used heavily world wide and is much better option for project management, versioning and collaboration. The best thing is, it's open source and freely available and is documented and improved regularly.

I spent past week configuring and hacking GForge. Apparently, It requires a good understanding of many Linux related stuff. Being sufficiently complex, it takes good system administration skills to get it running perfectly. I'm not proclaimed Linux expert, neither plan to be one; but I agree it was fun playing with this cool stuff in Linux among others.

Well If you are not curious on details, you may skip rest of the post.

There are multiple ways of setting it up, in increasing order of difficulty:

1. You can download Pre-configured GForge VMWare Image on Fedora core.
2. If you use debian based linux distributions, auto install all the dependencies and gforge using one simple command.
3. Download official tar-ball and configure it by hands on any Linux.

I tried all of them on Ubuntu and Fedora, and It worked well in most cases. Doing everything from scratch, i.e. installing it from "official" tarball, is bit difficult but not amazingly hard (more than linux, you need to understand how various things like DNS, Postgres, MailMan, WikiWikiWeb, CVS, SVN, LDAP etc. work).

If you want to deploy it on Debian or debian based distro, you've to update your sources.list to lookup/install and upgrade Gforge for debian, read more here. Be careful in doing 'apt-get install', it may render your OS useless, if GForge and dependencies are not configured smartly. Especially you'll face problems related to OpenLDAP and Postgres SQL, also these sources has older versions of Postgres which are bit difficult to upgrade without dependency issues. For example, if you fail to install postgres properly, you'll not be able to login which expects LDAP authentication. LDAP, in turn, depends on DB which is not well-configured, what will happen is everytime you try to login at tty it will give you "Can't connect to DB". If you run in such problems, You can login as single user from GRUB loader and delete ldapnss and other ldap related files from /etc to get it running normally, but this can be really dirty exercise.

The easiest most is to play with VMWare image. Recently, (ok, not so recently) some contributors at gforge.org made virtual appliance of GForge on Fedora Core 3, you may download it from here. It's pretty easy to setup and get going with it.

However, There are several problems when you run VMWare image in VMWare Player:
1. At first you've to setup the virtual host for 80 and 443 (HTTP, HTTPS) in apache configuration.

2. If you run this image on other network, you'll have problems of failed login with out any warning or error message you'll see glorious "Cookies must be enabled past this point" and nothing will happen. This is because your VMWare Image IP is different than what it is configured for apache, and browser will stop accepting cookie from the server. This whole IP thing is problematic with VMWare because VMware host does not automatically provide internal DNS service for the virtual machines, and IP allocation is based on the host network adapters bridged with virtual VMWare adapters, which may have different addressing schemes. In short, IP of gforge image keeps changing based on the computer you run it on.

In this case you can either reconfigure Virtual host settings in httpd.conf to match current IP or you can force static ip by adding following lines in your /etc/rc.local file.

/sbin/ifconfig eth0 [staticip] netmask [subnetmask]

This will ensure that you've static IP everytime you run image. Once run, you can access the gforge portal from host OS browser. That's it, there you go.

I'll write more about GForge interesting internal stuff sometime later.