PDF Manipulation Tools

I wanted to manipulate a few PDF files recently and was on the lookout for suitable tools. More specifically, I wanted to convert a few double-page PDF files (containing two pages of text on a single page) into single-page PDF files. I also wanted to drop some of the pages in order to have the files contain just the text that I was interested in. Fortunately for me, there are several freely-available tools that do the job well.

Read More...

Mohan Embar

I met Mohan Embar this weekend. He used to maintain the Windows port of GCJ. It was nice to finally be able to associate a face and a voice with the name, since our interaction so far had only been over email. He turned out to be much thinner, more soft-spoken and more boyish than I had imagined.

I think I ended up asking him a bit too much about how he managed to remain a freelance programmer for so long, that too in Milwaukee, since it is something that interests (as well as scares) me.

GCJ for MinGW

I have updated my article "Building GCJ for Windows" and the associated scripts to work with the current SVN mainline sources of GCC (to be released as GCC 4.2). They might also work with GCC 4.1 sources, but I have not checked it myself. The article also has some tips for building GCC natively on Windows using the MSYS toolkit, especially to make the built and installed GCC relocatable (see below).

A major portion of the effort went in to ensure that the resultant toolchain was actually relocatable (that is, the installation can be archived and then extracted elsewhere, possibly on a different machine, and everything can still be expected to be working). The proper locations of the Windows headers and runtime libraries and the flags to pass to the GCC configuration scripts were something that took a lot of trial and error (and a lot of help from Mark Mitchell and Danny Smith) to get right, since I was trying to do something less common (building cross and crossed-native compilers) for a platform that gets the attention of very few GCC hackers as such, if at all.

I had stopped working on GCJ for Windows quite a while back and the reason I had to update my article and scripts was that there seemed to be a lot of people trying to build GCJ for Windows themselves using the latest released or in-trunk sources (and my instructions and scripts) and they were running into all sorts of issues. Unfortunately, GCJ on Windows has become worse than it used to be which is understandable since there is no one who is actively working on it to improve it. It is also a shame since even though it is a closed platform with an ugly design it appears to have the most number of users enthusiastically willing to try out GCJ.

We must do something about this situation.

QEMU
For a fan of Linux trying to make GCJ for Windows work, a very useful property of GCC is that it can be built on Linux as a cross compiler or as a crossed-native compiler targetting Windows. For a person with a relatively old machine and limited free time to hack on GCJ, this is also important since the build on Linux is way faster and far more reliable than that on Windows itself using MSYS. Equally important is the ability to test out the binaries created in this process without having to reboot the machine into Windows or having access over the network to another machine running Windows. Wine doesn't quite help since I need an environment that is as faithful to the real thing as possible.

QEMU running Windows on Linux comes to my rescue here. When run with the -kernel-kqemu option using the QEMU Accelerator ("kqemu"), the guest OS runs at very close to native speeds without adversely affecting the performance of the host OS. It has a built-in TFTP server that allows you to easily transfer files from the host machine into the guest system (there are also other ways of achieving this using QEMU, but this is the simplest). It's almost magical and is immensely useful. It's no wonder that virtualisation is becoming so popular these days and every developer who has tried it out sings its praises. If you are an "enterprise software" developer, you should already know what I am talking about. If you haven't tried it out yet, you really should. Virtualisation offers you the freedom and the flexibility to play around that is very useful and quite addictive.

GCJ and ECJ

RMS has finallly agreed to using ECJ to generate bytecode for GCJ!

That sound you hear is the huge collective sigh of relief heaved by Free Java hackers everywhere.

GCJ: Quo Vadis?

Andi Vajda asked whether GCJ would cease to exist if Sun were to release the source code for Java and its tools under a really Free licence. I have also seen such questions asked on Slashdot, OSNews and other fora.

The response from Andrew Haley mirrors what I personally think of the situation - as long as there are hackers willing to maintain it, GCJ would continue to exist. Miguel de Icaza says that among the two types of hackers who usually work on Free Software in their spare time, GCJ and GNU Classpath only attract the Free Software idealogues and not those who want to get something free (gratis) working for them since Sun's JDK already works well for them and is free. Tom Tromey has a more philosophic take on the current situation - almost ascetic in fact. All this is enough to make a GCJ or GNU Classpath hacker reflect on the current state of Free Java, the utility of his contribution to it and the impact of a fairly Free release of Java source code from Sun. Despite being an extremely erratic contributor working on the fringes of Free Java, I cannot help doing the same.

It was about four years ago that I first flirted with GCJ. I wondered, for no particular reason as is usually the case with me, if it was possible to use GCJ to create native GUI applications with SWT in Java on Windows. I found out that the support was almost there and with a little effort and a lot of support from the GCJ hackers (especially Tom), I was able to add in that support and contribute it back to GCJ where it was accepted with minor modifications. That was my epiphany with Free Software. Until then I was more impressed by the fact that I could get so much decent-quality software for free than the liberty to make modifications to such software. But now I began to realise that having the source code available to you meant that you could change it yourself to fix its shortcomings and share your improvements with the other users. It also meant that the availability of the software did not depend on the solvency of the vendor or its willingness to maintain it. There were many other factors in favour of Free Software that became apparent to me over time. Suffice it to say that I finally understood what that twitchy, smelly and passionate preacher of Free Software was talking about all the time.

In time, my original itch died out (like so many of my digressions in life) but I continued to work on GCJ. I quickly moved from Windows to Linux (since that was the only platform that I enjoyed working on) and began fixing front-end bugs more out of a desire to help folks than to fix anything that was affecting any of my personal or professional work. This played a part in the fact that my track record with GCJ has been absolutely abyssmal (except perhaps in the area of contributing the most noise to the GCJ mailing lists). My pathetic time-management skills and a propensity to be carried away by even the slightest distraction have also played a big role. Since I worked on GCJ in my free time at home and I did not want to compromise too much on my personal life (spending time with my family, watching movies, reading books, meeting friends, getting enough sleep, etc.), I rarely found the time to debug and test anything except the most trivial of bugs. Finally, my tendency to "wait and watch" (first for GCJX and now for ECJ) has not helped matters much either.

There are some inherent problems with GCC and GCJ too. The GCJ front-end seems to have been written in a hurry in order to get as many things working as fast as possible with not much thought given to overall maintainability. The people who originally wrote it have moved on to other things in life leaving others with little idea of how it all works. Subsequent hackers (including me) have always made incremental changes to fix immediate issues rather than perform any big refactoring of the code, with the natural result that the code has become even more unwieldy now than before. To fully bootstrap GCC, especially with checking enabled, and then to run its testsuite takes an awful amount of time, especially on slightly older hardware (like my otherwise perfectly capable P3-based system). This is a huge barrier for most prospective hackers. (I reserve my rant about the disastrous effects of bundling several language front-ends and their ever-bloating runtime libraries into a single compiler system for another day.) Until recently, the ubiquitous tree data structure was used in funky ways for almost everything in GCC. There is precious little documentation for a programme of this complexity and some parts of this documentation is out-of-date. The best way to understand stuff in GCC is to read through the source code, to watch the operation of the relevant parts in a debugger and to ask questions on the mailing lists when you do not understand something even after doing all this.

All these are problems that can be overcome one way or the other. The biggest problem with GCJ however is the sheer paucity of hackers willing to work on it to improve it compared to the number of people willing to use it and reporting problems with it. This situation is particularly severe for Windows. Were it not for Red Hat's sponsorship of some critical GCJ hackers (and the heroic efforts of Tom in particular), GCJ would have been in a very bad shape by now. This situation really makes me realise how true Miguel's observations are with respect to hackers of Free Software and Free Java.

A Free Java from Sun would not obviate the need for GCJ though. I personally feel that ahead-of-time compilation to native code providing more opportunities for aggressive optimisations (platform-agnostic as well as platform-specific) and a more straightforward integration with C/C++ via CNI are enough to show the utility of GCJ orthogonal to the status of the freedom provided by Sun's JDK.

This post has already become the longest I have ever posted, so I will reserve my rant about how Java the language and its bloated "standard" runtime is not even worth spending so much time and effort on in the first place, for another day.

GCJ

I seemed to be in my elements on the GCJ list this week, provoking a thread on the lack of good support in GCJ for Windows and eliciting a reply from the GCC Steering Committee on the status of the proposal to integrate ECJ into GCJ.


(Originally posted on Advogato.)

ECJ for GCJ: Still in limbo

It has been almost a month since Tom formally proposed integrating ECJ in GCJ to the GCC Steering Committee (SC). There has been no word from the SC yet on this request. However, the SC did ask the GCC developers to avoid gratuitously including source code from external projects in GCC. One consequence of this for GCJ was the removal of fastjar from the GCC source tree. I'm not sure if the SC's decision was coincidental or in fact a result of deliberations triggered by Tom's request.


(Originally posted on Advogato.)

GCJ and ECJ

Tom has asked the GCC Steering Committee to provide their verdict on the proposed use of the Eclipse compiler for Java in GCJ. This follows his earlier proposal to abandon GCJX for GCJ and adopt ECJ instead. As of this writing, there has been no response from the GCC SC yet.


(Originally posted on Advogato.)

"A look at GCJ 4.1"

Mark Wielaard has written another article for LWN.net titled "A look at GCJ 4.1" (where he also looks at GCJ 4.2 and beyond). It is subscribers-only for the moment (for a week), but if you are interested in Linux in any way, you should seriously consider subscribing to LWN. It's quite good.

(Originally posted on Advogato.)

ECJ for GCJ?

Tom proposed killing GCJX and replacing it with the Eclipse compiler for Java (Eclipse JDT Core plug-in, known informally as ECJ). He has been almost single-handedly working on GCJX for more than a year and it looks pretty good already, so it is pretty courageous of him to be the one to propose using something else instead of GCJX in the overall interests of GCJ.

ECJ seems pretty good and very actively maintained. It must be one of the fastest Java compilers around and fully supports the new language features introduced in JDK 1.5. So it is a very good move for GCJ.


Using ECJ does introduce GCC bootstrapping issues though. However, it should be possible to easily overcome these issues. The bigger issues are political and legal in nature. Let us hope these are resolved favourably.


I personally feel a little sad though. This removes another "fun" part of GCJ even though it is pragmatically a better thing to do, especially considering the precious little resources that the GCJ project has. I feel that GCJ is becoming more and more an "integration" project combining the best-of-breed in Free software for a given task - the Java language compiler would be ECJ, the garbage collector is Boehm-GC, the runtime library is GNU Classpath and the optmisation and code-generation is done by GCC. Of course, this can hardly be characterised as bad and is in fact quite a sensible thing to do given the limited amount of resources that the Free software world has at its disposal, but...


(Originally posted on Advogato.)

Subversion, Old Dog, New Tricks

It turns out that I do not need too much of extra disc space for working on trunk and gcjx-branch using SVN compared to CVS after all. This is because I used to always create a snapshot of GCC sources and use it as a working copy for fear of messing up my checked-out sources. Since SVN always keeps a copy of the pristine sources around (which is the major cause of the increased disc space usage) and it is easy and fast to use svn diff to figure out the damage and to use svn revert to restore sanity, I no longer need to continue with my weird model of development. It is also quite simple to just ignore everything from the GCC SVN repository except for the interesting stuff - for the gcjx-branch, my checkout only has the bare minimum stuff needed to bootstrap C, C++ and Java and run the libjava testsuite, while for trunk I have removed all the Ada stuff since I can't build Ada anyways. Of course, all this would probably have been possible with CVS as well, but there weren't nice instructions in the GCC Wiki for lazy souls like me for doing this with CVS.


(Originally posted on Advogato.)

Dumping Parse Trees

GCJX now accepts an "-fdump-tree" option that prints out the abstract syntax tree of a Java source file to stdout.

(Originally posted on Advogato.)

A Walk Among The Trees

I implemented a simple pretty-printer for the AST used in GCJX over this weekend. It works just like the debug_tree() function in GCC and is accessed by using the dump_tree() function in GCJX. For convenience, I just change "-fdump-methods" to call dump_tree() instead of dump_method(). That way I can easily see the ASTs created for various Java source files. For someone like me who is new to GCJX in particular and compiler construction in general, this can be quite enlightening.

While debugging the GCJ front-end, I have found debug_tree() to be an immensely useful tool. I hope dump_tree() proves similarly useful for GCJX.


By the way, I added a small page to the GCC Wiki describing how to go about debugging GCJX.


(Originally posted on Advogato.)

Miscellany

I have been granted write access to the CVS repositories of GNU Classpath and Mauve. I have also started messing with GCJX (again) that was started by Tom Tromey. It has come a long way since the last time I had taken a look, but it sure can use some help from willing hackers. Would you like to help develop a first-class Java compiler?

By the way, Tom has become one of the Java front-end maintainers for GCC.

(Originally posted on Advogato.)

More on WFL Nodes

I found the original patch from Jeff Law that removed WFL nodes from the rest of GCC and includes an analysis of why WFL nodes are bad. Note in particular his analysis of the Java front-end, where 30% of all generated tree nodes are EWFL nodes! WFL nodes are thus a waste of space and time, not to mention a source of unnecessarily-added complexity in the front-end that ultimately only frustrates hackers. So I repeat my war cry, "Death to WFL nodes"!

By the way, GCJ already gets the column numbers pretty wrong on almost all diagnostics, so it's not as if we lose much by losing EWFL nodes.

(Originally posted on Advogato.)

EXPR_WITH_FILE_LOCATION

I hate EWFL tree nodes in GCJ. So many of the ICEs (internal compiler errors) I have seen in GCJ are because some piece of code expects or doesn't expect an EWFL node. To put it simply, the current front-end wants a WFL-wrapped expression node whenever there is a need to emit a warning or an error about that expression, but not otherwise.

This can easily frustrate anyone wishing to fix some of these ICEs in the hopes of making GCJ better. For example, here I am discovering that many ICEs in the Jacks testsuite are because the body of an empty block ({}) or statement is not being wrapped in an EWFL for diagnostics about unreachable statements, finding that it is trivially fixed and then discovering that doing this creates a whole mess of new ICEs on other tests, which have to be individually addressed in this manner potentially creating yet other ICEs in other places, ad nauseum.


To quote Jeff Law (gcc/ChangeLog.tree-ssa), "Death to WFL nodes"!


(Originally posted on Advogato.)

GNU Classpath Copyright Assignment

My copyright assignment papers for GNU Classpath were cleared by the FSF today.

(Originally posted on Advogato.)

R.I.P.: Old Bytecode Verifier

The old bytecode verifier used by the GCJ compiler has now finally been removed. It was a small and straightforward verifier but had a few bugs that made it difficult to use GCJ with random JARs. At the same time, the newer bytecode verifier (written in C++) used by the GCJ interpreter gij had far fewer bugs and could handle almost all JARs found in the wild. Since no one was fixing the bugs in the old verifier, GCJ could not work with many JARs for a long time and thus was unusable for a large number of potential users. Bryce made the new bytecode verifier work with the GCJ compiler to support the work on the new Binary Compatibility ABI. After some time I made the new verifier the default for even the old C++ ABI.

I feel guilty and sad now because once upon a time I had resolved to fix some of these bugs but never actually got around to fixing them. I had studied the source code and had read several papers on bytecode verification, especially some of the excellent ones by Alessandro Coglio, but never implemented any of the techniques. Not good.


(Originally posted on Advogato.)

Scratching An Itch: Terry Laurenzo, GCJ and Generics

One of the stumbling blocks in supporting generics in native programs created by GCJ is the fact that the C++ method-name-mangling used by GCJ does not encode the return type of the method and thus cannot support the Java 1.5 kludge for implementing generics (PR9861).

Terry was just another bloke who was trying to make his program work with GCJ when he hit this issue. Unlike most other blokes however, he has decided to do something about it. Cool!


(Originally posted on Advogato.)

Miscellany

I come back from a two-week vacation in Bhopal and find that GNU Classpath crosses a million lines of source code and that Tom has done the big GNU Classpath merge into libgcj that should make importing changes from GNU Classpath much easier than before. I also found out that people were still discussing the semantics of overflow of signed integers in C on the GCC list and that Zack Weinberg has left GCC development.

Daniel Berlin has set up an automated patch queue for GCC patches. All you have to do now is to include a line like:

:ADDPATCH java:

in your patch message and it would be added to the patch queue for the area "java". Cool!

Nvu, the new incarnation of Mozilla Composer and a standalone program like Firefox and Thunderbird, has reached version 1.0. I took it for a spin and was able to easily create a few nice-looking HTML documents. Apparently it also supports CSS, XHTML, etc. but I don't know much about them yet to find out how good it is at supporting them.


After having ignored all this while the RSS and other feed aggregation capabilities of Mozilla Thunderbird, I now find this feature absolutely indispensible and liberating. Now I can waste even more of my time reading weblogs and sites that I otherwise would not have bothered to visit with such regularity.


Barry Andrews became yet another person to let out a primal cry of joy and pride after he successfully built GCJ 4.0.1 for Win32 by following my document. Based on his suggestions, I have refactored the document a bit and have highlighted some of the more important points.


(Originally posted on Advogato.)