Calculating Interest Rates

You want to buy that fancy LCD TV that costs Rs 60,000 but you do not have that much money with you. You see an advertisement in a newspaper for the TV from a dealer who offers to sell it to you if you make a down-payment of Rs 10,000 and pay Rs 4,380 every month for one year. You see another advertisement in the newspaper for the same TV from another dealer who offers to sell it to you if you make a down-payment of Rs 20,000 and pay Rs 1,330 every month for three years. How do you calculate the rate of interest each dealer is charging you for what is, in effect, a loan?

In "Calculating EMIs", we derived the formula for calculating the "Equated Monthly Installment" (EMI) on a loan. If "E" represents the EMI, "P" represents the principal amount in the loan, "r" represents the monthly rate of interest (one way of arriving at it is to divide the annual rate of interest, quoted as a percentage, by 1,200) and "n" represents the number of months in the tenure of the loan, then:

     E = P × r × (1 + r)ⁿ / ((1 + r)ⁿ - 1)

In the current example, we know the values for "E", "P" and "n" and wish to calculate "r". Unfortunately it is not that simple to calculate "r" using just the high-school algebra that most of us manage to remember. Fortunately there is a simple algorithm that can help us in this situation.

Let us first rewrite the formula above as an equation:

     P × r × (1 + r)ⁿ / ((1 + r)ⁿ - 1) - E = 0

Our task now is to find the roots of this equation - that is, the values of "r" that will make the left-hand-side (LHS) of this equation evaluate to zero.

To find the roots of a given equation "f(x) = 0", the algorithm in question can be described as follows:

1. Find a value "a" for which "f(a)" evaluates to a negative value.


2. Find a value "b" for which "f(b)" evaluates to a positive value.


3. Let "c" be the average of "a" and "b".


4. If "f(c)" is close enough to zero, "c" is the desired root.


5. Otherwise, if "f(c)" is a negative value, substitute "c" for "a" and repeat the procedure from step #3 and if "f(c)" is a positive value, substitute "c" for "b" and repeat the procedure from step #3.

Note that this is just a binary search algorithm. By "close enough to zero", we mean that the absolute value of "f(c)" is less than some value, usually called "epsilon", that can be as small as we please. The algorithm given above can be rewritten as a function in a pseudo-language as follows:

  guessRoot( f, a, b)
  {
    c := (a + b) / 2;

    if( absoluteValue( f( c)) < EPSILON)
      return c;
    else if( f(c) < 0)
      return guessRoot( f, c, b);
    else
      return guessRoot( f, a, c);
  }

You can implement this in your favourite programming language along with a function that calculates the LHS of the equation given earlier. You can choose a value of "epsilon" according to your preference - the smaller the value of "epsilon", the more accurate is the result and the longer it takes to compute it. The time taken for the computation is also affected by how wide is the range between "a" and "b". Note that Newton's method is a much faster way of computing the roots of such equations, though it involves calculating derivatives.

How do you arrive at the values for "a" and "b"? This differs for each function. For our example, we can start with a low guess of "0.001%" ("0%" gives an undefined result) as the annual rate of interest and a high guess of "100%" and this gives us a negative and a positive value for the LHS respectively. With an "epsilon" of "0.00001", a C programme computes the answer in around 25 iterations.

In our example, the first dealer is offering us an effective loan of Rs 50,000 for 12 months with an EMI of Rs 4,380 and the effective annual rate of interest comes to about 9.32%. The second dealer is offering us an effective loan of Rs 40,000 for 36 months with an EMI of Rs 1,330 and the effective annual rate of interest comes to about 12.08%. In terms of the interest rates being charged by the dealers, you should now be able to tell that the first dealer has a better proposition for you when compared to the second dealer.

Advogato Diary Imported

I have imported all the old entries from my Advogato diary into this blog. These entries are labelled "advogato diary".

It was very simple to do since Advogato used to provide a simple way to import all my entries as an XML file (it does not work any more) and Blogger has a nice and simple API that allows me to post entries automatically, even allowing me to back-date and label them appropriately.

However, I did encounter an unexpected problem during this import. It turns out that if you post too many entries in a single day (in my case the limit seems to be 50), Blogger thinks that you are creating a SPAM blog and turns on "word verification" for posts (a CAPTCHA). While you can still post entries manually, the API provides you no way of retrieving and resolving the CAPTCHA using human input. After 24 hours the word verification is automatically switched off and you can again post using a programme. I had to therefore spread out the import over 5 days since I had 231 entries to import.

By the way, if you worry about search engine rankings for your pages, you might want to note that such mass imports cause Google to downgrade your site for having duplicate content if the old blog is still accessible.

Finally, Advogato is still alive and being maintained.

Blog Tweaks

I tweaked this blog in the last couple of days in the hopes of making it a little better - a little better-looking and a little better-behaved. Read on for the details.

The tweaks include:

• Using a better-looking template. The old template was a bit boring and quite minimal. It made it hard for most people to read all the text I was spewing. The new template looks better (at least to me). It has a narrower column width to display the text, somewhat similar to those in newspapers and magazines, which makes it easier for most people to read the text.


• Showing only the initial paragraph from each post on the main page. You can read the full post using the "Read More..." link at the bottom of each such paragraph. This makes it easier to skip over posts that you are not interested in reading.


• Showing only a preview of the post in the feeds. I used to feel bad about my banal verbiage eating up lots of space on Planet Classpath and other such "planets". This change should let people easily skip over my posts if they don't care for what they see in the preview and navigate to the page containing the full post if they do. It should also benefit people who have subscribed to this blog using a blog aggregator. To do this a bit better than what was possible with Blogger's own feed mechanism (but still not entirely satisfactorily), I have had to redirect the Blogger feed for this blog to the FeedBurner feed for this blog.


• Giving at least something back to Google for providing this great service for free. I used to feel bad about being yet another leech on Google's resources. I signed up for Google AdSense via Blogger. Now each page on this blog shows textual advertisements relevant to the context of the page.

I do realise that there are negative aspects of each of these changes that some folks are not going to appreciate. However, I believe that each of these changes is for the better, all things considered.

Update (2007-08-10): Anusha did not like the fact that you navigate to a different page when you click on "Read More...". It is also not fair to the reader since the entire post is already there on the main page, but hidden from view. So now I have changed the blog template to expand and collapse the rest of the post in place using a combination of techniques shown here, here and here, with a few adjustments of my own.

Update (2007-08-29): The feeds now have the full contents of the posts once again.

Disabling atime Updates

A recent article on KernelTrap highlights the high cost of supporting atime ("last-accessed time") updates on Linux file-systems. It has been suggested that desktop machines should just mount their file-systems using the "noatime" option to avoid this overhead.

Each time you read a file, its atime has to be updated. This can quickly become costly if you have applications that access a lot of small files. Most modern desktop environments, office suites, compilers (think of C/C++ headers), browsers, etc. fall into this category, so Linux takes a lot of unnecessary performance hit for data that is of interest only to a very small set of applications like tmpwatch. (Apparently even Windows has the same issue with NTFS.)

I have now changed the "/etc/fstab" on my PC to mount its file-systems using the "noatime" option. It does seem to have slightly improved the responsiveness of the desktop, though this could just be a placebo effect. On the other hand, in the KernelTrap article people have presented measurements that demonstrate the actual performance improvements brought about by using this option.

Blog Comments

Joel Spolsky links to a post by Dave Winer on blog comments as well as providing his own views on the subject.

Dave opines:

If it was one voice, unedited, not determined by group-think -- then it was a blog, no matter what form it took. If it was the result of group-think, with lots of ass-covering and offense avoiding, then it's not.
[...]
Well actually, my opinion is different from many, but it still is my opinion that it does not follow that a blog must have comments, in fact, to the extent that comments interfere with the natural expression of the unedited voice of an individual, comments may act to make something not a blog.

while Joel adds:

When a blog allows comments right below the writer's post, what you get is a bunch of interesting ideas, carefully constructed, followed by a long spew of noise, filth, and anonymous rubbish that nobody ... nobody ... would say out loud if they had to take ownership of their words.
[...]
Dave is absolutely right. The way to give people freedom of expression is to give them a quiet place to post their ideas. If other people disagree, they're welcome to do so... on their own blogs, where they have to take ownership of their words.

The timing of Joel's post couldn't have been better as I was recently wondering about the same issues myself. I was seriously considering disallowing comments on my blog posts. Some of the main reasons were:

• Impurity - it no longer remains purely my own ramblings (Dave makes the same point). My utterly inane ramblings get combined with the inane ramblings of other, mostly anonymous, folks. It starts to look like a mailing list where I start a thread and others join in.


• Overhead - I have had to review and moderate every comment since the time spammers discovered this blog and started abusing the comments facility to post links to their sites in order to boost their ranks with search engines (CAPTCHAs don't seem to deter them). I would like to avoid this unnecessary overhead.


• Liability - I seem to unnecessarily become liable for the contents of the comments since they are available from my blog. I moderate comments simply to weed out spamming efforts, not to censor or alter them. Reasonable folks would agree that the respective posters of the comments should be liable for their content, but as we all know, reasonable folks are a sad minority in this world.


• Noise - while I try to put some thought and effort into the material posted here, it gets diluted by the utterly trite comments that sometimes follow it, especially when people post under the cover of anonymity (Joel makes the same point). Insightful or interesting comments are a sad rarity on my blog.


• Lock-in - the ability to collect and collate comments is one of the major reasons I am forced to be with Blogger or similar blogging platforms. I would ideally like to be able to merge this blog with my web-site and only upload static pages to my web-site. I would then not depend on anything other than the very basic hosting facilities and this would let me easily switch hosting providers.

However, comments are not all bad, of course. Some of the main reasons I continue to allow comments on this blog include:

• Feedback - at worst, it tells you that at least some people took the trouble of navigating to your blog and reading your blog post. At best, a "Thank you!" warms your day up and a "This sucked!" goads you into writing better. In any case, you get to know that your efforts have not entirely been wasteful.


• Ease - comments allow a reader of your blog to quickly and easily leave feedback for you. Emails are a little burdensome for this purpose, not to mention a bit formal. Making everyone respond to your blog post via their own blog posts (as Joel seems to suggest) looks too awkward to me - you would have a very hard time keeping up with the responses and most readers would just give up trying to leave feedback for you (perhaps that is indeed the effect Joel intends).


• Scale - as Clay notes, if you are a small-time blogger (like yours truly), the signal-to-noise ratio in your comments is very likely to be much better than that on more popular blogs and web-sites that allow comments. For the same reason, the volume of comments is also likely to remain manageable enough for you to be able to moderate them.


• Anonymity - some people are just not comfortable with revealing their identities to you, but would still like to leave a comment for you - perhaps anonymity provides them the security needed to provide frank opinions, perhaps they are shy, perhaps they don't want to sign up with Google just to be able to leave a comment for you, perhaps they just don't want to be seen as a person caught reading blogs in general or your blog in particular, etc.


• Enhancement - some of the best comments are those that expand on the blog post by providing further information, clarifications, alternative ideas, etc. This enhances the value of your blog and makes it more appealing to your readers.

On the whole, blog comments appear quite useful to me despite their obvious warts. I will continue to allow comments on this blog, even those posted anonymously, as long as it remains manageable. I hope I continue to remain small enough to escape the attention of the trolls.

Comments?

The Amiga

Ars Technica has just published Part 1 of what looks like a very interesting series of articles on the history of the Amiga series of personal computers.

The Amiga was quite unlike the other PCs of its time and could supposedly handle multimedia with an ease that put the IBM PCs of that time to shame. Sadly, I never had the chance to work with an Amiga myself. As is usual in the computer industry however, mere technical brilliance does not guarantee survival and popularity and in the end the IBM PC prevailed, while Commodore, makers of the Amiga, went bankrupt. Being an early user and fan of the BBC Micro, I can also bitterly attest to this sad turn of events that made the IBM PC the overwhelmingly dominant PC. Even though the Intel 8086 CPU was awkward to work with, DOS was an abomination for an operating system and the IBM PC was quite limited in its capabilities, none of this could hold the IBM PC back from reigning supreme and from killing off other personal computers (the Apple Macintosh being a notable exception).

Some time back, I saw the second volume of MindCandy. This volume was about the Amiga demo-scene while the first volume was about the IBM PC demo-scene. I had been following the IBM PC demo-scene since about 1993 to about 2000, so the first volume also evoked nostalgia apart from being fun and awe-inspiring. The second volume was no less awe-inspiring - watch Lapsuus by Maturefurk and then consider the fact that it was running on an Amiga with a Motorola 68060 CPU that was running at 75 MHz at best! Amazing coding skills at display on an amazing piece of hardware.

Update (2007-08-14): Part 2 is now on-line.

Update (2007-08-22): Part 3 is now on-line.

Update (2007-10-22): Part 4 is now on-line.

Update (2007-12-12): Part 5 is now on-line.

Update (2008-02-11): Part 6 is now on-line.

Update (2008-05-13): Part 7 is now on-line.

ICFPC 2007

I spent most of this weekend taking part in the ICFP contest for 2007. As in 2005, I took part in the contest with Yumpee as my team-mate. Our team was called "Kalianpur Bakaits" (an IIT-K reference) in the beginning, but we changed it to "The Great Indian Rope Trick" later, for reasons that should become obvious in a while.

This year's task was very similar to that from last year, but it was a bit tougher and even more fun. The organisers surely must have spent a lot of time and effort in putting together the task description and the supporting materials. There were no mistakes discovered in the task description for this year's contest (unlike those for the previous years' contests). The mailing list also had an unusually low-traffic during the contest - perhaps it was because the task description was precise enough, perhaps it was because a lot of people were finding it difficult just to get started or perhaps the contest was affected by the release of the latest Harry Potter book during the same weekend.

This year's task was to save an alien named "Endo". Endo was a "Fuun" travelling in a spaceship named "Arrow", when it had a mishap with an "interstellar garbage collector" that picked it and then dumped it on Earth. This caused critical damage to both Endo and Arrow. Our job was to repair Endo's DNA in such a way that Arrow is able to restore it to its former condition using its limited remaining power.

Endo's DNA is a long sequence comprising four bases, 'I', 'C', 'F' and 'P'. This sequence can be interpreted to create an RNA sequence comprising the same four bases that in turn can be used to build the proteins necessary to restore Endo. Prefixes comprising the four bases can be attached to a DNA sequence to modify its behaviour - we had to find a cheap and low-risk prefix that Arrow could process to repair Endo's DNA.

This translates to creating an interpreter for the DNA sequence that produces an RNA sequence, creating another interpreter for this RNA sequence that creates an image out of it and finally discovering a prefix that could restore Endo. The organisers provided us the DNA sequence for Endo, a "source" image resulting from interpreting this DNA sequence and a "target" image representing a fully-restored Endo. We were asked to create a DNA prefix that results in an image that is as close to the target image as possible, while being as short and as cheap to process as possible.

The task description was detailed enough for us to readily build a DNA to RNA converter as well as an RNA to image converter. The challenge was to do the processing quickly enough since the given DNA sequence required about 2 million iterations over a string that was about 7.5MB long. I created the DNA-to-RNA converter, while Yumpee built the RNA-to-image converter.

Since we had to splice and join huge lists while interpreting the DNA, I started off with a C implementation that used Lisp-style "cons" cells. While it seemed to perform well (but not good enough for the task), it had hard to track bugs that caused segmentation faults, not to mention memory leaks. I then tried using Java for creating the interpreter, which resulted in a correct but even slower interpreter. By this time, I had wasted two out of the three days allotted for the task. We had not made much progress while a lot of teams seemed to have discovered the same "magical" prefix comprising 28 bases that had boosted their rankings.

This is when I finally decided to act on Yumpee's suggestion of looking into using "ropes" as the underlying data structure. He had been suggesting it repeatedly since the time he had seen the task description, but I kept putting it off since I was not familiar with it. I decided to check out the "cords" implementation of ropes that comes with Boehm-GC. I modified my original interpreter in C to use cords instead of my home-grown and buggy cons-cells implementation.

The difference in performance was startling! The very first implementation ran through all the 2 million iterations in about 10 minutes on my PC and in about 4 minutes on Yumpee's PC. The code had also become much simpler to read and much closer to the pseudo-code provided in the task description. I debugged it a little to remove a bug that resulted from a misreading of the specification and then handed it over to Yumpee. By this time, he had a working RNA-to-image converter (he was also busy barking up the wrong trees till that time) and had also discovered the magic 28-base prefix. Unfortunately for us, we had just nine hours left to finish the task.

We discovered that the fun had just started. Embedded in the DNA were a series of clues that formed a virtual treasure-hunt, involving encrypted pages, EBCDIC-encoded text, fractals, callable functions, etc. We discovered a lot of clues but we couldn't find a prefix in time that would improve our rankings. It felt good that we could make so much progress by the end of the contest, unlike the last time. However, I felt like an idiot for not having listened to Yumpee earlier and saving myself and our team a lot of wasted time.

As I always end up saying after such contests, I hope we have a better luck the next time.

Some more reports: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, etc.

Update (2007-07-26): I tweaked the code for the DNA-to-RNA converter a little bit to avoid creating a lot of unnecessary cords and that reduced its running time to about 1 minute 35 seconds on my PC (about 7 times better than before) and about 30 seconds on Yumpee's PC. Right now the garbage collector itself is the biggest consumer of CPU cycles (about 45% on the whole), but disabling it leads to the programme quickly running out of memory on my PC.

"Finance, Investments 'n' Trading"

"Finance, Investments 'n' Trading" is a weblog by Shobhit that tries to bring some sanity to the general mania currently surrounding the Indian stock markets.

You should read his articles in chronological order (he provides a helpful "Table of Contents" article for this purpose). He might sound too pessimistic to some and I do not necessarily agree with everything that he says, but the articles are still a good and recommended read, particularly if you actively buy and sell stocks or are thinking of getting into it.

The Indian stock markets have been in a prolonged and an almost-continuous bull run for about three to four years. So many people have apparently made so much money with such ease that everyone from students and housewives to army men and retired government workers want to jump into the bandwagon for fear of being left out. It does not help that newspapers, magazines and TV channels provide a disproportionate coverage of the "excitement" surrounding the Indian stock markets as if it is a form of sport.

When caution and rational reason are abandoned in times of general euphoria, many people are likely to get their hands burnt (some times without even realising that they have made a net loss, taking inflation, taxes and trading charges into account). The only people who consistently make money in such situations are the stockbrokers (for example, read Warren Buffett's "Chairman's Letter" to Berkshire Hathaway Shareholders for 2005 (PDF) from page 17 onwards).

I don't think that investing in stocks is necessarily a bad thing or that the current bull run is not backed by a real growth in the Indian economy. I just wish that people think a bit more rationally, keep realistic expectations of returns and figure out how to calculate their real loss or gain from trading, before jumping in.

I wish that more people read and understand the advice given in Benjamin Graham's superb book "The Intelligent Investor".

Expiry Dates

If medicines and food items become dangerous to consume after their expiry dates, shouldn't pesticides become more powerful after their expiry dates?

Dragon Ball Z

Many years ago, a cartoon TV channel in India started showing Dragon Ball Z. They showed 53 episodes of this series comprising the Vegeta Saga and the Namek Saga. Just as the series got really interesting, they yanked it off the air without notice and without even a word of apology.

Loyal viewers of the series were aghast. They pleaded with the channel to show them the rest of the series as well, rather than leaving them on tenterhooks. The channel could not care less. The viewers were frustrated and cursed the channel using the choicest of expletives. They did not have much choice.

Several years later, the same cartoon channel starts showing Dragon Ball Z once again. Some of the old viewers hope the channel would show the entire series this time and watch the episodes once again to revive their memories.

They were naïve. The channel showed the same 53 episodes of the series and then abandoned it yet again at the same critical point in the story.

They were idiots. The channel as well as the viewers.