nikhil's posterous

Toggling Proxy settings quickly with Pentadactyl and Firefox

If you use the excellent Pentadactyl plugin for Firefox to get vim super-powers to the browser, here is a quick and painless way to toggle between a direct connection to the internet or using the default proxy settings. Add this to your ~/.pentadactylrc

command proxy -nargs=1 :set! network.proxy.type=
nmap up :proxy 1
nmap np :proxy 0

Now pressing up (Use Proxy) will enable Manual Proxy Settings while pressing np (No Proxy) will use Direct Connection. network.proxy.type can take other values which might be suited to your setup. You can change the key bindings too.

Tic-Tac-Toe with Redis and Backbone.js

This post is a technical overview of my Tic-Tac-Toe implementation. It is a zero server-side logic, pub-sub based, realtime, multiplayer game which uses Redis and Backbone as the key enablers of real-time and moving logic to the client side, respectively. Now that all the buzz-words are out of the way, the source code is on Github. There are rough edges with reliable communication, security holes do exist, but for the most part it works well.

I would specially like to thank Nicolas Favre-Félix for Webdis without which this would’ve been impossible.

Basic functioning

On visiting the home page the user has an option to either join a game, or play with a friend. The join game option pairs the player with another player who also wants to play (if you are the only one online, you’ll have to wait). If you play with a friend you get a link you can send him/her so you can play together.

The players are asymmetric in the sense that there is a ‘hoster’ and a ‘joiner’, whose roles I’ll get into in a bit.

Technology Stack

Tic-Tac-Toe game me the opportunity to experiment with a ton of technologies/products I hadn’t handled before. The stack goes like this.

• A Linode VM hosts the service
• Redis powers base Pub/Sub
• Webdis provides a REST API to Redis so that the clients can directly talk to it
• nginx serves static files
• haproxy routes requests to nginx or Webdis depending on the path
• Backbone is used for MVC
• Raphaël is used to draw the grid and pieces using SVG
• jQuery is used for AJAX and the impromptu is used for modal dialogs
• underscore is a utility belt
• UUID.js for generating UUIDs

Yep, thats a lot of stuff for something so simple, so I’ve tried to rationalize it below :P

Clients

All clients are identified by a UUID. The ‘host’ player’s UUID is used as the name of the Redis Pub/Sub channel. The ‘joiner’ will also subscribe to this channel and publish to it. This is why the ‘host’, ‘join’ bifurcation, so that a common channel can be used for communication. All messages are JSON objects.

All communication is initiated by the ‘joiner’. The host player keeps waiting. The joiner starts the game, after which both players keep sending each other the moves made by the humans playing. After each move, both sides check for a win/lose/draw. Again, the joiner sends a gameover message when it detects somebody has won (or it’s a draw). The host then verifies that this is actually true, and responds back with it’s own gameover message. Then both parties change their UI accordingly. This causes the slight delay between the actual win and the notification.

Server side

The first question is ‘why both nginx and haproxy?’ I could’ve run Webdis and nginx on two different ports since Webdis supports CORS, but Opera does not support it. Running just nginx web-facing and then proxying to Webdis based on path is also not possible since nginx does not currently support HTTP chunked replies, while haproxy does. Chunked replies are used by Webdis to relay Pub/Sub messages.

Implementing ‘Join Game’

To allow two people to be paired, a Redis list is maintained. When a client clicks Join Game it attempts to LPOP a UUID off the list. If none are found (no other players), it RPUSHes itself onto the list.

Client side

The client side has all the logic and so is more interesting. I won’t go deep (you can read the source), but will discuss the key areas.

Once a game has been initiated, both clients open a XMLHttpRequest to the subscriber channel. They watch for readystatechange events and try to parse the messages. This is the Subscriber which fires events when it receives valid messages.

The Publisher component is used to send messages, while Webdis is used for other Redis commands (currently only LPOP and RPUSH).

The GameRouter is the controller, sets up models and views, watches for gameover and beginning a host or join. HTML5 pushState or hashes are used to offer permalinks for games. /host/UUID is for hosting and /play/UUID is the joiner. Backbone’s Router and History is used to cleanly handle this. One feature I wish there was, is a way to query what the current route is via Backbone instead of mucking with window.location.

The GridModel drives the game once it begins, triggering gameover events and checking the grid after every move, also handling the turns. GridView handles rendering the SVG grid and pieces based on GridModel and processes clicks while HUDView notifies the user of his turn, piece and game state. Events are an integral part of this entire setup and Backbone makes it very simple and modularized.

One thing that stands out in my use of Backbone is the absence of Sync. I’ve not used it, although it integrates well with Backbone. The game model didn’t seem suited to it, being more event-based rather than the model reflecting the game state. Collection is also not used since there is only one grid.

Issues

The lack of any server side code does lead to certain issues. None of them are serious when it comes to Tic-Tac-Toe but some affect the user experience and others are security issues.

Usability

On the usability front, the current code is very fickle. The joiner only tries to initiate the connection the first time it starts. If the host fails to respond for some reason (e.g. network connectivity) then both parties will keep waiting. If a joiner refreshes his page mid-way through a game, the game will restart for both parties (a good idea if you are a joiner and you are losing \:P). If the host refreshes the page, the joiner has to refresh after him.

The slight lag to decide win/lose/draw was probably unnecessary. Rather than verifying both sides are on the same page, the notification could’ve been shown directly since this is just a game. The UX would’ve benefited.

Security

Webdis sports a HTTP interface to Redis, but there is no real authentication support, which means the Redis instance hosting Tic-Tac-Toe is also effectively a public domain database (although the command set is restricted). Ideally this should run only in a trusted intranet. Ideally Webdis itself would be capable of serving files and deliver something like a nonce which it would then use to ensure that only its own connections are allowed to relay messages. Similarly nothing is stopping someone from grabbing the UUID and then playing future moves using a bot of some kind to always play optimally. In the case of Tic-Tac-Toe this is just a minor prick, but for actual client-side MVC applications this requires fixing.

Conclusion

Tic-Tac-Toe demonstrates that with a data structure server and HTTP interface to it, web applications where all logic is client-side are possible. Using the server as a Pub/Sub relay also allows near real-time performance, with WebSockets or SPDY possibly leading to better performance. Security policies still mean that true peer-to-peer isn’t possible. Authenticity and authorization also remain to be solved. On trusted intranets, such applications could be used for non-critical tasks. Meanwhile, games like this at least can be safely implemented and played as long as you have no sore losers :).

Why Indian students should attend college

In recent months, the number of posts extolling dropping out of college, or of people recounting their experiences (mostly positive, probably because the negatives won’t share) has increased substantially on Hacker News (I believe Steve Jobs effects on humanity extend here too). Meanwhile the The Story of Average Indian ‘Techie’, What’s your GPA? and other posts bring to the fore some things I do agree with:

• Most computer science curricula are outdated or just poor quality
• The majority of students are in it for the money
• The professors are almost always bad

In fact I didn’t particularly like most of my CS courses either. There were a few gems like System Software and Computer Networks at DA-IICT, but the rest were totally out of sync with the real world. So if you are a precocious hacker should you drop out of college in India? (Assuming your Indian parents will let you do that!)

NO!

Try your utmost to get into a good college with good infrastructure. Here is why you would want to do so. Not only is the infrastructure itself important, it also attracts the smartest people. Do well in college while improving your own skills and knowledge.

My reasons are based on personal experience, and in ways document some of my shortcomings too :P

Like minded people

Unlike the dense technological concentration of Silicon Valley, India doesn’t have technological hotspots. Even in Bangalore, very few people are passionate about technology and are hacking on open source software or launching startups (while this is pretty high in India terms, it is nothing in Bangalore terms). FOSS talent is instead concentrated in the students of engineering colleges. (I focus on FOSS because it’s a good way to filter out passionate people.) You will get to discuss problems, hack on code and be inspired by these people. A concentration of geeks also leads to geek events like hackathons. During various college fests there will be programming contests and so on. You’ll get to have fun. Finally there will be a lot of smart people doing things other than computer science. But they will be equally as passionate as you are, they will be liberal and forward looking and it will be a pleasure to interact with them. Oh and please don’t think of every person you approach as a potential future employer, employee or general networking and increasing contacts kind of person. Sometimes you (and certain people in the Valley too) just need friends. Face it, do you want to spend the next few years talking to your mom about why REST-ful APIs are the bomb or why this is funny?

Facilities

High-speed Internet access in India is still not too common, but colleges will usually have a great LAN setup, a lease line to the Internet and generally good connectivity. Use these to experiment with your peer-to-peer applications, host websites, or write the next great DDoS program (I’m not advocating this). That said they also may have ridiculously bureaucratic system administrators, censorship and the like. You just have to deal with in (and in some cases, circumvent).

Your college will also have a certain ‘relic from the past’ which is far more useful than any of our modern day, 140 characters technology. The library. Specific technology education is always best done via Internet, but general concepts and deep theory is still found in books. Use it well, you will regret the day you leave college and books will have to be purchased. (To overseas readers – there is hardly a public library system in India.) Oh, and do remember the fiction section.

Motivation and Persistence

When you are working on personal projects it’s very easy to give up or change tracks. You also tend to focus only on the things you like. College courses will force you to persist at subjects you don’t like, and keep you onto one thing for 3-4 months. Valuable lessons when your first commercial project is 90% done and you don’t want to polish it up because node.js just came along and is much more fun to play with. Do a great and challenging final project and end your education on a high note.

Find some other interests

Your life isn’t going to be just about CAP theorems, cache invalidation and naming your projects. You will have to interact with society. Take a humanities course. Learn to loosen up a bit — travel, listen to music and play sports. Waste time with friends once in a while. Don’t burn out before you’ve even started. Your whole life before 25 should not be spent being a workaholic. Sometimes I think the Valley propagates Minimum Viable Product, pitching to VCs and beer and pizza far too much. You don’t want to end up like this. You might also want to try some of these things.

So if you were thinking of dropping out, just give it a second thought. If you are still convinced you should drop out, do it. But please let me know at nsm.nikhil@gmail.com why you did so.

A High-Five Experiment

Final exam time is always a time of stress for students. When you have 6 exams in 6 days, the time just before the exam is one of general apprehension and attempts at mentally revising all the key points. Everybody notices that in the glum looks students have walking into the exam hall.

Fortunately, the fourth year in DA-IICT is pretty light, and due to a multitude of projects, me and Naman had only a couple of papers. Inspired by Charlie Todd’s TED video about Improv Everywhere we decided to inject some cheer into the poor souls of the other batches.

Four signs went up on Thursday, Nov 24, along the long, curved staircase leading to the hall.

They were nicely spaced out so only one could be seen at a time, and we were standing at the end of it, right at the entry doors.

As the students started coming in and the high-fives started flying, accompanied by words of encouragement (and the occasional “Heap sort is O(nlgn)”), it was very satisfying to see those faces light up in the happiness of an unexpected surprise and lose some of the pressure.

So go out and spread some cheer! You can freely use and modify the above images for a noble cause :) There will of course always be people who are unable to take in why someone would do something like this, they’ll walk past you without an upward glance. Others will eye you with suspicion even as they raise a half-hearted hand. Grumpy administrators will tell you you aren’t supposed to be in the examination area if you don’t have an examination. But on that day I saw the most introverted student give a resounding high-five back and walk with a spring in his step. It was worth it.

Have some humanity

This article was first published in Entelechy (Issue 29, September 2011), the in-house DA-IICT magazine.

“Technology [is] the knack of so arranging the world that we don’t have to experience it.”

-- Max Frisch

DA-IICT is one of the few colleges in India to make humanities courses mandatory for students. It is sad then that most students treat it as a course to pass, and not as a way to gain insight into the world they’ll spend the rest of their lives in. I am going to try to convince you that humanities courses are perhaps more essential than even the technical courses.

Observe the typical young engineer as he gets placed and eventually graduates from college. Engineers used to have dreams. That is why we have the Taj Mahal and the Golden Gate bridge. Today the most skilled engineers end up sitting at a desk writing non-user-friendly software for some mega-corp. Or they start a startup which aims to make another form of real communication virtual and ‘social’ without considering the repercussions.

The humanities have always been a has-been simply because they offer no financial value. A poet does not produce a life-saving vaccine or the next Fortune 500 company.

The fundamental schism lies in the fact that engineers want concrete answers to problems, while the humanities never answers anything. It is about concepts and interpretations and I think engineers find that hard to fathom. Trust me, try it once, it is fun.

We engineers are children of the binary, decisions are absolute, choices are fundamental. That is not how the real world works. The humanities teach us to look hard into those gray areas, and how they end up shaping history. I remember in the Environmental Studies class when the professor remarked that in the case of the Narmada project, you could not simply relocate the tribals. An economist or engineer is trained to see the world in terms of resources and equations and profits and margins. Our problems are so simple. We think that by throwing more hardware at it, or building better technology things will fix themselves. That to build a dam, we can simply move the people out and give them good homes. But we have no way to measure social cost. So the moving of tribals seemed a trivial problem. But the land they live on has been theirs for thousands of generations and they associate traditions and religion with it. It would be like evicting you from your home. All technological problems are finally attempts to improve society and the context in which they are implemented is essential.

Even if you don’t want to be the decision maker or ideal citizen or a analysis spewing geek when all that your friends wanted to watch in the movie was the explosions, there is one concrete reason that you should take atleast a few humanities courses.

Writing. The Indian education system especially thrives on canned solutions for much of school. Even the technical courses in college do not require writing papers or projects. But much of real-world engineering today is a team activity where written communication is a very important skill. For all your career you will be writing documentation, making reports and presenting findings to your boss. A good command of English and an ability to deliver crisp writing can help immensely. The humanities courses will be the only ones where you will have to analyse some aspect of art or literature, critique it and back up your opinions with arguments. Since you can’t do a copy-paste in humanities (since it doesn’t have any concrete answers), it is a good lesson in writing.

Finally remember that as bits seep more and more into our lives, our cultures are framed by the file formats and user interfaces and other mechanisms that we will make. And they will enforce the way we think. Do we want to end up in Orwell’s 1984 or Huxley’s Brave New World? Facebook friends vs real friends, privacy vs sharing, customer-friendly or corporate-friendly, patent laws and other important ‘wars’ are going to start erupting. Yet I find engineers have no awareness for any of this as they sit in their cubes creating the most widely propagated products that ever existed, constantly connected via a medium whose drivers are human. All these are areas where theologians and philosophers and lawyers have been arguing for centuries, in the eternally fluid and muddy concepts of property, privacy and ethics. Except they used to be able to make these decisions before the technology spread. Now App stores and locked-in products arise everyday, social networks grow exponentially and international surveillance is easy as pie, and law makers cannot catch up, so the engineer will have to specify those decisions by product design itself. There you and I will enter into the indefinite world of humanities because these problems have never arisen before. Only someone who understands both technology and humanities can solve this, otherwise we end up with abominations like the Digital Millenium Copyright Act or Software Patent Law based on real patent laws when it doesn’t fit the software model. This requires an ability to mull over these concepts and use the various interpretations debated in the past and the present. In some literary passage somewhere may lie the perfect system you strive for. The times, they are a changin.

Talk the Walk

Prelude

A few months ago, on the way back to Gandhinagar from attending DocTypeHTML5.in I wondered if it would be a good idea to walk from Ahmedabad to Gandhinagar to get home. The distance is 25km. The people accompanying me immediately shot down the idea :) but it stayed in my mind as something that had to be done at some point. So no, we didn’t just wake up at 4:30am one day and decide to walk to Ahmedabad. There was a plan. Over a busy semester I forgot about it, but in the summer I knew that the plan had to be executed as soon as I was back to college.

Are you crazy?

When I bounced the idea of a few people, pretty much all of the first reactions where “Are you crazy?” followed by refusal to accompany me. This kind of reaction was disappointing. Only two people stepped forward. One had had a similar idea in her head. The other was ready to do it. After significant rescheduling, the plan was finally put into motion on September 25, 2011. The latter had to drop out due to unforeseen circumstances.

The route

Our definition of ‘reaching’ Ahmedabad was to walk along the Sarkhej-Gandhinagar highway till Iscon Mega Mall. The Google Maps distance is 24.5km. The Google Maps walking estimate was 4 hours 58 minutes. The anticipation itself was a major stimulant.

We left the DA-IICT main gate at 5:23am. Here is a log of all the landmarks, distances, and when we reached there.

Adalaj                 |  7.3km |  6:45
Waterside              |  9.0km |  6:57
Sardar Patel Ring Road | 12.8km |  7:40
Nirma University       | 13.8km |  7:50
Gota Circle            | 15.3km |  8:25
Gujarat High Court     | 19.2km |  8:53
Gurdwara (Acropolis)   | 23.2km |  9:35
Iscon                  | 24.5km | 10:15

At Acropolis we took a 15 minute restroom break. We did not sit. If I had sat down, I probably wouldn’t have gotten up again. We were now officially in Ahmedabad, the rest was just a formality to satisfy predecided constraints.

The last 2km leg was finished in about 20 minutes. To be precise, we went up the south escalator and touched the glass door. The time was 10:15am. 4 hours 52 minutes with a 15 minute break.

The feeling: priceless!

(We took an auto back to DA-IICT after resting for 10 minutes.)

Tips

• Don’t think about the distance, or how much is left and where you are. There is a difference between observing and thinking. We observed landmarks and kept a log, but if you keep thinking it’s 20 more kilometres, now it’s 10 more kilometres, especially when you are starting to feel the fatigue, you will want to give up.

• Company is good – Not only is it an excellent way to pass some of the 5 hours, but it keeps the encouragement flowing. Again, avoid talking about the route, try playing games.

• Less food, more drink – I had expected to be incredibly hungry through and after finishing the walk. This turned out to be wrong. Between 2 people the only food we had all morning was 2 chocolate bars and 4 biscuits. I don’t remember feeling hungry for atleast 3-4 hours even after the walk. So don’t bother carrying too much food (fortunately we didn’t). Carry a lot of water (fortunately we did) and some sugary drinks. Each of us had 2 litres of water, and about 500ml of mango drink. The lack of salt was noticed though, and we should have carried some salty drink like lime water. That said, even if you aren’t feeling hungry, keep snacking lightly after the walk.

• Take care of your feet – Wear good shoes. I was initially planning to wear my Vibram Five Fingers, but didn’t because I have never walked more than 3-4km in them. I was a bit skeptical about the Saucony Kilkenny XC4 when I bought them 2 months ago, but they turned out to be brilliant shoes, never feeling uncomfortable or a burden. You will get blisters. Wear soft socks. If you tend to sweat a lot, powder your feet before you leave. Since we refused to take a break, friction did eventually catch up. I think my first blisters appeared about 15km in. By the end I had 6 reminders of the walk :) Carry flip flops to change into at the end of the walk. Your feet will love you for it.

• When you reach the Wall, ignore it – During my first 8km run last year, I realized what I had only heard from others. The Wall exists, and it is all mental. Your body is an exceptional piece of engineering that can keep going for a lot more than a mere 25km. But eventually your mind will start cribbing and noticing the pain and say things like “I’ve already done 15, this is enough for now”. That is your Wall. Punch a fist into that wall, gather up the pieces and stuff it in a little corner. Left foot, right foot.

Awesome! Call me the next time you go

We succeeded. Objectively 25km is nothing, but subjectively it was the longest I’ve ever walked, and more than most normal people ever walk and so I consider it an achievement. When people found out, the “Are you crazy?"s were still the first question, followed by awe, followed by "Why did you do it?” and then “I want to do it too. Call me the next time you go”. Why did we do it? There was no point, no success to be achieved, no money to be won. It’s just that for me, life is about pushing myself. Where people shy away from pain, I embrace it. There is a difference between the pain of pushing oneself and the pain of real injury, and I think everyone should try for the former, but not push on till the latter. It turns out, that as soon as somebody proves it is possible, people see that and they go ‘I can do this too’ Till then, negativity keeps gnawing away, however much you explain it in terms of distances and times. The shortest answer is doing. That’s enough preaching :p

There is unlikely to be a next time. I enjoyed the journey, but there was no end goal, and there are better ways of reaching Ahmedabad :) This challenge is done, it’s time to move on.

Life's Little Things: Chocolate

If you can afford to buy a slab of chocolate once in a while, enjoy it. Few things in life come in small packages yet have great joy inside them.

Life's Little Things: Board Shorts

If you tend to forget your shorts at home when you go swimming, board shorts that dry quick and can be squeezed hard are an invaluable investment.

Toolset

Inspired by The Setup and Pratul’s post, here is how I ‘get my work done’.

Hardware

I currently use a four-core i7 15" Macbook Pro with 8GB of RAM and a 500GB HDD. I love it. For backups I have a Western Digital 160GB hard drive. I use cheap Skullcandy earphones when I need them (I usually prefer speakers).

Operating System

I used to be on Arch Linux until I got the MBP. Now I use OS X Snow Leopard. I still run Arch + KDE on a VirtualBox instance to occasionally hack on KDE. With 8GB of RAM, both keep running snappy :)

Running OSX might come as a surprise considering my FOSS roots. But my software stack is such that it makes no difference what UNIX I use really.

Software

Considering I use the computer all day, the number of applications I actually run is tiny.

I use Firefox for browsing. I hide all chrome except the tab bar, and use the excellent Pentadactyl extension to get vim key bindings and other keyboard driven goodness to Firefox.

For viewing PDFs, I use Preview. The rare photo management is done with iPhoto. Music needs are satisfied with Clementine (my iTunes resides in the Trash), while VLC handles video.

I do use the excellent Notational Velocity for note taking and idea jotting and the like. It fast, it stays out of the way and is easy to sync if I have to.

iCal, synced with Google Calendar, is used for todos and submission reminders.

And that is all the GUI apps I usually run. Which is why it doesn’t matter what UNIX I use, since most of my work is in the shell.

Office suite you say? I find Google Docs suffices, to the point that if somebody sends me MS formats, I upload them to Google Docs. The Gmail web interface is also unbeatable so mail stays there.

Accessories include Dropbox, Temperature Monitor Lite, gfxCardStatus to manually control which graphics card is in use, ShiftIt to position windows, afloat to allow ‘Always on Top’ like KWin, Growl for notifications and Tunnelblick for VPN management.

Now we come to the terminal. My terminal emulator is iTerm. I have only one window running maximized, with tmux acting as my ‘window manager’ for shells. It has a slightly customized statusbar, with the solarized colour scheme. The shell is zsh with a custom zshrc based on stuff I pick up around the Internet, with the zsh-git prompt.

This exact same terminal configuration is also available in the Linux VM (including vim configuration) so that I don’t need to switch my behaviour for the different OSes.

For IRC I use irssi although I use it as a total newbie.

And the rest of my day is spent in vim. I do all my long writing in vim — code, articles, blog posts, configuration files — everything. I use the molokai theme rather than solarized for the editor.

Vim plugins

First Tim Pope’s pathogen is indispensible to easily manage other plugins.

a.vim is a convenient little script to toggle between headers and sources.

ack.vim allows the invocation of ack from vim and presents the results in the Quickfix buffer. I map the keystroke ‘sd’ to ack.vim.

fugitive is useful for Git integration with vim, but I don’t use it that much yet except to do blames.

fuzzyfinder is another indispensible script for me. I map ‘sf’ to fuzzyfinder so I can quickly invoke that.

Other plugins I use include nerdcommenter and vim-surround.

My Dream Setup

I would love a standing desk in my hostel room with an external monitor connected to my laptop, the way I worked at Mozilla. I am considering buying a Netgear Ultra 2 NAS to have redundant storage and sharing. Otherwise the MBP works great and is sufficient for now.

Harry Potter and the Deathly Hallows Part 2

I first saw DH2 on Friday night, and my thoughts after watching it were:

On Wednesday night I saw it again, determined to find out why I disliked it so much, and whether I was just being too hard on it. Interestingly, most Potter fans seem to have liked it this time, which was not the case before. Mike Patterson though has this interesting writeup pointing out in very specific instances why the movie was bad. I myself have only raised my rating a couple of notches. Seen independently, the movie is decent, but after multiple readings of the book and ten years of association with Harry Potter it is plain bad.

To get the good parts out of the way, the Prince’s Tale was beautifully done, Alan Rickman brings his marvellous portrayal of Snape to a great ending. The movie was visually pretty good, and Hogwart’s defending itself was well done, though a bit artificial in the performances of the professors.

But in the bid to add action and drama to the movie, certain themes of the book were pushed aside, and that made this movie just another action thriller. There was some weak acting in places and just terrible situations too, but to focus:

Imperio

We start at Gringotts and Harry casts the Imperius Curse on the goblin. To lend some perspective, use of the Unforgiveable Curses is akin to murder in the Muggle world. Harry has demonstrated countless times before that he is not comfortable with hurting others (Expelliarmus anyone?). In fact, it is Griphook who has to put the idea in his head.

“Act now, act now,” whispered Griphook in Harry’s ear, “the Imperius Curse!” – Harry Potter and the Deathly Hallows, Page 531 of the US edition

and

And another memory darted through his mind, of the real Bellatrix Lestrange shrieking at him when he had first tried to use an Unforgivable Curse: “You need to mean them, Potter!” – Harry Potter and the Deathly Hallows, Page 533 of the US edition

The use of Imperio in the book is an act of desperation, because certain things are simply evil, even in the magical world, and making a person go against his will is one of them. In the movie, we aren’t even shown Harry’s face as he casts the spell, and when Ron casts it again in the depths of Gringotts, he does it trivially. Obviously you cannot explain thought processes on screen, but a touch of hesistation, an expression, goes a long way into conveying the seriousness of the act. After all the goblin does die in the movie due to the curse.

Wands

Wands are the crux of the books, the one way humans get access to magic. They are ‘alive’, and bond and grow up with the wizard. Wands are respected, even Death Eaters don’t just destroy their enemy’s wand. The breaking of the holly and phoenix feather wand is immensely important. That wand has kept him alive 6 years. Its breaking is also symbolic of how Harry has to sever connections with Voldemort to defeat him. The re-joining of his wand is the last thing Harry does in the book, signifying a new beginning. Where is the holly wand in the movie?!

Now to turn to the Elder Wand, the Deathstick, a wand with more lore than any other. The scene in the Headmaster’s office in “The Flaw in the Plan” is to show that Harry has the selflessness to give up that power and accept Death, but he still does not destroy it because it is an artifact, and Harry realizes that and puts it back in the grave, aware that its power is not really neutralized just yet.

In the movie, he just snaps it and throws the pieces away. You DO NOT snap a wand!

The Kiss

“Is this the moment?” Harry asked weakly… – Harry Potter and the Deathly Hallows, Page 625 of the US edition

Ron and Hermione do not kiss because they destroyed a Horcrux! The Kiss serves multiple purposes in the book. Tension between Ron and Hermione has been brewing since Book 3. Ron has always been attracted to Hermione, but she holds back, unsure of Ron’s seriousness. Book 6 and 7 are about Ron maturing and starting to think about others. Ron abandoning Harry and Hermione and then returning is the test of his growth.

Meanwhile Hermione has always been the one to watch out for others. She also has a fanatical desire for equality of all species. Rowling clearly wanted to point out against discrimination in our society, and what better way than to use a Muggle-born girl with no pre-conceptions. S.P.E.W. is just her being formal, but all through Book 4-7 Hermione has stood up for the down-trodden – even when Kreacher is lying she supports him.

Ron remembering that the house-elves need to be rescued flips the switch. It convinces her that Ron truly has changed and is capable of the compromises required in a relationship. Second his acceptance for something she supports is crucial for her to commit.

The Kiss was not spontaneous, it was building up for 4 books. The only thing spontaneous was the absurdity of the situation. All of this was lost in the movie.

The Flaw in the Plan…

was badly dealt with.

So there is the most powerful wizard of all time, he has the most powerful wand of all time, WHY would he try to rip out Harry’s face with bare hands?

And why would you show both Bellatrix’s and Voldemort’s bodies as some kind of container for their souls, so that once the curse hits, they just disintegrate? Voldemort was after all, only human and the falling of his frail body with “mundane finality” was supposed to indicate this.

Then the theme of love and remorse that is a cornerstone of the book is never raised. That Voldemort’s spells are no longer binding on anybody because Harry has effectively ‘pulled a Lily’ on everyone is skipped. That love and sacrifice defeat Death is ignored. I would have loved to see Voldemort scream “Accidents” like he does in the book, to show his ignorance of “house elves and children’s tales, of love, loyalty, and innocence”.

Expelliarmus and Avada Kedavra, Harry and Voldemort’s trademark moves were for a reason. For Voldemort it represents his conviction that Death is the worst damage you can inflict, while Disarming fits Harry’s character of not wishing death even to his attacker. In the movie the spells were never uttered (and we know that Harry is bad at non-verbal spells). The spells seemed to be different too, since Avada Kedavra does not disintegrate the body.

Dumbledore

Ah, were to start with this. Book 7 took this great great man that was Albus Dumbledore and tore him down, producing the flawed human every one of us is. Harry and Dumbledore are two characters shaped by death and Dumbledore’s reaction proves opposite to Harry’s reaction for a while. Dumbledore is the man who campaigns for equality, non-discrimination and all other things that are Voldemort. He is the character who brings out the deeper flaws in the wizarding world to casual readers. And contrary to being all this through wisdom, Rowling gave him these attributes by making him a man filled with remorse.

Rowling is trying to show throughout the books is that its the choices that matter!

If that wasn’t enough, Dumbledore is linked to the Hallows and Horcruxes and essentially scripts this entire storyline in a very high-level manner. His theories about the connection between Harry and Voldemort and how Magic is connected to conscious phenomena like Love is the only channel the reader has of deeper insights into the Wizarding World. To drop all mention of Dumbledore in the movie was representative of Hollywood’s bid to add only action and bling to known blockbusters, creating movies that do not make the reader think or disturb them.

Many readers judge of the power of a book by the shock it gives their feelings. – Henry Wadsworth Longfellow, Kavanagh, Chapter XIII.

In re-reading this article, I feel I haven’t been fully able to convince the reader due to a lack of writing skills. I’d take that up in specific comments if any :)

Summing it up the books are about death and love and innocence and good and evil, but also about moments and small scenes that create intricate characters in our minds, and frame unspoken laws about the world of the book. You cannot make a good movie without keeping those in mind.