Automation Abstraction


For all things web development you have to be able to test them and confirm that they work. With the progression of AJAX and DOM altering JavaScript testing these web applications has become more and more of a chore. Unit tests and frameworks provide a certain level of quality assurance from the back but the user’s experience needs to be confirmed as well. The last four years of work at Broadcom has afforded me a number of opportunities to diversify and expand my skills palette, but no matter what I do I always find myself back in the role of a tester, unfortunately.

My passion has always been for solving problems, but finding them is often something I find tedious and boring. I prefer to allow others to do the heavy lifting to afford myself a certain degree of freedom to delve into the reasons for the failure. This is not how the world works though. Some poor schmuck has to take time out of his day to press into the inner workings of some application somewhere and find a bug that has been reported by a user back in 1949 that has very precise and yet imperfect details for its trigger. That person has died and left me their cloak of visibility and anal retentive nature. I don’t like it; it makes my thighs look big.

Finding the bug is often a problem because you have to move straight to edge cases, many of which may not even have existed when the application was originally architected, but have been added in. Picture a sky scraper towering to the sky. Imagine how difficult it will be when you are charged with inserting a floor for security between floors 8 and 9. Now the architect comes to you talking about the front door needing to be spruced up, the arch isn’t tall enough and is turning away the tall people that live in your city. After a while, you are no longer looking at the same building that was originally specified to a degree of anal retentiveness that can only be rivaled by Julia Child’s hunch doctor. This application that may have been perfect at one point is now leaning to one side, has a bungee cable system stapled in and a slide going down from the forty-first to the fifteenth level because of elevators. Try to find the bugs in that monstrosity – oh wait… they are everywhere.

Needless to say, testing is a pain. This is why I’ve turned to automation tools like WATiR, Spreadsheet, Ruby, Python, AutoIT, Mechanize, WGET and shell scripting to minimize my work. Most of the skyscrapers I have to work on are web based and heavily dependent upon JavaScript to get their jobs done. As pretty as they have become with all of the funky transitions and fades, the application still needs to be confirmed to work. Even more frustrating at times is the complete lack of any tools beside WATIR, which is effectively a com wrapper for IE6/IE7, and Selenium which is JavaScript based, to really interact with anything written in and around the web since the 2.0 movement. Here I stand, listening to my instructors of off about Web 4.0 and 5.0 and I am one of the people who rely on 2.0 technologies for my living. It’s quite funny to me.

This past week I’ve taken part in upgrading a testing environment to work on IE7 that has been using IE6 up to this point. I am well aware of all of the IE6 issues you may or may not have enjoyed dealing with. Being that I have been a web developer for some 13 years now I too hate IE6 with a passion that only the Nazi’s have enjoyed, but you can’t take the player out of the game any more than you can remove IE6 from the corporate work front.

So we’ve spun a new VM off to get started with IE7 development of the SQA toolkit I started some 2 years ago, checked out the latest code and all this, but in running it we ran into a slew of errors. Of the 116 test cases, 97 failed right away. What could this possibly be? We ran these very same tests not a week ago and everything was within our boundaries of success. Needless to say, automation has its downside – you have effectively abstracted away all of the fun and time consuming frustration of typing in all of the individual minutia to gain the simpler interaction level. This is one of the major growing pains I’ve had to shoulder and adapt to because as the tool has grown and evolved we handed it off to some developers in China and they took it in another direction of exceptions and unit testing that I had never even realized was necessary, or available. Needless to say, things had to change.

The errors we were getting were comprehensive, spanning every module and every level of depth. One thing that I was taught early was to take small steps to be able to confirm that your tool works and then make sure that the test itself is covering its bases. A small step translates to a level of abstraction that provides exception handling:

Log in as user [email protected]

  • Confirm that the user exists in the database and has the proper credentials
  • Are there permissions issues to take into account? Confirm that the permissions are correct and appropriate for the user.
  • Is there a problem with the load balancer and cookies/session handling?

This list can go on and on for eons, but recognize that there is always another question to be asked. If you ever run into a situation where you cannot go further you just need to consult this list of questions and you will likely find another road bump to be able to address your problem. After a day and a half of head throbbing testing we uncovered an issue with running the applications side by side where all three of these issues mentioned above are working fine, but in running the test cases side by side the application saw that as a security vulnerability and locked the accounts out. After uncovering these issues though we are now pushing forward to be able to confirm that things are working throughout the system and are back on the train towards completion and confirmation that IE7 works just as horribly as IE6.

Any step in any direction away from broken is a step in the correct direction.