After using Virtualbox (a free virtual machine) with MultipleIE and IETester installed, we thought we had the Internet Explorer (IE) browser testing issue covered for the most part. Though not perfect (MultipleIE is no longer supported and IETester is in alpha) both applications seemed to replicate the general functionality and habits of IE6+. Over a sustained period of time we found that inconsistencies occurred when comparing test results from IE natively to the results shown from these applications. Something more robust was required.
We ruled out using one of the numerous automated screenshot tools out there such as IE NetRenderer as it was a priority that we could replicate any bugs/issues specific to a browser be it layout or functionality. Another route we took was looking at utilising Microsoft’s VHD Images. Awesome we thought, we can now download and run multiple Windows images inside Virtualbox, each one with a different version of Internet Explorer. Well that’s what we thought. Unfortunately Microsoft changed their policies recently, so anyone trying to run any of these images on a Mac/Linux platform will never be able to get them fully working (tell us if we’re wrong).
Our chosen solution:
crossbrowsertesting.com
As it says on their website:
Test your website live via your browser and remote VNC sessions. Now you can interactively test your ajax, javascript, and flash on all operating systems and browsers from anywhere!
The service provides interactive live browser testing and automated browser screenshots. The operating systems available for usage include Mac OS, Ubuntu and Windows and each has a different group of browsers to test upon (take a look at the full config list). The pricing structure is based upon the total time you use up during your VNC sessions each month. Starting at $19.95 (as of 16/06/10) for 150 minutes, the pricing seems competitive.
The Working Process
Our starting point is navigating to the URL of the website we’ll like to test. We then utilise the Bookmarklet provided by Cross Browser Testing.
On clicking the Bookmarklet we’re presented with a JavaScript driven overlay that allows you to launch the currently viewed site in a specific browser inside a VNC session. You can also generate screenshots of the currently viewed site in selected browsers and view your account profile.
Well we’ve hit a crossroads, what should we show you first? The Automated Screenshots? An Interactive Live Test with IE7? Or perhaps what your account shows you?
Automated Screenshots
Good choice. The automated screenshots page loads up in a new browser window and looks a little like this:
From here you can enter the URL you wish to generate screenshots for (automatically filled via the Bookmarklet) and add/remove the required browsers, operating systems and screen resolutions (in other words different Configs). Click the Take Screenshots button and you’ll be presented with the results (after a short loading time).
The results page displays thumbnails of each requested Config. You can then share these with the usual social media tools, view a full size version of the screenshot, retake the screenshot or launch into a fully fledged Live Test. If you scroll down to the end of the results you can also download all the screenshots in a ZIP file (very useful), create a slideshow, look at the options of accessing the screenshots and the surrounding functionality via the API, download a crossbrowsertesting.com badge or copy a public url.
Interactive Live Test
Ok you want to see the VPN stuff I take it. In this example we clicked on the WinXP / IE7 Bookmarklet overlay which launches the VPN queue timer in a new window. Pretty much this gives you an idea of how long you’ll have to wait for a Config to be made available to you. From our experience the longest we’ve ever had to wait is around 30 seconds (which is excellent).
Once the Config is ready the connection options are made available to you. Chicken of the VNC is our preferred VNC client but we have tried using OSX Screen Sharing and it works just as well. You copy the VNC Host IP & port into your VNC client and enter your Cross Browser Testing account password. The options across the top are to end the testing session, refresh the page, take a snapshot/screenshot of the config, record a video of your session and to send text to your session window. Any snapshots/screenshots/videos you produce during a session are added into your account for future access.
Lo and behold a VNC connection showing the requested website in the requested Config. *Awesome*
Once you’ve finished all the testing you need to do, close the VNC client and click the end session button in the Config options browser window. You’ll then be presented with a mini feedback form, asking you to review your session. We’ve had the odd minor speed issue but we’ve never had a session that’s been below a 4/5. The guys are also very good at responding to feedback, both positive and negative so always leave them a note if you come across anything out of the ordinary.
Your account
From your account (accessed via the website or the Bookmarklet) you can view the usual account information, your time usage, all previous session information (time used, date, Config etc) and all previous screenshots.
Sum it up
Cross Browser Testing has become a key part to our workflow. It allows us to react to time critical situations where a bug has been found in a specific browser. A good example is when a client phoned up about a Magento 1.3.1 installation doubling up the quantities in IE8. We were able to quickly launch the site in IE8 via the Cross Browser Testing Bookmarklet and replicate the bug. Five minutes later the issue was fixed after replacing a piece of code (event bubbling was occurring).
We would recommend Cross Browser Testing to all developers that are in a professional environment.
