9 automation testing tools to consider for web applications in 2018

Automation-testing-tools

The year 2017 saw an impressive number of important events as far as UI automation testing tools go.

In 2017, we witnessed the end of developer support for Selenium IDE and PhantomJS. We’ve also had a full year to evaluate how the 2016’s acquisition by Micro Focus has influenced UFT. Finally, the releases of Angular 5 and React Fiber have introduced new challenges for QAs, making GUI testing tools even more relevant.

Given all these news, what automation frameworks and tools will become the mainstream options for guarding the user interface against bugs in 2018? Read on for an overview of top web automation solutions across three categories:

UI automation testing frameworksDesktop automation tools for enterprisesCloud testing platforms
Selenium WebDriverTestCompleteScreenster*
ProtractorUFT Pro (LeanFT)Ghost Inspector
CucumberEggplant FunctionalUsetrace

*Yep, thats us — and we’ve got awesome features to back up our claim for leadership.

UI automation frameworks. Selenium WebDriver, Protractor, and Cucumber

Code-based testing frameworks have their drawbacks, but they will persist driving UI automation in teams that have enough time to write UI tests. For these teams, Selenium, Protractor, and Cucumber will remain the preferred options 2018.

Selenium WebDriver: old but gold still got it

In 2017, people still consider it a must for every specialist involved in GUI testing to learn Selenium. And there’s a good reason to. Selenium WebDriver underlies the W3C WebDriver standard, it’s native to major browsers, and it’s used in all higher-level automated testing frameworks.

Being low-level, Selenium requires a solid grasp of good testing practices. The low-level nature of this tool also makes Selenium less productive than higher-level tools. Inspite of these two facts, Selenium has a reputation of the “golden standard” of automation testing of the GUI, which is why many teams still choose it over dedicated high-level frameworks. Specifically, many React teams prefer to test web applications with Selenium.

Protractor: test automation framework for Angular apps

While React doesn’t seem to have a UI testing champion, the situation is different in the Angular realm. When it comes to the automated testing of Angular-based web applications, Protractor enjoys the status of the go-to solution. Specifically, software testing engineers name several advantages of Protractor:

  • Protractor is the brainchild of the people who wrote AngularJS.
  • The framework has a clever way of handling Angular waits.
  • Protractor’s accessors make for an effective Angular-specific locator mechanism.
  • It’s always nice to work with a framework that ships with a test runner out of the box. In case with Protractor, the test runner is Jasmine, and there are many Jasmine fans out there.

Cucumber: BDD champion for UI

BDD frameworks stand apart from other test automation solutions. If you buy into the idea of “business-facing” Gherkin scripts, Cucumber is pretty much the only game in town. Capybara, the web-based automation framework for Cucumber, is also something you’d want to use for BDD-centric acceptance testing.

Desktop automation testing tools for enterprise: TestComplete, UFT Pro (LeanFT), and EggPlant

It’s not uncommon for large enterprises to gravitate towards all-in-one suites of automated testing tools that work for both web and mobile testing and have every feature one can imagine. The all-in-one nature of these products also manifests itself in how they automate tests.

IBM Rational Functional Tester, for instance, combines coded tests with screenshot comparison and “storyboard testing” which uses natural language for test editing. In the meantime, Telerik Test Studio emphasizes the comprehensive coverage of GUI, API, Load, and Performance testing.

While both of these two GUI testing tools offer a slew of neat features, this review will focus on other solutions. Specifically, we will zoom in on the tools featured in the Gartner’s Quadrant for automated software testing. The table below offers a comparison of TestComplete, UFT Pro (LeanFT), and EggPlant Functional.

Automation testing tool

TestComplete

UFT Pro (LeanFT)

Eggplant Functional

DeveloperSmartBearMicro Focus (the app was previously owned by Hewlett-Packard)Testplant Inc.
Position in the Gartner’s Quadrant for 2017ChallengersLeadersVisionaries
Runs onWindows (desktop app)Windows, Mac, LinuxWindows, Mac OS X, or Linux
Web testingChrome, Firefox, Internet Explorer/Edge, SafariChrome, Firefox, Internet Explorer/Edge, SafariChrome, Firefox, Internet Explorer/Edge, Safari
Mobile testingiOS, AndroidiOS, AndroidiOS, Android
Visual testingImage comparisonNo (image comparison and record-playback are available with a UFT license)Object recognition, image search, image recognition, and text recognition
Test scriptingJavaScript, JScript, Python, VBScript, DelphiScript, C#Script, C++ScriptJava, C#SenseTalk (proprietary), or Java, C#, and Ruby
Selenium supportYes, via Java’s JUnitYes, via Object Identification Center, Objects locators & utilitiesNo
Supported IDEMicrosoft Visual Studio, EclipseMicrosoft Visual Studio, Eclipse, IntelliJMicrosoft Visual Studio, Eclipse
CI supportJenkinsJenkinsJenkins, Bamboo
Known issuesSlow to support new IE versions, weak with Java appletsLarge memory consumption footprint, extensive infrastructureThe software is still buggy (even though the TestPlant team fixes them fast).
Pricing modelNode-Locked License tied to a machine

Floating User License tied to a user account

Seat-based license / 3 months
Custom pricing for enterprises
Pricing information only available via contacting TestPlant
Free versionFree trialFree trialFree trial

 

Cloud testing solutions for codeles and low-code automation: Screenster, Ghost Inspector, UseTrace

In the last couple of years, a slew of web services and platforms emerged to enable efficient low-code automation of UI and functional testing. Like Selenium-based frameworks, these testing tools drive the browser to emulate user interactions with websites and web apps. Unlike Selenium and similar solutions, these tools test the whole UI instead of targeting separate user journeys.

Even though people sometimes compare cloud testing platforms to Selenium IDE, their functionality goes beyond simplistic test recording and screenshot comparison.

For instance, modern cloud-based UI testing tools are able to recognize on-page objects and UI areas with dynamic content. They can automatically ignore visual differences caused by rendering and handle timeouts with little to no human input. Being web-based, these tools also feature collaboration functionality. Finally, they support coded test scripts and continuous integration.

The table below will compare the three tools that offer advanced functionality for powerful test automation.

Screenster

Usetrace

Ghost Inspector

FunctionalityFunctional and smoke testing (UI)
Visual verificationYesNo (Screenshot comparison only)No (Screenshot comparison only)
Content verificationYes (automatic)Yes (manual)No
Navigation verificationYesNoNo
Automatic locatorsYesNoNo
Self-healing locatorsYesNoNo
Automatic waitsYesYesYes
Management of dynamic UI areasYesNoNo
Built-in Selenium commands for in-browser codingYesNoYes
Importing / exporting testsImporting Selenium testsExporting tests as JSONImporting and exporting Selenium tests
Parameterizable tests / VariablesYesYesYes
Support of different screen sizesYesYesYes
Creating a basic test takes < 3 minutesYesYesYes
Fast test editingYesYesYes
Reusable testsYesYesYes
Supported browsersChrome, Firefox, IEChrome, Firefox, IEChrome, Firefox
Collaboration functionalityShared access to dashboards and test editingShared access to dashboards, tagging testsShared access to dashboards and test editing
ReportsScreenshots and test stepsScreenshots and test stepsVideo, screenshots, test steps console information
NotificationsEmailEmail, Flowdock, HipChat, JIRA, SlackEmail, Slack, Webhooks, HipChat, PagerDuty
Installation/setup footprintFully web-basedFully web-basedRequires a dedicated Chrome plugin
Test serverCloud-based or localCloud-based; NGrok tunnels to local machinesCloud-based; NGrok tunnels to local machines
Concurrent test runsYesYesYes
CI supportJenkins, TeamCity, Travis, BambooJenkins, Bamboo, CircleCI, Codeship, Visual Studio OnlineJenkins, CircleCI
REST APIYesYesYes

 

Automated functional testing of the GUI with Screenster

Screenster, Ghost Inspector, and UseTrace narrow down on test automation on the UI level. Instead of offering you the broad functional scope of enterprise platforms, these tools do one thing, but they do it really well. Here’s what our very own product Screenster can offer for powerful website and web application testing:

1. Pain-free test maintenance

Screenster enables you to record and execute automated tests without hand-coding. While this is the same sort of functionality that you get with some enterprise IDEs, there’s one major difference. The tests generated by the record-playback tools of enterprise IDEs typicaly come as auto-generated code which is often a pain to handle.

Instead of making you fumble through autogenerated test code, Screenster allows for quick and fully codeless test editing. Given how frequent the UI changes can be, you’ll appreciate how much sense this feature actually makes.

2. Self-healing tests with smart locators

When creating the baseline of the UI, Screenster automatically builds complex locators that include all HTML attributes and structure-based selectors for every element. Should a UI change of any kind break one of the selectors, Screenster will automatically substitute it with another one. This way, you can perform test automation without having to worry about UI updates.

3. Intelligent visual testing with verification of visuals, DOM, and content

Screenster automates UI testing by capturing the DOM structure and the content of the page, in addition to taking screenshots of every UI state. All of this information is stored as baselines of each UX step, and Screenster checks new UI versions against these baselines during every test run.

When it comes to visual testing,Screenster does more than mere screenshot comparison. The platform distinguishes between style changes and content updates, providing you with valuable information on the nature of every UI bug.  When looking at screenshots, Screenster sorts out the differences associated with anti-aliasing. It also detects UI elements with dynamic content and provides you with a simple functionality for managing these regions.


There’s a lot that can be said about the advantages of low-code cloud-based solutions, but giving them a try is worth a thousand words. Check out our demo, and you will see if cloud-based solutions have what it takes to rank among the top automation testing tools of 2018.

 

Want to try Screenster on the cloud?

Try Online
9 automation testing tools to consider for web applications in 2018 was last modified: September 14th, 2018 by Ilya Goncharov
Thoughts on “9 automation testing tools to consider for web applications in 2018”
  1. Looks like a great list, thanks! I liked that you’re talking about different web testing tools, no just the usual roster of frameworks. There’s a typo, though: in the last table, you have a full stop in one of the th tags.

  2. Ghost Inspector is okay, but I kinda wish it could do more… Let’s see if Screenster has all that you’re writing about

  3. I worked with TestComplete and UFT (back when it was owned by HewlettPackard). The two are pretty much similar. Do the same things slightly differently, but you’re basically gate the same out of the them. I also have limited experience with Katalon. The observation I have with tools like these is that teams that move to them rarely go back to Selenium

  4. That’s the whole point. actually! There’s nothing inherently bad about coding your tests — as long as you have the people and the budget to do this. But most small teams don’t really have that — and they either write very few tests or run very few manual tests. Codeless IDEs could’ve done a lot in those cases where people use Protractor nowadays. But codeless IDEs still have a long way to go

  5. looking at this list of automation testing tools, TestComplete seemed really awesome back in the day. now its foothold is basically undermined by apps like yours

  6. Comparing Screenster vs Ghost Inspector, you’re obviously doing more in the visual comparison department. Your Content Comparison algorithm is something that really sets you apart. Hope you’ll keep on improving!

  7. Would be great to see more tools compared in the last section. There’s, like, a dozen modern automated testing tools for web applications in addition to Screenster, Ghost Inspector, and Usetrace. For instance, I heard about LeapTest and iMacros on quite a few occasions

  8. Would be nice to see open-source tools like Sahi Pen Source and Katalon Studio covered as parts of this list.

  9. While I like the general idea behind open source automation testing tools (as opposed to just frameworks), there’s always a catch. Katalon isn’t really free: if you want decent technical support (and believe me you’ll need because it’s a smallish product built by a smallish company), that’ll be 2.5K/year charged as “business support” package. I’ve never tried Sahi but I heard that people end up using the Pro package (not sayin’ it’s a bad thing thought). Basically, there’s really no such thing as free lunch, and I’d rather know the price upfront…

  10. Speaking of testing tools for web applications, this post should definitely mention Sahi

  11. I wonder is these website automation tools work for wordpress, and I I really need them for a wordpress site.

  12. Jest seems like a decent test automation framework. The thing works with TypeScript which is getting popular in 2018, and there’s built-in focus on React which is even more popular. There are other goodies as well, like code coverage reports tool and a mocking library. It’s also used by Facebook (naturally), as well as Twitter, Spotify, and Airbnb. I’d say Jest is worth considering is you’re working with JavaScript in 2018.

  13. For some basic visual testing (i.e. screenshot-taking), you can use Modules like Protractor-snapshot. It’s more dev-centric, though. But than again, few companies that I’ve worked for have huge teams of real dedicated QAs for software testing (I’m guessing, that’s more typical of larger companies, startups don’t hire testers now-a-days).

Leave a Reply

Your email address will not be published. Required fields are marked *

WordPress Image Lightbox Plugin