Thursday, 6 November 2014

9 Tests with FluentLenium

As well as work projects I like to have a personal project which lets me try new things. As past blog entries may have suggested the current one is a web site, running on NinjaFramework. Having got the hang of the core underlying technology I had produced a chunk of MVP site. Now, rather belatedly, it was time to introduce tests on the web interface as well as the code. Ninja has FluentLenium packaged in. It functions as a fluent wrapper to Selenium. FluentLenium was new to me and so I set about breaking it. Here are the edited highlights of that process and a few lessons learnt (breaking things is a really good way to learn).

My starting point was the simplest possible test: load the home page, check that it had some text that was about right (key words in key places rather than full text). Use the example code as a basis.
I started with the HtmlUnitDriver as the engine. It worked.
Next I tried navigating to the about page. Still nothing clever behind the scenes. Then pressing the back button. Ah. Need to switch javascript on. Now it moaned bitterly in warnings about a lot of things, mostly because I'm using Bootstrap.
Another two "load a simple page" tests followed. The warnings were a bit tiresome. The code was getting scrappy. Although the API was FluentLenium they were reading more like a script. Time for a bit of structure, with common elements of the scripts factored into methods and a bit more of the fluent style.
Next test was moving into testing user registration, login, and logout. This involved a form, inputting data, pressing a button that (via javascript) would POST the data and AJAX the response. HtmlUnitDriver was no longer the tool for the job - its JavaScript just isn't up to it. I already had firefox on my local test VM. I was able to add it to my CI slave, the only problem being getting a compatible version of firefox (see here). That also made all the warning noise go away, which was nice.
But, this threw me into the world of asynchronous interactions and waiting for things to happen at the same time as AJAX updates to pages and cookie setting that would vary depending on the success / failure of the operation. The script approach grew messier and the tests less reliable. Pretty soon I had a test of register - login - logout - log back in that just wouldn't work. I could see the web page on the screen updating, but the tests were either telling me that it hadn't changed; or that the cached copy of the page was stale. Later (with lots of await statements and some pruning of the tests) I had passing tests on my laptop which broke on CI.

The details of the route out aren't important, although there is a snapshot encapsulated on StackOverflow. The eventual approach is the interesting thing:
First, embracing FluentLenium and its approach and not poking about with the underlying Selenium. There was a big refactoring into using the Page design pattern, and @Page annotations. This, obviously, started with commenting out the breaking code and reworking the things that I already had working. The separation between user-level script and web-page-internal tests really helped clarity. There was enough similarity between the two test classes that they started to share some code in a super class (mostly starting the driver); and also between the pages (all the static final Strings that refer to the web page and tests for logged-in state that come out of common framework HTML rather than being page specific). Indeed, in a site that uses templates this is to be hoped for.
Next, using the @AjaxElement. Well, I'd tried that along the way, but it didn't help. Because of the next thing. But, as part of the overall picture it works and makes the rest work!
Then, simplifying the tests. The scripted approach encouraged adding bits to the stories represented by the test. I needed to remember that the existing test was testing something. The additions were new tests. In particular this meant I was coming back to @Page objects and expecting change, but finding breakage. Each page field could only be used at one place in the test; once navigation moved on the field was no longer helpful. Getting an @AjaxElement works once; not for a whole string of prod / test cycles. Having more than one field for the various visits to a given page was considered, but could always be avoided so far by having the tests test just one thing.
Finally and most trickily, making the assert call to test that the @AjaxElement is what it ought to be from the test object rather than within a method of the page object. Not an immediately obvious step. So this (for me) is broken:
class Page {
  @AjaxElement e;
  public void checkE() {
    assertSomething(e.getText().contains("stuff"));
  }
  ...
}
class ETest {
  @Page
  EPage p;

  @Test
  public void test_eStuff() {
    goto(p);
    p.checkE();
  }
}

and this is not:
class EPage
  @AjaxElement e;
  public void getE() {
    return e.getText();
  }
}
class ETest {
  @Page
  EPage p;

  @Test
  public void test_eStuff() {
    goto(p);
    assertSomething(p.getE().getText().contains("stuff"));
  }
}

There was also some fixing of HTML id tags to aid testing that went on, but no real change to the pages (although I did contemplate this at one point).
Since that breakthrough I've added another two tests of user account activity, each with as much ease as I added that second test to load the about page. The code reads well and the collection of classes isn't expanding so fast. Tests still pass on CI. I'm happy. New features can be developed against tests like this, in outside-in style.

No comments:

Post a comment