Effective iOS app UI testing with AutomationTools

Introduction

UI tests for iOS apps using XCUITest have received increasing attention in recent years as the platform has matured and uptake has increased. UI tests are an integral part of the testing pyramid and should be considered in the test strategy for all apps/products. So, we created AutomationTools, a framework with a set of UI tests best practices and common functions that can be used across projects. 

Why AutomationTools?

The JustEat iOS app is highly modularised for better code architecture. One advantage this gives us is that all modules can have their own unit and UI tests. However, this means each module duplicates a lot of setup steps for tests.
Also, we often run experiments or A/B tests in our apps when we introduce new features. A/B tests can be run for a simple UI change on a screen, or change the user journey entirely. It can be challenging to run UI tests while A/B tests are active as the tests do not know which variant the app is running.  When using the XCUITest framework, the way you inject state into the app under test is via the use of launch arguments and environment variables. The launch arguments are passed in to XCUIApplication’s launch method as an array of strings and the environment variables are passed in as a dictionary. While this can be simple, it can quickly become complicated when using the same mechanism to configure experiment variants, state and flags to facilitate the UI test (e.g. simulating that the user is already logged in). 
AutomationTools is a framework that allows the easy setup of state configuration for our UI test suites, and also provides base classes and extensions to remove the need for duplicated setup code across test suites.

The Code

AutomationTools is divided into two sections, Core and HostApp. Core consists of the test case side of state configuration, useful base classes and utilities. HostApp provides functionality that acts as a bridge between the test case and the client app by unwrapping the flags that were injected by the testcase.

automationtools

 

Core

JustEatTestCase 

This is a base class that inherits from XCTestCase. Test cases should inherit from it, rather than directly from XCTestCase. It holds a handy reference to an instance of XCUIApplication which should be used either directly or injected into page objects.  It contains the most common overrides of the setUp() and tearDown() methods, which add in the condition to not continue after failure and to terminate the app during teardown. These overrides can be overridden again by any test cases that require them to be different, but for us this has proved to be a very small number of tests.

PageObject

For easy maintenance, avoiding code duplication and to speed up development of test cases, we use the Page Object Model. PageObject.swift is a base class that should be inherited from by all page object classes. It holds an injected instance of  XCUIApplication to be used when defining the elements in the page object classes (etc.) and also holds an injected instance of the current XCTestCase to allow the use of predicates when waiting for elements or other conditions in page object methods.

Extensions

ExtensionXCTestCase

This contains basic waiting utility methods to be used in tests. We have tried to keep this to a sensible minimum.

ExtensionXCUIApplication

This extends XCUIApplication and provides a launchApp method which takes in arrays of Flags and makes use of the LaunchArgumentBuilder discussed above. It takes in arguments for featureFlags, automationFlags, envVariables and otherArgs.

launchApp(featureFlags: [Flag] = [],
                   automationFlags: [Flag] = [],
                   envVariables: [String: String] = [:],
                   otherArgs: [String] = [])

FeatureFlags and automationFlags have been discussed already. Environment variables are exactly what they sound like. The OtherArgs parameter can be used to pass in any other test setup or state, e.g. configuration for mock data. It should be noted that this is all a nice way of configuring an array of strings to pass into XCUIApplication’s launch app method and that you will need to handle all of these flags and implement them in your host app. Please see the next section.

LaunchArgumentsBuilder

As mentioned above, we run test cases by flagging features or experiment variants on or off as required. Passing this information into the launchApp method as launch arguments gets  complicated when handling multiple features, variants and options at the same time.
LaunchArgumentsBuilder allows you to define an array of flags to configure this state which you want your test to run with. It takes in separate arrays for featureFlags and for automationFlags, and prepends each flag to distinguish them easily. AutomationBridge.swift discussed later contains the reverse functionality for the host app to use to unmarshall the flags for use in the appropriate state configuration.
Flags are structs defined as:

public struct Flag {
    public let key: String
    public let value: Any
}

It is important to distinguish between the feature and automation flags:

  • Feature flags should be used to drive decision making paths in code to allow easier testing (for our products at Just Eat we use JustTweak, a framework for feature flagging and A/B testing for iOS apps) with an ephemeral configuration.
  • Automation flags should be used by the app to set up the environment (e.g. directly deeplink to a specific screen, simulate the user to be logged in at startup, etc.)

HostApp

AutomationBridge

This class contains the functionality to unmarshall the flags that were passed into the launchApp method by the testcase. The client should make use of it to obtain these flags and then implement their state configuration in the client app.
AutomationBridge contains a method called isRunningAutomationTests which will return a boolean if the runningAutomationTests flag was set in the launchArguments. This should be referenced when performing UI test specific setup etc. in the client app code.
It also publicly exposes the ephemeral configuration that is used to set up feature flags in the client code. Again, we suggest using JustTweak to set up a TweaksConfigurationsCoordinator with an ephemeral configuration that should be given the highest priority. 

Conclusion

AutomationTools has allowed us to easily configure complex state in our UI tests. It has ensured consistency in our test suites and the included base classes have made it quicker than ever to get started with new projects.
While each project has its own unique requirements, using AutomationTools in your test suite should make it easy for all of your future projects to have a consistent approach. You will be ready to configure more complex state in your app when tests require it!
For more specific information and for code examples, please see the project README.