Table of Contents
Photo by EttiAmos on iStock
Introduction
In this article, we will review the configuration required to automate our test suite using a remote test lab and a CI/CD pipeline.
Starting project
The app…
Our application used for automated testing is forked from the fooderlich project, a social recipe app available at raywenderlich site.
Lately, Ana Polo has turned it into a mixture of recipes and cats app. Although the recipes are neither for cats nor include them as the main course…
Either way, the app uses a bottom navigation bar as primary navigation mechanism and it contains 3 modules:
…and its test suite
Fooderlich app also contains some integration tests, stored under the “integration_test/app_test.dart” directory.
By the way, if you want a test refresher, you can check the previous article about golden testing.
More info about the project architecture, project structure and tests can be found here.
How will we automate our tests?
Main components
Our test automation strategy will rely on the following tools:
- A remote test lab, the environment the tests will be run on
- A pipeline that will take care of building, running and testing the app
Remote test lab
Firebase Test Lab is a remote testing environment that allows us to run tests on a wide range of devices.
It supports different types of tests, mainly:
- robo: tests that are exploratory and help us uncover bugs by performing certain random actions on the app. The actions performed and/or the number of screens “explored” can be easily configured using the console.
- instrumentation: tests run over real devices or emulators that check certain flow or features in the app. Although they are expensive in terms of resources, they provide more accurate data than other types of tests, since we’re operating directly with the underlying mobile platform (we are not “mocking” the native SDK like we do with unit tests)
Pipeline
On the other hand, Codemagic is a CI/CD tools that makes it easy to build, run, test and distribute mobile apps.
It was built initially as an Flutter-only tools, so it is custom-tailored for Flutter apps.
After building and creating the executable file of the app, Codemagic will connect with Firebase in order to upload it and run our tests automatically.
Let’s see how we can put all these moving pieces together!
Workflow
These are the major steps required to automate our testing pipeline:
- Create a new Firebase Project
- Configure the Firebase project by adding an account service
- Enable Cloud Tool Results on GCloud
- Update the underlying native Android project
- Configure the underlying native Android Test Runner
- Set up a build workflow on Codemagic
As you can see, the first steps require some remote configuration, whereas the last involve local configuration (inside our project).
1. Create a new Firebase Project
Add a new project in the Firebase Console as described here.
Since we’ll use the new project only as a container over which the remote tests are performed, there is no need to add neither Android nor iOS apps. In fact, no further configuration is required either.
However, Android or iOS projects would be necessary if our app used any other Firebase feature, such as 3rd party authentication.
2. Adding a Service Account
Service accounts are used to identify a particular feature inside our Firebase project and grant or deny access to it. Simply explained, think of them as authorization elements.
In fact, service accounts are derived from “Identity and Access Management (IAM)” and they allow us to define the type of access (a.k.a. role) that somebody or something (a.k.a. identity) has over a given resource.
As previously discussed, our integration tests will be executed on Firebase Remote Lab, but they will be launched from the Codemagic CI/CD pipeline. So we will need a service account to grant access permissions.
To create a new service account in Firebase, go to
Home -> Project config -> Service accounts
and then click on the “<n> service accounts” item, as shown below:
Then you’ll be redirected to the Google Cloud Console. On the left menu, select:
Service accounts -> Create service account
When creating the account:
- on step 1, enter both name and description
- on step 2, select the function
Basic -> Editor
- on step 3, you can click “Skip” for now
More info about account creation can be found here.
Once created, the service account key can be downloaded as a JSON file by selecting:
Service account -> Context menu -> Manage keys -> Add new key
The contents of this file will be needed when configuring the CI/CD pipeline, so for now just keep in mind the folder you stored it on.
One more thing! Before we can use the service account key, we must convert it to base64 by running the following command:
cat /<path_to_download_directory>/gcloud_key_file.json | base64 | pbcopy
The last command (pbcopy) only saves the generated content to our clipboard so we can easily copy-paste elsewhere.
Cloud Tools Result API is required too in order to allow the communication between the different processes on the pipeline and the remote test lab.
It can be activated through GCloud, using the following link. Just click the button and then we’re good to go.
Coming up…
On the next article, we will review the remaining steps of the previous workflow and we’ll also check the reports generated once our test suite is run.
Sample project
Until then, check out the following repository containing the complete project:
https://github.com/begomez/Flutter-Test-Automation
Write you next time!
