A few years ago, here at Bumble — the parent company that operates Badoo and Bumble apps — we started to shift towards using autotests more in our development, testing and release processes. We have, however, encountered a few challenges along the way. Before we start, you might be curious to know why we have been investing in Test Automation […]
A few years ago, here at Bumble — the parent company that operates Badoo and Bumble apps — we started to shift towards using autotests more in our development, testing and release processes. We have, however, encountered a few challenges along the way.
Before we start, you might be curious to know why we have been investing in Test Automation in the first place. The answer is simple: our mobile applications have weekly releases. For the QA department, this means we need to test the feature itself, integrate it into the application and do the regression. Effectively, complex regression testing must be performed at least once per week, which is frankly exhausting.
Test Automation was our response to this situation and was intended to make releases faster and more reliable (read the full story in more detail here).
As you know, when trying to resolve an issue in software development we often use a pattern — a general, reusable solution to a commonly occurring problem within any given context. Test Automation is no exception, there is even a handy Test Automation patterns wiki reference. In this article, our main focus will be on Process patterns for Test Automation. As their name suggests, they show how the Test Automation process should be set up or how it can be improved.
Without any further introductions, here are the issues that we’ve struggled with the most and some recommendations on how to overcome them.
Initially, our QA team wasn’t very confident with automation tests. As often happens in engineering, our QA team is diverse: we all have different backgrounds, experiences, and domains of expertise. It’s important to highlight that back then not everybody in the QA team had been exposed to Test Automation. When we decided to invest more in autotests, it was a steep learning curve for some of us. To help our QAs overcome their new challenge, we encouraged them to ask the Test Automation team for help. The reasoning behind this was that if you get stuck on a particular problem, you shouldn’t ponder too long on it — others may have faced the same issues and so perhaps there’s already a solution. Still, in order to avoid overwhelming the Test Automation engineers with loads of requests, we decided to formalise the approach. The ASK FOR HELP Process pattern seemed to be a wonderful route to go — Ask for help instead of wasting time trying to do everything yourself. After all, no single team is going to have all the skills needed at any one time but instead, having a pool of experts to turn to, is the way to go.
As it turned out, merely having a pattern to follow can cause problems. The Test Automation team started getting even more questions than usual (for example, on issues around environment setup). Over time, we often noticed the same questions were being asked by different team members. It was taking significant time on the part of our Test Automation team to respond to all those people because even a simple well-understood issue requires context switching.
However, for most cases, we do have documentation and solutions for common issues on our local Wiki. It may be not the best way to document all possible errors and explanations but we thought it would be useful to have them stored somewhere.
So, we came up with the idea of local StackOverflow. Our current solution is based on the Open Source solution — Scoold. It is designed to act as the first line of inquiry. It works like the common StackOverflow but is limited to the people inside the company. Whenever somebody has a problem, it will be enough to use the local StackOverflow: either to find the answer or to post a new question and get a reply from the expert team.
Whenever possible we seek to share code across the apps/platforms to facilitate maintenance and support. But as with any other system developed over the years by many contributors, different parts can be written in different styles. Our main language for end-to-end autotests is Ruby (to explore the history of Mobile Test Automation in our company click here). In Ruby, intentionally, there are several ways to do the same thing. This is because Ruby’s creator Yukihiro Matsumoto wanted developers to use whichever way was natural to them to get the job done. Furthermore, as you remember, we all had different previous experiences and so might have coded in different ways. Consequently, these all meant it may be challenging to work on someone else’s code.
SET STANDARDS process pattern — Set and follow standards for the automation artefacts — was chosen to deal with the problem. Without that, it might be that many people work on the same project, but all use different styles and methods (imagine they “speak different languages”). For this reason, they are unable to share their work efficiently. We started by gathering feedback from the individuals who would be using the automation framework and tools. In this way, we could ensure that the conventions we adopted would serve all the users. Moreover, anyone involved in the decision-making would be sure to be more supportive of it in the future. You wouldn’t like having some arbitrarily imposed standards if people don’t understand the reasoning behind them. Thus, we’ve worked up the common guidelines that are shared and adhered to by all team members and helped us to find a common language between us all.
What we have done:
As a team, we are trying both to learn concepts and enhance expertise so we encourage everybody to share their knowledge with others. This could be a root cause analysis of the tricky bug; a case study; a sneak-peek into instrumentation internals; or anything considered to be potentially beneficial to the others.
This pattern can be described as SHARE INFORMATION pattern — Ask for and give information to managers, developers and other testers. For us, this is the way to benefit from a wider pool of knowledge that will improve Test Automation.
Thus, we’ve launched weekly QA lightning talks. Any person can suggest a topic and present it for 10–15 mins, with Q&As to follow. Because meetings are brief and time-limited, there’s enough time to learn something new without disrupting too much of the day. If anyone misses a presentation, there’s still a chance to catch up later, as all recordings are stored in the internal library. Lightning talks are not limited to Testing and Test Automation only but can be on a broader topic. For instance, once our colleague talked about their experience of being a Homebrew maintainer. One of the topics I personally have presented was about me being a volunteer ‘armchair mapper’ for Humanitarian OpenStreetMap. What I do there is create maps for the poorly mapped areas. These later get used by various organisations such as British Red Cross or Medecins Sans Frontieres in their fieldwork. A shortened version of this talk was later featured in the SoapBox session at the Eurostar 2020 conference.
One further point I’d like to mention is something that resulted from the previous steps that we’ve taken. Namely, one outcome was that QAs have become ready to take (and gradually taken) ownership of writing, maintaining and analysing the failed automation tests.
So, the intention now was to go a step further — with QAs taking over the Test Automation Framework and some tools related tasks, while the Test Automation team moves towards more Research, Innovation, and building internal testing tools. Such a shift would be impossible if QAs failed to learn the framework, test inner specifics or did in-depth work with the codebase — beyond their day-to-day tasks. Regular activities were being done to a decent standard, but it seemed that going beyond them would not be easily achievable without help.
To achieve our objective we’ve used the PAIR UP Process pattern — less experienced team members pair up with more experienced team members.
The result is that we’ve launched Internal Mentorships for QA Engineers within the Test Automation team. What exactly is it? QA engineers join the Test Automation team for one sprint, usually a 2-weeks period, and work on the tasks from the Test Automation team backlog, not on their own, but with help and mentoring from a Senior Test Automation Engineer. The main idea behind internal mentoring — is learning. To highlight this, we’ve devised the following manifesto which listed all goals and expectations:
While there is value in the items on the right, we value the items on the left more.
More information on internal Mentorships, why and how exactly we do them, can be found in my recent talk here.
In conclusion, to resolve a commonly occurring problem in software development it’s usually a good idea to investigate patterns in the software design. As for us, by adhering to the Process patterns for Test Automation we were able to resolve the following challenges:
We believe that following these patterns is crucial in making your Test Automation process efficient.
Having listed the pros and potential pitfalls of the selected approaches, we also wanted to show how you can apply the patterns.
In this article, we have only shed light on some of the Process patterns for Test automation. But we hope it will serve as an inspiration and you’ll benefit from following our recommendations.
Please don’t hesitate to share in the comment section the issues you faced in your Test Automation journey and how you resolved them. We would love to hear from you.
How 3 Process Patterns will change the way you do Test Automation was originally published in Bumble Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.