Conflicting Priorities for a Tester in an AI-driven World

Recently, we had the privilege of talking to Anna on various aspects related to the software testing. It was a very interesting talk and so we divided our conversation with her in two parts. In the first part, we shared Anna’s views on the Software testing leadership and the role of the conferences

In this final part, let us talk about the context, conflicting priorities, and the challenges of testing applications powered by the new technologies such as Artificial Intelligence and Machine Learning.

Anand: Anna, let us talk about the changing role of a tester. In the old days, remit of the tester was clear. Test the application and find defects. However, in an integrated team, tester is a part of a larger team and there are conflicting priorities from the PM or Scrum Master, developers, DevOps, Product Owner and so on. How it has impacted the role of a tester?

Anna: There are conflicting priorities from different stakeholders because the meaning of quality is different for every stakeholder. One of the exercises I do during my training is to identify stakeholders. Like really identify all of your stakeholders, who depend on your work, and figure out what quality means to them. This exercise can bring many interesting points and reveals how quality is different for different people.

For example, company reputation could be very important for the C-level people; delivery dates could be very important for the marketing group because they build their campaigns around those dates; and operations or product team can have completely different meaning for quality. Operations Support team do not want 300 people calling on Monday morning to ask the same question – as the customers don’t know how to navigate the software with the broken workflow. An Information Security team could be another stakeholder. They want to ensure that there is no leakage of information.

In order for you to balance all those requests, you need to first discover what quality means to different groups of stakeholders, and then you can have the conversation of conflicting priorities.

This is why; I keep on pushing people into the leadership thinking. You have to understand your business as much as you understand your system. Sometimes business decision is to deliver faster with reasonable quality and that is okay too as they may have appropriate support on the business side to address it. It is possible for business to tell the user that the software is released with some issues — because it was important to release. Ultimately, it is the customer experience that matters. This is something that testers must learn, because if they do not, they may be pushing against the wall. As a test leader, you are expected to learn about the business and sometimes release new version in production with some known bugs. Be friends with business. Understand that they do not want to release with the bugs, but they may have some reasons. Understand those reasons.

Anand: Conflicting priorities are about understanding the context as well. When a tester starts working on a new project, what advice do you have for them to understand the context?

Anna: There are few focus areas you need to go through in order to learn and understand the context of your testing assignment.

Understand your mission and your stakeholders. First of all, why are you brought into this project? Who are you working for? Who needs your services? Find answers to these questions in order to understand the focus and the scope of your work. A hint: do not make your own assumptions about your testing mission, always discuss and confirm with the identified stakeholders.

Understand your resources. Resources are: time, tools, environments and skillsets of the people who can help with your testing mission. For example, if you have a developer on your team who will write a log interpreter tool for you – it will save you time to troubleshoot bugs you found. Also, such tool will help you learn about the message exchange/implementation details of the system without going to all the meetings where such details are being discussed. Another example: operations support admins know production system and users well, as they deal with customers on a daily basis. They can help with testing and will discover interesting bugs – as they know a lot about user patterns and pain points. You can also review test coverage with them, as they will give you some ideas.

Understand your product. Discover what market niche your product is created for, understand the competition, and learn about your product’s competitive advantage — because that is how your product is being sold. That information will help you to adjust your testing focus and identify the risky areas that can compromise the product quality.

Understand your users. Who are the customers of your product? What is their skillset? Which issues are they solving with your software? You need to park your experience as a tester and think like a user and the experience they will have with the product. You may know your product inside out, but your users may not. Understand their limitations.

Understand the depth of system changes. Discover and evaluate risks of the changes that are happening in the program. There could be new features, changes to the existing workflows, there could be changes in the architecture or technology stack. Those are different context drivers and they will have an impact on the way you test.

Understand your technology. You need to grasp the risks as well as the advantages (possibilities) of the new technology that your product is built with, and learn about the tools which programmers use when working with such technology. That knowledge will greatly enhance your testing strategy, as you may need to develop new testing methods and build your own testing tools for your mission.

Anand: You must have worked at many levels. How do you identify that there are some issues with the test function? How would you know if an organization needs help with the testing?

Anna: Basically, what you look for is complains. If there are no complains and everyone is happy, there is nothing that a consultant can bring. If people are complaining, it means they are ready for a change. Normally, people do not want to suffer and that’s what you want to look for. You need to talk with the testers and find why they are not happy and you need to talk with the developers / project managers / business owners and understand why they are not happy with test function.

There is always an opportunity for improvement, no matter what. I’ll highlight few areas where to search and what to look for, when addressing the issues.

Can the process be improved? We can always find a way to fit a tester in a process in a better way. Sometimes testers are excluded from decision making meetings, and sometimes they are brought in too late in the software delivery process and miss the opportunity to discover various issues. For example: discovering an inconsistency in the proposed solution at the discussion stage is much more efficient than finding the same inconsistencies at the product development stage – after the code has been written. Testers in general have a better understanding of the whole system and they can pinpoint such inconsistencies easily.

Are your test metrics/reports useful for the target audience? Have a look at how information is provided from the test team to the rest of the organization. I have seen many times that the metrics do not tell the right story at all. For example, you report that a tester executed 20 test cases – what does it mean? Nobody knows! What stakeholders look for is the information which helps them to make a decision. Provide information about the state of software in a meaningful way to your stakeholders, and you raise their respect and trust for test function.

How does the company evaluate testers’ skills and productivity? For example, is there a common expectation that all bugs have to be found by testers before release, and testers are blamed for bugs in production? Is tester being judged by the number of bugs found? Such expectations can create counterproductive habits in testers, lower their morale and introduce an additional tension between testers and developers.

Are your testers resistant to change? Sometimes, testers do not like flexibility. Some people are comfortable with where they are and there is a resistance to new ideas, sometimes subconscious. People are not comfortable with the change because their approaches will need to change, their routines, their communication and so on. Pad the path to change anticipating the learning curve, and provide support to people in test organization so that they are more comfortable with changing their habits.

Anand: When you identify opportunities to improve, do you take a top down approach or a bottom up approach in introducing these changes? Do you start with the management or the testers?

Anna: Unless you are a manager, you can not tell someone “this is how you should do it”. As a consultant, you can only highlight the opportunity. I have often seen that the biggest complaint may be the biggest opportunity. A consultant needs to look at organization’s major pain points and then address them.

But again, it depends on your role. If those people are reporting to you, you can tell them this is how we are going to work tomorrow. If they are not your reports, you can only offer suggestions. It depends on how you are brought into this organization. A consultant would discuss where the opportunities are, suggest few efficient ways to approach the desired outcomes, offer learning materials to add more understanding to the issues at hand, share case studies, discuss the pitfalls and will generally help the organization create a roadmap to success.

Anand: Anna, Let us talk about the new technologies. Many organisations are adapting Artificial Intelligence and Machine Learning based solutions. In this new world of AI powered, voice activated applications, what challenges a tester should expect?

Anna: The challenges that I have seen with new technologies – the number one is getting ready for it. I run a meetup for that: looking at the new technologies and how as testers we approach them. Usually, new technologies do not come our way straight away. They are usually coming from the top – either from the R&D, Engineering Excellence group or whatever.

Someone recently spoke about AI in the testing department at my meetup. It is very rare now for testers to get involved in ML/AI validation, but AI is becoming mainstream. Within few years, you will have AI in many industries such as Retail, Service, Vertical Farming, Finances, Urban Development – there is AI in everything. IMO, testing is more important in AI than the development because you need to build a testset that can challenge the AI.

Number one in approaching testing of the new technology is to gain the trust of the developers you work with. Number two, you have to really understand the risks of the new technology. In the olden days, if a feature didn’t work, it was a big deal. Now, it’s not. Ethical issues are a big risk now. With AI, there is so much interference with the personal life that ethics becomes an important matter. As a tester, you need to understand that, you need to include mitigation of ethical risks into your testing strategy.

Another thing is the IoT. IoT is very interesting – it watches you. It doesn’t look at your history, it watches you as you live your life. Imagine, your fridge is watching you, it knows what is inside and what you may want to order. With such an environment, you have privacy issues, ethics issues. It is very important to identify the risks and think through potential problems. For example, if machine accepts a verbal command and if you have an accent, would machine understand you? It is a big risk. It would be very important for testers to be aware of what is going on.

We now have virtual reality, augmented reality, driverless cars. Testers would eventually need to test features around that. Testers need to be proactive about these future tasks. Get ready for the new ideas, think about what issues there might be? Who, if not testers, are going to think about those issues?

To me, these are challenges, but they are also opportunities because these are the places where testers can shine. They can identify and highlight the potential risks. They can bring their mindset and skillset to the table to protect organizations from the risks related to privacy, ethics and endangerment of people’s lives.

In such environments, it helps if a tester can “speak the language” of the developer. Know the tools they are using, learn their language, understand their challenges. It is such a new sphere that testers can even come up with their own test methodologies. None of it exists; you can be a pioneer in testing new technologies and become an inventor of new test methods as well.

Anand: What advice would you give to somebody who wants to grow professionally and really wants to become good at it?

Anna: If you want to grow professionally, first of all – be open. Be open to new ideas. Be open to the fact that in some areas your experience doesn’t matter and your expertise doesn’t apply. There is a good exercise that context-driven testers play: a game called Where would it not work? It goes like that: name a testing practice or a process (“best practice”) and come up with situations where this practice or process wouldn’t work, would not apply. It is a great mental exercise to open up your mind to new ideas and to question the notion of the “best” practices.

Second: become part of a professional community. If you don’t belong to such community, you will spend your professional life reinventing the wheel. There are lots of good ideas and methodologies floating around. Community provides a safe avenue to discuss new ideas and practices.

Of course, become really good at software testing. Ideally, find a mentor. Ask mentors to give you hints, give you challenges. Talk to them about their expertise, learn from them.

Participate in conferences. My upcoming conference ConTEST NYC is focused on new things in software testing. There is always something new that people are sharing. That’s the main advantage of coming to the conferences. You learn and network with people who are thinking like you. A lot of things about software testing is gut feeling. Many people outside the software testing world may not understand what testers know intrinsically. It helps to be able to discuss your experience with people who may have the same perception.

And last, but not least: develop your communication skills. That is becoming a requirement now, and it wasn’t there five years ago. Now, it’s “a must have” skill for IT professional. Get prepared to work with others and to explain what you are doing. We, the software testers, rarely know the right way to tell our story, explain our value. Learn to provide your input and help this profession grow.

Anand: Thanks a lot, Anna. There are plenty of things for people to take on from this conversation. Any parting thoughts?

Anna: First of all, I invite everyone to my conference ConTEST NYC. It is happening end of November in New York, USA. ConTEST is focused on new and emerging technologies, ideas and practices in the field of software testing and quality engineering.

When you join the group of people who discuss emerging trends in your profession, you are taking the stand of the thought leader and contributor advancing the craft. You come back to your organization as a leader enhanced with the new ideas and expertise, and that’s very valuable to the company who sent you to the conference. Growing and supporting test leadership is my passion and I strongly encourage everyone to participate.

Finally, don’t worry about the doomsayers who say testing is dead and the job is going away. Ideas, skills and the need of testing is not going away. Critical thinking is always going to be needed. Keep learning and don’t be left behind.

Anand: Thanks Anna for taking the time out to talk with us. I am sure your experience and insight would be extremely useful for many people. I have been following your work for a long time now and I will look forward to your conferences and other contributions in the software testing community.

To our readers, hope you found this conversation informative and insightful. Please, share this conversation with the relevant people in your organization or network on Facebook, Twitter, or Linked-In. Stay in touch!