Today, I chat with Anna Royzman (@QA_nna) - Conference Chair for this year’s annual conference of the Association for Software Testing (AST) scheduled on August 11th-13th in New York City, and the Chair of AST's test leadership Special Interest Group (Leadership SIG). She is a member of the context-driven school of testing community and is an international conference speaker with over 15 years of experience in testing and leading test teams in New York's Financial Trading Industry. Anna is passionate about her quest for quality and discovering new techniques to create effective quality-driven teams.
Q Hi Anna! Let’s start with a topic close to your heart and the subject of your recent presentations. Is the role of Testing / Testers changing? How?
A I think most traditional views on testing established a lot of boundaries for the test profession. Those views had to be revised -- for two reasons. First, the software development landscape is changing: process methodologies, such as Agile, eliminate some traditional roles that historically were performed by testers and test organizations. Second, testers, with their unique skills and approaches (such as mastering the art of asking the right questions, not taking anything for granted, analyzing assumptions) and passion for quality are capable to add a lot of value to the software development teams on all stages of product conception through delivery.
I was affected by this change through my own career: several years ago my position as a Director of Functional Testing was eliminated, as my company reoriented into business units while adopting Agile methodology. I found that my test strategist skills became very important in the new position of the single tester on an Agile team, which then transformed into the Test Lead role. Establishing a test strategy, which the whole team could execute, required deep knowledge of testing methodologies, along with the flexibility to adapt to ever-changing project needs. Becoming a quality advocate in cross-discipline teams was challenging, as it required me to find my own arguments in encouraging quality processes. Making everyone else on the team a quality advocate required my former managerial and facilitator skills.
The changing role of software tester is the focus of my research and latest conferences’ presentations. I first talked about my experience and findings at CAST 2013, and the audience related to my story: testers find themselves in new roles in many organizations throughout the industry. To succeed, the testers need to find their own identity in the new cross-functional environment, they have to master new skills, and they will discover new exciting opportunities as well.
Q What is the biggest skill in your opinion that is lacking in most Testers today? Why? How can one get better at these skills?
A In my opinion, the biggest skill which is lacking in most testers today is the ability to advocate and facilitate the process where testing becomes an integral part of every stage of the software development life cycle, be it Waterfall, Agile or any other. I mentioned ‘facilitate’ for a reason, as not all testing can and will be done by testers. Recently, I attended the LeanUX conference, whose attendees are mainly user experience specialists. They also spend a lot of time on testing to validate whether design assumptions will produce the desired outcomes. UX designers have strategies and tools for testing their assumptions. In this way, their approaches are very similar to those of testers.
Let's look at the bigger picture: validation of business claims, system architecture, functional specifications, software framework or support processes are equally important for the project’s success. Testers may not be in the best position to conduct a specific type of validation or review, due to their skillset or number of people on the project, but they should be able to encourage and facilitate the testing and validation processes. To have more weight in such conversations, it's good for the testers to learn the various strategies used by different disciplines to validate their assumptions. At my company for instance, a team of testers initiated "NASA training" (simulations of system failures) for operations support staff, thus testing both production support process as well as their own tools developed for system monitoring.
To become quality advocates, testers need to develop their soft skills. Speaking about testing and quality with people in different positions requires knowledge of their ‘business’ language and understanding about how they think. You would not talk about quality in the same way to a C-level manager and to a lead programmer. We at Leadership SIG, the special interest group at the Association for Software Testing, started interviewing software testing experts on how to approach various levels of management. Our project is called “Talking to Management about Testing”. It’s scheduled to air in the next month.
Q What are different career opportunities a tester has? How important is it to have a role model to reach different levels in the career?
A Career opportunities for testers vary in different organizations, but in general, you can describe them as: growing in the professional track (becoming an expert in specific types of testing, a test architect), growing in the managerial track (from line manager to head of testing for the organization), or expanding one's expertise in some other area (such as an agile coach, scrum master, project manager, tech lead or business analyst). All these opportunities are rewarding in some ways, and have their pitfalls as well. They obviously require mastering a different set of skills to progress.
Finding a role model is important, as it gives you a reference point and reality check (by comparison), as well as the motivation to grow professionally. Another suggestion I would give to testers who want to advance their career is getting a mentor. Choosing a mentor who is also a role model, gives you an unfair competitive advantage in your career. I wrote an article called "Peer Pressure" (published in Tea Time with Testers, September 2013), that talks about the leaders in my community that are my role models, and who became my peers and mentors as well.
Without aspiring to be like my role models, I would remain at the same level as I was in 2007 (arrogantly, I thought I knew everything by then). Looking back, I can see the gap that I bridged by being humble while ambitious; these two qualities allowed me to learn from my role models, and, at some point, reach their level as well.
Q What is more important for a tester - to have management skills or technical skills?
A I consider the most important skill for a tester is to be able to select appropriate testing strategies, and apply them efficiently within the context of their specific project and team goals. As far as management vs. technical skills – I have always believed that everyone has to follow their heart and find a job that fits their own talent.
My most recent training is built on seven tester types - the categorization developed by James Bach, which is derived from his observations during coaching and evaluating testers. Finding own personal type is important, as it allows the tester to understand his or her unique advantages, and realize weaknesses. Based on the natural preferences in how they approach testing, test professionals can find appropriate ways in utilizing their talents.
When I build my testing team, I try to complement each other’s skills: some people are better at administrative/managerial tasks, and some are better at technical tasks; each team member has something they are passionate about and something they do best. I try to arrange and delegate the tasks that are most satisfactory for each team member. Such approaches benefit the whole team as well, as everyone naturally contributes to the team’s success.
Q What advice will you give to encourage more women in testing?
A Since 2012, I speak at 3-4 international testing conferences each year, and every conference I go to represents visibly equal number of women and men attendees. However, I don’t see too many women speakers at these conferences, presenting some interesting, unique content. Also, I don’t see many women in testing getting publicly recognized for their work.
My advice to encourage more women in testing is to find the role models amongst women who are known experts in software testing and give them more exposure. Creating more visibility for women’s achievements in testing and celebrating them will also draw more young female professionals into this field.
I would also encourage women testers to visit schools and have face-to-face interactions with students, sharing their career experience. I teach few courses with STeP (Software Testing Education Program) at Per Scholas, the non-profit IT education school for adults. Usually, a quarter of my time with students is spent on Q&A about my career, how I reached the level I am at, and what skills are needed to succeed. Oftentimes, I give more personal advice to women students, seeing that some of them are less assertive, compared to male students in the same class. Encouraging by personal example is the key.
Q Do mental models help or inhibit testing?
A Mental models improve one’s capabilities to design better theories in understanding and predicting system’s behavior. Testing is about designing and executing experiments that are proving a theory about certain behavior of the system under test. Abstract and cognitive skills are necessary for the testers, if they want to rise above the perception of ‘button clickers’ that could be replaced with automation.
Many years ago, upon becoming a manager, I “banned” the test case spreadsheets in my team, replacing them with the test coverage document – an outline of tester’s ideas on how to address a specific testing assignment. Nowadays, a method called mind mapping became very popular in the community of skilled testers; it’s the brainstorming exercise allowing tester to create a visual diagram to logically organize their testing ideas and information. There are tools that allow turning a mind map into an outline.
Why did I ban the test case spreadsheets in the first place? It was impossible to review, as the quantity was overweighting quality. In order for me to understand whether the tester put enough thoughts into the developing a “plan for attack”, I would need to grasp the concepts of how they think about the assignment. Reviewing an outline – a mental model of tester’s understanding of the assignment - allows me to quickly analyze the tester’s approach without spending hours in reading meaningless copy-pasted text. It also helps with the follow-up conversations -- identifying risks, pinpointing redundancies or missing coverage.
Q What are the skills required to innovate? Can one be coached / taught how to be innovative?
A First, I think innovation needs the right environment to nourish. For that reason, I believe that building the right culture will motivate people to be innovative. Here comes skill #1: recognize whether your corporate culture is encouraging innovation, and become a drive for change if it doesn’t. I learned how to be a good change agent at AYE quartet workshops that are taught by Jerry Weinberg, Johanna Rothman, Esther Derby and Don Gray.
The second important skill – understand your context. Do not innovate in a vacuum, do some research; invite more people to vet your idea out. There could be another team or project working on the ideas you had, or hidden restrictions which prevent your ideas from implementation, or there could be another way of solving the ‘problem’. I’ll give you a personal example. My company runs semi-annual hackathons, where technologists can introduce innovative ideas and build the team to work on their implementation for two days (Hackathon is a very powerful innovation concept, as the ‘green field’ mentality that team embraces while working on someone’s idea for the short period of time motivates the team members in becoming innovative when they go back to work). I had an idea to redesign a company directory, as the old one was outdated. Once I announced my idea, multiple people started coming to me complaining that their major issue with the company directory was that they didn’t know where everyone sits and where the conference rooms are (we occupy 4 floors). People were late for meetings as they had trouble finding conference rooms. My hackathon team attracted not only technologists, but also human resources and office services representatives as well; and we brainstormed together. The solution was elegant, and it was half-technical and half-physical. We decided to add the seating information to the company directory, and also install physical floor maps for every floor, together with navigation signs.
Finally, the brain has to be open and trained for innovative ideas to flow in. Educate yourself on various models, understand systems thinking, and train your brain constantly. There are several methodologies that are designed to fuel innovation; my favorite is TRIZ (translated as Theory of Inventive Problem Solving). I also recommend exercising your core cognitive abilities on a regular basis. Lumosity works well for me. Testers from Context Driven Testing community favor brain games - we play them at the conferences and testers gatherings.
Q What is your opinion on certifications in the testing field? Which ones provide value in your opinion? Why?
A I don’t have much confidence in testing certifications what are based on memorizing the definitions and are not supported by experiential learning and demonstration of the learned skills. The one that I can recommend is called Black Box Software Testing (BBST); the Association for Software Testing offers it. It is a highly interactive online course in guided learning, which also requires for the students to work together, participate in discussions, and provide feedback to each other. In my opinion, BBST takes the learning to the next level – as the groups of students not only acquire skills interacting as a testing team, but also master providing and justifying their feedback (high quality feedback to the fellow students finals is a requirement to pass the course). Those skills are invaluable at the workplace. Besides gaining deeper understanding in the software testing concepts, I also got a lot of value from this course as a test manager.
Q How do you identify if a QA is trusted or not in the team? How can you build trust between different roles in the team, including management? What are your tips to build trust in distributed teams?
A When the QA is trusted in the team, they are part of solution discovery and decision-making. The visible signs of that: the tester is invited to meetings with developers, designers, business analysts, participates in discussions, and is welcomed to provide their own opinion. If the project kick off or requirements review meeting gets rescheduled to accommodate the tester – that’s an indication of trust.
Trust is never built overnight. It is created when the team members understand and value the unique focus the testers bring to the table, and have confidence in the testers’ skills. First and foremost, such skills are defined by the ability to analyze information and question assumptions. Those skills, when paired with domain knowledge, transform a tester into a subject matter expert who can assist a team to avoid many pitfalls on the road to successful product delivery. Professionals of other disciplines welcome the tester who is a domain expert, an analytical thinker, and who exhibits good communication and social skills. When this tester also has a proven record of good work, he or she is naturally trusted.
In my opinion, the key components to building trust between different roles in the team, including management, are: having everyone aligning to the business goals, maintaining complete transparency and establishing a good track record of successful deliveries. A tip to building trust in distributed teams is to have joint learning sessions: technical walkthroughs, requirements reviews, retrospectives, conferences with business and management (to learn business priorities), conferences with sales reps and production support (to hear customer stories), etc. Such sessions establish common vocabulary, get people familiar with each other, and keep all team members on the same page. It is a good first step toward building trust.
Q Should testers think beyond (and deeper) than Functional Testing?
A Bugs are inconsistencies in assumptions or implementation as compared to stakeholders’ expectations. The majority of bugs result from either misunderstanding of stakeholders needs, insufficient skill set in designing or developing software, limited domain or technical knowledge, miscommunication, stress or plain sloppiness. Look at the "testing" as the process of finding such inconsistencies in assumptions or in implementation -- as compared to stakeholders’ expectations. If you view testing this way, ask yourself a question: can the bugs be residing only at the level of implementing functional requirements? That would be far from reality.
I’ll give you a few examples where the first half of the sentence is “software is working per specification, but…”
... It’s not solving customer’s problems (the “bug” is in business assumptions)
... It’s not scalable with additional end users (the “bug” is in technical architecture)
... The users need handholding by customer support each time they try to complete their task (the “bug” is in user experience design)
Should testers think about those bugs? Absolutely. Especially if they want to advance their expertise and gain respect as Testers,
Q For a long running program of (distributed) work, what are your tips on Test Management?
A Tip #1 is a specific mission statement for the testing team on every stage of every project within the program. That statement has to be available for everyone to see, not only testers. Having a mission statement allows to reduce unreasonable expectations as well as align testers activities with the project goal. Consider this example: one project is in R&D state; another is ready for production (shipment). Obviously, the goals of testing on these two projects will vary significantly, and thus the mission statement for each one shall be different.
Tip #2 is transparency and communication. The testing team’s activities and tasks should be visible to everyone. I run a distributed testing team, and use a Jira Kanban Board to keep track of my team’s tasks and assignments. The Jira plug in allows tracing testing sessions and linking them to the assignments. Tracing testing sessions is a Session-Based Test Management practice, which I favor over maintaining spreadsheets with pass/fail results. Also, it wouldn’t be a surprise to say that testers are not only working on testing assignments: they are involved in environment upgrades, troubleshooting production issues, writing help files for operations support, maintaining automation test scripts, and in other activities which are not considered to be ‘purely testing’. Keeping all the tasks in one place is good for the team, as everyone can see what others are working on. Our QA Kanban Board is also used to communicate the status on our daily standup meetings with the cross-functional project team, and with management.
Tip #3 is defining the process guidelines, making them publicly available, and keeping timely retrospectives to analyze and improve the process. Note: defining guidelines is not the same as defining procedures. The teams should be able to adjust the processes that will enhance their effectiveness and working together, especially when they are distributed. The process guidelines should be targeted to this goal, allowing for teams to innovate. The practice of frequent discussions about process improvement is great for team building as it motivates individual team members to self-organize and reach the common goal. Frequent retrospectives are invaluable for distributed teams, as otherwise people feel alienated by the time and space boundaries. Such activities also help to integrate new team members that are moving from one project to another.
Q What is the future of Software? How does testing fit into it?
A To get a grasp of what is waiting for us in the future I recommend attending designer conferences. Those people are hired by companies to conceptualize future products and offerings. The way I see the indicative trends - software will become more interactive, more integrated, and incredibly personal. Also, there will be even more competition to win the customer over, pushing for higher quality and faster time to market. Testing will need to align to the new demands. Start thinking how would you test the Artificial Intelligence of the future, what experiments would you design to test an “intelligent agent” – a self-educating system that “perceives its own environment and takes actions that maximize its chances of success”. It’s a good forward-thinking exercise to prepare you for the future of testing.
Thanks Anna for your perceptive thoughts!
Check out other great interviews in this series with James Bach, Lorinda Brandon, Markus Gärtner, Matt Heusser and Justin Rohrman.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.