Tag: Teamwork
Too Busy for User Acceptance Testing (UAT)?
I Love UAT
One of my favorite parts of testing is getting the opportunity to coordinate UAT. I love it because we’re getting feedback from the people who are actually going to use the system once it goes live. Even if a team is great about getting user input, everything up to this point is theoretical. All of us can think we know what we want, but once we actually see it, touch it, and live with it, we don’t know if what we asked for is what we really wanted or needed.
I’m planning to write a few articles about what I’ve learned during my UAT experiences, and I hope it will help others with their efforts.
Who Should Perform UAT?
The obvious answer to who should perform UAT is “users”. What users should perform UAT, though? Like many things, “it depends.” For a public-facing application, you may want people from the general public who don’t have a lot of deep knowledge about the system or industry. For an internal application, you likely want experienced users. However, those experienced users will often, understandably, plead that they are “too busy” to help test.
At one of my past clients, I requested the time of experienced users to test a back-end system for processing college applications. Like most companies, they were a bit understaffed and perpetually a bit behind at getting the work done. To the team, and to the manager who was trying to guard their time, my request was just one more bit of unnecessary busywork added to their plate.
The manager offered to hire a temporary worker to perform UAT so their most experienced workers wouldn’t be pulled away from their important work.
The Users Will Live with the Results
Given what I’ve seen others do with UAT in the past, this team may have believed that UAT was simply going to be formality where they would run tests that I had already verified met the written requirements. UAT is sometimes simply a step done to mark a checkbox in the few days before going live. However, when I run a UAT effort, it is much more than that.
I knew that having a temp worker wouldn’t give our team the insight we needed. A temp worker could certainly execute a pre-written test script, but that is not a useful way to perform UAT in my experience. Our team needed to know if this software truly met the needs of the team. We genuinely wanted to know, and we planned to incorporate the feedback into the final product.
I had a heart-to-heart with the manager and the team lead. I explained to them that their team would be using this application to do all of their work in the not-too-distant future. The team, in order to make the most of their investment, would likely be using the software for years. Shortly after the software went live, I’d be gone, but their team would be living with the results.
I explained that we truly were going to take their input into account to ensure the best software for doing their work. Then I asked if they still wanted a temp to be responsible for giving that input and making those decisions that would affect their team for years.
Once they understood the true impact of UAT, the answer was obvious. The manager and lead both made time for UAT, and they even had another employee participate. They brought on a temporary worker to handle some of the more routine clerical tasks instead of having them perform UAT.
We did incorporate their feedback into the final product, and the client was very happy with the end product. It was a true improvement to their old system.
If you could use some guidance with user acceptance testing, please reach out to me at [email protected]
Embedded Quality – Development Team
When I’ve introduced Embedded Quality to the developers on a project, a common reaction is fear and loathing. They often have a visceral, negative reaction to the words “Quality Assurance,” and the idea of working more closely with the Quality Assurance team instills a feeling of dread in them.
As a reminder, I originally outlined the basic concepts of Embedded Quality in this blog post. The concepts are:
- Quality Assurance starts on Day 1
- Quality Assurance is part of the Core Project Team
- Quality Assurance is performed by qualified experts
- General User Acceptance Testing does not begin until Core Project Team QA is complete
- Strong Foundation – No code is “Complete” until it is tested and works correctly
Although the developers I’ve worked with generally have a negative first reaction to the concept of Embedded Quality, once they’ve experienced it, they almost always request it on their next project. The question is, “how can we convince developers to try Embedded Quality the first time?”
Overcoming Common Concerns
I’ve found that it’s typically harder to overcome the developers’ concerns than the Quality Assurance team’s concerns because the developers usually start with the mindset that they don’t want to use Embedded Quality. However, there are ways to address their concerns.
The Code Isn’t Ready to Test
The developers will often say that the code really isn’t ready to test until very close to the end of the project. However, there are some good reasons why this should not be the case.
- The project schedule generally shows sections of code being completed before the end of the project. If a developer says that the code is complete, it’s difficult to imagine why it can’t be tested.
- It’s much easier to identify and fix bugs in small portions of code.
- Finding and fixing bugs early means that new code is built on a solid foundation.
Having the Quality Assurance team test code early in the process is analogous to having a building inspector view various portions of a building before it is complete. It makes a lot of sense for the inspector to check the foundation, the electrical, the plumbing, and the HVAC system before the building is anywhere near complete. Fixing these items after the building is complete is much more costly and difficult than fixing them earlier.
It Will Take Too Much of My Time
Another concern that developers have is that they will have to spend too much time explaining things to the testers. Because the code isn’t complete, they are afraid that the testers will report bugs for items that are simply not yet finished. Also, they will have to spend more time explaining how to test certain items when the complete UI may not be available.
These concerns are legitimate and show the need for having a test team of qualified experts. The testers should have a mindset of collaboration with the developers rather than antagonism. The testers will work closely with the developers to understand what code can be tested and what code is not ready. Of course, any code that is not ready cannot be considered complete for project planning purposes.
It is also true that the developer will have to spend more time explaining how to test some incomplete code. However, if the Quality Assurance team is qualified and experienced, they should be able to quickly learn any tools needed for testing. In my experience, any extra time that the developers spend helping the Quality Assurance team is saved many times over by finding the problems early.
I Don’t Need the Extra Pressure of Someone Looking Over My Shoulder
Many developers have had bad experiences working with testers who take delight in pointing out the flaws of others. These types of testers tend to make comments like, “I don’t know what those developers were thinking,” when they chat with each other in the break-room. This attitude really does put unnecessary pressure on the developers and contributes to a hostile relationship between developers and testers.
This is why it’s critical to ensure that the Quality Assurance team is made up of true professionals. I always remind the developers that Quality Assurance is part of the core team, which means that our goal is to ensure that the project gives a good impression to everyone outside of the team. I tell the developers that I know that they’re good developers and that bugs are a normal part of the process. The developers shouldn’t feel bad if the Quality Assurance team finds bugs. In fact, they should be happy because every bug that is found by the Quality Assurance team is one not found by the project sponsor or the end users. The project sponsor and the end users are the people who are generally the harshest critics of the development team.
Other Concerns?
I’d like to know what concerns you would have introducing Embedded Quality into your organization. What concerns would you anticipate from others? I’ll do my best to address those concerns in future posts.
Offshore Agile Testing…a Request from Management
A friend of mine asked me for some advice about how to run an Agile software development team with the developers and business users working in the US and the testers working for an outsourcing company in India.
My short answer was, “You can’t”.
The Agile Manifesto and Offshore Testing
I’ve actually had similar requests a number of times over the past few years. Managers want to be “Agile” and want to use “Offshore resources” so they ask their project teams to implement the combination of the two hot concepts. In the cases that I’ve seen, management has tried to send only the testing portion of development offshore. They wanted the onshore, local team to be “Agile”, but they also want to send code “over the wall” to an offshore test team. Unfortunately, these two concepts do not work together.
The Agile Manifesto principles include “Business people and developers must work together daily throughout the project;” “Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done;” “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation;” and “The best architectures, requirements, and designs emerge from self-organizing teams.”
These 4 (out of 12) principles are very difficult, if not impossible, to implement in an environment where the testers are separated from the rest of the team by geography and time zones. In addition, when only the test team is outsourced to another company, the testers often are required to communicate with the rest of the team via a single point of contact rather than directly with developers and business people.
To make things even more difficult, I’ve found that offshore, outsourced teams are typically set up to run a full, lengthy regression test for each build. In one case, I worked with a team that was trying to do Agile development with a test team that insisted on running a 6 week regression test for every build. This ran completely counter to the Agile Manifesto principle of “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”
Can Agile Work with Offshore Teams?
I believe that Agile development can work with offshore teams if the teams are structured correctly and significant effort is put forth in building trust. Here are some techniques that I’ve used effectively to help build an Agile, globally distributed team.
- Have a full team consisting of all roles in each geographic location. This means that developers, testers, and business experts must be physically located in each location. The business expert may be a Product Owner Advocate rather than the actual Product Owner.
- Ensure that everyone on the global team has time to work together in person early in the project. This means flying people around the world which can be costly, but it makes a huge difference in the ability for team members to trust each other and work together.
- If the team is divided into drastically different time zones, schedule specific weeks throughout the project where the global team will work at the same time regardless of time zone. During these weeks, the team should make a concerted effort to communicate in real time with their counterparts around the world.
In a future post, I’ll talk about some additional techniques that are not specific to Agile that I’ve used to make global teams more effective.
Further Reading
Over the years, I’ve found these articles helpful in understanding the issues related to Agile distributed teams and how to address those issues. Maybe you’ll find them useful as well. To me, the two articles available for purchase from IEEE Computer Society were definitely worth the small cost.
Agile Strategies for Geographically Distributed Teams
Global Development and Delivery in Practice
Agile Offshore Techniques – A Case Study (Available for purchase)
Using and Agile Software Process with Offshore Development
Follow the Sun: Distributed Extreme Programming Development (Available for purchase)
Your Experiences
I’d like to hear any of your experiences with global Agile software development. What worked and what didn’t?
Contact me if you’d like help improving the effectiveness of your global team.
When the Firefighter is the Arsonist – Tales from the Trek
A number of years ago, I read a headline that said, “Ex-Forest Lake fire fighter pleads guilty to arson“. The story went on to say,
“Once considered a hero as a firefighter, John Berken was accused of starting a fire that burned 2,000 acres of nature area.”
This story got me thinking about a common pattern that I have seen on software development projects.
Heroic Measures
Many teams glorify the team members who take heroic actions to save a project. These team members, who work nights and weekends for weeks, or even months, at a time to pull a project success from the jaws of defeat, are held up as positive examples for the company and lavished with praise.
However, too often, the very same people who perform the heroic deeds are the people who created the near-disaster in the first place.
As a QA Sherpa, one of my top goals is to prevent the need for heroic actions at the end of a project. I feel happiest when we successfully implemented Embedded Quality and find no important bugs during the last few weeks before going live. In this role, I’m particularly attuned to those “heroes” who actually created the problems in the first place.
These are the people who dramatically underestimated their work, skipped unit testing, refused to work with the testing team to practice Embedded Quality, and generally eschew any form of teamwork or foresight. Despite all these negative traits, they end up being celebrated for “going above and beyond the call of duty”.
True Firefighters
True firefighters do need to be heroic at times, and it’s important that they are ready and able to be heroes when necessary. However, their bigger goal is to prevent fires in the first place. In addition to preventing fires, they work hard to put measures in place to limit the damage of fires when they do occur.
The bulk of a fire department’s efforts are not spent putting out fires. The bulk of the time is spent educating people about how to prevent fires, ensuring that the fire code is effective and that buildings follow the fire code by implementing fire walls, fire doors, sprinkler systems, alarms, and fire extinguishers.
Similarly, a solid development team should be spending their time implementing automated unit testing, working with the test team to implement Embedded Quality, performing peer code reviews, and getting feedback from key stakeholders throughout the project. All of these actions work both to reduce the odds of a disaster and to limit the damage when a disaster occurs.
As I said earlier, heroes are still needed at times, but you should take a second look at those heroes when the disasters are too frequent. If a city has way more fires than the norm, the fire department probably isn’t doing their job well. If every project requires heroic measures, it’s time to look a little closer at why that’s the case.
Feedback
I’m interested in hearing about your experiences with fire fighters who may also be arsonists. Have you worked on a project where team members who seemed to sabotage the project early on were held up as heroes at the end of the project? If so, do you think it was due to the individuals on the project or due to a broken system?
Setting Expectations – Lessons From a Little League Umpire
I’ve found through the years that projects go more smoothly when expectations are set at the beginning. Whether I’m in a Business Analyst or Quality Assurance role, I’ve found that the project goes more smoothly when the processes are laid out clearly up-front and the known limitations of the project are called out and addressed before the project starts. When people are not aware of the time that they need to dedicate to the project or the limits of the project scope until the project is well underway, they can get quite upset.
My most memorable lesson on the importance of setting expectations is still the one I learned while umpiring little league in high school.
Strike One!
I umpired my first game when I was 14 years old. The league was a city-run little league program, and some of the divisions had kids as old as me. Other than watching a lot of baseball, the only preparation the city gave me was handing me an armful of equipment and a copy of the rulebook. I read the rulebook several times and felt that I was as ready as I could be.
I set up the bases and put on my equipment for the first game. I said hi to the two managers and nervously took my spot behind the plate. My calls were a little shaky, but I was surviving. Surviving until the third batter, that is.
I lost my concentration on a pitch and uncertainly called, “Strike?” The batter’s manager yelled at me, “How can that be a strike?!? It hit the dirt before it even crossed the plate!”
I sheepishly said, “It did? I guess it was a ball then.”
Needless to say, the managers, players, and parents argued with me vehemently on nearly every call after that. I wasn’t sure I’d ever want umpire another game. However, the manager of the umpires was desperate for warm bodies to call the games and convinced me to finish out the season. I never had a game as bad as the first one, but every game was stressful and the managers, parents, and players argued with me regularly.
Year Two – Setting Expectations
In the next year, I read a book called “Strike Two” by former Major League umpire Ron Luciano. His stories about handling some of the toughest personalities in baseball gave me the insight and confidence that I needed to try umpiring again. The difference this year was that I was going to set some expectations before every game.
I met with both managers before every game and explained the following:
- I’m the only umpire, and I only see each play once. Don’t bother arguing any judgment call such as ball vs strike or safe vs out. I won’t change my call.
- I call a big strike zone. If any part of the ball is over any part of the plate from the top of the batter’s shoulders to the bottom of their knees, I’m calling it a strike.
- I’m not a professional umpire. I may make an error in the rules. If either manager believes that I made an error in the rules, let me know. I will call a time-out and meet with both managers to review the rule book and make an adjustment to the call if warranted.
Taking the time to set these expectations made a huge difference in my ability to manage the game. The managers would tell their players that I have a big strike zone and that arguing with me was pointless. The players swung a lot more, put the ball in play a lot more, and I got very few arguments.
In addition, after the games, parents would tell me that it was one of the best games they went to all year, especially for the younger age groups. Apparently, in many other games, the batters for each team would simply draw walks until they reached the 10-run mercy rule for the inning. This was boring for everyone involved. Because players were swinging more when I umpired, the ball was put into play more, and the kids and parents had a lot more fun.
Your Experiences
I’d like to hear about your experiences with setting expectations. Have you found that setting expectations up front helps your projects? Have you received any resistance to setting expectations at the start of a project?
Influencers – Jayme Edwards
There are a number of blogs that I enjoy reading about all areas of the software development process. The software development process includes project management, business analysis, development, and testing, of course. In addition, delivering software that works requires consideration of concepts around management, sales, and business organization.
I’d like to share some of the blogs that I find influential when thinking about how to build software that works.
Jayme Edwards – Consulting and Continuous Delivery Expert
Jayme Edwards is someone who I’ve known for a while. We’ve had many great discussions over the years about consulting and continuous delivery. I’ve gained a lot of insight from him directly and from his blog posts.
One blog post that recently resonated with me was titled Establishing Trust to Make IT Development Process Changes. His points about authenticity and setting realistic expectations up front are keys to a successful client/consultant relationship. In addition, he referenced Peter Block’s Book, Flawless Consulting which lays out three roles that a consultant can have when interacting with a customer: Expert, Pair of Hands, or Collaborator. I have been in all three roles, and although I enjoy them all, I especially enjoy working with a client in the Collaborator role.
Jayme Edwards’s is also incredibly knowledgeable in the area of Continuous Delivery. He is intimately familiar with the theory of Continuous Delivery. More importantly, I have seen him successfully move multiple clients into a successful Continuous Delivery model. Because Jayme Edwards has put the theory into practice many times, I find his posts on the topic to be particularly insightful and credible. If you’re interested in learning about Continuous Delivery, you should check out his posts.
Feedback
What do you think of Jayme Edwards’s blog and the concepts he writes about? What blogs and resources do you find particularly interesting and useful for building software that works?
The User Acceptance Testing Death Spiral
In a past role, I joined a test team that was in a User Acceptance Testing (UAT) “Death Spiral” that had caused the user base to lose confidence in the integration testing team. Based on conversations that I’ve had with others, I believe that the UAT Death Spiral is a common scenario that people encounter, and it can destroy a test team. It took some work, but we were able to pull out of the downward trajectory and regain a functional, productive partnership between the business folks and the integration testing team.
What is the User Acceptance Testing Death Spiral?
Our company had an understaffed integration testing team, aggressive deadlines, and a culture that valued meeting deadlines above all other goals. This meant that software often had known major bugs when UAT started. Even worse, there were typically areas of code that hadn’t been tested at all by the integration testing team before UAT started. It was common for the UAT team to find major, obvious bugs.
Because major bugs made it past the integration test team, the business felt a need to create a more formal, robust UAT team that would catch the numerous errors missed by the integration test team. The business folks assumed that the bugs were missed because the integration team wasn’t very good at their job.
As UAT became more robust, they realized that they needed more time to complete their testing. The company culture of tight deadlines meant that release dates could not be extended to accommodate in-depth UAT. Instead, the business insisted that the integration test cycle be shortened and that UAT start earlier.
I think you can see where this is going…
When UAT started earlier, they found even more bugs and the business lost even more confidence in the integration test team. The business then insisted on testing more of the functionality in UAT and starting UAT even earlier. This death spiral continued until the business had lost complete confidence in the integration test team. However, because the UAT team was not made up of experienced testers, the business was not finding all of the bugs either.
Basically, the integration team didn’t have time to do their job, the business was spending a huge amount of resources to test everything, even features already tested by the integration test team, and the quality of software in production was as poor as ever.
Breaking Free
Breaking free of this downward spiral was more than just a logistical problem. It was a political problem as well. The integration test team feared that the UAT team was trying to take over their jobs and the UAT team felt that the integration team wasn’t competent. Rebuilding trust was critical for any new process to be successful.
I worked with both the integration and UAT test teams to plan a new strategy. The strategy was that the integration test team would first test anything that did not have a user interface. In addition, the integration test team would write, maintain, and run automated regression tests. Basically, they would test the areas that required their expertise. Only after these areas were tested and any major bugs were fixed would the UAT team start their work. We would divide up the test cases to reduce any overlap of testing between the two teams as much as possible.
Even though the UAT team agreed that this plan made sense theoretically, they feared that removing the redundant testing would mean that bugs were missed and worried that starting UAT later would mean that they wouldn’t have time to complete their work. I convinced them to give the plan a try on a smaller project. If the advantages to the new plan didn’t materialize, it would be easier to adjust for the lost time on a small project.
Fortunately, on the small project, everything fell into place. The integration test team was able to adequately test their portion of the plan before UAT started, and the UAT team knew exactly what parts were and weren’t tested by the integration team. The UAT team had a shorter test cycle both because they didn’t run as many redundant tests and because the initial quality of code was much better. Each bug takes time to find, fix, and retest.
Best of all, the software went live and had no problems in production.
For the rest of my time with this team, we followed the new process. This resulted in higher quality code with lower cost, and it had the added benefit of greatly improving the working relationship between the two teams.
Your Experiences
I’d like to hear your experiences with User Acceptance Testing issues. Have you been in a situation where the business users lost confidence in the development team or integration test team? If so, what do you think were the root causes of the issues?