On projects that replace existing systems, project sponsors have often insisted that we didn’t need any business analysis because the existing system was “fully documented” in the form of the old system. I wrote about one problem with that mindset, which is that the new system will have all the issues that the old system had.
Another problem is that it takes a lot of time to figure out how the existing system work. The development team will end up taking time to analyze the existing system regardless of whether analysis time is in the budget or not. Contrary to what project sponsors often believe, users of the system don’t know all the details of how a system works, and code is not “self-documenting”.
Most complex systems have a variety of users. A relatively simple sales system has different users who place orders, manage orders, ship orders, manage the financials of the orders, report on orders, manage inventory, and handle returns. It’s time consuming for an analyst to research, understand, and record how all the parts of the system work from the users’ points of view.
Even after talking to all the users, an analyst won’t know the full functionality of the system. In my experience, it’s rare to find users who know the details of every calculation, decision, workflow, and automated interface the system has behind the scenes. In the few cases where I’ve found that a system was well documented, I’ve been told that the documentation was outdated, so I couldn’t trust it.
I’ve had project sponsors ask why the developers can’t just look at the code and rebuild the same system in a new technology. They assume that the rebuild is almost a copy and paste of the existing code. Although code examination is one of the techniques for understanding an existing system, developers are not able to quickly reverse engineer functionality from code in a complex system, even if the code is well organized. Any developer who’s made even a small change to a complex, unfamiliar codebase will tell you that understanding the code is time consuming.
Interviewing users, reviewing the exitisting user interface, and reviewing the existing code base are all important ways to understand the existing sytem. However, these actions clearly take time, and need to be budgeted for.
What challenges have you had getting budget approved for business analysis? What have you done to address those challenges? I’d love to hear your tales from the trek.
I’ve been on several projects where the goal is to replace an existing, mature system with a brand new system in a new technology. This is generally a huge investment in time and money, fraught with risk, and results in almost the same functionality. Still, it can be necessary due to the old software being dependent on unsupported or non-secure technology and exponentially increasing maintenance costs. In a way, it’s like making the decision to replace an old car rather than repair it.
On nearly every project of this type that I have experienced, the project sponsors have insisted that we could cut out, or at least drastically reduce, business analysis. Their reasoning was that because the new system simply had to work exactly like the old system, the new system was already “fully documented” in the form of the old system.
There are a number of reasons why assuming that the old system “fully documents” the new system is a mistake. I’ll address one of them here.
Often, software that’s running on outdated, unsupported technology has a lot of bugs and poor functionality that haven’t been addressed. Any changes to the system have become risky because the system is so fragile. If the only direction given to the development team is to rebuild the existing system as is with no additional analysis, the team will literally be building the same bugs and functionality into the new system. There’s no good way for the development team to know if something should be changed without at least some investment in business analysis.
The end result is that the users and project sponsors are even more frustrated with their investment in new software only to end up with the same problems. Even worse, sometimes, building the bugs into the new system costs more money. Imagine how much more it would cost to try to literally rebuild an old car exactly as it is rather than build a new car with the same basic functionality. It would take effort to make the brakes squeak just like the old car, have it shake at high speeds just like the old car, and randomly stall in the middle of intersections just like the old car.
The money spent rebuilding old bugs would be much better spent on some business analysis.
What challenges have you had with generating requirements when replacing existing systems? What actions have you tried to address the challenges? I’d love to hear your stories from the trek.
I’ve found through the years that projects go more smoothly when expectations are set at the beginning. Whether I’m in a Business Analyst or Quality Assurance role, I’ve found that the project goes more smoothly when the processes are laid out clearly up-front and the known limitations of the project are called out and addressed before the project starts. When people are not aware of the time that they need to dedicate to the project or the limits of the project scope until the project is well underway, they can get quite upset.
My most memorable lesson on the importance of setting expectations is still the one I learned while umpiring little league in high school.
I umpired my first game when I was 14 years old. The league was a city-run little league program, and some of the divisions had kids as old as me. Other than watching a lot of baseball, the only preparation the city gave me was handing me an armful of equipment and a copy of the rulebook. I read the rulebook several times and felt that I was as ready as I could be.
I set up the bases and put on my equipment for the first game. I said hi to the two managers and nervously took my spot behind the plate. My calls were a little shaky, but I was surviving. Surviving until the third batter, that is.
I lost my concentration on a pitch and uncertainly called, “Strike?” The batter’s manager yelled at me, “How can that be a strike?!? It hit the dirt before it even crossed the plate!”
I sheepishly said, “It did? I guess it was a ball then.”
Needless to say, the managers, players, and parents argued with me vehemently on nearly every call after that. I wasn’t sure I’d ever want umpire another game. However, the manager of the umpires was desperate for warm bodies to call the games and convinced me to finish out the season. I never had a game as bad as the first one, but every game was stressful and the managers, parents, and players argued with me regularly.
In the next year, I read a book called “Strike Two” by former Major League umpire Ron Luciano. His stories about handling some of the toughest personalities in baseball gave me the insight and confidence that I needed to try umpiring again. The difference this year was that I was going to set some expectations before every game.
I met with both managers before every game and explained the following:
Taking the time to set these expectations made a huge difference in my ability to manage the game. The managers would tell their players that I have a big strike zone and that arguing with me was pointless. The players swung a lot more, put the ball in play a lot more, and I got very few arguments.
In addition, after the games, parents would tell me that it was one of the best games they went to all year, especially for the younger age groups. Apparently, in many other games, the batters for each team would simply draw walks until they reached the 10-run mercy rule for the inning. This was boring for everyone involved. Because players were swinging more when I umpired, the ball was put into play more, and the kids and parents had a lot more fun.
I’d like to hear about your experiences with setting expectations. Have you found that setting expectations up front helps your projects? Have you received any resistance to setting expectations at the start of a project?
Throughout the years, I’ve led many user acceptance testing (UAT) efforts. There are a couple of common ways to plan the testing that I’ve found to be problematic. The methods allow someone to say they’ve planned UAT but add very little value. The end result is that unhappy users report problems after the software is live, and the project manager asks why they didn’t find the problems during UAT.
After years of experimentation, I found a method that adds a lot of value to the testing process with less effort than the other methods.
I’ve been involved with quite a few projects where the client wanted the users of the software to write the test scripts for UAT. This was often done to reduce development costs. The folks planning the project believed that they would save money by having business employees write the test scripts for UAT rather than have someone from the development team work on them. They also believed, often correctly, that the business folks understood the goals of the system best, so they were the most qualified to write the UAT scripts.
The problem with this is that most business users have more than a full time job already. In addition, most business users have never planned a testing effort or have written a test script. It’s difficult enough for business users to find time to execute tests, but nearly impossible for them to find time to plan testing. Even if they do find the time to plan it, their lack of experience results in test scripts that test very little.
On other projects, I’ve been asked to write test scripts that the users can run during UAT. Usually, I’m asked for the same scripts that the test team ran during system testing. Under this scenario, it’s unlikely that the users will find any issues that the test team didn’t already find. This method doesn’t uncover functional issues that the test team missed. In addition, I’ve found that the business users rarely take the time to follow the test script closely and end up not doing much testing at all.
The UAT planning method that I’ve found adds the most value with the least effort is Guided Functional Checklists. This method is quite simple:
That’s all there is to the planning. The result is an organized list of what should be tested based on the business goals of the users rather than based on the development team’s view of the project. Because this takes little of the users’ time, they are able to make time to complete the planning. By not having a detailed script, users are able to ensure they can meet their business goals in the way that they will in the field, which is often a bit different than the development team expects. This allows everyone to identify programming or training issues before going live.
What UAT planning challenges have you faced? How did you address the challenges? I’d love to hear your tales from the trek.