Just because you’re only changing some SQL around doesn’t mean that you can take shortcuts with testing. You’ll want to run an entire stem-to-stern test of your PeopleSoft system to insure that everything works as expected.
One thing to keep in mind: 99% of your defects will be with custom and customized code. The delivered code won’t generate nearly as many problems, so if you’re deciding where to spend your testing resources definitely focus on the custom processes.
As I mentioned in the Build phase, Unit Testing is critical. It’s arguably the most important testing that you will do so come up with a mechanism to track unit test results with the objects modified and make sure you have 100% custom code coverage.
Testing the data migration process is important too. Databases differ in subtle ways and you’ll want to make sure your normal and extended character sets make it across correctly. Naturally you’ll want to run row count reports, but you’ll need to go beyond that. You’ll want to make sure the data in the underlying tables are identical and nothing has been lost in translation. One simple way is to use ODBC to join to tables in the source and target databases and create queries that insure the individual columns are identical. Unfortunately that approach is extremely slow. Another approach is to run hash algorithms on key character fields and summarize the totals, comparing the results on both the source and target database.
System/Integration Testing is also very important. For one thing, you’ll want to have confidence that the system behaves the way you expect it to. And interfaces will generate problems themselves. One common problem is that your interface programs probably assume the default date format for a database platform, and interfaces that don’t specify a date format can choke when the incoming date format doesn’t match what’s expected, or they can send the wrong date format to an output file. Switching from a Non-Unicode database platform to Unicode can cause other problems. You’ll want to execute all of your interfaces and make sure results are valid.
User Acceptance Testing is important as well. Users who know the processes should spend some time in the system making sure all of the critical functionality works and they feel comfortable everything is working as expected. They’ll need to be on the lookout for date format issues, performance issues, data entry discrepancies, etc. They should also spend quality time with their reporting to make sure no new errors were introduced during the build phase.
And finally Performance Testing should be conducted to flesh out any new problems introduced by the new platform. The performance testing should include Online Performance testing, Batch performance testing, and if possible data from PeopleSoft Performance Monitor should be captured and compared to baseline performance data from the old system.
Online performance testing is typically conducted using a tool like LoadRunner or Rational Performance Tester. You record scripts based on a set of highly used business processes and play them back in volume. While the test is executing your admin team will monitor system performance on the various servers and look for bottlenecks. At the end of the day this is necessary for a performance testing effort (especially if you’re making multiple changes like migrating to new hardware and/or a PeopleTools release). However, it’s not sufficient to identify all performance issues.
One of the big deficiencies of online performance testing is that it doesn’t test batch jobs, or if it does test them it is very limited. For batch testing, you’ll want to define a testing mechanism that is realistic according to real, observed jobs that run in your production environment. You’ll want to make sure that the jobs have real data to process. And you’ll want to make sure the jobs are sequenced in such a way that dependencies are tracked to some extent. After all of that, you’ll want to come up with a way to track and report results. I’ll write more about the approach and toolset I’ve used to get realistic batch tests in the future so stay tuned.
The other deficiency of online performance testing is that PeopleSoft is just too complicated of an application to expect a robot to crawl all of the web pages looking for slowness. Sure, during your System and User Acceptance testing phases users will have manually gone through all of the important online pages, but if they didn’t report performance problems it’s likely performance problems will end up in Production. That’s another place where having PeopleSoft Performance Monitor running in the background during testing can be helpful. You can run reports against the various SQL and PeopleCode that gets executed during testing and compare them to your baseline. If certain items are too much slower, you can log a defect and fix them.