What a weekend.  Not necessarily in a good way.  We finished testing Maintenance Packs 6, 7 and bundles 16 and 17 on Friday and according to the project schedule we had this weekend to get it migrated to the other six PeopleSoft environments so it would be ready to go for User Acceptance Testing.  Since I originally applied the maintenance I was volunteered to migrate it everywhere.

I’m not crazy about applying maintenance this late into the project, but we found a critical issue a few weeks before beginning User Acceptance Testing that was fixed in Bundle #16.  Just to get us to that point required Maintenance Packs 6 and 7.  Then Oracle released Bundle #17 which fixed another important error, so we decided to throw it in.

So we put the maintenance into our Demo environment, and let the users make sure it fixed the problems they were interested in.  Once we had their blessing, we created a new environment from one of the environments that had converted data, configuration, and transactions.  Then we ran compare reports and applied the maintenance packs and bundles to this new one.

I tested what I could, but to really test you need people with knowledge of how the organization is using the functionality.  We had our functional team leads hit it for a few days to make sure the bugs were still fixed and the big major processes work.  Once we had that assurance, we scheduled this weekend’s work.

I always learn a lot when I go through one of these processes, and having been through it a few times I thought I’d share some of my observations.

Keep good notes as you apply the maintenance pack to demo, then again as you apply it to your testing environment.  Record when scripts bomb out, and what the fixes are.   Record things about manual steps like which are necessary and which ones aren’t.  This will help tremendously when it’s time to push it out to all of the environments.

Before pushing maintenance into an environment, run a compare report and review it to make sure nothing is different or requires special handling.

Always run a DDDAUDIT and SYSAUDIT before applying maintenance in an environment.  You don’t necessarily need to fix everything, but you’ll want to know what issues the maintenance caused and what was there before.

Set up the Environment Management Framework beforehand.  Be sure each DB has a unique GUID, be sure your PSEMAgents start, your hub works, and if you go through the Apply Change Packages in Change Assistant you can see all of your environments you need to work with.  I was wishing I had spent more time on this before the weekend, things would have gone quicker.

Work from Home.  I don’t know about you, but there’s no way I’d want to put in a 20+ hour weekend in the office.  Weekends are no fun at the office.  There’s nobody to talk to.  Food is hard to get.  Lights and air conditioning aren’t always on, and tend to go off every couple of hours.  And let’s face it:  Sometimes scripts are running and there’s not much to do for 15-30 minutes at a time.   If you’re home you can raid the fridge or get reacquainted with your family.

Plan to run the upgrade on a remote server (or servers) at work.  Get the correct access and install Change Assistant in advance.  Your VPN connection or your broadband connection will at some point let you down, and/or your broadband network won’t be fast enough to open large projects and build them.  Insist on this, even if you have to work in the office.  Servers located near your database in the server room just perform better than your workstation will on the LAN, and if your workstation locks up your upgrade will keep on going.

To use all of the functionality of Change Assistant, I should have applied the maintenance using the “Apply with Database Compare/Copy” option between Demo and our MP Test environment.  This would have run the compare report for us and after we reapplied customizations it would have re-packaged the update with the new customizations.  I may try this approach some day, but with 2 maintenance packs, 2 bundles, and 7 other prerequisite fixes that seemed like too many unnecessary extra steps.

I did use the “Apply with Database Compare/Copy” to migrate from our MP Test environment to the other 6 environments.  I just had to mark all of the steps as done in the “Apply Project” chapter except “Copy project from source db”.  This way we migrated the project from the environment where the customizations had been reapplied.  Also, database copies are faster than copying projects from files.

Personally, I don't let Change Assistant deploy files for me.  Not that there's anything wrong with that, but it's SLOW!  I create a subdirectory with folders for CRW, SQR, CBL, and carefully copy the objects from the filereferences directory into my new folders, paying careful attention about the order of the bundles I copy.  In Change Assistant, I leave the "Deploy Files" chapter but delete the individual steps like Deploy CRW, Deploy CBL.  I tried to delete Deploy Files, but afterwards on Tools 8.48.08 the job kept getting hung up after the last step and wouldn't continue to the next job.  

Don’t depend on the DBA to run your scripts.  Let change assistant do it.  I know the DBA’s mean well, and they rightly like to know what DDL is running on their system.  But when you’re working in a short window on the weekend you have to bend the rules.  You can’t depend on anyone to sit at their computer on the weekend waiting for a set of scripts to arrive.  And even if they do, they’ll have to modify the script to write log files and exit when errors happen, and review the scripts when they’re done for errors.   Then they’ll have to notify you that they’re done, and you might not be at the computer either.  Change Assistant writes log files, terminates the script if there are errors, and scans the log file for errors before continuing.  No errors, things keep going.  And CA can be configured to send you an e-mail if something bad happens.  This eliminates latency.  Make sure the DBA is available though.  Change Assistant can’t fix database problems like when a tablespace fills up or a rollback segment is too small.

Don’t try to do other work while change assistant is running.  Applying a bunch of maintenance doesn’t always require tons of concentration, but when a script bombs out you need to be relatively fresh to troubleshoot it and move on.  If your mind is still working on another unrelated problem, switching gears and fixing the maintenance pack problem could slow you down.

Run an “Alter Audit” in each environment after the maintenance is complete.  It’ll usually pick up changes.  In our case, PeopleSoft modified several field sizes which impacted records that weren’t in the maintenance projects.  Building an alter script on all tables in the database resolves these discrepancies. 

Run the VERSION app engine program after you’re done.  It doesn’t take long, you can launch it from App Designer, and if there are any version problems left over by the upgrade (shouldn’t happen but sometimes it does) it’ll fix it.

Shut down the App Server and Process Scheduler before you begin, and clear cache before you bring them back up.

Run Portal Security Sync once everything comes back up but before anyone logs in.  This will clean up the portal security tables.

What did I miss? 

Comments (0)

Write comment

Name

Email

Website

Title

Comment

Write the displayed characters

Add Comment