PeopleSoft – PUM – Failure to Start

During the last Windows Update on the PUM server the Virtual Box Image failed to start correctly. In the Logs you will see:

Starting PeopleSoft Application Server Domain APPDOM: [FAILED]

and/or

Starting PeopleSoft PIA Domain peoplesoft: [FAILED]

This environment had been running for sometime prior to this, and so doing a little investigate it turned up that there was no drive space left on the associated “/” root drive.

The quickest way to resolve this is to log into the PUM environment as root, and cleanup the files in the application server and process scheduler directories.  We have found that there is “core” files that are generated every time the PUM server is improperly shutdown and typically these files are very large.  Removing them is often the quickest way to get space back.  Look at the following directory locations:

Remove core dump files (core.*), trace files, log files and caches files from the following directories:

/
/home/psadm2/psft/pt/8.5x/appserv/APPDOM
/home/psadm2/psft/pt/8.5x/appserv/prcs/PRCSDOM/CACHE
/home/psadm2/psft/pt/8.5x/webserv/peoplesoft/applications/peoplesoft/PORTAL.war/ps/cache
/opt/oracle/psft/db/oracle-server/diag/rdbms/<db-name>/<db-name-uppercase>/trace

Once complete re-check the drive space availability and you should see a significant amount of space available again which will allow log writing to occur again.

During the last upgrade pass I was attempting to apply patches to the upgraded environment and ran into an unusual error when trying to build the change package within Change Assistant.

As it turns out when we build the image, we did not force it to always use the same IP address.  So every time it starts the IP address has a tendency to change.  As it turns out this can cause some problems with the configuration that was done before on previous passes.  Depending on your setup you may find that the Oracle Database Listening will not work, which in turn makes it unavailable for the application server and process scheduler and the database is not listening to requests.  So you will need to change the listening and tnsnames files, as well as the tnsnames file on the server where change assistant is running.

However, the error with building the change packages was caused because the mapped drive for the “pi_home” on the virtual image was mapped as a network drive using the IP address.  Once the mapping was changed and the pi_home was available the change package error was eliminated.

PeopleSoft has introduced new functionality in 8.53 and the 9.2 applications called the Data Migration Workbench. This functionality allows for easier migration of data which is directly related to the use of the new PeopleSoft Upgrade Manager (PUM). There has always been issues try to migrate “non” PeopleSoft meta-data objects from environment to environment. So items like data mover scripts and SQR’s always had to be manually migrated.

Part of the Data Migration Workbench is Application Data Sets (ADS). These data sets can be compared and copied, however, there is a whole new security element just for ADS. PeopleSoft delivered the permission list PTPT3500, which ties to the role ADS Designer. Make sure that your Upgrade user has access to this role/permission. Also make sure that the ADS permissions are turned on. Navigate to PeopleTools->Security->Permissions&Roles->Permission Lists, select the PTPT3500 permission, and go to the new Data Migration tab. Add in any access group permissions and under the copy compare permissions make sure it is set to Full Access.

During the application patching I was doing, the PTADS{BUG#} compare step was failing.

PeopleTools 8.53.06 – Application Engine
Copyright (c) 1988-2013 Oracle and/or its affiliates.
All Rights Reserved
Begin Application Engine Process (257,401)
Compare project from repository/area/project: USKANWD21374/Area/PTADS16299507 (257,400)
Message Set Number: 257
Message Number: 400
Message Reason: Compare project from repository/area/project: USKANWD21374/Area/PTADS16299507 (257,400)
User does not have permission to perform Compare from File (257,308)
Message Set Number: 257
Message Number: 308
Message Reason: User does not have permission to perform Compare from File (257,308)
Load from file failed (257,298)
Message Set Number: 257
Message Number: 298
Message Reason: Load from file failed (257,298)
End Application Engine Process (257,402)
Application Engine program PTADSAEPRCS ended normally

After turning on this new security, I ran it again and received the same error. Just to ensure that I was on the right track, I went and deleted the 2-Tier cache (usually c:\ps\{dbname}), and re-ran and everything worked!