Okay, this is a little embarrassing but it points out an important concept.  Sometimes reviewing the basics can save you a lot of time and frustration.

During my last go-live weekend we cut over to a whole new infrastructure for the new HCM 9.2 environment.  This included an external web server for the e-recruiting module that is in use.  The new web server was configured and up and running and the SSL certificates were in place and for testing purposes I put an entry in the hosts file to verify everything worked.  On go-live we had to change the external DNS and NATs for the new servers so that it could work in the DMZ.  According to the network guys everything was done and the only thing they could tell me was that my Weblogic Web Server was not running or not accepting requests.

Now clearly, I was on the actual server and had tested everything so, I know that the Weblogic was working and it was communicating to the application servers.  I could RDC to the server and even map UNC paths to the other servers to get files to and from the old servers.  Clearly communications were working.  It just simply wouldn’t allow http and https traffic.

Turns out that this server was created from a completely different VM template than all my other servers.  When I looked closely I found that the Windows Firewall was turned on and was blocking the web traffic.   Unfortunately this was after about 5 hours of having the network guys looking into the problem.  Now they had to do some work but the problem clearly could have been resolved VERY quickly if I had just thought to look at the settings on the server.  Oh Windows Firewall how you frustrate me.

Okay, Oracle Secure Enterprise Search has a lot of moving parts when integrated with PeopleSoft. I have posted a couple of notes on it already but I just spent several hours troubleshoot O-SES where the diagnostics check would ping, deploy, crawl, and search, but it was not returning a search result. The logs showed that there was a routing issue, and the O-SES logs showed a couple of different errors, and when looking on my Oracle I found some solutions but after chasing my tail for several hours, I thought, maybe I should just go back to the beginning and double check my routings.

Sure enough I found one I missed. Make sure that all the routings under the services: ADMINSERVICE, ORACLESEARCHSERVICE, PT_SES, PTFP_FEED, PTSF_SECURITY, PTSF_SES_FEED, PTSF_META_DATA are active. I have attached a SQL script that you can run which shows many of the key elements that you are checking for.

SELECT A.IB_SERVICENAME, A.IB_OPERATIONNAME, A.DEFAULTVER, A.RTNGTYPE, B.PARAMETERNAME, B.MSGNAME, C.ACTIVE_FLAG,
(SELECT COUNT(*) FROM PSAUTHWS S1 WHERE S1.IB_OPERATIONNAME = A.IB_OPERATIONNAME) AS “PERMISSION COUNT”,
B.QUEUENAME, D.ROUTINGDEFNNAME, D.SENDERNODENAME, D.RECEIVERNODENAME, D.EFF_STATUS AS “ROUTE STATUS”,
E.PARAMETERNAME, E.INMSGNAME, E.INMSGVERSION, E.XFRMNAME , E.OUTMSGNAME , E.OUTMSGVERSION , E.ALIASNAME
FROM PSOPERATION A, PSOPRVERDFNPARM B, PSOPRVERDFN C, PSIBRTNGDEFN D, PSRTNGDFNPARM E
WHERE A.IB_OPERATIONNAME = B.IB_OPERATIONNAME
AND A.DEFAULTVER = B.VERSIONNAME
AND A.IB_OPERATIONNAME = C.IB_OPERATIONNAME
AND A.DEFAULTVER = C.VERSIONNAME
AND A.IB_OPERATIONNAME = D.IB_OPERATIONNAME
AND A.DEFAULTVER = D.VERSIONNAME
AND D.ROUTINGDEFNNAME = E.ROUTINGDEFNNAME
AND A.IB_SERVICENAME IN (‘ADMINSERVICE’, ‘ORACLESEARCHSERVICE’, ‘PT_SES’, ‘PTFP_FEED’, ‘PTSF_SECURITY’, ‘PTSF_SES_FEED’, ‘PTSF_META_DATA’)
ORDER BY A.RTNGTYPE, C.ACTIVE_FLAG, D.EFF_STATUS, A.IB_OPERATIONNAME

When trying to get Oracle Secure Enterprise Search working on an environment there happens to be several places that call the search, one is on the classic menu which some people are still using. However, the search often will return an error using the O-SES.

An error has occurred while performing search, please contact your administrator.

In order to fix this you need to make a call to a different iscript: Navigate to PeopleTools -> Portal -> Structure and Content, select “Portal Objects”, then “Pagelets”, then “PeopleSoft Applications”. You should see a cref link for “Menu – Classic”, click on edit and change the iScript Parameter from ISCRIPT2 to ISCRIPT1. Save, Logout and try again! You might have to bounce the application server to make this fix work, but in the case I was working on, it worked without a bounce.

Just ran into a crazy little problem trying to get Oracle Secure Enterprise Search working at a client site that was using a self-signed internal SSL certificate.

In order to get Firefox to accept the SSL connection I had to add the Trusted CA (The Clients CA Certificate) to the Firefox keystore.  I had tried to load the certificates into the Windows keystore but apparently Firefox has its own.

Next When trying to deploy the search index I was receiving the following error:

Service Exception: ns2:AdminAPIRuntimeFault : EQA-15011: The plug-in manager raised an unexpected error. (262,1018) PT_SEARCH.SESIMPL.MESSAGE.AdminResponse.OnExecute Name:AdminResponse PCPC:1452 Statement:20
Called from:PT_SEARCH.SESIMPL.AdminService.OnExecute Name:doService Statement:848
Called from:PT_SEARCH.SESIMPL.AdminService.OnExecute Name:createAll Statement:794
Called from:PT_SEARCH.SESIMPL.AdminService.OnExecute Name:AddPSFTSources Statement:106
Called from:PTSF_DP_SBO_WRK.PTSF_DEPLOY_BTN.FieldChange Statement:139

I found a case on my oracle support: E-SES 11.2.2.2: Deploy Step Fails when Using SSL-Enabled PeopleSoft Integration Gateway: Error “EQA-15011: The plug-in manager raised an unexpected error” (Doc ID 1600736.1), which basically says that the cacerts keystore for the O-SES does not contain the trusted root identity for the SSL Certificate that is being used on the PeopleSoft System. So export the the Root and Intermediate certificates to a PEM format and import them into the cacerts keystore. There turns out to be many cacerts in the oracle directories, and I updated all of them with the new trusted certificates.

Lastly, I bounced the O-SES installation using the searchctl restart command.

Here is a handy little script to figure out what permissions have been granted to a user in SQL Server. For example the connect id user for PeopleSoft needs to have select access to the PSACCESSPRFL, PSOPRDEFN and PSSTATUS tables.  The default connect id is ‘people’.

select p.name, p.type_desc, pp.permission_name, pp.state_desc, pp.class_desc, object_name(pp.major_id)
from sys.database_principals p left join sys.database_permissions pp on pp.grantee_principal_id = p.principal_id
where p.name = 'people'