Remote login via rman working for one instance but not another

Question

I have Oracle 12c installed on two servers. On server UAT I have an instance RMCAT that hosts an RMAN recovery catalog. On server EXT I have two instances, extdev02 and extuat01. I can use the following command on server UAT to connect to my RMCAT catalog, and target of extdev02, but when I try to connect to extuat01 it gives an ORA-01017

UAT> rman target= sys/tiger@extdev02  catalog= rmuser/tigerman@rmcat

Recovery Manager: Release 12.1.0.2.0 - Production on Wed May 30 22:59:26 2018

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.

connected to target database: EXTDEV02 (DBID=4287636681)
connected to recovery catalog database

RMAN> exit


Recovery Manager complete.

UAT> rman target= sys/tiger@extuat01  catalog= rmuser/tigerman@rmcat

Recovery Manager: Release 12.1.0.2.0 - Production on Wed May 30 22:59:42 2018

Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-00554: initialization of internal recovery manager package failed
RMAN-04005: error from target database:
ORA-01017: invalid username/password; logon denied

I get OK results from tnsping from UAT server to both instances on EXT, and I can connect to a non-dba user from UAT server to both instances. I recreated the Oracle password file using orapw on EXT server instance EXTUAT01 and that did not help.

Instance names are extdev02 and extuat01 on server EXT. Here are my password files on that server:

/u01/app/oracle/product/12.1.0.2/db_1/dbs> ls -o orapw*
-rw-r-----. 1 oracle 18432 Jun  1 00:07 orapwextuat01
/u01/app/oracle/product/12.1.0.2/db_0/dbs> ls -o orapw*
-rw-r-----. 1 oracle  7680 Feb  6 20:22 orapwextdev02

Note they are in two different ORACLE_HOME directories. The value for remote_login_passwordfile is EXCLUSIVE for both instances

Resolved Problem

The server UAT hosting the RMAN catalog instance RMCAT was rebooted a few days ago to add memory, and now I can connect! If I had a wayback machine, I would go back and see if trying to restart the listener on UAT host would have helped; in any case, it works. Thanks for the suggestions; I am putting in the answers just for posterity.

Comments Answered

• Not a Grid nor RAC nor Dataguard instance environment.
• selecting from v$pwfile_users:

extdev02:

AtlasExt:SYS@extdev02> select * from v$pwfile_users;

USERNAME  SYSDBA  SYSOPER SYSASM  SYSBACKUP  SYSDG   SYSKM   CON_ID
--------- ------- ------- ------- ---------- ------- ------  ------
SYS       TRUE    TRUE    FALSE   FALSE      FALSE   FALSE   0

extuat01 (note: I ran orapw specifying sysbackup to no avail)

AtlasExt:SYS@extuat01> select * from v$pwfile_users;

USERNAME  SYSDBA  SYSOPER SYSASM  SYSBACKUP  SYSDG  SYSKM    CON_ID
--------- ------- ------- ------- ---------- ------ -------- ------
SYS       TRUE    TRUE    FALSE   FALSE      FALSE  FALSE    0
SYSBACKUP FALSE   FALSE   FALSE   TRUE       FALSE  FALSE    0

• I was not able to connect from UAT to EXT instance extuat01 via SQL*Plus; but after the reboot I was:

extdev02 from UAT:

AtlasUAT:SYS@extdev02> select host_name, instance_name from v$instance;

HOST_NAME                 INSTANCE_NAME
------------------------- ----------------
plsekatlasexta01          extdev02

extuat01 from UAT:

18:25:07 AtlasUAT:SYS@extuat01> select host_name, instance_name from v$instance;

HOST_NAME                 INSTANCE_NAME
------------------------- ----------------
plsekatlasexta01          extuat01

• Verified extuat01 and extdev01 processes were running out of the same home:

Found pmon PID:

plsekatlasexta01> ps -fu oracle  | grep pmon | grep -v grep
oracle    1976     1  0 Jun01 ?        00:00:43 ora_pmon_extuat01
oracle   32373     1  0 May16 ?        00:02:45 ora_pmon_extdev02
plsekatlasexta01> cat /proc/1976/environ | tr '\000' '\n' | grep -E 'ORACLE_(HOME|SID)'
ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/db_1
ORACLE_SID=extuat01
plsekatlasexta01> cat /proc/32373/environ | tr '\000' '\n' | grep -E 'ORACLE_(HOME|SID)'
ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/db_0
ORACLE_SID=extdev02

Answer 1

The server UAT hosting the RMAN catalog instance RMCAT was rebooted a few days ago to add memory, and now I can connect to both database instances on EXT server.

I do not remember whether I tried reloading / restarting the TNS listener on the UAT host when it was not working, but now that the server was rebooted it is now working; I assume the reboot got the listener working. Thanks to all for the helpful comments.