Litmus Horizons

Practical insights on AI, Oracle Cloud, and enterprise architecture.

Orphaned vServers at EMOC? No problem…

Orphaned vServers at EMOC? No problem… (title card)

About 4 years ago (2014) I faced this issue, a mistreated mishandled Exalogic suffered a Repo Corruption, thus a restore was in order but ghosts were lurking around …
If you faced this issue before, you know what I mean… but if you don’t, check the explanation.

Case Explanation

The affected Exalogic contains existing guest vServers, and the Exalogic Control repo gets corrupted.
Well, it gets even worse… your only “good” backup is the one that was taken right after Oracle ACS installed the rack (months ago, thus empty)… so there is no definition of any guest vServer in it… at least not at the vDC Accounts.
Restoring that “good backup” is your only option to at least restore EMOC functionality. This old backup is good but lacks of all the guest vServer definitions and details that were created after the good backup was taken (not so good, right?).
EMOC shows up, yay ! … not so much… all pre-existing guest vServers are gone from all the vDC Accounts… in other words, EMOC doesn’t see them at the vDC Account, only shows them as ORPHAN under the EMOC Assets accordion.
OVMM Console also shows them as ORPHAN vServers, and after a quick rediscovery, OVMM Console shows the guest vServers with the proper logical names (such a nice guy ah!). Though EMOC never gets the memo and thus the guest vServers never show up at the vDC Account in EMOC. We are stuck with EMOC not being able to admin the ORPHAN guest vServers.

OVMM Console Repo Refresh

Solution

The assumption here is that you have guest vServers on your Accounts and that you want/need to keep them.
I think that at this point our only bet is to use this exaBR Doc:
Restoring User vServers Created Using EECS 2.0.6 Guest Base Template 

And pretend that the orphaned vServers are corrupted (they are not), thus re-creating from the standard template (or whichever template it was used to create them) and restoring the same network vIPs/vNets/Routes.

Disclaimer: use at your own risk, I have tested it and it worked in my case, but there could be minimal differences that can make the solution not to work for you. Feel free to post your comments.

Action Plan for each orphaned vServer:
Preparation:

  1. Make sure to have a previous itemized vServer backup for each orphan vServer available using exaBR.
  2. Collect the /etc/sysconfig/network-scripts including routes.
  3. Collect from ECU the vNets/vIPs assigned to each orphan vServer.
  4. From the existing vm.cfg files extract the simple name and other configuration details. ie: @cn01 # find /OVS -wholename ‘*VirtualMachines/*/vm.cfg’ -exec grep -H ‘simple_’ {} \;

Execution:

  1. If the orphan vServers are up, then stop all of them.
  2. Re-Create the vServer as instructed at the exaBR Doc step: 5.2.1.4, a) through d).
  3. Backup the new /etc/sysconfig/network-script this will be restored later.
  4. Restore the vServer using exaBR as instructed at the exaBR Doc step: 5.2.1.5, ie: @cn01 #./exabr restore 192.168.1.2 -b 201309121315
  5. Backup (tar) /etc/sysconfig/network-script and static routes “route-*” for later use.
  6. Remove the new /etc/sysconfig/network-script from the restored vServer.
  7. Restore /etc/sysconfig/network-script from Execution item #3 and add the “route-*” file from Execution item #5
  8. Stop/Start the new vServer from EMOC.
  9. Check the “ifconfig |grep inet” to match the collected data from Preparation item #3
  10. Check the routes to match the collected data from Execution item #5
  11. Repeat for all Orphan guest vServers.

This should bring EMOC up to date with the existing (orphaned) Guest vServers showing up at each vDC Account.

IMPORTANT:
Take a new baseline control-stack backup using exaBR, and remember to take backups before and after significant modifications/changes, such as fixes, patches, and Oracle Support Action Plans.

Sergio

Comments

Leave a Reply