Windows Clustering Failover Issue


environment -

windows 2008r2 enterprise host machine a

windows 2008r2 enterprise host machine b

hyper - v virtual machine - windows 2008 standard (this vm on cluster high availability)

i have working environment 1 nic added virtual machine. there no issue when failover clustered virtual machine 1 host host machine.

after added 2nd nic virtual machine settings.

failover primary secondary node fails on cluster @ 98% generating 2 event id 1069 , 1205.

issue because of new network card after add virtual cannot figure causing.

if remove network card failover works again.

i keep both nic under same name on both host , host  b machine.

cluster log during failover

00000fa0.0000166c::2017/03/17-02:45:52.893 err   [rhs] s_rhsrpccreaterestype: error_not_ready(21)' because of 'startup routine restype msmq returned 21.'
00001670.0000194c::2017/03/17-02:45:52.893 warn  [rcm] failed load restype 'msmq': error 21.
00001670.0000194c::2017/03/17-02:45:52.893 warn  [rcm] rcm::rcmapi::restypecontrol: restype msmq's dll not present on node.  attempting find node...
00000fa0.0000166c::2017/03/17-02:45:52.908 err   [rhs] s_rhsrpccreaterestype: error_not_ready(21)' because of 'startup routine restype msmq returned 21.'
00001670.00001bac::2017/03/17-02:45:52.908 warn  [rcm] failed load restype 'msmq': error 21.
00000fa0.0000166c::2017/03/17-02:45:53.205 err   [rhs] s_rhsrpccreaterestype: error_not_ready(21)' because of 'startup routine restype msmqtriggers returned 21.'
00001670.0000194c::2017/03/17-02:45:53.205 warn  [rcm] failed load restype 'msmqtriggers': error 21.
00001670.0000194c::2017/03/17-02:45:53.205 warn  [rcm] rcm::rcmapi::restypecontrol: restype msmqtriggers's dll not present on node.  attempting find node...
00000fa0.0000166c::2017/03/17-02:45:53.205 err   [rhs] s_rhsrpccreaterestype: error_not_ready(21)' because of 'startup routine restype msmqtriggers returned 21.'
00001670.00001bac::2017/03/17-02:45:53.205 warn  [rcm] failed load restype 'msmqtriggers': error 21.
00000fa0.0000166c::2017/03/17-02:46:29.365 info  [res] virtual machine <virtual machine victests01>: current state 'online', event 'startlm'
00000fa0.00000c30::2017/03/17-02:46:29.506 info  [res] virtual machine <virtual machine victests01>: state change 'online' -> 'migrationsourcesetup'
00001670.00001974::2017/03/17-02:46:29.506 info  [gum] node 1: processing requestlock 2:21
00001670.00001974::2017/03/17-02:46:29.506 info  [gum] node 1: processing grantlock 2 (sent 1 gumid: 1636)
00001670.00001dfc::2017/03/17-02:46:30.801 warn  [rcm] resourcecontrol(unknown (0x160000c) obj: .mu. flags:2097164 code:0 access:<<insert {4} refers argument not supplied. 4 argument(s) provided>>) virtual machine victests01 returned 997.
00001670.00001dfc::2017/03/17-02:46:30.816 info  [gum] node 1: processing requestlock 1:1184
00001670.00001974::2017/03/17-02:46:30.816 info  [gum] node 1: processing grantlock 1 (sent 2 gumid: 1638)
00001670.00001dfc::2017/03/17-02:46:30.910 info  [rcm] rcm::rcmgum::setgrouppreferredowners(victests01,<vector len='2'>
00001670.00001dfc::2017/03/17-02:46:30.910 info      <item>2</item>
00001670.00001dfc::2017/03/17-02:46:30.910 info      <item>1</item>
00001670.00001dfc::2017/03/17-02:46:30.910 info  </vector>
00001670.00001dfc::2017/03/17-02:46:30.910 info  )
00001670.00001974::2017/03/17-02:46:31.237 info  [gum] node 1: processing requestlock 2:23
00001670.00001974::2017/03/17-02:46:31.237 info  [gum] node 1: processing grantlock 2 (sent 1 gumid: 1657)
00001670.00001bac::2017/03/17-02:46:59.193 info  [gum] node 1: processing requestlock 1:1203
00001670.00001974::2017/03/17-02:46:59.193 info  [gum] node 1: processing grantlock 1 (sent 2 gumid: 1659)
00001670.000009e8::2017/03/17-02:46:59.193 info  [nm] received request client address 10.3.175.34.
00001670.00001de8::2017/03/17-02:46:59.317 info  [nm] received request client address 10.3.175.34.
00000fa0.00001080::2017/03/17-02:47:43.933 info  [res] virtual machine <virtual machine victests01>: current state 'migrationsourcesetup', event 'migrationsourceofflineresult', result 0x00000000
00000fa0.00001080::2017/03/17-02:47:43.933 info  [res] virtual machine <virtual machine victests01>: state change 'migrationsourcesetup' -> 'migrationsourceoffline'
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] rcm::rcmapi::movegroup: (victests01, 2)
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] rcm::rcmgroup::move: (victests01, 2)
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] rcm::rcmgroup::move: bringing group 'victests01' offline first...
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] transitiontostate(virtual machine configuration victests01) online-->waitingtogooffline.
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] rcm::rcmgroup::updatestateifchanged: (victests01, online --> pending)
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] bringing dependent resource 'virtual machine victests01' offline before provider resource 'virtual machine configuration victests01'.
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] transitiontostate(virtual machine victests01) online-->offlinecallissued.
00001670.00001bac::2017/03/17-02:47:43.949 info  [rcm] 'virtual machine configuration victests01' cannot go offline yet; 'virtual machine victests01' in state offlinecallissued.
00000fa0.00001dc8::2017/03/17-02:47:43.949 info  [res] virtual machine <virtual machine victests01>: current state 'migrationsourceoffline', event 'offline'
00001670.00001cb4::2017/03/17-02:47:43.949 info  [rcm] handlemonitorreply: offlineresource 'virtual machine victests01', gen(32) result 0.
00001670.00001cb4::2017/03/17-02:47:43.949 info  [rcm] transitiontostate(virtual machine victests01) offlinecallissued-->offlinesavingcheckpoints.
00001670.00001cb4::2017/03/17-02:47:43.949 info  [rcm] transitiontostate(virtual machine victests01) offlinesavingcheckpoints-->offline.
00001670.00001cb4::2017/03/17-02:47:43.949 info  [rcm] transitiontostate(virtual machine configuration victests01) waitingtogooffline-->offlinecallissued.
00000fa0.00001dc8::2017/03/17-02:47:43.949 info  [res] virtual machine configuration <virtual machine configuration victests01>: current state 'online', event 'offline'
00000fa0.00001dc8::2017/03/17-02:47:43.949 info  [res] virtual machine configuration <virtual machine configuration victests01>: state change 'online' -> 'offlinepending'
00001670.00001cb4::2017/03/17-02:47:43.949 info  [rcm] handlemonitorreply: offlineresource 'virtual machine configuration victests01', gen(1) result 997.
00001670.00001cb4::2017/03/17-02:47:43.949 info  [rcm] transitiontostate(virtual machine configuration victests01) offlinecallissued-->offlinepending.
00000fa0.0000101c::2017/03/17-02:47:43.965 info  [res] virtual machine configuration <virtual machine configuration victests01>: current state 'offlinepending', event 'configdeleted', result 0x00000000
00000fa0.0000101c::2017/03/17-02:47:43.965 info  [res] virtual machine configuration <virtual machine configuration victests01>: state change 'offlinepending' -> 'offline'
00000fa0.00000cb4::2017/03/17-02:47:43.965 info  [rhs] resource virtual machine configuration victests01 has come offline. rhs report resource status rcm.
00001670.00001cb4::2017/03/17-02:47:43.965 info  [rcm] handlemonitorreply: offlineresource 'virtual machine configuration victests01', gen(1) result 0.
00001670.00001cb4::2017/03/17-02:47:43.965 info  [rcm] transitiontostate(virtual machine configuration victests01) offlinepending-->offlinesavingcheckpoints.
00001670.00001cb4::2017/03/17-02:47:43.965 info  [rcm] transitiontostate(virtual machine configuration victests01) offlinesavingcheckpoints-->offline.
00001670.00001cb4::2017/03/17-02:47:43.965 info  [rcm] rcm::rcmgroup::updatestateifchanged: (victests01, pending --> offline)
00001670.00001dfc::2017/03/17-02:47:43.965 info  [rcm] rcm::rcmgum::groupmoveoperation(victests01,2)
00001670.00001920::2017/03/17-02:47:44.011 warn  [rcm] rcm::rcmapi::resourcecontrol: retrying: victests01, 5908.
00001670.00001974::2017/03/17-02:47:44.121 info  [gum] node 1: processing requestlock 2:25
00001670.00001974::2017/03/17-02:47:44.121 info  [gum] node 1: processing grantlock 2 (sent 1 gumid: 1705)
00001670.0000194c::2017/03/17-02:47:44.230 info  [nm] received request client address 10.3.175.34.
00001670.00001704::2017/03/17-02:47:46.929 info  [rcm] rcm::rcmgum::groupmoveoperation(victests01,1)
00001670.0000194c::2017/03/17-02:47:46.929 info  [rcm] transitiontostate(virtual machine configuration victests01) offline-->onlinecallissued.
00001670.0000194c::2017/03/17-02:47:46.929 info  [rcm] rcm::rcmgroup::updatestateifchanged: (victests01, bringingtopersistentstate --> pending)
00001670.0000194c::2017/03/17-02:47:46.929 info  [rcm] rcm::rcmresource::online: virtual machine victests01's provider 'virtual machine configuration victests01' coming online (state onlinecallissued).
00001670.0000194c::2017/03/17-02:47:46.929 info  [rcm] transitiontostate(virtual machine victests01) offline-->waitingtocomeonline.
00000fa0.0000166c::2017/03/17-02:47:46.929 info  [res] virtual machine configuration <virtual machine configuration victests01>: current state 'offline', event 'online'
00000fa0.0000166c::2017/03/17-02:47:46.929 info  [res] virtual machine configuration <virtual machine configuration victests01>: state change 'offline' -> 'onlinepending'
00001670.00001920::2017/03/17-02:47:46.929 info  [rcm] handlemonitorreply: onlineresource 'virtual machine configuration victests01', gen(1) result 997.
00001670.00001920::2017/03/17-02:47:46.929 info  [rcm] transitiontostate(virtual machine configuration victests01) onlinecallissued-->onlinepending.
00000fa0.0000101c::2017/03/17-02:47:48.270 info  [res] virtual machine configuration <virtual machine configuration victests01>: current state 'onlinepending', event 'configregistered', result 0x00000000
00000fa0.0000101c::2017/03/17-02:47:48.270 info  [res] virtual machine configuration <virtual machine configuration victests01>: state change 'onlinepending' -> 'online'
00001670.00001de8::2017/03/17-02:47:48.270 info  [gum] node 1: processing requestlock 1:1249
00001670.00001974::2017/03/17-02:47:48.270 info  [gum] node 1: processing grantlock 1 (sent 2 gumid: 1709)
00000fa0.00001b24::2017/03/17-02:47:48.333 info  [rhs] resource virtual machine configuration victests01 has come online. rhs report status change rcm
00001670.0000194c::2017/03/17-02:47:48.333 info  [rcm] handlemonitorreply: onlineresource 'virtual machine configuration victests01', gen(1) result 0.
00001670.0000194c::2017/03/17-02:47:48.333 info  [rcm] transitiontostate(virtual machine configuration victests01) onlinepending-->online.
00001670.0000194c::2017/03/17-02:47:48.333 info  [rcm] transitiontostate(virtual machine victests01) waitingtocomeonline-->onlinecallissued.
00000fa0.0000166c::2017/03/17-02:47:48.333 info  [res] virtual machine <virtual machine victests01>: current state 'migrationsourceoffline', event 'online'
00000fa0.0000166c::2017/03/17-02:47:48.333 info  [res] virtual machine <virtual machine victests01>: state change 'migrationsourceoffline' -> 'onlinepending'
00001670.0000194c::2017/03/17-02:47:48.333 info  [rcm] handlemonitorreply: onlineresource 'virtual machine victests01', gen(32) result 997.
00001670.0000194c::2017/03/17-02:47:48.333 info  [rcm] transitiontostate(virtual machine victests01) onlinecallissued-->onlinepending.
00000fa0.00001b24::2017/03/17-02:47:48.348 info  [res] virtual machine configuration <virtual machine configuration victests01>: 'virtual machine configuration victests01' registered configuration virtual machine.
00000fa0.00000c30::2017/03/17-02:47:48.348 err   [res] virtual machine <virtual machine victests01>: 'virtual machine victests01' live migration did not succeed @ source.
00000fa0.00001080::2017/03/17-02:47:48.598 info  [res] virtual machine <virtual machine victests01>: current state 'onlinepending', event 'migrationsourceresult', result 0x80004005
00000fa0.00001080::2017/03/17-02:47:48.598 info  [res] virtual machine <virtual machine victests01>: current state 'onlinepending', event 'startupresult', result 0x00000000
00000fa0.00001080::2017/03/17-02:47:48.598 info  [res] virtual machine <virtual machine victests01>: state change 'onlinepending' -> 'online'
00000fa0.00000c30::2017/03/17-02:47:48.598 info  [res] virtual machine <virtual machine victests01>: 'virtual machine victests01' started virtual machine.
00000fa0.00000c30::2017/03/17-02:47:48.598 info  [rhs] resource virtual machine victests01 has come online. rhs report status change rcm
00001670.00001dfc::2017/03/17-02:47:48.598 info  [rcm] handlemonitorreply: onlineresource 'virtual machine victests01', gen(32) result 0.
00001670.00001dfc::2017/03/17-02:47:48.598 info  [rcm] transitiontostate(virtual machine victests01) onlinepending-->online.
00001670.00001dfc::2017/03/17-02:47:48.598 info  [rcm] rcm::rcmgroup::updatestateifchanged: (victests01, pending --> online)
00001670.00001dfc::2017/03/17-02:47:48.598 info  [rcm] group move 'victests01' has completed.
00001670.00001704::2017/03/17-02:47:48.645 err   mscs::gumagent::executequeuedupdate: error_file_not_found(2)' because of '::regdeletekey( handle, name.c_str() )'(resources\b9ff7f25-b80d-4f5c-905e-f0a0b1b2a3d7\internal\vmprivaterestartaction)
00001670.0000194c::2017/03/17-02:47:48.691 err   [dm] dmdeletekey: error_file_not_found(2)' because of 'gum handler completed failed'
00001670.00001704::2017/03/17-02:47:48.691 err   mscs::gumagent::executequeuedupdate: error_file_not_found(2)' because of '::regdeletekey( handle, name.c_str() )'(resources\b9ff7f25-b80d-4f5c-905e-f0a0b1b2a3d7\internal\vmprivaterestartthreshold)
00001670.0000194c::2017/03/17-02:47:48.707 err   [dm] dmdeletekey: error_file_not_found(2)' because of 'gum handler completed failed'
00001670.00001704::2017/03/17-02:47:48.769 err   mscs::gumagent::executequeuedupdate: error_file_not_found(2)' because of '::regdeletekey( handle, name.c_str() )'(resources\b9ff7f25-b80d-4f5c-905e-f0a0b1b2a3d7\internal\configprivaterestartaction)
00001670.0000194c::2017/03/17-02:47:48.769 err   [dm] dmdeletekey: error_file_not_found(2)' because of 'gum handler completed failed'
00001670.00001704::2017/03/17-02:47:48.785 err   mscs::gumagent::executequeuedupdate: error_file_not_found(2)' because of '::regdeletekey( handle, name.c_str() )'(resources\b9ff7f25-b80d-4f5c-905e-f0a0b1b2a3d7\internal\configprivaterestartthreshold)
00001670.0000194c::2017/03/17-02:47:48.801 err   [dm] dmdeletekey: error_file_not_found(2)' because of 'gum handler completed failed'
00001670.00001dfc::2017/03/17-02:47:48.816 info  [rcm] rcm::rcmgum::setgrouppreferredowners(victests01,<vector len='2'>
00001670.00001dfc::2017/03/17-02:47:48.816 info      <item>1</item>
00001670.00001dfc::2017/03/17-02:47:48.816 info      <item>2</item>
00001670.00001dfc::2017/03/17-02:47:48.816 info  </vector>
00001670.00001dfc::2017/03/17-02:47:48.816 info  )
00001670.00001704::2017/03/17-02:47:48.847 err   mscs::gumagent::executequeuedupdate: error_file_not_found(2)' because of '::regdeletekey( handle, name.c_str() )'(resources\b9ff7f25-b80d-4f5c-905e-f0a0b1b2a3d7\internal\failoverthreshold)
00001670.00001dfc::2017/03/17-02:47:48.863 err   [dm] dmdeletekey: error_file_not_found(2)' because of 'gum handler completed failed'
00001670.00001704::2017/03/17-02:47:48.863 err   mscs::gumagent::executequeuedupdate: error_file_not_found(2)' because of '::regdeletekey( handle, name.c_str() )'(resources\b9ff7f25-b80d-4f5c-905e-f0a0b1b2a3d7\internal\priority)
00001670.00001dfc::2017/03/17-02:47:48.863 err   [dm] dmdeletekey: error_file_not_found(2)' because of 'gum handler completed failed'
00001670.00001704::2017/03/17-02:47:48.879 err   mscs::gumagent::executequeuedupdate: error_file_not_found(2)' because of '::regdeletekey( handle, name.c_str() )'(resources\b9ff7f25-b80d-4f5c-905e-f0a0b1b2a3d7\internal\preferrednodes)
00001670.00001dfc::2017/03/17-02:47:48.879 err   [dm] dmdeletekey: error_file_not_found(2)' because of 'gum handler completed failed'


muhammad mehdi


as have identified second nic vm issue, helpful describe configuration of both virtual switches within cluster , how connected external switches.

did run cluster validation wizard see if warns of configuration issues?  validation wizard useful debugging tool.  besides, should run after change in cluster in case need open case microsoft - support requires it.


. : | : . : | : . tim




Windows Server  >  High Availability (Clustering)



Comments

Popular posts from this blog

some help on Event 540

WMI Repository 4GB limit - Win 2003 Ent Question

Event ID 1302 (error 1307) DFS replication service encountered an error while writing to the debug log file