NLB initiating convergence on host 2 because host 1 is converging for an unknown reason
hi - hoping somone out there can me bizarre issue has cropped twice in past week, following on 6 months of no issues. of sudden, no-one can browse 1 of hosts, not web or unc using cluster name.
i have 2 web servers (vmware guests) web01 and web02. each vm has 2 nics , in multicast nlb cluster name of web.
at time of issue, these records appeared in system event log of web01:
- nlb cluster [192.168.109.110]: nlb initiating convergence on host 1 because host 2 leaving cluster.
- nlb cluster [192.168.109.110]: host 1 converged host(s): 1. active member of nlb cluster , start load balancing traffic default host. default host host lowest host priority. handles traffic isn't covered of defined port rules
here entry on web02:
- nlb cluster [192.168.109.110]: nlb initiating convergence on host 2 because host 1 converging unknown reason.
does know why happen? , more importantly, how stop occuring again?
====================================
here nlb display output web01:
nlb cluster control utility v2.5 (c) 1997-2007 microsoft corporation.
cluster 192.168.109.110
=== configuration: ===
current time = 09/05/2012 09:09:59
parametersversion = 5
currentversion = v2.5
effectiveversion = 00000201
installdate = 0x4f319462
hostpriority = 1
clustername = web.xxxxxxx.xxx.uk
clusteripaddress = 192.168.109.110
clusternetworkmask = 255.255.255.0
dedicatedipaddresses/ = 192.168.109.108/255.255.255.0
dedicatednetworkmasks
mcastipaddress = 0.0.0.0
clusternetworkaddress = 03-bf-c0-a8-6d-6e
iptomacenable = enabled
multicastsupportenable = enabled
igmpsupport = disabled
multicastarpenable = enabled
masksourcemac = enabled
alivemsgperiod = 1000
alivemsgtolerance = 5
maxconnectiondescriptors = 262144
filtericmp = disabled
clustermodeonstart = started
persistedstates = none
nbtsupportenable = enabled
unicastinterhostcommsupport = enabled
bdateaming = no
teamid =
master = no
reversehash = no
identityheartbeatperiod = 10000
numberofrules (1):
vip start end prot mode pri load affinity
--------------- ----- ----- ---- -------- --- ---- --------
all 0 65535 both multiple eql single
=== event messages: ===
#358478 id: 0x0000001d type: 4 category: 0 time: 08/05/2012 16:38:46
nlb cluster [192.168.109.110]: host 1 converged host(s): 1,2. active member of nlb cluster , start load balancing traffic default host. default host host lowest host priority. handles traffic isn't covered of defined port rules.
#358476 id: 0x00000064 type: 4 category: 0 time: 08/05/2012 16:38:40
nlb cluster []: nlb driver attached (bound) adapter '{b00362dc-dd61-446e-9d83-8837c0580920}'.
#358475 id: 0x00000005 type: 4 category: 0 time: 08/05/2012 16:38:40
nlb cluster [192.168.109.110]: host active member of nlb cluster. host priority (unique host identifier) 1. start load balancing traffic converges rest of cluster hosts.
#358474 id: 0x0000003f type: 4 category: 0 time: 08/05/2012 16:38:40
nlb cluster [192.168.109.110]: nlb initiating convergence on host 1 because host 1 joining cluster.
#358473 id: 0x0000004b type: 4 category: 0 time: 08/05/2012 16:38:40
nlb cluster [192.168.109.110]: nlb host state updated in registry. current state persist after system restarts, if nlb has been configured so.
#358421 id: 0x0000001d type: 4 category: 0 time: 08/05/2012 14:52:42
nlb cluster [192.168.109.110]: host 1 converged host(s): 1. active member of nlb cluster , start load balancing traffic default host. default host host lowest host priority. handles traffic isn't covered of defined port rules.
#358420 id: 0x00000045 type: 4 category: 0 time: 08/05/2012 14:52:38
nlb cluster [192.168.109.110]: nlb initiating convergence on host 1 because host 2 leaving cluster.
#357563 id: 0x0000001d type: 4 category: 0 time: 04/05/2012 17:03:30
nlb cluster [192.168.109.110]: host 1 converged host(s): 1,2. active member of nlb cluster , start load balancing traffic default host. default host host lowest host priority. handles traffic isn't covered of defined port rules.
#357562 id: 0x0000003f type: 4 category: 0 time: 04/05/2012 17:03:25
nlb cluster [192.168.109.110]: nlb initiating convergence on host 1 because host 2 joining cluster.
#357561 id: 0x0000001d type: 4 category: 0 time: 04/05/2012 17:03:03
nlb cluster [192.168.109.110]: host 1 converged host(s): 1. active member of nlb cluster , start load balancing traffic default host. default host host lowest host priority. handles traffic isn't covered of defined port rules.
=== ip configuration: ===
windows ip configuration
host name . . . . . . . . . . . . : web01
primary dns suffix . . . . . . . : xxxxxxx.xxx.uk
node type . . . . . . . . . . . . : hybrid
ip routing enabled. . . . . . . . : no
wins proxy enabled. . . . . . . . : no
dns suffix search list. . . . . . : xxxxxxx.xxx.uk
ethernet adapter nlb nic:
connection-specific dns suffix . :
description . . . . . . . . . . . : intel(r) pro/1000 mt network connection #2
physical address. . . . . . . . . : 00-50-56-a8-3c-a3
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
ipv4 address. . . . . . . . . . . : 192.168.109.108(preferred)
subnet mask . . . . . . . . . . . : 255.255.255.0
ipv4 address. . . . . . . . . . . : 192.168.109.110(preferred)
subnet mask . . . . . . . . . . . : 255.255.255.0
default gateway . . . . . . . . . :
dns servers . . . . . . . . . . . : 192.168.121.11
192.168.122.11
netbios on tcpip. . . . . . . . : enabled
ethernet adapter local area connection:
connection-specific dns suffix . :
description . . . . . . . . . . . : intel(r) pro/1000 mt network connection
physical address. . . . . . . . . : 00-50-56-a8-74-e7
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
ipv4 address. . . . . . . . . . . : 192.168.109.71(preferred)
subnet mask . . . . . . . . . . . : 255.255.255.0
default gateway . . . . . . . . . : 192.168.109.1
dns servers . . . . . . . . . . . : 192.168.121.11
192.168.122.11
primary wins server . . . . . . . : 192.168.121.11
secondary wins server . . . . . . : 192.168.122.11
netbios on tcpip. . . . . . . . : enabled
tunnel adapter local area connection* 8:
media state . . . . . . . . . . . : media disconnected
connection-specific dns suffix . :
description . . . . . . . . . . . : isatap.{49d7406b-9dd8-4a0a-9a79-55eba768014d}
physical address. . . . . . . . . : 00-00-00-00-00-00-00-e0
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
tunnel adapter local area connection* 9:
media state . . . . . . . . . . . : media disconnected
connection-specific dns suffix . :
description . . . . . . . . . . . : teredo tunneling pseudo-interface
physical address. . . . . . . . . : 02-00-54-55-4e-01
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
tunnel adapter local area connection* 11:
media state . . . . . . . . . . . : media disconnected
connection-specific dns suffix . :
description . . . . . . . . . . . : isatap.{b00362dc-dd61-446e-9d83-8837c0580920}
physical address. . . . . . . . . : 00-00-00-00-00-00-00-e0
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
=== current state: ===
host 1 has entered converging state 1 time(s) since joining cluster
, last convergence completed @ approximately: 08/05/2012 16:38:47
host 1 converged default following host(s) part of cluster:
1, 2
========================================================
and same web02:
nlb cluster control utility v2.5 (c) 1997-2007 microsoft corporation.
cluster 192.168.109.110
=== configuration: ===
current time = 09/05/2012 09:13:15
parametersversion = 5
currentversion = v2.5
effectiveversion = 00000201
installdate = 0x4f3194b9
hostpriority = 2
clustername = web.xxxxxxx.xxx.uk
clusteripaddress = 192.168.109.110
clusternetworkmask = 255.255.255.0
dedicatedipaddresses/ = 192.168.109.109/255.255.255.0
dedicatednetworkmasks
mcastipaddress = 0.0.0.0
clusternetworkaddress = 03-bf-c0-a8-6d-6e
iptomacenable = enabled
multicastsupportenable = enabled
igmpsupport = disabled
multicastarpenable = enabled
masksourcemac = enabled
alivemsgperiod = 1000
alivemsgtolerance = 5
maxconnectiondescriptors = 262144
filtericmp = disabled
clustermodeonstart = started
persistedstates = none
nbtsupportenable = enabled
unicastinterhostcommsupport = enabled
bdateaming = no
teamid =
master = no
reversehash = no
identityheartbeatperiod = 10000
numberofrules (1):
vip start end prot mode pri load affinity
--------------- ----- ----- ---- -------- --- ---- --------
all 0 65535 both multiple eql single
=== event messages: ===
#209910 id: 0x0000001c type: 4 category: 0 time: 08/05/2012 16:38:44
nlb cluster [192.168.109.110]: host 2 converged host(s): 1,2. active member of nlb cluster , start load balancing traffic.
#209909 id: 0x0000003f type: 4 category: 0 time: 08/05/2012 16:38:39
nlb cluster [192.168.109.110]: nlb initiating convergence on host 2 because host 1 joining cluster.
#209908 id: 0x0000001d type: 4 category: 0 time: 08/05/2012 16:38:16
nlb cluster [192.168.109.110]: host 2 converged host(s): 2. active member of nlb cluster , start load balancing traffic default host. default host host lowest host priority. handles traffic isn't covered of defined port rules.
#209907 id: 0x00000045 type: 4 category: 0 time: 08/05/2012 16:38:12
nlb cluster [192.168.109.110]: nlb initiating convergence on host 2 because host 1 leaving cluster.
#209879 id: 0x00000041 type: 4 category: 0 time: 08/05/2012 14:52:37
nlb cluster [192.168.109.110]: nlb initiating convergence on host 2 because host 1 converging unknown reason.
#208976 id: 0x0000001c type: 4 category: 0 time: 04/05/2012 17:03:30
nlb cluster [192.168.109.110]: host 2 converged host(s): 1,2. active member of nlb cluster , start load balancing traffic.
#208974 id: 0x00000064 type: 4 category: 0 time: 04/05/2012 17:03:23
nlb cluster []: nlb driver attached (bound) adapter '{418dcded-2d90-4836-b51c-b5e3b8bcaf81}'.
#208973 id: 0x00000005 type: 4 category: 0 time: 04/05/2012 17:03:23
nlb cluster [192.168.109.110]: host active member of nlb cluster. host priority (unique host identifier) 2. start load balancing traffic converges rest of cluster hosts.
#208972 id: 0x0000003f type: 4 category: 0 time: 04/05/2012 17:03:23
nlb cluster [192.168.109.110]: nlb initiating convergence on host 2 because host 2 joining cluster.
#208971 id: 0x0000004b type: 4 category: 0 time: 04/05/2012 17:03:23
nlb cluster [192.168.109.110]: nlb host state updated in registry. current state persist after system restarts, if nlb has been configured so.
=== ip configuration: ===
windows ip configuration
host name . . . . . . . . . . . . : web02
primary dns suffix . . . . . . . : xxxxxxx.xxx.uk
node type . . . . . . . . . . . . : hybrid
ip routing enabled. . . . . . . . : no
wins proxy enabled. . . . . . . . : no
dns suffix search list. . . . . . : xxxxxxx.xxx.uk
ethernet adapter nlb nic:
connection-specific dns suffix . :
description . . . . . . . . . . . : intel(r) pro/1000 mt network connection #2
physical address. . . . . . . . . : 00-50-56-a8-24-1d
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
ipv4 address. . . . . . . . . . . : 192.168.109.109(preferred)
subnet mask . . . . . . . . . . . : 255.255.255.0
ipv4 address. . . . . . . . . . . : 192.168.109.110(preferred)
subnet mask . . . . . . . . . . . : 255.255.255.0
default gateway . . . . . . . . . :
dns servers . . . . . . . . . . . : 192.168.121.11
192.168.122.11
netbios on tcpip. . . . . . . . : enabled
ethernet adapter local area connection:
connection-specific dns suffix . :
description . . . . . . . . . . . : intel(r) pro/1000 mt network connection
physical address. . . . . . . . . : 00-50-56-a8-72-35
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
ipv4 address. . . . . . . . . . . : 192.168.109.72(preferred)
subnet mask . . . . . . . . . . . : 255.255.255.0
default gateway . . . . . . . . . : 192.168.109.1
dns servers . . . . . . . . . . . : 192.168.121.11
192.168.122.11
primary wins server . . . . . . . : 192.168.121.11
secondary wins server . . . . . . : 192.168.122.11
netbios on tcpip. . . . . . . . : enabled
tunnel adapter local area connection* 8:
media state . . . . . . . . . . . : media disconnected
connection-specific dns suffix . :
description . . . . . . . . . . . : isatap.{49d7406b-9dd8-4a0a-9a79-55eba768014d}
physical address. . . . . . . . . : 00-00-00-00-00-00-00-e0
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
tunnel adapter local area connection* 9:
media state . . . . . . . . . . . : media disconnected
connection-specific dns suffix . :
description . . . . . . . . . . . : teredo tunneling pseudo-interface
physical address. . . . . . . . . : 02-00-54-55-4e-01
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
tunnel adapter local area connection* 11:
media state . . . . . . . . . . . : media disconnected
connection-specific dns suffix . :
description . . . . . . . . . . . : isatap.{418dcded-2d90-4836-b51c-b5e3b8bcaf81}
physical address. . . . . . . . . : 00-00-00-00-00-00-00-e0
dhcp enabled. . . . . . . . . . . : no
autoconfiguration enabled . . . . : yes
=== current state: ===
host 2 has entered converging state 3 time(s) since joining cluster
, last convergence completed @ approximately: 08/05/2012 16:38:48
host 2 converged following host(s) part of cluster:
1, 2
=============================================
any appreciated! thanks.
hi tiger,
this 1 real head scratcher thing see in setup wasn't 100% right binding order of nlb nic's being below lan nic.
however, think found culprit behind problem. vmware template had been created same ip address of nlb cluster. quite pain in find of course, it's not switched on of time, ip subnet scans didn't pick up.
when deployed vm template, without using customisation specification, had same ip address of cluster. it's @ point problem manifested per original posting.
thanks help.
Windows Server > High Availability (Clustering)
This blog is nice and very informative. I like this blog.
ReplyDeleteblog Please keep it up.