No traffic from Server A to Server B (and vise versa) but all other servers can talk to them (and vise versa)

Question

I can not seem to get traffic from Server A to Server B, and no traffic from Server B to Server A. On the other hand all other servers can talk to them and they can talk to all other servers. Just looking for suggestions on how to debug it...

Layout:

NORCL1N1 (physical)
`-NORCL2N1 (virtual)
`-NOROIT (virtual) AKA Server A

NORCL1N2 (physical)
`-NORCL2N2 (virtual) AKA Server B

Server Information:

NORCL1N1

Type: Microsoft Virtual Network Switch Adapter (Connected)
Mac: 18-03-73-F0-39-CA
IP: 10.71.1.4

Type: Microsoft Failover Cluster Virtual Adapter (Disconnected)
Mac: FE-5B-20-52-41-53
IP: N/A

NORCL2N1

Type: Microsoft Failover Cluster Virtual Adapter (Connected)
Mac: 02-15-5D-01-67-00
IP: 169.254.1.126

Type: Microsoft Virtual Machine Bus Network Adapter (Connected)
Mac: 00-15-5D-01-67-00
IP: 10.71.1.5, 10.71.1.12, 10.71.1.13

NOROIT

Type: Microsoft Virtual Machine Bus Network Adapter (Connected)
Mac: 00-15-5D-01-67-03
IP: 10.71.1.15
AKA: Server A

NORCL1N2

Type: HyperV Virtual NIC (Connected)
Mac: 18-03-73-F0-3A-AB
IP: 10.71.1.6, 10.71.1.8, 10.71.1.9, 10.71.1.10, 10.71.1.11

Type: Microsoft Failover Cluster Virtual Adapter (Connected)
Mac: 1A-03-73-F0-3A-A9
IP: 169.254.2.127

NORCL2N2

Type: Microsoft Failover Cluster Virtual Adapter (Disconnected)
Mac: 02-15-5D-01-6B-00
IP: 169.254.2.241

Type: Microsoft Virtual Machine Bus Network Adapter (Connected)
Mac: 00-15-5D-01-6B-00
IP: 10.71.1.7
AKA: Server B

Pinging everything from everything works great with the exception of pinging NORCL2N2 to NOROIT and vice versa...

I disabled all firewalls, checked all ips (tried looking into the possibility there might be another computer using the same ip), I even tried restarting the app server (NOROIT)... Can not seem to come up with a reason why traffic between the two servers are having issues. Any ideas?

Answer 1

I think I figured it out...

It seems like it had to do with routing. I had to manually enter in persistent routes to make sure the servers in question knew how to get to each other.

For example on NOROIT (open command prompt and type)...

route add -p 10.71.1.7 mask 255.255.255.255 10.71.1.1 metric 1

Then on NORCL2N2 type...

route add -p 10.71.1.15 mask 255.255.255.255 10.71.1.1 metric 1

This is a "band aid" solution and something else... something bigger is an issue... I think it might have something to do with the Hyper-V server and the virtual nics...

It seems that even if I added a static route for 10.71.1.0 mask 255.255.255.128 it would not know how to get to the servers... Only after adding each one was I able to get it working.

Update / SOLUTION (this is what fixed the issue):

It was much simpler then I was making it out to be... It seems on the switch that I was using (Dell Layer2) the ports being used by the servers NORCL1N1 and NORCL1N2 were set up on a different VLAN. I thought the ports were damaged on the switch at first but instead somehow there VLANs were changed when I took them out of a LAG (used for teaming). I originally had 3 NICS on NORCL1N1 set up as a team and in turn had 3 ports on the switch set up in a LAG. I turned towards using the NICs on the server WITHOUT teaming involved (due to issues with virtual macs, ip conflicts, etc) and wanted to dedicate a NIC for virtuals, one for cluster-heartbeat, and 1 for the host machine. After doing that and removing the teaming from the server, then removing the LAG from the switch, I thought I was good to go... Not so fast.. I had to go back to each port on the switch and reset the VLAN the ports belonged to again...

Seems everything is fine now! Hope this helps someone out...