I run couple of VMware EXSi hosts in a cluster as test benches for virtualization and for hosting my homelab servers.
As I did not want to spend on a loud and expensive 10GB switch I decided to configure my FreeNAS server as a bridge.
This way I have 10GB connectivity to FreeNAS for shared storage on my both EXSi servers.

To accomplish this I got couple of cheap 10GB single port SFP+ PCI-e network cards from ebay and one dual port SFP+ card for the FreeNAS server.
Latest version of the FreeNAS supports alot more cards so finding a suitable one shouldn’t take much time.
For connection I used DAC’s as they draw less power than fiber transceivers and they also have slightly lower latency.

First let’s create new interfaces on FreeNAS. I’m using 192.168.200.100 address for the 10GB network.
Both interfaces need to have the same IP-address.

Then we add some tunables so that bridge will be brought up in boot:

Variable: autobidge_bridge10
Value: cxb0 cxb1
Type: rc

Variable: autobridge_interfaces
Value: bridge10
Type: rc

Variable: cloned_intefaces
Value: bridge10
Type: rc

Variable: ifconfig_bridge10
Value: up 192.168.200.100 netmask 255.255.255.0 mtu 1500
Type: rc

Variable: ifconfig_cxb0
Value: up mtu 1500 -tso -lso
Type: rc

Variable: ifconfig_cxb1
Value: up mtu 1500 -tso -lso
Type: rc

Like this:

That’s all. Now reboot FreeNAS server and check connectivity to the EXSi servers.

Then let’s try vMotion from vCenter.

All good! 4 seconds to migrate one of the testlab DC’s using shared storage to another node on the cluster.

Live migration from local- to another local-storage took couple minutes for a 40GB disk

This is a simple and cheap way to play around with VMware and shared storage and vMotion.

Parts used:

2x Mellanox ConnectX-2 single port 10GB SFP+ network cards (HP 671798-001)

1x Chelsio t320 dual port 10GB SFP+ network card

1x 1M Cisco branded DAC

1x 3M Cisco branded DAC