I thought ethernet was simple. I was wrong.

I’ve been saying that I hated WiFi because it’s too complicated and too unstable, and that ethernet is simple and reliable because it works the second you plug it in. I’d like to modify that statement – I still hate WiFi because it’s too unstable, and I’m starting to hate ethernet for being too complicated.

A while ago I wrote something about setting up an ESXi host, however due to the humongous level of electricity it uses and the deafening noise that the 13 fans inside the 2U case makes, I had to decommission the IBM rack server. Since then my home network has been relying on a Raspberry Pi 2 (later upgraded to 3) for the DNS, DHCP, SSH, and the occasional web server roles. A few days ago I found a pretty neat Dell Optiplex 9010 SFF desktop computer that was used in an enterprise environment with good CPU resource compared to my Pi (A potato has more CPU resource than the Raspberry Pi), and is comparable to the IBM. Due to its desktop form factor and that it was designed to be power efficient, the electricity usage is minimal, as is demonstrated by the built in 250W power supply. In comparison, the IBM rack server had 2 x 1000W power supply. So I snatched it off eBay, installed some RAM, a Intel PRO/1000 PT dual ethernet card, a larger hard drive, ESXi 6.5, and fired it up straight away.

Having painfully failed to set up link aggregation between the host and my D-Link DGS-3100-24 switch, I moved onto something I wanted to do for a long time – networking upgrade. It’s always bothered me that I was using a Linksys E3000 as my router. It’s very stable, but gets incredibly hot, which means its CPU will throttle and my internet speed will tank. By moving the routing role to a more powerful device, the “routers” can just act as Access Points, therefore reducing the level of load on each device. However, the internet connection is connected to inside the house, in a cabinet next to the power box. Since I didn’t want to pull another cable from that cabinet to the garage (where the servers are), and that I already had a switch there and 4 x gigabit ethernet LACP link to another switch in the garage, I wanted to utilise that link using VLAN. The diagram below shows the current set up of my house’s network. I have named this the Home Network 1.0.

Virtual Local Area Network is something I’ve seldom heard of before, never mind played with, so you can imagine the level of blankness I had in my head when I started to learn about this stuff. The jargons that VLAN involves, like VID, PVID, GVRP, Trunk, LACP/Static LAG, L2/L3, Ingress, Tag/Untag, just to name a few. It’s, as you probably can feel already, very complicated. From what I can work out, I need to create a VLAN on Switch A with (e.g.) Port 20 and link aggregation group 1 (the one going between two switches) as a trunking link, then I need to create the same VLAN on Switch B with the LAG 1 as the trunking link, and LAG 2 (the one going between the switch and the ESXi host), so I can access the VLAN on the ESXi host. In order to illustrate this, I created another diagram of what I wanted:

The green lines represent VLAN 1 (the default one that most devices are on), the red line from Modem to Switch A represents VLAN 2 (that nobody else can access), and the purple lines represent links that carries both VLAN 1 and VLAN 2.

One problem I had was I didn’t know how to set up LAG1 to carry both VLAN 1 and VLAN 2. This should be very easy, but for some reason I couldn’t get it to work. That’s definitely going to be my next step forward. As I was running out of time, and had lots of work to do, I took a short cut, and botched a solution which works for now. I’m calling it – Home Net v1.7. It’s demonstrated below:

I guess this will have to do for now. There’s no redundancy between the modem and the switches, which makes me a bit uncomfortable, but hey – it works for now. Another problem that I need to solve is that LAG 2 currently doesn’t bond bandwidth, which means that the transfer from and to ESXi is capped at 1Gbps, instead of the bonded bandwidth of 2Gbps. This is quite annoying, and I will definitely rectify this problem during the upgrade to HNv2.0.

Currently I am relatively happy about the set up. I’m using DD-WRT on a virtual appliance on ESXi to act as my router, and all the routers act as access point only. I’m running a Ubuntu 16.04.2 LTS virtual instance to manage the DNS, DHCP, and my Plex server. Next step is to implement backup services onto the ESXi host (I already have a good candidate for this job, but just need to get it to work), fix LAG 2, and the LAG 1 trunking problem. Can I just say… it’s great fun learning about network layers and the cool concepts. Much more fun than studying for exams which is really what I should be doing right now…

EDIT: Home Network v2.0 is up and running after buying a different switch, and it’s beautiful… LAG1 and LAG2 are both fixed, and tested for redundancy as well as bandwidth. Next step is to install a pfSense virtual machine on my ESXi host by request of some old man.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.