I’ve done a lot of work with enterprise systems over the years. The Secret to resilience in the Enterprise is very simple. Make sure you have two of everything.
A typical Enterprise server usually has 2 power supplies, at least 2 network cards, two physical CPUs, two system drives (C: and D:) and each of these system drives consists of 2 mirrored hard drives for the OS Disk C: and 2 mirrored hard drives for the Data Drive D: It will usually have Serial-Attached SCSI Storage, connected via a fibre-channel storage array via two additional network cards and there will usually be a couple of extra network cards for migrating virtual images, backups or monitoring.
It doesn’t stop there. That’s just one server. Usually there will be a primary server and secondary server for each function. What’s more is that the physical servers are usually stored in two different locations. the Primary Data Center and the Secondary Data Centre. There will usually be two data lines connecting the Primary and Secondary Data Centre and these will usually be provided by two different cable providers. It’s not uncommon to have the electricity to be supplied by two different suppliers.
Even the staff will have redundancy. Each role will have a junior in training, most likely someone who can step in at a minutes notice. It’s the law of 2.
Needless to say, it’s expensive to run a Enterprise data centre. But it doesn’t have to be. The resilience attitude has rubbed off on me. I’ve made my home network pretty resilient but I’m very cost conscious. What is more is that I have an ulterior motive. I hate backups. I want resilience without backups if possible. Making backups and testing backups and restoring backups is very time consuming. Old fashioned backups onto tape often failed too. It’s fine if you are running a big enterprise and can afford to hire a backup monkey to do this and verify your backups every night/week/month but I am a busy man and I want everything live, now, and immediately.
In my Home Office Network I have two Servers. One server is my main workhorse and the other is my main storage server. Both have dual disks. The main workhorse has 2 x 2TB HDD Disks configured as a RAID-1 mirror and a 512GB Evo 850 Pro SSD. The Storage server has 2 x 512 GB disks for the OS and 2 x 3TB HDD (see figure 1 below) also configured as RAID-1 mirror. Since they are mainly used for Storage, they don’t need to be fast. All the disks are Western Digital Red which as a bit slower but work well. I’ve lost a few drives over the years, but they are easy to rebuild and replace.
Each of the kids have their own gamer PC with hand-me-down SSD drives for the OS (either 64GB or 127GB drives) and I expose a 127GB iSCSI virtual drive to each PC from the Storage server as an F: Drive, so that they have extra storage – although virtual drives are quite slow – especially compared to SSD. The kids don’t complain. Everything important to the kids is stored on OneDrive or the F: Drive so even if their PC dies, they don’t lose much more than their Downloads folder and some minecraft textures from the AppData folder (reminds me, I must make a backup, grrrr!).
Each server runs Hyper-V and hosts a tiny VM running Windows Server 2016. I have a Active Directory Domain setup and extenddd to the cloud for Azure Active Directory for Single Sign On to Office. These two tiny servers run my DNS Servers, (DHCP Servers (an active DHCP server and a hot standby) and Active Directory Service. I am running DFS for File Sharing. Again redundancy makes everything work beautiful.
Even with this, I use Cloud Storage too. I have a set of One Drive for Business accounts setup for different projects. Projects that are critical to my business. This gives me more resilience and also give me the ability to access any of these files outside of the Home Office via the mobile SharePoint app. It’s really great.
Frankly, it’s a cheap as chips to run too. The servers are pretty old now and nothing special. I think the current process in the Storage server is x quad core Xeon processor that I purchased on ebay for £35. Most of the other bits are consumer components. like this rock solid PC case from foxconn that cost £22 each including power supply and in 7 years, only one power supply has failed out of 5 computers I originally build in 2011.
I have been running this setup for 7 years now and it has survived motherboard failures, hard drive failures and cpu failures with almost no down time. It’s has very basic resilience based on duplication. It might seem like a bit of an over kill for a home network, but I love playing with tech, and so it works for me and it gives me peace of mind.