Introduction
Let's assume you are a computational enthusiast, you like to code your own tools, test software your community releases and, yes, you have a limited budget.
Probably you already have a laptop or a desktop computer which is good enough to surf the web, but definitively is struggling under load and it cannot satisfy your needs. You are thinking to buy a new machine but, with bitcoin miners hitting the market and the semi-conductor shortage, you reasonably do not want to pay the hardware three times its market value. If this is your situation, have you considered to buy a decommissioned server rack from the second hand market? You probably know that companies around the world refreshe their IT facilities every three or four years and when they do so, the second hand market is flooded by valuable and fully working racks you can take advantage of.
Some companies sell their used hardware to third parties, which refurbish the components and put the items again on the market.
Some others simply put their used servers on Ebay or similar auction websites and it's there where you can do your deal, you just need to be patient
and wait for the good occasion!
Some time ago I was struggling with my 4 procs laptop and I decided to look for a used server to build an HPC test environment and give a boost to my computational capabilities.
I scanned the market roughly for one month and, in the end, I found out a German company selling their servers for a very competitive price.
I manage to get a fully working Dell R730:
- Intel® Xeon® processor E5-2600 v3/v4 product family
- Up to 3TB (24 DIMM slots): 4GB/8GB/16GB/32GB/64GB/128GB DDR4 up to 2400MT/s
- 2 x power supplies
- 2 x GPU support (nVidia or AMD)
- RAID controller
- Internal hard drive bay and hot-plug backplane: Up to 16 x 2.5" HDD: SAS, SATA, nearline SAS SSD: SAS, SATA, Up to 8 x 3.5" HDD: SAS, SATA, nearline SAS SSD: SAS, SATA
- H: 8.73 cm (3.44 in), W: 44.40 cm (17.49 in), D: 68.40 cm (26.92 in)
I got my machine for 700EUR with the following specs:
- 2 x Intel® Xeon® processor E5-2630 v3 (16 procs, 32 threads, 2.3Ghz base clock)
- 64Gb RAM running at 1833MT/S
- 2 x PSU
- 2 x SAS SFF (1.2TB in total)
- H730 RAID controller
- 2 x SAS SSF (1.2Tb storage)
But I readily upgraded the server using the following parts:
- 2 x Xeon® processor E5-2695 v4 (18 procs, 36 threads, 2.1Ghz base clock)
- 128Gb RAM running at 2400MT/S
- 2 x PSU
- 2 x SSD (1TB storage), used to store operative system and caching data.)
I spent in total 1400EUR for a machine that can easily afford CFD simulations up to 50M cells, which is more than enough to run benchmarks and have fun with many simulations!
Server install, upgrade and configuration
The server install is where all the fun starts. The machine can be put in a properer server rack you can store in closet or you can sit it somewhere in a room. In my case I put my Dell R730 under the desk of my old studio. It's best have your machine in a different room as under heavy load it becomes a bit loud, however I was thinking worst to be honest.
Make sure to have at hand these cables before start the install:
and also these two important elements:
You will need two power cables to plug your power supplies, the Ethernet cables will be used to connect the server to your home network, the VGA cable is required for the initial server configuration, while the switch will be used
to connect all your network cables.
The wifi range extender is not mandatory if you can connect the switch directly to the network (e.g. you have an Ethernet plug in the wall).
In my case that was not possible and I used this cheap extender to build an hot spot with Ethernet access.
Once all the connections are set, you can fire up your machine and start the configuration process. This is probably the longest part and you need to:
- Install your favorite operative system:
Linux, Windows or Proxmox, the operative system install is not different from any other computer you have used before: flash a usb, change the boot order and follow the OS instructions.
In my case I chose to install centOS. If you are using a RAID configuration, then you need to configure it before installing your OS on virtual disk!
- Update your firmware:
Your machine will come to you probably reset back to the factory configuration, so you will need to update the firmaware to the latest version.
Things here are a bit tricky since the default ftp method provided by Dell is not working anymore.
A new HTTPS method is provided, but the original version of the life cylce controller does not support it.
So you need to manually update the life cycle controller first (e.g. downloading and running its executable in Linux) and upgrade the firmware later on, using the new version of the life cycle controller.
While the second operation will be quite straightforward and automatic, the first one needs a bit of care. Look out the video in the link.
- Configure the RAID controller: this step is quite easy. You can configure your RAID from the life cycle controller or from the BIOS as the server boots up.
Make sure to connect all the necessary disks (SAS, SATA or SSD) before start to configure.
- Set up IP addresses and configure the remote access controller (iDrac):
this step shuold be quite straightforward. Jump inside the network configure and assign a static IP and a port number to your to your iDrac.
The port number is set to a defualt value, but I suggest you to change it to make your network more secure.
Do not worry, those steps are quite straighforward. Take your time and follow the instruction provided in the link above.
Network set up
If you want to get the most from your home lab, you need to configure properly your network such that you can connect to your machine anywhere in the world.
This means essentially three things:
- Assign to your home router a DDNS: your home router changes IP address time to time. This is for security reasons and it makes
difficult to connect to your home server since you cannot be sure your home IP has not changed. To get rid of this problem, you need to
assing a name to your home router such that you can use it in place of IP. In order to do so you need to create an account on one of the many DDNS
providers such as NoIP and then link your account to the router. This procedure is quite router specific,
, but every router has a dedicated panel to set a DDNS. Look out in your router documentation. After this procedure you will be able to ping your home network
by name (e.g. mylab.ddns.net:portNumber).
- Port forwarding : Once your home IP address has been linked to a fixed name, you will need assign and open a port for each device connected
to your home network you would like to access. For a server you will need to open at least two ports, one for the remote access controller and one for ssh (022 by default).
You can set the ports number in the network management in your machine and then open them up in the router. Each router has its specific procedure, look at the documention or do
a quick search in google for your own model!
Job scheduler
Now your server is up and running and you can start using it as you like. A nice feature to add is a job schedure (e.g. SLRUM, PBS)
managing your resources and putting in queue all your jobs. I found this quite useful and handy, especially if you plan to run while you are not sit at your desk.
In my case I use SLURM, which free and widely used in my HPC facilities. You might think it is difficult to set up, but this is really no the case!
Refer to this
step by step guide I found online to sort this problem out with relative ease.