Guest blog post by Jake Drew
BAB - The Ultimate Gaming Workstation Server
What makes a computer blistering fast? The answer really depends on what you want to do with it and can even be quite complex depending on your requirements. Take for instance bitcoin mining. Custom bitcoin mining rigs can appear very unusual since many prefer to use graphics cards for the bulk of their bitcoin processing power.
Bitcoin Mining Rig (http://exabay.com/en_us/mining_hardware.php)
Since the majority of bitcoin mining activities depend on hashing, the GPU becomes a very performance optimal, cost effective, and low voltage solution for bitcoin mining programs. However, you can quickly see by looking above that the "Ultimate" workstation could look very different depending on what it is built for.
That all being said... Now, I would like to introduce you to our very own "Ultimate" gaming workstation server named "BAB" which affectionately stands for the "Bad A$$ Box". Our requirements for the BAB build were somewhat unusual. We wanted to create a massively powerful parallel processing memory monster which had the stability and coolness of a high-end server or workstation mixed with a cutting edge gaming rig.
BAB Build Requirements
- Process very large volumes of gene sequence data for my Ph.D. related machine learning research. For example, some of my input files are over 88GB in size.
- Create a machine which runs very cool at high processor loads for long periods of time. For instance, run all available cores at > 90% utilization for days or even weeks at time.
- Leave plenty of room for upgrading both memory and processing power as needed. Will the motherboard hold 256GB, if I happen to need it? Could I throw 12 more cores on my box, if my program is still running too slow?
- Act as a stand-in server should the need arise.
- Produce high quality graphics. Could my son play any of the latest video games on this machine? Could I plug a 4K monitor into this machine, if I were lucky enough to get one for my birthday?
The BAB build meets all of the above requirements and more. Honestly, I did not set out to build a computer. After much research however, I found the marketplace to be highly lacking for my unusual requirements. I also found server grade equipment to be highly overpriced.
BAB Hardware Overview
BAB Harware at a Glance
- Motherboard - Supermicro X9DRL-EF
- Power - EVGA Supernova 750 G2
- Processor - (2) Intel Xeon E5-2630 v2 Hexacore 2.6-3.1 GHz
- Hard Drive - (3) Samsung 840EVO 250GB mSATA SSD in RAID 0
- Storage - (2) 1TB Western Digital 7200 RPM
- Memory - 128GB ECC 1333Mhz DDR3L PC3L-10600
- Graphics - ASUS GTX 760 Striker Platinum 4GB GDDR5
- Sound - Creative Sound Blaster Audigy Fx
The Corsair AIR540 High Airflow Mid-Tower Case was selected for this project. The case was primarily chosen to meet requirement #2 providing ample airflow to all of the selected hardware components. BAB sports a total of eleven fans. Six of these fans pull air into the case, and three push air out. In addition, 4 fans are used to pull air over two Corsair H75 High Performance Liquid CPU Coolers. This is one "cool" box! The graphics card also includes two dedicated fans (for those of you who are counting).
Motherboard and Processors
The large memory and processing requirements for this project ruled out some of the more artistic looking motherboards which are available for modern gaming rigs. Most of the high-end gaming motherboards I considered, unfortunately, only supported up to 64GB of memory.
The Supermicro X9DRL-EF was selected as a cost effective alternative which supports up to 512GB of ECC DDR3 RAM and 2 sockets for the Intel Xeon E5-2600 v2 family of processors, which are available including up to 12 cores per chip / socket. Remember, this is a total of up to 48 virtual cores or threads in windows using the dual sockets on the Supermicro X9DRL-EF motherboard.
For BAB, we selected two Intel Xeon E5-2630 v2 hexacore processors with 6 cores per chip / socket. This provides a total of 24 virtual cores or threads on Windows which have performed very well during my own gene sequence classification benchmarks!
The two Intel E5-2630 v2 processors installed in BAB perform at 2.6 (base) - 3.1 GHz (max turbo frequency). In addition, using two of these processors provides 30MB of L3 cache. This can also be seen in the image directly above.
When it comes to memory, this machine beats many servers! Using 128GB of DDR3 RAM, BAB can process the heaviest of memory intensive workloads. This machine also leaves ample opportunity to use any excess memory for the latest in gaming techniques including loading highly utilized files such as game maps or databases to a RAMDISK partition. While some hardware retailers may tell you that no one could ever use such a large amount of memory, I would argue that a simple in-memory database can eat up 128GB in no time!
Furthermore, solid state drive manufacturers such as Samsung are now producing software optimized drives which can utilize memory to dramatically optimize SSD perofrmance. Samsung's Rapid Mode operates at the block and file system level to analyze application and data usage and eliminates system performance bottlenecks by dynamically leveraging system DRAM as a read/ write cache. Improved reads and writes on a SSD can exponentially increase your system's overall performance.  I imagine many software companies will eventually follow suit in using such memory optimizations, including memory mapped files, if they have not begun already.
Solid State Drives and Storage
One of the fastest ways to improve overall system performance is to increase the speed at which your operating system can store and retrieve data from the hard drive. While Solid State Drives offer one of the best ways to achieve this goal, even greater performance gains are available.
For BAB, we used 3, 250GB Samsung EVO840 mSATA solid state drives. To further improve our read / write performance, all three drives were hardware configured in RAID 0. This means all three drives act as one single drive providing a maximum performance increase of up to 300% over a single drive alone. However, such spectacular results are rarely seen in a real world setting. Nonetheless, you can definitely increase your read / write speeds using a RAID0 configuration in most cases. It is important to note that if a single drive fails within a RAID0 array, you will lose all of your data. It is always good to keep those backups up to date!
3D Printing To The Rescue
One of the unique modifications we made during this build was to 3D print custom drive trays for our SATA to mSATA converters which allows them to easily slide into three of the 2.5" SSD trays which came with the Corsair AIR540 High Airflow Mid-Tower Case.
The Trays were designed using Google Sketchup  which is a freely available 3D modeling program. We originally printed the trays in red, but chose green to match our fan lights when the first print did not work out so well.
It took about 10 hours for the Makerbot Mini to print all three of the trays. The printing process is very time consuming and takes much patience. There are numerous reasons why a print job can fail, and we experienced several of them. My son Nathan Drew deserves much credit for paying careful attention to the Makerbot during our printing runs.
The final trays work and look great in our drive bays. The 3 mSATA drives are combined in RAID 0 for around 750GB of operating system and speed sensitive storage. We also have 2, 1 TB 7200 RPM disk drives for media and archival storage.
Graphics Card and Install
The BAB build includes a ASUS GTX 760 Striker Platinum graphics card with 4GB of dedicated GDDR5 memory. As previously mentioned, this graphics card includes two dedicated fans and a very nice backlit "Republic of Gamers" logo which turns from green, then orange, to red depending on the graphic card's workload. In addition, ASUS provides the GPU Tweak software interface for both overclocking and monitoring the Striker Platinum card. The GTX 760 is not the fastest card available on the market to date, but it is a very cost effective top performer which supports up to 4096 x 2160 digital resolution.
One the challenges with using a server grade motherboard for such a project is the lack of PCI express 16x slots which almost every good graphics card requires for installation. In particular, the Supermicro X9DRL-EF motherboard came with no 16x slots, only 1x and 8x. We originally purchased the ASRock BTC Prokit which included 1 PCIe x1 to x16 Riser Card.
Unfortunately, the bulky size of the converter was making for a very ugly install as it would not allow our ASUS GTX 760 Striker Platinum graphics card to slide nicely into the case's rear PCIe slots. To make for a perfect build, we simply used a dremel to modify one of our existing PCIe 8x slots on the motherboard. In addition, using the 8x slot should provide additional processing bandwidth when compared to the 1x slot.
Modifying the 8x slot allowed the graphic card to slide perfectly into the case's rear PCIe slot for a flawless install. It also allowed us to place the graphics card in the perfect asthetic and functional spot on motherboard. The graphics card now sits directly in line with one fan pulling cool air in from outside the case and flowing nicely across the Striker Platinum's two on-board fans. Since the modified 8x PCIe slot does not provide the same physical support as a "locked in" 16x slot, we added a single strategically placed zip tie to the very back of the card for extra support. The tie is nearly invisible inside the case, and it makes us feel better about the install.
The primary difference between the PCIe 8x and 16x slots is bandwidth. However, in actuality, the 8x slot should provide ample bandwidth to support this particular graphics card. It would take a whole article for me to explain why. So instead, I will simply refer anyone interested to the following video. 
Since the installation, we have heavily bench-marked the graphics card, and we are very pleased with its performance. In fact, we were able achieve a very stable overclock using the following GPU Tweak values and little help from the article here .
Benchmarks were performed with the UserBenchmark and AS SSD Benchmark applications. The following results were achieved:
Who Uses Xeon Processors in Workstations?
It is interesting to note that Apple uses the Xeon E5 3.5GHz processors in the Mac Pro flagship workstation which includes 16GB of DDR3 ECC RAM in the 6-core version.  While the Mac Pro typically includes the superior Samsung XP941 PCIe flash based storage, the above triple RAID0 benchmarks demonstrate that speeds comparable to PCIe flash based storage can be achieved. When compared to these Apple PCIe SSD benchmarks, the BAB triple RAID0 configuration falls around the middle of Apple's performance stack while offering over 750GB of rapid storage for much less than a comparable amount of PCIe based storage would cost. However, I still plan on using a PCIe based SSD solution in my next build. As of today the 512GB version of the Samsung XP941 sells for around $510 on Amazon. Hopefully, these prices will come down to a more reasonable amount in the near future.
Pictures of the BAB Build
Please feel free to learn more about me at:
- Samsung, Solid State Drive Rapid Mode, http://www.samsung.com/global/business/semiconductor/minisite/SSD/downloads/document/Samsung_SSD_Rapid_Mode_Whitepaper_EN.pdf, Accessed on 01/16/2014.
- Google, Sketchup, http://www.sketchup.com/ , Accessed on 01/16/2014.
- LinusTechTips, PCIe Lanes - PCIe 8x vs 16x in SLI, https://www.youtube.com/watch?v=rctaLgK5stA , Accessed on 01/16/2014.
- HARDOCP, Overclocking the ASUS ROG Striker GTX 760, http://www.hardocp.com/article/2014/07/07/asus_rog_striker_platinum_gtx_760_4gb_video_card_review/3#.VLm8fivF9dM, Accessed on 01/16/2014.
- Apple, Mac Pro Specs, http://www.apple.com/mac-pro/specs/, Accessed on 01/16/2014.