High System Interrupts

I just finished fixing a problem with one of my VMs using hardware pass-through. It was one of the machines getting the second USB controller. When I went into the OS (Windows), it was very slow. When checking task manager, it showed the first CPU floating around 30% and the second CPU was at idle. When looking at the processes, there wasn't anything consuming CPU cycles. Then, I saw that it was hardware system interrupts within the Resource Monitor.

I isolated it to the USB controller vice the video card by removing both and adding them back one at a time. I tried a couple different scenarios with switching the controller to other VMs and whatnot, but with no luck.

To resolve the problem, I de-selected all the hardware for pass-through, rebooted, reconfigured the pass-through and rebooted a second time. I checked my VMs to ensure they were still configured with the correct hardware items and powered everything back on. I think it might have happened when I was adding additional hardware for pass-through and it just got out of sync somewhere.

Within Linux, I'm fairly certain you can use iostat to track down culprits with high system interrupts. I think it was the tool I used a few years ago to track down something (it was either the analog phone card or a dying hard drive from what I can remember).

Storage Issues (RAID)

So, it's ideal to have RAID IMO - primarily for redundancy. I've had many hard drives fail to live without the redundancy.

My inventory includes 4 - 2 TB drives (somewhat old and somewhat slow), 4 - 250 GB (very old and very slow), and 2 -2 TB drives (new and fast). I was hoping to setup the larger drives on their own RAID (RAID 1 for the 2 new drives and RAID 5 for the 4 TB drives). I also have an older 4 bay SANS Digital SAN.

I bought a Dell perc 5i RAID card with two sets of SAS to SATA cables, to support 4 SATA drives per cable. Bad news is that it apparently doesn't work with VT-D enabled, which is the cornerstone of my entire virtual environment. When I installed it, I was able to create the logical drive, but ESX kept loosing the connection to it. It would see it until I started moving files to it, then it would disappear from inventory. When I would add it back, it wouldn't see that there was a VMFS partition on it and it would create a new one.

My options (at least what I can think of) is to use the 4 large drives for a software RAID within my FreeNAS VM, use the drives individually, or use my external SAN that I was using previously. The only problem with the external SAN is that it's not going to be as fast as the internal RAID card.

I decided to use the external SAN. I'll have to be extra careful not to allow the external SATA cable to come unplugged, but it gives me the flexibility to put VMs on the RAID array without having to worry about FreeNAS making the iSCSI / NFS mount point available to ESX to power on additional machines. And, it'll be a little more mobile.

I think that I will try and incorporate the Dell RAID card into a future build. I'll keep my eye open for a decently powered server that can support FreeNAS and add some hard drives to the RAID. Then, I can setup an iSCSI target to be used by the ESX host. It would even set me up to migrate VMs between ESX future ESX hosts.

Parts Purchased!

After much debating, I finally finished my research and purchased all my parts! It was a combination between using Micro Center, New Egg, Amazon and PC Outlet.

I had a few change of requirements mid way through my research. Everything in the end will cost about $1600 not including the 2 older SANs I have.

Before purchasing any of the items below, please see my "lessons learned" page. I'm more likely to keep the lessons learned page up to date then coming back to this page.

***Quick list:*** 
Trip Lite 1500VA/900 Watt UPS, Corsair CX750M, Cooler Master HAF 912 case, ASRock X79 Extreme 6, 64 GB Crucial Ballistix, Intel i7 3820 CPU, 2 - 2 TB Seagate Barracuda ST2000DM001 HDDs, Asus BluRay writer, Asus Radeon HD 7770, Hauppauge HVR-2250, Digium TDM4000 (already own), and Sound Blaster Audigy (already own).

***Explained:***
Motherboard - ASRock X79 Extreme 6

Requirements included a good warranty, Direct Path support, 64 GB memory and lots of SATA with RAID options. I did an analysis of what I think I might run as VMs, and it came out to about 40 GB of memory. I couldn't find any AMD boards that supported the FX-8xxx line with 8 DIMM slots (they all had 4 and nobody makes 16GB memory chips as of yet).

I'll be able to fully populate all the 10 SATA ports (2 as RAID 1 with the new Seagate drives), 4 as RAID 10 from old SATA I 250GB drives I have, 1 SATA for the bluray, 1 SATA for SAN #1, 1 SATA for an older 320 GB 2.5" drive (for online backups of important stuff), and 1 eSATA for SAN #2. The motherboard RAID controller isn't supported within ESX, however SATA ports are supported for individual drives.
http://www.newegg.com/Product/Product.aspx?Item=N82E16813157289

CPU - Intel i7 3820

Was going to go with the 8 core 3.6 ghz FX-8150 (advice taken from http://thehomeserverblog.com). Only requirement is a good warranty and IOMMU support. This CPU has another 4GHz processing power than the 8120. But, couldn't find any boards that supported 64 GB with 4 DIMM slots.

So, went with the i7 3820. This was the lowest end Intel processor that supported Direct Path and 64 GB.
http://www.newegg.com/Product/Product.aspx?Item=N82E16819115229

Memory - 64 GB Crucial Ballistix Sport

Though about 4 - 8GB DDR3 - I think I really need at least 32 GB. 64 would be a lot more ideal, but I don't know if the boards that support IOMMU also support 64 GB.

Had to go with an Intel board. The lowest end ASRock that supported 64 GB was the X79 Extreme 6. Went with the Crucial Ballistix Sport line DDR3 1600 as it had good ratings and was a good price (~$50 on sale).
http://www.newegg.com/Product/Product.aspx?Item=N82E16820148540

Power supply - Corsair CX750M

Based on online calculator with 2 videos gaming cards, 2 pci cards, 3 pci-e cards, 4 Sata drives, 4 fans, and 10 USB devices with 30% capacitor aging: 652 watt. 

Started thinking about the Corsair CX line. The 750 watt one for now for around $90. Little pricey, but don't want components frying and want a decent warranty. I then came across the NZXT series, which looked really nice (HALE820-N) for around the same price as the Corsair.

Ended up going with the Corsair CX750M as it had enough power for the Intel processor and it was on sale at Micro Center.
http://www.newegg.com/Product/Product.aspx?Item=N82E16817139051

Case - Cooler Master HAF 912

Started looking at the Antec 300, as I would like something a little stylish with lots of cooling (will be installing in a room without a lot of temperature control. It would also be nice to have the capability to do water cooling (don't know that much about this yet).

When I came across the NZXT power supplies, I started looking at their cases. I really liked the Guardian and Lexa-S, but it was a little more than what I was expecting (~$90).  But, it looks really nice and it has lots of cooling.

Then a coworker turned me back towards Antec, but the DF-85. I'm currently looking at a deal on Ebay for both the DF-85 and Atec's 1000W power supply for $155 with shipping. Pending some questions from the seller on its condition.
http://www.newegg.com/Product/Product.aspx?Item=N82E16811119233

Decided on the Cooler Master HAF 912 as it met my basic requirements 1) 6 internal 3.5" HDDs; 2) 4 5.25 slots for BD drive, fan controller, Sound Blaster panel and USB 3.0 ports provided from mother board; and 3) had enough fans to keep everything cool.

UPS - Tripp Lite 1500VA/900watt SmartUPS

 Started looking at the APC 750va, but might not run long enough (http://www.apc.com/products/runtime_for_extendedruntime.cfm?upsfamily=29). I then started looking at the 1500VA range, and came across a Tripp Lite Smart UPS that might do the trick.

I went with the Tripp Lite Smart UPS as I hope it'll support everything I need for the price ($160).
http://www.newegg.com/Product/Product.aspx?Item=N82E16842111461

Network Cards
The motherboard will come with a single gigabit NIC, but it would be nice to have a second one if I needed it. I don't plan to setup dedicated networks for traffic such as vMotion as I'll only have a single server. But, I think I'd like to configure a port on my router for DMZ traffic and forward that to a virtual machine running a firewall. This will allow me to 1) stay current with router defense and 2) allow me to do whatever I need to with port forwarding without effecting current house traffic (i.e., reboot or pull it offline for a security breach).

I'm looking at a $30 option on NewEgg for a Syba SY-PEX24028 which has 2 gigabit ports. This should allow me to fulfill the virtual router requirement while leaving another port available.
http://www.newegg.com/Product/Product.aspx?Item=N82E16833328022

Video Card
I'm not sure if I actually need a video card for the ESXi console or not (for it to actually boot up and run correctly more than if I would actually use it or not - once it's setup, I don't know how often I'd actually use it except if something gets messed up and needs fixing). I'd rather not waste a slot if I don't need to, and I'd like to pass my video card to a VM for use within it. I'm hoping that if I ever needed to get to the console, I could shut my VM down, remove the hardware pass through, and view ESXi's console. Worst case, I could have spare card available to put into the server replacing another card I'd be using full time.

I don't really do that many games, but it's fun to play a few games every now and then if the price is right (i.e., close to free such as Star Trek Online, Mechwarrior Online,  or  Tribes Online). I can play some of these with my 4 year old laptop (I think it's a Radeon HD 4560 card), so it would be nice to at least continue to play these on the VM. And since I don't play games that often, I won't have to worry about spending extra money for a second machine that I wouldn't use that often.
http://www.newegg.com/Product/Product.aspx?Item=N82E16814121632


Sounds Cards:
I'm going to try and re-use the PCI Sound Blaster Audigy ZS platinum.
The 5.1 version: http://en.wikipedia.org/wiki/Sound_Blaster_Audigy#Sound_Blaster_Audigy_2_ZS

Phone Cards:
I already have a TDM400 card by Digium. I will try and re-use it. I have 4 FXS modules which fully populates the card. I will try and get a FXO module to replace the FXO PCI card I'm using.
PCI version of: http://www.digium.com/en/products/telephony-cards/analog/4-port

TV Tuner:
I have a USB tuner from Hauppauge, but I would like to have a 2 port PCI-e card. I'd like to dedicate 1 port for over the air recording and a second for any other signals coming in via coax cable.
http://www.newegg.com/Product/Product.aspx?Item=N82E16815116037

USB Extenders:
I read about HDMI and USB extenders, which shouldn't be more than $80. This will allow me to setup the machine in another room and easily move the keyboard / mouse between rooms to connect to the HDMI supported TVs.

Kicking Off Research

August 18, 2013

I've been researching the components to use for a couple weeks now. I think I've narrowed everything down, but keep going back and forth between a couple pieces (which processor? which motherboard?). It's been a struggle between cost (hopefully less than $1000), how long I expect to actually use this (hopefully 8+ years) , and how much future growth I might need in a couple years. Spending an extra $100 now could allow me to add more memory or support additional HDDs to use the computer for a couple extra years.

I've also been trying to sell all my old computer parts (~5 to 10 years old). I need to both clean up all my old stuff (I have several totes full) while making some cash for my future build. I've been spending a lot of time wiping all my old IDE HDDs with TrueCrypt with it's encryption / 7 pass option to hopefully keep any old data unrecoverable (I'd like to get $5 for each one). And with my old Cisco 2500 series routers, Digium FXS card, and numerous old motherboards / PCI / AGP cards, there's lots to sell. I'm hoping to get around $300 for everything.

A site that I found to be really helpful is PC Part Picker (http://pcpartpicker.com). I've only begun to start using it, but it's seems to be helping with compatibility checking as well as finding the cheapest parts from multiple vendors.

First Post

Current date, August 8th, 2013: I kicked off this blog so that I could organize my thoughts with what I currently have and what I would like to do. I've started a few top level pages to track major efforts. The other goal is to potentially assist other users in their endeavors.

Comments are welcome, but I can't guarantee that I can respond quickly. I usually only have a couple hours a week to work on my computer stuff. Thanks for reading!