Pages Navigation Menu

Mod Something

Building begins!

Building begins!

More hardware came in so I can now finally start building but first some surprising news. Apparently shipping got the tracking numbers mixed up and my SC828TQ showed up! (Our new vacuum did not.) Since this did come in I will end up using it as a test system running ESXi.

Finally the Supermicro X10SRL-F motherboard came in.

3x LSI 9211-8i HBA. These have 2x SFF-8087 ports. I will only need 1 of these for the SC846 (SFF-8087 to SFF-8087) and SC828TQ (SFF-8087 to 4x SATA) but I will need 3 of them for the ENDSE server since it has 6 expanders (4 drives each) which each have a SFF-8087 port. (SC846 has 1 expander for all 24 drives).

2x 16GB SanDisk Cruzer Fit. I will have the FreeNAS OS mirrored on these. I will pick up more for the other systems at a later date.

I slapped the CPU into the board and installed the Supermicro 4U cooler.

Then installed the motherboard and HBA into the SC846 chassis. As you can see on the right side there is 1 fan not hooked up. I am waiting on the 4 pin extension cable to come in so I can hook it up to the motherboard.

The rest of my 2TB WD Red drives came in. I went ahead and installed all 17 of them into the HDD trays.

Since I don’t have the tray labels yet I had to go old school and get out my P-Touch and make labels.

As you can see in the above picture, there is an awesome light coming from inside the rack. Two years ago my parents got me a LED kit with remote. I thought about how dark the rack is when I’m trying to work in it and thought this would be a good fit for that LED kit. I installed the strips on 3 sides in the front and back.

Before I could do anything I had to wire up the network connectivity in the rack. Below is a close up shot at my switch. Cable management is not complete as I am still in the building phase but you can get an idea of how I have things separated. Port 1 is my connection from the router, Ports 2-5 are the NICs on each server and the last 3 are for the IPMI ports on the motherboards. (If you have never used IPMI before it is pretty freaking sweet.)

I then installed FreeNAS on the 2x 16GB Fits and made sure the system saw all 17 HDDs. Once I confirmed that I went ahead and flashed each LSI 9211-8i (one by one) from P18 to P20 IT mode. If you have one and need to do this upgrade I found a really good step by step guide here.

Now once all that is done I needed to start testing the system. I ran Memtest86 for about 12 hours, then CPUBench for about 12 hours and then started HDD tests. There is a really good thread on this topic on the FreeNAS forums. The SMART tests are really easy but I had a bit of trouble trying to figure out the badblocks part. Well, starting the process was easy but switching between instances took me a bit of research to figure out. Here is a good cheat sheet for ‘tmux’ commands.

Here you can see I have all 17 drives performing the badblocks test. As of this writing it has been going for 31 hours so far. I figured it would take about 36-48 hours to complete the test. If you have any questions on ‘burn-in testing’ let me know. I am not a pro at this but I have done quite a bit of research to make sure I am doing things right.

badblocks status

Once all the testing is done and I can confirm that all the drives are good I will create my volume and start configuring datasets and plugins. There is a FreeNAS Raid calculator out there that tells you how efficient your array is but of course I can’t find the link. Basically for me it looked better to do RAIDZ2 + Hot Spare than to do RAID Z3. I am not sold on that but I will think real hard about it. (And probably ask on the FreeNAS forms, haha)

My next post will just be for the Pfsense build. All the parts came in and I built it but ran into a few snags so I will share some “lessons learned” and hopefully stop someone from making the same mistakes as I did.

Cheers till next time,

Fuga

 

Leave a Comment