In my lab, more than anything, quiet and power-sipping are the two factors that matter most to me. That means most of my servers are towers instead of rack-mount gear.

So when a recent post on /r/datahoarder outlined shoving 15 drives into a DVD duplicator tower, I jumped all over the idea.

One thing I didn’t want was to be forced to run multiple SFF-8088 cables. So early-on, I decided a SAS expander was needed.

Parts List

Copystars 9 Bay Duplicator case ($80 new on eBay)
HP SAS Expander ($18 used on eBay). This is the one with the SFF-8088 port on the back
Some random SAS->SATA breakout cables ($12 new for a pair on eBay)
SeaSonic M12II-620 EVO, 620W modular PSU ($60 new from Newegg)
A way to turn on the PSU without a motherboard. A bright orange switch ($7 on Amazon)
A pair of Rosewill 3 x 5.25-Inch to 4 x 3.5-Inch bays ($50 each from Newegg)
Powered PCIe riser ($10 Amazon)

Right away you might notice that I only ordered enough enclosures for eight drive bays. Initially, that was all that I needed. My long term plan was to be able to turn one of the 5.25″ bays into 6 or 8 2.5″ docks for SSDs.

I ordered the Rosewill bays because of the fans on them. Most of the other 3.5″ drive bays had VERY small fans on them. I’ve had some cooling problems in the past, and I wanted to make sure my new DAS didn’t have any issues.

The parts arrive

I feel like this project was doomed from the start. As the parts started to arrive, there were immediate problems:

That’s a surface-mount capacitor on the HP SAS Expander. It was floating around in the bag. While it probably would have worked, I wasn’t willing to risk data to it. So without a working SAS expander, which was going to take another week to arrive, about all I could do was assemble the rest of the case.

Grrr, the case…

So I don’t know why, but I ordered the Copystars case instead of the BestDuplicator case mentioned in the Datahoarders reddit post above. There is almost NO room in the Copystars case, whereas the BestDuplicator has room for days. The only nice thing about the Copystars case is that it has CONSIDERABLY more cooling in it, due to a pair of nice big (and relatively silent) fans.

While I had planned on needing to replace the PSU, the one that shipped with this Copystars case was truly dreadful:

300w, with 15A on the 12 volt rail. Nowhere near enough for even 8 drives.

Unfortunately, even with the stock PSU, I don’t think it would have been possible to fit the SAS Expander (which is HUGE). Nor would the final set of drive bays be available. So enter the velcro:

Need more parts!

While I was waiting for a new SAS expander, Seagate_Surfer on Reddit responded to a post I wrote about the potential pitfalls of buying Enterprise drives on Amazon. Due to the generosity of Seagate, I had some brand new 10TB drives on the way!!

I want to take this moment to shill for Seagate a little. I’ve bought TONS (in the hundreds) of drives over the years. For many of those years, I was exclusively buying Western Digital Reds and was constantly forced to RMA them at the 1.5 year mark. Upwards of 20% or more.

Since I started buying Seagates, I’ve had zero RMAs. Even the drives from the aforementioned post above were just starting to throw errors, they hadn’t even failed yet.

I really do feel Seagate took care of me by honoring a warranty that they were under absolutely no requirement to honor. They’ve got a customer for life in me, both for personal and in business.

Of course this meant I needed to fill in the third set of drive bays, so it was off to Amazon for another Rosewill kit.

I decided because of the limited SAS lanes available over a single cable, putting 2.5″ bays in this case for SSDs probably didn’t make sense. Because of that, I also picked up an SFF-8088 to SFF-8087 cable. With how I was required to mount the SAS expander in the case, my original plan of just a straight SFF-8088 cable wasn’t going to work.

A few days later, everything arrived and was mounted.

Cable gore:

Not perfect but it works:

I hate that this cable isn’t removable, but it’s what worked for now:

Doesn’t really look too bad:

At this point, I had gone through 2 Expanders, ordered an extra set of SFF-8087->8088 cables and an additional set of drive bays. And this project was well into its third week.

All of the failures

Once everything was installed, I immediately fired it up and started plugging in drives.

NOTHING

Nothing was detected. The lights on the drive bays came on, but OmniOS never registered drives as being connected. And of course I had sealed up the case because I blindly assumed everything would work.

After much plugging and replugging, I determined the SAS expander was bad. So it was back to eBay for a new SAS expander. This time I went with the IBM ServeRAID.

A week later, it arrived:

And was installed:

The design of this card makes it a bit more difficult to shove the third set of bays all the way in, but it fit (potato pic):

More Failures

With the new SAS expander, plugging in drives caused them to start showing up in OmniOS. But as with everything in this project, one step forward resulted in two steps back.

In two of the Rosewill enclosures, the third drive bay was bad and refused to connect. Note that one of these came from Newegg, and one of them came from Amazon, which means it’s probably a common problem with the enclosure. So while I continued to use them, I started the replacement process

The bigger concern was actually the apparent failure of one of my Seagate IronWolf drives. These things are built like TANKS and I had never seen problems before on DOZENS of them.

But ZFS was throwing checksum errors and smart tests started getting aborted:

The hard and transfer error counters were rising:

And the UDMA CRC Error count was going up and up

Some quick googling told me this was probably due to a bad cable, and sure enough, swapping the SATA connection from the dead drive bay to the drive throwing errors caused all the problems to go away.

I’ve found those breakout cables to be a little fragile. So I guess I’m not surprised that one of them failed in all my shuffling between replacing the expander and trying to troubleshoot why the third bays in two of the enclosures were bad.

So back to Amazon for another SAS->SATA breakout cable.

The Casualty List

I ordered more breakout cables (and ordered extras just to be safe), and Amazon has replaced the failed Rosewill enclosure. I’m now almost a month into this project, and while the DAS is stable and I’m using it, I’m still waiting on Newegg for a replacment for the final piece.

The list of failed parts so far: