I am constantly on the hunt for new switches.  Between finding the right combination of power consumption, noise, and 10Gb, it seems like there is always something new on the horizon.

In business, for years my go-to switch has been the Cisco 2960x.  While I often hear "Nobody ever got fired for buying Cisco", my relationship with these 2960x switches has been tumultuous to say the least.

They seem prone to rather unique hardware failures.  And it always seems to run fine until either a firmware upgrade, or even just a simple reload on the switch.  The latest one occurred when I tried to bring home a de-commed 2960x unit, and I was presented with an error message basically suggesting I had stolen the switch or the firmware.

Of course that is false.  The stupid switch was covered under SmartNET until about a year ago, and the firmware is a universal image.  And it had been running fine until the previous day.  And dealing with the Cisco TAC on an uncovered switch... is just not fun, as it seems the remedy to this particular problem is a replacement.

I still needed a switch at home (I am running out of ports), and more importantly, I need to replace some stacks of switches in data-centers that look a little like this:

Yes.  That's terrible, but I'm scared as hell to touch them.  As mentioned above, these switches like to fail, and they tend to do it on reboots.  

So I started going down the list of vendors to use:

• Cisco.  Well, I'm a bit burned on them for this level of switch.  Plus, if I can avoid dropping $4-6000 per switch stack, that would be okay.
• Unifi.  Eww
• Mikrotik.  I like these switches, but no stacking and not enough ports.
• HP/Juniper/etc.  Somewhat of the same problem as Cisco.  I'm not saying I'm trying to cheap-out, but if I could get some quality gear and not drop thousands that would be a bonus.

Enter the S3900-48T4S

I think anybody that is even peripherally in the Networking world has probably heard about FS.com.  They are a bit of a knee-jerk reaction when the question is asked "where do I get cheap and guaranteed compatible optics, DACs, fiber, etc".  

They also have some switches.  Some run Cumulus Linux, while others run their own custom operating system.  My problem, and why I had never considered them before was pretty simple.  Perception.

You see, around Reddit from time-to-time these switches get recommended.  But the recommendations are always just tossed out there at random, by accounts with little or no history other than recommending the switches.  Point being, the recommendations come across as VERY paid-shill.

After a few emails with the FS rep, I decided to take a chance and order a switch for testing.  It came from Seattle, so four days later (should have been three, thanks FedEx), it arrived, a little beat up but generally safe:

I tore into the box and found everything one would expect with a new switch.  Rack ears, a grounding wire, DB9->RJ45, and a pair of power cables:

The nitty gritty details

The switch itself, from a fashion perspective, is actually somewhat boring.  A lot of my gear is Mikrotik, so I'm used to a bit of aesthetic.  But let's face it, I'm here for function, not form.  Otherwise, this switch is fairly comparable to the 2960x switches I've been using.

• $410 shipped to my door. That's a pretty great price for a new, stackable, silent, power-sipping switch.
• 48 GbE RJ45 ports.
• 4xSFP+ 10GbE ports.
• RJ45 console. Used my regular old Cisco rollover here.
• Dual power-supplies.
• Stackable

While eventually I will be stacking, I'm not currently.  It turns out the stacking on this switch just uses the last two SFP+ ports.  Far from a deal-breaker, but I like the idea of dedicated stacking modules so you have access to all four SFP+ ports.  Obviously that would have bumped up the price a little.

Some details when it's running:

• Other than a few seconds at startup, basically silent.
• 40 watts measured with a Kill-a-watt. It seems to be fairly stable no matter the load, but I haven't stress-tested it too much yet. The dual PSUs seem to run in a main/backup configuration. The main pulls around 39 watts. The backup PSU runs just at a single watt.
• Boots and is passing data in around a minute. Let's just say that's not the case with the 2960x switches I've been using.

Software and Configuration

To be honest, the configuration side of things was where I was a bit worried.  With Cisco's IOS, anything you might want to do is a simple web search away.  As I've discovered, the S3900-48T4S runs FSOS, which for all practical purposes, might as well be IOS.  

This switch also supports L2+, meaning it can handle basic routing tasks.  The 2960x was limited to 16 static routes, and I couldn't figure out how many this switch handles.  But with summary routes, it's usually a bit of a moot point for how I manage routing.  

The original firmware on the switch was a bit wonky.  For example, it was configure instead of configure terminal or conf t like someone might be accustomed to.  There were a number of other oddities like that, that upgrading the firmware to the latest version seemed to resolve.  The original firmware seemed to be for a S5600, which means under the hood, all these switches are probably the same:

Quirks that I've run into

• SSH access requires DSA. Just eww. I talked to support, and it sounds like it might be something that can be fixed. The end result was something like this in my .ssh/config file for the host:

      KexAlgorithms diffie-hellman-group1-sha1
      Ciphers aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
      HostKeyAlgorithms= ssh-dss

• You don't need to add VLANs to the local VLAN database first like you would on IOS. Adding tagged and untagged VLANs to interfaces will automatically manage this. This was a surprise (and somewhat annoying) until I noticed what was happening. Stuff in the VLAN database would automatically change without me doing anything. I consider this a huge quality-of-life improvement after I figured out what it was doing.
• Interacting with Port-channels automatically modifies the underlying interfaces. For example, on this config for a LACP trunk, other than the channel-group config, all these commands were originally entered on the interface port-channel 1:

    interface ethernet 1/51
     no negotiation
     switchport mtu 9216
     switchport mode trunk
     switchport allowed vlan add 1-4000 tagged
     channel-group 1 mode auto
    !
    interface ethernet 1/52
     no negotiation
     switchport mtu 9216
     switchport mode trunk
     switchport allowed vlan add 1-4000 tagged
     channel-group 1 mode auto
    !
    interface port-channel 1
    !

Again, that could be something that might be considered a quality-of-life improvement.

The switch also has what I would consider to be a fairly competent HTML GUI.  The comparable WebUI on a 2960X is generally considered to be a bit of a joke, but the one on this switch is usable.  HTTPs management with a custom SSL certificate.

Conclusion

So far, I'm pretty satisfied with this switch.  Any questions I've had or problems I've run into have been answered promptly by FS's support.  At this point I'll continue stress testing it and playing with different configurations to decide if it's worth replacing a bunch of Cisco with them.