I think I’m ready to elevate from bonded 1G networking to 10GB. I tend to have used/old netgear equipment at home, purchased on the cheap. I think I’m finally seeing enough latency when trying to run OS’s on my proxmox server with storage on my TrueNAS that I want to move to 10G networking.
Looking to replace my existing 12-port L3 switch (6 ports are occupied) with either RJ-45 or SFP based 10G, I think I will need realistically 16 ports. Storage will specifically be moved from my 48 port L2 witch with 4-1G bonds, carrying storage and networking, to probably 2 10G bonds dedicated to storage, which currently means I need an additional 6 ports right now, so 12 total used, then planning for future expansion, 16 port minimum.
I’ve been crawling ebay, but the barrier to entry is way higher than my foray from unmanaged to managed networking. I keep landing on Netgear M4300, where the price ranges from about $1000-$2400. I can save up for these, but is there a better way to get into 10G? I’ve seen some HP Procurve switches that seem way more cost reasonable, but my networking peers have told me to completely stay away from HP networking.
Any thoughts are appreciated!
Check out the FS stuff. It’s pretty cheap brand new
https://www.fs.com/c/10g-enterprise-switches-3256
They SFPs are cheap and work very well too.
Agree, I’m pretty happy with their gear.
I’ve bought several products from FS (no switches but cables) and they are amazing quality
This. FS or there’s some shockingly good (looking) stuff on Aliexpress I’ve been considering and have heard good (perhaps sponsored) things about.
How does it compare to e.g., Netgear in terms of functionality and reliability?
I’ve only used their SFPs and DACs, which have been perfectly fine. Can’t speak to their networking equipment.
If you can live with only 4 10G ports, those types of switches are far cheaper than 16 port ones, I’ve gotten them for ~$100AU off eBay auctions before.
And if you’re fine with spending a bit more you could get a 8 port ubiquiti aggregation switch, and then upgrade to a bigger switch once the price comes down.
Mikrotik stuff is pretty good and quite inexpensive. They all run RouterOS, which is their linux distro for switches/routers. Fully managed and can do basically anything L2 or L3 you could want. Worth noting that availability and pricing has been pretty variable from their resllers for a while, and you will pay a premium via Amazon. Here’s a list of relevant product product pages with the port counts and MSRPs:
- A router with 12x SFP+ and 2x SFP-28/25g for $600, I run one of these as my router.
- A switch with 16x SFP+ for $500 Probably the right switch for you
- A switch with 24x SFP+ and 2xQSFP/40g for $600 I run one of these as my core switch
Also these may help for some corner cases or change your ideas on what you want to build:
- A tiny switch with 4x SFP+ for $150, I have a few of these where I want multiple things plugged in to 10g, but don’t want to run a cable for each of them as the devices aren’t usually using a full 10g at the same time.
- A switch with 8x SFP-28/25g and 2x QSFP-28/100g for $1000, interesting because each QSFP-28 can be broken out into 4x SFP-28, so a total of 16 25g ports. Just imagine making the jump to 25g instead of 10g, wouldn’t it be glorious?
- A switch with 48x 1GBASE-T, 4x SFP+, and 2x QSFP+ for $600 Who needs anything else? Break-out the QSFPs into 8 SFP+ ports, you’ve got 4 SFP+ ports natively, and then another 48 ports of 1g copper. Could be pretty interesting. There is also a variant with PoE output for $400 more in case you need to run cameras/APs/whatever and don’t want to run injectors.
- A super weird 2x SFP-28/25g network card that is actually a full router for $200 I haven’t figured out a use for these yet, but I kinda want one. Otherwise my SFP-28 ports are being wasted at just 10g, right?
- A switch with 16x SFP-28/25g and 2x QSFP-28/100g for $1600 Total overkill, but I just had to list it since it falls in the middle of the price range you were considering but is so much beefier than the Netgear offering.
DAC cables. Essentially copper with the transceiver’s on the end. Ideally, your ports are sfp, so you can decide to do fiber or copper. Gives you the most flexibility.
The Brocade ICX switches are very affordable for the performance. Great way to get a mix of 1G PoE, 10G, and 40G for under $200. Sadly they do consume a fair bit of power and most are loud out of the box. Although, there is a fan mod you can do to make them run quiet and cool.
Here is a link to more info: https://forums.servethehome.com/index.php?threads/brocade-icx-series-cheap-powerful-10gbe-40gbe-switching.21107/
I have an actual bedroom closet with a 45U rack and dedicated power and cooling, so noise/power are not a concern, I’ll look into it. How is the mgmt of them?
Management is mostly done through CLI. You need a generic console cable to get it on the network and then you can do everything remotely. There is a web ui as well, I mostly use it for quick status checks.
I have only used an ICX6610-48P-E. As this was my first non-Unifi managed switch, I spent a few hours setting it all up. Now, I just go back to the guides on ServeTheHome whenever I want to change VLANs or something complicated.
My home server is connected to the switch using a Mellanox ConnectX-3 and a QSFP DAC cable. It was about the same price as a 10G card and link so I figured why not go 40G to future proof it.
Do you want managed or unmanaged switches? I run some Microtik managed 10g switches and I can provide model numbers when I get home tonight.
Microtik has a 16 port SFP+ managed switch (CRS317-1G-16S+RM) for around $500 USD.
My personal setup is a CRS312-4C+8XG-RM which has 8 10G RJ45 and 4 SFP+/RJ45 Ports. You can usually get this switch for around $500-600 USD. For 10G into 1G devices I got a CRS326-24G-2S+RM which has 2 SFP+ 10G port into 24 Gig RJ45 ports.
Just as a warning, most managed switches assume you are using these in a data center or office; they can get quite loud. In the case of Microtik switches there is an ebay seller that offers quieter fans that you can replace inside the box (that void your warranty).
I went the middle road and settled for 2.5G - still a nice boost, and a lot cheaper. Might be worth a consideration if you can’t find cheap enough 10G gear.
Edit: Cost me around 300 dollars to upgrade my network. 2x 8 port switches, and 4x network cards.
Edit2: As a bonus, I could continue using the 1gbit devices directly plugged into the switch, since 2.5G use ethernet and the switches support 1g too.
