I came up across that thread, that seems to have some good points both for migrating from Prom and against that.
And as some one who is considering a redesign of a montirong stack for my home lab, I wonder what would be opinion of wider community here?
Especially it would be great to get some opinion from people who did the switch (ideally in Production, but that doesn't matter) and what was good and/or bad about that decision?
Yeah, I have something like 12 karma or so, and the account is newly registered (a few days ago). Unfortunately, HN doesn't seems to provide any workarounds for that.
You can make it work with 2.5GbE and maybe even 10 GbE with some luck. But looking at its current state and price I personally would agree that fiber is more future proof.
There is a more concrete example - in the article I've showed iperf3 output, which gave 23.5 Gbit/s - that is actual data. Raw speed was ~24.9-25. Everything else is overhead in that case.
At some point in my life, I was forced to use 7/0.7 Mbps ADSL after having 50/50 Mbps and that ADSL was very oversubscribed (high packet loss and huge speed drops to <2 Mbps during weekends or evenings). Later on, I had FTTH, but I couldn't control the router (fibre was terminating at the landlord's house, and I had cat5e from the router to the place where I lived, and the landlord kept the router in the worst place possible so it was overheating during the summer).
Since then, I have tried to be in control of network hardware, and I now avoid *DSL, even though VDSL is decent nowadays. But the reason for 25 Gbps for me is "just because", as ISP charges the same monthly amount as I pay now (there is a setup fee for 25, though).
The main application here is to force myself into solving those problems :) I have plans on what I'll do after. As with a 25 Gbps router comes a 25 Gbps home NAS, then I would want to have decent latency for packet processing and I hope all of that would force me to do something interesting :)
True, but in my case, reliability and even price (per month) of going to 10 Gbps vs. 25 Gbps is the same.
And what I see as a use case - is when multiple people in my household would do heavy downloads simultaneously. Also, I hope I will get better-downloading speed (Steam, GoG, etc.) as with modern games, it becomes more critical, and I'll be keener to play and remove a game if I want to put in on pause, compared to keeping 3-4 100+ GB games installed at the same time.
But about your problem - cards manage to negotiate 5 Gbps, or even negotiation fails? If it's the latter, then it might be cables.
Overall, for me, it works to simplify the network as much as possible and first achieve results on a direct connection to the router when it's nearby and then add more and more.
Oh, and also measure as much as possible (that's why I'm sad about the state of system metrics on OSX, for example)
The cool thing about fibre is that it is way more future-proof than good old copper.
By the way, I also went for a Thunderbolt enclosure and network card for Mac. It is a bit pricy (the enclosure costs way more than yet another ConnectX).
Well... I did more than I've described in the post (I'm the article's author). It just became too large, so I've decided to stop and publish a second part once I run more tests and fix some problems.
So far, I have yet to figure out all the problems and settings with Netronome (or give up on it). Still, it is Netronome-specific as I can easily get line rate over TCP single stream with Mellanox cards, and the CPU doesn't even break a sweat (however, I have relatively simple routing rules).
I want to try out programmable NIC, and as far as I understand, Netronome has some support there; I just wanted to get a simple setup running first and then optimise/improve it. I tend to perfect the setup, and if I let myself do more and more and more - I will always need more time to finish it.
I still need to measure and optimise power consumption correctly, but it drew about 70W from the socket during the tests, according to my simple power meter. CPU never ran at boost frequency, and a lot of other stuff is in a low power state.
After reading your article, I've started to think if I want to get myself a splicer :D Before, I felt that field connectors would work just fine and splices were too expensive...
With the size of recent games and patches, 1 Gbit starts to feel a bit too low. :)
And about Macbook... In my current setup, I'm using Thunderbolt -> PCIe and one of the Mellanox cards; it gives excellent and stable 16 Gbps with stock firmware and DriverKit-based drivers (that comes with OSX), though it feels that DriverKit driver doesn't use any offloading that it supports (while running iperf between Macbook and router, it consumes roughly 3 cores entirely on Macbook, while the router doesn't even ramp up the single core to its boost frequency).
But you are right; rewiring is a hassle, though Cat5e usually works with 10Gbit over reasonable lengths (10-30 meters, maybe a bit more if it is perfect quality cable).
Also, even if I decided not to go for fibre, I'd still switch to at least 10Gbps because I'm not alone in that network :)
True, in terms of price, duplex is cheaper (though compared to overall router cost that is negligible). In my case, I wanted to be able to swap SFPs if input one dies (for any reason), and it doesn't make a lot of sense to buy different modules for different purposes, even if that would save a bit of money.
There is another difference - if I want to go for field connectors and cut it myself, the Simplex connector would cost about 4.15$, and the Duplex would be 6.16$, which would further increase the price difference.
But thanks for the advice, it is indeed a viable option.
As one of the maintainers of go-graphite stack, I'd really like to know more about what didn't work for Gramnarly. Based on values they've provided I can deduce that it was carbonapi + go-carbon, though I wonder if they've considered migrating to graphite-clickhouse. I understand that now it is too late, but maybe more details would help to fix same problem for other people.
P.S. It is also nice to see that feature that I've added long time ago just to have simpler dev setup for myself (vm-backend support) is actually used by people, even though it required few fixes :) (or to be precise - used at least at some point, it is not very clear if they've migrated to built-in one)
And as some one who is considering a redesign of a montirong stack for my home lab, I wonder what would be opinion of wider community here?
Especially it would be great to get some opinion from people who did the switch (ideally in Production, but that doesn't matter) and what was good and/or bad about that decision?