Best routers out there ?

[Monkeh]

It does support link aggregation, so you can get ~2Gbps.

This assumes that you're dealing with multiple streams, and that the CPU port is greater than 1Gbps, which is questionable.

[NiHaoMike]

Having said that, there are genuine uses for >1Gbps Wi-Fi capability, but that is generally reserved for many clients on enterprise radios with aggregated Ethernet links or 10 Gbps backhaul. You won't see that in a consumer product.

At least until wireless VR takes off and creates a demand for high resolution, low latency wireless video.

[madires]

It's wifi, the number on the box is always a lie..

... by a factor of two. For WiFi the net throughput is about the half of the gross throughput advertised (per band).

[Monkeh]

It's wifi, the number on the box is always a lie..

... by a factor of two. For WiFi the net throughput is about the half of the gross throughput advertised (per band).

At best.

[Halcyon]

It does support link aggregation, so you can get ~2Gbps.

Link aggregation only serves to increase the total amount of bandwidth shared among all clients. It won't increase the bandwidth to/from a single client device, you're still limited by the speed of a single port (in this case 1 Gbps).

This is a common assumption people often make when it comes to aggregating NICs. To put it another way, let's say you had a desktop PC and a NAS, both with 2x 1 Gbps NICs aggregated. The transfer speed between those two devices would still only be 1 Gbps max, however it means that the NAS still has 1 Gbps "spare" capacity to service other clients on  the network.

Having said that, there are genuine uses for >1Gbps Wi-Fi capability, but that is generally reserved for many clients on enterprise radios with aggregated Ethernet links or 10 Gbps backhaul. You won't see that in a consumer product.

At least until wireless VR takes off and creates a demand for high resolution, low latency wireless video.

Latency on current 802.11 networks is negligible if your wireless access points are decent and you've properly planned and built your network to suit your RF environment. Even when 802.11ax becomes the norm, almost no one will notice a difference. I can see benefits of 802.11ax, but not until 10 Gbit Ethernet becomes standard in homes.

[David Hess]

It does support link aggregation, so you can get ~2Gbps.

Link aggregation only serves to increase the total amount of bandwidth shared among all clients. It won't increase the bandwidth to/from a single client device, you're still limited by the speed of a single port (in this case 1 Gbps).

This is a common assumption people often make when it comes to aggregating NICs. To put it another way, let's say you had a desktop PC and a NAS, both with 2x 1 Gbps NICs aggregated. The transfer speed between those two devices would still only be 1 Gbps max, however it means that the NAS still has 1 Gbps "spare" capacity to service other clients on  the network.

Round-robin link aggregation will increase the throughput of a single connection but it is not generally supported because it is really hard on the TCP/IP stack due to packet reordering.  Linux supports it natively but Windows only supports it as part of the network driver if at all.

[NiHaoMike]

Latency on current 802.11 networks is negligible if your wireless access points are decent and you've properly planned and built your network to suit your RF environment. Even when 802.11ax becomes the norm, almost no one will notice a difference. I can see benefits of 802.11ax, but not until 10 Gbit Ethernet becomes standard in homes.

The low latency target for VR means a lot of bandwidth since heavy video compression is no longer possible, easily many Gbps. Either consumer 802.11ay APs will get USB-C ports to plug into GPUs or some arrangement will be made to connect GPUs to SFP sockets - probably adapters to convert USB-C to SFP at first and GPUs getting SFP sockets built in later on.

[Halcyon]

Latency on current 802.11 networks is negligible if your wireless access points are decent and you've properly planned and built your network to suit your RF environment. Even when 802.11ax becomes the norm, almost no one will notice a difference. I can see benefits of 802.11ax, but not until 10 Gbit Ethernet becomes standard in homes.

The low latency target for VR means a lot of bandwidth since heavy video compression is no longer possible, easily many Gbps. Either consumer 802.11ay APs will get USB-C ports to plug into GPUs or some arrangement will be made to connect GPUs to SFP sockets - probably adapters to convert USB-C to SFP at first and GPUs getting SFP sockets built in later on.

I think it's important to remember that latency and throughput are two different things. You can still have fast links with high latency (and the reverse is also true). I also think that this is "pie in the sky" thinking (at least for the moment). When you're talking traversing data links across continents, the latency of a good wireless connection is almost negligible, typically under 10 milliseconds.

[Jeroen3]

The latency of a fedex truck with hard drives is very high, but the throughput is also very high.

So... if WiFi does theoretically 5 Gbps, when connected to only 802.11ax clients, does that mean you can serve 4 clients on 1 Gbps to LAN half duplex?
Or will you hit the limit of the processor?

[Zucca]

Link aggregation only serves to increase the total amount of bandwidth shared among all clients. It won't increase the bandwidth to/from a single client device, you're still limited by the speed of a single port (in this case 1 Gbps).

This is a common assumption people often make when it comes to aggregating NICs. To put it another way, let's say you had a desktop PC and a NAS, both with 2x 1 Gbps NICs aggregated. The transfer speed between those two devices would still only be 1 Gbps max, however it means that the NAS still has 1 Gbps "spare" capacity to service other clients on  the network.

Thanks Halcyon for the information and to moderate this new forum branch, I am loving it so far.
Back to the topic, so those LACP:

My setup is as follows, and it works well:

Internet
| (1gig fiber)
Asus RT-AC88U (Running Asus Merlin firmware)
| (2gig LACP link aggregation)
24 port PoE switch
| (4gig LACP link aggregation)
Synology RS815+ NAS

are not increasing the bandwidth because are 1:1 devices connections? Or the magic will happens when one IP/device has to communicate with several differents IP addresses/devices through i.e. switches?

[Jeroen3]

Ethernet keeps frame order. The maximum speed between two points should then be the slowest physical link, which is one of the bonded links.
However, mikrotik offers a custom one: balance-rr, that does allow one stream to be split up. Messing with the order, putting a higher load on the clients reassembling the data. No problem for UDP though.
So pairing is a bad idea for clients, good for backhaul.

Wait for MPTCP?

[madires]

Link aggregation only serves to increase the total amount of bandwidth shared among all clients. It won't increase the bandwidth to/from a single client device, you're still limited by the speed of a single port (in this case 1 Gbps).

Sorry, but I have to disagree. The way the traffic is distributed across multiple links is determined by the load sharing algorithm implemented. In the past there were quite simple methods which could cause the scenario you've described. But today's algorithms are more sophisticated and consider several parameters to optimize load sharing. Several network elements offer fine tuning by configuration, i.e. you can tell the device which parameters to use. Of course, if you have a server supporting only some poor algorithm it won't be able to utilize multiple links properly and would create an asymmetrical load sharing with the LAN switch. Another thing to know is that even the most sophisticated algorithms aren't able to utilize all links 100%, but they do a decent good job.

[madires]

are not increasing the bandwidth because are 1:1 devices connections? Or the magic will happens when one IP/device has to communicate with several differents IP addresses/devices through i.e. switches?

You have to look into how each device handles the load sharing across the LACP links. There's no simple answer like yes or no.

[free_electron]

My system is running perfectly fine . Not a single hiccup.

Cabinet shows the entire setup. It is an IKEA KALLAX cabinet , single column, two stacked on top of each other. (one is modified to shorten it by one compartment)
The bottom compartment is a set of drawers that hold spare ethernet cable,s usb cables, power cables etc.
The slot above that holds a UPS to power the cabinet.
Above that are the four old NAS machines (routernas.jpg). They each hold two 2TB drives and a DVD burner. One has a Blu-Ray burner.
Above that is the Amplifi router in a slot by itself.
Above that are two NAS boxes each holding two 4TB drives with two blank slots ( they can hold 4 drives each)
The bay above that has two western Digital NAS machines. The white one is 2TB , the black one has two 2TB drives in it. There is room to bring one up from the garage ( a 1TB)
The last bay has the Apple Time Capsule and the Home automation ( not everything is there yet. I will move the other boxes from their various locations into this cabinet.) and one more Switch that serves this cabinet.

When all data is moved the LG NAS machines will essentially become 'cold storage'. They will serve as backup for the two main NAS machines. (once a month they sync in overwrite/prune mode). I use a piece of software called ViceVersa that is installed on the laptop running the NVR. The drives are powered off and the NAS responds to wake-on-lan from the laptop.
The main NAS machines already mirror each other.
The White WD will become a cloud drive.(Accessible from the big bad internet. It doesn't hold anything important. If it gets hacked ,or stuff stolen from it : no loss.)
The Black WD will replace the Apple Time Capsule. That thing is at the end of its life. It only has a 1TB drive and it is full ... My Mac has a 2TB drive with 850GB used. So there is not much 'time machining' to be done with a 1TB drive.
The Mac holds all the family pictures / video's etc. It is only used for that purpose.

I have 4 copies of everything.
- 1 on the computer
- 2 on the main nas boxes ( 1 per nas box, they clone each other )
- 1 in cold storage on the old nas

why ? because hard disks are essentially spinning rust that can and will die at the most unexpected point , when you need it the most. I worked in harddisks for 10 years of my life. i know what can go wrong.
and SSD is not any better. Contrary, it is worse ... if a harddisk goes bad you can at least attempt data recovery by swapping boards or headstacks. You have a 80 to 90 % chance it is recoverable. The only really unrecoverable is those glass harddisks (laptops) . if they shatter : game over.
The one below took 4 years to recover , but they got the data off...
Not bad considering it came in falling from space when the Space Shuttle Columbia burned up.
[url]https://gizmodo.com/charred-hard-drive-from-space-shuttle-columbia-recovere-388465[url]

If an SSD goes bad .. essentially kiss your data goodbye. There is no swapping of a fried board or headstack... a fried chip is a fried chip. if a flash chip no longer talks there is no (cost-effective. you could try e-beam probing to read bits , but then restoring that to real working data is another thing due to wear leveling algorithms . and in case of three level storage , or stacked dies it becomes a real nightmare...) way to get data out.

oh, and before you say : what if the house burns down ? I do have a fifth backup 'chez apple icloud'

[Halcyon]

Ethernet keeps frame order. The maximum speed between two points should then be the slowest physical link, which is one of the bonded links.
However, mikrotik offers a custom one: balance-rr, that does allow one stream to be split up. Messing with the order, putting a higher load on the clients reassembling the data. No problem for UDP though.
So pairing is a bad idea for clients, good for backhaul.

Wait for MPTCP?

Correct, splitting frames and taking them out of order essentially breaks the "fabric" of Ethernet, but as you and David Hess also suggested, there are some tricky ways around this but it involves both hardware and software support. More often than not (even on Enterprise gear) you'll be using LACP or similar link aggregation.

Sorry, but I have to disagree. The way the traffic is distributed across multiple links is determined by the load sharing algorithm implemented. In the past there were quite simple methods which could cause the scenario you've described. But today's algorithms are more sophisticated and consider several parameters to optimize load sharing. Several network elements offer fine tuning by configuration, i.e. you can tell the device which parameters to use. Of course, if you have a server supporting only some poor algorithm it won't be able to utilize multiple links properly and would create an asymmetrical load sharing with the LAN switch. Another thing to know is that even the most sophisticated algorithms aren't able to utilize all links 100%, but they do a decent good job.

That's right, but you have to consider how it shares that load, usually it's based off the client's IP and/or MAC address. When you aggregate ports, you're creating a virtual adapter which has it's own single unique MAC address. You're fundamentally limited by how Ethernet works, if you have frames coming in on an interface out of order, things break. As I mentioned, there are some special ways to get around this, but it's not often done. In the Enterprise world, you just upgrade your networking hardware to 10 Gbps or more and not mess around with this.

Link aggregation is generally used for redundancy or increasing link capacity to multiple clients. That's what it was designed for.

But don't take it from me, do a test yourself on your own NAS or between desktop machines, you'll see exactly what I mean.

[free_electron]

Correct, splitting frames and taking them out of order essentially breaks the "fabric" of Ethernet, but as you and David Hess also suggested, there are some tricky ways around this but it involves both hardware and software support. More often than not

wait ? what ?

Ethernet doesn't care as this is handled on individual protocol levels.

a TCP packet has a sequence number that allows you to reconstruct the payload if you get out-of-order packets or, if you get duplicate packets when a retransmit was requested and the original packet arrives.

[David Hess]

Correct, splitting frames and taking them out of order essentially breaks the "fabric" of Ethernet, but as you and David Hess also suggested, there are some tricky ways around this but it involves both hardware and software support. More often than not

wait ? what ?

Ethernet doesn't care as this is handled on individual protocol levels.

a TCP packet has a sequence number that allows you to reconstruct the payload if you get out-of-order packets or, if you get duplicate packets when a retransmit was requested and the original packet arrives.

It only requires software support.  Ethernet will happily deliver the packets but the problem is on the receiving end where the TCP part of the IP stack has to put the packets back into the correct order which is very demanding on the processor at high data rates.  Windows does not even bother to support it natively so the ethernet card driver has to handle it.  But it is built into Linux and I assume BSD.

[NiHaoMike]

I think it's important to remember that latency and throughput are two different things. You can still have fast links with high latency (and the reverse is also true). I also think that this is "pie in the sky" thinking (at least for the moment). When you're talking traversing data links across continents, the latency of a good wireless connection is almost negligible, typically under 10 milliseconds.

Many of the tricks used for video compression are not compatible with low latency, plus there's not much that can be done to reduce the bitrate of sudden scene changes and still have low latency. If streaming VR (PC to headset) could be handled by 802.11ac, they would have already done it...

[Halcyon]

Correct, splitting frames and taking them out of order essentially breaks the "fabric" of Ethernet, but as you and David Hess also suggested, there are some tricky ways around this but it involves both hardware and software support. More often than not

wait ? what ?

Ethernet doesn't care as this is handled on individual protocol levels.

a TCP packet has a sequence number that allows you to reconstruct the payload if you get out-of-order packets or, if you get duplicate packets when a retransmit was requested and the original packet arrives.

Sorry, I should have been more specific.

TCP can handle the odd out of order packet or two, but if it's consistent, you'll start getting retransmissions occurring. This is something you don't want to happen because it impacts performance.

Although this is starting to get outside the scope of normal residential or low-end commercial link aggregation.

I think it's important to remember that latency and throughput are two different things. You can still have fast links with high latency (and the reverse is also true). I also think that this is "pie in the sky" thinking (at least for the moment). When you're talking traversing data links across continents, the latency of a good wireless connection is almost negligible, typically under 10 milliseconds.

Many of the tricks used for video compression are not compatible with low latency, plus there's not much that can be done to reduce the bitrate of sudden scene changes and still have low latency. If streaming VR (PC to headset) could be handled by 802.11ac, they would have already done it...

What do you mean? A quick Google reveals a bunch of 802.11 VR headsets. It appears Wi-Fi is very much able to handle the data rates and latency.

[Berni]

WiFi speeds do have to be taken with a grain of salt.

With the faster wifi standards its often required to have multiple clients in order to get that speed, additionally the clients might need to support simultaneously using multiple channels. Router manufacturers also tend to have a habit of summing together the bandwidth of multiple modes. such as 2.4GHz and 5Ghz and advertising the sum of the speeds.

And even when you have all the ducks in a row the theoretical speeds never seam to be reached even if the devices are just a meter apart. The speed then degrades even further when there are other networks shouting over its channel spectrum and degrades even further once you put a wall or two in between you and the access point. But not only does the speed drop off a lot, you also start getting lost packets, this takes time for the wifi interfaces to notice and retransmit, so you get big spikes in the latency and that mess up anything that needs a good consistent ping time like video calls, multiplayer games, streaming etc... or god forbid live game streaming.

[madires]

That's right, but you have to consider how it shares that load, usually it's based off the client's IP and/or MAC address. When you aggregate ports, you're creating a virtual adapter which has it's own single unique MAC address. You're fundamentally limited by how Ethernet works, if you have frames coming in on an interface out of order, things break. As I mentioned, there are some special ways to get around this, but it's not often done. In the Enterprise world, you just upgrade your networking hardware to 10 Gbps or more and not mess around with this.

Most non-SOHO switch ASICs use load balancing algorithms based on L3 or L4 hashes. IIRC, linux comes with four different load balancing methods to choose from. There are also adaptive algorithms, e.g. Juniper. In the enterprise world load balancing works really well because of the large number of different clients/servers and sessions. And when upgrading the throughput of your network one key point is cost. If LAG is less expensive then moving to the next speed level, why pay more?

Back to the NAS example with a LAG of two GigEs for NAS and PC to explain load balancing. If the PC establishes just one single TCP connection with the NAS the traffic will go through one GigE because the load balancing algorithm has only one hash from that single TCP connection to decide which GigE to use. When the PC establishes a second connection with the NAS we have two TCP sessions creating two hashes for load balancing. Now the load balancing algorithm can send traffic for session #1 via GigE #1 and traffic for session #2 via GigE #2 controlled by the hashes.

[Jeroen3]

Correct, splitting frames and taking them out of order essentially breaks the "fabric" of Ethernet, but as you and David Hess also suggested, there are some tricky ways around this but it involves both hardware and software support. More often than not

wait ? what ?

Ethernet doesn't care as this is handled on individual protocol levels.

a TCP packet has a sequence number that allows you to reconstruct the payload if you get out-of-order packets or, if you get duplicate packets when a retransmit was requested and the original packet arrives.

You're mixing the layers. Ethernet is L2, TCP is L4.
Ethernet isn't supposed to change the order of frames, FIFO. TCP allows for fragmentation of packets on L3.

Since SMB multichannel allows multiple stream the balancing per stream should work fine. If your NAS supports it.

[NiHaoMike]

What do you mean? A quick Google reveals a bunch of 802.11 VR headsets. It appears Wi-Fi is very much able to handle the data rates and latency.

There are VR headsets with built in processing, but those don't match the realism offered by a full PC.

[free_electron]

Correct, splitting frames and taking them out of order essentially breaks the "fabric" of Ethernet, but as you and David Hess also suggested, there are some tricky ways around this but it involves both hardware and software support. More often than not

wait ? what ?

Ethernet doesn't care as this is handled on individual protocol levels.

a TCP packet has a sequence number that allows you to reconstruct the payload if you get out-of-order packets or, if you get duplicate packets when a retransmit was requested and the original packet arrives.

You're mixing the layers. Ethernet is L2, TCP is L4.
Ethernet isn't supposed to change the order of frames, FIFO. TCP allows for fragmentation of packets on L3.

Since SMB multichannel allows multiple stream the balancing per stream should work fine. If your NAS supports it.

Ethernet is basically fire and forget. Packets will be blasted out in the order they are received from the OS. They will be in sequence , but they may be a mix of multiple streams.
Ethernet itself has no handshake or retransmit mechanism. That is handled on protocol level.

How packets travel through various switches, routers , fiber optic, undersea , satellite is an unknown. That is where stuff may go out of order. So by the time it arrives somewhere the packet order is no longer guaranteed. It is up to information contained in the payload to make sure the entire message can be reconstructed and packets can be put back in order.

[Halcyon]

WiFi speeds do have to be taken with a grain of salt.

100%. A general rule of thumb I use is half the maximum speed with a good set up (that means decent hardware, good signal strength and someone who knows how to configure it). On consumer gear, I often find that number far less than half.