RTFB.log

If you are running a Torrent, FTP or whatever kind of server behind your DSL link, you have probably noticed that interactive sessions get really slow (Web browsing, SSH…). As a first solution, you can of course configure your Torrent or FTP server to limit its upload but it’s not fully optimized as this wouldn’t use 100 % of your upload bandwidth.

(French version of this article)

Before introducing a solution, let’s briefly explain what is happening. Typical Ethernet modems handle packets with the same priority. Only IP packets with a specific “Type of Service” (ToS) benefit from a special treatment and are sent in priority. This allows services like VoIP to stream data in real-time.

The goal of this article is to show how a Linux router can be configured to control your upload bandwidth by setting up different priorities to the network traffic. This task is achieved by using the Linux traffic control functionality (tc).

Controlling your download traffic, while possible with TCP connections, is much more complicated. Indeed, you can’t access your ISP routers in order to set up your own configuration. Then, your only possibility is to drop received packets in order to decrease the speed of TCP connections. This works because the TCP protocol automatically adapts the speed of a session. This article doesn’t get into download bandwidth control but you should find interesting links in resources. Finally, remember that ISP are implementing QoS so you don’t have to worry about the priority of your VoIP/RTP/… connections.

Overview

In this article, I suppose a Linux box is already configured as a router:

Note: the IPv6 configuration is not covered in this article but it should be pretty similar.

In order to control the order in which packets are sent to the Internet, packets need to be queued and sorted at the Linux Box level:

By default, packets are queued at the modem level, that’s why this article firstly shows how to force the queuing to be done by the Linux box. Then, tc is mentioned to sort packets and iptables to mark them.

Force queuing at the Linux Box level

The first thing to do is to configure the Linux Box so that it fully controls the outgoing bandwidth. As already mentioned, outgoing traffic needs to be queued by the Linux Box and not by the DSL modem.

Unfortunately, the only way is to limit the Linux Box upload bandwidth so that the modem queue never gets filled (in fact, there might exist some modem allowing to set the size of the queue but I’m not even sure it will help).

To do so, Linux offers a Traffic Control service (tc). tc allows to define a tree of qdiscs (where packets are actually queued) and classes. A class contains one or several qdisc(s) and allows to define where/how packets are queued. In this article, two qdiscs are going to be used:

• HTB (Hierarchy Token Bucket): HTB allows to shape network traffic;
• SFQ (Stochastic Fairness Queueing): SFQ allows packets of each session to be send in turn (all sessions are then treated equally).

For more information, please read the man pages of tc, tc-htb and tc-sfq.

So, here are the commands to limit your upload to $RATEUP ($DEV == eth1):

ip link set dev $DEV qlen $QLEN mtu $MTU

tc qdisc add dev $DEV root handle 1: htb default 1
  tc class add dev $DEV parent 1: classid 1:1 htb rate ${RATEUP}kbit
    tc qdisc add dev $DEV parent 1:1 handle 10: sfq perturb 10

In a few words, these commands:

1. decrease the size of the queue and set the MTU to something more appropriate for the DSL connection ($QLEN=100 and $MTU=modem_value);
2. replace the default queue by a HTB qdisc;
3. add a class to the HTB qdisc to limit the bandwidth to ${RATEUP}kbit;
4. add a SFQ qdisc to this class.

Sort packets with tc

The HTB class system allows to define several queues (qdisc) with different priorities. This way, packets waiting to be sent are sorted and sent (the higher priority is 0):

tc qdisc add dev $DEV root handle 1: htb default 12
  tc class add dev $DEV parent 1: classid 1:1 htb rate ${RATEUP}kbit

    tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${RATEMIN}kbit \
      ceil ${RATEUP}kbit prio 0
    tc class add dev $DEV parent 1:1 classid 1:11 htb rate ${RATEMIN}kbit \
      ceil ${RATEUP}kbit prio 1
    tc class add dev $DEV parent 1:1 classid 1:12 htb rate ${RATEMIN}kbit \
      ceil ${RATEUP}kbit prio 2
    tc class add dev $DEV parent 1:1 classid 1:13 htb rate ${RATEMIN}kbit \
      ceil ${RATEUP}kbit prio 3

      tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
      tc qdisc add dev $DEV parent 1:11 handle 11: sfq perturb 10
      tc qdisc add dev $DEV parent 1:12 handle 12: sfq perturb 10 # def. queue
      tc qdisc add dev $DEV parent 1:13 handle 13: sfq perturb 10

tc filter add dev $DEV parent 1: prio 0 protocol ip handle 10 fw flowid 1:10
tc filter add dev $DEV parent 1: prio 0 protocol ip handle 11 fw flowid 1:11
tc filter add dev $DEV parent 1: prio 0 protocol ip handle 12 fw flowid 1:12
tc filter add dev $DEV parent 1: prio 0 protocol ip handle 13 fw flowid 1:13

This script creates 4 queues with priorities going from 0 to 3 and with a minimum rate ($RATEMIN). This minimum rate allows low priority applications to always have access to the Internet, that is applications won’t fail with a network timeout because another high priority application is using the whole bandwidth.

The queue defined with the classid 1:12 is the default one and is used if no tc filter command defines another behavior. In this example, tc filter commands allow to sort packets according to the tag previously set by iptables (then, untagged packets use the default qdisc). The next section explains how packets are tagged.

Mark packets with iptables

If you already know how to use iptables, marking packets is really easy. Rules must be added to the mangle table and to the POSTROUTING chain. Here is a small example which shows how packets could be sorted with the previous tc configuration:

# High priority for ping packets
iptables -t mangle -A POSTROUTING -p icmp -j MARK --set-mark 10

# Default for low port outgoing connections (clients)
iptables -t mangle -A POSTROUTING -p tcp --dport 0:1024 -j MARK --set-mark 11

# Default for low port ingoing connections (servers)
iptables -t mangle -A POSTROUTING -p tcp --sport 0:1024 -j MARK --set-mark 12

# High priority for DNS, SIP and ACK packets
iptables -t mangle -A POSTROUTING -p udp -j MARK --set-mark 10
iptables -t mangle -A POSTROUTING -p tcp -m length --length :64 -j MARK \
  --set-mark 10 # Small packets are generally ACKs
iptables -t mangle -A POSTROUTING -p tcp --dport sip -j MARK --set-mark 10
iptables -t mangle -A POSTROUTING -p tcp --dport sip-tls -j MARK --set-mark 10

# High priority for interactive sessions
iptables -t mangle -A POSTROUTING -p tcp --dport ssh -j MARK --set-mark 11
iptables -t mangle -A POSTROUTING -p tcp --sport ssh -j MARK --set-mark 11

# Low priority for FTP data and torrent outbound traffic
iptables -t mangle -A POSTROUTING -p tcp --sport MINPORT:MAXPORT -j MARK \
  --set-mark 14
iptables -t mangle -A POSTROUTING -p tcp --sport 6881:6999 -j MARK \
  --set-mark 14

The configuration is now finished but you need to run tests in order to determine the best value of $RATEUP (tc script). To do so, run a ping to a remote host and a program which uploads data. As long as the ping stays low, you can try to increase $RATEUP. If the ping is high, you need to decrease $RATEUP.

Conclusion

You can now impress your friends by keeping a small ping while running a lot of services behind your DSL connection :)

tc offers a lot more possibilities that can be found by reading man pages and links given in the Resources section. You can also download my tc-restore script which sets the tc configuration on a Debian like system.

Resources

• My tc-restore script
• man pages of tc, tc-htb, tc-sfq…
• http://lartc.org and especially http://lartc.org/howto/lartc.qdisc.html
• http://tldp.org/HOWTO/Traffic-Control-HOWTO/index.html