server -- delay -- isp -- mbox -- client

delay:
tc qdisc replace dev ${host}.l root netem delay 50ms limit 100000
tc qdisc replace dev ${host}.r root netem delay 50ms limit 100000

isp:
tc qdisc replace dev ${host}.l root handle 1:0 netem rate 900mbit limit 100000
tc qdisc replace dev ${host}.r root handle 1:0 netem rate 900mbit limit 100000
tc qdisc replace dev ${host}.r parent 1:1 handle 10:1 bfifo limit 1024k

I played with mbox's qdiscs (3 variations).

mbox:

tc qdisc replace dev ${host}.l root fq maxrate 890mbit

tc qdisc replace dev ${host}.r root fq maxrate 890mbit

rrul_be_nflows, 32 upstream, 32 downstream. In every case cake wins.

George