So I was going to upload a couple of very large wireshark dumps, but then I looked at something again and realized I was reading one of steam's poorly documented performance metrics wrong. It appears that more often than not now, my problem is actually host or client side on the cpu/encoding, so my network setup is good. I think their may have also been some client side network driver issues. It also seems that even slight alternate network usage can affect the stream (such as leaving a web browser open) all though this seems strange, I can't imagine how that could be impacted one way or another by the router. So thanks for the help. Looking forward to more improvements on your end.
--