It is rather complex, you have to make the FIOS router act as a double bridge.
One bridge passes traffic from outside via COAX (ONT) to one RJ45 (CAT5) to WDR3800
Other bridge passes traffic from another WDR3800 port back to coax network to set top box

Does this help?