Remember that even if it was the ISP, it could be something from your side like poor connections or some kinda resistance. Is the connection a DSL variant? These connections are more prone to interference that other all-time connections. Try to not accuse them head on.
Typically Statistics under Status on the router page show such problems under errors. If it's in WLAN, it's highly possible the ISP.