Update: Aug. 13, 12:32 p.m. – WolfieNet-Guest Wi-Fi Access Restored
WolfieNet-Guest is now operational after one of Stony Brook’s internet service providers was brought back online.
———
Update: Aug. 13, 11:30 a.m. – All Services Restored Except for WolfieNet-Guest
Stony Brook guests are currently unable to connect to the WolfieNet-Guest Wi-Fi network as part of a lingering effect of yesterday’s wide-spread Internet connectivity issues. DoIT’s Data Network Services team is actively working to bring this service back up.
Yesterday, Stony Brook University users may have noticed inconsistencies while browsing certain websites as well as six 2-3 minute outages while DoIT’s networking team worked with SB’s internet service providers (IPS’s) to restore services by switching over to SB’s secondary and tertiary network providers.
The initial problem was caused when a flood of new routes were advertised on the Internet. These new routes brought the number of total routes over 512k which is the routing table limit for many Internet routers, creating problems for most of the major ISP’s in North America. Once Stony Brook realized there were problems with its primary ISP, it switched over to its secondary ISP and when that did not remediate the problem, it then had to turn to its tertiary ISP. Switching to its tertiary ISP resolved most of the problems that end users were experiencing.
The remaining browsing problems people were still experiencing were triggered by the initial routing advertisement which cascaded down to the campus border router bringing its table beyond capacity and causing it to act erratically in routing traffic to the Internet. Once Stony Brook rebooted its hardware at 8:15 p.m. all traffic then began routing as expected.
Here is a further description of the root cause of yesterday’s issues and its impact across the Internet: http://www.zdnet.com/internet-hiccups-today-youre-not-alone-heres-why-7000032566/
———
Update: Aug. 12, 8:40 p.m. – All Services Restored
A flood of new networking routes overwhelmed internet service provider (ISP) routers, which caused multiple service disruptions throughout the day on Aug. 12. Stony Brook’s Data Network Services team, working with SB’s internet service providers, have since resolved all issues. All systems (DoIT websites, including the IT System Status page, SOLAR, Blackboard, as well as other Stony Brook sites) should now be fully accessible from both on and off campus.
———
Update: Aug. 12, 8:12 p.m. – Network Outage beginning 8:15 p.m.
Due to severe internet traffic issues today that are beyond our control, we are forced to take down our primary router for a period of 10-15 minutes starting at 8:15pm. This means people outside of the Stony Brook campus network will not be able to reach our websites. It also means people inside the Stony Brook network will not be able to access websites off campus. Users on campus should be able to reach most Stony Brook websites.
——-
Stony Brook University is currently experiencing network-based Internet access issues for both on and off campus users. Users have reported they have inconsistent access to DoIT web sites (including IT System Status), SOLAR, Blackboard, as well as other Stony Brook sites.
We are working closely with our internet service providers to resolve the problem. Status updates will be provided on the IT Status site for those who can access it (http://it.stonybrook.edu/
Cole W. Camplese
Vice President, Information Technology & CIO
Stony Brook University
Shame on all of you and your team!
This is an extreme inconvenience to those who do computational research at Stony Brook. The internet was painfully slow, we couldn’t ssh into various clusters (on campus and off campus) and the wireless was intermittent. I had to go home because I couldn’t work on campus. People in my group were frustrated and furious and we all hate Stony Brook because the internet and wifi is consistently bad, especially during the semester.
I hope you take ***drastic*** steps to prevent this from happening again.
This is completely unacceptable in 2014 at a research university like Stony Brook.
Hi Dan:
We feel your frustration. Believe it or not, thanks for leaving this comment. Please know that yesterday’s Internet connectivity issues went beyond Stony Brook and were largely out of our control.
We posted the following update that explains more in depth what actually occurred yesterday – http://it.stonybrook.edu/status/isp-routing-issue – Our networking team acted as quickly as they could to troubleshoot the issues and worked with Stony Brook’s internet service providers to restore service as soon as possible.
Here is a further description of the root cause of yesterday’s issues and its impact across the Internet: http://www.zdnet.com/internet-hiccups-today-youre-not-alone-heres-why-7000032566/
However, we would like to learn more from you about why you say the Internet and Wi-Fi is consistently bad. What building/room are you located in? Have you ever reported this to our Client Support Help Desk at (631) 632-9800 or supportteam@stonybrook.edu to make them aware of the situation and so they can alert our Networking Team to investigate further? Please get back to us so we can try to make things better in your area.
Sincerely,
Kerrin Perniciaro
Manager, IT Communications
Division of Information Technology
Stony Brook University