INC0341032 | WiscNet | Multiple sites Impacted | PacketLoss
Incident Report for WiscNet
Resolved
The widespread issues with packet loss have been identified and resolved, with the exception of a smaller subset of members which will be tracked on a separate WiscNet page: https://status.wiscnet.net/incidents/t7408mmn5gmq.

WiscNet understands the root cause to be: broadcast loops were identified on different locations within the AT&T and TEACH networks borders. The broadcast loops were caused by migrated sites left with two active circuits (BadgerNet and TEACH). At this point, we will close this Status Page. If you are experiencing problems, please contact WiscNet technical support at 608-442-6761, option 2 or email us at support@wiscnet.net. Thank you for your patience!
Posted Apr 09, 2024 - 11:54 CDT
Update
The investigating regarding packet loss continues. WiscNet sent a list of members who reported issues to the TEACH Network operations manager for the next steps of troubleshooting to start, possibly at the site level. If you are experiencing problems please contact us at support@wiscnet.net or call us at 608-442-6761, option two to report your issues. Again, the packet loss is sporadic, it happens at different levels, at different times of the day, across all members who use BadgerNet or TEACH to connect to WiscNet. Some see impact, others don't. All parties are working diligently to find the root cause and fix for this issue. Thank you for your patience.
Posted Apr 05, 2024 - 12:16 CDT
Update
We are continuing to investigate this issue.
Posted Apr 01, 2024 - 16:43 CDT
Update
WiscNet Engineers continue to see packet loss for multiple members that use the TEACH and BadgerNet transport networks to connect to our backbone. For the last two days, GLS and AT&T technicians have been troubleshooting an outage that impacts a subset of the TEACH network users, and in the process are keeping the more widespread issue of packet loss in mind. We will continue to provide updates on a regular basis until this issue is resolved. Please contact us at support@wiscnet.net or call us at 608-442-6761, option 2.
Posted Mar 29, 2024 - 11:40 CDT
Update
Our partners are further investigating how configuration rules are handling traffic as it transverses the networks. This issue doesn't appear to significantly impact performance for the majority of our members at this point. We will continue to provide updates until the packet loss stops. Again, if you are experiencing issues, please contact our Support Team at support@wiscnet.net or all us at 608-442-6761 option 2.
Posted Mar 26, 2024 - 14:08 CDT
Update
The investigations continue. If you are experience any issues and have not reported them to our support team, please do do by sending an email to support@wiscnet.net or by calling us at 608-442-6761, option 2. If outside business hours, please call our NOC at 608-442-6761, option 1
Posted Mar 22, 2024 - 09:17 CDT
Update
We received update that AT&T engineers plan on doing testing throughout the night to further investigate the packet loss issue. We will continue to post updates to this page as we hear from our partners. Thank you for your patience.
Posted Mar 20, 2024 - 14:03 CDT
Update
WiscNet apologizes for the delay, We continue to work with our partners in identifying the cause of the widespread packet loss. WiscNet engineers have provided logs which are being reviewed. We will provide updates as we get them. Thank you

We appreciate you understanding in this matter, Thank you for your patience.

WiscNet
Posted Mar 19, 2024 - 16:45 CDT
Update
Unfortunately, this issue remains under investigation. WiscNet has provided logs of our monitoring system to our partners and have opened a general trouble ticket to track the update on the investigation of this widespread issue. The update as off minutes ago is that GLS has escalated the ticket to engineers.

What we know: the packet loss affects different sites at different times of the days and at different impact. It is very important that you contact us if you are experiencing problems so that we can isolate any possible issues with our side of things, and open an individual trouble case with the GLS help desk to further investigate the transport network. Please contact us at support@wiscnet.net or call us at 608-442-6761 option 2 if you are experiencing any problems.

Thank you for your collaboration and patience.

WiscNet
Posted Mar 19, 2024 - 10:43 CDT
Update
Hello,

WiscNet is investigating the ongoing issue and is working with our partners to find the exact location of the loss of traffic.


We will be posting the findings from our partners at least once a day, please stay tuned. Thank you for your patience.


If the issue for some reason changes greatly in further degradation or a return to expected results, please reach out to WiscNet Support.


If observations or concerns are found outside business hours, please reach out to the WiscNet NOC.


---------------------------------------------------------------------------------------------------------------------------------------------------

WiscNet Support;

Hours: 7:45 AM - 4:45 PM (M-F)
Phone: (608)-442-6761 ext. 2
Email: support@wiscnet.net


WiscNet NOC;

Hours: M - F, 24x7
Phone: (608)-442-6761 ext. 1
Email: noc@wiscnet.net
Posted Mar 15, 2024 - 12:27 CDT
Investigating
Greetings,

Please be advised that we are currently investigating network issues regarding service issues. We will provide updates as they become available.

Please contact the WiscNet NOC if you have any further questions.

608-442-6761 option 1
noc@wiscnet.net
Posted Mar 14, 2024 - 13:02 CDT
This incident affected: Transit Share (University of Wisconsin SysNet) and Adams-Friendship Area School District, Albany School District, Algoma School District, Almond-Bancroft School District, Antigo Unified School District, Argyle School District, Arrowhead UHS School District, Bangor School District, Beecher-Dunbar-Pembine School District, Belleville School District, Bellin College Inc, Belmont Community School District, Beloit School District, Benton School District, Birchwood School District, Black Hawk School District, Blackhawk Technical College, Bonduel School District, Boscobel Area School District, Bowler School District, Boyceville Community School District, Brighton #1 School District, Brillion School District, Bristol School District #1, Burlington Area School District, Cambridge School District, Cassville School District, Catholic Memorial High School, CESA 1, CESA 4, CESA 5, CESA 6, CESA 7, CESA 8, CESA 9, Chilton School District, Coleman School District, Concordia University, Crivitz School District, Cuba City School District, Darlington Community School District, Deerfield Community School District, De Forest Area School District, Dodgeland School District, Dodgeville School District, East Troy School District, Edgewood High School, Elkhart Lake-Glenbeulah School District, Elmwood School District, Erin School District, Fall River School District, Flambeau School District, Florence County School District, Fox Point J2 School District, Freedom Area School District, Geneva J4 School District, Genoa City J2 School District, Gillett School District, Gilmanton School District, Glenwood City School District, Good Shepherd Lutheran School, Goodman-Armstrong Creek School District, Grafton School District, Granton Area School District, Grantsburg School District, City of Green Bay, Green Lake School District, Hartford J1 School District, Hayward Community School District, Highland School District, Holmen School District, Horicon School District, Hortonville Area School District, Howards Grove School District, Hurley School District, Hustisford School District, Iola-Scandinavia School District, Iowa-Grant School District, Ithaca School District, Johnson Creek School District, Kettle Moraine School District, Kewaskum School District, Kewaunee School District, Kiel Area School District, Kimberly Area School District, Kohler School District, City of La Crosse, Lac Courte Oreilles Ojibwa Community College, Lac du Flambeau #1 School District, Lake Country School District, Lake Mills Area School District, Lakeland UHS School District, Lakeshore Technical College, Laona School District, Linn J6 School District, Lodi School District, Luther High School, Manawa School District, Manitowoc Public Utilities, Manitowoc School District, Maple Dale-Indian Hill School District, Marian University of Fond du Lac, Marion School District, Markesan School District, Marquette University, Marquette University High School, Medford Area Public School District, Menominee Indian School District, Mequon-Thiensville School District, Mercer School District, Merton Community School District, Mid-State Technical College, Milton School District, Milwaukee Art Museum, Milwaukee County Federated Library System, Minocqua J1 School District, Mishicot School District, Monticello School District, Mukwonago School District, Necedah Area School District, New Glarus School District, New London School District, Niagara School District, Nicolet UHS School District, Norris Academy, North Lakeland School District, Northcentral Technical College, Northland Pines School District, Northwood School District, Norwalk-Ontario-Wilton School District, Oconto Falls Public School District, Oconto Unified School District, Oregon School District, Osceola School District, Parkview School District, Paris J1 School District, Pecatonica Area School District, Peshtigo School District, Pewaukee School District, Phelps School District, Phillips School District, Pittsville School District, Plum City School District, Plymouth Joint School District, Port Edwards School District, Port Washington-Saukville School District, Portage Community School District, Poynette School District, Prairie du Chien Area School District, Prairie Farm Public School District, Prentice School District, Prescott School District, Princeton School District, Racine Lutheran High School, Racine Unified School District, Randall J1 School District, Randolph School District, Raymond #14 School District, Reedsville School District, Rib Lake School District, Richmond School District, Rio Community School District, Ripon College, River Ridge School District, River Valley School District, Riverdale School District, Rock Prairie Montessori, Rosholt School District, Sacred Heart of Jesus and Mary School, Sacred Heart School of Theology, St. Catherines High School, Saint Croix Central School District, Saint Croix Falls School District, St. Francis Xavier Catholic School Systems, St. Mary Catholic Schools, Salem School District, Sauk Prairie School District, Sharon J11 School District, Shawano School District, Sheboygan Area School District, Sheboygan Falls School District, Shell Lake School District, Shiocton School District, Shoreland Lutheran High School, Shullsburg School District, Siren School District, Solon Springs School District, Southwest Wisconsin Technical College, Southwestern Wisconsin School District, Spring Valley School District, Stone Bank School District, Stoughton Area School District, Stratford School District, Sturgeon Bay School District, Sun Prairie Area School District, Suring Public School District, Swallow School District, Three Lakes School District, Tigerton School District, Tomorrow River School District, Tri-County Area School District, Turtle Lake School District, Two Rivers Public School District, Union Grove J1 School District, Union Grove UHS School District, Wabeno Area School District, Walworth J1 School District, Washington-Caldwell School District, Waterford Graded J1 School District, Waterford UHS School District, Waterloo School District, City of Waukesha, Waunakee Community School District, Waupaca County, Waupaca School District, Wausaukee School District, Wauzeka-Steuben School District, Webster School District, West Salem School District, Westfield School District, Weyauwega-Fremont School District, White Lake School District, Whitefish Bay Dominican High School, Whitefish Bay School District, Winter School District, Wisconsin Center for the Blind and Visually Impaired, Wisconsin Department of Military Affairs, Wisconsin Heights School District, Wisconsin Lutheran College, Wisconsin School for the Deaf, Wonewoc-Union Center School District, Woodruff J1 School District, Wrightstown Community School District.