SERVICE(S) AFFECTED:
Metro Fiber
STATUS:
Resolved
INCIDENT SUMMARY:
On January 25, 2018 during the addition of a routine routing update for a Metro Fiber customer, a majority of the SLO fiber network went into an unresponsive state. All remotes and connections were up, but border gateway protocol (BGP) was not operating correctly. A restart of the routing instance brought both devices back into a neighbor relationship and traffic was restored. All system logs are intact and a core dump has been logged and sent to Juniper for analysis. The duration of the event was approximately seven minutes, from 14:46 to 14:53.
CUSTOMER IMPACT:
Loss of service.
LOCATION:
SBP1 (3620 Sacramento Drive, San Luis Obispo, CA)
NEXT UPDATE:
No further update regarding this incident, however if it is determined we need to patch firmware or do a version change, we will issue a maintenance notification and alert you to a scheduled maintenance window.
SERVICE(S) AFFECTED:
Voice, Direct Internet Access (DIA), MPLS connections (EPIPE, VPLS, VRPN)
STATUS:
Resolved
INCIDENT SUMMARY:
On January 25, 2018, one of our MPLS backbone nodes in our primary data center in SLO entered a faulty state. The node in question showed neighbor adjacencies with the rest of the network via open shortest path first (OSPF), but all MPLS services that rely on those neighbor relationships were down. Failover precautions at the MPLS level didn’t engage due to the underlying OSPF relationship remaining intact. A forced control switch module (CSM) failover cleared the erroneous state and services were restored. The duration of the event was approximately 30 minutes, from 16:31 to 17:01.
CUSTOMER IMPACT:
Loss of service.
LOCATION:
Lompoc, Santa Maria, Arroyo Grande, Pismo Beach, San Luis Obispo, Atascadero, Templeton, Paso Robles, Oak Shores, CA
NEXT UPDATE:
Error logs have been captured and sent to the vendor for root cause analysis. We will send an update when more information is available.