Category: System Status

Resolved – 10-17-2022 3:25PM

Update – 10-17-2022 2:00PM

The root cause of the outage on the coastal portion of our network has been found and we are working with the various players to resolve.  Current hopeful ETR at time of writing this is 6PM today.

Thanks for your understanding.

– Jacob

Our fiber vendor was able to determine around 4:40PM today that there is a fiber break between Blue Lakes and Upper Lake.  They are currently sending repair crews to the location to splice and replace the damaged fiber.

In the interim, I was able to upgrade the congested backup link to Blue Lakes to allow more capacity in the area while the fiber is repaired.

We do not yet have an ETR on the fiber repair.

Thanks,

– Jacob

At around 12:15 this afternoon the fiber between Blue Lakes and our head-end in Fremont went down for unknown reasons. Traffic is now flowing over a backup wireless link very slowly to the area.

I have opened a ticket with the fiber provider and will send
an update once we know more.
Thanks

• Jacob

A generator was delivered to the affected site this morning and service was restored to all affected subscribers (that themselves have power) at 7:32AM. 

PGE’s current estimated time of grid power in the area is 2PM today. 

It was determined when on-site this morning that our backup battery bank has a failed battery, hence the short run-time of only ~2.5 hours.  I plan on replacing the battery bank this afternoon after grid power is restored.  The site will continue running on generator until that time.

Thanks,

– Jacob

A power outage that began at around 2:30 this morning is currently causing an outage at a site that serves subscribers in Redwood Valley, Potter Valley, and Northern Ukiah.  

We are currently en-route with a generator to resolve the issue until power can be restored by PGE.  It is unknown why our backup battery system did not last longer at this time.

Thanks for your understanding.
– Jacob

2022-05-20 – As of 5:30PM we are running the site on a combination of battery and generator. PGE power is expected to be restored by 4PM on 5-21.

At ~11:00AM this morning PGE power went out at one of our coastal points of presence, causing a service interruption for coastal customers. The current ETR from PGE is 2:15PM this afternoon. We will update this message if the ETR changes.

Thanks for your understanding.

– Jacob

2022-05-11 – ATT repaired the damaged fiber this morning at 6:20AM and all services are restored.

As many are probably already aware, ATT suffered a fiber cut affecting Fort Bragg this morning.  This fiber cut has also affected our network on the coast, as we utilize ATT fiber to transport service back to our Ukiah location.  ATT does not currently have an estimated time of repair, but we have an open ticket with them and will update our customers when (if) we receive an updated ETR from them.

Once the fiber cut is repaired, service to our coastal network should be fully restored.

Thanks for your understanding.

– Jacob​

Battery power was lost at the site serving these areas at 1PM.  PGE has updated their power restoration estimate to 4:30PM today. 

Once power is restored, internet service will be restored as well.

Thank you for your understanding.

– Jacob

PGE Power was lost yesterday morning 12-27 at around
10AM at a mountaintop site that serves subscribers along the Ridgewood
Grade, Ridgewood Ranch, and parts of Willits.  Our equipment has been
running on battery since power was lost.

PGE has an estimated time of restoration of 10PM tonight, but due to heavy snow on the mountain that ETR may be ambitious. Due to the heavy snow accumulations we can’t currently get to the site to run our own backup generator. It is my hope that grid power can be restored before our batteries run flat.

Thanks for your understanding

– Jacob

=== PGE Restored power at 1:30PM ===​

At about 10:40 this morning PGE suffered a power outage that is affecting the Bald Hill tower site above Fort Bragg.  For unknown reasons, the backup generator at the location doesn’t appear to have started automatically like designed.   

This outage is affecting subscribers in Fort Bragg and North to Westport.  

An update will be sent as soon as PGE provides an ETR for electrical service at the site.

Thanks,

– Jacob