Category: System Status

PGE PSPS Update – State of the network

Well in to the third full day without power I thought I’d send an email to update everyone on the state of things related to our network and answer a few common questions.
The vast majority of the network has been fully operational during this extended outage.  A large number of our sites are solar powered and thus not affected.  However, our head-end in Ukiah and a handful of critical macro sites that serve large geographical areas are grid powered.  These grid powered sites have been running continuously on generator and backup batter power for the duration of the outage and will continue to do so.  
Some smaller micro-relay sites that serve small number of subscribers have either been partially impacted by the power outage, or totally down during the power outage.  The vast majority of subscribers served from these micro-relays would not have noticed though, as they are also without an alternative power source.  The number of customers served by such sites that haven’t been continuously up, account for 13% of our total subscriber count.  Unfortunately there are a large number of these small sites and it is unrealistic for us to move generators and fuel around to all of them for such a wide-spread and extended outage.
Of the remaining 87% of customers, almost 50% have been online during the power outage at some point.  Either because they are already off-grid, or they have used a generator or other alternative power source to get online.
Areas currently affected by the power outage (on our end of things) include the following:
– Fort Bragg  to Westport – all subscribers – Redwood Valley – Uva Drive, North State Street, and surrounding area (11 subscribers) – Downtown Ukiah – Between Stanley and Clay – Laytonville – all subscribers – Old River Road – From Ruddick Cunningham South – McNab Ranch – Subscribers served from McNab Ridge Winery – Hopland – about 20 subscribers – Blue Lakes – about 25 subscribers east of Irvine Ave
Affected areas above will resume normal operation when power is restored in those locations.
It’s always my hope to provide the most reliable service to all subscribers.  These power shutoffs are a great test of plans that have been put in place, and expose many areas in which we can improve.   Some very positive infrastructure changes in our network will result from this experience.
I want to send a special thank you to those on Orr Springs road that jumped at the opportunity to help with keeping things afloat this weekend when a call went out. It is truly appreciated.
If you have any questions, please don’t hesitate to call or email.
Thanks,
– Jacob

Anticipated PGE outages – how this may impact your service

As of 8:45PM Monday evening PGE is anticipating disrupting power to large areas of Lake and Mendocino Counties beginning at 4AM on 10-9-2019.  Current PGE PSPS info is available here:
www.pge.com/en_US/safe… <www.pge.com/en_US/safety/emergency-preparedness/natural-disaster/wildfires/public-safety-event.page?WT.mc_id=Vanity_pspsupdates>
I have anticipated these possible outages and have portable generators at critical grid-powered sites to minimize any service disruptions to our subscribers, particularly those that serve public safety customers.  In addition to this, a majority of our sites are off-grid and aren’t affected by grid outages.   We do however, have a number of small relays that serve small groups of people that only have small battery backups.  These small relays will go offline in the event of an extended power outage.  
What I can’t anticipate unfortunately, is how these large-area power outages may affect the networks of upstream fiber providers that we lease connectivity from between Ukiah and the Bay Area.   I expect that their facilities should have adequate battery and generator runtime to get through the outages without issue. Unfortunately though, this is something we won’t know until we know.
During an extended power outage our office phones and computers will be offline.  We will however be available via email at *support@seakay.org* if you have any questions.
Thank You,
– Jacob

Friday morning software updates -McNab Ranch and Hopland

Friday morning beginning at 3AM a number of devices serving subscribers on McNab Ranch and in Hopland are scheduled to receive important software updates.  The various updates are scheduled to happen automatically between 3 and 4AM and should complete on their own.  Subscribers may experience a few brief service interruptions during this window while the updates take place.
Thanks,
– Jacob

Completed – Intrusive maintenance tonight starting at 11:59PM

Tonight I will be migrating subscriber traffic to some new hardware and making some other infrastructure changes at our head-end in Ukiah.  During this transition, service will be interrupted to some subscribers for 15 minutes to two hours depending on where in our network they are located.  My goal is to finish this work by 2AM.  If it cannot be completed within the maintenance window, a follow-up window will be scheduled.
Thanks for your understanding.
– Jacob

Completed at 1:20AM