Re: Trouble with BFG?
[Re: Sly One]
#518686
07/03/09 11:46 AM
07/03/09 11:46 AM
|
Joined: Aug 2006
Posts: 1,368 Ontario, Canada
jldcat
Addicted Boomer
|
Addicted Boomer
Joined: Aug 2006
Posts: 1,368
Ontario, Canada
|
Wow, that's too bad! This happened around a year or so again too didn't it? Bad luck. I'm sure BFG will be up as soon as they can.
The breeze from one door closing opens another.
|
|
|
Re: Trouble with BFG?
[Re: Sly One]
#518693
07/03/09 11:56 AM
07/03/09 11:56 AM
|
Joined: Oct 2000
Posts: 40,644 southeast USA
Jenny100
GB Reviewer Glitches Moderator
|
GB Reviewer Glitches Moderator
Sonic Boomer
Joined: Oct 2000
Posts: 40,644
southeast USA
|
If you "titter" for #fisherfire you can see a new update every 10 seconds or so. Estimates are 1000 sites, 2 radio stations and a TV station were knocked out with this fire.
WE are in the process of moving to our backup datacenter. This will not be quick or easy. FIsher expects to be down another 12 to 24 hours, but we shoudl be back up before them. Thank you for letting us know what is going on, BFG-Guy. It sounds bad. I hope you're up and running again soon.
|
|
|
Re: Trouble with BFG?
[Re: C65candy]
#518694
07/03/09 11:58 AM
07/03/09 11:58 AM
|
Joined: Oct 2007
Posts: 399
BFG-Guy
Settled Boomer
|
Settled Boomer
Joined: Oct 2007
Posts: 399
|
Some more information from the North American Network Operators Group discussion board:
Fisher Plaza, a self-styled carrier hotel in Seattle, and home to multiple datacenter and colocation providers, has had a major issue in one of its buildings late last night, early this morning.
The best information I am aware of is that there was a failure in the main/generator transfer switch which resulted in a fire. The sprinkler system activated. From speaking to the fire battalion chief, I am under the impression that Seattle Fire did use water on the fire as well, but I am unsure of this.
Given the failure location, generator power was not available, and cooling failed. UPS power to systems continued, and I can personally vouch that they held out for well over an hour. When we were able to access our equipment, ambient air temps were well over 100 degrees in the room our equipment is located in.
At least some, if not many circuits were affected. Several large co-location providers and other datacenters are located in the facility, these facilities have no power.
As this was the main/generator switch, and it is now highly damaged, the circuits in the area are damaged, and the entire area is doused in water, a rapid restoration of power does not seem likely. Fisher Plaza's phone numbers now result in fast-busy signals, so I have no recent update from them directly.
Interestingly, this building is also the production studios for several Seattle TV and radio stations.
There is no ETA for resolution. We are not waiting for Fisher Plaza to fix this, as it now appears unlikely to happen any time soon. We are in the process of bringing up our disaster recovery data center in New Jersey and at another location in Seattle. If all the stars align this will be within 4-6 hours, but most likely 12 to 24 as we have to migrate huge database servers that will take some time. We will be operating at 100% soon...just not sure how soon. Picman p.s. at least we are realizing some value from the millions of dollars worth of equipment sitting around waiting in our backup facilities "just in case". This is the perfect storm you plan for but never really expect to happen. (Silver lining in an otherwise dark  ) I live on the hill near this facility and there are two radio station vans and a TV van attmpting to broadcast from thier mobile studios in a nearby park.
BFG-Guy  "A New Game Every Day!" keeps boredom away.
|
|
|
Re: Trouble with BFG?
[Re: BFG-Guy]
#518699
07/03/09 12:03 PM
07/03/09 12:03 PM
|
Joined: Nov 2004
Posts: 9,848 San Diego, CA
Sorta Blonde
BAAG Specialist
|
BAAG Specialist
Joined: Nov 2004
Posts: 9,848
San Diego, CA
|
Best of luck to you on the recovery efforts BFG-Guy! Hope it all works out smoothly. BFG is one of my favorite places. Crossing fingers and toes that all goes well!
WARNING! This person is extremely blonde...please type SLOWLY.
|
|
|
Re: Trouble with BFG?
[Re: BFG-Guy]
#518703
07/03/09 12:07 PM
07/03/09 12:07 PM
|
Joined: Jul 2007
Posts: 18,262 Massachusetts
GBC
Graduate Boomer
|
Graduate Boomer
Joined: Jul 2007
Posts: 18,262
Massachusetts
|
Sending hugs out to you BFG-Guy.  Hope things get better for you soon.
Gail
|
|
|
Re: Trouble with BFG?
[Re: Marian]
#518711
07/03/09 12:37 PM
07/03/09 12:37 PM
|
Joined: Oct 2007
Posts: 399
BFG-Guy
Settled Boomer
|
Settled Boomer
Joined: Oct 2007
Posts: 399
|
we have 20+ IT and engineers from BFG on site planning our move to our disaster recovery center. I am going offline (getting on an airplane soon) but our COO and tech ops people are very good and they will have us back up soon. (Soon as defined as within a day, hopefully much sooner).
BFG-Guy  "A New Game Every Day!" keeps boredom away.
|
|
|
Re: Trouble with BFG?
[Re: BFG-Guy]
#518712
07/03/09 12:39 PM
07/03/09 12:39 PM
|
Joined: Oct 2000
Posts: 40,644 southeast USA
Jenny100
GB Reviewer Glitches Moderator
|
GB Reviewer Glitches Moderator
Sonic Boomer
Joined: Oct 2000
Posts: 40,644
southeast USA
|
We are in the process of bringing up our disaster recovery data center in New Jersey and at another location in Seattle. If all the stars align this will be within 4-6 hours, but most likely 12 to 24 as we have to migrate huge database servers that will take some time.
We will be operating at 100% soon...just not sure how soon.
Picman
p.s. at least we are realizing some value from the millions of dollars worth of equipment sitting around waiting in our backup facilities "just in case". This is the perfect storm you plan for but never really expect to happen. (Silver lining in an otherwise dark) Thanks for the additional information, BFG-Guy. It's good to know you have a good disaster recovery plan in place and will be back online soon. I'd been wanting to get my 5 games for the "Big Fish Detective" keychain offer, so hopefully it won't be as long as 24 hours.
|
|
|
Re: Trouble with BFG?
[Re: Marian]
#518713
07/03/09 12:39 PM
07/03/09 12:39 PM
|
Joined: Jun 2005
Posts: 1,653 Waukegan, Il.
exalthim2
Addicted Boomer
|
Addicted Boomer
Joined: Jun 2005
Posts: 1,653
Waukegan, Il.
|
Good Luck BFG. So sorry this happened but it's a blessing that you were prepared with a back up. Prayers are going up for your smooth rcovery Kim
You don't have to attend every argument you're invited to.
|
|
|
Re: Trouble with BFG?
[Re: Marian]
#518728
07/03/09 01:27 PM
07/03/09 01:27 PM
|
Joined: May 2008
Posts: 99 Pacific NW
ChrisMM
Shy Boomer
|
Shy Boomer
Joined: May 2008
Posts: 99
Pacific NW
|
Oh, wow. Man, I am so sorry--my regards to your team--this is not the way to spend a holiday weekend.
Last edited by ChrisMM; 07/03/09 01:37 PM.
|
|
|
Re: Trouble with BFG?
[Re: connie]
#518789
07/03/09 03:55 PM
07/03/09 03:55 PM
|
Joined: May 2008
Posts: 168 East Suussex, UK
Johnnycowgirl
Settled Boomer
|
Settled Boomer
Joined: May 2008
Posts: 168
East Suussex, UK
|
quote on twitter: ''Temp repair is "on schedule to be completed by late this evening" for #fisherfire East'' Where's Bigfish, anyone know?
Inge Ravenhearst 1 ++ Dracula: Path of the Dragon 1,2,3 + The Shadow of the Dragon 4 + Blood Legacy 5 partial until stuck in the fog
|
|
|
Re: Trouble with BFG?
[Re: Johnnycowgirl]
#518790
07/03/09 03:59 PM
07/03/09 03:59 PM
|
Joined: Nov 2007
Posts: 550 Maryland
Margie
Settled Boomer
|
Settled Boomer
Joined: Nov 2007
Posts: 550
Maryland
|
quote on twitter: ''Temp repair is "on schedule to be completed by late this evening" for #fisherfire East'' Where's Bigfish, anyone know? Big Fish is in Seattle.
|
|
|
Re: Trouble with BFG?
[Re: Margie]
#518796
07/03/09 04:08 PM
07/03/09 04:08 PM
|
Joined: Jun 2001
Posts: 3,805 Kentucky, USA
acornia
Addicted Boomer
|
Addicted Boomer
Joined: Jun 2001
Posts: 3,805
Kentucky, USA
|
Am glad to here that it will be up soon. We will all be waiting patiently like a cat waiting to pounce. So as soon as the site is up, expect your own deluge of customers. lol If you like animals, be sure to ride the bus in the animal park at Great Adventure. Do not use a car and if you do use a car, bypass the monkey section.
Never resist a generous impulse.
|
|
|
Re: Trouble with BFG?
[Re: acornia]
#518800
07/03/09 04:28 PM
07/03/09 04:28 PM
|
Joined: Sep 2004
Posts: 631 South Carolina
jody319
Settled Boomer
|
Settled Boomer
Joined: Sep 2004
Posts: 631
South Carolina
|
BFGguy! Sorry about the fire, water damage and hopefully no one was hurt, but thank goodness y'all were "up" enough to have alternate locations planned ahead. SMART people y'all are. Hope your trip is for pleasure and not work and that you can truly put this at least temporarily out of your mind. We are a patient  (just kidding) people...good luck to all of you there and in New Jersey... 
|
|
|
Re: Trouble with BFG?
[Re: GPM]
#518824
07/03/09 05:53 PM
07/03/09 05:53 PM
|
Joined: Jul 2000
Posts: 50,763 near Yosemite
Marian
Global Moderator
|
Global Moderator
Sonic Boomer
Joined: Jul 2000
Posts: 50,763
near Yosemite
|
Here's a link to a Seattle news station that shows where this all occurred (watch the video on the lefthand side of the page: Fisher Plaza fire It sounds like Big Fish might end up being one of the "lucky" ones in this building, as apparently some businesses might have sustained loss of data.
|
|
|
|
|