View previous topic :: View next topic |
Author |
Message |
Da Nag
Joined: 24 Oct 2003 Posts: 2820 City/Region: Port Angeles
State or Province: WA
C-Dory Year: 2008
C-Dory Model: 16 Angler
Vessel Name: Taco
Photos: <a>Da Boats</a>
|
Posted: Sat Sep 26, 2009 1:25 pm Post subject: Ouch...server dead for almost 24 hours |
|
|
We're back. Sorry, folks...my bad.
What was intended to me a minor update yesterday, quickly turned into severe ugliness. A patch applied to the server required a reboot, which normally takes a minute or so.
The server failed to come back up...which is an absolute PITA to deal with remotely, as I'm not sitting in front of the computer to see what is going on. My only recourse, was to deal with two levels of tech support, playing a long and frustrating "game" of telephone.
Support is by email only, and to make matters worse, my support contact doesn't do the troubleshooting in front of the computer. Instead, it went something like this:
Me: "Please try this."
Wait 5 minutes to 2 hours...
Support: "I'll forward that to the data center techs."
Wait 10-30 minutes...
Support: "Didn't work. Should we rebuild the server?"
Me: "Hell no! Try this."
Repeat the above for about 24 hours...
Anyway, everything appears to be functioning again, and lessons were learned from this experience. In particular, I put together documentation for the hosting provider, that allows them to bring the server up via a CD when the server won't boot on its own. In this "rescue mode", the server doesn't function normally - but it does allow me to get onto it remotely, and fix things directly without the game of telephone mentioned above...
Welcome back, friends... _________________ Will, C-Brat Nerd |
|
Back to top |
|
|
Doryman
Joined: 03 Oct 2006 Posts: 3807 City/Region: Anacortes
State or Province: WA
C-Dory Year: 2006
C-Dory Model: 255 Tomcat
Vessel Name: Lori Ann
Photos: Lori Ann
|
Posted: Sat Sep 26, 2009 1:37 pm Post subject: Re: Ouch...server dead for almost 24 hours |
|
|
Da Nag wrote: | What was intended to me a minor update yesterday, quickly turned into severe ugliness. A patch applied to the server required a reboot, which normally takes a minute or so.
The server failed to come back up...which is an absolute PITA to deal with remotely, as I'm not sitting in front of the computer to see what is going on. My only recourse, was to deal with two levels of tech support, playing a long and frustrating "game" of telephone.
Support is by email only, and to make matters worse, my support contact doesn't do the troubleshooting in front of the computer. |
Bill -- been there, done that -- but fortunately hands-on. Trying to anticipate all contingencies when writing up instructions can be very frustrating. Thanks for your efforts!
What was the date and time of the backup that you restored from? That will let us know how much activity was lost.
Warren _________________ Doryman
M/V Lori Ann
TomCat 255, Hull #55, 150 Yamahas
Anacortes, WA
|
|
Back to top |
|
|
Comfortably Numb
Joined: 30 Aug 2007 Posts: 146 City/Region: Greenville
State or Province: IL
C-Dory Year: 2008
C-Dory Model: 22 Cruiser
Vessel Name: Comfortably Numb
Photos: Comfortably Numb
|
Posted: Sat Sep 26, 2009 1:40 pm Post subject: |
|
|
I understood about 10% of what you said, Bill! But, that's why you guys are in charge. Thanks for all the good work...you done good. _________________ Mark
Comfortably Numb
Governor Bond Lake
Greenville, Illinois
“For this is how the suicide of nations begins, when sentimentality prevails over sense.”
- Roger Scruton |
|
Back to top |
|
|
Casey
Joined: 02 Nov 2003 Posts: 1094 City/Region: The Villages(FL)
State or Province: FL
C-Dory Year: 2006
C-Dory Model: 23 Venture
Vessel Name: "Dessert 1st"
Photos: Dessert 1st
|
Posted: Sat Sep 26, 2009 1:46 pm Post subject: |
|
|
I didn't even understand 10% !
On the otherhand, what I DID understand was that I was beginning to shake from Brat Withdrawl; it was not a pretty sight!
Glad we're reconnected with the Brat Community!
THANK YOU, THANK YOU, THANK YOU, THANK YOU, THANK YOU!!!!!
Best,
Casey _________________ 2013 CC23 "Katmai" renamed "Dessert 1st"
2006 CC23 "Katmai" (purchased August 2009)
2003 CD22 "Naknek" (sold May 2008)
Last edited by Casey on Sat Sep 26, 2009 1:48 pm; edited 1 time in total |
|
Back to top |
|
|
Da Nag
Joined: 24 Oct 2003 Posts: 2820 City/Region: Port Angeles
State or Province: WA
C-Dory Year: 2008
C-Dory Model: 16 Angler
Vessel Name: Taco
Photos: <a>Da Boats</a>
|
Posted: Sat Sep 26, 2009 1:48 pm Post subject: Re: Ouch...server dead for almost 24 hours |
|
|
Doryman wrote: | What was the date and time of the backup that you restored from? That will let us know how much activity was lost. |
Didn't have to go to backup - there was zero data loss. Once I had access to the server, I was able to repair the reason it wasn't coming back up on its own.
For the nerds in the audience...the ultimate problem, was a grub failure. In my best attempt at laymen's terms - the code that tells the server how to start, became corrupted by the patch I applied. It didn't affect any of the data on the server - just the startup process. Once repaired, the server came back up unaffected by the problem. |
|
Back to top |
|
|
BrentB
Joined: 15 Jul 2006 Posts: 4419 City/Region: Greenwood
State or Province: IN
Photos: BrentB
|
Posted: Sat Sep 26, 2009 1:51 pm Post subject: |
|
|
thanks again for your hard work and I added something to the Tip jar _________________ Brent Barrett |
|
Back to top |
|
|
Jack in Alaska
Joined: 17 Aug 2004 Posts: 1190 City/Region: Anchorage/Ninilchik
State or Province: AK
C-Dory Year: 2005
C-Dory Model: 26 Pro Angler
Vessel Name: HIGH TIDE II
Photos: HIGH TIDE II
|
Posted: Sat Sep 26, 2009 2:05 pm Post subject: |
|
|
DaNag................
You are just great at keeping us "fat fingered" C-Brat junkies up and running. Thanks for your expertise and your effort in doing that for us. I am only a 5% understander. I also suffered withdrawal pains but am over it now.
Jack in Alaska _________________ On the HIGH TIDE-II, wife Carolyn and I.....Another summer fishing on the HIGH TIDE II in the Cook Inlet at Cape Ninilchik, Alaska.
HIGH TIDE-II; 2005 26' ProAngler; 2003 200 Honda / 2009 9.9 Honda high thrust
No. CD026021I405; AK-5008-AK
MSSI No. 338143486(cancelled)
HIGH TIDE; 1983 Angler Classic 22'; 90 Honda/ 9.9 Tohatsu-sold 2009 to son Dan (flatfishfool)
Stolen & stripped in Aug. 18
Bare hull & trailer sold in Nov.
Last edited by Jack in Alaska on Sat Sep 26, 2009 2:14 pm; edited 1 time in total |
|
Back to top |
|
|
colobear
Joined: 23 Jan 2005 Posts: 2154 City/Region: Denver
State or Province: CO
C-Dory Year: 2006
C-Dory Model: 22 Cruiser
Photos: C-Cakes
|
Posted: Sat Sep 26, 2009 2:06 pm Post subject: |
|
|
Thanks Bill. We sometimes need something like this to help us remember all the hard work you and Mike do to provide this great site...and how important this site is to us.
Thank you! _________________ Patti and Barry
formerly C-Cakes, now
rving around N. America |
|
Back to top |
|
|
Lucky Day
Joined: 10 Oct 2006 Posts: 215 City/Region: Churchton
State or Province: MD
C-Dory Year: 2004
C-Dory Model: 25 Cruiser
Vessel Name: Lucky Day
Photos: Lucky Day
|
Posted: Sat Sep 26, 2009 2:09 pm Post subject: |
|
|
Losing c-brats.com is a little like the shock of losing electricty. Wow. Glad that's over. Had to click on "Donate Now" for the effort that y'all put in . . . |
|
Back to top |
|
|
Da Nag
Joined: 24 Oct 2003 Posts: 2820 City/Region: Port Angeles
State or Province: WA
C-Dory Year: 2008
C-Dory Model: 16 Angler
Vessel Name: Taco
Photos: <a>Da Boats</a>
|
Posted: Sat Sep 26, 2009 2:14 pm Post subject: |
|
|
Thanks much for the kind words.
But, be careful with that Tip Jar. I just might realize, that screwing things up around here can be a money maker. Heck...maybe next time, I'll take a week to fix things - just might get enough for a TomCat. |
|
Back to top |
|
|
Doryman
Joined: 03 Oct 2006 Posts: 3807 City/Region: Anacortes
State or Province: WA
C-Dory Year: 2006
C-Dory Model: 255 Tomcat
Vessel Name: Lori Ann
Photos: Lori Ann
|
Posted: Sat Sep 26, 2009 2:31 pm Post subject: |
|
|
Da Nag wrote: | Thanks much for the kind words.
But, be careful with that Tip Jar. I just might realize, that screwing things up around here can be a money maker. Heck...maybe next time, I'll take a week to fix things - just might get enough for a TomCat. |
OTOH, given enough tips, you could afford to host the site where you have hands-on access!
Warren |
|
Back to top |
|
|
BrentB
Joined: 15 Jul 2006 Posts: 4419 City/Region: Greenwood
State or Province: IN
Photos: BrentB
|
Posted: Sat Sep 26, 2009 2:33 pm Post subject: |
|
|
Da Nag wrote: | Thanks much for the kind words.
But, be careful with that Tip Jar. I just might realize, that screwing things up around here can be a money maker. Heck...maybe next time, I'll take a week to fix things - just might get enough for a TomCat. |
sorry I am a bad tipper |
|
Back to top |
|
|
BRAZO
Joined: 29 Jul 2008 Posts: 650 City/Region: Full-time Travel
State or Province: NV
C-Dory Year: 2008
C-Dory Model: 22 Cruiser
Vessel Name: Meyer Meyer
Photos: BRAZO
|
Posted: Sat Sep 26, 2009 3:05 pm Post subject: |
|
|
Quote: | What was intended to me a minor update yesterday, quickly turned into severe ugliness. A patch applied to the server required a reboot, which normally takes a minute or so.
|
As a Computer Programmer for 25 years - I know the feeling, been there done that - but in my case it would take down the mainframe for 2500 employees for a bit, not good.
Very cool that c-brats have someone like Da Nag with the knowledge and the willingness to get it done.
Many thanks. _________________ 2008 CD22 Twin 50hp
Full-time Travel - Sprinter4x4, International, C-DORY 22 |
|
Back to top |
|
|
MyCattMaxx
Joined: 16 Sep 2009 Posts: 9
C-Dory Year: 2009
C-Dory Model: R-25 Tug
|
Posted: Sat Sep 26, 2009 3:15 pm Post subject: |
|
|
Yup, I kept trying to get on here and TugNuts.
I assume you both are on the same server? |
|
Back to top |
|
|
Da Nag
Joined: 24 Oct 2003 Posts: 2820 City/Region: Port Angeles
State or Province: WA
C-Dory Year: 2008
C-Dory Model: 16 Angler
Vessel Name: Taco
Photos: <a>Da Boats</a>
|
Posted: Sat Sep 26, 2009 3:17 pm Post subject: |
|
|
MyCattMaxx wrote: | I assume you both are on the same server? |
Yup. |
|
Back to top |
|
|
|