Discussion Forums  >  Buzztouch News and Announcements

Replies: 14    Views: 178

David @ buzztouch
buzztouch Evangelist
Profile
Posts: 6866
Reg: Jan 01, 2010
Monterey, CA
78,840
11/25/12 09:26 PM (13 years ago)

Thanksgiving Weekend + Cranky Servers Loooong Saturday night.

Some of you may have seen a message I posted in the forum and in account control panels about some issues being reported with the buzztouch.com hosted control panel. This has been resolved. The juicy details........ I got word about 5:30 AM on November 25 (thanks @cpmilez) that some app owners were experiencing problems uploading files and icons and creating Custom HTML screens. Both of these processes require writing files to disk on our backend. Both of these processes were failing intermittently. After snooping around a few servers and querying tens of thousands of directories for "permission issues" (over the course of many hours) I was able to isolate the problem to one harddrive that was intermittently failing and not allowing write operations. It appears that the harddrive on this server had been experiencing problems for about 48 hours...randomly. This meant that about 1,000 apps may have been affected (apps that use this servers file system for files and documents). The best solution I could come up with was to take this server offline, configure a replacement server, then transfer the most recent backup file system to the new machine. Of course this meant that I needed to use the most recent backup that was completed before the troubles began. I settled on restoring from a backup image that was 48 hours old. It appears that all of the data has been successfully migrated to the new server and no data has been lost. You can't imagine how much effort (and dollars) we put into backing up your information. We do it because issues like this can and DO arise. If this file system hick-up affected you or your customers we appreciate your understanding. If it did not, consider yourself lucky :-) For those of you that celebrated Thanksgiving this past week, we hope you enjoyed it as much as The Books did. We enjoyed a peaceful hang-out on Thursday, just the four of us (no other family) and ate around 3:00 PM after a great trip to the beach and a hike in the woods in Big Sur - a rugged but gorgeous region in Central California. I'll keep an eye on the newly configured file system server and post if I see any issues. David
 
GoNorthWest
buzztouch Evangelist
Profile
Posts: 8197
Reg: Jun 24, 2011
Oro Valley, AZ
1,000,000
like
11/25/12 09:30 PM (13 years ago)
Sounds like an awesome Thanksgiving time with family! Glad you had time to enjoy that. Thanks for the super quick turnaround on problems like these. Another reason why buzztouch simply kicks the butt of all other app making sites out there! Rock on! Mark
 
Outbreak
buzztouch Evangelist
Profile
Posts: 452
Reg: May 30, 2011
Colorado
14,620
like
11/25/12 10:06 PM (13 years ago)
David Thanks again for your efforts. Nothing should be expected to be perfect in any type of arena. It's how it handled when the unexpected does occur. You guys do a fantastic job of preparing for and handling setbacks. Thank you. Shane
 
Niraj
buzztouch Evangelist
Profile
Posts: 2943
Reg: Jul 11, 2012
Cerritos
37,930
like
11/25/12 10:32 PM (13 years ago)
Wow -- great sleuthing! A sincere thanks and pleasant dreams.
 
Susan Metoxen
buzztouch Evangelist
Profile
Posts: 1706
Reg: May 01, 2011
Hopkins, Minnes...
26,260
like
11/25/12 10:38 PM (13 years ago)
I wonder how we can add some similar protection to our self hosted servers. Would we be able to restore from a backup? Maybe that would be a good topic for a Webinar.
 
GoNorthWest
buzztouch Evangelist
Profile
Posts: 8197
Reg: Jun 24, 2011
Oro Valley, AZ
1,000,000
like
11/25/12 10:42 PM (13 years ago)
That's a great point, Susan! It appears that buzztouchmods.com has an addition to the self-hosted control panel that allows you to backup your database, which is what's the most important to have: https://www.buzztouchmods.com/downloads.php?cat_id=2&download_id=35 I'm sure your web hosting company hopefully also does backups of your server from time to time? Mark
 
David @ buzztouch
buzztouch Evangelist
Profile
Posts: 6866
Reg: Jan 01, 2010
Monterey, CA
78,840
like
11/25/12 10:47 PM (13 years ago)
Two things to consider here. Database: This holds all the necessary info that creates your config.txt file. No files, images, or other assets are stored in the database. File System: /files/all folders inside it... The files system holds files that are uploaded to the File Manager or document type data that some plugins create. Example, when you create a new Custom Text/HTML screen an actual .html document is created in the directory for the app. This is in /files/applications. Backups: This means you have two things to consider backing up. The database (like @GoNorthWest mentioned) and the File System. Both are important. Both are easy to backup but for sure there a zillion opinions about the best way to do this. Self Hosted 2.1.8 may make this type of thing easier on you (wink).
 
chadh0130
Apple Fan
Profile
Posts: 352
Reg: Nov 05, 2011
Rhode Island
4,020
like
11/25/12 11:47 PM (13 years ago)
I'm glad to hear you and your family enjoyed the holiday! David, thank you from the bottom of my heart for all that you do for this community and giving me an opportunity to make a career using your platform :)
 
cpmilez
Aspiring developer
Profile
Posts: 63
Reg: Jan 03, 2012
Ipswich, UK
6,030
like
11/26/12 03:24 AM (13 years ago)
Thank you for the fast response and turn around on this. If only other sites provided support like yourself! I'm sorry to report I still have a write permission issue :-( I have updated my original post.
 
MikeP
Aspiring developer
Profile
Posts: 20
Reg: Jul 14, 2012
NYC
6,100
like
11/26/12 06:20 AM (13 years ago)
Thanks for working through this David. We appreciate your support!
 
LA
Aspiring developer
Profile
Posts: 3280
Reg: Aug 16, 2012
Jerseyville, IL
42,900
like
11/26/12 07:07 AM (13 years ago)
David @buzztouch, Thank you for the time you put in for us (buzztouch users) might take for granted that goes on in the back end of buzztouch. Sometimes, unless I, expect to wake up and buzztouch will be there for me to log in and learn/build apps. Again thank you. LA
 
Calypso Kid
Aspiring developer
Profile
Posts: 780
Reg: Mar 09, 2012
Upstate New Yor...
18,200
like
11/26/12 07:13 AM (13 years ago)
Thanks for everything David. So happy to be here at Buzztouch!
 
Jake Chasan
Veteran developer
Profile
Posts: 1685
Reg: May 13, 2011
location unknow...
29,650
like
11/26/12 07:46 AM (13 years ago)
Hi David, Thank you for fixing the server issues. Just got this error when I clicked on my account: Some users are experiencing some trouble connecting to buzztouch.com, we are looking into the issue and hope to have it resolved soon. An error occurred trying to access a resource? (1212) EDIT: It is working now Thanks, Jake
 
ATRAIN53
Code is Art
Profile
Posts: 1755
Reg: Nov 17, 2011
Chicago
26,450
like
11/26/12 07:54 AM (13 years ago)
intermittent hardware failure... That has cost me a few long nights as well. Thanks for the details. Like how you used the forum to post the alert. I hate when providers try to hide issues. Just let customers know stuff is broke and you're fixing it. Because when the dust settles customers still want to know. Glad you had a good turkey day! GO IRISH!
 
Alex@TM
Apple Fan
Profile
Posts: 956
Reg: Dec 20, 2011
London, UK
10,560
like
11/26/12 04:24 PM (13 years ago)
Great stuff David. No idea if I suffered any issues but thanks greatly either way :)
 

Login + Screen Name Required to Post

pointerLogin to participate so you can start earning points. Once you're logged in (and have a screen name entered in your profile), you can subscribe to topics, follow users, and start learning how to make apps like the pros.