Online since 2002. Over 3300 puzzles, 2600 worldwide members, and 270,000 messages.

TwistyPuzzles.com Forum

It is currently Fri Jul 25, 2014 2:24 pm

All times are UTC - 5 hours



Post new topic Reply to topic  [ 26 posts ] 
Author Message
 Post subject: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 3:28 pm 
Offline
User avatar

Joined: Thu Jan 24, 2002 1:10 am
Location: Toronto, Canada
Our forum was shut down for a few hours due to an unusually heavy server load. My host thinks it may be an attack, but I suspect it's search engine spiders going nuts. Frankly, I have no good ideas of what to do about it yet. I could disable all bots from reading the forum, but that's a drastic move that I don't think I want to take. A spider bot forum style would probably help. I'll look into installing the latest forum software upgrades this evening just in case there is a security hole.

If anyone has any ideas, post them quick, before it goes down again! Heh!

Sandy


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 3:32 pm 
Offline
User avatar

Joined: Fri Feb 08, 2008 1:47 am
Location: near Utrecht, Netherlands
What about keeping track of which IP's make unusually large amounts of requests to the forum and then ban those IP's? Or the IP range if that is required. You could have a simple script make log files for each seperate day. The log file wouldn't need to be extensive, just keep track of the amount of hits for each IP.

_________________
Tom's Shapeways Puzzle Shop - your order from my shop includes free stickers!
Tom's Puzzle Website


Buy my mass produced puzzles at Mefferts:
- 4x4x6 Cuboid for just $38
- Curvy Copter for just $18
- 3x4x5 Cuboid for just $34


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 3:55 pm 
Offline
User avatar

Joined: Thu Jan 24, 2002 1:10 am
Location: Toronto, Canada
Banning the bots from the forum is rather easy since the forum puts all the bots into a single group. You've probably seen the bots listed in the "who's online" section on the "Board index" page. The forum settings for the bots group is basically a black and white "allow" or "ban", though, which makes it undesirable. The consequences to banning spider bots is that search engine users will no longer be able to discover the site. According to the site stats, more people come here from google.com/search than anywhere else (although a large Chinese forum site is close behind). I'd like those users to continue finding the site's content!

It's a pickle. If the host shuts down the site again, I may have no choice.

Sandy


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 3:57 pm 
Offline
User avatar

Joined: Fri Feb 08, 2008 1:47 am
Location: near Utrecht, Netherlands
I wouldn't want all bots being banned from the site. I search on google regularly to find back topics on this site.

Isn't it possible like I suggested to track down the 'problem bots' and just ban those? You don't need to ban using the PHPBB settings, you could also ban them using htaccess files, which would allow you to specify individual IP (ranges).

_________________
Tom's Shapeways Puzzle Shop - your order from my shop includes free stickers!
Tom's Puzzle Website


Buy my mass produced puzzles at Mefferts:
- 4x4x6 Cuboid for just $38
- Curvy Copter for just $18
- 3x4x5 Cuboid for just $34


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 10:58 pm 
Offline
User avatar

Joined: Thu Jan 24, 2002 1:10 am
Location: Toronto, Canada
Just in case you haven't noticed... I'm updating the forum this evening!

Sandy


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:00 pm 
Offline

Joined: Mon Aug 18, 2008 10:16 pm
Location: Somewhere Else
I've been noticing little changes as I navigate between topics. It's kind of funny, actually.


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:01 pm 
Offline
User avatar

Joined: Thu Oct 16, 2008 9:46 pm
Location: Littleton CO
it is a lot better
but how do you get to articles from the forum?


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:01 pm 
Offline

Joined: Mon Jan 05, 2009 9:05 pm
Location: Fort Meade, MD
The top of the screen is looking much better to me already!
& so far everything is looking real clean!

_________________
you know you missed me.


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:04 pm 
Offline
User avatar

Joined: Mon Aug 11, 2008 7:19 pm
Location: Green Bay, Wisconsin
Awesome. I like how now there's a GIANT "board index" button at the top of the page. (the phpbb thingy.)


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:05 pm 
Offline
User avatar

Joined: Mon Nov 19, 2007 9:44 pm
Location: St. Louis, MO
I love this new layout, its cleaner in my opinion

_________________
http://www.youtube.com/avenueq14


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:07 pm 
Offline

Joined: Mon Jan 05, 2009 9:05 pm
Location: Fort Meade, MD
eye2eye wrote:
it is a lot better
but how do you get to articles from the forum?


type it out and add it to your hot bar.


Btw I love that board presence is back up. How about avitars?

_________________
you know you missed me.


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:16 pm 
Offline
User avatar

Joined: Fri Dec 12, 2008 12:16 am
I like the new layout. :mrgreen:

_________________
PB's:
2x2: 6.17 (lucky)| 3x3: 29.79 (Extremely Lucky)
4x4: 3:18.62 | 5x5: 8:22.90
Megaminx: 9:01:94| 7x7: 39:51.98
Magic: 1.14 | Pyraminx: 9.76
2x2-5x5 Relay: 16:43.51


Image


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Wed Feb 25, 2009 11:22 pm 
Offline
User avatar

Joined: Sun Mar 09, 2008 10:27 pm
Location: Florida, USA
Same here I love it very nice :D

_________________
Image
^^Made by me^^
Want a custom sig? Send me a PM.


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 12:50 am 
Offline
User avatar

Joined: Wed Feb 18, 2009 7:03 pm
Location: Oklahoma, USA
Hehe, it's kinda interesting to watch as the changes are made...

I think it will look good once the changes are done.


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 1:13 am 
Offline
User avatar

Joined: Thu Jan 24, 2002 1:10 am
Location: Toronto, Canada
Okay, forum updates complete for the time being. Let me know what I broke.

Sandy


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 1:16 am 
Offline
User avatar

Joined: Mon Sep 09, 2002 2:19 pm
Location: Yaroslavl, Russia and Maryland, USA
I am not sure yet, but I might be missing the "mark all posts read" link.

_________________
Aleksey


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 1:25 am 
Offline
User avatar

Joined: Thu Oct 16, 2008 9:46 pm
Location: Littleton CO
Well the site was down again


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 1:53 am 
Offline
User avatar

Joined: Wed Dec 29, 2004 7:57 am
Location: Στο Κάστρο του Αγίου Γεωργίου
Aleksey wrote:
I am not sure yet, but I might be missing the "mark all posts read" link.

I'm sure already that I miss it.

_________________
...................Image..................
http://cube.helm.lu - http://gallery.helm.lu


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 1:59 am 
Offline
User avatar

Joined: Tue Mar 25, 2008 2:51 am
Location: Malibu, California
The "mark topics read" link in each sub-forum is up and running for me. Is this what you're talking about or was there some other button I never saw?

_________________
I am taking a break from the forum. You can reach me by PM if needed.


Last edited by Danny Devitt on Thu Feb 26, 2009 2:00 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 2:00 am 
Offline
User avatar

Joined: Sat Apr 21, 2007 11:21 pm
Location: Marin, CA
I see the "Mark all forums read" command on the board index page at the top right.

_________________
Jason Smith posted here as 'io' through 2012.
Visit Jason Smith's PuzzleForge on Shapeways!
Jason Smith's Puzzles - YouTube Channel.


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 2:03 am 
Offline
User avatar

Joined: Wed Dec 29, 2004 7:57 am
Location: Στο Κάστρο του Αγίου Γεωργίου
I see both now at the right top, whereas it used to be at the bottom left.

_________________
...................Image..................
http://cube.helm.lu - http://gallery.helm.lu


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 2:14 am 
Offline
User avatar

Joined: Thu Jan 24, 2002 1:10 am
Location: Toronto, Canada
Think of it as a puzzle!

Sandy


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 2:25 am 
Offline
User avatar

Joined: Mon Sep 09, 2002 2:19 pm
Location: Yaroslavl, Russia and Maryland, USA
Sandy wrote:
Think of it as a puzzle!Sandy

Yep! The puzzle solved! 8-)

_________________
Aleksey


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 3:14 am 
Offline
User avatar

Joined: Sun Apr 16, 2000 3:17 pm
Location: Hong Kong
When I try to post on the general topics, the top url address bar is yellow. I have to close a window that opens up. it says this website is phishing? whatever that means. I've never had this before.

_________________
Rox's Rambling Blog
Katsmom's Puzzling Videos


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 4:02 am 
Offline
User avatar

Joined: Fri May 06, 2005 10:13 am
Location: Norway
Hi. Include a robots.txt file to your website. This can specify what bots are allowed to crawl/index what parts of the website. Most legitimate crawlers would follow the "guidelines" politely. You could for instance allow google and yahoo spiders only ...

Per

PS!

Most likely you can analyse bot traffic through your hosting control panel stats section !

_________________
"Life is what happens to you while you are busy making other plans" -John Lennon, Beautiful Boy


Top
 Profile  
 
 Post subject: Re: Forum Downtime - Wednesday Feb 25
PostPosted: Thu Feb 26, 2009 9:22 am 
Offline
User avatar

Joined: Thu Jan 24, 2002 1:10 am
Location: Toronto, Canada
Last night while writing and e-mail to Sausage, I had what I believe to be a great insight into a major contributor to the original downtime. The forum style templates were all being stored in the database. I'm not sure exactly how this happened, but it was like that for a while. Each page that loads in this forum requires at least 3 templates, and often more, which means that many more hits on the database in addition to whatever else is requested from the database to display the page itself. When I updated the forum style last night, I switched this back to using the file system. I expect the load on the database server will be cut in half or more through that one action alone. So for now, Google can continue having it's way with the forum, since I'm paying for the bandwidth it eats up. I will definitely be adding in a stripped down bot style, though, to reduce the load even further. But for now, the pressure is off.

Roxanne, I haven't seen the fishing message in either Explorer 7 or Firefox 2. Can you give me a step-by-step process to reproduce the error? Also, let me know your browser, browser version, and operating system.

Sandy


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 26 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  

Forum powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group