Esmi2":3ia45vpn said:
What precipitated my paying is that the site was down for a couple of days (for me, anyway) which provided a stark reminder of how much I appreciate it and miss it when it's gone.
First up - thank you!
Regarding the outage...it wasn't technically an outage, as the site would respond - but it was taking up to a minute per page load. Still meandering about on the boat after Friday Harbor, my options for checking in and fixing things weren't as easy as usual.
This issue was rather interesting, never encountered it before. Basically, we were "attacked" by TikTok. They, like many places (Google, Yahoo, MSN, etc.) have spiders that crawl the Internet, gobbling up all data. They've become more aggressive in the age of AI, no doubt.
Anyway, TyBoo noticed we had hundreds of guest users online at any given time - and searching the logs made clear these were TikTik's bots. Well behaved bots, like Google's, do two things. First up, they obey a file put on the site telling them what is OK to crawl and index, and whether they are allowed to crawl/index at all. Second, when they do crawl - they do so slowly. A single connection every few seconds.
TikTok's bot is the most ill-behaved piece o' crap out there. It completely ignores the file that instructs what they can crawl. And then, it bombards the site with about 50 requests per second...something our little hobby site can't handle.
The solution: We're now hosted behind Cloudflare, which speeds things up and has tools to kill off persistent pests like these crawlers. Things look back to normal this AM, but follow up here if you see any issues.
