Longshot question about a non-DU discussion forum

[Orrex]

I'm the admin a hobbyists' forum on the Hyperboards server, and about a week ago Hyperboards suddenly and summarily announced that they would be shutting down permanently on December 1st, with a total loss of all data.

I can reproduce much of the information because I maintain it in a database on my own system, but a great many posts contain information that isn't similarly backed up.

The format is pretty standard BB-code stuff, and I was wondering if there's some sort of app or utility for downloading or otherwise harvesting/archiving information posted in that format.

Does such a tool exist, or am I out of luck?


Any suggestions you can offer are much appreciated. This is an inconvenience for me, but other forums on Hyperboards stand to lose tens of thousands of memers' posts spanning fifteen years or so.


cross-posting to the Lounge.

[steve2470]

1. if you get no answers here or in the Lounge, you might try this forum

https://www.democraticunderground.com/?com=forum&id=1098

Website, DB, & Software Developers

[Orrex]

2. Thanks for the tip. I'll check it out.

[mahigan]

3. I know I have downloaded a website scraper

some time ago but I also know I have never used it. I'll try to find it later today. Meanwhile, you might be able to find something at one of these links. [link:https://www.bigdatanews.datasciencecentral.com/profiles/blogs/top-30-free-web-scraping-software| or here [link:https://www.hongkiat.com/blog/web-scraping-tools/|

Please let us know if you find something that might work.

[Egnever]

4. Time is running out

Do you have access to the backend of the site or just the manegment console?

If you can get to the folder structure of the site I would jump on there using FTP if possible and download all of the folders immediately. That at least should give you the ability to cull through the data at your own pace instead of being under the gun.

I am not aware of a specific program to do what you are asking websites aren't really my thing discussion boards even less so. Still they are still setup in a standard folder structure and if you can access the server directly you should be able to copy all of the data.

[Orrex]

5. That's a good suggestion, but alas I don't appear to have FTP access

The admins are, in a word, inaccessible, and they've made no comment beyond the 10-day shutdown warning.

The Wayback Machine happily preserves a lot of what I thought would be lost, so I have some wiggle room there.

[Egnever]

7. ugh

that really sucks I am sorry you are facing this. Make sure you check all the tools theydo make available to see if there is away to access that back end. sometimes there are goofy http programs provided by the host as well.

Best of luck sorry I can't be more help.

[Egnever]

8. hmm another thought

Do you have access to any web design software like Dreamweaver or something similar? You might be able to use one of them to log into the site to gain access to the folder structure..

[Orrex]

9. I'll check on that--thanks again!

[mahigan]

6. Egnever is correct

If you actually have access to the server, downloading the the folders with ftp is your best bet. Scraping the site is your last alternative. I did a little looking in my download directory and the scraper I downloaded but never used is called NeoDownloader. Another I came across is called Visual Web Ripper. Please bear in mind that I haven't used either of these applications myself - they may do what you need but but try Egnever's solution first. Good luck.