Polish users cut off from TalkChess

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Guenther
Posts: 3662
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Polish users cut off from TalkChess

Post by Guenther » Sun Dec 06, 2020 12:15 pm

PK wrote:
Sun Dec 06, 2020 8:27 am
@Guenther, you have a nice crawler over there! I have one friend who can investigate options of putting at least programming part of the forum somewhere, so stay in touch.
Haha, I did not assume someone would have linked an external 158GB file to the forum...
http://talkchess.com/forum3/viewtopic.p ... se#p806561
(added a max size for non htm files now @30000000 Bytes)

Guenther
Posts: 3662
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Polish users cut off from TalkChess

Post by Guenther » Tue Dec 15, 2020 8:56 am

Guenther wrote:
Sun Dec 06, 2020 12:15 pm
PK wrote:
Sun Dec 06, 2020 8:27 am
@Guenther, you have a nice crawler over there! I have one friend who can investigate options of putting at least programming part of the forum somewhere, so stay in touch.
Haha, I did not assume someone would have linked an external 158GB file to the forum...
http://talkchess.com/forum3/viewtopic.p ... se#p806561
(added a max size for non htm files now @30000000 Bytes)
Well, after crawling for around nine days I had to abandon this for now. It blocked my (old) computer for too long and the result somehow was not convincing enough. Moreover after reaching around 85% it needed another day to add 2-3 percent, with still growing link numbers.
(Noticed also that it would not be possible to synchronize with already available links e.g. in CPW, because of a randomly added hex number with 4 digits, in the middle of the new created links)

Probably I should have found better settings at the beginning.
May be I will try it again later, with more cautious settings and less depth.
OTH it would be much simpler, if ChessUSA would create a database backup and make it available somewhere.

Pi4Chess
Posts: 220
Joined: Mon Nov 16, 2020 11:13 am
Full name: Manuel Rivera

Re: Polish users cut off from TalkChess

Post by Pi4Chess » Fri Dec 18, 2020 4:32 pm

Guenther wrote:
Sat Dec 05, 2020 5:27 pm

I am currently in the process of saving the most important parts of the forum by an old tool, which still seems to work like a charm.
(I don't want to put too much stress on the host server so I limited the bandwidth for downloading)

This will take a while and I have to check first, if the saved archives work at all later.
If all works out as desired, I will report back in a few days.
I don't think I will save everything and will filter manually 'not so useful' content, at least from the General forum.

Image

I have no clue how big it will be and hopefully it won't crash my computer over night ;-)
Oh, I forgot to say I started of course with the Programmers forum :)
After this I will try the General forum. The others are not that important.

Edit:
A first check is very promising - everything is working as expected in the saved archives structure!

BTW ofc I also tried wayback on this and in fact talkchess was saved hundredths of times in the past, BUT the saved archives
never go deeper than the thread/post titles, which means those are useless.

First guess for the programmers part ~5GB (total links will still increase further)
This will make the General part something like 20-50GB
It reminds me of old Website aspirator HTTrack ^^ It was not easy to configure parameters but you could exclude link to files more than x Mb i remember.

User avatar
Sam Hull
Posts: 5804
Joined: Tue Mar 14, 2006 8:19 am
Location: The Cherokee Nation
Full name: Sam Hull

Re: Polish users cut off from TalkChess

Post by Sam Hull » Wed Dec 23, 2020 4:57 pm

PK wrote:
Sat Dec 05, 2020 7:35 am
peter wrote:
Sat Dec 05, 2020 6:15 am
But even less I would simply let the data stored here go without even trying to save them in one way or the other.
I agree. But Talkchess is no longer a place where you can reliably store and retrieve information.
Email Quentin Turner at YMC&G directly with your problems and concerns (info@chessusa.com). If there is a sufficient volume of requests I would expect there to be some action. I have forwarded all requests sent to me about unblocking, but the last couple of months I have gotten no replies.

-Sam-

User avatar
mclane
Posts: 18368
Joined: Thu Mar 09, 2006 5:40 pm
Location: US of Europe, germany
Full name: Thorsten Czub
Contact:

Re: Polish users cut off from TalkChess

Post by mclane » Wed Dec 23, 2020 11:41 pm

Sam you are not the solution. If there would be a solution to the problem you would not be informed.
What seems like a fairy tale today may be reality tomorrow.
Here we have a fairy tale of the day after tomorrow....

Post Reply