When will the chess programmers write an engine that plans ?

[Zenmastur]

Ha, the funny thing is that mass storage isn't even needed, Bernstein Learning shows how little the engine needs to save in a file for it to work. We're talking about 64MB per year or something even storage from 10 years ago could manage without sweating.

At 2.5Mnps if you're not storing anything until some minimum depth of search is reached that's probably true.

I did a multiPV 12 search a couple of nights ago and left it run overnight. All lines got to depth 72 and a few reached depth 73. If it were storing all positions searched greater than depth 50, it's just a wild guess, but I pretty sure it would use more than 64MB a year if it were done every night.

[Ovyron]

As wild guesses go my upper-bound is 2GBs/year, even I have space for that...

An interesting part is the "Merge Persistent Hash" option, when it was implemented the idea was for different users to check different lines and then share the files and merge them together, perhaps with one master file that got the analysis shard by everyone. Depending of the people contributing the file could become much larger, but everyone with the file could have the engine reaching very high depths with accurate move choices very fast (as if they weren't the engine would learn and the next iteration will approach that.)

I wonder why this never got off the ground, much more time was devoted finding out if some engine's rating was 3402 or 3398...

[Dann Corbit]

Ha, the funny thing is that mass storage isn't even needed, Bernstein Learning shows how little the engine needs to save in a file for it to work. We're talking about 64MB per year or something even storage from 10 years ago could manage without sweating.

Hyatt beat him to it, and the main problem is it doesn't really work. By that I mean in a real chess game, unplayed moves happen pretty soon and then you are back to square zero, and not only that you don't have as much spiffy stuff stored in the hash because you have not been analyzing as much, with all those special deep cutoffs in the hash.

[Ovyron]

It does really work if you specialize in certain openings, that TCEC version with learning was still outpacing Stockfish 7 with all its learning on them. Since Stockfish 8 there have been important shifts in evals, and entire openings would die (<0.19 evals were learned for positions that were actually >1.00), and it has gotten worse since then (Stockfish 10 with learning stands no chance against Stockfish dev), but this is something worth looking for if improvements plateau (I've seen conflicting reports about this, is Stockfish 12 still going to be 50 elo stronger than 11? Will we have to wait until 2023? Can the exponential pace be kept?)

[Zenmastur]

Ha, the funny thing is that mass storage isn't even needed, Bernstein Learning shows how little the engine needs to save in a file for it to work. We're talking about 64MB per year or something even storage from 10 years ago could manage without sweating.

Hyatt beat him to it, and the main problem is it doesn't really work. By that I mean in a real chess game, unplayed moves happen pretty soon and then you are back to square zero, and not only that you don't have as much spiffy stuff stored in the hash because you have not been analyzing as much, with all those special deep cutoffs in the hash.

If the point of it is to save time it clearly does work. The retrieval times, assuming the stored positions are NOT in memory, can be less than 1ms. With them in memory at the start of the game your talking micro-seconds. Even fast games in a testing framework aren't so fast that positions stored on disk wouldn't be of benefit.

While it may be true that the the engine will see positions not in the position memory that's not the same as “being back to square zero”. It's very likely that the stored positions are much better than something that could be calculated in the short time available for a given move. This means avoiding past inaccuracies and mistakes.

It takes most engine a while to calculate an accurate move. If this has already been done in a previous encounter and the amount of time available to make a move is the same (or less) then it makes sense to use the stored data. If the amount of time available to make the move is much greater than the time it took to calculate the stored move then it makes sense to calculate a new move, make it across the board and then store the new results.

When you reach a position not stored there is plenty of time for the engine to fill the TT with positions that are much more relevant to the current position than if you had expended this time calculating several past moves from scratch.

In short, I don't buy the “it doesn't work” routine. It may not be a panacea but it's definitely useful. I personally don't use this method due to a lack of a “good” engine that supports it. I haven't purchased an engine in many years. I think ChessMaster series was the last time I forked over money for an engine (27 years ago?). I could be persuaded to buy an engine like Houdini or Komodo were they to offer a robust version of this that had a published position format that anyone could use/and or program to. I figure the open format would gain more support than a proprietary approach and more engines would eventually support it.

Regards,

Zenmastur

[jp]

I for one have always been interested in a chess program that can emulate human play, including some of the human weaknesses.

The challenge is to emulate strong humans. The more a programmer is happy to sacrifice in playing strength, the easier it becomes to make it fake (weak) humans.

Yes, I am biased towards strength rather than weakness, but we would need to have human error sprinkled into the chess moves. Some mistakes should be refutable [by that I mean by other humans, and not engines], others not as easily, just like with a strong human player's moves. AFAIC, I would like to see a human-emulator somewhere between master and average GM strength. One's mileage may vary, of course.

Some of the commercial engines' attempts have been appalling. Fritz had this "sparring mode" that played random unhumanlike errors. One current engine appears to be playing its second-best move all the time, so when there's an obvious mate coming from the first move but only that one, it ridiculously ignores the mate.

[carldaman]

Yes, Fritz sparring features fall well short of expectations. I would like to take another look at MB's Weakfish with reduced strength settings.

[carldaman]

A LC0+Bernstein Learning engine would be very interesting (the reason Leela was so useless in my 1.g4 games was that one would show it how a move she scored as -0.90 was actually -4.00, but going to a previous position it'd show -0.90 again, without a way to recognize this line lost from the root so it'd suggest a different plan for black.)

That's a pathetic analysis shortcoming for Leela. I understand it uses a different hash table 'architecture' because of the different search type, etc, but still, it seriously limits one's analysis options by not propagating the score backward.

[jp]

Yes, Fritz sparring features fall well short of expectations. I would like to take another look at MB's Weakfish with reduced strength settings.

That reminds me of something. There were tests that only a couple of decades ago were done with pen and paper for a few hours. Then the educational bodies decided that their computer programs were so clever that the standard tests would be replaced with shorter tests done on computers. The computer program (which I assume is the new standard test) is supposed to be able to judge in real time the level of the test taker, so that the questions can be adjusted to the test taker's level, allowing an accurate scoring in only a small number of questions.

Anyone who's taken one of these tests will know that it's about as effective as Fritz's sparring mode. Everyone knows that when the program starts asking you the questions you'd ask your five-year-old son that means you made a blunder and now the computer thinks you're an idiot and the only chance of getting a decent score (that you deserve) on the test is to answer the next six stupid questions at Stockfish microbullet speed, so that it then can return to asking normal questions.


When will humans learn to stop over-estimating the abilities of computer programs? It's like that in computer chess too.

[Uri]

I believe that even the top chess engines are still much much weaker than the best human chess players.

Our software has some serious limits, otherwise where do all of the improvements and upgrades come from?

Humans understand chess strategy much better than even the best chess engines available and humans even have much more chess knowledge and positional understanding than a chess engine could possibly ever have.

Maybe in a million years from now I may be proven wrong but until then I won't hold my breath.