Crowd Sourced processing of Super GMs + Hans for engine correlation

[adnoh]

I have uploaded analysis results for Stockfish 11.

https://onedrive.live.com/?authkey=%21A ... B668FBB6A7

One PGN failed and it is again So-2378164.pgn.analysis.

[adnoh]

I made the changes to setoption for Chess960 and reran the 2 PGN that failed for both Komodo 13.3 and Stockfish 11 and they are now fine.

Code: Select all

    engine.uci_ready()
    engine.write_line('setoption name MultiPV value 3\n')
    engine.write_line('setoption name UCI_Chess960 value true\n')
    engine.uci_ready()

I have updated the OneDrive folders for the 2 engines and they are now both complete, with 4,962 analysis files each.

https://onedrive.live.com/?authkey=%21A ... B668FBB6A7

[AndrewGrant]

What is the solution to wanting to do a multipv search but also really wanting that search to include a given played move
We have a played move M, and the multipv=3 search produces scores for X, Y, and Z.
If M is one of X,Y,Z, then cool everything is nice. But if its not then we need to do a search with M.
But you could then search M and get a higher score than X Y and Z. Which then makes it look like your multipv search was wrong.

I would like to append a score for the played/suggested move to the analysis. But there are a number of ways to do it, all of which seem to contradict themselves. The only true valid one being multipv=allmoves, which is far too costly.

[AndrewGrant]

Alright so I've deleted all of the Fischer Random games, and I have deleted all of the games that had illegal moves at the end of them. I guess this is some usual method of marking a draw? But they are deleted now.

I've pushed updates to...

1. Resumable in case something does go wrong. Files already done and seen will be skipped.

2. Move the PGNs to their own directory.

3. Continue to expand the multipv search until the played move is seen.
This was the best method I could come up with to get a fair score for a given move.

4. Only include the final analysis. I don't even know why I bothered to keep lower depths. It just makes things more confusing.

[RobertJBarker3]

I've been doing some analysis and I think depth 20 is just way to shallow. I'm looking at a move right now that is a blunder at depth 20 and 1st line move at depth 44.

[AndrewGrant]

I've been doing some analysis and I think depth 20 is just way to shallow. I'm looking at a move right now that is a blunder at depth 20 and 1st line move at depth 44.

I mean you are certainly right. but you are not going to do depth 40 analysis with multipv on 5,000 games unless you want to fire up a few thousand cores.

[RobertJBarker3]

I've been doing some analysis and I think depth 20 is just way to shallow. I'm looking at a move right now that is a blunder at depth 20 and 1st line move at depth 44.

I mean you are certainly right. but you are not going to do depth 40 analysis with multipv on 5,000 games unless you want to fire up a few thousand cores.

I think we might have no choice. If we do too shallow of an analysis, the data will be useless. We're probably better off checking fewer games with more precision. I'll think about this more tomorrow, but if we can make a queue for pngs and outsource it to whoever (begging on reddit maybe), then we might be able to get enough games.
I just think if we go too shallow, it'll all be done for nothing.

[AndrewGrant]

I've been doing some analysis and I think depth 20 is just way to shallow. I'm looking at a move right now that is a blunder at depth 20 and 1st line move at depth 44.

I mean you are certainly right. but you are not going to do depth 40 analysis with multipv on 5,000 games unless you want to fire up a few thousand cores.

I think we might have no choice. If we do too shallow of an analysis, the data will be useless. We're probably better off checking fewer games with more precision. I'll think about this more tomorrow, but if we can make a queue for pngs and outsource it to whoever (begging on reddit maybe), then we might be able to get enough games.
I just think if we go too shallow, it'll all be done for nothing.

I don't think the data is going to be very useful anyway. I'm expecting to show that all players play roughly the same number of engine moves. Players have similar cp losses in their games. Players have similar rates of playing the only moves, etc.

My motivation here is that there are a ton of people abusing "data" and "statistics" to find the conclusion they want. And they do it with tiny data and cherry picking. Well take 5,000 games from Top 7 + Hans and then do the same thing. Good luck. If there are cheated in the mix, you won't find them with such basic methods.

The data is "Interesting", however. And I would need evidence to accept that d20 engines are far inferior to humans. Because I don't think that is the case. Depth 20 is a few seconds per move. Good luck playing against that in 3+0.

[RobertJBarker3]

I'm done with the stockfish7 analysis. Do I push the results on a branch or something?