Zenmastur wrote: ↑
Thu Jul 11, 2019 10:21 am
That seems a bit odd. I would think it would be easy to figure out when Leela is better and when SF is clearly better. When one side is ahead SF can find good plans easily.
Are you a programmer? Perhaps you could program something that is better than current hybrid efforts. Like Aiquiry but running A/B and NN in parallel. Somehow a current problem seems to be that both need to be of about the same level in strength, otherwise, if Leela is on a weak GPU when its move is played it's a worse move than what Stockfish alone would have played, so that the entire game would have been played better with Stockfish alone. An hybrid system is needed that manages to play as good as one of the entities alone, or better, no matter how bad are the suggestions of the other entity most of the time.
Zenmastur wrote: ↑
Thu Jul 11, 2019 10:21 am
There are a huge number of ways that NN's could be used in an AB engine to improve it's play. I can even imagine using small NN to find new rules that then can be coded normally without putting NN in the engine proper. Mostly for simple things that most engine programmers are just guessing at now. I suspect that LMR and pruning would be much better ( meaning the could be used more aggressively) if controlled by a relatively simple NN.
It seems at the current time most people haven't moved to Leela because they already have fast CPU and because the improvement from going from CPU to GPU is not worth the investment (as Leela is better than SF, but not by much.) So if the Leela people can run is subpar, a hybrid effort that makes it play better is not useful to them.
What we need (at least until Leela's edge is significant) is a hybrid implementation that uses Leela (on whatever weak GPU the user already has) to improve Stockfish's play (in the fast CPU people already got for computer chess.) But those haven't focused on improving A/B's engine source code yet.
But it seems to be echoing your sentiment that extra searching does little for AB engines under some circumstances.
I'd go as far as saying "under most circumstances." It's unbelievable that I'm performing only 30 ELO worse than people with fast Intel i7s. I think this is another problem hidden by generic book testing, where sub-optimal positions are reached and in those the extra resources are used effectively, but when the weak side plays into optimal positions the extra resources don't make any difference.
There is a “Proper” way to create a book for an engine based on it's true performance as opposed to it's expected win percentage based on it's scoring function. When it's had no experience in a particular line it can fall back on it's expected win percentage based on it's scoring function. But, in order for this to work effectively the program has to be able to choose it's own opening play.
Heh, recently I was commenting about how someone built a Cluster with that approach in mind, and the Cluster eventually arrived at "1.e4 e5 2.Qf3!?" with its learning and reaching won positions against its opponents, and people were commenting about how sinful that was and about how that variation was bad no matter what, by chess principles. I wonder if opening theory of the future will look absolutely alien compared to what we have today, just like it happened in Go.
Well, I haven't seen a game where one side won and I couldn't find one or more moves where the losing side made errors. But if you have one that qualifies I'd like to take a look at it.
I didn't say the losing side wasn't making errors, I said A/B engines don't see those errors in analysis. What happens is the A/B engine thinks its side has the advantage, and thinks the best play for the opponent is to play for a draw, and it can't refute the draw, so it shows 0.00 scores in analysis. But Leela knows she has the advantage, and then the A/B engine realizes this, it's too late.
I believe here's such a game where Leela defeats a Stockfish derivative running on 12 cores, where I believe A/B engines would show a move as a "mistake" even though the mistake was played several moves earlier:
The opposite also happens, here's a game where Leela running on the fastest GPUs I know of lost against a Stockfish derivative running on hardware slower than mine:
What hardware do you use? NPS?
Q8200 2.33 GHz (I bought this one specifically because it was easy to overclock and it's a quiet CPU, but I never did it because for all my needs it was fast enough.)
128MB RAM (never found any advantage whatsoever from giving more than this to engines)
3CPU (the GUI is buggy and requires an entire core dedicated to it to remain stable...)
Around 2000 kn/s in average.
I generally like very deep analysis in my games. My problem is that the is so much analysis that it's sometimes hard to find the best move after all the analysis is done. The only game I lost in CC was one in which I transposed moves twice in the same game and made another that was not in any of my analysis. It took three mistakes in one game to force a loss. The last mistake was a transposition mistake I made after I had spent a week on it and found a line that would have equalized the position. Boy was I mad at myself after that move!
Have you tried playing at the LSS? (chess-server) I have found that people tend to cause more trouble there when compared to similarly rated opponents from other sites, and if you already have an ICCF rating, they allow you to start there so you skip the three years it'd require you to beat weaklings to reach it.
Great spirits have always encountered violent opposition from mediocre minds.