lkaufman wrote: ↑Tue Dec 18, 2018 12:53 am
I don't think that we can continue to gain five elo per month with regular Komodo. But we have been averaging close to fifty elo per month with MCTS, so even if this drops to something reasonable like twenty per month, after a year everyone should be happy!
Do you think you can sustain 20 Elo per month once Komodo MCTS reaches the current strength of regular Komodo?
If so that would make Komodo incredibly strong in very little time. The Stockfish team took basically took 10 months to gain 50 Elo going from SF9 to SF10. That's 5 Elo per month. 20 Elo per month is 4 times the rate of improvement. It would be unheard of. It would be insane.
It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak. It is likely that progress will be choppy, maybe 10 elo one month, then suddenly 30 or 40 elo when we make a breakthru. We might stall out when reaching normal Komodo level, but there is no particular reason to think so, the searches are VERY different. If we find a way to use NN and GPU the sky is the limit.
So we are looking at a potential new revolution in computer chess. Stay tuned!
Imo to qualify as a revolution, it would need to be 250 elo above stockfish, increasing at a rate of 200-250 a year with no signs of slowing down. A tall order but not impossible.
lkaufman wrote: ↑Wed Dec 19, 2018 7:34 am
It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak.
There was also no obvious limit for alpha zero, bit it started to struggle not soon after it just surpassed stockfish.
It made me think there is a natural limit in chess elo.
lkaufman wrote: ↑Wed Dec 19, 2018 7:34 am
It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak.
There was also no obvious limit for alpha zero, bit it started to struggle not soon after it just surpassed stockfish.
It made me think there is a natural limit in chess elo.
Good point - it's interesting to note the rise in chess Elo from 1995 to 2005 and then from 2005 to 2015 - the second period certainly kept pace with the first period. For years , Crafty was considered the best open source engine, almost by itself really, and that changed, with Fruit and Glaurung, both released approximately released at the same time around 2004. We have a lot to be thankful for those two authors and those authors before them and since then that were willing to share as that has helped with the increases in strength since 2004. But ultimately, yes there will be a limit when chess is solved. It will not happen in my lifetime I suppose, but it will happen.
I had the same problem it seemed to happen anytime I tried to use a .bin book. Once i removed that .bin program seem to have no problems when I ran it as an administrator.
I would like to throw out some comments about Monte Carlo Tree Search. Alpha zero came along and inspired team Komodo to try something similar by the looks of it. It is showing promise and they are in the process of abandoning Alpha Beta search. It looks like Rybka had this feature a few years ago. Quote " Monte Carlo Analysis in Rybka 4: yields precise evaluations by playing thousands of ultra-fast games in a few minutes in a given position. This is very much like using game result statistics, something human players do when choosing their opening variations." Here is an interesting PDF. A lot of it is over my head. http://www.ke.tu-darmstadt.de/lehre/arb ... z_Oleg.pdf
Leo wrote: ↑Wed Dec 19, 2018 5:21 pm
I would like to throw out some comments about Monte Carlo Tree Search. Alpha zero came along and inspired team Komodo to try something similar by the looks of it. It is showing promise and they are in the process of abandoning Alpha Beta search. It looks like Rybka had this feature a few years ago. Quote " Monte Carlo Analysis in Rybka 4: yields precise evaluations by playing thousands of ultra-fast games in a few minutes in a given position. This is very much like using game result statistics, something human players do when choosing their opening variations." Here is an interesting PDF. A lot of it is over my head. http://www.ke.tu-darmstadt.de/lehre/arb ... z_Oleg.pdf
[/quote
Actually I was responsible for the Monte Carlo feature in Rybka; I asked Vas to make it for me to help determine what evals correlated with playouts from various positions, and later he decided just to make it a feature for users. But it has nothing to do with Komodo MCTS or any other current MCTS. Current engines don't use random playouts, they use evals (either from Neural Networks or from short normal searches) to estimate the likely results of random playouts. Much more practical.
lkaufman wrote: ↑Wed Dec 19, 2018 7:34 am
It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak.
There was also no obvious limit for alpha zero, bit it started to struggle not soon after it just surpassed stockfish.
It made me think there is a natural limit in chess elo.
The natural limit in chess elo is purely a function of how testing is done. If no opening books are used we have the repeat game problem. If books that lead to equal positions are used the draw percentage is overwhelming and so the elo limit is near. If books that represent human GM play are used it's not as bad, but still pretty drawish. But if books that lead to advantages near the win/draw line (maybe 0.7 eval on Komodo) are used, the sky is the limit, as a slight improvement might lead to almost all wins for White and almost all draws for Black for an elo gain approaching 200. How we handle the draw problem is critical to the future of chess, of computer chess, and to the limits of the Elo rating scale.
lkaufman wrote: ↑Tue Dec 18, 2018 12:53 am
I don't think that we can continue to gain five elo per month with regular Komodo. But we have been averaging close to fifty elo per month with MCTS, so even if this drops to something reasonable like twenty per month, after a year everyone should be happy!
Do you think you can sustain 20 Elo per month once Komodo MCTS reaches the current strength of regular Komodo?
If so that would make Komodo incredibly strong in very little time. The Stockfish team took basically took 10 months to gain 50 Elo going from SF9 to SF10. That's 5 Elo per month. 20 Elo per month is 4 times the rate of improvement. It would be unheard of. It would be insane.
It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak. It is likely that progress will be choppy, maybe 10 elo one month, then suddenly 30 or 40 elo when we make a breakthru. We might stall out when reaching normal Komodo level, but there is no particular reason to think so, the searches are VERY different. If we find a way to use NN and GPU the sky is the limit.
So we are looking at a potential new revolution in computer chess. Stay tuned!
Imo to qualify as a revolution, it would need to be 250 elo above stockfish, increasing at a rate of 200-250 a year with no signs of slowing down.
I don't need all that!
All i want is that it either wins or draws with stockfish in every game, i.e. rises well above SF.
Once it has done that just one more time, [like it had done in the past] i will feel it has perfected its chess.
What happens later won't bother me.
But once it happens (whilst sf is like it is now), i doubt sf will anyway overtake it again.
There will be no harm after that, if mcts will continue to rise 5-10 elo a month also.
But the kind of chess I've been waiting for, once it is achieved, i don't need more, till infinity. It would be a complete product. A perfect analysis tool! There are other things in life too.
why MCTS search in Komodo is non-determinicstic despite using 1 thread only? I see some randomness during the search. Each run gives slightly different output. For example Leela's search is deterministic on single thread. Here is Komodo 12.3 on 1 CPU in starting position:
