Komodo 12.3 is out

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
duncan
Posts: 10366
Joined: Mon Jul 07, 2008 8:50 pm

Re: Komodo 12.3 is out

Post by duncan » Wed Dec 19, 2018 12:54 pm

lkaufman wrote:
Wed Dec 19, 2018 6:34 am
Gary Internet wrote:
Tue Dec 18, 2018 2:02 pm
lkaufman wrote:
Mon Dec 17, 2018 11:53 pm
I don't think that we can continue to gain five elo per month with regular Komodo. But we have been averaging close to fifty elo per month with MCTS, so even if this drops to something reasonable like twenty per month, after a year everyone should be happy!
Do you think you can sustain 20 Elo per month once Komodo MCTS reaches the current strength of regular Komodo?

If so that would make Komodo incredibly strong in very little time. The Stockfish team took basically took 10 months to gain 50 Elo going from SF9 to SF10. That's 5 Elo per month. 20 Elo per month is 4 times the rate of improvement. It would be unheard of. It would be insane.

It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak. It is likely that progress will be choppy, maybe 10 elo one month, then suddenly 30 or 40 elo when we make a breakthru. We might stall out when reaching normal Komodo level, but there is no particular reason to think so, the searches are VERY different. If we find a way to use NN and GPU the sky is the limit.
So we are looking at a potential new revolution in computer chess. Stay tuned!

Imo to qualify as a revolution, it would need to be 250 elo above stockfish, increasing at a rate of 200-250 a year with no signs of slowing down. A tall order but not impossible.

duncan
Posts: 10366
Joined: Mon Jul 07, 2008 8:50 pm

Re: Komodo 12.3 is out

Post by duncan » Wed Dec 19, 2018 1:51 pm

lkaufman wrote:
Wed Dec 19, 2018 6:34 am

It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak.
There was also no obvious limit for alpha zero, bit it started to struggle not soon after it just surpassed stockfish.

It made me think there is a natural limit in chess elo.

MikeB
Posts: 3467
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: Komodo 12.3 is out

Post by MikeB » Wed Dec 19, 2018 2:27 pm

duncan wrote:
Wed Dec 19, 2018 1:51 pm
lkaufman wrote:
Wed Dec 19, 2018 6:34 am

It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak.
There was also no obvious limit for alpha zero, bit it started to struggle not soon after it just surpassed stockfish.

It made me think there is a natural limit in chess elo.
Good point - it's interesting to note the rise in chess Elo from 1995 to 2005 and then from 2005 to 2015 - the second period certainly kept pace with the first period. For years , Crafty was considered the best open source engine, almost by itself really, and that changed, with Fruit and Glaurung, both released approximately released at the same time around 2004. We have a lot to be thankful for those two authors and those authors before them and since then that were willing to share as that has helped with the increases in strength since 2004. But ultimately, yes there will be a limit when chess is solved. It will not happen in my lifetime I suppose, but it will happen.

NJDenson
Posts: 75
Joined: Tue May 23, 2006 5:01 pm

Re: Komodo 12.3 is out

Post by NJDenson » Wed Dec 19, 2018 2:57 pm

I had the same problem it seemed to happen anytime I tried to use a .bin book. Once i removed that .bin program seem to have no problems when I ran it as an administrator.

Leo
Posts: 839
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo

Re: Komodo 12.3 is out

Post by Leo » Wed Dec 19, 2018 4:21 pm

I would like to throw out some comments about Monte Carlo Tree Search. Alpha zero came along and inspired team Komodo to try something similar by the looks of it. It is showing promise and they are in the process of abandoning Alpha Beta search. It looks like Rybka had this feature a few years ago. Quote " Monte Carlo Analysis in Rybka 4: yields precise evaluations by playing thousands of ultra-fast games in a few minutes in a given position. This is very much like using game result statistics, something human players do when choosing their opening variations." Here is an interesting PDF. A lot of it is over my head. http://www.ke.tu-darmstadt.de/lehre/arb ... z_Oleg.pdf
Advanced Micro Devices fan.

User avatar
Nordlandia
Posts: 2454
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Komodo 12.3 is out

Post by Nordlandia » Wed Dec 19, 2018 5:57 pm

Is probe depth value of 2 still applying for 5-men syzygy?

lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 12.3 is out

Post by lkaufman » Wed Dec 19, 2018 6:09 pm

Leo wrote:
Wed Dec 19, 2018 4:21 pm
I would like to throw out some comments about Monte Carlo Tree Search. Alpha zero came along and inspired team Komodo to try something similar by the looks of it. It is showing promise and they are in the process of abandoning Alpha Beta search. It looks like Rybka had this feature a few years ago. Quote " Monte Carlo Analysis in Rybka 4: yields precise evaluations by playing thousands of ultra-fast games in a few minutes in a given position. This is very much like using game result statistics, something human players do when choosing their opening variations." Here is an interesting PDF. A lot of it is over my head. http://www.ke.tu-darmstadt.de/lehre/arb ... z_Oleg.pdf
[/quote

Actually I was responsible for the Monte Carlo feature in Rybka; I asked Vas to make it for me to help determine what evals correlated with playouts from various positions, and later he decided just to make it a feature for users. But it has nothing to do with Komodo MCTS or any other current MCTS. Current engines don't use random playouts, they use evals (either from Neural Networks or from short normal searches) to estimate the likely results of random playouts. Much more practical.
Komodo rules!

lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 12.3 is out

Post by lkaufman » Wed Dec 19, 2018 6:21 pm

duncan wrote:
Wed Dec 19, 2018 1:51 pm
lkaufman wrote:
Wed Dec 19, 2018 6:34 am

It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak.
There was also no obvious limit for alpha zero, bit it started to struggle not soon after it just surpassed stockfish.

It made me think there is a natural limit in chess elo.
The natural limit in chess elo is purely a function of how testing is done. If no opening books are used we have the repeat game problem. If books that lead to equal positions are used the draw percentage is overwhelming and so the elo limit is near. If books that represent human GM play are used it's not as bad, but still pretty drawish. But if books that lead to advantages near the win/draw line (maybe 0.7 eval on Komodo) are used, the sky is the limit, as a slight improvement might lead to almost all wins for White and almost all draws for Black for an elo gain approaching 200. How we handle the draw problem is critical to the future of chess, of computer chess, and to the limits of the Elo rating scale.
Komodo rules!

S.Taylor
Posts: 8348
Joined: Thu Mar 09, 2006 2:25 am
Location: Jerusalem Israel

Re: Komodo 12.3 is out

Post by S.Taylor » Wed Dec 19, 2018 6:40 pm

duncan wrote:
Wed Dec 19, 2018 12:54 pm
lkaufman wrote:
Wed Dec 19, 2018 6:34 am
Gary Internet wrote:
Tue Dec 18, 2018 2:02 pm
lkaufman wrote:
Mon Dec 17, 2018 11:53 pm
I don't think that we can continue to gain five elo per month with regular Komodo. But we have been averaging close to fifty elo per month with MCTS, so even if this drops to something reasonable like twenty per month, after a year everyone should be happy!
Do you think you can sustain 20 Elo per month once Komodo MCTS reaches the current strength of regular Komodo?

If so that would make Komodo incredibly strong in very little time. The Stockfish team took basically took 10 months to gain 50 Elo going from SF9 to SF10. That's 5 Elo per month. 20 Elo per month is 4 times the rate of improvement. It would be unheard of. It would be insane.

It would be unrealistic to predict 20 elo per month gain once we catch normal Komodo, but the main point is that there is no obvious limit or sense that we are anywhere near any kind of peak. It is likely that progress will be choppy, maybe 10 elo one month, then suddenly 30 or 40 elo when we make a breakthru. We might stall out when reaching normal Komodo level, but there is no particular reason to think so, the searches are VERY different. If we find a way to use NN and GPU the sky is the limit.
So we are looking at a potential new revolution in computer chess. Stay tuned!

Imo to qualify as a revolution, it would need to be 250 elo above stockfish, increasing at a rate of 200-250 a year with no signs of slowing down.
I don't need all that!
All i want is that it either wins or draws with stockfish in every game, i.e. rises well above SF.
Once it has done that just one more time, [like it had done in the past] i will feel it has perfected its chess.
What happens later won't bother me.
But once it happens (whilst sf is like it is now), i doubt sf will anyway overtake it again.
There will be no harm after that, if mcts will continue to rise 5-10 elo a month also.
But the kind of chess I've been waiting for, once it is achieved, i don't need more, till infinity. It would be a complete product. A perfect analysis tool! There are other things in life too.

Karol Majewski
Posts: 94
Joined: Sat Jul 09, 2011 2:18 pm

Re: Komodo 12.3 is out

Post by Karol Majewski » Wed Dec 19, 2018 7:15 pm

Hi Larry,

why MCTS search in Komodo is non-determinicstic despite using 1 thread only? I see some randomness during the search. Each run gives slightly different output. For example Leela's search is deterministic on single thread. Here is Komodo 12.3 on 1 CPU in starting position:

First run:
0.02: +0.39/9 1.e4 Nc6 2.d4 d5
0.04: +0.35/11 1.e4 e5 2.Nc3 Nc6
0.06: +0.33/12 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6
0.10: +0.28/13 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.O-O
0.18: +0.20/14 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Nc3 Nxe4
0.32: +0.17/15 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.d3 Bc5 5.O-O
0.36: +0.18/16 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.d3 Bc5 5.O-O
0.50: +0.17/16 1.c4 c5 2.e3 e6 3.a3 Nf6
1.02: +0.20/17 1.c4 c5 2.e3 e6 3.a3 Nf6
1.18: +0.20/17 1.e3 Nf6 2.d4 d5 3.c4 c6 4.cxd5 cxd5
1.56: +0.20/18 1.e3 c5 2.Nf3 d5 3.c4 d4 4.exd4 cxd4 5.b4 a5
3.06: +0.21/19 1.e3 c5 2.Nf3 a6 3.d4 e6 4.d5
3.46: +0.23/20 1.e3 c5 2.Nf3 a6 3.d4 e6 4.d5
4.48: +0.20/20 1.Nf3 c5 2.e4 e6 3.c4 Nc6 4.Be2 Nf6 5.Nc3 d5 6.exd5 exd5 7.cxd5
7.24: +0.18/21 1.Nf3 c5 2.e4 e6 3.h3 Nc6 4.Bb5 Nd4 5.Be2 d5 6.Nxd4
8.10: +0.18/22 1.Nf3 c5 2.e4 e6 3.h3 Be7 4.Nc3 Nc6 5.Bb5 Nf6 6.e5


Second run:
0.02: +0.39/9 1.e4 Nc6 2.d4 d5
0.04: +0.35/11 1.e4 e5 2.Nc3 Nc6
0.06: +0.33/12 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6
0.12: +0.25/13 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.O-O
0.18: +0.20/14 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Nc3 Nxe4
0.32: +0.17/15 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.d3 Bc5 5.O-O
0.36: +0.17/16 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.d3 Bc5 5.O-O
0.52: +0.17/16 1.c4 c5 2.e3 e6 3.a3 Nf6
1.02: +0.20/17 1.c4 c5 2.e3 e6 3.a3 Nf6
1.18: +0.20/17 1.e3 Nf6 2.d4 d5 3.c4 c6 4.cxd5 cxd5
1.56: +0.20/18 1.e3 c5 2.Nf3 d5 3.c4 d4 4.exd4 cxd4 5.b4 a5
3.06: +0.22/19 1.e3 c5 2.Nf3 a6 3.d4 e6 4.d5
3.44: +0.24/20 1.e3 c5 2.Nf3 a6 3.d4 e6 4.d5
4.46: +0.20/20 1.Nf3 c5 2.e4 e6 3.c4 Nc6 4.Be2 Nf6 5.Nc3 d5 6.exd5 exd5 7.cxd5
7.20: +0.18/21 1.Nf3 c5 2.e4 e6 3.Nc3 Nc6 4.Bb5 Nd4 5.O-O a6 6.Bd3 Nxf3+ 7.Qxf3
11.04: +0.17/22 1.Nf3 c5 2.e4 e6 3.h3 Nc6 4.Bb5 Nd4 5.Be2 d5 6.d3 Be7


Best
Karol

Post Reply