Komodo 8 results summary

[shrapnel]

Setting drawscore to -7 has much more effect when Komodo is Black, because the opening (with or without book) usually leaves Black down by more than this, and so it will often cause Black to avoid a quick draw right out of book. With White Komodo will not often be down this much early in the game, and later if it is down that much it might mean that the opposing machine is playing better in that particular game and so a draw is much more desirable. So it makes some sense. But I would not expect it to be very noticeable.

Hmm...can't say that I follow all that very clearly, but I get the general idea. Glad to know that my online game observations make some sense.
Actually, I have yet to check out out Komodo with Black using Drawscore 0 against really strong opponents, as I had started out with default -7.
Actually putting Default -7 makes little sense, as most of my online opponents will obviously use the best software available. I suppose it makes sense only for the Rating Lists guys ; in the online gaming world it makes no sense at all, sorry to say ! The two opponents I played today didn't have very good hardware and I won easily with Komodo, Black, DrawScore 0.
Its interesting and a good thing that Komodo DrawScore 0 is NOT exactly the same as Houdini at Contempt =0, which became like a brick wall !
The hypothesis being that if DrawScore 0 is doing well with White against strong opposition, it should do so with Black as well.
I'll check it out .

[lucasart]

I have not yet started looking at a "tactical version". I looked into 960, but there was not enough time to add and test it before the Komodo 8 release. I have a list of things to work on, given time.

I don't understand what's taking so long with Chess960. I don't know how you plan to implement it, but if you do it right, it's very easy. The way I do it, is that I don't have casting rights at all. Instead I just have a single bitboard of castlable rooks (white and.black). Update them the obvious way: when a rook moves, gets captured remove it; when king moves remove own rooks.

You can find my implementation here:
https://github.com/lucasart/DiscoCheck
Quite clean and elegant as you can see. In fact, adding Chess960, was a code simplification, as it introduced uniform treatment of all castlings.

The only detail is the move to uci translation: stupid UCI was created before Chess960 was added, so you can't treat them uniformly, and see Chess as a particular case of Chess960. But that's easy to fix.

[shrapnel]

Hmm... quite disappointed with the performance of Komodo 8 with White with ANY DrawScore against quality opposition ; barely escape with draws.
Surprisingly, I do much better against the same opposition with Black, which makes no sense to me at all !
Contrary to what some here are saying, Komodo 8 performance against Stockfish dips noticeably in the late Middle Game. This area DEFINITELY needs to be worked on by the Engine Authors, in my opinion.
Considering Stockfish's amazing strength in the End-Game, this dip only makes matters worse for Komodo !
Only because I have the 6-Man Syzygy EGTBs, I managed to escape with a Draw.

[Leto]

Hmm... quite disappointed with the performance of Komodo 8 with White with ANY DrawScore against quality opposition ; barely escape with draws.
Surprisingly, I do much better against the same opposition with Black, which makes no sense to me at all !
Contrary to what some here are saying, Komodo 8 performance against Stockfish dips noticeably in the late Middle Game. This area DEFINITELY needs to be worked on by the Engine Authors, in my opinion.
Considering Stockfish's amazing strength in the End-Game, this dip only makes matters worse for Komodo !
Only because I have the 6-Man Syzygy EGTBs, I managed to escape with a Draw.

Can you post a pgn of a game in which this happens?

[shrapnel]

Why not ? Here you are...
[Event "120s/Move"]
[Site "?"]
[Date "2014.09.09"]
[Round "?"]
[White "Komodo 8 64-bit "]
[Black "Stockfish 5 64 SSE4.2"]
[Result "1/2-1/2"]
[ECO "D30"]
[PlyCount "108"]

{16384MB, 1337chess PRO v9 020914.ctg, i7 3930 k} 1. d4 d5 2. c4 {B} c6 3. Nf3
{0.18/23} e6 4. Bg5 {0.32/24} Be7 {0.26/26} 5. Bxe7 {0.23/26} Qxe7 {0.23/23} 6.
e3 {0.25/26} Nf6 {0.22/23} 7. Nc3 {0.23/26} O-O 8. Qc2 {0.23/24} b6 9. cxd5 {
0.21/25} cxd5 {0.19/25} 10. a3 {0.18/26} h6 11. Rc1 {0.20/27} Bb7 {0.25/24} 12.
Bd3 {0.20/28} Nbd7 13. O-O {0.14/26} Rfc8 {0.20/26} 14. Nd2 {0.18/26} a6 {0.17/
21} 15. Qa4 {0.18/26} Ne8 16. Rc2 {0.15/27} b5 17. Qb4 {0.19/29} Nd6 {0.22/26}
18. Nb3 {0.24/28} Nb8 {0.18/25} 19. Ne2 {0.20/28} Nc6 {0.21/25} 20. Qd2 {0.21/
27} Nc4 21. Qd1 {0.21/24} a5 22. Nc5 {0.24/26} N6e5 {0.16/21} 23. Nc1 {0.24/25}
Bc6 {0.14/22} 24. Re1 {0.19/23} Nxd3 25. N1xd3 {0.21/26} f6 26. Qg4 {0.15/27}
Re8 27. Ra1 {0.21/27} a4 {0.16/24} 28. Nf4 {0.14/26} e5 {0.15/24} 29. Ne2 {0.
16/26} Rac8 30. Rac1 {0.15/25} Qf7 31. h4 {0.15/27} Rf8 32. h5 {0.12/27} Rb8 {
0.14/25} 33. Qf3 {0.13/27} Rfd8 34. Ng3 {0.00/28} Bd7 {0.00/24} 35. Nf5 {0.00/
30} Bxf5 {0.00/26} 36. Qxf5 {0.00/29} Re8 37. g3 {0.00/30} Rbc8 {0.00/29} 38.
Kg2 {0.00/29} Ra8 39. Qg6 {0.00/30} Ra7 40. Rd1 {0.00/30} Rc8 41. Kg1 {0.00/30}
e4 {0.00/28} 42. Rdc1 {0.00/32} Rd8 43. Qf5 {0.00/31} Nd6 {0.00/30} 44. Qe6 {
0.00/31} Kf8 {0.00/29} 45. g4 {-0.10/30} Re7 46. Qxf7+ {-0.07/30} Kxf7 {-0.08/
26} 47. Kg2 {-0.10/29} Rg8 48. Rh1 {-0.17/36} Nc4 {-0.17/34} 49. Kg3 {-0.11/40}
Rc8 50. Kf4 {-0.11/38} Ra7 {-0.11/38} 51. Re2 {-0.11/42} Nd6 52. f3 {-0.11/37}
Re7 {-0.11/38} 53. Rc1 {-0.11/40} exf3 {-0.11/40} 54. Kxf3 {-0.11/42} Nc4 {-0.
11/41} 1/2-1/2

[Modern Times]

I compiled a list of all the ratings so far on lists that have both Komodo 7(a) and Komodo 8 as well as Stockfish 5, all with at least 400 games against many opponents.

On 4 CPUs, only CCRL 40/4, which has K8 at 53, SF5 at 46, H4 at 17.

Unfortunately Larry this initial rating didn't hold.

At this point Komodo 8 had about 522 games I think it was, and on the journey from there to 1,000 games it lost 14 Elo. So it now stands as +39 Elo from Komodo 7a 4CPU, and in second place behind Stockfish 5. The error bars are very big though even at 1,000 games, so all we can say is that Stockfish and Komodo 8 are pretty close, and either one could be stronger.

[Vinvin]

Larry, did you overlook this message ?

I think that the main effect of contempt (at least the way we do it) is so that when Komodo plays Black, it will not seek repetitions just because the book normally ends in a White plus. So the benefit should be pretty much just a question of elo difference, it shouldn't matter very much why the gap exists.

Hi Larry
While we are talking of Komodo playing Black, I have found a rather odd paradox in my online matches, which probably isn't very logical, but yet to be disproved, i.e., Komodo plays much better with Black with DrawScore -7 rather than with 0 !
Surprisingly, DrawScore 0 is more required when Komodo is using White, otherwise it tends to lose, at least in LTC games !
I have found this out in a series of online matches with a strong opponent who was using latest Stockfish Dev Version on a i7 like mine.
Any idea why this is happening ?
Also, Komodo seems to work very well without Book in LTC matches at least, probably because most Books have been made with the help of Houdini and Stockfish, which I suppose don't really suit Komodo's playing style and so actually end up hampering Komodo !

Setting drawscore to -7 has much more effect when Komodo is Black, because the opening (with or without book) usually leaves Black down by more than this, and so it will often cause Black to avoid a quick draw right out of book. With White Komodo will not often be down this much early in the game, and later if it is down that much it might mean that the opposing machine is playing better in that particular game and so a draw is much more desirable. So it makes some sense. But I would not expect it to be very noticeable.

To get a more clear view, what drawscore have to be set when :
Komodo 8 is black and opponent is behind by :
50 Elo ?
100 Elo ?
150 Elo ?
200 Elo ?

Komodo 8 is white and opponent is behind by :
50 Elo ?
100 Elo ?
150 Elo ?
200 Elo ?

[mjlef]

Larry has been sick this week, and so I will answer. Our default "Drawscore" is -7, which is good for play against weaker opponents. We raise this higher as the opponent strength goes up, and for a very strong opponent, set it to 0.

The experiments we did with changing this for white or black found nothing better than what we normally do.

[beram]

Larry has been sick this week, and so I will answer. Our default "Drawscore" is -7, which is good for play against weaker opponents. We raise this higher as the opponent strength goes up, and for a very strong opponent, set it to 0.

The experiments we did with changing this for white or black found nothing better than what we normally do.

Hi Mark, your contribution to the further development of Komodo is impressive. About 40 Elo in 3 months is a huge step .
Would you please tell us more about which latest changes did improve the performance of Komodo in particular

Kind regards, Bram

[mjlef]

Larry has been sick this week, and so I will answer. Our default "Drawscore" is -7, which is good for play against weaker opponents. We raise this higher as the opponent strength goes up, and for a very strong opponent, set it to 0.

The experiments we did with changing this for white or black found nothing better than what we normally do.

Hi Mark, your contribution to the further development of Komodo is impressive. About 40 Elo in 3 months is a huge step .
Would you please tell us more about which latest changes did improve the performance of Komodo in particular

Kind regards, Bram

That is a good question. First, this was a team effort, with Larry contributing many ideas. To make a very strong program stronger you have to try a lot of things. Sometimes we make over 100 versions before promoting. So part of the answer is what we did not add, since testing with thousands of games showed the idea not good. And from the beginning of June I have not been at my other job, which gave me more time to work on Komodo.

The basic categories are: speedups, evaluation improvements like better understanding of likely drawish positions, improved bounds and hashing, search improvements to get more depth, and some program restructuring. Some was recognizing you often can get the information you need some other way.

Probably the key is to really understand the search and what you can get away with. Lets say somewhere in the search, you are pruning moves like mad in the null window search. After searching all the moves (with many pruned and not really searched), what can you safely store in the hash if the best move searched was still under the null window? Should you store the null window -1 with an upper bound? Or can you be pretty sure you searched all the likely "best" moves and store whatever the best value you found as an upper bound? Or something in between? How accurate does it have to be to give a good search and not fail on researches? What is in Komodo does not seem to match what anyone else does, but it works. What you do in the null window searches has to be mostly right.

The other key is really good testing. Many thousands of games to prove to some level that an idea works. Larry is a wizard and keeps his computers cooking 24 hours a day.