Will a new Komodo come out in the near future?

[lkaufman]

and now KomodoMCTS 2301.00 gets wiped off the board by AllieStein V0.3dev-n4....
Like it's loss to Xiphos 0.5.2, it was not even close.

Both games were with Black. I took a few minutes and ran thru each...other similarities:

1. Komodo did not appreciate it was getting itself into positions where there would be no activity (Look at its light squared bishop in each case - against AllieStein it sits on g6 and shoots thru thin air...controlling what turns out to be a useless diagonal.)
2. It does not try for the standard (I've played the Sveshnikov type systems for years seen about move 14)...f5 to get more control of the center squares...perhaps the seeds of defeat ironically lie in 10...Bc8 instead of 10...Bf5, when it could tuck the bishop into g6, move the Nf6 and play...f5 to get some control of the central squares. The Bishop might then find employment on f7, looking at those White center pawns. I don't know, just spitballing here. Perhaps though the book used contained ...Bc8. Not necessarily 'bad', but that bishop flounders as the game unfolds.

I've also played the Benoni since I was a kid (not any more...too problematic against better players when I do not play OTB myself as much as I once did) and its loss to Xiphos was truly sad. Why on earth play 8...Nbd7 instead of simply 0-0? It adds nothing to the game and does not aid in trying to find the activity needed when playing the Benoni. Perhaps though this is simply the book move it was forced to play and the line chosen was simply to see how engines handled hopelessly cramped positions?

Is perhaps contempt set high for this phase of the event?
Are the programmers testing if Komodo can out-think' competition at this level instead of out calculate them?
Or perhaps it simply does not handle (maybe none of the engines do?) cramped, passive positions?

Personally, I do not care at all that it would lose games to these engines - Komodo has always been an 'analysis tool' for me (and others) and any decent player knows not to trust an engine at certain points - like when the position might get cramped.I currently use Komodo 11.3.

To satisfy my curiosity, I would like to know what the parameters are for this version of Komodo and why. Also, one can tweak and run millions of games at fast time controls and let statistics push where they are going - one can argue that one really doesn't need a GM level player involved in the programming for that...but how much do the programmers pay attention to these losses at slower time controls?

As a many time purchaser of Komodo (NEVER bought Houdini or any other commercial engine in the past decade), I am curious as to what the authors of Komodo (and other keen-eyed individuals) think of these two defeats...and what the parameters currently in use for TCEC are.

In general it is not productive for us to spend time trying to "fix" a loss, unless it is due to a bug. It is usually easy enough to make a change to the program to correct some inferior move played in a game, but most of the time the fixed version will play worse overall. So for example if the problem in the alliestein game was failure to play ...f5, tdhe reason was presumably that Komodo judged the harm to king safety to exceed the benefit. The fix would be to reduce the value of the f7 pawn in the king shield, but almost surely the net result would be an elo loss. We could try to give Komodo very specific info, such as "if black has pawns on e5 and d6 and a light-squared bishop and the kings are on the kingside .... then the ...e5...f5 duo gets a bonus", but I think that a lot of such very specific rules would end up hurting elo overall due to slowdown. That's where the NNs have the advantage; the GPU lets them get all this info almost for free. Once good GPUs become cheap enough, we'll have to find a way to use them to remain competitive.

[Werewolf]

That's where the NNs have the advantage; the GPU lets them get all this info almost for free. Once good GPUs become cheap enough, we'll have to find a way to use them to remain competitive.

I read around the Hydra project and they also claimed they could get positional knowledge for free with FPGAs

[leavenfish]

and now KomodoMCTS 2301.00 gets wiped off the board by AllieStein V0.3dev-n4....
Like it's loss to Xiphos 0.5.2, it was not even close.

Both games were with Black. I took a few minutes and ran thru each...other similarities:

1. Komodo did not appreciate it was getting itself into positions where there would be no activity (Look at its light squared bishop in each case - against AllieStein it sits on g6 and shoots thru thin air...controlling what turns out to be a useless diagonal.)
2. It does not try for the standard (I've played the Sveshnikov type systems for years seen about move 14)...f5 to get more control of the center squares...perhaps the seeds of defeat ironically lie in 10...Bc8 instead of 10...Bf5, when it could tuck the bishop into g6, move the Nf6 and play...f5 to get some control of the central squares. The Bishop might then find employment on f7, looking at those White center pawns. I don't know, just spitballing here. Perhaps though the book used contained ...Bc8. Not necessarily 'bad', but that bishop flounders as the game unfolds.

I've also played the Benoni since I was a kid (not any more...too problematic against better players when I do not play OTB myself as much as I once did) and its loss to Xiphos was truly sad. Why on earth play 8...Nbd7 instead of simply 0-0? It adds nothing to the game and does not aid in trying to find the activity needed when playing the Benoni. Perhaps though this is simply the book move it was forced to play and the line chosen was simply to see how engines handled hopelessly cramped positions?

Is perhaps contempt set high for this phase of the event?
Are the programmers testing if Komodo can out-think' competition at this level instead of out calculate them?
Or perhaps it simply does not handle (maybe none of the engines do?) cramped, passive positions?

Personally, I do not care at all that it would lose games to these engines - Komodo has always been an 'analysis tool' for me (and others) and any decent player knows not to trust an engine at certain points - like when the position might get cramped.I currently use Komodo 11.3.

To satisfy my curiosity, I would like to know what the parameters are for this version of Komodo and why. Also, one can tweak and run millions of games at fast time controls and let statistics push where they are going - one can argue that one really doesn't need a GM level player involved in the programming for that...but how much do the programmers pay attention to these losses at slower time controls?

As a many time purchaser of Komodo (NEVER bought Houdini or any other commercial engine in the past decade), I am curious as to what the authors of Komodo (and other keen-eyed individuals) think of these two defeats...and what the parameters currently in use for TCEC are.

In general it is not productive for us to spend time trying to "fix" a loss, unless it is due to a bug. It is usually easy enough to make a change to the program to correct some inferior move played in a game, but most of the time the fixed version will play worse overall. So for example if the problem in the alliestein game was failure to play ...f5, tdhe reason was presumably that Komodo judged the harm to king safety to exceed the benefit. The fix would be to reduce the value of the f7 pawn in the king shield, but almost surely the net result would be an elo loss. We could try to give Komodo very specific info, such as "if black has pawns on e5 and d6 and a light-squared bishop and the kings are on the kingside .... then the ...e5...f5 duo gets a bonus", but I think that a lot of such very specific rules would end up hurting elo overall due to slowdown. That's where the NNs have the advantage; the GPU lets them get all this info almost for free. Once good GPUs become cheap enough, we'll have to find a way to use them to remain competitive.

Well, again, I am talking about the similarity to the two defeats...lack of space (again, check the first loss in this round to see how often a piece/pawn even crossed into 'enemy territory' (!)). It does not handle lack of space well in the two games. If it is judging ...f5 as 'poor king safety' THIS EARLY in a game, that is arguable too aggressive a function. The position is not exactly 'open'. So, I do not consider this the 'fixing of a loss' rabbit hole you seem to allude to. It is just not getting out of the 'opening' into a playable middle game.

Maybe if your presumption is correct (and given that opening books are used), perhaps you could set a 'king safety scale' of (lets say for sake of argument) 3 on a scale of 1-10 within the first 20 or 25 moves, defaulting to say 7 after that (?) - less aggressive early, more aggressive when things begin to 'open up' as a matter of course as they will in a game. That might keep Komodo from crawling into move starved positions where perhaps positions are beyond saving with the strength of the opposition these days (?).

Again, I'm not sure what specific parameters are being used...that might help make more sense of these losses.
To what do you attribute the losses?

[lkaufman]

and now KomodoMCTS 2301.00 gets wiped off the board by AllieStein V0.3dev-n4....
Like it's loss to Xiphos 0.5.2, it was not even close.

Both games were with Black. I took a few minutes and ran thru each...other similarities:

1. Komodo did not appreciate it was getting itself into positions where there would be no activity (Look at its light squared bishop in each case - against AllieStein it sits on g6 and shoots thru thin air...controlling what turns out to be a useless diagonal.)
2. It does not try for the standard (I've played the Sveshnikov type systems for years seen about move 14)...f5 to get more control of the center squares...perhaps the seeds of defeat ironically lie in 10...Bc8 instead of 10...Bf5, when it could tuck the bishop into g6, move the Nf6 and play...f5 to get some control of the central squares. The Bishop might then find employment on f7, looking at those White center pawns. I don't know, just spitballing here. Perhaps though the book used contained ...Bc8. Not necessarily 'bad', but that bishop flounders as the game unfolds.

I've also played the Benoni since I was a kid (not any more...too problematic against better players when I do not play OTB myself as much as I once did) and its loss to Xiphos was truly sad. Why on earth play 8...Nbd7 instead of simply 0-0? It adds nothing to the game and does not aid in trying to find the activity needed when playing the Benoni. Perhaps though this is simply the book move it was forced to play and the line chosen was simply to see how engines handled hopelessly cramped positions?

Is perhaps contempt set high for this phase of the event?
Are the programmers testing if Komodo can out-think' competition at this level instead of out calculate them?
Or perhaps it simply does not handle (maybe none of the engines do?) cramped, passive positions?

Personally, I do not care at all that it would lose games to these engines - Komodo has always been an 'analysis tool' for me (and others) and any decent player knows not to trust an engine at certain points - like when the position might get cramped.I currently use Komodo 11.3.

To satisfy my curiosity, I would like to know what the parameters are for this version of Komodo and why. Also, one can tweak and run millions of games at fast time controls and let statistics push where they are going - one can argue that one really doesn't need a GM level player involved in the programming for that...but how much do the programmers pay attention to these losses at slower time controls?

As a many time purchaser of Komodo (NEVER bought Houdini or any other commercial engine in the past decade), I am curious as to what the authors of Komodo (and other keen-eyed individuals) think of these two defeats...and what the parameters currently in use for TCEC are.

In general it is not productive for us to spend time trying to "fix" a loss, unless it is due to a bug. It is usually easy enough to make a change to the program to correct some inferior move played in a game, but most of the time the fixed version will play worse overall. So for example if the problem in the alliestein game was failure to play ...f5, tdhe reason was presumably that Komodo judged the harm to king safety to exceed the benefit. The fix would be to reduce the value of the f7 pawn in the king shield, but almost surely the net result would be an elo loss. We could try to give Komodo very specific info, such as "if black has pawns on e5 and d6 and a light-squared bishop and the kings are on the kingside .... then the ...e5...f5 duo gets a bonus", but I think that a lot of such very specific rules would end up hurting elo overall due to slowdown. That's where the NNs have the advantage; the GPU lets them get all this info almost for free. Once good GPUs become cheap enough, we'll have to find a way to use them to remain competitive.

Well, again, I am talking about the similarity to the two defeats...lack of space (again, check the first loss in this round to see how often a piece/pawn even crossed into 'enemy territory' (!)). It does not handle lack of space well in the two games. If it is judging ...f5 as 'poor king safety' THIS EARLY in a game, that is arguable too aggressive a function. The position is not exactly 'open'. So, I do not consider this the 'fixing of a loss' rabbit hole you seem to allude to. It is just not getting out of the 'opening' into a playable middle game.

Maybe if your presumption is correct (and given that opening books are used), perhaps you could set a 'king safety scale' of (lets say for sake of argument) 3 on a scale of 1-10 within the first 20 or 25 moves, defaulting to say 7 after that (?) - less aggressive early, more aggressive when things begin to 'open up' as a matter of course as they will in a game. That might keep Komodo from crawling into move starved positions where perhaps positions are beyond saving with the strength of the opposition these days (?).

Again, I'm not sure what specific parameters are being used...that might help make more sense of these losses.
To what do you attribute the losses?

I suppose that the problem is that our own testing book probably has very few positions like these where Black must find an active plan or lose gradually due to space deficit. I don't like rules based on move number; positions can open up very early or very late. Probably we can adjust our king shield rules to increase the likelihood of playing ...f5 when appropriate, but unless we change our test book, it will most likely not test well. TCEC starts with a lot of positions most engines won't like (to minimize draws, a worthy goal), so the results will be different than a rating list that uses "good" openings. It's hard to teach an engine (or a human) when it is necessary to play actively in a poor position and when it is best to just defend. I definitely don't believe that a poor plan is better than no plan, as some claim. But maybe something can be done to encourage ...f5.

[leavenfish]

and now KomodoMCTS 2301.00 gets wiped off the board by AllieStein V0.3dev-n4....
Like it's loss to Xiphos 0.5.2, it was not even close.

Both games were with Black. I took a few minutes and ran thru each...other similarities:

1. Komodo did not appreciate it was getting itself into positions where there would be no activity (Look at its light squared bishop in each case - against AllieStein it sits on g6 and shoots thru thin air...controlling what turns out to be a useless diagonal.)
2. It does not try for the standard (I've played the Sveshnikov type systems for years seen about move 14)...f5 to get more control of the center squares...perhaps the seeds of defeat ironically lie in 10...Bc8 instead of 10...Bf5, when it could tuck the bishop into g6, move the Nf6 and play...f5 to get some control of the central squares. The Bishop might then find employment on f7, looking at those White center pawns. I don't know, just spitballing here. Perhaps though the book used contained ...Bc8. Not necessarily 'bad', but that bishop flounders as the game unfolds.

I've also played the Benoni since I was a kid (not any more...too problematic against better players when I do not play OTB myself as much as I once did) and its loss to Xiphos was truly sad. Why on earth play 8...Nbd7 instead of simply 0-0? It adds nothing to the game and does not aid in trying to find the activity needed when playing the Benoni. Perhaps though this is simply the book move it was forced to play and the line chosen was simply to see how engines handled hopelessly cramped positions?

Is perhaps contempt set high for this phase of the event?
Are the programmers testing if Komodo can out-think' competition at this level instead of out calculate them?
Or perhaps it simply does not handle (maybe none of the engines do?) cramped, passive positions?

Personally, I do not care at all that it would lose games to these engines - Komodo has always been an 'analysis tool' for me (and others) and any decent player knows not to trust an engine at certain points - like when the position might get cramped.I currently use Komodo 11.3.

To satisfy my curiosity, I would like to know what the parameters are for this version of Komodo and why. Also, one can tweak and run millions of games at fast time controls and let statistics push where they are going - one can argue that one really doesn't need a GM level player involved in the programming for that...but how much do the programmers pay attention to these losses at slower time controls?

As a many time purchaser of Komodo (NEVER bought Houdini or any other commercial engine in the past decade), I am curious as to what the authors of Komodo (and other keen-eyed individuals) think of these two defeats...and what the parameters currently in use for TCEC are.

In general it is not productive for us to spend time trying to "fix" a loss, unless it is due to a bug. It is usually easy enough to make a change to the program to correct some inferior move played in a game, but most of the time the fixed version will play worse overall. So for example if the problem in the alliestein game was failure to play ...f5, tdhe reason was presumably that Komodo judged the harm to king safety to exceed the benefit. The fix would be to reduce the value of the f7 pawn in the king shield, but almost surely the net result would be an elo loss. We could try to give Komodo very specific info, such as "if black has pawns on e5 and d6 and a light-squared bishop and the kings are on the kingside .... then the ...e5...f5 duo gets a bonus", but I think that a lot of such very specific rules would end up hurting elo overall due to slowdown. That's where the NNs have the advantage; the GPU lets them get all this info almost for free. Once good GPUs become cheap enough, we'll have to find a way to use them to remain competitive.

Well, again, I am talking about the similarity to the two defeats...lack of space (again, check the first loss in this round to see how often a piece/pawn even crossed into 'enemy territory' (!)). It does not handle lack of space well in the two games. If it is judging ...f5 as 'poor king safety' THIS EARLY in a game, that is arguable too aggressive a function. The position is not exactly 'open'. So, I do not consider this the 'fixing of a loss' rabbit hole you seem to allude to. It is just not getting out of the 'opening' into a playable middle game.

Maybe if your presumption is correct (and given that opening books are used), perhaps you could set a 'king safety scale' of (lets say for sake of argument) 3 on a scale of 1-10 within the first 20 or 25 moves, defaulting to say 7 after that (?) - less aggressive early, more aggressive when things begin to 'open up' as a matter of course as they will in a game. That might keep Komodo from crawling into move starved positions where perhaps positions are beyond saving with the strength of the opposition these days (?).

Again, I'm not sure what specific parameters are being used...that might help make more sense of these losses.
To what do you attribute the losses?

I suppose that the problem is that our own testing book probably has very few positions like these where Black must find an active plan or lose gradually due to space deficit. I don't like rules based on move number; positions can open up very early or very late. Probably we can adjust our king shield rules to increase the likelihood of playing ...f5 when appropriate, but unless we change our test book, it will most likely not test well. TCEC starts with a lot of positions most engines won't like (to minimize draws, a worthy goal), so the results will be different than a rating list that uses "good" openings. It's hard to teach an engine (or a human) when it is necessary to play actively in a poor position and when it is best to just defend. I definitely don't believe that a poor plan is better than no plan, as some claim. But maybe something can be done to encourage ...f5.

First, thanks for the response!
Second...totally agree with your comment about 'poor plan' vs 'no plan'. To go down a wrong road when you can keep your options open (ex...Nbd7 in the Benoni) just gets you into positions that are not very playable.
Third, with positions reversed, Xiphos played the more natural 9...Re8...but lost just about as badly as Komodo. You are of course right that there is a reason these more dubious openings are included. not 'crawling into a shell and saying "beat me"' desirable. My guess is that Stockfish would have opted for the more open 9...b5, which as I recall can lead to something close to dynamic equality for Black.
Fourth, 26. Nc5 in the loss to AllieStein....pretty move by White, but it looks like any engine can spot it, but the seeds of defeat in such positions are often sewn long before such shots are possible.

And finally...as I mentioned, I am only interested in Komodo for analysis purposes, I could (partially) care less that it can lose like a child to any given engine on any given day - they all could. I would never want you to lose sight of what makes Komodo appealing to those who buy it. MCTS is interesting and may yet breath new life into Komodo. I will certainly purchase again when it seems to do so. A 'playing version' and one optimized for 'analysis' would sure look like the way to go.