Page 1 of 2
Faster testing ?
Posted: Thu Aug 15, 2019 1:07 pm
by xr_a_y
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).
It would be quite cheap testing and a database can be built pretty quickly.
Re: Faster testing ?
Posted: Thu Aug 15, 2019 1:50 pm
by Guenther
xr_a_y wrote: ↑Thu Aug 15, 2019 1:07 pm
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).
It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html
Minic is missing though, but not for long I guess ;-)
Re: Faster testing ?
Posted: Thu Aug 15, 2019 2:34 pm
by xr_a_y
Guenther wrote: ↑Thu Aug 15, 2019 1:50 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 1:07 pm
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).
It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html
Minic is missing though, but not for long I guess
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly
Re: Faster testing ?
Posted: Thu Aug 15, 2019 6:18 pm
by lkaufman
xr_a_y wrote: ↑Thu Aug 15, 2019 2:34 pm
Guenther wrote: ↑Thu Aug 15, 2019 1:50 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 1:07 pm
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).
It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html
Minic is missing though, but not for long I guess
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly
N moves in M minutes is an obsolete and extremely wasteful way to test. It is no longer used in human chess as far as I know (for one or two time controls, yes, but always with a sudden death time control at the end), and for engine testing it is a terrible waste of resources to play out hopelessly drawn endgames for hundreds of moves at the same pace as the part of the game where the result is being decided. I know that that CCRL and CEGT still use it to maintain historical connection to the old machines which were tested before increment became normal, but they could switch to increment tc averaging the same time for 60 moves and combine the data without much risk of distortion, reverting to 40/x for old engines that did not support increment.
Re: Faster testing ?
Posted: Thu Aug 15, 2019 7:22 pm
by xr_a_y
lkaufman wrote: ↑Thu Aug 15, 2019 6:18 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 2:34 pm
Guenther wrote: ↑Thu Aug 15, 2019 1:50 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 1:07 pm
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).
It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html
Minic is missing though, but not for long I guess
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly
N moves in M minutes is an obsolete and extremely wasteful way to test. It is no longer used in human chess as far as I know (for one or two time controls, yes, but always with a sudden death time control at the end), and for engine testing it is a terrible waste of resources to play out hopelessly drawn endgames for hundreds of moves at the same pace as the part of the game where the result is being decided. I know that that CCRL and CEGT still use it to maintain historical connection to the old machines which were tested before increment became normal, but they could switch to increment tc averaging the same time for 60 moves and combine the data without much risk of distortion, reverting to 40/x for old engines that did not support increment.
Draws can be decided by end game table or by a specific rules, no need to let the engines plays hundred of moves. Mean Ccrl game length is quite correct i think. Is that the only argument against this kind of TC?
Re: Faster testing ?
Posted: Thu Aug 15, 2019 7:40 pm
by lkaufman
xr_a_y wrote: ↑Thu Aug 15, 2019 7:22 pm
lkaufman wrote: ↑Thu Aug 15, 2019 6:18 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 2:34 pm
Guenther wrote: ↑Thu Aug 15, 2019 1:50 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 1:07 pm
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).
It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html
Minic is missing though, but not for long I guess
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly
N moves in M minutes is an obsolete and extremely wasteful way to test. It is no longer used in human chess as far as I know (for one or two time controls, yes, but always with a sudden death time control at the end), and for engine testing it is a terrible waste of resources to play out hopelessly drawn endgames for hundreds of moves at the same pace as the part of the game where the result is being decided. I know that that CCRL and CEGT still use it to maintain historical connection to the old machines which were tested before increment became normal, but they could switch to increment tc averaging the same time for 60 moves and combine the data without much risk of distortion, reverting to 40/x for old engines that did not support increment.
Draws can be decided by end game table or by a specific rules, no need to let the engines plays hundred of moves. Mean Ccrl game length is quite correct i think. Is that the only argument against this kind of TC?
Tablebases only help when the number of men is 7 or less; there are plenty of clearly drawn endgames with more than 7 men, and often one side is "better" so no adjudication rule makes much sense. For example bishops of opposite color with four pawns vs. three but obviously no way to progress. The more general argument is that the chance of a move deciding the game is much smaller at move 100 than at move 30 so allocating the same amount of time per move just doesn't make sense. Increment handles this in a pretty good way. Full disclosure: I'm not neutral on this, as I am on record as the inventor of the use of increment for chess (despite Bobby Fischer's claims to same).
Re: Faster testing ?
Posted: Thu Aug 15, 2019 7:59 pm
by xr_a_y
Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...
I guess I shall work on sudden death + incr in Minic anyway...
Re: Faster testing ?
Posted: Thu Aug 15, 2019 10:58 pm
by lkaufman
xr_a_y wrote: ↑Thu Aug 15, 2019 7:59 pm
Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...
I guess I shall work on sudden death + incr in Minic anyway...
As long as the base to increment ratio is reasonable, there should be no more risk of making bad moves in the endgame than in the middlegame, since on average you get a lot more depth in the endgame in the same time, and also because the chance of the result still being in doubt is lower. Rating lists seem to have settled on 100 to 1 for the ratio, while tournaments like CCC and TCEC usually use larger ratios, close to the 180 or 240 ratios typical of human GM events. We mostly use 150 to 1 ratio as a compromise between these ratios in our own testing.
Re: Faster testing ?
Posted: Fri Aug 16, 2019 6:17 am
by mwyoung
lkaufman wrote: ↑Thu Aug 15, 2019 10:58 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 7:59 pm
Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...
I guess I shall work on sudden death + incr in Minic anyway...
As long as the base to increment ratio is reasonable, there should be no more risk of making bad moves in the endgame than in the middlegame, since on average you get a lot more depth in the endgame in the same time, and also because the chance of the result still being in doubt is lower. Rating lists seem to have settled on 100 to 1 for the ratio, while tournaments like CCC and TCEC usually use larger ratios, close to the 180 or 240 ratios typical of human GM events. We mostly use 150 to 1 ratio as a compromise between these ratios in our own testing.
I see more human tournaments using delay. When playing at fast time controls. Is this something that will be implemented in computer chess?
Re: Faster testing ?
Posted: Sat Aug 17, 2019 12:02 am
by Raphexon
mwyoung wrote: ↑Fri Aug 16, 2019 6:17 am
lkaufman wrote: ↑Thu Aug 15, 2019 10:58 pm
xr_a_y wrote: ↑Thu Aug 15, 2019 7:59 pm
Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...
I guess I shall work on sudden death + incr in Minic anyway...
As long as the base to increment ratio is reasonable, there should be no more risk of making bad moves in the endgame than in the middlegame, since on average you get a lot more depth in the endgame in the same time, and also because the chance of the result still being in doubt is lower. Rating lists seem to have settled on 100 to 1 for the ratio, while tournaments like CCC and TCEC usually use larger ratios, close to the 180 or 240 ratios typical of human GM events. We mostly use 150 to 1 ratio as a compromise between these ratios in our own testing.
I see more human tournaments using delay. When playing at fast time controls. Is this something that will be implemented in computer chess?
No need to implement delay for computers since they don't have a reaction time like humans, and a computer moves pieces at instant speed.