Faster testing ?

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
xr_a_y
Posts: 704
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Faster testing ?

Post by xr_a_y » Thu Aug 15, 2019 11:07 am

Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).

It would be quite cheap testing and a database can be built pretty quickly.

User avatar
Guenther
Posts: 2956
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Faster testing ?

Post by Guenther » Thu Aug 15, 2019 11:50 am

xr_a_y wrote:
Thu Aug 15, 2019 11:07 am
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).

It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html

Minic is missing though, but not for long I guess ;-)
Current foe list count : [91]
http://rwbc-chess.de/chronology.htm

User avatar
xr_a_y
Posts: 704
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Faster testing ?

Post by xr_a_y » Thu Aug 15, 2019 12:34 pm

Guenther wrote:
Thu Aug 15, 2019 11:50 am
xr_a_y wrote:
Thu Aug 15, 2019 11:07 am
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).

It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html

Minic is missing though, but not for long I guess ;-)
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly :oops:

lkaufman
Posts: 3646
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Faster testing ?

Post by lkaufman » Thu Aug 15, 2019 4:18 pm

xr_a_y wrote:
Thu Aug 15, 2019 12:34 pm
Guenther wrote:
Thu Aug 15, 2019 11:50 am
xr_a_y wrote:
Thu Aug 15, 2019 11:07 am
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).

It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html

Minic is missing though, but not for long I guess ;-)
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly :oops:
N moves in M minutes is an obsolete and extremely wasteful way to test. It is no longer used in human chess as far as I know (for one or two time controls, yes, but always with a sudden death time control at the end), and for engine testing it is a terrible waste of resources to play out hopelessly drawn endgames for hundreds of moves at the same pace as the part of the game where the result is being decided. I know that that CCRL and CEGT still use it to maintain historical connection to the old machines which were tested before increment became normal, but they could switch to increment tc averaging the same time for 60 moves and combine the data without much risk of distortion, reverting to 40/x for old engines that did not support increment.
Komodo rules!

User avatar
xr_a_y
Posts: 704
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Faster testing ?

Post by xr_a_y » Thu Aug 15, 2019 5:22 pm

lkaufman wrote:
Thu Aug 15, 2019 4:18 pm
xr_a_y wrote:
Thu Aug 15, 2019 12:34 pm
Guenther wrote:
Thu Aug 15, 2019 11:50 am
xr_a_y wrote:
Thu Aug 15, 2019 11:07 am
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).

It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html

Minic is missing though, but not for long I guess ;-)
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly :oops:
N moves in M minutes is an obsolete and extremely wasteful way to test. It is no longer used in human chess as far as I know (for one or two time controls, yes, but always with a sudden death time control at the end), and for engine testing it is a terrible waste of resources to play out hopelessly drawn endgames for hundreds of moves at the same pace as the part of the game where the result is being decided. I know that that CCRL and CEGT still use it to maintain historical connection to the old machines which were tested before increment became normal, but they could switch to increment tc averaging the same time for 60 moves and combine the data without much risk of distortion, reverting to 40/x for old engines that did not support increment.
Draws can be decided by end game table or by a specific rules, no need to let the engines plays hundred of moves. Mean Ccrl game length is quite correct i think. Is that the only argument against this kind of TC?

lkaufman
Posts: 3646
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Faster testing ?

Post by lkaufman » Thu Aug 15, 2019 5:40 pm

xr_a_y wrote:
Thu Aug 15, 2019 5:22 pm
lkaufman wrote:
Thu Aug 15, 2019 4:18 pm
xr_a_y wrote:
Thu Aug 15, 2019 12:34 pm
Guenther wrote:
Thu Aug 15, 2019 11:50 am
xr_a_y wrote:
Thu Aug 15, 2019 11:07 am
Is there some interest in faster testing (ultra-bullet style)? something that would be like a CCRL 40/1 scale (meaning 4 times faster than CCRL 40/4, so around 20sec or 15sec on recent hardware).

It would be quite cheap testing and a database can be built pretty quickly.
This is already pretty good:
http://www.fastgm.de/60-0.60.html

Minic is missing though, but not for long I guess ;-)
Oh thanks, I did forgot this one. One thing anyway is that I prefer "N moves in M minutes" rather than sudden-death even with increment but this is mainly because I don't really know how to handle it properly :oops:
N moves in M minutes is an obsolete and extremely wasteful way to test. It is no longer used in human chess as far as I know (for one or two time controls, yes, but always with a sudden death time control at the end), and for engine testing it is a terrible waste of resources to play out hopelessly drawn endgames for hundreds of moves at the same pace as the part of the game where the result is being decided. I know that that CCRL and CEGT still use it to maintain historical connection to the old machines which were tested before increment became normal, but they could switch to increment tc averaging the same time for 60 moves and combine the data without much risk of distortion, reverting to 40/x for old engines that did not support increment.
Draws can be decided by end game table or by a specific rules, no need to let the engines plays hundred of moves. Mean Ccrl game length is quite correct i think. Is that the only argument against this kind of TC?
Tablebases only help when the number of men is 7 or less; there are plenty of clearly drawn endgames with more than 7 men, and often one side is "better" so no adjudication rule makes much sense. For example bishops of opposite color with four pawns vs. three but obviously no way to progress. The more general argument is that the chance of a move deciding the game is much smaller at move 100 than at move 30 so allocating the same amount of time per move just doesn't make sense. Increment handles this in a pretty good way. Full disclosure: I'm not neutral on this, as I am on record as the inventor of the use of increment for chess (despite Bobby Fischer's claims to same).
Komodo rules!

User avatar
xr_a_y
Posts: 704
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Faster testing ?

Post by xr_a_y » Thu Aug 15, 2019 5:59 pm

Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...

I guess I shall work on sudden death + incr in Minic anyway...

lkaufman
Posts: 3646
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Faster testing ?

Post by lkaufman » Thu Aug 15, 2019 8:58 pm

xr_a_y wrote:
Thu Aug 15, 2019 5:59 pm
Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...

I guess I shall work on sudden death + incr in Minic anyway...
As long as the base to increment ratio is reasonable, there should be no more risk of making bad moves in the endgame than in the middlegame, since on average you get a lot more depth in the endgame in the same time, and also because the chance of the result still being in doubt is lower. Rating lists seem to have settled on 100 to 1 for the ratio, while tournaments like CCC and TCEC usually use larger ratios, close to the 180 or 240 ratios typical of human GM events. We mostly use 150 to 1 ratio as a compromise between these ratios in our own testing.
Komodo rules!

mwyoung
Posts: 1601
Joined: Wed May 12, 2010 8:00 pm

Re: Faster testing ?

Post by mwyoung » Fri Aug 16, 2019 4:17 am

lkaufman wrote:
Thu Aug 15, 2019 8:58 pm
xr_a_y wrote:
Thu Aug 15, 2019 5:59 pm
Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...

I guess I shall work on sudden death + incr in Minic anyway...
As long as the base to increment ratio is reasonable, there should be no more risk of making bad moves in the endgame than in the middlegame, since on average you get a lot more depth in the endgame in the same time, and also because the chance of the result still being in doubt is lower. Rating lists seem to have settled on 100 to 1 for the ratio, while tournaments like CCC and TCEC usually use larger ratios, close to the 180 or 240 ratios typical of human GM events. We mostly use 150 to 1 ratio as a compromise between these ratios in our own testing.
I see more human tournaments using delay. When playing at fast time controls. Is this something that will be implemented in computer chess?
Professing themselves to be wise, they became fools,
Take on me. foes 0

Raphexon
Posts: 83
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: Faster testing ?

Post by Raphexon » Fri Aug 16, 2019 10:02 pm

mwyoung wrote:
Fri Aug 16, 2019 4:17 am
lkaufman wrote:
Thu Aug 15, 2019 8:58 pm
xr_a_y wrote:
Thu Aug 15, 2019 5:59 pm
Ok I get your point but i'd be curious to see some stats on the subject (match lenght, tpye of draws...).
Sudden-death matches can also suffer from blunder in end game or failure to find a mate sequence being too short on time and living on a small increment...

I guess I shall work on sudden death + incr in Minic anyway...
As long as the base to increment ratio is reasonable, there should be no more risk of making bad moves in the endgame than in the middlegame, since on average you get a lot more depth in the endgame in the same time, and also because the chance of the result still being in doubt is lower. Rating lists seem to have settled on 100 to 1 for the ratio, while tournaments like CCC and TCEC usually use larger ratios, close to the 180 or 240 ratios typical of human GM events. We mostly use 150 to 1 ratio as a compromise between these ratios in our own testing.
I see more human tournaments using delay. When playing at fast time controls. Is this something that will be implemented in computer chess?
No need to implement delay for computers since they don't have a reaction time like humans, and a computer moves pieces at instant speed.

Post Reply