Cute Chess 0.9.4 released

[Laskos]

Well, 10+0.01 is not really hyper fast. I was already doing 12-sec games with Joker80 (because of time odds) in 2008, without any problems, on a much slower computer than we can use today. But your conclusion in any case is that Cutechess GUI is appreciably 'heavier' (about a factor 3-4) as Cutechess-cli.

What exactly do the CPU percentages mean? Is that percentage from a single thread? If that remains below 100%, how could it affect the games if you reserved one thread for the GUI?

1+0.01, not 10+0.01. On 7 separate threads from 8 logical cores available. CPU usage is from total 4-core CPU (8 logical cores). So, more than 1 thread for 20%, almost 2. The Cutechess load increases considerably at such fast games, be them fixed time, depth or nodes. Yes, GUI seems about 3 times heavier. The load becomes more acceptable at some normal games like 10+0.1, but I wanted to stress test the GUI.

[hgm]

Paul was using 10+0.01, and allmy previous remarks referred to that case.

I also note that you are using logical cores, which means that the different tasks affect each other anyway. You cannot expect reliable testing under such conditions even when you keep two cores in reserve for the GUI. Because when the GUI sleeps, the OS will distribute the hyper threads over those cores to keep them apart. So some engines will have a private physical core, while others have to share one.

If you want to run like this you should set process affinities. This is what I always do when I do concurrent testing on WinBoard: make sure that each game can only use a different set of two virtual cores on the same physical one.

[mcostalba]

The lesson to learn: don't compare results in GUI and CLI below 3+0.03. In fact I will probably not switch right now to GUI, and use CLI. I cannot even see in GUI whether there are losses on time. It would be nice for it to count number or regularly finished games, time losses, illegal move losses, adjudications, mates and draw reasons.

I think you are just measuring the scaling of the engines at very fast TC.

With the 16-20% CPU usage by the GUI it means that there is less available CPU time for the engines (both of them), so it is like to test with Cutechess-CLI at say 0.7s+0.01s, eventually you would get similar results.

With higher TC the different scaling affects ELO much less, but IMO is not a problem of overhead but of available CPU time, compared to wall time.

[mcostalba]

Perhaps once the book is out of book (first time), do not allow the gui to use the book again, so that the use of book moves is guaranteed to be equal.

That is a book problem. Trying to solve it in the GUI would be a mistake. If you want to use a neutral 6-move book for starting tourney games, all lines in the book should be 6 moves. Otherwise white can do the last book move, tso that black has one less.

Yes I agree here, when SF had book management it was checking for a book move at every move, even at endgame time

This is the most consistent and natural policy and the same policy is used by Cutechess, and it is ok to do so IMO.

[Laskos]

The lesson to learn: don't compare results in GUI and CLI below 3+0.03. In fact I will probably not switch right now to GUI, and use CLI. I cannot even see in GUI whether there are losses on time. It would be nice for it to count number or regularly finished games, time losses, illegal move losses, adjudications, mates and draw reasons.

I think you are just measuring the scaling of the engines at very fast TC.

With the 16-20% CPU usage by the GUI it means that there is less available CPU time for the engines (both of them), so it is like to test with Cutechess-CLI at say 0.7s+0.01s, eventually you would get similar results.

With higher TC the different scaling affects ELO much less, but IMO is not a problem of overhead but of available CPU time, compared to wall time.

Yes, I modified overhead to 15ms for both engines to have some more meaningful games. But it's the scaling indeed to blame for discrepancy between GUI and CLI, and I wanted to show that in hyper-fast games the scaling might become important. At slower games the scaling issue is much much tamer.

[hgm]

I created the polyglot book using Cute Chess from games with TC 3m+2s inc, around 1220 games only.

If you create a book from games that are all longer than 6 moves, it should not be possible to get out of book before move 7 if both players use that book, right?

[Ferdy]

I created the polyglot book using Cute Chess from games with TC 3m+2s inc, around 1220 games only.

If you create a book from games that are all longer than 6 moves, it should not be possible to get out of book before move 7 if both players use that book, right?

I am not sure, perhaps it depends on what moves are imported into the book (W and L stats), also depends on how the gui selects a book move for the engine.

[ilari]

Perhaps once the book is out of book (first time), do not allow the gui to use the book again, so that the use of book moves is guaranteed to be equal.

That is a book problem. Trying to solve it in the GUI would be a mistake. If you want to use a neutral 6-move book for starting tourney games, all lines in the book should be 6 moves. Otherwise white can do the last book move, tso that black has one less.

Yes I agree here, when SF had book management it was checking for a book move at every move, even at endgame time

This is the most consistent and natural policy and the same policy is used by Cutechess, and it is ok to do so IMO.

Yes, it wouldn't make any sense to have position hashes instead of move trees in Polyglot if transpositions between lines and jumping back into book wasn't allowed.

Cutechess-cli's interface points out the difference between an opening suite and a Polyglot book better than Cute Chess GUI: opening suites have starting positions or a sequence of moves that set the starting position for the game to provide variety, and Polyglot books are tools that help an engine to play better chess in the opening phase of the game.

[ilari]

I created the polyglot book using Cute Chess from games with TC 3m+2s inc, around 1220 games only.

If you create a book from games that are all longer than 6 moves, it should not be possible to get out of book before move 7 if both players use that book, right?

I am not sure, perhaps it depends on what moves are imported into the book (W and L stats), also depends on how the gui selects a book move for the engine.

Cute Chess ignores the losing player's moves when creating an opening book, so there will be "holes". I think my previous post serves as an explanation for why this is done.

[ilari]

The lesson to learn: don't compare results in GUI and CLI below 3+0.03. In fact I will probably not switch right now to GUI, and use CLI. I cannot even see in GUI whether there are losses on time. It would be nice for it to count number or regularly finished games, time losses, illegal move losses, adjudications, mates and draw reasons.

I think you are just measuring the scaling of the engines at very fast TC.

With the 16-20% CPU usage by the GUI it means that there is less available CPU time for the engines (both of them), so it is like to test with Cutechess-CLI at say 0.7s+0.01s, eventually you would get similar results.

With higher TC the different scaling affects ELO much less, but IMO is not a problem of overhead but of available CPU time, compared to wall time.

Yes, I modified overhead to 15ms for both engines to have some more meaningful games. But it's the scaling indeed to blame for discrepancy between GUI and CLI, and I wanted to show that in hyper-fast games the scaling might become important. At slower games the scaling issue is much much tamer.

If you don't actually need to watch the games live, there's a way to drop Cute Chess GUI's CPU usage to the cutechess-cli level: make sure there's a non-tournament game open where nothing is happening (eg. the default Human-Human game that Cute Chess opens initially) and switch to that game's tab. I just tested this on an average i5 laptop with 2 concurrent games at tc 3: with a tournament tab activated Cute Chess used about 20% CPU and with the Human-Human game active, CPU usage dropped to 3%.