Can somebody do simply this? By far the strongest engine!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Can somebody do simply this? By far the strongest engine!

Post by Laskos »

pohl4711 wrote: Sat May 16, 2020 1:20 pm
Laskos wrote: Sat May 16, 2020 1:08 pm
Thanks, that's it! Works in Cutechess-CLI, now performing a sanity check to see whether I set the options correctly and whether it doesn't have a large overhead.
Please report here about your experiences of Chimera in cutechess-cli. Perhaps, I can do a testrun for my NN vs SF performance-testing in 2 weeks or so, using Chimera with lc0 and Komodo vs. Stockfish and see, if there is an Elo-gain compared to lc0 alone...

https://www.sp-cc.de/nn-vs-sf-testing.htm
It doesn't work as described in Fritz GUI too. Showing weird NPS from move one which tend to converge to lower Lc0 NPS after some thinking, nothing happens at set to switch to SF move 40, same mainly Lc0 all the way (by GPU usage as well). Same in Cutechess-CLI. I don't know what's this mess, hope not a trojan, can somebody come with simple option to switch from one engine to another at a fixed number of moves (or fixed material on board)? For a skilled programmer it shouldn't be hard at all.
gordonr
Posts: 194
Joined: Thu Aug 06, 2009 8:04 pm
Location: UK

Re: Can somebody do simply this? By far the strongest engine!

Post by gordonr »

Dann Corbit wrote: Sat May 16, 2020 3:27 pm If one has a much better score for a move, then play that.
I'm not sure about that. I've seen LC0 make tactical blunders and Stockfish often overestimate positions. I think it's useful for them to somehow check each other's analysis.

For correspondence play, I wrote some Java code that gets a set of engines to propose their best move for a given position. If they disagree, they then slowly play out their preferred move against each other in "shoot out" fashion. The code monitors the evals to see if any engine has been persuaded by another engine. I find that more often than not, one engine will be "refuted" by another. However, there are cases where the playing out of lines goes on without agreement being reached and I have to make a decision otherwise.

Unfortunately this was developed for correspondence time controls and is often run overnight or even for a few days. I wasn't even concerned about my correspondence rating but rather more just a way to explore different engines at long time controls (my main interest). I get to see the types of positions where they refute each other. I realise that this method will need a minimal time control, though I've not experimented to find out, but it has highlighted to me the benefit of engines checking each other's analysis.
corres
Posts: 3657
Joined: Wed Nov 18, 2015 11:41 am
Location: hungary

Re: Can somebody do simply this? By far the strongest engine!

Post by corres »

Dann Corbit wrote: Sat May 16, 2020 3:27 pm Since LC0 uses almost exclusively GPU and very little CPU and Stockfish uses exclusively CPU, you can run them at the same time. The only losses are around two threads for SF and the memory for both programs is reduced.
If one has a much better score for a move, then play that.
In this mode you play a not coherent game and in a bad case you can not utilize the capacity of either engine.
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Can somebody do simply this? By far the strongest engine!

Post by Dann Corbit »

gordonr wrote: Sat May 16, 2020 4:40 pm
Dann Corbit wrote: Sat May 16, 2020 3:27 pm If one has a much better score for a move, then play that.
I'm not sure about that. I've seen LC0 make tactical blunders and Stockfish often overestimate positions. I think it's useful for them to somehow check each other's analysis.

For correspondence play, I wrote some Java code that gets a set of engines to propose their best move for a given position. If they disagree, they then slowly play out their preferred move against each other in "shoot out" fashion. The code monitors the evals to see if any engine has been persuaded by another engine. I find that more often than not, one engine will be "refuted" by another. However, there are cases where the playing out of lines goes on without agreement being reached and I have to make a decision otherwise.

Unfortunately this was developed for correspondence time controls and is often run overnight or even for a few days. I wasn't even concerned about my correspondence rating but rather more just a way to explore different engines at long time controls (my main interest). I get to see the types of positions where they refute each other. I realise that this method will need a minimal time control, though I've not experimented to find out, but it has highlighted to me the benefit of engines checking each other's analysis.
There are obviously lots of interesting things you can do with two engines giving opinions.
You could move immediately if they agree (which will be pretty often).
You could do a "mechanical fail high" and search another ply if one engine or the other has a different answer with a higher score, possibly forcing the other engine to ponder on the suggested move.

I think that the coming GPU model by AMD will render all of this moot when transparent memory access for CPU and GPU happens. They could even share the hash table.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Can somebody do simply this? By far the strongest engine!

Post by Rebel »

pohl4711 wrote: Sat May 16, 2020 11:52 am
Laskos wrote: Sat May 16, 2020 11:10 am I am unaware of a hybrid engine of the simplest kind, and I don't know what's the matter. Somebody better than me at coding can do this in an hour.
Simply: play using Lc0 first 35 moves and using SF after 35 moves to the end. On some RTX GPU and 4 to 32 core machine this combo will be by far the best engine.

Here are my results with RTX 2070 and 4 strong i7 cores:

200 games each match

Pure Lc0 net SV_3200 --- versus SF_11:
99/200

Combo Lc0 net SV_3200 to move 35, SF_11 after move 35 --- versus SF_11
123/200

An 85 Elo points improvement over pure Lc0.

As simple as that.
Lc0 is ridiculous in endgames, it always was and will probably improve only a little. In 200 games against Fritz 5.32, SF_11 scored 200/200, while Lc0 SV_3200 scored 193/200, Lc0 failing to convert in endgames 14 totally won positions even against Fritz 5.32 !!! It's ridiculous.

Are there volunteers to build such a combo engine in a matter of an hour?
This is, what you are looking for, isnt it?

Eman Chimera (previously known as Hybrid UCI) is a tool to run two or more UCI engines (one after another) to play the same game. Eman Chimera will switch to the appropriate engine based on move number of the current game being played

https://eman.zohosites.com/files/Eman%2 ... 0v1.10.zip

I know, that on playchess, some people use it, and SF takes over after 60 moves (35 moves seems a bit too early to me). So, it works in the FritzGUI. And what works in there, should run everywhere...
The switch should be done by the position on the board. About 6-8 months I wrote such a tool. SF10 and Lc0 both thinking, see screenshot below and then a simple referee program (I used ProDeo) makes the decision, in this board position, since it's the endgame, it takes the SF10 move, else the Lc0 move.

http://rebel13.nl/ZeroFishx.jpg
90% of coding is debugging, the other 10% is writing bugs.
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Can somebody do simply this? By far the strongest engine!

Post by Dann Corbit »

Rebel wrote: Sat May 16, 2020 10:31 pm
The switch should be done by the position on the board. About 6-8 months I wrote such a tool. SF10 and Lc0 both thinking, see screenshot below and then a simple referee program (I used ProDeo) makes the decision, in this board position, since it's the endgame, it takes the SF10 move, else the Lc0 move.

http://rebel13.nl/ZeroFishx.jpg
What would it do in the opening if LC0 saw a move with a 200 to 300 centipawn advantage to the move of Stockfish in the middle game? (You will see surprises like that on TCEC).

Stockfish can also (on rare occasions) find a tactical shot in the opening.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
carldaman
Posts: 2283
Joined: Sat Jun 02, 2012 2:13 am

Re: Can somebody do simply this? By far the strongest engine!

Post by carldaman »

Rebel wrote: Sat May 16, 2020 10:31 pm
pohl4711 wrote: Sat May 16, 2020 11:52 am
Laskos wrote: Sat May 16, 2020 11:10 am I am unaware of a hybrid engine of the simplest kind, and I don't know what's the matter. Somebody better than me at coding can do this in an hour.
Simply: play using Lc0 first 35 moves and using SF after 35 moves to the end. On some RTX GPU and 4 to 32 core machine this combo will be by far the best engine.

Here are my results with RTX 2070 and 4 strong i7 cores:

200 games each match

Pure Lc0 net SV_3200 --- versus SF_11:
99/200

Combo Lc0 net SV_3200 to move 35, SF_11 after move 35 --- versus SF_11
123/200

An 85 Elo points improvement over pure Lc0.

As simple as that.
Lc0 is ridiculous in endgames, it always was and will probably improve only a little. In 200 games against Fritz 5.32, SF_11 scored 200/200, while Lc0 SV_3200 scored 193/200, Lc0 failing to convert in endgames 14 totally won positions even against Fritz 5.32 !!! It's ridiculous.

Are there volunteers to build such a combo engine in a matter of an hour?
This is, what you are looking for, isnt it?

Eman Chimera (previously known as Hybrid UCI) is a tool to run two or more UCI engines (one after another) to play the same game. Eman Chimera will switch to the appropriate engine based on move number of the current game being played

https://eman.zohosites.com/files/Eman%2 ... 0v1.10.zip

I know, that on playchess, some people use it, and SF takes over after 60 moves (35 moves seems a bit too early to me). So, it works in the FritzGUI. And what works in there, should run everywhere...
The switch should be done by the position on the board. About 6-8 months I wrote such a tool. SF10 and Lc0 both thinking, see screenshot below and then a simple referee program (I used ProDeo) makes the decision, in this board position, since it's the endgame, it takes the SF10 move, else the Lc0 move.

http://rebel13.nl/ZeroFishx.jpg
Hi Ed,
Is this tool still up for download?
Thanks! :)
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Can somebody do simply this? By far the strongest engine!

Post by Rebel »

Dann Corbit wrote: Sat May 16, 2020 10:54 pm
Rebel wrote: Sat May 16, 2020 10:31 pm
The switch should be done by the position on the board. About 6-8 months I wrote such a tool. SF10 and Lc0 both thinking, see screenshot below and then a simple referee program (I used ProDeo) makes the decision, in this board position, since it's the endgame, it takes the SF10 move, else the Lc0 move.

http://rebel13.nl/ZeroFishx.jpg
What would it do in the opening if LC0 saw a move with a 200 to 300 centipawn advantage to the move of Stockfish in the middle game? (You will see surprises like that on TCEC).

Stockfish can also (on rare occasions) find a tactical shot in the opening.
I did not came to that kind of exceptions but what springs to mind is to keep scores of previous moves and if SF makes a jump and Lc0 not then pick SF.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Can somebody do simply this? By far the strongest engine!

Post by Rebel »

carldaman wrote: Sun May 17, 2020 12:03 am
Rebel wrote: Sat May 16, 2020 10:31 pm
pohl4711 wrote: Sat May 16, 2020 11:52 am
Laskos wrote: Sat May 16, 2020 11:10 am I am unaware of a hybrid engine of the simplest kind, and I don't know what's the matter. Somebody better than me at coding can do this in an hour.
Simply: play using Lc0 first 35 moves and using SF after 35 moves to the end. On some RTX GPU and 4 to 32 core machine this combo will be by far the best engine.

Here are my results with RTX 2070 and 4 strong i7 cores:

200 games each match

Pure Lc0 net SV_3200 --- versus SF_11:
99/200

Combo Lc0 net SV_3200 to move 35, SF_11 after move 35 --- versus SF_11
123/200

An 85 Elo points improvement over pure Lc0.

As simple as that.
Lc0 is ridiculous in endgames, it always was and will probably improve only a little. In 200 games against Fritz 5.32, SF_11 scored 200/200, while Lc0 SV_3200 scored 193/200, Lc0 failing to convert in endgames 14 totally won positions even against Fritz 5.32 !!! It's ridiculous.

Are there volunteers to build such a combo engine in a matter of an hour?
This is, what you are looking for, isnt it?

Eman Chimera (previously known as Hybrid UCI) is a tool to run two or more UCI engines (one after another) to play the same game. Eman Chimera will switch to the appropriate engine based on move number of the current game being played

https://eman.zohosites.com/files/Eman%2 ... 0v1.10.zip

I know, that on playchess, some people use it, and SF takes over after 60 moves (35 moves seems a bit too early to me). So, it works in the FritzGUI. And what works in there, should run everywhere...
The switch should be done by the position on the board. About 6-8 months I wrote such a tool. SF10 and Lc0 both thinking, see screenshot below and then a simple referee program (I used ProDeo) makes the decision, in this board position, since it's the endgame, it takes the SF10 move, else the Lc0 move.

http://rebel13.nl/ZeroFishx.jpg
Hi Ed,
Is this tool still up for download?
Thanks! :)
It never was available because I lost interest while it wasn't ready for release. Things like time control and draws by repetition etc. But it works and Kai's observation is quite to the point.
90% of coding is debugging, the other 10% is writing bugs.
duncan
Posts: 12038
Joined: Mon Jul 07, 2008 10:50 pm

Re: Can somebody do simply this? By far the strongest engine!

Post by duncan »

Rebel wrote: Sun May 17, 2020 12:55 am
carldaman wrote: Sun May 17, 2020 12:03 am
Rebel wrote: Sat May 16, 2020 10:31 pm
pohl4711 wrote: Sat May 16, 2020 11:52 am
Laskos wrote: Sat May 16, 2020 11:10 am I am unaware of a hybrid engine of the simplest kind, and I don't know what's the matter. Somebody better than me at coding can do this in an hour.
Simply: play using Lc0 first 35 moves and using SF after 35 moves to the end. On some RTX GPU and 4 to 32 core machine this combo will be by far the best engine.

Here are my results with RTX 2070 and 4 strong i7 cores:

200 games each match

Pure Lc0 net SV_3200 --- versus SF_11:
99/200

Combo Lc0 net SV_3200 to move 35, SF_11 after move 35 --- versus SF_11
123/200

An 85 Elo points improvement over pure Lc0.

As simple as that.
Lc0 is ridiculous in endgames, it always was and will probably improve only a little. In 200 games against Fritz 5.32, SF_11 scored 200/200, while Lc0 SV_3200 scored 193/200, Lc0 failing to convert in endgames 14 totally won positions even against Fritz 5.32 !!! It's ridiculous.

Are there volunteers to build such a combo engine in a matter of an hour?
This is, what you are looking for, isnt it?

Eman Chimera (previously known as Hybrid UCI) is a tool to run two or more UCI engines (one after another) to play the same game. Eman Chimera will switch to the appropriate engine based on move number of the current game being played

https://eman.zohosites.com/files/Eman%2 ... 0v1.10.zip

I know, that on playchess, some people use it, and SF takes over after 60 moves (35 moves seems a bit too early to me). So, it works in the FritzGUI. And what works in there, should run everywhere...
The switch should be done by the position on the board. About 6-8 months I wrote such a tool. SF10 and Lc0 both thinking, see screenshot below and then a simple referee program (I used ProDeo) makes the decision, in this board position, since it's the endgame, it takes the SF10 move, else the Lc0 move.

http://rebel13.nl/ZeroFishx.jpg
Hi Ed,
Is this tool still up for download?
Thanks! :)
It never was available because I lost interest while it wasn't ready for release. Things like time control and draws by repetition etc. But it works and Kai's observation is quite to the point.
Is Kai right that his crude ? ' Lc0 first 35 moves and using SF after 35 moves to the end. ' should just take an hour to code?