A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by kranium »

lucasart wrote:
ThatsIt wrote:... after 600 games:
http://computerschach.forumfrei.net/t23 ... itions#392

Best wishes,
G.S.
you are comparing different compiles of the *same source code*, so you don't need to play any games. If there was a 1 elo difference between the two, you would need tens of thousands of games to detect it anyway.
just run the program from the command line, and type this:

Code: Select all

uci
ucinewgame
position startpos
go depth 20
quit
and compare:
* node count, score and PV should be equal otherwise one compile at least is broken
* if so, then look at the time (displayed in ms)
and now you know *without any doubt* which compile is better, and you've saved your self a huge electricity bill of testing day and night for weeks...
PP likes to experiment
sometimes his compiles are not changed in any substantial way from the ippolit.wikispaces.com released tarball
but he sometimes changes piece values, adds CPU detection routines, assembly language codepath (SSE, etc.) libraries., etc. and more
for ex:

http://www.talkchess.com/forum/viewtopi ... ht=#436990

there are in fact substantial changes, i.e. quite a few things that may/should affect ELO...and the way the engine plays.

KLO normally does not experiment, and as a whole has produced very consistent and accurate compiles with the vast majority of code intact.
he does add CPU detection, etc.
and recently has been changing/tweaking the TC routines...

the only way to know for sure if the codebase is 100% original is to ask, as neither compiler releases their 'changed' source code

Windows users are limited to these 2 choices: KLO's compiles or PeterPans
under these circumstances it makes complete sense to test them...i simply recommend asking 1st, especially if you are a 'purist' and want a Windows binary exactly as
intended by the authors.

PS-
Frank Q. ran 1400 games testing IvanHoe 46 (PP vs KLO)
and KLO ended up on top by 6 ELO.
Last edited by kranium on Mon Jan 16, 2012 4:32 pm, edited 3 times in total.
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by kranium »

Sadly, Franks excellent list is no longer avaialble, but i had posted a copy of it on Immortal at one point, while the test was still running:

I reproduce it here:
1 1 Houdini 2.0c x64 3019 20 20 1220 82% 2753 25% Update, + 21
- 2 Houdini 1.5 x64 2998 14 14 2320 78% 2771 29%
- 3 Houdini 1.5 w32 2981 19 19 1240 79% 2756 29%
2 4 Rybka 4 x64 Exp. 42 2968 20 19 1212 79% 2731 26%
- 5 Rybka 4.1 x64 Exp. 79TD v.1 2963 20 19 1200 79% 2735 26%
3 6 Komodo 3.0 x64 2961 17 17 1500 77% 2752 32%
- 7 Rybka 4 x64 Exp. 61 2959 21 21 1000 78% 2741 27%
4 8 IvanHoe 999946f x64 2958 45 43 200! 74% 2776 32% TEST * STILL RUNNING
- 9 Rybka 4.1 x64 2955 15 15 1779 73% 2775 36%
5 10 Critter 1.2 x64 2954 16 15 1700 76% 2752 33%
- 11 IvanHoe B46fB x64 2951 44 42 200! 74% 2776 40% Update, + 11 * STILL RUNNING
6 12 Stockfish 2.1.1 JA x64 PHQ 2948 18 18 1292 76% 2747 32%
- 13 Houdini 1.03a x64 2943 21 21 1000 80% 2710 30%
7 14 Fire 2.2 xTreme x64 2940 16 16 1660 75% 2747 34%
- 15 IvanHoe B47cB x64 2940 15 15 1768 71% 2784 39%
- 16 Rybka 4 x64 2940 18 17 1480 80% 2696 28%
- 17 Fire 1.5 xTreme x64 2935 17 16 1400 73% 2761 36%
- 18 Critter 1.2 w32 2934 18 18 1200 75% 2753 36%
- 19 IvanHoe B49jA x64 2931 19 18 1160 76% 2737 34%
- 20 Stockfish 2.1.1 JA x64 2929 15 15 1817 73% 2755 35%
- 21 Komodo 2.03 JA x64 2926 17 16 1480 73% 2749 31%
- 22 IvanHoe B52aC x64 2918 20 20 1000 76% 2726 34%
- 23 RobboLito 0.09 x64 2918 17 16 1492 74% 2731 33% NEW

after 1400 games, it ended in a very similar relationship (IvanHoe 999946f KLO +6 compared to IvanHoe B46fB PP)

Please note:
the PP compile that Gerhard S. is testing (Ivanhoe 46bh) is different that the one Frank Q. tested (IvanHoe B46fB),
but both appear to be using the same KLO compile.

PS-
KLO compiles are more complete and require significantly more effort:
he compiles:
both 'Game Play' and 'Analysis' modes
Linux binaries as well as Windows
and he also includes the 2 java GUIs:
ComradesGUI and RobboExplorative
in addition, he ports the abundant documentation provided by the authors into PDFs.

PP normally compiles 'GamePlay mode'...
and compiles for Windows only
ThatsIt
Posts: 992
Joined: Thu Mar 09, 2006 2:11 pm

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by ThatsIt »

kranium wrote: [...snip...]
PS-
KLO compiles are more complete and require significantly more effort:
he compiles:
both 'Game Play' and 'Analysis' modes
...
PP also does.

Best wishes,
G.S.
ThatsIt
Posts: 992
Joined: Thu Mar 09, 2006 2:11 pm

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by ThatsIt »

kranium wrote: [...snip...]
KLO normally does not experiment, and as a whole has produced very consistent and accurate compiles with the vast majority of code intact.
he does add CPU detection, etc.
and recently has been changing/tweaking the TC routines...
[...snip...]
Who is right ?
http://talkchess.com/forum/viewtopic.ph ... 35&t=41919

Best wishes,
G.S.
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by kranium »

ThatsIt wrote:
kranium wrote: [...snip...]
PS-
KLO compiles are more complete and require significantly more effort:
he compiles:
both 'Game Play' and 'Analysis' modes
...
PP also does.

Best wishes,
G.S.

Yes, you're right...
my bad
ThatsIt
Posts: 992
Joined: Thu Mar 09, 2006 2:11 pm

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by ThatsIt »

kranium wrote: [...snip...]
not sure what the disagreement is...
can you be more specific?
suggest a modicum of research before testing any engine , had you done this with klo's offering you would have seen others state it was buggy (some even called this the worst engine ever )and even klo stated it lost on time and posted several so called fixs for it. sadly he is something of an arrogant muppet so it is very easy to be confused / mislead by the way he posts his engines on his own site
[...snip...]

Best wishes,
G.S.
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by kranium »

ThatsIt wrote:
kranium wrote: [...snip...]
not sure what the disagreement is...
can you be more specific?
suggest a modicum of research before testing any engine , had you done this with klo's offering you would have seen others state it was buggy (some even called this the worst engine ever )and even klo stated it lost on time and posted several so called fixs for it. sadly he is something of an arrogant muppet so it is very easy to be confused / mislead by the way he posts his engines on his own site
[...snip...]

Best wishes,
G.S.
nonsense...

KLO deserve nothing but credit, for the efforts i listed above,
and for maintaining his own FTP site in order to make these archives available...

http://chess.cygnitec.com/engine/ivanhoe/

his work has been of high quality, consistent, and dependable, and many many users have benefited
ThatsIt
Posts: 992
Joined: Thu Mar 09, 2006 2:11 pm

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by ThatsIt »

THX !!

Best wishes,
G.S.
Izak Pretorius
Posts: 34
Joined: Wed May 11, 2011 7:44 am

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by Izak Pretorius »

kranium wrote:
ThatsIt wrote:
kranium wrote: [...snip...]
not sure what the disagreement is...
can you be more specific?
suggest a modicum of research before testing any engine , had you done this with klo's offering you would have seen others state it was buggy (some even called this the worst engine ever )and even klo stated it lost on time and posted several so called fixs for it. sadly he is something of an arrogant muppet so it is very easy to be confused / mislead by the way he posts his engines on his own site
[...snip...]

Best wishes,
G.S.
nonsense...

KLO deserve nothing but credit, for the efforts i listed above,
and for maintaining his own FTP site in order to make these archives available...

http://chess.cygnitec.com/engine/ivanhoe/

his work has been of high quality, consistent, and dependable, and many many users have benefited
Hi Norman ;)

Correct.

This does not have to be turned into a competition,since Ivanhoe is not Peterpan's source neither is it KLO's ;) and KLO has done a wonderful job.

But expect more from me to come this year...

Best Regards
Peterpan
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: A privat IvanHoe 9.46bh test under CEGT-Blitz conditions

Post by kranium »

Izak Pretorius wrote:
kranium wrote:
ThatsIt wrote:
kranium wrote: [...snip...]
not sure what the disagreement is...
can you be more specific?
suggest a modicum of research before testing any engine , had you done this with klo's offering you would have seen others state it was buggy (some even called this the worst engine ever )and even klo stated it lost on time and posted several so called fixs for it. sadly he is something of an arrogant muppet so it is very easy to be confused / mislead by the way he posts his engines on his own site
[...snip...]

Best wishes,
G.S.
nonsense...

KLO deserve nothing but credit, for the efforts i listed above,
and for maintaining his own FTP site in order to make these archives available...

http://chess.cygnitec.com/engine/ivanhoe/

his work has been of high quality, consistent, and dependable, and many many users have benefited
Hi Norman ;)

Correct.

This does not have to be turned into a competition,since Ivanhoe is not Peterpan's source neither is it KLO's ;) and KLO has done a wonderful job.

But expect more from me to come this year...

Best Regards
Peterpan
good to hear Izak!

you've contributed an awful lot as well, as most know and appreciate...
keep up the excellent work!