Open Source Blitz: DoubleCheck 2.5.2

[lucasart]

Provisional rating after 130/500 games. I'm playing the exact same games v2.5 played, and removed all the games played by v2.5 to avoid any rating distortion. It's still early to say, but so far that's +46 elo for only 1 line of code added and 2 modified (passed pawn scoring improved).
Ratings

Code: Select all

Rank Name                  Elo    +    - games score oppo. draws 
   1 Critter 1.4          3250   33   32   350   74%  3052   34% 
   2 IvanHoe 999946h      3192   31   31   350   64%  3068   37% 
   3 Stockfish 2.2.1      3174   31   31   400   66%  3023   32% 
   4 Protector 1.4        2921   32   32   400   51%  2915   21% 
   5 Umko 1.2             2868   27   27   550   53%  2855   24% 
   6 Toga 1.4.1           2840   25   25   650   56%  2805   23% 
   7 Daydreamer 1.75      2741   25   24   550   61%  2662   29% 
   8 Fruit 2.1            2700   23   23   650   52%  2682   26% 
   9 Crafty 23.4          2671   25   25   600   42%  2745   23% 
  10 Cheng3 1.07          2655   26   26   500   54%  2629   25% 
  11 GNU Chess 5.07.173b  2644   23   23   600   47%  2667   28% 
  12 Arasan 13.4          2641   26   26   500   46%  2672   22% 
  13 Scorpio 2.7          2628   23   23   650   42%  2688   26% 
  14 Rodent 0.10          2626   26   25   500   53%  2605   25% 
  15 Pepito 1.59          2599   25   25   550   50%  2593   23% 
  16 Sloppy 0.2.2         2541   23   23   700   41%  2601   23% 
  17 Greko 9.0            2495   23   23   700   42%  2560   22% 
  18 DoubleCheck 2.5.2    2476   58   54   130   76%  2273   17% 
  19 Pawny 0.3.1          2475   28   28   450   50%  2477   19% 
  20 Olithink 5.3.0       2382   31   31   350   53%  2357   20% 
  21 Sungorus 1.4         2350   28   28   450   43%  2408   22% 
  22 EXchess 6.0.2        2344   30   30   400   45%  2385   20% 
  23 Jazz 501             2316   30   30   380   42%  2375   22% 
  24 Beowulf 2.4          2275   34   34   300   40%  2352   20% 
  25 KMT Chess 1.2.1      2246   34   35   300   35%  2357   19% 

Conditions
* Open Source and Portable engines only: no closed source, commercial, or windows only programs.
* Copyleft: Ideally licensed under the GNU GPL, or with copyright restrictions that are not excessive.
* 1min+1sec/move, 64 MB Hash, 1 Thread, 64-bit, Ponder off, no EGTB.
* 8 book moves: neither to little, nor too much. Allows engines to develop their own plan, while offering a large enough number of opening positions.
* Bayeselo, offset Fruit 2.1 = 2700 elo.

[mar]

Not bad at all Let's see if it holds. Good luck with the new version.

Martin

[lucasart]

Not bad at all Let's see if it holds. Good luck with the new version.

Martin

Well, clearly it doesn't seem to hold that well after more games: 283/500 games, only +17 elo compared to v2.4

Code: Select all

Rank Name                  Elo    +    - games score oppo. draws 
   1 Critter 1.4          3250   33   32   350   74%  3052   34% 
   2 IvanHoe 999946h      3192   31   31   350   64%  3068   37% 
   3 Stockfish 2.2.1      3174   31   31   400   66%  3023   32% 
   4 Protector 1.4        2921   32   32   400   51%  2915   21% 
   5 Umko 1.2             2868   27   27   550   53%  2855   24% 
   6 Toga 1.4.1           2840   25   25   650   56%  2805   23% 
   7 Daydreamer 1.75      2741   25   25   550   61%  2662   29% 
   8 Fruit 2.1            2700   23   23   650   52%  2682   26% 
   9 Crafty 23.4          2671   25   25   600   42%  2745   23% 
  10 Cheng3 1.07          2655   26   25   500   54%  2629   25% 
  11 GNU Chess 5.07.173b  2644   23   23   600   47%  2667   28% 
  12 Arasan 13.4          2641   26   26   500   46%  2672   22% 
  13 Scorpio 2.7          2628   23   23   650   42%  2688   26% 
  14 Rodent 0.10          2626   26   26   500   53%  2605   25% 
  15 Pepito 1.59          2599   25   25   550   50%  2593   23% 
  16 Sloppy 0.2.2         2541   23   23   700   41%  2601   23% 
  17 Greko 9.0            2495   23   23   700   42%  2560   22% 
  18 Pawny 0.3.1          2475   28   28   450   50%  2477   19% 
  19 DoubleCheck 2.5.2    2447   36   35   283   68%  2313   23% 
  20 Olithink 5.3.0       2384   30   30   383   52%  2363   20% 
  21 Sungorus 1.4         2349   26   26   500   42%  2411   23% 
  22 EXchess 6.0.2        2346   28   28   450   44%  2391   21% 
  23 Jazz 501             2313   29   29   400   42%  2376   22% 
  24 Beowulf 2.4          2270   34   34   300   40%  2347   20% 
  25 KMT Chess 1.2.1      2240   34   35   300   35%  2352   19% 

Isn't it frustrating to see these massive improvements in self-play turn into tiny ones when playing against a varied population of engines...
I think I'll review my testing procedure, and do something more varied than self-playing validation of new code.

[mar]

Well i got +5 in 1.07 against 1.06 in CEGT. In self-play it looked like +50...

Martin

[lucasart]

+31 in the end, not that bad

Code: Select all

Rank Name                  Elo    +    - games score oppo. draws 
   1 Critter 1.4          3250   33   32   350   74%  3052   34% 
   2 IvanHoe 999946h      3192   31   31   350   64%  3068   37% 
   3 Stockfish 2.2.1      3174   31   30   400   66%  3023   32% 
   4 Protector 1.4        2921   32   32   400   51%  2915   21% 
   5 Umko 1.2             2868   27   27   550   53%  2855   24% 
   6 Toga 1.4.1           2840   25   25   650   56%  2805   23% 
   7 Daydreamer 1.75      2741   25   25   550   61%  2662   29% 
   8 Fruit 2.1            2700   23   23   650   52%  2683   26% 
   9 Crafty 23.4          2671   25   25   600   42%  2745   23% 
  10 Cheng3 1.07          2655   26   25   500   54%  2629   25% 
  11 GNU Chess 5.07.173b  2645   23   23   600   47%  2667   28% 
  12 Arasan 13.4          2641   26   26   500   46%  2672   22% 
  13 Scorpio 2.7          2628   23   23   650   42%  2688   26% 
  14 Rodent 0.10          2627   26   26   500   53%  2606   25% 
  15 Pepito 1.59          2598   24   24   600   52%  2584   22% 
  16 Sloppy 0.2.2         2537   22   22   750   42%  2593   24% 
  17 Greko 9.0            2501   22   22   750   43%  2556   21% 
  18 Pawny 0.3.1          2483   27   26   500   51%  2480   19% 
  19 DoubleCheck 2.5.2    2461   27   26   500   57%  2408   21% 
  20 Olithink 5.3.0       2398   29   29   400   53%  2376   20% 
  21 Sungorus 1.4         2357   26   26   500   42%  2418   23% 
  22 EXchess 6.0.2        2355   28   28   450   44%  2399   21% 
  23 Jazz 501             2323   29   29   400   42%  2385   22% 
  24 Beowulf 2.4          2280   34   34   300   40%  2357   20% 
  25 KMT Chess 1.2.1      2251   34   35   300   35%  2362   19% 

[mar]

Sure a nice progress Congrats!