What is New in Toga II 1.3x4?

[Stephen Ham]

Dear gents,

I tend to play games at very long time controls. For example, 180/40 is my norm. I began by giving Toga II 1.3x4 the same book as 1.2.1a. While it sometimes matches 1.2.1a's results in tournaments (including two Rybkas, Spike 1.2 Turin, Shredder 9, Fruit 2.2.1, and Naum 2.0), it usually fell short of the ealier version.

So I've recently given 1.3x4 a different bok, only to see it steadily fall lower on the tournament roster.

Last night, for a change of pace, I played a very fast tournament (relative to what I normally play) - game 15+1" for each engine. The tourney just finished and Toga II 1.3x4 finished last with 2/7, while 1.2.1a was third with 4/7 (behind the two Rybkas which tied for first, and Fruit 2.2.1).

I'm just starting to analyze the moves now, but it seems to be that Toga II 1.3x4's problems are not just with egtbs. Instead, its move selection seems weaker than 1.2.1a.

Since I play games at very long time controls, my problem is that I generate a small selection of games to evaluate. Thus, while I don't have quantity, I have quality games. But it seems that regardless of book or time-control, Toga 1.3x4 has yet to surpass 1.2.1a in tournaments. However, I've yet to play them against each other in a match with the same book. There'd be value in doing that if it weren't for the fact that 1.3x4 is scheduled to be "fixed". So, I'll spend my quality time only with the "fixed" version.

All the best,
Steve

[mclane]

strange.
i have the opposite effect.
i give them both the performance.bin.

in any tournament i have put them so far, the new version is above the old version.

[Dr.Wael Deeb]

Dear gents,

I tend to play games at very long time controls. For example, 180/40 is my norm. I began by giving Toga II 1.3x4 the same book as 1.2.1a. While it sometimes matches 1.2.1a's results in tournaments (including two Rybkas, Spike 1.2 Turin, Shredder 9, Fruit 2.2.1, and Naum 2.0), it usually fell short of the ealier version.

So I've recently given 1.3x4 a different bok, only to see it steadily fall lower on the tournament roster.

Last night, for a change of pace, I played a very fast tournament (relative to what I normally play) - game 15+1" for each engine. The tourney just finished and Toga II 1.3x4 finished last with 2/7, while 1.2.1a was third with 4/7 (behind the two Rybkas which tied for first, and Fruit 2.2.1).

I'm just starting to analyze the moves now, but it seems to be that Toga II 1.3x4's problems are not just with egtbs. Instead, its move selection seems weaker than 1.2.1a.

Since I play games at very long time controls, my problem is that I generate a small selection of games to evaluate. Thus, while I don't have quantity, I have quality games. But it seems that regardless of book or time-control, Toga 1.3x4 has yet to surpass 1.2.1a in tournaments. However, I've yet to play them against each other in a match with the same book. There'd be value in doing that if it weren't for the fact that 1.3x4 is scheduled to be "fixed". So, I'll spend my quality time only with the "fixed" version.

All the best,
Steve

Hi Steve,
Thanks for the answer,I have the same impression and results so far

[Shaun]

Thanks for the reply Graham,
Let's hope that Shaun will spot some light on this issue as it seems that things are missed up regarding the new Toga's performance.....

Hi Wael,

from my tests (combined with other CCRL testers results):

at 40/4

We can see that 1.3x4 is leading however the versions are too close in strength for 100% certainty - however I am fairly convinced that 1.3x4 is stronger than 1.2.1a at blitz.

at 40/12

again 1.3x4 is rated higher than 1.2.1a but again too close for certainty and the gap has closed.

My prefered version at slower time controls is 1.3 beta 1 however there are some fixes in 1.3x4 so it would be great if a couple of setting changes can see 1.3x4 ahead of 1.3 beta1 - I am currently running history threshhold tests - the results will appear over the next few weeks.

I have not run enough games at slower time controls to make any more comments on 1.3x4 - once the egbb code is fixed I will test extensively at all time controls.

All the best

Shaun

[Dr.Wael Deeb]

Thanks Shaun for the detailed answer

[Shaun]

i have also no doubt. 40/4 is no serious test-level.

A couple of points.

The results at 40/4 and 40/40 are surprisingly similar - therefore I do not believe we can dismiss 40/4 testing.

40/4 (X2 4600+) adjusted, while fast still gives decent chess - on one of my machines the adjusted time control is 40/17 and that is a 1.2ghz PIII.

However saying that my main interest in 40/4 is for comparison purposes - when comparing versions I test a 40/4 to ensure nothing is broken then at 40/12 - I then look to see if a version performs relatively worse/better at 40/12 to help decide if it is worth testing at 40/40.

Shaun

[IWB]

Hello all

Unfortunately I can not post the whole list but I played 500 games with the new Toga. Conditions are:

256 MB Hash, only 3+4 pc TBs, 1 Tread, 50 Noomen positions, changing colors, 6 min + 3 sec per game, Ponder ON.

I know that TOGA 1.3x4 can not access the Tbs but that true for 1.2.1 as well.

Program Elo + - Games Score Av.Op. Draws
Toga II 1.2.1a : 2758 10 10 3100 51.0 % 2751 36.1 %
Toga II 1.3x4 : 2735 23 23 500 45.6 % 2766 42.4 %

As you can see the 1.3x4 is 23 ELo worse then the 1.2.1. I play 200 more games to finish the test but do not expect a big increase.

Bye
Ingo

[Dr.Wael Deeb]

Hello all

Unfortunately I can not post the whole list but I played 500 games with the new Toga. Conditions are:

256 MB Hash, only 3+4 pc TBs, 1 Tread, 50 Noomen positions, changing colors, 6 min + 3 sec per game, Ponder ON.

I know that TOGA 1.3x4 can not access the Tbs but that true for 1.2.1 as well.

Program Elo + - Games Score Av.Op. Draws
Toga II 1.2.1a : 2758 10 10 3100 51.0 % 2751 36.1 %
Toga II 1.3x4 : 2735 23 23 500 45.6 % 2766 42.4 %

As you can see the 1.3x4 is 23 ELo worse then the 1.2.1. I play 200 more games to finish the test but do not expect a big increase.

Bye
Ingo

Thanks Ingo for sharing your results with us,it almost confirms my own impression

[nuff]

It would be best for those running these tests to give an Axon Bench PowX result of their hardware and to attach the pgns of the tournaments run. Stating whether ponder=off or on and which book used would also be helpful.

[Spock]

It would be best for those running these tests to give an Axon Bench PowX result of their hardware and to attach the pgns of the tournaments run. Stating whether ponder=off or on and which book used would also be helpful.

That bench contains 16-bit code and doesn't run on 64-bit Windows
Something else may be a better choice ? One of the Crafty benches ?