Page 3 of 9

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 9:42 am
by jd1
Uri Blass wrote:
jd1 wrote:
lucasart wrote:
jd1 wrote:
jd1 wrote:Hi everyone!

This is this new release of Toga II 3.0, thanks to Jim Ablett for the compilation.

http://www.computerchess.info/tdbb/phpB ... hp?id=1402

Features:
- scores 64% against previous Toga versions at various time controls, so should be a decent strength improvement.
- Multi-PV fixed

For more details see readme.txt where a full list of changes may be found.

Hope you enjoy using Toga :)

Jerry
To clarify: it scored +100 elo (64%) against Toga II 2.02 in 6000 games, 2000 at 10' + 0.1" and 4000 at 1 min + 0.1 s / move. It also defeated Toga Returns 1.1 and Toga 1.4.1 SE by similar margins.
That's really impressive. Congratulations! I know that it's hard to gain 100 elo at this level. It's possible that the results are a little inflated by self-play, but still, even if it's only 80 elo in rating list conditions, it's a huge step forward :D
Thanks Lucas! Not as impressive as writing your own strong engine though :) Congratulations on Discocheck 4.1 too! I doubt it will be 80 elo though, because most of the gain is in a more efficient search - it gets 1 depth further on average. I think 1 extra iteration is worth more in self-play than against other engines. What do you think?
I think that we do not know what is most of the gain and I can clearly see evaluation changes in the readme(New Endgame Knowledge and Improved Mobility Evaluation and Piece Combo (QN vs QB) adjustment) .

I also think that 1 extra iteration means nothing because we do not know if it see more or less at equal depth(I am also not sure that you tested that it is 1 extra iteration at all time control and that there is no difference between a search of 1 second and a search of 1 minute).

LMR improved may mean that it prunes more lines so it is weaker at the same depth, but it also may mean that it stop to prune some important lines that earlier it pruned so it is stronger at the same depth.

Changes:
- Multi-PV fixed
- Aspiration windows
- Hash table used in quiescent search (for lookup as well as store)
- LMR improved (thanks to the authors of Stockfish and Protector)
- TT avoid null-move flag
- New Endgame Knowledge
- Improved Mobility Evaluation (Ben Tennison)
- Piece Combo (QN vs QB) adjustment
That is true, 1 extra iteration in an approximation - it depends on depth. And they are not identical either due to LMR changes, which works both ways. I am not sure whether it is stronger or weaker at the same depth.

The evaluation changes would not be more than 10 elo, according to my testing.

Aspiration windows, qsearch hashing, and using the Stockfish LMR formula were the major wins - about +30 elo self play each.

Jerry

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 2:46 pm
by melajara
Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 3:16 pm
by Adam Hair
melajara wrote:Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.
Given that the highest rated Toga on Ingo's list (Toga II 1.4 beta5c BB) has a rating of 2664 Elo, I do not think the estimate based on Jerry's self-testing is all that inflated.

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 3:37 pm
by Eelco de Groot
178.5 - 358.5 33.24% Perf=2726 537 out of 2850 games played
Yes, it is better than Zappa Mexico II for instance if it keeps this up for Ingo's list. Not bad for a program that does not have bitboards. I'm not sure Zappa does have bitboards actually but its SMP implementation was certainly pretty advanced compared with Toga :) SMP implementation does not count though for Ingo's list.

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 3:42 pm
by Uri Blass
Adam Hair wrote:
melajara wrote:Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.
Given that the highest rated Toga on Ingo's list (Toga II 1.4 beta5c BB) has a rating of 2664 Elo, I do not think the estimate based on Jerry's self-testing is all that inflated.
Lucas:
"I know that it's hard to gain 100 elo at this level. It's possible that the results are a little inflated by self-play, but still, even if it's only 80 elo in rating list conditions, it's a huge step forward"

56 elo(assuming it does not change) seems clearly smaller than 80 elo so the 100 elo seems to be slightly more than little inflated judging.

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 3:53 pm
by lucasart
Uri Blass wrote:
Adam Hair wrote:
melajara wrote:Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.
Given that the highest rated Toga on Ingo's list (Toga II 1.4 beta5c BB) has a rating of 2664 Elo, I do not think the estimate based on Jerry's self-testing is all that inflated.
Lucas:
"I know that it's hard to gain 100 elo at this level. It's possible that the results are a little inflated by self-play, but still, even if it's only 80 elo in rating list conditions, it's a huge step forward"

56 elo(assuming it does not change) seems clearly smaller than 80 elo so the 100 elo seems to be slightly more than little inflated judging.
OK, what's your point ? Can you stop trolling Jerry's thread ? His impressive work on Toga deserves better than your snotty remarks. It's very hard to improve an engine of this level by that much.

Besides, your 56 elo isn't meaningful: you're lookoing at an unfinished IPON result, and comparing the elo against the wrong base version (IPON didn't test Toga II 2.0). And I'm not even talking about the huge error bars!

When I said 80 elo, I just pulled this number out of thin air. It was just a way to illustrate the fact that self-testing often magnifies the elo difference. And if you read Jerry's posts, you will see that he was very careful about elo announcements, and clarified exactly what was tested and under what conditions. He never predicted the elo difference in a rating list condition. He's just wondering as much as everyone else, how 100 elo in self-play at relatively fast tc will translate in rating list conditions.

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 4:08 pm
by Uri Blass
I simply defended Mel Cooper.

I did not claim that the improvement is not significant.

I only claimed that the estimate of 100 elo or 80 elo was probably too optimistic that was Mel Cooper's point based on my understanding.

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 9:07 pm
by jd1
Uri Blass wrote:I simply defended Mel Cooper.

I did not claim that the improvement is not significant.

I only claimed that the estimate of 100 elo or 80 elo was probably too optimistic that was Mel Cooper's point based on my understanding.
Well, if you read my older post, you will see that I said that I thought 80 elo was too optimistic: "I doubt it will be 80 elo". But at least the improvement is significant this time.

Jerry

Re: Toga II 3.0 released

Posted: Sat Mar 23, 2013 10:27 pm
by Eelco de Groot
Yes Jerry you never mentioned any number for an expected elo-gain, that is very wise practice. Jerry I had a question, long time ago I made some code for Toga so that it could mate with Knight and Bishop against King, I know it is absolutely not going to give elos but it was just nice to get it working at the time. I am not sure I posted it on TDDB anywhere but implemented it in Toga CMLX on Rybka forum. Maybe it could be added sometime or maybe for a separate modified version of Toga?

Eelco

Re: Toga II 3.0 released

Posted: Sun Mar 24, 2013 2:58 pm
by Uri Blass
Eelco de Groot wrote:
178.5 - 358.5 33.24% Perf=2726 537 out of 2850 games played
Yes, it is better than Zappa Mexico II for instance if it keeps this up for Ingo's list. Not bad for a program that does not have bitboards. I'm not sure Zappa does have bitboards actually but its SMP implementation was certainly pretty advanced compared with Toga :) SMP implementation does not count though for Ingo's list.
bigger rating than zappa Mexico but interesting that it does not prevent zappa to win convincingly

Toga II 3.0 32b - Zappa Mexico II (2703) 54.0 - 82.0 39.71%