Toga II 3.0 released

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jd1
Posts: 266
Joined: Wed Oct 24, 2012 12:07 am

Re: Toga II 3.0 released

Post by jd1 » Sat Mar 23, 2013 8:42 am

Uri Blass wrote:
jd1 wrote:
lucasart wrote:
jd1 wrote:
jd1 wrote:Hi everyone!

This is this new release of Toga II 3.0, thanks to Jim Ablett for the compilation.

http://www.computerchess.info/tdbb/phpB ... hp?id=1402

Features:
- scores 64% against previous Toga versions at various time controls, so should be a decent strength improvement.
- Multi-PV fixed

For more details see readme.txt where a full list of changes may be found.

Hope you enjoy using Toga :)

Jerry
To clarify: it scored +100 elo (64%) against Toga II 2.02 in 6000 games, 2000 at 10' + 0.1" and 4000 at 1 min + 0.1 s / move. It also defeated Toga Returns 1.1 and Toga 1.4.1 SE by similar margins.
That's really impressive. Congratulations! I know that it's hard to gain 100 elo at this level. It's possible that the results are a little inflated by self-play, but still, even if it's only 80 elo in rating list conditions, it's a huge step forward :D
Thanks Lucas! Not as impressive as writing your own strong engine though :) Congratulations on Discocheck 4.1 too! I doubt it will be 80 elo though, because most of the gain is in a more efficient search - it gets 1 depth further on average. I think 1 extra iteration is worth more in self-play than against other engines. What do you think?
I think that we do not know what is most of the gain and I can clearly see evaluation changes in the readme(New Endgame Knowledge and Improved Mobility Evaluation and Piece Combo (QN vs QB) adjustment) .

I also think that 1 extra iteration means nothing because we do not know if it see more or less at equal depth(I am also not sure that you tested that it is 1 extra iteration at all time control and that there is no difference between a search of 1 second and a search of 1 minute).

LMR improved may mean that it prunes more lines so it is weaker at the same depth, but it also may mean that it stop to prune some important lines that earlier it pruned so it is stronger at the same depth.

Changes:
- Multi-PV fixed
- Aspiration windows
- Hash table used in quiescent search (for lookup as well as store)
- LMR improved (thanks to the authors of Stockfish and Protector)
- TT avoid null-move flag
- New Endgame Knowledge
- Improved Mobility Evaluation (Ben Tennison)
- Piece Combo (QN vs QB) adjustment
That is true, 1 extra iteration in an approximation - it depends on depth. And they are not identical either due to LMR changes, which works both ways. I am not sure whether it is stronger or weaker at the same depth.

The evaluation changes would not be more than 10 elo, according to my testing.

Aspiration windows, qsearch hashing, and using the Stockfish LMR formula were the major wins - about +30 elo self play each.

Jerry

melajara
Posts: 213
Joined: Thu Dec 16, 2010 3:39 pm

Re: Toga II 3.0 released

Post by melajara » Sat Mar 23, 2013 1:46 pm

Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.
Per ardua ad astra

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Toga II 3.0 released

Post by Adam Hair » Sat Mar 23, 2013 2:16 pm

melajara wrote:Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.
Given that the highest rated Toga on Ingo's list (Toga II 1.4 beta5c BB) has a rating of 2664 Elo, I do not think the estimate based on Jerry's self-testing is all that inflated.

User avatar
Eelco de Groot
Posts: 4157
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: Toga II 3.0 released

Post by Eelco de Groot » Sat Mar 23, 2013 2:37 pm

178.5 - 358.5 33.24% Perf=2726 537 out of 2850 games played
Yes, it is better than Zappa Mexico II for instance if it keeps this up for Ingo's list. Not bad for a program that does not have bitboards. I'm not sure Zappa does have bitboards actually but its SMP implementation was certainly pretty advanced compared with Toga :) SMP implementation does not count though for Ingo's list.
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

Uri Blass
Posts: 8586
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Toga II 3.0 released

Post by Uri Blass » Sat Mar 23, 2013 2:42 pm

Adam Hair wrote:
melajara wrote:Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.
Given that the highest rated Toga on Ingo's list (Toga II 1.4 beta5c BB) has a rating of 2664 Elo, I do not think the estimate based on Jerry's self-testing is all that inflated.
Lucas:
"I know that it's hard to gain 100 elo at this level. It's possible that the results are a little inflated by self-play, but still, even if it's only 80 elo in rating list conditions, it's a huge step forward"

56 elo(assuming it does not change) seems clearly smaller than 80 elo so the 100 elo seems to be slightly more than little inflated judging.

User avatar
lucasart
Posts: 3040
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: Toga II 3.0 released

Post by lucasart » Sat Mar 23, 2013 2:53 pm

Uri Blass wrote:
Adam Hair wrote:
melajara wrote:Indeed, VERY inflated judging from IPON current match (albeit 456/2850 games) with a provisional 2720 Elo performance.
Given that the highest rated Toga on Ingo's list (Toga II 1.4 beta5c BB) has a rating of 2664 Elo, I do not think the estimate based on Jerry's self-testing is all that inflated.
Lucas:
"I know that it's hard to gain 100 elo at this level. It's possible that the results are a little inflated by self-play, but still, even if it's only 80 elo in rating list conditions, it's a huge step forward"

56 elo(assuming it does not change) seems clearly smaller than 80 elo so the 100 elo seems to be slightly more than little inflated judging.
OK, what's your point ? Can you stop trolling Jerry's thread ? His impressive work on Toga deserves better than your snotty remarks. It's very hard to improve an engine of this level by that much.

Besides, your 56 elo isn't meaningful: you're lookoing at an unfinished IPON result, and comparing the elo against the wrong base version (IPON didn't test Toga II 2.0). And I'm not even talking about the huge error bars!

When I said 80 elo, I just pulled this number out of thin air. It was just a way to illustrate the fact that self-testing often magnifies the elo difference. And if you read Jerry's posts, you will see that he was very careful about elo announcements, and clarified exactly what was tested and under what conditions. He never predicted the elo difference in a rating list condition. He's just wondering as much as everyone else, how 100 elo in self-play at relatively fast tc will translate in rating list conditions.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

Uri Blass
Posts: 8586
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Toga II 3.0 released

Post by Uri Blass » Sat Mar 23, 2013 3:08 pm

I simply defended Mel Cooper.

I did not claim that the improvement is not significant.

I only claimed that the estimate of 100 elo or 80 elo was probably too optimistic that was Mel Cooper's point based on my understanding.

jd1
Posts: 266
Joined: Wed Oct 24, 2012 12:07 am

Re: Toga II 3.0 released

Post by jd1 » Sat Mar 23, 2013 8:07 pm

Uri Blass wrote:I simply defended Mel Cooper.

I did not claim that the improvement is not significant.

I only claimed that the estimate of 100 elo or 80 elo was probably too optimistic that was Mel Cooper's point based on my understanding.
Well, if you read my older post, you will see that I said that I thought 80 elo was too optimistic: "I doubt it will be 80 elo". But at least the improvement is significant this time.

Jerry

User avatar
Eelco de Groot
Posts: 4157
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: Toga II 3.0 released

Post by Eelco de Groot » Sat Mar 23, 2013 9:27 pm

Yes Jerry you never mentioned any number for an expected elo-gain, that is very wise practice. Jerry I had a question, long time ago I made some code for Toga so that it could mate with Knight and Bishop against King, I know it is absolutely not going to give elos but it was just nice to get it working at the time. I am not sure I posted it on TDDB anywhere but implemented it in Toga CMLX on Rybka forum. Maybe it could be added sometime or maybe for a separate modified version of Toga?

Eelco
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

Uri Blass
Posts: 8586
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Toga II 3.0 released

Post by Uri Blass » Sun Mar 24, 2013 1:58 pm

Eelco de Groot wrote:
178.5 - 358.5 33.24% Perf=2726 537 out of 2850 games played
Yes, it is better than Zappa Mexico II for instance if it keeps this up for Ingo's list. Not bad for a program that does not have bitboards. I'm not sure Zappa does have bitboards actually but its SMP implementation was certainly pretty advanced compared with Toga :) SMP implementation does not count though for Ingo's list.
bigger rating than zappa Mexico but interesting that it does not prevent zappa to win convincingly

Toga II 3.0 32b - Zappa Mexico II (2703) 54.0 - 82.0 39.71%

Post Reply