Why are the Ippo derivative stronger than Stockfish?

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Why are the Ippo derivative stronger than Stockfish?

Post by lkaufman » Thu Feb 24, 2011 1:32 am

It seems to me that both the Ippo family programs and SF use many similar ideas, and both are extremely well written and fast programs. Moreover SF had access to the Ippo open-source code for a long time now, and yet despite huge progress it still lacks behind all of these programs, though the gap is getting small. Our experience with Komodo is that SF ideas or something at least vaguely similar to them seem to work better for us than Ippo ideas, despite the fact that our eval is more like the Ippo family than like the SF eval, since I was responsible for the eval weights in both Rybka and Komodo. So I just cannot see any reason for the superiority of the Ippo programs over SF, and yet it's hard to argue with results. Any opinions out there?

User avatar
silentshark
Posts: 236
Joined: Sat Mar 27, 2010 6:15 pm
Contact:

Re: Why are the Ippo derivative stronger than Stockfish?

Post by silentshark » Thu Feb 24, 2011 12:36 pm

lkaufman wrote:It seems to me that both the Ippo family programs and SF use many similar ideas, and both are extremely well written and fast programs. Moreover SF had access to the Ippo open-source code for a long time now, and yet despite huge progress it still lacks behind all of these programs, though the gap is getting small. Our experience with Komodo is that SF ideas or something at least vaguely similar to them seem to work better for us than Ippo ideas, despite the fact that our eval is more like the Ippo family than like the SF eval, since I was responsible for the eval weights in both Rybka and Komodo. So I just cannot see any reason for the superiority of the Ippo programs over SF, and yet it's hard to argue with results. Any opinions out there?
is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?

mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 7:17 pm

Re: Why are the Ippo derivative stronger than Stockfish?

Post by mcostalba » Thu Feb 24, 2011 1:10 pm

silentshark wrote: is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?
Beeeep !!! Sorry, Wrong Answer. Thanks for having tried !

Because this question has been, for some reason, posted on the programmers subforum I assume the author was interested to give some fact and receive some fact back. Amazingly currently I don't see neither ;-)

Anyhow I'll keep looking at this post in the remote case something semi-serious emerges....

Joost Buijs
Posts: 986
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Why are the Ippo derivative stronger than Stockfish?

Post by Joost Buijs » Thu Feb 24, 2011 3:54 pm

mcostalba wrote:
silentshark wrote: is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?
Beeeep !!! Sorry, Wrong Answer. Thanks for having tried !
Why is this the wrong answer? I really think speed can be an issue here.

For instance lets compare Stockfish to Houdini and assume both engines do approximately the same work at each node. Probably most of the nodes counted will be quiescense nodes heavely influenced by the speed of the evaluation function.
On my core i7 980x Stockfish 2.01 JA runs at 7500 to 8000 knps while Houdini 1.5a runs at 15000 to 16000 knps. That means that Houdini does twice the amount of work in the same time. This can easely account for a difference of 70 ELO points.

zamar
Posts: 613
Joined: Sun Jan 18, 2009 6:03 am

Re: Why are the Ippo derivative stronger than Stockfish?

Post by zamar » Thu Feb 24, 2011 3:56 pm

lkaufman wrote:It seems to me that both the Ippo family programs and SF use many similar ideas, and both are extremely well written and fast programs. Moreover SF had access to the Ippo open-source code for a long time now, and yet despite huge progress it still lacks behind all of these programs, though the gap is getting small. Our experience with Komodo is that SF ideas or something at least vaguely similar to them seem to work better for us than Ippo ideas, despite the fact that our eval is more like the Ippo family than like the SF eval, since I was responsible for the eval weights in both Rybka and Komodo. So I just cannot see any reason for the superiority of the Ippo programs over SF, and yet it's hard to argue with results. Any opinions out there?
Their search algorithms still differ a lot. IPP* has different pruning rules close to the leaves compared to Stockfish. You can't just blindly copy Rule X from IPP* to SF. The search is one picture. And we don't want to copy the whole search, it would just be pointless.

Anyway we got a lot of good ideas from IPP*:

- Singular extensions
- Aggressive LMR
- Aggressive neg. SEE moves pruning
Joona Kiiski

mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 7:17 pm

Re: Why are the Ippo derivative stronger than Stockfish?

Post by mcostalba » Thu Feb 24, 2011 5:45 pm

Joost Buijs wrote: Why is this the wrong answer? I really think speed can be an issue here.

For instance lets compare Stockfish to Houdini and assume both engines do approximately the same work at each node. Probably most of the nodes counted will be quiescense nodes heavely influenced by the speed of the evaluation function.
On my core i7 980x Stockfish 2.01 JA runs at 7500 to 8000 knps while Houdini 1.5a runs at 15000 to 16000 knps. That means that Houdini does twice the amount of work in the same time. This can easely account for a difference of 70 ELO points.
I don't know how houdini counts the nodes. You should compare with Ippolit.

Bare speed is at least equal: do a perft test to check speed of move generation functions. Above that we have the actual search that is at the same level and the evaluation that perhaps is slower (because we don't use lazy evaluation), but I don't expect by so much. Even a 10% faster evaluation per node on average (that is a lot !) it means a 3-4% faster engine (because evaluation accounts for no more then 35-40% of total used time) and this translates in less then 5 ELO due to only speed difference, but probably even less.

bob
Posts: 20557
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Why are the Ippo derivative stronger than Stockfish?

Post by bob » Thu Feb 24, 2011 5:56 pm

Joost Buijs wrote:
mcostalba wrote:
silentshark wrote: is the level of optimisation the difference? Do the SF team favour clean code/ readable code/ concise code over fast code?
Beeeep !!! Sorry, Wrong Answer. Thanks for having tried !
Why is this the wrong answer? I really think speed can be an issue here.

For instance lets compare Stockfish to Houdini and assume both engines do approximately the same work at each node. Probably most of the nodes counted will be quiescense nodes heavely influenced by the speed of the evaluation function.
On my core i7 980x Stockfish 2.01 JA runs at 7500 to 8000 knps while Houdini 1.5a runs at 15000 to 16000 knps. That means that Houdini does twice the amount of work in the same time. This can easely account for a difference of 70 ELO points.
It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.

mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 7:17 pm

Re: Why are the Ippo derivative stronger than Stockfish?

Post by mcostalba » Thu Feb 24, 2011 6:12 pm

bob wrote: It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.
Yes, even worse than that. If node count is the double and would be all due to different evaluation, because evaluation does not account for more then 40% of total time, it would mean that SF evaluation is 4X slower then Houdini !!

This is almost nonsense. What I think is:

1) You should compare single core nodes because according to different SMP schemes node count could seem different on multi thread.

2) You should compare apples with apples: I don't know how Houdini counts, I know SF has recently changed node accounting and this showed in a -20% reported speed, but it is only an artifact because actual engine speed of 2.0.1 is always the same of 1.9, actually even a bit faster.

3) We already know hardware POPCOUNT gives a 3-4% boost and the JA versions don't have it.

At the end I think speed difference, if any, is really a misleading factor in trying to understand the differences. I think post of Joona makes a lot more sense.

Joost Buijs
Posts: 986
Joined: Thu Jul 16, 2009 8:47 am
Location: Almere, The Netherlands

Re: Why are the Ippo derivative stronger than Stockfish?

Post by Joost Buijs » Thu Feb 24, 2011 6:24 pm

bob wrote: It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.
I don't know the innards of both engines very well, but I assumed the evaluation function of Stockfish and Houdini to be roughly of the same complexity.

bob
Posts: 20557
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Why are the Ippo derivative stronger than Stockfish?

Post by bob » Thu Feb 24, 2011 11:35 pm

Joost Buijs wrote:
bob wrote: It doesn't mean that at all. It might mean stockfish does 2x the work at a single node. Whether that is good or bad is unknown. But it is not bad purely because it is slower.
I don't know the innards of both engines very well, but I assumed the evaluation function of Stockfish and Houdini to be roughly of the same complexity.
Evaluation is not the only thing. What about the code used to deal with pruning, reductions and such. Some use simple rules. Some toss in an eval call, + possibly a SEE call. Making a single node more expensive, but possibly more accurate as well...

Post Reply