Evaluating material advantages as position simplifys

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Howard E
Posts: 261
Joined: Wed Mar 08, 2006 8:49 pm

Evaluating material advantages as position simplifys

Post by Howard E »

An observation I made on Rybka 1 when it was released (2005 maybe ?) was that it went for reducing the number of pieces when having a material advantage like no other program I knew of. I devised a set of tests to see if other programs did the same. To my surprise I could not find one (mostly free programs ) that evaluated these seven positions the same as Rybka 1 did. Toga then and know for example evals #1 and #7 much the same.

Just look at position #1 and ask yourself, could I beat a strong program at Knight odds?

Now look at position #7 and ask the same question. Most of us would probably win the knight odds in #7 even against the best of programs.(or maybe not)

The point is, how many chess programs progressively show an advantage for the knight odds positions? (only about 20 seconds of eval time is required).

Back in 2005 I only found this behavior in Rybka.
Any other progarams have this wired into the eval?

Is it those Rybka tables that make this happen?
Rybka 3 - 32bit has +2.78 for #1 and +3.71 for #7.

I also tested for bishop odds and all the other pieces. With pawns too. Always the same result for Rybka - it "knows" that a piece or pawn up in an ending is more advantageous than during the opening.


[d]r1bqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq;
[d]r1bqkb1r/pppppppp/8/8/8/8/PPPPPPPP/RNBQKB1R w KQkq;
[d]r2qkb1r/pppppppp/8/8/8/8/PPPPPPPP/RN1QKB1R w KQkq;
[d]r2qk2r/pppppppp/8/8/8/8/PPPPPPPP/RN1QK2R w KQkq -;
[d]3qk2r/pppppppp/8/8/8/8/PPPPPPPP/1N1QK2R w Kk -;
[d]3qk3/pppppppp/8/8/8/8/PPPPPPPP/1N1QK3 w - -;
[d]4k3/pppppppp/8/8/8/8/PPPPPPPP/1N2K3 w - -;
Top
User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 8:30 pm
Location: Chicago, Illinois, USA

Re: Evaluating material advantages as position simplifys

Post by michiguel »

Howard E wrote:An observation I made on Rybka 1 when it was released (2005 maybe ?) was that it went for reducing the number of pieces when having a material advantage like no other program I knew of. I devised a set of tests to see if other programs did the same. To my surprise I could not find one (mostly free programs ) that evaluated these seven positions the same as Rybka 1 did. Toga then and know for example evals #1 and #7 much the same.

Just look at position #1 and ask yourself, could I beat a strong program at Knight odds?

Now look at position #7 and ask the same question. Most of us would probably win the knight odds in #7 even against the best of programs.(or maybe not)

The point is, how many chess programs progressively show an advantage for the knight odds positions? (only about 20 seconds of eval time is required).

Back in 2005 I only found this behavior in Rybka.
Any other progarams have this wired into the eval?

Is it those Rybka tables that make this happen?
Rybka 3 - 32bit has +2.78 for #1 and +3.71 for #7.

I also tested for bishop odds and all the other pieces. With pawns too. Always the same result for Rybka - it "knows" that a piece or pawn up in an ending is more advantageous than during the opening.


[d]r1bqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq;
[d]r1bqkb1r/pppppppp/8/8/8/8/PPPPPPPP/RNBQKB1R w KQkq;
[d]r2qkb1r/pppppppp/8/8/8/8/PPPPPPPP/RN1QKB1R w KQkq;
[d]r2qk2r/pppppppp/8/8/8/8/PPPPPPPP/RN1QK2R w KQkq -;
[d]3qk2r/pppppppp/8/8/8/8/PPPPPPPP/1N1QK2R w Kk -;
[d]3qk3/pppppppp/8/8/8/8/PPPPPPPP/1N1QK3 w - -;
[d]4k3/pppppppp/8/8/8/8/PPPPPPPP/1N2K3 w - -;
scores for gaviota (without search)
2.91, 3.31, 3.56, 3.89, 4.58, 4.78, 4.94

After 20 s search
3.26, 3.70, 4.36, 4.77, 5.16, 5.67, 5.49

Miguel
Top
Howard E
Posts: 261
Joined: Wed Mar 08, 2006 8:49 pm

Re: Evaluating material advantages as position simplifys

Post by Howard E »

Thanks for testing this. Your evals rise even more sharply than Rybka 3.
Top
Howard E
Posts: 261
Joined: Wed Mar 08, 2006 8:49 pm

Re: Evaluating material advantages as position simplifys

Post by Howard E »

Progression for Doch64 corei7 20 seconds per position

3.55 3.35 3.38 3.52 3.63 3.67 4.00
Top
Howard E
Posts: 261
Joined: Wed Mar 08, 2006 8:49 pm

Re: Evaluating material advantages as position simplifys

Post by Howard E »

Maybe it's just enough to test only the first and last position to see
if an engine gets the idea that a piece up means more in an endgame.

Protector3.2 3.42 and 3,47 for first and last
Top
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: Evaluating material advantages as position simplifys

Post by diep »

Howard E wrote:An observation I made on Rybka 1 when it was released (2005 maybe ?) was that it went for reducing the number of pieces when having a material advantage like no other program I knew of.
Pardon me?

Rybka's strong point is that it hardly exchanges during games and just shuffles forever. Especially compared to for example Diep and Shredder and Zappa, which are classical programs that do exchange quickly and increase scores.
Howard E wrote: I devised a set of tests to see if other programs did the same. To my surprise I could not find one (mostly free programs ) that evaluated these seven positions the same as Rybka 1 did. Toga then and know for example evals #1 and #7 much the same.

Just look at position #1 and ask yourself, could I beat a strong program at Knight odds?

Now look at position #7 and ask the same question. Most of us would probably win the knight odds in #7 even against the best of programs.(or maybe not)

The point is, how many chess programs progressively show an advantage for the knight odds positions? (only about 20 seconds of eval time is required).

Back in 2005 I only found this behavior in Rybka.
Any other progarams have this wired into the eval?
Really a lot do.
Howard E wrote: Is it those Rybka tables that make this happen?
Rybka 3 - 32bit has +2.78 for #1 and +3.71 for #7.
position 1 (debug mode of diep and single core, but doesn't matter as it is just about eval):

00:00 33 1 0 (1) 1 (0,0) 4.077 Ng1-f3
00:00 100 3 0 (1) 1 (0,0) 4.419 e2-e4
00:00 1333 40 0 (1) 2 (0,1) 3.404 e2-e4 e7-e5 (1)
00:00 9133 274 0 (1) 3 (0,2) 3.992 e2-e4 d7-d5 (2) Nb1-c3 (1) d5xe4 (-1) Nc3xe4 (-1)
++ g1-f3 procnr=0 terug=3993 org=[3992;3993] newwindow=[3992;520000]
00:00 9960 498 0 (1) 3 (0,2) 4.016 Ng1-f3 d7-d5 (2) e2-e3 (1)
++ b1-c3 procnr=0 terug=4017 org=[4016;4017] newwindow=[4016;520000]
00:00 14860 743 0 (1) 3 (0,2) 4.085 Nb1-c3 e7-e5 (2) d2-d4 (1)
++ d2-d4 procnr=0 terug=4086 org=[4085;4086] newwindow=[4085;520000]
00:00 17780 889 0 (1) 3 (0,2) 4.108 d2-d4 e7-e6 (2) Nb1-c3 (1)
...
00:03 96262 294562 0 (1) 10 (0,9) 3.769 Nb1-c3 Ng8-f6 (9) d2-d4 (8) d7-d5 (7) Bc1-f4 (6) Bc8-f5 (5) Nc3-b5 (4) e7-e5 (3) Bf4xe5 (2) Bf8-b4 (1) c2-c3 (0)
00:07 100067 771520 0 (1) 11 (0,10) 3.970 Nb1-c3 d7-d5 (10) d2-d4 (9) Bc8-f5 (8) Ng1-f3 (7) Ng8-f6 (6) Bc1-f4 (5) Nf6-g4 (4) Nc3-b5 (3) Ng4xf2 (2) Nb5xc7 (1) Qd8xc7 (0) Bf4xc7 (-1) Nf2xd1 (-1) Ke1xd1 (-1)
00:14 101696 1427812 0 (1) 12 (0,11) 3.745 Nb1-c3 d7-d5 (11) d2-d4 (10) Bc8-f5 (9) Bc1-g5 (8) c7-c6 (7) e2-e3 (6) Qd8-b6 (5) Ra1-b1 (4) Ng8-f6 (3) Bf1-d3 (2) Bf5-g4 (1)
++ d2-d4 procnr=0 terug=3746 org=[3745;3746] newwindow=[3745;520000]
00:23 102612 2420618 0 (1) 12 (0,11) 3.878 d2-d4 d7-d5 (11) Qd1-d3 (10) Ng8-f6 (9) Nb1-c3 (8) c7-c6 (7) Ng1-f3 (6) Bc8-g4 (5) Bc1-f4 (4) e7-e6 (3) Nf3-e5 (2) Bf8-b4 (1)

Position 7
putting engine to search errorlevel=0!
00:00 100 1 0 (1) 1 (0,0) 4.946 Nb1-a3
00:00 200 2 0 (1) 1 (0,0) 5.415 Nb1-c3
00:00 3900 39 0 (1) 2 (0,1) 5.199 Nb1-c3 e7-e5 (1)
00:00 17600 176 0 (1) 3 (0,2) 5.611 Nb1-c3 c7-c6 (2) d2-d4 (1)
00:00 50300 503 0 (1) 4 (0,3) 5.285 Nb1-c3 c7-c6 (3) d2-d4 (2) d7-d5 (1)
00:00 61733 1852 0 (1) 5 (0,4) 5.536 Nb1-c3 c7-c6 (4) d2-d4 (3) d7-d5 (2) e2-e4 (1)
00:00 130500 3915 0 (1) 6 (0,5) 5.380 Nb1-c3 c7-c6 (5) d2-d4 (4) d7-d6 (3) e2-e3 (2) d6-d5 (1)
00:00 198025 7921 0 (1) 7 (0,6) 5.566 Nb1-c3 c7-c6 (6) d2-d4 (5) d7-d5 (4) Ke1-d2 (3) Ke8-d7 (2) e2-e4 (1)
00:00 175337 14027 0 (1) 8 (0,7) 5.411 Nb1-c3 c7-c6 (7) d2-d4 (6) d7-d5 (5) Ke1-d2 (4) Ke8-d7 (3) e2-e4 (2) Kd7-e6 (1)
00:00 229558 39025 0 (1) 9 (0,8) 5.586 Nb1-c3 c7-c6 (8) d2-d4 (7) d7-d5 (6) Ke1-d2 (5) Ke8-d7 (4) f2-f3 (3) Kd7-e6 (2) e2-e4 (1)
00:00 246925 76547 0 (1) 10 (0,9) 5.475 Nb1-c3 c7-c6 (9) d2-d4 (8) d7-d6 (7) Ke1-d2 (6) e7-e5 (5) Kd2-d3 (4) e5xd4 (3) Kd3xd4 (2) c6-c5 (2) Kd4-d5 (1) Ke8-e7 (1)
00:01 264814 283351 0 (1) 11 (0,10) 5.566 Nb1-c3 c7-c5 (10) e2-e4 (9) d7-d6 (8) Ke1-e2 (7) Ke8-d7 (6) f2-f4 (5) e7-e6 (4) Ke2-e3 (3) Kd7-c6 (2) d2-d4 (1) c5xd4(-1) Ke3xd4 (-1)
00:02 273439 678131 0 (1) 12 (0,11) 5.468 Nb1-c3 c7-c6 (11) d2-d4 (10) d7-d6 (9) Ke1-d2 (8) e7-e5 (7) Nc3-e4 (6) Ke8-e7 (5) d4xe5 (4) d6xe5 (3) Ne4-c5 (2) b7-b6 (1)
++ d2-d4 procnr=0 terug=5469 org=[5468;5469] newwindow=[5468;520000]
00:04 269119 1149142 0 (1) 12 (0,11) 5.531 d2-d4 d7-d6 (11) Nb1-d2 (10) Ke8-d7 (9) Nd2-f3 (8) h7-h6 (7) Ke1-d2 (6) c7-c5 (5) d4xc5 (4) d6xc5 (3) Nf3-e5 (2) Kd7-e6 (1) Ne5-d3 (1)
00:06 272342 1661292 0 (1) 13 (0,12) 5.506 d2-d4 d7-d6 (12) Nb1-d2 (11) Ke8-d7 (10) e2-e4 (9) Kd7-e6 (8) Nd2-f3 (7) h7-h6 (6) Ke1-e2 (5) f7-f5 (4) d4-d5 (3) Ke6-f6 (2) e4xf5 (2) Kf6xf5 (1)
++ b1-c3 procnr=0 terug=5507 org=[5506;5507] newwindow=[5506;520000]
00:14 285717 4148622 0 (1) 14 (0,13) 5.508 d2-d4 d7-d6 (13) Ke1-d2 (12) e7-e5 (11) Kd2-d3 (10) Ke8-d7 (9) e2-e4 (8) h7-h6 (7) d4-d5 (6) g7-g6 (5) Nb1-d2 (4) h6-h5 (3) Nd2-c4 (2) f7-f5 (1)

So from 3.8 to 5.5 using latest outputs. Just rising of course in latest pos.

Howard E wrote: I also tested for bishop odds and all the other pieces. With pawns too. Always the same result for Rybka - it "knows" that a piece or pawn up in an ending is more advantageous than during the opening.
Interesting you missed something then.

That a bishop pair is like +0.9 to +1.2 bonus
and that when not having a bishop pair that knight versus bishop
is +0.36 to +0.6

So Rybka just never wants to exchange basically and shuffles forever.

It is not really new. Zappa world champs 2005 already had huge bonuses for example for bishop pair and i wondered why it worked for it yet not for Diep. Nowadays bigger bishop pair bonus (but really not even *close* yet to those big bonuses) works better for diep also. Might be because the rest is doing it as well nowadays.

In fact in 2004 diep's score increased dramatic against crafty when i would LOWER the bishop pair bonus. From 90% score to nearly 100% score.
Getting a near 100% score is *real tough*.

Of course with todays much better tuned crafty that no longer works, lowering the bishop pair. In fact tuning it higher worked better.

So you shouldn't confuse material scoring with other positional factors that dominate the position. Like another big catch is the bonus for a pawn in the center. That's really high in some programs. With a piece up that goes easier of course.

Note that those big bonuses triggering clear decisions with last 7 units of forward pruning on 1.2 pawn (if score >= beta+1.2 then return;) that of course increases search depth also a lot.
Howard E wrote: [d]r1bqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq;
[d]r1bqkb1r/pppppppp/8/8/8/8/PPPPPPPP/RNBQKB1R w KQkq;
[d]r2qkb1r/pppppppp/8/8/8/8/PPPPPPPP/RN1QKB1R w KQkq;
[d]r2qk2r/pppppppp/8/8/8/8/PPPPPPPP/RN1QK2R w KQkq -;
[d]3qk2r/pppppppp/8/8/8/8/PPPPPPPP/1N1QK2R w Kk -;
[d]3qk3/pppppppp/8/8/8/8/PPPPPPPP/1N1QK3 w - -;
[d]4k3/pppppppp/8/8/8/8/PPPPPPPP/1N2K3 w - -;
Top
Howard E
Posts: 261
Joined: Wed Mar 08, 2006 8:49 pm

Re: Evaluating material advantages as position simplifys

Post by Howard E »

Thanks Vincent, very informative. Happy to see that Shredder, Zappa, and your Diep (programs I do not have) show higher scores as pieces are removed. The explanation of the bishop pair is a good example that other factors play a role in overall evaluation of when to simplify.

Was there a time in Diep's early development when it evaluated position 1 and position 7 much the same?
Top

Return to “Computer Chess Club: General Topics”