4CPU and 4 different evals.

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

bhlangonijr
Posts: 482
Joined: Thu Oct 16, 2008 4:23 am
Location: Milky Way

Re: 4CPU and 4 different evals.

Post by bhlangonijr »

Uri Blass wrote:
rbarreira wrote:
Uri Blass wrote:I think that it is a bad idea.

chess is not democracy and I suspect that if you give players with similiar strength to vote about a move you can expect playing strength that is weaker than every one of them.
Are you saying that making a weak program sometimes play like Rybka or Stockfish will make it even weaker?
Not exactly(if you start from a weak program I do not expect it to be weaker by making move of rybka or stockfish but
I suspect that if the level of rybka and stockfish is almost the same you are going to get something weaker than both of them because stockfish may make one plan when rybka that is used to produce the next move is not going to follow the plan.
I think the idea is worth testing it. You should not always trust in your intuitions. The Mounty Hall problem is a good example of how our initial intuition fails badly sometimes- It is not always good stick to the plan :).

Regardless of using a transposition table, you can see the chess engine solving each move of the game as a discrete process. That said, if you have a given position to solve, what would you trust more: The analyses of the *best* engine, or consider various analyses made by not the best, but some other good chess engines?

I'd bet I am increasing my chances of doing the right choice by combining many " point of views" instead of trusting in only one. Although this is just yet another guess. :D

Regards,
User avatar
Robert
Posts: 20
Joined: Tue Oct 07, 2008 2:53 am
Location: Brasil

Re: 4CPU and 4 different evals.

Post by Robert »

URI is right.

In Magnus vs The World Match, was used a similar idea that favored Carlsen!
bhlangonijr
Posts: 482
Joined: Thu Oct 16, 2008 4:23 am
Location: Milky Way

Re: 4CPU and 4 different evals.

Post by bhlangonijr »

Robert wrote:URI is right.

In Magnus vs The World Match, was used a similar idea that favored Carlsen!
The G-Star event? :) It doesn't mean URI is right. It's like saying that you need only one game by means of calculate the Elo of a given player.
User avatar
Robert
Posts: 20
Joined: Tue Oct 07, 2008 2:53 am
Location: Brasil

Re: 4CPU and 4 different evals.

Post by Robert »

bhlangonijr wrote:
Robert wrote:URI is right.

In Magnus vs The World Match, was used a similar idea that favored Carlsen!
The G-Star event? :) It doesn't mean URI is right. It's like saying that you need only one game by means of calculate the Elo of a given player.

No I say more.

I say that need no one game to see which strategy favors player A or B.

This is not a matter of Elo.
bhlangonijr
Posts: 482
Joined: Thu Oct 16, 2008 4:23 am
Location: Milky Way

Re: 4CPU and 4 different evals.

Post by bhlangonijr »

Robert wrote: This is not a matter of Elo.
I know it is not. I was just making an analogy to explain that only one game or a couple of games is not enough to prove the point. As for if it is better or not I don't know and as I pointed out it would be nice test the idea.
rjgibert
Posts: 317
Joined: Mon Jun 26, 2006 9:44 am

Re: 4CPU and 4 different evals.

Post by rjgibert »

Robert Pope wrote:
rbarreira wrote:
Uri Blass wrote:I think that it is a bad idea.

chess is not democracy and I suspect that if you give players with similiar strength to vote about a move you can expect playing strength that is weaker than every one of them.
Are you saying that making a weak program sometimes play like Rybka or Stockfish will make it even weaker?
No, if you take a strong program like Rybka and make it sometimes play like a weak program, then it will make it even weaker.

Imagine you have 5 good programs of the same overall strength. But one is especially good in endgames, another in tactics, another in pawn
development, etc. Each time that a program sees an opportunity to leverage its own special skill, it will be outvoted by the others, who don't see the brilliancy. So what you end up with is a bland, weaker composite, with none of the brilliance of any of them.
Same argument, but instead of one stronger at this and another stronger at that, etc. you have one weaker at this and another weaker at that. The new conclusion: Instead of a weaker composite, you get a stronger composite.

Your argument was selectively biased. I selectively biased it the other way. If there is a reason to reject the idea, your reason is not it. I don't like the idea, because too much work is being duplicated. It's far too inefficient.
User avatar
Robert
Posts: 20
Joined: Tue Oct 07, 2008 2:53 am
Location: Brasil

Re: 4CPU and 4 different evals.

Post by Robert »

bhlangonijr wrote: I'd bet I am increasing my chances of doing the right choice by combining many " point of views" instead of trusting in only one. Although this is just yet another guess. :D

Regards,
If this can be done with high efficience (ie avoiding parallel work), i see no problem to be tested as idea.

My guess... :)
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: 4CPU and 4 different evals.

Post by bob »

jhaglund wrote:
Hi,
I am thinking about test like this:
1. Create 4 eval functions
2. Run 4 search any every search will return best move.
cpu 1 - e4
cpu 2 - Nf3
cpu 3 - d4
cpu 4 - e4
so e4 is the best move ...
This idea would work depending on if you replace the word "cpu" with "engine"

I like this approach:

a) Each search will take 1 root move per thread and force that move for that particular search.

b) Then compare each evaluation score from the _end_ of each PV line to order root moves.

c) Step-Forward and force PV_line1 temporarily and do a con-current search

d) Concatenation of the two PV_lines, taking end ply move evaluation score2.

if(last_ply_score1 < last_ply_score2)
score = last_ply_score2 - last_ply_score1;
if(last_ply_score2 < last_ply_score1)
score = last_ply_score1 - last_ply_score2;
if(last_ply_score1 == last_ply_score2)
score = last_ply_score1;
if(last_ply_score2 == last_ply_score1)
score = last_ply_score2;
else
score = MATE - score;

PV[score] = PV_line1[last_ply_score1] + PV_line2[last_ply_score2];

e) Order root moves based on score2, PV[score2].

f) Win.

But, I've already said something like this before...
http://talkchess.com/forum/viewtopic.ph ... 80&t=35066
:)
Which do you think would be more accurate:

(1) search to depth D in the normal way.

(2) starting at the root, do some shallower searches until you get to depth D/2, then starting there, search to something like depth D/2 + N (where N is small).

In (1) if you discover something unusual, you can change your mind all the way back to the root. In (2) your options are way limited because you have 1/2 of the tree that is now static and can not be modified to include more moves, after you realize that the ones you included are no good...

This is forward-pruning at it's most dangerous. It has been tried.

Greenblatt used a very simple form of this, by first doing a normal search to establish the final PV, then going to the end of that PV and doing a very shallow search to refine that PV before you make it "the PV". This doesn't work. Several of us tried it and it leads to gross tactical mistakes.
jhaglund
Posts: 173
Joined: Sun May 11, 2008 7:43 am

Re: 4CPU and 4 different evals.

Post by jhaglund »

Which do you think would be more accurate:

(1) search to depth D in the normal way.

(2) starting at the root, do some shallower searches until you get to depth D/2, then starting there, search to something like depth D/2 + N (where N is small).
(1) search to depth D in the normal way... but play out the pv line . Does it >=match your previous pv line evaluation score? or did it drop significantly?

Why can you not go back to the root if you force the pv line1,
and do another search and see it was a bad move at the same time?

pv_line1+pv_line2,

pv1 = pv_line1;
pv1 = e2e4 e7e5 g1f3 b8c6 f1b5

pv2 = pv_line2;
pv2 = a7a6 b5c6 b7c6 0-0 g8f6

pv = pv1+ pv2;
pv = e2e4 e7e5 g1f3 b8c6 f1b5 a7a6 b5c6 b7c6 0-0 g8f6

Say your pv2 sees that pv1 was a bust. There are a couple ways that can be tried.

Take your pv1, e2e4 e7e5 g1f3 b8c6 f1b5,
and work backards from the last pv_line1 ply, f1b5.

Increment to the next move ordered in your list. Assuming you order them from best score to worse. Next being, f1c4.

pv1 = pv_line1;
pv1 = e2e4 e7e5 g1f3 b8c6 f1c4

pv2 = pv_line2;
pv2 = f8c5 b1c3 g8f6 0-0 0-0

pv = pv1+ pv2;
pv = e2e4 e7e5 g1f3 b8c6 f1c4 f8c5 b1c3 g8f6 0-0 0-0

Still a bust? No. If yes, goto next move in list. If all moves in list fail, goto pv_line1[-ply][inc next move in list].

pv1 = e2e4 e7e5 g1f3,... b8c6 would change to the next move.

Use one thread per root move per pv search is what I would suggest to start with. This way you are not restrictive to one root being searched, and you'll know sooner if there is a move too avoid.

Then it would be another search concurrently done at the same time.
The root would be pv1, but shouldn't be limited to no changes if it fails with pv2. Working backwards only happens when your search says, "Hey! I've looked into the future. Your orginal normal search sucked. Try a new move." Work back toward the root changing the last ply of pv1 if it fails with pv2's evaluation. 6 root moves = 12 threads.
In (1) if you discover something unusual, you can change your mind all the way back to the root. In (2) your options are way limited because you have 1/2 of the tree that is now static and can not be modified to include more moves, after you realize that the ones you included are no good...
This is forward-pruning at it's most dangerous. It has been tried.

Greenblatt used a very simple form of this, by first doing a normal search to establish the final PV, then going to the end of that PV and doing a very shallow search to refine that PV before you make it "the PV". This doesn't work. Several of us tried it and it leads to gross tactical mistakes.
Ordering the moves at the root of pv1 would be by the outcome of pv2 evaluation. SFSB, Step-Forward-Step-Back.
6 ply search instead of 3.
10 ply search instead of 5.
18 ply search instead of 9.
30 ply search instead of 15.
48 ply search instead of 24.
Uri Blass
Posts: 10282
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: 4CPU and 4 different evals.

Post by Uri Blass »

You have a limited time.

The alternative to searching to depth D in the normal way cannot be searching to depth D in the normal way and do something else.