ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Programmer code of honor (update)
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Vincent Diepeveen



Joined: 09 Mar 2006
Posts: 1738
Location: The Netherlands
PostPost subject: Re: Programmer code of honor (update)    Posted: Fri Jul 13, 2012 6:00 pm Reply to topic Reply with quote
[quote="lkaufman"][quote="diep"]
lkaufman wrote:
diep wrote:

Larry, from all people in the world, i'm the last to even have time to reverse engineer anyone. First of all i'm one of those who invented a bunch of algorithms himself and several of my tricks boosted the elo of Rybka, therefore also the rybka clones.

And no i didn't like that.

To give one example of what i invented somewhere start 21th century and which i used in Diep for nullmove:

if( static eval >= beta + S ) then R = R+1;

I'm the one inventing that. I never had the machine power to test whether it worked well at bigger search depths. First of all diep never got those depths.

I just put it in Diep and for example world champs 2004 and onwards i played with it.

How did it end up in Rybka?
I don't know. Someone illegally took it over from me obviously.

In end 90s i invented extension, for some who were around back then, tactics was important mid 90s. It solved lots of tactical positions back then for Diep. Realize back then Diep got just a few plies. So did nearly everyone. Crafty got bigger depths end 90s, but didn't see anything tactical (cheapo quiescencesearch).

One extension i used there to big effect, it's very interesting to mention that 1 year after i put it into effect, that suddenly Rebel also had this extension.

I never posted about it.

Oh rybka rybka. How surprisingly. Ordering in root. Whether it delivers elo i don't know, as i never did do it differently since mid 90s, but how i order moves there, that's also how Rybka 3 is doing it. Not sure whether the clones took it over from rybka3 - didn't check. I never posted this anywhere.

Oh back onto Ivanhoe. It's doing up to a ply or 9-10 futility potentially.
Practical most futility will happen last 3-4 ply or so.

As it's exponential bonus to search depth, it will keep reducing of course and win plies.

It's total trivial that doing this futility to 9 deep loses elo.

It's needless to say that loses massive elo. I never understood all those guys doing futility. In Diep i can't use it at all. If you want to do something,
then just do a cheaper nullmove last 3 - 4 plies. Not deeper than that or you will miss tactics. This is on my todo list to build for Diep as well. A big attempt there failed in 2005 when i tried to replace my entire search last few plies with a fast highspeed simple search. It's easy to see for me that this is where ivanhoe messes up versus houdini.

Now of course you search deeper doing that futility yet it backfires.

As for the extension. Houdini sees tactics faster where it involves giving check.

As the evaluation is 100% similar, very obvious to see for chessplayers, and it solves all kind of the same positions like 4-5 ply sooner than ivanhoe, meanwhile ivanhoe searches 2 ply deeper.

It's obviously tactical better.

If you go play your own engine against another version of your own engine and one of the engines is tactical better, it will win always in selfplay.

That's kind of what happens at most rating lists now, as most 'top engines' are total clones.

This will change when one guy has fixed his evaluation function enough and gets some reasonable depth without messing up too much in search.

Ivanhoe just messes up in search further it's identical to Houdini, as evaluation is the same.

It's total trivial to me that biggest difference of Diep versus the clones is in opening. If i manage to fix that, a few plies more than the turtle gets now is enough for world domination - and i'm not interested in running single core contests nor interested in rapid levels at a quadcore from half a decade ago of course.

Fixing all the old junk knowledge to modern well debugged knowledge is NOT easy.

In a given position positionally the thing is doing great, yet in opening, just simply developing quick is strategically nearly always better. That's where the deep searchers pick up massive elo, including KOmodo of course.

Now you shouldn't lose that advantage - yet it's obvious that if you want besides just deep mainline checking more elostrength you also must pick up tactics.

With simple methods you can do that.

Now i didn't check komodo, but if your material values are DIFFERENT from the rybka/fruit clones, so not nearly exactly the same, and if it also has knowledge that's NOT inside ivanhoe and clones, then you'll already see a DIFFERENT behaviour there than Houdini will have versus the other clones that are so much the same in evaluation.

The easiest way to beat someone who is identically evaluating things, is by being tactical stronger.

As soon as someone has a different evaluation function, that tactical stronger matters less.

What still wins search depth meanwhile not getting tactical too much stronger is a cheapo nullmove last 3-4 plies. Cheapest thing you can possibly do there is razoring.

if( !incheck && eval >= beta + 1 pawn ) then return eval;

making things tactical stronger there- really everything has been invented there already.

A simple trick is to check whether you were in check +2 plies direction root.
Then just do normal nullmove with qsearch, provided that nullmove also detects giving checks.

With diep's small search depths, such things lose it too much elo for now.
So i just use normal nullmove there.



Several points:
1. "if( static eval >= beta + S ) then R = R+1;" My recollection is that this appeared in Ippo but not Rybka, but I could be mistaken.
2. I like your statement about seeing more tactics being decisive when eval functions are the same. I agree; this is perhaps the main problem with self-testing.
3. I also agree that reducing the plies of futility from 9 plies in Ivanhoe to some lesser number in Houdini would be an elo gain. I just didn't think it was a major one, as the margins for high-depth futility were pretty large. You seem to be saying that this is the main reason for Houdini's elo gain over Ivanhoe. Perhaps you are right, I don't know. At least I don't have a better theory.
4. Regarding "if( !incheck && eval >= beta + 1 pawn ) then return eval;" I think this is a subset or a special case of ProbCut, so probably the credit should go to the inventors of that algorithm.
5. Regarding check extension, you indicate elsewhere that this explains Houdini's tactical superiority over Ivanhoe. Here I'm a bit confused. The programs of the 90s generally extended all checks by a full ply. Then someone discovered that extending them only half a ply was slightly better, and that became the norm. Rybka and Ippo are intermediate, extending the check half a ply and the reply by another half ply if the majority of the material remains on the board. So are you saying that Houdini returned to extending checks by a full ply in general, as in the 90s programs? If so that would almost surely be an elo-loser, I think.


Probcut was an algorithm invented in 90s and proved to be losing elo in the long run.

Razoring was invented decades before that. In fact the earliest book i have is from around 1979 so i don't know exactly when was the first time it was invented. In some start of 80s book it already gets decribed here.

Yet i don't have much literature from days back then. Maybe Bob remembers that. Should be moved to another thread though here i feel.

If you do futility last 9 plies you will miss mate to give 1 example.
You also will miss passers promoting. You miss everything.

Besides missing the big things, also your pruning decision is based upon a guesstimate. Even razoring (which i am not doing in Diep) is more accurate there as it razors based upon a static evaluation.

It's just one example of idiocy in Ivanhoe, easy to improve.

These are trivial changes done so to speak in 5 minutes. Add some testing and you have houdini.

I bet he signed Ed's honor code Smile

For extensions, dig up some old ICGA journals - even Chrilly wrote something about it.

If you prune megamuch (like razoring and nullmove) then extensions are easier to do. I didn't say you should do braindead extensions. Adding some chess technical code wouldnt hurt there i guess.
Back to top
View user's profile Send private message Send e-mail Visit poster's website MSN Messenger
Display posts from previous:   
Subject Author Date/Time
Programmer code of honor (update) Ed Schroder Fri Jun 29, 2012 8:03 am
      Re: Programmer code of honor (update) Ricardo Barreira Fri Jun 29, 2012 7:09 pm
            Re: Programmer code of honor (update) Vincent Diepeveen Fri Jun 29, 2012 10:55 pm
            Re: Programmer code of honor (update) Ed Schroder Mon Jul 02, 2012 8:19 am
                  Re: Programmer code of honor (update) Dan Honeycutt Mon Jul 02, 2012 3:47 pm
      Re: Programmer code of honor (update) Vincent Diepeveen Fri Jun 29, 2012 10:41 pm
            Re: Programmer code of honor (update) Ed Schroder Mon Jul 02, 2012 8:21 am
      Re: Programmer code of honor (update) Don Dailey Thu Jul 05, 2012 4:22 pm
            Re: Programmer code of honor (update) Ed Schroder Thu Jul 05, 2012 10:02 pm
                  Re: Programmer code of honor (update) Don Dailey Thu Jul 05, 2012 10:14 pm
                        Re: Programmer code of honor (update) Ed Schroder Sat Jul 07, 2012 10:17 am
                              Re: Programmer code of honor (update) Don Dailey Sat Jul 07, 2012 10:41 am
                              Re: Programmer code of honor (update) Richard Vida Sat Jul 07, 2012 11:13 am
                        Re: Programmer code of honor (update) Marco Costalba Sat Jul 07, 2012 10:46 am
                              Re: Programmer code of honor (update) Uri Blass Sat Jul 07, 2012 11:25 am
                              Re: Programmer code of honor (update) Thomas Petzke Sat Jul 07, 2012 12:44 pm
                              Re: Programmer code of honor (update) Robert Hyatt Sat Jul 07, 2012 3:04 pm
                                    Re: Programmer code of honor (update) Uri Blass Sat Jul 07, 2012 3:17 pm
                                          Re: Programmer code of honor (update) Ed Schroder Sat Jul 07, 2012 7:07 pm
                                                Re: Programmer code of honor (update) Uri Blass Sat Jul 07, 2012 7:19 pm
                                                      Re: Programmer code of honor (update) Ed Schroder Sat Jul 07, 2012 8:08 pm
                                                            Re: Programmer code of honor (update) H.G.Muller Sat Jul 07, 2012 8:27 pm
                                                                  Re: Programmer code of honor (update) Ed Schroder Sat Jul 07, 2012 9:55 pm
                                                                        Re: Programmer code of honor (update) H.G.Muller Sun Jul 08, 2012 7:59 am
                                                                              Re: Programmer code of honor (update) Ed Schroder Sun Jul 08, 2012 9:55 am
                                                                                    Re: Programmer code of honor (update) H.G.Muller Mon Jul 09, 2012 9:17 am
                                                                                          Re: Programmer code of honor (update) Ed Schroder Mon Jul 09, 2012 10:40 am
                                                                                          Re: Programmer code of honor (update) H.G.Muller Mon Jul 09, 2012 11:51 am
                                                                                          Re: Programmer code of honor (update) Vincent Diepeveen Fri Jul 13, 2012 2:36 pm
                                                                        Re: Programmer code of honor (update) Vincent Diepeveen Fri Jul 13, 2012 2:30 pm
                                                                  Re: Programmer code of honor (update) Joerg Oster Sun Jul 08, 2012 9:35 am
                                                                        Re: Programmer code of honor (update) H.G.Muller Mon Jul 09, 2012 9:03 am
                                                                              Re: Programmer code of honor (update) Joerg Oster Mon Jul 09, 2012 9:41 am
                                                                              Re: Programmer code of honor (update) Thomas Petzke Mon Jul 09, 2012 6:03 pm
                                                            Re: Programmer code of honor (update) Vincent Diepeveen Fri Jul 13, 2012 2:26 pm
                                                      Re: Programmer code of honor (update) Thomas Petzke Sun Jul 08, 2012 10:06 am
                  Re: Programmer code of honor (update) Larry Kaufman Wed Jul 11, 2012 2:51 pm
                        Re: Programmer code of honor (update) Sune Fischer Wed Jul 11, 2012 3:29 pm
                              Re: Programmer code of honor (update) Larry Kaufman Wed Jul 11, 2012 3:54 pm
                                    Re: Programmer code of honor (update) Sune Fischer Thu Jul 12, 2012 10:40 am
                                          Re: Programmer code of honor (update) Vincent Diepeveen Thu Jul 12, 2012 10:47 am
                                                Re: Programmer code of honor (update) Sune Fischer Thu Jul 12, 2012 11:39 am
                                                      Re: Programmer code of honor (update) Vincent Diepeveen Thu Jul 12, 2012 12:19 pm
                                                            Re: Programmer code of honor (update) Ed Schroder Thu Jul 12, 2012 12:59 pm
                                                                  Re: Programmer code of honor (update) Vincent Diepeveen Thu Jul 12, 2012 8:51 pm
                                                                        Re: Programmer code of honor (update) Ed Schroder Thu Jul 12, 2012 9:27 pm
                                                            Re: Programmer code of honor (update) Rolf Thu Jul 12, 2012 1:55 pm
                        Re: Programmer code of honor (update) Ronald de Man Wed Jul 11, 2012 4:03 pm
                              Re: Programmer code of honor (update) Robert Houdart Wed Jul 11, 2012 4:52 pm
                                    Re: Programmer code of honor (update) Vincent Diepeveen Wed Jul 11, 2012 4:55 pm
                                          Re: Programmer code of honor (update) Robert Houdart Wed Jul 11, 2012 5:02 pm
                                                Re: Programmer code of honor (update) Vincent Diepeveen Wed Jul 11, 2012 5:12 pm
                                                      Re: Programmer code of honor (update) Robert Houdart Wed Jul 11, 2012 5:18 pm
                                                            Re: Programmer code of honor (update) Vincent Diepeveen Wed Jul 11, 2012 5:24 pm
                                                      Re: Programmer code of honor (update) Larry Kaufman Wed Jul 11, 2012 8:27 pm
                                                            Re: Programmer code of honor (update) Robert Houdart Wed Jul 11, 2012 9:35 pm
                                                            Re: Programmer code of honor (update) Vincent Diepeveen Thu Jul 12, 2012 10:06 am
                                                                  Re: Programmer code of honor (update) Lucas Braesch Thu Jul 12, 2012 11:26 am
                                                                        Re: Programmer code of honor (update) Vincent Diepeveen Thu Jul 12, 2012 12:03 pm
                                                                              Re: Programmer code of honor (update) Uri Blass Thu Jul 12, 2012 1:56 pm
                                                                              Re: Programmer code of honor (update) Lucas Braesch Thu Jul 12, 2012 3:24 pm
                                                                                    Re: Programmer code of honor (update) Vincent Diepeveen Thu Jul 12, 2012 9:09 pm
                                                                                          Re: Programmer code of honor (update) Lucas Braesch Fri Jul 13, 2012 3:59 am
                                                                                          Re: Programmer code of honor (update) Thomas Petzke Fri Jul 13, 2012 6:58 am
                                                                  Re: Programmer code of honor (update) Larry Kaufman Fri Jul 13, 2012 4:20 pm
                                                                        Re: Programmer code of honor (update) Vincent Diepeveen Fri Jul 13, 2012 6:00 pm
                                                Re: Programmer code of honor (update) H.G.Muller Wed Jul 11, 2012 5:59 pm
                                    Re: Programmer code of honor (update) Dan Honeycutt Wed Jul 11, 2012 7:58 pm
                                    Re: Programmer code of honor (update) Larry Kaufman Wed Jul 11, 2012 8:33 pm
                                          Re: Programmer code of honor (update) Robert Houdart Wed Jul 11, 2012 9:32 pm
                                          Re: Programmer code of honor (update) Dan Honeycutt Thu Jul 12, 2012 1:03 am
                                                Re: Programmer code of honor (update) Eelco de Groot Thu Jul 12, 2012 1:51 am
                                                      Re: Programmer code of honor (update) Dan Honeycutt Thu Jul 12, 2012 2:41 am
                                                            Re: Programmer code of honor (update) Rolf Thu Jul 12, 2012 1:28 pm
                                          Re: Programmer code of honor (update) Adam Hair Thu Jul 12, 2012 1:38 am
                                    Re: Programmer code of honor (update) Ed Schroder Thu Jul 12, 2012 9:04 am
                                    Re: Programmer code of honor (update) Rolf Thu Jul 12, 2012 1:12 pm
            Re: Programmer code of honor (update) Robert Houdart Thu Jul 05, 2012 11:07 pm
                  Re: Programmer code of honor (update) Don Dailey Fri Jul 06, 2012 12:18 am
                  Re: Programmer code of honor (update) H.G.Muller Sat Jul 07, 2012 10:45 am
                        Re: Programmer code of honor (update) Richard Vida Sat Jul 07, 2012 11:48 am
                              Re: Programmer code of honor (update) H.G.Muller Sat Jul 07, 2012 12:27 pm
                              Re: Programmer code of honor (update) OFF_TOPIC Alcides Schulz Sat Jul 07, 2012 12:39 pm
      Re: Programmer code of honor (update) Vincent Lejeune Mon Jul 09, 2012 10:25 am
            Re: Programmer code of honor (update) Ed Schroder Mon Jul 09, 2012 10:41 am
                  Re: Programmer code of honor (update) Vincent Lejeune Mon Jul 09, 2012 10:51 am
                        Re: Programmer code of honor (update) Ed Schroder Mon Jul 09, 2012 11:10 am
      Re: Programmer code of honor (update) Matthew Hull Fri Jul 13, 2012 5:56 am
            Re: Programmer code of honor (update) Sune Fischer Fri Jul 13, 2012 7:01 am
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads