Easy engine to use for testing

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

jdart
Posts: 4366
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Easy engine to use for testing

Post by jdart »

Yes, never underestimate the power of bug fixing. You would be very surprised how buggy an engine can be and still play legal, and even most of the time reasonable, moves. Be sure you are doing alpha-beta with null window and re-search, null move, and hash table probing completely correctly (or if lacking any of this, implement it).

--Jon
User avatar
lucasart
Posts: 3232
Joined: Mon May 31, 2010 1:29 pm
Full name: lucasart

Re: Easy engine to use for testing

Post by lucasart »

jdart wrote:Yes, never underestimate the power of bug fixing. You would be very surprised how buggy an engine can be and still play legal, and even most of the time reasonable, moves. Be sure you are doing alpha-beta with null window and re-search, null move, and hash table probing completely correctly (or if lacking any of this, implement it).

--Jon
elo-wise fixing bugs was the biggest contributor by a VERY LONG WAY !!

much more so than any eval or search "improvement". it's the one and only thing to worry about before you reach at least 2300 elo, I would say.

I think the one guy who tought this to the community is certainly Fabien Letouzey. When Fruit came out, people couldn't understand how such a simplistic engine could be so strong.

And the genious of Fabien was to write code in a very defensive way with a clear and simple design, so that bugs virtually cannot exist and go unnoticed.

The other genious of Fabien was in the evaluation, and is probably best summarized by Tord Romstad in this article:
http://www.talkchess.com/forum/viewtopi ... 33&t=15504

A must read before attempting to write an evaluation, I think :)
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Easy engine to use for testing

Post by Don »

lucasart wrote:
jdart wrote:Yes, never underestimate the power of bug fixing. You would be very surprised how buggy an engine can be and still play legal, and even most of the time reasonable, moves. Be sure you are doing alpha-beta with null window and re-search, null move, and hash table probing completely correctly (or if lacking any of this, implement it).

--Jon
elo-wise fixing bugs was the biggest contributor by a VERY LONG WAY !!

much more so than any eval or search "improvement". it's the one and only thing to worry about before you reach at least 2300 elo, I would say.

I think the one guy who tought this to the community is certainly Fabien Letouzey. When Fruit came out, people couldn't understand how such a simplistic engine could be so strong.

And the genious of Fabien was to write code in a very defensive way with a clear and simple design, so that bugs virtually cannot exist and go unnoticed.
I think it's much more than just a simple design and evaluation. There were a number of innovations - but part of his genius may have been that he made it look like it was just a simple thing. People who are good at things make it look easy.



The other genious of Fabien was in the evaluation, and is probably best summarized by Tord Romstad in this article:
http://www.talkchess.com/forum/viewtopi ... 33&t=15504

A must read before attempting to write an evaluation, I think :)
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
pkappler
Posts: 38
Joined: Thu Mar 09, 2006 2:19 am

Re: Easy engine to use for testing

Post by pkappler »

lucasart wrote:
jdart wrote:Yes, never underestimate the power of bug fixing. You would be very surprised how buggy an engine can be and still play legal, and even most of the time reasonable, moves. Be sure you are doing alpha-beta with null window and re-search, null move, and hash table probing completely correctly (or if lacking any of this, implement it).

--Jon
elo-wise fixing bugs was the biggest contributor by a VERY LONG WAY !!

much more so than any eval or search "improvement". it's the one and only thing to worry about before you reach at least 2300 elo, I would say.

I think the one guy who tought this to the community is certainly Fabien Letouzey. When Fruit came out, people couldn't understand how such a simplistic engine could be so strong.

And the genious of Fabien was to write code in a very defensive way with a clear and simple design, so that bugs virtually cannot exist and go unnoticed.

The other genious of Fabien was in the evaluation, and is probably best summarized by Tord Romstad in this article:
http://www.talkchess.com/forum/viewtopi ... 33&t=15504

A must read before attempting to write an evaluation, I think :)
Just wanted to say thanks for linking to that post! I hadn't seen it before, and it's quite good.

-Peter
User avatar
stevemulligan
Posts: 117
Joined: Wed Jul 20, 2011 2:54 pm
Location: Ottawa, Canada

Re: Easy engine to use for testing

Post by stevemulligan »

lucasart wrote: Even with a fairly simple but bugfree search, your engine should be a lot stronger than that, without eval (only material and basic parametric piece on square tables).
Hang on a sec. I have hash table, qsearch, null move, rotated bb's for move gen. The only thing I'm missing is aspiration search which I didn't add because it looked very tricky to debug and from what I understood, it didn't add a lot of Elo.

Are you saying that an eval with mobility like you describe in this thread and PST that you describe in this thread should be able to get > 2000 Elo?? If so then I have to start over :(
User avatar
Dan Honeycutt
Posts: 5258
Joined: Mon Feb 27, 2006 4:31 pm
Location: Atlanta, Georgia

Re: Easy engine to use for testing

Post by Dan Honeycutt »

stevemulligan wrote:
lucasart wrote: Even with a fairly simple but bugfree search, your engine should be a lot stronger than that, without eval (only material and basic parametric piece on square tables).
Hang on a sec. I have hash table, qsearch, null move, rotated bb's for move gen. The only thing I'm missing is aspiration search which I didn't add because it looked very tricky to debug and from what I understood, it didn't add a lot of Elo.

Are you saying that an eval with mobility like you describe in this thread and PST that you describe in this thread should be able to get > 2000 Elo?? If so then I have to start over :(
Hi Steve,

You can expect to start over but don't do that until you have a list of "I should have done this and that and now it is a lot of trouble to change." For weeding out bugs, while it is tedious, I find single stepping to be a great help. Set up a simple position where there is an obvious good move such as a hanging pawn to be captured and an even better move such as a nasty knight fork. Then follow your engine as it searches. It should find the pawn capture right away but as depth goes up a ply or two it should change to the knight move.

For opponents, I have two sub 2000 engines - Simon and my just released Java engine, Cupcake. Cupcake also has an option to limit the strength so you can pick a setting that gives a good match now and increase it as your engine improves. Simon and Cupcake are both available at Jim's site:

http://jimablett.net63.net/

Good luck
Dan H.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Easy engine to use for testing

Post by Don »

stevemulligan wrote:
lucasart wrote: Even with a fairly simple but bugfree search, your engine should be a lot stronger than that, without eval (only material and basic parametric piece on square tables).
Hang on a sec. I have hash table, qsearch, null move, rotated bb's for move gen. The only thing I'm missing is aspiration search which I didn't add because it looked very tricky to debug and from what I understood, it didn't add a lot of Elo.
My advice is that you have to test everything very tediously as you go. Use perft to test the move generator. I have a debug routine built into my program which throws up xboard with a target position in it. When appropriate I will throw up xboard to see if what is happening is what I think is happening. For example if you are working on king safety and counting how many checks are possible for a given color you can do tests which (usually at random positions) show you a board (which you could just print on the screen, but I prefer a real board) and you can manually check to see if the results agree with the computed result. I usually do the test for each color and often have several conditions to check odd corner cases and such.

The search is very easy to get wrong and could be buggy and you might never know. In the old days when programs were much simpler, we had problem sets that had deterministic solution times. You could count exactly how many ply it should take to solve this problem or that. You can do this too to see if you are way out of line with other programs but it's difficult to nail down the exact depth it should take to solve any particular problem because of things like LMR and aggressive forward pruning. But you can STILL do what the old program did by having simple switches that allow you to turn on or off just about anything. You should be able to switch LMR on and off for debugging as well as even hash tables, null move pruning, and any sort of forward pruning. If you do that a set of a few straightforward tactical positions should have predictable solve depths for debugging the search. It's not difficult to calculate the exact required depth to solve a problem once you get the hang of it if you understand the position. You don't need incredibly deep positions, just relatively simple tactics for this.



Are you saying that an eval with mobility like you describe in this thread and PST that you describe in this thread should be able to get > 2000 Elo?? If so then I have to start over :(
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
User avatar
stevemulligan
Posts: 117
Joined: Wed Jul 20, 2011 2:54 pm
Location: Ottawa, Canada

Re: Easy engine to use for testing

Post by stevemulligan »

stevemulligan wrote:and PST that you describe in ....
Wrong link, the PST thread was talking about was this one
User avatar
lucasart
Posts: 3232
Joined: Mon May 31, 2010 1:29 pm
Full name: lucasart

Re: Easy engine to use for testing

Post by lucasart »

stevemulligan wrote:
lucasart wrote: Even with a fairly simple but bugfree search, your engine should be a lot stronger than that, without eval (only material and basic parametric piece on square tables).
Hang on a sec. I have hash table, qsearch, null move, rotated bb's for move gen. The only thing I'm missing is aspiration search which I didn't add because it looked very tricky to debug and from what I understood, it didn't add a lot of Elo.

Are you saying that an eval with mobility like you describe in this thread and PST that you describe in this thread should be able to get > 2000 Elo?? If so then I have to start over :(
Yes
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Easy engine to use for testing

Post by Don »

stevemulligan wrote:
lucasart wrote: Even with a fairly simple but bugfree search, your engine should be a lot stronger than that, without eval (only material and basic parametric piece on square tables).
Hang on a sec. I have hash table, qsearch, null move, rotated bb's for move gen. The only thing I'm missing is aspiration search which I didn't add because it looked very tricky to debug and from what I understood, it didn't add a lot of Elo.

Are you saying that an eval with mobility like you describe in this thread and PST that you describe in this thread should be able to get > 2000 Elo?? If so then I have to start over :(
You don't have to start over, you just have to debug. There is probably some simple bugs.

I suspect the hash table or the search, a simple >= when it should be > or vice versa in comparisons against alpha or beta could break the search.

Play some fixed depth games between a version with hash table turned off. The score should be
close, otherwise there is a bug in the hash table implementation.

Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.