In addition to all the other advice, you should look at your node counts per ply and compare to other engines. Start with maybe something like GNUChess 5.x. If you are searching a lot more nodes per ply, something is wrong. IMO this is more likely than searching a similar number but just slower, although that is speculation.
Some of the programmers here might be willing to do a quickie code inspection for you (which could be a lot more fruitful than guessing what is going on). Private message me if interested.
--Jon
Easy engine to use for testing
Moderators: hgm, Rebel, chrisw
-
- Posts: 4367
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
-
- Posts: 117
- Joined: Wed Jul 20, 2011 2:54 pm
- Location: Ottawa, Canada
Re: Easy engine to use for testing
LittleBlitzer reports about 151k avg nodes per sec for me vs 550k for PortFish, another c# engine.
I posted the eval here, if this doesn't have any obvious bugs I'll post search.
At one point I modified an old version of crafty (15.17) and took out the aspiration search and all the extensions except for check and my node count was pretty close.jdart wrote:In addition to all the other advice, you should look at your node counts per ply and compare to other engines.
Some of the programmers here might be willing to do a quickie code inspection for you (which could be a lot more fruitful than guessing what is going on).
I posted the eval here, if this doesn't have any obvious bugs I'll post search.
-
- Posts: 4367
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Easy engine to use for testing
Eval code that was posted is only mobility, as far as I can tell.
You don't need much more, but least it should include:
1. Bonus for passed pawns.
2. Bonus for development (minors off the back rank, castling, etc.).
3. Some kind of king safety. Bonus for pawn shelter around the king, penalty for piece attacks near the king.
--Jon
You don't need much more, but least it should include:
1. Bonus for passed pawns.
2. Bonus for development (minors off the back rank, castling, etc.).
3. Some kind of king safety. Bonus for pawn shelter around the king, penalty for piece attacks near the king.
--Jon
-
- Posts: 117
- Joined: Wed Jul 20, 2011 2:54 pm
- Location: Ottawa, Canada
Re: Easy engine to use for testing
Oops, I'm around 421k nodes per sec after I turn off the debugger. Still slow but it's fast enough for me.stevemulligan wrote:LittleBlitzer reports about 151k avg nodes per sec for me vs 550k for PortFish, another c# engine.
Back to the fixed depth, I replaced the eval in PortFish with the simplified eval. This resulted in my being 37 Elo points under PortFish. Is that a small enough margin that I can feel confident I'm as bug free as PortFish?
I think Lucas's claim that I should get > 2000 Elo only applies if the engine is fast enough and under time control. Either that or my implementation was incorrect...
At fixed depth I only see 1800 Elo points.
-
- Posts: 4367
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Easy engine to use for testing
It is still possible your search is buggy but it is also possible that you've just shown that Lucas's claim is not true. My experience with weaker engines is that if they have no (or inadequate) king safety they will get taken apart by engines that do have such a term. All you have to do is "sac, sac and mate!" against such an engine. So maybe you could try adding such an eval term.
Re debugging:
I have a lot of debug and assert statements that are triggered in debug mode. All kinds of sanity checks, double-checking of things that should have been calculated a certain way, boundary conditions, etc. I don't run in this mode very often but it has certainly helped keep the program from not breaking badly.
Also, it's time-consuming but I've also found quite a few bugs by analyzing games my engine has played (at reasonable time controls, not the ultra blitz games you'd use for testing). Sometimes it's not really clear why it lost (with the stronger engines, frequently their position just gets incrementally worse over the game). But if the eval drops suddenly it could be that the program has miscalculated something. I had a bug not too long ago where it would think it was stalemated when it was not - that's a big issue, since it can mistakenly think it has a draw when it is losing.
--Jon
Re debugging:
I have a lot of debug and assert statements that are triggered in debug mode. All kinds of sanity checks, double-checking of things that should have been calculated a certain way, boundary conditions, etc. I don't run in this mode very often but it has certainly helped keep the program from not breaking badly.
Also, it's time-consuming but I've also found quite a few bugs by analyzing games my engine has played (at reasonable time controls, not the ultra blitz games you'd use for testing). Sometimes it's not really clear why it lost (with the stronger engines, frequently their position just gets incrementally worse over the game). But if the eval drops suddenly it could be that the program has miscalculated something. I had a bug not too long ago where it would think it was stalemated when it was not - that's a big issue, since it can mistakenly think it has a draw when it is losing.
--Jon
-
- Posts: 21
- Joined: Tue Sep 07, 2010 6:37 pm
Re: Easy engine to use for testing
thanksAdam Hair wrote:Hi Yves,yl84 wrote:Hi,Adam Hair wrote:Here is a list of engines: http://adamsccpages.blogspot.com/p/also ... t.html?m=1stevemulligan wrote:I've been working on my standard chess engine (c#) for what feels like a very long time and I'm ready to start testing against other engines. What is a good choice of opponents to use against a beginner like me. I'm at the point where I want to try to make my eval a bit smarter.
I started with Warrior but I'm wondering if there are easier engines for me to play against? Against Warrior I get about 10% wins, 40% draws, 50% loss. Maybe that W/L ratio is ok for testing eval changes? I'm not sure...
Also any tips on how to make my eval smarter would be much appreciated.
The engines with more than 300 games are relatively stable. Tscp is a common engine to measure against at this level. MSCP is also a good engine. Perhaps Crafty, set at an appropriate skill level, would be a good choice.
Most authors, if they are not conducting self-testing, will use a pool of 8 to 10 reliable opponents. I can go over my notes and see which of these engines appear to be most stable.
Adam
Adam I'm also interested if you can include my engine for testing in your rating list. My chess engine Milady is relatively weak. Now it is the version 3.04 the newest, available for download at http://milady-chess.blogspot.fr/
Cheers
Yves
I am sorry that I did not see your request to me at Open Chess until today. After I finish my experiment concerning move selection, Elo, and search depth, I will test Milady for the Also-Rans list. It maybe a couple of weeks or more before I do.
Adam
-
- Posts: 71
- Joined: Fri Mar 16, 2012 10:16 am
Re: Easy engine to use for testing
Don't forget the Stockfish/Portfish cuts the tree horribly. Even Portfish beats engines having 3x/4x NPS. A lot depends on what depth that NPS goes to...stevemulligan wrote:Oops, I'm around 421k nodes per sec after I turn off the debugger. Still slow but it's fast enough for me.stevemulligan wrote:LittleBlitzer reports about 151k avg nodes per sec for me vs 550k for PortFish, another c# engine.
Back to the fixed depth, I replaced the eval in PortFish with the simplified eval. This resulted in my being 37 Elo points under PortFish. Is that a small enough margin that I can feel confident I'm as bug free as PortFish?
I think Lucas's claim that I should get > 2000 Elo only applies if the engine is fast enough and under time control. Either that or my implementation was incorrect...
At fixed depth I only see 1800 Elo points.
Balint