Looking for advices

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
xr_a_y
Posts: 413
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Looking for advices

Post by xr_a_y » Fri Dec 08, 2017 9:45 pm

Dear all,
I've started writing my own chess engine nearly a year ago, just for fun. I really want to thank CPW contributors, forums members, ... for making this science freely accessible today. So, thank you all !

Today my engine has all the classic stuff (at least my own interpretations of them):
  • alpha-beta framework (not negamax)
    • PVS (root and alpha-beta)
      Window
      IID
    TT (mainly for sorting, seems buggy if used for cut-off)
    QTT (idem)
    EvalTT
    LazyEval
    Sort with
    • Piece position evaluation
      MVV-LVA
      Killer
      History
      SEE (not working ...)
      Checks
    QSearch
    • SEE (not working)
      Delta prunning (not working)
    Futility pruning (extended with adaptative margin)
    Razoring (reduction)
    Static null move (reversed futility pruning) : drop to QSearch
    Null move prunning (adaptative)
    Extension
    • Single reply
      Check
      Promotion (and near promotion)
      End game
      Very end game
      PV
    LMR (adaptative)
    Book
    • Small internal
      Big, read from file (too slow !)
    Evaluation
    • material
      position
      bishop pair
      mobility (global and by piece) (too expensive)
      pawn push
      pawn shield
      castling
      rook connected
      rook on 7th rank
      center control (too expensive)
      king troppism (very expensive)
      pawn structure
      open file
      • double
        isolated
        passed
        protected passed
        connected
        king too far (buggy)
    XBOARD support (partial)
The engine is quite slow (around 500knps single thread, and by node I mean all node and qnode). The current bottleneck is the "getthreats" function (that many people called "attack"), especially if I activate more complexe evaluation (like king troppism, center control, ...)

I cleary see that my LMR + other pruning technics parameters are not optimal, as the engine often make mistakes when all of them are activated.

I also remark that my evaluation is quite unstable in the sense that is black see -45 then white see +80, then black -120, then white +20. I often see more than a pawn difference in the evaluation of move n and n+1.

At the end, the engine has a elo rating around 2200 in classic tests suite. And when playing agaist fairy-max, it lose more often than it win.

I thus can work on at least three things :
1°) make the engine faster (I mean sequentially using more bitboard and also using multiple threads)
2°) make my evaluation smarter : I cleary lack some more pawn structure knowledge for exemple
3°) try to find better parameters for my pruning

I really need some advices here. I really hope the engine can 2500 by the end of the year :-)

Best regards

AlvaroBegue
Posts: 913
Joined: Tue Mar 09, 2010 2:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: Looking for advices

Post by AlvaroBegue » Fri Dec 08, 2017 10:06 pm

I would start by simplifying the engine dramatically and working on making sure everything you add doesn't have bugs and adds performance. You should be able to beat Fairy-Max with half as many features as you listed.

Test that each of the features you implemented actually adds Elo, by playing quick games between versions with and without the feature. If it's unclear, take the feature out. You can always try it later, but having a bunch of half-working things in the program only makes progress harder.

2500 by the end of the year is ambitious, though...

Dann Corbit
Posts: 9416
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Looking for advices

Post by Dann Corbit » Fri Dec 08, 2017 10:40 pm

Make sure it is correct before making it faster.
Then, when you want to make it faster, profile first.
After you profile, try to improve the algorithms that are slow.

If you post the code, I imagine people can critique it for you.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

lauriet
Posts: 162
Joined: Sun Nov 03, 2013 8:32 am
Contact:

Re: Looking for advices

Post by lauriet » Sat Dec 09, 2017 12:08 am

I can only agree with the comments so far.
You need to start with a simple implementation and progressively adding each feature making sure you understand how it is working and it is doing what you think it is......bug free.
From your post, it is obviously not bug free. If you can't figure out the bugs then you are not understanding your code as well as you should. This is not a criticism of you, just advice.
So "incremental implementation" is the way to go.

Laurie

Sven
Posts: 3758
Joined: Thu May 15, 2008 7:57 pm
Location: Berlin, Germany
Full name: Sven Schüle
Contact:

Re: Looking for advices

Post by Sven » Sat Dec 09, 2017 12:20 am

Hi Vivien,

welcome to TalkChess! It seems you've done a great amount of programming work so far, congrats! Now according to your description there are really a lot of things going wrong. That's quite normal at this stage :-)

BUT: As soon as you have realized what is going wrong it would be "not normal" to continue in the same way as you did before. You should become aware that there is a better way to write a strong chess engine. The preferred way is not to have 50 features with 25 bugs (or so) but to have few features without bugs, then add the next one and remain bug-free, then again the next one, and so on.

Therefore I seriously suggest that you disable at least the following features from your list before doing anything else:

TT+QTT
SEE
Delta pruning
all extensions except check extension
opening book
LazyEval
*all* evaluation features except material and "position" (assuming this is what others often call "piece-square table")
and maybe also some of the search features (or are you really sure that null move, IID, LMR, Futility pruning, Razoring are working perfectly as intended?)

If your move generator is "proven" to be correct (based on "perft" tests) then establish a first rating estimate by playing 1000 very fast games against FairyMax. Then I would enable your TT and debug it carefully until it works for both TT cutoff and move ordering, this is crucial. Adding a working TT should give a substantial improvement, test it by playing - for instance - 1000 very fast games against the non-TT version.

Continue like that, adding only one feature at a time and doing a lot of tests and debugging until you fixed all bugs. I always found it easier to add and test search features with a minimalistic evaluation because even small, subtle evaluation bugs in complex code can have a huge effect and lead to being unable to get any improvement from adding some search features. In my view a correct basic engine with a decent search is a good starting point, a better evaluation can be added later.

Regarding Elo ratings, try to be realistic. Making less than 50% vs. FairyMax does not sound like a 2200 rating (and calculating a rating only based on solving some tactical positions may be quite misleading), more like clearly below 2000. If you follow most of the advice given here your engine will become stronger soon.

Playing many very fast test games also requires to have a bug-free time control implementation. Use a fixed set of opening positions, and use a tool like cutechess-cli for playing the games. Building a simple but working test environment is crucial and will actually decide about your success. At some point 1000 games will not be sufficient any longer due to the high error bars compared to the small improvements you are going to make. So be prepared to play even much more than these 1000 games.

I wish you good luck for your project!

Henk
Posts: 5508
Joined: Mon May 27, 2013 8:31 am

Re: Looking for advices

Post by Henk » Sat Dec 09, 2017 12:02 pm

Huh. No question about how to compute backpropagation step in a deep convolutional network ? No Monte Carlo Simulation ? Just going on in old fashioned way.

Or was Alpha zero just some kind of Aprils fool joke. Then they fooled me.

User avatar
xr_a_y
Posts: 413
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Looking for advices

Post by xr_a_y » Sat Dec 09, 2017 3:02 pm

Thank you all for the help. I've just started a tournament between different configurations to track down the potential issues. I'll post the code on github as soon as i'm not to ashamed of it ;-)

It seems there is something wrong in my pvs when applied at none root node.

Best regards

User avatar
xr_a_y
Posts: 413
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Looking for advices

Post by xr_a_y » Sun Mar 25, 2018 1:44 pm

I'm back !

after a disappointing Winter and a bottom place in March tournament, my engine "Weini" is now able to defeat fairymax quite well : 60-30-22 in a 12sec/40moves tournament (this is around +100elo).

But it takes LMR, SMP (4 threads), pondering and some bug fix of course as you have all advised me last year...

There is still two issues I don't understand well :
*why LMR at root is not good (at all: -75elo)
*why using TT at root is not good (at all: -60elo)

Another thing I don't get well (and it makes my engine slow) is the cost of "IsInCheck" function during threats detection. I know many engines pre-compute pinned pieces once and for all for a position but I found it's even slower in mine.

Still, thank you all for your advices, I now test everything with cutechess tournament.

Best regards

Daniel Anulliero
Posts: 663
Joined: Fri Jan 04, 2013 3:55 pm
Location: Nice

Re: Looking for advices

Post by Daniel Anulliero » Mon Mar 26, 2018 4:34 am

Henk wrote:Huh. No question about how to compute backpropagation step in a deep convolutional network ? No Monte Carlo Simulation ? Just going on in old fashioned way.

Or was Alpha zero just some kind of Aprils fool joke. Then they fooled me.
It is far better to Hear some EXPERTS advices and write an old fashion engine for the begenning :wink:
We all saw Weini played at hgm's tournament , it is young but sure it can kill Skipper
:wink:
So vI viens , good work anyway :)

Henk
Posts: 5508
Joined: Mon May 27, 2013 8:31 am

Re: Looking for advices

Post by Henk » Mon Mar 26, 2018 2:26 pm

Play a game against your own engine and against LCZero. Which one did you like best ?

Post Reply