Compare your engine's performance: Corrections +++

Discussion of chess software programming and technical issues.

Moderator: Ras

User avatar
Ras
Posts: 2703
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: Compare your engine's performance: Corrections +++

Post by Ras »

klx wrote: Fri Aug 20, 2021 10:16 pmDo you guys use a transposition table? If so, how do you ensure correctness?
In the mate finder, I only use the hash tables for positions that are scored with 0, i.e. that don't lead to mate within the specified depth.
Rasmus Althoff
https://www.ct800.net
Chessnut1071
Posts: 313
Joined: Tue Aug 03, 2021 2:41 pm
Full name: Bill Beame

Re: Compare your engine's performance: Corrections +++

Post by Chessnut1071 »

Ras wrote: Sat Aug 21, 2021 8:04 pm
klx wrote: Fri Aug 20, 2021 10:16 pmDo you guys use a transposition table? If so, how do you ensure correctness?
In the mate finder, I only use the hash tables for positions that are scored with 0, i.e. that don't lead to mate within the specified depth.
I use hash tables (Zobirst) and history by ply depth. It's slow but perfectly accurate. Warning, you need a very large hash table for deep searches and it takes a lot of time. Evaluation function uses 3 metrics for each side, history, material consequence and check, with check being overwhelming significant. Looking for ways to increase speed. Some methods are 1000 - 10,000x faster than my approach; however, mine finds every mate. I'm wondering if the faster methods are 100% efficient too.
klx
Posts: 179
Joined: Tue Jun 15, 2021 8:11 pm
Full name: Emanuel Torres

Re: Compare your engine's performance: Corrections +++

Post by klx »

Chessnut1071 wrote: Sat Aug 21, 2021 9:46 pmLooking for ways to increase speed. Some methods are 1000 - 10,000x faster than my approach
No offense, but the speed stated in your first post (25,000 engine calls / second) is garbage. Not sure what you're doing, but you should aim for > 1 million / second.

That only explains about 100x improvement though. If you're seeing 10,000x difference, it sounds like an algorithmic issue (ie: reduced branching factor).

Also see my recent post on how to speed up the TT: http://talkchess.com/forum3/viewtopic.php?f=7&t=78007
[Moderation warning] This signature violated the rule against commercial exhortations.
Pio
Posts: 338
Joined: Sat Feb 25, 2012 10:42 pm
Location: Stockholm

Re: Compare your engine's performance: Corrections +++

Post by Pio »

klx wrote: Mon Aug 23, 2021 10:06 pm
Chessnut1071 wrote: Sat Aug 21, 2021 9:46 pmLooking for ways to increase speed. Some methods are 1000 - 10,000x faster than my approach
No offense, but the speed stated in your first post (25,000 engine calls / second) is garbage. Not sure what you're doing, but you should aim for > 1 million / second.

That only explains about 100x improvement though. If you're seeing 10,000x difference, it sounds like an algorithmic issue (ie: reduced branching factor).

Also see my recent post on how to speed up the TT: http://talkchess.com/forum3/viewtopic.php?f=7&t=78007
25000 nodes per second is not garbage. Just making an engine from scratch with an error free move generator is a big achievement. The speed does not mean much at all. Try beating Stockfish while it plays at 25000 nps.

A high branching factor will actually make the nodes per second go up a little bit since the interior nodes will be fever.
klx
Posts: 179
Joined: Tue Jun 15, 2021 8:11 pm
Full name: Emanuel Torres

Re: Compare your engine's performance: Corrections +++

Post by klx »

Pio wrote: Mon Aug 23, 2021 11:41 pm 25000 nodes per second is not garbage. Just making an engine from scratch with an error free move generator is a big achievement.
Sorry, don't mean to be rude. 25k nodes / second is ~120,000 clock cycles per node. If it were an engine of my creation I would not be proud of that performance. But it's all relative, and your right, just creating a chess engine is of course a big step in itself. Perhaps we can conclude, to answer the OP's question, that time-per-node is one area that can be improved.
Pio wrote: Mon Aug 23, 2021 11:41 pmThe speed does not mean much at all. Try beating Stockfish while it plays at 25000 nps.
The counter argument is, who would win between Stockfish 25k NPS vs Stockfish 70M NPS, all else the same?
[Moderation warning] This signature violated the rule against commercial exhortations.
Chessnut1071
Posts: 313
Joined: Tue Aug 03, 2021 2:41 pm
Full name: Bill Beame

Re: Compare your engine's performance: Corrections +++

Post by Chessnut1071 »

Pio wrote: Mon Aug 23, 2021 11:41 pm
klx wrote: Mon Aug 23, 2021 10:06 pm
Chessnut1071 wrote: Sat Aug 21, 2021 9:46 pmLooking for ways to increase speed. Some methods are 1000 - 10,000x faster than my approach
No offense, but the speed stated in your first post (25,000 engine calls / second) is garbage. Not sure what you're doing, but you should aim for > 1 million / second.

That only explains about 100x improvement though. If you're seeing 10,000x difference, it sounds like an algorithmic issue (ie: reduced branching factor).

Also see my recent post on how to speed up the TT: http://talkchess.com/forum3/viewtopic.php?f=7&t=78007
25000 nodes per second is not garbage. Just making an engine from scratch with an error free move generator is a big achievement. The speed does not mean much at all. Try beating Stockfish while it plays at 25000 nps.

A high branching factor will actually make the nodes per second go up a little bit since the interior nodes will be fever.
Chessnut1071
Posts: 313
Joined: Tue Aug 03, 2021 2:41 pm
Full name: Bill Beame

Re: Compare your engine's performance: Corrections +++

Post by Chessnut1071 »

klx wrote: Mon Aug 23, 2021 10:06 pm
Chessnut1071 wrote: Sat Aug 21, 2021 9:46 pmLooking for ways to increase speed. Some methods are 1000 - 10,000x faster than my approach
No offense, but the speed stated in your first post (25,000 engine calls / second) is garbage. Not sure what you're doing, but you should aim for > 1 million / second.

That only explains about 100x improvement though. If you're seeing 10,000x difference, it sounds like an algorithmic issue (ie: reduced branching factor).

Also see my recent post on how to speed up the TT: http://talkchess.com/forum3/viewtopic.php?f=7&t=78007
Question: If I give your 1 million nps engine 230 chess puzzles can it get all 230 solutions correct? By pruning specific types of moves, I can achieve much faster speeds too; however, it comes at a cost of accuracy. In fact, I can 2x the speed if I accept 92% accuracy. I started out at around 89,000 nps, but, as I included more puzzles, the speed dropped to around 23,500.
klx
Posts: 179
Joined: Tue Jun 15, 2021 8:11 pm
Full name: Emanuel Torres

Re: Compare your engine's performance: Corrections +++

Post by klx »

Chessnut1071 wrote: Tue Aug 24, 2021 3:09 am Question: If I give your 1 million nps engine 230 chess puzzles can it get all 230 solutions correct? By pruning specific types of moves, I can achieve much faster speeds too; however, it comes at a cost of accuracy. In fact, I can 2x the speed if I accept 92% accuracy. I started out at around 89,000 nps, but, as I included more puzzles, the speed dropped to around 23,500.
Again, I did not mean to offend. Garbage was perhaps a strong word. Congrats on solving all those puzzles.

I have not written an engine, but I'd expect at least 1 million NPS single threaded. The total speed to solve a puzzle would come from two factors: time-per-node and number of nodes. I'd expect adding pruning to reduce the total node count, but also reduce the NPS since the pruning checks don't come for free. Have you made your code public? Happy to take a look at it and offer some suggestions. If your engine is alpha-beta based and your seeing 23,500 NPS there is guaranteed to be room for improvement, eg: excessive allocations (this is a common one).
[Moderation warning] This signature violated the rule against commercial exhortations.
amanjpro
Posts: 883
Joined: Sat Mar 13, 2021 1:47 am
Full name: Amanj Sherwany

Re: Compare your engine's performance: Corrections +++

Post by amanjpro »

klx wrote: Tue Aug 24, 2021 3:30 am
Chessnut1071 wrote: Tue Aug 24, 2021 3:09 am Question: If I give your 1 million nps engine 230 chess puzzles can it get all 230 solutions correct? By pruning specific types of moves, I can achieve much faster speeds too; however, it comes at a cost of accuracy. In fact, I can 2x the speed if I accept 92% accuracy. I started out at around 89,000 nps, but, as I included more puzzles, the speed dropped to around 23,500.
Again, I did not mean to offend. Garbage was perhaps a strong word. Congrats on solving all those puzzles.

I have not written an engine, but I'd expect at least 1 million NPS single threaded. The total speed to solve a puzzle would come from two factors: time-per-node and number of nodes. I'd expect adding pruning to reduce the total node count, but also reduce the NPS since the pruning checks don't come for free. Have you made your code public? Happy to take a look at it and offer some suggestions. If your engine is alpha-beta based and your seeing 23,500 NPS there is guaranteed to be room for improvement, eg: excessive allocations (this is a common one).

I think you better write an engine before claiming these strong claims, or discovering your great ideas in the other threads...

My engine is 2730 CCRL and unless on very fast machines it cannot make to 1M NPS

So yeah, speak from experience next time
User avatar
Rebel
Posts: 7388
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Compare your engine's performance: Corrections +++

Post by Rebel »

klx wrote: Tue Aug 24, 2021 3:30 am I have not written an engine
That's pretty obvious from your posts. You might want to tone down a bit.
90% of coding is debugging, the other 10% is writing bugs.