Pruning and reductions

Discussion of chess software programming and technical issues.

Moderator: Ras

gaard
Posts: 463
Joined: Mon Jun 07, 2010 3:13 am
Location: Holland, MI
Full name: Martin W

Pruning and reductions

Post by gaard »

Are there any pruning or reduction techniques where I can test the effectiveness or potential Elo gain aside from playing many games? I can see the increased depth from LMR right away, but not so much from null move. What kind of metrics should I be looking for in terms of notes per second and time to depth?
User avatar
lithander
Posts: 915
Joined: Sun Dec 27, 2020 2:40 am
Location: Bremen, Germany
Full name: Thomas Jahn

Re: Pruning and reductions

Post by lithander »

I used a few test-sets of positions with a stored best-move in the epd format. Searching them all with a current build to a given depth (let's say 12) gives you two important information: How long does it take (e.g. 1000s) and how many positions are getting solved. (e.g. 75/100)

Now with effective pruning you'd expect the time to get significantly better while the number of solved positions ideally is staying the same or only gets slightly worse. That way you can rule out a few bad ideas early on (number solved position tanks or time doesn't really improve) but in the end only playing many games can confirm that you're indeed making your engine better.
Minimal Chess (simple, open source, C#) - Youtube & Github
Leorik (competitive, in active development, C#) - Github & Lichess
jdart
Posts: 4409
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Pruning and reductions

Post by jdart »

Game play is really the standard and best way to measure relative strength. You can use a SPRT test to determine if a candidate change is statistically better than the base version.

This does generally require either a lot of time or a hardware, but there is no getting around that.

The potential gain from LMR or null move is quite large though, so those should be relatively easy tests to make.