Singular Extensions

[rvida]

Currently in Critter I am doing something in-between (5) and (6). I am pretty sure this singular-move thing can _significantly_ boost Crafty's strength when done right no matter which approach you choose. Be prepared to revise your LMR and "close-to-leaves" pruning. Of course it depends on your own implementation, but it can very easily blow up the search tree. I have struggled with it almost 2 months to get it right. Well, "almost" right... I still occasionally suffer from tree explosions (like 112 plies at iteration 20).
My first implementation was worthy only about 7-10 elo points. That was unmodified search only with singular-move extensions at PV and CUT nodes.
The real benefit shows itself only after you push your usual LMR to the limits. For me it ended up in the range of 35-40 elo points.

Richard

[mcostalba]

BTW if idea (6) doesn't work for you it means is an implementation bug

Just like the null-move verification search doesn't work for me? And when I comment it out of SF I get a 2-3 elo improvement as well?

I am glad you take it with sense of humor...but don't be worried Bob, it is not difficult to implement, you can do it (well perhaps not at first try, you already failed at first try years ago, but I think now you'll be more lucky).

[liuzy]

I admit that Crafty is a good engine and stockfish is much better.

[Mangar]

PLEASE let this stay a singular extension thread and not a engine x is better than engine y and test condition thread.

Please

Greetings Volker

[bob]

BTW if idea (6) doesn't work for you it means is an implementation bug

Just like the null-move verification search doesn't work for me? And when I comment it out of SF I get a 2-3 elo improvement as well?

I am glad you take it with sense of humor...but don't be worried Bob, it is not difficult to implement, you can do it (well perhaps not at first try, you already failed at first try years ago, but I think now you'll be more lucky).

I'm hoping the issue from 90's was initial search depth. We were happy to see 7-8 plies in blitz back then. SE dropped that significantly (EBF was way higher back then as well).

I am getting close to testing the TT approach first, but as I said, Crafty has changed a _ton_ since 1996-1997. Taking what I did then and making it work today has been a bit of a challenge, the current search is highly optimized to avoid doing anything unnecessary or duplicative. I've found it quite easy to break things inserting even this relatively small amount of code.

More later.

[diep]

Any other comments, corrections, additions or suggestions???

Yes, idea (6) is not correctly laid-out, in particular this part is to be removed:

Code: Select all

If the score appears to be "reasonably better" than the current search bound

I suggest to better document yourself reading the sources (instead of being told about them) before to start your tests.

Not sure what you are talking about. This is code that I have from 10-15 years ago when several of us were experimenting with it. I am not quoting how anybody is doing this today, other than to mention that ip*, robo* used the _idea_ that you guys copied. I didn't give a single idea above that I do not already have code for (unfortunately most of it is going to need significant modification since crafty has changed a ton since the 1997 time frame when much of this was written and tested... When I played with this, one limit I tried was to check the TT bound. Didn't seem very useful to do the SE test on a position that returned a bound somewhat lower than current window...

BTW if idea (6) doesn't work for you it means is an implementation bug

Just like the null-move verification search doesn't work for me? And when I comment it out of SF I get a 2-3 elo improvement as well?

Correction: it was around 1999-2000 that GCP extensively experimented with singular extensions in Crafty. Did you get any feedback from him at the time and/or source code changes?

I'm not aware of how succesful his attempts were.

Seems he also had dug up how Hsu implemented it...

Vincent

No feedback that I can recall. Only detailed conversations during the SE extensions stuff was between Bruce, myself and Dave Kittinger.

Oh i recall now that i have the year wrong. It was more like 2001 or 2002.
I had big discussios with GCP on it. I remember crafty at the time lost too many plies with it to break even with it. Diep's using SE, some sort of Bruce implementation, however i still want to get rid of 'em as they deteriorate branching factor. I find that principle not ok.

Like objectively after e4 h5 Qxh5 then Rxh5 is a singular extensions.

It gives a cutoff and it is the only move and extending singular captures gives really a big boost in tactical strength. Yet the idea of extending Rxh5 here doesn't appear to be objectively correct to me.

Some versions it worked, then for some years they were turned off, latest setting is turned on - if i remember well.

First working implementation of SE diep had in 1994 in first diep version. I had invented SE myself actually, but not invented a reduction of search depth yet, so they were quite pricey. Well who cares if you just get 4 ply anyway. Which reduction would you want to have if your total search depth after 3 minutes is 5 or 6 ply?

I can remember the joy when they announced a mate in 6 against Bionic in my first tournament - which Bionic didn't see very quickly of course

Vincent

What was bionic? Only one I remember was Bionic Impakt, which was a crafty clone. Didn't happen until 1996 or so however...

Bruce had good luck with SE (the cheapo version). It never seemed to pan out for me, but it may have simply been bad luck that I never hit on a decent tuning. I plan on addressing that version of SE once I finish testing the IP*-like version...

That was indeed Bionic Impakt from our Belgium friend Hans Secelle. At that time a selfwritten program, written in some sort of compiled basic language and it got quite some nps in fact, considering the language and basic compilers at the time. It systematically during the game outsearched Diep by 1 ply. Something like Diep 5 ply versus Bionic 6.

Getting outsearched 2 ply those days was a sure death.

That year i was so happy to receive a new fast computer, a 486 dx 2, in order to figure out this tournament that everyone ran at pentium 90Mhz machines which were exactly factor 3 times faster than the 486 dx 2 i had.

I later found out when i got a 100Mhz P5.

I'm guessing Bionic ran at a P5-90Mhz, but would need to read tournament report to figure that out. Maybe Hans remembers.

It was october 1994 and at Wallstreet the dotcom balloon was still expanding.

For most software, the P5 was the ultimate processor for which the software had been designed. Back in 1994 no commercial programmer was talking to me, neither to other 'amateurs'; the first big change there was 1996 and especially after world champs 1997, in my case.

Vincent

[bob]

So far, testing on the TT-extension has not been productive. It is easy to get a break-even version, but so far none have been better than the non-SE version. I've tried a bunch of different singular margins, from 10 to 100. I've tried several "reduced depths" from things like depth/2 to depth-N where N is a constant, or a variable. I have tried extending only on "LOWER" tt best moves, and I have tried "!UPPER" which would include LOWER + EXACT TT best moves. I've experimented with the in check issues, since my testing showed that a single-legal-move extension was a small Elo loss, and a normal singular test will extend since with only one legal move, it is, by definition, "singular". I have tried this both ways (call it singular, or not). Still poking around.

Does anyone have any particular position (or positions) that SE works well on? Preferably at reasonably shallow depths so I can dump the tree and see what is happening. SE slightly helps tactically. At very fast time limits, normal crafty can get 298 of 300 at WAC (using one minute total search time for all moves combined). SE improves that to 299. But that is the only benefit I have found, so far. But as I mentioned, this is only for TT-singular. There are other variants ready to test once I exhaust this. And no, I have not touched anything else. If this allows better LMR aggressiveness, good, but if it doesn't help some by itself, I don't believe it will help overall.

Still testing...

[Ralph Stoesser]

So far, testing on the TT-extension has not been productive. It is easy to get a break-even version, but so far none have been better than the non-SE version. I've tried a bunch of different singular margins, from 10 to 100. I've tried several "reduced depths" from things like depth/2 to depth-N where N is a constant, or a variable. I have tried extending only on "LOWER" tt best moves, and I have tried "!UPPER" which would include LOWER + EXACT TT best moves. I've experimented with the in check issues, since my testing showed that a single-legal-move extension was a small Elo loss, and a normal singular test will extend since with only one legal move, it is, by definition, "singular". I have tried this both ways (call it singular, or not). Still poking around.

Does anyone have any particular position (or positions) that SE works well on? Preferably at reasonably shallow depths so I can dump the tree and see what is happening. SE slightly helps tactically. At very fast time limits, normal crafty can get 298 of 300 at WAC (using one minute total search time for all moves combined). SE improves that to 299. But that is the only benefit I have found, so far. But as I mentioned, this is only for TT-singular. There are other variants ready to test once I exhaust this. And no, I have not touched anything else. If this allows better LMR aggressiveness, good, but if it doesn't help some by itself, I don't believe it will help overall.

Still testing...

At what depths have you tried it?
It's probably no a good idea to use SE too close to the leafs.

[bob]

So far, testing on the TT-extension has not been productive. It is easy to get a break-even version, but so far none have been better than the non-SE version. I've tried a bunch of different singular margins, from 10 to 100. I've tried several "reduced depths" from things like depth/2 to depth-N where N is a constant, or a variable. I have tried extending only on "LOWER" tt best moves, and I have tried "!UPPER" which would include LOWER + EXACT TT best moves. I've experimented with the in check issues, since my testing showed that a single-legal-move extension was a small Elo loss, and a normal singular test will extend since with only one legal move, it is, by definition, "singular". I have tried this both ways (call it singular, or not). Still poking around.

Does anyone have any particular position (or positions) that SE works well on? Preferably at reasonably shallow depths so I can dump the tree and see what is happening. SE slightly helps tactically. At very fast time limits, normal crafty can get 298 of 300 at WAC (using one minute total search time for all moves combined). SE improves that to 299. But that is the only benefit I have found, so far. But as I mentioned, this is only for TT-singular. There are other variants ready to test once I exhaust this. And no, I have not touched anything else. If this allows better LMR aggressiveness, good, but if it doesn't help some by itself, I don't believe it will help overall.

Still testing...

At what depths have you tried it?
It's probably no a good idea to use SE too close to the leafs.

I have tried leaving SE out of last 4 plies, which is where I left off when I tested this years ago. But since that was only a guess on a significantly different program version, I am now running a set of tests on 2, 3, 4, 5 and 6, just to see if there is some point that makes a difference.

[Daniel Shawul]

Maybe you would like to try larger depths with lmr and other pruning? At depth 6 I had too much slow down..but that was for scheme (5).
Also there is question of testing at the appropriate tc to get the effect for larger depths.