Howard E wrote:Thanks everyone for your reply.
This cut and paste from the chess wiki link ...
Other Approaches
Many different approaches have been tried that do not directly split the search tree. These algorithms have not enjoyed popular success due to the fact that they are not scalable. Typical examples include one processor that evaluates positions fed to it by a searching processor, or a tactical search that confirms or refutes a positional search.
Has anyone tried this, where one or two threads are just devoted to tactical analysis. Then somehow a "master brain algorithm" decides
which move to make? I imagine this would be problematic yet if successfully implemented could have payoffs.
Let me respond onto that.
For most chessengines of today combining this doesn't make sense as basically the evaluation function is so limited that we can already see it as a stripped engine, so combining it with a tactics only engine doesn't make sense at all, as your nps will not speedup much, at most a few percent if not less.
For an engine like Diep that's factor 10-20 slower in nps (if not more) than the fast beancounters, it makes more sense to take a look there as there is a potential speedup that's very huge.
Where to effectively use such fast beancounter? Not to replace ENTIRE search, yet just the last few plies of course. If we're already up a full piece and have just 2 or 3 plies to search what's the odds that we have a lost position there?
Very small huh?
So if we just check with a fast tactical search there, we do not need to do that with the slow diep search huh?
That's about the idea below... ...now more nerdie talk below...
I spent around a month or 6+ to speedup diep in 2004-2005. I had a sponsor for world champs 2005 and the promise of a book from Erdo. yet his book was only for tactical strong engines that positional sucked a tad. It was all tactical tricks, run with passed pawns and pray for the mistake of the opponent.
Diep's nps was rather limited so i tried to combine diep's slow positoinal search with a fast beancounter search.
Now of course not in parallel, but each core in itself simply would do next.
The observation was that 90% of all positions at depthleft 3-5 ply was over 3 pawns away from beta. So 3 pawns above beta.
In itself usually you can prune that, just the few cases that it would give a cutoff you cannot.
What todays software is doing is just throw it away and pray for the best.
Yet my search depth and/or huge evaluation either isn't enough or the evaluation is spending a lot of time you don't want to throw away that cheaply.
So then with a quick tactical search i tried to get a cutoff there.
In itself this plan is not new. In 80s/90s already some engines tried this approach.
As for Diep back then. At home at my k7, at a single cpu of it from 2.1Ghz, the normal Diep search was searching around a 100k nps. The fast beancounter search i wrote which took a lot of months of fulltime work, it was 2.5 million nps though (includes giving checks in qsearch and a limited evaluation). This all in 32 bits obviously.
So it would be a tremendeous speedup for Diep to search like that.
Realize all this ONLY in positions where we are more than 3 pawns above the current bestscore of the opponent (beta). So in 99% of cases it really is useless to search further there.
Normal diep search wastes on average 7 nodes to that, and in total it's more than 50% of all nodes searched by Diep, so the actual speedup would win handsdown a ply and when replacing the entire search there it would win diretly 3 ply searchdepth for Diep.
yet despite the searchdepth win, it didn't work very well.
I didn't keep my plan especially secret back then. Stefan Meyer-Kahlen had a short comment about it: "it's difficult to combine 2 different searches".
Now i still feel with some toying something would be possible, yet fact is that i didn't get it to work simply. Maybe with more time put into it i would get it to work, yet for now Stefan's comments you can write in big capitals.
combining different searches that get produced by different evaluation functions is very difficult simply.
The first problem you run into for example is that Diep picks up a lot of tactics in this manner, whereas a fast beancounter search simply sees less there. So even replacing Diep's search tehre by a fast beancounter, already tactical you're directly a ply or what weaker in many occasions as Diep is spending way more time on selecting which moves to try.
So that factor 20 increase in nps directly some factor 4 or so you lose to fact Diep picks up more within its search thanks to evaluation and trying more moves, already in a TACTICAL manner.
Then next problem is that the 'bestmove' that such fast search finds is not close to what diep thinks the best move. So it also suffers in move ordering.
Everywhere you have problems. Now i still feel it is solvable to some extend, yet the effort you need to do for that is real complicated of combining a normal search with full slow eval with a fast tactical search.
With todays hardware i have here i AM capable of better testing such things. Back then the problem was simply i didn't have enough hardware to test enough games to be sure of anything
The end of it, is that with endless talks i managed to convince Anthony to use Erdo's book and Erdo to give his book to Anthony, and Anthony became world champion then with Zappa with 10.5 out of 11.
3 weeks in advance of world champs 2005 i concluded all my experiments failed for combining the 2 searches and that i had to stop research on it as i also wanted to score a few points.
So improvement of evaluation had totally stopped till then and that proved to be devastating as the few 'changes' i had done for more agressive scoring of the passed pawns total backfired.
Arturo's book, historical more suited to Diep is what i used world champs 2005 wasn't ready in time either also proving a disaster at the worldchamps 2005 for Diep.
Interesting is that Diep managed to win from Fruit pretty easily based upon some chessknowledge, so i'm very convinced that massive chessknowledge is the way to go.
However it has to be said that optimizing it is very complicated as no one else is doing that.
We also see that whatever Stefan MK is doing there is not exactly effective as compared to the elorating one would project Shredder.
Obviously one needs total different algorithms and different approaches for engines with a lot of chessknowledge and past nearly 20 years the only one who really put a lot of time into this is me and doing that on your own is not easy if it doesn't get paid.
Whereas most of the simplistic beancounters just write over whatever the hell someone else is doing and any trick one manages to squeeze out works for the other as well usually.
With more chessknowledge there is also more tricks possible - yet it's fulltime work - don't make any mistakes there. If you have that time it will work superior of course though - yet you do need a clever person who can invent new stuff otherwise it will fail.
Last systematic improvement of evaluation code in diep was start 2006, so we will see how much elo i manage to get it higher within afew months now.
Each few years doing some effort there is in fact not much if you consider the huge amount of code the evaluation is.
