Hardware vs Software

[Zach Wegner]

The source for Crafty 8.11 is available on http://wbec-ridderkerk.nl/html/download.htm

[bob]

A more harmonious (and hard to find) tuning combination of commonly known elements is no more a software improvement than a more pleasing (and hard to find) combination of dials on a Moog is a synthesizer improvement.

Well, I'm fairly certain that the tuning is not a hardware improvement. So you are saying there are parts of a chess program that are neither hardware nor software? Figuring out how to maximize the combination of technique A and technique B sounds to me like a definite software improvement.

This is turning into a semantic argument. So let me start the discussion over with this caveat:

"software improvement. (noun). A new idea introduced into a chess program that was unknown previously, and which improves the playing strength in a significant way."

Tuning evaluation numbers is _not_ software improvement. The software is not being changed at all, just constants are getting twiddled with. Software improvements could include any of the following ideas:

trans/ref tables
killer moves
history ordering
null-move pruning
LMR pruning
PVS modification to alpha/beta
singular extensions (not one-reply extension).
endgame databases
Learning (book/position/etc)
parallel search
forward pruning ideas (this is different from reductions and null-move)

If you want to get into evaluation ideas:
passed pawn races
outside passed pawns
pawn majorities
weak pawns
king safety which is a very complex idea
MG/EG interpolation scoring/ scaling
lazy evaluation
incremental evaluation
piece coordination


Etc...

Tweaking evaluation numbers is _not_ in the category we are talking about. Tweaking evaluation numbers is an elo here, an elo there. Many of the above type changes could be 40-50 elo here, 40-50 there.

[Uri Blass]

The big pgn is good enough and I can probably find easily games when the weak version of Crafty did not lose.

I would like to look at the games
I have 2 questions

1)Did you use 1,3,5,9 for values of pieces or something different?
2)Am I correct to assume that you did not use piece square tables in the evaluation?


Edit:I think that possible problems with the fast time control may be that some programs can lose on time in 10 seconds+.1 second per move so the question is also if there are no losses on time.

[Pablo Vazquez]

Cool, thx!

[bob]

The big pgn is good enough and I can probably find easily games when the weak version of Crafty did not lose.

I would like to look at the games
I have 2 questions

1)Did you use 1,3,5,9 for values of pieces or something different?
2)Am I correct to assume that you did not use piece square tables in the evaluation?


Edit:I think that possible problems with the fast time control may be that some programs can lose on time in 10 seconds+.1 second per move so the question is also if there are no losses on time.

1, 3, 5 9 were not used. The standard crafty piece values were used, which (based on memory) is 100, 325, 325, 500, 970 which I guess you could represent as 1.0, 3.25, 5.0 and 9.7 in your terminology. No piece square tables or any other scoring, just "return (wtm) ? Material : -Material;" at the top of Evaluate().

[bob]

The source for Crafty 8.11 is available on http://wbec-ridderkerk.nl/html/download.htm

Thanks, I will grab that and add it to my collection. I think that version might have had the "threat extension" stuff which was interesting...

[Michael Sherwin]

It may be interesting to do the same comparison for Glaurung to see how much rating Glaurung earns from LMR and null move.

if you have the time, go for it. I don't have the time to study the source to see what needs to be commented out.

I don't have time to run any tests, but if either of you (or somebody else) wants, I can make a special version where LMR, null move and the tapered super-qsearch (which I suspect is also worth a significant number of Elo points) can easily be switched on and off. Alternatively, if it makes testing easier, I can add some compile-time switches which makes it easy to disable the features you want.

By the way, it's amusing to see that LMR is now generally accepted as effective. Back when I started advocating it, the technique was largely abandoned since many years, and those few programmers I managed to convince to give it a try mostly reported that it didn't work for them.

Tord

This is just a guess, but:

Most tactics are very shallow (within say, 12 ply) and on slower hardware LMR may hurt, because too many tactics are missed. On faster hardware the vast majority of shallow tactics is seen anyway despite LMR and the extra plies gives sight of rare deep tactics as well as better 'strategical planning'.

[Uri Blass]

It may be interesting to do the same comparison for Glaurung to see how much rating Glaurung earns from LMR and null move.

if you have the time, go for it. I don't have the time to study the source to see what needs to be commented out.

I don't have time to run any tests, but if either of you (or somebody else) wants, I can make a special version where LMR, null move and the tapered super-qsearch (which I suspect is also worth a significant number of Elo points) can easily be switched on and off. Alternatively, if it makes testing easier, I can add some compile-time switches which makes it easy to disable the features you want.

By the way, it's amusing to see that LMR is now generally accepted as effective. Back when I started advocating it, the technique was largely abandoned since many years, and those few programmers I managed to convince to give it a try mostly reported that it didn't work for them.

Tord

This is just a guess, but:

Most tactics are very shallow (within say, 12 ply) and on slower hardware LMR may hurt, because too many tactics are missed. On faster hardware the vast majority of shallow tactics is seen anyway despite LMR and the extra plies gives sight of rare deep tactics as well as better 'strategical planning'.

I do not agree
You assume that LMR cause the program to be slower in finding simple tactics and I think that in most cases programs can see shallow tactics faster with LMR.

Uri

[Michael Sherwin]

It may be interesting to do the same comparison for Glaurung to see how much rating Glaurung earns from LMR and null move.

if you have the time, go for it. I don't have the time to study the source to see what needs to be commented out.

I don't have time to run any tests, but if either of you (or somebody else) wants, I can make a special version where LMR, null move and the tapered super-qsearch (which I suspect is also worth a significant number of Elo points) can easily be switched on and off. Alternatively, if it makes testing easier, I can add some compile-time switches which makes it easy to disable the features you want.

By the way, it's amusing to see that LMR is now generally accepted as effective. Back when I started advocating it, the technique was largely abandoned since many years, and those few programmers I managed to convince to give it a try mostly reported that it didn't work for them.

Tord

This is just a guess, but:

Most tactics are very shallow (within say, 12 ply) and on slower hardware LMR may hurt, because too many tactics are missed. On faster hardware the vast majority of shallow tactics is seen anyway despite LMR and the extra plies gives sight of rare deep tactics as well as better 'strategical planning'.

I do not agree
You assume that LMR cause the program to be slower in finding simple tactics and I think that in most cases programs can see shallow tactics faster with LMR.

Uri

Not in the lines that have been reduced by multible LMR reductions. Then add a further reduction with null move and many shallow tactics can be missed. But, not if the engine is searching deeper because of faster hardware.

[CRoberson]

I ran this overnight. I simply made Evaluate() return the material score only. It was almost exactly a 400 point drop in Elo from the version with the most recent evaluation.

Code: Select all

Crafty-22.9R01     2650    5    5 31128   51%  2644   21% 
Crafty-22.9R02     2261    5    6 31128    9%  2644    7% 

Ok, here is another test. No book. Combine that with the full Crafty
and the raw material Crafty.