What's the command line to extend the hash size of Crafty to 256 mb in Arena? It's using the default 9 mb. I only have 32 bits though and little slower hardware relative to what Bob has, so the results may not appear close to 85's anyway.swami wrote:Or Bob has an ultra-superior hardware at his disposal. Let me run to see how Crafty does on my Core2Quad...Dann Corbit wrote:Crafty seems on par with Rybka for this test set.
It seems that Crafty has excellent knight understanding.
STS (v3.0) - Knight Outposts/Centralization/Repositioning
Moderators: hgm, Rebel, chrisw
-
- Posts: 6640
- Joined: Thu Mar 09, 2006 4:21 am
Re: STS (v3.0) - Knight Outposts/Centralization/Repositionin
-
- Posts: 718
- Joined: Fri Mar 20, 2009 8:59 pm
Re: STS (v3.0) - Knight Outposts/Centralization/Repositionin
Moneypenny 10 seconds:
Moneypenny 2 minutes:
Code: Select all
Number of tests: 100
Max score: 1000
Engines score: 602
Number with full score: 45
Found in: 5 sec = 15, 10 sec = 30,
Time used for tests (wrong answer=10s): 811s
Code: Select all
Number of tests: 100
Max score: 1000
Engines score: 641
Number with full score: 54
Found in: 1 min = 15, 2 min = 39,
Time used for tests (wrong answer=120s): 9685s
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: STS (v3.0) - Knight Outposts/Centralization/Repositionin
I ran on both the 8-core box I have been using for 3 years or so as well as on my core2 duo laptop. Not much difference. The 8-core is 2.33ghz, my laptop is 2.0ghz.swami wrote:Or Bob has an ultra-superior hardware at his disposal. Let me run to see how Crafty does on my Core2Quad...Dann Corbit wrote:Crafty seems on par with Rybka for this test set.
It seems that Crafty has excellent knight understanding.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: STS (v3.0) - Knight Outposts/Centralization/Repositionin
You can edit your crafty.rc file and add hash=384M or 192M (256M is not possible in Crafty).swami wrote:What's the command line to extend the hash size of Crafty to 256 mb in Arena? It's using the default 9 mb. I only have 32 bits though and little slower hardware relative to what Bob has, so the results may not appear close to 85's anyway.swami wrote:Or Bob has an ultra-superior hardware at his disposal. Let me run to see how Crafty does on my Core2Quad...Dann Corbit wrote:Crafty seems on par with Rybka for this test set.
It seems that Crafty has excellent knight understanding.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: STS (v3.0) - Knight Outposts/Centralization/Repositionin
dual socket 4-core intel, 2.33ghz, the same machine I have been using for about 3 years now...swami wrote:Thanks for the test run, Bob. That's very impressive result, may I know what hardware was used?bob wrote:I ran a quick test with the following details:swami wrote:Chapter 3 Of the Strategic Test Suite: Knight Outposts, Centralization and Repositioning is now available for download.
* Consists of carefully selected 100 questionairres on Knight OutPosts/Centralization/Repositioning
* All of the questions in this test suite are selected by me and thoroughly verified by Dann Corbitt. Each question has been analysed by top engines for hours.
* All the answers carry similar traits:I initially selected about 190 problems and sent them to Dann, of which 100 has passed criteria. Dann made the final selection and done all that analysis with the help of the engines.
- (1) A Knight move to take control of the center or posit itself in the center. (Centralization)
(2) A Knight move to step back and reposition itself to lead the different (or better) path. (Repositioning)
(3) A Knight move to take control of the opponent's weak squares (Outposts)
Recommended time: Anywhere between 10 seconds - 2 minutes for each test.
Download it here: STS 3.0
http://computerchessblogger.googlepages.com/sts
Feel free to report the results from your engine/ favourite engine, suggestions are welcome.
Special Thanks to Pedro Castro, Allard Siemelink, Zach Wegner for the testing and correcting mistakes throughout the process. And Thanks to those who reported bugs and tested out the V 1.0 of the test suite called "Undermining", and v2.0 called "Open Files and Diagonals".
And Extra Special Thanks to Ulysses for creating the beautiful logo!
Test Suite Released time: 3rd of June, 2009
Swaminathan and Dann Corbit.
10 seconds per move.
I told Crafty that if it produced the same (correct) move for 5 consecutive iterations, to consider this one done and move on to the next position. That speeds things up quite a bit although I can run without it if it matters.
Here's the results:Crafty missed #13 (chose Nd5), #17 (liked Ncxe4), #57 (likes Qg6), #93 (likes Nd5)Code: Select all
total positions searched.......... 100 number right...................... 96 number wrong...................... 4 percentage right.................. 96 percentage wrong.................. 4 total nodes searched.............. 1430919140 average search depth.............. 8.1 nodes per second.................. 15638460 total time........................ 1:31
Ran it again at 30 secs/position, but had to keep correct move for 6 iterations before stopping. Got 97 out of 100. Picked up #93...
I ran it on my 2.0ghz laptop and will post those numbers later when I get home..
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: STS (v3.0) - laptop data
Here's the results from my core2-duo 2.0ghz laptop, 30 seconds per move, stopping when it has the correct move for six consecutive iterations, or else after 30 seconds have elapsed.
Code: Select all
test results summary:
total positions searched.......... 100
number right...................... 95
number wrong...................... 5
percentage right.................. 95
percentage wrong.................. 5
total nodes searched.............. 1544007086
average search depth.............. 9.3
nodes per second.................. 3847992
total time........................ 6:41
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: STS (v3.0) - complete data (3 tests)
Since I have seen some odd results Here are the three sets of positions run on my 8-core box. Order is undermine, then file-diag, then knights. 30 secs/position, stop if correct move is chosen on 6 consecutive iterations:
based on the search time needed, the last batch is by far the easiest...
Code: Select all
total positions searched.......... 100
number right...................... 85
number wrong...................... 15
percentage right.................. 85
percentage wrong.................. 15
total nodes searched.............. 16161816467
average search depth.............. 13.1
nodes per second.................. 16536365
total time........................ 16:17
total positions searched.......... 100
number right...................... 89
number wrong...................... 11
percentage right.................. 89
percentage wrong.................. 11
total nodes searched.............. 17665398896
average search depth.............. 13.2
nodes per second.................. 17172546
total time........................ 17:08
total positions searched.......... 100
number right...................... 96
number wrong...................... 4
percentage right.................. 96
percentage wrong.................. 4
total nodes searched.............. 3482397961
average search depth.............. 9.1
nodes per second.................. 16628774
total time........................ 3:29
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: STS (v3.0) - Knight Outposts/Centralization/Repositionin
BTW after running on my laptop with just 2 cores at 2ghz, you ought to be able to beat the 95 I see on this machine, at least...bob wrote:You can edit your crafty.rc file and add hash=384M or 192M (256M is not possible in Crafty).swami wrote:What's the command line to extend the hash size of Crafty to 256 mb in Arena? It's using the default 9 mb. I only have 32 bits though and little slower hardware relative to what Bob has, so the results may not appear close to 85's anyway.swami wrote:Or Bob has an ultra-superior hardware at his disposal. Let me run to see how Crafty does on my Core2Quad...Dann Corbit wrote:Crafty seems on par with Rybka for this test set.
It seems that Crafty has excellent knight understanding.
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: STS (v3.0) - complete data (3 tests)
I've seen other fairly strong programs have equal problems with this set that they have had with the others. I think crafty is just really good with knight moves for whatever reason.bob wrote:Since I have seen some odd results Here are the three sets of positions run on my 8-core box. Order is undermine, then file-diag, then knights. 30 secs/position, stop if correct move is chosen on 6 consecutive iterations:based on the search time needed, the last batch is by far the easiest...Code: Select all
total positions searched.......... 100 number right...................... 85 number wrong...................... 15 percentage right.................. 85 percentage wrong.................. 15 total nodes searched.............. 16161816467 average search depth.............. 13.1 nodes per second.................. 16536365 total time........................ 16:17 total positions searched.......... 100 number right...................... 89 number wrong...................... 11 percentage right.................. 89 percentage wrong.................. 11 total nodes searched.............. 17665398896 average search depth.............. 13.2 nodes per second.................. 17172546 total time........................ 17:08 total positions searched.......... 100 number right...................... 96 number wrong...................... 4 percentage right.................. 96 percentage wrong.................. 4 total nodes searched.............. 3482397961 average search depth.............. 9.1 nodes per second.................. 16628774 total time........................ 3:29
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: STS (v3.0) - complete data (3 tests)
I am not sure why. It's knight scoring is about as simple as it can get...Dann Corbit wrote:I've seen other fairly strong programs have equal problems with this set that they have had with the others. I think crafty is just really good with knight moves for whatever reason.bob wrote:Since I have seen some odd results Here are the three sets of positions run on my 8-core box. Order is undermine, then file-diag, then knights. 30 secs/position, stop if correct move is chosen on 6 consecutive iterations:based on the search time needed, the last batch is by far the easiest...Code: Select all
total positions searched.......... 100 number right...................... 85 number wrong...................... 15 percentage right.................. 85 percentage wrong.................. 15 total nodes searched.............. 16161816467 average search depth.............. 13.1 nodes per second.................. 16536365 total time........................ 16:17 total positions searched.......... 100 number right...................... 89 number wrong...................... 11 percentage right.................. 89 percentage wrong.................. 11 total nodes searched.............. 17665398896 average search depth.............. 13.2 nodes per second.................. 17172546 total time........................ 17:08 total positions searched.......... 100 number right...................... 96 number wrong...................... 4 percentage right.................. 96 percentage wrong.................. 4 total nodes searched.............. 3482397961 average search depth.............. 9.1 nodes per second.................. 16628774 total time........................ 3:29