Thats really weird , Mark.
You do ponder ON matches with clearly thread overload.
You "test " engines ...23 games, 43 games, 37 games, 102 games...aso. There is NO system at all in your tests.
3 minutes, 5 minutes, 90 minutes.....
But of course, in YOUR world all others are blind to see "the system" and the "easy" overload of threads - except you.
And now, you start moaning about other tester groups.
This is really mind sick.
For you, every kind of attention seems to be worthy. Event it is just a bad one. I am sure, this guides you all the way in your life.
You could grow so much in here, its a great scource of knowledge. Just your ego will not accept anything here.
C.K.
CCRL Chess Engine Match Standards. How obsolete are they?
Moderators: hgm, Rebel, chrisw
-
- Posts: 782
- Joined: Tue Dec 01, 2009 11:10 am
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: CCRL Chess Engine Match Standards. How obsolete are they?
Even my overloaded CPU is testing better then a smartphone you carry in your pocket.Hugo wrote: ↑Mon Jul 01, 2019 5:01 pm Thats really weird , Mark.
You do ponder ON matches with clearly thread overload.
You "test " engines ...23 games, 43 games, 37 games, 102 games...aso. There is NO system at all in your tests.
3 minutes, 5 minutes, 90 minutes.....
But of course, in YOUR world all others are blind to see "the system" and the "easy" overload of threads - except you.
And now, you start moaning about other tester groups.
This is really mind sick.
For you, every kind of attention seems to be worthy. Event it is just a bad one. I am sure, this guides you all the way in your life.
You could grow so much in here, its a great scource of knowledge. Just your ego will not accept anything here.
C.K.
40,000,0000 nps game average.
How do you feel as a tester about CCRL using a testing standard that is weaker than a smartphone?
And then misleading the public about the real time controls used in the CCRL ratings list.
We all should expect better from CCRL. The premier testing group. Don't you think so....
Last edited by mwyoung on Mon Jul 01, 2019 5:50 pm, edited 1 time in total.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.
-
- Posts: 1357
- Joined: Wed Mar 08, 2006 10:15 pm
- Location: San Francisco, California
-
- Posts: 261
- Joined: Sun Nov 13, 2016 10:37 am
Re: CCRL Chess Engine Match Standards. How obsolete are they?
I find their testing very good. At the same time I consider your testing disgusting, sorry. This is IMHO.mwyoung wrote: ↑Mon Jul 01, 2019 5:43 pmEven my overloaded CPU is testing better then a smartphone you carry in your pocket.Hugo wrote: ↑Mon Jul 01, 2019 5:01 pm Thats really weird , Mark.
You do ponder ON matches with clearly thread overload.
You "test " engines ...23 games, 43 games, 37 games, 102 games...aso. There is NO system at all in your tests.
3 minutes, 5 minutes, 90 minutes.....
But of course, in YOUR world all others are blind to see "the system" and the "easy" overload of threads - except you.
And now, you start moaning about other tester groups.
This is really mind sick.
For you, every kind of attention seems to be worthy. Event it is just a bad one. I am sure, this guides you all the way in your life.
You could grow so much in here, its a great scource of knowledge. Just your ego will not accept anything here.
C.K.
40,000,0000 nps game average.
How do you feel as a tester about CCRL using a testing standard that is weaker than a smartphone?
And then misleading the public about the real time controls used in the CCRL ratings list.
We all should expect better from CCRL. The premier testing group. Don't you think so....
Zevra 2 is my chess engine. Binary, source and description here: https://github.com/sovaz1997/Zevra2
Zevra v2.5 is last version of Zevra: https://github.com/sovaz1997/Zevra2/releases
Zevra v2.5 is last version of Zevra: https://github.com/sovaz1997/Zevra2/releases
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: CCRL Chess Engine Match Standards. How obsolete are they?
Why would we want such a low testing standard from CCRL. Subject experts tell me when you use such a weak standards. Your draw rate decreases making the rating list more subject to rating errors.
What do you think?
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: CCRL Chess Engine Match Standards. How obsolete are they?
No, that is alright.sovaz1997 wrote: ↑Mon Jul 01, 2019 5:53 pmI find their testing very good. At the same time I consider your testing disgusting, sorry. This is IMHO.mwyoung wrote: ↑Mon Jul 01, 2019 5:43 pmEven my overloaded CPU is testing better then a smartphone you carry in your pocket.Hugo wrote: ↑Mon Jul 01, 2019 5:01 pm Thats really weird , Mark.
You do ponder ON matches with clearly thread overload.
You "test " engines ...23 games, 43 games, 37 games, 102 games...aso. There is NO system at all in your tests.
3 minutes, 5 minutes, 90 minutes.....
But of course, in YOUR world all others are blind to see "the system" and the "easy" overload of threads - except you.
And now, you start moaning about other tester groups.
This is really mind sick.
For you, every kind of attention seems to be worthy. Event it is just a bad one. I am sure, this guides you all the way in your life.
You could grow so much in here, its a great scource of knowledge. Just your ego will not accept anything here.
C.K.
40,000,0000 nps game average.
How do you feel as a tester about CCRL using a testing standard that is weaker than a smartphone?
And then misleading the public about the real time controls used in the CCRL ratings list.
We all should expect better from CCRL. The premier testing group. Don't you think so....
If it was not for subject matter experts explaining to me that 40 million nps for the ab engine , 80 Knps for Lc0 game average is not good.
I would not have realized just how bad CCRL testing standard of dumming down their hardware speed to less then a smartphone was hurting their ratings.
I want to thank the subject matter experts for their help.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: CCRL Chess Engine Match Standards. How obsolete are they?
Each of the big three (CCRL, CEGT, FastGm) rating lists (I omit SSDF only because by the time they have a reliable rating for an engine, it is already obsolete!) has to work with the people and hardware that they have. FastGM has the most scientific lists for single CPU only by virtue of large samples, round robin tests, and increment use. The others have to mix results from different testers and hardware, so there is a bit more "noise" in the data, but they do a very good job despite this. Sure, the 40/40 CCRL list should be renamed 40/15, but that's a minor quibble, we all know the truth here. As long as you compare 1 GPU with 1 CPU the GPU they use seems reasonable. The use of Bayeselo (by CCRL only) artificially contracts the rating differences, but they might argue that this makes them more realistic in terms of how they would rate vs humans. The use of repeating time controls is a serious waste of resources, but is done to preserve historical continuity. Personally I would advocate just switching to increment play with the base set to the 40 move time and the increment to half that in seconds, and just combining the data from now on as the time controls will be equivalent for a typical 60 move game. So 40/15 would be 15' + 7.5" inc. This would allow perhaps 50% or so more games per hour at the same quality, significantly diminishing error margins. No need for ponder, that's mostly a waste of resources. But despite these issues, the three lists do a great job overall considering that they are volunteers, and most of us involved with computer chess very much appreciate what they do.
Komodo rules!
-
- Posts: 991
- Joined: Thu Mar 09, 2006 2:11 pm
Re: CCRL Chess Engine Match Standards. How obsolete are they?
Hi Larry,
"we" (CEGT)
use 3'+1" pb=on and 5'+3" also pb=on additionally
to our "repeating time controls lists" to compare
if there are differences between them or if there
are engines which are better/worser in this regard.
Thats no waste of resources in my view.
Best wishes,
G.S.
(CEGT team)
"we" (CEGT)
use 3'+1" pb=on and 5'+3" also pb=on additionally
to our "repeating time controls lists" to compare
if there are differences between them or if there
are engines which are better/worser in this regard.
Thats no waste of resources in my view.
Best wishes,
G.S.
(CEGT team)
-
- Posts: 88
- Joined: Thu Oct 06, 2016 9:17 pm
- Location: India
Re: CCRL Chess Engine Match Standards. How obsolete are they?
To me this seems more of an attack directed at CCRL team than actually getting to the point. From what I read, the issue is the advertised 40/4 | 40/40 which are actually scaled to some CPU that was decided when the website was started. They clearly state, right at the start, approximates for modern CPUs and the benchmarking methodology. I see no bait and switch here at all. The rating list is still indicative of relative strengths of engines to a pretty good accuracy.
If you have such valuable criticism to make, why not start a list of your own instead of making the entire established community conform to your personal interpretation of their list. In fact, make it pay-to-view as well because it's going to be the one true list with exact rating values measured for each new CPU that comes out right? Don't forget to get a few GMs into the pool to better reflect FIDE rating numbers so that we're not fooling anyone about SF's 3400 rating. I'm sure everyone would flock to that
If you have such valuable criticism to make, why not start a list of your own instead of making the entire established community conform to your personal interpretation of their list. In fact, make it pay-to-view as well because it's going to be the one true list with exact rating values measured for each new CPU that comes out right? Don't forget to get a few GMs into the pool to better reflect FIDE rating numbers so that we're not fooling anyone about SF's 3400 rating. I'm sure everyone would flock to that
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: CCRL Chess Engine Match Standards. How obsolete are they?
I was told by subject matter expert that CCRL standards is very poor.mkchan wrote: ↑Mon Jul 01, 2019 9:46 pm To me this seems more of an attack directed at CCRL team than actually getting to the point. From what I read, the issue is the advertised 40/4 | 40/40 which are actually scaled to some CPU that was decided when the website was started. They clearly state, right at the start, approximates for modern CPUs and the benchmarking methodology. I see no bait and switch here at all. The rating list is still indicative of relative strengths of engines to a pretty good accuracy.
If you have such valuable criticism to make, why not start a list of your own instead of making the entire established community conform to your personal interpretation of their list. In fact, make it pay-to-view as well because it's going to be the one true list with exact rating values measured for each new CPU that comes out right? Don't forget to get a few GMs into the pool to better reflect FIDE rating numbers so that we're not fooling anyone about SF's 3400 rating. I'm sure everyone would flock to that
And it is clear reading the posts of the discussion. The practice of CCRL claiming the ratings test as 4/40 and 40/40 needs to be addressed. CCRL can fix this issue today.
So valuable criticism has been made.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.