LS-ratinglist (news & comments)

Discussion of computer chess matches and engine tournaments.

Moderators: Harvey Williamson, Dann Corbit, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
pohl4711
Posts: 1413
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: LS-ratinglist (news & comments)

Post by pohl4711 » Sun Oct 13, 2013 11:20 am

lkaufman wrote: I know you do the best you can with your hardware. I hope you'll be able to upgrade your hardware before too long; maybe on an overclocked new i7 your time limit might be enough to minimize the Houdini bias.
Larry
We will see. But overclocking is (in my opinion) no good idea for a PC, which is running 24/7...
And if I a take a look on the benchmarks of the i7-3930k (not overclocked), this system is around 2.1 times faster than one of my test-notebooks. But it uses 6 cores instead of 4 (my notebooks), so per core this i7-3930k is only 1.5 times faster than my machines. So Houdini would calculate around 3 MN/s instead of 2 MN/s and Komodo 2.2 MN/s instead of 1.4 MN/s (in the middlegame, singlecore-mode).
This difference is not big enough (in my opinion) to justify buying new hardware and restart the LS-ratinglist on that new hardware. So at the moment, I believe, I should wait for a new CPU-generation with more "power per core", before upgrading my hardware.

And dont forget, that the idea of the LS-ratinglist is, to see, how strong the engines play with small resources (time and hardware)...and that my budget for computerchess is very small (like my budget in general)...

Best - Stefan

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: LS-ratinglist (news & comments)

Post by Don » Sun Oct 13, 2013 11:33 am

For the most part the time controls we apply to computer games are a reflection of our own sensibilities. So we consider any time control that is comfortable for humans as reasonable for computers. Although we can play faster and slower than this, the range of what is comfortable in a single sitting is something like 3 minutes + increment to 40 moves in 2 1/2 hours. The extreme ends of this range is even too extreme for many.

This is pretty arbitrary however because computers could play game in 1 second pretty comfortably (at least most) and they could be set to think for days at a time without groaning with discomfort. And as computers continue to get faster, a "bullet" time control today is like a tournament time control yesterday. In fact a bullet chess Houdini would crush a 24h per move program of the 70's.

So it still surprises me that there is so much difference between one program and another with respect to time control. Assuming there continues to be major advances in computer hardware and we get a few more doubling's of CPU speed eventually will there be a Houdini-like program of tomorrow that plays clearly better at "bullet" than at tournament time controls?

To me this all proves that we are (unintentionally) not building our programs correctly. In fact what we are doing is optimizing them for today's hardware.

If you could take Stockfish back in time to the 70's it would probably not run on the hardware that is available but if we made the appropriate modifications, we could make it run. My guess is that it would be far better than anything that was available then, but would be grossly non-optimal for the day. It wasn't tuned to run well at 1000 nodes per second. And the program we have today would be far from optimal for a machine that was 1000 times faster.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: LS-ratinglist (news & comments)

Post by Don » Sun Oct 13, 2013 11:43 am

pohl4711 wrote:
lkaufman wrote: I know you do the best you can with your hardware. I hope you'll be able to upgrade your hardware before too long; maybe on an overclocked new i7 your time limit might be enough to minimize the Houdini bias.
Larry
We will see. But overclocking is (in my opinion) no good idea for a PC, which is running 24/7...
And if I a take a look on the benchmarks of the i7-3930k (not overclocked), this system is around 2.1 times faster than one of my test-notebooks. But it uses 6 cores instead of 4 (my notebooks), so per core this i7-3930k is only 1.5 times faster than my machines. So Houdini would calculate around 3 MN/s instead of 2 MN/s and Komodo 2.2 MN/s instead of 1.4 MN/s (in the middlegame, singlecore-mode).
This difference is not big enough (in my opinion) to justify buying new hardware and restart the LS-ratinglist on that new hardware. So at the moment, I believe, I should wait for a new CPU-generation with more "power per core", before upgrading my hardware.

And dont forget, that the idea of the LS-ratinglist is, to see, how strong the engines play with small resources (time and hardware)...and that my budget for computerchess is very small (like my budget in general)...

Best - Stefan
Is overclocking that bad if you have really good cooling and you don't push it too hard?
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

pohl4711
Posts: 1413
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: LS-ratinglist (news & comments)

Post by pohl4711 » Sun Oct 13, 2013 11:43 am

Don wrote: To me this all proves that we are (unintentionally) not building our programs correctly. In fact what we are doing is optimizing them for today's hardware.
Agree.
I would never do testing with complete outdated hardware because of this. And before my hardware reaches that point, I will buy new hardware (of course). But at the moment my hardware is still good enough and much faster hardware (like TCEC uses) is too expensive for me.
So I believe, that the best solution (for me) is, to wait for the next Intel-CPU-generation.
In the last 2 decades, I bought a new PC, when it was around 3 times faster than my old one (and do not cost more than 1000€ (=1300$))

Best - Stefan

pohl4711
Posts: 1413
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: LS-ratinglist (news & comments)

Post by pohl4711 » Sun Oct 13, 2013 11:51 am

Don wrote:
Is overclocking that bad if you have really good cooling and you don't push it too hard?
I am not sure.
But because the price of electricity is very high here in Germany, I use notebooks (at the moment), because of their low power consumption. So I cant do any overclocking right now.

Stefan

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: LS-ratinglist (news & comments)

Post by Don » Sun Oct 13, 2013 11:54 am

pohl4711 wrote:
Don wrote: To me this all proves that we are (unintentionally) not building our programs correctly. In fact what we are doing is optimizing them for today's hardware.
Agree.
I would never do testing with complete outdated hardware because of this. And before my hardware reaches that point, I will buy new hardware (of course). But at the moment my hardware is still good enough and much faster hardware (like TCEC uses) is too expensive for me.
So I believe, that the best solution (for me) is, to wait for the next Intel-CPU-generation.
In the last 2 decades, I bought a new PC, when it was around 3 times faster than my old one (and do not cost more than 1000€ (=1300$))

Best - Stefan
Cutting edge comes at a premium. If you want the most bang for the buck the best strategy for the highest amortized performance over time is to upgrade often and get something good but not something that is on the "high side" of the point of diminishing returns. You will notice that you can get something pretty good fairly cheap, but to get something just a little bit better costs a LOT more. You can usually see where the point of diminishing returns is.

I learned this the hard way. In the 80's I would buy some cutting edge expensive computer thinking it would last me. A year or two later, a friend would go to K-mart or some budget outlet and by some inexpensive PC for his family, and it would already outperform what I had. And it was a fraction of the cost of the machine I had purchased a couple of years earlier.

It's still true, don't spend too much money but upgrade often and on average you will be closer to the cutting edge than you would if you went all out to get cutting edge.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: LS-ratinglist (news & comments)

Post by Don » Sun Oct 13, 2013 12:00 pm

pohl4711 wrote:
Don wrote:
Is overclocking that bad if you have really good cooling and you don't push it too hard?
I am not sure.
But because the price of electricity is very high here in Germany, I use notebooks (at the moment), because of their low power consumption. So I cant do any overclocking right now.

Stefan
The cost of fuel in the US is something people complain about all the time, but I have to assume they have never been to Europe, they would probably stop complaining. I've been in Europe many times and was staggered at the cost.

But electricity is something most of us take for granted. It's something we almost never give a second to. A lot of people leave their lights on all day and night and it's common for some families to leave the television on all day, whether they are watching it or not. Electricity is not free, but it's probably ridiculously cheap compared to Europe.

However we are careful. We turn off lights and conserve because it still makes a difference. But I have several computers and they are on 27/7 running tests - but we still don't flinch too much at the electric bill.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 8:19 pm
Location: Oslo, Norway

Re: LS-ratinglist (news & comments)

Post by Tord Romstad » Sun Oct 13, 2013 6:27 pm

I think the electricity prices vary widely within Europe, too. My electricity bills have never bothered me. I guess electricity is cheaper here than in most of Europe (we have lots of water power and a low population), even though Norway is the most expensive country in Europe in pretty much all other respects.
Don wrote:But I have several computers and they are on 27/7 running tests - but we still don't flinch too much at the electric bill.
As I said, the price of electricity doesn't bother me, but our 24 hour days most certainly do. Are you telling me you have 27 hour days in America? That's what I always wanted! This settles it; I'm moving to America as soon as I can.

User avatar
Graham Banks
Posts: 34652
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: LS-ratinglist (news & comments)

Post by Graham Banks » Sun Oct 13, 2013 6:36 pm

Tord Romstad wrote:....Are you telling me you have 27 hour days in America? That's what I always wanted! This settles it; I'm moving to America as soon as I can.
:lol:
gbanksnz at gmail.com

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: LS-ratinglist (news & comments)

Post by Don » Sun Oct 13, 2013 7:19 pm

Tord Romstad wrote:I think the electricity prices vary widely within Europe, too. My electricity bills have never bothered me. I guess electricity is cheaper here than in most of Europe (we have lots of water power and a low population), even though Norway is the most expensive country in Europe in pretty much all other respects.
Don wrote:But I have several computers and they are on 27/7 running tests - but we still don't flinch too much at the electric bill.
As I said, the price of electricity doesn't bother me, but our 24 hour days most certainly do. Are you telling me you have 27 hour days in America? That's what I always wanted! This settles it; I'm moving to America as soon as I can.
Yes, here in the USA we have 27 hours days. You can get a lot more done but it can make you really cranky too, which is probably why our politicians hate each other so much.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

Post Reply