Summer may be a problem! Since they all would be in a spare room with two windows I was just going to keep the door shut and use lots of window fans. In winter time the cost to run them should be negligible!
Lots of testers use celerons to test with and it seems that their results are just as valid. Tony Thomas test on a celeron and also uses almost exclusively 1'1" time limits and his results are very close to the truth.
There are engines that do not do well at blitz on slow machines, however, since I am only testing for a relative difference between beta versions and not for accuracy of rating there should not be a problem.
And bare minimum nodes as I originally described them should even be less expensive than what has been suggested so far. The cheaper the better. So, a simple celeron (sempro?) box for US $250 would be fantastic!
After having said all that, it would be a shame to not run 64 bit test! But once again everything is relative and if the 32 bit version does better then the 64 bit version should also be better.
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
You might try connecting with a nearby school or university. They often upgrade and will auction off older machines, many or all of the same model at the same time.
sje wrote:You might try connecting with a nearby school or university. They often upgrade and will auction off older machines, many or all of the same model at the same time.
Good idea!
Thanks!
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
But now that Microsoft started its own clusters scheme, it will be just be a DIY project for you or anyone interested. These 2 links would be a good start.
I like to test at 60/60 time controls. 60 minutes on the clock, 60 seconds per move added. That turns into 4-6 hours per game. or 6-8 games per day. 30 days isn't enough games to determine if version X is better than version X-1 at that speed. It takes more than a hundred games or two with two opponents that are fairly close in rating.
Fasinating stuff, however, it is either more than I need or does not do what I want and I am not sure which.
This is what I am asking for:
Ten boxes without monitors or keyboards in one room connected to a main computer in another room so that if I click on a boxs icon that is in the other room it can be controled from the main computer just enough so that the box can be turned on or off and a beta can be transfered to that box and a gauntlet started or stoped. No parallel computing needed!
Mike
If you are on a sidewalk and the covid goes beep beep
Just step aside or you might have a bit of heat
Covid covid runs through the town all day
Can the people ever change their ways
Sherwin the covid's after you
Sherwin if it catches you you're through
bob wrote:I like to test at 60/60 time controls. 60 minutes on the clock, 60 seconds per move added. That turns into 4-6 hours per game. or 6-8 games per day. 30 days isn't enough games to determine if version X is better than version X-1 at that speed. It takes more than a hundred games or two with two opponents that are fairly close in rating.
Hi,
At a rate 6-8 games per day you'll get to 210 games in 30 days, so 30 days should be enough to determine wich oponent is better.
bob wrote:I like to test at 60/60 time controls. 60 minutes on the clock, 60 seconds per move added. That turns into 4-6 hours per game. or 6-8 games per day. 30 days isn't enough games to determine if version X is better than version X-1 at that speed. It takes more than a hundred games or two with two opponents that are fairly close in rating.
Hi,
At a rate 6-8 games per day you'll get to 210 games in 30 days, so 30 days should be enough to determine wich oponent is better.
If the two versions are close, 200 games is nowhere near enough to say which version is better. I'm working on something for the JICGA right now that shows some interesting results based on this very topic. I can produce some 200+ game matches where version A wins one, version B wins the other. Both versions playing 80 games against each of 3 opponents, using 40 starting positions, each played once with each color. 240 games total. Not always enough. In fact, rarely ever enough unless something is broken to make one version clearly worse.
bob wrote:I like to test at 60/60 time controls. 60 minutes on the clock, 60 seconds per move added. That turns into 4-6 hours per game. or 6-8 games per day. 30 days isn't enough games to determine if version X is better than version X-1 at that speed. It takes more than a hundred games or two with two opponents that are fairly close in rating.
Hi,
At a rate 6-8 games per day you'll get to 210 games in 30 days, so 30 days should be enough to determine wich oponent is better.
If the two versions are close, 200 games is nowhere near enough to say which version is better. I'm working on something for the JICGA right now that shows some interesting results based on this very topic. I can produce some 200+ game matches where version A wins one, version B wins the other. Both versions playing 80 games against each of 3 opponents, using 40 starting positions, each played once with each color. 240 games total. Not always enough. In fact, rarely ever enough unless something is broken to make one version clearly worse.
Hi,
From your previous post I understood that you tough something between 100 and 200 games was enough. I probably misunderstood what you meant.
Out of curiosity, how many games (and wich kind of results) do you think are needed to assert that version N+1 is an improvment over version N?
