LittleBlitzer question

[Werewolf]

Can LB run multiple matches in parallel? For example, with a 6 core machine could you run 3 matches simultaneously with ponder on?

Cute chess sounds powerful, but how do you learn how to use it? Sounds like a nightmare to get going with.

[Laskos]

Can LB run multiple matches in parallel? For example, with a 6 core machine could you run 3 matches simultaneously with ponder on?

Cute chess sounds powerful, but how do you learn how to use it? Sounds like a nightmare to get going with.

Yes, LB can run multiple matches in parallel, it's one of the main features there. You can also change the number of parallel matches during the testing (Cutechess can't do that). I found all the necessary info for Cutechess-cli online and in the readme file.

[lkaufman]

I've been trying to run matches using the LittleBlitzer gui and "Noomen 2012" pgn opening suite, but for some reason I can't figure out it ignores the book and just has the engines play on their own from the start. The path seems to be correct. Can anyone suggest what I might be doing wrong, or alternatively can anyone point me to some easily downloaded opening suite that is known to work with LittleBlitzer without any problems?
Thanks in advance.

Check that "PGN" is selected. I use Bob Hyatt's 4000 EPD opening positions and SWCR.pgn ~5000 8-mover games, both working well.[/quotm

Thanks. I got it working ok with another PGN database Don made a while go from GM openings. But now there is another problem.
Everything works fine on my normal i7 quad. But when I run it on machines with two actual processors (12 core and 16 core machines) it works properly in sudden death mode, but when I add an increment (one second or more) it causes frequent time forfeits by Stockfish and Houdini, but not by Komodo. I'm not a hardware expert, but I guess there is some communications lag between the processors that is not properly accounted for, so that the increment is not exactly right. Probably it doesn't affect Komodo because we build in a small cushion with the increment to avoid this sort of problem. Anyway, I don't want to see time forfeits in test matches, so I'm having to run only game/x matches or matches with very tiny increment (fraction of a second). Can you or anyone else explain this any better or suggest a work-around that would permit normal increment play?

[pohl4711]

But now there is another problem.
Everything works fine on my normal i7 quad. But when I run it on machines with two actual processors (12 core and 16 core machines) it works properly in sudden death mode, but when I add an increment (one second or more) it causes frequent time forfeits by Stockfish and Houdini, but not by Komodo. I'm not a hardware expert, but I guess there is some communications lag between the processors that is not properly accounted for, so that the increment is not exactly right. Probably it doesn't affect Komodo because we build in a small cushion with the increment to avoid this sort of problem. Anyway, I don't want to see time forfeits in test matches, so I'm having to run only game/x matches or matches with very tiny increment (fraction of a second). Can you or anyone else explain this any better or suggest a work-around that would permit normal increment play?

Hi Larry,

because I only use Quadcore i7-CPUs, I cant say anything about this LBG-problem with more than one CPU. But what I can say is, that it is no problem to start more than one LBG at the same time on the same machine. Just make a copy of the LBG-folder and start this new LBG after the first one.
Example for 2 CPUs with 6 cores (=12 cores)
- Be sure that Hyperthreading is turned off !!!
- Start the first LBG (with 6 threads/games running in parallel). If you look in the Task Manager, only the first 6 cores (= the first CPU) should be used (all cores 100%). The other 6 cores (=the second CPU) should be unused (0%, or 1-3% or something like that)
- Then start the second LBG (with 6 threads/games.). Now all 12 cores should be used with 100%, but there is no need for any communication/ data transfer between the CPUs (hopefully)...

Perhaps this works better?!? (Only use the Gauntlet-mode of the LBG, in RR-mode, the opening-positions are choosen per random and are not replayed with reversed colors) and dont use PGN-opening positions with en passant-moves, because the captured pawn doesnt disappear (bug in the LBG)...

On my website, you can download some more hints for LBG-users in a small textfile and the 500 PGN-opening-positions, which I use for my LS-ratinglist testwork (all without en passant-moves)
http://ls-ratinglist.beepworld.de/setti ... -links.htm

Best - Stefan

[lkaufman]

But now there is another problem.
Everything works fine on my normal i7 quad. But when I run it on machines with two actual processors (12 core and 16 core machines) it works properly in sudden death mode, but when I add an increment (one second or more) it causes frequent time forfeits by Stockfish and Houdini, but not by Komodo. I'm not a hardware expert, but I guess there is some communications lag between the processors that is not properly accounted for, so that the increment is not exactly right. Probably it doesn't affect Komodo because we build in a small cushion with the increment to avoid this sort of problem. Anyway, I don't want to see time forfeits in test matches, so I'm having to run only game/x matches or matches with very tiny increment (fraction of a second). Can you or anyone else explain this any better or suggest a work-around that would permit normal increment play?

Hi Larry,

because I only use Quadcore i7-CPUs, I cant say anything about this LBG-problem with more than one CPU. But what I can say is, that it is no problem to start more than one LBG at the same time on the same machine. Just make a copy of the LBG-folder and start this new LBG after the first one.
Example for 2 CPUs with 6 cores (=12 cores)
- Be sure that Hyperthreading is turned off !!!
- Start the first LBG (with 6 threads/games running in parallel). If you look in the Task Manager, only the first 6 cores (= the first CPU) should be used (all cores 100%). The other 6 cores (=the second CPU) should be unused (0%, or 1-3% or something like that)
- Then start the second LBG (with 6 threads/games.). Now all 12 cores should be used with 100%, but there is no need for any communication/ data transfer between the CPUs (hopefully)...

Perhaps this works better?!? (Only use the Gauntlet-mode of the LBG, in RR-mode, the opening-positions are choosen per random and are not replayed with reversed colors) and dont use PGN-opening positions with en passant-moves, because the captured pawn doesnt disappear (bug in the LBG)...

On my website, you can download some more hints for LBG-users in a small textfile and the 500 PGN-opening-positions, which I use for my LS-ratinglist testwork (all without en passant-moves)
http://ls-ratinglist.beepworld.de/setti ... -links.htm

Best - Stefan

Thanks for your suggestion, I'll give it a try on my next test with that machine.
Some questions:
1. I suppose my pgn database has at least a couple en passant captures. What will be the consequences? Will it just play the game normally except that the pawn will remain, or will it lead to a forfeit by one side, or something else? If it only affects a fraction of a percent of the games and in a neutral way I guess the harm done is minimal. Of course it's better to be "perfect".
2. You say you use a database of 500 games, which is enough for one thousand games without repeats. But you play ten thousand game matches. Doesn't that mean that many games will be duplicates for a number of moves past book, perhaps enough to decide the game? Shouldn't you use a 5000 position book? The book I use has almost 32,000 positions.
3. You say to turn hyperthreading off. What are the consequences of not doing so? Is it just a slight loss of efficiency, or does this somehow corrupt or bias the test? I would point out that it is likely that a majority of users leave hyperthreading on since they use the computers for many things, so this is the more relevant way to test unless there is some serious bias with it.

Thanks in advance. I consider your list to be the best way to measure progress from one version to another within a family, although when comparing unrelated engines the results don't correlate well with longer time control tests, as different engines scale differently.

Best regards,
Larry Kaufman

[pohl4711]

Thanks for your suggestion, I'll give it a try on my next test with that machine.
Some questions:
1. I suppose my pgn database has at least a couple en passant captures. What will be the consequences? Will it just play the game normally except that the pawn will remain, or will it lead to a forfeit by one side, or something else? If it only affects a fraction of a percent of the games and in a neutral way I guess the harm done is minimal. Of course it's better to be "perfect".
2. You say you use a database of 500 games, which is enough for one thousand games without repeats. But you play ten thousand game matches. Doesn't that mean that many games will be duplicates for a number of moves past book, perhaps enough to decide the game? Shouldn't you use a 5000 position book? The book I use has almost 32,000 positions.
3. You say to turn hyperthreading off. What are the consequences of not doing so? Is it just a slight loss of efficiency, or does this somehow corrupt or bias the test? I would point out that it is likely that a majority of users leave hyperthreading on since they use the computers for many things, so this is the more relevant way to test unless there is some serious bias with it.

Thanks in advance. I consider your list to be the best way to measure progress from one version to another within a family, although when comparing unrelated engines the results don't correlate well with longer time control tests, as different engines scale differently.

Best regards,
Larry Kaufman

1. The game will be played normally, but with the remaining pawn on the board.
2. I play 10000 games, but the tested engine plays 1000 games against 10 different opponents. So I need 500 opening postitions only (10 gauntlet-matches (against 10 different opponents/engines with 1000 games each)). In Gauntlet-Mode the LBG uses the 500 positions one after another (2 times (reversed colors)). It is no opening-book, it is a collection of opening-positions, which are used one after another. So 1000 games = 500 positions.
3. Only with hyperthreading off the LBG will use x cores for x games (parallel) with 100% and let the rest of the cores unused. In order to solve your problem with the LBG and 2 CPUs on one machine, this is the point (to use one LBG for one CPU and another copy of the LBG for the other CPU). With hyperthreading on all cores are used (but not necessary with 100%)...

Please excuse my awful english...

Best - Stefan

[syzygy]

- Be sure that Hyperthreading is turned off !!!

Be sure to use not more threads than there are physical core and an OS that is not completely outdated.

[lkaufman]

Thanks for your suggestion, I'll give it a try on my next test with that machine.
Some questions:
1. I suppose my pgn database has at least a couple en passant captures. What will be the consequences? Will it just play the game normally except that the pawn will remain, or will it lead to a forfeit by one side, or something else? If it only affects a fraction of a percent of the games and in a neutral way I guess the harm done is minimal. Of course it's better to be "perfect".
2. You say you use a database of 500 games, which is enough for one thousand games without repeats. But you play ten thousand game matches. Doesn't that mean that many games will be duplicates for a number of moves past book, perhaps enough to decide the game? Shouldn't you use a 5000 position book? The book I use has almost 32,000 positions.
3. You say to turn hyperthreading off. What are the consequences of not doing so? Is it just a slight loss of efficiency, or does this somehow corrupt or bias the test? I would point out that it is likely that a majority of users leave hyperthreading on since they use the computers for many things, so this is the more relevant way to test unless there is some serious bias with it.

Thanks in advance. I consider your list to be the best way to measure progress from one version to another within a family, although when comparing unrelated engines the results don't correlate well with longer time control tests, as different engines scale differently.

Best regards,
Larry Kaufman

1. The game will be played normally, but with the remaining pawn on the board.
2. I play 10000 games, but the tested engine plays 1000 games against 10 different opponents. So I need 500 opening postitions only (10 gauntlet-matches (against 10 different opponents/engines with 1000 games each)). In Gauntlet-Mode the LBG uses the 500 positions one after another (2 times (reversed colors)). It is no opening-book, it is a collection of opening-positions, which are used one after another. So 1000 games = 500 positions.
3. Only with hyperthreading off the LBG will use x cores for x games (parallel) with 100% and let the rest of the cores unused. In order to solve your problem with the LBG and 2 CPUs on one machine, this is the point (to use one LBG for one CPU and another copy of the LBG for the other CPU). With hyperthreading on all cores are used (but not necessary with 100%)...

Please excuse my awful english...

Thanks again. I understand all points. Stupid of me to forget that you were playing gauntlet-style against ten opponents! Also, your English is fine!



Best - Stefan

Best - Larry

[lkaufman]

- Be sure that Hyperthreading is turned off !!!

Be sure to use not more threads than there are physical core and an OS that is not completely outdated.

If you test with hyperthreading on (let's say on a one processor quad i7 machine), what are the consequences of setting threads to 8 instead of 4? I believe it makes the testing quite a bit more efficient (nearly 20%) but I'm less sure if it might introduce a bias somehow. When we test in Linux we often overprovision like this, with no apparent problems, but so far when testing in Windows (7 or 8) (with HT on) I've kept to your rule about not exceeding number of physical cores.

Larry

[syzygy]

- Be sure that Hyperthreading is turned off !!!

Be sure to use not more threads than there are physical core and an OS that is not completely outdated.

If you test with hyperthreading on (let's say on a one processor quad i7 machine), what are the consequences of setting threads to 8 instead of 4? I believe it makes the testing quite a bit more efficient (nearly 20%) but I'm less sure if it might introduce a bias somehow. When we test in Linux we often overprovision like this, with no apparent problems, but so far when testing in Windows (7 or 8) (with HT on) I've kept to your rule about not exceeding number of physical cores.

I'm not very sure about the consequences, but it might just come out fine.

What I meant was mainly that instead of turning HT off in the BIOS, it should be sufficient (and even better) to limit the number of threads to the number of physical cores. That way HT isn't really used, at least not on a good OS.

But if in your experience the results with 8 threads on 4 HT-enabled cores are reliable and gives you 20% more games in the same time, then I can give no good reason why you should not do this. Usually any imbalances should go both ways. However, do keep in mind that there might be a higher tendency for strange things to happen.