TalkChess.com

pedrox wrote: ↑Thu Aug 29, 2019 4:11 am But I see that PV is also shown on the console and this is not good for me,

Ratosh wrote: ↑Fri Aug 30, 2019 9:09 am Hello,

It's a nice project, but i had issues:
- ahash from log file should not consider number of games and even concurrency, that way would be easy to run a 200 game session run using 100 games from previous run.
- Let me use options that are not seen as UCI options, i don't wanna to expose all possible tunable constants as options. I just need to define them using tool input options.
- Can't make it work with more than 100 games, tested with 6 options.

jorose wrote: ↑Fri Aug 30, 2019 7:08 am Just wanted to mention I was successfully able to retune Winter's futility and static null move pruning margins with this tool. I measured a self play rating gain of +9 +- 3 Elo roughly.

Joerg Oster wrote: ↑Fri Aug 30, 2019 10:35 am

Ratosh wrote: ↑Fri Aug 30, 2019 9:09 am Hello,

It's a nice project, but i had issues:
- ahash from log file should not consider number of games and even concurrency, that way would be easy to run a 200 game session run using 100 games from previous run.
- Let me use options that are not seen as UCI options, i don't wanna to expose all possible tunable constants as options. I just need to define them using tool input options.
- Can't make it work with more than 100 games, tested with 6 options.

What happens with more than 100 games?
I do have problems, too, when the number of games keeps rising.
CPU usage is suddenly growing with peaks up to >90% under Linux.

Joerg Oster wrote: ↑Fri Aug 30, 2019 6:33 pm

Joerg Oster wrote: ↑Fri Aug 30, 2019 10:35 am

Ratosh wrote: ↑Fri Aug 30, 2019 9:09 am Hello,

It's a nice project, but i had issues:
- ahash from log file should not consider number of games and even concurrency, that way would be easy to run a 200 game session run using 100 games from previous run.
- Let me use options that are not seen as UCI options, i don't wanna to expose all possible tunable constants as options. I just need to define them using tool input options.
- Can't make it work with more than 100 games, tested with 6 options.

What happens with more than 100 games?
I do have problems, too, when the number of games keeps rising.
CPU usage is suddenly growing with peaks up to >90% under Linux.

It looks like the computational effort is steadily increasing with every iteration, especially when tuning several parameters at once.
I had to cancel another testrun planned with 2,000 iterations after only about 400 iterations, unfortunately. Further progress had become exceptionally expensive.
This is really bad news because this makes this tool almost unusable in practice. But maybe this can be fixed.

Good news is that the parameters were heading towards reasonable values.
(I was tuning some pawn values of Stockfish to validate the tuner is working ...)

Michel wrote: ↑Sat Aug 31, 2019 3:21 pm

Joerg Oster wrote: ↑Fri Aug 30, 2019 6:33 pm

Joerg Oster wrote: ↑Fri Aug 30, 2019 10:35 am

Ratosh wrote: ↑Fri Aug 30, 2019 9:09 am Hello,

It's a nice project, but i had issues:
- ahash from log file should not consider number of games and even concurrency, that way would be easy to run a 200 game session run using 100 games from previous run.
- Let me use options that are not seen as UCI options, i don't wanna to expose all possible tunable constants as options. I just need to define them using tool input options.
- Can't make it work with more than 100 games, tested with 6 options.

What happens with more than 100 games?
I do have problems, too, when the number of games keeps rising.
CPU usage is suddenly growing with peaks up to >90% under Linux.

It looks like the computational effort is steadily increasing with every iteration, especially when tuning several parameters at once.
I had to cancel another testrun planned with 2,000 iterations after only about 400 iterations, unfortunately. Further progress had become exceptionally expensive.
This is really bad news because this makes this tool almost unusable in practice. But maybe this can be fixed.

Good news is that the parameters were heading towards reasonable values.
(I was tuning some pawn values of Stockfish to validate the tuner is working ...)

I think that Bayesian optimization is computationally very heavy. From the description here https://arxiv.org/pdf/1807.02811.pdf it appears to involve at every step the inversion of a matrix whose size is the number of observations up to then.