Speeding Up The Tuner

[D Sceviour]

Even when your evaluator was piece-square only it couldn't be much faster than reading a FEN. But if you clear a few GB of memory that was not used in the first place... That is just wrong design. There is no reason at all to clear the hash table on setboard. There actually is never any need to clear the hash if you do not switch variants and calculate the key from scratch on setboard (i.e. use an absolute key). And even when you would use a relative key (i.e. start with a random value unrelated to the true xor sum of Zobrist keys, but update it incrementally in the normal way) it would in fact be self-invalidating. If you do want to clear it, though, the 'new' command would be the recommended place; GUIs should always send one before a setboard.

Fixed. Actually, the hash clear was done after the ReadParse() and not inside the setboard() function. FRC\960 variant was added in the last version, and a lot more clearing and error checking was added including a reconstruction of the PST's after every fen read. So nothing has changed in regard to using a binary file to speed things up for the tuner.

[hgm]

Note that FRC isn't really a variant, so there is no need for clearing the hash when switching between Chess and Chess960; everything in there remains valid.

[Sven]

Even when your evaluator was piece-square only it couldn't be much faster than reading a FEN. But if you clear a few GB of memory that was not used in the first place... That is just wrong design. There is no reason at all to clear the hash table on setboard. There actually is never any need to clear the hash if you do not switch variants and calculate the key from scratch on setboard (i.e. use an absolute key). And even when you would use a relative key (i.e. start with a random value unrelated to the true xor sum of Zobrist keys, but update it incrementally in the normal way) it would in fact be self-invalidating. If you do want to clear it, though, the 'new' command would be the recommended place; GUIs should always send one before a setboard.

Fixed. Actually, the hash clear was done after the ReadParse() and not inside the setboard() function. FRC\960 variant was added in the last version, and a lot more clearing and error checking was added including a reconstruction of the PST's after every fen read. So nothing has changed in regard to using a binary file to speed things up for the tuner.

In the tuning code of Jumbo I clear the pawn hash and recalculate all eval tables (e.g. PST) once per parameter change, and leave it intact for all positions being evaluated with the new parameter set. I also optimized that a bit by considering which parameters really affect the pawn hash, the PST etc.

What I did not do yet is to optimize the time for setting up positions, like you do now. Even if FEN parsing is very fast and does not do any error checking, it is always slower than copying some bytes, so I guess I can even save a bit more by reading binary positions. The saving for N positions and K different parameter sets being tried could roughly be N*K*(parsing time per FEN - binary copying time per position) - ignoring cache effects - and should be significant.

[D Sceviour]

I make a public repo: https://github.com/KierenP/ChessTrainingSets.

So far there is only one tuning file. It would be nice to see more contributions. I am looking for 4 million positions already filtered with a quiescent score.

[Kieren Pearson]

I make a public repo: https://github.com/KierenP/ChessTrainingSets.

So far there is only one tuning file. It would be nice to see more contributions. I am looking for 4 million positions already filtered with a quiescent score.

Unfortunately github has a limit of 100mb files and 1Gb total. If anyone can highly compress larger datasets they find contact me and I’ll put them up

[chrisw]

I make a public repo: https://github.com/KierenP/ChessTrainingSets.

So far there is only one tuning file. It would be nice to see more contributions. I am looking for 4 million positions already filtered with a quiescent score.

Unfortunately github has a limit of 100mb files and 1Gb total. If anyone can highly compress larger datasets they find contact me and I’ll put them up

Yup. When I compressed my data with a view to sharing it this way, size was in the territory of 2.5 Gb, if we want a repository that’s two-way, eg with dataset creators (voluntarily) sharing as much as they are taking then we need nominal space running towards 50 Gb or so.
The alternative is that everybody has to boot up their Python, download Python chess, download games databases, write some relatively simple EPD generator and then run those EPDs past some kind of (probably own home brew) quiescence function, and then again past a search and eval. The final stage involves several days of a normal PC.

[D Sceviour]

The alternative is that everybody has to boot up their Python, download Python chess, download games databases, write some relatively simple EPD generator and then run those EPDs past some kind of (probably own home brew) quiescence function, and then again past a search and eval. The final stage involves several days of a normal PC.

Texel author (Peter Osterlund) created about 5 million positions for Texel's tuning. I can pass them along if anyone is interested. The files follow Peter' suggestion for tuning. They are merely full games with each move turned into a fen. Peter saw no use for altering any other data. The fens do not include a game result. I assume the quiescent function was used at that time to create a game result, or something like that.

This brings another question. What is the purpose of passing any position through a quiescent function? I would think for evaluation tuning, the evaluator has to produce results closest to the game result.

[chrisw]

The alternative is that everybody has to boot up their Python, download Python chess, download games databases, write some relatively simple EPD generator and then run those EPDs past some kind of (probably own home brew) quiescence function, and then again past a search and eval. The final stage involves several days of a normal PC.

Texel author (Peter Osterlund) created about 5 million positions for Texel's tuning. I can pass them along if anyone is interested. The files follow Peter' suggestion for tuning. They are merely full games with each move turned into a fen. Peter saw no use for altering any other data. The fens do not include a game result. I assume the quiescent function was used at that time to create a game result, or something like that.

This brings another question. What is the purpose of passing any position through a quiescent function? I would think for evaluation tuning, the evaluator has to produce results closest to the game result.

The idea is to add the quiescence PV (if there is one) to the original EPD and then use that as the tuning data. Reason is that you’re tuning your eval function and in real search environment your eval won’t see the pre-quiescent EPD but instead the position after any (good) captures play out.
If your qsearch doesn’t want to play out captures, then you use the original EPD. It’s probably better and important to use your own qsearch, btw.
It’s a compromise forced on AB qsearch engines by the nature of having to split world view into position plus material.

[D Sceviour]

What is the purpose of passing any position through a quiescent function? I would think for evaluation tuning, the evaluator has to produce results closest to the game result.

The idea is to add the quiescence PV (if there is one) to the original EPD and then use that as the tuning data. Reason is that you’re tuning your eval function and in real search environment your eval won’t see the pre-quiescent EPD but instead the position after any (good) captures play out.
If your qsearch doesn’t want to play out captures, then you use the original EPD. It’s probably better and important to use your own qsearch, btw.
It’s a compromise forced on AB qsearch engines by the nature of having to split world view into position plus material.

In real search time, the evaluator does not know the PV continuation from exchanges. It has to rely on SEE values or threats. A quiescent search would destroy all that evaluation work and give results that do not apply to the fen position. One way to test this is to compare two tunings - (1) with an evaluation score, and (2) with a quiescent score. Then, discover which one produces a higher elo strength. Anybody done this?