Haven't happend to you?

[Kempelen]

...... that you fix a bug in your engine and happen that it plays worse...... I'm frustrated.

[Ovyron]

I don't have an engine but yes, I've been aware of bugs that improved strength for some reason. I recall Thinker had a bug that improved its playing style.

[ilari]

...... that you fix a bug in your engine and happen that it plays worse...... I'm frustrated.

Sometimes fixing one bug can create new bugs elsewhere, which is why regression testing is so important. I've never had a bug which increased playing strength, but I've done bugfixes that decreased it.

[Kempelen]

...... that you fix a bug in your engine and happen that it plays worse...... I'm frustrated.

Sometimes fixing one bug can create new bugs elsewhere, which is why regression testing is so important. I've never had a bug which increased playing strength, but I've done bugfixes that decreased it.

I am really curious about how you make regresion testing. RT means you make test about all functionalities you implemented in the past. How do you do that? do you make a big epd with the testing positions you used to test each functionality when you want to test the last change?

[ilari]

...... that you fix a bug in your engine and happen that it plays worse...... I'm frustrated.

Sometimes fixing one bug can create new bugs elsewhere, which is why regression testing is so important. I've never had a bug which increased playing strength, but I've done bugfixes that decreased it.

I am really curious about how you make regresion testing. RT means you make test about all functionalities you implemented in the past. How do you do that? do you make a big epd with the testing positions you used to test each functionality when you want to test the last change?

I have a lot of little unit tests which can automatically and quickly detect when something is seriously broken. With these tests I can verify that the core routines like move generation, makemove, undomove, transposition table, move and FEN notation, opening book, etc. work correctly. This step only takes about 10 seconds.

If the module tests pass, I run test suites like WAC and WCSAC, and compare the results, speed, branching factor and hash table hit rate to the previous version.

Sometimes I also run a couple of tests under Valgrind to check for memory leaks, and if performance drops unexpectedly I create a cpu time profile with gprof to find the bottlenecks.

Finally, if the test suite results look good I run about 1000 quick test games against a few opponents.

[bob]

...... that you fix a bug in your engine and happen that it plays worse...... I'm frustrated.

That happens, although it is far more likely that you find a bug, and fix it, and the program plays no better...