Genetical tuning

[Ferdy]

Just 2 questions:
1 have you written you own framework for GA ?

Yes but this is based from the idea of Stefano. It is still in progress and experiment with it. It works on TC 15s+100ms real game test, even with only piece values involved, see my previous post.

2 how do you calculate fitness? Eval, quiescent search or other?

I use qsearch score. And use the eval in the given epd to calculate the error instead of epd result as was done in Texel.
Sample epd.

Code: Select all

r7/ppk1br2/2p1pn1p/4p3/P3P3/3RBP2/1PPNK2P/7R b - - c9 "0-1"; sm "f7g7"; ce -24;

That ce is what I compare with the qsearch score from engine. That ce value is the actual score from the engine during the actual game. So that ce is more reliable than the qsearch score from the engine under test.

3 what about elo verification?

I already have a system similar to Texel but uses the ce eval, and it works for me. But now I try this GA thing with some info from Stefano. I don't know how far Stefano has explored with this. I am not concerned about elo at this time I am still observing if it can still reduce the error at higher population and genes, and of course trying to find the secrets on crossover and mutation. The one I posted before was the only elo verification I have done. I did not bother testing that with longer TC but it was already a winner at TC 15s+100ms.

[stegemma]

[...]I don't know how far Stefano has explored with this.[...]I did not bother testing that with longer TC but it was already a winner at TC 15s+100ms.

Your approach is more scientific and pragmatic than mine. I've just tested and corrected the base algorithm using only my intuition and a few sessions (my sessions are longer because of the tournaments that I run). I find useful for this purpose to watch at the growing generations at runtime (looking at the progressive log file) as I were watching a chess game directly at the chess club. That way, you have the time to think but you are "pressed" by the incoming new move (generation) and your mind works faster and more focused

Some dynamic of the system can be caught better if you look at an "animation" of the parameters. I used to copy the values of the generations in a spreadsheet and then to color the square of the modified ones, to have a visual image of what happens. Drawing a graph on screen, with 3D looking bars, would help a lot but it requires more work.

My final goal is still to make the GA suitable for running during game (as said, that was the "scope" of Satana), to calibrate the parameters at the actual position. Your idea to use set of positions could help for this but it should be improved almost to selects positions that are similar to the actual one. in thos aspect, maybe a neural network could help but that is out of my present knowledge...

[Ferdy]

Some dynamic of the system can be caught better if you look at an "animation" of the parameters. I used to copy the values of the generations in a spreadsheet and then to color the square of the modified ones, to have a visual image of what happens. Drawing a graph on screen, with 3D looking bars, would help a lot but it requires more work.

I am trying to create one, generation at x, error in y, size of individual is increased when it experienced more crossovers.

[stegemma]

This looks very good. It seems to be a sinusoid that converge to the optimal value. It seems that the variability between individual will grow with generations but the average error tends to be minimized.

It would be nice to compare this with some other method.

[Ferdy]

Tried to improve the plots. Now with more engines, params and a new method of introducing training pos. Instead of a fixed training sets given per generation I varied it by randomizing. Here is the initial plot, Manila uses the default value of Deuterium, the other individuals are using randomized parameter values, as their first values. Bigger area of circles suggests the individual received more crossovers.





Here is the plot after 1st, 2nd and 3rd generations.
Here we can see Manila was first after the 1st gen. It is using the default values. Last was Beijing., In the 2nd gen Beijing has improved a lot after crossovers from Manila and Washington. (I wonder what will happen now in South China Sea) . The size of Beijing has increased twice because it received genes from Manila and Washington. Manila also has increased in size and is struggling to reduce the average error. Washington does not change its size, meaning it does not received new genes from other individual when crossover process was done after first generation. But look its average error, it is also showing difficulties, this is because the training sets in the second generation is different from the training sets given in the first generation. It seems there are some individuals that also experienced difficulties in the 2nd gen, Tokyo, London and Paris are visible.






And this is the plot for last generation or 50. Though different sets of training positions are given, the population were able to reduce the average error.





Here are the parameters involved, found in params_to_tune.txt file.

Code: Select all

option name OffensivePercent type spin default 100 min 100 max 200
option name DefensivePercent type spin default 100 min 100 max 200
option name KnightValueMg type spin default 325 min 300 max 350
option name KnightValueEg type spin default 315 min 300 max 350
option name BishopValueMg type spin default 328 min 300 max 350
option name BishopValueEg type spin default 327 min 300 max 350
option name RookValueMg type spin default 504 min 480 max 530
option name RookValueEg type spin default 498 min 480 max 530
option name QueenValueMg type spin default 996 min 950 max 1200
option name QueenValueEg type spin default 999 min 950 max 1200

Other info.

Code: Select all

Population            : 16
Target generations    : 50
Fitness positions     : 6000
Base engine           : Deuterium v2015.1.35.239 64bit POP

Parameters or genes considered to optimize fitness of individuals:
 0, OffensivePercent
 1, DefensivePercent
 2, KnightValueMg
 3, KnightValueEg
 4, BishopValueMg
 5, BishopValueEg
 6, RookValueMg
 7, RookValueEg
 8, QueenValueMg
 9, QueenValueEg

Chromosome:
['OffensivePercent', 'DefensivePercent', 'KnightValueMg', 'KnightValueEg', 'BishopValueMg', 'BishopValueEg', 'RookValueMg', 'RookValueEg', 'QueenValueMg', 'QueenValueEg']

History of top performers per generation:
generation   1, name       Manila, par [100, 100, 325, 315, 328, 327, 504, 498, 996, 999], aveErrCp 114.12030
generation   2, name WashingtonDC, par [177, 110, 335, 313, 344, 323, 525, 514, 1177, 1034], aveErrCp 115.33674
generation   3, name        Tokyo, par [104, 127, 335, 312, 333, 311, 484, 527, 1028, 1171], aveErrCp 114.59438
generation   4, name        Tokyo, par [104, 127, 335, 315, 336, 348, 504, 526, 1169, 988], aveErrCp 114.03829
generation   5, name        Tokyo, par [104, 127, 335, 315, 336, 348, 504, 526, 1169, 988], aveErrCp 113.91199
generation   6, name        Tokyo, par [104, 127, 335, 315, 336, 348, 504, 526, 1169, 988], aveErrCp 114.14157
generation   7, name        Tokyo, par [104, 127, 335, 315, 336, 348, 504, 526, 1169, 988], aveErrCp 114.23751
generation   8, name         Oslo, par [104, 127, 335, 315, 336, 348, 504, 526, 967, 1140], aveErrCp 113.98722
generation   9, name         Oslo, par [104, 127, 335, 315, 336, 348, 504, 526, 967, 1140], aveErrCp 114.17203
generation  10, name        Tokyo, par [107, 111, 335, 307, 336, 348, 504, 526, 1169, 988], aveErrCp 113.56593
generation  11, name       London, par [107, 111, 335, 315, 336, 348, 504, 526, 967, 1140], aveErrCp 113.76947
generation  12, name       London, par [107, 111, 335, 312, 333, 348, 504, 526, 967, 1140], aveErrCp 113.77239
generation  13, name        Tokyo, par [107, 111, 335, 307, 336, 348, 504, 526, 1169, 988], aveErrCp 113.76378
generation  14, name         Oslo, par [107, 111, 335, 313, 336, 348, 504, 526, 967, 1140], aveErrCp 113.32818
generation  15, name        Tokyo, par [107, 111, 335, 307, 336, 348, 504, 526, 1169, 988], aveErrCp 113.54158
generation  16, name  BuenosAires, par [107, 111, 335, 307, 336, 348, 504, 526, 1044, 1067], aveErrCp 113.75125
generation  17, name  BuenosAires, par [107, 111, 335, 307, 336, 348, 504, 526, 1044, 1067], aveErrCp 113.66713
generation  18, name  BuenosAires, par [107, 111, 335, 307, 336, 348, 504, 526, 1044, 1067], aveErrCp 113.98427
generation  19, name        Tokyo, par [107, 111, 335, 307, 336, 348, 504, 526, 1169, 988], aveErrCp 114.04292
generation  20, name    Amsterdam, par [107, 111, 335, 307, 336, 348, 504, 526, 1169, 988], aveErrCp 113.51611
generation  21, name        Tokyo, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 1017], aveErrCp 113.60590
generation  22, name        Tokyo, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 1017], aveErrCp 113.49311
generation  23, name      Beijing, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 993], aveErrCp 113.87451
generation  24, name      Beijing, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 993], aveErrCp 113.57404
generation  25, name      Beijing, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 993], aveErrCp 113.25582
generation  26, name      Beijing, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 993], aveErrCp 113.87818
generation  27, name      Beijing, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 993], aveErrCp 113.85483
generation  28, name    Amsterdam, par [107, 111, 335, 307, 336, 348, 504, 526, 1169, 988], aveErrCp 113.52490
generation  29, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.42669
generation  30, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.54144
generation  31, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.56912
generation  32, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.61096
generation  33, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.83537
generation  34, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.45757
generation  35, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.63644
generation  36, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.79182
generation  37, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.73589
generation  38, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.76826
generation  39, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.40456
generation  40, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 114.01088
generation  41, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.56913
generation  42, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.84524
generation  43, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.73708
generation  44, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.58399
generation  45, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.39519
generation  46, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.69475
generation  47, name    Amsterdam, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.81290
generation  48, name    Amsterdam, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.83456
generation  49, name    Amsterdam, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.31809
generation  50, name    Amsterdam, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.93207

Top 5 performing individuals at given generation:
generation 25, name      Beijing, par [107, 111, 335, 315, 336, 348, 504, 528, 1177, 993], aveErrCp 113.25582
generation 49, name    Amsterdam, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.31809
generation 14, name         Oslo, par [107, 111, 335, 313, 336, 348, 504, 526, 967, 1140], aveErrCp 113.32818
generation 45, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.39519
generation 39, name         Oslo, par [107, 111, 335, 307, 336, 348, 504, 526, 986, 1096], aveErrCp 113.40456

Test duration : 00h:07m:32s:924ms

[stegemma]

Very good! The best engines perform better than "Manila", the original values.

The "curve" seems to be in a limit but maybe with some other hundred of generations something could happens to find better individuals (but this could be an intrinsic limit of the engine itself).

[Ferdy]

Got some improvement in visual especially the random crossovers, now it is shown aside from the default crossover of top 1 and 2. Browsing thru the plot from generation to generation, generally if the top ranking individual transfers it genes to lower ranked individual, the lower ranked individual will improve. Now this will get interesting if generation is high. Here are 2 sample plots.

The first 3 generations. Brussels and Manila helps the last rank BuenosAires. Manila is using default values and the others are using random values initially, including Brussels, but here Brussels performs better than the Manila. But before the genes of manila were transfered to BuenosAires, the genes of Manila was altered by Tokyo thru random crossover which has taken place first before the top 1 and 2 crossover are performed.
At generation 2 BuenosAires' performance has improved. The performance of Manila has been affected in generation 2, perhaps due the the genes from Tokyo which is a poor performer in generation 1. Brussels was not affected by crossover but it performs better than in generation 1. This also shows that the training sets in generation 2 is easier to Brussels, since different training sets were given in every generation.




Generations close to the middle. I am going to stop story telling for now .

[stegemma]

Still good: after 11 generations Buenos Aires performs almost like initial Manila (that we lose on generation 2) while the others perform not more than 115 where in generation 1 the worst was 118 (lower is better).

[Ferdy]

[...]
I am researching on mutation, and this would look good only if I can trace the full history of the individuals like, does it able to reduce the aveErr if value of pawn increases or decreases, same with other params. Once this is known mutation can be introduced on certain parameter.

There are a lot to try also on different kinds of crossover, the one I am using is called single point crossover, with crossover point randomly selected.
[...]

This is the same type of crossover that I use: a single point crossing. I don't think that other kind of crossing could works. In effect, the idea of crossing the last one with the best two gives you some kind of two point crossing and because before crossing the last individual has been set casually, you don't need to add any kind of mutation. The first attempt I've made was with a complex crossing and mutation algorithm but it was too hard to configure. In this last version, you don't need to worry about % of mutations, because they are "embedded". As I've said, the random values grow from bottom to up and the good ones grow from up to bottom. The best individual is a little protected but without a true elitism, so that anything could happen.

Here is a weakness of the population if no mutation is done.
Individual: manila, par1 70, par2 80, par3 90, par4 100, par5 110
Individual: japan, par1 78, par2 88, par3 98, par4 108, par5 118
Individual: paris, par1 88, par2 98, par3 108, par4 118, par5 128
... and so on.

Say our population is not that big 100, also
say par1 value has range of 200, from -30 to 170,
since our population is only 100 we cannot distribute all the possible values of par1.
Here we have manila par1 70 ...
If we are not able to generate randomly the par1 = 1 to any of the individuals, our population will miss this par entirely,
no matter how many generations you generate because during crossover we just replace the value of individual, say genes of manila will crossover to genes of paris at
par1, that would be paris par1 = 70. Par1 = 1 and other numbers will not be tested, but if we introduce mutation, I can change the manila par1 = 70 to manila par1 = 70-1 = 69,
so this gene par1 is mutating and could possibly hit par1 = 1 as generations continues. The mutation that I plan is only +/- 1, and will be randomly selected to what gene it will be applied on the individual that
gets the new gene. This is something similar to what Texel tuning was doing,
par1 = par1 + 1 >> test and get ave_error
if not good try,
par1 = par1 - 1 >> test and get ave_error
if still not good, try
par1 = par1 + 2 >> test and get ave_error
if still not good, try
par1 = par1 - 2 >> test and get ave_error
if still not good, we leave it as it is and go for next par,
par2 = par2 + 1 ...

When par1 reaches -30 it is not allowed to mutate to -31, because par1 has only a range of min = -30 to max = 170.

[stegemma]

But the last engine gets always random values, before merging with first+second one. In fact, those random genes can have any value, in the predefined ranges of anyone of them. This is the "bottom-up" growing of random values, so I don't see any problem, related to parameters ranges.