New Tool for Tuning with Skopt

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
thomasahle
Posts: 72
Joined: Thu Feb 27, 2014 7:19 pm

Re: New Tool for Tuning with Skopt

Post by thomasahle » Sun Sep 15, 2019 2:52 pm

pedrox wrote:
Sat Sep 14, 2019 11:12 am
Once I have played a series of games and I have the file data.log, if I run the test again and use the file, in each run I have different results and the results seem quite random.
This happens because summarizing works by sampling a number of random values. As far as I can see the arguments you get out are quite similar.

I realize this might actually be annoying, so I've pushed a new version that summarizes using gradient descent and should be more deterministic (though still not entirely, as it picks a 100 random starting points for the decent.)

voffka
Posts: 118
Joined: Sat Jun 30, 2018 8:58 pm
Location: Ukraine
Full name: Volodymyr Shcherbyna
Contact:

Re: New Tool for Tuning with Skopt

Post by voffka » Sat Feb 22, 2020 8:02 pm

Hello thomasahle,
thomasahle wrote:
Sun Sep 15, 2019 2:52 pm
pedrox wrote:
Sat Sep 14, 2019 11:12 am
Once I have played a series of games and I have the file data.log, if I run the test again and use the file, in each run I have different results and the results seem quite random.
This happens because summarizing works by sampling a number of random values. As far as I can see the arguments you get out are quite similar.

I realize this might actually be annoying, so I've pushed a new version that summarizes using gradient descent and should be more deterministic (though still not entirely, as it picks a 100 random starting points for the decent.)
Thanks for the tool!

After some manual hand tuning I decided it is time to master your tool and I got the following error on a latest version from master:

Code: Select all

Starting 2 games 504/1000 with {'Option_1': 376, 'Option_2': 11}
Summarizing best values
play_game Cancelled
play_game Cancelled
play_game Cancelled
play_game Cancelled
Traceback (most recent call last):
  File "tune.py", line 523, in <module>
    asyncio.run(main())
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 579, in run_until_complete
    return future.result()
  File "tune.py", line 497, in main
    summarize(opt, steps=1)
  File "tune.py", line 332, in summarize
    new_opt.tell(opt.Xi, opt.yi)
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\site-packages\skopt\optimizer\optimizer.py", line 443, in tell
    return self._tell(x, y, fit=fit)
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\site-packages\skopt\optimizer\optimizer.py", line 502, in _tell
    acq_func_kwargs=self.acq_func_kwargs)
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\site-packages\skopt\acquisition.py", line 42, in _gaussian_acquisition
    func_and_grad = gaussian_lcb(X, model, kappa, return_grad)
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\site-packages\skopt\acquisition.py", line 142, in gaussian_lcb
    mu, std = model.predict(X, return_std=True)
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\site-packages\skopt\learning\gaussian_process\gpr.py", line 324, in predict
    y_var -= np.einsum("ki,kj,ij->k", K_trans, K_trans, K_inv)
  File "<__array_function__ internals>", line 6, in einsum
  File "C:\Users\volodymyr\AppData\Local\Programs\Python\Python37-32\lib\site-packages\numpy\core\einsumfunc.py", line 1356, in einsum
    return c_einsum(*operands, **kwargs)
ValueError: iterator is too large
My command line is the following:

Code: Select all

python tune.py igel -concurrency=4 -nodes 3000 -book C:\Users\volodymyr\Documents\Sources\fastchess\tools\gaviota-starters.pgn -games-file tune.pgn -opt Option_1 -opt Option_2 -n 1000 -base-estimator GBRT -acq-noise 5 -log-file data.log

Post Reply