ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

CLOP for Noisy Black-Box Parameter Optimization
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Edward Yu



Joined: 17 Nov 2008
Posts: 34

PostPost subject: Re: CLOP - a trial for xiangqi engine    Posted: Fri Oct 07, 2011 11:58 am Reply to topic Reply with quote

Hi Remi,

Thank you for your excellent software!

As a trial for my xiangqi engine, I have written the following script pawnval.py and pawnval.clop. Note that it is under Windows XP.
Code:
#!/usr/bin/env python
#############################################################################
"""
 DummyScript.py

 This is an example script for use with parallel optimization. In order to
 apply clop algorithms to your own problem, you should write a script that
 behaves like this one.

 Arguments are:
  #1: processor id (symbolic name, typically a machine name to ssh to)
  #2: seed (integer)
  #3: parameter id of first parameter (symbolic name)
  #4: value of first parameter (float)
  #5: parameter id of second parameter (optional)
  #6: value of second parameter (optional)
  ...

 This script should write the game outcome to its output:
  W = win
  L = loss
  D = draw

 For instance:
  $ ./DummyScript.py node-01 4 param 0.2
  W
"""
#############################################################################
import sys
import math
import random
import time
import os
#
# Log script invocations
#
f = open('pawnval.log', 'a')
print(sys.argv, file=f)

#
# Print doc if not enough parameters
#
if len(sys.argv) < 5:
     print(__doc__)
     sys.exit()

#
# Sleep for a random amount of time
#
# random.seed(int(sys.argv[2]))
# time.sleep(random.random() * 2)

#
# Parse parameter values
#
i = 4
params = []
while i < len(sys.argv):
    # params.append(float(sys.argv[i]))
    params.append(int(sys.argv[i]))
    i += 2

#
# Compute winning probability
#
# d2 = 0
# for i in range(len(params)):
#    delta = params[i] - 0.3456789
#    d2 += delta * delta * 10

# draw_elo = 100
# draw_rating = draw_elo * math.log(10.0) / 400.0
# win_p = 1.0 / (1.0 + math.exp(d2 + draw_rating))
# loss_p = 1.0 / (1.0 + math.exp(-d2 + draw_rating))

#
# Draw a random game according to this probability
#
# r = random.random()
# if r < loss_p:
#    print("L")
# elif r > 1.0 - win_p:
#    print("W")
# else:
#    print("D")

# Run a game with param1=params[0]
fo = open("eychessu.ini","w")
fo.write("[param]\n");
fo.write("param1=" + str(params[0]) + "\n");
fo.write("param2=" + str(params[1]) + "\n");
fo.write("param3=" + str(params[2]) + "\n");
fo.write("param4=" + str(params[3]) + "\n");
fo.write("param5=" + str(params[4]) + "\n");
fo.close()

if os.path.isfile("92h-3dc0.CHK") :
   os.remove("92h-3dc0.CHK");
if os.path.isfile("3dc-92h0.CHK") :   
   os.remove("3dc-92h0.CHK");
os.system("uccileag.exe < ucci3dc.in > ucci3dc.out");

score = 0
roundnum = 0
for line in open("ucci3dc.out"):
    parts = line.split(' ')
    # print len(parts)   
    x = 0
    while x+1<len(parts) :       
       parts[x]=parts[x].strip()
       # print(parts[x])
       if roundnum == 0 :
          if parts[x] == '1/2-1/2' :
             score = 1
             roundnum = 1
          elif parts[x] == '0-1' :
             score = 0
             roundnum = 1
          elif parts[x] == '1-0' :
             score = 2
             roundnum = 1
           
       elif roundnum == 1 :           
          if parts[x] == '1/2-1/2' :
             score = score + 1
             roundnum = 2
          elif parts[x] == '0-1' :
             score = score + 2
             roundnum = 2
          elif parts[x] == '1-0' :
             # score = score + 0
             roundnum = 2
                 
       elif parts[x] == 'Final' :
          if score == 2 :
             print("D")   
          if score < 2 :
             print("L") 
          if score > 2 :
             print("W")                 
       x=x+1
   

os.rename("ucci3dc.out", "ucci3dc.out"+sys.argv[2])
# print("roundnum=",roundnum)
# print("score=",score)


Code:
#
# pawnval.clop
#
# Example of experiment definition
#

# Name (used for .log and .dat files)
Name pawnval

# Script for running a game. See DummyScript.py for details.
Script c:/python32/python.exe pawnval.py

# Parameter(s) to be optimized
# <parameter_type> <name> <min> <max>
# <parameter_type> may be:
#  LinearParameter
#  IntegerParameter
#  GammaParameter
#  IntegerGammaParameter
# For GammaParameter, quadratic regression is performed on log(x)
# Warning: 123 and not 123.0 should be used for IntegerParameter

# int g_param1 = 24; //VP default 24
# int g_param2 = 95; //VPR default 95
# int g_param3 = 32; //ENDPAWNBONUS default 32
# int g_param4 = 32; //FUTPAWN default 32
# int g_param5 = 28; //ENDFUTPAWN default 28

IntegerParameter  vp  20 40
IntegerParameter  vpr 90 100
IntegerParameter  epb 20 40
IntegerParameter  fp  20 40
IntegerParameter  efp 20 40

# LinearParameter p2  0.0 1.0
# LinearParameter p3  0.0 1.0
# LinearParameter p4  0.0 1.0
# LinearParameter p5  0.0 1.0

# This could be the list of machine names in a distributed experiment.
# In order to run 4 games in parallel, 2 on machine1, 2 on machine2:
Processor machine1
# Processor machine1
# Processor machine2
# Processor machine2

# Call the script "Replications" times in a row with identical parameter values
# Replications may be used to alternate colors, for instance.
# Seed % Replications would indicate color.
# Replications 2

# Parameters of statistical model of outcome
# For binary outcome (Win/Loss, no draws), use "DrawElo 0"
# For chess, use "DrawElo 100"
DrawElo 100

# Regression parameters
# H 3 is recommended (it is the default value)
# Correlations may be "all" (default) or "none"
# Even if variables are not correlated "all" should work well. The problem is
# that the regression might become very costly if the number of variables is
# high. So use "Correlations none" only if you are certain parameters are
# independent or you have so many variables that "all" is too costly.
H 3
Correlations all


The experiment is still running, only 38 games so far. I can only use 1'+0" games because the UCCI emulator (uccileag.exe) can only support 1 minute games. I don't know if it really converges, or the result is better than my manual parameters.

Anyway, it is a very good software. At least now I have an automatic engine paramenter tuning procedure. Anyone interested in xiangqi engine tuning can contact me at eykm(at)yahoo.com

Thanks,

Edward Yu
Back to top
View user's profile Send private message
Display posts from previous:   
Subject Author Date/Time
CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 01, 2011 10:32 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Joona Kiiski Fri Sep 02, 2011 8:52 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 02, 2011 9:45 am
                  Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Fri Sep 02, 2011 5:34 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Zach Wegner Fri Sep 02, 2011 4:44 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 02, 2011 7:58 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rein Halbersma Fri Sep 02, 2011 10:07 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 9:24 am
                  Re: CLOP for Noisy Black-Box Parameter Optimization Rein Halbersma Sat Sep 03, 2011 5:01 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 5:28 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Andres Valverde Sat Sep 03, 2011 10:28 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 10:40 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sat Sep 03, 2011 6:54 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 7:34 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Ernest Bonnem Sun Sep 04, 2011 4:49 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Don Dailey Wed Sep 07, 2011 4:57 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 5:47 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Don Dailey Wed Sep 07, 2011 6:09 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 6:26 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization F. Bluemers Wed Sep 07, 2011 6:39 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 8:19 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 08, 2011 4:44 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Larry Kaufman Fri Sep 09, 2011 3:36 am
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 6:11 am
                              Re: CLOP for Noisy Black-Box Parameter Optimization Larry Kaufman Fri Sep 09, 2011 2:43 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 7:47 pm
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 8:24 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 8:30 am
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 8:50 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sun Nov 06, 2011 1:11 pm
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 4:53 pm
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:14 pm
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sun Nov 06, 2011 11:11 pm
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Nov 07, 2011 11:18 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:05 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 7:52 pm
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 10, 2011 9:49 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Sun Sep 25, 2011 8:18 am
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Sep 25, 2011 8:47 am
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Mon Sep 26, 2011 10:55 am
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Sep 26, 2011 1:40 pm
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Tue Sep 27, 2011 7:54 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Tue Sep 27, 2011 8:39 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Thu Sep 29, 2011 2:04 pm
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 29, 2011 3:38 pm
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Daniel Shawul Mon Oct 17, 2011 2:03 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Mon Oct 17, 2011 7:51 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 7:56 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 8:02 am
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 8:11 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Daniel Shawul Mon Oct 17, 2011 9:54 am
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 11:10 am
      Re: CLOP for Noisy Black-Box Parameter Optimization david nash Wed Oct 05, 2011 12:19 am
      Re: CLOP - a trial for xiangqi engine Edward Yu Fri Oct 07, 2011 11:58 am
            Re: CLOP - a trial for xiangqi engine Rémi Coulom Fri Oct 07, 2011 1:17 pm
                  Re: CLOP - a trial for xiangqi engine Edward Yu Fri Oct 07, 2011 4:01 pm
                        Re: CLOP - a trial for xiangqi engine Rémi Coulom Fri Oct 07, 2011 4:30 pm
                              Re: CLOP - a trial for xiangqi engine Edward Yu Sat Oct 08, 2011 3:59 am
                                    Re: CLOP - a trial for xiangqi engine Rémi Coulom Sat Oct 08, 2011 7:49 am
                                          Re: CLOP - a trial for xiangqi engine Edward Yu Tue Oct 11, 2011 4:04 pm
                                                Re: CLOP - a trial for xiangqi engine Rémi Coulom Tue Oct 11, 2011 4:22 pm
                                                      Re: CLOP - a trial for xiangqi engine Daniel Shawul Tue Oct 11, 2011 4:49 pm
                                                            Re: CLOP - a trial for xiangqi engine Rémi Coulom Tue Oct 11, 2011 5:27 pm
                                                                  Issue.... Michel Van den Bergh Sat Oct 15, 2011 2:51 pm
                                                                        Re: Issue.... Rémi Coulom Sun Oct 16, 2011 6:30 am
                                                                              Re: Issue.... Michel Van den Bergh Sun Oct 16, 2011 8:10 am
                                                                                    Re: Issue.... Rémi Coulom Sun Oct 16, 2011 9:02 am
                                                                                          Re: Issue.... Michel Van den Bergh Sun Oct 16, 2011 3:10 pm
                                                                                          Re: Issue.... Rémi Coulom Sun Oct 16, 2011 7:25 pm
                              Re: CLOP Michel Van den Bergh Sat Oct 08, 2011 6:00 am
                                    Re: CLOP Rémi Coulom Sat Oct 08, 2011 7:53 am
                                          Re: CLOP Rémi Coulom Sat Oct 08, 2011 8:27 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Tue Oct 11, 2011 8:22 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Tue Oct 11, 2011 1:38 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Sun Oct 23, 2011 7:39 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Oct 23, 2011 8:55 pm
                              Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Sun Oct 23, 2011 10:47 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 9:42 am
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 9:58 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 10:17 am
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 10:30 am
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 10:44 am
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 10:56 am
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 11:04 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 11:56 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 4:16 pm
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 5:48 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Sergei S. Markoff Fri Nov 04, 2011 3:02 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Nov 05, 2011 7:45 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sat Nov 05, 2011 10:49 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Ilari Pihlajisto Sat Nov 05, 2011 11:17 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sun Nov 06, 2011 12:06 am
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 8:00 am
                              Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sun Nov 06, 2011 10:20 am
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:10 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads