New Engine

[Zlaire]

I really have no way of proving or showing off my programming talent to prospective employers. I'm hoping to release a beta version of a chess engine I wrote

There's the catch. I don't intend to release my source, but I'm aware of the GPL on Stockfish

Are you going to show your source to your employees? If so, what's the point of keeping it private?

If you're aware of GPL, why do you even ask if it's ok to copy code? (which, by the GPL license, would obviously require you to publish your code)

I'd say keep your source open while you learn and you can copy freely from wherever and be happy.


I guess I don't get this post at all. To sum it up it would be something like:

"Hi, I want to impress people and I've copied this and that, but I have no intention of giving anything back to the community. Is that ok?"

The answer is apparently no.

[ChrisFlorin]

My honor is worth more than your accusations. I wrote some compression algorithms a few years ago, but everything I read about compression indicated it was a patent minefield. I guess in the last year or two most of you think that nobody can have original work anymore. Even cloners accuse other people of cloning their clones. It's funny to me the person willing to help me the most was one of the people who I actually copied from. I really had no intention of releasing an engine, or any program, which violated any copyright, patent, or intellectual property of anybody. There are so many tutorials and example code, that sometimes it's hard to tell what is free domain and what isn't.

Code: Select all

/// Check for console input. Original code from Beowulf and Olithink
//	I copied it from Stockfish.

#ifndef _WIN32

int data_available()
{
  fd_set readfds;
  struct timeval  timeout;

  FD_ZERO(&readfds);
  FD_SET(fileno(stdin), &readfds);
  timeout.tv_sec = 0; // Set to timeout immediately
  timeout.tv_usec = 0;
  select(16, &readfds, 0, 0, &timeout);

  return (FD_ISSET(fileno(stdin), &readfds));
}

#else

int data_available()
{
    static HANDLE inh = NULL;
    static bool usePipe;
    INPUT_RECORD rec[256];
    DWORD dw, recCnt;

    if (!inh)
    {
        inh = GetStdHandle(STD_INPUT_HANDLE);
        usePipe = !GetConsoleMode(inh, &dw);
        if (!usePipe)
        {
            SetConsoleMode(inh, dw & ~(ENABLE_MOUSE_INPUT | ENABLE_WINDOW_INPUT));
            FlushConsoleInputBuffer(inh);
        }
    }

    // If we're running under XBoard then we can't use PeekConsoleInput() as
    // the input commands are sent to us directly over the internal pipe.
    if (usePipe)
        return PeekNamedPipe(inh, NULL, 0, NULL, &dw, NULL) ? dw : 1;

    // Count the number of unread input records, including keyboard,
    // mouse, and window-resizing input records.
    GetNumberOfConsoleInputEvents(inh, &dw);

    // Read data from console without removing it from the buffer
    if (dw <= 0 || !PeekConsoleInput(inh, rec, cf_min(dw, 256), &recCnt))
        return 0;

    // Search for at least one keyboard event
    for (DWORD i = 0; i < recCnt; i++)
        if (rec[i].EventType == KEY_EVENT)
            return 1;

    return 0;
}

#endif

I found this code snippet in my own code that I intended to use when I implemented the infinite analysis feature, which I never got around to. Stockfish took it from Beowulf/Olithink, who may have gotten it from somewhere else. Stockfish gives credit in its source, but who owns this code? And what license is it under? Where is this GPL license line drawn? What about everything on chessprogramming.wikispaces.com? I could easily rewrite the SEE function and a pawn evaluation in assembly and be done with it, but now that I've made this post, I'll be permanently labeled a copier and cheat. My honor means more to me than anything; I'm sorry that I intended on making some money possibly selling something while I was going to school rather than making $7.99/hour cooking food.

My original intention was to see what everybody thought on the difference between that of my code and Crafty/Stockfish. My SEE function looks pretty different, especially the part where I save the last valuable attacker for a given square. I haven't seen that ANYWHERE, but since I posted it here, what's the license? Is it included with the rest of Blackmail in GPLv3 released below? Maybe there's a few other small pieces of code I've forgotten about.

Code: Select all

		if ( AreStringsEqual(command, "XBOARD") ) {
			comm->protocol = PROTOCOL_XBOARD;
			GlobalMode = GLOBALMODE_ENGINE;

			return GLOBALSTATE_NOOP;
		} else if ( AreStringsEqual(command, "USER") ) {
			comm->protocol = PROTOCOL_USER;
			GlobalMode = GLOBALMODE_ENGINE;

			RecvCommunication_User(comm, "new");

			return GLOBALSTATE_NOOP;
		} else if ( AreStringsEqual(command, "SERVER") ) {
			comm->protocol = PROTOCOL_SERVER;
			GlobalMode = GLOBALMODE_SERVER;

			return GLOBALSTATE_NOOP;
		} else if ( AreStringsEqual(command, "QUIT") ) {
			return GLOBALSTATE_QUIT;
		}

What about snippets like this? How many different ways can you seriously think of to check which command a user/interface has sent to an engine? There's this way, in Crafty and Beowulf, and then there's the Stockfish way, which essentially works like a stream tokenizer and makes more sense for the uci protocol.

I've also read many articles written by others, such as Larry Kaufman (something about pawns if I remember right, including a game lost by black due to a tripled pawn), the Mediocre chess blog (the author of which has posted here), and countless (I mean countless, I could never give an entire list) others. I have read every thread on this forum, save the very long ones, especially where arguments broke out. I know enough to know that I don't understand butterfly bitboards, and don't like 0x88 move generation over the bitboard approach I'm currently using. There was a post about a service Intel provided where you could test thread concurrancy on a multithreaded application which would run for an hour on their test server. But the service doesn't exist anymore

Anyway, the source code for my engine can be downloaded from the following link. The GPLv3 is included in the .zip file

http://www.chrisflorin.com/blackmail/source.zip

To use Blackmail, type the user command similar to using xboard or uci (uci is not supported). Using Blackmail in user mode is similar to xboard, you must use "usermove" before you type your move. Most other xboard commands work in user mode, but there are a few others:

checks: displays all moves which are checks
perft [depth]: displays the perft count for the given position
perftsplit [depth]: displays the perft count for each move in the given position
score: returns the current eval (divide by 65536 to normalize a pawn to 1.00)
see [move]: returns the see score for a given move (type a move just like usermove)
analyzeall: displays the exact score and pv line for all moves for the given position. uses the same time clock as a normal search would

The only clock types which work are sd (search depth) and st (search time), which are used exactly the same as in winboard. The defaults are 30 and 15 respectively.

The hashtable defaults to 8M entries, which is 128M, and 64K pawn entries. There is no command to change this; it must be changed in the source. Chess Engine VII.cpp: 78.

If you run a few games overnight or something, it crashes in Arena, but only every few games. When doing Blackmail vs. Blackmail tests, they would both crash or neither would crash; it's weird and was never only one of them. I could never figure out why and have been dealing with it.

I thought it was pretty cool that Onno Garms posted an idea of having an engine process PV nodes, ALL nodes, and CUT nodes differently, but he didn't know of an engine that did. Blackmail does. I wanted to tell him myself. You'll find the Bad Move Reduction in Blackmail commented out, it didn't work out too well.

You'll find some code that I started making game databases, opening books, and end game table bases. I was satisfied with the thinking aspect of my engine, and had intended starting auto tuning, but we'll see if I continue. Perhaps I'll find another project, but I don't know. Artificial Intelligence is one of my passions, and if this is anything like compression, I'm out. It's funny how I was warned this would happen, but refused to listen. I told myself that with something original, I could pull through. Maybe I'll just go back to assembly programming where example code doesn't even exist especially since once you see something, it cannot be unseen.

So, here's my contribution to the community. Thanks for the help. Special thanks goes to the authors of Beowulf, Crafty and Stockfish for their code. Special thanks also goes out to everybody who researches for a living and posts their results free for all to see. Also to authors of articles and code on chessprogramming.wikispaces.com, and anybody involved on helping people learn and understand chess programming on these forums. Oh, and Rebel for move ordering and evaluation ideas. I'm sure I'm missing a few, it's late and I have to get up for work in 6 hours.

There is nothing regarding Ippolit in Blackmail; I looked at the source for 3 minutes and decided it was not in my best interest to get anything from it. Sorry guys, but the code just looked ugly to me.

[Evert]

There are so many tutorials and example code, that sometimes it's hard to tell what is free domain and what isn't.

My understanding is that this is actually pretty simple: whoever writes the code owns the copyright to the code. In order for someone else to use the code, they need a licence that states what they can do with it. If there is no licence specified, then you cannot use the code.

I found this code snippet in my own code that I intended to use when I implemented the infinite analysis feature, which I never got around to. Stockfish took it from Beowulf/Olithink, who may have gotten it from somewhere else. Stockfish gives credit in its source, but who owns this code? And what license is it under?

That'll be the GPL.

What about everything on chessprogramming.wikispaces.com?

That's a good question. My guess is that my first answer applies: you cannot use the code directly without a licence. My advice is to treat any such code as pseudo-code, which illustrates the idea. Then you write your own (which, for some trivial code, may end up looking the same).

but now that I've made this post, I'll be permanently labeled a copier and cheat.

Why?

[mcostalba]

My original intention was to see what everybody thought on the difference between that of my code and Crafty/Stockfish. My SEE function looks pretty different, especially the part where I save the last valuable attacker for a given square.

What if behind an attacking queen there is a bishop of the same color ? Are you sure your trick is compatible with discovered x-ray attackers ?

Anyway, the source code for my engine can be downloaded from the following link. The GPLv3 is included in the .zip file

http://www.chrisflorin.com/blackmail/source.zip

Thanks! I will give it a look.

I thought it was pretty cool that Onno Garms posted an idea of having an engine process PV nodes, ALL nodes, and CUT nodes differently, but he didn't know of an engine that did.

This never worked for us, but is considered common practice, for instance see OpenCritter or even Ippolit, although I agree with you sources are (propousely) obfuscated and a pain to read.

I'm sure I'm missing a few, it's late and I have to get up for work in 6 hours.

My 2 cents, don't take too seriously this chess engine programming, if it is fun for you ok, but if it is such an heavy effort it really doesn't worth the pain. If you are aiming to find a god job in software development I'd really think that there are much better ways to interest possible employers than showing them your shining new chess engine.

[Zlaire]

now that I've made this post, I'll be permanently labeled a copier and cheat.

Erm no? Far from it. As long as you're open with what you're doing no one is going to accuse you of anything.

What's been brought up here is the way the GPL license works. I.e. if you're not going to release your code, don't use GPL licensed code. Simple as that, and I believe that was your original question (although you seem to disagree with the answers).

-

I still say keep your source open, copy to your heart's content, and then put your personal touch on your engine.

Or, keep your engine closed and use only concepts and ideas. That is don't copy anything, no matter how much you alter it afterwards. This can be a fun way to develop too, and you're free to use your engine however you like.

[lucasart]

"Hi, I want to impress people and I've copied this and that, but I have no intention of giving anything back to the community. Is that ok?"

That's a very good summary of why GPL exists. You cannot just take from the free software community and give nothing in return. The free software community is made of thousands of people who spent countless hours on developing free softwares. And the reason why they do it, is precisely to protect the freedom of the community. So if you think it's ok to take everything from the community and turn it into proprietary software, then you too deserve to join the hall of shame of computer chess programming.

[Don]

Chris,

I do not think your honor is in question. You are doing fine.

You are victim of the pro-clone propaganda, they hammer you with FUD in order to make it seem that these issues are too complicated to resolve and that everything needs to be changed. In a sense they have won if you now feel frustrated.

There is a small movement by a few to have no rules or restrictions at all about taking other peoples software so they are very quick to highlight and exaggerate the issues you are confused about - it's the standard straw man argument used by politicians - misrepresent the other persons position so that it's easier to attack.

I am not aware of any computer chess idea that is patented. These ideas are all in the public domain and the do not belong to anyone in particular. Getting an idea does not make you "owner" of the idea. However CODE is copyrighted, not patented. But keep in mind that copyright and patents are two different things. This is a big source of confusion. If you don't understand that distinction, you will be victim of all the silly arguments which are based on refusing to acknowledge the difference.

The straw man they have constructed in a nutshell (which I am going to exaggerate to make the point) is that if you oppose plagiarism, it's because you are old-fashioned and petty and do not want others to make progress or share ideas. Of course that is not our position at all, but that is the straw man they like to attack.

Please don't let them confuse you or hold you back from having fun with computer chess. We need more diversity, not less, and I hope to see some good stuff from you.


Don

[ChrisFlorin]

I do not think your honor is in question. You are doing fine.

Thank you.

You are victim of the pro-clone propaganda, they hammer you with FUD in order to make it seem that these issues are too complicated to resolve and that everything needs to be changed. In a sense they have won if you now feel frustrated.

Yes, last night, I was very tired and frustrated.

I am not aware of any computer chess idea that is patented. These ideas are all in the public domain and the do not belong to anyone in particular. Getting an idea does not make you "owner" of the idea.

Thank god, but how would someone "own" an idea should they have one?

However CODE is copyrighted, not patented. But keep in mind that copyright and patents are two different things. This is a big source of confusion. If you don't understand that distinction, you will be victim of all the silly arguments which are based on refusing to acknowledge the difference.

I do not understand this distinction at all, perhaps this is a problem.

The straw man they have constructed in a nutshell (which I am going to exaggerate to make the point) is that if you oppose plagiarism, it's because you are old-fashioned and petty and do not want others to make progress or share ideas. Of course that is not our position at all, but that is the straw man they like to attack.

I don't oppose plagiarism, but I sort of assumed some things that I guess I shouldn't have assumed. In college (and high school a little) composition classes, they are very strict with plagiarism and giving credit to original authors, but nobody teaches this in engineering. I know enough to know that it's obviously illegal to copy computer programs, but the line is blurry with respect to computer code. Yes, Stockfish is GPL, blah blah blah, but what about tutorials? Example code? Forum posts asking how to do something? MSDN code? What if there's just not another way to do something that is easy and intuitive? For example, the code I use to disable buffering that I originally found in Beowulf, and stumbled across in the MSDN looking for something else completely unrelated?

Please don't let them confuse you or hold you back from having fun with computer chess.

Confuse me, yes. Hold me back? We'll see. My original intention with respect to releasing code was to release a "shell" of an engine, complete with protocol communication, board representation, move generation, the most basic PVS search, and material evaluation, etc. i.e. everything required to get a basic engine working under another gui.

I have rewritten Blackmail from scratch 7 times since my hard drive failure. This 7th time, I didn't even write a search function, evaluation, anything until my perft times beat those of Stockfish. And I still made improvements to it during search/evaluation development. Every step of the way, if anything slowed Blackmail too much, I scrapped it or rewrote it several times. Blackmail still has no concept of center/square control simply because I cannot implement it efficiently.

But, I never wanted to be accused of cloning. I had a dream last night in my frustration, Vasij accused me of cloning Rybka in a tournament, and I pulled out a usb drive and told him to put up or shut up. I don't even know what he looks like, it's funny how the brain fills information in.

We need more diversity, not less, and

As does any genetic algorithm

I hope to see some good stuff from you.

You will.

The code has been pulled from download since technically, I guess I can't even release Blackmail under GPL until I resolve all of the issues with example code I copied, mostly from chessprogramming.wikispaces.com. This site should really come with some disclaimers. Anyway, I'll be working on that. Any code that I simply get out of my head, such as the SEE code will be rewritten in assembly. I fully intend on releasing example code for public domain; I simply don't understand or see the point in copyrighting "tutorial class" code. I would love to make a copy of my code/engine to the authors of Crafty, Stockfish and Beowulf to look over; not only for integrity purposes, but perhaps they can learn something from an engine written by someone who has reviewed their code pretty thoroughly.

Just be careful, and don't copy the code and say it is your own! Clones are frowned upon by the computer chess community as a whole.

I found this in the Getting Started section, but am too lazy to look comprehensively. I'm not convinced this is warning enough to not copy code from the website to the average programmer. Although it does warn against clones, not every programmer is aware of code copyright laws as I have demonstrated. There should be an ethics for engineers class or something

Anyway, when I make the required changes for release, I'll let you guys know. Until then, I would recommend anybody who downloaded my code to delete it and not distribute it since it contains illegal code and distribution of it would be illegal. Hopefully the FBI isn't watching.

[ChrisFlorin]

What if behind an attacking queen there is a bishop of the same color ? Are you sure your trick is compatible with discovered x-ray attackers ?

No, I'm not sure of this. I changed it to not save the pt, and to set pt to PAWN every iteration and am currently testing an elo change. Right now, the version I wrote is winning 159.5/276 105-62-109 in a test I've been running since you raised this question last night. Perhaps Blackmail benefits from not looking at discovered x-ray attacks. I'll review the code a little more in a debugger when I get a chance and let you know.

Thanks! I will give it a look.

I took the download down. PM me when you're ready and able to review it and I'll make sure you can still look at it. Your word that it will remain private will be enough for me.

I thought it was pretty cool that Onno Garms posted an idea of having an engine process PV nodes, ALL nodes, and CUT nodes differently, but he didn't know of an engine that did.

This never worked for us, but is considered common practice, for instance see OpenCritter or even Ippolit, although I agree with you sources are (propousely) obfuscated and a pain to read.

I purposely started Blackmail's search algorithm using this method although I'll admit there isn't much difference in the way Blackmail handles CUT nodes and ALL nodes. Blackmail only performs IID on CUT nodes. Also, Blackmail only saves PV nodes, and actual CUT nodes in the hashtable (even if the nodetype is CUT, if it does not actually produce a cutoff, then it is not saved).

My 2 cents, don't take too seriously this chess engine programming, if it is fun for you ok, but if it is such an heavy effort it really doesn't worth the pain.

It is such a heavy effort sometimes, but it's worth the pain in my opinion. Just sitting back watching it play on its own in Arena is well worth everything I've put into it and helps me remember that it will be worth more future development. Plus it's fun to watch; it sometimes puts my roommate to sleep

If you are aiming to find a god job in software development I'd really think that there are much better ways to interest possible employers than showing them your shining new chess engine.

Maybe, but with nothing but 12 years of restaurant cooking/management on my resume, it isn't easy. I worked for my dad a little, but he's my dad. I have a Flash game I wrote somewhere, and a small website written in php/mysql just to say I can really. I actually did have an interview last year, literally a month after my hard drive crash. My interviewer asked me about one of my projects (a 3d engine), and I felt silly and could see the look on his face with my "excuses." The interview promptly ended.

[ChrisFlorin]

What about everything on chessprogramming.wikispaces.com?

That's a good question. My guess is that my first answer applies: you cannot use the code directly without a licence. My advice is to treat any such code as pseudo-code, which illustrates the idea. Then you write your own (which, for some trivial code, may end up looking the same).

Blackmail:

Code: Select all

	//4) Perform mate-distance pruning
    alpha = cf_max(-INFINITY + currentdepth, alpha);
    beta = cf_min(INFINITY - currentdepth, beta);

	if (alpha >= beta) {
        return alpha;
	}

Stockfish:

Code: Select all

    // Step 3. Mate distance pruning
    alpha = Max(value_mated_in(ply), alpha);
    beta = Min(value_mate_in(ply+1), beta);
    if (alpha >= beta)
        return alpha;

Glaurang (pv-nodes): (Tord posted this in March 2006; I have not checked the actual source of Glaurang)

Code: Select all

    alpha = Max(value_mated_in(ply), alpha); 
    beta = Min(value_mate_in(ply+1), beta); 
    if(alpha >= beta) 
      return alpha;

Glaurang (non-pv nodes):

Code: Select all

    if(value_mated_in(ply) >= beta) 
      return beta; 
    if(value_mate_in(ply+1) < beta) 
      return beta-1;

Zach Wegner:

Code: Select all

/* mate-distance pruning */ 
        sb->alpha = MAX(sb->alpha, -MATE + sb->ply); 
        sb->beta = MIN(sb->beta, MATE - sb->ply - 1); 
        if (sb->alpha >= sb->beta) 
            RETURN(sb->alpha);

chessprogramming.wikispaces.com:

Code: Select all

mating_value = SCORE_MATE - RootDistance;
 
if (mating_value < beta) {
   beta = mating_value;
   if (alpha >= mating_value) return mating_value;
}

mating_value = -SCORE_MATE + RootDistance;
 
if (mating_value > alpha) {
   alpha = mating_value;
   if (beta <= mating_value) return mating_value;
}

Miguel A. Ballicora:

Code: Select all

   alpha = adjust_in (alpha); 
   beta  = adjust_in (beta); 

   /*---------------------------* 
      Unreachable Check Mates 
    *---------------------------*/ 

   if (alpha >= (MATE_VALUE-1)) { 
      return adjust_out (alpha); 
   } 

   /* cut off, I cannot be mated on time faster than beta */    
   if (beta <= (-MATE_VALUE + 2) && !position_isincheck(po)) { 
      return adjust_out (beta);    
   }
 
static eval_t 
adjust_in (eval_t x) 
{ 
   ASSERT (-INFINITY_VALUE <= x && x <= INFINITY_VALUE); 
   if (MATE100_VALUE < x && x < MATE_VALUE) 
      return x + 1; 
   if (-MATE_VALUE < x && x < -MATE100_VALUE) 
      return x - 1; 
   return x; 
} 

static eval_t 
adjust_out (eval_t x) 
{ 
   ASSERT (-INFINITY_VALUE <= x && x <= INFINITY_VALUE); 
   if (MATE100_VALUE < x && x < INFINITY_VALUE) 
      return x - 1; 
   if (-INFINITY_VALUE < x && x < -MATE100_VALUE) 
      return x + 1; 
   return x; 
} 

Tord did actually mention to Zach that his code was similar to which Zach replied

I've wanted to add mate_in macros, I might not anymore now that I see you have them.

Blackmail uses the SearchStack concept, but not as heavily as Stockfish, just for HashMove, ExcludedMove, etc. but I suppose Zach uses it also for alpha and beta scores. I'm not sure which engine he has written, but is his GPL also? Is he not allowed to use mate_in/mated_in macros? If his engine IS GPL, why wouldn't he use the macros? Am I allowed to use them? It was kinda in the back of my mind. How different does the code have to be?

Code: Select all

	//4) Perform mate-distance pruning
	if (cf_max(-INFINITY + currentdepth, alpha) >= cf_min(INFINITY - currentdepth, beta)) {
        return alpha;
	} else {
        alpha = cf_max(-INFINITY + currentdepth, alpha);
        beta = cf_min(INFINITY - currentdepth, beta);
    }

Is this sufficient? I'm disinclined to use if statements to change alpha and beta because a branchless ? is faster. I use cf_max and cf_min rather than max and min because my first macros conflicted with one of the main header files and I didn't want to deal with it; that isn't my way of getting around copying code. I will say Tord's idea of processing PV and nonPV nodes differently is pretty cool and I will probably change Blackmail's implementation depending on which node type a particular node is. I also hate if/for/while statements without brackets; I've created some nasty and hard to find bugs while not using brackets.