Honestly, I don’t see much diversity, I consider it more like variations on a theme. The same basic algorithms are used by everyone, they suffer from the same nullmove weaknesses etc. You can call it inbreeding and I can call it survival of the fittest ideas or equivalently, convergence towards the optimal solution. There are several factors contributing to this, one is probably the nature of the problem, but another is the flow of information. When strong programs gets disassembled, ideas get reused and programs converge. To get true diversity we need to develop independently in vacuum or at least protect the codes better.Don wrote: There is also a factor that most here are completely ignorant about which involves computer science and I believe applies here with a real vengeance. It's called the "local optimum" which means that a better solution exists, but it's probably not close to the point you are starting from. Put in everyday language it means that as long as you promote using Ivanhoe as your starting point and believe that everyone should start from that exact code and build in a straight line from that, you are not going to move towards better solutions. Learning algorithms in general struggle with that exact problem and often incest becomes an issue that exacerbates the problem. The general cure to this is "diversity."
What interests me, is the pattern we have been seeing for the last decade or so. It seems that the most successful methodology has been to build upon existing, strong and well written codes.
Maybe every 10 years we should pick the top 10 engines and only build on those New diversity can emerge over time from the same starting point, it would just be at a higher level.
I feel the same way, but objectively it’s just a very inefficient attitude from a scientific or strength/time point of view. When you take a look at all the most successful engines you really can’t say they are the work of a single individual. They have either been developed in teams, be it parallel or sequential in time, or the engines are based heavily on ideas of others through eg. disassembly.Don wrote: It's not in my own nature to do that. I told Larry once that we could probably take a program like Ivanhoe and easily add 100 ELO by implementing a lot of the ideas in Komodo and fixing the things that are obvious to us. We could also do this with Stockfish. I think Larry secretly would have liked to do this had it been legitimate, but to me it was like taking all the fun out of it. EVEN if it had been fully acceptable and ethical, I could not get satisfaction from doing it this way. I would be a fraud and I would feel that I was being dishonest. Of course I could just be up front about it and make an open source fork of the code, but then it's not my stuff. I would rather have a program that completely sucks than to have the best program out there that is basically someone else's work with my name on it.
Sometimes I wonder if our feelings on this matter could be standing in the way of progress. It doesn’t seem rational to force everyone to start from scratch and waste 2-3 years, when they could possibly jump right in and start improving an existing top engine. As long as proper credit is given to the original authors I would have no problem with it. It doesn’t seem that much different than eg. disassembling a top engine and using the ideas in your own engine. Why go through these hoops when the result is practically the same? In the latter case proper credit is often not granted making it somewhat more dishonest IMHO.