Something new coming down the highway

[Mike Sherwin]

It may be a new Diablo Countach or maybe just an old clunky Geo Metro but it is new as far as chess engines go.

About 30 years ago I had an idea to make a statistical driven search. I got part way done but had to give up on the idea because of life circumstances. When I started to program chess again around 2003 I decided on writing a more conventional chess engine that eventually turned into RomiChess. But, I always wanted to pursue my original idea. Then my mom came down with Alzheimer's and all chess activity came to a screeching halt as I had to take care of her. However, when my own health started to suffer from the strain of taking care of my mom I had no choice but to put her in a nursing home. Shortly after that I was shoveling snow and had a heart attack. I managed to get back into the house but could not make it to the phone. I must have laid there for a very long time because when I came to I had pissed and s__t myself. I was able to make it into the bathroom and clean myself off. Then I went and got into bed. I slept 20 hours a day for I do not know how long. No, I never went to the hospital. Then I remembered that my step father had a large stash of coumadin so I started to take one twice a week. Slowly my strength started to return. I lost a ton of weight.

Anyway, I have been struggling ever since to program chess once more. I would give up and then decide, no I do not want to give up. Now after all these years (thanks to help by TommyTC) I have written a very minimal example of a statistical driven search. It is not as far along as what I had 30 years ago. Thirty years ago it self played 1 e2e4 e7e5 2 f1c4 f8c5 3 g1f3 b8c6, etc. And it could more than hold its own against CM5000. I don't remember everything I did back then. It will be a rediscovery process. It understood king safety which means that I must have included checks in the statistics. This is so far the most basic idea of counting beta cutoffs and nothing more. So just consider this very first self played game as a first peek! Understand that this engine, Bricabrac, has absolutely no evaluation function whatsoever!!! Enough blah blah blah, here is the game.
[pgn][Event "Computer chess game"]
[Site "DESKTOP-HFVHK2B"]
[Date "2020.10.01"]
[Round "?"]
[White "mjshe"]
[Black "mjshe"]
[Result "*"]
[BlackElo "2400"]
[ECO "D00"]
[Opening "Hodgson Attack"]
[Time "10:15:03"]
[Variation "2...f6"]
[WhiteElo "2400"]
[TimeControl "40/7200:40/7200:7200"]
[Termination "unterminated"]
[PlyCount "65"]
[WhiteType "human"]
[BlackType "human"]

1. d4 d5 2. Bg5 f6 3. Bf4 g5 4. Bg3 f5 5. f4 Bh6 6. e3 g4 7. Bb5+ Bd7 8.
Qd3 b6 9. Nd2 Bxb5 10. Qxb5+ Qd7 11. a4 Qxb5 12. axb5 c5 13. Ke2 c4 14. h4
a6 15. b4 Nf6 16. h5 Ne4 17. Nxe4 fxe4 18. f5 Rf8 19. f6 Rxf6 20. Be5 Rf8
21. g3 Nd7 22. Rxa6 Rxa6 23. bxa6 b5 24. c3 Rf5 25. a7 Nb6 26. Bc7 Na8 27.
Be5 Kd7 28. Rh2 Rf8 29. Rf2 Rxf2+ 30. Kxf2 Ke6 31. Ke2 Kd7 32. Kd2 Ke6 33.
Ke2 *
[/pgn]
P.S. The search may still have bugs and probably does.

Fixed some bugs but did not improve the data collection yet. That is next.
[pgn][Event "Computer chess game"]
[Site "DESKTOP-HFVHK2B"]
[Date "2020.10.06"]
[Round "?"]
[White "mjshe"]
[Black "mjshe"]
[Result "*"]
[BlackElo "2400"]
[ECO "B02"]
[Opening "Alekhine Defence"]
[Time "19:06:18"]
[WhiteElo "2400"]
[TimeControl "120+6"]
[Termination "unterminated"]
[PlyCount "36"]
[WhiteType "human"]
[BlackType "human"]

1. e4 Nf6 2. Qf3 c5 3. Bc4 Qc7 4. Bb5 Qd6 5. Qf5 Kd8 6. Be2 Nc6 7. c3 h5 8.
b3 b6 9. Bf3 Qd3 10. Qf4 g6 11. Nh3 Bh6 12. Qh4 Ne5 13. Ng1 Ba6 14. c4 g5
15. Qg3 Qd4 16. Nc3 h4 17. Nge2 Nd3+ 18. Kf1 hxg3 *
[/pgn]
So not great play yet. That will come. But play is very different after bug fixes.

[TommyTC]

Hi Mike,

With your latest code, I compiled and entered "go" as the first command. A bad thing happened

I started the program again, and entered "sd" all by itself. A bad thing happened

If you enter "sd abc", a bad thing happens

Even though this is 2020, some of us (including me) have difficulty with C and C++ and something as "simple" as getting input from the keyboard into variables within a program.

Also, I wasn't real familiar with "using namespace std;", but my first Google searches seem to indicate it isn't the best programming practice.

A hint to solving the first problem is contained within the second and third problems I listed above.

I will start looking at your code "published" on Oct 6 over the coming days.

Tommy

[Mike Sherwin]

Hi Mike,

With your latest code, I compiled and entered "go" as the first command. A bad thing happened

I started the program again, and entered "sd" all by itself. A bad thing happened

If you enter "sd abc", a bad thing happens

Even though this is 2020, some of us (including me) have difficulty with C and C++ and something as "simple" as getting input from the keyboard into variables within a program.

Also, I wasn't real familiar with "using namespace std;", but my first Google searches seem to indicate it isn't the best programming practice.

A hint to solving the first problem is contained within the second and third problems I listed above.

I will start looking at your code "published" on Oct 6 over the coming days.

Tommy

Oh hi Tommy, Glad to see you here! Yes I have not bulletproofed the input code yet. But, that is on my todo list. Everything is so difficult for me that I just want to get things working as simple as I can so I can get the next thing working. But since you pointed that out I'll do it next!

THANKS
Mike

Here is Bricabrac's first complete self played game. I added some more stats. I think that they helped but I have no idea what the correct values for each should be so it is going to take a long time to tune them. If I remember 30 years ago I think that I remember having null move and the stats for null move failures helped a lot!

I'll be forking this engine to two, one more traditional and one statistical. Bricabrac will remain the statistical engine. That is why I named it Bricabrac, "miscellaneous small articles collected for their antiquarian, sentimental, decorative, or other interest". As in collecting miscellaneous stats because it is interesting.
[pgn][Event "Computer chess game"]
[Site "DESKTOP-HFVHK2B"]
[Date "2020.10.06"]
[Round "?"]
[White "mjshe"]
[Black "mjshe"]
[Result "0-1"]
[BlackElo "2400"]
[ECO "A00"]
[Opening "Van Kruij's Opening"]
[Time "20:22:36"]
[Variation "1...d5 2.e3"]
[WhiteElo "2400"]
[TimeControl "120+6"]
[Termination "normal"]
[PlyCount "162"]
[WhiteType "human"]
[BlackType "human"]

1. e3 d5 2. Bb5+ Bd7 3. Qe2 b6 4. Qd3 Bxb5 5. Qxb5+ Qd7 6. a4 Qxb5 7. axb5
Nd7 8. Nf3 h6 9. Nd4 Nb8 10. c4 e6 11. Ra2 a6 12. Nc2 c5 13. bxc6 Nxc6 14.
Nba3 Na7 15. b3 Bc5 16. Ra1 dxc4 17. Nxc4 b5 18. Ne5 Bd6 19. Nd3 Nc6 20.
Nd4 Nxd4 21. exd4 g6 22. Ra5 Ne7 23. Nc5 Bxc5 24. dxc5 Ng8 25. Bb2 Rh7 26.
b4 Ne7 27. Ra3 f5 28. Re3 Nd5 29. Re5 Rha7 30. f3 Nf4 31. Kf1 Re7 32. Bc3
h5 33. g3 Nd5 34. h4 f4 35. Rh2 fxg3 36. Rg2 a5 37. bxa5 Nxc3 38. dxc3 Rxa5
39. Re1 Ra3 40. c6 Kf8 41. Re3 Ra1+ 42. Re1 Rxe1+ 43. Kxe1 Kf7 44. Kd1 Ra7
45. f4 Ra5 46. Kc1 Kg7 47. Kb2 Ra7 48. Re2 Kf7 49. Re4 g2 50. Re1 Ra5 51.
Rg1 Ra6 52. Kb3 Kg7 53. Kb4 Rxc6 54. Rxg2 Rc8 55. Re2 Kf7 56. Rd2 Kf8 57.
Rd7 Rc4+ 58. Kxb5 Rxf4 59. Rd8+ Ke7 60. Rd4 Rxd4 61. cxd4 Kd8 62. Kc6 Ke8
63. Kd6 Kf7 64. Kd7 Kf6 65. Kd6 Kf7 66. Kd7 Kf6 67. Kd6 Kf5 68. d5 exd5 69.
Kxd5 Kf4 70. Ke6 g5 71. hxg5 Kxg5 72. Ke5 h4 73. Ke6 h3 74. Kf7 h2 75. Ke6
h1=Q 76. Ke5 Qb7 77. Ke6 Qb5 78. Kd6 Kf6 79. Kc7 Ke6 80. Kc8 Kd6 81. Kd8
Qd7# 0-1
[/pgn]

[duncan]

I managed to get back into the house but could not make it to the phone. I must have laid there for a very long time because when I came to I had pissed and s__t myself. I was able to make it into the bathroom and clean myself off. Then I went and got into bed. I slept 20 hours a day for I do not know how long. No, I never went to the hospital. Then I remembered that my step father had a large stash of coumadin so I started to take one twice a week. Slowly my strength started to return. I lost a ton of weight.

Have you still not seen a doctor?

[Mike Sherwin]

I managed to get back into the house but could not make it to the phone. I must have laid there for a very long time because when I came to I had pissed and s__t myself. I was able to make it into the bathroom and clean myself off. Then I went and got into bed. I slept 20 hours a day for I do not know how long. No, I never went to the hospital. Then I remembered that my step father had a large stash of coumadin so I started to take one twice a week. Slowly my strength started to return. I lost a ton of weight.

Have you still not seen a doctor?

I believe that was in late 2010. My mom made me go to the doctor for a checkup when I was 20 in 1977. Then she made me go again when I was 40. I told her that I'd go again when I was 60. In 2012 my mom died and I broke my word and did not go to the doctor when I was 60. Maybe I'll go if I make it to 80, NOT . I was born with a hole in my heart that was at that time not fixable. The doctors said that I'd probably not make to 40. When I was in my 20's I played 3 set tennis matches. If a match went to the third set I always lost the match. When I went to the doctor when I was 20 he told me if I kept playing tennis I wouldn't live long. I kept playing tennis! I've been staring death in the face all my life and have beaten the odds. The game of life has to end sometime and given the hand I was dealt I played a real good game. So to me seeing the doctor is non consequential, or something like that.

[Mike Sherwin]

Fixed a bug in RootSearch.

Well I was making a mess of the statistics so it became clear that I had to backup a bit. All the stat collection was commented out except for the CHECKMATE stat. I figured let's just start with the goal of the game and build it up one stat at a time. It is still not a great game but toward the end of the game white started the checkmating sequence long before it could see actual checkmate! And now it has become a lot more interesting. Search depth was set at 8.
[pgn][Event "Computer chess game"]
[Site "DESKTOP-HFVHK2B"]
[Date "2020.10.07"]
[Round "?"]
[White "mjshe"]
[Black "mjshe"]
[Result "1-0"]
[BlackElo "2400"]
[ECO "A00"]
[Opening "Van Kruij's Opening"]
[Time "10:32:54"]
[WhiteElo "2400"]
[TimeControl "120+6"]
[Termination "normal"]
[PlyCount "53"]
[WhiteType "human"]
[BlackType "human"]

1. e3 Nf6 2. Bc4 e6 3. Qf3 d6 4. g4 a5 5. Nh3 h5 6. g5 d5 7. Na3 dxc4 8.
gxf6 gxf6 9. Nf4 h4 10. Nxc4 h3 11. a3 f5 12. Ne5 Qd6 13. d4 Bg7 14. Nfd3
Rh4 15. Nc4 Qa6 16. Rg1 Bf8 17. b3 Qa7 18. Nf4 b5 19. Ne5 a4 20. Rg8 Qa5+
21. Bd2 Rxf4 22. exf4 Qa6 23. Qh5 Ke7 24. Qxf7+ Kd6 25. Qxf8+ Kd5 26. Qc5+
Ke4 27. f3# 1-0
[/pgn]

[Mike Sherwin]

RED ALERT! ... RED ALERT! ... RED ALERT!
Search Depth 8, added scaled back Q stats
[pgn][Event "Computer chess game"]
[Site "DESKTOP-HFVHK2B"]
[Date "2020.10.07"]
[Round "?"]
[White "mjshe"]
[Black "mjshe"]
[Result "*"]
[BlackElo "2400"]
[ECO "A00"]
[Opening "Van Kruij's Opening"]
[Time "19:23:14"]
[Variation "Krejcik Variation"]
[WhiteElo "2400"]
[TimeControl "120+6"]
[Termination "unterminated"]
[PlyCount "34"]
[WhiteType "human"]
[BlackType "human"]

1. e3 Nf6 2. Bc4 e6 3. Nc3 Na6 4. h4 h5 5. d4 Ng4 6. d5 Bc5 7. Bb3 c6 8.
dxe6 fxe6 9. Nf3 Be7 10. Ke2 O-O 11. a3 c5 12. g3 Rxf3 13. Kxf3 Qf8+ 14.
Kg2 Qxf2+ 15. Kh3 Qf6 16. Rf1 Nf2+ 17. Rxf2 Qxf2 *
[/pgn]
On move 12 ... Rxf3 SF12 third choice at depth 8
On move 13 ... Qf8 SF12 first choice at depth 5, not locked in until depth 21
On move 15 ... Qf6 SF12 third choice at depth 19 (15 ... Qf7 though)

This from a material only searcher that collects one CHECKMATE stat!

At some point:

QUESTION
How is calculus used in statistics, and how important is it for a career/theoretical statistician? For an applied statistician? For a practitioner of statistics?

ANSWER
Alvin Grissom II, Computational Linguist
Answered March 25, 2014 · Upvoted by Peter Flom, Independent statistical consultant for researchers in behavioral, social and medical sciences and David Joyce, Professor Emeritus of Mathematics at Clark University
Originally Answered: How is calculus used in statistics?
It depends on the kind of statistics.

At a very basic level, one example of calculus being used in statistics involves integrating over sections of a probability distribution. This will yield the cumulative probability over those values, which is simply the area under the curve. The indefinite integral of the probability distribution will yield a cumulative distribution function, which is simply a function that yields the cumulative probability for an arbitrary value (and thus is always non-decreasing).

As with other curves, you can use the standard calculus techniques to find inflection points, maxima, minima, etc.

Essentially, integrals will take the place of finite sums whenever continuous values are used: for example, "marginalizing out" values.

Calculus is extremely prevalent in more advanced statistical applications. Often, this is the application of a basic concept to a more complex function, however.