Xiangqi - MoonRiver the begining

[phhnguyen]

AMD Ryzen 7 1700 Eight-Core Processor
64 MB Hash
Timer: 40/300+1
Total 200 games Roundrobin
GUI: MoonRiver

The first four freeware-strong Xiangqi engines I could find and run safely. Fairy-Stockfish runs with and without NNUE.

[lkaufman]

How does the level of this top Xiangqi pc engine compare with the best human player? Is it like chess and shogi, where the engines are so far beyond the level of any human that only a handicap makes a competitive game, or is it still reasonably close in Xiangqi?

[phhnguyen]

How does the level of this top Xiangqi pc engine compare with the best human player? Is it like chess and shogi, where the engines are so far beyond the level of any human that only a handicap makes a competitive game, or is it still reasonably close in Xiangqi?

Since there is not any direct match between those Xiangqi engines and the best human player, I make a guess:
- a commercial Xiangqi engine won the best human player (or a grandmaster?) about 10 years ago
- with NNUE tech, a good freeware engine now could beat the best commercial ones without NNUE. For example, SA Chess 1.6 (it is a freeware but the next version 1.8 become commercial) used to be 2nd place in a software contest in China about 5 years ago and now it has been beaten by a large margin. GGchess was a commercial too.
- thus the best freeware Xiangqi engines such as Pikafish could win the best human players with a high probability

However, a guess is just a guess. Xiangqi requires some hard knowledge that may not be studied or applied enough for engines such as opening and especially endgames. Some Xiangqi endgames may have 11 - 15 pieces - very hard to build EGTB or to code. We may wait for magic from NNUE but it may not happen.

I believe if prepared long enough and then plays with the right strategy (for example, find and use trap opening lines, convert games into winning large-piece-number-endgames), a good human player could still beat the best Xiangqi engines (regardless of commercial/freeware)!

[phhnguyen]

I made a mistake to set the option to turn NNUE off for Classic Fairy-Stockfish thus in the above test the Classic and NNUE are quite close. The author of Fairy-Stockfish said that the one with NNUE should be 500 Elo stronger than the one without (Classic).

Fixed and re-run the tournament. Look like now the result is reasonable. The Classic Fairy-Stockfish is far weaker than famous-commercial engines (SAchess and GGchess) but the NNUE one is much stronger than them all and it took second place. Pikafish seems to be number one with a large margin, compared to the following ones.

[lkaufman]

How does the level of this top Xiangqi pc engine compare with the best human player? Is it like chess and shogi, where the engines are so far beyond the level of any human that only a handicap makes a competitive game, or is it still reasonably close in Xiangqi?

Since there is not any direct match between those Xiangqi engines and the best human player, I make a guess:
- a commercial Xiangqi engine won the best human player (or a grandmaster?) about 10 years ago
- with NNUE tech, a good freeware engine now could beat the best commercial ones without NNUE. For example, SA Chess 1.6 (it is a freeware but the next version 1.8 become commercial) used to be 2nd place in a software contest in China about 5 years ago and now it has been beaten by a large margin. GGchess was a commercial too.
- thus the best freeware Xiangqi engines such as Pikafish could win the best human players with a high probability

However, a guess is just a guess. Xiangqi requires some hard knowledge that may not be studied or applied enough for engines such as opening and especially endgames. Some Xiangqi endgames may have 11 - 15 pieces - very hard to build EGTB or to code. We may wait for magic from NNUE but it may not happen.

I believe if prepared long enough and then plays with the right strategy (for example, find and use trap opening lines, convert games into winning large-piece-number-endgames), a good human player could still beat the best Xiangqi engines (regardless of commercial/freeware)!

Another question: If Pikafish plays against itself or another equal strength engine, with some source of variety such as a short opening book from GM games (or perhaps just relying on MP for variety?), what percentage of games end in draws? Of course the answer depends on time limit and number of threads, presumably rising with more time and more threads, so if you have any data on this please specify the time control and number of threads. This would probably also give us an indication of the level of play; I believe that the percentage of draws among grandmasters is similar to chess or perhaps somewhat higher than in chess, so the percentage of drawn games between equal engines can be compared to the percentage of draws among grandmasters to give an idea of the relative level of play. In chess with say 8 threads and "Rapid" time controls (like 15' + 10" or the equivalent) it's past 99%, so that's also an interesting comparison to make.

[phhnguyen]

How does the level of this top Xiangqi pc engine compare with the best human player? Is it like chess and shogi, where the engines are so far beyond the level of any human that only a handicap makes a competitive game, or is it still reasonably close in Xiangqi?

Since there is not any direct match between those Xiangqi engines and the best human player, I make a guess:
- a commercial Xiangqi engine won the best human player (or a grandmaster?) about 10 years ago
- with NNUE tech, a good freeware engine now could beat the best commercial ones without NNUE. For example, SA Chess 1.6 (it is a freeware but the next version 1.8 become commercial) used to be 2nd place in a software contest in China about 5 years ago and now it has been beaten by a large margin. GGchess was a commercial too.
- thus the best freeware Xiangqi engines such as Pikafish could win the best human players with a high probability

However, a guess is just a guess. Xiangqi requires some hard knowledge that may not be studied or applied enough for engines such as opening and especially endgames. Some Xiangqi endgames may have 11 - 15 pieces - very hard to build EGTB or to code. We may wait for magic from NNUE but it may not happen.

I believe if prepared long enough and then plays with the right strategy (for example, find and use trap opening lines, convert games into winning large-piece-number-endgames), a good human player could still beat the best Xiangqi engines (regardless of commercial/freeware)!

Another question: If Pikafish plays against itself or another equal strength engine, with some source of variety such as a short opening book from GM games (or perhaps just relying on MP for variety?), what percentage of games end in draws? Of course the answer depends on time limit and number of threads, presumably rising with more time and more threads, so if you have any data on this please specify the time control and number of threads. This would probably also give us an indication of the level of play; I believe that the percentage of draws among grandmasters is similar to chess or perhaps somewhat higher than in chess, so the percentage of drawn games between equal engines can be compared to the percentage of draws among grandmasters to give an idea of the relative level of play. In chess with say 8 threads and "Rapid" time controls (like 15' + 10" or the equivalent) it's past 99%, so that's also an interesting comparison to make.

I have found another Xiangqi engine (called CCStockfish) that is similar-level to Pikafish. The book is from good/GM players. With 200 direct-match games (Pikafish vs CCStockfish), each engine won only 2 (2 wins, 2 losses), and the rest (196 games) are drawn (timer 300/40+0.5).




High-rank Xiangqi games are quite similar to chess games about the rate of draws between 30-40%).

I have still been verifying and tuning settings for Xiangqi matches as well as finding and checking new engines. Will update you if there is any news/interesting.

[lkaufman]

I have found another Xiangqi engine (called CCStockfish) that is similar-level to Pikafish. The book is from good/GM players. With 200 direct-match games (Pikafish vs CCStockfish), each engine won only 2 (2 wins, 2 losses), and the rest (196 games) are drawn (timer 300/40+0.5).




High-rank Xiangqi games are quite similar to chess games about the rate of draws between 30-40%).

I have still been verifying and tuning settings for Xiangqi matches as well as finding and checking new engines. Will update you if there is any news/interesting.

Thanks. I'm not quite sure how to interpret (timer 300/40 + 0.5); it would seem to mean 7.5 seconds base time plus half a second increment, but (300/40) is an odd way to say 7.5, so perhaps it means something else? If this is indeed bullet speed (something like a second per move or so on average), it strongly suggests that these engines play bullet xiangqi far better than the top humans play at serious tournament time controls, just as is the case in chess and shogi. Probably it means that they could give the top humans knight odds at least with equal time, I imagine. Also, when you say the the draw rate in top human games is 30 to 40%, at what time limit is this? In chess, between top humans this would be the draw rate in Rapid perhaps, but at standard tournament time controls the draw rate is much more than 50% at top level.

[phhnguyen]

Thanks. I'm not quite sure how to interpret (timer 300/40 + 0.5); it would seem to mean 7.5 seconds base time plus half a second increment, but (300/40) is an odd way to say 7.5, so perhaps it means something else? If this is indeed bullet speed (something like a second per move or so on average), it strongly suggests that these engines play bullet xiangqi far better than the top humans play at serious tournament time controls, just as is the case in chess and shogi. Probably it means that they could give the top humans knight odds at least with equal time, I imagine. Also, when you say the the draw rate in top human games is 30 to 40%, at what time limit is this? In chess, between top humans this would be the draw rate in Rapid perhaps, but at standard tournament time controls the draw rate is much more than 50% at top level.

I use a similar way to Cutechess to brief timers: that is a standard clock of 300s (5 minutes) per/for 40 moves, a bonus of 0.5s for each move. Yes, they are bullet games.

I totally agree that software has much more advance than humans in fast games.

I got the stats (about WDL rate) based on games of human-high-rank players such as from Megabase 3.45m (by Schroder) with W: 39.0%, D: 30.5%, L: 30.5%. I guess the major of those games are standard timers but there are also some fast games and software games too. That is similar to Xiangqi high rank database I have (a mix between human players, some software, standard and fast games).

[lkaufman]

Thanks. I'm not quite sure how to interpret (timer 300/40 + 0.5); it would seem to mean 7.5 seconds base time plus half a second increment, but (300/40) is an odd way to say 7.5, so perhaps it means something else? If this is indeed bullet speed (something like a second per move or so on average), it strongly suggests that these engines play bullet xiangqi far better than the top humans play at serious tournament time controls, just as is the case in chess and shogi. Probably it means that they could give the top humans knight odds at least with equal time, I imagine. Also, when you say the the draw rate in top human games is 30 to 40%, at what time limit is this? In chess, between top humans this would be the draw rate in Rapid perhaps, but at standard tournament time controls the draw rate is much more than 50% at top level.

I use a similar way to Cutechess to brief timers: that is a standard clock of 300s (5 minutes) per/for 40 moves, a bonus of 0.5s for each move. Yes, they are bullet games.

I totally agree that software has much more advance than humans in fast games.

I got the stats (about WDL rate) based on games of human-high-rank players such as from Megabase 3.45m (by Schroder) with W: 39.0%, D: 30.5%, L: 30.5%. I guess the major of those games are standard timers but there are also some fast games and software games too. That is similar to Xiangqi high rank database I have (a mix between human players, some software, standard and fast games).

Forty moves in five minutes is blitz, not bullet, but anyway much faster than serious human games. The stats you are quoting cover a pretty wide range of strength; when I refer to "top" players I mean something like the top fifty in the world. In chess, when these players play each other at standard chess, the draw percentage is somewhere around 70% I believe. So what is the draw percentage in games among the top 50 xiangqi players at time limits like 2 hours or more per side for the game (or at least 2 minutes per move)? Or are such games rarely played anymore?

[phhnguyen]

Forty moves in five minutes is blitz, not bullet, but anyway much faster than serious human games. The stats you are quoting cover a pretty wide range of strength; when I refer to "top" players I mean something like the top fifty in the world. In chess, when these players play each other at standard chess, the draw percentage is somewhere around 70% I believe. So what is the draw percentage in games among the top 50 xiangqi players at time limits like 2 hours or more per side for the game (or at least 2 minutes per move)? Or are such games rarely played anymore?

Interesting idea about getting stats!

ATM, I have some difficulties getting the top list of Xiangqi players. A large number of them are from China (Xiangqi has another name Chinese Chess), documented in Chinese, the language I don't know and can't read, especially it is so hard for me to search. I have some databases of Xiangqi's top games with all names of players written in Latin characters. Someone has translated many of them from Chinese. It is good for me (to understand) but also harder to match out their real names. Thus I have noted out as work to do (find the list of top Xiangqi players and do some stats) when I have a chance.