how will Leela fare at the WCCC?

[yanquis1972]

Already in move 35. Houdini, Komodo and SF evaluate position 0,00 almost instantly and keep it. No need to continue . Definitely unfair from Komodo team. But they are already announcing in homepage "2018 World Computer Blitz Champion".

Some comments: Komodo indeed had a score of 0 early in the game, but lc0 id not have a score of zero. This indicated to me that lc0 did not know the position was a draw. So we played on. If lc0 had a score of zero early on, then it would have been clear it understood the position, and I am sure we would have accepted a draw much earlier. The lc0 operator told use they did not have Syzygy, and I even discussed with him some ways they could add them. And I hope they do.

someone actually did it quite awhile ago, but beyond that i know nothing. i don't know if it was capable of, or if you're referring to, probing during search, or just having them on the burner for particular impediments or peculiarities (such as this one). they'd be useful even now for shortening unattended, non-training games, but i think the larger issue is one of principle; this is the first publicly scheduled, open display of a zero knowledge NN & i'd put demonstrating the WIP concept over maximizing points at the expense of not using it in critical situations. once it's established itself, purity will be sacrificed for convenience & eventually, strength.

[hgm]

I'm not sure about the shogi programs, have they agreed on USI for tournaments?

To participate in the championship of the Computer Shogi Association your program must support CSA protocol, which is a socket-based protocol.

[hgm]

I watched the game in Sweden. Mark did, indeed, tell the Leela operator that he could offer a draw. He did so immediately. The Komodo operator refused the offer by playing a move.

It is really strange that operators are allowed to offer or decide about acceptance of draws. IIRC there used to be rules where an operator wasn't allowed to do anything that the computer did not explicitly ask it to do. Not even synchronizig clocks. I think that is the way it should be; the computers should make the game decisions.

[IanO]

Go uses a simple protocol (GTP) and doesn't have the "opinionated" issue (of course the standard protocol being very simple causes much more important problems outside tournament play, IMHO).

The computer Go tournaments were, in fact, one of the first to support a standardized communication protocol even before such conveniences as networks and socket communication: the Go Modem Protocol (GMP). All it required was an RS-232 serial connection, allowing interoperation of programs on very widely differing architectures.

And even the current Go protocols have a lot of warts having to do with disputed group resolution at the end of the game, depending on the ruleset in use.

[mjlef]

I watched the game in Sweden. Mark did, indeed, tell the Leela operator that he could offer a draw. He did so immediately. The Komodo operator refused the offer by playing a move.

It is really strange that operators are allowed to offer or decide about acceptance of draws. IIRC there used to be rules where an operator wasn't allowed to do anything that the computer did not explicitly ask it to do. Not even synchronizig clocks. I think that is the way it should be; the computers should make the game decisions.

I would be fine with the computers deciding, provide we could:

a. let the engines know who they are playing. What we might choose to do against a strong program would not be the same as a weak program.
b. add some support to UCI for this. Unlike winboard it has not offer/accept draw/resign.

Right now we already influence the engines via settings, opening book choices. And if we switched to having the engines decide, I would simply add another parameter or two telling the engine what the score would need to be to offer/accept a draw, and for how many moves, and maybe what game stage.... so I would be influencing its choice once again.

[hgm]

As a programmer you (directly or indirectly) always control what the program does. The idea of these tournaments, however, used to be that the games were 'fire and forget': no human interference whatsoever during a game. So it is perfectly OK to adapt your opening book and contempt setting to the opponent before the game. But not during it. The latter would open the door to cheats of the form "I see that my engine under-estimates its winning chances in the end-game it happened to reach, so let me increase contempt to make sure it will avoid moves that immediately draw".

You can let your program know who it is playing, or at least how strong an opponent it is playing. AFAIK there is no rule that forbids that. I thought the 'contempt' option was designed to already do exactly that.

Likewise, the WCCC rules do not put any restriction on the protocol you use, anymore than it puts restrictions on the language you have to program in, the kind of extensions and reductions you are allowed to use, etc. If a participants chooses to use a programming language that is slow, a GUI that is clumsy, or a protocol that is deficient, that is entirely his own problem, and certainly not a reason to adapt the rules or exempt him from those. Otherwise progams written in an interpreted language, or running on a phone could claim extra time, etc.

So the "provided we could" seems a bit out of context here. There is nothing "provided". You can, and if you wanted, you should have...

[Gian-Carlo Pascutto]

And even the current Go protocols have a lot of warts having to do with disputed group resolution at the end of the game, depending on the ruleset in use.

This only happens if you use a broken ruleset, such as the Japanese one (which I don't believe can even be called a ruleset, but that's another holy war). Luckily, most tournaments don't do that, so we (can) play until resignation or until the undisputed count on the board is always a win.

Side story: Leela Zero (not chess) was playing in a major Chinese tournament, operated by a local person who supplied 10x1080Ti's. He claims it crashes occasionally on this hardware (I've never seen it, but I also don't have 10x1080Ti's!) and thus decided that running the engine automated (which was entirely possible!) was risky to lose on time should a crash happen. So he played the moves manually and promptly misclicked in several games. In Go tournaments the move stands so Leela lost basically instantly. This probably cost many thousands of USD in prize money.

If you are playing to win you need an experienced operator (on site or doing the setup), no way around it.

[Gian-Carlo Pascutto]

Likewise, the WCCC rules do not put any restriction on the protocol you use

It does not now, but we're obviously talking in the context of an "improved" tournament where the engines run automated, and consequently, some kind of standard communication protocol is enforced.

As I mentioned before, even though UCI is the most popular, it is probably too opinionated to be good for that, and as Mark points out, lacks essential features for tournament play.

[hgm]

Well, I was actually talking about the Leela - Komodo game, where operators were apparently involved in complicated reasoning for whether the circumstances that had materialized made it wise to offer a draw, or accept one. The rules I knew would not allow such a thing, and if they have been changed to allow it now, I don't consider that a change for the good. Operators should only be allowed to offer a draw if the engine request them to do so. In which case the operator of the opponent should relay the offer to his engine so it can decide upon it, or reject it by default in the engine doesn't have the capability to accept draw offers.

To prevent having to wait out the 50-move rule in cases of an obviously drawn position, the operators can request the referee if it is OK to agree a draw, as a sort of voluntarty adjuication. Even resigning should be subject to referee approval if the engine doesn't resign by itself.

As to a future protocol for automated play, an obvious solution would be to use the old, but time-tested ICS protocol. This supports all features needed for tournament play, and supported by a number of GUIs or command-line adapters, so that is basically requires zero effort of the participants to comply with such a requirement.

[Guenther]

Well, I was actually talking about the Leela - Komodo game, where operators were apparently involved in complicated reasoning for whether the circumstances that had materialized made it wise to offer a draw, or accept one. The rules I knew would not allow such a thing, and if they have been changed to allow it now, I don't consider that a change for the good. Operators should only be allowed to offer a draw if the engine request them to do so. In which case the operator of the opponent should relay the offer to his engine so it can decide upon it, or reject it by default in the engine doesn't have the capability to accept draw offers.

IMHO the TD should declare a draw, if the (materially) weaker side uses tablebases or both sides use tablebases in a 6men endgame.
(Of course this could be subject to a change in the future, if bigger tablebases are openly available)

In this case no intervention from any participant/operator itself would be necessary.