how will Leela fare at the WCCC?

[mjlef]

Already in move 35. Houdini, Komodo and SF evaluate position 0,00 almost instantly and keep it. No need to continue . Definitely unfair from Komodo team. But they are already announcing in homepage "2018 World Computer Blitz Champion".

Some comments: Komodo indeed had a score of 0 early in the game, but lc0 id not have a score of zero. This indicated to me that lc0 did not know the position was a draw. So we played on. If lc0 had a score of zero early on, then it would have been clear it understood the position, and I am sure we would have accepted a draw much earlier. The lc0 operator told use they did not have Syzygy, and I even discussed with him some ways they could add them. And I hope they do.

[mjlef]

Allowing the possibility of 'gamesmanship' in a computer tournament is a little perplexing.
Relies on 'good' behaviour', I guess. When money/commercialism is involved, this is probably asking too much of some people.

More than a little perplexing. Leela and operator lost on time through their own setup. There was no power outage, no dizzy spell by the operator, in other words, no mitigating circumstances other than failing to be prepared. The clock is a part of the game. Take ownership of mistakes made, and don't blame the opponent for benefiting from them.

I meant earlier in the game when t was clear Komodo only say a draw
I posted a reply earlier but it seems gone now. So I will try to repeat it. Some more info below.

The lc0 operator was using some simplified GUI they wrote. Well, it was not graphical but it has things like "wp" for a white pawn (very much what I did when I started chess programming). I told him the writer of the UCI spec was in the tournament and they should write a UCI version and try out some of the many good chess GUIs (this was during the game).
There was no opening book for lc0, so it started thinking on the first move.

I spoke with him after the game and suggested he reduce his increment. Someone else had already suggested this. He had set his increment to, I believe, 10 seconds per move for the first game. For the second game he had already set it to 5 seconds, because 10 seconds (with the actual 15 sec increment in the time control) only allows 5 seconds per move. I also believe moves were entered as e2e4<Return> in the unique GUI, which takes more time than a single or double click in a standard GUI. These things all took up more time. Note he did not seem upset by the time loss.

You might ask why we played on once Komodo reached the 6 piece Syzygy draw. Our experience with MTCS nn engines is they do not play the endgame as well as traditional programs. We also discussed Syzygy with the lc0 operator and he said they had not implement them yet. If they had them, we would have proposed a draw ourselves long before since why play on when we know the result? When you know little about a new opponent, the safest thing to do is play on. If it was say Shredder, then game would have ended a lot sooner.

About two moves before the end of the game game I reminded the lc0 operator that he could alwys request a draw. I was not looking at the clock when I said this. He proposed it, and I went over to our operator to start discussing if we should accept. The clock fell and it lost on time.

Earlier in the game I saw lco was taking a lot less time than Komodo, so I did not keep track of its time. And I did not know he had such a large increment which only allowed 5 seconds (on average) to enter a move. I am also not sure if he tried adjusting the time. Operating a chess engine in a tournament srequire a lot of scanning of everything to make sure all is good.

I spoke with Larry later in the day and he agreed with what we had done. I think nearly all chess programmers learn a lot from the first tournament. I sure did. Preparation and practice is important. I will try to follow the time better and suggest people adjust time if it is getting off.

Mark

You had bishop and wrong colour rook pawn.
Leela had rook and pawn. Pawns blocked on same file. IIRC


Do not misunderstand, I am most definitely not suggesting you broke the rules or did not have the right to play on.
Equally, the primary failing is definitely by the leela team.
But the game had been drawn for some time.

I meant earlier in the game, Programs without Syzygy can goof and exchange down to a position that could be win. When an opponent has had questionable endgame skills in the past, and you know it is not using Syzygy, it i proper to continue playing. Another time when is when the programs both know it is a draw, and the GM commentator asks to continue the game a bit so he can explain the position to the audience. That happened in WCSC during a game with a wrong color bishop and rook pawn. I saw the GM walking to our table and guessed out loud what he wanted.

[mjlef]

Oh man,the first time I see those photos where people are manually entering moves. no IT professionals to set up two computers via LAN? I can even do that!!

The rules of this tournament are rooted in the dawn of the computer era (70's) long before networks and standardized serial communication protocols. Playing on a physical chess board was considered the easiest way to interoperate between such widely varying platforms such as remote secured supercomputers and dedicated chess computers.

As to whether we should change this rule, that is up to the ICGA, and I don't want to spark this argument for the 2421st time. (Seriously, check the archives and even Usenet rec.games.chess.computer!)

To blunder it all away would require worse chess than any computer I've ever owned, even from the 1980's. It just wouldn't happen. Surely.

There is no way in hell Leela loses that outside of somehow getting disqualified or losing on time and it seems disingenuous to even suggest it.

There is always a third opponent in every computer chess game: the esteemed Mr. Murphy. The WCCC archives are littered with dozens of abruptly lost games because the program crashed. I mean, it is no different than a high performance engine blowing a gasket during an race, even if the car is ahead. This happens in all computer chess events, even the latest TCEC had to disqualify Chiron after it crashed three times, and there were even some games where the crash came just a few moves before mate!

The clear difference is that a human operator can affect, in fact change, the result.

The komodo operator won the game, not komodo.
What people think about that is a matter for them, but it does not change the fact.

If a program loses on time or crashes then so be it. It is the fault of the program, so to speak.
I find it bizarre that the result of a WCCC match, and perhaps the whole event, can be changed, or even affected, by human intervention/error/ability to move and type fast. Perhaps I simpley misunderstood the meaning of the ' WCCC'.

The game for lc0 because of too little operator time being set by the operator. In speed chess, an operator can lose the game by making the wrong move on the board or when they run out of time. For WCSC and WCCC we are loowed to correct a position on the boar after alerting the TD. Humans need to know their own limitations and set the time on the machine appropriately. Part of the game its the human operator. If you are going to play without an opening book, programs should allocate even more operator time.

[mjlef]

Allowing the possibility of 'gamesmanship' in a computer tournament is a little perplexing.
Relies on 'good' behaviour', I guess. When money/commercialism is involved, this is probably asking too much of some people.

More than a little perplexing. Leela and operator lost on time through their own setup. There was no power outage, no dizzy spell by the operator, in other words, no mitigating circumstances other than failing to be prepared. The clock is a part of the game. Take ownership of mistakes made, and don't blame the opponent for benefiting from them.

I posted a reply earlier but it seems gone now. So I will try to repeat it. Some more info below.

The lc0 operator was using some simplified GUI they wrote. Well, it was not graphical but it has things like "wp" for a white pawn (very much what I did when I started chess programming). I told him the writer of the UCI spec was in the tournament and they should write a UCI version and try out some of the many good chess GUIs (this was during the game).
There was no opening book for lc0, so it started thinking on the first move.

I spoke with him after the game and suggested he reduce his increment. Someone else had already suggested this. He had set his increment to, I believe, 10 seconds per move for the first game. For the second game he had already set it to 5 seconds, because 10 seconds (with the actual 15 sec increment in the time control) only allows 5 seconds per move. I also believe moves were entered as e2e4<Return> in the unique GUI, which takes more time than a single or double click in a standard GUI. These things all took up more time. Note he did not seem upset by the time loss.

You might ask why we played on once Komodo reached the 6 piece Syzygy draw. Our experience with MTCS nn engines is they do not play the endgame as well as traditional programs. We also discussed Syzygy with the lc0 operator and he said they had not implement them yet. If they had them, we would have proposed a draw ourselves long before since why play on when we know the result? When you know little about a new opponent, the safest thing to do is play on. If it was say Shredder, then game would have ended a lot sooner.

About two moves before the end of the game game I reminded the lc0 operator that he could alwys request a draw. I was not looking at the clock when I said this. He proposed it, and I went over to our operator to start discussing if we should accept. The clock fell and it lost on time.

Earlier in the game I saw lco was taking a lot less time than Komodo, so I did not keep track of its time. And I did not know he had such a large increment which only allowed 5 seconds (on average) to enter a move. I am also not sure if he tried adjusting the time. Operating a chess engine in a tournament srequire a lot of scanning of everything to make sure all is good.

I spoke with Larry later in the day and he agreed with what we had done. I think nearly all chess programmers learn a lot from the first tournament. I sure did. Preparation and practice is important. I will try to follow the time better and suggest people adjust time if it is getting off.

Mark

You had bishop and wrong colour rook pawn.
Leela had rook and pawn. Pawns blocked on same file. IIRC


Do not misunderstand, I am most definitely not suggesting you broke the rules or did not have the right to play on.
Equally, the primary failing is definitely by the leela team.
But the game had been drawn for some time.

We would have offered a draw if it was clear lc0 also say a draw score. But its score was way off.

[mjlef]

The Leela operator's mom slipped and hit her head here ins Stockholm last night, and they spent the night in the emergency room. They are now taking his Mom home, so there will be no Leela game this morning. I do not know how serious the injury is but it is hopeful since they are allowing her to travel.

I just heard they have found another operator and awaiting approval of the lc0 team. So hopefully Leela will continue this afternoon. The programmers and TD are considering how we might be able to fill in the missing game later. Maybe playing from the hotel, or we could try starting the other games sooner to free up time. As I hear more, I will keep you posted.

Mark

[frankp]

Allowing the possibility of 'gamesmanship' in a computer tournament is a little perplexing.
Relies on 'good' behaviour', I guess. When money/commercialism is involved, this is probably asking too much of some people.

More than a little perplexing. Leela and operator lost on time through their own setup. There was no power outage, no dizzy spell by the operator, in other words, no mitigating circumstances other than failing to be prepared. The clock is a part of the game. Take ownership of mistakes made, and don't blame the opponent for benefiting from them.

I posted a reply earlier but it seems gone now. So I will try to repeat it. Some more info below.

The lc0 operator was using some simplified GUI they wrote. Well, it was not graphical but it has things like "wp" for a white pawn (very much what I did when I started chess programming). I told him the writer of the UCI spec was in the tournament and they should write a UCI version and try out some of the many good chess GUIs (this was during the game).
There was no opening book for lc0, so it started thinking on the first move.

I spoke with him after the game and suggested he reduce his increment. Someone else had already suggested this. He had set his increment to, I believe, 10 seconds per move for the first game. For the second game he had already set it to 5 seconds, because 10 seconds (with the actual 15 sec increment in the time control) only allows 5 seconds per move. I also believe moves were entered as e2e4<Return> in the unique GUI, which takes more time than a single or double click in a standard GUI. These things all took up more time. Note he did not seem upset by the time loss.

You might ask why we played on once Komodo reached the 6 piece Syzygy draw. Our experience with MTCS nn engines is they do not play the endgame as well as traditional programs. We also discussed Syzygy with the lc0 operator and he said they had not implement them yet. If they had them, we would have proposed a draw ourselves long before since why play on when we know the result? When you know little about a new opponent, the safest thing to do is play on. If it was say Shredder, then game would have ended a lot sooner.

About two moves before the end of the game game I reminded the lc0 operator that he could alwys request a draw. I was not looking at the clock when I said this. He proposed it, and I went over to our operator to start discussing if we should accept. The clock fell and it lost on time.

Earlier in the game I saw lco was taking a lot less time than Komodo, so I did not keep track of its time. And I did not know he had such a large increment which only allowed 5 seconds (on average) to enter a move. I am also not sure if he tried adjusting the time. Operating a chess engine in a tournament srequire a lot of scanning of everything to make sure all is good.

I spoke with Larry later in the day and he agreed with what we had done. I think nearly all chess programmers learn a lot from the first tournament. I sure did. Preparation and practice is important. I will try to follow the time better and suggest people adjust time if it is getting off.

Mark

You had bishop and wrong colour rook pawn.
Leela had rook and pawn. Pawns blocked on same file. IIRC


Do not misunderstand, I am most definitely not suggesting you broke the rules or did not have the right to play on.
Equally, the primary failing is definitely by the leela team.
But the game had been drawn for some time.

We would have offered a draw if it was clear lc0 also say a draw score. But its score was way off.

The report I saw was that the leela operator offered a draw which you declined, in order to win on time. Perhaps this was false reporting?

[Gian-Carlo Pascutto]

About two moves before the end of the game game I reminded the lc0 operator that he could alwys request a draw. I was not looking at the clock when I said this. He proposed it, and I went over to our operator to start discussing if we should accept. The clock fell and it lost on time.

The report I saw was that the leela operator offered a draw which you declined, in order to win on time. Perhaps this was false reporting?

Did he not already answer this? See above.

By setting the operator time very low, Leela had more time to think than Komodo, so "saving" Leela after having taking this advantage would have been unfair to Komodo. There was also still a possibility that Leela would have a bug and hang or similar (I noticed it lost a TCEC pre-season game on time too, just a few days ago, and that's playing automated!). I assume they were confident that Komodo playing from EGTB would never lose.

I can't say whether I would have made the same decision in the same circumstances (especially as Leela couldn't really lose by over-pressing a win...) but it was not at all improper from the Komodo team to play on.

[frankp]

About two moves before the end of the game game I reminded the lc0 operator that he could alwys request a draw. I was not looking at the clock when I said this. He proposed it, and I went over to our operator to start discussing if we should accept. The clock fell and it lost on time.

The report I saw was that the leela operator offered a draw which you declined, in order to win on time. Perhaps this was false reporting?

Did he not already answer this? See above.

By setting the operator time very low, Leela had more time to think than Komodo, so "saving" Leela after having taking this advantage would have been unfair to Komodo. There was also still a possibility that Leela would have a bug and hang or similar (I noticed it lost a TCEC pre-season game on time too, just a few days ago, and that's playing automated!). I assume they were confident that Komodo playing from EGTB would never lose.

I can't say whether I would have made the same decision in the same circumstances (especially as Leela couldn't really lose by over-pressing a win...) but it was not at all improper from the Komodo team to play on.

As I have said:
they did nothing against the rules;
the leela team are primarily at fault for not testing/preparing (enough);
the decision to win by gamesmanship (my view) I find unsporting (my view). The game was won by the operator not komodo.

And again, in a tournament called the WCCC in 2018, I find it surprising that human intervention is allowed to affect the result - faster operation = more time and in this instance changed the game result. But that is my misunderstanding of the meaning of the 'title' WCCC. Probably does go to the best computer+hardware combo 'on the day', but maybe also the best computer+hardware+operator, with the latter able to have a significant impact.

Congratulations to the komodo team: they proved the better team on the day - no sarcasm intended. Sadly, I think something else was lost in the winning.

[Harvey Williamson]

Congratulations to the komodo team: they proved the better team on the day - no sarcasm intended. Sadly, I think something else was lost in the winning.

That is also my feeling.

I watched the game in Sweden. Mark did, indeed, tell the Leela operator that he could offer a draw. He did so immediately. The Komodo operator refused the offer by playing a move.

[Gian-Carlo Pascutto]

I find it surprising that human intervention is allowed to affect the result - faster operation = more time and in this instance changed the game result.

It's an artifact from there being huge commercial interests at some point in the past, which made agreeing on a common server and protocol problematic. Maybe nowadays you could just agree on UCI, but even then I'm not sure people like Bob would agree because UCI is in some ways opinionated. I think the closest thing we have is CECP/WinBoard on some text-based server? I'm a bit out of the loop there.

Go uses a simple protocol (GTP) and doesn't have the "opinionated" issue (of course the standard protocol being very simple causes much more important problems outside tournament play, IMHO). I'm not sure about the shogi programs, have they agreed on USI for tournaments?

Now that the ICGA has prize money, they could surely have enforced the contenders on a common protocol. If they'd bothered.

Congratulations to the komodo team: they proved the better team on the day - no sarcasm intended. Sadly, I think something else was lost in the winning.

Komodo's operator is a veteran of >17 years and one of the best and most experienced people in those tournaments, save for maybe Stefan Meyer-Kahlen. I don't think he'll give it a second thought.