@HG: auto moveing and lift,put,drag

[Daniel Shawul]

You are probably on a vacation, so this is for when you get back. You recall about the "lift,hover,put" combination to highlight piece movement and make auto moves ? I sure forgot some about it. Anyway it is a little bit difficult to implement it directly on a smartphone for some obvious reasons which I mention below. Also with the use of external referee engine I am going to implement a method (probably mentioned it some time ago) that I think will be better. First the issues

a) smarthpones are hard to use for drag and drop so hover is not that much of use. On an android app, I send only lift from the GUI so that the highlight stays there. If I send a 'put' right away after a 'lift', the highlight goes off because it assumes a drag-n-drop movement. So so far I only used lift but that may also be gone. See (c)

b)There is also the need to make leg-moves holding on the cntrl-key which makes things even more difficult

If you recall, the use of referee engine avoided encoding legal move indicators in the colorfen which we used in lack of other alternatives.
The solution I am planning to use uses the referee engine to finish off moves once (instead of sending series of clicks), also highlight moves (in a different way). The former removes a huge headache regarding move formats that involve commas; in this new situation the GUI doesn't even need to know if there are commas or not, since a move is sent once in full so it is not necessary to understand its contents.

Step 1) The referee keeps a list of legal moves
Step 2) The gui sends clicks of the user to the referee. This is not the click command used in the previous version. In this method the user can start clicking from the 'to' square of the move or even in the middle.
Step 3) From each click it recieves, it reduces the move list zooming in what the user wants to move. When only one move is left the referee sends the whole move to the Gui so that it makes the move. If no moves contain all the clicked squares the move is deemed illegal and the whole process is repeated.
Step 4) Highlighting is done differently. Say the user clicked an empty square he intends to land some pieces. All the moves that contain that square as a 'from', 'to', 'middle' as in checkers are accepted. So we merge all the squares in the moves and highlight them. If the user clicked his own piece first, then it is the same as previous method, because moves from that piece are only highlighted. So the set (the move list, the highlighted squares) can start from a wide range, and gets reduced with more clicks until one or none is left.

Required new command: 'click' from GUI to Referee
After that the GUI listens for a confirmation from the Referee about the click which could be. a) Illegal move b) A real move that could be f.i a 4 pawn capture in checkers just after a click c) confirmation like a 'pong' so that the GUI moves on. I don't know what that could be right now ? Go text protocol sends a \n\n confirmation after every command. But maybe I missed something here and it may not be required at all.
Edit Ineed. Right after I post this I see the solution. The 'highlight' is sent if the move is 'Not legal' or 'Legal'. That is really not a universal set lol. When the move is during construction, a highlight is always sent. Highlight is not needed when the fate of the move is known, so this fits in perfectly. So to summarize
* GUI sends: a click command (could be 'click' itself)
* Referee replies with either an Illegal move, Legal move or a Highlight.

Advantages if it passes the test:
a) lift,put,hover and click can be avoided. The hover may still be required for on PC games though.
b) wide option of making a move for the user. User can make moves with a few cleverly selected clicks. Current winboard alien only accepts moves that start with a click on the piece to move.
c) GUI don't need to understand if a move is multi-leg making it completely alien.
c) ctrl-key combinations avoided
d) easier to implement too i guess

I haven't actually implemented it but it seems it should work. We should agree on the commands so nothing in the previous version breaks down. Using 'click' in that regard seems a bad idea.

cheers
Daniel

[Daniel Shawul]

I implemented it now and all I can say is that it works really well. It is very easy to implement on both sides (gui & engine) and works really well. Clicking on to-squares, intermediate squares etc to make the legal root move that has that square as its destination is fun. For multi-leg captures of checkers that branch at the 3rd branch clicking on the from and destination resolves the branch. Something I forgot to mention is that this method also offers nice solution for poping up "promotion menu". It can decide to do that when it sees its moves are all promotions and tell the gui to offer the user a menu with its options. Again no decisions required from the GUI.
I was hung up on a bug for an hour or so because of a non reentrant behaviour of the referee engine thread. It was calling itself when it gets a move for the user (eg click e2e4 -> move e2e4), so I needed to call it from the GUI thread to update its moves. Console output to show the behavior I explained before. Only one command required "click" from gui to referee. It makes the move when it determines it is final, sends "illegal move" when an unreachable combo is clicked, and highlights when there are multiple options like in case of click h3.

Code: Select all

            a b c d e f g h 
            * * * * * * * * * * * * * * * * * * * * * * * * 
            * * * * * * * * * * * * * * * * * * * * * * * * 
            * * * * * * * * * * * * * * * * * * * * * * * * 
            * * * * * * * * * * * * * * * * * * * * * * * * 
          8 r n b q k b n r * * * * * * * * * * * * * * * * 8 
          7 p p p p p p p p * * * * * * * * * * * * * * * * 7 
          6 . . . . . . . . * * * * * * * * * * * * * * * * 6 
          5 . . . . . . . . * * * * * * * * * * * * * * * * 5 
          4 . . . . . . . . * * * * * * * * * * * * * * * * 4 
          3 . . . . . . . . * * * * * * * * * * * * * * * * 3 
          2 P P P P P P P P * * * * * * * * * * * * * * * * 2 
          1 R N B Q K B N R * * * * * * * * * * * * * * * * 1 
            * * * * * * * * * * * * * * * * * * * * * * * * 
            * * * * * * * * * * * * * * * * * * * * * * * * 
            * * * * * * * * * * * * * * * * * * * * * * * * 
            * * * * * * * * * * * * * * * * * * * * * * * * 
            a b c d e f g h 

                [Material: 4900 4900 ] 
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 

click e4 
move e2e4 
click h3 
highlight 8/8/8/8/8/7Y/7Y/6Y1 
click h2 
move h2h3 
click e5 
Illegal move 

One additional modification is when the user changes his mind about which move he want to make. Say he clicked e2 to move to e4 (e3 & e4 will be highlighted), then he decided to make d2d4, clicking on d2 would produce an illegal move first, then when he clicks again on d2 the highlight for d2 will be produced. These two can be merged by automatically switching from moving e2 to moving d2 (i.e forgetting about previously clicked squares) when the user makes an impossible combo.
SO this removes the need for sending "illegal move". That also gave me a problem because I stopped the game when the user makes an illegal move. The human can never make an illegal move because it is the GUI's duty to make sure that don't happen. So we have now only a "move" or "highlight" as responses from the referee. The clicks will go on until the human finally makes a legal move.
Edit I guess we still need "illegal move" when the user clicks on an empty square (no squares to highlight and no legal move). But anyway it can still be used as a shortcut to make faster move selection...

[hgm]

Short comment (from phone).

Lift and put seem upward compatible with your GUI-engine click. Why not use that, and just respond differently to it? Engine-GUI move in stead of click seems good idea. If GUI wants pong, it can send ping.

[Daniel Shawul]

Short comment (from phone).

Lift and put seem upward compatible with your GUI-engine click. Why not use that, and just respond differently to it? Engine-GUI move in stead of click seems good idea. If GUI wants pong, it can send ping.

Here is what I understood from implementing it. First the ping-pong turns out to be not necessary despite my fear. In the previous case where I had the referee judge moves from engines as illegal or legal, GUI had to wait after sending move to the referee until a response comes back (either "Illegal" or "setup"). It can not just send the move to the next player because it may be illegal, and the next player could have faltered because of that.
But in the case of a human player, the gui is not sending moves but _clicks_ to construct a move. 20 clicks could be sent and still one legal move will come back, the gui simply doesn't move anywhere unless a legal move comes back. Possible responses that come back are in this case ("highlight","move e2e4","Illegal move"). In case of highlight, and Illegal move, the gui simply displays information, but when it receives a move , it automatically sends it back to the referee, from which point onwards things go on like the case of an engine sending a move. It is kind of redundunt to send the move back to the referee since it generated it anyway, but this is what avoided the ping-pong during move construction.
I had an interesting problem with threading in this case, because if I use the referee engine thread to send a move to gui and then back to itself it gets stuck because I had a wait_for_refree in the code. So dead lock there.

About the lift and put, those two were required to be treated as separate (instead of single click), because there was a hover in between. And what the engine does in between to send highlight information is interrelated. What I do when receiving clicks is very easy : generate all moves from scratch for every click (independently), and check every move against a list of clicked squares that it keeps. Then if there is only one move that contains all it sends a move otherwise a highlight. This is very easy compared to a lift-hover-put combination, that constructs moves leg by leg.
Another advantage I found is this also provides a solution for promotion popups without involving the gui again. When the referee finds out it is a promotion move according to its rule which could be really alien (like a promo on 6th rank), it can send the GUI a special message like "promotion Q R B N at h6" and it would offer that. In the previous case the GUI had to understand that by itself ,like winboard considers moves to rank 8 as promotions automatically. That gave us problems with checkers if I recall. More things can be sent from the referee-to-gui to offer options. I am convinced this approach is the right way to go for the alien edition because it has reduced the gui simply to displaying stuff and not get involved in understanding the moves, or move construction, fen etc..

More when you come back. Have a good one.

[hgm]

How would the referee know you did a drag-drop move?Send 2 clicks? Why mask the difference between press and release?

[Daniel Shawul]

For smarthphones drag-n-drop may not be appropriate because while the user is swiping with his finger to drag a piece it will cover the piece (you don't get any satisfaction out of it). Having said that , there are equivalents to a mouse pressed,released, swiped screen events in android too. But the method is definately slower thank click-click. With this new method you can make e2e4 by clicking e4 only (which is SAN representation). You can make long capture moves in checkers that branch out after say 3 captures, by clicking one square that resolves the ambiguity. But you can not drag a piece, because you have to click the from square _first_ to do so. So it is one or the other but for smartphones I find the latter method preferable. For PC implemenation, we used the "hover" to mark danger squares like in Ultima and Atomic. I have not thought through everything well, and I don't know if it is possible to do every thing equally using both methods.

An interesting situation happened in amazons. At the initial possition the first click highlights all squares!, because you can reach virtually any square. The second click reduces the range and a crossed ray highlight (Queen type move), and finally in the third click the amazon moves and throws the arrow to complete the move.

[hgm]

Referee and protocol should be platform independent. Only the GUI should know it runs on Android. In Amazons the referee must highlight only Q to-squares to be useful. Beware that the system should also work without a highlighting referee; highlighting is optional feature. What if second leg is optional (e.g. Chu Shogi Lion)?

[Daniel Shawul]

Referee and protocol should be platform independent. Only the GUI should know it runs on Android. In Amazons the referee must highlight only Q to-squares to be useful. Beware that the system should also work without a highlighting referee; highlighting is optional feature. What if second leg is optional (e.g. Chu Shogi Lion)?

Yes they should be independent and I didn't think the gui would need to change too, as I was already on my way implementing lift and hover stuff. But dragging pieces wastes time and probably wears creen too.. For Amazons the lift-n-put version did exactly what you said, which also makes sense since the amazon gets picked up first. However with the new version, I can't do that reasonably since any empty square that the move lands on could be clicked first. I thought it was a bug at first when the screen filled up with highlights. Note that starting clicks from the square where the arrow lands far reduces the highlighted squares , mostly ray highlights. So it is upto the user to narrow his domain as to what he plans to do with his first click. Clicking the first square opens up door to ~17000 moves which is the problem!

If the second leg move is optional as in chu shogi, then the shorter and longer versions of the move would be included in the list and it should be able to highlight them as usual. If he made clear indication of doing the shorter version of the move by his first click, it should be no problem I guess..

No other information is communicated through the colorfen (f.i legality checking). Highlighting is optional and the gui could ignore them if that is not allowed as in the case of FICS. Which reminds me that, at no point did I need to use a referee there! The server sends you positions when you recieve moves, accept takeback offers etc... Referee is also optional when you know the two engines 'never' make illegal moves, That along with console mode is useful for internal testing of engines.

[hgm]

Somewhat better internet access today!

I am not sure how exactly your current implementation is. (I.e. does your GUI only send something to the every on button press events, or also for button release?) But let me explain what I am concerned about:

I fully understand that drag-drop is cumbersome/undesirable of smart-phone devices. But that should just mean that a smart-phone GUI does not allow the user to perform drag and drop, and thus will never send the corresponding commands (such as 'hover') to the engine. But there are other GUIs running on more powerful hardware, where drag-drop is preferred, and both the protocol and the referee should also support that. It is not clear to me that this is the case in your current referee implementation.

It seems to me that it can never hurt to diversify the GUI->Engine 'click' command into two flavors: press and release. The GUI would normally drag pieces that you click, if they are of the side to move. I am not sure if that requires more smartness than even a dumb GUI provides. It seems the GUI must have an idea of who is to move, or it could not maintain the clocks. But it could be that it is enough to let the GUI send a 'click' command for every release event on a different square than the corresponding press (and stay silent if the up-click is on the same square). But the engine would never know when exactly you are dragging, and this might be a problem (e.g. how to respond to 'hover'). In WinBoard I use a release on a different square to clear any highlights (because the leg of the move for which the highlights were intended is now finished), so that the engine never has to worry about clearing highlights. On a static click the highlights stay on after the release event, however, because the leg entry has still to be completed by a second click.

Another requirement I would like to impose is that the system can still provide basic functionality without a special referee engine, i.e. using an engine that supports nothing but original WB protocol. (In which case the engine should be able to act as its own referee.) You would lose the highlighting and one-click moving, for sure, but these are merely frills. Move entry and legality checking should always work, though. So there must be some simple default mechanism for the GUI to synthesize moves from the clicks, without using any rule knowledge.

I agree that the Ctrl key for partial moves is a pain, though. (I'd much rather use it for something else...) Making the engine or referee responsible for this would be nice. I am just not sure that this is always possible, though. Take again the example of the Chu-Shogi Lion. It can do two King moves (even if both are captures) in one turn. But it can also do just one. So how can you enter that? After clicking Lion and first victim there is already a legal move, but there could also be a continuation (eight, actually). But if you want to stop there, how can you tell it to the engine? I guess reverse entry could be a convention to make this work. Click any number of empty or opponent squares before you click a piece of your own, and it will capture them all with a multi-leg move (i.e. send 'usermove LEG1,LEG2,LEG3,...' to the engine). With an engine that supports 'click' there would be much more flexibility: the GUI could send the individual squares in any order, and keep that up until the engine/referee responds with 'move' (in the mean time performing highlighting as requested). An engine would be aware of the possible ambiguities in move termination, and could apply its own conventions to resolve them (e.g. to terminate entry that might have continued, click the same piece again as last time).

Problems I don't clearly see a solution for at the moment are:
- With reverse multi-leg entry, how can you force a reset of the entry process after the user makes an erroneous click? (Both with stand-alone GUI or engine-mediated.) It should always be possible for the user to change his mind before the full move is committed. Perhaps also click the last selected target a second time, just as WB allows you to click your own piece a second time to deselect it? I guess multiple (square-border or arrow) highlights could be supported, that go on even if you click opponent pieces or empty squares, and would all go off on a deselect click.
- If the user does want one-click moving to be off, how to achieve that? It would be nasty if a click-supporting engine has to know it is off, yet a GUI would have to send all the clicks to evoke highlighting responses. Just shelve the referee's move until the user has clicked all the squares stipulated by the move? And send 'undo' if he aborts or deviates?
- It would be nice if highlighting could somehow be used to determine clicks illegal already in the GUI: waiting for an 'Illegal move' engine response to undo the just entered move produces notable glitches in the display.

[Daniel Shawul]

Somewhat better internet access today!

Welcome back

I am not sure how exactly your current implementation is. (I.e. does your GUI only send something to the every on button press events, or also for button release?)

It sends only one click on a Mouse pressed event (none on release). Pressed-hovered-released are required only when are going to make a move leg by leg, with the clicks method any click pattern of randomly sorted squares of the move will do. For hovering, a piece has to be clicked first.

But let me explain what I am concerned about:

I fully understand that drag-drop is cumbersome/undesirable of smart-phone devices. But that should just mean that a smart-phone GUI does not allow the user to perform drag and drop, and thus will never send the corresponding commands (such as 'hover') to the engine. But there are other GUIs running on more powerful hardware, where drag-drop is preferred, and both the protocol and the referee should also support that. It is not clear to me that this is the case in your current referee implementation.

The referee i.e Nebiyu has all the previous code, only my simplistic gui for android doesn't provide support for hover. The guis click are meant to indicate selected squares to the referee.

It seems to me that it can never hurt to diversify the GUI->Engine 'click' command into two flavors: press and release. The GUI would normally drag pieces that you click, if they are of the side to move. I am not sure if that requires more smartness than even a dumb GUI provides. It seems the GUI must have an idea of who is to move, or it could not maintain the clocks. But it could be that it is enough to let the GUI send a 'click' command for every release event on a different square than the corresponding press (and stay silent if the up-click is on the same square). But the engine would never know when exactly you are dragging, and this might be a problem (e.g. how to respond to 'hover'). In WinBoard I use a release on a different square to clear any highlights (because the leg of the move for which the highlights were intended is now finished), so that the engine never has to worry about clearing highlights. On a static click the highlights stay on after the release event, however, because the leg entry has still to be completed by a second click.

Yes i could have used 'put' instead of new command 'click' I guess. I am not sure why but the code for the lift-put-hover in nebiyu is complicated. I suspect it is because winboard doesn't use an external referee, and the engine playing the same game is used as referee. So I had to keep a different search stack , keep track of commas and other things... So rather than modifying that I prefred to use a different command 'click' and do a simple comparsion of list of clicked squares to check which moves have them. ALSO the response from the refree in case of click is different from the 'put' of the previous version. Now the refree is sending moves back to the gui , so a new command was necessary.

Another requirement I would like to impose is that the system can still provide basic functionality without a special referee engine, i.e. using an engine that supports nothing but original WB protocol. (In which case the engine should be able to act as its own referee.) You would lose the highlighting and one-click moving, for sure, but these are merely frills. Move entry and legality checking should always work, though. So there must be some simple default mechanism for the GUI to synthesize moves from the clicks, without using any rule knowledge.

Yes a bug I made and still didn't fix is that in FICS mode I do not use the refree. After I reverted to this new way, the app started crashing because even in FICS mode it needs refree! Well I added the previous code that sends moves on the 2nd click, and constructs them in plain e2e4 notation.
But this is not good because other games with 3 or more clicks could be served, and also complete lack of highlighting. A more serious problem I observed is that boards are not displayed until servers sends them so it could do away with a refree that will update the board as soon as the human enters the moves... So I am thinking to use a referee as well. Without the referee you can only come up with solutions that are good for a specific kind of game.

I agree that the Ctrl key for partial moves is a pain, though. (I'd much rather use it for something else...) Making the engine or referee responsible for this would be nice. I am just not sure that this is always possible, though. Take again the example of the Chu-Shogi Lion. It can do two King moves (even if both are captures) in one turn. But it can also do just one. So how can you enter that? After clicking Lion and first victim there is already a legal move, but there could also be a continuation (eight, actually). But if you want to stop there, how can you tell it to the engine?

Good question I didn't notice that ..

I guess reverse entry could be a convention to make this work. Click any number of empty or opponent squares before you click a piece of your own, and it will capture them all with a multi-leg move (i.e. send 'usermove LEG1,LEG2,LEG3,...' to the engine). With an engine that supports 'click' there would be much more flexibility: the GUI could send the individual squares in any order, and keep that up until the engine/referee responds with 'move' (in the mean time performing highlighting as requested). An engine would be aware of the possible ambiguities in move termination, and could apply its own conventions to resolve them (e.g. to terminate entry that might have continued, click the same piece again as last time).

I will think about this for a while. It seems to me that some signaling maybe requrired to make the shorter move, since the longer continuations always contains the squares in side the shorter ones. Otherwise it will get stuck (never makes a move) because it can't resolve it. We can use 'clicking on the from' square _again_ as a signal to tell it to make the move now, that means the move that contains exactly all the squares that are clicked and no more. Is that what you meant ?

Problems I don't clearly see a solution for at the moment are:
- With reverse multi-leg entry, how can you force a reset of the entry process after the user makes an erroneous click? (Both with stand-alone GUI or engine-mediated.) It should always be possible for the user to change his mind before the full move is committed. Perhaps also click the last selected target a second time, just as WB allows you to click your own piece a second time to deselect it? I guess multiple (square-border or arrow) highlights could be supported, that go on even if you click opponent pieces or empty squares, and would all go off on a deselect click.

That is implemented already. If the user make a wrong continuation, Nebiyu resets its clicked squares list that it keeps inside and the clicking is restarted from the last square. This works already.

- If the user does want one-click moving to be off, how to achieve that? It would be nasty if a click-supporting engine has to know it is off, yet a GUI would have to send all the clicks to evoke highlighting responses. Just shelve the referee's move until the user has clicked all the squares stipulated by the move? And send 'undo' if he aborts or deviates?

I see. We will have to think a way for that. Right now I turn both on or off..

- It would be nice if highlighting could somehow be used to determine clicks illegal already in the GUI: waiting for an 'Illegal move' engine response to undo the just entered move produces notable glitches in the display.

I forgot how we used highilight to mark illegal squares in the previous method. I guess that was on hover or not? Anyway this is another thing to discuss.