Mac OS CMD clicking

[khildebr]

From another post I understand that to send a command+C for copying text I have Robot press Alt+C. But when trying to command+click it doesn't work. I have Robot click with modifiers=Alt, but this doesn't work. It ends up not even clicking. If I however manually Alt+click through the viewer in Robot it does work. But when I have Robot record my mouse actions and replay them it doesn't work.

[rpes]

Can you try a composed mouse click?

Press Alt release="true" wait=100
Mouse click to=<coords>
Press Alt release="true"

Perhaps it's just a timing issue and the target system requires some time elapsed between the Alt key and the mouse click.

[khildebr]

Press Alt release="true" wait=100
Mouse click to=<coords>
Press Alt release="true"

I tried this but Alt doesn't seem to be getting pressed. I also tried it with the first release set to false. The click is now being registered but it doesn't appear that Alt is actually being pressed when its clicking.

[rpes]

Oh, the "release=true" in the first command was a typo. It should have been "release=false" as you correctly figured out.

I tested the sequence with the RFB logging on:

1. Go to Edit->Preferences and select the "RFB (VNC) 3.x Client" panel
2. Set the "Console debug logging" at least to "Init, config & client messages"
3. Check the console (command prompt) or the application log of the sequence of mouse & keyboard events.

The log for the code above shows this:

Code: Select all

C2S: Processing key event: [KEY_PRESSED,keyCode=18,keyText=Alt,keyChar=Undefined keyChar,keyLocation=KEY_LOCATION_STANDARD,rawCode=0,primaryLevelUnicode=0,scancode=0]
C2S: KeyEvent sent at 15:50:56,922: pressed ALT, [0]
C2S: Mouse event sent at 15:50:57,698, [MOUSE_MOVED,(85,139),absolute(0,0),button=0,clickCount=1] X [5, 0, 0, 85, 0, -117, 0]
C2S: Mouse event sent at 15:50:57,729, [MOUSE_MOVED,(89,129),absolute(0,0),button=0,clickCount=1] X [5, 0, 0, 89, 0, -127, 0]
C2S: Mouse event sent at 15:50:57,760, [MOUSE_MOVED,(92,120),absolute(0,0),button=0,clickCount=1] X [5, 0, 0, 92, 0, 120, 0]
C2S: Mouse event sent at 15:50:57,791, [MOUSE_MOVED,(96,110),absolute(0,0),button=0,clickCount=1] X [5, 0, 0, 96, 0, 110, 0]
C2S: Mouse event sent at 15:50:57,822, [MOUSE_PRESSED,(100,100),absolute(0,0),button=1,modifiers=Button1,clickCount=1] X [5, 1, 0, 100, 0, 100, 0]
C2S: Mouse event sent at 15:50:58,125, [MOUSE_RELEASED,(100,100),absolute(0,0),button=1,modifiers=Button1,clickCount=1] X [5, 0, 0, 100, 0, 100, 0]
C2S: Processing key event: [KEY_RELEASED,keyCode=18,keyText=Alt,keyChar=Undefined keyChar,keyLocation=KEY_LOCATION_STANDARD,rawCode=0,primaryLevelUnicode=0,scancode=0]
C2S: KeyEvent sent at 15:50:58,357: released ALT, [0]

This proves that the Alt key gets pressed before and released after the mouse click. There are two points to try out:

1. Perhaps the server doesn't like the Alt key being pressed before the mouse gets moved to the click location. Can you try with this code?

Mouse move to=x:100,y:100
Press Alt release="false" wait=100
Mouse click to=x:100,y:100
Press Alt release="true"

2. If even this doesn't work could you please enable the log and then post the logs for (1) manual Alt+click and (2) the automated Alt+click using the code above? This would clearly show whether there is any difference.

Thanks a lot!

[khildebr]

This is the log for the manual Alt+click.

Code: Select all

C2S: Processing key event: [KEY_PRESSED,keyCode=18,keyText=?,keyChar=Undefined keyChar,modifiers=?,extModifiers=?,keyLocation=KEY_LOCATION_LEFT,rawCode=0,primaryLevelUnicode=0,scancode=0]
C2S: KeyEvent sent at 10:57:09,769: alt pressed ALT, [4, 1, 0, 0, 0, 0, -1, -23, 0]
C2S: Mouse event sent at 10:57:12,873, [MOUSE_PRESSED,(691,225),absolute(1544,332),button=1,modifiers=?+Button1+Button2,extModifiers=?+Button1,clickCount=1] X [5, 1, 2, -77, 0, -31, 0]
C2S: Mouse event sent at 10:57:13,017, [MOUSE_RELEASED,(691,225),absolute(1544,332),button=1,modifiers=?+Button1+Button2,extModifiers=?,clickCount=1] X [5, 0, 2, -77, 0, -31, 0]
C2S: Processing key event: [KEY_RELEASED,keyCode=18,keyText=?,keyChar=Undefined keyChar,keyLocation=KEY_LOCATION_LEFT,rawCode=0,primaryLevelUnicode=0,scancode=0]
C2S: KeyEvent sent at 10:57:13,641: released ALT, [4, 0, 0, 0, 0, 0, -1, -23, 0]

This is the code for the Alt+click using the code in your reply (Just changed where it clicks)

Code: Select all

C2S: Processing key event: [KEY_PRESSED,keyCode=18,keyText=?,keyChar=Undefined keyChar,keyLocation=KEY_LOCATION_STANDARD,rawCode=0,primaryLevelUnicode=0,scancode=0]
C2S: KeyEvent sent at 10:50:02,464: pressed ALT, [0]
C2S: Mouse event sent at 10:50:02,565, [MOUSE_MOVED,(696,224),absolute(0,0),button=0,clickCount=1] X [5, 0, 2, -72, 0, -32, 0]
C2S: Mouse event sent at 10:50:02,575, [MOUSE_MOVED,(696,224),absolute(0,0),button=0,clickCount=1] X [5, 0, 2, -72, 0, -32, 0]
C2S: Mouse event sent at 10:50:02,585, [MOUSE_PRESSED,(696,224),absolute(0,0),button=1,modifiers=Button1,clickCount=1] X [5, 1, 2, -72, 0, -32, 0]
C2S: Mouse event sent at 10:50:02,615, [MOUSE_RELEASED,(696,224),absolute(0,0),button=1,modifiers=Button1,clickCount=1] X [5, 0, 2, -72, 0, -32, 0]
C2S: Processing key event: [KEY_RELEASED,keyCode=18,keyText=?,keyChar=Undefined keyChar,keyLocation=KEY_LOCATION_STANDARD,rawCode=0,primaryLevelUnicode=0,scancode=0]
C2S: KeyEvent sent at 10:50:02,651: released ALT, [0]

The mouse moved before the Mouse press because I have the option to generate a mouse move before a click selected. I also tried it with this option turned off but there was still no difference other than the two move events not being in the log before the mouse press.

Still the manual Alt+click works but the automated one doesn't seem to be Pressing the Alt button even though the two logs are quite similar.

[rpes]

I finally reproduced the failure and I succeed to identify the cause. As I didn't find a use for Alt+click I tested the behavior of Alt+drag which is supposed to drag the window on Linux.

It turned out that Java for some reason internally converted a left mouse click combined with Alt to a middle mouse click. As a consequence of this behavior the Robot simulated a left mouse click but the server received a middle one (with no Alt key pressed).

I updated the code to prevent this situation and I successfully tested an Alt+mouse drag action on Linux. I was wondering whether you could upgrade to the latest 2.3.5Beta from today and confirm whether it works on your environment? It would be great if you could try out both the composed action (alt press, mouse click, alt release) and the single command ("Mouse click modifiers=Alt"). Both should work fine now.