Android: Pen creates phantom mouse events when relative mode set

[AntTheAlchemist]

With SDL_HINT_PEN_MOUSE_EVENTS disabled, and SDL_SetWindowRelativeMouseMode on, pen events leak into mouse motion and button events.

Tested with a real pen-tablet, but this can also be seen on a Chromebook / desktop emulator in Android Studio, which seems to emulate pen events from physical mouse input.

[icculus]

Tried this on a physical Android phone with the latest in main and a physical tablet plugged into the phone's USB port, and I'm not getting mouse events at all.

My test case is just test/testpen.c with relative mouse mode and the hint set, plus some debug logging added:

diff --git a/test/testpen.c b/test/testpen.c
index 214ee0594..2f32d088b 100644
--- a/test/testpen.c
+++ b/test/testpen.c
@@ -39,6 +39,9 @@ SDL_AppResult SDL_AppInit(void **appstate, int argc, char *argv[])
 {
     int i;
 
+SDL_SetHint(SDL_HINT_PEN_MOUSE_EVENTS, "0");
+SDL_SetHint(SDL_HINT_EVENT_LOGGING, "2");
+
     SDL_srand(0);
 
     /* Initialize test framework */
@@ -89,7 +92,9 @@ SDL_AppResult SDL_AppInit(void **appstate, int argc, char *argv[])
         SDL_UpdateTexture(white_pixel, &rect, pixels, 16 * sizeof (Uint32));
     }
 
-    SDL_HideCursor();
+//    SDL_HideCursor();
+
+SDL_SetWindowRelativeMouseMode(state->windows[0], true);
 
     return SDL_APP_CONTINUE;
 }
@@ -212,6 +217,10 @@ SDL_AppResult SDL_AppEvent(void *appstate, SDL_Event *event)
             break;
         }
 
+        case SDL_EVENT_MOUSE_MOTION:
+            SDL_Log("MOUSE MOTION!");
+            return SDL_APP_SUCCESS;
+
         case SDL_EVENT_QUIT:
             return SDL_APP_SUCCESS;
 

I am getting finger events through the tablet, which I assume SDL synthesized as I didn't set the hint to disable...but no mouse events at all, and nothing in adb logcat that says "MOUSE MOTION!" for that specific event firing.

A lot of this, specifically:

05-28 16:58:50.493 28842 28842 I SDL/APP : SDL EVENT: SDL_EVENT_PEN_AXIS (timestamp=17031284795 windowid=2 which=3 pen_state=2147483649 x=1398.93 y=727.318 axis=PRESSURE value=0.41221)
05-28 16:58:50.493 28842 28842 I SDL/APP : SDL EVENT: SDL_EVENT_FINGER_MOTION (timestamp=17031304795 touchid=18446744073709551614 fingerid=1 x=0.597832 y=0.673443 dx=0.000000 dy=0.000000 pressure=0.412210

To rule out an app bug, can you see if this test case triggers it on your end?

(My other question, if this does still trigger for you with this test case: are the unwanted mouse events being synthesized by SDL, or is whatever version of Android helpfully generating them itself, so SDL thinks it's getting both pen and mouse events and doesn't know better...?)

[icculus]

Building testpen here was just these shell commands, in case there's something more complex I skipped, like a serious AndroidManifest.xml or whatever.

cd SDL
./build-scripts/create-android-project.py org.icculus.testpen test/testpen.c src/test/SDL_test_* test/testutils.*
cd build/org.icculus.testpen
./gradlew installDebug && adb logcat |grep SDL/APP

[AntTheAlchemist]

I think I found the culprit. The pen creates a MotionEvent.ACTION_HOVER_MOVE event and sends it to onCapturedPointerEvent, which sends it to SDLActivity.onNativeMouse. onCapturedPointerEvent needs to check tool type, like onTouch?

[icculus]

You might be right; for whatever reason I'm not firing this event handler here.

I'm wondering if we just need to have onCapturedPointerEvent() be simply...

    @Override
    public boolean onCapturedPointerEvent(MotionEvent event)
    {
        return SDLActivity.getMotionListener().onGenericMotion(this, event);
    }

...and be done with it, since it seems to do the same thing but with more robust handling.

[icculus]

I set up an emulator, in case the Samsung phone was the culprit, but I still don't ever get onCapturedPointerEvent calls.

I tried bumping the minimum API to 26 in the manifest, that didn't change anything, either. I don't know how to coerce Android to use this event here.

That being said, I don't suppose you can try this patch? It might just fix the actual problem, but I can't test it here for whatever reason.

diff --git a/android-project/app/src/main/java/org/libsdl/app/SDLSurface.java b/android-project/app/src/main/java/org/libsdl/app/SDLSurface.java
index 196cf04ec..e28870fac 100644
--- a/android-project/app/src/main/java/org/libsdl/app/SDLSurface.java
+++ b/android-project/app/src/main/java/org/libsdl/app/SDLSurface.java
@@ -402,45 +402,7 @@ public class SDLSurface extends SurfaceView implements SurfaceHolder.Callback,
     @Override
     public boolean onCapturedPointerEvent(MotionEvent event)
     {
-        int action = event.getActionMasked();
-        int pointerCount = event.getPointerCount();
-
-        for (int i = 0; i < pointerCount; i++) {
-            float x, y;
-            switch (action) {
-                case MotionEvent.ACTION_SCROLL:
-                    x = event.getAxisValue(MotionEvent.AXIS_HSCROLL, i);
-                    y = event.getAxisValue(MotionEvent.AXIS_VSCROLL, i);
-                    SDLActivity.onNativeMouse(0, action, x, y, false);
-                    return true;
-
-                case MotionEvent.ACTION_HOVER_MOVE:
-                case MotionEvent.ACTION_MOVE:
-                    x = event.getX(i);
-                    y = event.getY(i);
-                    SDLActivity.onNativeMouse(0, action, x, y, true);
-                    return true;
-
-                case MotionEvent.ACTION_BUTTON_PRESS:
-                case MotionEvent.ACTION_BUTTON_RELEASE:
-
-                    // Change our action value to what SDL's code expects.
-                    if (action == MotionEvent.ACTION_BUTTON_PRESS) {
-                        action = MotionEvent.ACTION_DOWN;
-                    } else { /* MotionEvent.ACTION_BUTTON_RELEASE */
-                        action = MotionEvent.ACTION_UP;
-                    }
-
-                    x = event.getX(i);
-                    y = event.getY(i);
-                    int button = event.getButtonState();
-
-                    SDLActivity.onNativeMouse(button, action, x, y, true);
-                    return true;
-            }
-        }
-
-        return false;
+        return SDLActivity.getMotionListener().onGenericMotion(this, event);
     }
 
     @Override

[AntTheAlchemist]

It's maddening that you can't repro even on the emulator. Let me strip it down and do some more digging to see what's different about my build.

[icculus]

Any chance you got a moment to look at this? Either if the patch fixes it for you, or if there's some other indication as to why you get the events and I don't.

(If the patch works, it's good enough for me, since it would solve both your codepath and the one my phone and emulator ended up taking.)