Fix pen input not being friendly with touch

[frenzibyte]

Fixes pen & touch osu! playstyle on mobile tablets no longer working in the past release.

---

• Closes Cannot play osu! with pen + touch input osu#31570

Back before SDL was bumped, SDL was propagating pen input on iOS in two forms:

• mouse move input when the pen is within the screen's region but not touching it.
• touch input when the pen is directly touching the screen.

This was working perfectly fine with our implementation of OsuTouchInputMapper in osu! for an elegant pen + touch playstyle. But since we bumped SDL and it became handling pen input natively on iOS and now became directly mapped to mouse input by our implementation in SDL3Window / #6488, everything fell apart.

I initially began on this by looking into updating the implementation of OsuTouchInputMapper to expect pen input by mouse events, but bailed out after finding multiple catastrophic input-level issues by the idea of sending pen input as mouse input (see #6508 / https://discord.com/channels/188630481301012481/589331078574112768/1332383057642000426).

I then attempted to dive right into the o!f input subsystem and try to get things in a right place, then I realised I'm essentially about to propose a top-down refactor of major input components, and I'm absolutely in no place, time, or even interest in doing any of that (nor do I think that anyone will be supporting this effort right at this very moment).

Finally, I decided to just remind myself of how osu! has always been working until now and mimicked that behaviour for now, calling it a day and allowing ourselves to focus on much better and prioritised tasks. In fact, I think the behaviour in this PR might even make more sense with regards to the idea that we want pen and touch input to work harmoniously.

I have tested this on my iPad and it's working perfectly fine everywhere.

[Walavouchey]

was sort of hoping this would also fix my use case of surface pro pen + touch input on windows (screen capture), but that has never worked unlike the ios pen

ideally i think mouse aim + touch tapping should be supported as well but we'll see

[frenzibyte]

Are you testing with SDL3?

[Walavouchey]

hmm, same results. made a discussion #6510 and put the logs there

[hwsmm]

You may need to wait for the next SDL3-CS release since SDL3 added Windows Ink support after our last update.

I also tried it on Android, but I think Pen+Touch playstyle never worked on Android (at least on my device, Galaxy Tab S7 FE) due to how palm rejection is implemented. My device just rejects all touch inputs while the pen is near the screen.

On osu!, this PR enables Touch device mod while pen is down, and disables it while pen is hovering. I think it may need a fix.

[Susko3]

I don't like this change, but it seems this is the simplest fix to make osu! work, so it can go in as a hack. I would like to see the following:

• Add a FrameworkSetting flag that games (osu!) can set a default
• Place the pen handling introduced in this PR behind the flag
• Restore master behaviour when the flag is disabled
• Add pen-as-mouse-and-touch tests to TestSceneOsuTouchInput
  • It should be easy to modify the tests so they check actual pen input (i.e. make a helper function that calls InputManager.BeginTouch(), so that it can be substituted for InputManager.PressPen() when someone starts to implement proper pen input)

[frenzibyte]

• Add a FrameworkSetting flag that games (osu!) can set a default
• Place the pen handling introduced in this PR behind the flag
• Restore master behaviour when the flag is disabled

I don't know about any of this. Why are we bringing back something that must be working perfectly fine with this PR, it's just introducing a flag that I can't imagine anyone will be bothered enough to use it?

If anything, I can move the application of touch input within PenHandler itself so as to not bother the SDL implementation enough with that, but that's code quality at best.

• Add pen-as-mouse-and-touch tests to TestSceneOsuTouchInput

  • It should be easy to modify the tests so they check actual pen input (i.e. make a helper function that calls InputManager.BeginTouch(), so that it can be substituted for InputManager.PressPen() when someone starts to implement proper pen input)

I have something like that prepared already, I'll PR it out.

[bdach]

Not that I can bring myself to particularly care about any of this, but this here looks immediately incorrect and a complete stretch just to make apple pencil specifically work. This has no semantic sense elsewhere. SDL generally classifies drawing tablet pens as pens, so treating it as a mobile stylus specifically is an overreach that can bite us in the rear later.

Maybe if you constrained this to only be a mobile thing I could accept it.

[frenzibyte]

Well it's entirely weird how SDL pen input can originate from "indirect" / drawing tablet devices in the first place when we have local handling for those devices specifically via OTD handlers, but alas.

This logic makes sense for any kind of touchscreen with a stylus on it, be it Apple or Android or even Microsoft Surface with a Surface Pen I guess.

However, since there's no way to determine whether pen input originates from an on-screen pen or a drawing tablet pen, I can only resort to an is-mobile conditional.

[frenzibyte]

Blocking this momentarily while an alternative fix may supersede this.