Wayland fractional scaling discussion

[simonioc]

This issue was reported countless times this week, sorry for being pedantic, but i think it will affect native Wayland clients adoption if it's not resolved.
This is a continuation of #12079 (comment) , which was kind of resolved but the clients are still left to being blurred out, because they don't know about fractional scaling.
Also i think some things would completely break when their logical pixel size does not match the real one.
I think it would be better for clients to be explicit about supporting fractional scale, because many people developing the programs may not have it enabled and it wouldn't be a test case for them.

[simonioc]

Unfortunately i am not informed enough, but how is it handled on platforms such as Windows and OSX whose fractional scaling implementation dates back a few years more than Wayland?
Do they break things in order to support it or they transparently work around it?

[slouken]

I'm going to leave this open so we can discuss it with @Kontrabant when he gets back, but here is how high DPI is handled on other OSes in SDL:

Windows and Linux X11

The window coordinate system is always in pixels, and you can query the display content scale, which is an optional scale that the application should use to scale up windows and UI elements so they are visible and easily interactable on the screen.

MacOS

The window coordinate system is always in points, and you can request that your window have high pixel density, which means that your back buffer will be an integer multiple of the window size.

Wayland

I'm not sure which camp this lands in, and I know it's complicated by whether the viewporter extension is available. I think we've landed on making fullscreen games take the full screen regardless of what resolution they choose, but I think we should revisit how we do this.

There are several options for this:

• Only providing the native resolution for the window / desktop (how does this interact with window coordinates? Is that always 1:1?)
• Only provide integer multiples of the native resolution (allows lower resolution content to scale up without blurriness, assuming we can choose whether linear or point sampling is used)
• Provide all common resolutions that a game might use and center/scale+letterbox/stretch the game content into the native resolution.

There are good reasons to choose any of these, which is why this is a discussion, not just a bug report.

Here are some of the considerations, as food for thought:

• What happens on ultra-wide displays, e.g. 32x9 aspect ratios? Games often do not support this aspect ratio. Do you really want to stretch the game twice as wide as it normally is?
• What happens for games that only support low resolutions? The poster child for this is classic DOOM, which runs at 320x200. This is VERY small on a 4K monitor.
• What happens for fill-bound games on a very large monitor? A game like Cyberpunk might get 120 FPS at 1080p, struggle to get 45 FPS at 4K, and drop to a whopping 10-15 FPS displaying on an 8K monitor.

[JustinWayland]

I will contribute from a quick search online that at the very least KDE should support the viewporter extension given that it's built on Qt, which definitely has it.

Another quick search revealed that GNOME also had it at least five years ago. Dunno if it has it now.

So at the very least both big Wayland desktop environments should support the viewporter extension. COSMIC, Hyprland and Sway are for someone else to figure out.

[JustinWayland]

Actually, looking at this the only compositor that currently doesn't support the Viewporter extension is Gamescope. It was implemented relatively recently, but I think most stable distros don't even have SDL3 yet. So long as we aren't running under SDL2, we should have access to the viewporter protocol.

[Kontrabant]

Only provide integer multiples of the native resolution (allows lower resolution content to scale up without blurriness, assuming we can choose whether linear or point sampling is used)

There is no way for client apps to specify this, and the problem here is that rounding up scale values to integers can result in drawing a lot of extra pixels, which is a problem on Mac, and there will still be some blur, as the backbuffer isn't 1:1.

Only providing the native resolution for the window / desktop (how does this interact with window coordinates? Is that always 1:1?)

This is what the scale-to-display hint tries to do, but it can get ugly with multi-monitor desktops, there are ways it can break (some compositor developers believe that a window shouldn't know anything about the display it is on), and in general, it won't feel "native".

Provide all common resolutions that a game might use and center/scale+letterbox/stretch the game content into the native resolution.

We already do this for exclusive fullscreen modes, and have a hint to toggle aspect-correct output. The problem is, some compositors don't strictly follow the spec and center/mask fullscreen windows, which is why the default is to stretch to fill the display at the moment (Gnome didn't have spec-compliant behavior for a long while, and KDE still doesn't). XWayland stretches fake fullscreen modes for the same reason, and Wine has hit this as well, with fullscreen windows that don't match the native display aspect not being centered/masked on KDE.

Other than that, scaled desktops on Wayland basically works the same as on a Mac, with apps/games needing to render at the backbuffer resolution for pixel-perfect output.

Actually, looking at this the only compositor that currently doesn't support the Viewporter extension is Gamescope. It was implemented relatively recently, but I think most stable distros don't even have SDL3 yet. So long as we aren't running under SDL2, we should have access to the viewporter protocol.

SDL already makes extensive use of viewports.

[JustinWayland]

SDL already makes extensive use of viewports.

Excellent.

Sorry that you have to deal with this during your vacation.

[JustinWayland]

Stupid idea: the only thing that would want scale-to-display would be games. Games also don't expect to be run on more than one monitor, right?

What if we were to add a hint that would indicate to the underlying compositor that the window should only render on a single monitor? That would allow games to say "We don't want to deal with multi-monitor edge cases."

Of course, we would need to see if upstream Wayland would even accept such a hint, so we might have to page some developers on Wayland compositors.

Or is this unrealistic?

[Reonu]

I just want to chime in and say that a window size in pixels should always be that window size in pixels (for games specifically, of course)

If I run Cyberpunk and set it to a 1280x720 window, I expect Cyberpunk to be in a window that is 1280x720 pixels in size regardless of whether my DPI scaling is set to 100%, 125% or 600%. If I set Doom to a 320x240 window then I expect Doom to run in a 320x240 window. If I'm playing fullscreen, whether it's via borderless window or exclusive fullscreen, the game needs to know that my resolution is 2560x1440, it can't think that the resolution is 2048x1152. If I enable DLSS in a game it needs to be able to know what resolution to scale to, DLSS becomes useless if it's not scaling to the native resolution, etc etc. Games aren't office apps or whatever, they're games, they're all designed around the idea that you set a resolution and that's the resolution.

This is how it works in Windows. When, occasionally, a game isn't able to properly detect or set the resolution in a high DPI environment, it's seen as a problem and the solution is to enable the DPI compatibility setting in the .exe properties since it's considered broken behavior. This broken behavior seems to be the default in SDL under native Wayland...?

I doubt native Wayland will ever be adopted for games if it doesn't work that way. I don't know who is at fault for this mess, so I'm not trying to blame anybody. If it's an upstream Wayland issue then I hope somebody can make them see how big of a problem this is.

Sorry for the rant.

[slime73]

As far as I understand, Wayland works very similarly to macOS which generally has seamless preservation of the visual scale and relative position of content on screen. Windows took several iterations of different half-measures, so it's not exactly comparable. Apps that are prepared to deal with content scales are expected to opt-in on every OS though.

they're all designed around the idea that you set a resolution and that's the resolution.

In a multi-window environment such as a desktop operating system, operating systems and displays have moved to a model that includes scale alongside pure resolution. It is possible for a game to ignore that, but I wouldn't say it's a good idea with modern displays being so varied.

[Reonu]

As far as I understand, Wayland works very similarly to macOS which generally has seamless preservation of the visual scale and relative position of content on screen. Windows took several iterations of different half-measures, so it's not exactly comparable. Apps that are prepared to deal with content scales are expected to opt-in on every OS though.

If I understand it correctly, games basically aren't DPI scaled on Windows at all, and that's what's expected and what makes sense. Basically, the game is rendered and presented on-screen as if the DPI scaling was set to 100%. Can't this be achieved on Wayland? This is the expected and logical behavior for games.

[slime73]

Can't this be achieved on Wayland?

Have you tried SDL_WINDOW_HIGH_PIXEL_DENSITY (SDL3's window flag) or SDL_WINDOW_ALLOW_HIGHDPI (SDL2's window flag)?

SDL3 generally provides enough information for you to choose whichever way you want to handle scaled displays, and it tends to go with what the rest of the OS does in terms of the window position's coordinate space etc.

[Reonu]

Can't this be achieved on Wayland?

Have you tried SDL_WINDOW_HIGH_PIXEL_DENSITY (SDL3's window flag) or SDL_WINDOW_ALLOW_HIGHDPI (SDL2's window flag)?

Yes, I mentioned in #12161 that I tried that and the issue remains. The game looks blurry on my 1440p monitor at 125% scaling with KDE Plasma 6.2.5 in native Wayland, but it looks sharp and perfect in xWayland as expected. The game is also the wrong window size in native Wayland mode. We're on SDL2 btw.

Here are some screenshots, note that in both cases I haven't touched the window size, it just booted into the wrong size in native Wayland. The difference might be subtle if you aren't used to spotting these things but hopefully you can tell that the edges are "softened" in the Wayland screenshot since the window is being scaled up for no reason. Of course you need to see both screenshot at 100% size, both will look blurry if you expand them to fit your screen.

xWayland:

Native Wayland:

[Reonu]

I suspect @misyltoad will have interesting thoughts on this as well so I'm summoning them

[misyltoad]

Ideally in SDL_WINDOW_HIGH_PIXEL_DENSITY you'd be pixel for pixel.

In Gamescope, that's what I try to do anyway in my WaylandBackend for that, and I thought I was actually doing the same things SDL was here... Do you have the same issues with that?..

Like, I thought I used wp_viewporter + wp_fractional_scaling in basically the exact same ways. Maybe there's a regression somewhere.

[simonioc]

I just want to chime in and say that a window size in pixels should always be that window size in pixels (for games specifically, of course)

If I run Cyberpunk and set it to a 1280x720 window, I expect Cyberpunk to be in a window that is 1280x720 pixels in size regardless of whether my DPI scaling is set to 100%, 125% or 600%. If I set Doom to a 320x240 window then I expect Doom to run in a 320x240 window. If I'm playing fullscreen, whether it's via borderless window or exclusive fullscreen, the game needs to know that my resolution is 2560x1440, it can't think that the resolution is 2048x1152. If I enable DLSS in a game it needs to be able to know what resolution to scale to, DLSS becomes useless if it's not scaling to the native resolution, etc etc. Games aren't office apps or whatever, they're games, they're all designed around the idea that you set a resolution and that's the resolution.

I agree with this, Wayland behavior should be as close as possible to X11 and Windows, Mac is on a special position because they already break things often and the majority of people there run a high dpi display.
I'm sure there's a way for applications to work around this but i don't think that's possible for proprietary games that haven't been updated since 2014 (there are lots of those on Steam).
Open source projects too wouldn't be happy because they don't want to expect scaling dictated from the system, considering many do handle it themselves.