Fixing OBS Offset Cursor Issue, Part 3 (investigating with Python-C++ interfaces!)

Poking and prodding at the Windows API

Background

There is an issue with cursors being rendered offset for DPI-scaled windows in OBS, illustrated in this video clip:

I’ve been looking into this as it was suggested that my proposed fix of letting the user specify an offset amount was more of a workaround of the underlying issue. It’s a bit like a coffee machine, apparently.

To make a useful and comprehensive fix I wanted to do a bit more research, as there are a few points outstanding.

Why are some games unaffected?

The last question I posed previously as a stinger was: why are some games’ cursors unaffected by the scaling issue?

The answer seems straightforward from what I can tell: they render their own cursor. They presumably do this and hide the Windows cursor (ie unset the CURSOR_SHOWING flag). When OBS comes to render the cursor, it notes that flag and skips rendering it.

Simple!

Can we use DPI-query APIs to get the window’s scale?

Determining a window’s scale factor programmatically was mooted to obviate the need for the user to manually set a scaling factor, which would be potentially confusing UX for users who have the issue, and unnecessary for those who don’t.

Windows provides a GetDpiForWindow function to do just this. While obs-studio already has the interface to the API defined, it’s a bit slower to work from C++ to poke around at Windows API functions. So I used the native python ctypes library to poke at it from a a python REPL. More on that elsewhere!

Unfortunately for scaled games I was still getting a DPI value of 96 for any game I cared to test. I checked Windows explorer, which did return a DPI value of 168 as expected (96 * 175% scaling).

Boo.

But why? I had a notion, and checked GetWindowsDpiAwarenessContext for the processes. They returned some interesting numbers, which mapped to enum values: 1073766416 = DPI_AWARENESS_CONTEXT_UNAWARE_GDISCALED, 24592 = DPI_AWARENESS_CONTEXT_UNAWARE (cf DPI_AWARENESS enumeration).

So they were being reported as DPI unaware. That’s a bit odd, particularly for relatively modern games. made well after Windows introduced DPI awareness settings (they changed a little after Windows 8, but they were there!).

I double-checked with Task Manager, which can show ‘DPI awareness’ (in the ‘Details’ tab) for processes. That confirmed what I saw. I checked Mahjong Nagomi, and saw that actually did have DPI awareness. Oho! It turns out that I never actually scaled that game, and was simply running in 2550×1440.

So it would seem that by turning on scaling, the process/window becomes DPI-unaware. As a consequence of this, any query of the window’s DPI will return 96:

DPI unaware. This process does not scale for DPI changes and is always assumed to have a scale factor of 100% (96 DPI). It will be automatically scaled by the system on any other DPI setting.

DPI_AWARENESS

See also: Windows lies about DPI over on Stack Overflow.

This seem counter-intuitive, but as over at MSDN:

DPI unaware applications render at a fixed DPI value of 96 (100%). Whenever these applications are run on a screen with a display scale greater than 96 DPI, Windows will stretch the application bitmap to the expected physical size. This results in the application appearing blurry.

High DPI Desktop Application Development on Windows

Since we’re forcing Windows to do scaling on a DPI-aware application, we’re basically taking that DPI-awareness, chucking it away and saying “do a simple bitmap scale on this instead”.

Where does the OBS cursor bug occur?

While poking around the Windows APIs and the OBS code, I got to wondering where the actual issue is. The relevant workflow for the cursor seems to be:

1. Get the cursor info
   
   This uses GetCursorInfo, which returns a CURSORINFO structure. That structure contains the cursor coordinates in a POINT structure- ptScreenPos, which is the screen coordinates of the cursor.
2. Render the cursor
   
   To do this, game_capture_render_cursor() in OBS calls ClientToScreen to transform client coordinates — in this context, ‘client’ means a window’s draw area — supplying 0, the top-left part of the client area. This is then used as a (negative) offset to cursor_draw(), transforming the screen coordinate to the position on the screen of the window- the client coordinate.
3. The rub:
   
   OBS believes the game window to be unscaled- ie a 1920×1080 window is actually that size instead of the scaled value of 3360×1890.
   
   Using GetClientRect confirmed that- it returned the unscaled size.

So because of the above, the cursor position is obtained relative to the screen, but the output is relative to the unscaled window size, and so cursor position/movement is magnified by the scaling factor in rendered output.