integration/Docs/Luprex-Window-Management.md

Introduction

Unreal has several input mode-related subsystems that interact with each other in complicated ways. These subsystems include:

• keyboard focus
• mouse capture
• enhanced input routing
• pointer visibility
• window z-order

Unreal is littered with conditionals that cause these bits of state to affect each other in unpredictable, often illogical ways. If you set these bits of state in the wrong order, or to the wrong values, it is all too easy to get unreal into a non-functioning state. The system is much too fragile.

For this reason, I have implemented a window management system that orchestrates all of this from a centralized location, in a way that guarantees reasonably predictable, sane behavior.

Core Design Choices

Our window management system, in order to keep things simple, has to make some assumptions about how Luprex games work. So, here are the rules.

The presumption is that most of the time, you're interacting with the 3D world, and importantly, we assume that if you're using keyboard and mouse, you're using the mouse to control the camera - aka "mouselook."

We also assume that as you interact with the 3D world, you will occasionally be popping up GUI widgets that can coexist with mouselook. These mouselook-compatible widgets don't need a mouse pointer, they don't need you to click on anything. They rely on buttons alone. We assume that most of the GUI elements you interact with will be mouselook-compatible, in order to allow you to stay immersed in the 3D world.

But we also assume that there may be moments when you want to pop up a very complicated widget, for example, a big inventory management screen, for which a mouse pointer would be very helpful. For occasions like these, a widget can declare "ShowPointer".

When one of these ShowPointer widgets is on the screen, the entire system switches into point-and-click mode. In point-and-click mode, the pointer is visible. Mouse movements move the pointer. Mouse movements do not get translated into mouselook.

Widgets have a z-order: one widget is always "in front." In mouselook mode, only the front widget can get keyboard focus. In mouselook mode, the window management system will put focus on the front widget, and it will keep it there. If you want some other widget to have focus, you'll have to bring that widget to the front.

In point-and-click mode, the keyboard focus rules differ. When you raise a widget to the front, the window management system will give it focus. But, if you click the mouse pointer on a different widget - say, on a text box in a different widget - keyboard focus can get transferred.

When a ShowPointer window is on the screen, not only does the system shift to point-and-click mode, but it also keeps all ShowPointer windows in front of any non-ShowPointer window.

Basically, you can think of the system as a 3D world with its mouselook-compatible widgets as one layer, and the point-and-click stuff as a second layer on top of that. When the point-and-click layer gets out of the way, then you can drive the 3D world.

The Root Canvas

Typically, in Unreal, when you create a new top-level widget, you insert it into the viewport using AddToViewport. But to use our window management system, you must instead insert top-level widgets into a 'root canvas', using AddWidgetToRoot.

The main reason for the creation of the root canvas is that it gives us a place to store window-management related hints, and window-management related state.

The root canvas object attaches a RootCanvasSlot to each top-level widget. The RootCanvasSlot is a place where we can store management-related hints for that widget. The contents of the RootCanvasSlot include the following:

• ShowPointer: If true, this is a point-and-click widget. When this widget is in front, the pointer is visible, and the system switches to point-and-click mode.

• BlockInput: If this window is in front, all enhanced input events in other objects are blocked.

• EnableEnhancedInput: If false, enhanced input events in this widget are disabled.

• BringToFrontCount: Effectively, a timestamp indicating the last time this window was brought to the front.

In addition, the top-level widget itself contains some window-management related properties. Currently, these are:

• DesiredFocusWidget: Indicates which sub-widget, if any, should be given focus.

That is all the state variables that control our new window management system. If your blueprint is managing these properties, then it is doing everything it needs to do.

The function SetWidgetWindowManagement can set all of these properties in a single operation. That one function is all you need to control the entire window management system.

Handling Keyboard and Gamepad Buttons

Here is a summary of how keyboard/gamepad button handling in unreal works. We have tweaked this slightly, but this is mostly just ordinary unreal input handling:

When you press a keyboard or gamepad button, the button first goes to any widget that has keyboard focus. If that widget doesn't declare the button to be "handled", then button is offered to other widgets higher in the widget heirarchy. If no widget handles the button, the button then goes to the "enhanced input subsystem."

The enhanced input system puts the button through an "input mapping context." Basically, that's a many-to-one map that translates buttons into more abstract "enhanced input events." Here's a fragment of a typical input mapping context:

Key W --> IA_Move_Forward Key S --> IA_Move_Backward Left_Thumbstick_Forward --> IA_Move_Forward Left_Thumbstick_Backward --> IA_Move_Backward

What the mapping context buys you is that you can handle events like "IA_Move_Forward" without having to care whether the player is driving with the WASD keys or with the gamepad left thumbstick.

Typically, enhanced input events go to all of the following: the player controller, the character, and user-defined widgets. All of these consumers of enhanced input are automatically registered to receive enhanced input, which means that all they have to do is implement a handler in their event graph, and they're ready. Other actors can also receive enhanced input, but that requires jumping through some hoops.

It's interesting that a widget can implement a handler for a raw keyboard button, and then declare the button "not handled". If the button proceeds to the enhanced input system, and if the widget has a handler for enhanced input, the widget can receive the same button again, in a different form!

There is a priority order among consumers of enhanced input events: user widgets first (front-to-back), then the player controller, then the character. A consumer of enhanced input has the option of blocking input to lower-priority consumers.

This is all almost entirely unchanged from Unreal's default behavior. We've only made two tiny tweaks: we send enhanced input to widgets in front-to-back order, and, widgets disable enhanced input by setting a flag instead of by unregistering their input component.

Handling mouse buttons

Mouse buttons behave differently than keyboard buttons.

Widgets have an OnMouseDown handler. This is only active in point-and-click mode. OnMouseDown only fires when three things are true: the pointer is visible, the pointer is inside the rectangle of a widget, and the widget is marked hit-testable.

If no OnMouseDown event fires, or if OnMouseDown declares the mouse down to be "not handled," then the mouse down makes it to the enhanced input subsystem.

Once the mouse down reaches the enhanced input system, it starts being treated the same as keyboard and gamepad buttons. It can be mapped to an enhanced input event by the input mapping context, and then from there, it can be handled by any enhanced input event handler in a blueprint.

The upshot of all this is: if you want to think of a mouse button as "just another button," then the way to achieve that is to handle the mouse button using an enhanced input handler.

We have very slightly tweaked the default behavior of unreal. If the pointer is visible, and you click on a widget that is hit-testable, but which has no OnMouseDown handler, we provide a default OnMouseDown behavior: we bring the widget to the front. Because our system grants keyboard focus to the widget in front, this will grant focus, if the widget can accept it.

Handling Mouse Movement

In point-and-click mode, mouse movement moves the pointer and doesn't generate any events at all.

There is one exception: mouse capture. If you click on a hit-testable widget, that widget will "capture" the mouse until you release the mouse button. As long as the widget has capture, it receives OnMouseMove events. This is mainly intended to implement click-and-drag, scroll bar scrolling, and other movements like that.

Unreal has a lot of complicated mouse capture and mouse lock options and modes. We don't support any of that. We support only the basics: automatic capture when you click. If you need more, we'll have to improve the Luprex window management system.

In point-and-click mode, mouse movements do not go to the enhanced input system at all.

When the system is in mouselook mode, mouse movements go directly to the enhanced input system. They get mapped by the input mapping context and turned into enhanced input events. Handling these events is how mouselook works.

Handling Analog Joysticks

Analog joysticks (including gamepad thumbsticks) generate events that go directly to the enhanced input subsystem. They get mapped to enhanced input events. From there, they can be handled by any consumer of enhanced input.

Functions you Should NOT CALL!

If you're using our Luprex window management system, there are several things your blueprint should NOT do:

• DO NOT use SetKeyboardFocus, SetUserFocus, or any other function with Set-Focus in the name. Instead, set the DesiredFocusWidget inside a top-level widget, and our window management system will decide who gets focus.

• DO NOT use SetShowMouseCursor, or set the bShowMouseCursor flag. Instead, set the ShowPointer flag in the configuration of any top-level widget.

• DO NOT use UserWidget::RegisterInputComponent or UserWidget::UnregisterInputComponent. These will be ignored. Instead, set or unset the flag EnableEnhancedInput, which effectively does the same thing.

• DO NOT use SetZOrder. If you try, you will be overridden by our window management code. Currently, the only control we're giving over window z-order is 'BringToFront'. If you need more control, we'll have to enhance the window management system.

• DO NOT use SetMouseCaptureMode, SetMouseLockMode, SetHideCursorDuringCapture, CaptureMouse, ReleaseMouseCapture, LockMouseToWidget, ReleaseMouseLock. We simply don't support controlling mouse capture and mouse lock at this level of granularity. Trying to use these will fight our window management code. If you need this, we'll have to enhance the window management system.

• DO NOT use AddToViewport or AddToPlayerScreen. Top level widgets should be inserted into the root canvas using AddWidgetToRoot.

• DO NOT use SetIgnoreInput. You will be overridden. Our window management system relies on the enhanced input system being active, turning it off would cause everything to fail. However, a widget can handle keyboard or character events, causing them not to be propagated, it can also block events to any window lower in the z-order.

• DO NOT use SetInputModeXXX. Be aware that there is no "input mode" enum or "input mode" variable anywhere in Unreal. What these functions actually do is set a large number of state variables - keyboard focus, mouse capture, and so forth - from a single call. Naturally, then, these will fight our window management system.

Most local event-handling functions are allowed

There are many functions that gate or route events locally - ie, within a single UserWidget, or within a single Actor. Controlling and gating events within a single localized entity does not create window-management confusion. Because of that, all of these are still allowed:

• You CAN use EnableInput/DisableInput on actors, to turn enhanced input events on/off for that actor.

• You CAN use PushInputComponent/PopInputComponent on the player controller, if you want to register something that's NOT a widget to receive enhanced input events. Seems esoteric, but it still works.

• You CAN use methods of UUserWidget to bind or unbind input events.

More broadly, functions that an actor or widget uses to manipulate its own input component or input events are no problem.