integration/Docs/Luprex-Window-Management.md

Introduction

Unreal has several input mode-related subsystems that interact with each other in complicated ways. These subsystems include:

• keyboard focus
• mouse capture
• enhanced input routing
• pointer visibility
• window z-order

Unreal is littered with conditionals that cause these bits of state to affect each other in unpredictable, often illogical ways. If you set these bits of state in the wrong order, or to the wrong values, it is all too easy to get unreal into a non-functioning state. The system is much too fragile.

For this reason, I have implemented a window management system that orchestrates all of this from a centralized location, in a way that guarantees reasonably predictable, sane behavior.

Core Design Choices

Our window management system, in order to keep things simple, has to make some assumptions about how Luprex games work. So, here are the rules.

Top-level UserWidgets get inserted into a "Root Canvas", instead of into the viewport. The root canvas implements most of the functionality of our window management system.

The keyboard focus rule is simple: the UserWidget in front according to the z-order gets keyboard focus. The window management system will put focus on the front widget and will keep it there. The only way to give a UserWidget keyboard focus is to raise it to the front of the z-order.

Mouse movements events are handled in two different ways: the system can shift between "mouselook" mode and "point-and-click" mode. Every top-level UserWidget declares whether it wants a mouse pointer or not. If the front UserWidget wants a pointer, the system shifts into point-and-click mode.

In point-and-click mode, enhanced input mouse move events cannot happen. In mouselook mode, widget OnMouseDown and OnMouseMove events cannot happen. In both modes, you can track mouse movement, but you have to use different mechanisms.

Widgets that declare that they want a pointer are automatically put in front of widgets that don't want a pointer. Because of this rule, the system essentially separates into the "mouselook" layer underneath, and the "point-and-click" layer on top. When the point-and-click layer gets out of the way, then you can drive the 3D world.

State Variables of the Window Management System

I have made an effort to keep the number of state variables that you have to control to an absolute minimum, and to concentrate them all in one place. That place is the "Root Canvas Slot."

Typically, in Unreal, when you create a new top-level widget, you insert it into the viewport using AddToViewport. But to use our window management system, you must instead insert top-level widgets into a 'root canvas', using AddWidgetToRoot.

The root canvas object associates a RootCanvasSlot to each top-level widget. The RootCanvasSlot is a place where we can store window management-related hints for that widget. The contents of the RootCanvasSlot include the following:

• ShowPointer: If true, this is a point-and-click widget. When this widget is in front, the pointer is visible, and the system switches to point-and-click mode.

• BlockInput: If this window is in front, all enhanced input events in other objects are blocked.

• EnableEnhancedInput: If false, enhanced input events in this widget are disabled.

• BringToFrontCount: Effectively, a timestamp indicating the last time this window was brought to the front. This is the main factor determining the z-order of the widgets.

In addition, the top-level widget itself contains some window-management related properties. Currently, these are:

• DesiredFocusWidget: Indicates which sub-widget, if any, should be given focus. When the system grants focus to the frontmost UserWidget, the focus actually goes here.

There are deliberately no other variables that control our new window management system. If your blueprint is managing these properties, then it is doing everything it needs to do.

The function SetWidgetWindowManagement can set all of these variables in a single operation. That one function is all you need to control the entire window management system.

Handling Keyboard and Gamepad Buttons

Here is a summary of how keyboard/gamepad button handling in unreal works. We have tweaked this slightly, but this is mostly just ordinary unreal input handling:

When you press a keyboard or gamepad button, the button first goes to any widget that has keyboard focus. If that widget doesn't declare the button to be "handled", then button is offered to other widgets higher in the widget heirarchy. If no widget handles the button, the button then goes to the "enhanced input subsystem."

The enhanced input system puts the button through an "input mapping context." Basically, that's a many-to-one map that translates buttons into more abstract "enhanced input events." Here's a fragment of a typical input mapping context:

Key W --> IA_Move_Forward Key S --> IA_Move_Backward Left_Thumbstick_Forward --> IA_Move_Forward Left_Thumbstick_Backward --> IA_Move_Backward

What the mapping context buys you is that you can handle events like "IA_Move_Forward" without having to care whether the player is driving with the WASD keys or with the gamepad left thumbstick.

Typically, enhanced input events go to all of the following: the player controller, the character, and user-defined widgets. All of these consumers of enhanced input are automatically registered to receive enhanced input, which means that all they have to do is implement a handler in their event graph, and they're ready. Other actors can also receive enhanced input, but that requires jumping through some hoops.

It's interesting that a widget can implement a handler for a raw keyboard button, and then declare the button "not handled". If the button proceeds to the enhanced input system, and if the widget has a handler for enhanced input, the widget can receive the same button again, in a different form!

There is a priority order among consumers of enhanced input events: user widgets first (front-to-back), then the player controller, then the character. A consumer of enhanced input has the option of blocking input to lower-priority consumers.

This is all almost entirely unchanged from Unreal's default behavior. We've only made two tiny tweaks: we send enhanced input to widgets in front-to-back order, and, widgets disable enhanced input by setting a flag instead of by unregistering their input component. Other than that, this is all just stock unreal.

Handling mouse buttons

Mouse buttons behave differently than keyboard buttons.

Widgets have an OnMouseDown handler. This is only active in point-and-click mode. OnMouseDown only fires when three things are true: the system must be in point-and-click mode, the pointer must be inside the rectangle of a widget, and the widget must be marked hit-testable.

If no OnMouseDown event fires, or if OnMouseDown declares the mouse down to be "not handled," then the mouse down makes it to the enhanced input subsystem.

Once the mouse down reaches the enhanced input system, it starts being treated the same as keyboard and gamepad buttons. It can be mapped to an enhanced input event by the input mapping context, and then from there, it can be handled by any enhanced input event handler in a blueprint.

The upshot of all this is: if you want to think of a mouse button as "just another button," then the way to achieve that is to not write an OnMouseDown handler, but instead, to deal with it through enhanced input.

We have tweaked the default behavior of unreal. If the system is in point-and-click mode, and you click on a widget that is hit-testable, but which has no OnMouseDown handler, we provide a default OnMouseDown behavior: we bring the top-level UserWidget to the front. Because our system grants keyboard focus to the widget in front, this will also grant focus.

Handling Mouse Movement

In point-and-click mode, mouse movement moves the pointer and doesn't generate any events at all.

There is one exception: mouse capture. If you click on a hit-testable widget, that widget will "capture" the mouse until you release the mouse button. As long as the widget has capture, it receives OnMouseMove events. This is mainly intended to implement click-and-drag, scroll bar scrolling, and other movements like that.

Unreal has a lot of complicated mouse capture and mouse lock options and modes. We don't support any of that. We support only the basics: automatic capture when you click. If you need more, we'll have to improve the Luprex window management system.

In point-and-click mode, mouse movements do not go to the enhanced input system at all.

When the system is in mouselook mode, mouse movements go directly to the enhanced input system. They get mapped by the input mapping context and turned into enhanced input events. Handling these events is how mouselook works.

Handling Analog Joysticks

Analog joysticks (including gamepad thumbsticks) generate events that go directly to the enhanced input subsystem. They get mapped to enhanced input events. From there, they can be handled by any consumer of enhanced input.

Functions you Should NOT CALL!

If you're using our Luprex window management system, there are several things your blueprint should NOT do:

• DO NOT use SetKeyboardFocus, SetUserFocus, or any other function with Set-Focus in the name. Instead, just be aware that the frontmost UserWidget will get focus. It can delegate that focus to one of its components by setting DesiredFocusWidget.

• DO NOT use SetShowMouseCursor, or set the bShowMouseCursor flag. Instead, set the ShowPointer flag in the RootCanvasSlot of any top-level widget.

• DO NOT use UserWidget::RegisterInputComponent or UserWidget::UnregisterInputComponent. These will be ignored. Instead, set or unset the flag EnableEnhancedInput in the RootCanvasSlot, which effectively does the same thing.

• DO NOT use SetZOrder. If you try, you will be overridden by our window management code. Currently, the only control we're giving over window z-order is 'BringToFront'. If you need more control, we'll have to enhance the window management system.

• DO NOT use SetMouseCaptureMode, SetMouseLockMode, SetHideCursorDuringCapture, CaptureMouse, ReleaseMouseCapture, LockMouseToWidget, ReleaseMouseLock. We simply don't support controlling mouse capture and mouse lock at this level of granularity. Trying to use these will fight our window management code. If you need this, we'll have to enhance the window management system.

• DO NOT use AddToViewport or AddToPlayerScreen. Top level UserWidgets should be inserted into the root canvas using AddWidgetToRoot.

• DO NOT use SetIgnoreInput. You will be overridden. Our window management system relies on the enhanced input system being active, turning it off would cause everything to fail. However, a widget can handle keyboard or character events, causing them not to be propagated, it can also block events to any widget lower in the z-order, and to the player controller and character.

• DO NOT use SetInputModeXXX. Be aware that there is no "input mode" enum or "input mode" variable anywhere in Unreal. What these functions actually do is set a large number of state variables - keyboard focus, mouse capture, and so forth - from a single call. Naturally, then, these will fight our window management system.

Most local event-handling functions are allowed

There are many functions that gate or route events locally - ie, within a single UserWidget, or within a single Actor. Controlling and gating events within a single localized entity does not create window-management confusion. Because of that, all of these are still allowed:

• You CAN use EnableInput/DisableInput on actors, to turn enhanced input events on/off for that actor.

• You CAN use PushInputComponent/PopInputComponent on the player controller, if you want to register something that's NOT a widget to receive enhanced input events. Seems esoteric, but it still works.

• You CAN use methods of UUserWidget to bind or unbind input events.

More broadly, functions that an actor or widget uses to manipulate its own input component or input events are no problem.