Compare commits

...

3 Commits

Author SHA1 Message Date
960abba07f Intellisense clangd fix 2026-04-22 23:32:50 -04:00
a689d59ea0 More work on focus, and good docs 2026-04-22 22:52:04 -04:00
a964211cc8 Fix some issues with new root canvas stuff 2026-04-22 17:02:24 -04:00
14 changed files with 522 additions and 457 deletions

Binary file not shown.

View File

@@ -1,172 +0,0 @@
# Better Debugging With LLDB (in VS Code + CodeLLDB)
## The Problem
When debugging Unreal with VS Code + CodeLLDB, the **Variables** pane and
the **Watch** pane use two completely different evaluation paths:
- **Variables pane** walks a tree built by lldb's *synthetic children
providers* (Unreal's Python formatters for TArray, TMap, FName, FString,
and now our TObjectPtr/TSharedPtr/TWeakPtr). Values are looked up by
offset/type — no compilation. Base classes appear as named children
("SCompoundWidget", "UWidget"); smart-pointer inners get unwrapped;
container elements get indexed.
- **Watch pane** (without intervention) runs text through the `/se` (simple)
or `/nat` (Clang) evaluator. `/nat` fails for most Unreal paths;
`/se` transpiles to Python and walks the SBValue tree, but — as
originally shipped — fails on pointer auto-deref and can't find base
classes by type name.
When you right-click in Variables and **Add to Watch**, CodeLLDB sends
a path like `Top.Widget` (built itself for synthetic children, uniform
`.`, no pointer awareness). Without the fix below, that path fails in
Watch even though it identifies a real value Variables can show.
## What We Did
All changes live in `tools/UEDataFormatter.py`, loaded via `initCommands`
in every launch config in `Integration.code-workspace.tpl.json`.
### 1. Patched `codelldb.value.Value.__getattr__`
At module load, we monkey-patch CodeLLDB's `Value` class — the Python
wrapper it uses inside `/se` and `/py` evaluation — to:
- **Auto-deref pointers** before descending into a named child.
- **Fall back to iterating children** by `GetName()` when
`GetChildMemberWithName` returns invalid. This catches base classes
(which appear as named children when iterated but can't be looked up
by name).
**Result:** plain "Add to Watch" from the Variables pane produces a path
like `Top.Widget.SomeField` and Watch evaluates it correctly — same
expandable tree you'd see in Variables. No `/py fv(...)` wrapper, no
VS Code extension, no manual prefix typing.
Blast radius: every `/se` and `/py` expression that goes through
`Value.__getattr__`. The change is strictly more permissive (paths that
used to fail now succeed), but it is a global behavior change to
CodeLLDB internals. Breaks if CodeLLDB renames `Value` or the
`__sbvalue` slot; re-applied automatically on extension reload since
our patch runs on `command script import`.
### 2. Synth + Summary Providers for Smart Pointers
UE's engine formatter covers containers and TWeakObjectPtr, but not the
smart pointer family. We added providers for:
- **`TObjectPtr<T>`** — shows `nullptr` / `unresolved` / wrapped object's
summary. Expanding flattens straight to the target's members (no
intermediate `*DebugPtr` click).
- **`TSharedPtr<T>`** and **`TSharedRef<T>`** — shows `nullptr` / target
summary; expands straight to target's members.
- **`TWeakPtr<T>`** — checks
`WeakReferenceCount.ReferenceController.SharedReferenceCount` before
dereferencing. Expired weak refs show `expired` rather than garbage
from a dangling pointer.
All registration regexes are anchored with `^` — otherwise the greedy
`.+` matches nested occurrences (a `TArray<TObjectPtr<X>, ...>` would
get dispatched to the TObjectPtr provider instead of TArray).
### 3. Dynamic Type Resolution
Every launch config now sets:
settings set target.prefer-dynamic-value no-run-target
lldb reads the vtable at each polymorphic value and shows the runtime
type's members. A `UObject*` that actually points to a `UUserWidget`
expands to the full `UUserWidget` subtree, not just `UObject`.
`no-run-target` avoids running code in the debuggee, which is important
during synthesis.
### 4. Universal SIGTRAP Handling
`process handle SIGTRAP --notify false --pass false --stop false` is now
in every launch config. Unreal raises SIGTRAP internally in a number of
places (soft asserts, ensure-style checks); without this, the debugger
stops constantly.
## Reloading Without a Session Restart
If you edit `tools/UEDataFormatter.py`, reload in the Debug Console:
script import importlib; importlib.reload(UEDataFormatter); UEDataFormatter.__lldb_init_module(lldb.debugger, {})
The reload re-executes module code (updating the patch and the provider
classes). The explicit `__lldb_init_module` call re-runs the provider
registrations — lldb only fires `__lldb_init_module` on initial import,
not on reload.
## Summary of Workflow
| Action | Where | How |
|---|---|---|
| Explore a value | Variables pane | Click disclosure triangles |
| Track a value across steps | Watch pane | Right-click variable → Add to Watch (just works) |
| One-shot inspection | Debug Console | `v Widget.Object->SCompoundWidget.SWidget` (`v` = `frame variable`; use `->` for pointers explicitly) |
| Reload formatter edits | Debug Console | `script import importlib; importlib.reload(UEDataFormatter); UEDataFormatter.__lldb_init_module(lldb.debugger, {})` |
## Notes on the Design
### Why this works
CodeLLDB's `/se` evaluator transpiles user expressions into Python that
operates on `Value` objects. `Value.__getattr__` drives every `.field`
access. By making that method auto-deref and iterate for base classes,
every downstream mechanism (Watch, Debug Console, hover, conditional
breakpoints) inherits the fix.
### Why `fv` is no longer needed
Earlier we had an `fv(path)` helper plus a plan for a companion VS Code
extension to wrap "Add to Watch" results in `/py fv(...)`. The
`Value.__getattr__` patch makes the default path work, so that whole
layer is obsolete.
### Why not patch `SBValue` instead
Tempting, but much larger blast radius — affects every tool, every
adapter, every Python script using lldb. Patching `Value` confines the
change to CodeLLDB's expression pipeline.
## Ideas for Further Improvement
### Propose `/fv` mode to CodeLLDB upstream
Prefix dispatch is in CodeLLDB's Rust binary, so we can't add a new
prefix from Python. A clean feature request would be to add `/fv`
`SBFrame::GetValueForVariablePath(code)` as a native evaluator. That
would give direct access to lldb's own frame-variable walker without
the Python/Value indirection — though it has its own limitations
(no arithmetic, no casts).
### More synth providers
- **`TOptional<T>`** — hide the storage bytes and `bIsSet`; expose the
contained value (or `unset`) directly.
- **`TVariant<...>`** — expose the currently-held alternative as the
single child.
- **`FText`** — show the resolved localized string as the summary.
- **`FSoftObjectPtr` / `FSoftClassPtr`** — show the asset path.
### Enrich TObjectPtr summary
When resolved, show both class and name (`UUserWidget 'W_HUD_0'`)
instead of just the name. The class is reachable via
`ClassPrivate->NamePrivate`.
### A `fdump` helper
A Debug Console helper that prints an entire subtree as indented text —
useful for grabbing a snapshot of complex state into a log or comment.
### Get `Copy as Expression` to emit `->` for pointers
The path CodeLLDB builds for synthetic children uses `.` uniformly,
regardless of whether intermediate values are pointers. That's why
`v Top.Widget` fails but `v Top->Widget` works. A feature request to
have CodeLLDB emit `->` when traversing a pointer would make paths
`frame variable`-compatible out of the box.

View File

@@ -1,129 +0,0 @@
# Keyboard Focus and Input Modes
Luprex provides a "window management system" that takes full ownership of: keyboard focus, mouse capture, pointer visibility, enhanced input routing, and input mode. Top-level widgets don't call `AddToViewport` or configure input modes directly — instead, they are added to a single root canvas, and the player controller arbitrates the per-tick state based on which widget is in front.
## Key types
- **`AlxPlayerControllerBase`** — owns the top-level `RootWidget`, builds the input stack, and reconciles focus / capture / pointer state every tick.
- **`UlxRootCanvasPanel`** — the one `UCanvasPanel` subclass that lives inside `RootWidget`. Every top-level UI widget in the game is a child of this canvas.
- **`UlxRootCanvasSlot`** — the slot type used for children of `UlxRootCanvasPanel`. Extends `UCanvasPanelSlot` with input-mode fields. A widget's slot carries **both** layout config (anchors, offsets, ZOrder) **and** its window-management declarations.
These types collapse what would otherwise be two parallel systems (layout + input-mode requests) into one. Adding a widget to the canvas = declaring its window-management intent.
## The core rule: ZOrder is authoritative
**The widget with the highest ZOrder wins everything.** It dictates:
- Whether the mouse pointer is shown.
- Whether the viewport captures the mouse.
- Which widget (or sub-widget) receives keyboard focus.
- Which input components end up on top of the input stack.
Draw order and input priority are the same fact. A widget you can see on top is the one that gets input.
## Slot fields (`UlxRootCanvasSlot`)
Inherited from `UCanvasPanelSlot`:
- **`ZOrder`** (int32) — Higher = drawn on top / higher input priority. Can be updated at runtime via `Slot->SetZOrder(N)`.
- **`Anchors`, `Offsets`, `Alignment`** — standard UMG canvas layout.
Added by `UlxRootCanvasSlot`:
- **`ShowPointer`** (bool) — When this widget is in front, the mouse cursor is visible and the viewport releases capture. When false, the cursor is hidden and the viewport captures the mouse (game-style input).
- **`BlockInput`** (bool) — When this widget's input component is pushed onto the stack, it blocks lower-priority components from receiving events.
- **`EnableEnhancedInput`** (bool, default true) — Whether this widget's `UInputComponent` is included on the input stack at all. Corresponds to "Enhanced Input events fire on this widget."
One property that lives on the **widget** rather than the slot:
- **`UUserWidget::DesiredFocusWidget`** — the sub-widget to receive keyboard focus when this widget is in front. Null means "focus the viewport."
## API
### Adding a widget
```cpp
// Static, BP-callable with DefaultToSelf + HideSelfPin,
// so in Blueprint it looks like a method on UUserWidget.
AlxPlayerControllerBase::AddWidgetToRoot(UUserWidget *Widget);
```
Adds the widget as a child of the player controller's `RootCanvas` at ZOrder 0 with default flags. If the widget is already a child, does nothing.
### Updating a widget's settings
```cpp
UlxRootCanvasPanel::SetWidgetWindowManagement(
UUserWidget *Widget,
bool ShowPointer, bool BlockInput, bool EnableEnhancedInput,
bool BringToFront, UWidget *DesiredFocusWidget);
```
Updates an already-added widget's slot. Writes all the input-mode flags, sets `DesiredFocusWidget`, and optionally bumps ZOrder to `GetMaxZOrder() + 1` if `BringToFront` is true. Logs an error if the widget isn't yet a root widget.
### Forcing a focus re-apply
```cpp
AlxPlayerControllerBase::RestoreFocusToFrontWidget(const UObject *Context);
```
Clears `LastWidgetGrantedFocus`, which causes the next tick of `UpdateInputMode` to re-apply focus based on the current front widget. Use after mutating `DesiredFocusWidget` or when a modal is dismissed and focus should return to the underlying widget.
## Per-tick flow
On every tick, the player controller runs two passes:
### `BuildInputStack`
Assembles the list of `UInputComponent`s that will receive input events this frame:
1. Controlled pawn's input component(s).
2. Level script actors.
3. The player controller's own input component.
4. `CurrentInputStack` (anything pushed via `PushInputComponent`, filtered to exclude input components already managed by a root canvas slot).
5. Root canvas widget input components, pushed in ZOrder-ascending order (so highest ZOrder ends up on top of the stack, which gets processed first).
Each widget-managed IC has its `bBlockInput` flag set from its slot's `BlockInput` before being pushed.
### `UpdateInputMode`
Reads the front widget's slot and applies the declared state:
1. Sorts the canvas slots by ZOrder; picks the front widget.
2. Calls `SetShowMouseCursor(Slot->ShowPointer)`.
3. Configures viewport capture mode based on `ShowPointer`:
- **Pointer on**: releases capture (only if held), unlocks mouse, sets `CaptureDuringMouseDown`.
- **Pointer off**: high-precision mouse movement, locks mouse to viewport, `CapturePermanently`.
4. Resolves focus:
- If `!ShowPointer`, re-applies focus every tick (keeps stealing focus back from anything that steals it).
- If `ShowPointer` and the front widget (identity) hasn't changed since the last grant, lets click-to-focus take over.
- Otherwise grants focus once to the widget's `DesiredFocusWidget` (or the viewport if none).
## Focus arbitration
`LastWidgetGrantedFocus` (a `TObjectKey<UWidget>`) tracks the last focus target the system applied. The comparison is identity-based: if the current front widget's `DesiredFocusWidget` differs from `LastWidgetGrantedFocus`, focus is re-granted.
To force a re-grant without a widget change (e.g., after mutating `DesiredFocusWidget`), call `RestoreFocusToFrontWidget`. This sets the key to null, so the next tick's comparison will always re-grant.
## Click-to-focus
When the user clicks on a focusable widget, the `LuprexViewportClient` notifies the player controller via `ClickToFocus(Widget)`. On the next `UpdateInputMode` tick, if the system isn't going to re-grant focus for other reasons, it forwards focus to the clicked widget.
This only takes effect in pointer mode. In non-pointer mode, focus is continuously re-applied to the front widget's `DesiredFocusWidget`, overriding click behavior.
## What NOT to do
- **Don't call `AddToViewport` on top-level game widgets.** All top-level widgets must be children of `RootCanvas`. Going directly to the viewport bypasses the entire system.
- **Don't set input mode via `SetInputModeGameOnly` / `SetInputModeUIOnly` / `SetInputModeGameAndUI`.** The window management system owns these. Adjust `ShowPointer` / `BlockInput` on the slot instead.
- **Don't call `SetShowMouseCursor` directly.** Same reason — driven by the front widget's `ShowPointer`.
- **Don't set focus manually.** Set `DesiredFocusWidget` on the widget; the system will pick it up.
## Design notes
**Why slot fields, not separate requests?** Earlier versions of this system kept a parallel `FlxInputModeRequests` array of per-widget state. Tying that to slot lifecycle (widget added = entry exists; widget removed = entry gone) eliminated a whole class of stale-entry and lifecycle bugs.
**Why ZOrder as the priority key?** The alternative is a separate priority field, which can diverge from visual order. Using ZOrder means "the widget you see on top is the one with input priority" is true by construction, not by convention.
**Why identity-based focus invalidation?** Tracking a sequence number worked but required callers to call an explicit re-request API whenever `DesiredFocusWidget` changed. Using `TObjectKey<UWidget>` lets the system notice any change in the intended focus target automatically.
**Why `ShowPointer` and not a whole `EInputMode` enum?** Two flags cover the behaviors that actually matter (`ShowPointer`, `BlockInput`). A three-state enum with `GameOnly` / `UIOnly` / `GameAndUI` would force callers into categories that don't map cleanly to how this system layers widgets.

View File

@@ -0,0 +1,323 @@
# Introduction
Unreal has several input mode-related subsystems that
interact with each other in complicated ways. These
subsystems include:
- keyboard focus
- mouse capture
- enhanced input routing
- pointer visibility
- window z-order
Unreal is littered with conditionals that cause these bits
of state to affect each other in unpredictable, often
illogical ways. If you set these bits of state in the wrong
order, or to the wrong values, it is all too easy to get
unreal into a non-functioning state. The system is *much*
too fragile.
For this reason, I have implemented a window management
system that orchestrates all of this from a centralized
location, in a way that guarantees reasonably predictable,
sane behavior.
# Core Design Choices
Our window management system, in order to keep things
simple, has to make some assumptions about how Luprex games
work. So, here are the rules.
The presumption is that most of the time, you're interacting
with the 3D world, and importantly, we assume that if you're
using keyboard and mouse, you're using the mouse to control
the camera - aka "mouselook."
We also assume that as you interact with the 3D world, you
will occasionally be popping up GUI widgets that can
coexist with mouselook. These mouselook-compatible
widgets don't need a mouse pointer, they don't need you to
click on anything. They rely on buttons alone. We assume
that most of the GUI elements you interact with will be
mouselook-compatible, in order to allow you to stay
immersed in the 3D world.
But we also assume that there may be moments when you want
to pop up a very complicated widget, for example, a big
inventory management screen, for which a mouse pointer would
be very helpful. For occasions like these, a widget can
declare "ShowPointer".
When one of these ShowPointer widgets is on the screen,
the entire system switches into point-and-click mode.
In point-and-click mode, the pointer is visible. Mouse
movements move the pointer. Mouse movements do *not* get
translated into mouselook.
Widgets have a z-order: one widget is always "in front." In
mouselook mode, only the front widget can get keyboard
focus. In mouselook mode, the window management system will
put focus on the front widget, and it will keep it there.
If you want some other widget to have focus, you'll have to
bring that widget to the front.
In point-and-click mode, the keyboard focus rules differ.
When you raise a widget to the front, the window management
system will give it focus. But, if you click the mouse
pointer on a different widget - say, on a text box in a
different widget - keyboard focus can get transferred.
When a ShowPointer window is on the screen, not only
does the system shift to point-and-click mode, but it
also keeps all ShowPointer windows in front of any
non-ShowPointer window.
Basically, you can think of the system as a 3D world with
its mouselook-compatible widgets as one layer, and the
point-and-click stuff as a second layer on top of that.
When the point-and-click layer gets out of the way, then you
can drive the 3D world.
# The Root Canvas
Typically, in Unreal, when you create a new top-level
widget, you insert it into the viewport using
AddToViewport. But to use our window management
system, you must instead insert top-level widgets into a
'root canvas', using AddWidgetToRoot.
The main reason for the creation of the root canvas is that
it gives us a place to store window-management related
hints, and window-management related state.
The root canvas object attaches a RootCanvasSlot to each
top-level widget. The RootCanvasSlot is a place where we can
store management-related hints for that widget. The contents
of the RootCanvasSlot include the following:
- `ShowPointer`: If true, this is a point-and-click widget.
When this widget is in front, the pointer is visible,
and the system switches to point-and-click mode.
- `BlockInput`: If this window is in front, all enhanced
input events in *other* objects are blocked.
- `EnableEnhancedInput`: If false, enhanced input events in
*this* widget are disabled.
- `BringToFrontCount`: Effectively, a timestamp indicating
the last time this window was brought to the front.
In addition, the top-level widget itself contains some
window-management related properties. Currently, these are:
- `DesiredFocusWidget`: Indicates which sub-widget, if any,
should be given focus.
That is all the state variables that control our new window
management system. If your blueprint is managing these
properties, then it is doing everything it needs to do.
The function SetWidgetWindowManagement can set all of these
properties in a single operation. That one function is all
you need to control the entire window management system.
# Handling Keyboard and Gamepad Buttons
Here is a summary of how keyboard/gamepad button handling in
unreal works. We have tweaked this slightly, but this is
mostly just ordinary unreal input handling:
When you press a keyboard or gamepad button, the button
first goes to any widget that has keyboard focus. If that
widget doesn't declare the button to be "handled", then
button is offered to other widgets higher in the widget
heirarchy. If no widget handles the button, the button then
goes to the "enhanced input subsystem."
The enhanced input system puts the button through
an "input mapping context." Basically, that's a many-to-one
map that translates buttons into more abstract "enhanced
input events." Here's a fragment of a typical input
mapping context:
Key W --> IA_Move_Forward
Key S --> IA_Move_Backward
Left_Thumbstick_Forward --> IA_Move_Forward
Left_Thumbstick_Backward --> IA_Move_Backward
What the mapping context buys you is that you can handle
events like "IA_Move_Forward" without having to care
whether the player is driving with the WASD keys or with
the gamepad left thumbstick.
Typically, enhanced input events go to *all* of the
following: the player controller, the character, and
user-defined widgets. All of these consumers of enhanced
input are automatically registered to receive enhanced
input, which means that all they have to do is implement a
handler in their event graph, and they're ready. Other
actors can *also* receive enhanced input, but that requires
jumping through some hoops.
It's interesting that a widget can implement a handler for a
raw keyboard button, and then declare the button "not
handled". If the button proceeds to the enhanced input
system, and if the widget has a handler for enhanced input,
the widget can receive the same button again, in a
different form!
There is a priority order among consumers of enhanced input
events: user widgets first (front-to-back), then the player
controller, then the character. A consumer of enhanced
input has the option of blocking input to lower-priority
consumers.
This is all almost entirely unchanged from Unreal's default
behavior. We've only made two tiny tweaks: we send enhanced
input to widgets in front-to-back order, and, widgets
disable enhanced input by setting a flag instead of by
unregistering their input component.
# Handling mouse buttons
Mouse buttons behave differently than keyboard buttons.
Widgets have an OnMouseDown handler. This is only active in
point-and-click mode. OnMouseDown only fires when three
things are true: the pointer is visible, the pointer is
inside the rectangle of a widget, and the widget is marked
hit-testable.
If no OnMouseDown event fires, or if OnMouseDown declares
the mouse down to be "not handled," then the mouse down
makes it to the enhanced input subsystem.
Once the mouse down reaches the enhanced input system, it
starts being treated the same as keyboard and gamepad
buttons. It can be mapped to an enhanced input event by the
input mapping context, and then from there, it can be
handled by any enhanced input event handler in a blueprint.
The upshot of all this is: if you want to think of a
mouse button as "just another button," then the
way to achieve that is to handle the mouse button using
an enhanced input handler.
We have very slightly tweaked the default behavior of
unreal. If the pointer is visible, and you click on a
widget that is hit-testable, but which has no OnMouseDown
handler, we provide a default OnMouseDown behavior: we
bring the widget to the front. Because our system
grants keyboard focus to the widget in front, this
will grant focus, if the widget can accept it.
# Handling Mouse Movement
In point-and-click mode, mouse movement moves the pointer
and doesn't generate any events at all.
There is one exception: mouse capture. If you click on a
hit-testable widget, that widget will "capture" the mouse
until you release the mouse button. As long as the widget
has capture, it receives OnMouseMove events. This is
mainly intended to implement click-and-drag, scroll
bar scrolling, and other movements like that.
Unreal has a *lot* of complicated mouse capture and mouse
lock options and modes. We don't support any of that. We
support only the basics: automatic capture when you click.
If you need more, we'll have to improve the Luprex window
management system.
In point-and-click mode, mouse movements do not go to the
enhanced input system at all.
When the system is in mouselook mode, mouse movements go
directly to the enhanced input system. They get mapped by
the input mapping context and turned into enhanced input
events. Handling these events is how mouselook works.
# Handling Analog Joysticks
Analog joysticks (including gamepad thumbsticks) generate
events that go directly to the enhanced input subsystem.
They get mapped to enhanced input events. From there,
they can be handled by any consumer of enhanced input.
# Functions you Should NOT CALL!
If you're using our Luprex window management system, there are
several things your blueprint should *NOT* do:
- DO NOT use SetKeyboardFocus, SetUserFocus, or any other
function with Set-Focus in the name. Instead, set
the DesiredFocusWidget inside a top-level widget, and our
window management system will decide who gets focus.
- DO NOT use SetShowMouseCursor, or set the bShowMouseCursor
flag. Instead, set the ShowPointer flag in the configuration
of any top-level widget.
- DO NOT use UserWidget::RegisterInputComponent or
UserWidget::UnregisterInputComponent. These will be ignored.
Instead, set or unset the flag EnableEnhancedInput, which
effectively does the same thing.
- DO NOT use SetZOrder. If you try, you will be overridden
by our window management code. Currently, the only control
we're giving over window z-order is 'BringToFront'. If
you need more control, we'll have to enhance the window
management system.
- DO NOT use SetMouseCaptureMode, SetMouseLockMode,
SetHideCursorDuringCapture, CaptureMouse, ReleaseMouseCapture,
LockMouseToWidget, ReleaseMouseLock. We simply don't support
controlling mouse capture and mouse lock at this level of
granularity. Trying to use these will fight our window
management code. If you need this, we'll have to enhance the
window management system.
- DO NOT use AddToViewport or AddToPlayerScreen. Top level
widgets should be inserted into the root canvas using
AddWidgetToRoot.
- DO NOT use SetIgnoreInput. You will be overridden. Our
window management system relies on the enhanced input
system being active, turning it off would cause everything
to fail. However, a widget can handle keyboard or
character events, causing them not to be propagated, it
can also block events to any window lower in the z-order.
- DO NOT use SetInputModeXXX. Be aware that there is no
"input mode" enum or "input mode" variable anywhere in
Unreal. What these functions actually do is set a large
number of state variables - keyboard focus, mouse capture,
and so forth - from a single call. Naturally, then, these
will fight our window management system.
# Most *local* event-handling functions are allowed
There are many functions that gate or route events locally -
ie, within a single UserWidget, or within a single Actor.
Controlling and gating events within a single localized
entity does not create window-management confusion. Because
of that, all of these are still allowed:
- You CAN use EnableInput/DisableInput on actors, to turn
enhanced input events on/off for that actor.
- You CAN use PushInputComponent/PopInputComponent on the
player controller, if you want to register something that's
NOT a widget to receive enhanced input events. Seems
esoteric, but it still works.
- You CAN use methods of UUserWidget to bind or unbind
input events.
More broadly, functions that an actor or widget uses to
manipulate its *own* input component or input events
are no problem.

27
Docs/TASKS/Dictation.txt Normal file
View File

@@ -0,0 +1,27 @@
I need you to act as a secretary taking dictation. You will
be helping me to edit a markdown file.
I have a voice-to-speech program which is running in the background.
It records, and it sends my words to you. Most of what I say to you
will be meant as text to be put into the markdown file. But
occasionally, I will give you verbal instructions, for example:
"Reformat that bullet list into a numbered list"
It is your job to figure out intelligently which of the things I
say to you is meant as a directive, and which is meant as words to
go into the markdown file.
Do not edit the markdown file after every sentence. Instead,
quietly listen until you have a significant edit to make. I'd say,
roughly, when you have a paragraph, then it's time to make an edit.
I may occasionally commandeer the keyboard and edit the markdown
file myself. In those cases, you should notice that the file changed,
and read my changes.
It is also your job to make small corrections without comment.
If you see a really big mistake, stop and ask me what to do.

View File

@@ -7,10 +7,11 @@
#include "LuprexViewportClient.h"
#include "Common.h"
#include "PlayerControllerBase.h"
#include "RootCanvas.h"
#include "Engine/GameInstance.h"
#include "Framework/Application/SlateApplication.h"
#include "Layout/WidgetPath.h"
#include "Widgets/SViewport.h"
#include "Slate/SObjectWidget.h"
UlxViewportClient::UlxViewportClient(const FObjectInitializer &ObjectInitializer)
: Super(ObjectInitializer)
@@ -18,44 +19,44 @@ UlxViewportClient::UlxViewportClient(const FObjectInitializer &ObjectInitializer
UE_LOG(LogLuprexIntegration, Display, TEXT("UlxViewportClient constructed"));
}
bool UlxViewportClient::TryBringToFront(const FWidgetPath &Path)
{
UGameInstance *GI = GetGameInstance();
if (!GI) return false;
AlxPlayerControllerBase *PC = Cast<AlxPlayerControllerBase>(
GI->GetFirstLocalPlayerController(GetWorld()));
if (!PC) return false;
for (int32 Idx = 0; Idx < Path.Widgets.Num(); ++Idx)
{
SWidget &SW = Path.Widgets[Idx].Widget.Get();
if (SW.GetType() != FName(TEXT("SObjectWidget"))) continue;
UUserWidget *Widget = static_cast<SObjectWidget&>(SW).GetWidgetObject();
if (Widget && Widget->GetParent() == PC->RootCanvas)
{
UlxRootCanvasPanel::BringToFront(Widget);
return true;
}
}
return false;
}
bool UlxViewportClient::InputKey(const FInputKeyEventArgs &EventArgs)
{
UE_LOG(LogLuprexIntegration, Display, TEXT("UlxViewportClient::InputKey key=%s event=%d"),
*EventArgs.Key.ToString(), (int32)EventArgs.Event);
// Only act on left mouse button presses that bubbled up to the
// viewport unhandled. Walk the widget path under the cursor and
// find the nearest focusable ancestor of whatever was hit. If it
// isn't the game viewport itself, hand it to the player controller
// to apply on its next UpdateInputMode pass; that's the point in
// the frame where we can override SViewport's own click-focus
// behaviour without fighting it.
// viewport unhandled. If the click landed on a descendant of a
// top-level widget in the root canvas, bring that top-level widget
// to the front.
if ((EventArgs.Event == IE_Pressed) && (EventArgs.Key == EKeys::LeftMouseButton))
{
FSlateApplication &Slate = FSlateApplication::Get();
FVector2D MousePos = Slate.GetCursorPos();
FWidgetPath Path = Slate.LocateWindowUnderMouse(
MousePos, Slate.GetInteractiveTopLevelWindows());
if (Path.IsValid())
{
TSharedPtr<SViewport> ViewportWidget = GetGameViewportWidget();
for (int32 Idx = Path.Widgets.Num() - 1; Idx >= 0; --Idx)
{
TSharedRef<SWidget> Widget = Path.Widgets[Idx].Widget;
if (!Widget->SupportsKeyboardFocus()) continue;
if (ViewportWidget.IsValid() && Widget == ViewportWidget) break;
if (UGameInstance *GI = GetGameInstance())
{
if (AlxPlayerControllerBase *PC = Cast<AlxPlayerControllerBase>(
GI->GetFirstLocalPlayerController(GetWorld())))
{
PC->ClickToFocus(Widget);
}
}
break;
}
}
if (Path.IsValid() && TryBringToFront(Path)) return true;
}
return Super::InputKey(EventArgs);

View File

@@ -14,6 +14,7 @@
#include "CoreMinimal.h"
#include "Engine/GameViewportClient.h"
#include "Layout/WidgetPath.h"
#include "LuprexViewportClient.generated.h"
UCLASS()
@@ -25,4 +26,7 @@ public:
UlxViewportClient(const FObjectInitializer &ObjectInitializer);
virtual bool InputKey(const FInputKeyEventArgs &EventArgs) override;
private:
bool TryBringToFront(const FWidgetPath &Path);
};

View File

@@ -135,7 +135,7 @@ void AlxPlayerControllerBase::RestoreFocusToFrontWidget(const UObject *Context)
{
// This will trigger UpdateInputMode to shift focus back to
// the front window, if the front window wants focus.
FromContext(Context)->LastWidgetGrantedFocus = nullptr;
FromContext(Context)->RootCanvas->LastWidgetGrantedFocus = nullptr;
}
void AlxPlayerControllerBase::AddWidgetToRoot(UUserWidget *Widget)
@@ -163,8 +163,7 @@ void AlxPlayerControllerBase::AddWidgetToRoot(UUserWidget *Widget)
if (Widget->GetParent() == PC->RootCanvas) return;
UlxRootCanvasSlot *Slot = PC->RootCanvas->AddChildToRootCanvas(Widget);
Slot->SlotUnderConstruction = false;
PC->RootCanvas->AddChildToRootCanvas(Widget);
}
void AlxPlayerControllerBase::BuildInputStack(TArray<UInputComponent*>& InputStack)
@@ -239,7 +238,7 @@ void AlxPlayerControllerBase::BuildInputStack(TArray<UInputComponent*>& InputSta
}
// Now add the widget-managed input components.
for (UlxRootCanvasSlot *Slot : ReverseIterate(WidgetSlots))
for (UlxRootCanvasSlot *Slot : WidgetSlots)
{
if (Slot->EnableEnhancedInput)
{
@@ -269,15 +268,14 @@ void AlxPlayerControllerBase::UpdateInputMode()
if (!SlateUser.IsValid()) return;
// Get the desired configuration from the first widget.
// TODO: Maybe we don't have to sort the whole array.
TArray<UlxRootCanvasSlot*> WidgetSlots = RootCanvas->GetSortedUserWidgets();
RootCanvas->UpdateZOrders();
// Get the desired configuration from the top widget.
UUserWidget *Widget = nullptr;
UWidget *Focus = nullptr;
bool ShowPointer = false;
if (!WidgetSlots.IsEmpty())
if (UlxRootCanvasSlot *Top = RootCanvas->GetTopWidget())
{
UlxRootCanvasSlot *Top = WidgetSlots[0];
Widget = Cast<UUserWidget>(Top->GetContent());
Focus = Widget->GetDesiredFocusWidget();
ShowPointer = Top->ShowPointer;
@@ -321,32 +319,22 @@ void AlxPlayerControllerBase::UpdateInputMode()
// viewport client notifies us of that fact. We then focus the
// widget if possible.
//
if ((!ShowPointer) || (LastWidgetGrantedFocus != Focus))
if ((!ShowPointer) || (RootCanvas->LastWidgetGrantedFocus != Focus))
{
if (Focus)
{
if (TSharedPtr<SWidget> SlateFocus = Focus->GetCachedWidget())
{
SlateOperations.SetUserFocus(SlateFocus.ToSharedRef());
LastWidgetGrantedFocus = Focus;
RootCanvas->LastWidgetGrantedFocus = Focus;
}
}
else
{
SlateOperations.SetUserFocus(ViewportWidgetRef);
LastWidgetGrantedFocus = nullptr;
RootCanvas->LastWidgetGrantedFocus = nullptr;
}
}
else if (TSharedPtr<SWidget> ClickedWidget = ClickToFocusTarget.Pin())
{
SlateOperations.SetUserFocus(ClickedWidget.ToSharedRef());
}
ClickToFocusTarget.Reset();
}
void AlxPlayerControllerBase::ClickToFocus(TSharedRef<SWidget> Widget)
{
ClickToFocusTarget = Widget.ToWeakPtr();
}
void AlxPlayerControllerBase::UpdateLookAt()

View File

@@ -80,9 +80,6 @@ public:
UPROPERTY()
FHitResult CurrentLookAt;
// The last widget whose focus request was granted.
TObjectKey<UWidget> LastWidgetGrantedFocus;
// The single top-level UUserWidget added to the viewport. All
// top-level UI widgets are children of RootCanvas inside it.
UPROPERTY()
@@ -94,14 +91,5 @@ public:
UPROPERTY()
UlxRootCanvasPanel *RootCanvas = nullptr;
// The viewport client uses this to notify us that the user
// clicked on a focusable widget.
void ClickToFocus(TSharedRef<SWidget> Widget);
private:
TWeakPtr<SWidget> ClickToFocusTarget;
public:
bool MustCallLookAtChanged = false;
};

View File

@@ -18,33 +18,6 @@ UlxRootCanvasSlot::UlxRootCanvasSlot(const FObjectInitializer& ObjectInitializer
SetOffsets(FMargin(0.0f, 0.0f, 0.0f, 0.0f));
}
void UlxRootCanvasSlot::SetZOrderReliable(int32 Order)
{
if (SlotUnderConstruction)
{
PRAGMA_DISABLE_DEPRECATION_WARNINGS
ZOrder = Order;
PRAGMA_ENABLE_DEPRECATION_WARNINGS
}
else
{
SetZOrder(Order);
}
}
int32 UlxRootCanvasSlot::GetZOrderReliable()
{
if (SlotUnderConstruction)
{
PRAGMA_DISABLE_DEPRECATION_WARNINGS
return ZOrder;
PRAGMA_ENABLE_DEPRECATION_WARNINGS
}
else
{
return GetZOrder();
}
}
UClass* UlxRootCanvasPanel::GetSlotClass() const
{
@@ -70,16 +43,24 @@ UlxRootCanvasSlot* UlxRootCanvasPanel::AddChildToRootCanvas(UWidget* Content)
return Cast<UlxRootCanvasSlot>(Super::AddChild(Content));
}
int32 UlxRootCanvasPanel::GetMaxZOrder() const
void UlxRootCanvasPanel::OnSlotAdded(UPanelSlot* InSlot)
{
int32 MaxZOrder = 0;
UlxRootCanvasSlot *Slot = CastChecked<UlxRootCanvasSlot>(InSlot);
Slot->BringToFrontCount = ++BringToFrontCounter;
Super::OnSlotAdded(InSlot);
}
UlxRootCanvasSlot* UlxRootCanvasPanel::GetTopWidget()
{
UlxRootCanvasSlot* Top = nullptr;
for (UPanelSlot* PanelSlot : Slots)
{
UlxRootCanvasSlot *TypedSlot = Cast<UlxRootCanvasSlot>(PanelSlot);
check(TypedSlot);
MaxZOrder = FMath::Max(MaxZOrder, TypedSlot->GetZOrderReliable());
UlxRootCanvasSlot* Slot = CastChecked<UlxRootCanvasSlot>(PanelSlot);
if (Cast<UUserWidget>(Slot->Content) == nullptr) continue;
if ((Top == nullptr) || (*Top < *Slot)) Top = Slot;
}
return MaxZOrder;
return Top;
}
TArray<UlxRootCanvasSlot*> UlxRootCanvasPanel::GetSortedUserWidgets()
@@ -95,12 +76,35 @@ TArray<UlxRootCanvasSlot*> UlxRootCanvasPanel::GetSortedUserWidgets()
}
Result.StableSort([](const UlxRootCanvasSlot &A, const UlxRootCanvasSlot &B)
{
return A.GetZOrder() > B.GetZOrder();
return A < B;
});
return Result;
}
void UlxRootCanvasPanel::UpdateZOrders()
{
for (UPanelSlot *PanelSlot : Slots)
{
UlxRootCanvasSlot *Slot = Cast<UlxRootCanvasSlot>(PanelSlot);
check(Slot);
int32 ZOrder = (Slot->ShowPointer ? 1000000 : 0) + Slot->BringToFrontCount;
Slot->SetZOrder(ZOrder);
}
}
void UlxRootCanvasPanel::BringToFront(UUserWidget *Widget)
{
if (!IsValid(Widget)) return;
UlxRootCanvasSlot *Slot = Cast<UlxRootCanvasSlot>(Widget->Slot);
if (!Slot) return;
UlxRootCanvasPanel *Panel = Cast<UlxRootCanvasPanel>(Slot->Parent);
if (!Panel) return;
Slot->BringToFrontCount = ++Panel->BringToFrontCounter;
// This refocuses the widget, even if it was already in front.
Panel->LastWidgetGrantedFocus = Panel;
}
void UlxRootCanvasPanel::SetWidgetWindowManagement(class UUserWidget *Widget,
bool ShowPointer, bool BlockInput, bool EnableEnhancedInput, bool BringToFront, UWidget *DesiredFocusWidget)
{
@@ -120,7 +124,15 @@ void UlxRootCanvasPanel::SetWidgetWindowManagement(class UUserWidget *Widget,
Slot->ShowPointer = ShowPointer;
Slot->BlockInput = BlockInput;
Slot->EnableEnhancedInput = EnableEnhancedInput;
Widget->SetDesiredFocusWidget(DesiredFocusWidget);
if (BringToFront) Slot->SetZOrderReliable(Panel->GetMaxZOrder() + 1);
if (DesiredFocusWidget)
{
if (!Widget->SetDesiredFocusWidget(DesiredFocusWidget))
UE_LOG(LogLuprexIntegration, Error, TEXT("SetWidgetWindowManagement: focus widget must be a child of widget"));
}
else
{
Widget->SetDesiredFocusWidget(NAME_None);
}
if (BringToFront) Slot->BringToFrontCount = ++Panel->BringToFrontCounter;
}

View File

@@ -2,13 +2,16 @@
//
// RootCanvas.h
//
// UlxRootCanvasPanel is a UCanvasPanel subclass whose
// slots (UlxRootCanvasSlot) carry input-mode configuration
// in addition to layout. The PlayerController scans these
// slots, sorted by ZOrder, to arbitrate pointer visibility,
// capture, focus, and input-component blocking for
// top-level widgets. ZOrder therefore serves double duty:
// it determines draw order AND input priority.
// Luprex provides a "window management system" for root
// widgets. In this system, all top-level widgets have to go
// into the root canvas (instead of the viewport). The
// window management system monitors the widgets within the
// root canvas and continuously updates the z-orders, the
// pointer visibility, the mouse capture mode, the keyboard
// focus, and enhanced input event routing based on hints
// and directives given by the widgets.
//
// To learn more, read Docs/Luprex-Window-Management.md
//
////////////////////////////////////////////////////////////
@@ -16,6 +19,7 @@
#include "CoreMinimal.h"
#include "Common.h"
#include "UObject/ObjectKey.h"
#include "Blueprint/UserWidget.h"
#include "Components/CanvasPanel.h"
#include "Components/CanvasPanelSlot.h"
@@ -29,11 +33,6 @@ class UWidget;
//
// UlxRootCanvasSlot
//
// Luprex provides a "window management system" for root widgets.
// This system is documented in Docs/Keyboard-Focus-and-Input-Modes.md
// The Root Canvas Slot is how widgets ask the window management system
// to engage certain behaviors.
//
////////////////////////////////////////////////////////////
UCLASS()
class INTEGRATION_API UlxRootCanvasSlot : public UCanvasPanelSlot
@@ -60,32 +59,24 @@ public:
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Luprex|Input Mode")
bool EnableEnhancedInput = true;
// Knowing whether the slot is under construction helps us to work
// around some bugs in Unreal. See below.
bool SlotUnderConstruction = true;
// Sequence number assigned when this slot was last brought to front.
int32 BringToFrontCount = 0;
// Reliable version of SetZOrder. There is a bug in the normal version of
// SetZOrder: it crashes if you use it during OnConstruct. We have a
// workaround, but it requires using a different getter and setter.
UFUNCTION(BlueprintCallable, Category = "Luprex|Input Mode")
void SetZOrderReliable(int32 Order);
// Reliable version of GetZOrder. There is a bug in the normal version of
// SetZOrder: it crashes if you use it during OnConstruct. We have a
// workaround, but it requires using a different getter and setter.
UFUNCTION(BlueprintCallable, Category = "Luprex|Input Mode")
int32 GetZOrderReliable();
// Widget Z Ordering: widgets that show the pointer go on top of windows
// that don't, and within a group, widgets that have greater BringToFrontCount
// go on top. Sorting an array with this puts the top windows at the end
// of the array.
bool operator<(const UlxRootCanvasSlot &Other) const
{
if (ShowPointer != Other.ShowPointer) return !ShowPointer;
return BringToFrontCount < Other.BringToFrontCount;
}
};
////////////////////////////////////////////////////////////
//
// UlxRootCanvasPanel
//
// A UCanvasPanel that uses UlxRootCanvasSlot for its
// children instead of the plain UCanvasPanelSlot. Layout
// behavior is identical to UCanvasPanel; only the slot
// type differs.
//
////////////////////////////////////////////////////////////
UCLASS()
class INTEGRATION_API UlxRootCanvasPanel : public UCanvasPanel
@@ -94,18 +85,18 @@ class INTEGRATION_API UlxRootCanvasPanel : public UCanvasPanel
public:
// Convenience wrapper around AddChild that returns the
// derived slot type, so callers don't have to cast.
// Add a child to the canvas. Also brings the child to the front
// for the first time, ensuring that every widget has a unique
// BringToFront counter.
UFUNCTION()
UlxRootCanvasSlot* AddChildToRootCanvas(UWidget* Content);
// Find children of type UserWidget. Return them in a sorted
// order, with the highest Zorder first.
// order, with the bottom window first and the top window last.
TArray<UlxRootCanvasSlot*> GetSortedUserWidgets();
// Return the largest ZOrder across all slots, or 0 if empty.
// Used as the basis for placing new widgets on top.
int32 GetMaxZOrder() const;
// Return the highest-priority UserWidget slot, or nullptr if there are none.
UlxRootCanvasSlot* GetTopWidget();
// This function updates several window-management-related properties
// which are stored in the UserWidget and the lxRootCanvasSlot. Note that
@@ -119,6 +110,17 @@ public:
bool ShowPointer, bool BlockInput, bool EnableEnhancedInput,
bool BringToFront, UWidget *DesiredFocusWidget);
// Bring the widget to the front of the Z order within its group.
// This is implemented using the BringToFront counter.
UFUNCTION(BlueprintCallable, Category = "Luprex|Window Management",
meta = (DefaultToSelf = "Widget"))
static void BringToFront(class UUserWidget *Widget);
// Recompute and apply ZOrder values for all slots based on ShowPointer and
// BringToFrontCount. Must be called from a context where SetZOrder is safe -
// i.e. not during OnConstruct.
void UpdateZOrders();
// Fetch the UlxRootCanvasSlot for a widget that is parented to a
// UlxRootCanvasPanel. Returns nullptr via the WrongType exec pin
// if the widget isn't a root widget (no slot, or slot is not a
@@ -127,10 +129,22 @@ public:
meta = (DefaultToSelf = "Widget", ExpandEnumAsExecs = "Result"))
static UlxRootCanvasSlot *GetRootCanvasSlot(class UUserWidget *Widget, ElxSuccessOrWrongType &Result);
// The last widget whose focus request was granted.
TObjectKey<UWidget> LastWidgetGrantedFocus;
protected:
// UPanelWidget
// We inherit most of our code from CanvasPanel. This causes the
// CanvasPanel code to allocate slots of type UlxRootCanvasSlot.
virtual UClass* GetSlotClass() const override;
// We override OnSlotAdded in order to be able to be able to
// fully finish initializing the slot before the widget
// OnConstruct method has a chance to execute.
virtual void OnSlotAdded(UPanelSlot* InSlot) override;
// Monotonic counter incremented each time any slot is brought to front.
int32 BringToFrontCounter = 0;
};

View File

@@ -309,7 +309,7 @@ def build_compile_commands_from_luprex():
parts = line.split()
if (parts[0] == "g++") and ("-c" in parts):
source = os.path.abspath(os.path.join(INTEGRATION, parts[-1]))
entries.append({ "file": source, "command": line, "directory": INTEGRATION })
entries.append({ "file": source, "command": line.replace("g++ ", "g++ -D__INTELLISENSE__ ", 1), "directory": INTEGRATION })
return entries
@@ -349,7 +349,7 @@ def build_compile_commands_from_integration():
ccdir = f"{UNREALENGINE}/Engine/Source"
for cpp in sorted(cpp_to_rsp.keys()):
rsp = cpp_to_rsp[cpp]
args = [clang, "@"+rsp]
args = [clang, "-D__INTELLISENSE__", "@"+rsp]
entries.append({ "file": cpp, "arguments": args, "directory": ccdir })
return entries

View File

@@ -16,26 +16,32 @@ _FUObjectItemType = None
_GNameBlocksDebug = None
_GObjectArrayForDebugVisualizers = None
def _init_globals(target):
def _init_globals(frame, bp_loc, dict):
global _FNameEntryType, _FNameEntryStride, _FUObjectItemType
global _GNameBlocksDebug, _GObjectArrayForDebugVisualizers
_FNameEntryType = target.FindFirstType('FNameEntry')
if not _FNameEntryType.IsValid():
print('UEDataFormatter: FNameEntry type not found')
_FNameEntryStride = _FNameEntryType.GetByteAlign()
target = frame.GetThread().GetProcess().GetTarget()
_FUObjectItemType = target.FindFirstType('FUObjectItem')
if not _FUObjectItemType.IsValid():
print('UEDataFormatter: FUObjectItem type not found')
if _FNameEntryType is None:
t = target.FindFirstType('FNameEntry')
if t.IsValid():
_FNameEntryType = t
_FNameEntryStride = t.GetByteAlign()
_GNameBlocksDebug = target.FindFirstGlobalVariable('GNameBlocksDebug')
if not _GNameBlocksDebug.IsValid():
print('UEDataFormatter: GNameBlocksDebug not found')
if _FUObjectItemType is None:
t = target.FindFirstType('FUObjectItem')
if t.IsValid():
_FUObjectItemType = t
_GObjectArrayForDebugVisualizers = target.FindFirstGlobalVariable('GObjectArrayForDebugVisualizers')
if not _GObjectArrayForDebugVisualizers.IsValid():
print('UEDataFormatter: GObjectArrayForDebugVisualizers not found')
if _GNameBlocksDebug is None:
v = target.FindFirstGlobalVariable('GNameBlocksDebug')
if v.IsValid():
_GNameBlocksDebug = v
if _GObjectArrayForDebugVisualizers is None:
v = target.FindFirstGlobalVariable('GObjectArrayForDebugVisualizers')
if v.IsValid():
_GObjectArrayForDebugVisualizers = v
############################################################
@@ -677,8 +683,11 @@ def _register_provider(cat, pattern, summary_fn=None, synth_cls=None):
def __lldb_init_module(debugger, dict):
print("Running lldb_init_module")
_init_globals(debugger.GetSelectedTarget())
debugger.HandleCommand('target stop-hook add --python-function ' + __name__ + '._init_globals')
debugger.HandleCommand('type category delete ' + __name__)
frame = debugger.GetSelectedTarget().GetProcess().GetSelectedThread().GetSelectedFrame()
if frame.IsValid():
_init_globals(frame, None, {})
cat = debugger.CreateCategory(__name__)
_register_provider(cat, '^FString$', summary_fn='UEFStringSummaryProvider')