### []() Displaying Widget Blueprints

Unreal provides a class, "Widget Blueprint," which is intended for GUI elements. The Unreal Editor includes an editor for widget blueprints, in which you can position and align buttons, check boxes, text boxes, and other widgets. We intend to use widget blueprints for our GUI elements.

A widget blueprint can represent a window full of buttons and checkboxes and the like, but it doesn't necessarily need to be that complicated. It could also be a menu, or even just a label.

Widget blueprints, like all Unreal blueprints, contain an event graph. Therefore, you can write Unreal blueprint code to get any behavior you want from them. They can update their contents periodically, and they can respond to mouse clicks or hotkeys.

This paper explains how we use widget blueprints as the foundation for Luprex's GUI system.

## How Blueprint Code calls Lua Functions

Before we explain how gui elements are made, I need to tell you about a general-purpose pathway that we have created that allows Unreal blueprints to make calls into Lua. There are two kinds of calls: "invokes", and "probes."

An "invoke" is when a blueprint calls into Lua for the purpose of changing the state of the world. An invocation needs to be transmitted to the server, so that it may affect the master copy of the world. Invocations are queued, serialized, and executed in a precise order according to the rules of "predictive reexecution" - we have a separate paper on this topic. Because of this, the "invoke" is not executed the instant that the blueprint calls an invoke-function. The invoke therefore doesn't return anything to the blueprint.

A "probe", on the other hand, does have a return value: that's the purpose of a probe, to return information to the blueprint. A probe is not allowed to alter the state of the world. Ideally, probes will be executed in a sandbox that prevents them from changing the state of the world. Probes are only executed by the world model on the client - they are not forwarded to the server, because the don't need to be.

It is possible for a malicious hacker to alter his blueprints, he can therefore cause any function to be invoked, with any arguments. Therefore, when an invocation comes into the server, the server must account for the possibility that the invocation is an attempted hack.

Because of this, we have added a security mechanism: a permit_invoke function that allows you to name a class and a function, to enable blueprints to invoke that particular function. Without the permit_invoke, the invocation will be rejected. However, that still leaves the possibility of the hacker deliberately passing invalid arguments. Because of this, any function that is invocable must error-check its arguments extensively.

Probes do not have the same security issues. The worst a probe can do is crash the hacker's client. So we have no security mechanism for probes.

When you do a probe or an invoke, the blueprint can pass parameters to the Lua code. Actor and place are always passed automatically. After that, the blueprint can pass four types of parameters: floating point, strings, vectors, and booleans. The blueprint may pass as many of these as desired.

When the blueprint calls a probe-function, the probe must return a value. The return value is always an array containing floating point, strings, vectors, and booleans.

To call a Lua function, use the blueprint node 'Invoke Lua Function' or 'Probe Lua Function.'

## Look-At Widgets

By default, luprex puts a crosshair on the screen. When you aim the crosshair at an object, you are said to be "looking at" that object. Sometimes, looking at an object causes a "look-at widget" to pop up. Here's a typical example of a look-at widget, which displays the hotkeys for that object:

![](Images/Displaying%20Widget%20Blueprints_html_fc056206a4099c85.jpg)

Luprex contains all the following mechanisms to support the idea of look-at widgets:

- *Event Calculate Look At*. Once per frame, luprex will call event *Calculate Look At* to determine which object you are looking at. The event is written in blueprint.

- *Current Look-At Object*. The luprex gamemode contains a global variable that stores a pointer to the object you're currently looking at.

- *Event Look-At Changed*. When the value of the *Current Look-At Object* changes, event *Look-At Changed* gets triggered. This event is responsible for popping up the look-at widget, if any. The event is written in blueprint.

- *The Lua Function engio.getlookat*. When the crosshair moves to a new object, the blueprint code asks Lua what blueprint to pop up. It does so by probing the Lua function *engio.getlookat*.
- *Current Look-At Widget*. The luprex gamemode contains a global variable that stores a pointer to the current look-at widget. When the look-at changed event creates a look-at widget, it stores a pointer to the widget in this global variable.

The global variables are provided by our C++ code. The code that invokes the *Calculate Look-At* and *Look-At Changed* are also written in C++. Everything else is either blueprint or Lua. What that means is that there's very little about this system that can't be overridden or redefined by the scripter.

The following sections give more information about event *Calculate Look-At*, event *Look-At Changed*, and the Lua function *engio.getlookat*.

## Event Calculate Look-At

Part of the Luprex system is class lxGameMode. It is expected that all games that use Luprex will either use lxGameMode, or some other blueprint class derived from lxGameModeBase.

lxGameMode contains a custom event, *Calculate Look At*. This event gets called once per frame to figure out which object the player is looking at. Open the Unreal Editor now, open class lxGameMode, and take a look at the code for *Calculate Look-At*. The following is an explanation:

Event *Calculate Look-At* fetches the XY position of the crosshair. It does a *Line Trace Through Pixel* to cast a ray from the camera, through the crosshair, and out into the world. The line trace returns a *HitResult* struct. The *HitResult* contains the ID of the object that was hit by the ray. The *HitResult* also contains other potentially useful information, such as the world XYZ coordinate where the ray hit the object.

The final step in Event *Calculate Look-At* is to call *Set Look-At*, which stores the *HitResult* in a global variable. Once the *HitResult* has been stored, you can fetch it from anywhere using the function *Get Look-At*. You can also use the function *Get Look-At Actor* if you don't need the whole *HitResult*, and you just want to know which Actor was hit.

The *HitResult* might contain an Actor that is known to Lua - ie, a Luprex tangible. It might also hit an object which is just scene decoration, something that was put into the scene using the Unreal Editor without any involvement of Lua. It could also hit the ground plane. If the ray didn't hit anything at all, then the *HitResult* might be empty. All of these possibilities are valid. *Get Look-At* can return any of these.

## Overriding Event Calculate Look-At

The normal implementation of *Calculate Look-At* traces a ray through the crosshair. But what if you don't want a crosshair in your game? What if you want to write a point-and-click game that uses a visible mouse pointer instead? In that case, it would probably make sense to change *Calculate Look-At* so that instead of casting a ray through the crosshair, it would cast a ray through the mouse pointer instead.

Of course, you could just open up the Unreal Editor and edit the blueprint code for *Calculate Look-At*. However, I don't recommend editing code provided by the Luprex system. Instead, I would suggest creating a new GameMode class that derives from lxGameMode: perhaps you could call it PointAndClickGameMode. Then, you can override event *Calculate Look-At* using inheritance.

Actually writing the new version of Calculate Look-At would be simple: just copy the code for the existing routine, then replace Crosshair XY with Mouse XY.

Imagine a slightly different game design - a game design where to "look at" an object, you just bump into it while walking. In this game design, there's no crosshair or mouse pointer. In such a design, you might entirely disable event *Calculate Look-At* by overriding it with blank code. Instead, you might set up collision detection events, and in those collision detection events, call *Set Look-At*. This is entirely legitimate. You can call *Set Look-At* from anywhere.

The routine *Set Look-At* doesn't directly call event *Look-At Changed*. Instead, if it detects that the look-at object changed, it sets a flag. Later, the flag causes event *Look-At Changed* to be called.

## About The Line Tracing Code

The standard implementation of *Calculate Look-At* uses a routine *Line Trace Through Pixel* to cast a ray from the camera, through the crosshair, and out into the world. *Line Trace Through Pixel* is a routine provided by Luprex. It is a variation of *Line Trace by Channel*, which is a standard Unreal built-in. It would be entirely possible to accomplish the same thing using *Line Trace by Channel*, but it would require more blueprint code.

In the default implementation of *Calculate Look-At*, the line trace is configured to use *Trace Channel: LookAtDetection*. You can use the Unreal Editor to edit the blueprint for an actor, and specify whether that actor collides with *LookAtDetection* rays. You will find this checkbox in the blueprint by selecting the mesh component, and then looking in the *Details* panel for *Collision/Collision Presets*. If you don't touch this checkbox, then by default, all actors will collide with the *LookAtDetection* ray. If you mark this checkbox false, then the crosshair line trace will pass right through that actor.

The trace channel *LookAtDetection* is a trace channel that we added to the project in the *Collision* section of the *Project Settings*. Initially, we tried using trace channel *Visibility*, which is an Unreal built-in trace channel that (according to the documentation) collides with any visible object. However, some visible objects are omitted, your player character in particular. That is not what we want. We eventually decided to have a trace channel specifically for *LookAtDetection* in order to make it possible to configure it precisely without interfering with anything else.

## Event Look-At Changed and engio.getlookat

After *Calculate Look-At* figures out which object you're looking at, it calls *Set Look-At,* to store the object in a global variable. *Set Look-At* also checks whether the look-at object has changed, and if so, it sets a flag. The flag eventually causes event *Look-At Changed* to be called. Open the Unreal Editor now, open class lxGameMode, and take a look at the code for event *Look-At Changed*. The following is an explanation:

The first thing that *Look-At Changed* does is log an informational message. You can therefore see the look-at object changing in the Unreal debugging window.

Then, *Look-At Changed* calls *Clear Look-At Widget*, which deletes any previous look-at widget. If this were not here, then looking away from an object would not cause the look-at widget to go away.

Next, if the object you're looking at is a Lua tangible, *Look-At Changed* probes the Lua function *engio.getlookat*. The purpose of this call is to ask Lua what widget blueprint to display. The function *engio.getlookat* is expected to return a string (the blueprint name), and an array of additional values that will be passed through to the widget blueprint's constructor.

Once we have a blueprint name, event *Look-At Changed* calls *Create Look-At Widget by Name*. This function creates a new look-at widget. *Create Look-At Widget by Name* also adds the widget to the viewport, and it calls *Set Look-At Widget* to store the widget as the current global look-at widget.

The blueprint language doesn't really have "constructors." Instead, to give the widget a chance to initialize itself, event *Look-At Changed* calls the custom event *Read Lua Configuration*. The widget can then use data returned from Lua to set itself up.

It is theoretically possible to override event *Look-At Changed.* Suppose, for example, that you want to add a glowing highlight to whatever object the player is looking at. You could add code to *Look-At Changed* to modify the material of the look-at object.

## The Standard Hotkey Widget

The hotkey widget for an object stays on the screen until you turn the camera, moving the crosshairs off of that object. As soon as you point the crosshairs away, the hotkey widget is automatically destroyed.

The look-at widget is highly scriptable and configurable. Almost every part of its behavior can be altered by editing either lua code or blueprint code. In order to explain it in a comprehensible way, we're going to start by explaining the *default behavior*.

By default, the look-at widget is implemented by blueprint class *lxLookAtWidgetBP*, which is a built-in part of the luprex codebase. When *lxLookAtWidgetBP* pops up, it probes a builtin lua function *gui.gethotkeys.* This returns a vector, which might look like this:

```lua
{ "PX", "Close" }
```

PX stands for gamepad X. When the player presses the gamepad X button, the *lxLookAtWidgetBP* captures that hotkey. It invokes the builtin lua function *gui.presshotkey(place, "Close").*

The built-in system supports an enumeration of hotkeys, including these:

M1 - the left mouse button

M2 - the middle mouse button

M3 - the right mouse button

PX - the X button on the xbox gamepad, or the square button on the PS gamepad

PY - the Y button on the xbox gamepad, or the triangle button on the PS gamepad

PB - the B button on the xbox gamepad, or the circle button on the PS gamepad

PA - the A button on the xbox gamepad, or the cross button on the PS gamepad

There are many more. The full list will be included elsewhere. In addition to the hotkeys listed above, you can use single letters to represent keyboard keys.

To add a hotkey to a lua class, the scripter must write lua code similar to this:

```lua
makeclass("brickoven")

gui.addhotkeys(brickoven, "Close", "PX")

function brickoven.do_close(place)

… implement the hotkey here …

end

function brickoven.do_show_menu(place)
```

The builtin lua function *gui.addhotkeys(place…)* stores one or more hotkeys in the class table. If you were to pretty-print the class table for *brickoven*, you would see this:

```lua
{

__class = "brickoven",

__hotkeys = { PX = "Close" },

hotkey_close = <function>

}
```

If you're playing a game using keyboard and mouse, you won't be able to press gamepad keys. Likewise, if you're playing with gamepad, you won't be able to press mouse buttons. Therefore, the lua script should include hotkeys for both, like this:

```lua
gui.addhotkeys(brickoven, "Close", "PX", "M3")
```

That means that you can invoke the close function by pressing either gamepad X, or the right mouse button. Class *lxLookAtWidgetBP* will attempt to determine which input device you're using, and it will show the correct hotkeys for the input device you're currently using.

The implementation of *gui.gethotkeys(place)* is very straightforward: it gets the class from the tangible, then it gets the hotkey list from the class. The implementation of *gui.presshotkey(place, hotkey, action)* is equally straightforward: it figures out which function to call based on the action string. It does so by simplifying the action string: it lowercases it, it replaces whitespace with underscore, and it removes punctuation.

## Modifying the Look-At Widget Blueprint

If the scripter doesn't like the appearance of the provided *lxLookAtWidgetBP*, one option is to edit the blueprint. By editing the blueprint, it is possible to change the appearance in any way desired.

One possible modification is to actually make the widget entirely invisible. This might be a reasonable thing to do in a point-and-click game: when you aim the mouse pointer at an object, a look-at widget pops up, but it's invisible. But the widget is there, and it responds to the "hotkeys" left-mouse and right-mouse.

If you examine *lxLookAtWidgetBP*, you will find the calls to *gui.gethotkeys* and *gui.presshotkey.* In theory, it is possible to remove these calls and replace them with some completely different lua calls. You can completely change the way look-at widgets communicate with lua. I don't recommend that, but it is possible if for some reason the current design is not sufficient.

In general, it's not ideal for the scripter to actually edit our provided classes. Instead, it's better to write a class that derives from *lxLookAtWidgetBP*, and then edit the derived class.

## Using Multiple Look-At Widget Blueprints

What if you want to use *lxLookAtWidgetBP* for some of the objects in your game, but you have a few objects for which you want to use a different blueprint? In that case, you will need to tell luprex which blueprint to pop up for which object.

When unreal is about to pop up a look-at widget, it first probes the lua function *gui.getlookwidget(place).* The built-in version of this function always returns nil, which in turn causes unreal to use the default blueprint, *lxLookAtWidgetBP*. If you wish, you can edit the code for *gui.getlookwidget*. For example, you might put this in your lua script:

```lua
function gui.getlookwidget(place)

local class = getclass(place)

return { class.lookwidget }

end
```

Then, having done that, you could write code that looks like this:

```lua
makeclass("brickoven")

brickoven.lookwidget = "BrickOvenLookWidgetBP"
```

Any class that doesn't have a look widget specified will end up falling back to *lxLookAtWidgetBP*.

## Adding a Menu System

## More Complex Widgets

Currently, there is no way to pop up any widget other than a look-at widget. The following section is for future reference.

The configuration for a widget blueprint could be complicated. Consider this menu:

![](Images/Displaying%20Widget%20Blueprints_html_24d3706fb604c218.jpg)

This menu could be a single widget blueprint, *MenuWidgetBP*. In its *BeginPlay* event, it could probe *gui.getmenu(actor, place),* which might return { "Resume", nil, "Save Game", "Load Game", nil, "Session", "Difficulty", "Options", nil, "Main Menu", "Quit Game" }. This design would make it possible for one widget blueprint to represent all the menus in the game.

We will provide common blueprints like *HotkeyWidgetBP* and *MenuWidgetBP* in our distribution to help the scripter get started. However, these won't be a privileged part of the system: they're really just example blueprints. If the user doesn't like the appearance of the menus rendered by our *MenuWidgetBP*, then he can alter the blueprint's code, or write his own *MyMenuWidgetBP*. If he wants to have several different types of menus in his game, he can have multiple menu widget blueprints.

Here's an interesting window from a role-playing game:

![](Images/Displaying%20Widget%20Blueprints_html_ae2696951c4fbe2a.png)

Since there are hundreds of spells in this game, it would not be good to have a separate widget blueprint for each one. Instead, it's probably best to have a *SpellWidgetBP* that can render any spell. This might probe *spell.getinfo(actor, place, spell),* which might return a vector like this:

```
{

"Heading", "Ice Storm", 4, "Evocation",

"Damage Type", "2D8", "Bludgeoning",

"Bonus Damage", "4D6", "Cold",

"Description", "A hail of rock-hard ice pounds …",

"Saving Throw", "DEX", "Targets still take half damage",

"Range", 300, "feet",

}
```

However, instead of making a *SpellWidgetBP*, there's another option. The window above is pretty simple: just text and images. There's nothing interactive. There are no resizable elements. There's nothing that scrolls. One could consider making a widget blueprint that can render any simple window: text, images, and buttons only. Such a widget blueprint might be called *SimpleDialogWidgetBP.* This widget blueprint might probe a lua function, gui.getlayout(actor, place), which might return a vector like this:

```
{

"window", "WindowStyle3", 0, 0, 500, 500,

"text", "Ice Storm", "12pt", "white", 20, 20, 200, 40,

"text", "Level 4 Evocation Spell", "9pt", "grey", 20, 45, 200, 65,

…

}
```

There's no right answer as to whether *SimpleDialogWidgetBP* or *SpellWidgetBP* is the best choice for this window. *SpellWidgetBP* has the disadvantage that you have to create a separate widget blueprint just for spells. On the other hand, having a separate widget for spells might make some things easier. Typing coordinates into Lua is awkward, having a SpellWidgetBP would make it possible to use the Unreal Editor to lay out the window. It might also be possible to have the window resize to fit the length of the description. It's also possible to implement it both ways: start by using *SimpleDialogWidgetBP*, and then when you decide you want to add a little polish, switch it over.

It may even be desirable to make a full-blown *ScriptableWidgetBP*, that takes a little language that is even more sophisticated. If the scripter wants to do so, he can create his own little languages to control the windows in his game.

## Building a Menu System

Let's walk through the design of a menu system.

This menu system will only contain one blueprint: *MenuWidgetBP.* When this blueprint pops up, it *probes* a Lua function *getmenu* to get the menu configuration. Then later, when the user clicks a menu item, it *invokes* a Lua function *execmenu* to execute the menu item, passing the selected menu item as a string. Here's what the two functions would look like, with the example of a brick oven:

```lua
makeclass("brickoven")

brickoven.onclick = "MenuWidgetBP"

function brickoven.getmenu(actor, place)

return { "Add Clay", "Add Straw", "Add 1 Fuel", "Add 10 Fuel", "Ignite the Oven" }

end

function brickoven.execmenu(actor, place, selection)

if (toofaraway(actor,place)) then return end

if (selection == "Add Clay") then place.addclay(actor, place) end

if (selection == "Add Straw") then place.addstraw(actor, place) end

if (selection == "Add 1 Fuel") then place.addfuel(actor, place, 1) end

if (selection == "Add 10 Fuel") then place.addfuel(actor, place, 10) end

if (selection == "Ignite the Oven") then place.ignitetheoven(actor, place) end

end

permit_invoke("brickoven", "execmenu")
```

That's easy to understand, and it would absolutely be possible to implement *MenuWidgetBP* to work like that. But it's not very convenient for the Lua scripter. We can do significantly better. We're going to make a series of improvements to that. The first step is adding a level of indirection:

```lua
makeclass("gui")

function gui.getmenu(actor, place)

local class = getclass(place)

return class.getmenu(actor, place)

end

function gui.execmenu(actor, place, selection)

local class = getclass(place)

class.execmenu(actor, place, selection)

end
```

The idea here is that instead of *MenuWidgetBP* calling *brickoven.getmenu*, it calls *gui.getmenu*. Then, *gui.getmenu* fetches the class of the place (brickoven), and then it calls *brickoven.getmenu*. Effectively, *MenuWidgetBP* is still probing *brickoven.getmenu* and invoking *brickoven.execmenu*, but it's doing so through our two dispatchers. The functionality is exactly the same, so far.

The advantage of having the centralized dispatchers, *gui.getmenu* and *gui.execmenu*, is that we can now move certain things into the dispatcher. First, we're going to move the *permit_invoke* to the dispatcher, and also the security check *toofaraway(actor, place):*

```lua
makeclass("gui")

function gui.getmenu(actor, place)

local class = getclass(place)

return class.getmenu(actor, place)

end

function gui.execmenu(actor, place, selection)

if (toofaraway(actor, place)) then return end

local class = getclass(place)

class.execmenu(actor, place, selection)

end

permit_invoke("gui", "execmenu")
```

So now, instead of needing a *permit_invoke* in class *brickoven*, and another in class *furnace*, and another in class *kiln*, we only need one in class *gui*. Likewise, instead of needing the *toofaraway* check in every class, we only need one in class *gui*.

The next improvement we can make is to eliminate the need for separate *getmenu* and *execmenu* functions, by writing a single *makemenu* function that operates in two modes (*getmenu* mode or *execmenu* mode). This is the updated code: we've combined *getmenu* and *execmenu*, we've eliminated the *toofaraway* check (because it's in the dispatcher), and we've eliminated the *permit_invoke* (again, because it's in the dispatcher):

```lua
makeclass("brickoven")

brickoven.onclick = "MenuWidgetBP"

function brickoven.makemenu(actor, place, menu)

gui.menuitem(menu, "Add Clay", function () place.addclay(actor,place) end)

gui.menuitem(menu, "Add Straw", function () place.addstraw(actor, place) end)

gui.menuitem(menu, "Add 1 Fuel", function () place.addfuel(actor, place, 1) end)

gui.menuitem(menu, "Add 10 Fuel", function () place.addfuel(actor, place, 10) end)

gui.menuitem(menu, "Ignite the Oven", function () place.ignitetheoven(actor, place) end)

end
```

This function takes a parameter "menu". This is an object that indicates whether *makemenu* is running in *getmenu* mode or *execmenu* mode. In *getmenu* mode, *gui.menuitem* adds the menu item to a list of menu items. In *execmenu* mode, *gui.menuitem* conditionally calls the lambda. To make this work, we have to alter our centralized dispatchers. We will still have separate *getmenu* and *execmenu* dispatchers, both of which will call *makemenu*:

```lua
makeclass("gui")

function gui.getmenu(actor, place)

local menu = { "mode":"getmenu", "config": {} }

place.makemenu(actor, place, menu)

return menu.config

end

function gui.execmenu(actor, place, selection)

if toofaraway(actor, place) then return end

local menu = { "mode":"execmenu", "selection" : selection }

place.makemenu(actor, place, menu)

end

permit_invoke("gui", "execmenu")

function gui.menuitem(menu, item, lambda)

if menu.mode == "getmenu" then

table.insert(menu.config, item)

end

if menu.mode == "execmenu" then

if menu.selection == item then lambda() end

end

end
```

So far, we've simplified the following: we only need to write one *makemenu* function per class like *brickoven*, we don't need to write security checks in the *makemenu* function, and we don't ever need to write *permit_invoke* other than the one in class gui.

There is one last thing left to simplify: we can eliminate the need to write *brickoven.onclick = "MenuWidgetBP".* We will make "MenuWidgetBP" the default *onclick* whenever a *makemenu* function exists. We can accomplish that by overriding the built-in function *gui.get_onclick*:

```lua
function gui.get_onclick(actor, place)

local class = getclass(place)

if class.onclick == nil and class.makemenu ~= nil then

return { "MenuWidgetBP" }

else

return { class.onclick }

end

end
```

There is one more thing we could simplify. The *gui.menuitem* function requires us to pass in a lambda. We could imagine modifying *gui.menuitem* so that if you pass nil for the lambda, it will automatically search for a function whose name matches the menu-item. For example, it could take the menu item "Add Clay", strip out the white space and uppercase yielding "addclay", and then it could look for a function brickoven.addclay. This would allow us to optionally omit the lambda and just write a function whose name matches the menu item.

The most important thing about this menu system is: it's all implemented in "user space." It was created by writing non-privileged Lua code and non-privileged blueprint code – no C++. If we were to ship this menu system with our distribution, the scripter could modify it at will. Or, he could build his own menu system using similar or different principles, and his menu system could coexist peacefully with ours.
