186 lines
6.4 KiB
Markdown
186 lines
6.4 KiB
Markdown
# UE Wingman
|
|
|
|
UE Wingman is a tool that allows an AI to control the unreal
|
|
editor. When you're using it, it feels like the AI is right
|
|
there looking at the editor with you. You'll be able to
|
|
watch as it creates graph nodes and wires them together,
|
|
you'll see it add components to your blueprints, you'll see
|
|
it design widget hierarchies for you, and you'll see it write
|
|
shaders for your materials.
|
|
|
|
The tool is not complete, not by a long shot. There are
|
|
tons of Unreal Editor functions that the AI just can't
|
|
access yet. Even so, I think it's useful: it has pretty
|
|
comprehensive support to allow the AI agent to help create
|
|
blueprints, widget blueprints, and materials.
|
|
|
|
## How Does it Work?
|
|
|
|
This tool adds a command interpreter plugin to the Unreal
|
|
Editor. You can type commands, and the plugin in the editor
|
|
will execute them. You can actually type commands directly
|
|
from the command-line. Here's an example of what you might
|
|
see:
|
|
|
|
```
|
|
$ ue-wingman.py Graph_Dump Graph=/Game/Testing/BP_Test,graph:EventGraph
|
|
|
|
node K2Node_Event_0: Event BeginPlay
|
|
output-pins OutputDelegate
|
|
|
|
node K2Node_Event_2: Event Tick
|
|
output-pins OutputDelegate, DeltaSeconds
|
|
```
|
|
|
|
There are tons of commands built in: Graph_Dump,
|
|
GraphNode_Create, GraphPin_Connect,
|
|
BlueprintComponent_Create, Widget_Create, and so forth.
|
|
Using these commands, it's possible to examine and modify
|
|
blueprints, widgets, and materials.
|
|
|
|
But, of course, these commands aren't really intended for humans.
|
|
They're intended for an AI agent. The AI is given access to these
|
|
commands using what's called "Model Context Protocol," which is
|
|
a goofy name for "a mechanism that an AI can use to send
|
|
commands to other software."
|
|
|
|
## Why Choose this Particular Unreal Engine MCP?
|
|
|
|
There are a *lot* of Unreal Engine MCPs out there. Some of
|
|
them are, shall we say, not carefully engineered. I'm a
|
|
reasonably skilled software engineer and I've designed
|
|
this plugin to be robust and capable of sustained development.
|
|
|
|
This MCP is also designed to be as broadly general as
|
|
possible. I've seen MCPs that claim "can create 22 different
|
|
kinds of graph nodes!" This makes me ask: why not just
|
|
provide the *entire catalog* of all possible graph nodes?
|
|
I've seen MCPs claim "you can edit 15 different material
|
|
expression properties!" Why not provide access to *all*
|
|
editable material expression properties? I've tried to make
|
|
every tool in this MCP as capable as possible, with as few
|
|
limits as possible.
|
|
|
|
Some of the MCPs out there expose the entire Unreal API to
|
|
the AI agent. That is not entirely safe. AI agents will
|
|
very often carelessly call API functions they shouldn't,
|
|
crashing your editor or corrupting your assets. All the
|
|
commands supported by this MCP are error-checked. It is
|
|
intended that they won't let the AI do something that would
|
|
crash your editor.
|
|
|
|
This MCP is very extensible. Adding a new command requires
|
|
a relatively small amount of code. I'm hoping some others
|
|
in the community will eventually start contributing new
|
|
commands.
|
|
|
|
## Installation
|
|
|
|
There are three parts to UE Wingman:
|
|
|
|
* The Unreal Plugin, which does 99% of the work.
|
|
|
|
* The python program "ue-wingman.py" which a human can
|
|
use to send commands to the plugin.
|
|
|
|
* The python program "ue-wingman-mcp.py", which an AI
|
|
can use to send commands to the plugin.
|
|
|
|
If you build Unreal from source, the best way to install the
|
|
plugin is to drop the entire UEWingman source folder into
|
|
your Plugins folder. Then do a build. Restart the editor, and
|
|
go into your plugins configuration. Enable the UE Wingman
|
|
plugin. You're done.
|
|
|
|
If you don't build from source, then unfortunately, you're
|
|
out of luck. Precompiled plugins must be built for every
|
|
different OS, for every different engine version. I just
|
|
don't have the means to do that right now.
|
|
|
|
After installing the plugin, you need to install the two
|
|
python programs. They are both short and simple: all they
|
|
do is establish a network connection to the plugin, and then
|
|
send the command you typed. They require python 3.6 or later,
|
|
and no other dependencies.
|
|
|
|
To install the human version, ue-wingman.py, just drop it into
|
|
a folder on your PATH.
|
|
|
|
To install the AI version, ue-wingman-mcp.py, you have to
|
|
usually set up some config file for your AI agent. I use
|
|
Claude Code, for that, you have to create a file ".mcp.json"
|
|
in your project folder, and it needs to have this inside it:
|
|
|
|
```
|
|
{
|
|
"mcpServers": {
|
|
"ue-wingman": {
|
|
"command": "python3",
|
|
"args": ["Plugins/UEWingman/ue-wingman-mcp.py"]
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
You can usually ask your AI agent for help creating this
|
|
config file.
|
|
|
|
## The "User Manual"
|
|
|
|
You might be interested in seeing the "user manual" for the
|
|
plugin. To get that, you type this:
|
|
|
|
```
|
|
$ ue-wingman.py UserManual
|
|
```
|
|
|
|
Of course, you're not the intended user: your AI agent is.
|
|
When the AI agent starts up ue-wingman-mcp, it is
|
|
automatically told to read the user manual. From there, the
|
|
User Manual says, among other things, that the AI agent can
|
|
get a listing of built-in commands. You can see that too:
|
|
|
|
```
|
|
$ ue-wingman.py ShowCommands
|
|
```
|
|
|
|
With these two commands at your disposal, you'll have a better
|
|
understanding of what exactly your AI agent is doing with this
|
|
plugin, and how it all works.
|
|
|
|
## Fun things to Try
|
|
|
|
I really started enjoying this plugin when I asked my agent
|
|
to make me a "cool looking material, something psychedelic
|
|
and weird." It made a neat kaleidoscope thing. I then
|
|
asked it to make me an animated rendering of the mandelbrot
|
|
set. It's fun to watch it do things like that.
|
|
|
|
## History and Credits
|
|
|
|
When I myself needed an MCP for unreal development, I did a
|
|
survey of the plugins out there. I ended up choosing a
|
|
plugin called "Blueprint MCP" by David Gundry:
|
|
|
|
<https://github.com/mirno-ehf/ue5-mcp>
|
|
|
|
It was not bad, but it had some limitations, and I started
|
|
doing work to improve it. Incrementally, I ended rewriting
|
|
pretty much the whole thing. So this whole project is actually
|
|
a fork of Blueprint MCP. There's very little of the original
|
|
code remaining. However, you will find snippets here and there.
|
|
|
|
Even though I ended up rewriting most of the code, it really
|
|
was useful to have a functioning starting point. It meant I
|
|
could improve one thing at a time, without having to try to
|
|
get everything working all at once. So I'm quite grateful
|
|
to David Gundry and his work.
|
|
|
|
## Software License
|
|
|
|
UE Wingman is licensed under the MIT license, a copy of which
|
|
is enclosed. Its predecessor (of which it is a fork) was also
|
|
under the MIT license, so everything works out.
|
|
|
|
|