Files

UE Wingman

UE Wingman is a tool that allows an AI to control the unreal editor. When you're using it, it feels like the AI is right there looking at the editor with you. You'll be able to watch as it creates graph nodes and wires them together, you'll see it add components to your blueprints, you'll see it design widget hierarchies for you, and you'll see it write shaders for your materials.

The tool is not complete, not by a long shot. There are tons of Unreal Editor functions that the AI just can't access yet. Even so, I think it's useful: it has pretty comprehensive support to allow the AI agent to help create blueprints, widget blueprints, and materials.

How Does it Work?

This tool adds a command line interpreter plugin to the Unreal Editor. You can type commands, and the plugin in the editor will execute them.

 $ ue-wingman Graph_Dump /Game/Testing/BP_Test,graph:EventGraph
 
 node K2Node_Event_0: Event BeginPlay
     output-pins OutputDelegate
 
 node K2Node_Event_2: Event Tick
    output-pins OutputDelegate, DeltaSeconds

The ue-wingman command has tons of subcommands: Graph_Dump, GraphNode_Add, GraphPin_Connect, BlueprintComponent_Add, Widget_Add, and so forth. Using these commands, it's possible to examine and modify blueprints, widgets, and materials.

But, of course, these commands aren't really intended for humans. They're intended for an AI agent.

Why Choose this Particular Unreal AI Plugin?

There are a lot of Unreal Engine AI plugins out there. Some of them are, shall we say, not carefully engineered. I'm a reasonably skilled software engineer and I've designed this plugin to be robust and capable of sustained development.

This plugin is also designed to be as broadly general as possible. I've seen plugins that claim "can create 22 different kinds of graph nodes!" This makes me ask: why not just provide the entire catalog of all possible graph nodes? I've seen plugins claim "you can edit 15 different material expression properties!" Why not provide access to all editable material expression properties? I've tried to make every tool in this plugin as capable as possible, with as few limits as possible.

Some of the MCPs out there expose the entire Unreal API to the AI agent. That is not entirely safe. AI agents will very often carelessly call API functions they shouldn't, crashing your editor or corrupting your assets. All the commands supported by this MCP are error-checked. It is intended that they won't let the AI do something that would crash your editor.

This MCP is very extensible. Adding a new command requires a relatively small amount of code. I'm hoping some others in the community will eventually start contributing new commands.

Installation

There are two parts to UE Wingman:

  • The Unreal Plugin, which does 99% of the work.

  • The python program "ue-wingman.py"

The python program is actually less than 100 lines of code: all it does is package up its command line arguments, send them to the plugin, and let the plugin do the work. Then it prints the output.

If you build Unreal from source, the best way to install the plugin is to drop the entire UEWingman source folder into your Plugins folder. Then do a build. Restart the editor, and go into your plugins configuration. Enable the UE Wingman plugin. You're done.

If you don't build from source, then unfortunately, you're out of luck. Precompiled plugins must be built for every different OS, for every different engine version. I just don't have the means to do that right now.

After installing the plugin, you need to install the two python programs. They are both short and simple: all they do is establish a network connection to the plugin, and then send the command you typed. They require python 3.6 or later, and no other dependencies.

To install the human version, ue-wingman.py, just drop it into a folder on your PATH.

The "User Manual"

You might be interested in seeing the "user manual" for the plugin. To get that, you type this:

$ ue-wingman.py Documentation_Manual

Of course, you're not the intended user: your AI agent is. You should put a note into your agent's system prompt to let it know about the ue-wingman.py command, and to let it know that it can type ue-wingman.py Documentation_Manual. This in turn will tell your agent about this command:

$ ue-wingman.py Documentation_Commands

Using these commands, you can learn more about what this plugin can do.

Fun things to Try

I really started enjoying this plugin when I asked my agent to make me a "cool looking material, something psychedelic and weird." It made a neat kaleidoscope thing. I then asked it to make me an animated rendering of the mandelbrot set. It's fun to watch it do things like that.

History and Credits

When I myself needed an MCP for unreal development, I did a survey of the plugins out there. I ended up choosing a plugin called "Blueprint MCP" by David Gundry:

https://github.com/mirno-ehf/ue5-mcp

It was not bad, but it had some limitations, and I started doing work to improve it. Incrementally, I ended rewriting pretty much the whole thing. So this whole project is actually a fork of Blueprint MCP. There's very little of the original code remaining. However, you will find snippets here and there.

Even though I ended up rewriting most of the code, it really was useful to have a functioning starting point. It meant I could improve one thing at a time, without having to try to get everything working all at once. So I'm quite grateful to David Gundry and his work.

Software License

UE Wingman is licensed under the MIT license, a copy of which is enclosed. Its predecessor (of which it is a fork) was also under the MIT license, so everything works out.