Rename files in Docs, and add new Doc about print statements.

CLAUDE.md

@@ -44,7 +44,7 @@ Two update channels flow into the synchronous models:
 1. **Command acknowledgements** — for the client's own actions, keeping the two synchronous models in lockstep.
 2. **Difference transmission** — for everything else (other players' actions, server-side events, tangibles entering/leaving visibility).
 
-See `Docs/Predictive Reexecution.md` for the full explanation.
+See `Docs/Predictive-Reexecution.md` for the full explanation.
 
 ## Architecture: Lua / Unreal Separation
 
@@ -69,14 +69,14 @@ On the Unreal side, **Tangible Actor blueprints** (TangibleStaticMesh, TangibleS
 
 ## Architecture: Lua Environment
 
-- **Patched Lua runtime** — deterministic table iteration, deterministic table length, flag bits on tables, generalized less-than, C++ exceptions instead of longjmp, and more. See `Docs/A Summary of our Lua Patches.md`.
+- **Patched Lua runtime** — deterministic table iteration, deterministic table length, flag bits on tables, generalized less-than, C++ exceptions instead of longjmp, and more. See `Docs/A-Summary-of-our-Lua-Patches.md`.
-- **LuaStack API** — custom C++ API replacing the standard Lua C API. Uses `LuaDefStack`/`LuaExtStack` with `LuaArg`/`LuaVar`/`LuaRet` slots mapped to stack positions. See `Docs/Our In-House Lua API.md`.
+- **LuaStack API** — custom C++ API replacing the standard Lua C API. Uses `LuaDefStack`/`LuaExtStack` with `LuaArg`/`LuaVar`/`LuaRet` slots mapped to stack positions. See `Docs/Our-In-House-Lua-API.md`.
 - **LuaDefine macro** — declares Lua-callable C++ functions and auto-registers them in a global registry for automatic insertion into the Lua environment.
 - **eng::malloc heap** — custom deterministic memory allocator for the driven portion, ensuring reproducible addresses during replay.
 
 ## Architecture: Determinism
 
-The driven portion must be fully deterministic so that synchronous models stay in lockstep and event replay works. Rules: no true random numbers, no iterating unordered maps, no real-time clocks, no threads (with carefully sandboxed exceptions). See `Docs/The Event-Driven Structure of the Engine.md`.
+The driven portion must be fully deterministic so that synchronous models stay in lockstep and event replay works. Rules: no true random numbers, no iterating unordered maps, no real-time clocks, no threads (with carefully sandboxed exceptions). See `Docs/The-Event-Driven-Structure-of-the-Engine.md`.
 
 ## Architecture: GUI System
 
@@ -84,21 +84,21 @@ Blueprints call into Lua via two mechanisms:
 - **Invokes** — change world state, forwarded to server, executed in order per predictive reexecution rules.
 - **Probes** — read-only, return data to blueprints, run locally on client.
 
-Look-at widgets, hotkeys, and menus are built on top of this. The menu system is implemented entirely in "user space" Lua and blueprint code. See `Docs/Displaying Widget Blueprints.md`.
+Look-at widgets, hotkeys, and menus are built on top of this. The menu system is implemented entirely in "user space" Lua and blueprint code. See `Docs/Displaying-Widget-Blueprints.md`.
 
 ## Key Documentation
 
-- `Docs/Predictive Reexecution.md` — how the four world models stay in sync
+- `Docs/Predictive-Reexecution.md` — how the four world models stay in sync
-- `Docs/The Event-Driven Structure of the Engine.md` — driver/driven separation, determinism, replay
+- `Docs/The-Event-Driven-Structure-of-the-Engine.md` — driver/driven separation, determinism, replay
-- `Docs/Multipass Difference Transmission.md` — the algorithm for syncing Lua table graphs
+- `Docs/Multipass-Difference-Transmission.md` — the algorithm for syncing Lua table graphs
-- `Docs/Animation Queues and Tangible Actors.md` — how blueprints interpret animation queues
+- `Docs/Animation-Queues-and-Tangible-Actors.md` — how blueprints interpret animation queues
-- `Docs/Our In-House Lua API.md` — the LuaStack API (LuaDefStack, LuaExtStack)
+- `Docs/Our-In-House-Lua-API.md` — the LuaStack API (LuaDefStack, LuaExtStack)
-- `Docs/A Summary of our Lua Patches.md` — all modifications to the Lua runtime
+- `Docs/A-Summary-of-our-Lua-Patches.md` — all modifications to the Lua runtime
-- `Docs/Major Data Structures.md` — World, tangibles, threads, classes, source database
+- `Docs/Major-Data-Structures.md` — World, tangibles, threads, classes, source database
-- `Docs/Displaying Widget Blueprints.md` — GUI system (invokes, probes, look-at widgets, menus)
+- `Docs/Displaying-Widget-Blueprints.md` — GUI system (invokes, probes, look-at widgets, menus)
-- `Docs/Global Variables.md` — different types of global data and their transmission rules
+- `Docs/Global-Variables.md` — different types of global data and their transmission rules
-- `Docs/Correct Implementation of Blocking Operations and NoPredict.md` — how to handle blocking ops
+- `Docs/Correct-Implementation-of-Blocking-Operations-and-NoPredict.md` — how to handle blocking ops
-- `Docs/Difference Transmission with Threads.md` — why concurrent diff transmission is hard
+- `Docs/Difference-Transmission-with-Threads.md` — why concurrent diff transmission is hard
 
 ## Session Startup
 

Docs/Print-Statement-Handling.md

@@ -0,0 +1,142 @@
+### Handling Print Statements
+
+This document describes how "print" statements are handled within
+Luprex.  It documents the full path by which print statements
+originate with Lua, and gradually travel to the Unreal virtual
+console and debug log.  There are two types of print statements:
+
+- *dprint*, for messages that go to the debug logs.
+- *print*, for messages that go to a window in the user's GUI.
+
+The following sections explain the differences, and how each
+of these is implemented.
+
+## dprint, for messages that go to the debug logs
+
+The lua code can use 'dprint' to print a message into Unreal's
+debugging logs.  A 'dprint' goes to the same places
+that a debugging message within Unreal goes.  That includes:
+
+- Visual studio's debug message window.
+- The Unreal Editor's debug message window.
+- Unreal's debug message log file.
+
+In the lua server, currently, dprints just go to stderr.  We're
+probably going to enhance that by adding a log file as well.
+
+The unreal logs are not managed by predictive reexecution.
+To put it differently, once a message goes into the unreal logs,
+it can't be corrected by difference transmission.  Because of
+this, if a dprint is reexecuted, you will get multiple possibly
+conflicting messages in the unreal logs.
+
+## Implementation of dprint
+
+The luprex DLL contains a function util::dprintview, along with
+a couple of convenience wrappers like util::dprintf.  This
+function is used to make a debugging print.  It is used in a
+number of places throughout Luprex.  It can also be called from
+lua, via the 'dprint' function.
+
+The Luprex DLL exports an API: EngineWrapper::hook_dprint.  This
+function accepts a pointer to a C callback function, which takes
+a string as a parameter.  The driver is expected to call
+hook_dprint early in 'main', to set the dprint callback. The
+pointer to the callback function is stored in the global
+variable util::dprint_hook.  The function util::dprintview
+breaks the string into lines, and calls util::dprint_hook
+once per line.
+
+In the Unreal client, the dprint callback is set to a function
+that calls UE_LOG, the unreal error logging macro, with a log
+category of LogLuprex.  So therefore, in the Unreal client,
+calling dprint is equivalent to calling UE_LOG.
+
+In the server, the dprint callback is set to a function that
+outputs the string to the console (ie, stdout).
+
+
+The print-callback is the *only* place where the Luprex DLL actually
+calls into the driver (via a callback).  Normally, the driver is
+supposed to call into the Luprex DLL, but not the other way around.
+This one exception is necessary to handle the case where the
+Luprex DLL is about to crash and wants to print a message before it
+does.
+
+## print, for messages that go to a window in the user's GUI
+
+The client is expected to have a GUI of some sort
+that includes a text console, where messages can be
+displayed.  The print statement puts a message into this GUI
+console.
+
+Every person who is logged in has their own GUI text console.
+Therefore, it is possible to print a message onto any one
+of those.  When the lua code executes a print statement,
+it sends its output to the GUI console of the *actor*.
+
+The contents of the GUI text console is part of the world model.
+Therefore, it can be corrected by difference transmission.
+If a print-statement is predicted incorrectly, the user's
+GUI text console will temporarily contain the wrong text, but
+it will get fixed by difference transmission.
+
+## Implementation of print
+
+Inside the Luprex DLL, class PrintBuffer is used to store
+the contents of the GUI console.  Every logged in player has
+a PrintBuffer as part of their character tangible.
+Difference transmission is capable of amending the contents
+of the PrintBuffer.
+
+When the luprex thread scheduler starts executing a thread,
+it sets up an ostringstream to collect any print statements.
+When the thread pauses or ends, the contents of the ostringstream
+are copied into the PrintBuffer.  The code to create the
+stringstream and to copy it into the PrintBuffer are both
+in the thread scheduling code in world-core.cpp.
+
+The stringstream isn't *always* copied into the PrintBuffer.
+There are exceptions: for example, in a 'probe',
+print statements don't go into the PrintBuffer, they get
+rerouted to dprint instead.
+
+The difference transmitter handles PrintBuffers specially.
+It doesn't just fix the contents of the PrintBuffer. It also
+leaves an 'authoritative' bit inside the PrintBuffer to
+indicate which print statements are confirmed as authoritative,
+and conversely, to indicate which ones are still just
+predictions.  Because of this, the PrintBuffer knows the
+difference between a "final" print and a "tentative" print.
+
+The client and server in lpxclient.cpp and lpxserver.cpp both
+contain objects  of class PrintChanneler.  This class is designed
+to monitor a PrintBuffer to see if anything has changed.
+Specifically, it is capable of answering the question: does
+the PrintBuffer contain any new authoritative print statements?
+
+Periodically, lpxserver.cpp and lpxclient.cpp will check
+their PrintChannelers to see if there are any new authoritative
+print statements.  If so, they will set a flag called "have_prints"
+in the DrivenEngine.  Unreal periodically polls this flag using
+EngineWrapper::get_have_prints.
+
+If the flag is set, Unreal then calls 
+EngineWrapper::play_access(... CHANNEL_PRINTS ...).
+This asks the PrintChanneler to fetch all new authoritative prints.
+Then, CHANNEL_PRINTS will send an invoke via the standard
+predictive reexecution channels to FLUSH_PRINTS - ie, to forget
+the print statements that it just channeled.
+
+When the PrintChanneler returns prints to Unreal, Unreal
+passes those prints to the LuprexGameMode blueprint by calling
+LuprexGameMode::AddConsoleOutput.  The LuprexGameMode is
+currently responsible for implementing the GUI text console.
+That will probably change at some point.
+
+The LuprexGameMode maintains an object of class UlxConsoleOutput
+which keeps a record of what's in the GUI Console.  When
+AddConsoleOutput feeds new prints into the LuprexGameMode,
+those prints get added to the UlxConsoleOutput.  This stores
+the contents of the console as one big string. From there,
+the string is copied into a text widget.

Integration.code-workspace.tpl.json

@@ -54,7 +54,8 @@
             "**/Binaries": true,
             "**/DerivedDataCache": true,
             "**/*.generated.h": true,
-            "**/*.gen.cpp": true
+            "**/*.gen.cpp": true,
+            "**/*.d": true
         }
     },
     "extensions": {

Source/Integration/AnimQueue.h

@@ -125,11 +125,9 @@ public:
 	UFUNCTION(BlueprintPure, meta = (BlueprintAutocast), Category = "Luprex|Animation Step")
     static int64 AnimationStepID(const FlxAnimationStep& step) { return step.Hash; }
 
-	// Scan an animation step for key-value pairs of the form mat_XXXX={x,y,z}.
+	// Using mat_xxxx values from the animation step, update the actor's
-	// For each match, create a dynamic material instance on the actor's mesh
+	// material parameters. Doing this may involve creating or replacing
-	// components and set the vector parameter XXXX.  Materials are restored to
+	// dynamic material instances for the actor.
-	// their base (non-dynamic) state before applying, so parameters from
-	// previous calls do not persist.
 	//
 	UFUNCTION(BlueprintCallable, Meta = (DefaultToSelf = "actor"), Category = "Luprex|Animation Step")
 	static void AnimationStepApplyMaterials(const FlxAnimationStep& step, AActor* actor);
@@ -137,6 +135,8 @@ public:
 	// Look for a mesh=name key-value pair in the animation step.
 	// If found, load the named mesh and apply it to the actor's
 	// mesh component.  The actor must have exactly one mesh component.
+	// If FallbackToBP is true, and mesh=name is not present, looks
+	// for a bp=name pair instead.
 	//
 	UFUNCTION(BlueprintCallable, Meta = (DefaultToSelf = "actor"), Category = "Luprex|Animation Step")
 	static void AnimationStepApplyMesh(const FlxAnimationStep& step, bool FallbackToBP, AActor* actor);
@@ -144,6 +144,8 @@ public:
 
 ////////////////////////////////////////////////
 //
+// An animation step that doesn't actually store the step,
+// it just contains a pointer to the string.
 // 
 ////////////////////////////////////////////////
 
@@ -243,7 +245,7 @@ private:
 	FlxStreamBuffer Decoder;
 
 public:
-	// Initialize the FlxAnimationStepDecoder from the FlxAnimationStepView.
+	// Initialize the FlxAnimationStepDecoder.
 	// 
 	FlxAnimationStepDecoder(std::string_view body) : Decoder(body) {}
 

Source/Integration/ConsoleOutput.h

@@ -4,21 +4,34 @@
 
 #include "ConsoleOutput.generated.h"
 
-
 //////////////////////////////////////////////////////////////
 //
 // ConsoleOutput
 //
-// This class stores the text that's in the unreal console.
+// When the lua code executes a print statement, the text
-// It stores it as one great big string, which contains
+// eventually gets passed to the GameMode blueprint: see
-// newlines to denote line breaks.
+// Docs/Print-Statement-Handling.md for more information.
+//
+// The GameMode blueprint is expected to create a virtual
+// console of some sort to display the print statements.
+// This class, ConsoleOutput, is a class that the GameMode
+// can optionally use to help implement that virtual console.
+//
+// This class just collects the print statements and keeps
+// a record of what text is in the virtual console.  The
+// text is stored as one big string.
 //
 // This class also contains a 'dirty' bit.  Each time somebody
 // appends a line of text to the console, the dirty bit is
 // automatically set.  The bit can be checked using 'IsDirty'
-// and cleared using 'ClearDirty'.  This makes it so that
+// and cleared using 'ClearDirty'.  Assuming that the GameMode
-// you don't have to update the unreal widget unless the
+// is maintaining a text widget of some sort, the GameMode
-// text has actually changed.
+// can transfer the contents of this buffer into the text
+// widget only when the dirty bit is set.
+//
+// Note that the GameMode is not obligated to use this class.
+// If the GameMode wants to use some other framework to
+// implement the virtual console, that's perfectly fine.
 //
 //////////////////////////////////////////////////////////////