integration/Docs/Different-Kinds-of-Lua-Variables.md

Global Variables in Luprex

Normally, lua programs store global variables in the global environment table. In luprex, however, global data needs to be differentiated into several different kinds:

• Some global data needs to be difference transmitted, some doesn't.

• Some global data needs special garbage collection strategies.

• Some global data needs to be instanced per-user, some doesn't.

Simply throwing all these different types of data into a single table causes a problem, because it doesn't give us the means to treat these different types of data differently. So instead, we have different tables for different types of global data.

This document explains all the different things that would normally go into a lua global environment table, and it describes what we do with each of those different kinds of data.

Function Definitions, Class Definitions, and Constant Definitions

In a normal lua interpreter, function definitions are stored in the global environment table. Luprex sticks with that convention. Classes and constants also get stored in the global environment table. For example, suppose you write this code in a lua source file:

makeclass("deque")

function deque.push_right(deque, value)

…

end

deque.default_size = 10

The three statements above are all actually storing data into _G, the global environment table. The code above could be rewritten to explicitly mention the global environment table:

_G["deque"] = {} – this is an oversimplification of makeclass

_G["deque"]["push_right"] = function(deque, value) … end

_G["deque"]["default_size"] = 10

When you think about it that way, a typical lua source file is really a long list of statements that insert closures, and sometimes classes and constants, into the global environment table. That's how it works in normal lua, and that's also how it works in luprex.

We need to transmit the lua code to all connected clients. In lua, closures are opaque objects. There is no API to do anything to them, other than call them: they can't be examined. So we simply can't transmit closures.

That means we need a different strategy. Instead of transmitting the closures, we transmit the source code. When we transmit the source code to a client, that client compiles the source code itself. When the client loads the source code, the client inserts class definitions, function definitions, and constant definitions into its own copy of the global environment table.

For this reason, the class definitions, function definitions, and constant definitions that are present in the global environment table don't need to be difference transmitted. Instead, we transmit the source code, and that's sufficient.

Suppose that you edit a lua source file, and remove the definition of the function deque.push_right. Removing the function from the source doesn't automatically remove it from the global environment table. Even if you reload the source file, the function deque.push_right will linger in the global environment. We added a garbage collection strategy to handle this problem: before reloading the lua source, all the function and constant definitions in the global environment are removed. The global environment is rebuilt cleanly from scratch.

The global environment table has these properties:

• Table: The Global Environment Table

• Contents: Function definitions, class definitions, and constant definitions from the lua source.

• Difference Transmission: The source is transmitted, the client makes its own copy from source.

• Per-User Instancing: There's only one such table, shared by all users.

• Garbage Collection: We purge the table before reloading the source.

Interactive Temporary Variables

When you're typing commands into a lua interpreter interactively, you inevitably will need to create temporary variables. For example, suppose that you're an administrator and you want to teleport yourself somewhere. You might type these commands into the interactive lua prompt:

a = tangible.actor() — get a pointer to my actor

tangible.animate{tan=a, anim={action="warpto", xyz={100,100,100}}}

You just created a temporary variable, a. In a normal lua interactive shell, these temporary variables go into the global environment table.

But consider what would happen if the temporary variables from hundreds of interactive sessions were to just accumulate in the luprex global environment table. It would be a mess.

Another problem is that two administrators might be typing commands at the same time. They might both use the temporary variable "x" to mean different things, and they would interfere with each other.

For this reason, when you log into a luprex server with an administrative account that has permission to type interactive lua commands, it creates a new interactive environment table which is private to you. When you log out again, the table is discarded. Lua treats the interactive environment table as if it were the global environment table for any commands you type. The interactive environment table is initially empty, but it has a metatable that points to the true global environment table, so typing the name of a function or class will work - you can "read through."

The interactive environment table is a useful tool in that it makes it feasible to use temporary variables. But it is important to remember that it is not a sandbox. It is very easy to unintentionally inject junk into the real lua environment table.

For example, suppose you type this into the interactive prompt:

function foo(x) … end

This is fine. It stores a function foo in your interactive environment table. Nobody else can see it, and it gets garbage collected when you log out. But if you type this:

function foo.bar(x) … end

This inserts a function bar into the already-existing class foo. If class foo is a class defined in a lua source file, and therefore, class foo is visible to all users, then the new function foo.bar will be visible to all users. Doing this would be a bad idea. Remember, the system has a mechanism to garbage collect any function that has been removed from the lua source files - the function foo.bar is not in a lua source file. It will get removed from the global environment as soon as somebody reloads the lua source. It also won't get transmitted to clients, again, because it's not in a lua source file.

The upshot is this: don't define functions, classes, or constants in interactive sessions. Put them into source files, and reload the source files.

The interactive environment table has these properties:

• Table: The Interactive Environment Table

• Contents: Temporary variables created in interactive shells.

• Difference Transmission: Transmitted as part of the actor tangible.

• Per-User Instancing: Each logged-in administrator gets his own.

• Garbage Collection: We purge the table when the administrator logs out.

Note: this table is not implemented yet.

Broadcast Globals

By "broadcast globals," I mean global variables that can be updated during the course of the program. When you do update one, the new value gets broadcast to all connected clients. When a new user logs in, he receives the current values of all broadcast globals. When using these, it is important to be thoughtful about the bandwidth requirements of using them.

Broadcast globals are stored in their own table. This table is accessed using these two functions:

global.set("x", value)

value = global.get("x")

When you call global.set("x", value), the value is immediately serialized. It is this serialized value that is stored. When you call value = global.get("x"), the stored value is deserialized and returned.

It is important to remember those semantics, for performance reasons. Both the global.set and global.get operations are O(N) in the size of the data being stored. It is also important to remember that both functions have copy semantics: if you store an object in the table, any mutations you apply to the object don't affect what's in the table: the table contains a serialized copy of the object, not the original.

Simple values can be stored in a very straightforward way: int, string, float, boolean, tokens, nil - all of these can be stored and retrieved without thinking about it. But more complex values require a little thought.

Tables can be stored, even if they recursively contain more tables, and even if the table references contain cycles. The entire data structure will be packaged up. Be aware that if the data structure contains a pointer to another data structure, that other data structure will get traversed and packaged up too. Storing a large data structure will take time proportional to the size of the data structure.

Tangibles are a special case. Storing a tangible in a broadcast global does not serialize the contents of the tangible. Instead, only the tangible's ID is serialized. When a client uses global.get to fetch the value, the the function returns the client's copy of the tangible. Tangible contents are difference transmitted separately.

Classes are also a special case. You can store a class in a broadcast global, but only the class name is actually serialized. Since every client has a copy of the source code, using global.get will fetch the client's copy of the class.

Closures can not be stored in broadcast globals at all. Our system is not capable of transmitting closures across the network. Attempting to store one will cause an error to be thrown in the routine global.set.

For more details about object serialization, read the documentation of the table.serialize function.

The Broadcast Globals Table has these properties:

• Table: The Broadcast Globals Table

• Contents: Anything you store using the function global.set.

• Difference Transmission: Difference transmitted to all clients, immediately upon update.

• Per-User Instancing: None, all users share this data.

• Garbage Collection: None. This data must be manually cleaned up.

Global Once Flags

Sometimes, in a source file, it is useful to say "execute this code only if it has never been executed before." You can accomplish this in luprex using code that looks like this:

if global.once("initialize-database") then

initialize_database()

end

When the source file is loaded, if the global.once expression has never been evaluated before, then the database will get initialized.

Sometimes you want to trigger the 'once' code to execute again. In the example above, for example, you might want to reinitialize the database. In that case you can do this:

global.clearonce("initialize-database")

After doing that, the next time you reload the source file, the database will get initialized again.

Currently, the global once flags are implemented on top of broadcast globals. It is not clear whether this is the right choice. I am not sure that it is necessary to difference transmit global once flags. I may reimplement these to have their own table which only exists on the master, and which is not predicted on the client. But for now, they work on top of broadcast globals. That should work okay for now.

Since these are implemented on top of broadcast globals, they don't have their own table yet.