integration/luprex/cpp/core/source.hpp

////////////////////////////////////////////////////////////
//
// SOURCEDB
//
// This module manages the loading of lua source files into the Lua environment.
// Since the source files can be reloaded over and over, this module doesn't
// just load the source files.  Instead, it does "source rebuilds" in which it
// purges everything from the global environment, then it reinstalls everything
// that should be there.  That way, if you delete something from a lua source
// file, it gets removed from the lua global environment.
//
// THE MAKECLASS OPERATOR
//
// This module provides a new lua 'builtin' operator: "makeclass". This creates
// a table and stores it in the global environment. The new table is meant to be
// used as a class.  Three fields are initially created:
//
//    __index --> points back to the class.  Makes it convenient to use the
//                class as a metatable.
//
//    action  --> a subtable of additional methods.
//
// If you invoke 'makeclass' on a class that already exists, the existing table
// is "repaired" -  __index field is restored and the action
// subtable, if not present, is recreated.  If there are already functions or
// constants inside the class, they are not affected.
//
//
// THE LUA SOURCE DATABASE
//
// Class SourceDB only contains a single pointer to the lua environment. That's
// because all the data for SourceDB is stored in the lua registry.
// Specifically, the registry contains these keys:
//
//     1. The source database proper (registry key "sourcedb")
//     2. The snapshot of builtins (registry key "source_snapshot_builtins")
//
// The source database proper is a table where the keys are filenames, and the
// values are records containing information about that file:
//
//     "foo.lua" : {
//         "name":        "foo.lua",
//         "fingerprint": "12893129385854",
//         "code":        "function xyz ...",
//         "loadresult":  <function-23>,
//         "sequence":    13
//     }
//
// Let's break that down a little.  The "name" field is just the filename.  The
// "fingerprint" is an encoding of the file modification date and the file
// length, it is used to detect whether the file has been altered on disk
// without having to reread the file.  The "code" is the entire content of the
// source file.  The "loadresult" is the closure that results from calling the
// lua 'load' function on the code.  If load fails, then the "loadresult" is an
// error message. Finally, "sequence" indicates the order in which the source
// files are meant to be loaded.
//
// The operation SourceDB::update refreshes the source database from disk.  It
// doesn't reread files whose fingerprints have not changed. In a synchronous
// model, we don't call SourceDB::update - instead, we update the source
// database by difference transmission.
//
// Note that updating the source database has *no effect* on the lua global
// variables (or lua function definitions).  The SourceDB::update operation
// calls lua's "load" on the code, but all that does is return a loaded closure.
// Nothing happens to the lua invironment until you *invoke* the loaded closure.
// That doesn't happen until you do a SourceDB::rebuild operation, described
// below.
//
//
// SOURCE REBUILDS, IN DEPTH
//
// The function SourceDB::rebuild clears and then rebuilds the entire contents
// of the lua global environment.  The reason to clear the environment is that
// if we didn't clear it, then removing a function from the lua source would not
// remove it from the lua environment.  Rebuilding the lua environment is a
// multi-step process:
//
// * Delete everything from the global environment except class tables.
//
//     - Class tables are kept, but the contents are cleared.
//     - If a class has an "actions" subtable, that is kept and cleared.
//     - Anything else is deleted.
//
// * Lua Builtin functions are reinstalled in the global environment.
//
//     - To make this possible, the snapshot of builtins is used.
//
// * C++ functions registered with "LuaDefine" are reinstalled.
//
//     - LuaDefine creates a registry, that registry is iterated over.
//
// * Lua code is reinstalled by running the closures in the source DB.
//
//     - Simply call the "loadresult" closure to reinstall functions into the
//       global environment.
//
// Note that if you've stored any global data in the lua environment, it's gone.
// So therefore, we have to provide a separate "safe" space for global data
// structures.  That is provided elsewhere, in the module "globaldb".
//
//
// UNITTESTS
//
// We reserve the lua class name 'unittests' for storing unit tests.  Any
// function placed into this class is considered a unit test.  We don't separate
// unit tests from the rest of the code - they're compiled right into the main
// binary.
//
// Unit tests can be either lua functions, or Lua-registered C functions. Unit
// tests are executed by calling 'source_run_unittests'.
//
// Each unit test is run in a protected 'pcall' environment.  Any errors are
// printed out to console. (At least for now).
//
////////////////////////////////////////////////////////////

#ifndef SOURCE_HPP
#define SOURCE_HPP

#include "wrap-string.hpp"

#include "util.hpp"
#include "luastack.hpp"
#include "streambuffer.hpp"
#include "debugcollector.hpp"

class SourceDB : public eng::nevernew  {
private:
    lua_State *lua_state_;

public:
    void init(lua_State *L);
    
    // Update
    //
    // Update the database using the specified lua source code.
    // Compiles these files using lua's "load" function.
    //
    void update(const util::LuaSourceVec &source);

    // modules
    //
    // Returns a list of all the modules.  The first item in the list
    // is always the string "CORE" which represents the lua core
    // functionality with all the builtins.  This is then followed by
    // all the lua sourcefiles in the correct order.
    //
    eng::vector<eng::string> modules();
    
    // rebuild_module
    //
    // To rebuild the lua environment, fetch the module list, then
    // call rebuild_module on each module in turn.  This will return
    // an error message for the module, or empty string if no error.
    //
    // This is a thin wrapper around traceback_pcall.  The return
    // value is the return value of traceback_pcall.
    //
    eng::string rebuild_module(const eng::string &mod);
    
    // rebuild_core
    //
    // This is equivalent to rebuild_module("CORE").  Clears the environment
    // and installs all the builtins.  No error conditions.
    //
    void rebuild_core();

    // Difference transmission.
    //
    // Note: The patch routine applies the differences to the source
    // database, and if there are any changes, it does a source rebuild.
    // The patch routine returns true if anything was modified.
    //
    void diff(const SourceDB &auth, StreamBuffer *sb);
    bool patch(StreamBuffer *sb, DebugCollector *dbc);

    // run_unittests
    //
    // Run all the lua unit tests.  Print any errors to console.  If there
    // are any errors, exits the program.
    //
    void run_unittests();

    // Get/Set code (for unit testing).
    //
    // These functions are direct getters/setters for values in the source
    // database.  They are intended only for unit testing.
    //
    void set(const eng::string &fn, const eng::string &code, int sequence);
    eng::string get(const eng::string &fn);

    // Add builtins to the global function registry.
    //
    static void register_lua_builtins();
    
    // Get function documentation.
    //
    static eng::string function_docs(const LuaCoreStack &LS, LuaSlot slot);

    // Serialize and unserialize a source vector.
    //
    static void serialize_source(const util::LuaSourceVec &sv, StreamBuffer *sb);
    static void deserialize_source(util::LuaSourceVec *sv, StreamBuffer *sb);
};

#endif // SOURCE_HPP