////////////////////////////////////////////////////////////
//
// SOURCEDB
//
// This module manages the loading of lua source files into the Lua environment.
// Since the source files can be reloaded over and over, this module doesn't
// just load the source files.  Instead, it does "source rebuilds" in which it
// purges everything from the global environment, then it reinstalls everything
// that should be there.  That way, if you delete something from a lua source
// file, it gets removed from the lua global environment.
//
// THE MAKECLASS OPERATOR
//
// This module provides a new lua 'builtin' operator: "makeclass". This creates
// a table and stores it in the global environment. The new table is meant to be
// used as a class.  Three fields are initially created:
//
//    __index --> points back to the class.  Makes it convenient to use the
//                class as a metatable.
//
//    action  --> a subtable of additional methods.
//
// If you invoke 'makeclass' on a class that already exists, the existing table
// is "repaired" -  __index field is restored and the action
// subtable, if not present, is recreated.  If there are already functions or
// constants inside the class, they are not affected.
//
//
// THE LUA SOURCE DATABASE
//
// Class SourceDB only contains a single pointer to the lua environment. That's
// because all the data for SourceDB is stored in the lua registry.
// Specifically, the registry contains these keys:
//
//     1. The source database proper (registry key "sourcedb")
//     2. The snapshot of builtins (registry key "source_snapshot_builtins")
//
// The source database proper is a table where the keys are filenames, and the
// values are records containing information about that file:
//
//     "foo.lua" : {
//         "name":        "foo.lua",
//         "fingerprint": "12893129385854",
//         "code":        "function xyz ...",
//         "loadresult":  <function-23>,
//         "sequence":    13
//     }
//
// Let's break that down a little.  The "name" field is just the filename.  The
// "fingerprint" is an encoding of the file modification date and the file
// length, it is used to detect whether the file has been altered on disk
// without having to reread the file.  The "code" is the entire content of the
// source file.  The "loadresult" is the closure that results from calling the
// lua 'load' function on the code.  If load fails, then the "loadresult" is an
// error message. Finally, "sequence" indicates the order in which the source
// files are meant to be loaded.
//
// The operation SourceDB::update refreshes the source database from disk.  It
// doesn't reread files whose fingerprints have not changed. In a synchronous
// model, we don't call SourceDB::update - instead, we update the source
// database by difference transmission.
//
// Note that updating the source database has *no effect* on the lua global
// variables (or lua function definitions).  The SourceDB::update operation
// calls lua's "load" on the code, but all that does is return a loaded closure.
// Nothing happens to the lua invironment until you *invoke* the loaded closure.
// That doesn't happen until you do a SourceDB::rebuild operation, described
// below.
//
//
// SOURCE REBUILDS, IN DEPTH
//
// The function SourceDB::rebuild clears and then rebuilds the entire contents
// of the lua global environment.  The reason to clear the environment is that
// if we didn't clear it, then removing a function from the lua source would not
// remove it from the lua environment.  Rebuilding the lua environment is a
// multi-step process:
//
// * Delete everything from the global environment except class tables.
//
//     - Class tables are kept, but the contents are cleared.
//     - If a class has an "actions" subtable, that is kept and cleared.
//     - Anything else is deleted.
//
// * Lua Builtin functions are reinstalled in the global environment.
//
//     - To make this possible, the snapshot of builtins is used.
//
// * C++ functions registered with "LuaDefine" are reinstalled.
//
//     - LuaDefine creates a registry, that registry is iterated over.
//
// * Lua code is reinstalled by running the closures in the source DB.
//
//     - Simply call the "loadresult" closure to reinstall functions into the
//       global environment.
//
// Note that if you've stored any global data in the lua environment, it's gone.
// So therefore, we have to provide a separate "safe" space for global data
// structures.  That is provided elsewhere, in the module "globaldb".
//
//
// UNITTESTS
//
// We reserve the lua class name 'unittests' for storing unit tests.  Any
// function placed into this class is considered a unit test.  We don't separate
// unit tests from the rest of the code - they're compiled right into the main
// binary.
//
// Unit tests can be either lua functions, or Lua-registered C functions. Unit
// tests are executed by calling 'source_run_unittests'.
//
// Each unit test is run in a protected 'pcall' environment.  Any errors are
// printed out to console. (At least for now).
//
////////////////////////////////////////////////////////////

#pragma once

#include "wrap-string.hpp"

#include "util.hpp"
#include "luastack.hpp"
#include "streambuffer.hpp"
#include "debugcollector.hpp"

class SourceDB : public eng::nevernew  {
private:
    lua_State *lua_state_;
    static void register_lua_builtins();

public:
    void init(lua_State *L);
    
    // modules()
    // 
    // Returns a list of all the lua modules, in the proper order.
    //
    eng::vector<eng::string> modules();
    // Update
    //
    // Try to compile and load the specified source.  Then, rebuild the
    // global environment.
    // 
    // If this generates any errors, puts back the old code, and rebuilds
    // the global environment using the old code.
    // 
    // Returns any error messages.  If this returns empty string, it means
    // there were no errors and the code was successfully update.  If there
    // are any error messages, it means we restored the old code as best as
    // possible.
    //
    eng::string update(const util::LuaSourceVec &source);

    // Difference transmission.
    //
    // Note: The patch routine applies the differences to the source
    // database, and if there are any changes, it does a source rebuild.
    //
    //
    void diff(const SourceDB &auth, StreamBuffer *sb);
    bool patch(StreamBuffer *sb, DebugCollector *dbc);

    // Get/Set code (for unit testing).
    //
    // These functions are direct getters/setters for values in the source
    // database.  They are intended only for unit testing.
    //
    void set(const eng::string &fn, const eng::string &code, int sequence);
    eng::string get(const eng::string &fn);

    // Get function documentation.
    //
    // Returns false if it has no documentation for the specified function.
    //
    bool function_docs(const LuaCoreStack &LS, LuaSlot slot, std::ostream &ostream);

    // Search the documentation.
    //
    // Search all the documentation for the specified substring.
    //
    // Returns false if it found nothing and output nothing to the stream.
    //
    bool search_docs(const eng::string &substring, std::ostream &ostream);

    // Serialize and unserialize a source vector.
    //
    static void serialize_source(const util::LuaSourceVec &sv, StreamBuffer *sb);
    static util::LuaSourceVec deserialize_source(std::string_view datapack);

private:
    //////////////////////////////////////////////////////////////////////////
    //
    // Internal implementation stuff.
    //
    //////////////////////////////////////////////////////////////////////////
    
    // rebuild
    //
    // Rebuild the global environment from the sourcedb:
    // 
    // * Clears the environment
    // * Installs the builtins
    // * Executes all the closures in the sourcedb.
    // * Regenerates the function-names table.
    //
    // The closures may generate errors, if so, this returns the error
    // messages.  Note that if there are errors, there is no automatic cleanup.
    //
    eng::string rebuild();

    // get_source
    //
    // Get the source code for a given module.
    // 
    eng::string get_source(const eng::string &fn);

    // rebuild_module
    //
    // To rebuild the lua environment, fetch the module list, then
    // call rebuild_module on each module in turn.  This will return
    // an error message for the module, or empty string if no error.
    //
    // This is a thin wrapper around traceback_pcall.  The return
    // value is the return value of traceback_pcall.
    //
    eng::string rebuild_module(const eng::string &mod);
    

    // rebuild_funcnames
    //
    // Traverses the global environment and populates the registry "funcnames"
    // table, mapping each closure to its name.
    //
    void rebuild_funcnames();

};



