integration/luprex/core/cpp/planemap.cpp

// PlaneTree is an octree-like data structure.
//
// THE COORDINATE SPACE
//
// The tree stores objects which have (x,y,z) coordinates where each of x,y,z is
// an unsigned 16-bit integer. The API uses floating-point coordinates, but
// these are converted to uint16_t for use in the tree.
//
// In an octree, each node has a 2x2x2 array of children (total: 8 children). In
// this tree, however, each node has a 4x4x4 array of children (total: 64
// children).
//
// The depth of the tree is fixed.  It always has one leaf level (level 0), plus
// 8 internal levels (levels 1-8).  The root of the tree is level 8. Tangibles
// are always are always stored in the leaf-level. Since there are 8
// subdivisions, at 4x4x4 per subdivision, that divides the space into 65536 x
// 65536 x 65536. Hence the 16-bit integer coordinates.
//
// Here are the sizes of the levels:
//
//  * Level 8 (internal) is 1 cubed.
//  * Level 7 (internal) is 4 cubed.
//  * Level 6 (internal) is 16 cubed.
//  * Level 5 (internal) is 64 cubed.
//  * Level 4 (internal) is 256 cubed.
//  * Level 3 (internal) is 1024 cubed.
//  * Level 2 (internal) is 4096 cubed.
//  * Level 1 (internal) is 16385 cubed.
//  * Level 0 (leaf) is 65536 cubed.
//
// NODE IDS AND THE NODE TABLES
//
// Every tree node has a unique "NodeID", which consists of it's level and the
// lowest (x,y,z) coordinate in its region.  Every tree node is stored in a hash
// table that maps NodeID to the data for that tree node.
//
// Tree nodes don't contain any child pointers or parent pointers.  Instead, you
// can calculate the NodeIDs of the parent and children from the NodeID of a
// node. So, you can just look up parents and children in the hash table. Even
// though PlaneTree really is a tree, it doesn't use pointers at all.
//
// There are two separate hash tables - one for storing internal nodes, and one
// for storing leaf nodes.  The only data stored in an internal node is a
// bitvector indicating which of its 64 children are nonempty.  The only data
// stored in a leaf node is a list of PlaneItems.   The list of PlaneItems is
// stored as an intrusive doubly-linked circular list.
//
// THE PLANETREE BOUNDING BOX, OUTLIERS, AND ADAPTIVE RESIZING
//
// The planetree as a whole has a bounding box.  For now, the bounding box is
// fixed at (-65536, -65536, -65536) to (65536, 65536, 65536).  That makes the
// size of the leaf cells two meters.  However, it is my plan to eventually have
// the bounding box selected adaptively.
//
// "Outliers" are objects that are outside the PlaneTree's bounding box. When an
// object is an outlier, its (X,Y,Z) coordinates are clamped to the tree's
// bounding box for purposes of storing it in the tree.  This puts all the
// outliers into the cells on the edge of the tree.  When we need to scan a
// region that includes areas outside the tree's bounding box, we scan the edge
// cells.
//
// The tree keeps a count of the number of outliers.  The intended use for these
// outlier counts is to implement adaptive logic that says "are there too many
// outliers? If so, expand the tree's bounding box."  But we haven't written the
// adaptive algorithm yet, so the outlier counts don't serve any purpose yet.
//
// We probably *don't* want to capture 100% of the outliers.  It seems likely
// that the programmer will deliberately put a few objects way off in the
// distance.  For example, you might imagine somebody creating a "LoginManager"
// object and putting it way off at (1000000, 0, 0) in order to get it out of
// the way. If we were to expand the bounding box to include such objects, we
// would make the tree's bounding box unreasonably large.  I have an idea for a
// heuristic: expand the bounding box until it covers 90% of the tangibles, then
// expand it by 25% more, then stop.  I have another idea for a different
// heuristic: keep expanding the bounding box as long as making a small
// increment will capture significantly more tangibles, then stop. I don't know
// if either of those heuristics will work well.
//
// Any adaptive algorithm we create must take another factor into account: we
// don't want the cell size to be too small. Overly small cell sizes create lots
// of extra work when an object moves around.  So sometimes, it's advantageous
// to deliberately expand the bounding box way beyond what is actually needed to
// capture the outliers, just to make the cell size large enough.  Doing this
// isn't wasteful: octrees are very good at handling big empty spaces inside the
// bounding box.  As long as the cell size is large enough, but not too large,
// and there aren't too many outliers, everything will work well. Unfortunately,
// I haven't devised a good heuristic to decide when the cell size is "right."
//
// Another thought I have, periodically, is that writing a lot of logic just to
// choose a cell size seems like a lot of unnecessary work: we could just
// configure the cell size for the game in the script.
//
// BYPASSING THE TREE
//
// In many cases, it's possible to jump directly to a leaf node and start
// processing.  For example, let's say you want to move a tangible.  You know
// its current position, therefore, you know exactly the NodeID of the leaf node
// where it is stored.  There is no need to walk the tree to get there: you can
// just access it directly using the hash table.
//

#include "luastack.hpp"
#include "util.hpp"
#include "planemap.hpp"

#include <algorithm>
#include <cmath>

using NodeID = uint64_t;
using ChildBits = uint64_t;
using IdVector = util::IdVector;

// These need to be static floats to encourage gcc to generate
// efficient code in NodeInfo.
static float k_lo = 0.0f;
static float k_hi = 65535.0f;

static constexpr ChildBits child_bit(int i) {
    return (uint64_t(1) << i);
}

static constexpr ChildBits child_bit(int x, int y, int z) {
    return child_bit((x << 4) | (y << 2) | z);
}

static constexpr uint8_t node_get_level(NodeID node) {
    return node >> 48;
}

static constexpr uint16_t node_get_x(NodeID node) {
    return uint16_t(node >> 32);
}

static constexpr uint16_t node_get_y(NodeID node) {
    return uint16_t(node >> 16);
}

static constexpr uint16_t node_get_z(NodeID node) {
    return uint16_t(node >> 0);
}

static constexpr NodeID node_lxyz(uint8_t level, uint16_t x, uint16_t y, uint16_t z) {
    return (NodeID(level) << 48) | (NodeID(x) << 32) | (NodeID(y) << 16) | (NodeID(z) << 0);
}

static constexpr NodeID node_parent(NodeID node) {
    uint8_t level = node_get_level(node);
    return ((node & node_lxyz(0, 0xFFFC, 0xFFFC, 0xFFFC)) >> 2) | node_lxyz(level + 1, 0, 0, 0);
}

static constexpr uint8_t node_childindex(NodeID node) {
    uint64_t masked = node & node_lxyz(0, 0x0003, 0x0003, 0x0003);
    return (masked >> 0) | (masked >> 14) | (masked >> 28);
}

static constexpr ChildBits node_childbit(NodeID node) {
    return child_bit(node_childindex(node));
}

static constexpr NodeID node_child(NodeID node, uint16_t x, uint16_t y, uint16_t z) {
    int level = node_get_level(node);
    return ((node & node_lxyz(0, 0xFFFF, 0xFFFF, 0xFFFF)) << 2) | node_lxyz(level - 1, x, y, z);
}

static constexpr NodeID node_nthchild(NodeID node, int i) {
    return node_child(node, (i >> 4) & 3, (i >> 2) & 3, (i >> 0) & 3);
}

static void print_node_id(NodeID node, std::ostream *os) {
    int level = node_get_level(node);

    if (level >= 8) {
        (*os) << "L" << level << ":root";
        return;
    }
    
    uint16_t x = node_get_x(node);
    uint16_t y = node_get_y(node);
    uint16_t z = node_get_z(node);
    auto fmt = util::hex.width(4 - (level >> 1)).fill('0');
    (*os) << "L" << level << ":" << fmt.val(x) << "," << fmt.val(y) << "," << fmt.val(z);
}

enum BBoxCheck { INSIDE_BBOX, JUST_OUTSIDE_BBOX, WAY_OUTSIDE_BBOX };

struct NodeInfo {
    NodeID node;
    BBoxCheck bbcheck;
    NodeInfo(float scale, float x, float y, float z) {
        float sx = (x * scale);
        float sy = (y * scale);
        float sz = (z * scale);
        float dist = std::max({std::abs(sx), std::abs(sy), std::abs(sz)});
        if (dist >= 32768.0f) {
            bbcheck = (dist > (32768.0f + 8192.0f)) ? WAY_OUTSIDE_BBOX : JUST_OUTSIDE_BBOX;
            float clampx = std::min(k_hi, std::max(k_lo, sx + 32768.0f));
            float clampy = std::min(k_hi, std::max(k_lo, sy + 32768.0f));
            float clampz = std::min(k_hi, std::max(k_lo, sz + 32768.0f));
            node = node_lxyz(0, clampx, clampy, clampz);
        } else {
            node = node_lxyz(0, sx + 32768.0f, sy + 32768.0f, sz + 32768.0f);
            bbcheck = INSIDE_BBOX;
        }
    }
};

template <int LEVEL>
struct TreeLevel {
    constexpr static int child() { return LEVEL - 1; }
};

template <>
struct TreeLevel<0> {
    constexpr static int child() { return 0; }
};

// Class PlaneTree.  Everything here is 'public', but this class
// is only visible inside this one C++ file.
class PlaneTree : public eng::opnew {
public:
    void set_radius(float r);
    
    using NodeID = uint64_t;
    using ChildBits = uint64_t;
    using IdVector = util::IdVector;

    // The PlaneMap that this tree is a part of.
    PlaneMap *planemap_;

    // The name of this plane.
    eng::string plane_;

    // Internal nodes in the tree just have bits indicating
    // which children exist.  
    eng::bytell_hash_map<NodeID, ChildBits> internal_nodes_;

    // Leaf nodes in the tree contain a doubly-linked
    // intrusive ring.
    eng::bytell_hash_map<NodeID, PlaneItem *> leaf_nodes_;

    // The radius of the bounding box.
    double radius_;

    // A conversion factor to convert float coordinates to
    // integral coordinates.  Equal to 32k / radius.
    float scale_;
    
    // total number of items in the planetree.
    int total_count_;
    
    // total number of outliers in the planetree. Outliers are
    // classified as just outside or way outside the bbox.
    int just_outside_bbox_;
    int way_outside_bbox_;

public:
    // The following state is initialized whenever we do a scan.
    // It is only relevant during the scan.
    const PlaneScan *scan_config_;
    IdVector *scan_result_;
    int scan_bbxlo_[9], scan_bbxhi_[9];
    int scan_bbylo_[9], scan_bbyhi_[9];
    int scan_bbzlo_[9], scan_bbzhi_[9];

public:
    ChildBits get_internal_node(NodeID id) const {
        auto iter = internal_nodes_.find(id);
        if (iter == internal_nodes_.end()) return 0;
        return iter->second;
    }

    PlaneItem *get_leaf_node(NodeID id) const {
        auto iter = leaf_nodes_.find(id);
        if (iter == leaf_nodes_.end()) return 0;
        return iter->second;
    }
    
    // Untrack all planeitems.  This is for unit testing and for destructors. We
    // don't use PlaneItem::untrack, because that would create problems with
    // removing items from a list while iterating over that list.
    void untrack_all() {
        for (auto &l : leaf_nodes_) {
            PlaneItem *first = l.second;
            PlaneItem *pi = first;
            assert(pi != nullptr);
            while (true) {
                PlaneItem *next = pi->next_;
                pi->tree_= nullptr;
                pi = next;
                if (pi == first) break;
            }
        }
        leaf_nodes_.clear();
        internal_nodes_.clear();
    }

    // This just sets the radius.
    // It verifies that the tree is empty first.
    void set_radius_of_empty_tree(float r) {
        assert(total_count_ == 0);
        radius_ = r;
        scale_ = 32768.0 / r;
    }

    // Get a PlaneTree by plane name.
    static PlaneTree *get(PlaneMap *pmap, const eng::string &plane) {
        std::unique_ptr<PlaneTree> &result = pmap->planes_[plane];
        if (result == nullptr) {
            result.reset(new PlaneTree(pmap, plane));
        }
        return result.get();
    }

    // Remove an item from the specified leaf node.
    // Returns true if we removed the last item from the leaf.
    bool remove_planeitem_from_leaf(NodeID node, PlaneItem *item) {
        if (item->next_ == item) {
            leaf_nodes_.erase(node);
            // Note: these next two assignments are only needed for sanity checking.
            item->next_ = nullptr;
            item->prev_ = nullptr;
            return true;
        } else {
            item->prev_->next_ = item->next_;
            item->next_->prev_ = item->prev_;
            // Note: these next two assignments are only needed for sanity checking.
            item->next_ = nullptr;
            item->prev_ = nullptr;
            return false;
        }
    }

    // Insert an item into the specified leaf node.
    // Returns true if we inserted the first item into the leaf.
    bool insert_planeitem_into_leaf(NodeID node, PlaneItem *item) {
        PlaneItem *&newcell = leaf_nodes_[node];
        if (newcell == nullptr) {
            newcell = item;
            item->next_ = item;
            item->prev_ = item;
            return true;
        } else {
            PlaneItem *next = newcell;
            PlaneItem *prev = newcell->prev_;
            item->next_ = next;
            item->prev_ = prev;
            prev->next_ = item;
            next->prev_ = item;
            return false;
        }
    }

    // Update the parent to reflect the fact that the child was added.
    // This will propagage all the way up the tree.
    void insert_child_into_childbits(NodeID child) {
        uint8_t childlevel = node_get_level(child);
        uint8_t parentlevel = childlevel + 1;
        NodeID parent = node_parent(child);
        ChildBits &inode = internal_nodes_[parent];
        bool waszero = (inode == 0);
        inode |= node_childbit(child);
        if (waszero) {
            if (parentlevel < 8) insert_child_into_childbits(parent);
        }
    }

    // Update the parent to reflect the fact that the child was removed.
    // This will propagage all the way up the tree.
    void remove_child_from_childbits(NodeID child) {
        uint8_t childlevel = node_get_level(child);
        uint8_t parentlevel = childlevel + 1;
        NodeID parent = node_parent(child);
        auto iter = internal_nodes_.find(parent);
        assert(iter != internal_nodes_.end());
        iter->second &= (~node_childbit(child));
        if (iter->second == 0) {
            internal_nodes_.erase(iter);
            if (parentlevel < 8) remove_child_from_childbits(parent);
        }
    }

    // Update the counters to reflect the removal of one item from the tree.
    void decrement_planeitem_counters(BBoxCheck bbcheck) {
        total_count_ -= 1;
        if (bbcheck == JUST_OUTSIDE_BBOX) just_outside_bbox_ -= 1;
        if (bbcheck == WAY_OUTSIDE_BBOX) way_outside_bbox_ -= 1;
    }

    // Update the counters to reflect the insertion of one item into the tree.
    void increment_planeitem_counters(BBoxCheck bbcheck) {
        total_count_ += 1;
        if (bbcheck == JUST_OUTSIDE_BBOX) just_outside_bbox_ += 1;
        if (bbcheck == WAY_OUTSIDE_BBOX) way_outside_bbox_ += 1;
    }

    // Remove a planeitem from whatever tree it is in, preserving
    // all invariants.  The planeitem ends up being an untracked PlaneItem.
    static void remove_planeitem(PlaneItem *item) {
        PlaneTree *tree = item->tree_;
        assert(tree != nullptr);
        NodeInfo info(tree->scale_, item->x_, item->y_, item->z_);
        tree->decrement_planeitem_counters(info.bbcheck);
        if (tree->remove_planeitem_from_leaf(info.node, item)) {
            tree->remove_child_from_childbits(info.node);
        }
        item->tree_ = nullptr;
    }

    // Insert a planeitem into whatever tree is specified, preserving
    // all invariants.  The planeitem must be an untracked PlaneItem.
    void insert_planeitem(PlaneItem *item) {
        PlaneTree *tree = this;
        assert(item->tree_ == nullptr);
        NodeInfo info(tree->scale_, item->x_, item->y_, item->z_);
        tree->increment_planeitem_counters(info.bbcheck);
        if (tree->insert_planeitem_into_leaf(info.node, item)) {
            tree->insert_child_into_childbits(info.node);
        }
        item->tree_ = tree;
        item->plane_ = tree->plane_;
    }

    void print_indented_internal_node(NodeID node, ChildBits cb, std::ostream *os) {
        int level = node_get_level(node);
        int indent = 8 - level;
        (*os) << "|";
        for (int i = 0; i < indent; i++) (*os) << " ";
        print_node_id(node, os);
        if ((cb == 0) && (level != 8)) {
            (*os) << " (invalid empty node)";
        }
    }

    void print_indented_leaf_node(NodeID node, PlaneItem *first, std::ostream *os) {
        (*os) << "|        ";
        print_node_id(node, os);
        (*os) << " ";
        util::IdVector ids;
        collect_planeitem_ids(first, &ids);
        std::sort(ids.begin(), ids.end());
        util::print_id_vector(ids, os);
    }

    void print_tree_r(NodeID node, std::ostream *os) {
        int level = node_get_level(node);
        if (level == 0) {
            print_indented_leaf_node(node, get_leaf_node(node), os);
        } else {
            ChildBits cb = get_internal_node(node);
            if ((level & 1) == 0) {
                print_indented_internal_node(node, cb, os);
            }
            for (int i = 0; i < 64; i++) {
                if (cb & child_bit(i)) {
                    NodeID child = node_nthchild(node, i);
                    assert(node_childindex(child) == i);
                    print_tree_r(child, os);
                }
            }
        }
    }

    void calculate_search_bboxes(const PlaneScan &scan) {
        // Convert the scan's bounding box to integral coordinates.
        NodeInfo bblo(scale_, scan.lo_.x, scan.lo_.y, scan.lo_.z);
        NodeInfo bbhi(scale_, scan.hi_.x, scan.hi_.y, scan.hi_.z);
        
        // Calculate the bounding box at each level of the tree.
        NodeID ibblo = bblo.node;
        NodeID ibbhi = bbhi.node;
        for (int i = 0; i <= 8; i++) {
            scan_bbxlo_[i] = node_get_x(ibblo);
            scan_bbylo_[i] = node_get_y(ibblo);
            scan_bbzlo_[i] = node_get_z(ibblo);
            scan_bbxhi_[i] = node_get_x(ibbhi);
            scan_bbyhi_[i] = node_get_y(ibbhi);
            scan_bbzhi_[i] = node_get_z(ibbhi);
            ibblo = node_parent(ibblo);
            ibbhi = node_parent(ibbhi);
        }
    }

    // Calculate the size of the search bboxes.
    int64_t scan_xsize(int i) const { return 1 + scan_bbxhi_[i] - scan_bbxlo_[i]; }
    int64_t scan_ysize(int i) const { return 1 + scan_bbyhi_[i] - scan_bbylo_[i]; }
    int64_t scan_zsize(int i) const { return 1 + scan_bbzhi_[i] - scan_bbzlo_[i]; }

    eng::string search_bboxes_debug_string(const PlaneScan &scan) {
        calculate_search_bboxes(scan);
        eng::ostringstream oss;
        for (int i = 8; i >= 0; i -= 2) {
            auto fmt = util::hex.width((8 - i) / 2);
            oss << "|Level " << i << " ";            
            oss << fmt.val(scan_bbxlo_[i]) << ","
                << fmt.val(scan_bbylo_[i]) << ","
                << fmt.val(scan_bbzlo_[i]) << " - "
                << fmt.val(scan_bbxhi_[i]) << ","
                << fmt.val(scan_bbyhi_[i]) << ","
                << fmt.val(scan_bbzhi_[i]);
        }
        return oss.str();
    }

    static inline void scan_push_id(int64_t id, int64_t special, IdVector *result) {
        if (id != special) {
            result->push_back(id);
        }
    }

    static void scan_planeitem(PlaneItem *pi, const PlaneScan &scan, IdVector *result) {
        switch (scan.shape_) {
            case PlaneScan::BOX: {
                if ((pi->x() >= scan.lo_.x) && (pi->x() <= scan.hi_.x) &&
                    (pi->y() >= scan.lo_.y) && (pi->y() <= scan.hi_.y) &&
                    (pi->z() >= scan.lo_.z) && (pi->z() <= scan.hi_.z)) {
                    scan_push_id(pi->id(), scan.special_, result);
                }
                break;
            }
            case PlaneScan::SPHERE: {
                float dx = (pi->x() - scan.center_.x) * scan.invradius_.x;
                float dy = (pi->y() - scan.center_.y) * scan.invradius_.y;
                float dz = (pi->z() - scan.center_.z) * scan.invradius_.z;
                if (dx*dx + dy*dy + dz*dz <= 1.0) {
                    scan_push_id(pi->id(), scan.special_, result);
                }
                break;
            }
            case PlaneScan::CYLINDER: {
                if ((pi->z() >= scan.lo_.z) && (pi->z() <= scan.hi_.z)) {
                    float dx = (pi->x() - scan.center_.x) * scan.invradius_.x;
                    float dy = (pi->y() - scan.center_.y) * scan.invradius_.y;
                    if (dx*dx + dy*dy <= 1.0) {
                        scan_push_id(pi->id(), scan.special_, result);
                    }
                }
                break;
            }
        }
    }

    // Recursive part of the planetree scan.
    // Note: template expansion terminates because
    // TreeLevel<0>::child returns zero again.
    template <int LEVEL>
    void scan_node(NodeID node, std::ostream *debug) {
        if (LEVEL == 0) {
            auto iter = leaf_nodes_.find(node);
            assert (iter != leaf_nodes_.end());
            PlaneItem *first = iter->second;
            assert(first != nullptr);
            if (debug != nullptr) {
                print_indented_leaf_node(node, first, debug);
            }
            PlaneItem *pi = first;
            while (true) {
                PlaneItem *next = pi->next_;
                scan_planeitem(pi, *scan_config_, scan_result_);
                if (next == first) break;
                pi = next;
            }
        } else {
            constexpr int CHILDLEVEL = TreeLevel<LEVEL>::child();
            NodeID firstchild = node_nthchild(node, 0);

            int xlo = std::max(0, scan_bbxlo_[CHILDLEVEL] - int(node_get_x(firstchild)));
            int ylo = std::max(0, scan_bbylo_[CHILDLEVEL] - int(node_get_y(firstchild)));
            int zlo = std::max(0, scan_bbzlo_[CHILDLEVEL] - int(node_get_z(firstchild)));
            int xhi = std::min(3, scan_bbxhi_[CHILDLEVEL] - int(node_get_x(firstchild)));
            int yhi = std::min(3, scan_bbyhi_[CHILDLEVEL] - int(node_get_y(firstchild)));
            int zhi = std::min(3, scan_bbzhi_[CHILDLEVEL] - int(node_get_z(firstchild)));

            auto iter = internal_nodes_.find(node);
            assert (iter != internal_nodes_.end());
            ChildBits cb = iter->second;
            assert (cb != 0);
            if (debug != nullptr) {
                print_indented_internal_node(node, cb, debug);
            }

            for (int x = xlo; x <= xhi; x++) {
                for (int y = ylo; y <= yhi; y++) {
                    for (int z = zlo; z <= zhi; z++) {
                        if (cb & child_bit(x, y, z)) {
                            NodeID child = node_child(node, x, y, z);
                            scan_node<CHILDLEVEL>(child, debug);
                        }
                    }
                }
            }
        }
    }

    // Scan a planetree.
    void scan(const PlaneScan &sc, IdVector *result, std::ostream *debug) {
        scan_config_ = &sc;
        scan_result_ = result;
        calculate_search_bboxes(sc);

        // We must only call 'scan_node' on nodes that actually exist.
        // So we check if the tree is empty, and if so, we don't scan the
        // root node.
        if (!internal_nodes_.empty()) {
            NodeID root = node_lxyz(8, 0, 0, 0);
            scan_node<8>(root, debug);
        }
    }

    eng::string scan_steps_debug_string(const PlaneScan &sc) {
        eng::ostringstream oss;
        IdVector result;
        scan(sc, &result, &oss);
        oss << "|Result: ";
        std::sort(result.begin(), result.end());
        util::print_id_vector(result, &oss);
        return oss.str();
    }

    void collect_planeitem_ids(PlaneItem *first, IdVector *ids) {
        if (first != nullptr) {
            PlaneItem *pi = first;
            while (true) {
                PlaneItem *next = pi->next_;
                ids->push_back(pi->id());
                if (next == first) break;
                pi = next;
            }
        }
    }

    eng::string tree_debug_string() {
        eng::ostringstream oss;
        print_tree_r(node_lxyz(8,0,0,0), &oss);
        return oss.str();
    }

    eng::string outliers_debug_string() {
        eng::ostringstream oss;
        oss << "total:" << total_count_ << " justout:" << just_outside_bbox_ << " wayout:" << way_outside_bbox_;
        return oss.str();
    }   

    // Construct a PlaneTree.  
    PlaneTree(PlaneMap *pmap, const eng::string &plane) {
        planemap_ = pmap;
        plane_ = plane;
        total_count_ = 0;
        just_outside_bbox_ = 0;
        way_outside_bbox_ = 0;
        set_radius_of_empty_tree(pmap->default_radius_);
    }

    // Destructor: the PlaneTree doesn't own the PlaneItems, so it doesn't
    // delete them, but it needs to untrack all the PlaneItems. 
    ~PlaneTree() {
        untrack_all();
    }
};

void PlaneItem::track(PlaneMap *pmap) {
    // If we're already in a PlaneMap, and it's not the
    // PlaneMap we want to be in, remove from the old PlaneMap.
    if ((tree_ != nullptr) && (tree_->planemap_ != pmap))  {
        PlaneTree::remove_planeitem(this);
    }

    // If we're supposed to be in a PlaneMap, and we're not
    // already in the PlaneMap, insert it.
    if ((tree_ == nullptr) && (pmap != nullptr)) {
        PlaneTree::get(pmap, plane_)->insert_planeitem(this);
    }
}

void PlaneItem::set_pos(const eng::string &plane, float x, float y, float z) {
    // If we're not in a PlaneMap, nothing to do but set the variables.
    if (tree_ == nullptr) {
        plane_ = plane;
        x_ = x;
        y_ = y;
        z_ = z;
        return;
    }

    // When moving within a plane (not warping), use set_xyz, which is faster.
    if (plane_ == plane) {
        set_xyz(x, y, z);
        return;
    }

    // We're warping from one plane to another.  That means we're removing
    // ourself from one planetree and inserting ourself into a different one.
    PlaneTree *newtree = PlaneTree::get(tree_->planemap_, plane);
    PlaneTree::remove_planeitem(this);
    x_ = x;
    y_ = y;
    z_ = z;
    newtree->insert_planeitem(this);
}

void PlaneItem::set_xyz(float x, float y, float z) {
    // If we're not in a PlaneMap, nothing to do but set the variables.
    if (tree_ == nullptr) {
        x_ = x;
        y_ = y;
        z_ = z;
        return;
    }

    // We could implement this function using 'PlaneTree::remove_planeitem'
    // and 'PlaneTree::insert_planeitem', which would be less code, but it
    // would also be slower.
    NodeInfo old_cell(tree_->scale_, x_, y_, z_);
    NodeInfo new_cell(tree_->scale_, x, y, z);

    // Update the variables.
    x_ = x;
    y_ = y;
    z_ = z;

    // Update the outliers counters (unlikely).
    if (old_cell.bbcheck != new_cell.bbcheck) {
        tree_->decrement_planeitem_counters(old_cell.bbcheck);
        tree_->increment_planeitem_counters(new_cell.bbcheck);
    }

    // If we have changed cells, update the tree.
    // We have to remove the child from the old leaf before inserting
    // it into the new leaf, because the 'next' and 'prev' pointers are
    // intrusive and we need them to be unused to do the insert.
    // However, inserting the child into the childbits first is faster
    // and poses no problems.
    if (new_cell.node != old_cell.node) {
        bool leaf_removed = tree_->remove_planeitem_from_leaf(old_cell.node, this);
        bool leaf_created = tree_->insert_planeitem_into_leaf(new_cell.node, this);
        if (leaf_created) tree_->insert_child_into_childbits(new_cell.node);
        if (leaf_removed) tree_->remove_child_from_childbits(old_cell.node);
    }
}

PlaneItem::PlaneItem() {
    id_ = 0;
    tree_ = nullptr;
    next_ = nullptr;
    prev_ = nullptr;
    x_ = y_ = z_ = 0.0;
}

PlaneItem::~PlaneItem() {
    untrack();
}

PlaneMap::PlaneMap() : default_radius_(32768.0) {}
PlaneMap::~PlaneMap() {}


IdVector PlaneMap::scan(const PlaneScan &sc) const {
    IdVector result;
    int startpos = 0;
    if (sc.special_ != 0) {
        if (!sc.omit_special_) {
            result.push_back(sc.special_);
            startpos = 1;
        }
    }

    if (sc.omit_nowhere_ && (sc.plane_ == "nowhere")) {
        return result;
    }

    auto piter = planes_.find(sc.plane_);
    if (piter != planes_.end()) {
        const std::unique_ptr<PlaneTree> &tree = piter->second;
        tree->scan(sc, &result, nullptr);
    }

    if (sc.sorted_) {
        std::sort(result.begin() + startpos, result.end());
    }
    return result;
}

eng::string PlaneMap::tree_debug_string(const eng::string &plane)  {
    return PlaneTree::get(this, plane)->tree_debug_string();
}

eng::string PlaneMap::outliers_debug_string(const eng::string &plane)  {
    return PlaneTree::get(this, plane)->outliers_debug_string();
}

eng::string PlaneMap::search_bboxes_debug_string(const eng::string &plane, const PlaneScan &scan)  {
    return PlaneTree::get(this, plane)->search_bboxes_debug_string(scan);
}

eng::string PlaneMap::scan_steps_debug_string(const eng::string &plane, const PlaneScan &scan) {
    return PlaneTree::get(this, plane)->scan_steps_debug_string(scan);
}

void PlaneMap::untrack_all() {
    for (const auto &pair : planes_) {
        pair.second->untrack_all();
    }
}

static eng::string tdb(PlaneMap &pm) {
    return pm.tree_debug_string("p");
}

static eng::string odb(PlaneMap &pm) {
    return pm.outliers_debug_string("p");
}

// The default radius is set such that float coordinates map directly to
// integer coordinates, with an offset of 0x8000.  This makes unit testing
// a lot easier.
//
LuaDefine(unittests_planemap, "", "some unit tests") {
    PlaneMap pm;
    PlaneItem pi123, pi456;
    PlaneScan scan;
    pi123.set_id(123);
    pi456.set_id(456);

    // Test that PlaneItems can be manipulated when they're not
    // yet tracking a PlaneMap.
    pi123.set_pos("p", 0x38, 0x16, 0x87);
    LuaAssert(L, pi123.plane() == "p");
    LuaAssert(L, pi123.x() == 0x38);
    LuaAssert(L, pi123.y() == 0x16);
    LuaAssert(L, pi123.z() == 0x87);
    
    // TESTS OF TREE MANIPULATION FOLLOW.
    
    // Test track.
    pi123.set_pos("p", 0x38, 0x16, 0x87);
    pi123.track(&pm);
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,8,8"
        "|    L4:80,80,80"
        "|      L2:803,801,808"
        "|        L0:8038,8016,8087 123");

    // Test untrack.
    pi123.untrack();
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root");
    
    // Track two items at a time, not in the same cell.
    pi456.set_pos("p", 0x12, 0x17, 0xAC);
    pi123.track(&pm);
    pi456.track(&pm);
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,8,8"
        "|    L4:80,80,80"
        "|      L2:801,801,80a"
        "|        L0:8012,8017,80ac 456"
        "|      L2:803,801,808"
        "|        L0:8038,8016,8087 123");
    
    // Move one of the items into the same cell as the other.
    pi456.set_xyz(0x38, 0x16, 0x87);
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,8,8"
        "|    L4:80,80,80"
        "|      L2:803,801,808"
        "|        L0:8038,8016,8087 123,456");

    // Move item 456 back out of the cell.
    pi456.set_xyz(0x27, 0x11, 0x31);
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,8,8"
        "|    L4:80,80,80"
        "|      L2:802,801,803"
        "|        L0:8027,8011,8031 456"
        "|      L2:803,801,808"
        "|        L0:8038,8016,8087 123");
    
    // Move item 123 to follow 456.
    pi123.set_xyz(0x27, 0x11, 0x31);
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,8,8"
        "|    L4:80,80,80"
        "|      L2:802,801,803"
        "|        L0:8027,8011,8031 123,456");
    
    // TESTS OF OUTLIER CLAMPING FOLLOW.

    // Move item 456 close to, but not quite on the positive edge.
    pi123.untrack();
    pi456.set_xyz(0x23, 0x7FFE, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:0 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,f,8"
        "|    L4:80,ff,80"
        "|      L2:802,fff,802"
        "|        L0:8023,fffe,8027 456");

    // Move item 456 so that it's on the positive edge, but not an outlier.
    pi456.set_xyz(0x23, 0x7FFF, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:0 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,f,8"
        "|    L4:80,ff,80"
        "|      L2:802,fff,802"
        "|        L0:8023,ffff,8027 456");

    // Move item 456 so that it's even closer to the positive edge, but not an outlier.
    pi456.set_xyz(0x23, 0x7FFF + 0.99, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:0 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,f,8"
        "|    L4:80,ff,80"
        "|      L2:802,fff,802"
        "|        L0:8023,ffff,8027 456");

    // Move item 456 so that it's just barely a positive outlier.
    pi456.set_xyz(0x23, 0x8000, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:1 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,f,8"
        "|    L4:80,ff,80"
        "|      L2:802,fff,802"
        "|        L0:8023,ffff,8027 456");

    // Move item 456 so that it's considerably past the positive edge.
    pi456.set_xyz(0x23, 0x8048, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:1 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,f,8"
        "|    L4:80,ff,80"
        "|      L2:802,fff,802"
        "|        L0:8023,ffff,8027 456");

    // Move item 456 so that it's way past the positive edge.
    pi456.set_xyz(0x23, 0x83748, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:0 wayout:1");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,f,8"
        "|    L4:80,ff,80"
        "|      L2:802,fff,802"
        "|        L0:8023,ffff,8027 456");

    // Move item 456 close to, but not quite on the negative edge.
    pi456.set_xyz(0x23, -0x7fff, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:0 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,0,8"
        "|    L4:80,00,80"
        "|      L2:802,000,802"
        "|        L0:8023,0001,8027 456");

    // Move item 456 so that it's on the negative edge, but not an outlier.
    pi456.set_xyz(0x23, -0x7fff - 0.5, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:0 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,0,8"
        "|    L4:80,00,80"
        "|      L2:802,000,802"
        "|        L0:8023,0000,8027 456");

    // Move item 456 so that it's just barely a negative outlier.
    pi456.set_xyz(0x23, -0x8000, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:1 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,0,8"
        "|    L4:80,00,80"
        "|      L2:802,000,802"
        "|        L0:8023,0000,8027 456");

    // Move item 456 so that it's significantly past the negative edge.
    pi456.set_xyz(0x23, -0x8048, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:1 wayout:0");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,0,8"
        "|    L4:80,00,80"
        "|      L2:802,000,802"
        "|        L0:8023,0000,8027 456");

    // Move item 456 so that it's way past the negative edge.
    pi456.set_xyz(0x23, -0x83048, 0x27);
    LuaAssertStrEq(L, odb(pm), "total:1 justout:0 wayout:1");
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:8,0,8"
        "|    L4:80,00,80"
        "|      L2:802,000,802"
        "|        L0:8023,0000,8027 456");

    // Test the calculation of search bboxes.
    // The two corners are deliberately not in low-high order.
    scan.clear();
    scan.set_bbox_given_two_corners(util::XYZ(0x23, 0x97, 0x103),
                                    util::XYZ(0x309, 0x412, 0x27));
    LuaAssertStrEq(L, pm.search_bboxes_debug_string("p", scan),
        "|Level 8 0,0,0 - 0,0,0"
        "|Level 6 8,8,8 - 8,8,8"
        "|Level 4 80,80,80 - 83,84,81"
        "|Level 2 802,809,802 - 830,841,810"
        "|Level 0 8023,8097,8027 - 8309,8412,8103");

    // TESTS OF SCANNING
    
    // Store a single object in the map to scan.
    pm.untrack_all();
    pi123.set_pos("p", 0x12, 0x34, 0x45);
    pi123.track(&pm);

    // Set up a scan with a radius of zero, centered
    // right on the one object.  Check the bboxes to make
    // sure they only include the one cell.
    scan.clear();
    scan.set_bbox_given_center_radius(util::XYZ(0x12, 0x34, 0x45), 0.0);
    LuaAssertStrEq(L, pm.search_bboxes_debug_string("p", scan),
        "|Level 8 0,0,0 - 0,0,0"
        "|Level 6 8,8,8 - 8,8,8"
        "|Level 4 80,80,80 - 80,80,80"
        "|Level 2 801,803,804 - 801,803,804"
        "|Level 0 8012,8034,8045 - 8012,8034,8045");

    // Run the scan with radius zero.  It should find the one object.
    LuaAssertStrEq(L, pm.scan_steps_debug_string("p", scan),
        "|L8:root"
        "| L7:2,2,2"
        "|  L6:8,8,8"
        "|   L5:20,20,20"
        "|    L4:80,80,80"
        "|     L3:200,200,201"
        "|      L2:801,803,804"
        "|       L1:2004,200d,2011"
        "|        L0:8012,8034,8045 123"
        "|Result: 123");

    // Bump the scan over a half-unit.  It should have the same
    // bboxes as before, since a half-unit isn't enough to shift
    // from one cell to the next.
    scan.clear();
    scan.set_bbox_given_center_radius(util::XYZ(0x12 + 0.5, 0x34, 0x45), 0.0);
    LuaAssertStrEq(L, pm.search_bboxes_debug_string("p", scan),
        "|Level 8 0,0,0 - 0,0,0"
        "|Level 6 8,8,8 - 8,8,8"
        "|Level 4 80,80,80 - 80,80,80"
        "|Level 2 801,803,804 - 801,803,804"
        "|Level 0 8012,8034,8045 - 8012,8034,8045");

    // Run the scan with radius zero, but one half-step away
    // from the object.  It should encounter the cell containing the
    // one object, but the object should get removed in the final
    // filtering step.
    LuaAssertStrEq(L, pm.scan_steps_debug_string("p", scan),
        "|L8:root"
        "| L7:2,2,2"
        "|  L6:8,8,8"
        "|   L5:20,20,20"
        "|    L4:80,80,80"
        "|     L3:200,200,201"
        "|      L2:801,803,804"
        "|       L1:2004,200d,2011"
        "|        L0:8012,8034,8045 123"
        "|Result: ");

    // Next, expand the scan radius to huge.  Examine the bboxes
    // to make sure they cover the entire PlaneTree.
    scan.clear();
    scan.set_bbox_given_center_radius(util::XYZ(0x12, 0x34, 0x45), 100000.0);
    LuaAssertStrEq(L, pm.search_bboxes_debug_string("p", scan),
        "|Level 8 0,0,0 - 0,0,0"
        "|Level 6 0,0,0 - f,f,f"
        "|Level 4 00,00,00 - ff,ff,ff"
        "|Level 2 000,000,000 - fff,fff,fff"
        "|Level 0 0000,0000,0000 - ffff,ffff,ffff");
    
    // Walk the tree using the expansive search bboxes.  It should
    // find the one object, and it should still only traverse the same
    // cells, because those are the only cells that exist.
    LuaAssertStrEq(L, pm.scan_steps_debug_string("p", scan),
        "|L8:root"
        "| L7:2,2,2"
        "|  L6:8,8,8"
        "|   L5:20,20,20"
        "|    L4:80,80,80"
        "|     L3:200,200,201"
        "|      L2:801,803,804"
        "|       L1:2004,200d,2011"
        "|        L0:8012,8034,8045 123"
        "|Result: 123");

    // Add another object to the tree.  Then, scan again
    // using the expansive search.  It should find both objects.
    pi456.set_pos("p", 0x14, 0x35, 0x30);
    pi456.track(&pm);
    LuaAssertStrEq(L, pm.scan_steps_debug_string("p", scan),
        "|L8:root"
        "| L7:2,2,2"
        "|  L6:8,8,8"
        "|   L5:20,20,20"
        "|    L4:80,80,80"
        "|     L3:200,200,200"
        "|      L2:801,803,803"
        "|       L1:2005,200d,200c"
        "|        L0:8014,8035,8030 456"
        "|     L3:200,200,201"
        "|      L2:801,803,804"
        "|       L1:2004,200d,2011"
        "|        L0:8012,8034,8045 123"
        "|Result: 123,456");

    // Set up a tree with a single object that's outside
    // the tree's bounding box (an outlier).  Print the tree
    // to verify that the object ended up on the edge.
    pm.untrack_all();
    pi123.set_pos("p", 0x100000, 0x16, 0x23);
    pi123.track(&pm);
    LuaAssertStrEq(L, tdb(pm),
        "|L8:root"
        "|  L6:f,8,8"
        "|    L4:ff,80,80"
        "|      L2:fff,801,802"
        "|        L0:ffff,8016,8023 123");
    
    // Now set up a scan around the target outlier.  Print out
    // the scan bboxes.  It should be scanning the edge cell that
    // contains the outlier.  It also contains a few other cells
    // because the radius is nonzero.
    scan.clear();
    scan.set_bbox_given_center_radius(util::XYZ(0x100000, 0x16, 0x23), 0.2);
    LuaAssertStrEq(L, pm.search_bboxes_debug_string("p", scan),
        "|Level 8 0,0,0 - 0,0,0"
        "|Level 6 f,8,8 - f,8,8"
        "|Level 4 ff,80,80 - ff,80,80"
        "|Level 2 fff,801,802 - fff,801,802"
        "|Level 0 ffff,8015,8022 - ffff,8016,8023");

    // Confirm that the scan finds the outlier.
    LuaAssertStrEq(L, pm.scan_steps_debug_string("p", scan),
        "|L8:root"
        "| L7:3,2,2"
        "|  L6:f,8,8"
        "|   L5:3f,20,20"
        "|    L4:ff,80,80"
        "|     L3:3ff,200,200"
        "|      L2:fff,801,802"
        "|       L1:3fff,2005,2008"
        "|        L0:ffff,8016,8023 123"
        "|Result: 123");

    return 0;
}