integration/Docs/Object-Oriented-Lua.md

How to do Methods and Inheritance in Standard Lua

Standard Lua has the ability to do classes and inheritance. Let me explain how.

Lua has a colon operator, for method lookup. It looks like this:

  local result = v1:dotproduct(v2)

By default, this looks for the method "dotproduct" inside the table v1. But you don't really want to put a whole bunch of function objects (like dotproduct) into v1, and into every other vector. Instead, you want to put those function objects in a separate table, the class table:

  vector = {}
  function vector.dotproduct(v1, v2) ... end
  function vector.crossproduct(v1, v2) ... end

Fortunately, Lua can do that. Lua has a metamethod, INDEX, that basically says: "if you don't find what you're looking for in this table, look in that table instead." We can use the INDEX metamethod to tell lua: if you're looking a method "dotproduct" in v1, and it's not there, look in class vector instead.

So to do that, you only need two steps. First, we're going to decide that class vector isn't just hold the methods for vectors. We're also going to use it as the metatable for all vectors:

  v1 = {x=100, y=100, z=100}
  v2 = {x=200, y=200, z=200}
  setmetatable(v1, vector)
  setmetatable(v2, vector)

Now that vector is the metatable for all vectors, you can put metamethods into class vector and they will affect all vectors. We're going to put an __INDEX metamethod in there:

  vector = {}
  vector.__INDEX = vector
  function vector.dotproduct(v1, v2) ... end
  function vector.crossproduct(v1, v2) ... end

That says: if you're doing a lookup on a vector, and the lookup fails, look in class vector instead.

OK, in case you didn't quite follow, let me walk you through step by step what happens when you evaluate the expression v1:dotproduct(v2).

First, the colon operator looks for "dotproduct" in v1. It's not going to find it: we're not going to put all those function objects into every single vector.

When the table lookup code fails to find "dotproduct" in v1, it goes to a fallback codepath: it checks whether v1 has a metatable. It does: vector is the metatable for all vectors. So v1 does indeed have a metatable. The table lookup code then checks whether the metatable contains INDEX: again, it does, class vector contains INDEX. Finally, the table lookup checks what INDEX is pointing to: class vector again. So the table lookup code, having failed to find "dotproduct" in v1, looks for it in class vector. The method lookup succeeds.

The little trick "vector.__INDEX = vector" certainly feels arcane, but it works. I like to hide it with a little syntactic sugar:

  def makeclass(name)
    _G[name] = {}
    _G[name].__INDEX = _G[name]
  end

Now I can say this, and it hides all the metatable magic:

  makeclass("vector")
  function vector.dotproduct(v1, v2) ... end
  function vector.crossproduct(v1, v2) ... end

A useful property of this design is that vector is both the metatable for all vectors, and also the class table for all vectors. One consequence of that is that I can put both regular methods and metamethods in there:

  makeclass("vector")
  function vector.__eq(v1, v2) ... end
  function vector.dotproduct(v1, v2) ... end
  function vector.crossproduct(v1, v2) ... end

Let's say you also want inheritance: you want vector to inherit from vectorbase. That's another thing you can accomplish using INDEX again: "if you can't find what you're looking for in vector, look in vectorbase." Here's the code for that:

  makeclass("vectorbase")
  function vectorbase.whatever(...) ... end

  makeclass("vector")
  function vector.dotproduct(v1, v2) ... end
  function vector.crossproduct(v1, v2) ... end

  setmetatable(vector, vectorbase)

That 'setmetatable' directs all failed lookups from vector into vectorbase. I've set up a search path.

That's really all there is to it. Honestly, it's reasonably straightforward.

Why I Don't Love It: High Bug Potential

Let's say you want to create a class, "json". This is for receiving and transmitting json over HTTP:

  json={}
  function json.validate() ... end
  function json.serialize(output) ... end

Now, suppose you receive this json from an HTTP client:

  {
    "jsonrpc" : "2.0",
    "validate" : true
  }

You convert the json into a lua table request, call setclass(request, json), and then you try to validate the request:

  bool legal = request:validate()

This produces an error: "true" is not a callable function. That's because when it looks for the "validate" method, it instead finds the "validate" data that is in the request.

The underlying problem is this: method lookups should only look in the class table, not in the request. Meanwhile, data lookup should be only in the request, not in the class table. But the INDEX metamethod doesn't differentiate between method lookups and data lookups. All lookups are treated the same. So all accesses go to both tables.

Using this design for object-orientation means you're always at risk of data hiding your methods, at unexpected times. It also means you can get false readings on instance variables if the actual instance variable is nil.

Why I Don't Love It: Speed

How much does this cost:

  local result = v1:dotproduct(v2)

The answer is: it has to do three table lookups for just the method lookup:

• v1["dotproduct"]
• metatable["__INDEX"]
• vector["dotproduct"]

That also applies to data lookups. Suppose I do this:

  local param = request.param

That's a simple data lookup. But if the request is of class json, and the request doesn't contain "param", then it does three table lookups, trying to find the data "param":

• request["param"]
• metatable["__INDEX"]
• json["param"]

Looking for param in the class table, aside from being risky, is wasting CPU time. This makes lua, an already slow interpreted language, significantly slower.

A Patch

To solve these problems, we need to establish a rule: method lookups should only go to the class table, and data lookups should only go to the data table.

To make that happen, I introduce a new metamethod, CLASS (with leading underscores). This metamethod changes only the behavior of the method look up operator (colon):

  v1:dotproduct(v2)

The behavior is: if you do a method lookup, then the colon operator looks to see if v1 has a metatable with the CLASS metamethod in it. If so, then it looks for the method in the metatable of v1 instead of looking in v1.

Let me show you how this is intended to be used. We create the class table in pretty much the same way, but instead of putting INDEX in there, we put CLASS:

  vector = {}
  vector.__CLASS = true
  function vector.dotproduct(v1, v2) ... end
  function vector.crossproduct(v1, v2) ... end

We again intend to make vector the metatable for all vectors. So again, when you create a vector, you do this:

  v1 = {x=100, y=100, z=100}
  v2 = {x=200, y=200, z=200}
  setmetatable(v1, vector)
  setmetatable(v2, vector)

So far, it looks almost identical. But when you do v1:dotproduct(v2), the method lookup checks whether v1 has a metatable (it does), and whether that metatable has CLASS in it (it does). So therefore, it doesn't look for dotproduct in v1. It only looks for it in the metatable of v1, which is class vector.

The CLASS metamethod has no effect on data lookups. That incudes the dot operator, and the array lookup operator:

  local param = request.param

  local param = request["param"]

Even if request above has the json class as its metatable, these lookups are just plain lua table lookups, and that's all. Class json isn't involved.

This cleanly separates the two namespaces: data is looked up in the data table, methods are looked up in the methods table.

So that leaves inheritance. We use a slightly different version of the CLASS metamethod:

  vector = {}
  vector.__CLASS = vectorbase
  function vector.dotproduct(v1, v2) ... end
  function vector.crossproduct(v1, v2) ... end

Saying CLASS=true means "this is a class." Saying CLASS=vectorbase means "this is a class, derived from vectorbase". If you do that, then the method look-up operator will look in vector first, then vectorbase, and it will follow the inheritance chain upward.

How does the CLASS metamethod compare, performance-wise, to using the INDEX metamethod to achieve object-orientation? Let's start with data lookups. Using INDEX, this can take up to three table lookups:

  local param = request.param

However, using CLASS instead, this is only ever one table lookup. You might assume that since request has a metatable, that lua would at least have to do a table lookup to see if there's an INDEX metamethod, even if there's not one. However, lua has a clever optimization: lua remembers that INDEX is not present in class json, and it doesn't look a second time. It only ever does the INDEX lookup once, for the entire program.

Method lookups like v1:dotproduct(v2) are also accelerated. Using INDEX, it takes three table lookups to find "dotproduct" in class vector. Using CLASS, however, it usually takes only one table lookup. It skips looking in v1 entirely. And, like INDEX, lua remembers that CLASS=true is present in class vector, and it doesn't look a second time. So it goes straight to looking for "dotproduct" in class vector. It only has to lookup CLASS if the method is not found, and we have to walk the inheritance tree.

Finally, I'd like to say something about readability:

  vector.__CLASS = true

Summary

Metamethod CLASS achieves object-oriented method lookup, but with two advantages over using INDEX:

• Safely separates the two namespaces: data, and methods.

• CLASS is faster than using INDEX.