Add token literals to the lua parser
This commit is contained in:
@@ -248,3 +248,18 @@ GC Finalizers and weak tables both introduce nondeterminism into Lua execution.
|
||||
Update 1: I'm using GC finalizers in some cases to clean up userdata objects. I think it's safe as long as the only thing the finalizer does is free memory. (NOTE: WHERE?)
|
||||
|
||||
Update 2: I don't remember using userdata objects at all. I am not sure that Update 1 is the truth any more.
|
||||
|
||||
## Token Literal Syntax Patch
|
||||
|
||||
Tokens are lightuserdata values encoding short alphanumeric strings as base37 numbers (see `Tokens-A-New-Lua-Type.md`). Previously, tokens could only be created in C++ and inserted into the Lua environment via `LuaTokenConstant`. This patch adds a literal syntax to the Lua parser so that tokens can be written directly in Lua source code using the `@` prefix:
|
||||
|
||||
```lua
|
||||
local x = @null
|
||||
local y = @found
|
||||
```
|
||||
|
||||
The lexer (llex.c) recognizes `@` followed by one or more alphanumeric characters (a-z, 0-9, case insensitive, max 12 characters). It encodes the string as a base37 number using the same encoding as `LuaToken::parse()` in luastack.hpp and produces a `TK_TOKEN` token. The parser (lparser.c) handles `TK_TOKEN` in `simpleexp()` by storing it as a lightuserdata constant in the function's constant table via `luaK_lightuserdataK()` in lcode.c.
|
||||
|
||||
Underscores are not valid in token literals. Writing `@foo_bar` produces a lexer error rather than silently splitting into token `@foo` and identifier `_bar`.
|
||||
|
||||
This patch is live and functioning.
|
||||
|
||||
Reference in New Issue
Block a user