Embedding Lua 5.4: Stack Discipline, Error Boundaries, and Cleanup
When Lua is embedded in a C or C++ process, the real interface is not the script text. It is the lua_State, virtual stack, and permissions granted by the host. The example uses Lua 5.4.8 and stays small. A dependable embedding layer maintains three invariants: every function documents its stack inputs and outputs, untrusted execution enters through a protected call, and the Lua state plus host resources have one traceable owner.
Begin with explicit ownership
luaL_newstate creates a state and returns null on failure. Once it succeeds, the host is responsible for one eventual lua_close. Several objects must not all believe they own the state, and the state cannot be closed while a Lua C function is still active on its call stack.
#include <stdio.h>
#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>
int run_script(const char *source) {
lua_State *L = luaL_newstate();
if (L == NULL) return 1;
luaL_openlibs(L);
int status = luaL_loadstring(L, source);
if (status == LUA_OK) {
status = lua_pcall(L, 0, 0, 0);
}
if (status != LUA_OK) {
const char *message = lua_tostring(L, -1);
fprintf(stderr, "lua: %s\n", message ? message : "unknown error");
lua_pop(L, 1);
}
lua_close(L);
return status == LUA_OK ? 0 : 1;
}
This version is appropriate only for trusted local scripts. For plugins or configuration, luaL_openlibs grants too much. Open only required libraries and omit file, process, dynamic-loading, and debug capability. A sandbox is a complete threat model, not the removal of one global name.
Write stack effects as contracts
Positive indices address from the bottom of the stack and negative indices from its top. Every helper should state the values required on entry, results pushed on success, and error values left on failure. Saving int top = lua_gettop(L) and checking or restoring the expected height before return catches many leaks early.
Validate inputs with lua_type or appropriate luaL_check* helpers. A C function called from Lua returns the number of values it pushed. When the host calls Lua, it pushes the function followed by arguments, and the lua_pcall argument and result counts must match. Do not retain a stack index across arbitrary pushes, pops, or calls into Lua.
Stack space is finite for an operation. Call lua_checkstack before pushing a variable or potentially large group of values. A failed capacity check is an error path, not permission to continue writing.
Stop errors at a protected boundary
lua_call allows a Lua error to long-jump. Host entry points normally use lua_pcall, which returns a status and leaves the error object on the stack. Loading and execution are distinct: luaL_loadfile or luaL_loadbuffer compiles a chunk, and success still requires a call. Record syntax, runtime, memory, and message-handler errors separately.
An error object is not guaranteed to be a string. Convert it safely and cap log size. A message handler can produce a traceback, but it has its own stack contract and can fail. Do not log secret host paths, keys, or an entire private script merely because an exception occurred.
Define a single conversion point from Lua status to the host’s error type. Cleanup should not depend on parsing message text. If the host uses exceptions, keep them outside C callbacks unless the binding explicitly and safely translates them.
Give host pointers a lifetime protocol
Full userdata can expose a host object, but it is not automatically a safe reference. Decide whether Lua owns the resource or borrows it. Check its metatable on every method. Make __gc and explicit close release at most once. If the host destroys a borrowed object first, invalidate the userdata so later calls return an error instead of dereferencing a dangling pointer.
Lua 5.4 to-be-closed variables help scripts express scoped cleanup, but the host must still handle errors, cancellation, and closing the whole state. A C++ wrapper also needs care around Lua error mechanics and destructor boundaries; do not assume a long jump unwinds every C++ object like an ordinary exception.
Bound time and memory
Untrusted code can loop forever or allocate until the process fails. A custom allocator can meter and refuse allocations. A debug hook can implement an instruction budget or cancellation checkpoint. Stress those paths because callbacks run precisely when resources are constrained. Expose filesystem, network, and system commands only through allowlisted host functions with validated arguments.
Threading needs an equally explicit rule. A Lua state is not a general object for simultaneous access by arbitrary host threads. Give it one execution owner or place a complete synchronization boundary around every interaction, including callbacks.
Test hostile paths
Cover syntax errors, runtime errors, non-string error objects, stack growth, allocator rejection, timeout, repeated close, and a host object destroyed first. Build the C layer with address and undefined-behavior sanitizers and assert stack height around every boundary. Fuzzing can explore Lua-to-C argument conversion, but create independent states so one script cannot contaminate the following case.
The Lua C API is compact and unforgiving. Give lua_State one owner, document every stack effect, contain script errors with lua_pcall, expose only required capabilities, and bound userdata, execution time, and memory. A narrow embedding boundary makes Lua a controlled extension language rather than an arbitrary-code tunnel into the host process.