The Bitsquid engine has a built in scripting component that allows game scripting in Lua. The scripting component is versatile enough that it should be possible to build an entire game in Lua. Dropping into C++ should only be necessary when maximum performance is needed. And in that case, only the performance critical parts need to be implemented in C++.
This document describes how Lua is connected to the C++ engine. It does not give a full reference of the Lua API, see the separate Lua API documentation for that. Also, it does not provide a detailed description of how the different systems that can be scripted in Lua (such as the Physics system) work. To understand such a system you should begin by looking at the system documentation to get a feel for the different concepts used and how they work together. Then you can refer to the API document for a description of the scripting interface.
There are three main contenders for a gameplay programming language:
Using C++ is already possible in the Bitsquid engine, since the core engine is written in C++. C# and Java are faster than the "dynamic" languages and offer better refactoring tools. But our impression is that they are not different enough from C++ to have a huge impact on gameplay programmer productivity. Also, they both come with huge, scary, semi-black box runtime environments – we want small runtimes that are easy to overview and control.
Note
We are still investigating this though, and in the future, we may offer Mono bindings.
Of the dynamic languages, Lua is a standout. It is very fast, has a very small and clean implementation that is easy to understand and a minimal runtime without sacrificing flexibility and power. The main drawbacks of Lua are:
When the game engine loads, it also loads single Lua file (the Lua boot file). After loading the boot file it calls the global Lua function init(). That functions is responsible for loading other Lua files and resources needed by the application. The name of the Lua boot file can be configured in the settings.ini file.
Note that you can't use require at the top of the boot file to bring in other Lua files, because those files will not have been loaded by the time the boot file is run. To require the files, you must first make sure that they are loaded. For example like this:
function init() local package = Application.resource_package "main_package" ResourcePackage.load(package) Resourcepackage.flush(package) require "main/game" end
Note that when you call require you always specify the script name without the .lua extension and with the full path to the file in the game folder. This is consistent with how other resources in the game are referenced.
The typical setup for the boot script is to first load all the resources required to display the loading screen. (The loading screen does not have to be a single static screen. It is a regular game world, just like an ordinary level and can use animation, sound, physics, etc. The only difference is that it is usually small so that it loads quickly.) Then, the loading screen is displayed while the main game is loading. Finally, when the loading has completed, the main game is started.
This is accomplished with the following steps:
Note that the Lua script files are typically only loaded once, at the start of the game. They don't use that much memory, so there is little point in isolating the Lua files needed at a particular level and streaming them in and out.
A good way of loading the lua files is to put them all in a single resource package and load that package early in the game (right after the loading screen has been displayed).
A minimal script for a Lua game looks like this:
package = nil world = nil viewport = nil shading_environment = nil camera = nil test = nil function init() package = Application.resource_package "test" ResourcePackage.load(package) ResourcePackage.flush(package) world = Application.new_world() viewport = Application.create_viewport(world, "default") shading_environment = World.create_shading_environment(world, "core/rendering/default_outdoor") World.set_shading_environment(world, shading_environment, "core/rendering/default_outdoor") local camera_unit = World.spawn_unit(game.world, "core/units/camera") camera = Unit.camera(camera_unit, "camera") Camera.set_local_position(camera, camera_unit, Vector3(0,-2,0)) test = World.spawn_unit(world, 'test', Vector3(0,0,0)) end function update(dt) World.update(world, dt) end function render() ShadingEnvironment.update(shading_environment) Application.render_world(world, camera, viewport, shading_environment) end function shutdown() Application.destroy_viewport(world, viewport) World.destroy_shading_environment(world, shading_environment) Application.release_world(world) ResourcePackage.unload(package) Application.release_resource_package(package) end
This script loads a resource package with the needed resources, creates a world, spawns a unit in it and creates a camera to render the world. Then it continually updates and renders the world. Finally it shuts down by destroying all the resources it has created.
Note that init(), update(), render() and shutdown() are global functions that are called by the game engine at appropriate points.
If you want to support more than one world (such as a loading screen world and a main world) you need to keep track of which world is currently being shown and make sure to update and render it in the update and render callbacks.
You can add your own code to the update() callback to move around the objects in the game world.
The script integration has been written with rapid prototyping in mind. Script errors do not (typically) crash the game. Instead, if an error occurs in the script, an error message is written to the console. You can fix the error and reload the script without restarting the game. Similarly you can tweak values and add functionality in the scripts and just reload them to see the effects of your changes, without having to reload the game.
An error in a Lua script will be written to the Console, together with a stack traceback. Given the error message and the stack traceback it should hopefully be easy to locate and fix the error.
When a script error occurs the game is automatically paused so that you can fix the error. Once you have fixed the error, you can reload the scripts and type "game unpause" in the Console to continue the execution.
By ticking the check box next to the command line in the Console window you can use the console to directly type and evaluate script commands. The result of the commands will be printed to the console. This is a good way of investigating the runtime, testing things out and debugging problems.
For example, to spawn a test box, just write:
World.spawn_unit(Game.world, "test/box")
Where Game.world is the variable that holds your game world. Or, to list all the functions in the Vector3 module, write:
for f,_ in pairs(Vector3) do print(f) end
If the stack traceback does not give enough information, you can use the Lua Debugger to find out more about the error.
The Lua Debugger is an integrated part of the Console, accessible through the menu option View > Lua Debugger. The Lua Debugger allows you to set breakpoints in the code and step through it as in a regular debugger. You can also evaluate specific lua expressions.
The Lua Debugger
You can edit your Lua files directly in the Lua debugger and use Save and Reload to save them, compile them, and load the new files into the running application. Save and Reload can thus be used as a quick fix-and-continue option to fix small problems. Note that for Save and Reload to work, you must setup the correct paths to your game executable and your data folders.
The game engine supports a generic debugging interface over TCP/IP for debugging Lua applications, so if you want to integrate a debugger into your favorite Lua editor, it shouldn't be too much work.
Lua files can be reloaded without restarting the application. You can reload a Lua file with the console command:
reload lua path/to/file
Or reload all lua files with just
reload lua
Note that the reload command does not automatically recompile and build the Lua files, so you must run your regular command line script for rebuilding the data directory before reloading the file.
For reloading to work, you must take some care when writing your Lua files to that they reload nicely. A Lua file is reloaded by just running the Lua file again. This means evaluating all the definitions, etc again.
Consider the following Lua script:
function update()
print("Hello")
end
Running this script will create a function in the global table _G["update"] that prints "Hello". If we edit the string and reload the script, the effect of the reload will be that the function at _G["update"] is changed to the new definition, which means that everyone calling update() will get the new code.
A second example:
Game = {}
Game.__index = Game
function Game:update()
print "Hello"
end
function init()
game = {}
setmetatable(game, Game)
end
function update()
game:update()
end
When this file is loaded a new table is stored in _G["Game"], then _G["Game"]["update"] is set to a specific function. When init() is run the game object game is set to use _G["Game"] as its meta table.
Consider what happens when we reload this file. At the start of the file, a new table will be created in _G["Game"] and an update function will be created for that table. But the existing object game's meta table is not changed. (Unless init() is called again.) It will still refer to the old Game object. So when game:update() is called, it will still print the old string "Hello".
We can fix this by changing the first line of the script to:
Game = Game or {}
Now, when the file is reloaded, _G["Game"] already exists, so no new object is created. This means that the update function in the existing Game table is changed, which means that game will see the change and use the new string.
By carefully considering what happens to the global variables when a file is reloaded in this manner, you can make sure that reloading is safe and useful.
The binding of C++ objects to Lua has been selected to balance programmer convenience with runtime efficiency.
The most convenient binding is to bind all C++ values as full userdata in Lua, since that means that we can give them meta tables and use them with regular object syntax and arithmetic operations.
object:call() c = a + b
However, using a full userdata is expensive. Full userdata objects are allocated on the heap and subject to garbage collection. The more such objects we have, the more time consuming garbage collection will be. Also, temporarily allocating full userdata objects for every Actor, Unit, Mesh, etc that you want to talk to, just to garbage collect them a few frames later is costly. This is especially true for operations that create a lot of temporary objects, such as mathematical operations on vectors and quaternions.
Note
Escape analysis might prevent creation of many such temporary objects. If a future version of Lua had good support for escape analysis we might get away with using full userdata objects for everything which would be convenient.
For this reason, the Bitsquid engine uses four different binding strategies:
This is the most common binding used in the Bitsquid engine. It is used for Units, Actors, etc. The object is bound as a light userdata, which means that the Lua side just stores a raw pointer to the underlying C object with no metatable and no type information.
Since light user data does not have a meta-table, you cannot use the regular object syntax to call them:
unit:set_name("x")
Instead, you must explicitly look up the method in the class table Unit, and pass the object unit as parameter:
Unit.set_name(unit, x)
Or, with the method lookup cached for greater efficiency:
local unit_set_name = Unit.set_name unit_set_name(unit, x)
Light userdata objects are never owned by Lua (since they don't have garbage collection). Instead, you must explicitly tell C to create and destroy the objects.
unit = World.spawn_unit(world, "car") World.destroy_unit(world, unit)
The C side provides limited type checking for light user data objects. If you run into a situation where the C side does not recognize a type mismatch, contact the engine team about adding additional type checks.
Full user data bindings are used for objects that need to be owned by the Lua side. (Since a full user data binding is the only way to get the object into the garbage collector.) The raycast objects created by PhysicsWorld.make_raycast() are an example, they don't need to be explicitly destroyed by the user, they are automatically collected by the garbage collector.
Full userdata are also used in a few cases for classes where there only exist a very small number of objects, such as World and PhysicsWorld. Since only a few instances of these classes ever exist, the cost of using a full userdata object instead of a light user data is negligable, and we can use it for the added type safety.
It is recommended to still use these full userdata objects as if they were light user data, i.e., instead of:
ray:cast(from, to, len)
Write:
Raycast.cast(ray, from, to, len)
This gives the gameplay code a more consistent look, which makes it easier to understand.
A singleton binding is used for objects that are true singletons. I.e. there only exists one such object in the application. An example is Application, a singleton representing the application itself and Mouse a singleton representing the (primary) mouse.
For singletons, no object needs to be passed to the module functions, since they always operate on the same object (the single object). So to check if a mouse button has been pressed, you just write: if Mouse.button_pressed(0) then. You do not have to pass any mouse object to the function.
A temporary binding is used for objects that need to be Lua owned, but where we create so many temporary objects that using a full user data is prohibitively expensive.
The typical example is math objects such as vectors, quaternions and matrices. Such objects cannot be C owned, because that would mean that we would have to allocate and release all the temporary objects by hand in Lua, which would be impossibly tedious and error prone. But they cannot be full user data objects either, since the strain on the garbage collector would be too high.
The best (though by no means perfect) solution is to have them as temporary objects. Each Vector3 is represented by a light userdata on the Lua side, which points into a buffer of temporary Vector3 objects. Each frame the buffer of temporary objects is reset so that they can be reused the next frame.
This means that on the Lua side, you can use Vector3:s with good performance, and without worrying about allocating and deallocating them.
But you must be aware that the objects are only temporary. I.e., you cannot store a Vector3 in a Lua variable and use it in the next frame. The Vector3 is only valid during the frame in which it was allocated. If you want to save it and use it later you must store it in a more permanent way. For example, you can store it in a unit's script data:
Unit.set_data(player, "position", Vector3.add(a,b))
Another way of storing a Vector3 permanently is to use the Vector3Box class. A Vector3Box is a "box" that can hold a Vector3 value. Vector3Box is implemented as a full userdata object, so it persists over frames:
box = Vector3Box() box:store( Vector3(10,0,0) ) pos = box:unbox()
Note that every time you call Vector3Box() you allocate a new heap object for the box, so you don't want to do that too much. Typically you should create all the boxes you want to use in the init() method of your class and then just use store() to update there contents.
There are corresponding classes for quaternions and matrices: QuaternionBox and Matrix4x4Box.
The Bitsquid engine provides a generic mechanism for storing data in game Units through the functions:
Unit.set_data() Unit.get_data() Unit.has_data()
The data is stored and accessed by a sequence of keys. A key is either a string or an integer. For example:
Unit.set_data(player, "score", "headshots", 1) local shots = Unit.get_data(player, "score", "headshots")
The stored data can be a Lua bool, a Lua number, a Lua string or any C side object (temporary objects get stored permanently) or a reference to a Lua table or Lua function. Storing data in the unit data store has two advantages over using regular Lua tables for data storage:
There are two ways that C++ calls into Lua. The first is the global calls to init(), update(), render() and shutdown(). By implementing those global functions you can customize your game. The second is through flow. When interesting things happen in the engine, such as when two physics objects collide or when an animation trigger is reached, flow events are generated.
As a Lua programmer you can set up script flow nodes. The nodes you setup will become available in the unit and level flow editors. Technical artists and level designers can connect events to the script nodes, so that whenever the events are triggered the script nodes are run.
The script nodes are specified in the two special files level.script_flow_nodes and unit.script_flow_nodes that should be put in the root of the project directory. They specify the script nodes that should be available in the level editor and the unit editor respectively.
The files are SJSON files. A typical example may look something like this:
nodes = [
{
name = "Tile Unblock"
function = "flow_callback_tile_unblock"
args = {
tile = "unit"
ground = "bool"
air = "bool"
}
}
{
name = "Tile Block"
function = "flow_callback_tile_block"
args = {
tile = "unit"
ground = "bool"
air = "bool"
}
}
]
The name is the name of the node in the editor. Function is the lua function that should be called. You can use dot-syntax to specify a function inside a global table, i.e. flow_callbacks.tile_block.
args is a list of arguments to the function. For each argument the type is specified. The level designer can feed data to the arguments by connecting out variables in flow to the corresponding input variables.
When the lua function is called, it will receive all the arguments in a table. If the level designer has not connected any input to a particular argument, that entry in the table will be nil.