TshwaneLua: TLex/tlTerm/tlDatabase/tlCorpus Scripting

TLex, tlTerm, tlDatabase and tlCorpus include integration of the Lua scripting language.

Getting Started with TshwaneLua Scripting
Sample Scripts
Common Mistakes to Avoid
Frequently Asked Questions

[New] GitHub repository of informal public Lua scripting samples for TLex, tlTerm, tlDatabase etc. »

Getting Started with TshwaneLua Scripting

TLex, tlTerm, tlDatabase and tlCorpus have a built-in programming (scripting) language. We have chosen to embed the OpenSource Lua scripting language. There are primarily two types of scripts:

1. Lua Script Attributes: A Lua script attribute is just another attribute in the document, but is special in that instead of ordinary data, it contains a piece of Lua script that is executed in order to generate the resulting output for that attribute value, very similar in principle to a 'formula' in spreadsheet applications.
2. Stand-alone Scripts: Stand-alone (or external) scripts exist in the form of a text file, and are loaded and executed via the "Tools / Execute Lua Script" menu option in TLex/tlTerm/tlDatabase.

Other types of scripts include "click scripts" and "search scripts". The main script types will be dealt with here.

1. Lua Scripting Attributes

A good place to start learning about Lua script attributes is to have a look at the "Lua Scripting Attribute" sample file in the Samples folder. To view the sample's script attribute contents, go to "Dictionary/Customise DTD", select the "Sense" element on the left, then select the "SenseNumLua" attribute under "Attributes of this element", click in the "Default/fixed value" box at the bottom right of the DTD editor, and press F12 to open the script in the overlay editor window. This sample demonstrates the use of a fixed Lua attribute value to generate automatic sense numbering in a manner currently not supported by the automatic numbering styles: Specifically, repeating the sense number of the parent at every subsense (e.g. "2a ... 2b ..."). The return value of the script is the value that is output in the Preview etc. (with the usual styles and so on applied, just as with any other attribute).

Note that for a Lua script attribute you can choose if the attribute value should be "fixed" or not. If "fixed" (as in the sample), the script is entered once - in the DTD editor - and the same script is always executed. If not fixed, a different script can be entered for the attribute on an individual entry/element basis.

To create a Lua DTD attribute, create an attribute as usual in the DTD editor, then select "Lua script" as the "Data type".

Note the use of the gCurrentNode global. This value always contains the node (i.e. element in the document tree) that the attribute on which the script is currently being executed belongs to. (Related to this is also a gCurrentEntry global, pointing to the owner entry.)

Note how all local variables are explicitly declared as such - in Lua, variables are global unless otherwise specified.

2. Stand-alone Scripts

Stand-alone (or external) scripts are basically text files containing Lua code. They can be executed via the "Tools / Execute Lua Script" menu option in TLex/tlTerm/tlDatabase. By default they use a ".lua" file extension but this is not necessary. Such a script can be used to perform pretty much any kind of manipulation on the database.

⚠️NB: The text file encoding for Lua scripts is always UTF8, without signature.

2.1 Stand-alone Entry Scripts

[Version 4] "Tools / Execute Lua Entry Script" executes the selected Lua script file for every entry currently visible in the Entry List(s) of the currently active document (e.g. all that pass the filter). gCurrentEntry will point to the current entry 'on' which the script is being run. tRequestLoad() is called for you, but not tRequestModify().

General Lua Tips

TshwaneLua Reference: The TshwaneLua API (Application Programming Interface) reference can be viewed via "Start menu / Programs / TLex (or tlTerm etc.) / TshwaneLua Scripting API". Good starting points are the "Detailed Description" sections of tcNode and tcDocument.

Note that we implement Lua 5.0. For general Lua information (e.g. syntax or standard Lua function reference), it is suggested that you make use of the Lua reference manual [PDF].

Keep in mind the use of F12 to open the overlay editor window when editing scripts - this is virtually indispensable.

In our API, indexes are always zero-based unless otherwise specified. Strings are always UTF8.

Sample Scripts

Tip: Always hit 'File/Create a backup' before running/testing any script that could make extensive or negative changes to your document.

Template script: This script calls a function on all lemmas/entries in the database.

-- TLex/tlTerm sample script
--
-- Call a function on all entries in the currently selected language/section
-- window that pass the currently applied filter.

function YourFunc(ENTRY)
	tLuaLog(ENTRY:GetLemmaSign());
end

-- Make sure we have a document open
local DOC = tApp():GetCurrentDoc();
if DOC == nil then
	return "No document open";
end

-- Iterate through all 'sections'/languages, and for each section, through all entries.
local i, j;
for i=0,DOC:GetDictionary():GetNumLanguages()-1,1 do
	local SECTION = DOC:GetDictionary():GetLanguage(i);
	for j=0,SECTION:GetNumEntries()-1,1 do
		local ENTRY = SECTION:GetEntry(j);
		YourFunc(ENTRY);
	end
end

return "";

Template script: This script calls a function in the currently selected language/section window on all entries that pass the currently applied filter.

-- TLex sample script
--
-- Call a function on all entries in the currently selected language window that
-- pass the currently applied filter.

function YourFunc(ENTRY)
	tLuaLog(ENTRY:GetLemmaSign());
end

-- Make sure we have a document open
local DOC = tApp():GetCurrentDoc();
if DOC == nil then
	return "No document open";
end

-- Get selected section/language window
local FRAME = tFrameWindow();
local SECTION = FRAME:GetSelectedSectionWindow();
local LANG = SECTION:GetLanguage();
-- Get filter tool
local FILTER = SECTION:GetFilter();
local i;
for i=0,LANG:GetNumEntries()-1,1 do
	local ENTRY = LANG:GetEntry(i);
	local include = true;
	if FILTER:IsFiltered() then
		include = FILTER:PassFilter(ENTRY);
	end

	if include then	
		YourFunc(ENTRY);
	end
end

return "";

Move attribute onto parent.lua This script moves all occurrences of an attribute from an element to an attribute with the same name on its parent, if the element is a child of the specified parent element.

Assign random colours to styles.lua

Access PCDATA text nodes.lua Demonstrates how to access 'text' nodes under the new 'Document Object Model' (TLex 4.1 or higher).

If you have created scripts that you feel may be useful to others and you'd like them uploaded here, you are welcome to send them to us!

Common Mistakes to Avoid

Function Parameters

When calling c++ object members from Lua e.g. Node:SetAttributeDisplayByString(Attribute, Value); the object is treated as a parameter.
In this case "Node" would be parameter 1, "Attribute" parameter 2 and "Value" parameter 3

It is important to keep this in mind as otherwise an error message such as error in function 'SetAttributeDisplayByString' argument #2 is 'number'; 'userdata' expected. will have you looking at the wrong parameter resulting in confusion.

Array Indexes

In C++ indexes always start from 0, in Lua they start from 1. Our Lua API follows the C++ convention so all indexes in our API start from 0

It is important to keep this in mind as you may sometimes need to adjust an index if using both our API as well as another Lua API at the same time.

NULL Pointers

In C++ 0 can often be used in place of NULL without any negative effect, this is not the case in Lua. Please always use nil when writing Lua scripts to refer to NULL values.

Unicode

All strings from our API are UTF8 and our API is fully Unicode aware, however Lua itself is not UTF8 (or Unicode) aware.
This means that while you can copy/compare and even manipulate the UTF8 strings safely inside Lua using certain parts of the Lua API, some parts will not work as expected, it is important to keep this in mind.

Example:
local Value = Node:GetAttributeDisplayAsString(Attribute);
Value = string.gsub(Value,"[愛恨]","");
The expected behaviour of the above is for all occurrences of either "愛" or "恨" to be removed from the text.
Unfortunately this is not the case, the reason for this is that these characters are represented by more than one byte: 愛"xE6 x84 x9B" 恨"xE6 x81 xA8" as Lua is not UTF8 aware, it will treat each byte as an individual character so what it actually sees: "[ææ¨]".

Note This is only the case for the API that comes with Lua, the above will not be an issue with any functions or methods in our own API.
The equivalent of the above in our own API would work as expected.
Example:
local Value = Node:GetAttributeDisplayAsString(Attribute);
Value = tRegReplace(Value,"[愛恨]","",true);
It is therefore better to use functions from our own API where possible when doing string manipulations.

TshwaneLua Frequently Asked Questions (FAQ)

Q: How to typecast an object?

If you have (for example) a variable of type tcNode called NODE which (NB) you know is of type tcReferences, you can typecast it using tolua.cast as follows:

local Reference = tolua.cast(NODE, "tcReferences");

Q: How to use functions that work with std::vectors?

local VECENTRIES = vector_tcEntry__:new_local();
LANG:FindEntries(NAME,VECENTRIES);

local NodeArray = vector_tcNode__:new_local();
Node:FindDescendantsOfElementType(SomeElement:GetID(), NodeArray);

Q: How to instantiate an object explicitly, like C++ 'new'?

See above.

Q: I get a "not enough memory" message when running a Lua script

It is possible when running Lua scripts that perform a lot of operations over large enough sets of data that all the available memory might be exhausted before control returns to the program and the garbage collector gets a chance to run. This can be avoided by calling the garbage collector yourself periodically from within your script with the following code: collectgarbage(0)

Q: I get a "C stack overflow" message

Try declaring variables and/or functions "local" (i.e. with local keyword in front).

Q: ODBC and Lua

- If writing a script that will process all entries in a database stored via ODBC, you must call tRequestLoadAll at the start of the script to ensure that all entries are fully loaded locally.

- Alternatively, you must call tRequestLoad individually on each entry you will process.

- If you want to modify an entry, you must first 'check it out'. To do this, call bool tRequestModify(Entry, true). You must check the return result - if it returns false, it generally means that somebody else on the network is busy working on that entry. Saving the database as usual 'checks in' any changes you made. The second parameter is a boolean 'bSilent' - if true, the function will not display any user interaction message boxes (such as asking the user to confirm the checkout, or error messages on failure).

Q: How to send an event?

Events can be triggered using the Trigger method on the event object.

In most cases the first parameter when calling events from Lua should be nil, the second parameter will be the object that the event relates to. e.g. If it is a Node changed event then the Node that has changed would be the second parameter.

Evt_StylesChanged:Trigger(nil, Styles);

Q: When my Lua script runs the preview or other parts of the program don't update to show the changes correctly, why is this?

Many parts of the program rely on events to be sent in order to know that a change has occurred and respond to the change.

You will need to add events to your script in order for the program to update correctly when the script is run. See above for an example of how to send an event.