SECS in technicolor
A colour-coded model of the SECS pipeline. Work in progress — boxes that share a colour are related.
28 articles in the notebook
ScriptEngine.
Script Engine Component System (SECS)
People define a script engine narrowly as "the thing that runs the scripts" — I'd argue it's the most important component of OpenSim. SECS is my proposal for a component system with interfaces worth building on.
DotNetEngine and xengine
Reflections on why a common way of implementing script engines was needed: many found it hard to build engine variations on DotNetEngine, and not all developers want to conform to one structure.
Objectifying LSL
OSSL takes an object-oriented turn: instead of only LSL2-style function calls you can now write Prim.Position.x += 10 or set Prim.Text directly. A more modular approach to the scripting language.
New languages
DotNetEngine now speaks VB.Net and JScript — just start your script with //vb or //js. LSL state support landed too.
New common framework
Most of DotNetEngine has moved into OpenSim.Region.ScriptEngine.Common, turning it into a general execution framework. DotNetEngine now only converts and compiles — Common does the running, and future engines get loading, threading and maintenance for free.
ScriptEngine with many regions per server
New detailed config options for DotNetEngine, and more importantly: maintenance, load/unload and execution threads can now be shared across regions instead of 3-4 threads per region — a big deal for large scale OpenSim installations.
ScriptServer
ScriptServer is "almost done": everything works except script-to-OpenSim communication. It lets script execution happen on a remote machine entirely outside OpenSim.exe.
Script engine implementation
I wrote up my opinions on how the future script engine should be implemented as a contribution to the Architecture Working Group wiki.
Communication
Notes on the protocol between region and script server: region initiates, a custom binary protocol streams async function calls in both directions, and replies are matched by ID. TCP client/server is already up.
A peek on new architecture
A diagram of where the stand-alone ScriptEngine planning is at. It was very complex; after redrawing it a few times it ended up like this.
Major rewrite number 2
Lots of alien squid-like drawings in my notebook later: the plan is to move the ScriptEngine into a separate daemon, independent from the region. Scripts stop moving when objects do, can run on separate servers or clusters, and can even be rented out.
Pure C# support
The ScriptEngine now mixes LSL and C# in the same script — and if a script starts with //C#, the LSL preprocessor is skipped entirely for 100% pure C# support. A few naming rules apply for events.
Small stresstest
114 scripts loaded in-world, one AppDomain each, most of them firing timer events every second or every five — and my 2.8GHz Prescott doesn't break a sweat. Screenshots of the world and the performance graphs.
So where are we now?
A summary for those who skip the techie details: LSL script support works, hundreds of scripts run simultaneously across multiple cores, events are queued, and the remaining gaps are clearly listed.
Introduction of a new class
Some LL commands, like llSetTimerEvent and llHTTPRequest, don't return immediately — they come back later as events. LSLLongCmdHandler is a new class with a dedicated thread that keeps track of these long-running requests.
Speed
Script events now execute multithreaded — one object at a time, so multicore CPUs finally get used — plus cached event binding to avoid reflection on every call.
AppDomains
.NET cannot unload an assembly, only a whole AppDomain — so scripts get an AppDomainManager that pools scripts into domains and recycles domains as scripts die.
New status after Monday (and a milestone)
ScriptEngine is now implemented in OpenSim: a default script in every object, a touch_start event, and llSay writing to the server console. Now the 350+ builtin ll-functions can be implemented and tested in-world.
Status of LSL compiler
Quick status: the LSL-to-C# translator is close to complete, the compiler and script manager work, and what remains is the builtin commands and events.
Small milestone reach today
The full event pipeline works end to end: pressing a button in the test app sends an event through the ScriptEngine, into the queue, and out to the correct compiled script. Screenshot included.
Status on initial SE implementation
The plan for the first simple version of script support: a simplified event queue with a custom threadpool, one shared AppDomain, and a basic LSL compiler that others can extend while I keep working on the core.
ScriptEngine - Events
Builtin commands are easy — incoming events need planning. An EventHandler translates OpenSim happenings into LSL-compatible events, and an Event Queue Manager executes them on a separate thread so the server never blocks.
ScriptEngine
The architecture for OpenSim script support: a dynamically loaded ScriptEngine assembly that hooks the server's events, a compiler that turns LSL into .NET assemblies, and the loading pipeline between them.
Current status
Static and global variables, push/pop and simple scripts with math and llSay now work, and the internal state is ready for serializing so a running script can be moved between servers. Next up: local variables, custom functions, states and jumps.
Current OPCODE implementation status
A raw status table over the LSO opcodes: which ones the IL processor understands and which ones are implemented so far in the LSL-to-.NET translation.
Looking ahead, fibres
Running thousands of scripts on a grid cannot mean one OS thread per script. Old ideas come to the rescue: cooperative multitasking à la Windows 3.11 and NT fibres, i.e. user space threading where each script carries its own execution state.
LSL scripting support - Progress report
The converter can now take LSL bytecode all the way to a compiled .NET .dll on disk, with events visible in Visual Studio and llSay mapped to Console.WriteLine. Messy code, but the goal is simply "Version 1: Functional".