Topic: Some Plugins

I figured that it would be a good idea to move the plugin discussion here. These plugins currently only work on the Github branch but I'm going to polish up the plugin system and commit it to the official repository soon.

I'll update this post with any new plugins I add, or anyone else wishes me to add here

MEvent Nistur
description
Requirements - none
Optional - none

MScriptableBehaviour Nistur
description
Requirements - none
Optional - MEvent, MScriptExt

MSaveFile Nistur
description
Requirements - none
Optional - MEvent, MScriptExt

MScriptExt Nistur*
description
Requirements - none
Optional - MEvent

MRecast Nistur
description
Requirements - none
Optional - none

MEmbedFile Nistur
description
Requirements - none
Optional - none

*extension of Maratis internal functionality written by Anaël

Last edited by Nistur (2013-07-16 13:39:54)

Re: Some Plugins

MEvent
link
Simple message system for use in Maratis. Overrides MGame so no other "games" can be added, but will send the following messages:
update - when the game gets an update call
lateUpdate - after the game has finished updating
preRender - before the game draws anything each frame
render - during the frame drawing
postRender - after everything else has drawn

To get the messages, create a class that inherits from MEventListener and use the MEventListenerDeclare macro. The event listener should implement void onEvent(const char*). Event listeners will not be fired in a fixed order, so you cannot guarantee which object will receive postRender first.

To broadcast messages, use:

MEventSend("SomeMessage")

TODO:

  • Change events to support ints/enums for faster checking

  • Lua support

  • Allow filtering of events so not every object gets every message

  • Support multiple broadcasters maybe?

Changelog
09.07.2013 - Fixed MEventListener so you can actually link to it properly

Last edited by Nistur (2013-07-11 19:48:53)

Re: Some Plugins

MScriptableBehaviour
link
Creating behaviours from within lua. Create a behaviors folder within the project and add scripts within that. Inside the script needs to be a table with the same name as the script. They can have the following functions: onBegin, update, onEnd

eg player.lua

player = {
    onBegin = function(obj, bh)
    end,
    update = function(obj, bh)
    end,
    onEnd = function(obj, bh)
    end,
}

obj - the object reference for passing to any of the standard Maratis lua functions
bh - behaviour ID for use with the standard Maratis lua functions (right now, useless)

Optional libraries: MEvent, MScriptExt
If MEvent and MScriptExt plugins are also included in a project with MScriptableBehaviour, a more object oriented API is available:

MScriptableBehavior "player" {
    speed = 3,

    onBegin = function(self)
    end,

    update = function(self)
        -- do something with self.speed
    end,

    onEnd = function(self)
    end,
}

Using this, each object has instance variables (within self) and if required, the object ID can be accessed by self.__id.object

TODO:

  • Don't load the script once per instance using it

  • Expose variables between lua and C++ (so that lua variables show up in the editor)

Changelog
11.07.13 - Added embedded lua for extended API

Last edited by Nistur (2013-07-11 19:55:34)

Re: Some Plugins

MSaveFile
link
Simple save file system.

MSaveFile* save = NULL;
MSaveFileGetNew(save, "file.sav", M_SAVE_FILE_MODE_BINARY);
if(save)
{
    save->setInt("some.key", 1);
    save->setFloat("some.other.key", 44.32f);
    save->setString("some.other.other.key", "something");

    int valInt = 0;
    float valFloat = 0.0f;
    char valStr[256];
    save->getInt("some.key", valInt);
    save->getFloat("some.other.key", valFloat);
    save->getString("some.other.other.key", valStr);

    save->destroy();
}
save = saveFileOpen("file.sav", "M_SAVE_FILE_MODE_BINARY")
if save ~= nil then
    saveFileSetInt(save, "some.key", 1)
    saveFileSetFloat(save, "some.other.key", 44.32)
    saveFileSetString(save, "some.other.other.key", "something")

    valInt = saveFileGetInt(save, "some.key")
    valFloat = saveFileGetFloat(save, "some.other.key")
    valString = saveFileGetString(save, "some.other.other.key")

    saveFileClose(save)
end

The file will automatically be loaded when opened and saved when closed. You can force save/loading by using MSaveFile::save/MSaveFile::load or saveFileSave/saveFileLoad
The mode must be specified when the save file is opened. The choices are:
M_SAVE_FILE_MODE_TEXT - will save in text (XML) format
M_SAVE_FILE_MODE_BINARY - will save in binary fomat
M_SAVE_FILE_MODE_ANY - will save in the existing format, or binary if no file exists
Specifying the mode will not affect loading the save file, it is possible to open an XML savefile with M_SAVE_FILE_MODE_BINARY and vice versa. When the file is saved the next time it will be overwritten with the specified format.

Optional libraries: MEvent, MScriptExt
MSaveFile contains embedded lua which it can add to the lua environment if the project contains both MEvent and MScriptExt plugins. It extends the lua functionality.

save = MSaveFile("file.sav")

save:setInt("some.key", 1)
save:setFloat("some.other.key", 44.32)
save:setString("some.other.other.key", "something")

valInt = save:getInt("some.key")
valFloat = save:getFloat("some.other.key")
valString = save:getString("some.other.other.key")

save:close()

Note: you can also use save:set which will allow you to pass any object, if it's a number or a string, it will save it directly as above, but can also save any table by looping recursively through its children. There is no matching working get function as MSaveFile doesn't yet allow iterating through the values

TODO:

  • Give a default directory to save in

  • Serialise tables in/out in lua

  • Add savefile versioning

  • Change flat binary files into hierarchical ones

Re: Some Plugins

Hey Nistur,

Very excellent job ! Many thanks for sharing. big_smile

I've tested your MScriptableBehaviour and found an interesting thing : now we can pass variables from other scripts to the update function, but we can't pass variables from onBegin to update. To clarify, what I did is set a variable in onBegin and use it in update.

Maybe it is related to current issue exposing variables between lua and C++

I'll try MSaveFile asap and give you feedback.

PS: you should take holidays more often!

Re: Some Plugins

com3D wrote:

Very excellent job ! Many thanks for sharing.

No problem smile I'm just hoping that other people can start adding to this list soon.

com3D wrote:

we can't pass variables from onBegin to update

Hrmmm, that's interesting. Can you send me a sample that shows this that I can test? I'm assuming it's something weird with how lua handles scopes. The behaviour doesn't actually have an object (and, therefore, data) associated with it. I haven't yet found how you can use more object oriented stuff when calling lua, although, now I have MScriptExt and embedded scripts working, I can relatively easily do this on the lua side I think...

com3D wrote:

I'll try MSaveFile asap and give you feedback.

MSaveFile is actually the only plugin that doesn't actually need Maratis, so I've added a small testsuite to it. This doesn't mean it's bug free by ANY means, but if you find any bugs, it'd be good to be able to reproduce this in the testsuite so we can make sure it doesn't happen again. I'll also try to add testsuites to the other plugins at some point maybe, but it will require stubbing out a lot of Maratis functionality to even get them to link.

com3D wrote:

PS: you should take holidays more often!

Haha. I actually did half of this stuff before my holiday, and half on the journey here. I've been doing this thing called "relaxing" this week mainly big_smile

As a slight aside, most of the plugins have two different build targets, a dynamic lib (so it can be used as a plugin directly) and a static lib which I am currently fiddling with linking into a über-lib, but would also (in theory) work for non-dynamic platforms, such as iOS. The plugins will have to be initialised/destroyed manually though from within the game

Re: Some Plugins

MScriptExt
link
Extensions to internal MScript system. Almost no changes have been made at this stage. The one enhancement is that, if run with MEvent it will send the event MScriptInit* which can be used by plugins to extend the lua environment by adding scripts after the base environment has been created but before the main script has been loaded.

*name not consistent with other events. Need to decide what to use

MEvent should now fire off "publish" which will be translated through to lua as "editor.publish". MScriptExt now has an editor script which should be automatically loaded and will allow the following:

local test_publish_event_function = function()
    print("test publish event function")
end

local test_publish_event_table =
{
    publish = function()
        print("test publish event function")
    end
}

editor.publish_event(test_publish_event_function)
editor.publish_event(test_publish_event_table)

Also, MScriptExt will search for all lua files within editor/ and load them (NOTE: currently this does this even when not running in editor, so it might be best to add if isEditor() then ... end around any editor scripts) This means that things such as publish events, which don't need to be published themselves, can be placed in editor scripts. Editor scripts can also be added to plugins to allow for flexibility to package their data however they see best.

Changelog
22.07.13 Added editor scripts and supported publish events
11.07.13 Updated to use luaL_loadbuffer within ::parse and added MResource interface

Last edited by Nistur (2013-07-22 19:17:27)

Re: Some Plugins

Nistur wrote:

Can you send me a sample that shows this that I can test?

Of course. I've tested with the SponzaFPS demo and the following Script1.lua file put in the behaviors folder:

Script1 =
{

    ----------------------
    -- SetUp Once / Constants
    ----------------------

    onBegin =

    function(object, behaviorID)

        speed=3

    end,

    ----------------------
    -- Scene Update
    ----------------------

    update =

    function(object, behaviorID) -- NB: behaviorID is currently inactive

        if isKeyPressed("A") then addCentralForce(object, {-speed, 0, 0}, "local") end
        if isKeyPressed("Z") then rotate(Head, {1, 0, 0}, 1, "local") end

    end

}

The behavior is applied onto the Player.

If I put the "speed=3" in the main lua script (joystick or mouse) or in update then it works.

Thanks

Re: Some Plugins

Oops, try again, I forgot to push the changes to github. onBegin was never being called. Sorry

Note: you can't currently use MScriptableBehaviour for multiple objects like this easily. But I had a thought, you could do something like this:

Script1_objects = {}
Script1 = {
    onBegin = function(object, behaviorID)
        Script1_objects[object] = {}
        Script1_objects[object].speed = 3
    end,

    update = function(object, behaviorID)
        -- do something with Script1_objects[object].speed
    end,
}

I'll try and make this a bit "friendlier" but right now there's not much else I can do

Last edited by Nistur (2013-07-08 13:42:24)

Re: Some Plugins

Many thanks, I'll check if everything is OK now.

Nistur wrote:

you can't currently use MScriptableBehaviour for multiple objects like this easily

Not sure if MScriptableBehaviour is robust enough, but currently it can handle objects from other scripts (like Head in SponzaFPS) without any problem. I'll check more extensively a little later, but for now it looks almost perfect!

Re: Some Plugins

No, sorry. That's correct. I wrote the code on June 10th but just forgot to push to github until earlier today.

Also, I'm working on extending MScriptableBehaviour a little. Got a bit distracted today, but should have a more object-like way of dealing with behaviours very soon smile

Re: Some Plugins

First tests look really promising & work fine. big_smile

However I found a little issue with how the script names are handled.

Even if scripts are put in 2 different folders (scripts & behaviors) and the script from scripts folder is not called with a dofile, you cannot use the same name for both, it makes the editor crash.
So I renamed the script in the behaviors folder with a "_Nistur" (for example Script_Nistur.lua) to fix this.

Then I did the same with a second new script (ZZZ_Nistur.lua) and it makes the editor crash again. If I remove the "_Nistur" then it works fine.

Last edited by com3D (2013-07-09 07:10:22)

Re: Some Plugins

com3D wrote:

Even if scripts are put in 2 different folders (scripts & behaviors) and the script from scripts folder is not called with a dofile

Whaaaa? I have to admit that I didn't test this, but I didn't think it would be a problem because, I _thought_ it should default to looking in the scripts folder.

com3D wrote:

Then I did the same with a second new script (ZZZ_Nistur.lua) and it makes the editor crash again. If I remove the "_Nistur" then it works fine.

Curiouser and curiouser.

Right. I have some use cases to test out. I'll get back to you with this smile

In other news, I almost have the "extended" MScriptBehaviour working. I have the lua looking like it's working, but the problem is currently that the way I'm embedding the file doesn't null terminate the file, so I'm either going to have to pass a size through to MScript when loading it, or stick a 0 byte at the end. Neither of which are particularly difficult, I just need to decide which. Null terminating would make most sense perhaps, but if I can pass through the size, because I'm using luaL_loadbuffer rather than lua_dostring then it means that I can embed compiled lua scripts in release builds which means smaller executables and (in theory) faster loading times.

The way that the extended MScriptableBehaviour will work is as follows:

MScriptableBehaviour "player" {
    health = 10,
    onBegin = function()
        self.speed = 12
    end,

    update = function()
        move(self.speed)
        if hit then
            self.health = self.health - 1
        end
    end,

    onEnd = function()
    end,
}

The system no longer passes through the object or behaviour IDs. they are stored inside the behaviour and can be retrieved with self.__id.object and self.__id.behaviour The reason behind this is that they shouldn't be used eventually (of course, initially they will be) but I'm hoping that someone will take my code and extend the lua MScriptableBehaviour and add in wrappers for all of these in the embedded script so from a behaviour you could just do something such as

self:setScale({10, 10, 10})

and you wouldn't have to worry about the IDs at all.

Anyway, a few more tests before that one gets submitted, but the script is almost all there. Just need to get the file loading to work smile

Re: Some Plugins

Update: deleting the script in the scripts folder doesn't improve the situation, I still cannot use the original name. And if in place of _Nistur I rename by _N for both scripts then it works.


I've found another small issue : cloning an object (using the getClone function) onto which a behavior has been applied makes the editor crash.

Re: Some Plugins

Right, something really weird is going on here. I've got some other work to deal with today, but after I'm done I'll look into it

Re: Some Plugins

Take your time, Nistur, and don't forget to have fun !

Re: Some Plugins

Right, sorry about the delay. I've checked up on the multiple scripts issue. I have an empty project that has three scripts in it, scripts/main.lua scripts/camera.lua and behaviors/camera.lua which, with the exception of main.lua (which also has a single dofile in it) all they do is print their own name. I have a cube with the camera behaviour added to it, and all 3 scripts print. I can't produce a crash at all.

Could you please zip up a sample that crashes for you please?

Edit: I also tried using the script you provided
Update: Updated MScriptableBehaviour to include extended lua API

Last edited by Nistur (2013-07-11 19:56:17)

Re: Some Plugins

Thanks, a zip might not be necessary.

Try creating a file named Hero.lua with no active script in it (just a blank update function or even an empty table) and apply it to the Player in Jules demo. With this name it crashes in any project I test, but if I name it for example Hero_N then no crash.

I'll test your new version immediately! tongue

By the way, did you try the getClone function ?

Last edited by com3D (2013-07-12 05:52:42)

Re: Some Plugins

I haven't got around to testing the getClone yet, I'll look into that next.

Oh, and the extended MScriptableBehaviour needs MScriptExt to be updated for it to work too. I think I will add versioning into the MResource so it won't try to work (and fail painfully) with previous versions.

Re: Some Plugins

Just a quick thing I remembered. It's possible in lua to have an implicit self parameter. So if it suits your programming style, you could do the following instead for MScriptableBehavior:

MScriptableBehavior "player" = {
    -- variables go here
    speed = 3,
}

function player:update()
    -- can use variables here
    move(self.speed) -- the self parameter is passed because we declared the function with a :
end

It does mean that you have to specify variables separately from functions, not sure how much I like that. But it's up to you smile

Re: Some Plugins

MRecast
link
Currently does nothing. Just a placeholder for when it does

Re: Some Plugins

MEmbedFile
link
This wraps up embedding files within code, which means that plugins can more easily provide their own data. Other plugins already sort of support this manually for some of their API scripts, which get loaded when MScript initialises. MEmbedFile will allow optional scripts, which can be loaded only when required with dofile as well as embedding images, or any other data.

To use:

MEmbedFile* file = NULL
MEmbedFileGetNew(file, "file/name", fileData, fileDataSize);

if(file)
{
    file->destroy();
    file = NULL;
}

The file is kept in the virtual file system until it is destroyed and can be accessed by any of the standard Maratis file IO functions.

Re: Some Plugins

I've run a getClone function with MScriptableBehaviour in debug mode to check what makes the editor crash.
Console returns :

ERROR lua script : unable to read file /behaviors/MScriptableBehaviour.lua

Re: Some Plugins

Hmmm, I think I know what that will be. It's something I've been meaning to fix for a while anyway, I think the issue is that when you start a new behaviour, it will automatically try to reload the script file (because up until recently, there was no way to get a callback when to add to the script state) That probably means that it's steamrollering over something important. It's an easy fix, but it's going to require MScriptableBehaviour depends on MEvent and MScriptExt to make it work.

In other news... I've added publish events! It requires an update of the community Maratis branch to support it, but games, and plugins, can add scripts to dictate how their data gets published. I'll update the MScriptExt post, and maybe write a short doc on the wiki about it.

Edit: I was thinking, as there are a number of plugins which pretty much depend on each other (or, their function is greatly reduced without each other) I might roll MEvent, MScriptExt and MScriptableBehaviour into one MExtensions plugin (basically what I've done with Mage) I'm not sure whether I want to add MSaveFile and MEmbedFile to that yet.

Last edited by Nistur (2013-07-23 07:15:08)

Re: Some Plugins

I'm fan for a unique plugin as it will be easier to update and keep in sync.

By the way, I'm having trouble building with premake (on linux), it seems that there are prerequisites to match, like environment variables, file locations...
Documentation for these settings &/or a bash script would be greatly appreciated (I'm currently building with a standard makefile).