101

Re: package file request

- Thanks Jurgel

- Ok, the bin mesh is reading, it is faster than I taught, sponza mesh is loading in 10 sec in xml, half a sec in bin, of course it is less easy to maintain and impossible to edit by hand.

Re: package file request

Anael: That's very impressive. I like big_smile Of course meshes should be loaded from xml during development, but when published, everyone wants faster loading smile It should be fully debugged and shouldn't need any maintenance then, surely.

When you submit the bin read/writing to svn along with the package stuff, I'll update the publishing to write the meshs to the npk smile

103

Re: package file request

Hi,

I looked at the fopen wrapper again, I wanted to check the isFileExist function, then I noticed something that I have a small problem with. You used if(! file || !file->Exists()) and then I understood that M_fopen doesn't return NULL when the file is not opened. To be sure to port safely the current loaders, it should really match the use of fopen (and in this case return NULL if the file is not opened).

Re: package file request

Ok, I can fix that quickly at lunch. There will be another version uploaded at 13:30 (GMT+0).

105

Re: package file request

smile Thank you,
I will then commit little by little, I'll start by sending your wrapper on svn.

Re: package file request

I've uploading the update now. I've just changed a little. I've removed the extra check in IsFileExist and modified M_fopen a little to account for this. I wanted to keep the design of MFile itself the same as it already is because it would be useful, for example for save files, as you can use it as an RIIA class and for that the class needs to exist, but can return errors on things like Read, Write, and can be checked with IsOpen. Hope this is acceptable.

Once the code is on svn, tested on all 3 OSs, I will start working on some platform independant updates, such as adding an API for the teakey. I would like to also expose publish steps to plugins, but that _would_ require some project file changes, so I'll not do that for now.

107

Re: package file request

Thank you,
I'm starting including MFile and commit step by step

108

Re: package file request

I am forced to do some small modifications to MFile and then to MPackageManagerNPK, there is a problem in the files new/delete, a file cannot be new in Maratis (by MPackageFileOpenHook) and be delete in MCore (by M_fclose) as MCore is a dynamic library the memory is not shared, at least on windows.

It's best if you don't make modifications the time everything is sent to svn (I think it's what you planned but just in case).

Re: package file request

That's ok. I hadn't noticed that problem. I guess the solution is to put a Close function into the FileOpenHook?

110

Re: package file request

Something like that, it's ok I had the same problem with MDataRef at the time, it will be the same system, a "getNew" static function for the derivated class and a virtual "destroy" function, to avoid using new and delete directly.

111

Re: package file request

!! the first pass is over,
MFile, MPackageManager, the file loaders you updated (I modified freetype loader at the same time so we don't need to convert the fonts), and the mesh bin export/load are on svn smile

http://code.google.com/p/maratis/source/list

The biggest part will be to commit your scons modifications, 3rdparties and your publishing code, then testing it on every platform and checking that everything still compiles on iOS : )

Re: package file request

Wow, that's great. It feels good to see my code online tongue Once the publishing code is in, I will take a look at writing the binary meshes to the package smile Shouldn't take long smile

I've had an idea about how to allow publishing steps from plugins. If the MPublishEvent interface is moved to MEngine, along with MPublishContext, which will just have

virtual void addEvent(MPublishEvent* pubEvent) = 0;

then  MEngine can hold the publish context (will always be 0 in the player), plugins can then, in StartPlugin can do:

if(MEngine* engine = MEngine::getInstance())
{
    if(MPublishContext* pub = engine->getPublishContext())
    {
        pub->addEvent(new SomeCustomPublishEvent);
    }
}

113

Re: package file request

Hi smile

your idea is perfect, I just don't know where we should put it, it is a tool thing so it should not be in MEngine, but it need to be accessed by plugin... Ideally we should have two different kind of plugins, editor plugins and engine plugins.

I didn't had time yesterday to finish the commits, I'll try to finish today, I will probably be very busy next week because of a new work coming.

Re: package file request

No worries, I wasn't trying to rush you.

With regards to where to put it, I see the problem, I guess one way around it is, as you say, to split plugins into editor and engine plugins. The problem with that is that some might cover both, for example, the game plugin which might want specific publish steps. Although I guess this could be avoided by pulling out the publish functionality into smaller plugins...

115

Re: package file request

Yes, I'm not sure yet, lets take some time to think about it, having different plugins can be also confusing for users, but at some point it will be probably needed to be able to extend the editor.

I commited your npk package manager, I just modified a bit the cleanup.
Next, your publishing, and (I forget to do) the level bin.

Re: package file request

Thanks, I guess I should have done some of that cleanup myself... I'm sorry.

I guess it would make it more difficult if you had 2 plugin directories? Something like ~/.Maratis/plugins for editor-only plugins and $PROJECT_DIR/plugins for game plugins which could potentially also have editor effects? Or game ones which have a dependancy, so StartPlugin returns bool, and it can query a list of editor/game dependencies, checking whether they're available and/or loaded (first? Would have to make sure there are no circular dependencies) But yes, I guess this conversation should be moved to the plugin thread. Not overly important now. It was just something that came to mind with the publishing.

117

Re: package file request

No problem for the clean, im just used to check 3 times this things.

Ok, lets keep it for later.
I'm seeing the end of the merging, is it ok for you if the publisher code is GPL (Maratis and MaratisPlayer are) ?

Re: package file request

I have no problem with the code being GPL. If I mistakingly made it zlib, it's because it originally was written in MEngine.

119

Re: package file request

Done ! smile
wow, that was huge, thanks a lot for the big work Nistur !

I tested on mac and windows (not on linux), and it seems to be working.
There was a bad access on mac, because the publisher was accessing Maratis::getInstance() before everything was started, so I made MPublisher a singleton. I finalised the mac .app publishing and did a general cleaning.

But I didn't had to time to start the level bin. Maybe next week-end.

Thank you again, that was huge !
If you want to improve the publishing code, you can now commit directly on svn.

Cheers,
Anaël.

Re: package file request

Hey, great! Thanks for sorting through my submission smile I will take a look at writing the mesh files directly to the package tomorrow smile Maybe expose the teakey and check that the game plugin can change it for publishing and loading. The only issue is that the plugin doesn't necessarily know that the package manager _will_ be the NPK one, so there would be no guarantee that the encryption needs 4 ints...

I noticed that, for the OS X publishing, you commented that all the data could be moved into the .app. I could probably do that pretty quickly too, although I would have no way to test it.

121

Re: package file request

smile cool,
for the key, you can just pass a string, each manager will interpret it the way it need.

For osx yes, it is working like that, but it also possible to put the data/dylib into the Resources/ app sub-path, so it is more hidden.
I tested by copying it by hand.

[EDIT] it's working on my Linux : http://www.maratis3d.org/download/M_Lin … -rev93.zip

Re: package file request

As requested, continued from PSP Thread

As I said, I find it quite difficult to follow a lot of the NPK code, but I believe once the package is opened, at least some of it is stored in RAM. I can't be totally sure yet, I would have to delve in a bit more. It's quite possible that it's doing some magic and only  loading the bits you request.

On a side point, when scanning through the headers to try and find the relevant structs, it turns out that there's a NPK read callback, can possibly tie that into an animated loading screen system... hmmmmmm smile

123

Re: package file request

I would be surprised that it's taking much ram, it's using zlib, but we should test it yes. With an example heavy in textures we can compared maratisPlayer used ram with package and without.

For the loading, good, I should also add a callback in maratis, at least to call lua and update the window.

Re: package file request

Unfortunately I was unable to do much testing during my lunch break as I had to sort out the billing on my website. I did, however, have a quick look through the npk code. Much as I'm still not 100% sure what it's doing, I've changed my mind and I think that the package isn't actually loaded. npk_read seems to open and read chunks of the package every time. So that would mean that leaving packages "mounted" should be acceptable. Will try and test further when I have time.

Re: package file request

I had a quick fiddle with the package manager. I added the package key to the game plugin, the way I'm doing it at the moment is passing a c-string and then splitting it into substrings, then using a hash function to create up to 4 ints and store it as the tea key. As StartPlugin is called before attempting to load a package, or before publishing, that means the key will have been set. I'm not entirely happy about using the hash function yet. I can't think of a better plan right now though. Ideas?

Also, I had a look at how to save the mesh file. Should I just load every xml file again and then write it out again? As there are potentially multiple levels, I think it might be problematic to only save out the meshs loaded, right?