426

(150 replies, posted in Engine)

Basically, when you run the executable, the OS usually copies the program into memory (loading dynamic libraries when needed) and then every function call has an address which the program can get to (for example &fopen will get you the address of the fopen function).

There are 2 ways that you can then intercept library functions, especially ones which are part of the standard C libraries like fopen. The first is to simply wrap it:

FILE* MFOpen(const char* path, const char* mode) 
{
    // some crazy stuff here involving package loading/checking etc
    ...
    else
    {
        return fopen(path, mode);
    }
}

would do the job. The second is to replace the library function with some patching. Basically, when you start the program, you get the address of the original fopen call, replace it with a JMP instruction to your own code. It's usually used for things like debuggers as well as cracking programs where they replace authentication calls on other processes. For what our situation I think it would work quite well though as it would basically replace all calls to fopen in all libraries and route it through our package process, if needed.

Much as the Maratis code could easily be modified to use the first method, it would then also require changing TinyXML and any other libraries which don't have handy buffer loading. On the other hand, the function patching is a little more work, but then the buffer loading would probably be entirely redundant as DevIL almost definitely uses fopen internally, which would, again, be patched through to the custom function.

Having the pak-entity name as the relative path seems most logical. Agreed.

I don't know what the performance penalties might be, but it makes sense that accessing and potentially decrypting/decompressing larger amounts of data might be memory/cpu intensive. Also, usually there's no reason to keep things like frontend data in memory, so maybe worth trashing it on larger projects. Packages could then be mounted on top of each other... with the file loader looking first in the latest package loaded, then the previous one, then the previous one (etc) and falling back to the filesystem if no packages are loaded or the file isn't found in any of them.

Setting the password in C++ seems like a reasonable option, as you load any game plugin in the editor, I should be able to create something I can pull out during the publishing process.

I realise I sometimes have a tendency to over-engineer things. If you think that anything is wrong, or you'd prefer something done in a different way, just say. I chose to patch the function as it would more easily allow addition of libraries later to use the package management, but I don't want to seem like I'm trying to make design decisions on this. Maratis is your baby wink

427

(150 replies, posted in Engine)

I did a bit of coding during my lunch break...
I actually asked the lead programmer on the project I'm working on how he patched a function in our in house tech, it's a couple of lines of code and he's said I could use it, so that's made life a bit easier. There's a couple of problems with it, in that it will currently only work on Win32 (it uses WriteProcessMemory and patches a different JMP instruction in) I had a look and I'm pretty sure that I can get it working on Linux, and I assume OSX without too much of a problem, but I haven't got a clue how ARM assembly works, and I'm going to have to do a deal more reading to make it work on x64 too.

EDIT: I just realised this is Win7 x64. I guess x86/x64 might just work... hmm...

For iOS it shouldn't be _too_ big a problem, everything is packaged withing the .ipa anyway, I mean, it's always nice to have the data packaged, especially as npk can {en,de}crypt the data which would protect against cracking the game, but I think, for now, for any ARM builds, it would be ok to not patch the functions.

Also, what sort of structures should the npk files have? I can duplicate the directory structure inside each package, are there any requirement to have multiple packages so they can be loaded/unloaded on the fly (frontend, ingame, level1, level2 etc) Does Maratis support this at all?

Also, a very minor thing in the grand scheme of things, but what do I call the files? I mean, for now I'm going to be calling it {projectname}.npk. Would it be preferable for this to be obfuscated in some way? something like PKG00001.maratis?
With regards to the encryption, does anyone want it? Do people want to choose their own key, or should I make it generate random keys for each publish? If so, where should it be stored? The logical place would probably be inside the .mproj file, as I don't think that would be inside the package, but if it's in there, it should be in some way obfuscated, otherwise there's no reason for the encryption.

Personally, I think the .mproj should be able to be embedded inside the player executable on publish, possibly not as xml. That would obviously take more work, but shouldn't be too bad.

I hope noone minded me taking a brain-dump here, I just thought I should write down things to get some sort of feedback what people wanted, so I know what I'm aiming towards. Of course, to begin with, I'm not adding encryption, just supporting one package, no x64 etc.

428

(150 replies, posted in Engine)

That's ok, I'll just keep git for my work and merge with the svn repo when I've got something working.

429

(150 replies, posted in Engine)

Back on topic tongue

I didn't get as much done as I would have liked on the train, but I made a start at least. I've also created a git repository fork of the maratis repository and I'm keeping a private repository for me to fiddle in. Partially because I can do mini-commits then and not break anything, partially because I don't have access to the main repo yet (not a hint, honest wink ) but mainly because, if I make any changes to Maratis for the game I'm working on, it makes sense that I wouldn't polute the main engine.

Anyway, for the time being at least, the only changes maratis will have in it will be the npk stuff, so if anyone wants me to stick it on a public repo, I can either put it on the Maratis google code project, or I can make a project myself on either google code or github or whatever.

Oh, and I've done a little bit of rearranging for the git repo, but any merging to/from the svn that would need to be done, I will take care of. I just got rid of the tags/branches directories and moved dev up one level as it was all implicit in git tongue

430

(150 replies, posted in Engine)

I haven't done any apple submissions myself, so I don't know how it works exactly, but as far as I know you just need to send them an .ipa file. What I meant was that you could develop on windows or Linux (or OS, using the editor, press publish which would then create the ipa. If the computer has libimobiledevice it could potentially even push the app to a connected device.

I don't pretend to know exactly how this would work, or suggesting it would be easy to do. I'm also not entirely sure how much I should say about this subject as it is very close to what my work does. Linking in any game plugin to an engine executable should be able to be done with gcc-arm. The rest is just data and can be packaged easily. The only thing that is missing is signing the app with a valid developer certificate which I believe scripts exist for.

This thread is getting off topic, should we migrate to another thread to continue this?

Update on npk: I am sitting on a train, my laptop booted and awaiting my command, I have the design drawn up for the package management. Time to get busy. I have 1.5hrs and only a bearded dragon to distract me. Wish me luck.

431

(1 replies, posted in Engine)

I'm trying to work things out by looking at the source code, but to try to keep an internal consistency, I'll write down here so they can be corrected and seen by other potential contributors.

  • All code files should include the license at the top

  • Name of the original author of the file should be added before the license

  • All classes start with M (for Maratis I assume)

  • Function names are camelCase, starting with lower case

  • Member variables are prefixed by m_ and begin lower case

As far as I can tell, use of singletons is acceptable, but reserved for the engine and hardware devices.

That about right so far?

432

(150 replies, posted in Engine)

I've just found out that TinyXML loads files itself... and guess what... you can only give it a filename. I just had a sinking feeling... but then I remembered that you (usually) compile TinyXML yourself for most projects, so I checked Maratis. Yay, that means that overriding fopen etc will work smile

Relief!

433

(150 replies, posted in Engine)

Yes! Thanks. The Editor in prod worked fine. The Player still throws the segfault. If I can figure out how to build a debug version I will take a look at why it does that. The precompiled player works though, although on my underpowered netbook with it's Intel 945GM graphics chip, it doesn't like any shaders.

Anyway, I figured out how to add 3rd party libs and have it linking in a build of npk which I built. Now to the fun!

434

(150 replies, posted in Engine)

Right. This is annoying. My laptop had been playing up with a couple of the packages I had installed as I hadn't updated it for a while. Subversion was expecting some libraries which weren't there and when I got them, more things broke. So I finally did a full update and now X doesn't start at boot and I can't connect to wifi networks. The good news is than svn works fine and I have the scons building maratis... sort of... it's now complaining it can't find XGrabKeyboard. This isn't going well. At least I have internet by tethering my phone...

I will probably give up on this in a bit, and then possibly try to beat it into submission on the train on the way back to London tonight. I have yet to get maratis compiling under any OS nicely sad As soon as it's sorted, I'll be rolling.

EDIT: I didn't give up. It was one of the examples (ManualUse) which didn't have X11 added to the libs, Added that to it and it worked fine. MaratisEditor and MaratisPlayer also needed X11 magic as well as dl. Yay. One step closer

EDIT EDIT: Nope. MaratisEditor now says the following:

[nistur@IMP dev]$ build/linux2/release/Maratis/Editor/MaratisEditor
AL lib: pulseaudio.c:612: Context did not connect: Access denied
GL_VERSION : 1.4 Mesa 7.11.2

and then just closes, whereas MaratisPlayer just segfaults. Woo.

MarKill - If I remember correctly, Apple used to have a total no-go on interpreted scripts for submission to the app store because they couldn't verify the safety of the app properly. They also disallowed any package that wasn't compiled and built on an official version of OSX with XCode. When Epic released their Citadel demo, it was clear that it had scripts and was developed with the Unreal toolchain. They couldn't disallow Epic Citadel as it provided them with a lot of publicity for mobile gaming, so they had to change their app store agreement. What they changed it to was that they would allow interpreted code for minor parts of the application, obviously leaving it vague. I can't remember the exact wording as it's been a while since I read it.
Anyway, the point was that Apple _probably_ wouldn't have anything against publishing the application from Windows. The action iOS package is fairly straight forward if I remember correctly, just a compressed directory with a set structure.

435

(150 replies, posted in Engine)

I've taken a look at DevIL and libsndfile and they both _do_ accept buffer inputs. So that shouldn't be a problem at all. DevIL asks for "Lumps" which is quite straight forward, libsndfile has a virtual file open thing which has some instructions on the API on how to use it, but wasn't immediately apparent. Regardless, they're now upgraded to "nice libraries" in my books smile

I'll maybe start with the npk integration tomorrow.

436

(150 replies, posted in Engine)

I looked at the PVRTC headers at work a few months ago when I was integrating a flash player for our frontend GUI system. I don't know what the license for PVRTC is, but it seems very simple to use. I'm currently not overly interested in doing much iOS stuff myself, especially as I believe with my contract, I'm not allowed to work on anything in competition with products developed there, so I am not sure whether I can offer to do much in this line of work. If I get some work done on the publish functionality using npk, I will try my best to leave it as open and easy to extend as possible.

As for publishing to iOS, it shouldn't be a problem at all, as Apple have now practically dropped half of their restrictive clauses in their agreement since Epic released the Citadel demo (technically they can still say no if they are feeling pedantic about interpreted code or being built on anything bar OSX) Especially with the format of Maratis, as long as the engine binary was compiled against the iOS SDK, data could easily be packaged under any OS, and even the game plugin could be cross compiled against the engine. Maybe this is a bit off topic for this thread, but I can't see why it couldn't be done with a little work.

437

(150 replies, posted in Engine)

Ahh yes, my pet hatred, libraries that expect a FILE* or const char* to do their dirty work. I found a nice image library a while ago which took a buffer and was using that in my engine. I will take a look and see what I can do with DevIL and libsnd.

You're entirely right with the package button. I had never thought of that because I've not worked with an editor before apart from when I was fiddling with big engines.

I am not suggesting doing it at this stage, but the publish button could also run something like NSIS to build an installer. The same with iOS deployment. So many possibilities. One step at a time though.

I had a look at the sample code and I agree, it looks like a very nice library. Let's see if everything else plays ball tongue

438

(6 replies, posted in Gossip)

I've been asking around for artists, so with any luck, I can have something together soon. I still need a name for the game so I can get a fitting domain name, but that will all come in time.

Also, much as the majority of my "free time" will be working on my game, I still have a love of side projects, apart from anything else, if I've been wrestling with a particularly gnarly problem for a while, it's nice to change what I'm looking at. For this reason, I can try and offer my services, primarily to try and add features to Maratis, with Anael's permission of course, but if anyone has anything that they might want adding, give me a shout and if I've got time, I'll give it a go. I'm interested in engine development, hence me trying to write my own, and have some experience through work with integrating 3rd party libraries.

EDIT:I've just reread what I've written and I'm sorry if my posts sound conceited. I'm not trying to just come in here and say "look at me, I'm awesome, I will do everything" I am far from competent enough to make something so complete or robust as Anael seems to, but I'm willing to learn and help where I can, and I definitely want to help contribute to this project smile

I just thought I'd say a quick hello to this forum.

My name is Philipp Geyer. I'm a game programmer. I graduated University of Abertay, Dundee a year and a half ago (Summer 2010) and started working for Ideaworks Game Studio in London. Since then, I've worked on 4 iOS titles including, most recently, Call of Duty: Black Ops Zombies.

In my "free time" or what little I have, I enjoy programming and fiddling around on my own personal projects. Much as I have little distractions, mainly in the field of Memetics where I'm trying to write a small library to allow evolution and dispertion of information within games, the main project I have been spending my time on is a game.

Due to a couple of reasons, the game will be targetted towards Windows/OS X/Linux with the possibility of handhelds and other devices coming later. The original plan was that I would develop a game engine along side the game, with some friends doing the gameplay programming, however, I have since given up on that as it appears I cannot trust any of my friends to do any work. I am not insane enough to believe that I can do both engine development and gameplay programming, hence me looking for an engine to use and coming across Maratis3D.

The schedule for my game was disrupted, unfortunately. The plan was to have my engine in a usable state for the game to begin development in earnest in Early September. Until that time my team and I have been busy with design work and have a good deal of game design and plot design worked out and written up. We also commissioned a couple of pieces of concept art. September, unfortunately, was the beginning of almost 3 months of crunch time for me and my friends were starting their last year at university.
TL;DR: Busy busy busy. Game was put on hold.

As the game was recently released and the first scary updates are out, I'm no longer quite as stressed at work, so I am resurrecting my project.

A little more about me. I use Linux predominantly and, while at university, have done several talks about gaming on Linux, both what games are available and what's available to develop games easily for it. I have also fiddled around with as many different hardware and software platforms that I have been able to get my hands on. I wouldn't say that I'm confident with all of them, but I'm at least familiar. I am a big fan of free and open source software, but as of yet, have not contributed, or released any myself. I do intend to try and give back to Maratis though whenever I can.

Finally, I'm just looking for a 3D artist to be able to bring my concept art to life, until I have something I can work with and play around with, I will be poking around the Maratis source code some more and getting more comfortable with it. When I have something visual up and running, I will probably start a devblog and link to it from the forum, adding pretty pictures to my posts smile

If anyone has any questions, give a shout. I hope I can give as much to the Maratis community that Maratis looks like it will give me.

440

(150 replies, posted in Engine)

My first post!

I found Maratis last week and have spent the past few days looking over the source, the examples, the tutorials and the forums. I've also emailed Anael about using Maratis for my own project. As it stands, this is the only bit which, from what I can see, is missing in order to be ready for publishing a game, is the package format.

Anael, it would seem, has already done a lot of research and has suggested "npk". I'm not sure if he's got it planned for integrating it right away, but I thought I'd just (re)start the conversation here so can discuss how to go ahead with this and possibly take over the integration of this library.

First of all, from what I can see at a quick glance, there are only a couple of places where files are loaded (a quick find/grep shows 8) so it shouldn't be too difficult to keep on top of those and override fopen/fread/fwrite with ones that access npk packages, Falling back to the normal filesystem if there's no package available.

Which then leads to "how to make the packages". NPK comes with a command line utility which allows creation of packages, but that doesn't exactly fit with the flow of the engine, which seems to be "things should just work" and also would cause problems if you updated an original file. What I think would be the way to go is to look for a package (probably named the same as the level?) and then, if it exists, check for a file inside it. If the file doesn't exist, or the timestamp is older than a file in the filesystem, then add the new file to the package, then load and return that.

I probably shouldn't have written such a long post. I hope noone minds. If Anael doesn't mind, I'll start looking into integrating this properly.