revghost v2

[guerro323]

A very big rework of revghost is coming (previously GameHost), which will focus even more on the ECS Host aspect.
The Host ECS has been swapped from DefaultEcs to flecs, which provide more features (and even an explorer!) and performance improvement that will be useful in the long run.

What is kept:

• Module centric development
• Hot reload of modules
• Storage concept (but converted to a more ECS representation)

What has been removed:

• Domain (it will be more favorable to create nested worlds)
• OOP AppSystem and Module
• OOP Dependencies (use traditional ECS patterns for resolving dependencies instead)

Roadmap:

• Finish the C# wrapper
• Finish the source generator for the wrapper
• revghost features
  • external module load/unload
  • bootstrapping
  • storage

[guerro323]

The C# wrapper is almost finished, the most important features that needs to be completed for it to be considered done are:

• Observers (systems that react to events such as adding components)
• Processors (a system that can only be executed in a system, wasn't present in flecs, but it is a necessity here since otherwise nested queries wouldn't be possible)
• More tests to help completing the required features

[guerro323]

Processors aren't an obligation for now, since I replicated the needs for them with using foreach for nested queries (and it works well!)

[guerro323]

Observers are now coded in.

Before releasing the wrapper, I'll remake the Low Level part use my own bindings and include binaries of flecs (so that it can be used on windows and macos without compiling it)

[guerro323]

Will be released very soon (approx some hours from now), there are still some bugs that needs to be resolved but they're not gamebreaker

[guerro323]

wrapper release: https://github.com/guerro323/revghost.flecs
work on revghost2 is starting now

[guerro323]

File and storage entities 👀

[guerro323]

The concept for Storage is a bit problematic since I couldn't do it the way I originally wanted (using multiple EcsIsA relations from different storage into one storage).
Instead I'll make it more basic with a hierarchy and priority system:

• There are three static entities: storages, contexts and contexts/root
• storages only contains storage entities
• contexts only contains contexts entities
• Storages need to be added manually by the user, by default a storage will point to the children context, if there was no context then contexts/root will be used
• Contexts entities contains files entities and possible storages (if it was a storage, once opened it will be in storages)
• A file entity can be a file or a folder
• The entities in contexts are added/removed from storages (see bottom for how it work)
• A file entity has a specific name: folder_hash.file_hash

Example of a world hierarchy:

flecs
    | ...
mods
    | ... 
**storages**
    | FileSystemExecutableFolder
    | Mod0
    | Mod1
    | FileSystemUserData
    | ...
    | ServerMod0
    | ...
    | CurrentMap
**contexts**
    | **root**
            | XXXX.YYYY # config/inputs.json
            | XXXX.YYYY # maps/topkek.zip
            | ...
    | player_guerro_skin
           | XXXX.YYYY # config/inputs.json (since the goal of this file may differ from the root context, having it isolated prevent that (for example this file can be material inputs))
           | XXXX.YYYY # tex/skin.tga
    | player_guerro_data
           | XXXX.YYYY # config/inputs.json (a multiplayer game could show the current input configuration of a player! having it isolated here prevent it from conflicting with root/config/inputs.json)
           | XXXX.YYYY # player_skin -> context: player_guerro_skin
    | ...

• Contexts are created manually via code, they can be useful to isolate data from the root context so that other storage can't interfere with it.
• A system in the framework will put the files from the storage:
  1. It first create the files entities from the first storage.
  2. Then on the next storage, it will either create the missing entities or override them

Usage example:

struct Configuration : IComponent {
	public int Value;	
}

var rootCtx = world["contexts.root"];
// Lookup for config.json, and put a placeholder if it either doesn't exist or didn't load yet
var configFile = rootCtx.LookupFile("config.json", placeholder: true);

// Create the reload state of our config entity.
// When the file will get loaded or reloaded we will refresh with these components
var reloadState = world.New()
                  .Add<Disabled>()
                  .Add<IoLoadFileContent>() // ParseJson need to get the content from somewhere!
                  .Add<ParseJsonToStruct, Configuration>();
// Then, create our config entity, set the target to our config file and attach the reload state to it.
// Note: the framework will complain if the target doesn't have a Disabled state.
var config = world.New("config")
                  .Add((IoFile.EntityId, configFile))
                  .Add((IoReloadTarget.EntityId, reloadState));

// pseudo code
while (!config.Has<Configuration>);

var cfgData = config.Get<Configuration>().Value;
// If the user update the file, changes will be automatically be reflected!
//
// Of course if you don't need to have the reload feature
// you can just move the components of 'reloadState' to the 'config' entity:
//
//  var config = world.New("config")
//                   .Add((IoFile.EntityId, configFile))
//                   .Add<IoLoadFileContent>()
//                   .Add<ParseJsonToStruct, Configuration>();
//

[guerro323]

The storage concept has now been finalised and is currently being implemented.

Once done, I'll go work on the module load/unload system, which will be relatively much easier to do.
The biggest difference is that there will be a root module per assembly again (need to register the loadable modules of this assembly) :

// -- as of now we can only load 'flecs modules'
world.Register<MyModule>(); //< only work if you referenced the module in your code

// -- revghost modules
// In your root module:
void Load(World world) {
     // only register the description of the module (so that it doesn't load it)
     world.Register<ModuleDescription<MyModule>>();
}

// -- In any other places (even outside of the assembly)
// Note that this will only load the module on next frame and if it was registered beforehand.
// There will be a warning logged to the console if the module was not registered and a skippable error if the module couldn't be loaded after 3 seconds.
world.New("mods.MyModule");

This goes hand in hand with how flecs and the wrapper work; if the module wasn't loaded/created, a system will pick up on the empty entity and try to see if it match in the register from the name.

Of course this is also gonna go hand in hand with the storage concept since it will utilize it (under the hood the mods entity is a context)

[guerro323]

Small change to the module concept:

• To load a module, you add a ModuleLoadRequest tag to a module description (located in mods_desc.*), which is different from just creating an empty entity with the same module name.

And progression of module loading:

(The only thing left for it to be finished is to add assembly loading, and see if it can unload well)

[guerro323]

Storage progress:

What's left for it to be finished is to make sure that the order is respected (order should be: dependencies->creation)

[guerro323]

Ordering is half here, for now there is only creation ordering

A cool unexpected feature is that you can nest contexts, and also override with a storage that was overriden (for example in my_custom_sub_ctx we can see that my_storage_a overrided the B storage (even though it's not the case on the other context, because we explicitly re-added it)

[guerro323]

I need to investigate why there is an 'infinite' loop on Windows, it seems it come when the storage module get loaded (maybe because of the file system watchers?)