discuss: consider switching to a proper game library

[andreionoie]

Currently in this project we're working with many low-level data structures, primitive types, battling synchronization, we need to be aware of garbage in hot paths etc. Development could possibly be simplified by using a game development library where all the low level details are already handled and optimized for us, and we are only working with those abstractions. Obviously, the library needs to permit some level of customization to handle any legacy quirks for preserving compatibility.

For example, something like Arch ECS https://arch-ecs.gitbook.io/arch/extensions/page-3/arch.system:

• knows about 2D positions and movement
• can separate monster AI behaviour systems
• can separate each player subsystem: Skills, Affects, Refine, Inventory, Guild, Marriage etc.
• is already optimized for performance in hot 100Hz loops

[MeikelLP]

I thought about something similar but with the Godot game engine. Not really used as a library but rather as a framework to build on top of especially for the collision stuff. This would also make it easier to implement real 3D maps.

Additionally I thought about implementing a Godot game client to add automated e2e testing. But that's just an idea for now.

However for now I'd like to focus on building the thing itself and not fiddle around with the technology all the time. I don't care about performance, memory leaks, etc for now. I'd like to get it more done and then change the baseline. I know it's probably harder but right now QCX is basically useless. It needs to add more features so it becomes a minimum viable product and not some tinkering project.

[andreionoie]

It's a nice idea, I think many parts of the codebase are good (packet definitions and SerDes, parsers, Auth server, DB / Redis abstractions, etc), but I don't know how feasible it is to make an actual playable MVP with the current architecture because:

• bug: Thread-safety issues between packet handlers and entity state #385 is a major issue which sometimes kills the connection on concurrent modification of NearbyEntities while iterating
  • would we really want to clone those collections and say that "we will refactor and optimize that later"?
• client gets spammed with packets and responds slower as game time accumulates (feature: Packet sending queue throttling / update coalescing #363)
  • the only fix I found was to do a full reconnect of the character (after 30 min in game it gets fully unplayable)
• inheritance mess: PlayerEntity inherits Entity, yet Entity methods define custom behaviour if its type is PlayerEntity (see Entity::Damage)
• poor codebase scalability: it would get pretty much unworkable if we directly pile on all the other missing systems for an MVP, without good abstractions

There needs to be some vision of how the base architecture of the GameServer should look like to be completely thread-safe and never throw exceptions on normal gameplay, and how all the systems can be written with modularity and testability.

For example the stat logic is very complex with dependencies between base stats and computed stats through affects and equipment; think about how you need to clamp HP and SP back to MAX_HP/SP whenever you take off earrings. You can implement this the simple way (and in my opinion buggy and not testable) by changing Entity.Health directly in the unequip method. Or you can implement a custom StatSystem which can generically hold definitions for all possible behaviours and dependencies, and then you test it in an isolated way with a few dummy stats only.

public static StatsBlueprint ForPlayer(Job job)
{
    var blueprintBuilder = new StatsBlueprint.Builder()
            .Core(EPoint.St, max: PlayerConstants.MaxCoreStatusPointValue, allowModifiers: true)
            .Core(EPoint.Ht, max: PlayerConstants.MaxCoreStatusPointValue, allowModifiers: true)
            .Core(EPoint.Dx, max: PlayerConstants.MaxCoreStatusPointValue, allowModifiers: true)
            .Core(EPoint.Iq, max: PlayerConstants.MaxCoreStatusPointValue, allowModifiers: true)

            .RuntimeCappedBy(EPoint.Hp, capPoint: EPoint.MaxHp)
            .RuntimeCappedBy(EPoint.Sp, capPoint: EPoint.MaxSp)
            .RuntimeCappedBy(EPoint.Stamina, capPoint: EPoint.MaxStamina)
            .Runtime(EPoint.Level, max: PlayerConstants.MaxPlayerLevel)
            .Runtime(EPoint.Gold, max: PlayerConstants.MaxPlayerGold)
            .Runtime(EPoint.Experience)
            .Runtime(EPoint.StatusPoints)
            .Runtime(EPoint.Skill)
            .Runtime(EPoint.SubSkill)
            .Runtime(EPoint.HpRecovery)
            .Runtime(EPoint.SpRecovery)

            // modifier-only collectors (zero or constant base + list of flat modifiers summed)
            .Derived(DerivedApplyTypePoints)
            .Derived(EPoint.MinWeaponDamageInternal)
            .Derived(EPoint.MaxWeaponDamageInternal)
            .Derived(EPoint.PartyTankerBonus)
            .Derived(EPoint.PartySkillMasterBonus)
            .Derived(EPoint.PartyDefenderBonus)
            
            .Derived(EPoint.AttackSpeed, constantBase: PlayerConstants.DEFAULT_ATTACK_SPEED)
            .Derived(EPoint.MoveSpeed, constantBase: PlayerConstants.DEFAULT_MOVEMENT_SPEED)
            .Derived(EPoint.CastingSpeed, constantBase: PlayerConstants.DEFAULT_CASTING_SPEED)

            // derived (computed) stats - accepting modifiers, can have dependencies for the base value computation (through ctx)
            .Derived(EPoint.MaxHp,
                ctx => job.StartHp + job.HpPerHt * ctx.Get(EPoint.Ht) + job.HpPerLevel * ctx.Get(EPoint.Level))
            .AggregateWithPercentageBonus(EPoint.MaxHp,
                pctBonusPoint: EPoint.MaxHpPercentage, pctMaxBonus: PlayerConstants.MaxHpPercentageBonus,
                postPctFlatBonusPoints: EPoint.PartyTankerBonus)

            .Derived(EPoint.MaxSp,
                ctx => job.StartSp + job.SpPerIq * ctx.Get(EPoint.Iq) + job.SpPerLevel * ctx.Get(EPoint.Level))
            .AggregateWithPercentageBonus(EPoint.MaxSp,
                pctBonusPoint: EPoint.MaxSpPercentage, pctMaxBonus: PlayerConstants.MaxSpPercentageBonus,
                postPctFlatBonusPoints: EPoint.PartySkillMasterBonus)

            .Derived(EPoint.MaxStamina,
                ctx => job.MaxStamina + job.StaminaPerHt * ctx.Get(EPoint.Ht))
            .Derived(EPoint.MinAttackDamage,
                ctx => ctx.Get(EPoint.MinWeaponDamageInternal))
            .Derived(EPoint.MaxAttackDamage,
                ctx => ctx.Get(EPoint.MaxWeaponDamageInternal))

            .Derived(EPoint.AttackGrade, ctx =>
            {
                var levelBonus = 2 * ctx.Get(EPoint.Level);
                var jobBonus = (ctx.Get(EPoint.St) * 4 + ctx.Get(job.AttackStatus) * 2) / 3;
                return levelBonus + jobBonus;
            })
            .AggregateWithFlatBonus(EPoint.AttackGrade, flatBonusPoints: EPoint.AttackGradeBonus)

            .Derived(EPoint.MagicAttackGrade, ctx =>
            {
                var iq = ctx.Get(EPoint.Iq);
                var level = ctx.Get(EPoint.Level);
                return level * 2 + iq * 2;
            })
            .AggregateWithFlatBonus(EPoint.MagicAttackGrade, flatBonusPoints: EPoint.MagicAttackGradeBonus)

            .Derived(EPoint.DefenceDisplayBaseInternal, ctx =>
            {
                var ht = ctx.Get(EPoint.Ht);
                var level = ctx.Get(EPoint.Level);
                return level + ht;
            })
            // EPoint.Defence is for client display only; the aggregated EPoint.DefenceGrade is used for calculations
            .Aggregate(EPoint.Defence, agg => agg
                .WithBase(EPoint.DefenceDisplayBaseInternal)
                .WithFlatPostPercent(EPoint.DefenceGrade, StatValuePortion.ModifiersOnly)
                .WithFlatPostPercent(EPoint.DefenceGradeBonus)
                .WithFlatPostPercent(EPoint.PartyDefenderBonus)
            )

            .Derived(EPoint.DefenceGrade, ctx =>
            {
                var ht = ctx.Get(EPoint.Ht);
                var level = ctx.Get(EPoint.Level);
                return level + ht * 8 / 10;
            })
            .AggregateWithFlatBonus(EPoint.DefenceGrade,
                flatBonusPoints: [EPoint.DefenceGradeBonus, EPoint.PartyDefenderBonus])

            .Derived(EPoint.MagicDefenceGrade, ctx =>
            {
                var ht = ctx.Get(EPoint.Ht);
                var iq = ctx.Get(EPoint.Iq);
                var level = ctx.Get(EPoint.Level);
                return level + (iq * 3 + ht) / 3;
            })
            .AggregateWithFlatBonus(EPoint.MagicDefenceGrade, flatBonusPoints: EPoint.MagicDefenceGradeBonus)
        ;   // intentional for prettier diffs; leave like this
        
        return blueprintBuilder.Build();
}

(I think it is pretty much guaranteed the GameServer will be full of bugs without proper testing, and testing behaviour directly on PlayerEntities is much harder and time consuming than testing individual systems)

[MeikelLP]

I know what you mean but I personally struggled a lot with the project when there is no "domain progress" I reworked some systems several times (like packet serialization to eliminate as much allocations as possible) but I drained my motivation multiple times and stopped me working on the project all together.

That's just a personal thing but I think the previous developers had a similar issue. I have no issues of rebuilding the whole base once enough features are packed because we currently don't know what we need. "Premature Optimization" is maybe not valid here but I think the gist is still correct. We don't know what we need to optimize until most features are interconnected imo.

Yes the issues you mentioned are critical and will stop the project from ever going live but it doesn't matter right now because the project is far from ever going live for all possible reasons.

[andreionoie]

So basically we need to find the best priority list. We can add a few more features on top but not too many before an explosion in complexity. Which features or subset of those features would we need to implement for a MVP, to better understand how we will need to refactor later? Affects and skills are pretty straightforward, stats are a bit harder but still workable.

[MeikelLP]

especially the questing system brings a lot of complexity as it's the baseline for most features

[andreionoie]

And what would be the prerequisites for quests? I know at least an affect API would need to be exposed

[MeikelLP]

The whole LUA system would need to be inplace. The legacy scripts must be supported (and they contain a lot of special treatment)