Servers

I've been giving thought to what kind of servers I want to see in this engine, lately. The reason for so many servers is to allow the world to be (infinitely) expandable, limited only by the available hardware (and not the speed of that hardware).

Some games (Ultima Online), split their world into different parts, having a server run each part of the world. These splits are static (same for every Ultima shard) and noticeable by the players: the client "skips" a little when you cross a server boundary; NPCs (until recently, it seems) couldn't cross them; and information across the boundary was incomplete (PCs/NPCs would "echo" their location on the other side). Everquest, from what I hear, has things even more blatant - you actually cross into a new zone, where the screen blanks/blinks/loads and you're in a new area, and nothing but players can cross this boundary (and it's not seamless).

I want to avoid these problems. They break immersion; when you play a massively-multiplayer game, or are in a virtual world, you want it to be a world, not a bunch of disjointed rooms or areas. Additionally, while the technical reason for having multiple servers is most likely to distribute load for many players, these methods fail if all the players get together in one area (for an in-game event, for example). Optimally, then, I'd like to have a system where, given enough resources (read: money), one could just keep adding machines to a world, and allow it to use the available hardware to share out the work. No fixed boundaries - they could be moved around, either at server-setup time, or maybe dynamically?

This is what I've come up with for now. It might change, as I might decide more servers are needed, or that, perhaps, some are not. If I had some sort of chart-drawing program, I'd post a picture, but for now, I hope I can describe the doodle I have in front of me well enough to make it understood.

Login servers

These servers are the only servers exposed to the outside world. They are connected to by the client, and are the gateway for all commands from the client to the virtual world. These servers do authentication, character creation, and perhaps account management. They provide the list of worlds (if there are more than one), and communicate further in the "chain" to the client server.

You can have a single login server (if your virtual world is small), or you can have multiple ones; in the case of more than one, there would probably be a proxy at the "front" of them all to distribute the client connections evenly amongst all the login servers. A client would connect to the one login server and stay connected to it for the life of their session. Also, the client server that the login server connects to would remain the same for the life of the client's session.

The number of login servers would be a function of how many (expected) users the worlds have (connected at any one time). More can be easily added, as only the proxy would need to know to add it to its list of possible choices. The login server needs to have fast throughput, as it's funneling commands to and from the client, but wouldn't need much in the way of actual processing power, or storage.

Client servers

Client servers can be thought of as clients, in that they do everything on behalf of the player, if only we could trust them not to cheat. At this point, we believe they are who they say they are (authenticated). In the client server, we would verify commands from the player (too many commands sent at once, etc.), and decide whether or not these commands were valid (permission to pick that up, close enough to pick it up, etc.) A client server can talk to more than one compute server at once, depending on how they're implemented, and possibly the world server.

You can have a single client server (if your virtual world is small), or you can have multiple ones; as with the login server, you would probably have a proxy for the client servers so the connections between the login servers and the client servers are shared evenly to distribute load. Multiple login servers can connect to a single client server, so they client server acts as multiple players. The client server could be on the same machine as the login server; that is, they could be two separate processes running on the same hardware, communicating through IPC (or TCP/IP loopback). They could, too, be the same process, though this prevents scaling.

The number of client servers would be a function of how many (expected) users the world has (connected at any one time). Where the login servers had to account for every player on every world, client servers just need to account for players on a single world. More client servers could be added, and the proxy would just need to be configured to know about them. The client server would need to have fast throughput, as it's funneling commands much as the login server is, as well as decent processing power, as it might have to do some sanity checks on the data received from the client (by way of the login server).

Compute servers

These are the core processors of the virtual world. They move NPCs, they trigger events, and they handle player actions. Really, they run scripts for the most part, and communicate with the world server.

You can have a single compute server (if your virtual world is small, or you can have multiple ones. They might be partitioned so one (or more) run the NPCs, another (group) for triggered events (spawn of monsters and resources), another (group) for handling client interaction, etc. Or, one (or more) might handle area A of the world, another (group) area B, and so on. Compute servers might also communicate with each other, to let one know that an object has moved between areas. In this way, the compute server acts as a client server, "requesting" that this object be allowed to step here or be placed there. The compute server, too, could be on the same machine as the client server (and the login server), if the world is small enough, each running as a different process. This could speed communication between them, again using IPC, local TCP/IP, etc.

The number of compute servers would depend on the complexity of the world; how large, how many events happen at any given time, how many triggers are waiting to be run, how many NPCs are meandering, etc. Adding another compute server would probably require bringing the world down to reconfigure which one handles which tasks, whereas login servers and client servers could probably be added live (where their proxies were just "refreshed" with the new server's information). While it's possible to design the compute servers so they, too, could be added dynamically, "live", that would require a lot of extra work. Indeed, allowing the compute servers to change their workload on-the-fly would be optimal, but a much bigger coding task.

World servers

This is basically the world database. It knows about every object, and every property about them. It backs up the data, it provides the data to the client servers, it changes the data on behalf of the client servers. It, most importantly, locks the data when two client servers are trying to affect the same object at the same time. It is nothing but a specialized database. Maybe even not that special... could be just mySQL, Oracle, or something similar.

You would likely have just one world server. While it could be implemented on any number of machines, perhaps for redundancy, perhaps to share the load (player information there, NPC information there, item information here), but the point-of-view to the compute servers would be of just one central database of information about the world. Whatever the internals of the world server, it would be its responsibility to keep the data sane: to never "lose" an object, to never allow duplication of an object (duping), etc. The world server, could, as before, be on the same hardware as the compute server (and the client server, and the login server...) if the world is small enough. Communication could then be improved in a world that's small enough to allow one physical machine to handle all these processes, but of course a world that small may not need such performance. It's the large-scale worlds which require multiple machines that also require efficient communication between them.

To be continued...