thinmatrix
patreon
September Update - Save System Changes
🕑 Added 2021-09-30 16:13:58 +0000 UTCHey everyone!
It's been another decent month of progress on the city-builder game, although I did get a little bit stuck on some issues regarding saving and loading the new activities. After a few unsuccessful redesigns of the serialization code I finally made the breakthrough last week and it's all working smoothly now. For those who are interested, I'll explain the problem and my solution in a bit more detail below.
The latest downloads for the game and the code are available in the posts below.
The Save System Previously
The system I had used for saving and loading town data before now had been rather simplistic. When the user saved the game it would go through the entities (buildings, people, cars, and any other objects in the world) one by one and write them to the save file. To write an entity it would basically just write all of that entity's important attributes one after another. To load a town, the opposite would happen, and the data would be read back in (in the same order that it was written) and the entities reconstructed from that data.
One slight complexity came when an entity referenced another entity (for example, a person entity may have a reference to the car that he's currently driving). When saving a reference to another entity, the other entity's ID would be saved. On loading, that ID would be read-in and could then be used to get the corresponding Entity instance.
This introduced the slight issue that the referenced entity might not have been loaded up yet, if it was saved further down in the save file. To solve this I implemented an entity requesting system, which allowed requests to be sent for entities during loading. If the entity had already been loaded then the request could be carried out straight away. If the entity hadn't been loaded yet the the request would be kept for later. After each entity was loaded, any requests that were waiting for it get carried out. (A request typically just involves saving a reference to the entity).
The Problem with Activities
The save system had worked perfectly well for the simple objects that had been saved up till now. However, the activities posed more of a challenge. The activities have a more complex tree structure, where each activity can be broken down into sub-activities, and each of them broken down into more sub-activities, etc. Moreover the activities often referenced quite a few other entities (for example, the "eating at a restaurant" activity might reference the restaurant, the current table/seat, the waiter, etc.) and these references were often shared between many of the sub-activities in an activity.
Saving the activities was no problem - I could just iterate through the tree structure and save the state of each activity node. Loading the data back up and reconstructing the activity tree proved to be more difficult.
When creating an activity node I would load up its attributes and request any entities that it referenced. I then wanted to create its child activity nodes and load up their state, but they sometimes needed the referenced entities to be created, and those entities may not be created yet. So I could make a request for those entities and when all the relevant entities are ready I could create the children activities and load their state - except I couldn't do that because everything has to be read in from the save file in the same order that it was written, and the activity children may have been saved before the other required entities. Long story short, I couldn't create an activity node and load its state until the entities it depends on are ready, but the data for the activity node had to be loaded from the save file before the data for the required entities could be loaded. Something had to change!
The Solution
I was hoping I wouldn't have to completely redo the save system, so I tried a few different slight redesigns but all of them ran into difficulties with the activities. In the end, the solution I've settled with changed up the format of the save data in the save file. Instead of one long stream of data, activities are now saved in a tree-like structure to mirror their own structure. It's a bit similar to a JSON format, but without the attribute names (so the save file size hasn't had to increase by much). Each sub-activity is represented by one node in the tree.
The main aim of this approach was to remove the requirement for reading the data in a specific order. Now when an activity (or sub-activity) is to be loaded, the data in the tree structure is pre-loaded and given to the activity-loader. The activity-loader is then free to process that information in its own time, while the rest of the loading can continue. It may be able to load up the whole activity tree straight away if all the relevant entities have already been loaded, or it might only be able to load up part of the tree to start with, or none at all. Then while the other entities are being loaded up, the activity trees will slowly start to be created, as and when the relevant entities become available. The entity requesting system works at the heart of this process, so every time a new entity is loaded it carries out the requests that were waiting for it, and relevant sub-activities that depended on it can be instantiated.
Plan for October
The save system is working very smoothly once again but I must admit there are still a few messy parts in the code that need tidying up, so I plan to spend one or two more weeks just finalizing the activity system. I don't mind spending so much time on this, seeing as it's probably going to be the system I work with most frequently in the future. Every new behavior that I want the people to have will require new activities to be created, so I need the system to be as easy to work with as possible.
For the rest of October I'll be doing more work on the roads. Firstly, I want to improve the placement tools because placing the roads one by one is rather annoying! There's also a bit of code restructuring to be done to allow for multiple road types, and I may also have a go at adding some simple road markings while I'm at it. I'll be making the next devlog video when I start work on these new road updates.
That's it for this month! Thanks for reading and thanks as always for your great support :)
Karl
Comments
KC Oaks
I came to suggest something similar to what Kenneth suggested. I have an application that also requires serializing object graphs. I write out all of the distinct entities, each with an identifier, in a flat list and store the relationships separately. Upon loading, I load all of the entities, build a dictionary, and then create the relationships by looking up each (already instantiated) entity.
ThinMatrix
Ah I see, that makes sense
If you generate and store the identifier immediately before serializing the entity, then when you are about to serialize the reference to the first entity in the second entity, it will already have the identifier ready to go.
ThinMatrix
Oh that's clever, I hadn't considered that! But how would it work with a circular reference, if two entities both reference each other?
Nice write up! Another way to tackle the reference issue, is to serialize the entities instantly when you hit a reference, with a unique identifier, and then only serializing the identifier for any subsequent reference. This way you're guaranteed that the entity is already loaded when it is needed during deserialization. Can't wait for the next devlog!
