Saving bytes instead of objects to write smaller files

This question is about reducing the size of files saved to disk in an infinite world 2D mining game.

I finally added code to my game to save and load distant chunks instead of keeping all of them in memory. Up to this point, I had only used the default NSCoding Apple library to save and load files from disk. In my initial testing, each Chunk consisted of 900 Blocks, and when saved to disk the file was 160kb. This would mean that worlds with just a few million blocks could be bigger than a gigabyte.

I tried a number of different methods to decrease the size of the files. I found some code on Stack Overflow to implement some gzip compression, and that reduced the size down to about 74kb. This was good, but not nearly good enough. The next thing I tried was to reduce the amount of information stored in each block. I removed all the NSStrings, and derived those from other values instead of saving them. I converted all the CGPoints to ints, and saved those instead. This got the file size down to about 52kb. This was almost small enough to work, but I knew I could do better.

The Blocks themselves only had a very small amount of information stored in them. I decided to manually compose the bytes needed to initialize a block, and then manually parse those bytes when the file was loaded.

I succeeded in writing some code to do this, and it got the file size down to 4kb. Each Block is 28 bytes, so this is much closer to a reasonable number. I should note that when I tried without the gzip compression, the file size rose to 25kb. I decided to use the Apple libraries to do this, so I am reading and writing to NSData. I believe there are better ways to achieve this, and I would love to hear about them.

DWBlock.m

```
#pragma mark - Custom encode and decode
-(instancetype) initWithData:(NSData *)data {
self = [super init];
if (self) {
int theBytesRecovered[7];
[data getBytes:theBytesRecovered];

_type = (i

Do chunkLength = chunkPosY-chunkPosX or length = posY-posX have a max limit?

Does damageLevel, type have a max value say 10.

Did you try using a structure and writing that to the file (as in C). A few benefits include:

• Bit field usage.

-
Easy retrieval and storing instead of storing block metadata.

struct NSData { 
    int type :4; // bitfield usage since the max type is 10(1010)
    int damageLevel:4; // bitfield usage since max damageLevel is 8(1000)
    int posX;
    int posY; // If length = posY-posX has a max level with limited bitfield usage; Use it.
    int chunkSize; //Is this a part of the actual data to be stored.
    int chunkPosX;
    int chunkPosY; // Same as above.
}

I have used structs in C and the size is almost equal to the calculated size, i.e.

sizeof(NSData) * number of NSData

One observation:

Since all your data are numbers, if each item's value does not exceed '999'(3 characters), you can serialize it to a string delimited with any character and save it.

2|5|22|35|123|24|98; // This will occupy less bytes instead of 28 bytes.
    decimal-value delimiter string-end-delimiter. // uses 19 bytes
    2|5|F6|23|7B|F8|62;
    hex-value  delimiter string-end-delimiter. //20Bytes

Note: kb = KiloBits; kB = KiloBytes.

(Unsure how 900Blocks * 28Bytes < 24KB)

StackExchange Code Review Q#67926, answer score: 3