Writing my own DBMS: Storing databases

I've been interested in writing something meaningful in C++ for a long time, yet I had a hard time not picking Java or C# for a new project... Now I've found something for which C++ seems to be the right tool: Writing a Database Management System.

I'm going to show you the code for the first step: Storing databases.

The code is written in Microsoft Visual C++ 17 and I'm trying to have code that is as modern as possible. I also didn't intend to build a serialization library, it just happened. This library exports its methods through a DLL.

stdafx.h

#pragma once

#include "targetver.h"

#define WIN32_LEAN_AND_MEAN             // Exclude rarely-used stuff from Windows headers
// Windows Header Files:
#include 

#include 
#include 
#include 
#include 
#include 

#ifdef DBMS_EXPORTS
#define DBMS_EXPORTS_API __declspec(dllexport)
#else
#define DBMS_EXPORTS_API __declspec(dllimport)
#endif

targetver.h

#pragma once

// Including SDKDDKVer.h defines the highest available Windows platform.

// If you wish to build your application for a previous Windows platform, include WinSDKVer.h and
// set the _WIN32_WINNT macro to the platform you wish to support before including SDKDDKVer.h.

#include 

DBMS.h

#pragma once

#include "stdafx.h"
#include "Database.h"

namespace DBMS
{    
    class DBMS
    {
    public:
        DBMS_EXPORTS_API DBMS(const std::string file);

        DBMS_EXPORTS_API Database& CreateDatabase(const std::string file, const std::string name);

        DBMS_EXPORTS_API std::optional> GetDatabaseByFile(const std::string file) const;

        DBMS_EXPORTS_API std::optional> GetDatabaseByName(const std::string name) const;

        DBMS_EXPORTS_API void DeleteDatabase(const Database& database);

    private:
        const std::string file;
        std::vector databases;

        void SaveToDisk() const;
    };
}

Database.h

```
#pragma once

#include "stdafx.h"

namespace DBMS
{
class Database
{
public:
DBMS_EX

Your code appears technically functional, but doesn't actually appear be very useful in storing any data.

• It appears to have a suitable database file identification system

• It appears to have a serialisation layer

• It does not appear to have any kind of query layer stub

• It does not appear to have a file layout layer stub

• It does not appear to have file consistency / rebuilding mechanism stub

Note that depending on what your objective is then you should consider the following:

• Relational DB => very, very difficult

• Variable length data => need to build indexes

• Sql or other text input => need a language parser, query constructor and evaluator

• In place database file modification (avoiding entire read in and write out of the database) => need to be able to have intertwined, extensible and space managed (to handle any deletions and insertions) data structures.

You absolutely can greatly increase performance / reduce server cost if you happen to have a data flow that can be optimised for; but at the cost of being locked into an inflexible, unproven data storage mechanism that you have to make and maintain yourself. I have made the following (propriety code) which had much higher performance than the general purpose database it replaced.

• append to /selection from tables only

• single pass table generation

• single file per table or index

• variable sized data using an index

• index re-generation though self describing parsable tables

• tables contain aligned (with file system efficient storage block multiple) framed data with CRC checks for corruption recovery

• chunked indexes with CRC for corruption recovery

• eventual commitment to disk (in memory write back single point of truth cache) with extent locking for concurrent usage with block writes for throughput performance

• all appends on a table are transactional and sequenced

• queries are asynchronous and not blocked by each other or appends)

• tcp binary database server implementation with non-text class based API

• live snapshotting for incremental backups with throttling disk bandwidth

• written in C#

StackExchange Code Review Q#155879, answer score: 2