snippetsqlMinor
How to store schema-less data efficiently in a relational database?
Viewed 0 times
efficientlystorerelationaldatabaselesshowdataschema
Problem
I'm trying to get a nice balance of searchability and speed. I'm experimenting with different methods.
Option 1:
Serialize the array as JSON/PHP serialized array and store it in a 'meta' column.
Option 2:
Store keys and values together in a stack.
Option 3:
3 tables: entites, keys, values. Store each key name only once.
Are there any pitfalls with using any of these strategies? I'd like to eliminate some of them if they have insurmountable disadvantages.
EDIT: Added some data to represent schema-less data
Option 1:
Serialize the array as JSON/PHP serialized array and store it in a 'meta' column.
id | name | meta
1 Bob {"city":"GoTown","birthdate":"1980\/8\/14","cat":"Felix"}
2 Alice {"city":"Streamville","birthdate":"1986\/6\/6","dog":"Rex"}Option 2:
Store keys and values together in a stack.
user_id | key | value
1 name Bob
1 city GoTown
1 birthdate 1980/8/14
1 cat Felix
2 name Alice
2 city Streamville
2 birthdate 1986/6/6
2 dog RexOption 3:
3 tables: entites, keys, values. Store each key name only once.
user_id | name
1 Bob
2 Alice
key_id | keyname
1 city
2 birthdate
3 cat
4 dog
user_id | key_id | value
1 1 GoTown
1 2 1980/8/14
1 3 Felix
2 1 Streamville
2 2 1986/6/6
2 4 RexAre there any pitfalls with using any of these strategies? I'd like to eliminate some of them if they have insurmountable disadvantages.
EDIT: Added some data to represent schema-less data
Solution
Some really nasty problems would happen with these tables (idea)
-
Data Redundancy (duplicated data) what you need to keep in synchronisation (and you do you check this because MySQL doesn't have an nice JSON to records function?)
-
You can't force correct values on the database (say goodbye to data integrity and hello to garbage-in garbage-out data) example key birthdate could have value "hello"
And you need some kind of pivot query to fetch the key/values
If you really need dynamic key values (EAV) storage there are some more options:
-
Data Redundancy (duplicated data) what you need to keep in synchronisation (and you do you check this because MySQL doesn't have an nice JSON to records function?)
-
You can't force correct values on the database (say goodbye to data integrity and hello to garbage-in garbage-out data) example key birthdate could have value "hello"
And you need some kind of pivot query to fetch the key/values
If you really need dynamic key values (EAV) storage there are some more options:
- Maybe MySQL 5.6 NoSQL using the InnoDB memcached Plugin
- XML -> query with xpath
- RDF
- NoSQL like MongoDB
- MariaDB dynamic columns and MariaDB also supports json.
Context
StackExchange Database Administrators Q#51256, answer score: 4
Revisions (0)
No revisions yet.