C++ ORM for SQLite

Related tags

Database hiberlite
Overview

Hiberlite ORM

Build Status Build status

C++ object-relational mapping with API inspired by the awesome Boost.Serialization - that means almost no API to learn.

Usage

Just compile and link all files under src/ to your project. Include the main header:

#include "hiberlite.h"

Features

Key features of Hiberlite are:

  • Boost.Serialization like API
  • no code generators / preprocessors
  • support for one-to-many and many-to-many relations
  • automatic code generation
  • lazy loading
  • smart pointers
  • no need to inherit from a single base class

In contrast to most serialization libraries with SQL serializers, C++ objects mapped with hiberlite behave similar to active record pattern - you are not forced to follow the "read all your data/modify some small part/write everything back" path.

For people who need reliable data storage, ACID transactions, simple random-access to their data files, and don't like coding in SQL.


Tutorial

While building a typical RDBMS application developer has to deal with the following tasks:

  1. Open a database
  2. Create database schema
  3. Populate the tables with data
  4. Read and/or modify that data

Hiberlite greatly reduces the implementation of each of these tasks.

Creating a database

Opening a database is as simple as

hiberlite::Database db;
db.open("sample.db");

Where

Database::open(std::string filename)

opens a sqlite3 file. If it doesn't exist it will be created. Another option is to have constructor automatically open/create the DB file:

hiberlite::Database db("sample.db");

Creating database schema

In C++ You deal with classes and objects. You know what objects You want to store in the database. And hiberlite will figure out how to store that data.

Defining data model

First You must prepare the data classes for use with hiberlite. Suppose we develop a social-network application. We define a person class as:

class Person{
public:
        string name;
        int age;
        vector<string> bio;
};

Now, to let hiberlite know about the internal structure of this class, we add an access method and an export declaration:

class Person{
    friend class hiberlite::access;
    template<class Archive>
    void hibernate(Archive & ar)
    {
        ar & HIBERLITE_NVP(name);
        ar & HIBERLITE_NVP(age);
        ar & HIBERLITE_NVP(bio);
    }
public:
    string name;
    double age;
    vector<string> bio;
};
HIBERLITE_EXPORT_CLASS(Person)

Inside the hibernate(Archive & ar) method we list all the data fields we want to persist in the database. Macro HIBERLITE_EXPORT_CLASS(Person) is invoked to register the class name with Hiberlite. For now just remember to invoke it once per class in your project. (Placing it in the Person.cpp is a good idea)

At this point hiberlite is able to map the Person to a database.

Creating tables

Database schema in hiberlite is simply a set of classes, that are stored in the database. A programmer defines that set of classes, and hiberlite determines the needed tables and their columns.

Each instance of Database maintains its own copy of the database schema (set of data classes). To insert a new class to that set, use registerBeanClass template method:

db.registerBeanClass<Person>();

In most applications several classes are stored in one database - so usually You will call registerBeanClass several times:

Database db("gmail2.0.db");
db.registerBeanClass<UserAccount>();
db.registerBeanClass<Letter>();
db.registerBeanClass<Attachment>();

When the classes are registered, Database can create tables to map them:

db.dropModel();
db.createModel();

createModel() executes several CREATE TABLE queries. dropModel() cleans the database - executes DROP TABLE IF EXISTS query for each table in the schema. Besides uninstallers, this method may be used to destroy the old tables (from a probably obsolete scheme) before calling createModel().

Saving data

When the database with proper schema is opened, we can put some Person objects in it:

Person x;
x.name="Amanda";
x.age=21;
x.bio.push_back("born 1987");
hiberlite::bean_ptr<Person> p=db.copyBean(x);
x.age=-1; //does NOT change any database record
p->age=22; //does change the database

copyBean(x) template method creates a copy of its argument and saves it to the database. It returns a smart pointer - bean_ptr.

bean_ptr is inspired by the boost::shared_ptr. The main difference is that in addition to deletion, bean_ptr guarantees that the referenced object will be also saved to the database, when no longer needed.

An internal primary key is autogenerated for each object in the database. It can be read with sqlid_t bean_ptr::get_id().

Another way to create a bean is to use createBean() template method:

hiberlite::bean_ptr<Person> p=db.createBean<Person>();
p->name="Amanda";
p->age=21;
p->bio.push_back("born 1987");

Loading data

There are several methods to query the database:

bean_ptr Database::loadBean(sqlid_t id) loads the bean with the given id.

bean_ptr<Person> p=db.loadBean<Person>(1);
cout << "first person is " << p->name << endl;

In this case object itself is not loaded, when bean_ptr is returned. bean_ptr is a lazy pointer, so the wrapped object will be loaded when first needed. In this example Person will be loaded later, when we try to access the name field.

std::vector< bean_ptr > Database::getAllBeans() returns a vector with all the beans of a given class C.

vector< bean_ptr<Person> > v=db.getAllBeans<Person>();
cout << "found " << v.size() << " persons in the database\n";

In this example objects are not loaded at all.

You can load the same object more than once - all the returned bean_ptr's will point the same bean.

Deleting beans

To remove an object from the database, call bean_ptr::destroy():

bean_ptr<Person> p==db.loadBean<Person>(1);
p.destroy();

Examples

All the above code is put together in sample.cpp. For more demonstration see the poor-mans unit tests : tests.cpp.

User-defined class First we define the class we plan to store in the database:

class MyClass{
    friend class hiberlite::access;
    template<class Archive>
    void hibernate(Archive & ar)
    {
        ar & HIBERLITE_NVP(a);
        ar & HIBERLITE_NVP(b);
        ar & HIBERLITE_NVP(vs);
    }
public:
    int a;
    double b;
    vector<string> vs;
};
HIBERLITE_EXPORT_CLASS(MyClass)

Note the friend declaration and the hibernate(...) template method - these two pieces of code are necessary for hiberlite to access the internal structure of the user-defined class.

HIBERLITE_NVP is a macro that creates a name-value pair with reference to its argument.

HIBERLITE_EXPORT_CLASS() defines the root table name for the class. More on this later.

How it is stored

hiberlite will use 2 tables to represent MyClass instances in the database:

CREATE TABLE MyClass
(
    a INTEGER,
    b REAL,
    hiberlite_id INTEGER PRIMARY KEY AUTOINCREMENT
)
CREATE TABLE MyClass_vs_items
(
    hiberlite_entry_indx INTEGER,
    hiberlite_id INTEGER PRIMARY KEY AUTOINCREMENT,
    hiberlite_parent_id INTEGER,
    item TEXT
)

Table MyClass is the root table for MyClass. It will contain one row per object. Columns a and b store values of the corresponding int and double fields.

HIBERLITE_EXPORT_CLASS(MyClass) macro simply declares that instances of MyClass must be stored int the MyClass root table and its subtables.

MyClass_vs_items table is the "subtable" of MyClass. It is used to store elements of the vs vector: hiberlite_parent_id references the hiberlite_id of the root table, hiberlite_entry_indx - is the zero-index of the string element of the vector, and item is the value of that element.

Objects in the database are called beans. To add a new bean to the database we call

hiberlite::bean_ptr<MyClass> p=db.copyBean(x);

copyBean() creates an internal copy of its argument (by invoking copy-constructor new MyClass(x)), saves it in the database and returns a bean_ptr, pointing to that copy. bean_ptr is a smart-pointer, it will perform reference-counting and care about saving and destroying your bean when it is no longer in use. But the object will not be lost - it is stored in the database. The only way to remove it is to call bean_ptr::destroy().

There are two other ways to create a bean in the database: bean_ptr Database::createBean() creates a bean using default constructor and returns a bean_ptr Database::manageBean(C& ptr) takes control over its argument and wraps it in the bean_ptr. You must not call delete ptr; after calling manageBean(ptr) - bean_ptr will do it when necessary.

Loading data

Call to

std::vector< hiberlite::bean_ptr<MyClass> > v=db.getAllBeans<MyClass>();

will return a vector with all objects in the database.

bean_ptr is a smart pointer, but I forgot to mention that it is also a lazy pointer. Beans itself are not loaded when with getAllBeans() - only their primary keys. To give user the access to the underlying object, C& bean_ptr::operator->() is overloaded. The object will be created and fetched from the database with the first use of operator->.

User-defined primitive types

Sometimes, numbers or strings are not suitable representations for fields of objects. In that case it is possible to extend hiberlite's conversion mechanism to support types other than the ones supported. For example:

given an enum:

enum class Status {
        OK,
        NOT_OK
};

the macro HIBERLITE_NVP will cause a compile error if a member of that type is used with it. In order to convert the enum to a number and back (conversion safety considerations may apply), it is possible to define custom conversions before defining the data model:

namespace hiberlite {
    template <class A>
    void hibernate(A& ar, Status& value, const unsigned int) {
        ar& db_atom<Status>(value);
    }

    template <>
    inline std::string db_atom<Status>::sqliteStorageClass() {
        return "INTEGER";
    }

    template <>
    template <class Stmt, class Arg>
    void db_atom<Status>::loadValue(Stmt& res, Arg& arg) {
        val = static_cast<Status>(res.get_int(arg));
    }

    template <>
    inline void db_atom<Status>::bindValue(sqlite3_stmt* stmt, int col) {
        sqlite3_bind_int(stmt, col, static_cast<int>(val));
    }
}

This allows the following to work:

struct Item {
    Status status;

    template <class Archive>
    void hibernate(Archive & ar) {
        ar & HIBERLITE_NVP(status);
    }
};
Comments
  • Does hiberlite handle inheritance?

    Does hiberlite handle inheritance?

    In my case I have a bunch of classes which inherit from a common base. Can hiberlite handle this? Specifically, can I do the following?

    HIBERLITE_EXPORT_CLASS(Base);
    HIBERLITE_EXPORT_CLASS(Derived);
    

    and then would db.getAllBeans<Base>(); return all instances of Derived as well?

    Thanks!

    opened by wtholliday 7
  • decltype problem

    decltype problem

    Got this error with gcc 4.7.3 and gcc 4.6.3

    sample.cpp:13:8: error: ‘Person::name’ cannot appear in a constant-expression sample.cpp:13:8: error: a function call cannot appear in a constant-expression sample.cpp:13:8: error: template argument 1 is invalid

    The problem is in HIBERLITE_NVP macro on decltype, if I change to macro from

    define HIBERLITE_NVP(Field) hiberlite::sql_nvp< decltype(Field) >(#Field,Field)

    to

    define HIBERLITE_NVP(Type, Field) hiberlite::sql_nvp< Type >(#Field,Field)

    works as a short term solution, I have no idea on how to fix this in order to keep the original implementation.

    Best regards.

    opened by rbarreiros 7
  • travis-ci, catch-test, warnings and errors, premake build script and generated files

    travis-ci, catch-test, warnings and errors, premake build script and generated files

    • continuous integration using travis-ci
    • fixes for various compilers
    • premake and premake script premake4.lua
    • generated visual studio 2012 solution and linux gmake scripts in the Build directory
    • ci-build passes
    opened by d-led 7
  • enums as object fields

    enums as object fields

    how to make enum class (or just old school enums) serializable?

    enum class Gender {
        male, female
    };
    
    class Person {
    
    Gender gender_;
    
    friend class hiberlite::access;
    
        template <class Archive>
        void hibernate(Archive & ar) {
            ar & HIBERLITE_NVP(gender_);
      }
    
    

    gives compilation error Visitor.h:12:4: Member reference base type 'ARGender' is not a structure or union static void hibernate(A& a, C& c) { c.hibernate(a); } I assume that it lacks template specialization similar to POD types like int, etc

    opened by psydevascender 5
  • Biicode integration

    Biicode integration

    Is anyone else interested in making this compatible with biicode. I've got an initial version of the block working on my fork: https://github.com/rharriso/hiberlite

    Travis integration doesn't seem to be cooperating.

    opened by rharriso 4
  • unsigned long long

    unsigned long long

    Hi there, I have a member of a model that I would like to use unsigned long long as the type, but compilation failed at hiberlite::access::hibernate's c.hibernate(a) call because "Member reference base type 'unsigned long long' is not a structure or union". I see that long long is allowed.

    Is it possible to resolve this? Also, is it possible to use enum and map it to an integer type for example? Thanks!

    opened by yuklai 3
  • constant expression in toSQLiteValue(const char* val)

    constant expression in toSQLiteValue(const char* val)

    First, I like your library! :)

    The second line in:

    int n=strlen(val);
    char s[4+2*n];
    

    is invalid though, as n is not a constant expression. You could do:

    char* s=new char[4+2*n];
    ...
    delete[] s;
    

    that will work in all compilers. I've noticed it when compiling with Visual Studio.

    opened by d-led 3
  • trivial test for unsigned long long & build config

    trivial test for unsigned long long & build config

    I noticed that the build config is not working out of the box anymore. Here's an update to premake5 and VS2013, as VS2013 seems to be the current stable version

    opened by d-led 2
  • typeof portability

    typeof portability

    in hiberdefs.h, the HIBERLITE_NVP macro contains typeof(Field). In c++11 mode you could use decltype.

    maybe something like that:

    #if (defined _MSC_VER && _MSC_VER< 1600)
    #include <boost/typeof/typeof.hpp>
    #define HIBERLITE_NVP(Field) hiberlite::sql_nvp< BOOST_TYPEOF(Field) >(#Field,Field)
    #else
    #define HIBERLITE_NVP(Field) hiberlite::sql_nvp< decltype(Field) >(#Field,Field)
    #endif
    
    opened by d-led 2
  • undefined reference for Database class.

    undefined reference for Database class.

    After compiling with gnu I recieve the error undefined reference Database(std::string) using your tutorial, as well as numerous other errors for every single call to the object of type Database. I thought including hiberlite.h would be sufficient. Im trying to compile this on windows using mingw,if that helps.

    opened by roland-hediger 2
  • Build error

    Build error

    I unzipped and tried to build by using the command : make in the root directory of hiberlite on my Windows 10 machine using cygwin. Initially, i get this error: image

    Then, as per the error, in the file BeanUpdater.h, I change the line int n=strlen(val); to int n=mbrlen(val);

    Then, when I try to again build, I get this: image

    I'm very new to C++ and to this project. Please help me out. Thanks

    opened by iamfarazbaig 1
  • DataBase Lock Error

    DataBase Lock Error

    Hi ,

    I'm using hiberlite, There are two tables in database , two process trying to read separate table from database in loop. I folllowed the sample.cpp code. But after few minutes of execution, there is DB LOCK Error. I don't understand, why while doing READ we are getting DBLOCK error. Thanks.

    opened by santosh116 2
  • Getting beans by another key other than the primary

    Getting beans by another key other than the primary

    I want this library could get a single bean by another key ,such as "itemid" or "name" . it seem only one way to get a bean with Database::loadBean function .

    opened by super999 2
  • Transactions

    Transactions

    Please, more control over transactions is required. Create is working too slow because of many transactions. The temporal solution in my case was commenting the following "dbExecQuery"s:

    template<class C>
    void Database::dbUpdate(bean_key key, C& bean)
    {
        try{
    //      dbExecQuery(key.con,"ROLLBACK TRANSACTION;");
        }catch(...){}
    //  dbExecQuery(key.con,"BEGIN TRANSACTION;");
    
        ChildKiller ck;
        ck.killChildren(key,bean);
        BeanUpdater u;
        u.update(key, bean);
    
    //  dbExecQuery(key.con,"COMMIT TRANSACTION;");
    }
    

    But thank you for the useful tool!

    opened by johnkea 1
Owner
Paul Korzhyk
Paul Korzhyk
DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files compatible with SQLite.

DB Browser for SQLite What it is DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files c

null 17.5k Jan 2, 2023
React-native-quick-sqlite - ⚡️ The fastest SQLite implementation for react-native.

React Native Quick SQLite The **fastest** SQLite implementation for react-native. Copy typeORM patch-package from example dir npm i react-nati

Oscar Franco 423 Dec 30, 2022
A project to create a simple ORM.

cpp-ORM Current build status : An ORM project. You can simply create persistent objects using databases. The object representation: Each object have t

Maxime Barbier 5 Dec 14, 2020
SQLean: all the missing SQLite functions

SQLite has very few functions compared to other DBMS. SQLite authors see this as a feature rather than a bug, because SQLite has extension mechanism in place.

Anton Zhiyanov 2.1k Jan 8, 2023
An SQLite binding for node.js with built-in encryption, focused on simplicity and (async) performance

Description An SQLite (more accurately SQLite3MultipleCiphers) binding for node.js focused on simplicity and (async) performance. When dealing with en

mscdex 14 May 15, 2022
Yet another SQLite wrapper for Nim

Yet another SQLite wrapper for Nim Features: Design for ARC/ORC, you don’t need to close the connection manually Use importdb macro to create helper f

Code Hz 9 Jan 4, 2023
A friendly and lightweight C++ database library for MySQL, PostgreSQL, SQLite and ODBC.

QTL QTL is a C ++ library for accessing SQL databases and currently supports MySQL, SQLite, PostgreSQL and ODBC. QTL is a lightweight library that con

null 173 Dec 12, 2022
Fork of sqlite4java with updated SQLite and very basic compiler hardening enabled.

Download latest version: sqlite4java-392 with SQLite 3.8.7, Windows/Linux/Mac OS X/Android binaries OSGi bundle 1.0.392 with sqlite4java-392 Files for

GrapheneOS 6 Oct 26, 2022
An updated fork of sqlite_protobuf, a SQLite extension for extracting values from serialized Protobuf messages.

This fork of sqlite_protobuf fixes some issues (e.g., #15) and removes the test suite that we do not use. It also comes with proto_table, a C library

Backtrace Labs 18 Oct 19, 2022
Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform.

serverless-sqlite Serverless SQLite database read from and write to Object Storage Service, run on FaaS platform. NOTES: This repository is still in t

老雷 7 May 12, 2022
Verneuil is a VFS extension for SQLite that asynchronously replicates databases to S3-compatible blob stores.

Verneuil: streaming replication for sqlite Verneuil1 [vɛʁnœj] is a VFS (OS abstraction layer) for sqlite that accesses local database files like the d

Backtrace Labs 307 Dec 21, 2022
The C++14 wrapper around sqlite library

sqlite modern cpp wrapper This library is a lightweight modern wrapper around sqlite C api . #include<iostream> #include <sqlite_modern_cpp.h> using n

null 720 Dec 29, 2022
Unofficial git mirror of SQLite sources (see link for build instructions)

SQLite Source Repository This repository contains the complete source code for the SQLite database engine. Some test scripts are also included. Howeve

null 2k Dec 25, 2022
A hook for Project Zomboid that intercepts files access for savegames and puts them in an SQLite DB instead.

ZomboidDB This project consists of a library and patcher that results in file calls for your savegame(s) being transparently intercepted and redirecte

Oliver 7 Aug 27, 2022
Lightweight C++ wrapper for SQLite

NLDatabase Lightweight C++ wrapper for SQLite. Requirements C++11 compiler SQLite 3 Usage Let's open a database file and read some rows: #include "NLD

Raven 0 Sep 20, 2019
Writing a sqlite clone from scratch in C++

如何用C++实现一个简易数据库 基于cstack/db_tutorial C语言版本 KCNyu 2022/2/2 作为笔者写的第一个系列型教程,还是选择基于前人的教程经验以及添加一些自己个人的探索。也许有很多纰漏之处,希望大家指正。 1. 数据库是什么? 数据库是“按照数据结构来组织、存储和管理数

shengyu.li 69 Dec 27, 2022
C++ ORM for SQLite

Hiberlite ORM C++ object-relational mapping with API inspired by the awesome Boost.Serialization - that means almost no API to learn. Usage Just compi

Paul Korzhyk 651 Dec 28, 2022
DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files compatible with SQLite.

DB Browser for SQLite What it is DB Browser for SQLite (DB4S) is a high quality, visual, open source tool to create, design, and edit database files c

null 17.5k Jan 2, 2023
React-native-quick-sqlite - ⚡️ The fastest SQLite implementation for react-native.

React Native Quick SQLite The **fastest** SQLite implementation for react-native. Copy typeORM patch-package from example dir npm i react-nati

Oscar Franco 423 Dec 30, 2022
ORM for consuming RESTful APIs in C++

restful_mapper ORM for consuming RESTful APIs in C++ Introduction restful_mapper connects business objects and Representational State Transfer (REST)

Logan Raarup 80 Dec 16, 2022