High Level API

The main API is provided by KiokuDB.pm. This is the entry point into KiokuDB and is the API for users. It brings together all of the other components.

When using KiokuDB you rarely need to delve deeper than this interface, but knowing how the pieces fit together is helpful for both knowing what to expect, and extending the system to better suit your purposes.


Processing Layer

The processing layer contains a number of components that integrate storage backed objects seamlessly into memory space.

The processing layer deals with objects in Perl's memory space and is backend independent.

The Linker

The linker manages the task of expanding entries into objects and connecting them into the in memory graph of objects.

Object expansion uses the type map to locate the per-class expansion callbacks, and calls them while maintaining all the links between objects consistent.

Inter object links are handled by recursively fetching the necessary entries from the backend, except when explicitly deferred..

The Collapser

The collapser is the storage-time counterpart of the linker. It walks the objects in memory and locates the appropriate collapsing routine for each class as it prepares a set of entries to storage.

The collapser handles memory cycles, weak references, handling of tied data structures, and various other nits and edge cases with Perl data structures.

The Type Map

The type map is responsible for finding appropriate callbacks for each class stored in the database. These callbacks assign IDs, break down the object for the collapser, and reassemble it during loading.

The default typemap will handle all Moose based objects, with rich support for metaclass based customization. This allows you to lazy load certain attributes, skip cache attributes during serialization, assign custom IDs, etc.

The default typemaps also contains entries for popular modules such as DateTime.

Optional out of the box support for other popular class builders is also provided.

Lastly, adding your own entries is trivial. You can reuse existing Storable hooks, use callbacks, or simply let the collapser break down the object for you.

Live Objects

Live objects are objects that KiokuDB has either loaded or stored. The live object manager maps Perl reference addresses to IDs and live entries allowing the linker and the collapser to maintain the integrity of the object graph.

The live object manager additionally handles the various scoping related tasks, such as keeping alive objects only pointed to by weak references, and handling entry state during transactions.

Indexing

Indexing is done before insertion to the database by extracting the relevant search keys in Perl space. Although Search::GIN is still in its infancy, it is capable of performing simple searches on your database on arbitrary search keys.

An alternative to in memory indexing is relying on the backend to provide searching facilities. The downsides are that this technique breaks the encapsulation of your objects, and also limits you to the data inside a single object, and does not let you index relationship based data.


Backends

Almost any keyed data store can be used as a backend for KiokuDB. BLOB or document oriented databases are especially suitable.

Since backends are isolated from the complexity of the object graph and all of Perl's quirks by the processing layer, the driver code can remain clean and simple.

Implementing a backend is easy. The design lets you add features incrementally instead of all at once, by using interface roles. The core API contains just 4 methods.

Secondly, there are numerous concrete roles to help backend authors reuse code, and a reusable test suite to check your code.

Berkeley DB

The Berkeley DB backend uses the BerkeleyDB module to interface with Oracle Berkeley DB. This is a high performance, full features transactional store, supporting GIN based queries.

DBI

The DBI backend utilizies Perl's DBI module to provide database independent transactional storage using simple, portable SQL.

This backend supports GIN indexing as well as simple SQL where clauses.

Tested databases include SQLite, PostgreSQL and MySQL.

File Backend

NOTE: This backend is currently broken. Patches are welcome to get it running again.

The files backend is the simplest storage provider. It uses Directory::Transactional to provide transaction support to store each object in its own file on disk.

This backend has no built in indexing mechanism.

CouchDB

Apache CouchDB can be used as a backend using the CouchDB backend. This utilizes AnyEvent::CouchDB to communicate with CouchDB.

Simple DB

The SimpleDB backend is still in development, and provides access to Amazon SimpleDB using the Amazon::SimpleDB module.