threaddb  2.0b
A file mapped memory container extension
Functions
threaddbC.h File Reference

List of interface methods to the database. More...

#include "threaddbTypes.h"

Go to the source code of this file.

Functions

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Create (void **pThreadDB_p, size_t PackageSize_p, size_t PackageCacheLimit_p, const char *pPackagesFolder_p)
 Interface for creating a new database object. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Import (void **pThreadDB_p, const char *pIndexFileUTF8_p, size_t PackageCacheLimit_p)
 Interface for creating a new database object based on a database index file. More...
 
DLLEXPORT_ void ThreadDB_Destroy (void *pThreadDB_p)
 Interface to destroy an existing database object. More...
 
const DLLEXPORT_ char * ThreadDB_GetVersionInfo (void *pThreadDB_p)
 Interface for retrieving the package version info. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Save (const char *pIndexFileUTF8_p, void *pThreadDB_p)
 Interface to save the stored data. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewThread (const char **pFileName_p, const char *pFolder_p, size_t MaxFileSize_p, void *pThreadDB_p)
 Interface to create and register a new worker thread. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewPackage (uint64_t *pPackageID_p, void *pThreadDB_p)
 Registers a new data item package. More...
 
DLLEXPORT_ size_t ThreadDB_GetThreadCount (void *pThreadDB_p)
 Returns the number of currently running threads. More...
 
DLLEXPORT_ size_t ThreadDB_GetPackageCount (void *pThreadDB_p)
 Returns the number of currently registered packages. More...
 
DLLEXPORT_ size_t ThreadDB_GetFileCount (void *pThreadDB_p)
 Returns the number of currently registered temporary database files. More...
 
DLLEXPORT_ size_t ThreadDB_GetPackageSize (void *pThreadDB_p)
 Returns the package buffer size. More...
 
const DLLEXPORT_ char * ThreadDB_GetDatabaseFileName (size_t FileIndex_p, void *pThreadDB_p)
 Returns the filename of a currently registered temporary database files. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_RelocateFileTo (size_t FileID_p, const char *pFilePathUTF8_p, threadDB_RelocationType RelocationType_p, void *pThreadDB_p)
 Renames and copies/moves a temporary database file to a different location or disk. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Store (uint64_t Package_p, size_t Size_p, const char pData_p[], threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
 Stores database package items in the specified package. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_SynchronizePackage (uint64_t Package_p, void *pThreadDB_p)
 Synchronizes the internal package buffers with the temporary database files. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Synchronize (void *pThreadDB_p)
 Synchronizes the internal package buffers with the temporary database files. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Open (threadDB_ReadInfo **pReadInfo_p, uint64_t Package_p, void *pThreadDB_p)
 This operation opens a package for stream reading. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_OpenItem (threadDB_ReadInfo **pReadInfo_p, const threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
 This operation opens a package for stream reading at the given item cursor. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Close (threadDB_ReadInfo **pReadInfo_p, void *pThreadDB_p)
 This operation closes a package after stream reading has finished. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverContinous (size_t *pReadBytes_p, size_t Size_p, char pData_p[], threadDB_ReadInfo *pReadInfo_p, void *pThreadDB_p)
 Stream reading operation through the data items stored in a package. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverRandom (size_t Size_p, char pData_p[], const threadDB_ItemInfo *pItemHandle_p, uint64_t Package_p, void *pThreadDB_p)
 Random access operation to the data items stored in a package. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverRandomContinous (size_t Size_p, char pData_p[], threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
 Random access operation to the data items stored in a package. More...
 
DLLEXPORT_ bool ThreadDB_PackageEnd (threadDB_ReadInfo *pReadInfo_p, void *pThreadDB_p)
 Stream reading operation through the data items stored in a package. More...
 
DLLEXPORT_ threadDB_ReturnCode ThreadDB_Replace (size_t Size_p, const char pData_p[], const threadDB_ItemInfo *pItemHandle_p, void *pThreadDB_p)
 Replace operation to modify the contents of a stored data item. More...
 
const DLLEXPORT_ char * ThreadDB_GetErrorMessage (void *pThreadDB_p)
 Information on error state. More...
 

Detailed Description

List of interface methods to the database.

This module contains the list of available methods provided. The interface is provided as a C header which allows to integrate the threadDB implementation as dynamic library (dll).

Function Documentation

◆ ThreadDB_Close()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Close ( threadDB_ReadInfo **  pReadInfo_p,
void *  pThreadDB_p 
)

This operation closes a package after stream reading has finished.

This operation is required to close a package and remove the pReadInfo_p after stream reading has finished.

Parameters
pReadInfo_pRead handle created during the ThreadDB_Open operation.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_Create()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Create ( void **  pThreadDB_p,
size_t  PackageSize_p,
size_t  PackageCacheLimit_p,
const char *  pPackagesFolder_p 
)

Interface for creating a new database object.

Creates a new database object and returns the related handle. The operation initializes the internal data structures and starts the thread for package buffer flushing. The PackageSize_p is the size of the memory block allocated for interim buffering of the stored data items. The PackageCacheLimit_p defines an upper limit of parallel held item buffers of size PackageSize_p. If the number of packages exceeds this limit the flush threads starts to withdraw package buffers from memory. This treatment requires additional thread synchronization taking place and could reduce the throughput significantly. The major part of consumed memory at a time is given by the number of packages times the package buffer size. Prior storing items in the database, it is important to create at least one worker thread using ThreadDB_NewThread.

Parameters
pThreadDB_pHandle to the created database object.
PackageSize_pRecord size for buffering the data items.
PackageCacheLimit_pMaximum number of package buffers held in memory at one point of time.
pPackagesFolder_pDirectory path to store the package header information
Returns
Error code.

◆ ThreadDB_Destroy()

DLLEXPORT_ void ThreadDB_Destroy ( void *  pThreadDB_p)

Interface to destroy an existing database object.

Destroys an existing database object, removes the related temporary database files and stops all running threads.

Parameters
pThreadDB_pHandle to the database object.

◆ ThreadDB_GetDatabaseFileName()

const DLLEXPORT_ char* ThreadDB_GetDatabaseFileName ( size_t  FileIndex_p,
void *  pThreadDB_p 
)

Returns the filename of a currently registered temporary database files.

Returns the filename of the registered temporary database files with index FileIndex_p. During execution of this command registering of new packages using ThreadDB_GetDatabaseFileName is synchronized.

Parameters
FileIndex_pIndex of the temporary database file.
pThreadDB_pHandle to the database object.
Returns
Filename of the temporary database file.

◆ ThreadDB_GetErrorMessage()

const DLLEXPORT_ char* ThreadDB_GetErrorMessage ( void *  pThreadDB_p)

Information on error state.

This function returns information on current error state of the database.

Parameters
pThreadDB_pHandle to the database object. return Information on the occured error.

◆ ThreadDB_GetFileCount()

DLLEXPORT_ size_t ThreadDB_GetFileCount ( void *  pThreadDB_p)

Returns the number of currently registered temporary database files.

Returns the number of currently registered temporary database files.

Parameters
pThreadDB_pHandle to the database object.
Returns
Number of registered database files.

◆ ThreadDB_GetPackageCount()

DLLEXPORT_ size_t ThreadDB_GetPackageCount ( void *  pThreadDB_p)

Returns the number of currently registered packages.

Returns the number of currently registered packages.

Parameters
pThreadDB_pHandle to the database object.
Returns
Number of registered packages.

◆ ThreadDB_GetPackageSize()

DLLEXPORT_ size_t ThreadDB_GetPackageSize ( void *  pThreadDB_p)

Returns the package buffer size.

Returns the size used to allocate internal package buffer blocks.

Parameters
pThreadDB_pHandle to the database object.
Returns
Size of the internal package buffers.

◆ ThreadDB_GetThreadCount()

DLLEXPORT_ size_t ThreadDB_GetThreadCount ( void *  pThreadDB_p)

Returns the number of currently running threads.

Returns the number of currently running threads.

Parameters
pThreadDB_pHandle to the database object.
Returns
Number of executing threads.

◆ ThreadDB_GetVersionInfo()

const DLLEXPORT_ char* ThreadDB_GetVersionInfo ( void *  pThreadDB_p)

Interface for retrieving the package version info.

Provides the current version of the ThreadDB project.

Parameters
pThreadDB_pHandle to the database object.
Returns
Version information string.

◆ ThreadDB_Import()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Import ( void **  pThreadDB_p,
const char *  pIndexFileUTF8_p,
size_t  PackageCacheLimit_p 
)

Interface for creating a new database object based on a database index file.

Creates a new database object based on a previously saved database. The operation initializes the internal data structures, starts the thread for package buffer flushing and builds up the internal package table. The PackageSize_p is the size of the memory block allocated for interim buffering of the stored package items. The pIndexFileUTF8_p needs to specify the full path to the location of the database index file. The pIndexFileUTF8_p path characters have to be provided in UTF8 representation. Threads registered on an imported database will create new temporary database files and the imported database files are kept mainly untouched.

Parameters
pThreadDB_pHandle to the created database object.
pIndexFileUTF8_pRecord size for buffering the data items.
PackageCacheLimit_pMaximum number of package buffers held in memory at one point of time.
Returns
Error code.

◆ ThreadDB_NewPackage()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewPackage ( uint64_t *  pPackageID_p,
void *  pThreadDB_p 
)

Registers a new data item package.

Creates a new package to receive the stored data items. Each package definition is held in memory and consumes an amount of about 180 Bytes. Storing data items within a package synchronization of threads takes place.

Parameters
pPackageID_pGenerated package ID.
pThreadDB_pHandle to the database object.
Returns
Unique package id used for the store operation.

◆ ThreadDB_NewThread()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_NewThread ( const char **  pFileName_p,
const char *  pFolder_p,
size_t  MaxFileSize_p,
void *  pThreadDB_p 
)

Interface to create and register a new worker thread.

Creates a new worker thread receiving and managing the stored item data. Each worker thread gets a temporary database file assigned to. It is possible to define multiple worker threads and define temporary folders on different discs. The created files have the format threaddb.XXXXXX with XXXXX reflecting an unique identifying number. A basic prinicple of storing data itmes is, that the data is directed to the first available thread registered. This leads to a priority to which thread/temporary file the data is written. It is therefore sensible to specify folders on discs with faster write speed (e.g. SSD discs) first.

Parameters
pFileName_pFull path to the created temporary database file in ASCII character representation.
pFolder_pPath to the temporary database file folder in ASCII character representation.
MaxFileSize_pMaximum size in bytes a database file is allowed to grow. Exceeding this limit a new temporary file will be created.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_Open()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Open ( threadDB_ReadInfo **  pReadInfo_p,
uint64_t  Package_p,
void *  pThreadDB_p 
)

This operation opens a package for stream reading.

This operation opens a package for stream reading the package contents. Stream reading is sensible for continious reading through the stored data items of the given package.

Parameters
pReadInfo_pGenerated ReadInfo handle.
Package_pindex of the package whose contents should be recovered.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_OpenItem()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_OpenItem ( threadDB_ReadInfo **  pReadInfo_p,
const threadDB_ItemInfo pItemHandle_p,
void *  pThreadDB_p 
)

This operation opens a package for stream reading at the given item cursor.

This operation opens a package for stream reading at the given stored item position.

Parameters
pReadInfo_pGenerated ReadInfo handle.
pItemHandle_phandle to the stored item.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_PackageEnd()

DLLEXPORT_ bool ThreadDB_PackageEnd ( threadDB_ReadInfo pReadInfo_p,
void *  pThreadDB_p 
)

Stream reading operation through the data items stored in a package.

This function provides information if the package end has been reached.

Parameters
pReadInfo_pread handle created during the ThreadDB_Open operation.
pThreadDB_pHandle to the database object.
Returns
End of package flag.

◆ ThreadDB_RecoverContinous()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverContinous ( size_t *  pReadBytes_p,
size_t  Size_p,
char  pData_p[],
threadDB_ReadInfo pReadInfo_p,
void *  pThreadDB_p 
)

Stream reading operation through the data items stored in a package.

This function allows to continiously read through the stored data items. The calling process has to preallocate the pData_p transfer buffer and provide the size of data to be read through the Size_p parameter. It is possible to read amongst several data items at once.

Parameters
pReadBytes_pNumber of read data bytes.
Size_pnumber of bytes to be read from the selected package.
pData_ppreallocated transfer buffer, requires to be preallocated and embrace at least Size_p bytes.
pReadInfo_pread handle created during the ThreadDB_Open operation.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_RecoverRandom()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverRandom ( size_t  Size_p,
char  pData_p[],
const threadDB_ItemInfo pItemHandle_p,
uint64_t  Package_p,
void *  pThreadDB_p 
)

Random access operation to the data items stored in a package.

This function allows to read stored data items individually. The calling process has to preallocate the pData_p transfer buffer and provide the size of data to be read through the Size_p parameter. This function reads only the selected item. Since this method reads also from internal buffers, synchronization is not required. Since this operation requires to individually seek for the entry on disk, streaming throughput es in general less in comparison to continous reading.

Parameters
Size_pnumber of bytes to be read from the selected package.
pData_ppreallocated transfer buffer, requires to embrace at least Size_p bytes.
pItemHandle_phandle to the stored item.
Package_ppackage identifier the selected item belongs to.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_RecoverRandomContinous()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RecoverRandomContinous ( size_t  Size_p,
char  pData_p[],
threadDB_ItemInfo pItemHandle_p,
void *  pThreadDB_p 
)

Random access operation to the data items stored in a package.

This function allows to read stored data items individually. The calling process has to preallocate the pData_p transfer buffer and provide the size of data to be read through the Size_p parameter. It is possible to read amongst several data items at once. The parameter pItemHandle_p points to the selected data item. During read the handle is moved on to the following item entry. This allows to read through a series of elements continously. Since this operation requires to individually seek for the specific entry on disk, streaming throughput is in general less in comparison to continous reading.

Parameters
Size_pnumber of bytes to be read from the selected package.
pData_ppreallocated transfer buffer, requires to embrace at least Size_p bytes.
pItemHandle_phandle to the stored item.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_RelocateFileTo()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_RelocateFileTo ( size_t  FileID_p,
const char *  pFilePathUTF8_p,
threadDB_RelocationType  RelocationType_p,
void *  pThreadDB_p 
)

Renames and copies/moves a temporary database file to a different location or disk.

The function allows to rename or copy/move a database file to a different location or disk. The command is synchronized and available also while store or read operations are in progress.

Parameters
FileID_pthe file index of the temporary database file to be renamed/moved.
pFilePathUTF8_pfull path to the destination to which the database file should be moved.
RelocationType_ptype of action copy/move to take place.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_Replace()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Replace ( size_t  Size_p,
const char  pData_p[],
const threadDB_ItemInfo pItemHandle_p,
void *  pThreadDB_p 
)

Replace operation to modify the contents of a stored data item.

This function allows to modify existing data item entries in the database. It replaces the given Size_p of bytes by the data provided in the pData_p transfer buffer. The Size_p parameter has to match the size of the previously stored data item.

Parameters
Size_pnumber of bytes to be replaced.
pData_ppreallocated transfer buffer, requires to be embrace at least Size_p bytes.
pItemHandle_phandle to the stored item to be replaced.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_Save()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Save ( const char *  pIndexFileUTF8_p,
void *  pThreadDB_p 
)

Interface to save the stored data.

Saves the database control structures to the given database index file. The related temporary database files are removed from the purging process executed during the database destruction phase.

Parameters
pIndexFileUTF8_pFull path to the database index file location in UTF8 representation.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_Store()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Store ( uint64_t  Package_p,
size_t  Size_p,
const char  pData_p[],
threadDB_ItemInfo pItemHandle_p,
void *  pThreadDB_p 
)

Stores database package items in the specified package.

The function allows to append a new data item to the package @ Package_p. The item is asynchronously copied and appended to the given package. The parameter pItemHandle_p handle can be omitted by specifying a NULL pointer. If available, the stored index entry of the item is provided. A pItemHandle_p allows random access of individual package items. The pItemHandle_p has to be allocated and managed separately and provided by the caller. The item handles are registered in the database and updated during the ThreadDB_Synchronize operation. Please note, that the item handle is usually invalid until the ThreadDB_Synchronize operation is executed. The Size_p parameter specifies the size of data to be transferred. The size of the data item is NOT stored in the database and has to be managed separately. It needs to be provided during the ThreadDB_RecoverContinous and ThreadDB_RecoverRandom operations. The maximum size specified by the Size_p parameter. Data items exceeding the specified PackageSize_p need are split into multiple records and stored performing several @ ref ThreadDB_Store calls. Please note that each data item requires to be temporary duplicated to allow asynchronous storage. A special form of the ThreadDB_Store call is to flush specific packages to disk. This can be achieved by calling the function with a Size_p of zero. In this case only the buffer of the selected Package_p is synchronized while other packages stay open for writing and are not prematurely flushed. This allows to start reading of package information while writing to other packages is still in progress.

Parameters
Package_ppackage index to which the data item should be appended.
Size_psize of the package item to be stored.
pData_ppointer to the data for being stored.
pItemHandle_poptional parameter for random data item access.
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_Synchronize()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_Synchronize ( void *  pThreadDB_p)

Synchronizes the internal package buffers with the temporary database files.

Synchronizes the package buffers by writing their contents to the database files. This operation also updates and validates the registered item handles specified during the ThreadDB_Store operation. Synchronization is required prior calling ThreadDB_RecoverContinous operation. This operation also releases the internal package buffers.

Parameters
pThreadDB_pHandle to the database object.
Returns
Error code.

◆ ThreadDB_SynchronizePackage()

DLLEXPORT_ threadDB_ReturnCode ThreadDB_SynchronizePackage ( uint64_t  Package_p,
void *  pThreadDB_p 
)

Synchronizes the internal package buffers with the temporary database files.

Synchronizes the selected package by writing its contents to the database files. This operation also updates and validates the registered item handle specified during the ThreadDB_Store operation. Synchronization is required prior calling ThreadDB_RecoverContinous or operation. This operation also releases the internal package buffer.

Parameters
Package_ppackage index to which should be synchronized.
pThreadDB_pHandle to the database object.
Returns
Error code.