DataStore Library

Personal Information Manager (PIM) applications such as address books and appointment calendars need fast, efficient data storage. Because these applications often share the same data, the data should be maintained on the system level and not by each application individually.

The DataStore Library provides a simple way for applications to store and access data in a centralized database. It shields programmers from low-level implementation of data storage.

To understand how to use the DataStore Library, you need only a basic understanding of GEOS programming.

  1 Introduction
  2 Creating a DataStore
    2.1 Adding Fields
    2.2 Adding Records
    2.3 Deleting Records
    2.4 Deleting Fields
  3 Deleting a DataStore
  4 Building an Index
  5 String Search
  6 Enumeration
  7 Timestamping
  8 Synchronization

DataStore Library: 1 Introduction

The DataStore Library provides a system for storing and manipulating collections of structured data. Data reside in numbered records with type-defined fields; each collection of records is called a datastore. An application can access different datastores simultaneously; similarly, multiple applications can access a single datastore concurrently.

The DataStore Manager manages synchronization by issuing "session tokens" whenever an application opens (or creates) a datastore. An application can have multiple sessions, each identified by a unique token.

The DataStore Library contains routines for creating and accessing data on the file level, record level, and field level. All the routines return an error value which is useful for error checking. For a full list of routines and their parameter lists, see the C Reference Book (routines are listed alphabetically and all DataStore routines begin with "DataStore").

DataStore Library: 2 Creating a DataStore

DataStoreCreate()

To create a new datastore, specify its attributes in a DataStoreCreateParams structure and call DataStoreCreate() .

typedef struct {
	TCHAR	*DSCP_name;
	DataStoreFlags	DSCP_flags;
	FieldDescriptor	*DSCP_keyList;
	word 	DSCP_keyCount;
	optr	DSCP_notifObject;
	DataStoreOpenFlags	DSCP_openFlags;
} DataStoreCreateParams;

DSCP _name

Name of the datastore. Used when opening, deleting or renaming a datastore. Any legal filename is acceptable.

DSCP _flags

Only the following flags may be passed:

• DSF_PRIVATE
  Restricts access to the datastore to the owner only ( i.e. , the application that created the datastore).
• DSF_TIMESTAMP
  Creates a timestamp field and makes it the first field of the datastore. This field is updated whenver a record is modified.
• DSF_NO_PRIMARY_KEY
  Specifies that the datastore will contain no key field. When passing this flag, you must set DSCP _keyCount to zero.

DSCP _keyList

Pointer to an array of fields that make up the key. The term "key field" is not used in the relational database sense. In a datastore, the key field does not uniquely identify a record; it simply determines storage order of the records if no callback is passed in DataStoreSaveRecord() (see "Adding Records," Adding Records for a complete discussion of storage order).

	typedef struct {
		FieldData		FD_data;
		TCHAR		*FD_name;
	} FieldDescriptor;

	typedef struct {
		FieldType			FD_type;
		FieldCategory			FD_category;
		FieldFlags			CFD_flags;
	} FieldData;

Fields of FieldType DSFT_TIMESTAMP and DSFT_BINARY may not be part of the key and the only FieldFlag which may be passed is FF_DESCENDING (default sort order is ascending).

Once you create a datastore, you cannot redefine the key later (by adding, deleting or changing the key field(s)) nor can you add a key to a datastore that was created without one.

DSCP _keyCount

Number of fields that make up the primary key.If you pass zero, then you must pass DSF_NO_PRIMARY_KEY in DSCP_flags .

DSCP _notifyObject

Object to be notified when changes to the datastore occur. This object will receive MSG_META_NOTIFY_WITH_DATA_BLOCK which will pass a DataStoreChangeNotification structure. See the GCN chapter.

Passing NullOptr means no object will receive notification.

DSCP _openFlags

Passing zero allows multiple applications to open the datastore simultaneously. Passing DSOF_EXCLUSIVE gives exclusive access to the calling application.

Adding Fields

Adding Records

Deleting Records

Deleting Fields

DataStore Library: 2.1 Creating a DataStore: Adding Fields

DataStoreAddField()

Once you've created a new datastore, you can add (non-key) fields at any time with DataStoreAddField() . When adding fields, keep in mind:

• The maximum size of a field name is MAX_FIELD_NAME_LENGTH or 40.
• When defining fields, note the difference between FieldType and FieldCategory . FieldType specifies the type of data stored in the field, such as DSFT_STRING; FieldCategory defines what type of information that data represents, such as a phone number (FC_TELEPHONE). See the C Reference Book for a full list of FieldTypes and FieldCategories .

The following code sample shows how to create a datastore that contains three fields, one of which is the key.

Code Display 9-1 Creating a New DataStore

	/* 
	 * When a new datastore is created, the DataStore Manager
	 * opens a "session" and returns a "session" token
	 */
	word	dsToken;

	/* 
	 * DataStoreCreateParams contains information about
	 * the new datastore, such as key field(s) and access level
	 */
	DataStoreCreateParams	params;

	/*
	 * FieldDescriptor contains information about
	 * a field, such as its name and type
	 */
	FieldDescriptor	field;

	/*
	 * This example will be an "Exchange Rate" datastore with three fields
	 */
	static TCHAR dsName[] = "Exchange Rates";
	static TCHAR field1[] = "country";
	static TCHAR field2[] = "currency";
	static TCHAR field3[] = "exchange rate";

	/*
	 * Define the "country" field; this will become the key field
	 */
	field.FD_name = field1;
	field.FD_data.FD_type = DSFT_STRING;
	field.FD_data.FD_category = FC_NAME;
	field.FD_data.FD_flags = 0;	/* use ascending sort order */

	/*
	 * Set the parameters for the new DataStore file:
	 *   - add a timestamp field (this becomes the first field)
	 *   - define the key field (the key cannot be changed later)
	 *   - designate which object is to receive notifications
	 * 	when the DataStore is changed
	 */
	params.DSCP_name = dsName;
	params.DSCP_flags = DSF_TIMESTAMP;
	params.DSCP_keyList = &field; /* "country" field defined above */
	params.DSCP_keyCount = 1;
	params.DSCP_notifObject = oself; /* "oself" refers to the object handling
			this message; in this case, it is the process object */
	params.DSCP_openFlags = 0; /* make the datastore sharable between apps */

	/* 
	 * Now create the new datastore file. If it is successfully created,
	 * add the additional fields.
	 */
	if(DataStoreCreate(&params, &dsToken) == DSE_NO_ERROR)
	{
		/* 
		 * DataStoreAddField() returns the FieldID of the newly
		 * created field to the passed FieldID variable (i.e., fid).
		 */

		FieldID		fid;

		/* add currency field */
		field.FD_name = field2;
		field.FD_data.FD_type = DSFT_STRING;
		field.FD_data.FD_category = FC_NONE;
		field.FD_data.FD_flags = 0;	/* use ascending sort order */
		DataStoreAddField(dsToken, &field, &fid);

		/* add exchange rate field */
		field.FD_name = field3;
		field.FD_data.FD_type = DSFT_FLOAT;
		field.FD_data.FD_category = FC_NONE;
		field.FD_data.FD_flags = 0;	/* use ascending sort order */
		DataStoreAddField(dsToken, &field, &fid);
	}

Below is a diagram of the "Exchange Rates" datastore created in the above code example. The following section discusses how to add records to the datastore.

Note that fields of type DSFT_FLOAT expect data of type FloatNum (a special GEOS data type that differs from the standard C float type; see the Math chapterfor additional information on FloatNum s).

DataStore Library: 2.2 Creating a DataStore: Adding Records

DataStoreOpen(), DataStoreNewRecord(), DataStoreSetField(), DataStoreSaveRecord()

To manage synchronization between applications accessing the same datastore, the DataStore Manager creates a "session" each time an application opens a datastore with DataStoreOpen() . Each session has its own "record buffer;" to modify a record, you must load it into the record buffer first. The loaded record is called the "current record." There can only be one record loaded in the buffer at a time. DataStoreNewRecord() adds a new record and makes that record the "current record."

To write to a record, call DataStoreSetField() . When you write to a record, you are actually writing to a copy of the record loaded in the record buffer. Any changes made to this copy do not become permanent until you call DataStoreSaveRecord() . To cancel changes, call DataStoreDiscardChanges() . Both routines flush the current record from the buffer.

Most routines that access fields (such as DataStoreSetField() ) take both a FieldID and a field name as parameters. You can reference the field by either parameter. To reference a field by its name, pass its name in the name parameter and zero in the FieldID parameter. To reference a field by its FieldID , pass its FieldID in the FieldID parameter andNULL in the name parameter. Fields are numbered from zero. (There are routines for obtaining the FieldID corresponding to a given field name and vice versa; see DataStoreFieldNameToID() and DataStoreFieldIDToName() in the C Reference Book.)

The following code example shows how to open an existing datastore, add a new record, and write data to two of its fields.

Code Display 9-2 Adding a Record to a DataStore

	/* Opening a datastore returns a token. */
	word			dsToken;

	/*
	 * Saving changes to a record returns both the record's RecordNum
	 * and RecordID
	 */
	RecordID			recordID;
	RecordNum			recordNum;

	/* 
	 * Open the "Exchange Rates" datastore with sharable access
	 * (i.e., no flag passed). "Oself" refers to the object which is to
	 * receive change notifications; in this case, assume oself refers to
	 * the process class.
	 */
	if(DataStoreOpen("Exchange Rates", oself, 0, &dsToken) == DSE_NO_ERROR)
	{
		/* Add a new record. */
		if(DataStoreNewRecord(dsToken) == DSDE_NO_ERROR)
		{
			/* 
			 * Write data to the country and exchange rate fields.
			 * You can refer to a field by its name or FieldID;
			 * the examples below show both methods. 
			 */
			TCHAR			countryBuffer[] = "Albania";

			/* refer to a field by its name */
			DataStoreSetField(dsToken, "country", 0, countryBuffer,
							strlen(countryBuffer));

			FloatNum			rateBuffer = .9234;

			/* refer to a field by its FieldID */
			DataStoreSetField(dsToken, NULL, 3, &rateBuffer,
							sizeof(rateBuffer));

			/* Save the record. */
			DataStoreSaveRecord(dsToken, 0, 0, &recordNum, &recordID);
		}
		/* Close the datastore. */
		DataStoreClose(dsToken);
	}

DataStoreSaveRecord() writes the RecordNum and RecordID of the saved record to the passed variables. A record's RecordNum is its relative place in the datastore; this value may change when records are added or deleted. A record's RecordID is its unique identifier and does not change. RecordNums are numbered from zero; RecordIDs are numbered from one.

You can use a callback function with DataStoreSaveRecord() to specify where in the datastore the record is to be saved. The calling routine passes the record to be inserted (rec1) and a record from the datastore (rec2) to the callback; the callback decides which of the two records comes before the other. (The callback cannot modify the records, however.)

The callback should return a value greater than zero if rec1 comes before rec2; otherwise, a value less than zero. Declaration of Callback Function in DataStoreSaveRecord() shows the declaration of the callback.

Code Display 9-3 Declaration of Callback Function in DataStoreSaveRecord()

sword SortCallback(RecordHeader *rec1, RecordHeader *rec2,
						word dsToken, void *cbData);

The actual data in the record follows the RecordHeader . Use DataStoreGetFieldPtr() or DataStoreFieldEnum() to access fields within the records.

typedef struct {
	RecordID	RH_uniqueID;
	byte	RH_fieldCount; /* # of fields */
	word	RH_size; /* # of bytes */
} RecordHeader;

If you do not specify a callback routine in DataStoreSaveRecord() , it will insert the record according to values in the key field(s). If two records have matching values in one key field, they will be inserted according to the first non-matching key field value. Records with matching key field(s) values are stored in the order they are added. Records with empty key fields are inserted at the beginning of the file.

If there is no callback or key, records are added to the end of the file.

DataStore Library: 2.3 Creating a DataStore: Deleting Records

DataStoreDeleteRecord(), DataStoreDeleteRecordNum()

To delete a record by its RecordID , call DataStoreDeleteRecord() . To delete a record by its RecordNum , call DataStoreDeleteRecordNum() . When deleting records, remember that RecordID s number from one and stay constant but that RecordNum s (like array elements) number from zero and change (whenever records are added or deleted). Be sure to use the correct value with the appropriate routine.

DataStore Library: 2.4 Creating a DataStore: Deleting Fields

DataStoreGetNumFields(), DataStoreRemoveFieldFromRecord(), DataStoreDeleteField()

To optimize storage, empty fields are not stored; thus, it is possible for a datastore to be defined with four fields but contain records with fewer than four fields.

DataStoreGetNumFields() returns the number of fields in the current record. DataStoreRemoveFieldFromRecord() deletes a field from the current record. (Note that passing zero in the size parameter in DataStoreSetField() will also cause the field to be removed from the record.)

To delete a field from the datastore itself, call DataStoreDeleteField() .

DataStore Library: 3 Deleting a DataStore

DataStoreDelete()

To delete a datastore, call DataStoreDelete() . This routine will delete the entire file or return DSE_DATASTORE_IN_USE if the datastore is in use or DSE_DATASTORE_NOT_FOUND if the datastore does not exist.

DataStore Library: 4 Building an Index

DataStoreBuildIndex()

You can create a secondary index ( i.e. , an index based on a non-key field) by calling DataStoreBuildIndex() . This routine builds an index based on a single field or on sort criteria specified in a callback function.

DataStoreBuildIndex() creates an array of RecordNum s (low word only) and stores the array in an LMem (local memory) block. The block contains an IndexArrayBlockHeader which holds data about the index (the number of records in the index, the offset to the beginning of the array, etc.). Following the block header is space for writing your own data, which is followed by the index itself. There is no limit to how much data you can write but the amount of data you write naturally decreases the amount of space available for the index. If the LMem block is too big to be allocated, the routine will return DSSE_MEMORY_FULL.

The following example shows how to build an index on a field and write data to the index's block header.

Code Display 9-4 Building a Secondary Index

	/* index parameters */
	DataStoreIndexCallbackParams	params;

	/* Building an index returns the handle of the allocated block */
	MemHandle	indexHandle;

	/* sample data to be added to the index block */
	TCHAR	indexData[]  = "Index created 7/1/96.";

	/*  Set up index parameters. */
	params.DSICP_indexField = 2; /* field on which to build index */
	params.DSICP_sortOrder = SO_DESCENDING; /* sort direction */
	params.DSICP_cbData = NULL; /* data to be passed to the callback
					function if a callback is being used */ 

	/*
	 * Open the datastore. (In this example, assume that dsToken
	 * has been declared as a global variable.)
	 */
	if(DataStoreOpen("Exchange Rates", oself, 0, &dsToken) == DSE_NO_ERROR)
	{
		/* Build the index. */
		if(DataStoreBuildIndex(dsToken, &indexHandle,
				sizeof(IndexArrayBlockHeader)+
				LocalStringSize(indexData)+sizeof(TCHAR),
				&params, NULL) == DSSE_NO_ERROR)
		{
			/* get a pointer to the block header */
			IndexArrayBlockHeader *pIndex;

			/* lock the block down */
			MemLock(indexHandle);

			/*
			 * dereference the handle to get a pointer
			 * to the block header
			 */
			pIndex = MemDeref(indexHandle);

			/* increment the pointer past the block header */
			pIndex++;

			/* now copy the sample data into the block */
			strcpy((TCHAR *) pIndex,  indexData);

			/* Now that we're through with the block,unlock it. */
			MemUnlock(indexHandle);
		}
		/* Close the datastore. */
		DataStoreClose(dsToken);
	}

You can also build an index based on a custom callback routine. The calling routine passes the DataStoreCallbackParams to the callback; the callback decides which of the two records ( DSICP_rec1 or DSICP_rec2 ) should go first. (If you use a callback, DataStoreBuildIndex() will ignore DSICP _indexField and DSICP _sortOrder .)

The callback should return:

-1 if DSICP_rec1 comes before DSICP_rec2
1 if DSICP_rec1 comes after DSICP_rec2

Code Display 9-5 Declaration of Callback Function in DataStoreBuildIndex()

sword SortCallback(word dsToken, DataStoreIndexCallbackParams *params);

DataStoreBuildIndex() works on datastores of 4,000 records or less. If you call this routine on a datastore larger than 4,000 records, the routine will return DSSE_INDEX_RECORD_NUMBER_LIMIT_EXCEEDED.

The application owns this index and is responsible for freeing the block. The DataStore Manager does not maintain the index in any way. Applications can synchronize a secondary index by rebuilding it whenever the application receives notification of a change that would affect the index.

DataStore Library: 5 String Search

DataStoreStringSearch()

To do a simple string search (on a specified field or field category), use DataStoreSearchString() . Starting at a specified record number, this routine searches through each record until it finds a match or until it runs out of records to search.

DataStoreSearchString() uses the following parameters:

typedef struct {
	SearchType	SP_searchType;
	RecordNum	SP_startRecord;
	dword	SP_maxRecords;
	FieldID	SP_startField;
	FieldCategory	SP_category;
	TCHAR	*SP_searchString;
	SearchFlags	SP_flags;
} SearchParams;

SP _searchType

Possible flags:

• ST_ALL
  Search all fields of FieldType DFST_STRING.
• ST_FIELD
  Search by FieldID . (Specify the FieldID in SP_startField , explained below.)
• ST_CATEGORY
  Search by FieldCategory . (Specify the FieldCategory in SP_category , explained below.)

SP _startRecord

RecordNum of record to begin search. This routine updates this field with the RecordNum of the last record examined.

SP _maxRecords

Maximum number of records to search. Allows the program to break out of the search if no matches are found. (The search can be continued by incrementing the value of SP_startRecord .) Passing -1 causes the routine to search all records.

SP _startField

FieldID of field to search if SP_searchType is set to ST_FIELD. If you specify a non-string field, the routine will return DSE_BAD_SEARCH_PARAMS.

FieldID of the field to begin the search if SP_searchType is set to ST_CATEGORY. If you specify a field that is not of the specified category, the routine will start with the next field it finds of the specified category.

SP _category

FieldCategory to search if SP_searchTyp e is set to ST_CATEGORY (otherwise this parameter is ignored).

SP _searchString

Null-terminated string to search.

SP _flags

Possible flags:

• SF_BACKWARDS
  Search backwards through the records.
• SF_START_AT_END
  Begin search from first record (going forward) or from last record if SF_BACKWARDS is also passed. If set, SP_startRecord is ignored.
• SF_IGNORE_CASE
  Ignore case when searching.
• SF_NO_WILDCARDS
  Ignore wild card characters.
  
  Many libraries including DataStore support the use of wild card characters for searching. See the Text object chapter for a discussion of wild card searching; also, see /Appl/Datastore/datastore.goc for a code example.
• SF_PARTIAL_WORD
  Treat search string as a substring. For example, if search string is "den," return "gardenia," "dentist," and "garden."

The following example shows how to set up a simple string search on a specified field.

Code Display 9-6 Searching a DataStore

	/* search conditions */
	SearchParams	params;

	/*  Specify search parameters. */
	params.SP_searchType = ST_FIELD; /* search by FieldID */
	params.SP_startRecord = 0; /* start search at first record */
	params.SP_maxRecords = -1; /* search all records until a match is found or 
					there are no more records to search */
	params.SP_startField = 1; /* search "country" field */
	params.SP_searchString = "Albania"; /* string to search for */
	params.SP_flags = SF_IGNORE_CASE; /* ignore case when searching */

	/* Open the datastore. */
	if(DataStoreOpen("Exchange Rates", oself, 0, &dsToken) == DSE_NO_ERROR)
	{
		/* Do the search. */
		if(DataStoreStringSearch(dsToken, &params) == DSDE_NO_ERROR)
		{
			 /*
			  * If a match is found, load the record and get the data
			  * from the "exchange rate" field.
			  * Note:  DataStoreStringSearch() returns the record
			  * number of the last examined record in SP_startRecord.
			  */
			if(DataStoreLoadRecordNum(dsToken, params.SP_startRecord,
							&recordID) == DSDE_NO_ERROR)
			{
				/* variables used for retrieving field data */
				FloatNum		rateBuffer, *pRateBuffer;
				RecordID		recordID;
				MemHandle		dummy;
				word		size;

				pRateBuffer = &rateBuffer;
				size = sizeof(rateBuffer);

				DataStoreGetField(dsToken, "exchange rate", 0,
					(void **)&pRateBuffer, &size, &dummy);
				/*
				  * Do something with the data then 
				  * flush the record from the buffer.
				  */
				DataStoreDiscardRecord(dsToken);
			}
		DataStoreClose(dsToken);
		}
	}

If the routine returns DSE_NO_MORE_RECORDS, it has reached the last record in the file (either the first or last record depending on the direction of the search). If the routine returns DSE_NO_MATCH_FOUND, it did not find a match within the set of records it searched. If it returns DSE_NO_ERROR, it writes the RecordNum of the matching record in SP_startRecord .

The DataStore Library does not implement global searches ( i.e. , searches through multiple datastores), though it is possible to implement this type of search at the application level by opening each datastore file and calling DataStoreStringSearch() on each one.

DataStore Library: 6 Enumeration

DataStoreFieldEnum(), DataStoreRecordEnum()

DataStoreFieldEnum() enumerates through fields of a record. This routine uses a Boolean callback to determine whether to continue enumeration. If the callback returns TRUE, enumeration stops.

DataStoreRecordEnum() enumerates through records of a datastore in storage order, starting at the specified RecordNum in the specified direction. This routine uses a Boolean callback routine to determine whether to continue enumeration. If the callback returns TRUE, enumeration ends; if FALSE, enumeration continues until the callback returns TRUE or until the routine reaches the last record.

There are two DataStoreRecordEnumFlags that can be passed in this routine:

• DSREF_BACKWARDS
  Enumerate backwards through the datastore.
• DSREF_START_AT_END
  Enumerate from first record (going forward) or from last record if DSREF_BACKWARDS is also passed.

The following example enumerates through the a datastore looking for the maximum value in a particular field.

Code Display 9-7 Enumerating Through a Datastore

	/* data to be passed to the callback routine */
	FloatNum	enumData = 0;

	/* record at which to start enumeration */
	RecordNum	rec = 0;

	/* 
	 * Open the datastore. For this example, assume dsToken
	 * is a global variable.
	 */
	if(DataStoreOpen("Exchange Rates", oself, 0, &dsToken) == DSE_NO_ERROR)
	{
		/* 
		 * Enumerate through the datastore starting at the first record
		 * (so pass zero in the flags parameter);
		 * find the maximum value of the "exchange rate" field.
		 */
		if(DataStoreRecordEnum(dsToken, &rec, 0, &enumData, EnumCallback)
							== DSE_NO_MORE_RECORDS)
		{
			/* do something with the value */
		}
		DataStoreClose(dsToken);
	}

	/*
	 * The callback compares the data in the "exchange rate" field to the value
	 * passed in with enumData. If the field data is greater than that of
	 * enumData, copy the field's data to enumData.
	 */

	Boolean EnumCallback(RecordHeader *record, void *enumData)
	{
		/* parameters for getting field data */
		FloatNum	rateBuffer, *pRateBuffer;
		FieldType	type;
		word	size;

		if(DataStoreGetFieldPtr(dsToken, record, 3, (void **)&pRateBuffer,
						&type, &size) == DSDE_NO_ERROR)
		{
			if(*pRateBuffer > *((FloatNum *)enumData))
			{
				*((FloatNum *)enumData) = *pRateBuffer;
			}
		}
		return FALSE; /* FALSE to continue enumeration */
	}

If the routine returns DSE_NO_MORE_RECORDS, it has reached the last record in the file. If it returns DSE_NO_ERROR, it writes the RecordNum of the last record examined in the startRecord parameter.

DataStore Library: 7 Timestamping

DataStoreGetTimeStamp(), DataStoreSetTimeStamp()

Record timestamping is important for synchronization and reconciliation of data between devices; between a portable device and a desktop PC, for example. Passing the FT_TIMESTAMP flag when creating a new datastore makes the first field a timestamp field. The DataStore Manager updates this field when the record has been modified.

You can read the data in the timestamp field by calling DataStoreGetField() and passing zero for the FieldID parameter and NULL for field name.

To retrieve the time and date a datastore was last changed, call DataStoreGetTimeStamp() . To modify a datastore's timestamp manually, call DataStoreSetTimeStamp() .

DataStore Library: 8 Synchronization

The DataStore Manager ensures synchronization between applications sharing the same datastore file by issuing a series of read and write locks. Routines that modify data request write locks while non-modifying routines request read locks. If the DataStore Manager cannot grant a lock because another session has locked the datastore, the routine will return DSE_DATASTORE_LOCKED.There can be up to thirty-two readers per datastore.