There are tools and APIs to make it easier to administer an application's stored data.
Administration Console
When using the Administration Console, remember that NDB is implemented by means of the Datastore and memcache. When NDB "reads" a value, it checks memcache first; it only reads from the Datastore if it doesn't find the value in memcache.
Thus, if you use the Administration console to edit an entity Datastore, the application may still use the "old" value if that value is in memcache. You might work around this by purging memcache. This is rather disruptive, however; if you need to tweak values "by hand" often, you might want to set up a UI for this in your application that writes values via the NDB API; writing via NDB keeps the Datastore and memcache in sync.
Metadata Queries
NDB supports a metadata query API. This allows an application to get
some general information about its Datastore. This API is in the
google.appengine.ext.ndb.metadata
module. It has functions:
get_namespaces(start=None, end=None)
: return a list of namespace namesget_kinds(start=None, end=None)
: return a list of kind namesget_properties_of_kind(kind, start=None, end=None)
: return a list of property names for the given kind nameget_representations_of_kind(kind, start=None, end=None)
: return a dict mapping property names for the given kind name to lists of representation names such as 'STRING', 'BOOLEAN' or 'INT64'.
These functions have optional start and end
arguments that can be used to restrict the query to a certain range.
Here, start is inclusive and end is exclusive.
Both default to None
. For example, to get all namespaces
starting with a lowercase letter you could call
get_namespaces('a', chr(ord('z') + 1))
.
All of these except get_namespaces()
are implicitly restricted to the current namespace.
Beware of the
cost of metadata queries.
Statistics Queries
The datastore maintains statistics about the data stored for an application, such as how many entities there are of a given kind, or how much space is used by property values of a given type. You can view these statistics in the Administration Console, under Datastore > Statistics.
You can also access these values programmatically within the application by querying for specially named entites using the datastore API. Each statistic is accessible as an entity whose kind name begins and ends with two underscores. For example, each app has exactly one entity of the kind __Stat_Total__
that represents statistics about all of the entities in the datastore in total. Each statistic entity has the following properties:
count
, the number of items considered by the statistic (a long integer)bytes
, the total size of the items for this statistic (a long integer)timestamp
, the time of the most recent update to the statistic (a date-time value)
Some statistic kinds also have additional properties, listed below.
An application can use model classes provided by the package
google.appengine.ext.ndb.stats
to access statistic entities.
from google.appengine.ext.ndb import stats global_stat = stats.GlobalStat.query().get() print 'Total bytes stored: %d' % global_stat.bytes print 'Total entities stored: %d' % global_stat.count
When the statistics system creates new statistic entities, it does not delete the old ones right away. The best way to get a consistent view of the statistics is to query for the GlobalStat
entity with the most recent timestamp
, then use that timestamp value as a filter when fetching other statistic entities.
The statistic entities are included in the calculated statistic values. Statistic entities take up space relative to the number of unique kinds and property names used by the application.
The statistics system will also create statistics specific to each namespace. Note that if an application does not use datastore namespaces then namespace specific statistics will not be created. Namespace specific stats are found in the namespace that they're specific to. The kind names for namespace specific stats are prefixed with __Stat_Ns_
and have the same corresponding suffix as application wide statistics kinds.
The complete list of available statistics is as follows:
Statistic | Stat Entity Kind | Description |
---|---|---|
all entities | __Stat_Total__
Python class: GlobalStat
Namespace specific entry: |
All entities. Additional properties:
|
all entities in a namespace | __Stat_Namespace__
Python class: NamespaceStat
Note that |
All entities in a namespace.
|
all entries in application defined indexes | __Stat_Kind_CompositeIndex__
Namespace specific entry: Python class: KindStat
Namespace specific entry: |
Entries in the composite index table; one stat entity for each kind of entity stored. Additional properties:
|
entities of a kind | __Stat_Kind__
Python class: KindStat
Namespace specific entry: |
Entities of a kind; one stat entity for each kind of entity stored. Additional properties:
|
root entities of a kind | __Stat_Kind_IsRootEntity__
Python class: KindRootEntityStat
Namespace specific entry: |
Entities of a kind that are entity group root entities (have no ancestor parent); one stat entity for each kind of entity stored. Additional properties:
|
non-root entities of a kind | __Stat_Kind_NotRootEntity__
Python class: KindNotRootEntityStat
Namespace specific entry: |
Entities of a kind that are not entity group root entities (have an ancestor parent); one stat entity for each kind of entity stored. Additional properties:
|
properties of a type | __Stat_PropertyType__
Python class: PropertyTypeStat
Namespace specific entry: |
Properties of a value type across all entities; one stat entity per value type. Additional properties:
|
properties of a type per kind | __Stat_PropertyType_Kind__
Python class: KindPropertyTypeStat
Namespace specific entry: |
Properties of a value type across entities of a given kind; one stat entity per combination of property type and kind. Additional properties:
|
properties with a name | __Stat_PropertyName_Kind__
Python class: KindPropertyNameStat
Namespace specific entry: |
Properties with a given name across entities of a given kind; one stat entity per combination of unique property name and kind. Additional properties:
|
properties of a type and with a name | __Stat_PropertyType_PropertyName_Kind__
Python class: KindPropertyNamePropertyTypeStat
Namespace specific entry: |
Properties with a given name and of a given value type across entities of a given kind; one stat entity per combination of property name, value type and kind that exists in the datastore. Additional properties:
|
Some statistics refer to datastore property value types by name, as strings. These names are as follows:
"Blob"
"BlobKey"
"Boolean"
"Category"
"Date/Time"
"Email"
"Float"
"GeoPt"
"IM"
"Integer"
"Key"
"Link"
"NULL"
"PhoneNumber"
"PostalAddress"
"Rating"
"ShortBlob"
"String"
"Text"
"User"
Note that __Stat_Namespace__
entities contain the same information found in __Stat_Ns_Total__
records. __State_Namespace__
entities are stored in the empty namespace and contain a subject_namespace
field describing the namespace to which they belong. __Stat_Ns_Total__
records are stored in the namespace to which they refer, and thus do not contain a subject_namespace
field. Hence, a query on kind __Stat_Namespace__
(from the empty string namespace) ordered descending by bytes
will list the namespaces that consume the largest storage first. Since queries across namespaces are not possible, any query for __State_Ns_Total__
entities will only ever produce at most single record.