Class: Keyspace | oops/core.py | ||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Keyspace manager.Keyspace's role is to keep track of a container node's persistent members. In the case of a Dict instance, it is to keep track of the dictionary's keys. In the case of an Obj instance, it is to keep track of its data rows. Keyspace is specially designed to scale for containers with many subnodes. Keyspace instances distribute their contents across many database rows to counter the overhead of pickling large lists. The reason for this is that the keyspace needs to be synced to disk every time it is modified. With this implementation, the keyspace is broken into chunks and only the modified chunks are pickled and saved when new keys are added. Keyspace is ultra-light on storage size, I/O and processing because it uses what I call the square key distribution algorithm. The rows containing the distributed keyspace are always smaller than the square root of the size of the keyspace. There are usually as many rows as there are elements per row, but when a new square size is reached (len(keyspace)==x**2), new elements are placed in a new row (so rows==row_size+1). When adding an element, only one row is synced to disk which has at most row_size+1 elements. When removing an element, at most 2 rows are synced to disk while sometimes only one row is synced to disk.
|