Last time I described how Openstack Swift stores data and started to explain Storage nodes, Consistent Hashing, and caching called an Account Ring. Let’s dig into the Rings a bit.
Openstack Swift Storage Account Rings
The Account Ring is used by the Proxy nodes to determine where each account is stored. There is one and only one Account Ring for a given Swift Cluster. Input to the Account Ring is the tuple: (account name). The account name is then hashed, and the hash is used as an index into the Account Ring. The result is the three Storage nodes that contain a copy of the account data. All three copies are identical; so it does not matter which copy (Storage node) is used by the Proxy node. In practice, a specific Proxy node, given repeated requests for a specific account, will round robin the Storage node requests to help balance the load.
Container Ring
The Container Ring is used in a similar manner. Input to the Container Ring is the tuple: (account name, container name). The (account name, container name) is then hashed, and the hash is used as an index into the Container Ring.
The result is the three Storage nodes that contain a copy of the container data.
And finally the Object Ring is also similar. Input to the Object Ring is the tuple: (account name, container name, object name). The (account name, container name, object name) is then hashed and the hash is used as an index into the Object Ring. The result is the three Storage nodes that contain a copy of the object data.
The second Swift component, is the Database. Swift uses SQLite as its database engine. An SQLite database instance is stored as a single file. Constrast this with MySQL, in which a database instance is represented as a file system directory and database tables are stored in files.
Swift Database Types
Swift creates two different database types for storage: Account and Container.
An Account Database consists of two tables: Account Stat and Container. A separate Account Database is created for each Swift account. Thus, an Account Stat Table has one and only one entry. The Container Table contains an entry for each container in the account. Account Databases are stored in files. An Account Database file name is constructed as:
/srv/<zone>/node/<device>/accounts/<hash>.db
where:
<zone> = zone number (1 – 4).
<device> = device name (i.e. sdb1).
<hash> = hash of the account name.
A Container Database consists of two tables: Container Stat and Object. A separate Container Database is created for each Swift container. Thus, a Container Stat Table has one and only one entry. The Object Table contains an entry for each object in the container. Container Databases are stored in files. An Container Database file name is constructed as:
/srv/<zone>/node/<device>/containers/<hash>.db
where:
<zone> = zone number (1 - 4).
<device> = device name (i.e. sdb1).
<hash> = hash of: (account name, container name).
Object data is also stored in files. A Object file name is constructed as:
/srv/<zone>/node/<device>/objects/<hash>.db
where:
<zone> = zone number (1 – 4).
<device> = device name (i.e. sdb1).
<hash> = hash of: (account name, container name, object name).
Swift Cluster
Tying this all together, a Openstack Swift Cluster consists of Proxy nodes and Storage nodes. A Storage node consists of multiple Storage devices. Storage Device data is accessed via the XFS file system. A Storage node thus contains a number of XFS file systems, one for each of its Storage device. A file system instance then, contains Account Databases, Container Databases, and Object files, which represents the accounts, containers, and objects that are stored on that storage device. Proxy nodes provide a REST interface to storage data. Proxy nodes do not store data and they use rings to determine the location (Storage nodes) of storage items.
Next week I will discuss the plusses and minusses of this schema, and present possible performance improvements.