Multiple Repositories in MagLev
MagLev stores all of its state, including application classes and code, in a repository. By default, MagLev has one repository named “maglev”. But there are times you may want to have multiple repositories. If you have multiple applications, you may want to store each in its own repository. Or, you may want one repository per developer, or different repositories for development, test, staging and deployment. MagLev provides some management tools that allow you to create, destroy, start, stop and perform other tasks on repositories.
Get Latest MagLev
While writing this post, I discovered a bug in the maglev-ruby command (the --stone option was being ignored). I also decided to add a stone_name method that returns the name of the stone the VM is connected to. If you want to try all of the examples in this post, you’ll need to ensure you have the latest MagLev. The install and upgrade instructions are in the GitHub README. You’ll need version 22804 or later.
Note on Terminology
A quick note about terminology before we begin. “repository” refers to the data (code, objects, etc.) safely stored on disk. The term “stone” is used rather loosely in GemStone parlance. It sometimes refers to the single stoned process that coordinates activities regarding a single repository. “stone” is sometimes used to refer to all the processes related to a particular repository. And sometimes, “stone” is used to (loosely) refer to the repository managed by the stone processes.
Default repository is “maglev”
$MAGLEV_HOME/Rakefile contains several tasks to manage repositories:
$ cd $MAGLEV_HOME $ rake -T stone (in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26) rake stone:all[task_name] # Invoke a task on all MagLev servers rake stone:create[server_name] # Create a new MagLev server and repository rake stone:destroy[server_name] # Destroy an existing MagLev server and repository rake stone:list # List MagLev servers managed by this Rakefile
Here we see the first loose use of “stone”: the stone:* rake tasks actually manage the repository files, not the running “stone” processes (we’ll get to them in a bit). To list all of the stones (repositories) you have currently installed:
$ rake stone:list (in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26) maglev
The default installation has one stone named “maglev”. The repository files live under $MAGLEV_HOME/data/maglev:
$ find data data data/extent0.ruby.dbf data/maglev data/maglev/extent data/maglev/extent/extent0.ruby.dbf data/maglev/tranlog data/maglev/tranlog/tranlog3.dbf
data/maglev/extent/extent0.ruby.dbf is the file that holds all of the data. This file has all of the objects and code that live in the repository. This is the file to backup! The tranlog subdirectory holds the transaction logs for the repository. These are the files that record each commit, and can be replayed in the event of a server crash to restore the repository. During a Maglev.commit_transaction, MagLev will wait until the tranlog entry for the current transaction is safely flushed to disk before it returns.
To see if the processes that manage the repository are running, you can use the status task (which is also the default rake task):
$ rake (in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26) Status Version Owner Pid Port Started Type Name ------ --------- --------- ----- ----- ------------ ------ ---- OK 3.0.0 pmclain 95141 57924 Jan 26 10:42 Stone maglev OK 3.0.0 pmclain 95142 57916 Jan 26 10:42 cache maglev@cairo.gemstone.com
Here we see that there is one running “stone” and it is named “maglev”. The cache line indicates that the Shared Page Cache (SPC) is up and running. All VMs on this machine that connect to the maglev stone will use this SPC to share data.
Create a new repository
If we want a new repository to use, we’ll have to do two things: (A) create some new repository files, most notably extent0.ruby.dbf, and (B) start the stone processes that will allow VMs to connect to the repository.
Create Repository Files
We do (A) with the stone:create[name] rake task:
$ rake stone:create[quux] (in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26) Creating server "quux"
At this point, we have new repository files and a configuration file for the stone named “quux”. The repository files are under $MAGLEV_HOME/data/quux, and the configuration file is $MAGLEV_HOME/etc/conf.d:
$ find data data data/extent0.ruby.dbf data/maglev data/maglev/extent data/maglev/extent/extent0.ruby.dbf data/maglev/tranlog data/maglev/tranlog/tranlog3.dbf data/quux data/quux/extent data/quux/extent/extent0.ruby.dbf data/quux/tranlog $ find etc/conf.d etc/conf.d etc/conf.d/maglev.conf etc/conf.d/quux.conf
Start new Repository Processes
If we try to run a VM against quux, we’ll get an error, since we haven’t yet started the server processes:
$ maglev-ruby --stone quux -e 'puts "Hello from #{Maglev::System.stone_name}."'
maglev-ruby: [Error] The MagLev server "quux" is not running.
To fix this run "rake quux:start" before running maglev-ruby.
The Rakefile detects installed stones and creates stone specific tasks, including the tasks to start and stop the stone. Now that we have a stone named “quux”, there are several new rake tasks automatically generated for that stone:
$ rake -T quux (in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26) rake quux:reload # Destroy the "quux" repository then load a fresh one rake quux:restart # Stop then start the "quux" server rake quux:restore_snapshot # Restore the "quux" repository from its previous snapshot rake quux:start # Start the "quux" server rake quux:status # Report status of the "quux" server rake quux:stop # Stop the "quux" server rake quux:take_snapshot # Stop the "quux" server then make a backup copy of its repository
Let’s start the quux server and check to see which stone processes are running:
$ rake quux:start (in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26) startstone[Info]: Starting Stone repository monitor "quux". startstone[Info]: GemStone server 'quux' has been started. $ rake (in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26) Status Version Owner Pid Port Started Type Name ------ --------- --------- ----- ----- ------------ ------ ---- OK 3.0.0 pmclain 740 61015 Jan 27 09:01 Stone maglev OK 3.0.0 pmclain 741 61007 Jan 27 09:01 cache maglev@cairo.gemstone.com OK 3.0.0 pmclain 1750 61206 Jan 27 09:57 Stone quux OK 3.0.0 pmclain 1751 61198 Jan 27 09:57 cache quux@cairo.gemstone.com
With the stone processes for both maglev and quux now running, we should be able to run off either stone. By default, maglev-ruby will start a VM that connects to the stone named “maglev”. To use a different stone, use the --stone name option:
# First, try quux:
$ maglev-ruby --stone quux -e 'puts "Hello from #{Maglev::System.stone_name}"'
Hello from quux
# Now try the default stone (maglev):
$ maglev-ruby -e 'puts "Hello from #{Maglev::System.stone_name}"'
Hello from maglev
Other management tasks
The other rake tasks do pretty much what you’d expect from the descriptions. Here is a brief description of the stone-specific tasks (there will be a set of these rake tasks for each stone you have installed):
- rake quux:reload: Shutdown the quux server processes, if they are running. Destroy the database (yes…data loss). Copy a fresh repository into data/quux/extent, then re-start the quux servers.
- rake quux:restart: Shutdown the server processes for quux and then restart them.
- rake quux:take_snapshot: Make an off-line backup: Shutdown the quux server processes, make a backup of the repository, re-start the quux server processes.
- rake quux:restore_snapshot: Restore the latest off-line backup: Shutdown the quux server processes, copy the previously saved repository backup into the extent directory, re-start the quux server processes.
- rake quux:status: Reports status on just the quux server (running or not).
Note: it is also possible to make on-line backups (a backup without stopping the server). See the Admin Manual in the MagLev Documentation for more information.
And you can simply delete a stone and its configuration by using the rake stone:destroy task. To get rid of our quux stone:
$ rake stone:destroy[quux] (in /Users/pmclain/GemStone/checkouts/git) stopstone[Info]: initiating "quux" shutdown... stopstone[Info]: Stone repository monitor "quux" has been stopped. Destroying server "quux" $ rake stone:list (in /Users/pmclain/GemStone/checkouts/git) maglev
