Maglevity

MagLev, Ruby, …

Multiple Repositories in MagLev

leave a comment »

MagLev stores all of its state, including application classes and code, in a repository. By default, MagLev has one repository named “maglev”. But there are times you may want to have multiple repositories. If you have multiple applications, you may want to store each in its own repository. Or, you may want one repository per developer, or different repositories for development, test, staging and deployment. MagLev provides some management tools that allow you to create, destroy, start, stop and perform other tasks on repositories.

Get Latest MagLev

While writing this post, I discovered a bug in the maglev-ruby command (the --stone option was being ignored). I also decided to add a stone_name method that returns the name of the stone the VM is connected to. If you want to try all of the examples in this post, you’ll need to ensure you have the latest MagLev. The install and upgrade instructions are in the GitHub README. You’ll need version 22804 or later.

Note on Terminology

A quick note about terminology before we begin. “repository” refers to the data (code, objects, etc.) safely stored on disk. The term “stone” is used rather loosely in GemStone parlance. It sometimes refers to the single stoned process that coordinates activities regarding a single repository. “stone” is sometimes used to refer to all the processes related to a particular repository. And sometimes, “stone” is used to (loosely) refer to the repository managed by the stone processes.

Default repository is “maglev”

$MAGLEV_HOME/Rakefile contains several tasks to manage repositories:

$ cd $MAGLEV_HOME
$ rake -T stone
(in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26)
rake stone:all[task_name]        # Invoke a task on all MagLev servers
rake stone:create[server_name]   # Create a new MagLev server and repository
rake stone:destroy[server_name]  # Destroy an existing MagLev server and repository
rake stone:list                  # List MagLev servers managed by this Rakefile

Here we see the first loose use of “stone”: the stone:* rake tasks actually manage the repository files, not the running “stone” processes (we’ll get to them in a bit). To list all of the stones (repositories) you have currently installed:

$ rake stone:list
(in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26)
maglev

The default installation has one stone named “maglev”. The repository files live under $MAGLEV_HOME/data/maglev:

$ find data
data
data/extent0.ruby.dbf
data/maglev
data/maglev/extent
data/maglev/extent/extent0.ruby.dbf
data/maglev/tranlog
data/maglev/tranlog/tranlog3.dbf

data/maglev/extent/extent0.ruby.dbf is the file that holds all of the data. This file has all of the objects and code that live in the repository. This is the file to backup! The tranlog subdirectory holds the transaction logs for the repository. These are the files that record each commit, and can be replayed in the event of a server crash to restore the repository. During a Maglev.commit_transaction, MagLev will wait until the tranlog entry for the current transaction is safely flushed to disk before it returns.

To see if the processes that manage the repository are running, you can use the status task (which is also the default rake task):

$ rake
(in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26)
Status  Version    Owner    Pid   Port   Started     Type  Name
------ --------- --------- ----- ----- ------------ ------ ----
  OK   3.0.0     pmclain   95141 57924 Jan 26 10:42 Stone  maglev
  OK   3.0.0     pmclain   95142 57916 Jan 26 10:42 cache  maglev@cairo.gemstone.com

Here we see that there is one running “stone” and it is named “maglev”. The cache line indicates that the Shared Page Cache (SPC) is up and running. All VMs on this machine that connect to the maglev stone will use this SPC to share data.

Create a new repository

If we want a new repository to use, we’ll have to do two things: (A) create some new repository files, most notably extent0.ruby.dbf, and (B) start the stone processes that will allow VMs to connect to the repository.

Create Repository Files

We do (A) with the stone:create[name] rake task:

$ rake stone:create[quux]
(in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26)
Creating server "quux"

At this point, we have new repository files and a configuration file for the stone named “quux”. The repository files are under $MAGLEV_HOME/data/quux, and the configuration file is $MAGLEV_HOME/etc/conf.d:

$ find data
data
data/extent0.ruby.dbf
data/maglev
data/maglev/extent
data/maglev/extent/extent0.ruby.dbf
data/maglev/tranlog
data/maglev/tranlog/tranlog3.dbf
data/quux
data/quux/extent
data/quux/extent/extent0.ruby.dbf
data/quux/tranlog

$ find etc/conf.d
etc/conf.d
etc/conf.d/maglev.conf
etc/conf.d/quux.conf

Start new Repository Processes

If we try to run a VM against quux, we’ll get an error, since we haven’t yet started the server processes:

$ maglev-ruby --stone quux -e 'puts "Hello from #{Maglev::System.stone_name}."'
maglev-ruby: [Error] The MagLev server "quux" is not running.
To fix this run "rake quux:start" before running maglev-ruby.

The Rakefile detects installed stones and creates stone specific tasks, including the tasks to start and stop the stone. Now that we have a stone named “quux”, there are several new rake tasks automatically generated for that stone:

$ rake -T quux
(in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26)
rake quux:reload            # Destroy the "quux" repository then load a fresh one
rake quux:restart           # Stop then start the "quux" server
rake quux:restore_snapshot  # Restore the "quux" repository from its previous snapshot
rake quux:start             # Start the "quux" server
rake quux:status            # Report status of the "quux" server
rake quux:stop              # Stop the "quux" server
rake quux:take_snapshot     # Stop the "quux" server then make a backup copy of its repository

Let’s start the quux server and check to see which stone processes are running:

$ rake quux:start
(in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26)
startstone[Info]: Starting Stone repository monitor "quux".
startstone[Info]: GemStone server 'quux' has been started.

$ rake
(in /Users/pmclain/GemStone/snapshots/MagLev-2010-01-26)
Status  Version    Owner    Pid   Port   Started     Type  Name
------ --------- --------- ----- ----- ------------ ------ ----
  OK   3.0.0     pmclain     740 61015 Jan 27 09:01 Stone  maglev
  OK   3.0.0     pmclain     741 61007 Jan 27 09:01 cache  maglev@cairo.gemstone.com
  OK   3.0.0     pmclain    1750 61206 Jan 27 09:57 Stone  quux
  OK   3.0.0     pmclain    1751 61198 Jan 27 09:57 cache  quux@cairo.gemstone.com

With the stone processes for both maglev and quux now running, we should be able to run off either stone. By default, maglev-ruby will start a VM that connects to the stone named “maglev”. To use a different stone, use the --stone name option:

# First, try quux:
$ maglev-ruby --stone quux -e 'puts "Hello from #{Maglev::System.stone_name}"'
Hello from quux

# Now try the default stone (maglev):
$ maglev-ruby -e 'puts "Hello from #{Maglev::System.stone_name}"'
Hello from maglev

Other management tasks

The other rake tasks do pretty much what you’d expect from the descriptions. Here is a brief description of the stone-specific tasks (there will be a set of these rake tasks for each stone you have installed):

  • rake quux:reload: Shutdown the quux server processes, if they are running. Destroy the database (yes…data loss). Copy a fresh repository into data/quux/extent, then re-start the quux servers.
  • rake quux:restart: Shutdown the server processes for quux and then restart them.
  • rake quux:take_snapshot: Make an off-line backup: Shutdown the quux server processes, make a backup of the repository, re-start the quux server processes.
  • rake quux:restore_snapshot: Restore the latest off-line backup: Shutdown the quux server processes, copy the previously saved repository backup into the extent directory, re-start the quux server processes.
  • rake quux:status: Reports status on just the quux server (running or not).

Note: it is also possible to make on-line backups (a backup without stopping the server). See the Admin Manual in the MagLev Documentation for more information.

And you can simply delete a stone and its configuration by using the rake stone:destroy task. To get rid of our quux stone:

$ rake stone:destroy[quux]
(in /Users/pmclain/GemStone/checkouts/git)
stopstone[Info]: initiating "quux" shutdown...
stopstone[Info]: Stone repository monitor "quux" has been stopped.
Destroying server "quux"

$ rake stone:list
(in /Users/pmclain/GemStone/checkouts/git)
maglev

Written by maglevdevelopment

January 29, 2010 at 2:16 pm

Posted in MagLev

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: