5. Configure MogileFS

A MogileFS setup comprises three components: storage nodes, trackers (mogilefsd), and the tracker's database(s). It is a distributed filesystem, allowing file storage and retrieval, and file replication across devices on different host machines. All three components can be run on multiple machines, so there is no single point of failure. A minimum of two machines is recommended. The time on all machines must be set accurately and kept in sync among the machines. You can run it on one machine, perhaps for a LiveJournal development installation.

5.1. Install MogileFS

First you need the MogileFS server code from SVN (http://code.sixapart.com/svn/mogilefs/trunk/). This provides the tracker and storage daemon. It is is automatically downloaded for you into your cvs subdirectory $LJHOME/cvs/mogilefs/server. You can build the code so it is ready for installation, with:

# cd $LJHOME/cvs/mogilefs/server
# perl Makefile.PL
# make
# make test

Assuming everything went well continue to the next step. If you received errors about missing module dependencies when running make test, obtain the missing dependencies through your Linux distribution or CPAN. Run the next command to install.

# make install

You also need the MogileFS utilities, these are automatically checked out into your cvs subdirectory $LJHOME/cvs/mogilefs/utils. The utilities themselves (mogadm and mogtool) are installed into your $LJHOME/bin directory. To set up the necessary libraries for the utilities, build and install the code as before:

# cd $LJHOME/cvs/mogilefs/utils
# perl Makefile.PL
# make
# make test
# make install

Next, you neeed the MogileFS client API (MogileFS::Client). from SVN (http://code.sixapart.com/svn/mogilefs/trunk/api/perl). This is required for the utilities. It is automatically checked out into your cvs subdirectory $LJHOME/cvs/mogilefs/utils and mapped into $LJHOMEcgi-bin/MogileFS. To set up the necessary libraries, build and install the code as before:

# cd $LJHOME/cvs/mogilefs/api/perl/MogileFS-Client
# perl Makefile.PL
# make
# make test
# make install

5.2. Set up MogileFS

First set up some directories for your MogileFS installation:

# mkdir /var/mogdata
# mkdir /var/mogdata/dev1
# chown -R lj:lj /var/mogdata

If you are using Perlbal, you should now set up some further links between directories:

$ ln -s $LJHOME/cvs/perlbal/lib/Perlbal $LJHOME/cvs/mogilefs/server/lib/Perlbal
$ ln -s $LJHOME/cvs/perlbal/lib/Perlbal.pm $LJHOME/cvs/mogilefs/server/lib/Perlbal.pm

Next, enable MogileFS in ljconfig.pl, by uncommenting the %LJ::MOGILEFS_CONFIG hash. The default setup in that hash is okay to use. Make sure you explicitly set the hosts key (it can be an arrayref of ip:port combinations) to point to your local MogileFS server. Add the classes for serving things like userpics over MogileFS, at the same time.

%LJ::MOGILEFS_CONFIG = (
    domain => 'example.com::lj',   # arbitrary namespace, not DNS domain
    hosts  => ['127.0.0.1:6001'],
    root => '/var/mogdata',
    classes => { userpics => 3,
        captcha => 2,
        # 'your_class' => 3,       # define any special MogileFS classes you need
        },
);

$LJ::CAPTCHA_MOGILEFS = 1;
$LJ::USERPIC_MOGILEFS = 1;

Next, set up database permissions for MogileFS.

$ mysql -uroot -p
mysql> GRANT ALL PRIVILEGES ON mogilefs.* TO 
     > mog@'localhost' IDENTIFIED BY 'mogpass';
     > \q

Now you should edit the tracker configuration file. You can copy the example file as a starting point.

$ cp $LJHOME/cvs/mogilefs/server/conf/mogilefsd.conf $LJHOME/etc/
$ nano $LJHOME/etc/mogilefsd.conf

You should change the values such as db_user and db_pass, to match the values you entered setting up the database in the previous step.

# daemonize = 1
db_dsn = DBI:mysql:mogilefs:host=localhost
db_user = mog
db_pass = mogpass
listen = 127.0.0.1:7001
conf_port = 7001
mog_root = /var/mogdata
listener_jobs = 2
delete_jobs = 1
replicate_jobs = 2
reaper_jobs = 1

Place the necessary directories in your path [4], and start up the daemons.

$ export PERL5LIB=/home/lj/cvs/Sys-Syscall/lib:
/home/lj/cvs/Danga-Socket/lib:/home/lj/cvs/perlbal/lib:
/home/lj/cvs/gearman/api/perl/Gearman-Client-Async/lib:
/home/lj/cvs/gearman/api/perl/Gearman/lib:
/home/lj/cvs/mogilefs/server/lib:
/home/lj/cvs/mogilefs/api/perl/MogileFS-Client/lib

$ cd $LJHOME/cvs/mogilefs/server
$ ./mogdbsetup --yes
$ ./mogstored -d
$ ./mogilefsd --daemon --config=$LJHOME/etc/mogilefsd.conf

$ sleep 1[5]
$ cd ..
$ utils/mogadm --lib=api/perl --trackers=127.0.0.1:7001 host add local --ip=127.0.0.1 --status=alive
$ utils/mogadm --lib=api/perl --trackers=127.0.0.1:7001 device add local 1
$ utils/mogadm --lib=api/perl --trackers=127.0.0.1:7001 domain add example.com::lj

You may find it easier to create a script to set the path and execute that sequence of commands, for starting-up the MogileFS daemons in future. You are free to figure that out yourself. The domain part in the last line needs to match what you put in your %LJ::MOGILEFS_CONFIG hash for the value of the domain key.

Last, update the database for MogileFS, to reflect your setup in %LJ::MOGILEFS_CONFIG.

$ bin/upgrading/update-db.pl -p


[4] The export command is one long line. It is separated onto multiple lines here for readability.

[5] The sleep command is only needed if you are using these commands in a script.