Category Archives: Uncategorized

Compiling Mozilla’s SpiderMonkey 1.8.5 and building an RPM out of it for CentOS 6.x

Hey Squirrels,

We recently came across a situation where we wanted to install CouchDB on CentOS 6.x

As you can imagine from the title of this post, one of the requirements was to install SpiderMonkey 1.8.5 – “Mozilla’s JavaScript engine written in C/C++“.

Here’s how we compiled it and made an RPM package out of it – you can find this package in our Squirrel5 RPM repository here.

1. Create a target directory – we’ll use this as a target for the files that will make up our RPM package.

mkdir /target

2. Get the prerequisites to compile the package:

yum -y install make gcc gcc-c++ perl zip rpm-build

3. Download, configure, compile the source and install it to our target directory

cd /tmp
wget http://ftp.mozilla.org/pub/js/js185-1.0.0.tar.gz
tar -xzf js185-1.0.0.tar.gz
cd js-1.8.5/js/src/
./configure --prefix=/target
time make -j`nproc`
# this took about 5 minutes on our 2 core VPS
make install

4. Now the source is compiled and installed under /target – let’s go take a look:

cd /target/lib
ls -al
[root@squirrel5test lib]# ls -al
total 9040
drwxr-xr-x 3 root root    4096 Dec 12 05:28 .
drwxr-xr-x 5 root root    4096 Dec 12 05:28 ..
-rwxr-xr-x 1 root root 5571078 Dec 12 05:24 libmozjs185-1.0.a
lrwxrwxrwx 1 root root      30 Dec 12 05:28 libmozjs185.so -> /target/lib/libmozjs185.so.1.0
lrwxrwxrwx 1 root root      32 Dec 12 05:28 libmozjs185.so.1.0 -> /target/lib/libmozjs185.so.1.0.0
-rwxr-xr-x 1 root root 3667244 Dec 12 05:24 libmozjs185.so.1.0.0
drwxr-xr-x 2 root root    4096 Dec 12 05:28 pkgconfig

5. See those symlinks? they’re not going to work ! if we pack this up the symlinks will keep pointing to a /target directory which will no longer exist. We’ll delete the symlinks and replace them with relative ones:

rm -f libmozjs185.so
rm -f libmozjs185.so.1.0
#recreate the symlinks
ln -s libmozjs185.so.1.0.0 libmozjs185.so.1.0
ln -s libmozjs185.so.1.0 libmozjs185.so

So now the output of ls -al should look like this:

[root@squirrel5test lib]# ls -al
total 9040
drwxr-xr-x 3 root root    4096 Dec 12 06:19 .
drwxr-xr-x 5 root root    4096 Dec 12 06:18 ..
-rwxr-xr-x 1 root root 5571078 Dec 12 06:18 libmozjs185-1.0.a
lrwxrwxrwx 1 root root      18 Dec 12 06:19 libmozjs185.so -> libmozjs185.so.1.0
lrwxrwxrwx 1 root root      20 Dec 12 06:19 libmozjs185.so.1.0 -> libmozjs185.so.1.0.0
-rwxr-xr-x 1 root root 3667244 Dec 12 06:18 libmozjs185.so.1.0.0
drwxr-xr-x 2 root root    4096 Dec 12 06:18 pkgconfig

6. Now install Ruby and fpm – we need those to create the RPM package:

yum -y install http://rpm.squirrel5.com/squirrel5-repo-1.0-1.x86_64.rpm
yum -y install ruby
gem install fpm --no-ri --no-rdoc

7. Use fpm to create an RPM out of the current directory:

cd /target
fpm --verbose -v 0.1 -n libmozjs185-dev -s dir -t rpm .=/usr/

You’ll see output like this:

Executing(%clean): /bin/sh -e /tmp/rpm-tmp.UIOsJQ {:level=>:info}
Created package {:path=>"libmozjs185-dev-0.1-1.x86_64.rpm"}

8. You can install the RPM package by typing:

yum -y install /target/libmozjs185-dev-0.1-1.x86_64.rpm

Done!

Sources:

An introduction to lsyncd

Hello squirrels,

Today we’re going to do a gentle introduction to lsyncd. lsyncd is a file synchronizer. Many folks confuse lsyncd with it’s cousin rsync. The difference is that rsync is a one-off solution (unless you automate it via cron etc.). lsyncd will use inotify to detect changes to files and will launch rsync for you. That’s right – lsyncd uses rsync.

Here’s how to set it up:

1. Get the lsyncd dependencies (one of them is rsync so we install the IUS repository to get the most recent version:

yum install -y https://dl.iuscommunity.org/pub/ius/stable/CentOS/6/x86_64/ius-release-1.0-14.ius.centos6.noarch.rpm
yum install -y rsync31u
yum install -y cmake lua-devel gcc gcc-c++ openssh-clients

2. Get the lsyncd source code – at the time of writing the latest version was 2.1.6:

cd /tmp
curl -LO https://github.com/axkibe/lsyncd/archive/release-2.1.6.tar.gz
tar -xzf release-2.1.6.tar.gz
cd lsyncd-release-2.1.6/

3. Now compile lsyncd:

cmake .
make
make install

Also copy the manual page in place in case you need it:

cp doc/lsyncd.1 /usr/local/share/man/man1/

This will install lsyncd under /usr/local/bin.

4. Check the version:

lsyncd -version
Version: 2.1.6

5. Create an SSH key pair to use specifically for lsyncd (optional but good idea)

ssh-keygen -f ~/.ssh/squirrel5-lsyncd -N ''

6. Also add the key in your authorized_keys file so that you can still ssh in after the sync is done

cat /root/.ssh/squirrel5-lsyncd.pub >> /root/.ssh/authorized_keys

7. Now add a config file under /usr/local/etc/lsyncd.conf:

In a previous post we showed how to use rsync to migrate CentOS from one machine to another. Let’s do the exact same thing using rsync – for reference here’s the rsync command we used:

time /usr/bin/rsync -aHAXv --delete-after --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/etc/sysconfig/network-scripts/*"} / root@222.222.222.222:/

This is what the exact same thing would look like in lsyncd:

settings {
	log = all,
        logfile    = "/var/log/lsyncd.log",
        statusFile = "/tmp/lsyncd.status",
        pidfile    = "/var/run/lsyncd.pid",
        nodaemon   = false,
}

sync{
     	default.rsyncssh,
        source="/",
        host="222.222.222.222",
        targetdir="/",
        delay=30,
        maxDelays=10000,
	delete=running,
        exclude={ "dev/*", "proc/*", "sys/*","tmp/*","run/*","mnt/*"
                ,"media/*", "lost+found", "etc/sysconfig/network-scripts/*" },
        rsync = {
                compress = false,
                checksum = true,
                 _extra = {"-e", "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /root/.ssh/squirrel5-lsyncd -l root"}
        }
}

As you can see the above includes the excluded directories and SSH options such as which private key to use and which user to connect as.

8. Start lsyncd by pointing it to the configuration file and setting the -nodaemon option so we can see all of it’s output:

lsyncd /usr/local/etc/lsyncd.conf -nodaemon

Note:

If you get an error like this:

06:05:41 Error: Terminating since out of inotify watches.
Consider increasing /proc/sys/fs/inotify/max_user_watches

Check what the number is currently:

cat /proc/sys/fs/inotify/max_user_watches
8192

or:

sysctl fs.inotify.max_user_watches
fs.inotify.max_user_watches = 8192

set it to 15 million:

sysctl fs.inotify.max_user_watches=15000000

9. We’re going to control lsyncd using monit – install it with:

yum install -y monit
chkconfig monit on && chkconfig --list monit

10. Add a very simple configuration for lsyncd in /etc/monit.d/lsyncd:

check process lsyncd pidfile /var/run/lsyncd.pid
 start program = "/usr/local/bin/lsyncd /usr/local/etc/lsyncd.conf"
 stop program  = "/bin/kill `/bin/cat /var/run/lsyncd.pid`"

11. Start lsyncd:

monit start lsyncd

12. Monitor it’s status:

monit status lsyncd
The Monit daemon 5.14 uptime: 4m 

Process 'lsyncd'
  status                            Running
  monitoring status                 Monitored
  pid                               1929
  parent pid                        1
  uid                               0
  effective uid                     0
  gid                               0
  uptime                            1m 
  children                          2
  memory                            2.3 MB
  memory total                      6.1 MB
  memory percent                    0.2%
  memory percent total              0.6%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Sat, 03 Dec 2016 20:05:27

Note:

If the output you get looks like:

monit status lsyncd
The Monit daemon 5.14 uptime: 2m 

Process 'lsyncd'
  status                            Initializing
  monitoring status                 Initializing
  data collected                    Sat, 03 Dec 2016 20:01:08

Don’t worry about it – give monit a couple more minutes and it will get the status 🙂

Sources: