Pure-ftpd capabilities for OpenVZ hosts

Trying to get pure-ftpd running on an OpenVZ virtual server and getting this message?

[ERROR] Unable to switch capabilities : Operation not permitted

Shut the VPS down, enter the host via SSH and enable needed capabilities for your VPS (where $VPS_ID is your VPS id, ie 101, 102…)


That’s it :-)

Set up a full development server for a Ruby on Rails project

Those who know me also know that one thing I’m specially concerned about is the importance of writing good documentation on everything you do. Specially for technical projects. Maybe it is not important for coders or designers, but for a sysadmin I think it should be definilety mandatory. If you get cool things working together but you don’t write any documentation, you won’t be able to play that music again.

Also, I am working in Ruby on Rails development since about 2 years ago. One of the things I have to do usually is to set up servers for hosting projects. And sometimes I have to set up a server for project development and tracking. These are the steps you can follow to set up a Ruby on Rails based server for development tracking.

1. Get a server

There are plenty of choices around the internet, some of them come completelly installed and prepared for deploying your app. They usually work the same way traditional LAMP hostings used to do, but using ssh key authentication instead of FTP sessions… some of them have also web based backend panels for deploying, migrating and managing your app gems.

I prefer to set up an empty GNU/Linux vanilla server, so that I can host everything I need, my way, no worries if PHP, Ruby or whatever. I can set up a relay mailserver, netfilter routing, traffic shaping, services for monitoring and differential backups that I can periodically rsync to my office local server. This is the way a sysadmin thinks, something that coders not always understand.

If you want to set up a development server I recomment you to take a look at most basic Linode VPS. Their control panel will let you run and destroy any Linux distro, setup and resize partitions, add failover addresses, disk space, processor and memory. And of course you’ll have full SSH root access. So I find it very useful and flexible for my needs.

Some considerations:

  • You’ll have to set up a DNS name for the server IP, i.e: http://devel.somedomain.net
  • You’ll need root access on the server. I’ll suppose you are working on a local computer, remotely logged via SSH to your server
  • I usually build my servers using Debian (Lenny for now), but a basic knowledge of any other distro package system will let you adapt my guide to your preferred system
  • I’ll suppose you’ve set up a specific environment for running your app on the development server (I usually call it beta)

2. Install screen

You may partially lose your work if for any reason your local computer hangs during the process. So it’s recommendable to install and use screen for working on a detachable console terminal. This way, if your computer shuts down, you’ll be able to boot again, ssh into your server and recover your screen session.

apt-get install screen

You’ll maybe want to set up your screen sessions to look nicer than withe on black, and give you some additional information of the remote system. I use a copy of the screenrc configuration file from my friend r0sk, it’s simple and nice and works sweet.

wget -c http://ivanhq.net/stuff/_screenrc
mv _screenrc ~/.screenrc

Enter screen


3. Base system

You’ll need an unprivileged user to host your web apps. I usually create a web user

mkdir /home/web
groupadd service
useradd -d /home/web -s /bin/bash -g service web
chown -R web:service /home/web/
passwd web        # type your preferred password here

Add contrib and non-free repositories to your Apt sources file (vim /etc/apt/sources.list)

deb http://ftp.us.debian.org/debian lenny main contrib non-free
deb-src http://ftp.us.debian.org/debian lenny main contrib non-free

Update your sources and then your system

apt-get update
apt-get upgrade

Edit your SSH server config file to disallow root login and enable SSH key authentication (vim /etc/ssh/sshd_config)

# change next line, swap "yes" for "no"
PermitRootLogin no
# uncomment next line
AuthorizedKeysFile      %h/.ssh/authorized_keys

Restart your SSH service

/etc/init.d/ssh restart

Install build-essentials for compiling and installing the environment dependencies

apt-get install build-essential

Install Postfix for using as MTA relay server, or MySQL Debian packages will install Exim (which I don’t really like)

apt-get install postfix

# select: internet site
# MTA's outgoing domain name: somedomain.net

4. Backports

Because of some software versions being old and deprecated on the APT sources, and some other not being present, an alternative repository can be set up for installing newer versions. The backports repository.

wget -O - http://backports.org/debian/archive.key | apt-key add -

Edit your APT sources file (vim /etc/apt/sources.list)

deb http://www.backports.org/debian lenny-backports main contrib non-free

Update your system again

apt-get update

5. MySQL

Install mysql service, and its ruby bindings package

apt-get install mysql-server mysql-client libmysqlclient15-dev libmysql-ruby

Create a database for your project (mysql -uroot -p)

mysql> CREATE DATABASE yourapp_beta;
mysql> GRANT ALL PRIVILEGES ON yourapp_beta.* TO yourapp_username@localhost IDENTIFIED BY 'your_superhacker_password123';

6. Ruby, Rubygems and Rails

Install the Ruby stuff, I prefer to use backports at this point to ensure latest versions.

apt-get -t lenny-backports install ruby1.8-dev ruby1.8 ri1.8 rdoc1.8 irb1.8 libreadline-ruby1.8 libruby1.8 libopenssl-ruby

Add unversioned links for the newly created binaries

ln -s /usr/bin/ruby1.8 /usr/local/bin/ruby
ln -s /usr/bin/ri1.8 /usr/local/bin/ri
ln -s /usr/bin/rdoc1.8 /usr/local/bin/rdoc
ln -s /usr/bin/irb1.8 /usr/local/bin/irb
ln -s /usr/bin/gem /usr/local/bin/gem

Install Rubygems from source, so you can update it directly whenever you want. Otherwise the system will update it for you, only when new packages for your distro are available.

cd /usr/local/src
wget -c http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz
tar zxvfp rubygems-1.3.7.tgz
cd rubygems-1.3.7
ruby setup.rb

Link gem binary to an unversioned name

ln -s /usr/bin/gem1.8 /usr/bin/gem

Update rubygems (not really needed, but just in case)

gem update
gem update --system

Install Rails

gem install rails

7. Image processing

Youll surelly want a plugin for cropping and thumbnailing images, maybe attachment_fu or paperclip. They both can work with Imagemagick.

apt-get install imagemagick

8. Apache

There are different web server layouts for hosting your application. Some people likes Nginx+Mongrel, some other prefer Apache+Passenger, others prefer Nginx+Passenger… I love Apache. Maybe because its syntax simplicity, its flexibility or maybe simply because as a sysadmin I’ve been working for years with it.

Install Apache

apt-get install apache2.2-common apache2-mpm-prefork apache2-prefork-dev libssl-dev

Rename default virtualhost

a2dissite default
mv /etc/apache2/sites-available/default /etc/apache2/sites-available/devel_somedomain_net

Edit that virtualhost (vim /etc/apache2/sites-available/devel_somedomain_net)and configure it with the minimum options to work. I’m adding a basic HTTP authentication so only people you give the password can see your project (it’s a development server heheh).

<VirtualHost *:80>
        ServerName devel.somedomain.net
        RailsEnv beta
        DocumentRoot /home/web/sites/devel.somedomain.net/current/public
        CustomLog /var/log/apache2/devel_somedomain_net/access.log combined
        ErrorLog /var/log/apache2/devel_somedomain_net/error.log
        <Directory /home/web/sites/devel_somedomain_net>
                AuthType Basic
                AuthName "Members Only"
                AuthUserFile /home/web/sites/.htpasswd_devel_somedomain_net
                <limit GET PUT POST>
                        require valid-user

Enable your new virtualhost

a2ensite devel_somedomain_net

As the unprivileged user running the service, create the application directory and the HTTP Auth passwords file

su - web
mkdir -p /home/web/sites/devel.somedomain.net
htpasswd -c /home/web/sites/.htpasswd_devel_somedomain_net username1
htpasswd /home/web/sites/.htpasswd_devel_somedomain_net username2


Create the apache log directory for your app

mkdir -p /var/log/apache2/devel_somedomain_net

Install Passenger

gem install passenger

Create a file for loading passenger module into Apache (vim /etc/apache2/mods-available/passenger.load). Be careful with versions on the names as they may have changed.

LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.2.15/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.2.15
PassengerRuby /usr/bin/ruby1.8

Enable passenger module

a2enmod passenger

Change the user and group executing Apache service to match your unprivileged user (vim /etc/apache2/envvars)

#export APACHE_RUN_USER=www-data
#export APACHE_RUN_GROUP=www-data
export APACHE_RUN_USER=web
export APACHE_RUN_GROUP=service

Restart Apache

/etc/init.d/apache2 restart

9. Git and Gitosis

If you don’t want to pay a Github account and you don’t want your source to be visible to everyone, you’ll want to host your own Git server. There are different choices for that, but I like Gitosis because it’s simple and I can manage it from my command line, only editing a config file.

Install Git

apt-get -t lenny-backports install git-arch

Gitosis is written in Python, so you need to install python’s setup-tools in order to install Gitosis

apt-get install python-setuptools

Clone and install Gitosis

cd /usr/local/src
git clone git://eagain.net/gitosis.git
cd gitosis
python setup.py install

Create a new unprivileged user for hosting Gitosis and its repositories. It does not need a password (so it can’t log in) but it needs a shell as all management occours using an SSH session.

adduser \
--system \
--shell /bin/sh \
--gecos 'git version control' \
--group \
--disabled-password \
--home /home/git \
usermod -g service git

Upload your ssh public key to the server, leave it in /tmp/ and call it user@host.pub being user@host the final string specified into the file itself (ivan@mbp-local, or something similar)

Initialize Gitosis using the SSH key you just uploaded, which will be the Gitosis admin

su - git
gitosis-init < /tmp/ivan\@ibelmonte-mbp.local.pub

In your local computer clone the Gitosis repository so you can manage it from now on

git clone git@devel.somedomain.net:gitosis-admin.git

Enter the gitosis-admin directory and check the contents. It has to show a filename called gitosis.conf and a directory called keydir.

10. Add a repository for your app

Edit the Gitosis config (vim gitosis.conf) and add a new group, formed with one user (or more, separated with spaces), and its repository

[group your_app_name]
members = user1@host1 user2@host2 ...
writable = your_repo_name

Make sure to put user’s keyfiles into the keydir directory, following the previously mentioned format. Upload your changes.

git add .
git commit -a -m "Added user2 to your_app group"
git push

In your local computer you can now create an application and add its files to the repo

rails your_app
cd your_app
git init
git remote add origin git@devel.somedomain.net:your_app.git
git add .
git commit -a -m "Initial import"
git push origin master:refs/heads/master

11. Capistrano

You’ll want to deploy your software to your newly configured server. Capistrano is the best choice I know, since it creates versioned directories to let you roll back and forth, remotely run tasks on the hosted version of your app, place a custom “under maintenance” page, etc…

Install Capistrano

gem install capistrano

Capify your application

cd /path/to/your/app
capify .

Create a recipe (vim config/deploy.rb)

# set a target name, so you can also use it to deploy to a production server

# set :targetname, 'www'
set :targetname, 'devel'
if targetname == 'devel'
  set :rails_env, 'beta'
set :application, "#{targetname}.somedomain.net"
set :deploy_to, "/home/web/sites/#{application}"
set :user, "web"
set :runner, "web"
set :repository, "git@devel.somedomain.net:your_app.git"
set :deploy_via, :remote_cache
set :copy_exclude, [".git", ".gitignore"]
set :scm, :git
set :use_sudo, false
# Master branch of course. If not, change
# set :branch, master
ssh_options[:forward_agent] = true
role :app, "#{targetname}.somedomain.net"
role :web, "#{targetname}.somedomain.net"
role :db,  "#{targetname}.somedomain.net", :primary => true
namespace :deploy do
  desc "Restarting passenger with restart.txt"
  task :restart, :roles => :app, :except => { :no_release => true } do
    run "touch #{current_path}/tmp/restart.txt"
  [:start, :stop].each do |t|
    desc "#{t} task is a no-op with mod_rails"
    task t, :roles => :app do ; end

12. SSH keys

In order to deploy your app without typing any password, you’ll have to set up a SSH key authentication between you and the unprivileged user of the server (web user).

Copy the content of your local ssh keyfile (commonly ~/.ssh/id_rsa.pub) and paste at the end of /home/web/.ssh/authorized_keys on the server. You’ll want to repeat this process for each developer so they can also deploy without typing a password.

You’ll also need to generate a SSH key for your web user on the server, and add it as a member of the repo in gitosis. Otherwise you won’t be able to deploy your app, because the server won’t let itself access to the repo for fetching.

13. Redmine

Redmine is one of those powerful tools that make your life easier when working with other developers, designers, webmasters and CEO’s… it helps you to keep tracking of issues, bugs, timings, files and documents, wikis and it also lets you graphically browse your repositories. If you’ve never tested it, now it’s your time.

NOTE: you’ll need to create a new entry on your DNS server. Set up a CNAME called redmine pointing to devel.somedomain.net

Redmine needs mod_rewrite support on Apache

a2enmod rewrite

You’ll need subversion to checkout the source code of Redmine

apt-get install subversion

Su to your unprivileged user

su - web

Checkout the code

cd sites
svn co http://redmine.rubyforge.org/svn/tags/1.0.1/ redmine.somedomain.net
cd redmine.somedomain.net

Create a database for redmine (mysql -uroot -p)

mysql> GRANT ALL PRIVILEGES ON redmine.* TO redmine@localhost IDENTIFIED BY 'redmine123';

Edit Redmine’s database configuration file (vim config/database.yml)

  adapter: mysql
  database: redmine
  host: localhost
  username: redmine
  password: redmine123
  encoding: utf8
  socket: /var/run/mysqld/mysqld.sock

Place a secret key for Redmine cookies on its environment file, inside the Initializer block (vim config/environment.rb)

config.action_controller.session = { :key => "_redmine_somedomain_com_session", :secret => "578359e95c3f2sj9enw10amkd81508ff" }

Migrate the database

env RAILS_ENV=production rake db:migrate

Edit the SMTP configuration for Redmine to be able to send notification emails (vim config/email.yml)

  delivery_method: :smtp
    address: "smtp.somedomain.net"
    port: '25'
    domain: "somedomain.net"
    authentication: :login
    user_name: "your_username"
    password: "your_password"

If you want to relay over Gmail or your domain’s email is hosted on Google apps, you’ll need to install a tls authentication plugin

ruby script/plugin install git://github.com/collectiveidea/action_mailer_optional_tls.git

Use this config for use with Gmail / Google apps

  delivery_method: :smtp
    tls: true
    address: "smtp.gmail.com"
    port: '587'
    domain: "gmail.com"   # use "somedomain.net" in case of Google Apps
    authentication: :plain
    user_name: "your_username"
    password: "your_password"

Exit su, and become root again


Create a new Apache virtualhost (vim /etc/apache2/sites-available/redmine_somedomain_net)

<VirtualHost *:80>
        ServerName redmine.somedomain.net
        RailsEnv production
        DocumentRoot /home/web/sites/redmine.somedomain.net/public
        CustomLog /var/log/apache2/redmine_somedomain_net/access.log combined
        ErrorLog /var/log/apache2/redmine_somedomain_net/error.log

Create an Apache log directory for redmine

mkdir /var/log/apache2/redmine_somedomain_net/

Enable your new virtualhost and restart Apache

a2ensite redmine.somedomain.net
/etc/init.d/apache2 restart

14. Start writing Ruby on Rails

Here comes the funny part. Enjoy! ;-)

Recursive crop with Imagemagick

A very simple and tiny shellscript for cropping images recursively with Imagemagick on Linux. In my case I am working on a Ruby on Rails website, managing images and its thumbnails with attachment_fu. After some months I decided to convert all user stored thumbnails to a cropped format. This is what this script was for.

# very little and tiny shellscript for cropping images
# massively and recursively with Linux and Imagemagick
# Make sure you have Imagemagick installed (ie: try to
# execute "convert" binary)
# USAGE: place this script in your images parent directory,
# edit and change the size names and widths
# Ivan Belmonte <ivan@ivanhq.net - http://ivanhq.net>

SIZES="icon:48 thumb:120" # change them for your needs
FORMATS="jpg JPG jpeg JPEG gif GIF png PNG"

# remove previously sized files
for SIZE in ${SIZES}; do
  SIZENAME=`echo ${SIZE} | cut -f 1 -d :`
  find . | grep -i _${SIZENAME} | xargs rm -f

# start resizing and cropping
for FORMAT in ${FORMATS}; do

  for FILE in `find . | grep ${FORMAT}`; do

    for SIZE in ${SIZES}; do

      SIZENAME=`echo ${SIZE} | cut -f 1 -d :`
      SIZEWIDTH=`echo ${SIZE} | cut -f 2 -d :`
      FILENAME=`echo ${FILE} | sed s/".${FORMAT}"//g`

      convert ${FILE} -resize \

      convert -gravity Center -crop \
      ${SIZEWIDTH}x${SIZEWIDTH}+0+0 \




echo "OK"

# enjoy :-)

Massive emailer

Yesterday I did a talk at Xing offices in Barcelona. It was about some tips on how to send emails massively from a Ruby on Rails application without being tagged as a spammer by Hotmail, Gmail or Yahoo. I’m not a dedicated expert on this, but I’ve been learning these things while developing last internet site, and I thought it may be also interesting for other developers.
Here are the slides (in english) of the presentation, formerly named “Massive emailing with Linux, Postfix and Ruby on Rails“.

Here is the PDF for downloading.

Remote pair programming

Pair programming es una técnica en la que dos programadores trabajan con una misma consola, sobre el mismo código. Normalmente uno hace la función de driver y el otro la de observer. El objetivo es evidente: crear un mejor código, a la vez que conseguir que ambos programadores lo conozcan con el mismo nivel de profundidad.

Hoy tenía un problema con el proyecto en el que estoy trabajando, y pedí a Mauro que me echase un cable a ver si entre los dos conseguíamos solucionarlo. Él estaba en su casa, y yo en la mía. Los ingredientes para cocinar un pair programming realmente fluído en remoto:

  • Port forwarding: un puerto para el acceso remoto por SSH y otro para el acceso al servidor de aplicaciones (acceso web)
  • screen: Usando una versión ligeramente modificada del screenrc de r0sk. Usamos tres consolas: [0] log del servidor, [1] vim con el código, [2] una bash para hacer accesos de mysql u otras cosas que necesitemos
  • vim: con los controles mapeados para trabajar con tabs, y el plugin de FuzzyFinderTextMate para buscar y abrir archivos… así se puede hacer todo en una sola consola
  • Skype: Para hablar todo el rato durante la sesión de programación

Los programadores maceros que usan TextMate suelen hacer el pair programming usando VNC, pero aunque no lo he probado pongo las dos manos en el fuego a que tiene que ser una cagada. La pareja screen+vim es brutal por lo lightweight y lo flexible, y consume muy poquito ancho de banda con lo que la comunicación del conjunto (ssh + skype) es perfecta.

La experiencia ha sido GENIAL, y hemos resuelto el problema en aproximadamente 1 hora. Es como trabajar en pair programming en local, tocando el mismo código y comentando todo el rato. Ha sido una pasada, y lo recomiendo a todos los programadores que trabajen separados físicamente del resto del equipo.

backupMussol 0.1

Ya son varios los amigos o colegas que me preguntan cómo hago los backups en los servidores del trabajo. En realidad uso sólamente tres herramientas: Mondo Rescue, DAR y el ya conocido rsync.

  • Mondo es una herramienta para crear imágenes del sistema autoarrancables (liveCD) y poderlas planchar en un CD o DVD (o en varios si es muy grande). Después se puede arrancar el sistema con ellas y restaurarlo así al estado en que se encontraba en el momento de sacar el backup.
  • DAR, por su parte, es una herramienta de backup diferencial.
  • Y rsync todos sabemos lo que es. Yo lo uso para hacer un mirror de los discos en lugar de usar RAID1, de esta forma tengo una ventana de tiempo de 24 horas para restaurar un backup en caso de fallo en el filesystem (ya que con un RAID1 se propaga al momento, y sí: lo he sufrido ya una vez, la primera y la última).

Todas estas herramientas las uso mediante un pequeño set de scripts, es muy sencillito y no tiene mucha complicación, así que he decidido liberarlos para que los podais usar si estais interesados.

He creado una página nueva (se ve en el menú lateral) llamada Mussol Scripts y lo he dejado ahí junto con un par más que había escrito anteriormente para otras cosas.

Espero que os sea de utilidad :-)

Serveralias en lighttpd

Valga cuanto menos a modo de recordatorio personal.

Un virtualhost habitual (atención al ==):

$HTTP["host"] == "www.something.net" {

Para que www.something.net, www.something.com o www.something.lo_que_sea apunten al mismo documentroot (atención al =~):

$HTTP["host"] =~ "www.something" {

Y para que www.something.net y www.otherthing.net apunten al mismo documentroot:

$HTTP["host"] =~ "(www.something.net|www.otherthing.net)" {

Project-open en Debian (update 1)

po-mini.jpgHe actualizado el mini tutorial de instalación de ]Project-open[ en Debian ETCH. Un chico llamado Martin Van Es me ha escrito explicando algunos trucos para hacer funcionar la versión 4 de Aolserver con la versión 8.x de PostgreSQL. Yo lo he probado y funciona perfectamente.

Aquí está el tuto. Como siempre, espero que sea de ayuda!

I have updated the mini-tutorial about installing ]Project-open[ on Debian ETCH. A guy called Martin Van Es sent me some emails with tips on making it work with Aolserver4 and PostgreSQL 8.1. I’ve tested it and works okay.

Here is the tutorial. I hope it helps!

Paquetes de Slackware con makepkg

Siempre cuando he impartido cursos de Linux lo he hecho bajo Slackware. Uno de los temas que me parecen más interesantes es la gestión de paquetes. Entre otras cosas porque es prácticamente lo que más se le ha echado en cara siempre a Slackware desde Debian o Red Hat.

Slackware tiene herramientas de gestión de paquetes, y son flexibles y funcionan perfectamente. Desde luego jamás he visto petar un paquete de slackware como he visto petar un paquete instalado con apt.
El sistema de paquetes de Slackware es muy sencillo: descomprimir el binario, copiar los contenidos a su sitio y guardar una base de datos de referencia para borrarlos cuando se desee. No falla. ¿Para qué más?

Ahora bien, no hay paquetes de slackware para todo lo que se desea. Así que aquí va una pequeña explicación de cómo crear paquetes para Slackware usando makepkg, una de las herramientas propias de la distribución.

1. Crear un área de trabajo para compilar el paquete

mkdir /work
mkdir /work/scripts
mkdir /work/builds

2. Descargar el fuente y compilarlo

cd /usr/local/src
tar zxvfp progname-progversion.tar.gz
cd progname-progversion/
./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var

3. Instalarlo dentro del área de trabajo

make install DESTDIR=/work/builds
cp -Rp doc/ /work/builds/usr/
# copiar más cosas que se quieran incorporar al paquete

4. Crear el paquete

cd /work/builds
makepkg progname-progversion-arch-build.tgz
# contestar SI a la pregunta sobre enlaces y permisos

5. Mover el paquete a su sitio y limpiar los restos

mkdir -p /var/spool/packages
mv progname-progversion-arch-build.tgz /var/spool/packages/
rm -rf *

6. Instalar el paquete

cd /var/spool/packages
installpkg progname-progversion-arch-build.tgz

7. Praise Bob!


]Project-open[ on Debian

po-mini1.jpgHe escrito una página con las instrucciones y los paquetes para instalar ]Project-open[ en Debian ETCH. Nunca había hecho paquetes de Debian, así que supongo que serán un poco primitivos. En cualquier caso son perfectamente funcionales.
También lo he posteado en la web de la empresa.
Será bien recibida cualquier aportación o comentario :-)


I’ve written a page with instructions and packages for installing ]Project-open[ on Debian ETCH. I’ve never built any debian package, so maybe they’re so primitive. But anyway they’re completely functional.
I’ve posted it on my company’s website too.
Comments will be appreciated :-)