Moebooru
Moebooru is a well maintained Danbooru-style image tag system, used by Yande.re, Konachan, and many others. It is written in Ruby, and can be a bit of a challenge to install. Since many other installation guides are insufficient, we've noted down our entire installation and configuration process for Eikonos.
Setting Up Moebooru
Get configuration tips from here:
Create Moebooru User
Create a specific non-login daemon user just for moebooru (Though it will have bash shell for setup purposes temporarily). Then create a systemd service for it.
sudo git clone https://github.com/moebooru/moebooru.git /var/www/booru.eikonos.org sudo useradd -s /bin/bash -d /var/www/booru.eikonos.org -r moebooru sudo chown -R moebooru:moebooru /var/www/booru.eikonos.org
Setup Postgresql
The examples below use PostgreSQL 9.6, but as of 2017-01-28, Moebooru just needs greater than 9.4.
Debian: (Alternatively, you can get it from jessie-backports and newer)
/etc/apt/sources.list.d/pgdg.list
deb http://apt.postgresql.org/pub/repos/apt/ YOUR_DEBIAN_VERSION_HERE-pgdg main
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc
# yum install http://yum.postgresql.org/9.6/redhat/rhel-7-x86_64/pgdg-redhat95-9.5-2.noarch.rpm
# yum install postgresql96 postgresql96-devel postgresql96-server libpqxx libpqxx-devel
libpqxx is required for libpq-ruby to build.
Now log in and create the moebooru user:
# su - postgres $ psql postgres # create user moebooru_user with password 'the_password' createdb;
Finally, edit pg_hba.conf from ident to md5 (except for the UNIX socket line) to allow users to log in using a password (required by moebooru's config), it should look like the following:
RHEL/CentOS: /var/lib/pgsql/9.6/data/pg_hba.conf Debian: /etc/postgresql/9.6/main/pg_hba.conf
# "local" is for Unix domain socket connections only local all all md5 # IPv4 local connections: host all all 127.0.0.1/32 md5 # IPv6 local connections: host all all ::1/128 md5 # Allow replication connections from localhost, by a user with the # replication privilege. #local replication postgres peer #host replication postgres 127.0.0.1/32 trust #host replication postgres ::1/128 trust
Once you have this file edited as seen above, restart postgresql.
sudo systemctl restart postgresql-9.6
Install Nodejs
NodeJS is necessary for the frontend. You should obtain the latest version, 7.x: by following these guides for your distro.
Setup Ruby
For most small sites, Ruby 2.3 and higher with the Unicorn server is sufficient.
A site with heavier traffic may find it helpful to use Rubinius with the Puma server make better use of concurrent multithreading, but this may require compilation.
Method 1: Set up Normal Ruby 2.3 or newer
Most distros have an outdated version of Ruby, so you must set up repositories from the official developers:
- Debian: Brightbox PPA - Ruby 2.3
- RHEL/CentOS: RHSCL - Ruby 2.3
Method 2: Install Rubinius with RVM
For RVM, just install any ol' ruby 2.x, we won't be using it after the compilation stage. Also install all the build dependencies.
Dependencies (Debian/Ubuntu)
sudo apt-get install ruby
sudo apt-get install build-essential openssl libreadline6 libreadline6-dev \
curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 \
libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison \
subversion pkgconfig
An LLVM version greater than 3.6+ is also required. You can get this in Debian Jessie from backports, or in Ubuntu from normal repositories.
# if on debian jessie, set up jessie-backports: https://backports.debian.org/Instructions/
sudo apt-get install llvm-3.8 clang-3.8 libclang-3.8-dev llvm-3.8-runtime llvm-3.8-dev llvm-3.8-tools libedit libedit-dev
Dependencies (RHEL/CentOS)
sudo yum install ruby
sudo yum install -y patch autoconf patch automake libtool bison sqlite-devel
One special build dependency is llvm-3.6+. Unfortunately, CentOS 7's EPEL repo only has 3.4, so we will need to get the latest build of LLVM from Fedora COPR.
/etc/yum.repos.d/daveisfera-llvm_3.7-epel-7.repo
[daveisfera-llvm_3.7] name=Copr repo for llvm_3.7 owned by daveisfera baseurl=https://copr-be.cloud.fedoraproject.org/results/daveisfera/llvm_3.7/epel-7-$basearch/ type=rpm-md skip_if_unavailable=True gpgcheck=1 gpgkey=https://copr-be.cloud.fedoraproject.org/results/daveisfera/llvm_3.7/pubkey.gpg repo_gpgcheck=0 enabled=1 enabled_metadata=1
Then just install llvm as normal:
sudo yum install clang llvm llvm-devel llvm-static libedit libedit-devel
Using RVM
http://rayhightower.com/blog/2014/02/06/installing-rubinius-using-rvm/
The right way to use it is just plain jane on the server. Installation is easy using rvm. Run these as the current user:
Refresh the rvm repos:
rvm get head
Install rubinius. Compilation will take a while.
rvm install rbx
After installation, choose Rubinius (rbx) as the default ruby version to use.
rvm list
This will list out various ruby versions.
rvm alias create default rbx
Finally, log out and log back in, and check your current ruby version to ensure that Rubinius is default.
ruby -v
Now that your ruby type is set up, install bundler.
gem install bundler
Mandatory Access Control
AppArmor (Debian/Ubuntu)
anyone have apparmor profiles?
SELinux Permissions (RHEL/CentOS)
If using SELinux (which we highly recommend), you will need the following policies, assuming that everything is installed to /var/www/booru.eikonos.org/:
needed for proxy pass
sudo chcon -Rt httpd_sys_content_t /var/www/ # allow nginx to access folders sudo setsebool httpd_can_network_connect 1 -P # allows reverse proxy sudo setsebool -P httpd_can_network_memcache 1 # allows memcache
needed to serve from moebooru user's directory, but only shared/ and public folders
setsebool -P httpd_enable_homedirs 1 sudo semanage fcontext -a -t httpd_sys_content_t '/var/www/booru.eikonos.org/shared(/.*)?' sudo restorecon -R -v /var/www/booru.eikonos.org/shared sudo semanage fcontext -a -t httpd_sys_content_t '/var/www/booru.eikonos.org/public(/.*)?' sudo restorecon -R -v /var/www/booru.eikonos.org/public
https://www.pckr.co.uk/selinux-nginx-and-reverse-proxying-2/
The last step is to run the final allows.
sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx > nginx.te
Open up the nginx.te file and see that it is correct (such that no suspicious rule allows are inside). Then run:
sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx sudo semodule -i nginx.pp
http://axilleas.me/en/blog/2013/selinux-policy-for-nginx-and-gitlab-unix-socket-in-fedora-19/
Setup Moebooru
Now, conduct the setup and get dependencies.
sudo yum install gcc gcc-c++ ImageMagick jhead libxslt-devel git libyaml-devel openssl-devel pcre-devel readline-devel
First, do a bundle config for pg since we're using a specific postgresql version:
bundle config build.pg --with-pg-config=/usr/pgsql-9.5/bin/pg_config
Generate your secret key, which is used for salts and such.
bundle exec rake secret
Install the ruby packages for the moebooru user only (under the directory ./vendor/bundle):
bundle install --path vendor/bundle
Create config/database.yml and config/local_config.rb from the .example files, and configure them accordingly. Then set chmod 700 so only the moebooru user can read the database password.
chmod 600 /var/www/booru.eikonos.org/config/database.yml
Initialize database with bundle exec rake db:reset (there will be some errors reported which is expected)
Run bundle exec rake db:migrate
Now, you need to provide the correct permissions to the public folder:
chmod 755 /var/www/booru.eikonos.org/public
Start the server (bundle exec unicorn or bundle exec puma if using JRuby/Rubinius)
Finally, set moebooru to a non login user:
sudo chsh -s /bin/false moebooru
Customize Header Image and Branding
By default, Moebooru comes with the Yande.re header image and branding, as the site developed the moebooru engine. You should definitely consider removing the original branding unless your site is private.
app/assets/images
public/favicon.ico
Enable Memcached
Memcached is a high performance caching solution and is needed to have Moebooru enumerate posts and create the /posts pagination bar. Follow these instructions to install Memcached. You need at least 2GB free RAM to provide.
sudo yum install memcached
http://www.liquidweb.com/kb/how-to-install-memcached-on-centos-7/
Edit /etc/sysconfig/memcached and set CACHESIZE=2048 (2GB RAM) if possible.
Then set memcached to start at every boot:
sudo systemctl restart memcached
Config-based Activation
Add these options to the following file. When you start moebooru again, memcached will be active.
config/local_config.rb
# The server and port where the memcache client can be accessed. Only relevant if you enable caching. CONFIG["memcache_servers"] = ["localhost:11211"] # This enables various caching mechanisms. You must have memcache (and the memcache-client ruby gem) installed in order for caching to work. CONFIG["enable_caching"] = true
ENV-based Memcached Activation
Activate it by appending a bash variable to the puma command: MB_MEMCACHE_SERVERS="<url-to-memcached>" . Here are some examples.
TCP: MB_MEMCACHED_SERVERS='127.0.0.1:11211' RAILS_ENV=production bundle exec puma -e production
UNIX Socket: MB_MEMCACHED_SERVERS='127.0.0.1:11211' RAILS_ENV=production bundle exec puma -C shared/puma.rb
SELinux Permissions
You will probably need to allow it through SELinux:
https://major.io/2011/09/07/getting-apache-php-and-memcached-working-with-selinux/
Production Mode
By default, Moebooru runs in development mode, which can be slow. Here are the steps to set up Production mode.
Preparation
First, you need to create the database, and pregenerate the javascript/css (do this every time you update):
RAILS_ENV=production bundle exec rake db:reset RAILS_ENV=production bundle exec rake assets:precompile
Then, you need to provide the correct permissions to the public folder:
chmod 755 /var/www/booru.eikonos.org/public
Serve static files with Nginx
Create an Nginx config under /etc/nginx/conf.d/. Make sure to change the server_name, and the root /var/www/booru.eikonos.org/public to root /YOUR/MOEBOORU/PATH/public.
Note: If you are using a different port for puma, (by adding
-p 3000to the serving command), also change the port accordingly below.
server {
listen 80;
server_name booru.eikonos.org;
# directory of static assets, first generate with the command:
# RAILS_ENV=production bundle exec rake assets:precompile
root /var/www/booru.eikonos.org/public;
try_files $uri/index.html $uri @app;
location @app {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the "It appears that your reverse proxy set up is broken" error.
proxy_pass http://127.0.0.1:9292;
proxy_read_timeout 90;
proxy_redirect http://127.0.0.1:9292 http://$server_name;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
Run the Server
Finally, to run the server (default is port 9292), run one of the following commands:
Ruby:
bundle exec unicorn -p 9292 -E production
Rubinius, JRuby:
RAILS_ENV=production bundle exec puma -e production
Serve with UNIX Socks in Production Mode
for even more effectiveness, use a UNIX sock: https://www.digitalocean.com/community/tutorials/how-to-deploy-a-rails-app-with-puma-and-nginx-on-ubuntu-14-04
Figure out the amount of CPU cores you have:
grep -c processor /proc/cpuinfo
Create the following folders in your application directory:
mkdir -p shared/pids shared/sockets shared/log
Place the following into <app_dir>/shared/puma.rb:
# Change to match your CPU core count
workers 8
# Min and Max threads per worker
threads 1, 6
app_dir = File.expand_path("../..", __FILE__)
shared_dir = "#{app_dir}/shared"
# Default to production
rails_env = ENV['RAILS_ENV'] || "production"
environment rails_env
# Set up socket location
bind "unix://#{shared_dir}/sockets/puma.sock"
# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true
# Set master PID and state locations
pidfile "#{shared_dir}/pids/puma.pid"
state_path "#{shared_dir}/pids/puma.state"
activate_control_app
on_worker_boot do
require "active_record"
ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
ActiveRecord::Base.establish_connection(YAML.load_file("#{app_dir}/config/database.yml")[rails_env])
end
Change the Nginx server config to the following:
upstream app {
# Path to Puma SOCK file, as defined previously
server unix:/var/www/booru.eikonos.org/shared/sockets/puma.sock fail_timeout=0;
}
server {
listen 80;
server_name booru.eikonos.org;
# directory of static assets, first generate with the command:
# RAILS_ENV=production bundle exec rake assets:precompile
root /var/www/booru.eikonos.org/public;
try_files $uri/index.html $uri @app;
location @app {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the "It appears that your reverse proxy set up is broken" error.
proxy_pass http://app;
proxy_read_timeout 90;
proxy_redirect http://app http://$server_name;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
Finally, to run the server, use the following:
bundle exec puma -C shared/puma.rb
Systemd Service
https://github.com/puma/puma/blob/master/docs/systemd.md
Save this to /usr/systemd/system/moebooru.service. There are two versions, one for TCP and one for unix socket. Change the WorkingDirectory accordingly.
TCP
[Unit] Description=Moebooru's Puma HTTP Server Requires=redis.service postgresql-9.5.service Wants=postgresql-9.5.service memcached.service After=network.target postgresql-9.5.service # Uncomment for socket activation (see below) # Requires=puma.socket [Service] # Foreground process (do not use --daemon in ExecStart or config.rb) Type=simple # Preferably configure a non-privileged user User=moebooru # Specify the path to your puma application root WorkingDirectory=/var/www/booru.eikonos.org # Helpful for debugging socket activation, etc. # Environment=PUMA_DEBUG=1 # The command to start Puma # Here we are using a binstub generated via: # `bundle binstubs puma --path ./sbin` # in the WorkingDirectory (replace <WD> below) # You can alternatively use `bundle exec --keep-file-descriptors puma` # ExecStart=<WD>/sbin/puma -b tcp://0.0.0.0:9292 -b ssl://0.0.0.0:9293?key=key.pem&cert=cert.pem # Alternatively with a config file (in WorkingDirectory) and # comparable `bind` directives ExecStart=/bin/bash -c 'RAILS_ENV=production /usr/bin/bundle exec puma -e production' Restart=always [Install] WantedBy=multi-user.target
UNIX Socket
Two Systemd services are needed: one for the socket and one for the application.
/usr/systemd/system/moebooru.service
[Unit] Description=Puma HTTP Server Requires=redis.service postgresql-9.5.service Wants=postgresql-9.5.service memcached.service After=network.target postgresql-9.5.service # Uncomment for socket activation (see below) # Requires=puma.socket [Service] # Foreground process (do not use --daemon in ExecStart or config.rb) Type=simple # Preferably configure a non-privileged user # User= # Specify the path to your puma application root # WorkingDirectory= # Helpful for debugging socket activation, etc. # Environment=PUMA_DEBUG=1 # The command to start Puma # Here we are using a binstub generated via: # `bundle binstubs puma --path ./sbin` # in the WorkingDirectory (replace <WD> below) # You can alternatively use `bundle exec --keep-file-descriptors puma` # ExecStart=<WD>/sbin/puma -b tcp://0.0.0.0:9292 -b ssl://0.0.0.0:9293?key=key.pem&cert=cert.pem # Alternatively with a config file (in WorkingDirectory) and # comparable `bind` directives # ExecStart=<WD>/sbin/puma -C config.rb Restart=always [Install] WantedBy=multi-user.target
Grab some code from here?
https://github.com/puma/puma/issues/976
SSL Certificates
While this is beyond the scope of this guide, you should strongly consider using SSL certificates, which are now free with Let's Encrypt.
https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-centos-7
Full Backup
PostgreSQL Database (run as moebooru user!):
pg_dump -U moebooru -W moebooru > eikonos.org-20170128.pg.sql # will ask for password
Images only:
tar -cvzf eikonos.org-20170128_images.tar.gz /var/www/booru.eikonos.org/public/data/image/
Code only, no binaries:
tar --exclude='/var/www/booru.eikonos.org/tmp' --exclude='/var/www/booru.eikonos.org/vendor' --exclude='/var/www/booru.eikonos.org/public' --exclude='/var/www/booru.eikonos.org/.rvm' --exclude='/var/www/booru.eikonos.org/.rbx' --exclude='/var/www/booru.eikonos.org/.bundle' --exclude='/var/www/booru.eikonos.org/.gem' -cvjf eikonos.org-20170128_var_www_booru-eikonos-org.tar.gz /var/www/booru.eikonos.org
