Editing Moebooru

From Bibliotheca Anonoma

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 1: Line 1:
{{Note|This guide is up to date and tested as of 2017-01-28. Moebooru does evolve gradually, so installation steps may change with time. If you find any divergences, please create an account and fix them.}}
= Setting Up Moebooru =


'''Moebooru''' is a well maintained Danbooru-style image tag system, used by Yande.re, Konachan, and many others. It is written in Ruby, and can be a bit of a challenge to install. Since many other installation guides are insufficient, we've noted down our entire installation and configuration process for [https://booru.eikonos.org Eikonos].
Get configuration tips from here:


== Assumptions ==
http://wiki.douglasqsantos.com.br/doku.php/deploying_a_rails_app_on_debian_jessie_with_capistrano_nginx_and_puma_en
 
In this guide, we make certain assumptions about your configuration. Your situation may vary.
 
* [[Nginx]] - The webserver used in this guide. Follow the instructions in that page to install Nginx. Then, you can find some [[Moebooru#Serve_static_files_with_Nginx|sample Nginx configurations for Moebooru]] in sections below.
  * www-data - A group used by all webservers. Make sure to add the nginx user to this group!
* {{ic|moe}} user - The unprivileged user that runs this app. After configuration, this will be converted to a nonlogin user.
* {{ic|/home/moe/booru/}} - The home folder of the moebooru user, where the application data will be stored from a {{ic|git clone}}.
** {{ic|/home/moe/booru/shared/}} - The location of the UNIX sockets that puma will use. It is probably safer to put it into {{ic|/var/run/moebooru}} instead, but it is what it is and we have SELinux policies to account for it.
* {{ic|/var/www/moebooru/public/}} - The public directory will be moved to {{ic|/var/www/}}, and symlinked to {{ic|/home/moe/booru/public/}}. This way, Nginx will not have control over the entire application directory.
** Make sure to restore this symlink every time an upgrade is run with {{ic|git pull}}.


== Create Moebooru User ==
== Create Moebooru User ==


Create a specific non-login daemon user just for moebooru (Though it will have bash shell for setup purposes temporarily).
https://github.com/moebooru/moebooru
 
{{bc|
sudo adduser moe      # create whatever password
sudo passwd -d moe    # delete password to prevent normal login
sudo -i -u moe        # become the moe user
git clone https://github.com/moebooru/moebooru.git /home/moe/booru
chown -R moe:moe /home/moe/booru
|lang=bash}}
 
As root, we also need to move all publicly served assets to {{ic|/var/www/}}.


{{bc|
Create a specific non-login daemon user just for moebooru (Though it will have bash shell for setup purposes temporarily). Then create a systemd service for it.
sudo mkdir -p /var/www/moebooru
sudo mv /home/moe/booru/public /var/www/moebooru/
sudo ln -s /var/www/moebooru/public /home/moe/booru/public
sudo chown moe:moe /home/moe/booru/public
sudo chmod 755 /var/www/moebooru/public
sudo chown root:www-data -R /var/www/moebooru
|lang=bash}}


<pre>sudo git clone https://github.com/moebooru/moebooru.git /var/www/booru.eikonos.org
sudo useradd -s /bin/bash -d /var/www/booru.eikonos.org -r moebooru
sudo chown -R moebooru:moebooru /var/www/booru.eikonos.org</pre>
== Setup Postgresql ==
== Setup Postgresql ==


The examples below use PostgreSQL 9.6, but as of 2017-01-28, Moebooru just needs greater than 9.4.
<pre>sudo rpm -ivh http://yum.postgresql.org/9.5/redhat/rhel-7-x86_64/pgdg-centos95-9.5-2.noarch.rpm
 
sudo yum install postgresql95 postgresql95-devel libpqxx libpqxx-devel</pre>
[https://www.postgresql.org/download/linux/debian/ Debian:] (Alternatively, you can get it from jessie-backports and newer)
<code>libpqxx</code> is required for libpq-ruby to build.
 
{{hc|/etc/apt/sources.list.d/pgdg.list|
deb http://apt.postgresql.org/pub/repos/apt/ YOUR_DEBIAN_VERSION_HERE-pgdg main
}}
 
{{bc|
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
}}
 
[https://www.postgresql.org/download/linux/redhat/ RHEL/CentOS:]
 
{{bc|
# yum install http://yum.postgresql.org/9.6/redhat/rhel-7-x86_64/pgdg-redhat95-9.5-2.noarch.rpm  
# yum install postgresql96 postgresql96-devel postgresql96-server libpqxx libpqxx-devel
}}
 
{{ic|libpqxx}} is required for libpq-ruby to build.


Then, initialize and start the database.
Now log in and create the moebooru user:


{{bc|
<pre># su - postgres
# /usr/pgsql-9.6/bin/postgresql96-setup initdb
# systemctl start postgresql-9.6
}}
 
Now log in and create the moebooru database and user with createdb permissions: (make sure to give it a very good password!)
 
{{bc|$ sudo su    # becoming the postgres user doesn't really work with sudo for some reason
# su - postgres
$ psql
$ psql
postgres # CREATE USER moebooru_user WITH PASSWORD 'your_password' CREATEDB;
postgres# create user moebooru_user with password 'the_password' createdb;</pre>
}}
Finally, edit <code>/var/lib/pgsql/9.5/data/pg_hba.conf</code> from <code>ident</code> to <code>md5</code> (except for the UNIX socket line) to allow users to log in using a password (required by moebooru's config):
Finally, edit {{ic|pg_hba.conf}} from {{ic|ident}} to {{ic|md5}} (except for the UNIX socket line) to allow users to log in using a password (required by moebooru's config), it should look like the following:


{{hc|'''RHEL/CentOS''': /var/lib/pgsql/9.6/data/pg_hba.conf    '''Debian''': /etc/postgresql/9.6/main/pg_hba.conf|<nowiki>
https://gist.github.com/groveriffic/4122225
# "local" is for Unix domain socket connections only
local  all            all                                    md5
# IPv4 local connections:
host    all            all            127.0.0.1/32            md5
# IPv6 local connections:
host    all            all            ::1/128                md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local  replication    postgres                                peer
#host    replication    postgres        127.0.0.1/32            trust
#host    replication    postgres        ::1/128                trust
</nowiki>}}
 
Once you have this file edited as seen above, restart postgresql.
 
<pre>
sudo systemctl restart postgresql-9.6
</pre>
 
{{Warning|If this postgresql server is on the same machine, make sure the firewall is configured to prevent remote access to postgresql ports. If this postgresql server is on another machine in the network/internet, make sure moebooru is connecting via SSL.}}


<blockquote>'''Advisory:''' If this postgresql server is on the same machine, make sure the firewall is configured to prevent remote access to postgresql ports. Also ensure that SSH keys are used instead of passwords. If this postgresql server is on another machine in the network/internet, make sure moebooru is connecting via SSL.
</blockquote>
== Install Nodejs ==
== Install Nodejs ==


NodeJS is necessary for the frontend. You should obtain the latest version, 7.x: [https://nodejs.org/en/download/package-manager/ by following these guides for your distro.]
NodeJS is necessary for the frontend. You should obtain the latest version, 6.x:
 
== Setup Ruby ==
 
''For most small sites'', Ruby 2.3 and higher with the Unicorn server is sufficient.
 
''A site with heavier traffic'' may find it helpful to use Rubinius with the Puma server [https://blog.engineyard.com/2014/ruby-app-server-arena-pt1to make better use of concurrent multithreading,] but this may require compilation, which can be fiendishly difficult.
 
{{Warning|If you can't get Rubinius to work or to compile your gems, don't waste your time on it anymore. Just use standard Ruby 2.3.}}
 
=== Method 1: Set up Normal Ruby 2.3 or newer ===
 
Most distros have an outdated version of Ruby, so you must set up repositories from the official developers:
 
* Debian: [https://www.brightbox.com/docs/ruby/ubuntu/ Brightbox PPA - Ruby 2.3]
* RHEL/CentOS: [https://www.softwarecollections.org/en/scls/rhscl/rh-ruby23/ RHSCL - Ruby 2.3]
 
=== Method 2: Installing Rubinius Binaries with chruby ===
 
{{hc|Debian/Ubuntu|
sudo apt-get install openssl libreadline6 libreadline6-dev \
curl zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 \
libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev libtool  \
pkg-config libgdbm-dev ncurses-dev libffi-dev
}}
 
{{hc|Red Hat/CentOS|
sudo yum install ruby
sudo yum install -y patch autoconf libtool sqlite-devel
}}
 
Rubinius is particularly tough to compile, and as of 2017-01-28 we have not been able to compile it with Debian Jessie. Instead, [https://rubinius.com/install/ Rubinius provides Ubuntu 14.04 binaries designed for use with Travis-Cl.]
 
First, download and install [https://github.com/postmodern/chruby chruby] as a user with sudo privileges.
 
{{bc|
wget -O chruby-0.3.9.tar.gz https://github.com/postmodern/chruby/archive/v0.3.9.tar.gz
tar -xzvf chruby-0.3.9.tar.gz
cd chruby-0.3.9/
sudo make install
}}
 
Next, log in as the {{ic|moe}} user, download the latest rubinius binaries, and extract them to {{ic|/home/moe/}}:
 
{{bc|
sudo -i -u moe
wget -O rubinius-3.70.tar.bz2 https://rubinius-binaries-rubinius-com.s3.amazonaws.com/ubuntu/14.04/x86_64/rubinius-3.70.tar.bz2
tar -xjvf rubinius-3.70.tar.bz2
}}
 
Then, activate chruby with rubinius by adding it to the bashrc.
 
{{hc|/home/moe/.bashrc|<nowiki>
source /usr/local/share/chruby/chruby.sh


RUBIES+=(
Run this script as root:
  "$HOME/rubinius/3.70"
)
</nowiki>}}


Finally, log out and log back into the {{ic|moe}} user, and set rubinius 3.70 as the default ruby version.
<pre>curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -</pre>
https://nodejs.org/en/download/package-manager/


{{bc|
== Setup Rubinius ==
chruby 3.70
}}
 
=== Method 3: Install Rubinius with RVM ===
 
{{Note|Unfortunately, until the [https://github.com/jruby/activerecord-jdbc-adapter/issues/708 ActiveRecord-JDBC-Adapter] required by JRuby is ported to Rails 5, we can't use it with Moebooru.}}
 
For RVM, just install any ol' ruby 2.x, we won't be using it after the compilation stage. Also install all the build dependencies.
 
==== Installation with RVM (RHEL/CentOS) ====
 
{{Note|Frustratingly, Rubinius refuses to build with Debian Jessie for some darn reason, so this is for RHEL/CentOS only.}}
 
{{bc|
sudo yum install ruby
sudo yum install -y patch autoconf patch automake libtool bison sqlite-devel bzip2 zlib-devel libyaml-devel readline-devel openssl-devel
}}
 
One special build dependency is llvm-3.6+. Unfortunately, CentOS 7's EPEL repo only has 3.4, so we will need to get [https://copr.fedorainfracloud.org/coprs/daveisfera/llvm_3.7/ the latest build of LLVM from Fedora COPR.]
 
{{hc|/etc/yum.repos.d/daveisfera-llvm_3.7-epel-7.repo|<nowiki>
[daveisfera-llvm_3.7]
name=Copr repo for llvm_3.7 owned by daveisfera
baseurl=https://copr-be.cloud.fedoraproject.org/results/daveisfera/llvm_3.7/epel-7-$basearch/
type=rpm-md
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://copr-be.cloud.fedoraproject.org/results/daveisfera/llvm_3.7/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1
</nowiki>}}
 
Then just install llvm as normal:
 
{{bc|
sudo yum install clang llvm llvm-devel llvm-static libedit libedit-devel
}}
 
{{Note|Make sure to run all these steps as the {{ic|moe}} user.}}
 
[https://rvm.io/ Follow the RVM installation instructions, as replicated below:]
 
{{bc|<nowiki>
sudo -i -u moe
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
curl -sSL https://get.rvm.io | bash -s stable
</nowiki>|lang=bash}}
 
Then, log out and log back in to reload bash and make the RVM command work:
 
{{bc|
exit
sudo -i -u moe
|lang=bash}}
 
Refresh the rvm repos. And since we have already installed dependencies in the sections above, disable the built-in dependency resolver:
 
{{bc|
rvm get head
rvm autolibs disable
}}
 
{{Note|If there are errors while compiling rubinius, check the compilation logs for any missing libraries, then install them using your distro's package manager.}}
 
Install rubinius. Compilation will take a while.
 
{{bc|
rvm install rbx
}}
 
After installation, choose Rubinius (rbx) as the default ruby version to use.
 
{{bc|
rvm list
}}
 
This will list out various ruby versions.
 
{{bc|
rvm alias create default rbx
}}
 
Finally, log out and log back in, and check your current ruby version to ensure that Rubinius is default.
 
{{bc|
ruby -v
}}
 
Now that your ruby type is set up, install bundler.
 
{{bc|
gem install bundler
}}
 
{{Note|If you change ruby editions later on after installing gems, make sure to run {{ic|gem pristine --all}} to delete the previous ones. After that, reinstall all gems from scratch.}}
 
<!--
==== Installation (Debian/Ubuntu) ====
 
{{bc|
sudo apt-get install ruby ruby-dev
sudo apt-get install build-essential openssl libreadline6 libreadline6-dev \
curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 \
libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison  \
subversion pkg-config libgdbm-dev ncurses-dev libffi-dev
}}
 
An LLVM version greater than 3.6+ is also required. You can get this in [https://backports.debian.org/Instructions/ Debian Jessie from backports], or in Ubuntu from normal repositories.
 
{{bc|
# if on debian jessie, set up jessie-backports: https://backports.debian.org/Instructions/
sudo apt-get install llvm-3.8 clang-3.8 libclang-3.8-dev llvm-3.8-runtime llvm-3.8-dev llvm-3.8-tools libedit2 libedit-dev
}}
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:53em">
Click '''Expand''' to display the {{ic|update-alternatives}} commands, which activate 3.8 as the default clang/llvm version.
<div class="mw-collapsible-content">
{{bc|
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.8 100
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.8 100
sudo update-alternatives --install \
        /usr/bin/llvm-config      llvm-config      /usr/bin/llvm-config-3.8  200 \
--slave /usr/bin/llvm-ar          llvm-ar          /usr/bin/llvm-ar-3.8 \
--slave /usr/bin/llvm-as          llvm-as          /usr/bin/llvm-as-3.8 \
--slave /usr/bin/llvm-bcanalyzer  llvm-bcanalyzer  /usr/bin/llvm-bcanalyzer-3.8 \
--slave /usr/bin/llvm-cov          llvm-cov        /usr/bin/llvm-cov-3.8 \
--slave /usr/bin/llvm-diff        llvm-diff        /usr/bin/llvm-diff-3.8 \
--slave /usr/bin/llvm-dis          llvm-dis        /usr/bin/llvm-dis-3.8 \
--slave /usr/bin/llvm-dwarfdump    llvm-dwarfdump  /usr/bin/llvm-dwarfdump-3.8 \
--slave /usr/bin/llvm-extract      llvm-extract    /usr/bin/llvm-extract-3.8 \
--slave /usr/bin/llvm-link        llvm-link        /usr/bin/llvm-link-3.8 \
--slave /usr/bin/llvm-mc          llvm-mc          /usr/bin/llvm-mc-3.8 \
--slave /usr/bin/llvm-mcmarkup    llvm-mcmarkup    /usr/bin/llvm-mcmarkup-3.8 \
--slave /usr/bin/llvm-nm          llvm-nm          /usr/bin/llvm-nm-3.8 \
--slave /usr/bin/llvm-objdump      llvm-objdump    /usr/bin/llvm-objdump-3.8 \
--slave /usr/bin/llvm-ranlib      llvm-ranlib      /usr/bin/llvm-ranlib-3.8 \
--slave /usr/bin/llvm-readobj      llvm-readobj    /usr/bin/llvm-readobj-3.8 \
--slave /usr/bin/llvm-rtdyld      llvm-rtdyld      /usr/bin/llvm-rtdyld-3.8 \
--slave /usr/bin/llvm-size        llvm-size        /usr/bin/llvm-size-3.8 \
--slave /usr/bin/llvm-stress      llvm-stress      /usr/bin/llvm-stress-3.8 \
--slave /usr/bin/llvm-symbolizer  llvm-symbolizer  /usr/bin/llvm-symbolizer-3.8 \
--slave /usr/bin/llvm-tblgen      llvm-tblgen      /usr/bin/llvm-tblgen-3.8
|lang=bash}}
[https://gist.github.com/jc00ke/b55cf7d16d584fbb2b92 Source: Github Gist - LLVM-Update-Alternatives]
</div></div>
 
==== Setup Rubinius with Zonio Repo (RHEL/CentOS) ====
 
{{Warning|Unfortunately Zonio does not keep their rubinius builds updated, so this method is depreciated.}}


Currently, just running normal ruby MRI should be sufficient for most tasks, but Rubinius can add significant performance boosts.
Currently, just running normal ruby MRI should be sufficient for most tasks, but Rubinius can add significant performance boosts.
Line 320: Line 46:
First, if you have normal ruby MRI currently installed, you should uninstall it.
First, if you have normal ruby MRI currently installed, you should uninstall it.


Add the Zonio repository to the new file <code>/etc/yum.repos.d/zonio.repo</code> to add Rubinius:
Add the Zonio repository to <code>/etc/yum.repos.d/zonio.repo</code> to add Rubinius:


<pre>[zonio]
<pre>[zonio]
Line 333: Line 59:
<pre>sudo yum install rubinius rubinius-devel
<pre>sudo yum install rubinius rubinius-devel
ruby -v</pre>
ruby -v</pre>
Finally, log in as the moebooru user and install bundler:
<pre>
sudo -u moebooru gem install bundler --user-install
</pre>
Finally, edit moebooru's <code>~/.profile</code> and add the correct PATH for your bundler (for example, <code>/home/moe/booru/.gem/rbx/2.2/bin</code>, may differ with your ruby version)
<pre>
PATH=$PATH:/home/moe/booru/.gem/rbx/2.2/bin
</pre>
{{Warning|Do not use the root user to install ruby gems! They will not be propogated to the right places.}}
Notice that you should not install normal ruby MRI afterwards. If you have a good reason to, follow these instructions:
Notice that you should not install normal ruby MRI afterwards. If you have a good reason to, follow these instructions:


https://zonio.net/rubinius_rpm_packages/
https://zonio.net/rubinius_rpm_packages/
-->
{{Note|If you encounter any issues with gems that say "Ignoring because extensions are not built", run {{ic|gem pristine --all}}}}
== Mandatory Access Control ==
=== AppArmor (Debian/Ubuntu) ===
anyone have apparmor profiles?


We may have to create our own through learning mode: http://www.howtogeek.com/118328/how-to-create-apparmor-profiles-to-lock-down-programs-on-ubuntu/
== SELinux Permissions ==


=== SELinux Permissions (RHEL/CentOS) ===
If using SELinux (which we highly recommend), you will need the following policies
 
If using SELinux (which we highly recommend), you will need the following policies, assuming that everything is installed to {{ic|/home/moe/booru/}}:


needed for proxy pass
needed for proxy pass
Line 375: Line 75:


<pre>setsebool -P httpd_enable_homedirs 1
<pre>setsebool -P httpd_enable_homedirs 1
sudo semanage fcontext -a -t httpd_sys_content_t '/home/moe/booru/shared(/.*)?'
sudo semanage fcontext -a -t httpd_sys_content_t '/var/www/booru.eikonos.org/shared(/.*)?'
sudo restorecon -R -v /home/moe/booru/shared
sudo restorecon -R -v /var/www/booru.eikonos.org/shared
sudo semanage fcontext -a -t httpd_sys_content_t '/var/www/moebooru/public(/.*)?'
sudo semanage fcontext -a -t httpd_sys_content_t '/var/www/booru.eikonos.org/public(/.*)?'
sudo restorecon -R -v /var/www/moebooru/public</pre>
sudo restorecon -R -v /var/www/booru.eikonos.org/public</pre>
https://www.pckr.co.uk/selinux-nginx-and-reverse-proxying-2/
https://www.pckr.co.uk/selinux-nginx-and-reverse-proxying-2/


Line 386: Line 86:


<pre>sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx &gt; nginx.te</pre>
<pre>sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx &gt; nginx.te</pre>
Open up the {{ic|nginx.te}} file and see that it is correct (such that no suspicious rule allows are inside). Then run:
View this and see that it is correct (such that no suspicious rule allows are inside). Then run:


<pre>sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx  
<pre>sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx  
sudo semodule -i nginx.pp</pre>
sudo semodule -i nginx.pp</pre>
http://axilleas.me/en/blog/2013/selinux-policy-for-nginx-and-gitlab-unix-socket-in-fedora-19/
http://axilleas.me/en/blog/2013/selinux-policy-for-nginx-and-gitlab-unix-socket-in-fedora-19/


Line 397: Line 96:
Now, conduct the setup and get dependencies.
Now, conduct the setup and get dependencies.


=== Dependencies (Debian/Ubuntu) ===
<pre>sudo yum install gcc gcc-c++ ImageMagick jhead libxslt-devel git libyaml-devel openssl-devel pcre-devel readline-devel</pre>
 
{{bc|sudo apt-get install build-essential libxml2-dev libxslt1-dev libpq-dev git jhead libgd2-noxpm-dev imagemagick}}
 
* '''build-essential''' - a meta-package that installs various tools and libraries necessary for program (mostly C++ and Perl) compilation and interpretation.
* '''libxml2''' - library that is required for XML support. It is necessary for moebooru’s external API to work.
* '''libxslt1''' - extension of libxml that allows better xml parsing and converting.
* '''libpq-dev''' - library that is necessary to build postgres connection module for ruby.
* '''git''' - version control system - we will use it for moebooru installation and updating.
* '''jhead''' - image processing tool that extracts EXIF information from image files.
* '''libgd2-noxpm''' - an image processing library.
* '''libgs2-noxpm-dev''' - an extension for compiling GD2 support for applications.
* '''imagemagick''' - collection of tools for image processing
 
=== Dependencies (RHEL/CentOS) ===
 
{{bc|sudo yum install gcc gcc-c++ ImageMagick jhead libxslt-devel git libyaml-devel openssl-devel pcre-devel readline-devel}}
 
=== Configure Moebooru ===
 
All commands in this section must be run as the {{ic|moe}} user:
 
{{bc|sudo -i -u moe}}
 
First, [http://stackoverflow.com/a/9235107 do a bundle config for pg] since we're using a specific postgresql version:
First, [http://stackoverflow.com/a/9235107 do a bundle config for pg] since we're using a specific postgresql version:


{{bc|<nowiki>
<pre>bundle config build.pg --with-pg-config=/usr/pgsql-9.5/bin/pg_config</pre>
bundle config build.pg --with-pg-config=/usr/pgsql-9.6/bin/pg_config   # Red Hat/CentOS Only
bundle config build.pg --with-pg-config=/usr/bin/pg_config              # Debian Only
</nowiki>}}
 
Install the ruby packages for the moebooru user only (under the directory <code>./vendor/bundle</code>):
Install the ruby packages for the moebooru user only (under the directory <code>./vendor/bundle</code>):


{{bc|bundle install --path vendor/bundle}}
<pre>bundle install --path vendor/bundle</pre>
 
Create <code>config/database.yml</code> and <code>config/local_config.rb</code> from the <code>.example</code> files, and configure them accordingly. Then set <code>chmod 700</code> so only the moebooru user can read the database password.
Obtain <code>config/database.yml</code> and <code>config/local_config.rb</code> from the <code>.example</code> files, and configure them accordingly. Then set <code>chmod 700</code> so only the moebooru user can read the database password.
 
{{bc|
cp config/database.yml.example config/database.yml
cp config/local_config.rb.example config/local_config.rb
chmod 700 config/database.yml
chmod 700 config/local_config.rb
}}
 
Generate your secret key, which is used for salts and such.
 
{{bc|bundle exec rake secret}}
 
Edit {{ic|config/database.yml}} using your favorite editor (such as nano), and replace {{ic|imouto}} with the password of your database user:
 
{{hc|config/database.yml|<nowiki>
login: &login
  adapter: postgresql
  username: moe
  password: imouto
  host: 127.0.0.1
 
development:
  database: moebooru_dev
  <<: *login
 
test:
  database: moebooru_test
  <<: *login
 
production:
  database: moebooru
  <<: *login
</nowiki>}}


{{bc|chmod 600 /home/moe/booru/config/database.yml}}
<pre>chmod 600 /var/www/booru.eikonos.org/config/database.yml</pre>
Initialize database <code>with bundle exec rake db:reset</code> (there will be some errors reported which is expected)


Initialize database with this command (there will be some errors reported with {{ic|ERROR:  must be owner of extension plpgsql}} which is normal, proceed)
Run <code>bundle exec rake db:migrate</code>
 
{{bc|bundle exec rake db:reset}}
 
{{Note|If you have trouble creating databases because they set to SQL_ASCII by default, [https://gist.github.com/amolkhanorkar/8706915 follow this guide to require UTF-8 to be set by default.] }}
 
{{Note|Whenever you update moebooru using {{ic|git pull}}, make sure to migrate the database table schema with the command below.}}
 
Then, migrate the database tables.
 
{{bc|bundle exec rake db:migrate}}


Now, you need to provide the correct permissions to the public folder:
Now, you need to provide the correct permissions to the public folder:


{{bc|chmod 755 /home/moe/booru/public}}
<pre>chmod 755 /var/www/booru.eikonos.org/public</pre>
Start the server (<code>bundle exec unicorn</code> or <code>bundle exec puma</code> if using JRuby/Rubinius)


Start the server: {{ic|bundle exec unicorn}} or {{ic|bundle exec puma}} if using Rubinius. Note that this will start the server in '''development mode''', which is somewhat slower. See the Production Mode section once you are ready to serve the site.
Finally, set moebooru to a non login user:


<pre>sudo chsh -s /bin/false moebooru</pre>
== Customize Header Image and Branding ==
== Customize Header Image and Branding ==
{{Note|You will have to restore these each time Moebooru is updated with {{ic|git pull}}, so keep them in a safe place.}}


By default, Moebooru comes with the Yande.re header image and branding, as the site developed the moebooru engine. You should definitely consider removing the original branding unless your site is private.
By default, Moebooru comes with the Yande.re header image and branding, as the site developed the moebooru engine. You should definitely consider removing the original branding unless your site is private.
Line 499: Line 128:
== Enable Memcached ==
== Enable Memcached ==


Memcached is a high performance caching solution and is needed to have Moebooru enumerate posts and create the /posts pagination bar. Follow these instructions to install Memcached. You need at least 2GB free RAM to provide.
Follow these instructions to install Memcached. You need at least 2GB free RAM to provide.


=== Installation (Debian/Ubuntu) ===
sudo yum install memcached
 
{{bc|sudo apt-get install memcached}}
 
https://kyup.com/tutorials/install-use-memcache/
 
=== Installation (RHEL/CentOS) ===
 
{{bc|sudo yum install memcached}}


http://www.liquidweb.com/kb/how-to-install-memcached-on-centos-7/
http://www.liquidweb.com/kb/how-to-install-memcached-on-centos-7/
=== Configuration ===


Edit <code>/etc/sysconfig/memcached</code> and set <code>CACHESIZE=2048</code> (2GB RAM) if possible.
Edit <code>/etc/sysconfig/memcached</code> and set <code>CACHESIZE=2048</code> (2GB RAM) if possible.


Then set memcached to start at every boot:
Then activate it by appending a bash variable to the puma command: <code>MB_MEMCACHE_SERVERS=&quot;&lt;url-to-memcached&gt;&quot;</code> . Here are some examples.
 
{{bc|sudo systemctl restart memcached}}
 
=== Config-based Activation ===
 
Add these options to the following file. When you start moebooru again, memcached will be active.
 
{{hc|config/local_config.rb|<nowiki>
# The server and port where the memcache client can be accessed. Only relevant if you enable caching.
CONFIG["memcache_servers"] = ["localhost:11211"]
# This enables various caching mechanisms. You must have memcache (and the memcache-client ruby gem) installed in order for caching to work.
CONFIG["enable_caching"] = true
</nowiki>}}
 
=== ENV-based Memcached Activation ===
 
{{Note|We're not really sure whether the below method of enabling caching works. Instead, we use config-based activation as stated above.}}
 
Activate it by appending a bash variable to the puma command: <code>MB_MEMCACHE_SERVERS=&quot;&lt;url-to-memcached&gt;&quot;</code> . Here are some examples.


TCP: <code>MB_MEMCACHED_SERVERS='127.0.0.1:11211' RAILS_ENV=production bundle exec puma -e production</code>
TCP: <code>MB_MEMCACHED_SERVERS='127.0.0.1:11211' RAILS_ENV=production bundle exec puma -e production</code>
Line 550: Line 150:
== Production Mode ==
== Production Mode ==


By default, Moebooru runs in development mode, which can be slow (since it's designed to allow debug and automatic recompilation). Here are the steps to set up Production mode.
By default, Moebooru runs in development mode, which can be slow. Here are the steps to set up Production mode.


=== Preparation ===
=== Preparation ===
Line 560: Line 160:
Then, you need to provide the correct permissions to the public folder:
Then, you need to provide the correct permissions to the public folder:


<pre>chmod 755 /var/www/moebooru/public</pre>
<pre>chmod 755 /var/www/booru.eikonos.org/public</pre>
 
=== Serve static files with Nginx ===
=== Serve static files with Nginx ===


Create an Nginx config under <code>/etc/nginx/conf.d/</code>. Make sure to change the <code>server_name</code>.
Create an Nginx config under <code>/etc/nginx/conf.d/</code>. Make sure to change the <code>server_name</code>, and the <code>root /var/www/booru.eikonos.org/public</code> to <code>root /YOUR/MOEBOORU/PATH/public</code>.


<blockquote>'''Note:''' If you are using a different port for puma, (by adding <code>-p 3000</code> to the serving command), also change the port accordingly below.
<blockquote>'''Note:''' If you are using a different port for puma, (by adding <code>-p 3000</code> to the serving command), also change the port accordingly below.
Line 574: Line 173:
     # directory of static assets, first generate with the command:
     # directory of static assets, first generate with the command:
     # RAILS_ENV=production bundle exec rake assets:precompile
     # RAILS_ENV=production bundle exec rake assets:precompile
     root /var/www/moebooru/public;
     root /var/www/booru.eikonos.org/public;


     try_files $uri/index.html $uri @app;
     try_files $uri/index.html $uri @app;
Line 595: Line 194:
=== Run the Server ===
=== Run the Server ===


Finally, to run the server at, for example, port 9292, run one of the following commands:
Finally, to run the server (default is port 9292), run the following command:
 
Ruby:
 
{{bc|bundle exec unicorn -p 9292 -E production}}
 
Rubinius:
 
{{bc|<nowiki>RAILS_ENV=production bundle exec puma -e production -p 9292</nowiki>}}


<pre>RAILS_ENV=production bundle exec puma -e production</pre>
* [http://stackoverflow.com/a/27318704 Source: StackOverflow - Rails 4: assets not loading in production]
* [http://stackoverflow.com/a/27318704 Source: StackOverflow - Rails 4: assets not loading in production]


Line 614: Line 206:


<pre>grep -c processor /proc/cpuinfo</pre>
<pre>grep -c processor /proc/cpuinfo</pre>
Create the following folders in your application directory:


Create a folder to store your UNIX sockets and PIDs in:
<pre>mkdir -p shared/pids shared/sockets shared/log</pre>
Place the following into <code>&lt;app_dir&gt;/shared/puma.rb</code>:


{{bc|
<pre># Change to match your CPU core count
sudo mkdir /var/run/moebooru
sudo chown -R moe:moe /var/run/moebooru
}}
 
Create the following folders in your application directory to store logs:
 
<pre>mkdir -p shared/log</pre>
Place the following into {{ic|<app_dir>/shared/puma.rb}}:
 
{{hc|shared/puma.rb|<nowiki># Change to match your CPU core count
workers 8
workers 8


Line 633: Line 217:
threads 1, 6
threads 1, 6


app_dir = File.expand_path("../..", __FILE__)
app_dir = File.expand_path(&quot;../..&quot;, __FILE__)
shared_dir = "#{app_dir}/shared"
shared_dir = &quot;#{app_dir}/shared&quot;
socket_dir = "/var/run/moebooru"


# Default to production
# Default to production
rails_env = ENV['RAILS_ENV'] || "production"
rails_env = ENV['RAILS_ENV'] || &quot;production&quot;
environment rails_env
environment rails_env


# Set up socket location
# Set up socket location
bind "unix://#{socket_dir}/puma.sock"
bind &quot;unix://#{shared_dir}/sockets/puma.sock&quot;


# Logging
# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true
stdout_redirect &quot;#{shared_dir}/log/puma.stdout.log&quot;, &quot;#{shared_dir}/log/puma.stderr.log&quot;, true


# Set master PID and state locations
# Set master PID and state locations
pidfile "#{socket_dir}/puma.pid"
pidfile &quot;#{shared_dir}/pids/puma.pid&quot;
state_path "#{socket_dir}/puma.state"
state_path &quot;#{shared_dir}/pids/puma.state&quot;
activate_control_app
activate_control_app


on_worker_boot do
on_worker_boot do
   require "active_record"
   require &quot;active_record&quot;
   ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
   ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
   ActiveRecord::Base.establish_connection(YAML.load_file("#{app_dir}/config/database.yml")[rails_env])
   ActiveRecord::Base.establish_connection(YAML.load_file(&quot;#{app_dir}/config/database.yml&quot;)[rails_env])
end
end</pre>
</nowiki>}}
 
Change the Nginx server config to the following:
Change the Nginx server config to the following:


{{hc|/etc/nginx/conf.d/booru.eikonos.org.conf|<nowiki>upstream app {
<pre>upstream app {
     # Path to Puma SOCK file, as defined previously
     # Path to Puma SOCK file, as defined previously
     server unix:/var/run/moebooru/puma.sock fail_timeout=0;
     server unix:/var/www/booru.eikonos.org/shared/sockets/puma.sock fail_timeout=0;
}
}


Line 672: Line 253:
     # directory of static assets, first generate with the command:
     # directory of static assets, first generate with the command:
     # RAILS_ENV=production bundle exec rake assets:precompile
     # RAILS_ENV=production bundle exec rake assets:precompile
     root /var/www/moebooru/public;
     root /var/www/booru.eikonos.org/public;


     try_files $uri/index.html $uri @app;
     try_files $uri/index.html $uri @app;
Line 681: Line 262:
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Proto $scheme;
       proxy_set_header X-Forwarded-Proto $scheme;
       # Fix the "It appears that your reverse proxy set up is broken" error.
       # Fix the &quot;It appears that your reverse proxy set up is broken&quot; error.
       proxy_pass http://app;
       proxy_pass http://app;
       proxy_read_timeout 90;
       proxy_read_timeout 90;
Line 690: Line 271:
     client_max_body_size 4G;
     client_max_body_size 4G;
     keepalive_timeout 10;
     keepalive_timeout 10;
}
}</pre>
</nowiki>}}
Finally, to run the server, use the following:
Finally, to run the server, use the following:


{{bc|bundle exec puma -C shared/puma.rb}}
<pre>bundle exec puma -C shared/puma.rb</pre>
 
== Separate subdomain for images and static content ==
 
It's often a better idea to have static content served from another subdomain entirely, where it will be cached for much longer than ever changing text. And sometimes the files could be located on an entirely different server.
 
Edit your {{ic|config/local_config.rb}} to activate the {{ic|assets.}} and {{ic|files.}} subdomains.
 
{{hc|config/local_config.rb|<nowiki>
# Servers for static files (assets and uploaded files)
CONFIG[:file_hosts] = { :files => "files.eikonos.org", :assets => "assets.eikonos.org" }
</nowiki>}}
 
Then, create these two nginx configs (customize the server_name to your subdomains, must start with {{ic|assets.}} and {{ic|files.}}):
 
{{hc|/etc/nginx/conf.d/moebooru_assets.conf|<nowiki>server {
    listen 127.0.0.1:8080; # tells Nginx to listen for traffic passed by Varnish if unencrypted http
    server_name assets.eikonos.org;
 
    # directory of static assets, first generate with the command:
    # RAILS_ENV=production bundle exec rake assets:precompile
    root /home/moe/booru/public;
 
    error_page 500 502 503 504 /500.html;
    keepalive_timeout 10;
 
    location ^~ / {
        expires 1M;
        access_log off;
        add_header Cache-Control "public";
        try_files $uri =404;
    }
 
    # users must access files.eikonos.org for full images, though thumbs are shared through assets
    location ^~ /data/images/ { deny all; }
}
</nowiki>}}
 
{{hc|/etc/nginx/conf.d/moebooru_files.conf|<nowiki>
server {
    listen 127.0.0.1:8080; # tells Nginx to listen for traffic passed by Varnish if unencrypted http
    server_name files.eikonos.org;
 
    # directory of full size files
    root /home/moe/booru/public/;
 
    error_page 500 502 503 504 /500.html;
    keepalive_timeout 10;
 
    location ^~ /data/ {
        expires 1M;
        access_log off;
        add_header Cache-Control "public";
        try_files $uri =404;
    }
    location ^~ /assets/ { deny all; } # users must access assets.eikonos.org for this
}
</nowiki>}}
 
== Systemd Service ==
== Systemd Service ==


Line 761: Line 283:
=== TCP ===
=== TCP ===


{{hc|/usr/systemd/system/moebooru.service|<nowiki>[Unit]
<pre>[Unit]
Description=Moebooru's Puma HTTP Server
Description=Moebooru's Puma HTTP Server
Requires=postgresql-9.6.service
Requires=redis.service postgresql-9.5.service
Wants=postgresql-9.6.service memcached.service
Wants=postgresql-9.5.service memcached.service
After=network.target postgresql-9.6.service
After=network.target postgresql-9.5.service


# Uncomment for socket activation (see below)
# Uncomment for socket activation (see below)
Line 775: Line 297:


# Preferably configure a non-privileged user
# Preferably configure a non-privileged user
User=moe
User=moebooru


# Specify the path to your puma application root
# Specify the path to your puma application root
WorkingDirectory=/home/moe/booru
WorkingDirectory=/var/www/booru.eikonos.org


# Helpful for debugging socket activation, etc.
# Helpful for debugging socket activation, etc.
Line 786: Line 308:
# Here we are using a binstub generated via:
# Here we are using a binstub generated via:
# `bundle binstubs puma --path ./sbin`
# `bundle binstubs puma --path ./sbin`
# in the WorkingDirectory (replace <WD> below)
# in the WorkingDirectory (replace &lt;WD&gt; below)
# You can alternatively use `bundle exec --keep-file-descriptors puma`
# You can alternatively use `bundle exec --keep-file-descriptors puma`
# ExecStart=<WD>/sbin/puma -b tcp://0.0.0.0:9292 -b ssl://0.0.0.0:9293?key=key.pem&amp;cert=cert.pem
# ExecStart=&lt;WD&gt;/sbin/puma -b tcp://0.0.0.0:9292 -b ssl://0.0.0.0:9293?key=key.pem&amp;cert=cert.pem


# Alternatively with a config file (in WorkingDirectory) and
# Alternatively with a config file (in WorkingDirectory) and
Line 797: Line 319:


[Install]
[Install]
WantedBy=multi-user.target
WantedBy=multi-user.target</pre>
</nowiki>}}
 
=== UNIX Socket ===
=== UNIX Socket ===


Two Systemd services are needed: one for the socket and one for the application.
Two Systemd services are needed: one for the socket and one for the application.


{{hc|/usr/systemd/system/moebooru.service|<nowiki>
<code>/usr/systemd/system/moebooru.service</code>
[Unit]
 
<pre>[Unit]
Description=Puma HTTP Server
Description=Puma HTTP Server
Requires=postgresql-9.6.service
Requires=redis.service postgresql-9.5.service
Wants=postgresql-9.6.service memcached.service
Wants=postgresql-9.5.service memcached.service
After=network.target postgresql-9.6.service
After=network.target postgresql-9.5.service


# Uncomment for socket activation (see below)
# Uncomment for socket activation (see below)
Line 830: Line 351:
# Here we are using a binstub generated via:
# Here we are using a binstub generated via:
# `bundle binstubs puma --path ./sbin`
# `bundle binstubs puma --path ./sbin`
# in the WorkingDirectory (replace <WD> below)
# in the WorkingDirectory (replace &lt;WD&gt; below)
# You can alternatively use `bundle exec --keep-file-descriptors puma`
# You can alternatively use `bundle exec --keep-file-descriptors puma`
# ExecStart=<WD>/sbin/puma -b tcp://0.0.0.0:9292 -b ssl://0.0.0.0:9293?key=key.pem&amp;cert=cert.pem
# ExecStart=&lt;WD&gt;/sbin/puma -b tcp://0.0.0.0:9292 -b ssl://0.0.0.0:9293?key=key.pem&amp;cert=cert.pem


# Alternatively with a config file (in WorkingDirectory) and
# Alternatively with a config file (in WorkingDirectory) and
# comparable `bind` directives
# comparable `bind` directives
# ExecStart=<WD>/sbin/puma -C config.rb
# ExecStart=&lt;WD&gt;/sbin/puma -C config.rb


Restart=always
Restart=always


[Install]
[Install]
WantedBy=multi-user.target
WantedBy=multi-user.target</pre>
</nowiki>}}
 
Grab some code from here?
Grab some code from here?


Line 853: Line 372:


https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-centos-7
https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-centos-7
== Full Backup ==
PostgreSQL Database (run as moebooru user!):
{{bc|
pg_dump -U moebooru -W moebooru > eikonos.org-20170128.pg.sql # will ask for password
}}
Images and thumbs and other generated items:
{{bc|
tar -cvzf eikonos.org-20170128_data.tar.gz /var/www/moebooru/public/data/
}}
Code only, no binaries or image data:
{{bc|<nowiki>
tar --exclude='/home/moe/booru/tmp' --exclude='/home/moe/booru/public' --exclude='/home/moe/booru/vendor' -cvjf eikonos.org-20170128_var_www_booru-eikonos-org.tar.gz /home/moe/booru
</nowiki>}}
== Restore/Move ==
=== PostgreSQL database ===
First, create the user with CreateDB permissions:
{{bc|
# su - postgres
$ psql
postgres # CREATE USER moebooru_user CREATEDB;
}}
then import the sql dump with that user.
== Troubleshooting ==
=== Pagination not showing in Post#index ===
{{Note|This issue only occurs when you are using the default cache instead of Memcached. We strongly recommend using Memcached for this reason.}}
https://github.com/moebooru/moebooru/issues/42
=== Atom RSS Canonical URL Problem ===
Bloo had met an issue with the [https://github.com/moebooru/moebooru/issues/71 Atom RSS URLs pointing to his IP instead of the domain name.] Here's the patch he used:
{{bc|
--- a/app/helpers/application_helper.rb
+++ b/app/helpers/application_helper.rb
@@ -126,7 +126,7 @@ module ApplicationHelper
    elsif post.is_a?(Array)
      posts = post
-      new_params = params.to_unsafe_h.merge :only_path => true
+      new_params = params.to_unsafe_h.merge :host=>'otakupaper.com'
      if posts.previous_page.present?
        html << tag("link", :href => url_for(new_params.merge :page => 1), :rel => "first", :title => "First Page")
diff --git a/app/helpers/post_helper.rb b/app/helpers/post_helper.rb
index 32cf04f..cf98cb5 100644
--- a/app/helpers/post_helper.rb
+++ b/app/helpers/post_helper.rb
@@ -18,7 +18,7 @@ module PostHelper
      "type"  => tag_options[:type] || "application/#{type}+xml",
      "title" => tag_options[:title] || type.to_s.upcase,
      "id"    => tag_options[:id],
-      "href"  => url_options.is_a?(Hash) ? url_for(url_options.merge(:only_path => false)) : url_options
+      "href"  => url_options.is_a?(Hash) ? url_for(url_options.merge(:host => CONFIG["server_host"])) : url_options
    )
  end
}}
== Sources ==
* Get configuration tips from here: http://wiki.douglasqsantos.com.br/doku.php/deploying_a_rails_app_on_debian_jessie_with_capistrano_nginx_and_puma_en
Please note that all contributions to Bibliotheca Anonoma are considered to be released under the Creative Commons Attribution-ShareAlike (see Bibliotheca Anonoma:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!
Cancel Editing help (opens in new window)

Templates used on this page: