FoolFuuka/Install/Debian

This guide was written by ATC of desuarchive, and is designed for Debian/Ubuntu Linux servers.

Prepare things

apt-get update
apt-get upgrade

Add the keys for HipHop Virtual Machine

sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0x5a16e7281be7a449
sudo add-apt-repository "deb http://dl.hhvm.com/ubuntu $(lsb_release -sc) main"

Add repo for MariaDB 10.1

sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db
sudo add-apt-repository 'deb [arch=amd64,i386] http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu trusty main'

Add repo for sphinxsearch

Note: Weird shit happens here

sudo add-apt-repository ppa:builds/sphinxsearch-stable

Then, Comment out repo in /etc/apt/sources.list.d/* and add to end:

  deb http://ppa.launchpad.net/builds/sphinxsearch-stable/ubuntu precise main 
  deb-src http://ppa.launchpad.net/builds/sphinxsearch-stable/ubuntu precise main 

sudo apt-get update

Install all the needed packages.

Nginx, HHVM, PHP5 Cli, PHP5 Curl, MariaDB, OpenJDK w/ compilier, Maven

sudo apt-get install git nginx hhvm php5-cli php5-curl php5-mysqlnd mariadb-server libmariadbclient-dev mariadb-client imagemagick openjdk-7-jre openjdk-7-jdk openjdk-7-jre-lib maven

Install composer for downloading FoolFuuka dependencies.

curl -sS https://getcomposer.org/installer | php

Move composer to bin

sudo mv composer.phar /usr/local/bin/composer

Install hhvm as a service so it launches at boot.

sudo update-rc.d hhvm defaults

Install hhvm with nginx

sudo /usr/share/hhvm/install_fastcgi.sh

Edit HHVM config to use a unix socket.

sudo nano /etc/hhvm/server.ini

Then remove the line starting with hhvm.server.port, and in its place add the following one:

  hhvm.server.file_socket=/var/run/hhvm/hhvm.sock

Set required php.ini vars.

sudo nano /etc/hhvm/php.ini

  date.timezone = UTC
  upload_max_filesize = 32M
  post_max_size = 32M

Restart HHVM

sudo service hhvm restart

Configure NGINX to use hhvm's new unix socket.

sudo nano /etc/nginx/hhvm.conf

In this file make sure that the fastcgi_pass directive points to the HHVM socket and looks like this:

  fastcgi_pass unix:/var/run/hhvm/hhvm.sock;

Download FoolFuuka in home folder and install dependencies.

cd ~
composer create-project foolz/foolfuuka foolfuuka --prefer-source
cd foolfuuka
composer dump-autoload --optimize
composer install 

Install Foolfuuka Plugins

cd ~/foolfuuka/app/foolz/foolframe/plugins
mkdir foolz
cd foolz
git clone -c http.sslVerify=false https://dev.foolz.us/foolframe-plugins/foolframe-plugin-articles.git

cd ~/foolfuuka/app/foolz/foolfuuka/plugins
mkdir foolz
cd foolz
git clone -c http.sslVerify=false https://dev.foolz.us/foolfuuka-plugins/foolfuuka-plugin-board-statistics.git
git clone -c http.sslVerify=false https://dev.foolz.us/foolfuuka-plugins/foolfuuka-plugin-dice-roll.git
git clone -c http.sslVerify=false https://dev.foolz.us/foolfuuka-plugins/foolfuuka-plugin-quests.git

Set Permissions and Group for FoolFuuka Directory

chmod -Rf 775 ~/foolfuuka
chown -Rf www-data:www-data ~/foolfuuka

Configure Nginx to host foolfuuka

sudo nano /etc/nginx/sites-enabled/default

        root /home/ubuntu/foolfuuka/public;
        index index.php index.html index.htm;

        location / {
                try_files $uri $uri/ /index.php$is_args$args;
        }

Restart Nginx

sudo service nginx restart

Create the database and install Foolfuuka user accounts.

Warning: Don't use this command: sudo mysql_install_db

sudo mysql_secure_installation

mysql -uroot -p
  CREATE USER 'asagi'@'localhost' IDENTIFIED BY 'pass';  
  CREATE USER 'foolfuuka'@'localhost' IDENTIFIED BY 'pass';
  CREATE DATABASE asagi DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_general_ci;
  CREATE DATABASE foolfuuka DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_general_ci;
  GRANT ALL PRIVILEGES ON `asagi`. * TO 'asagi'@'localhost';
  GRANT ALL PRIVILEGES ON `asagi`. * TO 'foolfuuka'@'localhost';
  GRANT ALL PRIVILEGES ON `foolfuuka`. * TO 'foolfuuka'@'localhost';
  FLUSH PRIVILEGES;
  exit

edit MariaDB to use utf8mb4 for correct unicode processing

sudo nano /etc/mysql/my.cnf

  [client]
  default-character-set = utf8mb4

  [mysql]
  default-character-set = utf8mb4

  [mysqld]
  character-set-client-handshake = FALSE
  character-set-server = utf8mb4
  collation-server = utf8mb4_general_ci

Restart MariaDB

sudo service mysql restart

Disable transparent_hugepage

su -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'

sudo nano /etc/default/grub

append transparent_hugepage=never to GRUB_CMDLINE_LINUX_DEFAULT, then save the file

sudo update-grub

Enable toku (Find and Uncomment the load module)

sudo nano /etc/mysql/conf.d/tokudb.cnf 

Raise file limit

sudo nano /etc/security/limits.conf

  * soft nofile 63545
  * hard nofile 63545
  * soft nproc 63545
  * hard nproc 63545

Increase mysql file limit:

sudo nano /etc/mysql/my.cnf

  # Put this after [mysqld]
  open-files-limit=40000

sudo service mysql stop
sudo reboot

mysql -u root -p
  set global default_storage_engine=TokuDB;
  SET GLOBAL log_bin_trust_function_creators = 1;

Now, Install FoolFuuka at the website using foolfuuka credentials

Install and compile Asagi from source.

cd ~
git clone https://github.com/FoolCode/asagi.git
cd asagi
mvn package assembly:single
mv target/asagi-0.4.0-SNAPSHOT-full.jar ./asagi.jar

Configure Asagi

cp asagi.json.example asagi.json
nano asagi.json

Insert the following JSON into that file:

###
{"settings": {
  "dumperEngine": "DumperJSON",
  "sourceEngine": "YotsubaJSON",

  "boardSettings": {
    "default": {
      "engine": "Mysql",
      "database": "asagi",
      "host": "localhost",
      "username": "asagi",
      "password": "afj8a428dauwj",
      "charset": "utf8mb4",
      "path": "/home/ubuntu/foolfuuka/public/foolfuuka/boards/",
      "useOldDirectoryStructure": false,
      "webserverGroup": "www-data",
      "thumbThreads": 3,
      "mediaThreads": 0,
      "newThreadsThreads": 3,
      "deletedThreadsThresholdPage": 8,
      "refreshDelay": 30,
      "throttleAPI": true,
      "throttleURL": "api.4cdn.org",
      "throttleMillisec": 11100,

      "threadRefreshRate": 10,
      "pageSettings": [
        {"delay": 30, "pages": [0, 1]},
        {"delay": 500, "pages": [2, 3, 4, 5, 6, 7, 8]},
        {"delay": 30, "pages": [9, 10]}
      ]
    },

    "3": {},
    "a": {},
    "aco": {},
    "adv": {},
    "an": {},
    "asp": {},
    "b": {},
    "biz": {},
    "c": {},
    "cgl": {},
    "ck": {},
    "cm": {},
    "co": {},
    "d": {},
    "diy": {},
    "e": {},
    "f": {},
    "fa": {},
    "fit": {},
    "g": {},
    "gd": {},
    "gif": {},
    "h": {},
    "hc": {},
    "hm": {},
    "his": {},
    "hr": {},
    "i": {},
    "ic": {},
    "int": {},
    "jp": {},
    "k": {},
    "lgbt": {},
    "lit": {},
    "m": {},
    "mlp": {},
    "mu": {},
    "n": {},
    "news": {},
    "o": {},
    "out": {},
    "p": {},
    "po": {},
    "pol": {},
    "qa": {},
    "qst": {},
    "r": {},
    "r9k": {},
    "s": {},
    "s4s": {},
    "sci": {},
    "soc": {},
    "sp": {},
    "t": {},
    "tg": {},
    "toy": {},
    "trash": {},
    "trv": {},
    "tv": {},
    "u": {},
    "v": {},
    "vg": {},
    "vp": {},
    "vr": {},
    "w": {},
    "wg": {},
    "wsg": {},
    "wsr": {},
    "x": {},
    "y": {}
  }
}}
###

Start asagi (There is a better way to do this).

Maybe we should use systemd for it...

UseParallelGC adds Parallelization to the garbage collector, which really helps for Java.

screen java -XX:+UseParallelGC -XX:+UseParallelOldGC -verbose:gc -jar asagi.jar

ATC of Desuarchive prefers these parameters, which have two customizations.

1. For the first parameter, Java by default will use 1/4th your system memory, so you can set your own hard memory roof to reduce usage or increase it. He used 6144m.
   • If you aren't archiving more than 8 boards, this probably won't be an issue that you need to deal with.
2. The permsize is about the permanent generation objects: depends on your heap size.
   • This setting is probably related to how many boards are on Desuarchive, but Sunako doesn't had that sort of issue.

screen java -Xmx6144m -XX:+UseParallelGC -XX:+UseParallelOldGC -verbose:gc -XX:MaxPermSize=512m -jar asagi.jar

Now, Configure asagi as the database and remove the table prefix from FoolFuuka.

Add boards, Configure Sphinx

http://IP/admin > Search > Generate Config.

Copy config out of FF and add your MySQL database information. Paste this to /etc/sphinxsearch/sphinx.conf

Build the indexes

sudo indexer 3_ancient a_ancient aco_ancient adv_ancient an_ancient asp_ancient b_ancient biz_ancient c_ancient cgl_ancient ck_ancient cm_ancient co_ancient d_ancient diy_ancient e_ancient f_ancient fa_ancient fit_ancient g_ancient gd_ancient gif_ancient h_ancient hc_ancient hm_ancient his_ancient hr_ancient i_ancient ic_ancient int_ancient jp_ancient k_ancient lgbt_ancient lit_ancient m_ancient mlp_ancient mu_ancient n_ancient news_ancient o_ancient out_ancient p_ancient po_ancient pol_ancient qa_ancient qst_ancient r_ancient r9k_ancient s_ancient s4s_ancient sci_ancient soc_ancient sp_ancient t_ancient tg_ancient toy_ancient trash_ancient trv_ancient tv_ancient u_ancient v_ancient vg_ancient vp_ancient vr_ancient w_ancient wg_ancient wsg_ancient wsr_ancient x_ancient y_ancient --rotate

sudo indexer 3_main a_main aco_main adv_main an_main asp_main b_main biz_main c_main cgl_main ck_main cm_main co_main d_main diy_main e_main f_main fa_main fit_main g_main gd_main gif_main h_main hc_main hm_main his_main hr_main i_main ic_main int_main jp_main k_main lgbt_main lit_main m_main mlp_main mu_main n_main news_main o_main out_main p_main po_main pol_main qa_main qst_main r_main r9k_main s_main s4s_main sci_main soc_main sp_main t_main tg_main toy_main trash_main trv_main tv_main u_main v_main vg_main vp_main vr_main w_main wg_main wsg_main wsr_main x_main y_main --rotate

Configure Crontab

  */3  * * * * indexer 3_delta a_delta aco_delta adv_delta an_delta asp_delta b_delta biz_delta c_delta cgl_delta ck_delta cm_delta co_delta d_delta diy_delta e_delta f_delta fa_delta fit_delta g_delta gd_delta gif_delta h_delta hc_delta hm_delta his_delta hr_delta i_delta ic_delta int_delta jp_delta k_delta lgbt_delta lit_delta m_delta mlp_delta mu_delta n_delta news_delta o_delta out_delta p_delta po_delta pol_delta qa_delta qst_delta r_delta r9k_delta s_delta s4s_delta sci_delta soc_delta sp_delta t_delta tg_delta toy_delta trash_delta trv_delta tv_delta u_delta v_delta vg_delta vp_delta vr_delta w_delta wg_delta wsg_delta wsr_delta x_delta y_delta --rotate --quiet
  0 */6 * * * indexer 3_main a_main aco_main adv_main an_main asp_main b_main biz_main c_main cgl_main ck_main cm_main co_main d_main diy_main e_main f_main fa_main fit_main g_main gd_main gif_main h_main hc_main hm_main his_main hr_main i_main ic_main int_main jp_main k_main lgbt_main lit_main m_main mlp_main mu_main n_main news_main o_main out_main p_main po_main pol_main qa_main qst_main r_main r9k_main s_main s4s_main sci_main soc_main sp_main t_main tg_main toy_main trash_main trv_main tv_main u_main v_main vg_main vp_main vr_main w_main wg_main wsg_main wsr_main x_main y_main --rotate --quiet

Tip: If you're not serving the archive to the public, you can just use hourly or daily crontabs instead of the big ugly delta above.

@hourly sudo -u sphinxsearch /usr/bin/indexer --rotate *_delta > /dev/null
@daily sudo -u sphinxsearch /usr/bin/indexer --rotate *_main > /dev/null
@monthly sudo -u sphinxsearch /usr/bin/indexer --rotate *_ancient > /dev/null