FoolFuuka/Install/Ubuntu16

From Bibliotheca Anonoma

This guide was written by ATC of desuarchive, and is designed for Debian/Ubuntu Linux servers. This guide is designed for Ubuntu 16.04 LTS servers (xenial).

Work in progress.


Prepare things

apt-get update
apt-get upgrade

Add the keys for HipHop Virtual Machine

from docs.hhvm.com 

sudo apt-get install software-properties-common
sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0x5a16e7281be7a449
sudo add-apt-repository "deb http://dl.hhvm.com/ubuntu $(lsb_release -sc) main"

Add repo for MariaDB 10.1

from downloads.mariadb.org 

sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8
sudo add-apt-repository "deb [arch=amd64,i386,ppc64el] http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu xenial main"

Add repo for NGINX Stable

from downloads.mariadb.org 

wget https://nginx.org/keys/nginx_signing.key
sudo apt-key add nginx_signing.key
sudo add-apt-repository "deb http://nginx.org/packages/ubuntu/ xenial nginx"
sudo add-apt-repository "deb-src http://nginx.org/packages/ubuntu/ codename nginx"

Add repo for sphinxsearch

Note: Todo

Install all the needed packages.

Nginx, HHVM, PHP5 Cli, PHP5 Curl, MariaDB, Maven (includes java), Sphinxsearch 

sudo apt-get update
sudo apt-get install git nginx hhvm mariadb-server mariadb-client imagemagick maven openjdk-8-jdk-headless

Install composer for downloading FoolFuuka dependencies.

curl -sS https://getcomposer.org/installer | php

Move composer to bin

sudo mv composer.phar /usr/local/bin/composer

Enable the hhvm service.

systemctl enable hhvm

Install hhvm with nginx

TODO

Edit HHVM config to use a unix socket.

sudo mkdir /var/run/hhvm/
sudo chown www-data:www-data /var/run/hhvm/
sudo nano /etc/hhvm/server.ini

Then remove the line starting with hhvm.server.port, and in its place add the following one: 

hhvm.server.file_socket=/var/run/hhvm/hhvm.sock

Add required php.ini vars.

sudo nano /etc/hhvm/php.ini
date.timezone = UTC
upload_max_filesize = 32M
post_max_size = 32M

Restart HHVM

sudo service hhvm restart

Download FoolFuuka and install dependencies.

cd /var/www/
git clone https://github.com/pleebe/FoolFuuka.git foolfuuka
cd foolfuuka
composer dump-autoload --optimize
composer install

Choose 'Y' to 3rd party assets.

Install Foolfuuka Plugins

You may need to find the repo on their github if their site doesn't work: https://github.com/FoolCode 

cd /var/www/foolfuuka/app/foolz/foolframe/plugins
mkdir foolz
cd foolz
git clone https://github.com/pleebe/foolframe-plugin-articles.git


cd /var/www/foolfuuka/app/foolz/foolfuuka/plugins
mkdir foolz
cd foolz
git clone https://github.com/pleebe/foolfuuka-plugin-board-statistics.git
git clone https://github.com/FoolCode/foolfuuka-plugin-dice-roll.git
git clone https://github.com/pleebe/foolfuuka-plugin-intel-share.git
git clone https://github.com/pleebe/foolfuuka-plugin-popup-report.git
git clone https://github.com/FoolCode/foolfuuka-plugin-quests.git
git clone https://github.com/pleebe/foolfuuka-plugin-spam-guard.git
git clone https://github.com/pleebe/foolfuuka-plugin-thread-chunk.git
git clone https://github.com/pleebe/foolfuuka-plugin-adverts.git
git clone https://github.com/pleebe/foolfuuka-plugin-external-links.git
git clone https://github.com/pleebe/foolfuuka-plugin-fortune.git
git clone https://github.com/pleebe/foolfuuka-plugin-cloudflare-cache-purge.git
git clone https://github.com/pleebe/foolfuuka-plugin-table.git

Set Permissions and Group for FoolFuuka Directory

sudo chown -Rf www-data:www-data /var/www/foolfuuka

Configure Nginx to host foolfuuka

sudo nano /etc/nginx/sites-enabled/default
        root /home/ubuntu/foolfuuka/public;
        index index.php index.html index.htm;

        client_max_body_size 32M; # set to the same as max PHP upload size in HHVM
        location / {
                try_files $uri $uri/ /index.php$is_args$args;
        }

        location ~ \.php$ {
                 fastcgi_pass unix:/var/run/php5-fpm.sock;
                 fastcgi_index index.php;
                 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                 include fastcgi_params;
        }

Restart Nginx

sudo service nginx restart


edit MariaDB to use utf8mb4 for correct unicode processing

sudo nano /etc/mysql/my.cnf
[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_general_ci


Create the database and install Foolfuuka user accounts.

sudo mysql_secure_installation
mysql -uroot -p
  CREATE USER 'asagi'@'localhost' IDENTIFIED BY 'pass';  
  CREATE USER 'foolfuuka'@'localhost' IDENTIFIED BY 'pass';
  CREATE DATABASE asagi DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_general_ci;
  CREATE DATABASE foolfuuka DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_general_ci;
  GRANT ALL PRIVILEGES ON `asagi`. * TO 'asagi'@'localhost';
  GRANT ALL PRIVILEGES ON `asagi`. * TO 'foolfuuka'@'localhost';
  GRANT ALL PRIVILEGES ON `foolfuuka`. * TO 'foolfuuka'@'localhost';
  FLUSH PRIVILEGES;
  exit

Restart MariaDB

sudo service mysql restart

Disable transparent_hugepage 

su -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled'
sudo nano /etc/default/grub

append transparent_hugepage=never to GRUB_CMDLINE_LINUX_DEFAULT, then save the file 

sudo update-grub

Enable toku (Find and Uncomment the load module) 

sudo nano /etc/mysql/conf.d/tokudb.cnf

Raise file limit 

sudo nano /etc/security/limits.conf
* soft nofile 63545
* hard nofile 63545
* soft nproc 63545
* hard nproc 63545

Increase mysql file limit: 

sudo nano /etc/mysql/my.cnf
# Put this after [mysqld]
open-files-limit=40000

sudo service mysql stop
sudo reboot
mysql -u root -p
set global default_storage_engine=TokuDB;
SET GLOBAL log_bin_trust_function_creators = 1;

Now, Install FoolFuuka at the website using foolfuuka credentials

Install and compile Asagi from source.

cd ~
git clone https://github.com/FoolCode/asagi.git
cd asagi
mvn package assembly:single
mv target/asagi-0.4.0-SNAPSHOT-full.jar ./asagi.jar

Configure Asagi

cp asagi.json.example asagi.json
nano asagi.json

Insert the following JSON into that file: 

{"settings": {
  "dumperEngine": "DumperJSON",
  "sourceEngine": "YotsubaJSON",

  "boardSettings": {
    "default": {
      "engine": "Mysql",
      "database": "asagi",
      "host": "localhost",
      "username": "asagi",
      "password": "afj8a428dauwj",
      "charset": "utf8mb4",
      "path": "/home/ubuntu/foolfuuka/public/foolfuuka/boards/",
      "useOldDirectoryStructure": false,
      "webserverGroup": "www-data",
      "thumbThreads": 3,
      "mediaThreads": 0,
      "newThreadsThreads": 3,
      "deletedThreadsThresholdPage": 8,
      "refreshDelay": 30,
      "throttleAPI": true,
      "throttleURL": "api.4cdn.org",
      "throttleMillisec": 11100,

      "threadRefreshRate": 10,
      "pageSettings": [
        {"delay": 30, "pages": [0, 1]},
        {"delay": 500, "pages": [2, 3, 4, 5, 6, 7, 8]},
        {"delay": 30, "pages": [9, 10]}
      ]
    },

    "3": {},
    "a": {},
    "aco": {},
    "adv": {},
    "an": {},
    "asp": {},
    "b": {},
    "biz": {},
    "c": {},
    "cgl": {},
    "ck": {},
    "cm": {},
    "co": {},
    "d": {},
    "diy": {},
    "e": {},
    "f": {},
    "fa": {},
    "fit": {},
    "g": {},
    "gd": {},
    "gif": {},
    "h": {},
    "hc": {},
    "hm": {},
    "his": {},
    "hr": {},
    "i": {},
    "ic": {},
    "int": {},
    "jp": {},
    "k": {},
    "lgbt": {},
    "lit": {},
    "m": {},
    "mlp": {},
    "mu": {},
    "n": {},
    "news": {},
    "o": {},
    "out": {},
    "p": {},
    "po": {},
    "pol": {},
    "qa": {},
    "qst": {},
    "r": {},
    "r9k": {},
    "s": {},
    "s4s": {},
    "sci": {},
    "soc": {},
    "sp": {},
    "t": {},
    "tg": {},
    "toy": {},
    "trash": {},
    "trv": {},
    "tv": {},
    "u": {},
    "v": {},
    "vg": {},
    "vp": {},
    "vr": {},
    "w": {},
    "wg": {},
    "wsg": {},
    "wsr": {},
    "x": {},
    "y": {}
  }
}}

Start asagi (There is a better way to do this).

Maybe we should use systemd for it...

UseParallelGC adds Parallelization to the garbage collector, which really helps for Java. 

screen java -XX:+UseParallelGC -XX:+UseParallelOldGC -verbose:gc -jar asagi.jar

ATC of Desuarchive prefers these parameters, which have two customizations.

  1. For the first parameter, Java by default will use 1/4th your system memory, so you can set your own hard memory roof to reduce usage or increase it. He used 6144m.
    • If you aren't archiving more than 8 boards, this probably won't be an issue that you need to deal with.
  2. The permsize is about the permanent generation objects: depends on your heap size.
    • This setting is probably related to how many boards are on Desuarchive, but Sunako doesn't had that sort of issue.
screen java -Xmx6144m -XX:+UseParallelGC -XX:+UseParallelOldGC -verbose:gc -XX:MaxPermSize=512m -jar asagi.jar

Now, Configure asagi as the database and remove the table prefix from FoolFuuka.

Install FoolFuuka

The installer is quite self explanatory, just make sure to remove the Boards prefix.

Also make sure to install imagemagick and set the path in the config to it.

Add boards, Configure Sphinx

To configure Sphinx Search, take the following links:

http://IP/admin -> Search -> Generate Config. http://IP/admin -> Boards -> Edit -> Check Use SphinxSearch as search engine. On the Search panel, change the working directory (on Ubuntu) to the following: 

/var/lib/sphinxsearch

Supplement Sphinxsearch

cd /var/run
sudo mkdir sphinxsearch
sudo chmod -Rf 777 sphinxsearch

Start Sphinxsearch

sudo nano /etc/default/sphinxsearch

Change the following line from 

START=no

to 

START=yes

Save and run the following command 

sudo /etc/init.d/sphinxsearch start


Right-click and copy this config out of Foolfuuka and add your MySQL database information for asagi. Change the following line from 

pid_file = /var/lib/sphinxsearch/searchd.pid

to 

pid_file = /var/run/sphinxsearch/searchd.pid

Then, paste this to /etc/sphinxsearch/sphinx.conf

If anything errors, kill any running instance of asagi and restart it

Build the indexes

sudo indexer *_ancient --rotate
sudo indexer *_main --rotate

Configure Crontab

sudo nano /etc/crontab

@hourly sudo -u sphinxsearch /usr/bin/indexer --rotate *_delta > /dev/null
@daily sudo -u sphinxsearch /usr/bin/indexer --rotate *_main > /dev/null
@monthly sudo -u sphinxsearch /usr/bin/indexer --rotate *_ancient > /dev/null


Adding Archives

Inject your sql file into the asagi database (assuming that your sql file is called "dump.sql"): 

mysql -u root -p asagi < dump.sql

Then, in the admin panel, simply add the board that corresponds to the board you injected.

Extra configuration

Depending on which distro you use, some configs might need to be changed.

Fixing thumbnail creation

Foolfuuka might not be able to generate thumbnails for images to be posted. If so, check: 

$ whereis convert
convert: /usr/bin/convert /usr/share/man/man1/convert.1.gz

and make sure it is the same value in Admin panel -> general -> Imagemagick Convert

Retrieved from "https://wiki.bibanon.org/index.php?title=FoolFuuka/Install/Ubuntu16&oldid=4114"