Editing Hayden Guide

Guide on how to setup and run hayden on a linux machine.


== Warnings ==
* Blame Ctrl-S for this document. (WIP)
* Hayden is not really reliably stable on linux right now and may decide to enter a state where it will reliably crash on startup.
* Hayden cares about the JSON config file syntax. If you missplace a comma it will probably reject the config file and crash.


== Prerequisites ==
How to set up a system so hayden will download, build, and run.
# Prep system for updates
#* Debian-based systems (ubuntu, debian, etc)
#: <syntaxhighlight lang="bash">
$ sudo apt update && sudo apt upgrade -y
$ sudo apt install -y git curl wget
</syntaxhighlight>
#* RHEL-based systems (redhat, fedora, centos)
#: <syntaxhighlight lang="bash">
$ sudo yum update -y
</syntaxhighlight>

=== Install dotnet ===
# Install Microsoft dotnet runtime
https://docs.microsoft.com/en-us/dotnet/core/install/linux-centos
<br>https://docs.microsoft.com/en-us/dotnet/core/install/linux-ubuntu
#* RHEL-based systems (redhat, fedora, centos)
#: <syntaxhighlight lang="bash">$ sudo dnf install dotnet-sdk-3.1</syntaxhighlight>
#* Debian-based systems (ubuntu, debian, etc)
#: <syntaxhighlight lang="bash">$ sudo apt-get install -y dotnet-sdk-3.1</syntaxhighlight>

=== Database install ===
Hayden works with MariaDB and I dont know if it works with others.
<br>You could use any MariaDB installation if you wanted to, however that is left to advanced users who know how and is outside the scope of this guide.

==== Docker+Mariadb - Install Docker ====
https://docs.docker.com/engine/install/#server
<br>https://docs.docker.com/engine/install/centos/
# Setup repository:
#: <syntaxhighlight lang="bash">
$ sudo yum install -y yum-utils
$ sudo yum-config-manager  --add-repo  https://download.docker.com/linux/centos/docker-ce.repo
</syntaxhighlight>
# Install docker engine:
#: <syntaxhighlight lang="bash">$ sudo yum install docker-ce docker-ce-cli containerd.io</syntaxhighlight>
# Start docker service:
#: <syntaxhighlight lang="bash">$ sudo systemctl start docker</syntaxhighlight>
# Verify docker works
#: <syntaxhighlight lang="bash">$ sudo docker run hello-world</syntaxhighlight>

==== Docker+Mariadb - Install docker-compose ====
https://docs.docker.com/compose/
# Install docker-compose:
#: <syntaxhighlight lang="bash">
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.26.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose
</syntaxhighlight>
# Symlink into $PATH:
#: <syntaxhighlight lang="bash">$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose</syntaxhighlight>
#* This is so that you do not need to type the absolute path to run the commands.
# Test installation:
#: <syntaxhighlight lang="bash">$ docker-compose --version</syntaxhighlight>

==== Docker+Mariadb - Setup Container ====
#: Prepare container definition
* Example docker-compose definition file:
*: <syntaxhighlight lang="bash">
# mariadb.yml
# https://hub.docker.com/_/mariadb/
version: '3.1'
services:
  db: # SQL DB server
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: "root_super_secret_code"
      MYSQL_DATABASE: hayden_db
      MYSQL_USER: hayden_un
      MYSQL_PASSWORD: "hayden_super_secret_cod"
    ports:
      - "10306:3306" # HOST:CONTAINER
    volumes:
      - "/home/USER/hayden-2020/mariadb_docker:/var/lib/mysql" # HOST:CONTAINER
      - "/home/USER/hayden-2020/mariadb_docker_dump:/dump" # HOST:CONTAINER
  adminer: # WebGUI for SQL DB server
    image: adminer
    restart: always
    ports:
      - 10088:8080
</syntaxhighlight>
#* Ports in the 10XXX range are used on the host side here in an attempt to avoid conflicting with other programs wanting specific ports.
# Bring up defined containers
#* Bring up defined containers in the foreground of your terminal:
#*: Not reccomended outside of testing and development, as it is easy to close by mistake. 
#*: <syntaxhighlight lang="bash">sudo docker-compose -f "/home/USER/hayden-2020/mariadb.yml" up</syntaxhighlight>
#* Bring up defined containers in the background ("-d" CLI flag):
#*: Reccomended, as this is harder to close by mistake.
#*: <syntaxhighlight lang="bash">sudo docker-compose -f "/home/USER/hayden-2020/mariadb.yml" -d up</syntaxhighlight>

==== Docker+Mariadb - Usage ====
* TODO 
* To connect to the DB through adminer:
*: TODO
* To make a DB dump:
*# Find the name or ID of the DB container
*#: 
*# Run the dump command targeting that container
*#: <syntaxhighlight lang="bash">$ sudo docker exec hayden-2020_db_1 sh -c 'exec mysqldump --all-databases -u"root" -p"YOUR_PASSWORD_HERE"' > "/home/USER/hayden-2020/mariadb-container-all-databases.ts`date -u +%s`.sql" </syntaxhighlight>
*#* Note that the dump command in the container outputs the DB dump SQL as text on its STDOUT; which is sent back from the container to the host; and that is redirected into a file on the host.
*#* Explaination:
*#** <code>$ sudo docker exec hayden-2020_db_1 sh -c</code> Runs a command inside a container.
*#** <code>'exec mysqldump --all-databases -u"root" -p"YOUR_PASSWORD_HERE"'</code> Dumps a copy of the DB to STDOUT
*#** <code> > "/home/USER/hayden-2020/mariadb-container-all-databases.ts`date -u +%s`.sql"</code> Redirects the output into a file with a timestamp in the name.
*#** <code>date -u +%s</code> Produces a timestamp in seconds since the epoch. e.g. <code>1597472305</code>.
*#* This is done using the DB root account as it always has full access to the entire database.

== Download and Build Hayden ==
# Prepare location:
#: <syntaxhighlight lang="bash">
$ mkdir -vp /home/USER/hayden_2020/
$ cd /home/USER/hayden_2020/
</syntaxhighlight>
# Clone (download) Hayden source code from Github:
#: <syntaxhighlight lang="bash">$ git clone https://github.com/bbepis/Hayden.git</syntaxhighlight>
# Build the Hayden solution file:
#: <syntaxhighlight lang="bash">$ dotnet build "/home/USER/hayden_2020/Hayden/Hayden.sln"</syntaxhighlight>
# Run with config file:
#: <syntaxhighlight lang="bash">$ "/home/USER/hayden_2020/Hayden/Hayden/bin/Debug/netcoreapp3.1/Hayden" "/home/USER/hayden_2020/hayden-config.json"</syntaxhighlight>

== Configuration ==
* Configuration for Hayden is done through a JSON file that is specified as a command-line argument when starting Hayden.
<syntaxhighlight lang="bash">$ "/home/USER/hayden_2020/Hayden/Hayden/bin/Debug/netcoreapp3.1/Hayden" "/home/USER/hayden_2020/hayden-config.json"</syntaxhighlight>
* Hayden is picy about the syntax of this JSON file.
** Commas only go between key:value pars. Trailing commas are not tolerated by Hayden.
* Some options are implimented but not shown in the example JSON file.
** The option to scan archive.json to save threads that have recently fallen off the board: <code"ReadArchive": true</code>
**: This value is a child of  "source", as shown here:
**: <syntaxhighlight lang="json">
{
	"source" : {
		"type" : "4chan",
		"boards" : [
			"vmg"
		],
		"apiDelay" : 0.8,
		"boardDelay" : 30,
		"ReadArchive": true
	},
</syntaxhighlight>
* Some databses will not let you connect as the DB root account from a system user other than root.
** As a result of this, you should create a DB user specifically for Hayden to use.
* The database connection is defined by this string value:
*: <syntaxhighlight lang="json">"connectionString" : "Server=DB_NETWORK_ADDRESS;Port=DB_PORT;Database=DATABASE;Uid=DB_USERNAME;Pwd=DB_PASSWORD;CharSet=utf8mb4;IgnorePrepare=false",</syntaxhighlight>
** e.g. <syntaxhighlight lang="json">"connectionString" : "Server=localhost;Port=3306;Database=asagi;Uid=hayden;Pwd=some_secret_value;CharSet=utf8mb4;IgnorePrepare=false",</syntaxhighlight>
* Image downloading is controlled by two options:
*: <code>"fullImagesEnabled" : true,</code>
*: <br><code>"thumbnailsEnabled" : false,</code>
** These values must either be <code>false</code> or<code>true</code>.


== Running ==
# Ensure DB is running
#* Hayden requires the database to be available to function.
#* See the database sections of this document for information on setting up and running a database in a container.
# Start Hayden
#* It is STRONGLY RECCOMENDED to run Hayden inside a Ccreen / Tmux / Byobu session so Hayden persists after your terminal disconnects.
#*: e.g. One of the following three commands:
#*: <syntaxhighlight lang="bash">$ screen</syntaxhighlight>
#*: <syntaxhighlight lang="bash">$ tmux</syntaxhighlight>
#*: <syntaxhighlight lang="bash">$ byobu</syntaxhighlight>
#* Start Hayden itself:
#** <syntaxhighlight lang="bash">$ "/home/USER/hayden_2020/Hayden/Hayden/bin/Debug/netcoreapp3.1/Hayden" "/home/USER/hayden_2020/hayden-config.json"</syntaxhighlight>

== Proxies ==
* UNTESTED BY AUTHOR
* Hayden supports proxies and these can be configured through the config file.
* The most common SSH server programs used by linux support acting as a proxy out of the box. You just have to tell it to start the proxy connection.
** To test if your machine is rigged for automated SSH access to some remote host, run this command:
**: <syntaxhighlight lang="bash">$ ssh username@host</syntaxhighlight>
** If it connects you to the remote host without needing to type things in, then your SSH key configuration is probably fine.
* To start a SOCKS proxy over SSH, giving you a proxy accessible at the address localhost:10000:
*: <syntaxhighlight lang="bash">$ ssh -D 10000 user@<remote_host_address></syntaxhighlight>
* To tell hayden to use a proxy:
*: <syntaxhighlight lang="json">
	"proxies" : [
		{
			"type" : "socks",
			"url" : "http://au393.nordvpn.com:1080",
			"username" : "bane@gmail.com",
			"password" : "bigguy4u"
		}
	],
</syntaxhighlight>
* To tell hayden to use a proxy:
*: <syntaxhighlight lang="json">
	"proxies" : [
		{
			"type" : "socks",
			"url" : "http://localhost:10000",
			"username" : "TODO",
			"password" : "TODO"
		}
	],
</syntaxhighlight>


== Tips ==
* Use absolute filepaths instead of relative filepaths whenever you can, as it makes it harder to screw up as bad.
*: e.g. <code>/home/dir/subdir/file</code> instead of <code>./subdir/file</code> instead of <code>subdir/file</code>
* Back up your database frequently and regularly (i.e. mysqldump or similar)
** Consider scheduling this via cron to occur automatically.
* Run your programs inside screen/tmux/byobu so they dont get closed from you disconnecting your terminal.
* Use a good password. <code>test</code> is a bad password.
** Spaces need extra consideration regarding escaping, so try to avoid them.
** Alphanumerical gibberish is hard to remember and to type in. <code>a2Ggkp5xx1</code>
** Passphrases are easy to remember and to type in. <code>White_horse_lion_wave</code>
** Write down your usernames and passwords so you dont lose them.
* Thumbnails can probably be generated from full images and thus can be turned off if you dont plan on running a public archive.
* Save any error messages to a plain text file via copy/paste otherwise you won't be able to get real help from any experts.

== Example files ==
* These are what the configuration files should broardly look like.

=== Example config.json ===
* The following configuration file code instructs Hayden to download the 4chan /vmg/ board, including the old threads temporarily retained by 4chan's archive.json API endpoint.
*: <syntaxhighlight lang="json">
{
	"source" : {
		"type" : "4chan",
		"boards" : [
			"vmg"
		],
		"apiDelay" : 0.8,
		"boardDelay" : 30,
		"ReadArchive": true
	},

	"proxies" : [
	],
	
	"backend" : {
		"type" : "Asagi",
		
		"connectionString" : "Server=localhost;Port=3306;Database=hayden_db;Uid=hayden_un;Pwd=hayden_pw;CharSet=utf8mb4;IgnorePrepare=false",
		
		"downloadLocation" : "/home/USER/hayden_2020/dl/",
		
		"fullImagesEnabled" : true,
		"thumbnailsEnabled" : true,
		
		"sqlConnectionPoolSize" : 12
	}
}
</syntaxhighlight>

=== Example docker-compose.yml ===
* The following configuration file defines two containers for docker-compose .
** A MariaDB database container <code>db</code>
** A adminer DB webUI container <code>adminer</code>
*: <syntaxhighlight lang="bash">
# https://hub.docker.com/_/mariadb/
version: '3.1'
services:
  db: # SQL DB server
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: "root_super_secret_code"
      MYSQL_DATABASE: hayden_db
      MYSQL_USER: hayden_un
      MYSQL_PASSWORD: "hayden_super_secret_cod"
    ports:
      - "10306:3306" # HOST:CONTAINER
    volumes:
      - "/home/USER/hayden-2020/mariadb_docker:/var/lib/mysql" # HOST:CONTAINER
      - "/home/USER/hayden-2020/mariadb_docker_dump:/dump" # HOST:CONTAINER
  adminer: # WebGUI for SQL DB server
    image: adminer
    restart: always
    ports:
      - 10088:8080
</syntaxhighlight>