Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision |
Your text |
Line 37: |
Line 37: |
| --warc-max-size=1G \ | | --warc-max-size=1G \ |
| --user-agent "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27" | | --user-agent "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27" |
| </pre>
| |
|
| |
| === grab-site ===
| |
|
| |
| Grab-site can be particularly hard to install due to all the updated libraries needed, but even on CentOS 7 it can be done.
| |
|
| |
| First, install all the dependencies.
| |
|
| |
| {{bc|
| |
| sudo yum install gcc make gcc-c++ sqlite-devel readline-devel re2-devel libxml2-devel libffi-devel gcc openssl-devel bzip2-devel
| |
| }}
| |
|
| |
| Then follow the guide to have python 3.7 compiled per user. Doing this allows the user to upgrade it by themselves without needing admin intervention, though it does take a lot more time to compile python.
| |
|
| |
| https://github.com/ludios/grab-site#install-on-ubuntu-1604-1804-debian-9-stretch-debian-10-buster
| |
|
| |
| <pre>
| |
| # yum install gcc make gcc-c++ sqlite-devel readline-devel re2-devel libxml2-devel libffi-devel gcc openssl-devel bzip2-devel
| |
| # cd /usr/src
| |
| # wget https://www.python.org/ftp/python/3.7.1/Python-3.7.1.tgz
| |
| # tar xzf Python-3.7.1.tgz
| |
| # cd Python-3.7.1
| |
| # ./configure --enable-optimizations --enable-loadable-sqlite-extensions
| |
| # make altinstall
| |
| </pre>
| |
| logout as root, then become your own personal user:
| |
| <pre>
| |
| $ cd ~
| |
| $ echo 'export PATH=$PATH:/usr/local/bin' >> ~/.bashrc
| |
| $ pip3.7 install --process-dependency-links --no-binary --upgrade git+https://github.com/ludios/grab-site --user # install as user
| |
| </pre> | | </pre> |
|
| |
|