Merge pull request #46 from extremeshok/dev

2.0
This commit is contained in:
eXtremeSHOK 2021-03-21 02:51:44 +02:00 committed by GitHub
commit 0ef0170cb3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
22 changed files with 1184 additions and 325 deletions

View File

@ -3,9 +3,6 @@ You are free to use, modify and distribute, however you may not remove this noti
Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
License: BSD (Berkeley Software Distribution)
Originially based on:
Copyright (c) 2007 - 2013, Bill Landry (unofficialsigs@gmail.com)
All rights reserved.
Redistribution and use in source and binary forms, with or without modification,

190
README.md
View File

@ -1,37 +1,76 @@
# xshok-proxmox :: eXtremeSHOK.com Proxmox (pve)
Scripts for working with and optimizing proxmox
## Maintained and provided by <https://eXtremeSHOK.com>
### Please Submit Patches / Pull requests
## Optimization / Post Install Script (install-post.sh aka postinstall.sh) *run once*
*not required if server setup with install-hetzner.sh*
* 'reboot-quick' command which uses kexec to boot the latest kernel set in the boot loader
* Force APT to use IPv4
Turns a fresh proxmox install into an optimised proxmox host
*not required if server setup with hetzner-install-proxmox.sh*
'reboot-quick' command which uses kexec to boot the latest kernel, its a fast method of rebooting, without needing to do a hardware reboot
* Disable the enterprise repo, enable the public repo, Add non-free sources
* Fixes known bugs (public key missing, max user watches, etc)
* Update the system
* Install ceph, ksmtuned, openvswitch-switch, zfsutils and common system utilities
* Increase vzdump backup speed, enable pigz and fix ionice
* Increase max Key limits, max user watches, max File Discriptor Limits, ulimits
* Detect AMD EPYC CPU and install kernel 4.15
* Detect AMD EPYC CPU and Apply EPYC fixes to kernel and KVM
* Install and configure ZFS-auto-snapshots (12x5min, 7daily, 4weekly, 3monthly)
* Detect AMD EPYC CPU and Apply Fixes
* Force APT to use IPv4
* Update proxmox and install various system utils
* Customise bashrc
* add the latest ceph provided by proxmox
* Disable portmapper / rpcbind (security)
* set-timezone UTC and enable timesyncd as nntp client
* Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy
* Protect the web interface with fail2ban
* Detect if is running in a virtual machine and install the relavant guest agent
* Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch)
* Limit the size and optimise journald
* Install kernel source headers
* Install kexec, allows for quick reboots into the latest updated kernel set as primary in the boot-loader.
* Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size
* Set language, if chnaged will disable XS_NOAPTLANG
* Increase max user watches, FD limit, FD ulimit, max key limit, ulimits
* Optimise logrotate
* Lynis security scan tool by Cisofy
* Increase Max FS open files
* Optimise Memory
* Pretty MOTD BANNER
* Enable Network optimising
* Save bandwidth and skip downloading additional languages, requires XS_LANG="en_US.UTF-8"
* Disable enterprise proxmox repo
* Remove subscription banner
* Install openvswitch for a virtual internal network
* Detect if this is an OVH server and install OVH Real Time Monitoring
* Set pigz to replace gzip, 2x faster gzip compression
* Detect OVH Server and install OVH RTM (real time monitoring)"
* Protect the webinterface with fail2ban (security)
* Optimize ZFS arc size depending on installed memory, Use 1/16 RAM for MAX cache, 1/8 RAM for MIN cache, or 1GB
* ZFS Tuning, set prefetch method and max write speed to l2arc
* Enable TCP BBR congestion control, improves overall network throughput
* Bugfix: high swap usage with low memory usage
* Enable TCP BBR congestion control
* Enable TCP fastopen
* Enable testing proxmox repo
* Automatically Synchronize the time
* Set Timezone, empty = set automatically by IP
* Install common system utilities
* Increase vzdump backup speed
* Optimise ZFS arc size accoring to memory size
* Install zfs-auto-snapshot
https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh
return value is 0
Or run *install-post.sh* after installation
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh -c -O install-post.sh && bash install-post.sh && rm install-post.sh
```
## TO SET AND USE YOUR OWN OPTIONS
User Defined Options for (install-post.sh) post-installation script for Proxmox are set in the xs-install-post.env, see the sample : xs-install-post.env.sample
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/xs-install-post.env.sample -c -O xs-install-post.env
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh -c -O install-post.sh
nano xs-install-post.env
bash install-post.sh
```
## Install Proxmox Recommendations
Recommeneded partitioning scheme:
* Raid 1 (mirror) 40 000MB ext4 /
@ -44,99 +83,24 @@ Recommeneded partitioning scheme:
* Remaining for lv xfs /var/lib/vz (LVM)
# Hetzner Proxmox Installation Guide #
*includes and runs the (install-post.sh) script*
* Select the Rescue tab for the specific server, via the hetzner robot manager
* * Operating system=Linux
* * Architecture=64 bit
* * Public key=*optional*
* --> Activate rescue system
* Select the Reset tab for the specific server,
* Check: Execute an automatic hardware reset
* --> Send
* Wait a few mins
* Connect via ssh/terminal to the rescue system running on your server and run the following
````
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-hetzner.sh -c -O install-hetzner.sh && chmod +x install-hetzner.sh
./install-hetzner.sh
````
* Reboot
* Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## LVM to ZFS
````
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh
./lvm-2-zfs.sh && rm lvm-2-zfs.sh
````
* Reboot
* Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## NETWORKING (vmbr0 vmbr1)
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh
./network-configure.sh && rm network-configure.sh
```
* Reboot
* Post Install: Now login via ssh as root and create a password, which will be used for the webinterface when logging in with pam authentication
see *hetzner* folder
# OVH Proxmox Installation Guide #
Select install for the specific server, via the ovh manager
* --INSTALL-->
* Install from an OVH template
* --NEXT-->
* Type of OS: Ready-to-go (graphical user interface)
* VPS Proxmox VE *(pick the latest non zfs version)*
* Language: EN
* Target disk arrray: *(always select the SSD array if you have ssd and hdd arrays)*
* Enable/Tick: Customise the partition configuration
* --NEXT-->
* Disks used for this installation: *(All of them)*
* (Remove all the partitions and do the following)
* Type: Filesystem: Mount Point: LVM Name: RAID: Size:
* * 1 primary Ext4 / - 1 20.0 GB
* * 2 primary Swap swap - - 2 x 8.0 GB *(minimum 16GB total, set recommended swap size)*
* * 3 LV xfs /var/lib/vz data 1 REMAINING GB *(use all the remaining space)*
* --NEXT-->
* Hostname: server.fqdn.com
* Installation script (URL): https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh
* Script return value: 0
* SSH keys: *(always suggested, however if this value is used a webinterface login will not work without setting a root password in shell)*
* --CONFIRM-->
After installation, Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## LVM to ZFS
````
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh
./lvm-2-zfs.sh && rm lvm-2-zfs.sh
````
* Reboot
* Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## NETWORKING (vmbr0 vmbr1)
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh
./network-configure.sh && rm network-configure.sh
```
* Reboot
* Post Install: Now login via ssh as root and create a password, which will be used for the webinterface when logging in with pam authentication
# Advance Installation Options #
Assumptions: Proxmox installed, SSD raid1 partitions mounted as /xshok/zfs-slog and /xshok/zfs-cache, 1+ unused hdd which will be made into a zfspool
* Connect via ssh/terminal to the new Proxmox system running on your server and run the follow
## Create ZFS from unused devices (createzfs.sh)
**NOTE: WILL DESTROY ALL DATA ON SPECIFIED DEVICES**
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/createzfs.sh -c -O createzfs.sh && chmod +x createzfs.sh
./createzfs.sh poolname /dev/device1 /dev/device2
```
## Create ZFS cache and slog from /xshok/zfs-cache and /xshok/zfs-slog partitions and adds them to a zpool (xshok_slog_cache-2-zfs.sh) *optional*
**NOTE: WILL DESTROY ALL DATA ON SPECIFIED PARTITIONS**
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/xshok_slog_cache-2-zfs.sh -c -O xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh
./xshok_slog_cache-2-zfs.sh poolname
```
* Reboot
see *ovh* folder
# ------- SCRIPTS ------
## Convert from Debian 10 to Proxmox 6 (debian10-2-proxmox6.sh) *optional*
Assumptions: Debian9 installed with a valid FQDN hostname set
* Tested on KVM, VirtualBox and Dedicated Server
* Will automatically detect cloud-init and disable.
* Will automatically generate a correct /etc/hosts
* Note: will automatically run the install-post.sh script
```
curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian10-2-proxmox6.sh && chmod +x debian10-2-proxmox6.sh
./debian10-2-proxmox6.sh
```
## Convert from Debian 9 to Proxmox 5 (debian9-2-proxmox5.sh) *optional*
Assumptions: Debian9 installed with a valid FQDN hostname set
* Tested on KVM, VirtualBox and Dedicated Server
@ -144,14 +108,14 @@ Assumptions: Debian9 installed with a valid FQDN hostname set
* Will automatically generate a correct /etc/hosts
* Note: will automatically run the install-post.sh script
```
curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian9-2-proxmox5.sh && chmod +x debian9-2-proxmox5.sh
curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian9-2-proxmox5.sh && chmod +x debian9-2-proxmox5.sh
./debian9-2-proxmox5.sh
```
## Enable Docker support for an LXC container (pve-enable-lxc-docker.sh) *optional*
There can be security implications as the LXC container is running in a higher privileged mode.
```
curl https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/pve-enable-lxc-docker.sh --output /usr/sbin/pve-enable-lxc-docker && chmod +x /usr/sbin/pve-enable-lxc-docker
curl https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/helpers/pve-enable-lxc-docker.sh --output /usr/sbin/pve-enable-lxc-docker && chmod +x /usr/sbin/pve-enable-lxc-docker
pve-enable-lxc-docker container_id
```
@ -173,7 +137,7 @@ Converts the a MDADM BASED LVM into a ZFS raid 1 (mirror)
**NOTE: WILL DESTROY ALL DATA ON LVM_MOUNT_POINT**
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh
./lvm-2-zfs.sh
```
@ -188,7 +152,7 @@ Creates a zfs pool from specified devices
**NOTE: WILL DESTROY ALL DATA ON SPECIFIED DEVICES**
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/createzfs.sh -c -O createzfs.sh && chmod +x createzfs.sh
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/createzfs.sh -c -O createzfs.sh && chmod +x createzfs.sh
./createzfs.sh poolname /dev/device1 /dev/device2
```
@ -198,7 +162,7 @@ Creates a zfs pool from specified devices
**NOTE: WILL DESTROY ALL DATA ON SPECIFIED PARTITIONS**
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/xshok_slog_cache-2-zfs.sh -c -O xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/xshok_slog_cache-2-zfs.sh -c -O xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh
./xshok_slog_cache-2-zfs.sh poolname
```
@ -223,21 +187,21 @@ ALSO CREATES A NAT Private Network as vmbr1
NOTE: WILL OVERWRITE /etc/network/interfaces
A backup will be created as /etc/network/interfaces.timestamp
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh
./network-configure.sh && rm network-configure.sh
```
## Creates default routes to allow for extra ip ranges to be used (network-addiprange.sh) *optional*
If no interface is specified the default gateway interface will be detected and used.
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/network-addiprange.sh -c -O network-addiprange.sh && chmod +x network-addiprange.sh
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-addiprange.sh -c -O network-addiprange.sh && chmod +x network-addiprange.sh
./network-addiprange.sh ip.xx.xx.xx/cidr interface_optional
```
## Create Private mesh vpn/network (tincvpn.sh)
tinc private mesh vpn/network which supports multicast, ideal for private cluster communication
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/tincvpn.sh -c -O tincvpn.sh && chmod +x tincvpn.sh
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/tincvpn.sh -c -O tincvpn.sh && chmod +x tincvpn.sh
./tincvpn.sh -h
```
### Example for 3 node Cluster

View File

@ -0,0 +1,156 @@
#!/usr/bin/env bash
################################################################################
# This is property of eXtremeSHOK.com
# You are free to use, modify and distribute, however you may not remove this notice.
# Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
################################################################################
#
# Script updates can be found at: https://github.com/extremeshok/xshok-proxmox
#
# Debian 10 to Proxmox 6 conversion script
#
# License: BSD (Berkeley Software Distribution)
#
################################################################################
#
# Assumptions: Debian10 installed with a valid FQDN hostname set
#
# Tested on KVM, VirtualBox and Dedicated Server
#
# Will automatically detect cloud-init and disable.
# Will automatically generate a correct /etc/hosts
#
# Note: will automatically run the install-post.sh script
#
# Thank you @floco
#
# Usage:
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian10-2-proxmox6.sh && chmod +x debian10-2-proxmox6.sh
# ./debian10-2-proxmox6.sh
#
#
################################################################################
#
# THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT
#
################################################################################
#todo : verify and check
# Set the local
export LANG="en_US.UTF-8"
export LC_ALL="C"
sh -c "echo -e 'LANG=en_US.UTF-8\nLC_ALL=en_US.UTF-8' > /etc/default/locale"
#create lock dir for aptitude
if [ -d "/run/lock" ] ; then
mkdir /run/lock
chmod a+rwxt /run/lock
fi
echo "Deinstalling any linux firmware packages "
firmware="$(dpkg -l | grep -i 'firmware-')"
if [ -n "$firmware" ]; then
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge firmware-bnx2x firmware-realtek firmware-linux firmware-linux-free firmware-linux-nonfree
else
echo "No firmware packages loaded"
fi
echo "Deinstalling the Debian standard kernel packages "
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge linux-image-amd64
echo "Removing conflicting packages"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge os-prober
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoremove
apt-get clean all
echo "Auto detecting existing network settings"
default_interface="$(ip route | awk '/default/ { print $5 }' | grep -v "vmbr")"
if [ "$default_interface" == "" ]; then
#filter the interfaces to get the default interface and which is not down and not a virtual bridge
default_interface="$(ip link | sed -e '/state DOWN / { N; d; }' | sed -e '/veth[0-9].*:/ { N; d; }' | sed -e '/vmbr[0-9].*:/ { N; d; }' | sed -e '/tap[0-9].*:/ { N; d; }' | sed -e '/lo:/ { N; d; }' | head -n 1 | cut -d':' -f 2 | xargs)"
fi
if [ "$default_interface" == "" ]; then
echo "ERROR: Could not detect default interface"
exit 1
fi
default_v4="$(ip -4 addr show dev "$default_interface" | awk '/inet/ { print $2 }' )"
default_v4ip=${default_v4%/*}
if [ "$default_v4ip" == "" ] ; then
echo "ERROR: Could not detect default IPv4 address"
echo "IP: ${default_v4ip}"
exit 1
fi
echo "Configure /etc/hosts"
if [ -f /etc/cloud/cloud.cfg ] ; then
echo 'manage_etc_hosts: False' | tee --append /etc/cloud/cloud.cfg
fi
sed -i "s/^ - update_etc_hosts/# - update_etc_hosts/" /etc/cloud/cloud.cfg
cat <<EOF > /etc/hosts
127.0.0.1 localhost.localdomain localhost
${default_v4ip} $(hostname -f) $(hostname -s) pvelocalhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
EOF
echo "Add Proxmox repo to APT sources"
cat <<EOF >> /etc/apt/sources.list.d/proxmox.list
# PVE packages provided by proxmox.com"
deb http://download.proxmox.com/debian/pve buster pve-no-subscription
EOF
wget -q "http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg" -O /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
apt-get update > /dev/null
echo "Upgrading system"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' dist-upgrade
echo "Installing postfix"
cat <<EOF | debconf-set-selections
postfix postfix/mailname string $(cat /etc/hostname)
postfix postfix/destinations string $(cat /etc/hostname), proxmox, localhost.localdomain, localhost
postfix postfix/chattr boolean false
postfix postfix/mailbox_limit string 0
postfix postfix/main_mailer_type select Local only
postfix postfix/mynetworks string 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
postfix postfix/protocols select all
postfix postfix/recipient_delim string +
postfix postfix/rfc1035_violation boolean false
EOF
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install -y postfix
echo "Installing open-iscsi"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install -y open-iscsi
echo "Installing proxmox-ve"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install -y proxmox-ve
echo "Remove legacy (4.19) kernel"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge linux-image-4.19*
echo "Force grub to update"
update-grub
echo "Done installing Proxmox VE"
echo "Creating admin user"
pveum groupadd admin -comment "System Administrators"
pveum aclmod / -group admin -role Administrator
pveum useradd admin@pve -comment "Admin"
pveum usermod admin@pve -group admin
# export NO_MOTD_BANNER=true
# echo "Fetching postinstall script"
# wget https://raw.githubusercontent.com/floco/xshok-proxmox/master/install-post.sh -c -O install-post.sh && chmod +x install-post.sh
# if grep -q '#!/usr/bin/env bash' "install-post.sh"; then
# bash install-post.sh
# fi
# echo "Setting admin user password"
# pveum passwd admin@pve

View File

@ -23,7 +23,7 @@
# Note: will automatically run the install-post.sh script
#
# Usage:
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian9-2-proxmox5.sh && chmod +x debian9-2-proxmox5.sh
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/debian-2-proxmox/debian9-2-proxmox5.sh && chmod +x debian9-2-proxmox5.sh
# ./debian9-2-proxmox5.sh
#
################################################################################

View File

@ -0,0 +1,4 @@
#Install PVE-Edge-Kernel
wget https://github.com/fabianishere/pve-edge-kernel/releases/download/v5.11.0-2/pve-edge-kernel-5.11.0-2_5.11.0-2+zen21_amd64.deb
apt install ./pve-edge-kernel-5.11.0-2_5.11.0-2+zen21_amd64.deb

View File

@ -15,9 +15,12 @@
#
# Note:
# There can be security implications as the LXC container is running in a higher privileged mode.
# Not advisable to run docker inside a LXC container.
# Correct way is to create a VM (qemu/kvm) which will be used exclusively for docker.
# ie. fresh ubuntu lts server with https://github.com/extremeshok/xshok-docker
#
# Usage:
# curl https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/pve-enable-lxc-docker.sh --output /usr/sbin/pve-enable-lxc-docker && chmod +x /usr/sbin/pve-enable-lxc-docker
# curl https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/helpers/pve-enable-lxc-docker.sh --output /usr/sbin/pve-enable-lxc-docker && chmod +x /usr/sbin/pve-enable-lxc-docker
# pve-enable-lxc-docker container_id
#
################################################################################

32
hetzner/README.md Normal file
View File

@ -0,0 +1,32 @@
# Hetzner Proxmox Installation Guide #
*includes and runs the (install-post.sh) script*
* Select the Rescue tab for the specific server, via the hetzner robot manager
* * Operating system=Linux
* * Architecture=64 bit
* * Public key=*optional*
* --> Activate rescue system
* Select the Reset tab for the specific server,
* Check: Execute an automatic hardware reset
* --> Send
* Wait a few mins
* Connect via ssh/terminal to the rescue system running on your server and run the following
````
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/hetzner/hetzner-install-proxmox.sh -c -O hetzner-install-proxmox.sh && chmod +x hetzner-install-proxmox.sh
./hetzner-install-proxmox.sh "your.hostname.here"
````
* Reboot
* Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## LVM to ZFS
````
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh
./lvm-2-zfs.sh && rm lvm-2-zfs.sh
````
* Reboot
* Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## NETWORKING (vmbr0 vmbr1)
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh
./network-configure.sh && rm network-configure.sh
```
* Reboot
* Post Install: Now login via ssh as root and create a password, which will be used for the webinterface when logging in with pam authentication

View File

@ -35,6 +35,8 @@
#
################################################################################
#todo: add nvme support /nvme0n1 and /dev/nvme1n1
#set size of swap partition or leave blank for autoconfig, USE NUMBER ONLY, will be in gbytes, 0 to disable
MY_SWAP=""
#set size of cache partition or leave blank for autoconfig, USE NUMBER ONLY, will be in gbytes, 0 to disable
@ -57,7 +59,7 @@ export LC_ALL="C"
installimage_bin="/root/.oldroot/nfs/install/installimage"
MY_HOSTNAME="$1"
if [ "$MY_HOSTNAME" == "" ]; then
if [ "$MY_HOSTNAME" == "pve.host.invalid" ]; then
echo "Please set a hostname"
echo "$0 host.name"
exit 1
@ -233,7 +235,7 @@ fi
sleep 5
# Detect the latest installimage file to use
installimage_file=$(find root/images/ -iname 'Debian-*-stretch-64-minimal.tar.gz' | sort --version-sort --field-separator=- --key=2,2 -r | head -n1)
installimage_file=$(find /root/images/ -iname 'Debian-*-buster-64-minimal.tar.gz' | sort --version-sort --field-separator=- --key=2,2 -r | head -n1)
if [ ! -f $installimage_file ] ; then
echo "Error: Image file was not found: ${installimage_file}"
echo "Please log an issue on the github repo with the following"
@ -242,7 +244,7 @@ if [ ! -f $installimage_file ] ; then
fi
#fetching post install
curl "https://raw.githubusercontent.com/hetzneronline/installimage/master/post-install/proxmox5" --output /post-install
curl "https://raw.githubusercontent.com/hetzneronline/installimage/master/post-install/proxmox6" --output /post-install
#Customising post install file
echo "wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh -c -O install-post.sh && bash install-post.sh && rm install-post.sh" >> /post-install

View File

@ -13,149 +13,387 @@
#
################################################################################
#
# Version: 2.0
#
# Assumptions: proxmox installed
#
# Notes:
# to disable the MOTD banner, set the env NO_MOTD_BANNER to true (export NO_MOTD_BANNER=true)
# openvswitch will be disabled (removed) when ifupdown2 is enabled
# ifupdown2 will be disabled (removed) when openvswitch is enabled
#
# Docker : not advisable to run docker on the Hypervisor(proxmox) directly.
# Correct way is to create a VM which will be used exclusively for docker.
# ie. fresh ubuntu lts server with https://github.com/extremeshok/xshok-docker
################################################################################
#
# THERE ARE NO USER CONFIGURABLE OPTIONS IN THIS SCRIPT
#
################################################################################
##### T O S E T Y O U R O P T I O N S ######
# User Defined Options for (install-post.sh) post-installation script for Proxmox
# are set in the xs-install-post.env, see the sample : xs-install-post.env.sample
##### D O N O T E D I T B E L O W ######
#### VARIABLES / options
# Detect AMD EPYC CPU and Apply Fixes
XS_AMDFIXES="yes"
# Force APT to use IPv4
XS_APTIPV4="yes"
# Update proxmox and install various system utils
XS_APTUPGRADE="yes"
# Customise bashrc
XS_BASHRC="yes"
# Add the latest ceph provided by proxmox
XS_CEPH="yes"
# Disable portmapper / rpcbind (security)
XS_DISABLERPC="yes"
# Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy
XS_ENTROPY="yes"
# Protect the web interface with fail2ban
XS_FAIL2BAN="yes"
# Detect if is running in a virtual machine and install the relavant guest agent
XS_GUESTAGENT="yes"
# Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch)
XS_IFUPDOWN2="yes"
# Limit the size and optimise journald
XS_JOURNALD="yes"
# Install kernel source headers
XS_KERNELHEADERS="yes"
# Install kexec, allows for quick reboots into the latest updated kernel set as primary in the boot-loader.
XS_KEXEC="yes"
# Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size
XS_KSMTUNED="yes"
# Set language, if chnaged will disable XS_NOAPTLANG
XS_LANG="en_US.UTF-8"
# Increase max user watches, FD limit, FD ulimit, max key limit, ulimits
XS_LIMITS="yes"
# Optimise logrotate
XS_LOGROTATE="yes"
# Lynis security scan tool by Cisofy
XS_LYNIS="yes"
# Increase Max FS open files
XS_MAXFS="yes"
# Optimise Memory
XS_MEMORYFIXES="yes"
# Pretty MOTD BANNER
XS_MOTD="yes"
# Enable Network optimising
XS_NET="yes"
# Save bandwidth and skip downloading additional languages, requires XS_LANG="en_US.UTF-8"
XS_NOAPTLANG="yes"
# Disable enterprise proxmox repo
XS_NOENTREPO="yes"
# Remove subscription banner
XS_NOSUBBANNER="yes"
# Install openvswitch for a virtual internal network
XS_OPENVSWITCH="no"
# Detect if this is an OVH server and install OVH Real Time Monitoring
XS_OVHRTM="yes"
# Set pigz to replace gzip, 2x faster gzip compression
XS_PIGZ="yes"
# Bugfix: high swap usage with low memory usage
XS_SWAPPINESS="yes"
# Enable TCP BBR congestion control
XS_TCPBBR="yes"
# Enable TCP fastopen
XS_TCPFASTOPEN="yes"
# Enable testing proxmox repo
XS_TESTREPO="no"
# Automatically Synchronize the time
XS_TIMESYNC="yes"
# Set Timezone, empty = set automatically by IP
XS_TIMEZONE=""
# Install common system utilities
XS_UTILS="yes"
# Increase vzdump backup speed
XS_VZDUMP="yes"
# Optimise ZFS arc size accoring to memory size
XS_ZFSARC="yes"
# Install zfs-auto-snapshot
XS_ZFSAUTOSNAPSHOT="yes"
################# D O N O T E D I T ######################################
echo "Processing .... "
# VARIABLES are overrideen with xs-install-post.env
if [ -f "xs-install-post.env" ] ; then
echo "Loading variables from xs-install-post.env ..."
# shellcheck disable=SC1091
source xs-install-post.env;
fi
# Set the local
export LANG="en_US.UTF-8"
if [ "$XS_LANG" == "" ] ; then
XS_LANG="en_US.UTF-8"
fi
export LANG="$XS_LANG"
export LC_ALL="C"
## Force APT to use IPv4
echo -e "Acquire::ForceIPv4 \"true\";\\n" > /etc/apt/apt.conf.d/99force-ipv4
## disable enterprise proxmox repo
if [ -f /etc/apt/sources.list.d/pve-enterprise.list ]; then
echo -e "#deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise\\n" > /etc/apt/sources.list.d/pve-enterprise.list
fi
## enable public proxmox repo
if [ ! -f /etc/apt/sources.list.d/proxmox.list ] && [ ! -f /etc/apt/sources.list.d/pve-public-repo.list ] && [ ! -f /etc/apt/sources.list.d/pve-install-repo.list ] ; then
echo -e "deb http://download.proxmox.com/debian/pve buster pve-no-subscription\\n" > /etc/apt/sources.list.d/pve-public-repo.list
# enforce proxmox
if [ ! -f "/etc/pve/.version" ] ; then
echo "ERROR: This script only supports Proxmox"
exit 1
fi
## Add non-free contrib to sources
sed -i "s/main /main non-free contrib/g" /etc/apt/sources.list
## Add the latest ceph provided by proxmox
echo "deb http://download.proxmox.com/debian/ceph-nautilus buster main" > /etc/apt/sources.list.d/ceph.list
## Refresh the package lists
apt-get update > /dev/null
## Remove conflicting utilities
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge ntp openntpd chrony ksm-control-daemon
## Fix no public key error for debian repo
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install debian-archive-keyring
## Update proxmox and install various system utils
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' dist-upgrade
pveam update
## Fix no public key error for debian repo
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install debian-archive-keyring
## Install openvswitch for a virtual internal network
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install openvswitch-switch
## Install zfs support, appears to be missing on some Proxmox installs.
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfsutils
## Install zfs-auto-snapshot
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfs-auto-snapshot
# make 5min snapshots , keep 12 5min snapshots
if [ -f "/etc/cron.d/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=12|g' /etc/cron.d/zfs-auto-snapshot
sed -i 's|*/[0-9]*|*/5|g' /etc/cron.d/zfs-auto-snapshot
fi
# keep 24 hourly snapshots
if [ -f "/etc/cron.hourly/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=24|g' /etc/cron.hourly/zfs-auto-snapshot
fi
# keep 7 daily snapshots
if [ -f "/etc/cron.daily/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=7|g' /etc/cron.daily/zfs-auto-snapshot
fi
# keep 4 weekly snapshots
if [ -f "/etc/cron.weekly/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=4|g' /etc/cron.weekly/zfs-auto-snapshot
fi
# keep 3 monthly snapshots
if [ -f "/etc/cron.monthly/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=3|g' /etc/cron.monthly/zfs-auto-snapshot
if [ -f "/etc/extremeshok" ] ; then
echo "ERROR: Script can only be run once"
exit 1
fi
## Install missing ksmtuned
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install ksmtuned
systemctl enable ksmtuned
systemctl enable ksm
# SET VARIBLES
## Install ceph support
echo "Y" | pveceph install
OS_CODENAME="$(grep "VERSION_CODENAME=" /etc/os-release | cut -d"=" -f 2 | xargs )"
RAM_SIZE_GB=$(( $(vmstat -s | grep -i "total memory" | xargs | cut -d" " -f 1) / 1024 / 1000))
## Install common system utilities
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install whois omping tmux sshpass wget axel nano pigz net-tools htop iptraf iotop iftop iperf vim vim-nox unzip zip software-properties-common aptitude curl dos2unix dialog mlocate build-essential git ipset
#snmpd snmp-mibs-downloader
## Detect AMD EPYC CPU and install kernel 4.15
if [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "EPYC")" != "" ]; then
echo "AMD EPYC detected"
#Apply EPYC fix to kernel : Fixes random crashing and instability
if ! grep "GRUB_CMDLINE_LINUX_DEFAULT" /etc/default/grub | grep -q "idle=nomwait" ; then
echo "Setting kernel idle=nomwait"
sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="/GRUB_CMDLINE_LINUX_DEFAULT="idle=nomwait /g' /etc/default/grub
update-grub
fi
echo "Installing kernel 4.15"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install pve-kernel-4.15
if [ "$XS_LANG" == "en_US.UTF-8" ] && [ "$XS_NOAPTLANG" == "yes" ] ; then
# save bandwidth and skip downloading additional languages
echo -e "Acquire::Languages \"none\";\\n" > /etc/apt/apt.conf.d/99-xs-disable-translations
fi
if [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "EPYC")" != "" ] || [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "Ryzen")" != "" ]; then
## Add msrs ignore to fix Windows guest on EPIC/Ryzen host
echo "options kvm ignore_msrs=Y" >> /etc/modprobe.d/kvm.conf
echo "options kvm report_ignored_msrs=N" >> /etc/modprobe.d/kvm.conf
if [ "$XS_APTIPV4" == "yes" ] ; then
# force APT to use IPv4
echo -e "Acquire::ForceIPv4 \"true\";\\n" > /etc/apt/apt.conf.d/99-xs-force-ipv4
fi
## Install kexec, allows for quick reboots into the latest updated kernel set as primary in the boot-loader.
# use command 'reboot-quick'
echo "kexec-tools kexec-tools/load_kexec boolean false" | debconf-set-selections
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install kexec-tools
if [ "$XS_NOENTREPO" == "yes" ] ; then
# disable enterprise proxmox repo
if [ -f /etc/apt/sources.list.d/pve-enterprise.list ]; then
sed -i "s/^deb/#deb/g" /etc/apt/sources.list.d/pve-enterprise.list
fi
# enable free public proxmox repo
if [ ! -f /etc/apt/sources.list.d/proxmox.list ] && [ ! -f /etc/apt/sources.list.d/pve-public-repo.list ] && [ ! -f /etc/apt/sources.list.d/pve-install-repo.list ] ; then
echo -e "deb http://download.proxmox.com/debian/pve ${OS_CODENAME} pve-no-subscription\\n" > /etc/apt/sources.list.d/pve-public-repo.list
fi
if [ "$XS_TESTREPO" == "yes" ] ; then
# enable testing proxmox repo
echo -e "deb http://download.proxmox.com/debian/pve ${OS_CODENAME} pvetest\\n" > /etc/apt/sources.list.d/pve-testing-repo.list
fi
fi
cat <<'EOF' > /etc/systemd/system/kexec-pve.service
# rebuild and add non-free to /etc/apt/sources.list
cat <<EOF > /etc/apt/sources.list
deb https://ftp.debian.org/debian ${OS_CODENAME} main contrib
deb https://ftp.debian.org/debian ${OS_CODENAME}-updates main contrib
# non-free
deb https://httpredir.debian.org/debian/ ${OS_CODENAME} main contrib non-free
# security updates
deb https://security.debian.org/debian-security ${OS_CODENAME}/updates main contrib
EOF
# Refresh the package lists
apt-get update > /dev/null 2>&1
# Remove conflicting utilities
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge ntp openntpd chrony
# Fixes for common apt repo errors
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install apt-transport-https debian-archive-keyring ca-certificates curl
if [ "$XS_APTUPGRADE" == "yes" ] ; then
# update proxmox and install various system utils
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' dist-upgrade
pveam update
fi
# Install packages which are sometimes missing on some Proxmox installs.
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfsutils-linux
if [ "$XS_UTILS" == "yes" ] ; then
# Install common system utilities
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install \
axel \
build-essential \
dialog \
dnsutils \
dos2unix \
git \
gnupg-agent \
grc \
htop \
iftop \
iotop \
iperf \
ipset \
iptraf \
mlocate \
msr-tools \
nano \
net-tools \
omping \
software-properties-common \
sshpass \
tmux \
unzip \
vim \
vim-nox \
wget \
whois \
zip
fi
if [ "$XS_CEPH" == "yes" ] ; then
# Add the latest ceph provided by proxmox
echo "deb https://download.proxmox.com/debian/ceph-octopus ${OS_CODENAME} main" > /etc/apt/sources.list.d/ceph.list
## Refresh the package lists
apt-get update > /dev/null 2>&1
## Install ceph support
echo "Y" | pveceph install
fi
if [ "$XS_LYNIS" == "yes" ] ; then
# Lynis security scan tool by Cisofy
wget -O - https://packages.cisofy.com/keys/cisofy-software-public.key | apt-key add -
## Add the latest lynis
echo "deb https://packages.cisofy.com/community/lynis/deb/ stable main" > /etc/apt/sources.list.d/cisofy-lynis.list
## Refresh the package lists
apt-get update > /dev/null 2>&1
## Install ceph support
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install lynis
fi
if [ "$XS_OPENVSWITCH" == "yes" ] && [ "$XS_IFUPDOWN2" == "no" ] ; then
## Install openvswitch for a virtual internal network
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install ifenslave ifupdown
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge ifupdown2
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install openvswitch-switch
else
## Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch)
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge openvswitch-switch
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install ifupdown2
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' purge ifenslave ifupdown
fi
if [ "$XS_ZFSAUTOSNAPSHOT" == "yes" ] ; then
## Install zfs-auto-snapshot
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install zfs-auto-snapshot
# make 5min snapshots , keep 12 5min snapshots
if [ -f "/etc/cron.d/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=12|g' /etc/cron.d/zfs-auto-snapshot
sed -i 's|*/[0-9]*|*/5|g' /etc/cron.d/zfs-auto-snapshot
fi
# keep 24 hourly snapshots
if [ -f "/etc/cron.hourly/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=24|g' /etc/cron.hourly/zfs-auto-snapshot
fi
# keep 7 daily snapshots
if [ -f "/etc/cron.daily/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=7|g' /etc/cron.daily/zfs-auto-snapshot
fi
# keep 4 weekly snapshots
if [ -f "/etc/cron.weekly/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=4|g' /etc/cron.weekly/zfs-auto-snapshot
fi
# keep 3 monthly snapshots
if [ -f "/etc/cron.monthly/zfs-auto-snapshot" ] ; then
sed -i 's|--keep=[0-9]*|--keep=3|g' /etc/cron.monthly/zfs-auto-snapshot
fi
fi
if [ "$XS_KSMTUNED" == "yes" ] ; then
## Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install ksm-control-daemon
if [[ RAM_SIZE_GB -le 16 ]] ; then
# start at 50% full
KSM_THRES_COEF=50
KSM_SLEEP_MSEC=80
elif [[ RAM_SIZE_GB -le 32 ]] ; then
# start at 60% full
KSM_THRES_COEF=40
KSM_SLEEP_MSEC=60
elif [[ RAM_SIZE_GB -le 64 ]] ; then
# start at 70% full
KSM_THRES_COEF=30
KSM_SLEEP_MSEC=40
elif [[ RAM_SIZE_GB -le 128 ]] ; then
# start at 80% full
KSM_THRES_COEF=20
KSM_SLEEP_MSEC=20
else
# start at 90% full
KSM_THRES_COEF=10
KSM_SLEEP_MSEC=10
fi
sed -i -e "s/\# KSM_THRES_COEF=.*/KSM_THRES_COEF=${KSM_THRES_COEF}/g" /etc/ksmtuned.conf
sed -i -e "s/\# KSM_SLEEP_MSEC=.*/KSM_SLEEP_MSEC=${KSM_SLEEP_MSEC}/g" /etc/ksmtuned.conf
systemctl enable ksmtuned
fi
if [ "$XS_AMDFIXES" == "yes" ] ; then
## Detect AMD EPYC CPU and Apply Fixes
if [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "EPYC")" != "" ]; then
echo "AMD EPYC detected"
#Apply EPYC fix to kernel : Fixes random crashing and instability
if ! grep "GRUB_CMDLINE_LINUX_DEFAULT" /etc/default/grub | grep -q "idle=nomwait" ; then
echo "Setting kernel idle=nomwait"
sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="/GRUB_CMDLINE_LINUX_DEFAULT="idle=nomwait /g' /etc/default/grub
update-grub
fi
fi
if [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "EPYC")" != "" ] || [ "$(grep -i -m 1 "model name" /proc/cpuinfo | grep -i "Ryzen")" != "" ]; then
## Add msrs ignore to fix Windows guest on EPIC/Ryzen host
echo "options kvm ignore_msrs=Y" >> /etc/modprobe.d/kvm.conf
echo "options kvm report_ignored_msrs=N" >> /etc/modprobe.d/kvm.conf
fi
fi
if [ "$XS_KERNELHEADERS" == "yes" ] ; then
## Install kernel source headers
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install pve-headers module-assistant
fi
if [ "$XS_KEXEC" == "yes" ] ; then
## Install kexec, allows for quick reboots into the latest updated kernel set as primary in the boot-loader.
# use command 'reboot-quick'
echo "kexec-tools kexec-tools/load_kexec boolean false" | debconf-set-selections
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install kexec-tools
cat <<'EOF' > /etc/systemd/system/kexec-pve.service
[Unit]
Description=boot into into the latest pve kernel set as primary in the boot-loader
Description=Loading new kernel into memory
Documentation=man:kexec(8)
DefaultDependencies=no
Before=shutdown.target umount.target final.target
Before=reboot.target
RequiresMountsFor=/boot
#Before=shutdown.target umount.target final.target
[Service]
Type=oneshot
ExecStart=/sbin/kexec -l /boot/pve/vmlinuz --initrd=/boot/pve/initrd.img --reuse-cmdline
RemainAfterExit=yes
ExecStart=/sbin/kexec -d -l /boot/pve/vmlinuz --initrd=/boot/pve/initrd.img --reuse-cmdline
[Install]
WantedBy=kexec.target
WantedBy=default.target
EOF
systemctl enable kexec-pve.service
echo "alias reboot-quick='systemctl kexec'" >> /root/.bash_profile
systemctl enable kexec-pve.service
echo "alias reboot-quick='systemctl kexec'" >> /root/.bash_profile
fi
## Remove no longer required packages and purge old cached updates
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoremove
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoclean
if [ "$XS_DISABLERPC" == "yes" ] ; then
## Disable portmapper / rpcbind (security)
systemctl disable rpcbind
systemctl stop rpcbind
fi
## Disable portmapper / rpcbind (security)
systemctl disable rpcbind
systemctl stop rpcbind
if [ "$XS_TIMEZONE" == "" ] ; then
## Set Timezone, empty = set automatically by ip
this_ip="$(dig +short myip.opendns.com @resolver1.opendns.com)"
timezone="$(curl "https://ipapi.co/${this_ip}/timezone")"
if [ "$timezone" != "" ] ; then
echo "Found $timezone for ${this_ip}"
timedatectl set-timezone "$timezone"
else
echo "WARNING: Timezone not found for ${this_ip}, set to UTC"
timedatectl set-timezone UTC
fi
else
## Set Timezone to XS_TIMEZONE
timedatectl set-timezone "$XS_TIMEZONE"
fi
## Set Timezone to UTC and enable NTP
timedatectl set-timezone UTC
cat <<EOF > /etc/systemd/timesyncd.conf
if [ "$XS_TIMESYNC" == "yes" ] ; then
## Automatically Synchronize the time
cat <<EOF > /etc/systemd/timesyncd.conf
[Time]
NTP=0.pool.ntp.org 1.pool.ntp.org 2.pool.ntp.org 3.pool.ntp.org
FallbackNTP=0.debian.pool.ntp.org 1.debian.pool.ntp.org 2.debian.pool.ntp.org 3.debian.pool.ntp.org
@ -163,32 +401,55 @@ RootDistanceMaxSec=5
PollIntervalMinSec=32
PollIntervalMaxSec=2048
EOF
service systemd-timesyncd start
timedatectl set-ntp true
service systemd-timesyncd start
timedatectl set-ntp true
fi
## Set pigz to replace gzip, 2x faster gzip compression
cat <<EOF > /bin/pigzwrapper
if [ "$XS_GUESTAGENT" == "yes" ] ; then
## Detect if is running in a virtual machine and install the relavant guest agent
if [ "$(dmidecode -s system-manufacturer | xargs)" == "QEMU" ] || [ "$(systemd-detect-virt | xargs)" == "kvm" ] ; then
echo "QEMU Detected, installing guest agent"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install qemu-guest-agent
elif [ "$(systemd-detect-virt | xargs)" == "vmware" ] ; then
echo "VMware Detected, installing vm-tools"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install open-vm-tools
elif [ "$(systemd-detect-virt | xargs)" == "oracle" ] ; then
echo "Virtualbox Detected, installing guest-utils"
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install virtualbox-guest-utils
fi
fi
if [ "$XS_PIGZ" == "yes" ] ; then
## Set pigz to replace gzip, 2x faster gzip compression
sed -i "s/#pigz:.*/pigz: 1/" /etc/vzdump.conf
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install pigz
cat <<EOF > /bin/pigzwrapper
#!/bin/sh
# eXtremeSHOK.com
PATH=/bin:\$PATH
GZIP="-1"
exec /usr/bin/pigz "\$@"
EOF
mv -f /bin/gzip /bin/gzip.original
cp -f /bin/pigzwrapper /bin/gzip
chmod +x /bin/pigzwrapper
chmod +x /bin/gzip
## Detect if this is an OVH server by getting the global IP and checking the ASN
if [ "$(whois -h v4.whois.cymru.com " -t $(curl ipinfo.io/ip 2> /dev/null)" | tail -n 1 | cut -d'|' -f3 | grep -i "ovh")" != "" ] ; then
echo "Deteted OVH Server, installing OVH RTM (real time monitoring)"
# http://help.ovh.co.uk/RealTimeMonitoring
# https://docs.ovh.com/gb/en/dedicated/install-rtm/
wget -qO - https://last-public-ovh-infra-yak.snap.mirrors.ovh.net/yak/archives/apply.sh | OVH_PUPPET_MANIFEST=distribyak/catalog/master/puppet/manifests/common/rtmv2.pp bash
mv -f /bin/gzip /bin/gzip.original
cp -f /bin/pigzwrapper /bin/gzip
chmod +x /bin/pigzwrapper
chmod +x /bin/gzip
fi
## Protect the web interface with fail2ban
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install fail2ban
# shellcheck disable=1117
if [ "$XS_OVHRTM" == "yes" ] ; then
## Detect if this is an OVH server by getting the global IP and checking the ASN, then install OVH RTM (real time monitoring)"
if [ "$(whois -h v4.whois.cymru.com " -t $(curl ipinfo.io/ip 2> /dev/null)" | tail -n 1 | cut -d'|' -f3 | grep -i "ovh")" != "" ] ; then
echo "Deteted OVH Server, installing OVH RTM (real time monitoring)"
# http://help.ovh.co.uk/RealTimeMonitoring
# https://docs.ovh.com/gb/en/dedicated/install-rtm/
wget -qO - https://last-public-ovh-infra-yak.snap.mirrors.ovh.net/yak/archives/apply.sh | OVH_PUPPET_MANIFEST=distribyak/catalog/master/puppet/manifests/common/rtmv2.pp bash
fi
fi
if [ "$XS_FAIL2BAN" == "yes" ] ; then
## Protect the web interface with fail2ban
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install fail2ban
# shellcheck disable=1117
cat <<EOF > /etc/fail2ban/filter.d/proxmox.conf
[Definition]
failregex = pvedaemon\[.*authentication failure; rhost=<HOST> user=.* msg=.*
@ -203,44 +464,36 @@ logpath = /var/log/daemon.log
maxretry = 3
# 1 hour
bantime = 3600
findtime = 600
EOF
cat <<EOF > /etc/fail2ban/jail.local
[DEFAULT]
banaction = iptables-ipset-proto4
EOF
systemctl enable fail2ban
##testing
#fail2ban-regex /var/log/daemon.log /etc/fail2ban/filter.d/proxmox.conf
systemctl enable fail2ban
##testing
#fail2ban-regex /var/log/daemon.log /etc/fail2ban/filter.d/proxmox.conf
fi
## Increase vzdump backup speed, enable pigz and fix ionice
sed -i "s/#bwlimit:.*/bwlimit: 0/" /etc/vzdump.conf
sed -i "s/#pigz:.*/pigz: 1/" /etc/vzdump.conf
sed -i "s/#ionice:.*/ionice: 5/" /etc/vzdump.conf
## Bugfix: pve 5.1 high swap usage with low memory usage
echo "vm.swappiness=10" >> /etc/sysctl.conf
sysctl -p
## Bugfix: reserve 512MB memory for system
echo "vm.min_free_kbytes = 524288" >> /etc/sysctl.conf
sysctl -p
## Remove subscription banner
if [ -f "/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js" ] ; then
sed -i "s/data.status !== 'Active'/false/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
sed -i "s/checked_command: function(orig_cmd) {/checked_command: function() {} || function(orig_cmd) {/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
# create a daily cron to make sure the banner does not re-appear
cat <<'EOF' > /etc/cron.daily/proxmox-nosub
if [ "$XS_NOSUBBANNER" == "yes" ] ; then
## Remove subscription banner
if [ -f "/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js" ] ; then
# create a daily cron to make sure the banner does not re-appear
cat <<'EOF' > /etc/cron.daily/xs-pve-nosub
#!/bin/sh
# eXtremeSHOK.com Remove subscription banner
sed -i "s/data.status !== 'Active'/false/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
sed -i "s/checked_command: function(orig_cmd) {/checked_command: function() {} || function(orig_cmd) {/g" /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
EOF
chmod 755 /etc/cron.daily/proxmox-nosub
chmod 755 /etc/cron.daily/xs-pve-nosub
bash /etc/cron.daily/xs-pve-nosub
fi
# Remove nag @tinof
echo "DPkg::Post-Invoke { \"dpkg -V proxmox-widget-toolkit | grep -q '/proxmoxlib\.js$'; if [ \$? -eq 1 ]; then { echo 'Removing subscription nag from UI...'; sed -i '/data.status/{s/\!//;s/Active/NoMoreNagging/}' /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js; }; fi\"; };" > /etc/apt/apt.conf.d/xs-pve-no-nag && apt --reinstall install proxmox-widget-toolkit
fi
if [ "$XS_MOTD" == "yes" ] ; then
## Pretty MOTD BANNER
if [ -z "${NO_MOTD_BANNER}" ] ; then
if ! grep -q https "/etc/motd" ; then
cat << 'EOF' > /etc/motd.new
This system is optimised by: https://eXtremeSHOK.com
@ -259,15 +512,20 @@ EOF
fi
fi
## Increase max user watches
# BUG FIX : No space left on device
echo 1048576 > /proc/sys/fs/inotify/max_user_watches
echo "fs.inotify.max_user_watches=1048576" >> /etc/sysctl.conf
sysctl -p /etc/sysctl.conf
## Increase max FD limit / ulimit
cat <<EOF >> /etc/security/limits.conf
# eXtremeSHOK.com Increase max FD limit / ulimit
if [ "$XS_LIMITS" == "yes" ] ; then
## Increase max user watches
# BUG FIX : No space left on device
cat <<EOF > /etc/sysctl.d/99-xs-maxwatches.conf
# eXtremeSHOK.com
# Increase max user watches
fs.inotify.max_user_watches=1048576
fs.inotify.max_user_instances=1048576
fs.inotify.max_queued_events=1048576
EOF
## Increase max FD limit / ulimit
cat <<EOF >> /etc/security/limits.d/99-xs-limits.conf
# eXtremeSHOK.com
# Increase max FD limit / ulimit
* soft nproc 256000
* hard nproc 256000
* soft nofile 256000
@ -277,52 +535,236 @@ root hard nproc 256000
root soft nofile 256000
root hard nofile 256000
EOF
## Enable TCP BBR congestion control
cat <<EOF > /etc/sysctl.d/10-kernel-bbr.conf
# eXtremeSHOK.com
# TCP BBR congestion control
net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr
EOF
## Increase kernel max Key limit
cat <<EOF > /etc/sysctl.d/60-maxkeys.conf
## Increase kernel max Key limit
cat <<EOF > /etc/sysctl.d/99-xs-maxkeys.conf
# eXtremeSHOK.com
# Increase kernel max Key limit
kernel.keys.root_maxkeys=1000000
kernel.keys.maxkeys=1000000
EOF
## Set systemd ulimits
echo "DefaultLimitNOFILE=256000" >> /etc/systemd/system.conf
echo "DefaultLimitNOFILE=256000" >> /etc/systemd/user.conf
## Set systemd ulimits
echo "DefaultLimitNOFILE=256000" >> /etc/systemd/system.conf
echo "DefaultLimitNOFILE=256000" >> /etc/systemd/user.conf
echo 'session required pam_limits.so' | tee -a /etc/pam.d/common-session-noninteractive
echo 'session required pam_limits.so' | tee -a /etc/pam.d/common-session
echo 'session required pam_limits.so' | tee -a /etc/pam.d/runuser-l
echo 'session required pam_limits.so' >> /etc/pam.d/common-session
echo 'session required pam_limits.so' >> /etc/pam.d/runuser-l
## Set ulimit for the shell user
cd ~ && echo "ulimit -n 256000" >> .bashrc ; echo "ulimit -n 256000" >> .profile
## Set ulimit for the shell user
echo "ulimit -n 256000" >> /root/.profile
fi
## Optimise ZFS arc size
if [ "$(command -v zfs)" != "" ] ; then
RAM_SIZE_GB=$(( $(vmstat -s | grep -i "total memory" | xargs | cut -d" " -f 1) / 1024 / 1000))
if [[ RAM_SIZE_GB -lt 16 ]] ; then
# 1GB/1GB
MY_ZFS_ARC_MIN=1073741824
MY_ZFS_ARC_MAX=1073741824
else
MY_ZFS_ARC_MIN=$((RAM_SIZE_GB * 1073741824 / 16))
MY_ZFS_ARC_MAX=$((RAM_SIZE_GB * 1073741824 / 8))
fi
# Enforce the minimum, incase of a faulty vmstat
if [[ MY_ZFS_ARC_MIN -lt 1073741824 ]] ; then
MY_ZFS_ARC_MIN=1073741824
fi
if [[ MY_ZFS_ARC_MAX -lt 1073741824 ]] ; then
MY_ZFS_ARC_MAX=1073741824
fi
cat <<EOF > /etc/modprobe.d/zfs.conf
if [ "$XS_LOGROTATE" == "yes" ] ; then
## Optimise logrotate
cat <<EOF > /etc/logrotate.conf
# eXtremeSHOK.com
daily
su root adm
rotate 7
create
compress
size=10M
delaycompress
copytruncate
include /etc/logrotate.d
EOF
systemctl restart logrotate
fi
if [ "$XS_JOURNALD" == "yes" ] ; then
## Limit the size and optimise journald
cat <<EOF > /etc/systemd/journald.conf
# eXtremeSHOK.com
[Journal]
# Store on disk
Storage=persistent
# Don't split Journald logs by user
SplitMode=none
# Disable rate limits
RateLimitInterval=0
RateLimitIntervalSec=0
RateLimitBurst=0
# Disable Journald forwarding to syslog
ForwardToSyslog=no
# Journald forwarding to wall /var/log/kern.log
ForwardToWall=yes
# Disable signing of the logs, save cpu resources.
Seal=no
Compress=yes
# Fix the log size
SystemMaxUse=64M
RuntimeMaxUse=60M
# Optimise the logging and speed up tasks
MaxLevelStore=warning
MaxLevelSyslog=warning
MaxLevelKMsg=warning
MaxLevelConsole=notice
MaxLevelWall=crit
EOF
systemctl restart systemd-journald.service
journalctl --vacuum-size=64M --vacuum-time=1d;
journalctl --rotate
fi
if [ "$XS_ENTROPY" == "yes" ] ; then
## Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' install haveged
## Net optimising
cat <<EOF > /etc/default/haveged
# eXtremeSHOK.com
# -w sets low entropy watermark (in bits)
DAEMON_ARGS="-w 1024"
EOF
systemctl daemon-reload
systemctl enable haveged
fi
if [ "$XS_VZDUMP" == "yes" ] ; then
## Increase vzdump backup speed
sed -i "s/#bwlimit:.*/bwlimit: 0/" /etc/vzdump.conf
sed -i "s/#ionice:.*/ionice: 5/" /etc/vzdump.conf
fi
if [ "$XS_MEMORYFIXES" == "yes" ] ; then
## Optimise Memory
cat <<EOF > /etc/sysctl.d/99-xs-memory.conf
# eXtremeSHOK.com
# Memory Optimising
## Bugfix: reserve 512MB memory for system
vm.min_free_kbytes=524288
vm.nr_hugepages=72
# (Redis/MongoDB)
vm.max_map_count=262144
vm.overcommit_memory = 1
EOF
fi
if [ "$XS_TCPBBR" == "yes" ] ; then
## Enable TCP BBR congestion control
cat <<EOF > /etc/sysctl.d/99-xs-kernel-bbr.conf
# eXtremeSHOK.com
# TCP BBR congestion control
net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr
EOF
fi
if [ "$XS_TCPFASTOPEN" == "yes" ] ; then
## Enable TCP fastopen
cat <<EOF > /etc/sysctl.d/99-xs-tcp-fastopen.conf
# eXtremeSHOK.com
# TCP fastopen
net.ipv4.tcp_fastopen=3
EOF
fi
if [ "$XS_NET" == "yes" ] ; then
## Enable Network optimising
cat <<EOF > /etc/sysctl.d/99-xs-net.conf
# eXtremeSHOK.com
net.core.netdev_max_backlog=8192
net.core.optmem_max=8192
net.core.rmem_max=16777216
net.core.somaxconn=8151
net.core.wmem_max=16777216
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.log_martians = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.secure_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.ip_local_port_range=1024 65535
net.ipv4.tcp_base_mss = 1024
net.ipv4.tcp_challenge_ack_limit = 999999999
net.ipv4.tcp_fin_timeout=10
net.ipv4.tcp_keepalive_intvl=30
net.ipv4.tcp_keepalive_probes=3
net.ipv4.tcp_keepalive_time=240
net.ipv4.tcp_limit_output_bytes=65536
net.ipv4.tcp_max_syn_backlog=8192
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_mtu_probing = 1
net.ipv4.tcp_rfc1337=1
net.ipv4.tcp_rmem=8192 87380 16777216
net.ipv4.tcp_sack=1
net.ipv4.tcp_slow_start_after_idle=0
net.ipv4.tcp_syn_retries=3
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 0
net.ipv4.tcp_wmem=8192 65536 16777216
net.netfilter.nf_conntrack_generic_timeout = 60
net.netfilter.nf_conntrack_helper=0
net.netfilter.nf_conntrack_max = 524288
net.netfilter.nf_conntrack_tcp_timeout_established = 28800
net.unix.max_dgram_qlen = 4096
EOF
fi
if [ "$XS_SWAPPINESS" == "yes" ] ; then
## Bugfix: high swap usage with low memory usage
cat <<EOF > /etc/sysctl.d/99-xs-swap.conf
# eXtremeSHOK.com
# Bugfix: high swap usage with low memory usage
vm.swappiness=10
EOF
fi
if [ "$XS_MAXFS" == "yes" ] ; then
## Increase Max FS open files
cat <<EOF > /etc/sysctl.d/99-xs-fs.conf
# eXtremeSHOK.com
# Max FS Optimising
fs.nr_open=12000000
fs.file-max=9000000
EOF
fi
if [ "$XS_BASHRC" == "yes" ] ; then
## Customise bashrc (thanks broeckca)
cat <<EOF >> /root/.bashrc
export HISTTIMEFORMAT="%d/%m/%y %T "
export PS1='\u@\h:\W \$ '
alias l='ls -CF'
alias la='ls -A'
alias ll='ls -alF'
alias ls='ls --color=auto'
source /etc/profile.d/bash_completion.sh
export PS1="\[\e[31m\][\[\e[m\]\[\e[38;5;172m\]\u\[\e[m\]@\[\e[38;5;153m\]\h\[\e[m\] \[\e[38;5;214m\]\W\[\e[m\]\[\e[31m\]]\[\e[m\]\\$ "
EOF
echo "source /root/.bashrc" >> /root/.bash_profile
fi
if [ "$XS_ZFSARC" == "yes" ] ; then
## Optimise ZFS arc size accoring to memory size
if [ "$(command -v zfs)" != "" ] ; then
if [[ RAM_SIZE_GB -le 16 ]] ; then
MY_ZFS_ARC_MIN=536870912
MY_ZFS_ARC_MAX=536870912
elif [[ RAM_SIZE_GB -le 32 ]] ; then
# 1GB/1GB
MY_ZFS_ARC_MIN=1073741824
MY_ZFS_ARC_MAX=1073741824
else
MY_ZFS_ARC_MIN=$((RAM_SIZE_GB * 1073741824 / 16))
MY_ZFS_ARC_MAX=$((RAM_SIZE_GB * 1073741824 / 8))
fi
# Enforce the minimum, incase of a faulty vmstat
if [[ MY_ZFS_ARC_MIN -lt 536870912 ]] ; then
MY_ZFS_ARC_MIN=536870912
fi
if [[ MY_ZFS_ARC_MAX -lt 536870912 ]] ; then
MY_ZFS_ARC_MAX=536870912
fi
cat <<EOF > /etc/modprobe.d/99-xs-zfsarc.conf
# eXtremeSHOK.com ZFS tuning
# Use 1/16 RAM for MAX cache, 1/8 RAM for MIN cache, or 1GB
@ -338,10 +780,20 @@ options zfs l2arc_noprefetch=0
# setting here : 500 * 1024 * 1024
options zfs l2arc_write_max=524288000
EOF
fi
fi
# propagate the setting into the kernel
update-initramfs -u -k all
# cleanup
## Remove no longer required packages and purge old cached updates
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoremove
/usr/bin/env DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::='--force-confdef' autoclean
echo "# eXtremeSHOK.com" > /etc/extremeshok
date >> /etc/extremeshok
## Script Finish
echo -e '\033[1;33m Finished....please restart the system \033[0m'
echo "Optimisations by https://eXtremeSHOK.com"

0
networking/README.md Normal file
View File

View File

@ -17,7 +17,7 @@
# NOTE: WILL APPLY CHANGES TO /etc/network/interfaces
#
# Usage:
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/network-addiprange.sh && chmod +x network-addiprange.sh
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-addiprange.sh && chmod +x network-addiprange.sh
# ./network-addiprange.sh ip.xx.xx.xx/cidr interface_optional
# ./network-addiprange.sh ip.xx.xx.xx / cidr interface_optional
# ./network-addiprange.sh ip.xx.xx.xx interface_optional

View File

@ -53,7 +53,7 @@ fi
if ! [ -f "network-addiprange.sh" ]; then
echo "Downloading network-addiprange.sh script"
curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/network-addiprange.sh && chmod +x network-addiprange.sh
curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-addiprange.sh && chmod +x network-addiprange.sh
fi
if ! grep -q '#!/usr/bin/env bash' "network-addiprange.sh"; then
echo "ERROR: network-addiprange.sh is invalid"

View File

@ -12,7 +12,7 @@
# License: BSD (Berkeley Software Distribution)
#
# Usage:
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/tincvpn.sh && chmod +x tincvpn.sh
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/tincvpn.sh && chmod +x tincvpn.sh
# ./tincvpn.sh -h
#
# Example for 3 node Cluster
@ -40,8 +40,8 @@
vpn_ip_last=1
vpn_connect_to=""
vpn_port=655
#my_default_v4ip=$(ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '192.168.' | grep -v '10.0.' | grep -v '10.10.' | grep -v '127.0.0.' | tail -n 1)
my_default_v4ip=""
my_default_v4ip=$(ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '192.168.' | grep -v '10.0.' | grep -v '10.10.' | grep -v '127.0.0.' | tail -n 1)
#my_default_v4ip=""
reset="no"

14
nvidia/nvidia-docker.sh Normal file
View File

@ -0,0 +1,14 @@
#!/bin/bash
# thanks @88plug
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
tee /etc/apt/sources.list.d/nvidia-docker.list
apt-get update
# Install nvidia-docker2 and reload the Docker daemon configuration
apt-get install -y nvidia-docker2
pkill -SIGHUP dockerd
reboot

69
nvidia/readme.md Normal file
View File

@ -0,0 +1,69 @@
# initial work by @88plug
xserver-xorg-dev dkms
#!/bin/bash
apt-get install build-essential pve-headers-$(uname -r) pkg-config libgtk-3-0 libglvnd-dev
update-grub
REBOOT
#Old Driver
#wget https://us.download.nvidia.com/XFree86/Linux-x86_64/455.38/NVIDIA-Linux-x86_64-455.38.run
#chmod +x NVIDIA-Linux-x86_64-455.38.run
#./NVIDIA-Linux-x86_64-455.38.run
wget https://us.download.nvidia.com/XFree86/Linux-x86_64/460.56/NVIDIA-Linux-x86_64-460.56.run
chmod +x NVIDIA-Linux-x86_64-460.56.run
./NVIDIA-Linux-x86_64-460.56.run
Installer will ask to create modeprobe file, say YES!
Reboot
Run ./NVIDIA-Linux-x86_64-455.38.run again
WARNING: nvidia-installer was forced to guess the X library path '/usr/lib' and X module path '/usr/lib/xorg/modules'; these paths were not queryable from the system. If X fails to find the NVIDIA X driver module, please
install the `pkg-config` utility and the X.Org SDK/development package for your distribution and reinstall the driver
YES to 32 bit dependencies
Would you like to run the nvidia-xconfig utility to automatically update your X configuration file so that the NVIDIA X driver will be used when you restart X? Any pre-existing X configuration file will be backed up.
NO
REBOOT
nvidia-smi!
Now run ./docker.sh to install nvidia-docker!
Unlock card with
sudo nvidia-xconfig -a --cool-bits=31 --allow-empty-initial-configuration
nvidia-smi -pl 200 -i 0
Now if you want to overclock you need a Xauthority/gdm
tasksel > install gnome desktop
then run this command
systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target
as gnome stupidly enables sleep by default!
finally reboot...you should now have a folder
/run/user/$SOMENUMBER/gdm/Xauthority
replace $SOMENUBMER in lines below! :)
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:0]/GPUFanControlState=1 -a [fan-0]/GPUTargetFanSpeed=80
sleep 3
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:1]/GPUFanControlState=1 -a [fan-1]/GPUTargetFanSpeed=80
sleep 3
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:2]/GPUFanControlState=1 -a [fan-2]/GPUTargetFanSpeed=80
sleep 3
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority sudo nvidia-settings -a [gpu:3]/GPUFanControlState=1 -a [fan-3]/GPUTargetFanSpeed=85
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:0]/GPUGraphicsClockOffset[3]=150'
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:0]/GPUMemoryTransferRateOffset[3]=600'
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:1]/GPUGraphicsClockOffset[3]=150'
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:1]/GPUMemoryTransferRateOffset[3]=600'
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:2]/GPUGraphicsClockOffset[3]=150'
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:2]/GPUMemoryTransferRateOffset[3]=600'
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:3]/GPUGraphicsClockOffset[3]=150'
DISPLAY=:0 XAUTHORITY=/run/user/121/gdm/Xauthority nvidia-settings -a '[gpu:3]/GPUMemoryTransferRateOffset[3]=600'

58
ovh/README.md Normal file
View File

@ -0,0 +1,58 @@
# OVH Proxmox Installation Guide #
Select install for the specific server, via the ovh manager
* --INSTALL-->
* Install from an OVH template
* --NEXT-->
* Type of OS: Ready-to-go (graphical user interface)
* VPS Proxmox VE *(pick the latest non zfs version)*
* Language: EN
* Target disk arrray: *(always select the SSD array if you have ssd and hdd arrays)*
* Enable/Tick: Customise the partition configuration
* --NEXT-->
* Disks used for this installation: *(All of them)*
* (Remove all the partitions and do the following)
* Type: Filesystem: Mount Point: LVM Name: RAID: Size:
* * 1 primary Ext4 / - 1 20.0 GB
* * 2 primary Swap swap - - 2 x 8.0 GB *(minimum 16GB total, set recommended swap size)*
* * 3 LV xfs /var/lib/vz data 1 REMAINING GB *(use all the remaining space)*
* --NEXT-->
* Hostname: server.fqdn.com
* Installation script (URL): https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/install-post.sh
* Script return value: 0
* SSH keys: *(always suggested, however if this value is used a webinterface login will not work without setting a root password in shell)*
* --CONFIRM-->
After installation, Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## LVM to ZFS
````
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/lvm-2-zfs.sh -c -O lvm-2-zfs.sh && chmod +x lvm-2-zfs.sh
./lvm-2-zfs.sh && rm lvm-2-zfs.sh
````
* Reboot
* Connect via ssh/terminal to the new Proxmox system running on your server and run the following
## NETWORKING (vmbr0 vmbr1)
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/networking/network-configure.sh -c -O network-configure.sh && chmod +x network-configure.sh
./network-configure.sh && rm network-configure.sh
```
* Reboot
* Post Install: Now login via ssh as root and create a password, which will be used for the webinterface when logging in with pam authentication
# Advance Installation Options #
Assumptions: Proxmox installed, SSD raid1 partitions mounted as /xshok/zfs-slog and /xshok/zfs-cache, 1+ unused hdd which will be made into a zfspool
* Connect via ssh/terminal to the new Proxmox system running on your server and run the follow
## Create ZFS from unused devices (createzfs.sh)
**NOTE: WILL DESTROY ALL DATA ON SPECIFIED DEVICES**
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/createzfs.sh -c -O createzfs.sh && chmod +x createzfs.sh
./createzfs.sh poolname /dev/device1 /dev/device2
```
## Create ZFS cache and slog from /xshok/zfs-cache and /xshok/zfs-slog partitions and adds them to a zpool (xshok_slog_cache-2-zfs.sh) *optional*
**NOTE: WILL DESTROY ALL DATA ON SPECIFIED PARTITIONS**
```
wget https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/xshok_slog_cache-2-zfs.sh -c -O xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh
./xshok_slog_cache-2-zfs.sh poolname
```
* Reboot

97
xs.env.sample Normal file
View File

@ -0,0 +1,97 @@
#!/usr/bin/env bash
################################################################################
# This is property of eXtremeSHOK.com
# You are free to use, modify and distribute, however you may not remove this notice.
# Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
################################################################################
#
# Script updates can be found at: https://github.com/extremeshok/xshok-proxmox
#
################################################################################
#
# User Defined Options for (install-post.sh) post-installation script for Proxmox
#
# Usage: place in the same folder as install-post.sh and rename to xs-install-post.env
#
#### VARIABLES
#
# yes = enabled
# no = disabled
#
#### VARIABLES / options
# Detect AMD EPYC CPU and Apply Fixes
XS_AMDFIXES="yes"
# Force APT to use IPv4
XS_APTIPV4="yes"
# Update proxmox and install various system utils
XS_APTUPGRADE="yes"
# Customise bashrc
XS_BASHRC="yes"
# Add the latest ceph provided by proxmox
XS_CEPH="yes"
# Disable portmapper / rpcbind (security)
XS_DISABLERPC="yes"
# Ensure Entropy Pools are Populated, prevents slowdowns whilst waiting for entropy
XS_ENTROPY="yes"
# Protect the web interface with fail2ban
XS_FAIL2BAN="yes"
# Detect if is running in a virtual machine and install the relavant guest agent
XS_GUESTAGENT="yes"
# Install ifupdown2 for a virtual internal network allows rebootless networking changes (not compatible with openvswitch-switch)
XS_IFUPDOWN2="yes"
# Limit the size and optimise journald
XS_JOURNALD="yes"
# Install kernel source headers
XS_KERNELHEADERS="yes"
# Install kexec, allows for quick reboots into the latest updated kernel set as primary in the boot-loader.
XS_KEXEC="yes"
# Ensure ksmtuned (ksm-control-daemon) is enabled and optimise according to ram size
XS_KSMTUNED="yes"
# Set language, if chnaged will disable XS_NOAPTLANG
XS_LANG="en_US.UTF-8"
# Increase max user watches, FD limit, FD ulimit, max key limit, ulimits
XS_LIMITS="yes"
# Optimise logrotate
XS_LOGROTATE="yes"
# Lynis security scan tool by Cisofy
XS_LYNIS="yes"
# Increase Max FS open files
XS_MAXFS="yes"
# Optimise Memory
XS_MEMORYFIXES="yes"
# Pretty MOTD BANNER
XS_MOTD="yes"
# Enable Network optimising
XS_NET="yes"
# Save bandwidth and skip downloading additional languages, requires XS_LANG="en_US.UTF-8"
XS_NOAPTLANG="yes"
# Disable enterprise proxmox repo
XS_NOENTREPO="yes"
# Remove subscription banner
XS_NOSUBBANNER="yes"
# Install openvswitch for a virtual internal network
XS_OPENVSWITCH="no"
# Detect if this is an OVH server and install OVH Real Time Monitoring
XS_OVHRTM="yes"
# Set pigz to replace gzip, 2x faster gzip compression
XS_PIGZ="yes"
# Bugfix: high swap usage with low memory usage
XS_SWAPPINESS="yes"
# Enable TCP BBR congestion control
XS_TCPBBR="yes"
# Enable TCP fastopen
XS_TCPFASTOPEN="yes"
# Enable testing proxmox repo
XS_TESTREPO="no"
# Automatically Synchronize the time
XS_TIMESYNC="yes"
# Set Timezone, empty = set automatically by IP
XS_TIMEZONE=""
# Install common system utilities
XS_UTILS="yes"
# Increase vzdump backup speed
XS_VZDUMP="yes"
# Optimise ZFS arc size accoring to memory size
XS_ZFSARC="yes"
# Install zfs-auto-snapshot
XS_ZFSAUTOSNAPSHOT="yes"

0
zfs/README.md Normal file
View File

View File

@ -1,5 +1,16 @@
#!/usr/bin/env bash
# https://blog.programster.org/zfs-add-intent-log-device
################################################################################
# This is property of eXtremeSHOK.com
# You are free to use, modify and distribute, however you may not remove this notice.
# Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
################################################################################
#
# Script updates can be found at: https://github.com/extremeshok/xshok-proxmox
# Based on https://blog.programster.org/zfs-add-intent-log-device
#
# License: BSD (Berkeley Software Distribution)
#
################################################################################
# Set the local
export LANG="en_US.UTF-8"

View File

@ -35,7 +35,7 @@
# NOTE: WILL DESTROY ALL DATA ON DEVICES SPECIFED
#
# Usage:
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/createzfs.sh && chmod +x createzfs.sh
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/createzfs.sh && chmod +x createzfs.sh
# ./createzfs.sh poolname /dev/sda /dev/sdb
#
################################################################################

View File

@ -18,7 +18,7 @@
# Assumes mounted MD raid partitions (linux software raid)
#
# Usage:
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh
# curl -O https://raw.githubusercontent.com/extremeshok/xshok-proxmox/master/zfs/xshok_slog_cache-2-zfs.sh && chmod +x xshok_slog_cache-2-zfs.sh
# ./xshok_slog_cache-2-zfs.sh MY_ZFS_POOL
#
# NOTES: remove slog with