Cheatsheet – TheTechLead

Users and Groups
SSH
Firewall
Run scripts when start up
Monit

Users & Groups

Create User with password
useradd -m <username>
passwd <username>

Create a Group
groupadd <group name>

Add user to group
usermod -aG <group name> <username>

Remove user from group
deluser <username> <group name>

Set ACL to allow a user to read folders
setfacl -m u:<username>:rwx,d:u:<username>:r <folder path>

SSH

Connecting
ssh <username>@<host IP or domain>
ssh -i <path to id_rsa file> <username>@<host IP or domain>

Generate SSH key
ssh-keygen

Add SSH public key to remote server
Manually paste public keys to: ~/.ssh/authorized_keys
Or: ssh-copy-id <username>@<ssh_host>
Note: Before ssh-copy-id, remote server must already create the underlying user. ssh-copy-id will prompt for password to login

Download files/folder via SSH
scp [-r] <username>@<remote server>:<path on remote server> <path on local>

Upload files via SSH
scp [-r] <path on local> <username>@<remote server>:<path on remote server>

Configure SSH timeout

vi /etc/ssh/sshd_config

# Hit "i" for INSERT mode on vi, edit below line
ClientAliveInterval  1200 # 1200 seconds

# Hit Esc to escape INSERT mode, type ":x" to save file
# Restart sshd
sudo systemctl reload sshd

Firewall

List all Rules of all Chains:
iptables -n -L -v --line-numbers

List all Rules of a specific Chain
iptables -L INPUT --line-numbers

Delete a Rule in a Chain at a line number
iptables -D INPUT 10

Allow Incoming Traffic , Insert Rule add specific line
iptables -I INPUT <line_number> -p tcp --dport 80 -s <source_ip> -j ACCEPT

Allow Outgoing Traffic, Append Rule add end of a Chain
iptables -A OUTPUT -d <destination_ip> --sport <source port> -j ACCEPT

[NAT] Allow LAN nodes to access public network via interface eth0
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

[NAT] Redirect Incoming traffic to internal node
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to 172.31.0.23:80

Run a script when startup

sudo vim /etc/rc.local

Edit the rc.local file with your desired commands like below :

#!/bin/sh
# add your commands here
# last line must be exit 0 
exit 0

Then activate it by:

sudo chmod -v +x /etc/rc.local
sudo systemctl enable rc-local.service

Monit

Let the server notify you when something goes wrong !

Origin: https://mmonit.com/monit/documentation/monit.html

Install
apt-get install monit -y

Start as a daemon once per n seconds
monit -d 30

Configuration file
~/.monitrc or /etc/monitrc

Specify configuration file :
monit -c <path to cf file>

Configuration file sample content

Open Httpd for Dashboard

set httpd port 2812 allow username:password
# with IP
set httpd
     port 2812
     use address 127.0.0.1
     allow username:password
# using htpasswd file with limited username
set httpd port 2812
      allow md5 /etc/httpd/htpasswd john paul ringo george

Configure Daemon

SET DAEMON <seconds>

Setup Alert methods via Email

set alert dev@yourcompanny.com
set mail-format {
      from: Monit Support <monit@foo.bar>
  reply-to: support@domain.com
   subject: $SERVICE $EVENT at $DATE
   message: Monit $ACTION $SERVICE at $DATE on $HOST: $DESCRIPTION.
            Yours sincerely,
            monit
 }
SET MAILSERVER
        <hostname|ip-address>
        [PORT number]
        [USERNAME string] [PASSWORD string]
        [using SSL [with options {...}]
        [CERTIFICATE CHECKSUM [MD5|SHA1] <hash>],

Setup Alert via Slack Webhook

Go to https://<yourteam>.slack.com/apps/manage/custom-integrations
Click Incoming WebHooks
Click Add Configuration
Select an existing channel or create a new one (e.g. #monit) – you can change it later
Click Add Incoming WebHooks integration
Copy the Webhook URL
Create file slack_notification.sh : touch /etc/slack_notification.sh .
Sample script :

#!/bin/bash
SERVER_NAME=$1
SERVICE=$2
STATUS=$3
TEST_WHEN="`date +%Y-%m-%d_%H:%M:%S`"
PRETEXT="$SERVER_NAME | $TEST_WHEN"
TEXT="$SERVICE - $STATUS"
SLACK_HOOK="Paste your Copied Slack web hook here"

curl -X POST --data-urlencode "payload={\"text\": \"$TEXT\"}" $SLACK_HOOK

Use in Configuration file

check program check-mysql ...
     if status != 0 then exec "/etc/slack_notification.sh ServerName ServiceName OK"

Monitor ports with Alert via Slack Hook

check host ServerA with address localhost
 if failed port 5433 protocol pgsql with timeout 30 seconds
  then exec "/etc/slack_notification.sh ServerA Postgresl FAIL"
   else if succeed exec "/etc/slack_notification.sh ServerA Postgresl OK"

Monitor process with Alert via Email

check process mysqld with pidfile /var/run/mysqld.pid
   if failed port 3306 protocol mysql then alert

Check remote host alive

check host Hostname with address www.yourremotehost.com
       if failed ping then alert

Check Disk amount usage

 check filesystem rootfs with path /
       if space usage > 90% then alert

Check Inode usage

 check filesystem rootfs with path /
       if inode usage > 90% then alert

Check CPU, Memory usage

check system $HOST
    if loadavg (5min) > 3 then alert
    if loadavg (15min) > 1 then alert
    if memory usage > 80% for 4 cycles then alert
    if swap usage > 20% for 4 cycles then alert
    # Test the user part of CPU usage 
    if cpu usage (user) > 80% for 2 cycles then alert
    # Test the system part of CPU usage 
    if cpu usage (system) > 20% for 2 cycles then alert
    # Test the i/o wait part of CPU usage 
    if cpu usage (wait) > 80% for 2 cycles then alert
    # Test CPU usage including user, system and wait. Note that 
    # multi-core systems can generate 100% per core
    # so total CPU usage can be more than 100%
    if cpu usage > 200% for 4 cycles then alert

Tag: Cheatsheet

Server Setup Cheatsheet