Linux Administration

How to Install SVN on Ubuntu

http://www.howtogeek.com/howto/ubuntu/install-subversion-with-web-access-on-ubuntu/
http://forums.techarena.in/guides-tutorials/1209507.htm

clean up linux disk space

sudo journalctl --vacuum-time=2d
sudo apt-get autoremove --purge

tar and gzip

Archive a set of files:

tar -cvf tarfile.tar /var/log/syslog /var/log/messages

Archive and compress (gzip) a set of files:

tar -cvzf file.tar.gz /var/log/syslog /var/log/messages

Archive and compress (bzip2) a set of files:

tar -cvjf file.tar.bz2 /var/log/syslog /var/log/messages

Extract a tar file:

tar -xvf file.tar
tar -xvzf file.tar.gz
tar -xvjf file.tar.bz2

Display the content of a tar file:

tar -tvf file.tar
tar -tvzf file.tar.gz
tar -tvjf file.tar.bz2

Replace a file in an existing tar file:

tar -rvf tarfile.tar filetoreplace

Update a file in an existing tar file:

tar -uvf tarfile.tar newfile

Copy all files in one directory to another directory on local host:

cd /etc; tar cf – . | (cd /etc.bak; tar xvpf -)

zip with split and password

interactive

zip --encrypt -s 4g -r file.zip files

Or

// recursive zip all folder
// with password
// split 4gb
zip --password password -s 4g -r output.zip input &
// with store only and 1gb split
zip -Z store -r -s 1g output.zip input/

no hangup even if logged out of ssh

nohup bash script.sh &> script.log &

DB backup

1. mysqldump -u root -p [root_password] [tabdaase_name] > dumpfilename.sql
   
2. mysqldump -u root -p password sugarcrm > sugarcrm.sql
   

//multiple

1. mysqldump -u root -ptmppassword --databases bugs sugarcrm > bugs_sugarcrm.sql
   

//restore
mysql -u root -p[root_password] [database_name] < dumpfilename.sql

Cron job backup mysql databse every week

# Back up db. Run once a week at midnight on Sunday morning
0 0 * * 0 mysqldump -u root -pPASSWORD DB_NAME | gzip > /tmp/wiki_database_`date +'%Y-%m-%d'`.sql.gz

Search specific text among files

grep -H "textToFind" /var/www/*
grep -H "textToFind" /var/www/sites/*/html/.htaccess

Search file

find /home/david -name 'index*'

SVN cheat sheet

http://www.abbeyworkshop.com/howto/misc/svn01/
Set up svn in Ubuntu
http://www.howtogeek.com/howto/ubuntu/install-subversion-with-web-access-on-ubuntu/

Apache Rewrite

CodeIgniter Rewrite Rules

<IfModule mod_rewrite.c>
	#Disable file listing
	Options -Indexes
    RewriteEngine On
    RewriteBase /
	#force www
	RewriteCond %{HTTP_HOST} ^hawkguide\.com [NC] 
	RewriteRule ^(.*) http://www.hawkguide.com/$1 [R=301,QSA,L] 
    #Removes access to the system folder by users.
    #Additionally this will allow you to create a System.php controller,
    #previously this would not have been possible.
    #'system' can be replaced if you have renamed your system folder.
    RewriteCond %{REQUEST_URI} ^system.*
    RewriteRule ^(.*)$ /index.php?/$1 [L]
    
    #When your application folder isn't in the system folder
    #This snippet prevents user access to the application folder
    #Submitted by: Fabdrol
    #Rename 'application' to your applications folder name.
    RewriteCond %{REQUEST_URI} ^application.*
    RewriteRule ^(.*)$ /index.php?/$1 [L]

    #Checks to see if the user is attempting to access a valid file,
    #such as an image or css document, if this isn't true it sends the
    #request to index.php
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule ^(.*)$ index.php?/$1 [L]
</IfModule>

Create User

adduser newuser

Add a existing user to a group usermod -a -G www-data jerry

ImageMagick merge images

montage f*.jpg -tile 1x22 -geometry 1576x930+0+0 tile_.jpg

listen ports

 netstat -a | egrep 'Proto|LISTEN'

Disable prompt when ssh to unknown host

echo -e "StrictHostKeyChecking no\n" >> ~/.ssh/config

Apache server log permission denied

In CentOS
Permission denied: [client xx:xx] AH00132: file permissions deny server access: /var/www/ Run

restorecon -R /var/www/foldername

squid 3.3 on centos

Configure SSL Bump on squid 3.3

List disk usage by directory

du -h <dir> | grep '[0-9\.]\+G'

Ubuntu 16.4 create swap

sudo fallocate -l 1G /swapfile
ls -lh /swapfile
sudo chmod 600 /swapfile
ls -lh /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo swapon --show

to auto mount

echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-16-04

How to auto run scripts at system start in Cent OS 7

/etc/rc.local or /etc/rc.d/rc.local are no longer executed by default due to systemd-changes. to still use those, you need to make /etc/rc.d/rc.local executable:

chmod +x /etc/rc.d/rc.local
sudo vim /etc/rc.d/rc.local
# then add yout commands to run

Mac fix mongodb max file limit issue / Adjust ulimit

echo kern.maxfiles=65536 | sudo tee -a /etc/sysctl.conf
echo kern.maxfilesperproc=65536 | sudo tee -a /etc/sysctl.conf
sudo sysctl -w kern.maxfiles=65536
sudo sysctl -w kern.maxfilesperproc=65536

Then set ulimit in bash_profile

echo "ulimit -n 65536 65536" >> ~/.bash_profile

Ubuntu set timezone

sudo dpkg-reconfigure tzdata

Mac show git branch name in terminal

Add following code into ~/.bash_profile

parse_git_branch() {
  git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ (\1)/'
}

export PS1="\u@\h \W\[\033[32m\]\$(parse_git_branch)\[\033[00m\] $ "

Prepare extra ~/bin folder for shortcut commands

edit ~/.bash_profile add line

export PATH="~/bin/:$PATH"

Auto generate git commit message with one file name

touch ~/bin/autocommit && chmod +x ~/bin/autocommit && vim ~/bin/autocommit 

Paste

#!/bin/bash
BRANCH_NAME=$(git symbolic-ref --short HEAD)
COMMIT_MSG=$(LANG=C git -c color.status=false status \
| sed -n -E -e '1,/Changes to be committed:/ d'\
            -e '1,1 d' \
            -e '/git reset/ d' \
            -e '/^Untracked files:/,$ d' \
            -e 's/^\s*//' \
            -e 's/      modified:   //' \
            -e '/./p')
FIRST_FILE=${COMMIT_MSG%%$'\n'*}
git commit -m "[${BRANCH_NAME}] ${FIRST_FILE}"

usage: just type autocommit and it will generate a commit message containing the current branch name and first file changed

Auto commit and push

git config --global push.default current
echo "autocommit && git push" > ~/bin/autopush
chmod +x ~/bin/autopush

usage: just type autopush

Mount hard drive

#find uuid
sudo blkid
#mount hard drive
sudo mount /dev/sda1 /mnt/ssd1


# auto mount
sudo vim /etc/fstab
#add 
UUID=0c18cc09-0028-4cb1-97ef-6222fe477432   /mnt/ssd   ext3    defaults     0        2

Better method

DRIVE=sdb1
FOLDER=ssd1
UUID=$(sudo blkid /dev/${DRIVE} -sUUID -ovalue)
echo $UUID
echo "UUID=${UUID} /mnt/${FOLDER} xfs defaults 0 1" | sudo tee -a /etc/fstab
sudo mount --uuid $UUID /mnt/${FOLDER}

fix corrupt hard drive file system

sudo gdisk /dev/sdb

sudo e2fsck /dev/sdb1

using gparted for over 2tb

https://www.cyberciti.biz/tips/fdisk-unable-to-create-partition-greater-2tb.html

sudo apt-get install -y gparted xfsprogs

DRIVE=sdb
FOLDER=ssd1

sudo parted /dev/${DRIVE}

mklabel gpt
mkpart primary 0G 100%
print
quit

sudo mkfs.xfs /dev/${DRIVE}1
# mount hard drive
UUID=$(sudo blkid /dev/${DRIVE}1 -sUUID -ovalue)
echo $UUID
echo "UUID=${UUID} /mnt/${FOLDER} xfs defaults 0 1" | sudo tee -a /etc/fstab
sudo mount --uuid $UUID /mnt/${FOLDER}

Test disk write speed

sudo chown -R $USER:$USER ./
dd if=/dev/zero of=./test1.img bs=1G count=1 oflag=dsync && rm ./test1.img

Splunk installation

download splunk then follow http://docs.splunk.com/Documentation/Splunk/7.0.3/Installation/InstallonLinux

Change port

sudo vim $SPLUNK_HOME/etc/system/local/web.conf

add

[settings]
httpport = 8866

Configure to start on boot time http://docs.splunk.com/Documentation/Splunk/latest/Admin/ConfigureSplunktostartatboottime

sudo $SPLUNK_HOME/bin/splunk enable boot-start
sudo service splunk start

List file sizes by directory (depth = 1)

du -ah --max-depth=1 | sort -n

Allow pm2 run via ssh and cron

n=$(which node); \
n=${n%/bin/node}; \
chmod -R 755 $n/bin/*; \
sudo cp -r $n/{bin,lib,share} /usr/local

method 2:

sudo rm -f /usr/local/bin/pm2 && sudo ln -s $(which pm2) /usr/local/bin/pm2

control cpu fan speed

see https://askubuntu.com/questions/22108/how-to-control-fan-speed

sudo apt-get install fancontrol lm-sensors
sudo sensors-detect
# answer yes to all questions

check temp

sensors

edit /etc/default/grub to add acpi_enforce_resources=lax to the GRUB_CMDLINE_LINUX_DEFAULT

then run

sudo update-grub

reboot

sudo service kmod start
sudo pwmconfig

view current config

/etc/fancontrol

finally start fan control

sudo service fancontrol start

auto start

sudo update-rc.d fancontrol enable

config explaination

MINTEMP: The temperature (°C) at which to SHUT OFF the CPU fan. Efficient CPUs often will not need a fan while idling. Be sure to set this to a temperature that you know is safe. Setting this to 0 is not recommended and may ruin your hardware!

MAXTEMP: The temperature (°C) at which to spin the fan at its MAXIMUM speed. This should be probably be set to perhaps 10 or 20 degrees (°C) below your CPU's critical/shutdown temperature. Setting it closer to MINTEMP will result in higher fan speeds overall.

MINSTOP: The PWM value at which your fan stops spinning. Each fan is a little different. Power tweakers can echo different values (between 0 and 255) to /sys/class/hwmon/hwmon0/device/pwm1 and then watch the CPU fan. When the CPU fan stops, use this value.

MINSTART: The PWM value at which your fan starts to spin again. This is often a higher value than MINSTOP as more voltage is required to overcome inertia.

raw disk backup

if: input file

of: output file (image)

dd if=/dev/mmcblk0 of=/media/usb500/pi_system_torrent.image

search for file name or extension

search for file name or extension find . -iname "*.jpg"

delete files by matches

find . -iname "*.jpg" -delete

rsync file from remote to local

nohup rsync -avh -e "ssh -x -T -o Compression=no" root@domain.com:/mnt/ssd1/* ./ >~/rsync.log &

The above command turn off compression so it will be faster.

search

grep --color -n -C 1 "search_text" lib/npm.js

regex

grep --color -n -e "search_text[abc|abe]" lib/npm.js

search all files recursive

grep -HR --color "loan" ./

backup whole system

sudo su
dd if=/dev/sdc | bzip2 > /mnt/ssdbackup/ubuntuBackUp201803.bz2
bzcat /mnt/ssdbackup/ubuntuBackUp201803.bz2 | dd of=/dev/sdc

Start socks proxy server

see https://github.com/adegtyarev/docker-dante use docker-compose

Install fusion io drive2 driver in ubuntu 16.04

fusionio driver

Port Forwarding Simple TCP Traffic

use rinetd https://github.com/boutell/rinetd

Cron start service if stopped

SERVICE=frpc

if pgrep "$SERVICE" >/dev/null
then
    echo "$SERVICE is running"
else
    echo "$SERVICE stopped"
    /root/frp/frpc -c /root/frp/frpc.ini &
fi

rinetd simple tcp traffic forwarding

This is useful for forwarding traffic to server behind vpn

Auto restart process if exit

#/bin/bash
RIGID=$(hostname)
echo $RIGID
STATUS=`ps -A | grep gethbsc`
# change channel to slack webhook
CHANNEL="https://hooks.slack.com/services/xxxxxxxx/xxxxxxx"
if [ ! -z "$STATUS" -a "$STATUS" != " " ] ; then
  echo $STATUS
else
  echo "Restart service"
  cd ~/bsc && bash ~/bsc/start.sh
  curl -X POST -H 'Content-type: application/json' --data "{\"text\":\"server=${RIGID} node down\"}" $CHANNEL
fi

setup apache reverse proxy

Setup apache2 reverse proxy