Tag: how-to

Setting up ssh reverse tunnels with reversi

# reversi-server and reversiclient
# Author: kazimof at zzero dot org

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# http://www.gnu.org/licenses/.

repo now moved to git.
https://github.com/kazimof/reversi/

WHAT DOES REVERSI DO?
———————
Uses reverse ssh tunnels to expose device ports on a reversi-client’s network to a reversi-server anywhere on the internet.

A reverse ssh tunnel (ssh -R) can be established quite easily, the problem comes with connection stability, where the ssh connection can and does hang from either the server or client end. Reversi tests the validity of the connection from the client every X seconds and re-establishes it if the connection is not fit for purpose.

EXAMPLE USE 1:
—————–
You need to manage a LAN but you have no access to their office broadband router to configure port forwarding.

EXAMPLE USE 2:
—————–
You have OpenWRT devices installed within a public mesh network and you need to be able to manage them.

EXAMPLE USE 3:
—————–
A company router has died or their broadband has been cut or they have moved premises, so they are on a temporary 3/4G connection waiting for a new broadband router with a static IP. Obviously the server not getting mail since port 25 is no longer forwarded by router, and webmail is unavailable to user for the same reason. Using an openwrt inside the network I forward ports 25 and 443 of the mail server to internet allowing the company mail server to receive mail and service web mail requests.

HOW DOES IT WORK?
—————–
reversi-clients are equipped with a password-free ssh key to a user on the reversi-server. Using this key the reversi-client is allowed to do 2 things:
1) run a script that forwards any local port on it’s network the reversi-server
2) run a script on the reversi-server passing arguements that inform the reversi-server about which node they are, which port it is using and what kind of response to expect from that port

On the client side the scripts are both run in gnu-screen, one does the reverse connection and the other acts as a connections monitor, killing and re-establishing the reverse tunnel according to the server response from the script referenced in 2) above.

On the server side the script referenced in 2) above tests to see whether the port is responding with the expected response, if not it assumes the connection has hung, identifies the appropriate PID and kills it. Script then returns a number specifying a time which the client monitor script should wait before retyring the connection.

DESIGN PARAMETERS
—————–
Needs to run on a variety of client OS, I am managing a mix of hundreds of openwrt devices, linux servers and OSX installs, I needed this solution for them all and I am too lazy to maintain multiple forks.
The 2 programs available to all platforms are:

screen
ssh

DESIGN
——
Reversi commprises of 2 components to be installed on the reversi server.

SERVER: hosts a script that clients are authorised to invoke to monitor and manage their reverse tunnel
RECOMMENDED INSTALL FOLDER: /home/reversi/reversi
CLIENT BUILDER: builds the scripts to be run on the client device
RECOMMENDED INSTALL FOLDER: /root/reversi/client-builder

See INSTRUCTIONS.TXT for installation info.

Scripts for monitoring Linux servers

Tired of regularly logging into our servers to find out if anything is wrong, I wanted to know before something like a spam attack or a disk-eater gets out of hand.

Wrote some scripts to automate the process and email me some stats. Yes there are tools to do this for you, but then you have to maintain those tools, and sometimes installing them and configuring them on so many servers can be a pain. This system needs only ssh and mutt, lightweight and available in almost all Linux distributions I know.

This, it’s simply bash installed on one server, and everything is “pushed” fresh to the servers every time it is run, so updates are automatic and easily deployed.

Design

The main script checkserver.sh copies (using scp)  the checkservices, checkdf, checkmailq and checkmaillog scripts and config files to the target host, runs them there and emails the output to you.

On your host servers there is nothing to install except openssh-server. Configure your password-less ssh-keys for all the servers in ~/.ssh/config on your monitoring server with the public keys in /root/.ssh/authorized_keys on the hosts. For help with that see our article: master-ssh-key

Scripts

 

To send notification emails you need a mutt wrapper, almost all distributions have mutt in their repos.

muttcc.sh

func_muttemail () {

#### establish current dir
APPDIR=$( cd “$( dirname “$0″ )” && pwd )
gotdate=`date +%Y-%m-%d-%H-%M`
DEBUGLOG=”$APPDIR/functions-email.log”
echo “ENTERING mutt.sh” | tee -a $DEBUGLOG

#### ASSIGN VARS
mailentity=”$1″
mailsubject=”$2″
maildatafile=”$3″
mailentitycc=”$4″
echo “$mailentity $mailsubject $maildatafile” | tee -a $DEBUGFILE
#send email
echo “cc specified: $4″ | tee -a $DEBUGLOG
if [ x”${mailentitycc}” = x ]
then
#the cc is not set so do not include mecc
mutt -F $APPDIR/muttrc -s “$mailsubject” “$mailentity” < $maildatafile
echo “no cc specified: $mailentitycc” | tee -a $DEBUGLOG
else
mutt -F $APPDIR/muttrc -s “$mailsubject” -c “$mailentitycc” “$mailentity” < $maildatafile
echo “cc specified: $mailentitycc” | tee -a $DEBUGLOG
fi

EX=$?
echo “$EX”
echo “##### EXITING mutt.sh #######” | tee -a $DEBUGLOG
return $EX
}

You’ll need to install mutt and have access to an smtp server. Mine is local and needs no authentication. Normally this fill will be the user’s “.muttrc” file, but in the wrapper we have specified it’s location.

muttrc

set from = “me@myserver.net”
set realname = “CHECKSERVER”
set use_envelope_from = yes
set smtp_url=smtp://mylocalmailserverhostname.net/
set ssl_starttls = no

checkservers.sh

#!/bin/bash
APPDIR=$( cd “$( dirname “$0″ )” && pwd )
. /$APPDIR/muttcc.sh
SERVERLIST=$APPDIR/server.list
MAILQTESTS=$APPDIR/checkmailq.txt
MAILLOGTESTS=$APPDIR/checkmaillog.txt
MAILQSH=$APPDIR/checkmailq.sh
MAILLOGSH=$APPDIR/checkmaillog.sh
DISKDF=$APPDIR/checkdiskdf.sh
CHECKLOG=/tmp/servercheck.log

while IFS=”:” read servername ifmailq ifmaillog ifdiskdf serviceslist
do
echo “=========== SERVERCHECK STARTS: $servername ==============”
echo “=========== SERVERCHECK STARTS: $servername ==============” > $CHECKLOG
echo “———————- disks —————— ” >> $CHECKLOG
if [ “$ifdiskdf” == “diskdf” ]
then
scp $APPDIR/checkdiskdf.sh root@$servername:/usr/local/sbin/
ssh -n root@$servername /usr/local/sbin/checkdiskdf.sh >> $CHECKLOG
else
echo “$servername diskdf not flagged for check” >> $CHECKLOG

fi
echo “———————- services —————— ” >> $CHECKLOG
echo “SERVICEFILE: $serviceslist”
case $serviceslist in
no)
echo “$servername services not flagged for check” >> $CHECKLOG
;;
*)
scp $APPDIR/checkservices-$serviceslist.txt root@$servername:/usr/local/sbin/checkservices.txt
scp $APPDIR/checkservices.sh root@$servername:/usr/local/sbin/
ssh -n root@$servername /usr/local/sbin/checkservices.sh >> $CHECKLOG
esac

echo “———————- mailq —————— ” >> $CHECKLOG
if [ “$ifmailq” == “mailq” ]
then
scp $APPDIR/checkmailq.sh root@$servername:/usr/local/sbin/
scp $APPDIR/checkmailq.txt root@$servername:/usr/local/sbin/
ssh -n root@$servername /usr/local/sbin/checkmailq.sh >> $CHECKLOG
else
echo “$servername mailq not flagged for check” >> $CHECKLOG
fi

echo “—————– maillog —————– ” >> $CHECKLOG
if [ “$ifmaillog” == “maillog” ]
then
scp $APPDIR/checkmaillog.sh root@$servername:/usr/local/sbin/
scp $APPDIR/checkmaillog.txt root@$servername:/usr/local/sbin/
ssh -n root@$servername /usr/local/sbin/checkmaillog.sh >> $CHECKLOG
else
echo “$servername maillog not flagged for check” >> $CHECKLOG
fi
echo “=========== SERVERCHECK ENDS: $servername ==============” >> $CHECKLOG
email_subject=”checkserver results: $servername”
echo “$email_subject”
tmp2=$(func_muttemail “myemail@address.com” “$email_subject” “$CHECKLOG” “ccemail@address.com”)
echo “=========== SERVERCHECK ENDS: $servername ==============”

done < “$SERVERLIST”

checkdiskdf.sh

#!/bin/bash
/bin/df -h
exit

checkmaillog.sh

#!/bin/bash
APPDIR=$( cd “$( dirname “$0″ )” && pwd )
#greps maillotgs and mailqueue fiels for various issues
MAILLOGTESTS=$APPDIR/checkmaillog.txt
MAILRESULT=$APPDIR/checkmailresult.log
MAILRESULTREPORT=$APPDIR/checkmailresult.log
MAILRESULTTEMP=$APPDIR/checkmailresulttemp.log
rm $MAILRESULTTEMP
while IFS=”:” read testname before after
do
datum=`date “+%b %d”`
grep -A$after -B$before “$testname” /var/log/maillog | grep “$datum” >> $MAILRESULTTEMP

done < $MAILLOGTESTS
resultlength=$(cat “$MAILRESULTTEMP” | wc -l)
echo “$resultlength”
if [ “$resultlength” -gt “1” ]
then
echo “issues found”
echo “mail log errors found. lines=$resultlength.” > $MAILRESULTREPORT
cat “$MAILRESULTTEMP” >> $MAILRESULTREPORT
cat “$MAILRESULTREPORT”
else
echo “nothing to report”
fi

checkmaillog.txt

refused to talk to me:0:0
Connection timed out:0:0
Name service error for name:0:0
sender non-delivery notification:0:0
to=<noreply@:0:0
421 Too many concurrent SMTP connections:0:0
testing something:0:0

checkmailq.sh

#!/bin/bash
APPDIR=$( cd “$( dirname “$0″ )” && pwd )
#greps maillotgs and mailqueue fiels for various issues

MAILQTESTS=$APPDIR/checkmailq.txt
MAILRESULT=$APPDIR/checkmailresult.log
MAILRESULTREPORT=$APPDIR/checkmailresult.log
MAILRESULTTEMP=$APPDIR/checkmailresulttemp.log
rm $MAILRESULTTEMP

while IFS=”:” read testname before after
do
/usr/bin/mailq | grep -A$after -B$before “$testname” >> $MAILRESULTTEMP
done < $MAILQTESTS
resultlength=$(cat “$MAILRESULTTEMP” | wc -l)
echo “$resultlength”
if [ “$resultlength” -gt “1” ]
then
echo “issues found”
echo “there is queued mail. lines=$resultlength.” > $MAILRESULTREPORT
cat “$MAILRESULTTEMP” >> $MAILRESULTREPORT
cat “$MAILRESULTREPORT”
else
echo “nothing to report”
fi

checkmailq.sh

#!/bin/bash
APPDIR=$( cd “$( dirname “$0″ )” && pwd )
#greps maillotgs and mailqueue fiels for various issues

MAILQTESTS=$APPDIR/checkmailq.txt
MAILRESULT=$APPDIR/checkmailresult.log
MAILRESULTREPORT=$APPDIR/checkmailresult.log
MAILRESULTTEMP=$APPDIR/checkmailresulttemp.log
rm $MAILRESULTTEMP

while IFS=”:” read testname before after
do
/usr/bin/mailq | grep -A$after -B$before “$testname” >> $MAILRESULTTEMP
done < $MAILQTESTS
resultlength=$(cat “$MAILRESULTTEMP” | wc -l)
echo “$resultlength”
if [ “$resultlength” -gt “1” ]
then
echo “issues found”
echo “there is queued mail. lines=$resultlength.” > $MAILRESULTREPORT
cat “$MAILRESULTTEMP” >> $MAILRESULTREPORT
cat “$MAILRESULTREPORT”
else
echo “nothing to report”
fi

checkmailq.txt

Connection timed out:1:1
Connection refused:1:1

checkservices.sh

#!/bin/bash

#!/bin/bash

APPDIR=$( cd “$( dirname “$0″ )” && pwd )

#greps maillotgs and mailqueue fiels for various issues

SERVICETESTS=$APPDIR/checkservices.txt

RESULTREPORT=$APPDIR/checkservicesresultreport.log

RESULTTEMP=$APPDIR/checkservicesresulttemp.log

rm $RESULTREPORT

rm $RESULTTEMP

while IFS=”:” read portnumber service

do result=$( lsof -i:$portnumber | head -n2 > $RESULTTEMP)

resultlength=$(cat “$RESULTTEMP” | wc -l )

if [ “$resultlength” -gt “1” ] then

#service is running there

echo “OK: $service on $portnumber” >> $RESULTREPORT else

echo “++++++ NOK: $service on $portnumber +++++++++” >> $RESULTREPORT

fi

done < $SERVICETESTS

cat “$RESULTREPORT”

checkservices-ALL.txt

22:ssh
443:https
993:imaps
3306:mysql
389:ldap

servers.list

This is your list of servers to process.

megatron:nomailq:nomaillog:diskdf:ALL
magdelena:mailq:maillog:diskdf:ALL

In my crontab. I don’t want email noise from cron so I use /dev/null 2> &1 to null that.

30 08 * * * /usr/local/sbin/backup_app/checkservers.sh > /dev/null 2>&1

Debugging

Debug email issues with functions-email.log

For output issues log into your remote servers and run the check*.sh files to see what comes out.

Room for improvement

tmp files are used per server in the main loop, so take care not to run this script again until complete or you’ll get nonsense.

How to choose secure passwords

Why use secure passwords?

 

  • If your server is exposed to the internet (this will be the case if you have remote access) – you can assume that there will be hundreds of hacking attempts on your server every day. These attacks usually are not coming from humans but by software. The software uses massive dictionaries of human names and potential passwords by trying out millions of combinations until something works. Once the software has access to your account it notifies the human hackers, who’s job it is to hack into your server to access information stored thereon.
  • Don’t assume that you are unimportant, that because your information is not valuable to anyone other than yourself that you will not be a target. Ransomeware systems could encrypt the entire system and demand money in exchange for a decryption service. Hackers also could utilise your system to send spam, an activity that would blacklist your server preventing your real emails from reaching their intended recipients.

    Unexpcted consequences of being hacked

  • How would you like it for all your contacts to be emailed (ostensibly from yourself) with adverts for Viagra or infected file attachments that destroy or damage data on their computers? They may open the attachments because they trusted you.
  • There’s money in this game, big money. A valid email address can be sold to spammers, fetching up to a pound each. If you have any emails in your systems, they are valuable.
  • Data protection liabilities. We all keep names and addresses of our clients and sometimes, under the Data Protection Act we are legally obliged to take care of this information.
  • Infections and Spam: As well as virus infections, slow computers, disrupted networks, you could end up inadvertently hosting objectionable material like as hardcore pornography, viruses or worse.
  • Being cut off by your ISP: Have you heard of “botnets” and “zombies”? A breach of your server like this can end up with your ISP shutting you down for participating in DDOS attacks.

    How do I change my password?

  • In mac/Linux do it through the user manager, or terminal and type: “passwd”  and ENTER. Answer blind (no typing shows) with old pass then new password (enter twice) and your done.
  • In Windows you can press CTRL-ALT-DEL, and click the button “Change Password”

Password good practise.

A perfect password is impossible, with quantum computing any password can be cracked eventually. However we are human, and we can only do our best.

Bear in mind that using an identical password for different services is highly inadvisable, if one of your services are compromised, the others will be too.

Good passwords can be a pain to remember, design and employ a system for your self and keep it to yourself. A good system will allow you to remember the password easily, and in the event that you end up having to reveal the password to anyone (for instance a friend needing access in emergency, or a technician you have asked to recover you computer) your other passwords will not be compromised.

Something simple that works well for many is using PREFIX, a BASENAME WITH SPECIAL CHARACTERS and a SUFFIX.  For example, I need to make a password for my gmail account, I might use:

2017%BaNaNa*1328PATIEgoogligoogli

  1. The PREFIX (2017) is the year I established this account.
  2. The BASENAME is common to all my passwords, I NEVER write it anywhere except at a password prompt. It’s made up of a few things that are easy to remember for me and quick to type, choose numbers and letters that have nothing to do with you personally or legally (as this might allow someone who knows a little about you already to guess). For this reason – NEVER use relative’s birthdays as part of your basename, they are very easy for attackers to discover.
  3. The SUFFIX is something unique to my gmail password only, I always think of this when I see google.
  • Some examples of reasonable (and memorable) passwords:
    7sing*to*me7
    99harryup1-oSCaR
    1Song^^And^^Dance
  • This is what a rubbish password looks like:
    password
    password1
    tuesday
    123
    harry
    10/12/81

Using a VPN on Mac OS 10.12.5 (Sierra)

1. Download and install https://tunnelblick.net/cInstall.html
(Select the option to display the icon in top menu)
2. Download certificates from the server
3. Right-click on the .ovpn file and choose Open With Tunnelblick.app
4. Enter the user password you are logged-in with. If this is a surprise, read http://www.macworld.co.uk/how-to/mac/what-do-if-forgotten-mac-password-3594395/
5. Click on Tunnelblick icon top-right
6. A Connect window will be displayed, click Connect
7. At the top Apple menu, click Go > Connect to Server
8. In the Server Address field, type the name of server
9. Enter username and password for your account on the server
10. The directories will be accessible from the displayed window

My computer won’t switch on!

Diagnose, fix it yourself in 5 minutes

These are the things you can do to troubleshoot this yourself. After each of these steps, press the ON button to test again. Diagnosis is only effective if you do things one at a time.

1. Press EJECT on the CDROM see if any mechanical sounding action results. If it ejects or tries to eject then power is probably OK, but the Operating System has crashed, or else the monitor is off or disconnected.
2. Check there is a light on the monitor / screen. If not then test with a different monitor.
3. Disconnect ANY USB devices (including Keyboard, Monitor, External USB disk, Printer etc)
4. Completely disconnect power plug. Wait 20 seconds. Reconnect.
5. Test another device with the same power cable / wall socket to ensure it is live.

Call your IT support. Make a note of the following findings to help them assist.

1. Do you hear any noises or see lights flashing briefly anywhere on the machine at the moment you plug it in?
2. When you attach an Ethernet cable between the Ethernet Wall Socket (if you have one) and the Ethernet socket on the machine, are there any lights?

If your organisation has no IT support, we may be able to help.

For the confident or the brave

If your willing/able to open the case and get under the hood you can probably get to the root of the problem yourself! Firstly
ensure that the power supply is live as per instructions above (must be some lights or noise when you plug in power). Sometimes a faulty peripheral can stop the machine from starting up so lets try to disconnect them all. Again it’s important, after each step below, to test the ON button again.

1. Open the case – on most modern machines you don’t need tools
2. Disconnect the CDROM
3. Disconnect the Hard Drive
4. (advanced only) Disconnect all the memory modules. Be very careful not to handle with un-gloved fingers the gold/silver strips on the modules. If response to power-on changes then probably you’ve got a faulty one. In this case, place them back in the machine one at a time repeating power-on test each time. When re-installing press firmly and gently, don’t forget to ensure the holding clips are closed properly. When behaviour changes you’ve probably found your faulty SIM. You can get like-replacements from https://crucial.com/ or contact us for a quote.