No Znx!

I am a constant user of screen and it always suprises me how many people use it and barely know how truly powerful it is. Here is a short list of some very neat tricks.

As always looking through info and man pages can provide a raft of information about tools and sometimes uncover tricks that you might of otherwise not known about!

This is a really simple script that can be used to backup all the databases in a MySQL server. Thought I’d put it here as it shows some basic scripting techniques that maybe will help people see how to write scripts. Basically all this does is save me from having to type out the date. It could be automated but that would require storing the user name and password somewhere which normally isn’t too smart an idea.

#!/bin/sh
# GNU GPL version 3.0 or above
# Copyright (c) 2009 Mark Sangster
 
if [ -z "$3" ]; then
  echo 'mybackup <user> <password> <host>' >&2
  exit 1
fi
 
MYUSER=$1; MYPASS=$2; MYHOST=$3
 
mysqldump -u $MYUSER -p $MYPASS -h $MYHOST -A | \
  bzip2 -9 > mysql-`date +%Y-%m-%d`.bz2

The script usage is very striaght-forward:

# mybackup user pass host

At which point you will find a file called “mysql-2009-01-27.bz2″ or similar.  Nothing to difficult!

A problem that seems to crop us with these now and then is that one of the recorded files in your Library becomes corrupt and seemingly impossible to delete. When you select it the box freezes and requires to be hard restarted by removing the power. There is however a solution.

• Select “Menu”
• Option 4 – “Setup”
• Option 6 – “Help”
• Type in 1397 (this is to enter the “Engineer” panel)
• Option 7 – “Disable Library Video Window”
• Go back to your library now and select then delete the offending file.

That’s it, the reason that it is freezing is that the small preview window is attempting to load the corrupted file thus freezing the box. Disabling that preview window allows you to delete the file without it loading.

Once complete, the preview window will automatically enable itself again.

A mate of mine was reading this and made some great comments about it! First up there is a Bash version of expect (THANK GOD!). Secondly fetchmail maybe a better solution to grabbing the mail to see if it was received, another good comment! Unfortunately for me I was working on a system with the tools that were available to me, that means no fetchmail and no empty installed.

Interestingly it didn’t initially work 100% of the time, after a little bit of testing it looked like Microsoft were temporarily throwing away some of the test e-mails. Interesting as it must mean that they look for repetitious e-mails in a bid to fight SPAM. We of course simply contacted MS and got it sorted and now the e-mails come in every hour.

What this does show us though, is that under UNIX/Linux there is a lot of different solutions that can be made, but sometimes you have to work with the tools that are there.

At my work we have two domains and recently one of those domains was blacklisted, thanks to some phishing e-mails hitting their marks. So I was tasked today to setup an automated e-mail send and check. To ensure that we can send between our domains.

First up I checked to see that I was able to get a POP3 connection to the other domain, as it is hosted by Microsoft for us. Whilst I was able to connect via POP3 I was unable to login. A quick check revealed that only POP3S was being accepted. This meant that I was going to need to utilize the simple s_client tool available with OpenSSL.

So to testing:

$ openssl s_client -connect SERVER:995
.... blah blah ....
+OK The POP3 Service is ready.
USER test.user@example.com
^C
$

Hrmm, what’s going on here, the server isn’t responding to my USER. A quick brain check and I spotted my issue, line endings!

$ openssl s_client -connect SERVER:995 -crlf
.... blah blah ....
+OK The POP3 Service is ready.
USER test.user@example.com
+OK
PASS mypassword
+OK User successfully logged in.
STAT
+OK 49 2758446
QUIT
DONE

That’s more like it. Now I know that I can connect, logged and run arbitrary commands. It may be perfectly OK to manually check like this but that isn’t the reason we started this, we want an automated solution.

The most obvious solution here is expect, ugh TCL! With the above information it is easy to write a simple expect script:

#!/usr/bin/env expect
spawn openssl s_client -connect SERVER:995 -crlf
expect "+OK The POP3 service is ready."
send "USER test.user@example.com\r"
expect "+OK"
send "PASS mypassword\r"
expect "+OK User successfully logged on."
send "QUIT\r"
expect "DONE"
expect eof

Simple enough, send a command and expect a response. Obviously it helps if you understand the protocol that you are dealing with. Running that script happily logs in and then quits. Sweet, we are halfway there already!

So to recap, I plan on sending an e-mail from the first domain to my test account on the second domain and then check that the e-mail actually arrived. Thus proving that the e-mail was getting through without issue. So when I do this I will wish to get the newest e-mail in my INBOX, which may not the be the first. So the script needs some modifying.

#!/usr/bin/env expect
spawn openssl s_client -connect SERVER:995 -crlf
expect "+OK The POP3 service is ready."
send "USER test.user@example.com\r"
expect "+OK"
send "PASS mypassword\r"
expect "+OK User successfully logged on."
send "STAT\r"
expect -re "OK (.*) .*" {
  set number $expect_out(1,string)
}
send "TOP $number 1\r"
expect "\."
expect "+OK"
send "QUIT\r"
expect "DONE"
expect eof

Now expect will grab the number of e-mails in the INBOX and request the headers of the newest one. So the final step is integrating this with a simple check. Lets just use a shell script to check with:

#!/bin/sh
TS=`date +%s`
echo "Subject: TESTING $TS
 
Hi,
 
This is a test please do not delete me!
 
Thanks" | mail test.user@example.com
sleep 60
expect collect.exp | grep $TS && \
 echo "Success The E-Mail Arrived!" ||\
 echo "Failure The E-Mail Was Not Found!"

That works, a test and we have a success! We find the TS in the subject line of the e-mail. There is a minor problem here, what happens if I e-mail the test and in-between I receive some other e-mails, hrmm! OK so why don’t we dump the headers of the first few e-mails, we certainly don’t expect a rush of e-mails on a test account within a minute. Let’s change the expect script just a bit more:

#!/usr/bin/env expect
 
spawn openssl s_client -connect SERVER:995 -crlf
expect "+OK The POP3 service is ready."
send "USER test.user@example.com\r"
expect "+OK"
send "PASS mypassword\r"
expect "+OK User successfully logged on."
send "STAT\r"
expect -re "OK (.*) .*" {
  set number $expect_out(1,string)
}
send "TOP $number 1\r"
expect "\."
expect "+OK"
set number [expr $number-1]
send "TOP $number 1\r"
expect "\."
expect "+OK"
set number [expr $number-1]
send "TOP $number 1\r"
expect "\."
expect "+OK"
send "QUIT\r"
expect "DONE"
expect eof

Placing the shell script into a crontab and we are done. Yet another example of how simple scripting can achieve anything.