Raam Dev » thoughts essays journal notes contact about subscribe rss

Posts Tagged: sysadmin

Lazy Linux: 10 Essential tricks for admins

Lazy Linux: 10 Essential tricks for admins is an awesome list of cool things you can do with Linux. I learned about trick #3 (collaboration with screen) from an admin at the datacenter where one of my servers is hosted. I remember being thrilled sitting there watching him do stuff on my server while sharing the keyboard to type messages back and forth in vi (think Remote Desktop or VNC, but on the console). Trick #5 (SSH back door) is something I’ve been using for years at work for remote diagnostics. It is an invaluable trick for getting around firewalls. Very cool stuff!

Being Greedy With Bash

Last night at my C/Unix class the professor quickly glossed over an interesting shell scripting technique that allows you to strip stuff off the beginning or end of a variable. I forgot about it until I saw the technique used again while editing a shell script at work today.

I didn’t know what the technique was called but I remembered the professor saying something about “greedy clobbering” and, since I cannot search Google for special characters, I Googled “Bash greedy” and luckily found 10 Steps to Beautiful Shell Scripts, which just so happened to contain the technique I was looking for (#5).

There are basically four versions of this technique:

${var#pattern}
Search var from left-to-right and return everything after the first occurrence of pattern

${var##pattern}
Search var from left-to-right and return everything after the last occurrence of pattern (be greedy)

${var%pattern}
Search var from right-to-left and return everything after the first occurrence of pattern

${var%%pattern}
Search var from right-to-left and return everything after the last occurrence of pattern (be greedy)

Here’s how it works. Let’s say you have a variable that contains the path to a file:

FILE=/home/raam/bin/myscript.sh

Now let’s say you wanted to extract the myscript.sh part from that variable. You could do some funky stuff with awk but there is a much easier solution built into Bash:

SCRIPTNAME=${FILE##*/}

Now $SCRIPTNAME will contain myscript.sh!

The ##*/ tells the shell to search left-to-right for everything before and including the slash (*/), be greedy while doing it so that all the slashes will be found (##), and then return whatever is left over (in this case, myscript.sh is the only thing remaining after the last slash).

AFAIK, this is a Bash-specific feature, but I’m not entirely certain and I wasn’t sure where I could look to find out. It’s amazing how four characters can do so much work so easily. The more I learn about what I can do with Bash, the more I wonder how I ever lived without all this knowledge!

Bounce-back Spam (Backscatter)

I really hate bounce-back spam! (I call it bounce-back spam, but the official name for it is Backscatter.) I’ve read, and been told by sysadmins, that there is not much that can be done about it. The Wikipedia page on bounce messages has a little section that explains why:

Excluding MDAs, all MTAs forward mails to another MTA. This next MTA is free to reject the mail with an SMTP error message like user unknown, over quota, etc. At this point the sending MTA has to inform the originator, or as RFC 5321 puts it:

If an SMTP server has accepted the task of relaying the mail and later finds that the destination is incorrect or that the mail cannot be delivered for some other reason, then it MUST construct an “undeliverable mail” notification message and send it to the originator of the undeliverable mail (as indicated by the reverse-path).

This rule is essential for SMTP: as the name says, it’s a simple protocol, it cannot reliably work if mail silently vanishes in black holes, so bounces are required to spot and fix problems.

Today, however, most email is spam, which usually utilizes forged Return-Paths. It is then often impossible for the MTA to inform the originator, and sending a bounce to the forged Return-Path would hit an innocent third party. This inherent flaw in today’s SMTP (without the deprecated source routes) is addressed by various proposals, most directly by BATV and SPF.

It looks like I’ll have to just deal with it. (I could set up filters and such, but then I might miss a real bounce-back and not know that my message didn’t go through!) I’m just grateful it comes in waves of a few hours every few weeks instead of non-stop! Has anyone else had to deal with this? If so, what did you do about it?

Backscatter Spam

I subconsciously converted a problem into a shell script

I have been writing a lot of shell scripts lately as part of the C/Unix class that I’m taking at Harvard Extension. My familiarity with how the Unix shell and the underlying system works has grown exponentially. When I came across a problem earlier today, I subconsciously turned the problem into a shell script without even thinking about it!

The problem: “How can I check to make sure my program is running every 30 minutes and restart it if it’s not?”

Answer:

# If myscript isn't running, restart it
ONLINE=`ps aux | grep -c myscript`
# 2 because grep myscript also finds 'grep myscript'
if [ $ONLINE -ne "2" ]; then
        $MYSCRIPT_PATH/restart_service.sh
fi

I’m sure there are many better ways to solve this problem, but the fact that I instantly translated the problem into shell scripting code (and that it worked as expected on my first try) astonished me. I can see how good programmers who write in a particular language, and know the in’s and out’s the like the back of their hand can turn problems into code seamlessly (or know exactly where to look to find answers if they’re unsure).

It’s really amazing how easily you can solve simple problems when you have a deeper understanding of how the system works.

That’s all. I just wanted to share my excitement. :)

Using ‘rsync –exclude-from’ to Exclude Files Containing Spaces

A few months ago I wrote a post about escaping filename or directory spaces for rsync. Well that wasn’t the end of rsync giving me problems with spaces.

When I used the --exclude-from rsync option to specify a list of exclusions, I figured using single or double-quotes around files/directories that contain spaces would be enough to escape them. However, after swashing through hundreds and hundreds of lines from rsync’s output, I discovered the excluded directories were still being synced!

When using --exclude-from, files and directories should not contain any single or double quotes, only a backslash:

/afs/*
/automount/*
/Users/raam/Documents/Virtual\ Machines/*

Where in the world is Raam?

Join the Facebook Community

Raam Dev » thoughts essays journal notes contact about subscribe rss

Powered by WordPress and other Open Source Software
Uncopyright by Raam Dev