Bash | Raam Dev

Posts Tagged: Bash

Blogging from the Command-Line

I’m a command-line person. If you can show me a command-line version of something I already do in a windowed environment, I’ll get stuff done faster. I often look for command-line solutions to tasks that become repetitive and feel as though time could be saved by doing them on the console.

A recent example of this is the posting of asides on my blog. Asides are often very short (one or two sentences at most — they appear on my blog without a title) and navigating the WordPress Administration interface in a web browser simply to post one or two sentences became very time consuming and distracting. Since I’m constantly editing files and code on the console using my favorite editor (vi), being able to quickly create and post an aside from the same environment would be awesome.

Before writing a tool that allowed me to post to my WordPress blog, I searched Google to see if someone else had already written something. Sure enough, I found blogpost, a script written in Python by Stuart Rackham:

blogpost is a WordPress command-line weblog client. It creates and updates weblog entries directly from AsciiDoc (or HTML) source documents. You can also delete and list weblog entries from the command-line.

It uses XML-RPC to post to WordPress blogs and also supports automatically uploading media files (images, videos, audio, documents) that are referenced within the AsciiDoc (or HTML) post file. Check the blogpost man page for full details.

Remember, my main goal here is to make posting short asides easier. I’m perfectly happy using the WordPress web interface to write longer posts. In fact, I prefer the web interface for longer posts because I get things like automatic spell checking (through OS X) and automatic draft saving (through WordPress).

After installing blogpost and modifying the configuration file to include my WordPress login details, I created a file called post.txt using the vi editor and, after saving the file and closing vi, I published the aside using blogpost:

$ blogpost.py --title="My Test Aside Post" -U --doctype='html' create post.txt creating published post 'My Test Aside Post'... id: 2758 url: http://blog.raamdev.com/2009/01/24/my-test-aside-post

$ blogpost.py cat --categories="Asides, Blog Entries, General" post.txt assigning categories: Asides,Blog Entries,General

Note that I only need really basic formatting (i.e., HTML for links), so I use the --doctype='html' option. This allows me to type raw HTML in vi when I’m editing the post file, just as I do now in WordPress (I don’t use the Visual Editor).

While the options and flexibility provided by blogpost are great, the process of publishing an aside needed to be more automated to solve my problem. Creating a new file in vi, typing all those options, running two separate commands, and then deleting the file every time I wanted to post a few sentences on my blog didn’t make a whole lot of sense. So I whipped together this little shell script to help automate the steps above:

#!/bin/sh
##
## aside.sh - automates publishing asides using blogpost.py
##

# Open a temporary file in the vi editor
vi aside.$$

# Display new aside before publishing
echo &quot;New Aside:&quot;
cat aside.$$
echo

# Prompt for an aside title
echo &quot;Enter a title for this Aside:&quot;
read TITLE
echo &quot;OK!&quot;
echo

# Using the temp file saved above, post the Aside
blogpost.py --title=&quot;$TITLE&quot; -U --doctype='html' create aside.$$
blogpost.py cat --categories=&quot;Blog Entries, Asides&quot; aside.$$

# Remove the temporary file
rm aside.$$

Now posting an aside to my blog is as simple as running ./aside.sh, typing the aside in vi, saving and quitting (:wq), and then typing a title. The rest of the work, including cleanup, is taken care of by the script!

Stuart did an excellent job with blogpost and if you have a blog and use the console (and why shouldn’t you?!) I recommend you check it out. The blogpost README is a great place to start, as it includes prerequisites and installation information.

Being Greedy With Bash

Last night at my C/Unix class the professor quickly glossed over an interesting shell scripting technique that allows you to strip stuff off the beginning or end of a variable. I forgot about it until I saw the technique used again while editing a shell script at work today.

I didn’t know what the technique was called but I remembered the professor saying something about “greedy clobbering” and, since I cannot search Google for special characters, I Googled “Bash greedy” and luckily found 10 Steps to Beautiful Shell Scripts, which just so happened to contain the technique I was looking for (#5).

There are basically four versions of this technique:

${var#pattern}
Search var from left-to-right and return everything after the first occurrence of pattern

${var##pattern}
Search var from left-to-right and return everything after the last occurrence of pattern (be greedy)

${var%pattern}
Search var from right-to-left and return everything after the first occurrence of pattern

${var%%pattern}
Search var from right-to-left and return everything after the last occurrence of pattern (be greedy)

Here’s how it works. Let’s say you have a variable that contains the path to a file:

FILE=/home/raam/bin/myscript.sh

Now let’s say you wanted to extract the myscript.sh part from that variable. You could do some funky stuff with awk but there is a much easier solution built into Bash:

SCRIPTNAME=${FILE##*/}

Now $SCRIPTNAME will contain myscript.sh!

The ##*/ tells the shell to search left-to-right for everything before and including the slash (*/), be greedy while doing it so that all the slashes will be found (##), and then return whatever is left over (in this case, myscript.sh is the only thing remaining after the last slash).

AFAIK, this is a Bash-specific feature, but I’m not entirely certain and I wasn’t sure where I could look to find out. It’s amazing how four characters can do so much work so easily. The more I learn about what I can do with Bash, the more I wonder how I ever lived without all this knowledge!

I subconsciously converted a problem into a shell script

I have been writing a lot of shell scripts lately as part of the C/Unix class that I’m taking at Harvard Extension. My familiarity with how the Unix shell and the underlying system works has grown exponentially. When I came across a problem earlier today, I subconsciously turned the problem into a shell script without even thinking about it!

The problem: “How can I check to make sure my program is running every 30 minutes and restart it if it’s not?”

Answer:

# If myscript isn't running, restart it
ONLINE=`ps aux | grep -c myscript`
# 2 because grep myscript also finds 'grep myscript'
if [ $ONLINE -ne "2" ]; then
        $MYSCRIPT_PATH/restart_service.sh
fi

I’m sure there are many better ways to solve this problem, but the fact that I instantly translated the problem into shell scripting code (and that it worked as expected on my first try) astonished me. I can see how good programmers who write in a particular language, and know the in’s and out’s the like the back of their hand can turn problems into code seamlessly (or know exactly where to look to find answers if they’re unsure).

It’s really amazing how easily you can solve simple problems when you have a deeper understanding of how the system works.

That’s all. I just wanted to share my excitement. :)

Google Reported Attack Site

I’m sure some of you must have seen this warning when you tried to visit my site. Fear not, I have fixed the problem. There was an old file on my domain that had a link to a site that was defined as “malicious” by Google, so they basically added my entire domain to the watch list. I removed the file and, after asking Google to check my site again using Google’s Webmaster Tools, they removed my domain from the list.

So, how did I find the few pages (among thousands of files on my site) that contained a link to the malicious site Google was blocking me for? I logged into my site via SSH and ran a command like the following:

for i in `find . -name "*.ht*"` ; do echo $i; cat $i | grep 195\.2\.252; done


This basically searched every single .htm or .html file inside my public_html directory and returned anything that contained the IP address I was looking for. Whenever there was a match, the filename that preceded the output was the offending file. I'm sure there's a more elegant way of doing this, but hell, I just wanted to fix the problem!
Although this was annoying to deal with, it made me feel good that Google is actually keeping track of these things and, with the help of Firefox, is warning people of such sites. Site owners must be vigilant in fixing such problems or they risk losing loads of traffic from Google (and from visitors with Firefox).



				
		

				Escaping Filename or Directory Spaces for rsync
				
					To rsync a file or directory that contains spaces, you must escape both the remote shell and the local shell. I tried doing one or the other and it never worked. Now I know that I need to do both!
So lets say I’m trying to rsync a remote directory with my local machine and the remote directory contains a space (oh so unfortunately common with Windows files). Here’s what the command should look like:
rsync 'raam@example.com:/path/with\ spaces/' /local/path/
The single quotes are used to escape the space for my local shell and the backslash is used to escape the remote shell. 
				
				
				
				

			

		
		
			← Older Entries

Posts Tagged: Bash

Blogging from the Command-Line

Being Greedy With Bash

I subconsciously converted a problem into a shell script

Google Reported Attack Site

Escaping Filename or Directory Spaces for rsync

Where in the world is Raam?

Join the Facebook Community

Latest photos from Flickr