Sysadmin | Raam Dev

Posts Tagged: sysadmin

Yahoo DNS Issues Cause Problems in the United States

Yahoo! appears to be inaccessible to people in the US. Visiting yahoo.com redirects to www.yahoo.com and fails to load. I confirmed it was at least somewhat limited to the US by trying the connection from a shell account on a server in Europe.

Using dig (a Unix DNS lookup utility), we can see from within the United States that there is a problem with DNS. There is no A record with an IP address listed in the ANSWERS section:

;; QUESTION SECTION: ;www.yahoo.com. IN A

;; ANSWER SECTION: www.yahoo.com. 129 IN CNAME www.wa1.b.yahoo.com.

And from the server in Europe:

;; QUESTION SECTION: ;www.yahoo.com. IN A

;; ANSWER SECTION: www.yahoo.com. 272 IN CNAME www.wa1.b.yahoo.com. www.wa1.b.yahoo.com. 33 IN CNAME www-real.wa1.b.yahoo.com. www-real.wa1.b.yahoo.com. 33 IN A 209.191.93.52
;; AUTHORITY SECTION: wa1.b.yahoo.com. 273 IN NS yf2.yahoo.com. wa1.b.yahoo.com. 273 IN NS yf1.yahoo.com.

If you try connecting directly to the missing IP address, you should at least be able to get the main Yahoo page: http://209.191.93.52. You might also try temporarily adding an entry to your /etc/hosts or C:\Windows\system32\drivers\etc\hosts if you want to continue being able to use the FQDN.

UPDATE: As of 15:50 EST, Yahoo appears to be working again. The outage appeared to start around 15:11 EST, so that’s a good 40 minutes of downtime.

HOWTO: Count Files Recursively with Exclusion on Linux

Find all files in this directory, including the files in sub-directories, and exclude all files that start with a period (dot files) and any directories named .thumbs. Then pass the list of results to the wc command to get a total count:

find . ! -name ".*" ! -path "*.thumbs*" -type f | wc -l

Mounting HFS+ with Write Access in Debian

When I decided to reformat and install my Mac Mini with the latest testing version of Debian (lenny, at the time of this writing) I discovered that I couldn’t mount my HFS+ OS X backup drive with write access:

erin:/# mount -t hfsplus /dev/sda /osx-backup [ 630.769804] hfs: write access to a journaled filesystem is not supported, use the force option at your own risk, mounting read-only.

This warning puzzled me because I was able to mount fine before the reinstall and, since the external drive is to be used as the bootable backup for my MBP, anything with “at your own risk” was unacceptable.

I had already erased my previous Linux installation so I had no way of checking what might have previously given me write access to the HFS+ drive. A quick apt-cache search hfs revealed a bunch of packages related to the HFS filesystem. I installed the two that looked relevant to what I was trying to do:

hfsplus - Tools to access HFS+ formatted volumes hfsutils - Tools for reading and writing Macintosh volumes

No dice. I still couldn’t get write access without that warning. I tried loading the hfsplus module and then adding it to /etc/modules to see if that would make a difference. As I expected, it didn’t. I was almost ready to give up but there was another HFS package in the list that, even though it seemed unrelated to what was trying to do, seemed worth a shot:

hfsprogs - mkfs and fsck for HFS and HFS+ file systems

It worked! I have no idea how or why (and I’m not interested enough to figure it out), but after installing the hfsprogs package I was able to mount my HFS+ partition with write access.

Update:

As Massimiliano and Matthias have confirmed in the comments below, the following solution seems to work with Ubuntu 8.04:

From Linux, after installing the tools suggested before, you must run:
mount -o force /dev/sdx /mnt/blabla

Otherwise, in my fstab, I have an entry like this:
UUID=489276e8-7f9b-3ae6-8c73-69b99ccaab9c /media/Leopard hfsplus defaults,force 0 0

Understanding the Linux Load Averages

I have been using Linux for several years now and although I have looked at the load averages from time to time (either using top or uptime), I never really understood what they meant. All I knew was that the three different numbers stood for averages over three different time spans (1, 5, and 15 minutes) and that under normal operation the numbers should stay under 1.00 (which I now know is only true for single-core CPUs).

Earlier this week at work I needed to figure out why a box was running slow. I was put in charge of determining the cause, whether it be excessive heat, low system resources, or something else. Here’s what I saw for load averages when I ran the top command on the box:

load average: 2.86, 3.00, 2.89

I knew that looked high, but I had no idea how to explain what “normal” was and why. I quickly realized that I needed a better understanding of what I was looking at before I could confidently explain what was going on. A quick Google search turned up this very detailed article about Linux load averages, including a look at some of the C functions that actually do the calculations (this was particularly interesting to me because I’m currently learning C).

To keep this post shorter than the aforementioned article, I’ll simply quote the two sentences that gave me a clear-as-day explanation of how to read Linux load averages:

The point of perfect utilization, meaning that the CPUs are always busy and, yet, no process ever waits for one, is the average matching the number of CPUs. If there are four CPUs on a machine and the reported one-minute load average is 4.00, the machine has been utilizing its processors perfectly for the last 60 seconds.

The machine I was checking at work was a single-core Celeron machine. This meant with a continuous load of almost 3.00 the CPU was being stressed much higher than it should be. Theoretically, a dual-core machine would drop this load to around 1.50 and a quad-core would drop it to 0.75.

There is a lot more behind truly understanding the Linux load averages, but the most important thing to understand is that they do not represent CPU usage. Rather they represent the load on the CPU by processes waiting for their chance to use the CPU. If you still can’t get your brain away from thinking in terms of percentages, consider 1.00 to be 100% load for single-core CPU’s, 2.00 to be 100% load for dual-core CPUs, and so on.

Update: John Gilmartin had some insightful feedback and shared a link to Understanding Load Averages where there’s a nice graphical description for how load averages work.

Creating a Bootable OS X Backup on Linux: Impossible?

I’ve had plans for a while now to set up a backup system using a Debian Linux server and rsync to back up my MacBook Pro laptop. At first glance, it seemed like it would be pretty straight forward. I’ve been able to make a bootable copy of my entire MBP using nothing but rsync (thanks to some very helpful directions by Mike Bombich, the creator of the popular, and free, Carbon Copy Cloner software). And by bootable copy I mean I could literally plug in the USB drive and boot my MBP from the drive (hold down the Alt/Option key while booting). Restoring a backup is as simple as running the rsync command again, but in the reverse direction. I know this solution works because I used it when I upgraded to a 320GB hard drive.

To start, I needed to create a big enough partition on the external USB drive using Disk Utility (formatted with Mac OS Extended (Journaled)). I then made a bootable copy of my MBP with one rsync command:

sudo rsync -aNHAXx --protect-args --fileflags --force-change \ --rsync-path="/usr/local/bin/rsync" / /Volumes/OSXBackup

But my dream backup system was more unattended. I wanted something that would periodically (a couple times a day) run that rsync command over SSH (in the background) and magically keep an up-to-date bootable copy of my MBP on a remote server.

I love Linux and I jump at any opportunity to use it for something new, especially in a heterogeneous network environment. So when I decided to set up a backup server, I naturally wanted to make use my existing Debian Linux machine (which just so happens to be running on an older G4 Mac Mini).

So, after making a bootable copy of my MBP using the local method mentioned above, I plugged the drive into my Linux machine, created a mount point (/osx-backup), and added an entry to /etc/fstab to make sure it was mounted on boot (note the filesystem type is hfsplus):

/dev/sda /osx-backup hfsplus rw,user,auto 0 0

All that’s left to do now is to run the same rsync command as earlier but this time specifying the remote path in the destination (root@myserver.example.com:/osx-backup/). This causes rsync to tunnel through SSH and run the sync. Unfortunately, this is where things started to fall apart.

OS X uses certain file metadata which must be copied for the backup to be complete (again, we’re talking about a true bootable copy that looks no different than the original). Several of the flags used in the rsync command above are required to maintain this metadata and unfortunately Linux doesn’t support all the necessary system calls to set this data. In particular, here are the necessary flags that don’t work when rsyncing an OS X partition to Linux:

-X (rsync: rsync_xal_set: lsetxattr() failed: Operation not supported (95))
-A (recv_acl_access: value out of range: 8000)
–fileflags (on remote machine: –fileflags: unknown option)
–force-change (on remote machine: –force-change: unknown option)
-N (on remote machine: -svlHogDtNpXrxe.iL: unknown option)

According to the man page for rsync on my MBP, the -N flag is used to preserve create times (crtimes) and the --fileflags option requires chflags system call. When I compiled the newer rsync 3.0.3 on my MBP, I had to apply two patches to the source that were relevant to preserving Mac OS X metadata:

patch -p1 <patches/fileflags.diff patch -p1 <patches/crtimes.diff

I thought that maybe if I downloaded the source to my Linux server, applied those same patches, and then recompiled rsync, that it would be able to use those options. Unfortunately, those patches require system-level function calls (such as chflags) that simply don’t exist in Linux (the patched source wouldn’t even compile).

So I tried removing all unsupported flags even though I knew lots of OS X metadata would be lost. After the sync finished, I tried booting from the backup drive to see if everything worked. It booted into OS X, but when I logged into my account lots of configuration was gone and several things didn’t work. My Dock and Desktop were both reset and accessing my Documents directory gave me a “permission denied” error. Obviously that metadata is necessary for a viable bootable backup.

So, where to from here? Well, I obviously cannot use Linux to create a bootable backup of my OS X machine using rsync. I read of other possibilities (like mounting my Linux drive as an NFS share on the Mac and then using rsync on the Mac to sync to the NFS share) but they seemed like a lot more work than I was looking for. I liked the rsync solution because it could easily be tunneled over SSH (secure) and it was simple (one command). I can still use the rsync solution, but the backup server will need to be OS X. I’ll be setting that up soon, so look for another post with those details.

Posts Tagged: sysadmin

Yahoo DNS Issues Cause Problems in the United States

HOWTO: Count Files Recursively with Exclusion on Linux

Mounting HFS+ with Write Access in Debian

Understanding the Linux Load Averages

Creating a Bootable OS X Backup on Linux: Impossible?

Where in the world is Raam?

Join the Facebook Community

Latest photos from Flickr