Tuesday, June 4, 2013

Unix: Using 'find' and 'xargs' on filenames that have spaces within them

To use find and xargs effectively together, use -print0 to separate filenames with a null character instead of new lines. Along with that, use -0 with xargs.

$ touch "my file"
$ find | xargs ls
ls: ./my: No such file or directory
ls: file: No such file or directory
$ find -print0 | xargs -0 ls
./my file 

Sunday, April 14, 2013

Unix: Script to compare a directory to a reference directory

Suppose there are two directories. We will call them as the reference and target directories. The requirement is to ensure that each and every entry - directory & file - in the reference directory is also available under a sub-tree of the target directory.

Illustration of requirement:

Given:
Reference Directory:   /tmp/a
Entry under reference directory:   /tmp/a/b/c

Then, if /home/mydir is considered as target directory, /home/mydir/b and /home/mydir/b/c should be present and are of the same type of entry i.e. directory or file.

It should not matter if there are additional directories or files under the target directory - this means as long as everything under the reference directory exists correspondingly (under the same subtree) in the target directory, the requirement is considered fulfilled.

On modern Unixes and Linux machines, we could start-off with:

diff -arq <reference_directory> <target_directory>

to further develop and refine a tool to check if our requirement is fulfilled. Alas many old Unix machines have diff versions that do not support the -a and -q options.

The following script can be a workaround if you have such a limitation.


#!/bin/sh

diff_dir_entry () {
   if ! [ -e $2 ]
   then
      echo "***ERROR: '$1' does not have corresponding '$2'."
   else
      if [ -d $1 ]
      then
         if ! [ -d $2 ]
         then
            echo "***ERROR: '$1' is a directory while '$2' is not."
         fi
      else
         if ! cmp $1 $2 >/dev/null 2>&1
         then
            echo "***ERROR: Files '$1' and '$2' differ."
         fi
      fi
   fi
}

dir_resolve()
{
   cd "$1" 2>/dev/null || return $?
   echo "`pwd -P`" # output full, link-resolved path
}

REF_DIR=`dir_resolve $1`
EXAM_DIR=`dir_resolve $2`

find $REF_DIR -mindepth 1 -print |
perl -sn -e'chomp; $examined_dir_entry=$_; $examined_dir_entry=~s{^$reference_rootdir}{$examined_rootdir}; print "$_ $examined_dir_entry\n"' -- -reference_rootdir=$REF_DIR -examined_rootdir=$EXAM_DIR |
while read DIRS; do diff_dir_entry $DIRS; done




Note:
1) Reference directory entries should not have white-space within their names.
2) Script has been tested with directories and regular files only; yet to test with soft-links.
3) Reference and target directories can be specified with absolute or relative pathnames.
4) If you have noticed, there will be a problem should the target directory be a sub-tree of the reference directory. I will have to include a constraint-check to ensure the user does not specify reference and target directories which are structured as such.


I would be very interested if someone could demonstrate a more compact way of solving this problem while still maintaining portability.