Re: [cobalt-users] .Glmpse (was Qube 2 hang solution found)

[Mike Vanecek <nospam99@xxxxxxxxxxxx>]

On Sun, 18 Nov 2001 10:01:14 +1100, Malcolm McLeary <mmcleary@xxxxxxx>
wrote:

:>> Any reason you took that approach as opposed to editing the
:>> .glimpse_include and .glimpse_exclude files?
:>
:>I felt that what I did restricted the scanning and traversing directories.
:>It was my understanding thattThe include and exclude files would only get
:>invoked after it scanned to determine whether the info was to be included in
:>the index.  
:>
:>I wanted to restrict the scanning process not just what was in the index.
:>
:>Cheers,  Malcolm

I made your recommended changes, i.e., 

# add directories to index below as needed (within quotes)
# my $gIndexDirs = '"/home/users" "/home/groups"';
my $gIndexDirs = '"/home/groups/home"';

However, I also made the following changes:

This one requires rebuilding the index from scratch thereby removing
deleted files from the index:

# glimpseindex command
# -f for incremental indexing
# -X for HTML titles
# my $gCommand = "/usr/bin/glimpseindex -f -H $gGlimpseIndexPath -X
$gIndexDirs";
my $gCommand = "/usr/bin/glimpseindex -b -H $gGlimpseIndexPath -X
$gIndexDirs";

This one removes any frontpage directories from the index. Note that
.glimpse_exclude needs to be deleted so that the below code will cause
it to be rebuilt.

# create the .glimpse_exclude if necessary
my $glimpseExclude="$gGlimpseIndexPath/.glimpse_exclude";
if( ! -e $glimpseExclude ) {
    open( EXCLUDE, ">$glimpseExclude" );
.
.
.
    # exclude frontpage stuff
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/_private\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/_vti_bin\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/_vti_cnf\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/_vti_inf.html\$\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/_vti_log\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/_vti_pvt\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/_vti_txt\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/cgi-bin\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/images\n";
#    print EXCLUDE "^$Htgroup_dir/$MainGroup/post_info.html\$\n";
    print EXCLUDE "/_private\n"; 
    print EXCLUDE "/_vti_bin\n";
    print EXCLUDE "/_vti_cnf\n";
    print EXCLUDE "/vti_inf.html\$\n";
    print EXCLUDE "/_vti_log\n";
    print EXCLUDE "/_vti_pvt\n";
    print EXCLUDE "/_vti_txt\n";
    print EXCLUDE "/cgi-bin\n";
    print EXCLUDE "/images\n";
    print EXCLUDE "/post_info\n";
    close( EXCLUDE );
}

This has significantly reduced the size of the files in .glimpse.
Further, my .log file now has zero errors. My index went from 1.5GB to
2K with the above changes.

Thank you kindly for the help. I strongly recommend you take a look at
the -b option as opposed to the -f option.

Further, anyone making these changes should do a ./buildSearchIndex.pl
to test them before letting cron run its daily update.

Mike.