How To Install Checkbot under Windows

Checkbot is a free, open-source application written by Hans de Graaff. It spiders a Web site looking for broken links. This page tells you how to install it under Windows.

Installing Checkbot isn't hard, in fact it's trivial under Unix/Linux. These instructions are a little lengthy because under Windows one has to install a number of tools that are more or less automatically present under *nix. The tools required (tar, gzip and Perl) are all free and available under the very generous Artistic and GNU GPL licensing schemes. Checkbot itself is under the Artistic license.

These notes were tested against Checkbot 1.73. If you use a different version of Checkbot, YMMV.

Installation Prep

Download and install ActivePerl. I chose to install it in c:\perl for simplicity's sake. I'll refer to that path throughout these instructions.
Download Gnu Gzip. When you download and run this executable, it will self-extract into a ReadMe file, some other documentation and gzip.exe. Put gzip.exe into your path, and it is installed. That's right, this amazing software installs without writing to the registry or forcing you to reboot your computer. Extraordinary! What will they think of next!
Download Gnu tar. Tar is even easier to install than gzip. The executable that you just downloaded is the whole shebang. Rename it to tar.exe, stick it somewhere in your path and you're done.
Download Microsoft's nmake. It is also a self-extracing executable. Put the extracted nmake.exe into your path.

OK! Now you've upgraded your Windows machine to something Unix-like ;) An alternative route is to install the suite of free Unix tools for Windows which gives you tar, gzip and a whole lot more but requires a little more work to install. Doing so is left as an exercise for the reader.

Installation

Checkbot makes use of several Perl libraries, specifically LWP 5.50 (LWP = libwww-perl), HTML::Parser 3.10, URI 1.10, and Net::FTP 2.00 (available in the libnet package). ActivePerl (which you just installed) comes with a number of bonus packages in addition to vanilla Perl and thankfully most (or all) of the ones listed above come as part of the ActivePerl distribution. The only one I'm not sure about is Net::FTP which is part of libnet. I downloaded and installed the libnet package before I realized it might be installed as part of ActivePerl.

You can determine what Perl packages you have installed with a nifty tool called PPM (Perl Package Manager) that comes with (is part of?) ActivePerl. Just type ppm at a command prompt. Typing query * inside ppm will show you all of the packages you've got installed. If you're lucky libnet will already be in the list. If not, you'll have to use ppm to fetch it. To install libnet via ppm:

Type search libnet. That will (after some time spent searching remote databases) return a numbered list of hits.
Type install # where # is a number from the list of hits generated in step 1.

Full documentation for ppm is available by typing help or in the ActivePerl FAQ.

Once you've got these libraries sussed out, you're ready to install Checkbot.

Installing Checkbot

Dust off your keyboard; you need to execute most of these steps from a command line prompt. It shows my age, I guess, that I usually refer to this as a DOS prompt.

Download the latest version of Checkbot into c:\perl\lib.
cd c:\perl\lib
Untar Checkbot: tar -xz < checkbot-1.73.tar.gz
Tar creates a directory called CHECKBOT.73 which is wrong. Rename the directory to checkbot-1.73 and delete checkbot-1.73.tar.gz.
cd checkbot-1.73 and observe that the filenames are ALL UPPERCASE BECAUSE THIS VERSION OF CHECKBOT IS FOR THE HARD OF HEARING. Also note that one filename (changelog) got whacked into 8.3 conformance (which is also the reason why you had to rename the directory in the previous step). You should see something like this:
```
Directory of C:\Perl\lib\checkbot-1.73

12/10/03  05:54p        <DIR>          .
12/10/03  05:54p        <DIR>          ..
08/31/03  05:36p                35,343 CHANGELO
08/31/03  05:25p                54,500 CHECKBOT
06/28/03  02:18p                 1,160 CHECKBOT.CSS
12/01/02  05:19p                 3,330 MAKEFILE.PL
12/27/02  11:25a                    74 MANIFEST
12/01/02  05:00p                 1,600 README
12/10/03  05:54p        <DIR>          T
08/13/03  09:15p                 6,184 TODO
              10 File(s)        102,191 bytes
                          4,382,733,824 bytes free
```
I think this is an artifact of tar and gzip's origins in a case-sensitive world and operating in a case-insenstive environment. In any case (pun intended), you should fix this. I'm not strictly sure it is necessary to do so, but Perl and nmake complained when I didn't. They're your friends; keep 'em happy.

Rename the files to the following. (Be aware that some versions of Windows Explorer don't show true filenames. If they're uppercase, Explorer will Initcap them. You need to go to a DOS command prompt to see what the files are really called.)

12/10/03  05:54p        <DIR>          .
12/10/03  05:54p        <DIR>          ..
08/31/03  05:36p                35,343 changelog          <-- changed
08/31/03  05:25p                54,500 checkbot           <-- changed
06/28/03  02:18p                 1,160 checkbot.css       <-- changed
12/01/02  05:19p                 3,330 Makefile.PL        <-- changed
12/27/02  11:25a                    74 manifest           <-- changed
12/01/02  05:00p                 1,600 README
12/10/03  05:54p        <DIR>          t                  <-- changed
08/13/03  09:15p                 6,184 TODO
              10 File(s)        102,191 bytes
                          4,382,733,824 bytes free

As of version 1.73, Checkbot can't deal with proxy servers that demand credentials. (This might change in later versions of Checkbot. If you get a version more mature than 1.73, check the release notes.) With 1.73, in order to run in an authenticated proxy environment, you need to hack Checkbot a little. Use Notepad to open the file checkbot (which is Perl code) and find this bit of code in the sub called performRequest:
my $ref_header = new HTTP::Headers(%header_hash); my $request = new HTTP::Request($try, $url, $ref_header); $response = $main::ua->simple_request($request);
After the line beginning my $request (which declares a new variable and instantiates an object), add the following code fragment:
# *** begin mod Your Name 11-dec-2003 ********************************** $request->proxy_authorization_basic($ENV{'http_proxy_user'}, $ENV{'http_proxy_password'}); # *** end mod Your Name 11-dec-2003 **********************************
Save the file. w00t! Look who's a Perl hacker!

Get ready to build Checkbot by swtiching back to the DOS prompt. Type

perl 
Makefile.PL

. You should get:

Checking for LWP........... ok
Checking for URI........... ok
Checking for HTML::Parser.. ok
Checking for MIME::Base64.. ok
Checking for Net::FTP...... ok
Checking for Digest::MD5 .. ok
Checking for Mail::Send .. failed
Can't locate Mail/Send.pm in @INC (@INC contains: C:/Perl/lib C:/Perl/site/
) at C:\Perl\lib\Checkbot-1.73\MAKEFILE.PL line 128.


Mail::Send is used to allow Checkbot to send email. You currently
can not use the --mailto option of Checkbot. Mail::Send can be found
in the MailTools package.

The missing modules can be obtained from CPAN.  Visit 
<URL:http://www.perl.com/CPAN/> to find a CPAN site near you.


Writing Makefile for checkbot

Note that the error above informs us that the optional "mailto" feature of Checkbot can't operate because we haven't supplied the Mail::Send package. I don't cover installation of that package, so if you need the mailto option, you'll have to figure it out own your own.

At the DOS prompt, type nmake. Nmake will respond with:


Microsoft (R) Program Maintenance Utility   Version 1.50
Copyright (c) Microsoft Corp 1988-94. All rights reserved.

        C:\Perl\bin\perl.exe -MExtUtils::Command -e cp checkbot blib\script
	\checkbot
        pl2bat.bat blib\script\checkbot

Type nmake install.

Microsoft (R) Program Maintenance Utility   Version 1.50
Copyright (c) Microsoft Corp 1988-94. All rights reserved.

        C:\Perl\bin\perl.exe -MExtUtils::Command -e cp checkbot blib\script
	\checkbot
        pl2bat.bat blib\script\checkbot
Installing C:\Perl\bin\checkbot
Installing C:\Perl\bin\checkbot.bat
Appending installation info to C:\Perl\lib/perllocal.pod

cd \
Type checkbot and checkbot will respond with its command-line options. You're good to go!

Tips

Authenticating At The Proxy. Checkbot 1.73 doesn't know how to deal with authenticated proxies. But thanks to the bit of code you added, Checkbot can read your credentials from environment variables. Before running Checkbot, type the following at the DOS prompt:
set http_proxy_user=your_userid set http_proxy_password=your_password
Note that setting environment variables from the command line as demonstrated above affects only that session, not the whole Windows environment. As soon as you close that DOS box, your changes disappear. This means that you have to run Checkbot in the same DOS box in which you changed the environment, but you're not creating a security risk by adding your userid/password to the machine's permanent environment information.
Regular expressions. Checkbot's match parameter uses regular expressions so you might be interested in this nifty tutorial on regular expressions.
The DOS prompt. Remember that DOS command prompts treat some characters as special and they're liable to trip you up if you don't realize they're being treated as metacharacters. The solution is to enclose the offending string in quotes. For instance, the ampersand & is a metacharacter, and if it appears at the end of your search URL, DOS will give strange "Bad command or filename" errors. Putting quotes around the URL solves the problem.
Bad:
checkbot http://MyCorporateIntranet.com/blahblahblah.asp?UserID=437070&
Good:
checkbot "http://MyCorporateIntranet.com/blahblahblah.asp?UserID=437070&"
Quotes never hurt, so if you're not sure whether a string contains troublesome metacharacters or not, stick it in quotes to be on the safe side.

Home :: Philip's Stuff :: Installing Checkbot under Windows