Blocking Software FAQ

  1. What kind of sites are blocked by the different blocking programs?
  2. How can I get a list of sites blocked by a given censorware program?
  3. I heard that a site was blocked by a particular program, but when I tested the program, it said that the site was not blocked. What happened?
  4. How can I find out if my site is blocked by any of the programs?
  5. Why is it that "keyword blocking" cannot really be turned off?

What kind of sites are blocked by the different blocking programs?
A: The controversy over blocking software does not center on the blocking of chicken breast recipes, breast cancer information, Anne Sexton, or "Superbowl XXX". It is true that these sites are accidentally blocked by blocking software programs that scan pages for certain keywords, as almost all of them do. However, the controversy centers on sites that are blocked not accidentally but deliberately. These are URL's that come pre-included on the list of sites to be blocked by the program, regardless of the content of the pages themselves. Some examples:

Q: How can I get a list of sites blocked by a given censorware program?
A: You cannot get a list of blocked sites by downloading a trial copy of the program. Even though most censorware programs have free-trial versions that come with a copy of the blacklist, the list is stored in an encrypted file that is not supposed to be readable to the user. The only censorware program that does not encrypt their blocked site list is
Net Nanny, however, their blocked site list is not available with the free trial version.

Your first option is trial and error. You can download the program and try to access different sites to see which ones are blocked. This is how Peacefire came up with lists of sites that were blocked by the different censorware programs when we examined them. You can go to http://www.peacefire.org/censorware/ and find the program on that page, for a list of some of the sites that were blocked when we tested it.

There have also been several cases where the encryption on a particular program's blacklist was broken, and the entire list of blocked sites was posted to the Internet. Peacefire broke the encryption on CYBERsitter's blocked site list and published a program called CSDecode in April 1997 that could be used to decrypt the list of sites blocked by CYBERsitter 2.12. Anyone could get a list of sites blocked by CYBERsitter by downloading CYBERsitter, downloading our CSDecode program, and running it against CYBERsitter's list of blocked sites.
You can get the entire list of sites blocked by CYBERsitter at:
http://www.xs4all.nl/~mjk/cybersitter.html
and the entire list of sites blocked by Net Nanny at:
http://www.xs4all.nl/~mjk/netnannysites.html
(even though the Net Nanny list didn't have to be decrypted). Peacefire is not affiliated with these sites; the URL's themselves have information on how to contact the author.

In March 1997, TIME Magazine online posted a tool called the "Censorware Search Engine", created by journalists who managed to obtain decrypted copies of the blocked site lists used by Cyber Patrol, SurfWatch, Net Nanny, CYBERsitter, and X-Stop. The user could enter a keyword and the search engine would reply with a list of URL's from the different blacklists that contained that keyword. CYBERsitter added TIME magazine to their list of blocked sites, apparently in retaliation. But the Censorware Search Engine was taken permanently offline in about August 1998 during an overhaul of the TIME Digital Web site.

Q: I heard that a site was blocked by a particular program, but when I tested the program, it said that the site was not blocked. What happened?
A: The company may have removed the site from their list since it was discovered to be blocked. Even if the company has removed the site from their blacklist, all users of the program have to download the latest version of the blacklist before it will take effect on their computer. Depending on who told you that the site was blocked, it might have been that they received an error of a different kind (e.g. "404 File Not Found" or "403 Forbidden") and thought that it was caused by the blocking software.

All blocked sites listed on Peacefire.org were obtained through trial and error, by testing the latest available copy of the program's blacklist (unless -- in the case of CYBERsitter -- we were able to break the encryption on the blacklist and examine it directly). In some cases, if we were worried that the company might un-block a site after our report was published and deny that the site was ever blocked in the first place, we asked reporters to go through the list of sites and verify that they were blocked, before our page was published.

Many examples of blocked sites listed on our pages are there to demonstrate that the censorware company is not reviewing pages before blocking them, such as the Vatican site blocked by X-Stop or the Breast Cancer Legislation site blocked by BESS. Even if these pages are un-blocked after the discovery is made public, they still would not have been blocked in the first place if the sites had been reviewed by a human first.

Q: How can I find out if my site is blocked by any of the programs?
A: The following three companies have published "lookup forms" on their Web sites where you can enter the URL of your page, and the form will tell you whether it's blocked or not:

Note that some of these forms may return incorrect information. For example, if a site is blocked automatically by Cyber Patrol because it has the word "sex" in the URL, the site may still be listed as "not blocked" if you enter the URL in the form on their site. To find out for sure if a program is blocking a Web site, you should download and install the software.

To find out whether your site is blocked by any other program, downloading the program and installing it is the only choice you have. Of all the companies that offer free downloads of their programs, the trial versions block all of the same sites that the full versions block (except for the trial versions of Net Nanny and WebSENSE).

Some programs -- usually proxy servers such as BESS, I-Gear and SmartFilter -- do not have trial versions that you can download, since they are for use by schools and companies but not by home users. In that case, you have to contact the company to ask them if they are blocking a Web site, or contact someone at a company or school that is using the software, and ask them to test the URL for you.

Q: Why is it that "keyword blocking" cannot really be turned off?
A: Normally, a site is blocked if (1) the URL is on the program's internally stored "blacklist" or (2) the program detects certain keywords on the page and blocks it automatically, whether the URL is on the blacklist or not. When a censorware program includes an option to "turn keyword blocking off", this means that condition (2) no longer applies, so the site will only be blocked if it's on the blacklist. Some users have wondered if the really embarrassing errors caused by blocking software -- such as blocking sites about breast cancer -- can be avoided if you disable keyword blocking.

The reason this doesn't work is because censorware companies use keywords to generate the blacklist itself. X-Stop, for example, uses a program called "MudCrawler" to search the Web for pages containing words like "xxx" in the title. As a result, sites are added to the blacklist that probably would not have been added if they had been reviewed by a human first. The Quakers home page and the AIDS Memorial Quilt were both discovered to be on X-Stop's blacklist. Cyber Patrol, which developed their own program called CyberSpider similar to X-Stop's MudCrawler, blocked a youth soccer league, MapleSoccer.org, because of a page that listed the teams in the categories "Boys under 12", "Boys under 14", etc. These sites were blocked even on computers where keyword blocking was disabled, because they were on the program's blacklist.

bennett@peacefire.org