Zapping Ads with Squid and adzapper

agent007 · Post by **agent007** » Wed Apr 23, 2003 10:33 pm

Squid is a very powerful caching proxy server that is capable of doing much
more than filtering ads but I will limit this tip to just the ad filtering.
Some of the other uses are user authentication and access restriction to the
Internet. In my house when you open any browser on any machine it pops up with
an authentication box. If one of my kids enter their ID and password they are
restricted to where they can go. My wife and I have unrestricted access. I
have written a nice little interface to easily add or remove sites from
several access lists.

Can u pls tell me as to how u filter out the bad sites VoidMain? I'm guessing
that u can also block sites that contain certain "terms" etc....Where will I
find those access lists? Also, is it possible to block certain images?

Like http://www.test-server/some-image.jpg

Hope u have a small tutorial on this.

thanks & rgds,
007

Void Main · Post by **Void Main** » Wed Apr 23, 2003 11:12 pm

I don't have a tutorial for it and there are ways to filter based on words but I don't use that type of filter. I just create my own access rules. My kids are limited to only being able to access sites that I have on the list of acceptable sites. I also have a list of sites that are banned no matter who you are (microsoft.com, msn.com, etc). I keep the banned sites or allowed sites in separate text files. Then I have a couple of rules that point to those files in my squid.conf:

squid.conf exerpt:

Code: Select all

acl bannedsites dstdomain "/etc/squid/acl/banned.sites"

acl unrestricted_users proxy_auth "/etc/squid/acl/unrestricted.grp"

acl kidsgrp proxy_auth "/etc/squid/acl/kids.grp"
acl kidsites dstdomain "/etc/squid/acl/kids.sites"
acl kidips dst "/etc/squid/acl/kids.ips"

http_access allow manager
http_access allow SSL_ports
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny badips

http_access allow unrestricted_users !bannedsites
http_access allow kidsgrp kidsites !bannedsites
http_access allow kidsgrp kidips !bannedsites

http_access allow localhost
http_access deny all

As you can see I created a directory /etc/squid/acl that contain my site and user files that I use in my access rules.

The "banned.sites" file excerpt:

Code: Select all

.doubleclick.net
.hitbox.com
.microsoft.com
.msn.com
.penthousemag.com
.unixsux.com
3ps.go.com
homepage.mac.com

The "unrestricted.grp" (auth usernames) excerpt:

Code: Select all

voidmain
thewife

The "kids.grp" (auth usernames) excerpt:

Code: Select all

youngvoid
youngvoidette

The "kids.sites" excerpt:

Code: Select all

.foxnews.com
.supercross.com
games.yahoo.com
sports.yahoo.com

The "kids.ips" excerpt:

Code: Select all

10.0.0.0/255.0.0.0
192.168.0.0/255.255.0.0

Of course you need to have a proxy authentication method set up in order to use this. You could set up an htaccess type of authentication using ncsa_auth or if you are running a Samba domain you can use smb_auth. Find the "authentication_program" section in your squid.conf. You should find several auth programs in /usr/lib/squid, just pick the one you want to use and you'll have to read a few docs on how to use it.

I also have written a CGI program to be able to easily add/remove users to the kids or unrestricted groups and add/remove sites from the banned and kids site files. I just go to http://proxy.voidmain.home/ and enter my password and it runs my CGI program. It automatically does a "service squid reload" after a file is modified. Of course there is nothing wrong with using "vi" on these files directly and reloading squid.

Hope that is roughly what you were asking about...

agent007 · Post by **agent007** » Thu Apr 24, 2003 4:18 am

Ok, lets say I fall into the "unrestricted.grp"....How is it possible to block porn stuff? Do I have to manually enter URL's into the "banned sites"? That would be really tedious......

thanks,
007

Void Main · Post by **Void Main** » Thu Apr 24, 2003 8:15 am

Yes you would with the way I do it. But it's exactly what I want and I put very few things in the banned list. I don't block porn sites, I just don't go to them. Trusted people go in the unrestricted group who are adult. It's more for places I want to boycott than a sweeping ban on a certain type of site. And then of course all advertisements are blocked via the ad-zapper script. I believe there is a similar thing for porn sites. You can add multiple redirect scripts. I'm sure some searching on google will find more redirect scripts to do just about anything you want.

agent007 · Post by **agent007** » Fri Apr 25, 2003 12:53 am

You can add multiple redirect scripts.

Yes!! Thats just what I wanted to know...U really got it right VoidMain. Do I separate the multiple redirects with a comma? and does it have to be in the same line?

thanks,
007

Void Main · Post by **Void Main** » Fri Apr 25, 2003 1:29 am

For some porn blocking see section 10.12, looks like some are acl based and some are redirect script based:

http://www.squid-cache.org/Doc/FAQ/FAQ-10.html

As far as multiple redirectors:

http://www.squid-cache.org/mail-archive ... /0199.html

May the force be with you...

[edit]
I suppose if you are a pr0n freak this would be a great way to come up with a good collection:
http://www.squidguard.org/blacklist/
[/edit]

agent007 · Post by **agent007** » Fri Apr 25, 2003 2:19 am

Right, so I want to block certain images..But haven't proved to be successful so far..Also, I dont want to create separate users or anything. The setup will be common for all.. Following are extracts from my config files....

SQUID.CONF

acl banned_sites dstdomain "/etc/squid/acl/banned.sites"
http_access deny banned_sites
http_access allow all

BANNED.SITES

http://www.fsckmicro$oft.com/images/fms_logo.gif

So, something is wrong somewhere.....but where?

thanks,
007

Void Main · Post by **Void Main** » Fri Apr 25, 2003 2:46 am

banned.sites must contain items like this:

Code: Select all

.somedomain.com
www.somefullhostname.co.uk

"dstdomain" means pretty much what it says, it blocks domains not URLs. To block URLs you probably want to use "url_regex" rather than "dstdomain":

http://www.squid-cache.org/Doc/FAQ/FAQ-10.html#ss10.4
http://www.squid-cache.org/Doc/FAQ/FAQ-10.html#ss10.16

agent007 · Post by **agent007** » Fri Apr 25, 2003 3:45 am

VoidMain,

U are a real whizz!! THanks a tonne! I'm really glad I got it going!!!!!!!!!

Squid is really great, the configuring and stuff is extremely powerful.....

thanks,
007

Void Main · Post by **Void Main** » Fri Apr 25, 2003 5:23 pm

Just a note about a link I provided above regarding the "multiple redirectors". That link does not work. However, there is a way to do it and I am in the process of chaining AdZapper for advertisements and squidGuard to filter porn, warez, etc, etc. Here is the link on the adzapper site on how to set it up:

http://adzapper.sourceforge.net/#chaining

Sorry about giving you bad info. I'll update my Ad Zapper page to include the porn zapping by chaining these two together as soon as I get everything set up right.

[edit]
squidGuard is really nice and blocks a lot of porn, warez, violence, etc. However, a quick google search and I found porn sites that it did not block so it's still not safe enough to let the kids roam free in my opinion. I still have them restricted to specific sites. If they need to go to other places for homework either I or my wife will enter our id/password into the browser auth and keep an eye on them.

I wish it were safe enough to just let them go but there is just too much stuff out there that I don't think they need to get into until they get older (if ever). Yes, I'm a Nazi sensor. Maybe I'll try some of the other porn block stuff.
[/edit]

Void Main · Post by **Void Main** » Fri Apr 25, 2003 10:49 pm

Ok, now I've got two redirectors and many ACL based block files going. It does a very good job of blocking bad stuff and allowing good stuff. But it's not perfect, and perfect is not good enough for me. I still don't have quite what I would like as with a mostly open system you are going to have some stuff slip through. I just can't afford to have that happen with my kids so I have to keep it a mostly closed system for the kids. One thing slipping through is unacceptable.

What we really need is a list of known good sites that do not contain porn which would be a huge list. Better yet make it work like "RBL" for Sendmail. I have seen mention for these types of lists but I believe they are all pay for services. Maybe I'll have to start my own....

Void Main · Post by **Void Main** » Sat Apr 26, 2003 12:43 am

BINGO!! I think I found the solution I've been looking for. It's a true content filter that works in conjunction with Squid. In fact it wouldn't even let me view this thread because of a weighted combination of the words "porn", "warez", etc... It let me view the other threads and it really seems to work quite well. I am so impressed that I believe I will unleash it on the kids sometime this weekend which should make them very happy. Here's the link:

http://dansguardian.org/

I'm using v2.6.0. Now I'm hoping I can somehow make it slightly less restrictive for me. I think I will have no problem doing this in the squid config if I can't in the DansGuardian config. DansGuardian listens on port 8080 by default and then talks to Squid on port 3128 (or whatever you have Squid configured to listen on). You set your proxy client setting to 8080 instead of 3128. I could set my browser to 3128 and the kids to 8080 and have it in the Squid config that users in the "kids" user group can only talk to squid from localhost. This would prevent them from setting their browser to 3128 as they would be blocked from going anywhere.

Now I have like 5 layers of blocking, pretty dang cool!

Tux · Post by **Tux** » Sat Apr 26, 2003 10:16 am

Hey void what kids you have?
Well, human obviously, but how many/how old?

Ja, i'm nosy.

Void Main · Post by **Void Main** » Sat Apr 26, 2003 10:30 am

I have two, one of each. One smart, one dumb, I mean one girl and one boy, they are actually both smart, as in 4.0 straight through every grade so far for both of them along with both of them being in the quest class (or whatever they call it). So I have to stay on my toes if I want to keep ahead of them which is getting harder to do each day. It wouldn't surprise me if some of the members of this forum are younger than my daughter who is a freshman (9th), my son is two grades behind her (7th).

Tux · Post by **Tux** » Sat Apr 26, 2003 10:36 am

Have you got 'em using Linux?