<cid:part1.06070505.00030407@unimacq.edu.au>
Title: How to block the loading of certain files from remote sites
How to block
the loading of certain files from remote sites
This is a documentation for how to block
other sites from leaching or ripping files from your server, consuming
your bandwidth, for users of the great Apache
HTTP daemon. Also known as Remote Loading, ever since Tripod started
blocking it, it seems everyone is wondering how they did it so they can
do the same for their server. I do not know how Tripod does it, but
found out how to atleast block
remote loading. I have searched for how to do this, and found nothing
but people wondering how. So I figured it out myself and puting it up
on the web so others can find out how to do the same. I started by
looking for modules/external progs, but then figured out how to do it
with just some configuration tweaks. Although I acually tested this on
a Win32 [barf] port of Apache,
it should work on the Linux/UNIX versions as well, as long as you have
the required modules (most likely default already) configured in when
you built it, or commented out if you installed it pre-compiled. I
tried this out on Apache
1.3.27 and it worked fine!
Of interest to most people, is to deny images from remote loading -
that is, someone who links to an images on your site from another site.
Images are desired to only be allowed if they are viewed on your own
pages, not someone else's. Another value is the blocking of files that
you do not want ripped, such as downloads or other large demand files
that you want to be accessible to members of your site and not remote
people linking to it. There are surely many ways to do this, and this
is but one. It is fairly simple and straight forward. You can change
the names and sites to your own liking to suit your needs, as well as
reverse the negatives around.
If installed from binary, make sure the following is uncommented. If
compiled, make sure mod_setenvif was added.
AddModule mod_setenvif.c
You will need to set where you want the site to load.
SetEnvIfNoCase Referer www\.monobasin\.net LOCAL_REFERER
So the above sets the environment variable `LOCAL_REFERER' true if
the referer in the HTTP header is `www.monobasin.net' and although it
seems to work fine with the `.' unescaped, Apache's
documentation uses them. You would want it to not be case sensitive
since domain
names are not. This way www.MonoBasin.net will set LOCAL_REFERER to
true as well as www.monobasin.net would.
<Files "*.jpg">
Order deny,allow
Deny from all
Allow from env=LOCAL_REFERER
</Files>
Then add in the above, substituting what you want blocked. This will
forbid all loading of files ending with `.jpg' unless they are referred
to by www.monobasin.net in the reverer.
You may want to add in another variable for your host without the
`www' if you enable such with your server. You can also set some more
allowable sites, such as `localhost' or some friends sites. Chances
are, you will more than likely need something like this:
SetEnvIfNoCase Referer localhost LOCAL_HOST
SetEnvIfNoCase Referer monobasin\.net LOCAL_REFERER
SetEnvIfNoCase Referer poopia ALLOWED_REFERER-0
SetEnvIfNoCase Referer www\.my-friends-site\.org ALLOWED_REFERER-1
SetEnvIfNoCase Referer www\.*\.nice-domains\.com ALLOWED_REFERER-2
# ...
<Files "*.jpg">
Order deny,allow
Deny from all
Allow from env=LOCAL_REFERER
Allow from env=LOCAL_HOST
Allow from env=ALLOWED_REFERER-0
Allow from env=ALLOWED_REFERER-1
Allow from env=ALLOWED_REFERER-2
</Files>
<Files "*.gif">
Order deny,allow
Deny from all
Allow from env=LOCAL_REFERER
Allow from env=LOCAL_HOST
Allow from env=ALLOWED_REFERER-0
Allow from env=ALLOWED_REFERER-1
Allow from env=ALLOWED_REFERER-2
</Files>
<Files "*.png">
Order deny,allow
Deny from all
Allow from env=LOCAL_REFERER
Allow from env=LOCAL_HOST
Allow from env=ALLOWED_REFERER-0
Allow from env=ALLOWED_REFERER-1
Allow from env=ALLOWED_REFERER-2
</Files>
In this case, you are blocking multipule file types from being
remotely loaded, except from several sites that you do not mind it
from, as well as various names for your own server. With this case,
your machines name is `poopia' and is listed because it is good to add
it in there so users on your LAN can use that name for your site
instead of the fully qualified domain
name. There is also allowing remote loading from all domains under
`nice-domains.com' as well. Note that you can change the format of the
above `<Files> directive to be in one line. I just find it
suitable in multiple lines, it is easier to read and edit. For instance
<Files "*.png">
Order deny,allow
Deny from all
Allow from env=LOCAL_REFERER
Allow from env=LOCAL_HOST
Allow from env=ALLOWED_REFERER-0
Allow from env=ALLOWED_REFERER-1
Allow from env=ALLOWED_REFERER-2
</Files>
is the same as
<Files "*.png">
Order deny,allow
Deny from all
Allow from env=LOCAL_REFERER env=LOCAL_HOST env=ALLOWED_REFERER-0 env=ALLOWED_REFERER-1 env=ALLOWED_REFERER-2
</Files>
Incase you are running virtual domains, or just simply do not want
it globally, you can put it within directives that narrow it down to
just what you want (or atleast close to it). For example, I recently
had to turn these one because I was getting about 20,000 hits a day of
leeched images, and it was dropping my half-duplex, 128k BIT/sec (bit,
not byte - yes it is that cheap) always-changing IP'd server here in my
apartment. All of the images being leeched were from the /warhawkfire
part, a friend's site I have setup. So I set up the following, which
works perfectly:
<Directory /www/warhawkfire >
<Files "*.gif">
Order Deny,Allow
Allow From ENV=LOCAL_REFERER
</Files>
</Directory>
A note on wildcards
Note that although you can use
wildcards, the string is still not absolute unless the entire string is
inputed. For example, if you wanted to block
all loading of `*.gif' files that were requested by a page x/g.htm (I
actually do) that could be from any
server server, all is needed is x/g/htm, not */x/g.htm. Wildcards
almost are always added in anyway, since the referer statement is
always variable, even from the same server, since it can be requested
by a new page. I do not particularly like how it is setup that way, and
would much rather have to enter in the full, absolute referer, with
wildcards, to satisfy it, such as http://www.monobasin.net/*. So as I
was saying earlier, the `*' is implied. If you had the following:
SetEnvIfNoCase Referer x/g.htm XGHTM
<Files "*.gif">
Order allow,deny
Deny from env=XGHTM
Allow from all
</Files>
that will block
loading of files ending with `.gif' that have anything
followed by x/g.htm. Setting the variable to */x/g.htm will cause an
error. However, /*/x/g.htm will not. Not the way it should be, but this
is how it is. (at least for Apache
1.3.30).
A note on security
It is worth mentioning incase there are some of you out there who think
that this will work in all cases. As with anything, cracking is
inevitable, and as I look at it - I am human and so is the other
person, so in reality, anything I make is capable of being cracked.
Period. And that goes for everything anyone else makes too. It would
take something made by someone other than human to make something
humans cannot figure out. Just like what we [sadly] do to many other
species. So what am I saying? Basically the protection this provides is
EXTREMELY WEAK, because it relies on the reverer part of the HTTP
request. This is easily spoofed, and to make things a little bit better
than what is listed here, but is more system resource intensive, is to
do a reverse DNS lookup of the host you are blocking for the IPA, and
see if it is really them. This would stop people from setting a simple
entry of your sites name in their hosts file, and on there own web
server (if they are running one locally) and type in your site - their
site shows up, but links to yours will satisfy your referral
check and give them the requested files. So if there was the added
protection of denying if the referer is true, but their IP is not
yours. Better than the Apache
configuration, the best is to have your firewall drop packets that come
from the Internet with your IPA, since this only [should] happen if you
request something from your own site, from your site, and DNS servers
are on the other side of the firewall. The IPA part of the request is
spoofable too, but a little harder. Enough though, you get the message.
If you do not want someting to be available to others, do not make it
available to others! Puting it on a public network, or a network
attatched to a public network, is making it available.
A note about different browsers
There are some issues with
certain browsers that I have noticed. Most notably, some do not keep
sending the referer for each request, just the first, and all
postceding requests are sent without a referer. So if you have images
remotely blocked and someone tries to view an image that is `legal' and
referred to from one of your site's own pages, it will load without
problems. However, if that person then does a refresh/reload, they will
get a 403 forbidden error. One browser I have noticed to to this is the
Konqueror web browser.
Other Points
When I initially looked to see if anyone knew how to block
out remotely referred requests, I saw much talk about Tripod, and how
do they do it. The way I came up with is pretty much the same, and does
not use any extra progs. The only thing different is what is returned
to the user who tries to remote load. Tripod gives an image that has
thier own logo and words. Although I never bothered, if you wanted to
do the same, you might be able to do so by using a custom error
document instead of having the standard 403 being used for these, and
then set "content-type: image/jpeg\n\n" in it, followed by the contents
of some jpeg image you wanted to show.
It is a good idea to maybe include something for some or all .cgi
Hope you find this usefull,ea
|