Apache Restricting Content From Download

The Scenario

After releasing our latest Youtube video https://youtu.be/QWjub-nKNL4 we had some extra content that we wanted to share, but not allow download, since it was mentioned in the video.

What to do, what to do.  Previously I tried looking for options on how to stop the right click download, but that wouldn't stop people using other ways to download the content, and owning the web server wanted a way that would restrict the straight forward methods of downloading the content and only allowing it to be streamed using the player on the web site.

A few Apache configuration lines later, and the use of JWPlayer I solved my problem.  The joys of having your own web server - which means you can't do this if you're on a hosted platform.

Most players are client based, so the requirement to not have direct access to the file is something I'm very familiar with, but didn't want to write my own player.  In the past I've streamed Word documents without the URL being available so that clients had to be logged in to access the document, and never knew the real URL, but this time I want to make sure the content is not downloadable directly.

The Research

First I needed to see if anyone had worked out how to prevent download of files in Apache (httpd), so a quick Google search for "apache htaccess deny download of specific files" came up with;

• https://wordpress.stackexchange.com/questions/91757/prevent-users-from-accessing-mp3s-in-my-uploads-folder
• https://www.svnlabs.com/blogs/how-to-prevent-downloading-and-leeching-media-files/

The answer code;

RewriteEngine On
RewriteCond %{HTTP_REFERER} !^http://(www\.)?yourwebsite\.com/ [NC]
RewriteCond %{REQUEST_URI} !hotlink\.(mp3|mp4|mov) [NC]
RewriteCond %{HTTP_COOKIE} !^.*wordpress_logged_in.*$ [NC]
RewriteRule .*\.(mp3|mp4|mov)$ http://yourwebsite.com/ [NC]

From this the lines I needed would be;

• RewriteEngine On
• RewriteCond %{HTTP_REFERER} !^http://......
• RewriteRule .*\.(mp3|wav)$ ....

I would need to make some modifications to this, but the .htaccess file was the way to go, since other directories I still wanted people to download from.  So some extra help from https://300m.com/stupid-htaccess-tricks/ to understand what goes on in the square brackets.

Secondly, because I wanted to use the .htaccess file I would need to modify my Apache configuration to AllowOverride All for the root directory.

Last but not least a player that would allow the media to be played through the web browser - https://www.jwplayer.com/.

Configuring Apache

To ensure Apache allows you to use the .htaccess files you need to change the main configuration files.  If you intend to be able to do different things in different directories then you should set your DocumentRoot so that the Directory setting has AllowOverride  All instead of the usual AllowOverride None.

Excerpt from the httpd.conf file;

DocumentRoot "/usr/local/apache2/htdocs"

<Directory "/usr/local/apache2/htdocs">

    Options Indexes FollowSymLinks

    IndexOptions FancyIndexing

    DirectoryIndex index.html

    AllowOverride All

    Require all granted

</Directory>

You should also ensure that the following module is enabled in the httpd.conf file;

LoadModule rewrite_module modules/mod_rewrite.so

This will ensure that the directives we add to the .htaccess file will work. 

Denying Download

Now locate the directory in you web server where you want to restricted the download of the content.  In this directory we will add the .htaccess file to prevent our MP3 and WAV files being downloaded.

The content of the .htaccess file;

RewriteEngine on

RewriteCond %{HTTP_REFERER} !^http://(www\.)?tps\.local [NC]

RewriteRule .*\.(mp3|wav)$ - [NC,F,L] 

The Lines Explained

RewriteEngine on

This line ensures that Apache can perform rewrites to the HTTP headers and allow the other 2 actions to work correctly.

ReWriteCond

This line checks that the REFERER in the header is tps.local for this example, but should be your domain name.  This ensures that only requests from your own domain are allowed.  The (www\.)? state that the www part might not be supplied.

The [NC] at the end of the line makes the whole statement case insensitive = No Case.

RewriteRule

This line defines what extensions we will be denying the download of.  In this case .mp3 and .wav.  The regular expression uses the or | notation to allow us to supply a list of mp3|wav  which is the same as saying mp3 or wav.  The .*\. before this will be anyfile name before the final full stop.  The $ at the end specifies the the mp3 or wav are the last characters at the end of the data.

The - symbol is there to replace the mp3 or wav file names with just an invalid filename of -.

Again at the end we see NC for case insensitive, F for forbidden to drive the 403 and L being that this is the last rule, don't do anything more.

Testing Download

Firstly we can try a successful download by using wget or curl or our web browser.

We should ensure an html file in the directory, and that should be displayed;

curl http://www.tps.local/index.html

It Works!

Now the mp3

curl http://www.tps.local/my.mp3

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<html><head>

<title>403 Forbidden</title>

</head><body>

<h1>Forbidden</h1>

<p>You don't have permission to access /media/meditation/body.mp3

on this server.</p>

</body></html>

Using JWPlayer

First you need to get yourself an account on JWPlayer.

Log in and from the dashboard menu on the right select Players;

By default you will have 2 example players.  Select one that suits your needs and make modifications to it;

Change items such as;

• Size
• Playback
• etc

Save your player and make a note of Cloud-Hosted Player Library URL as you will need this in your JavaScript code.

The following HTML is an example of using the player with a single file, let's say this is called index.html to make it easier to have the player launch when people enter the restricted directory;

<html>

<body>

<div id="myElement"></div>

<script type="text/JavaScript" src="https://cdn.jwplayer.com/libraries/example.js"></script>

<script type="text/JavaScript">

    jwplayer("myElement").setup({ 

        "playlist": [{

                "file": "my.mp3"

        }]

    });

</script>

</body>

</html>

Change the src to the URL in your JWPlayer account.  The file will play using the player.

For further customisation of JWPlayer see;

• https://developer.jwplayer.com/jwplayer/docs/jw8-javascript-api-reference
• https://developer.jwplayer.com/jwplayer/docs/jw8-javascript-api-reference#section-playlist  (shows you the attributes for the playlist in setup)

 

 

Terraform Runtime Data

Terraform and Runtime values

The other day I was asked if we could use operating system variables in Terraform like we can in Ansible?

An interesting question.  Obviously those who know Terraform will shout out and say, of course you can you just put TF_VAR_ in front of the variable in the OS and Terraform will find it.

However, that's not the same as being able to get hold of say, HOME or PWD, etc as can be done in Ansible using the lookup function;

vars:
  cwd: "{{ lookup('env', 'PWD' }}"

The above in Ansible obtains the value of the PWD operating system environment variable and stores it in an Ansible variable called cwd at runtime.

So, how on earth do I do such a thing in Terraform, rather than relying on people configuring variables up front, or using a Makefile or wrapper script?

The data sources

The data source allows you to obtain data from various sources, see https://www.terraform.io/docs/configuration/data-sources.html, of which the best data source for our requirement is the external data source.

Example

The downside is that your external data source needs to return JSON data, so you can't just run a command or echo a variable.  But you can create the relevant JSON data.  I wrote a simple use at https://bitbucket.org/stevshil/terraform/src/master/envvar/.

To explain that code directly here I took the following steps;

1. Create a shell script that will return JSON data, this way you don't have to work out how to escape characters, etc.  In the example code we created a script called mypwd containing the following;
   
   #!/bin/bash
   cat <<_END_
   {
     "dir": "$PWD"
   }
   _END_
   
   
2. Create the Terraform code (getvar.tf) to grab the printed output;
   
   data "external" "example" {
    program = ["bash","./mypwd"]
   }
   output "pwd" {
     value = data.external.example.result.dir
   }

You'll notice that using the external data source, and a given name (example in this instance) we call our shell script.  The bash is there just in case someone forgot to make the script executable.

The output in this case is just to show that the data was returned in an attribute called dir as you'll note in the shell script code where we output JSON data which has a key called dir that contains the value of the operating system variable called $PWD.  We could just as easily have written Python code, or have ran another shell command that outputs the data we desire.

To retrieve the value output from our script we use the normal Terraform object attribute reference, but because it is a data source we prepend the word data to the resource - data.external.example.result.dir, with the result element being part of the data source, but the dir is our JSON data key.  This reference can be used anywhere within your code when you need to use the runtime value.

Goodbye to the TF_VAR_ potential error prone values if you need to make use of attributes from the operating system.