Signed URls – Query string authentication

What it is?

Signed URL’s contain query string parameters that allow access to protected data without requiring any credentials from the end user. Furthermore, you can restrict access to the particular file by providing a timestamp, so that after a particular date the URL will be invalidated automatically.
You can generate URL’s with three different permissions:
– READ (GET)
– WRITE (PUT)
– DELETE (DELETE)

When do I need it?

There are plenty of scenarios when signed URL’s come in handy.
One of the most important advantages and use cases is to share files with other people for a restricted period of time. That can be achieved without any unnecessary login being required or any temporary credentials. All you need to do is generate a proper signed URL with specific permissions.
You can generate as many signed URLs as you want for each file. You can also grant permissions to the same file for different periods of time.

Where can I use it?

Two of the most common services providing signed URLs authentication are Amazon Web Services and Google Cloud Services.
There are also a few smaller providers you can use, or you can implement signed URLs on your server. You can find out how to do this here on BlazerSix.

How does it work?

It depends on the service where the signed URLs are implemented. Each service requires a few different pieces of information in the URL and sometimes different types of encoding, but the concept of encoding the query string is the same.
Basically you need to create a string with the specific information about the file you will give access to and encode it using your private key and the protocol specified by the provider, then add this to the URK as a query parameter, to authenticate access to the file.
In the examples below we will see how to create signed URL’s manually and by using a readily available library.
To create a signed URL you need to follow three crucial steps:
1. Construct the string to be signed
2. Sign the string using the specified algorithm
3. Assemble the URL

Examples

All examples are written in python.

1. AMAZON WEB SERVICES

The Python library to connect to the AWS services is called BOTO. It also contains a built-in way of creating signed URL’s so it is easy.
We can easily install BOTO lib using pip.
[code language=”python”]
$ pip install boto
[/code]
And now we’re ready to work with it.
[code language=”python”]
# import boto library
import boto
import time
# connect to the AWS in this particular example we are connecting to CloudFront
c = boto.connect_cloudfront(‘your-aws-access-key-id’, ‘your-aws-secret-access-key’)
# retrieve all buckets from CloudFront
rs = c.get_all_distributions()
# get for example first
ds = rs[0]
# parse it to proper distribution
distro = ds.get_distribution()
# get url of the file inside the bucket
url = (distro.get_objects()[0]).url()
# get the private key to sign the url
private_key_file = open(‘AWS test pk-your-aws-access-key-id.pem’, ‘r’)
# timestamp to set period of time the link will be active
expires = int(time.time()) + 300000
# create signed URL and share with other people
url = distro.create_signed_url(url, ‘your-aws-access-key-id’, private_key_file=private_key_file, expire_time=expires)
[/code]
So using AWS is really straight forward.

2. GOOGLE

To use google services we need to install gcloud for Python. We can also easily do this using pip.
[code language=”python”]
$ pip install gcloud
[/code]
After that we’re ready to start work.
Unfortunately gcloud doesn’t provide any built-in methods to create signed URLs. To create them we need to use our own implementation that follows Google’s instructions from the Google signing manual.
There are also some code samples written in python, that show how to create signed URLs in Python here.
I’ve reused some of the parts from that example to create the class presented below.
Lets assume we have a file in our Google Cloud bucket
[code language=”python”]
base_url = "https://storage.googleapis.com"
file_path = "/test-bucket/test-file.txt’"
[/code]
So, first we need to create the signature string. The format of it is specified in the Google Cloud documentation.
[code language=”python”]
signature_string = (‘{method}n’ # Required – What is the method we want to give access to(‘GET’, ‘PUT’, ‘DELETE’)
‘{content_md5}n’ # Optional – argument used as checksum of file
‘{content_type}n’ # Optional – argument specifying type of the file
‘{expiration}n’ # Required – Unix timestamp when the signature expires
‘{resource}’) # Required – file path to the file inside the bucket (like file_path)
[/code]
The next step is to sign the URL using our private key. We will use the cryptographic library for Python PyCrypto.
[code language=”python”]
# create sha hash from the signature string
shahash = Crypto.Hash.SHA256.new(signature_string)
# import an RSA key, encoded in standard form
priv_key = Crypto.PublicKey.RSA.importKey(PRIV_KEY_IN_DER_FORMAT)
# RSA digital signature protocol, allow to sign url
signer = Crypto.Signature.PKCS1_v1_5.new(priv_key)
# sign url by signer
signature_bytes = signer.sign(shahash)
# encode signed data by standard base64 protocol
signed_string = base64.b64encode(signature_bytes)
[/code]
Finally we need to add the timestamp to specify how long the URL should be valid for and then assemble the URL
[code language=”python”]
query_params = {
‘GoogleAccessId’: _client_id_email, # provided by google
‘Expires’: str(expiration), # Unix timestamp, for example ‘1416831132’
‘Signature’: signed_string
}
signed_query_string = urllib.urlencode(query_params) # use standard urllib to encode dictionary to query string
signed_url = base_url + file_path + ‘?’ + signed_query_string # ready to share signed URL
[/code]
That’s all we need to do!

Do I need to do it manually?

If you are using Google Cloud services you can use a small class that I wrote that will do all of this for you.
You can find it here on Github.
Using it is really straight forward (read the README before using it).
[code language=”python”]
from GCSSignedUrlGenerator import GCSSignedUrlGenerator
GC_KEY_FILE_NAME_DER = ‘<priv_key>.der’
GC_EMAIL = ‘<gc-email>@developer.gserviceaccount.com’
gc_key_file_der = open(GC_KEY_FILE_NAME_DER, ‘rb+’)
private_key_der = gc_key_file_der.read()
file_path = ‘/%s/%s’ % (‘test-bucket’, ‘test-file.txt’)
signer = GCSSignedUrlGenerator(GC_EMAIL, GCS_API_ENDPOINT, private_key_der)
signed_url = signer.makeSignedUrl(file_path)
print signed_url
‘https://storage.googleapis.com/test-bucket/test-file.txt?Expires=1416389894&GoogleAccessId=896388671115-fhb4f9o9k520c084tlj7r8g71ddsdi1n%40developer.gserviceaccount.com&Signature=Rshs1Acg5VXYsMtwEdQrAWGldLU9eCLb3bW5JN7xDkj7dzIaRIV8e4AoOjVisZ1JY%2BXHbO8RRDTZT4ubVHXdhoCWbmkcegnIXztAArWQeKoHTXmoayZEmcC72HAnFz9nPK23AYOmzo5scdn53yweJ8NWPtYgTdCQOb%2Fqve7PhFc%3D’
[/code]
Hope anyone will find this helpful.