Every non-anonymous request to S3 must contain authentication information to establish the identity of the principal making the request. In REST, this is done by first putting the headers in a canonical format, then signing the headers using your AWS Secret Access Key.
There are two ways to send your signature with a request. The first is to put your AWS Access Key ID and the signature you computed into the Authorization header:
Example
"Authorization: AWS " + AWSAccessKeyId + ":" + base64(hmac-sha1(VERB + "\n" + CONTENT-MD5 + "\n" + CONTENT-TYPE + "\n" + DATE + "\n" + CanonicalizedAmzHeaders + "\n" + CanonicalizedResource))
You can also send a signature as a URL-encoded query-string parameter in the URL for the request. This is useful if you want to enable a third party to access S3 on your behalf without your having to proxy the data transfer. For example, if you want to enable a user to download your private data directly from S3, you can insert a pre-signed URL into a web page before giving it to your user. The canonicalized string that you sign is the same, except that you replace the DATE field in the string with an Expires field that indicates when you want the signature to expire. The Expires field is given as the number of seconds since epoch time, and is also included as a query string parameter along with your AWS Access Key ID:
Example
GET /my-bucket/foo
?Signature=<urlencode(base64(hmac-sha1(VERB + "\n" + CONTENT-MD5 + "\n" + CONTENT-TYPE + "\n" + Expires + "\n" + CanonicalizedAmzHeaders + "\n" + CanonicalizedResource)))>
&Expires=<seconds since epoch>
&AWSAccessKeyId=<aws-id>
When authenticating through the Authorization header, you create the string to be signed by concatenating the request verb with canonicalized headers and the resource that the request is targeting.
The headers used for request signing are: content-md5, content-type, date, and anything that starts with x-amz-. The string to be signed is formed by appending the REST verb, content-md5 value, content-type value, date value, canonicalized x-amz headers (see recipe below), and the resource; all separated by newlines. (If you cannot set the Date header, use the x-amz-date header as described below.)
The resource is the bucket and key (if applicable), separated by a '/'. If the request you are signing is for an ACL or a torrent file, you should include ?acl or ?torrent in the resource part of the canonical string. No other query string parameters should be included, however.
When authenticating via query string parameters, you create the string to be signed by concatenating the request verb with canonicalized headers and the resource that the request is targeting.
The headers used for request signing are the same as those for authorization header authentication, except that the Date field is replaced by the Expires parameter. The Expires parameter is the time when you want the signature to expire, specified as the number of seconds since the epoch time.
Thus, the string to be signed is formed by appending the REST verb, content-md5 value, content-type value, expires parameter value, canonicalized x-amz headers (see recipe below), and the resource; all separated by newlines.
The resource is the same as that for authorization header authentication: the bucket and key (if applicable), separated by a '/'. If the request you are signing is for an ACL or a torrent file, you should include ?acl or ?torrent in the resource part of the canonical string. No other query string parameters should be included, however.
Lower-case header name
Headers sorted by header name
The values of headers whose names occur more than once should be white space-trimmed and concatenated with comma separators to be compliant with section 4.2 of RFC 2616.
remove any whitespace around the colon in the header
remove any newlines ('\n') in continuation lines
separate headers by newlines ('\n')
The string to sign (verb, headers, resource) must be UTF-8 encoded.
The content-type and content-md5 values are optional, but if you do not include them you must still insert a newline at the point where these values would normally be inserted.
Some toolkits may insert headers that you do not know about beforehand, such as adding the header 'Content-Type' during a PUT. In most of these cases, the value of the inserted header remains constant, allowing you to discover the missing headers using tools such as Ethereal or tcpmon.
Some toolkits make it difficult to manually set the date. If you have trouble including the value of the 'Date' header in the canonicalized headers, you can include an 'x-amz-date' header prior to canonicalization. The value of the x-amz-date header must be in one of the RFC 2616 formats (http://www.ietf.org/rfc/rfc2616.txt). If S3 sees an x-amz-date header in the request, it will ignore the Date header when validating the request signature. If you include the x-amz-date header, you must still include a newline character in the canonicalized string at the point where the Date value would normally be inserted.
The value of the Date or, if applicable, x-amz-date header must specify a time no more than 15 minutes away from the S3 webserver's clock.
The hash function to compute the signature is HMAC-SHA1 defined in RFC 2104 (http://www.ietf.org/rfc/rfc2104.txt), using your Secret Access Key as the key.
Example
For example, imagine that you want to sign the following request:
PUT /quotes/nelson HTTP/1.0 Content-Md5: c8fdb181845a4ca6b8fec737b3581d76 Content-Type: text/html Date: Thu, 17 Nov 2005 18:49:58 GMT X-Amz-Meta-Author: [email protected] X-Amz-Magic: abracadabra
The canonical string to be signed is:
PUT\nc8fdb181845a4ca6b8fec737b3581d76\ntext/html\nThu, 17 Nov 2005 18:49:58 GMT\nx-amz-magic:abracadabra\nx-amz-meta-author:[email protected]\n/quotes/nelson
Suppose your AWS Access Key ID is "44CF9590006BF252F707" and your AWS Secret Access Key is "OtxrzxIsfpFjA7SwPzILwy8Bw21TLhquhboDYROV". Then you could compute the signature as follows:
import base64
import hmac
import sha
h = hmac.new("OtxrzxIsfpFjA7SwPzILwy8Bw21TLhquhboDYROV",
"PUT\nc8fdb181845a4ca6b8fec737b3581d76\ntext/html\nThu, 17 Nov 2005 18:49:58 GMT\nx-amz-magic:abracadabra\nx-amz-meta-author:[email protected]\n/quotes/nelson",
sha)
base64.encodestring(h.digest()).strip()
The resulting signature would be "jZNOcbfWmD/A/f3hSvVzXZjM2HU=", and, you would add the Authorization header to your request to come up with the following result:
PUT /quotes/nelson HTTP/1.0 Authorization: AWS 44CF9590006BF252F707:jZNOcbfWmD/A/f3hSvVzXZjM2HU= Content-Md5: c8fdb181845a4ca6b8fec737b3581d76 Content-Type: text/html Date: Thu, 17 Nov 2005 18:49:58 GMT X-Amz-Meta-Author: [email protected] X-Amz-Magic: abracadabra
Example
Imagine that you can't set the Date header, and don't know what value your toolkit will assign that header. Then you need to include the x-amz-date header in your request:
GET /quotes/nelson HTTP/1.0 Date: XXXXXXXXX X-Amz-Magic: abracadabra X-Amz-Date: Thu, 17 Nov 2005 18:49:58 GMT
The canonical string to be signed is (note the included newlines even though there is no Content-Md5 or Content-Type header in the request):
GET\n\n\n\nx-amz-date:Thu, 17 Nov 2005 18:49:58 GMT\nx-amz-magic:abracadabra\n/quotes/nelson
Suppose your AWS Access Key ID is "44CF9590006BF252F707" and your AWS Secret Access Key is "OtxrzxIsfpFjA7SwPzILwy8Bw21TLhquhboDYROV". Then you could compute the signature as follows:
import base64
import hmac
import sha
h = hmac.new("OtxrzxIsfpFjA7SwPzILwy8Bw21TLhquhboDYROV",
"GET\n\n\n\nx-amz-date:Thu, 17 Nov 2005 18:49:58 GMT\nx-amz-magic:abracadabra\n/quotes/nelson",
sha)
base64.encodestring(h.digest()).strip()
The resulting signature would be "5m+HAmc5JsrgyDelh9+a2dNrzN8=", and, you would add the Authorization header to your request to come up with the following result:
GET /quotes/nelson HTTP/1.0 Authorization: AWS 44CF9590006BF252F707:5m+HAmc5JsrgyDelh9+a2dNrzN8= Date: XXXXXXXXX X-Amz-Magic: abracadabra X-Amz-Date: Thu, 17 Nov 2005 18:49:58 GMT
Example
Let's say you want to let someone else access http://s3.amazonaws.com/quotes/nelson via a web browser. Query String Auth URLs all have an Expires parameter which specifies until when the URL is still valid. The value of this parameter is expressed in seconds since epoch. On unix, you can run the command "date +%s", or in java, you can divide the result of System.currentTimeMillis() by 1000. So, if it's 1141889060 right now, and you want the url to be valid for 60 seconds, your Expires parameter would be 1141889120. This value will be used in place of the Date header when computing the canonical string.
The canonical string to be signed is:
GET\n\n\n1141889120\n/quotes/nelson
You know that when the browser makes the GET request, it won't provide a Content-Md5 or a Content-Type hader, nor will it set any x-amz- headers, so those parts are all kept empty.
Suppose your AWS Access Key ID is "44CF9590006BF252F707" and your AWS Secret Access Key is "OtxrzxIsfpFjA7SwPzILwy8Bw21TLhquhboDYROV". Then you could compute the signature as follows:
import base64
import hmac
import sha
import urllib
h = hmac.new("OtxrzxIsfpFjA7SwPzILwy8Bw21TLhquhboDYROV",
"GET\n\n\n1141889120\n/quotes/nelson",
sha)
urllib.quote_plus(base64.encodestring(h.digest()).strip())
Note that we also url-encoded the result this time. This is because the output from the base64 algorithm is not suitable for use as a query string parameter, so we add an additional layer of armor to make it acceptable.
The resulting signature would be "vjbyPxybdZaNmGa%2ByT272YEAiv4%3D", so the parameters we will use are:
| Key | Value |
|---|---|
| AWSAccessKeyId | 44CF9590006BF252F707 |
| Expires | 1141889120 |
| Signature | vjbyPxybdZaNmGa%2ByT272YEAiv4%3D |
And the resulting URL would be:
http://s3.amazonaws.com/quotes/nelson?AWSAccessKeyId=44CF9590006BF252F707&Expires=1141889120&Signature=vjbyPxybdZaNmGa%2ByT272YEAiv4%3D