Direct S3 Uploads with Ember and Rails
For the past several weeks, a few of us on the ThreatSim team at Wombat have been learning the ins and outs of Ember. We’re in the early stages of developing an Ember app to manage user records, and to practice for that, we started building a Red Pen clone, which we lovingly refer to as Julipen. We were working with a Rails backend, storing the images in Amazon S3 buckets.
Julipen requires that users be able to upload images, sometimes several at one time. I played around with using the pl-pload github README addon for the uploads. The reason I liked this addon was that it provides direct S3 uploads, comes with some built-in UI goodies, such as drag ‘n drop and a progress bar, and is pretty well maintained according to [ember observer] (http://emberobserver.com/addons/ember-plupload).
Being the beginner that I am, I found this addon pretty easy to use. The pl-uploader README was written in such a way that I spent most of my time copying and pasting and less time finding an excuse to go do laundry.
Behind the Scenes
Before working with the addon, we already had an app with the ability to “add” an image to the database. Any time a new image was created, we put a placeholder image on the UI, which made full use of Ember data and the Rails backend, but didn’t hit S3. The goal of this tutorial is to show you how to replace that basic functionality by uploading a real image to S3.
Let’s Get Started
The first step is to add the pl-ploader component code. Below, the plupload code is nestled inside a component we had already created, called create-shot
, for adding new individual images via a modal window. So essentially, I am adding a component to a component. The only change I made from the addon’s supplied code was to change the name of the action called, onfileadd="addShot"
, to match the code we had already been using for the placeholder image. This is where the bonus built-ins I mentioned are added oh-so-nicely.
{{#modal-dialog translucentOverlay=true}}
<div class="close">
<button {{action "closeModal"}} type="button" class="close-button">X</button>
</div>
<div class="add-shot">
<div class="field">
<span>Title:</span> {{input value=title}}
</div>
<div class="field">
<span>Description:</span> {{input value=description}}
</div>
{{#pl-uploader for="upload-image" extensions="jpg jpeg png gif" onInitOfUploader="onInitOfUploader" onfileadd="addShot" as |queue dropzone|}}
<div class="dropzone" id={{dropzone.id}}>
{{#if dropzone.active}}
{{#if dropzone.valid}}
Drop to upload
{{else}}
Invalid
{{/if}}
{{else if queue.length}}
Uploading {{queue.length}} files. ({{queue.progress}}%)
{{else}}
<h4>Upload Images</h4>
<p>
{{#if dropzone.enabled}}
Drag and drop images onto this area to upload them or
{{/if}}
<a id="upload-image">Add an Image.</a>
</p>
{{/if}}
</div>
{{/pl-uploader}}
</div>
{{/modal-dialog}}
Then, to get the action back to my calling route, I passed the action through the create-shot
component.
actions: {
closeModal: function() {
this.sendAction("closeModal");
},
addShot: function(file) {
let title = this.get('title') || file.get('name');
let description = this.get('description');
this.sendAction("addShot", title, description, file);
}
Before we get to adding the action on the calling route, we need to get our S3 bucket and the Rails API ready. This is where the copy & paste action gets really heavy…
Set up a new S3 bucket, and supply the following CORS configuration. This gives you a good chunk of basic permissions for uploading files to your bucket.
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Content-Type</AllowedHeader>
<AllowedHeader>x-amz-acl</AllowedHeader>
<AllowedHeader>origin</AllowedHeader>
<AllowedHeader>accept</AllowedHeader>
<ExposeHeader>Location</ExposeHeader>
</CORSRule>
</CORSConfiguration>
Now, to the Rails piece. The first step is to add your AWS credentials using Figaro. This goes in the application.yml
Figaro generates.
development:
aws_access_key: somelongkeylettersandnumbers
aws_secret_key: somemorelongkeylettersandnumbers
aws_bucket: the_name_of_the_bucket_you_set_on_s3
Next, create a file in the models called s3_direct.rb
and paste this into it. All this code gives you the url and credentials you are going to ask for from the Ember app, which are pulled together in the to_json
method below:
require 'openssl'
require 'base64'
require 'json'
class S3Direct
def initialize(aws_access_key, aws_secret_key, options = {})
require_options(options, :bucket, :expiration, :key, :acl)
@aws_access_key = aws_access_key
@aws_secret_key = aws_secret_key
@options = options
end
def signature
Base64.strict_encode64(
OpenSSL::HMAC.digest('sha1', @aws_secret_key, policy)
)
end
def policy
Base64.strict_encode64({
expiration: @options[:expiration].utc.iso8601,
conditions: conditions
}.to_json)
end
def to_json
{
url: "https://#{@options[:bucket]}.s3.amazonaws.com",
credentials: {
AWSAccessKeyId: @aws_access_key,
policy: policy,
signature: signature,
acl: @options[:acl],
key: @options[:key]
}
}
end
private
def conditions
dynamic_key = @options[:key].include?('${filename}')
prefix = @options[:key][0..(@options[:key].index('${filename}') - 1)]
conditions = (@options[:conditions] || []).map(&:clone)
conditions << { bucket: @options[:bucket] }
conditions << { acl: @options[:acl] }
conditions << { key: @options[:key] } unless dynamic_key
conditions << ['starts-with', '$key', prefix] if dynamic_key
conditions << ['starts-with', '$Content-Type', '']
end
private
def require_options(options, *keys)
missing_keys = keys.select { |key| !options.key?(key) }
return unless missing_keys.any?
raise ArgumentError, missing_keys.map { |key| ":#{key} is required to generate an S3 upload policy." }.join('\n')
end
end
And then create a folder in your API directory called s3_direct_controller.rb
and give it the following code. Make sure you replace the variables for your aws_access_key
and aws_secret_key
if you did not give them the same names I did.
class Api::S3DirectController < ApplicationController
def presign
render json: S3Direct.new(Figaro.env.aws_access_key, Figaro.env.aws_secret_key, {
bucket: Figaro.env.aws_bucket,
acl: 'public-read',
key: 'uploads/${filename}',
expiration: Time.now + 10 * 60,
conditions: [
['starts-with', '$name', '']
]
}).to_json
end
end
Whew! That was a lot of copy & paste! Now, back to Ember.
When we left off, we were just about to get to the juicy stuff…adding the addShot
action to the calling route. In the addShot
action, we start out by creating a new record for the shot. I already had this code from the previous placeholder code, so the rest is the song and dance we go through to get the url for the image on S3.
We kick that off by asking for the signed credentials from Rails. The RSVP.cast
business goes to the API (insert your real API link or environment variable here, instead of localhost) to ask for the signed credentials. The cast
method allows the response we get to be thenable, which we make use of using pluploader’s built-in magic with file.upload
, sending along the S3 url and some info from the presigned credentials in the response.
If that is successful, we chain one more promise to handle the response, which is where we add the url for the newly uploaded image to the record we created at the top of this action. If it is unsuccessful, we rollback the entire shot object.
const RSVP = Ember.RSVP;
const get = Ember.get;
const set = Ember.set;
actions: {
closeModal: function() {
this.transitionToProject();
},
addShot: function(title, description, file) {
this.store.findRecord('project',
this.paramsFor('project').project_id).then(
(project) => {
var shot = this.store.createRecord('shot', {
title: title,
description: description,
project: project
});
RSVP.cast(Ember.$.get('http://localhost:3000/api/s3_direct')).then((response) => {
return file.upload(response.url, {
data: response.credentials
});
}).then((response) => {
set(shot, 'source', response.headers.Location);
shot.save().then(() => {
Ember.Logger.log("save successful");
this.transitionToProject();
});
}), function() {
Ember.Logger.log("save was not successful")
shot.rollback();
};
});
}
Now go to the browser and upload some files!