How to upload multiple files parallelly to amazon S3

Are you writing a backend code?. Do you use Node.js as the backend?. Are you writing APIs that accept multipart/form-data request?. Do you use Amazon S3 for storing files? Are you facing issues uploading files to amazon s3?. Then, this article is for you. Yes, you have landed at the right place. Throughout this article, I will guide you how to upload files(be it single or multiple) to Amazon s3 in 10 easy steps.

Step 1:

Install “aws-sdk” npm package.

Install “multer” npm package.

That’s all you need.

Step 2:

‘Require’ them from the code and store them in variables.

const AWS = require(‘aws-sdk’);
const multer = require(‘multer’);
const upload = multer({ dest: ‘uploads/’ });
const fileSystem = require(‘fs’);

Step 3:

Create ‘s3’ object using Amazon web services “access key Id” and “secret access key”.

const s3 = new AWS.S3({
accessKeyId: process.env.aws_access_key_id,
secretAccessKey: process.env.aws_secret_access_key
});

Storing keys on “process.env” is out of the scope of this article. There are lot of articles regarding this on the internet.

Step 4:

Create a function “uploadFile” like below;

async function uploadFile(fileName, fileKey) {
return new Promise(async function(resolve, reject) {
const params = {
Bucket: ‘bucketName’, // pass your bucket name
Key: fileKey,
ACL: ‘public-read’,
Body: fileSystem.createReadStream(fileName.path),
ContentType: fileName.type
};

await s3.upload(params, function(s3Err, data) {
if (s3Err){
reject(s3Err);
}
console.log(`File uploaded successfully at ${data.Location}`);
resolve(data.Location);
});
});
}

Step 5:

Create an array of Promises;

var uploadFilePromises = [];

Step 6:

Create an API to serve the request from the client;

var cpUpload = upload.fields([{ name:’screenShots’, maxCount:1 },{ name:’apk’, maxCount:1 }]);

router.post(‘/updateApp’, cpUpload, async function (req, res, next) {
}

Here, cpUpload variable holds the fields in the request which has files. maxCount tells you the maximum number of files that the backend can accept for that particular field. Files could be accessed as follows;

var screenShot = request.files.screenShots;
var apk = request.files.apk;

Step 7:

Edit the API to include the following;

Create keys for the files respectively and call the “uploadFile” method with the ‘file’ and ‘file key’ as parameters. Push the function call into ‘uploadFilePromises’ variable that has been created in step 5.

We will try to upload both the `apk` and `screenshot` files parallelly. since, ‘request.files’ returns array, we have to get the first file using the index 0.

var apkFileKey = “apk”;
uploadFilePromises.push(uploadFile(apk[0], apkFileKey));
var screenShotFileKey = “screenShot”;
uploadFilePromises.push(uploadFile(screenShot[0], screenShotFileKey));

Step 8:

Use “Promise.all” method to upload the files parallelly.

Promise.all(uploadFilePromises).then(async (values) => {
console.log(values);
}, reason => {
console.log(reason);
});

Step 9:

Finally, the API would look like below;

var cpUpload = upload.fields([{ name:’screenShots’, maxCount:5 },{ name:’apk’, maxCount:1 }]);

router.post(‘/updateApp’, cpUpload, async function (req, res, next) {
var screenShot = req.files.screenShots;
var apk = req.files.apk;

var apkFileKey = “apk”;
uploadFilePromises.push(uploadFile(apk[0], apkFileKey));
var screenShotFileKey = “screenShot”;
uploadFilePromises.push(uploadFile(screenShot[0], screenShotFileKey));

Promise.all(uploadFilePromises).then(async (values) => {
console.log(values);
}, reason => {
console.log(reason);
});
}

Step 10:

That’s it !!!. Try it for yourself. Let me know, if you are facing any issues in the comment section below. And, if you liked this article, please give me a round of applause(Hit the clap icon below(as many times as you want :P)).

Happy Coding!!!.