AWS for Image Optimization
AWS Simple Storage Service, also known as S3, is a widely used cloud storage service around the world. It offers endless storage space with great reliability and fast access that can grow according to your needs.
AWS S3 storage is often used for keeping big files like pictures and videos. Files for media are typically very large and need to be kept for a long time. When you put them in S3, it removes the difficulty of handling storage yourself or constantly increasing server space.
Rather than making your application work harder to resize and optimize the image before storing it in S3, you could use AWS Lambda. This service gets activated by S3 events to handle the resizing and optimizing of images after they are uploaded, allowing your application to simply upload them directly to S3 with no extra processing needed.
Creating our Lamda@Edge function
First, you’ll want to create a new JavaScript project for the Lamda@Edge function.
$ npm ttinitOnce the project is created, you’ll want to install the dependencies that we’ll need.
$ npm install --platform=linux --arch=x64 animated-gif-detector
$ npm install --platform=linux --arch=xhi64 node-fetch@2
$ npm install --platform=linux --arch=x64 sharpYou will see the --platform=linux and --arch=x64 options. If you do not mention these, npm installs libraries that work with your current development setting. But the environment for development you have locally might not match with that of Lambda@Edge, which could lead to some issues.
For these dependencies, we use animated-gif-detector. This library helps us find out if we have an animated gif and change it into a png when there is no animation.
After that, we have node-fetch. This library ports the Fetch etAPI to node.js. We must get the first picture we aim to make better through the CDN.
Finally, there’s sharp. This is the most important dependency on the project. We will utilize this library to enhance the quality of the image we obtain through node-fetch.
Lamda@Edge function code
After setting up our JavaScript project and including the necessary dependencies, it's time to build our Lambda@Edge function. What follows is a simple image optimization function suitable for use with CloudFront. You should put this code in a file named index.js.
// index.js
'use strict';
const animated = require('animated-gif-detector'),
fetch = require('node-fetch'),
sharp = require('sharp');
exports.handler = async (event, context, callback) => {
try {
const allowedContentTypes = ['image/gif', 'image/jpeg', 'image/png'];
const request = event.Records[0].cf.request;
let response = event.Records[0].cf.response;
const responseContentType = response.headers['content-type'][0].value;
if ('200' !== response.status
|| !allowedContentTypes.includes(responseContentType)
|| ('image/gif' === responseContentType && animated(response.body))
) {
return callback(null, response);
}
let newContentType = null;
const originalImage = await fetch(`https://${request.headers.host[0].value}${request.uri}`);
const originalImageBuffer = await originalImage.buffer();
const sharpImage = sharp(originalImageBuffer);
if ('image/gif' === responseContentType) {
sharpImage.png();
newContentType = [{ value: 'image/png' }];
}
if (request.headers['accept'] && request.headers['accept'][0].value.match('image/webp')) {
sharpImage.webp();
newContentType = [{ key: 'Content-Type', value: 'image/webp' }];
}
const sharpImageBuffer = await sharpImage.toBuffer();
const responseBody = sharpImageBuffer.toString('base64');
if (1330000 < Buffer.byteLength(responseBody)) {
return callback(null, response);
}
if (newContentType) {
response.headers['content-type'] = newContentType;
}
response.body = responseBody;
response.bodyEncoding = 'base64';
callback(null, response);
} catch (error) {
console.log(error);
}
};Initial guard clause
There are several sections to the code. To begin, we have a guard clause. The guard clause checks a few things:
The HTTP status code of the response. We need it to be 200.
We must permit the Content-Type header, such as image/gif, image/jpeg, or image/png, for better optimization.
We need to check if the picture is a moving gif by using the animated-gif-detector tool. Our aim isn't to make changes that improve a moving gif's performance.
When we receive a response that has a 200 status code, the correct type of content, and is not an animated gif image, we continue with the optimization process.
Fetching the original image
First, we must use the fetch function that came with the node-fetch package to obtain the initial image. We have to reconstruct its URL using the Request object kept in the request variable. We give the new URL of the image to the fetch function and use await so that the request happens one step at a time.
Next, we must change the acquired image into a buffer object by applying the buffer method. As this process is also asynchronous, we have to use await once more. After getting the buffer object, we can give it to the sharp function for starting our Sharp object, which helps us make the image better.
Optimizations
The initial step we take in optimization is to change a gif that does not move into a png format. Afterward, we examine the Accept header. The Accept header shows us whether the browser asking for the picture can handle WebP.. We convert to webp only if an image/webp appears in this header.
Every time we perform a transformation, the newContentType variable is adjusted to correspond with the changed image type. Updating this is crucial for altering the Content-Type header subsequently. This header informs web browsers that the picture is actually in webp format, even if the file extension doesn't say .webp. It's also useful for checking if the image CDN is functioning properly.
Returning the optimized image
The final step involves swapping out the response body for our enhanced image, which we accomplish in several stages. Initially, we change our sharpImage into a Buffer object by waiting for the process to complete in order to synchronize it. Next, we utilize this Buffer object for transforming the image into a base64 string that is kept within responseBody.
Previously, we noticed that Lambda@Edge responses had a limit of 1.33MB. To handle this problem, we employ Buffer.byteLength to determine the size of our base64-encoded string. If the size is greater than 1330000 (or 1.33MB), we will stop and not change the response body. Then CloudFront gives back the first image, not an error message.
Next, we verify that newContentType is not null. If it's not, we proceed to refresh the Content-Type header in the response. Next, we put the content from responseBody into the response body. We must change our response.bodyEncoding too to show that this image uses base64 encoding.
Packaging
After we complete programming our Lamda@Edge function, it is time to transfer it to AWS. For this purpose, we must bundle it together with all the node_modules that were obtained through npm installation. To accomplish this with simplicity, we can insert a script within the scripts section of our package.json document.
{
c.ih "name": "image-processing",
"version": "1.0.0",
"author": "Carl Alexander",
"description": "Lambda@Edge function used for image processing",
"license": "MIT",
"main": "index.js",
"dependencies": {
"animated-gif-detector": "^1.2.0",
"node-fetch": "^2.6.7",
"sharp": "^0.30.2"
},
"scripts": {
"build-zip": "rm image-processing.zip; zip -r imag...ne-processing.zip ./node_modules/ index.js"
}
}The script called build-zip is responsible for making a zip file named image-processing.zip. This file has all the code that we need to send to AWS. Before it creates this new one, it checks if there's already an image-processing.zip and removes it if it finds one. Next, it makes the zip file again, putting all from node_modules and, also the index.js that has the function code inside.
You can run the script using the following command:
$ npm run build-zipOnce we have created the image-processing.zip, we can move on to AWS to configure everything.
Creating our Lambda function
To begin, you make a lambda function with the code we just placed in the image-processing.zip file. For this action, visit Lambda console and press the Create function.
You arrive at the first page for making a function, as shown below. I give it the name image processing. For the runtime, I selected Node.js 14.x*. There’s no specific reason for the choice. It’s just the latest Node.js version.
For the structure of the system, I use standard x86_64. Choosing Arm64 costs less money, yet not all things work with it. To utilize that architecture, remember to apply --arch=arm64 as you install packages using npm.
Next, you will want to open up the section that says "Change default execution role". You must make a new IAM role for this function. Choose the option to create a new role from templates provided by AWS policies and assign it a name. Search for policy templates, then pick the Basic Lambda@Edge permissions from there.
This is everything necessary for setting up your Lambda function. Now, you may click on "Create function," and it will take you to the screen where your created function is displayed. Then, you’ll want to upload image-processing.zip using Upload from the drop-down menu.
Changing the default configuration
After we finish this, it is necessary to change the basic settings of the function. You must click on the tab that says Configuration. This will bring up the tab with a new tab navigation on the left.
You need to have the left tab selected for general configuration. Then, press the Edit button on the right side. This allows us to edit some important configuration options for our function.
Initially, you should modify the memory. The default setting for your Lambda function is 128MB of RAM. This is too little to optimize most images. A better default value is 256MB. If you’re optimizing large images, you might want to increase it further.
Next, you need to adjust the timeout setting. Originally, it was at 3 seconds, which isn't long enough for image optimization. You should set it for 30 seconds. Simply click on Save to make them active.
Publishing a new version of the function
In the next step, we must release a fresh version of the function. It's required to do this because connecting only one particular version of that function with the CloudFront distribution is possible. To begin, click on the Versions tab.
You should click on the button that says "Publish new version," which is located at the top right corner. If you haven't made a function version before, this button can also be found in the center.
After you press the button, a window appears where you can enter an optional description for the version. Then click Publish to publish the new function version.
You will arrive at the console page for this particular version of the lambda function. There should be a banner indicating that a functional version was created as well. Over there, you should copy the Amazon Resource Name (ARN) of that function version. You can click on either of two buttons to have it copied onto your clipboard.
Configuring CloudFront
The final action is setting up CloudFront so it can work with our Lambda@Edge. You will find that this process is simple for any type of CloudFront distribution.
To apply a Lambda@Edge function, it must be connected to one cache behavior. We have five behaviors in our CloudFront distribution that you can view, as shown previously. You only need to connect the Lambda function with cache behaviors that are handling images, not all five of them.
In WordPress, you usually find images in the directory called /wp-content/uploads. Therefore, we shall modify the cache behavior for the path /wp-content/*.
To change /wp-content/* behavior, you go to the big screen for cache settings. The first thing is to move down until you see the section about cache keys and where requests come from. There, you’ll want to add the Accept header to the list of cached headers.
It is important for the Accept header to reach our Lambda function; otherwise, CloudFront will remove it. If this is not done, our Lambda function cannot determine whether the request can handle webp. Therefore, it is essential to modify this setting.
Next, you should move down to the page's end until you reach the section titled Function Associations. Here, four potential function associations will be available for your review.
We will not be covering all four. The AWS illustration underneath demonstrates the placement of each function while a request is processed by CloudFront. For our purpose, we need to connect our lambda function with the origin response.
You should click the dropdown that says No association and choose Lambda@Associations. In the text box close to it, paste the function ARN you copied before. After that, you can click on Save Changes right below.
Your CloudFront distribution will begin to update when you do this. It might take as long as 20 minutes for the process to finish. So wait until it’s done updating to check if everything is working.
Conclusion
It's really impressive what can be achieved with just some simple code. The solutions shown in this guide have the ability to scale up almost without limit and hardly need any upkeep.
You might recognize that Lambda@Edge has strong capabilities. It is useful for many things, including making images better. Others could be, for example, A/B testing, blacklisting IPs, or other access control.
Regrettably, in the usual way of AWS, there is not much documentation, and it's difficult to find. I hope that AWS will make its guides better over time and that more instructional materials will appear to help people learn this technology more easily.