Skip to content

Amazon S3 Media Storage (OrchardCore.Media.AmazonS3)

The Amazon Media Storage feature enables support for storing assets in Amazon S3 Bucket.

The feature replaces the default App_Data file-based media store with an Amazon Media Storage Provider.

Media is still served by the Orchard Core website, and the Media Cache module takes responsibility for fetching media, on the fly, from the Amazon S3 Bucket.

This allows the Amazon Media Storage feature to support image resizing on the fly through integration with ImageSharp.Web.

The URL generated by the AssetUrl helpers points to the Orchard Core website.

Configuration

The following configuration values are used by default and can be customized:

{
   "OrchardCore": {
       "OrchardCore_Media_AmazonS3": {
           // If you have AWS CLI installed and configured you may just specify a profile name.
           "Profile": "",
           // In case your AWS profiles are located not in the default place.
           "ProfilesLocation": "",
           "Region": "",
           // This section needed only if Orchard will be hosted not in the AWS Cloud
           // You can obtain all that information in the IAM Management Console
            "Credentials": {
              "SecretKey": "",
              "AccessKey": ""
           },
           // Optionally, set to a path to store media in a subdirectory inside your container.
           "BasePath": "/media",
           "CreateBucket": false,
           // Your AWS S3 Bucket name.
           "BucketName": ""
       }
  }
}

Refer also to the Configuration Section, and the Media Section for other Media related configuration settings.

There are two hosting options: inside and outside AWS Cloud. In case you are hosting Orchard Core inside AWS (EC2, EKS, etc.) you need to configure only BucketName and you may delete or comment out the other sections.

In case you are hosting Orchard Core outside of AWS, you should fill the Credentials section or if you have AWS CLI installed and configured on your server you may specify only configured profile name (default if a profile name was not chosen during AWS CLI configuration).

You can find region endpoints in the Official AWS S3 Documentation, see Region column. For example for the Frankfurt region you should use eu-central-1

AWS Credentials and its loading order

OrchardCore_Media_AmazonS3 is a subset of AWSOptions configuration and should be configured the same as a generic AWSOptions.

Credentials loading order

  1. Credentials property of `AWSOptions˙.
  2. Shared Credentials File (Custom Location). When both the profile and profile location are specified.
  3. SDK Store (Windows Only). When an instance of `AWSOptions˙ is provided and only the profile is set (profile location is null or empty).
  4. Shared Credentials File (Default Location). When an instance of `AWSOptions˙ is provided and only the profile is set (profile location is null or empty).
  5. AWS Web Identity Federation Credentials. When an OIDC token file exists and is set in the environment variables.
  6. CredentialsProfileStoreChain
    1. SDK Store (Windows Only) encrypted using Windows Data Protection API.
    2. Shared Credentials File in the default location.
  7. Environment variables. When the Access Key ID and Secret Access Key environment variables are set.
  8. ECS Task Credentials or EC2 Instance Credentials. When using IAM roles with ECS tasks and ECS instances.

Note

The AWS team wants to encourage using profiles instead of embedding credentials directly into appsettings.X.json files where they would accidentally get checked into source control. If you have an option to use profiles or environment variables - you should use it instead of direct credentials.

AWS S3 Bucket Configuration

If CreateBucket was configured as true and BucketName follows official Bucket naming rules, then a new bucket will be created. The new bucket will be created without Access Control Lists due to security reasons. If you create the bucket manually then you need to do it with ACLs enabled. When using a previously created bucket, you may need to configure ACLs manually:

  1. Open your bucket.
  2. Go to the Permissions tab.
  3. Edit "Block public access".
  4. Tick "Block all public access".

S3 Bucket policies

By default, AWS 3S Bucket has limitations for newly uploaded files. If you want media files to be available from the outside of AWS, you should set up your bucket permissions.

The simplest way of doing it is to add a policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AddPerm",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME/YOR-BASE-PATH/*"
        }
    ]
}
After this policy will be added to your bucket permissions all newly added files will have Read permission and will be available from the outside of the Amazon Cloud.

Templating Configuration

Optionally you may use Liquid templating to further configure Amazon Media Storage, perhaps creating a bucket per tenant, or a single bucket with a base path per tenant.

The ShellSettings property is made available to the Liquid template. The BucketName property and the BasePath property are the only templatable properties.

Note

When templating the BucketName using {{ ShellSettings.Name }}, the tenant's name will be automatically lowercased, however, you must also make sure the BucketName conforms to other Amazon S3 naming conventions as set out in Amazon's documentation.

Configuring a bucket per tenant

{
    "OrchardCore": {
        "OrchardCore_Media_AmazonS3": {
            "BucketName": "{{ ShellSettings.Name }}-media",
            "Region": "",
            "Credentials": {
                "SecretKey": "",
                "AccessKey": ""
            },
            "BasePath": "/media",
            "Profile": "",
            "ProfilesLocation": ""
        }
    }
}

Configuring a single bucket, with a base folder per tenant

{
    "OrchardCore": {
        "OrchardCore_Media_AmazonS3": {
            "BucketName": "",
            "Region": "",
            "Credentials": {
                "SecretKey": "",
                "AccessKey": ""
            },
            "BasePath": "{{ ShellSettings.Name }}/Media",
            "Profile": "",
            "ProfilesLocation" : ""
        }
    }
}

Media Cache

The Media Cache feature will automatically be enabled when Amazon Media Storage is enabled.

The Media Cache feature will cache files stored in Amazon S3 Storage, to support image resizing.

The Media Cache feature allows Purging of the Media Cache files stored locally.

You might choose to use the Purging feature if you are fronting the media assets with a CDN. After allowing a long enough period of time for the CDN to have fetched a significant number of Media assets, both resized, and full size, from the Media Cache you might consider purging the cache.

However please bear in mind that your CDN provider will likely have multiple Points of Presence worldwide, and each of these will maintain their own cache, so while a local CDN PoP might have the asset another PoP may not, until it is requested. At this stage the Media Cache will, if necessary, refetch the asset from Amazon S3 Storage, on the fly, and provide it to the CDN PoP.

CDN providers also clear their caches at pre-determined times of their own devising, so while CDNs are a valuable caching and performance asset, it is important that they are always be able to re-fetch the source file, as and when required, which the Media Cache Module will automatically handle.

Note

The Media Feature is designed to support one storage provider at a time, whether that is local File Storage (the default), Azure Blob Storage, or Amazon S3 Storage.


Last update: June 27, 2022