Skip to content

Infrastructure

Login to Azure

The website is hosted in Azure. Most infrastructure work requires access – head over to the Azure portal, or use the Azure CLI.

Application server

We use App Service Web App for Containers. We use Docker both in production and local development, with the only difference being usage of Supervisor in production, while the local development setup uses Django’s development server.

SSH access

See SSH access.

Database

Postgres v11, via Azure Database for PostgreSQL server.

Pulling data

You can fetch data from the prod or preprod shared environments to populate your local dev site. Here are the relevant fab tasks:

fab prod.pull-data             # Copies a **live** database from Azure to the local `db` container.
fab prod.pull-media            # Retrieve media files from Azure. Requires Azure CLI.
fab prod.sync-media            # Use `azcopy sync` to sync media files from Azure. Does not delete local files not on remote.

The database is behind a firewall. To allow access, go to the desired Azure Database for PostgreSQL server product in Azure, then in Connection security click "Add current client IP address" to allow your own IP. Note the database sync pulls data directly from the live database. This has a performance cost and should be avoided for production environments.

Cache

Redis v4, via Azure Cache for Redis, with SSL-only access.

File storage

Blob service in a Azure Storage account.

Scheduled tasks

We use Azure Functions for scheduled tasks.

CDN

The Fastly CDN sits in front of the site, providing caching and path-based proxying to different applications.

Services

There are two 'services' in Fastly:

  • Tate Preprod - used for the preprod site
  • Tate Prod - used for the production site

These are kept separate so changes can be tested in the preprod service before making the same change in the prod service.

Configuration

While Fastly supports uploading a complete VCL file, Fastly's UI and 'VCL Snippets' feature is ussed to ease maintenance.

Routing

In each service, three origin hosts are configured:

  • The Tate website on Azure
  • The object storage container on Azure
  • The 'legacy' WHAM backend (running Varnish)

Conditions are specified to route traffic to each of these hosts as appropriate.

  • The object storage container host is used if the host matches the relevant media subdomain for the environment (e.g. media.tate.org.uk).
  • The WHAM backend is used if the local variable use_wham is True. This is set in a VCL Snippet called 'Route WHAM Requests' if the request URL matches any of a long list of patterns (a snippet is used rather than a condition as there is a character limit on conditions in Fastly). Once routed to WHAM, its Varnish setup will then route the request to the right place.
  • If neither of the above conditions are met, the Tate website is used.

Caching

A default caching setting automatically sets a TTL of 3600 seconds for all requests.

If a Cookie header is present, caching is disabled - this prevents caching of user-specific content or other important site functionality.

As the site sets cookies for all requests through analytics and other trackers, this would mean no requests are cached at all. To work around this, under the Headers configuration section, a condition determines if any of the important cookies are set (csrftoken and sessionid). If neither are set, the Cookie header is removed so it never hits the origin and is never used to determine caching.

Some additional rules are applied around caching for special cases:

  • Within the Route WHAM Requests snippet, cache is bypassed if any paths match.
  • Fastly's segmented caching is used for all media assets and assets from WHAM to support assets over 20MB.

Responses & Redirects

  • The 'Force TLS and enable HSTS' feature is enabled to redirect HTTP traffic to HTTPS.
  • The Apex redirects feature is enabled to redirect non-www to www.
  • Custom error pages are served from Fastly, defined in the Responses section.
  • An access control list (ACL) 'Tate Admin' is used alongside a snippet ('Only allow admin requests from Tate admin ACL') to limit access to admin areas to those in the ACL.

WAF issues

The Azure hosting environment is protected by an implementation of Fortinet's Web Application Firewall.

In some instances, the firewall has blocked the submission of legitimate Wagtail backend editor form content. Several strings have been identified as being instrumental to a rule-infringement, as perceived by the WAF.

Wagtail's backend editor form uses conventional name/values in POST submissions. However, in keeping with the flexible nature of StreamField content, form values relating to StreamField-enabled fields, use JSON. Some online references suggest this may be a factor in the WAF's perception of a rule infringement. The blocking also occurs when the strings are used in non-JSON field values. However, editor form content will always include some JSON-based values so identifying a cause more precisely is not practical. For more details see: tate.utils.content_parser.py