# Guide to using Auto Escape

## Introduction

Auto Escape is an optional mode of execution in the Template System developed to provide a better defense against cross-site scripting (XSS) in web applications. In this mode, the Template System assumes responsibility for applying the proper escaping modifiers for each variable in your template (and templates it may include). As such, the template developer no longer needs to manually apply escaping modifiers to each variable, a process which is repetitive and error-prone particularly in larger and more complex applications.

In order for the Template System to properly determine the escaping modifiers to apply to variables, it activates a built-in HTML-aware parser during template initialization. The parser scans the template to determines the context in which each variable is being emitted. The scanning is performed at initialization-time only hence does not contribute to processing costs at template expansion time.

Refer to Security Considerations for additional security information on escaping directives and their use according to the context in which content is being expanded.

## Current Status

Auto Escape currently supports well `HTML` and `Javascript` templates. It also provides very basic support for `CSS`, `JSON` and `XML` templates where it applies the corresponding escaping modifier but without utilizing a parser. We may in the future provide parsers specific to these contexts and modifiers for them that are finer-grained.

## Overview

This section provides more background on Auto Escape. Refer to How to Auto Escape for information on how to leverage this mode in your application.

### Using template modifiers

In the simplest case, where you only want to use template-modifiers to achieve html-safety (that is, you only use `html_escape`, `url_query_escape`, and so forth), you can stop specifying template-modifiers in your template at all. The Auto Escape mode will generate the modifiers on its own and perform the correct escaping.

If you need other types of modifiers, including your own custom modifiers, you can still specify them in your template and they will continue to work just the same. The Auto Escape mode will simply apply the appropriate escaping modifiers after your own.

Other reasons for manually specifying variable modifiers include:

1. To indicate a variable is safe. You may add the `:none` modifier as the last modifier for a given variable, in which case the Auto Escape mode will leave it untouched and not apply escaping directives to it. Use only when you are sure the variable is trusted to be safe from XSS.
2. To select a different escaping modifier from a list of equivalent modifiers. Certain modifiers are equivalent from an escaping point of view in that they would all ensure given content is safe when escaped in a specific context. The Auto Escape mode choses the most common escaping modifier for that use but allows you to indicate a different choice as shown below:

   TemplateContext | Primary Modifier | Accepted alternatives
   --- | --- | ---
   `TC_HTML` | `:html_escape` | `:pre_escape` `:html_escape_with_arg=snippet` `:html_escape_with_arg=attribute` `:html_escape_with_arg=url` `:url_query_escape` `:url_escape_with_arg=html`
   `TC_HTML` and `TC_CSS` | `:cleanse_css` | `:url_escape_with_arg=css`
   `TC_XML` | `:xml_escape` | `:html_escape` `:html_escape_with_arg=attribute`
   `TC_JSON` | `:javascript_escape` | `:json_escape`

3. To add additional escaping modifiers. The Template System will never remove escaping directives you explicitly specify.

### Escaping Contexts

The table belows indicates which modifier Auto-Escape selects to escape a given variable, based on the context of the variable being inserted and, possibly, on more specific information such as whether the variable is inside quotation marks. The table only applies for the well-supported TemplateContext values (currently `TC_HTML` and `TC_JS`).

In a few cases, we do not have a modifier that is both safe against XSS and respecting of the semantics of the variable therefore we fail the template initialization. An error is logged indicating the cause of the failure.

Context | HTML Quoted? | Examples | Action Performed
--- | --- | --- | ---
Regular HTML Body and HTML comments | Any |
    <p>Hello {{USER}}</p>
| Escape `USER` using `:html_escape`
In URL attribute: Starts at pos 0 | Yes |
    <img src="{{URL}}">;
| Escape `URL` using `:url_escape_with_arg=html`
In URL attribute: Other | Yes |
    <a href="/foo?q={{QUERY}}">
| Escape `QUERY` using `:html_escape`
In URL attribute: Starting at pos 0 | No |
    <form action={{URL}}>
| Fail template initialization
In URL attribute: Other | No |
    <a href=/foo?q={{QUERY}}>My Link</a>
| Escape `QUERY` using `:url_query_escape`
In STYLE attribute | Yes |
    <div style="color:{{COLOR}};">
| Escape `COLOR` using `:cleanse_css`
In STYLE attribute | No |
    <div style=color:{{COLOR}};>
| Fail template initialization
In Javascript attribute: String literal | Yes |
    <a href="url" onclick="doFoo('{{ARG}}');">
| Escape `ARG` using `:javascript_escape`
In Javascript attribute: Non-string literal | Yes |
    <a href="url" onclick="doFoo({{ARG}});">
| Escape `ARG` using `:javascript_escape_with_arg=number`
In Javascript attribute: Any | No |
    <a href="url" onclick=doFoo('{{ARG}}');>
| Fail template initialization.
In all other attributes | Yes |
    <b class="{{CLASS}}">
| Escape `CLASS` using `:html_escape`
In all other attributes | No |
    <table border={{BORDER}}>
| Escape `BORDER` using `:html_escape_with_arg=attribute`
In Javascript code: In a string literal | Any |
    <script>var a = '{{VALUE}}';</script>
| Escape `VALUE` using `:javascript_escape`
In Javascript code: Non-string literal | Any |
    <script>var a = {{VALUE}};</script>
| Escape `VALUE` using `:javascript_escape_with_arg=number`
In a <style> tag | Any |
    <style>font-size={{FONTSIZE}};</style>
| Escape `FONTSIZE` using `:cleanse_css`

#### Comments:

- For values of URL-accepting attributes, we apply a different modifier if the variable is at the beginning of the attribute value or not because in the former case we consider the variable to represent a complete URL and hence also check that its scheme is safe (`HTTP` or `HTTPS`).
- We recommend that you enclose attribute values in quotes in particular:
  1. Under some conditions outlined in the table above, when variables are present in attribute values that are not enclosed in quotes, we may fail template initialization because we do not have escaping modifiers that are safe to use.
  2. Also the determination of which escaping modifier to apply for unquoted attribute values specifically is likely to change in the future.

The Auto Escape mode is designed to be simple to use and to produce correct correct escaping with minimal input. It does however come with restrictions governing the use of this template system.

### Limitations

- Templates may only be included in regular HTML text. Including a template within HTML comments or inside an HTML tag is currently not supported. In the example below, `HEADER` and `FOOTER` are properly included but `TAG` and `COMMENT` are not.

                <html><head>{{>HEADER}}</head><body>
                <{{>TAG}}>
                <p>
                <!-- {{>COMMENT}} -->
                <p>{{>FOOTER}}
                </body></html>

  If the template system detects that you are including a template in another HTML context, it will log a warning.

- Included templates may not cross HTML boundaries. They cannot start with one `TemplateContext` and end in another. For example, the template below crosses boundaries -- it starts in HTML and ends in Javascript -- and is therefore not a valid included template for auto-escaping.

              <html>
                <head>
                </head>
                <body>
                  Hello USER.
                    <script>
                       var a = 'not a nice template';

  To rectify this violation, continue the template by completing the javascript code and terminating it with a `</script>` tag.

- Error reporting. We currently report errors by logging a warning or an error during initial template initialization and parsing if we suspect something is incorrect. Please check for them, as they indicate an error that may cause auto-escaping to work improperly.

- Beware of double escaping. When you use the Auto Escape mode, you surrender all variable escaping to the template system. If you also perform your own escaping in your source code, you may face situations where content was escaped multiple times.

  In particular, do not use any of the legacy methods below or others that perform variable escaping.

  - `SetEscapedValue`
  - `SetEscapedFormatttedValue`
  - `SetEscapedValueAndShowSection`

- Supported HTML contexts. HTML (`TC_HTML`) and Javascript (`TC_JS`) templates are well supported. Other template contexts have only basic support. For these contexts, variables are escaped as follows:

  TemplateContext | Escaping Applied
  --- | ---
  TC_JSON | `:javascript_escape`
  TC_XML | `:xml_escape`
  TC_CSS | `:cleanse_css`

- New restrictions on the use of `BI_SPACE` and `BI_NEWLINE`. These system variables may not be re-assigned to a semantically different value via `dict->SetValue` or related methods, and they may not have modifiers attached to them in templates. For instance: it's ok to redefine `BI_NEWLINE` from `\n` to `\r\n`, since html treats them identically. But changing it to `hello, world!` is not allowed.

#### A new filename convention

The Auto Escape feature codifies simple conventions of template filenames:

1. CSS templates optionally include, in their filename, the keywords `style` or `stylesheet` or `css`.
2. Javascript templates optionally include, in their filename, the keywords `js` or `javascript`.

If we find one of these keywords in the filename but it does not match the template type given, we log a warning.

- Howto
- Tips
- Example