# How To Use the Google Template System

## Motivation

A template system can be used to separate output formatting specifications, which govern the appearance and location of output text and data elements, from the executable logic which prepares the data and makes decisions about what appears in the output.

Template systems lie along a continuum of power versus separation. "Powerful" constructs like variable assignment or conditional statements make it easy to modify the look of an application within the template system exclusively, without having to modify any of the underlying "application logic". They do so, however, at the cost of separation, turning the templates themselves into part of the application logic.

This template system leans strongly towards preserving the separation of logic and presentation. It is intentionally constrained in the features it supports and, as a result, applications tend to require quite a bit of code to instantiate a template. This may not be to everybody's tastes. However, while this design limits the power of the template language, it does not limit the power or flexibility of the template system. This system supports arbitrarily complex text formatting. Many Google applications, including the "main" Google web search, use this system exclusively for formatting output.

Finally, this system is designed with an eye towards efficiency. Template instantiation is very quick, with an eye towards minimizing both memory use and memory fragmentation.

## Overview

There are two parts to the Google Template System:

- Templates
- Data dictionaries

The templates are text files that contain the format specification for the formatted output, i.e, the template language. The data dictionaries contain the mappings from the template elements (markers) embedded in the templates to the data that they will format. Here's a simple template:

       <html><head><title>{{TITLE}}</title>{{META_TAGS}}</head>
       <body>{{BODY}}</body></html>

Here's a dictionary that one could use to instantiate the template:

       {"TITLE": "Template example",
        "BODY": "This is a simple template example.\nIt's boring",
        "DATE": "11/20/2005"}

If we instantiated the template with this dictionary, here's the output we would get:

       <html><head><title>Template example</title></head>
       <body>This is a simple template example.
    It's boring</body></html>

`{{TITLE}}` and `{{{BODY}}` are template elements, also called markers. In the dictionary, `TITLE`, `BODY`, and `DATE` are dictionary names, and the values associated with each one, such as `11/20/2005`, are dictionary values.

A few points are clear even from this simple example:

1. Dictionary keys and values are strings; the Google template system is not typed.
2. Dictionary values come already formatted. It was up to the application code to decide how to format the value for `DATE`, and to insert the date into the dictionary already formatted.
3. Not all dictionary values must be used by a template. `DATE` is entirely ignored.
4. Not all template elements may exist in the dictionary. In this example, `{{META_TAGS}}` is not found in the dictionary. This is perfectly legal; missing variable markers evaluate to the empty string.

### Templates

The template language has six types of markers:

1. VARIABLE markers, which are replaced by text based on dictionary values. All markers in the above example are variable markers. Variable markers look like this: `{{VARIABLE}}`
2. START SECTION and END SECTION markers, which delimit sections which may appear zero, one, or N times in the output. The number of times a section appears is determined by the data dictionaries, as explained below. Each time a section is expanded, it uses a different dictionary, so that the output values may be different from one iteration of a section expansion to another. Note that the specification of how sections expand is entirely dependent on the dictionary, as set up by the application; there is no way to specify a repeat rate in the template language itself. Section markers look like this: `{{#SECTION_NAME}}...{{/SECTION_NAME}}`
3. TEMPLATE-INCLUDE markers, which designate other templates to be expanded and inserted at the location where the marker appears. These are treated much like sections -- one may think of them as sections whose content is specified in a different file instead of inline -- and just like sections, can be expanded zero, one or N times in the output, each with a different dictionary. Template-include markers look like this: `{{>FILENAME}}`
4. COMMENT markers, which may annotate the template structure but drop completely out of the expanded output. Comment markers look like this: `{{! comment lives here -- cool, no?}}`
5. SET-DELIMITER markers, which change the marker delimiters from `{{` and `}}` to custom strings. (The only requirement is that these strings not contain whitespace or the equals sign.) This is useful for languages like TeX, where double-braces may occur in the text and are awkward to use for markup. Set-delimiter markers look like this: `{{=< >=}} <! Now markers are delimited by braces > <=| |=> |! And now markers are delimited by bars! |`
6. PRAGMA markers, which invoke additional built-in template features when processing the template. Pragma markers look like this: `{{%PRAGMA [name="value"...]}}`. Currently, AUTOESCAPE is the only pragma defined, see Auto Escape Mode for using it to enable Auto-Escape for a given template.

These marker types each have their own namespace. For readability, however, it is best to not overuse a single name.

Anything found in a template of the form `{{...}}` is interpreted as a template marker. All other text is considered formatting text and is output verbatim at template expansion time. Formatting text may consist of HTML tags, XML tags, linefeeds and other spacing characters, constant text, etc.

### Data Dictionaries

A data dictionary is a map from keys to values. The keys are always strings, each string representing either a variable, a section, or a template-include file. (Comments are not stored in the data dictionary!) These values correspond to the name of the associated template marker: a section `{{#FOO}}` in the template text is matched to the key `"FOO"` in the dictionary, if it exists. Note the case must match as well.

The value associated with a key differs according to key type. The value associated with a variable is simple: it's the value for that variable. Both keys and values can be any 8-bit character-string, and may include internal NULs (\0).

The value associated with a section is more complicated, and somewhat recursive: it's a list of data dictionaries. Come template-expansion time, the section is expanded once for each dictionary in the list, so if there are two dictionaries in the list, then the section text will occur in the output twice. The first time, all variables/etc. in the section will be evaluated taking into account the first dictionary. The second time, all variables/etc. will be evaluated taking into account the second dictionary. (See below for a definition of "taking into account.")

A template-include is a special type of section, so the associated value is the same: a list of dictionaries. Template-includes also have one other, mandatory associated piece of information: the filename of the template to include. This filename may be specified either as an absolute path, or as a relative path. (In the latter case, the path is taken relative to the template_root, as set by the application.)

The application program is responsible for building this data dictionary, including all nesting. It then applies this dictionary to a single template to produce formatted output.

### Expanding a Template

A program using Google Templates typically reads in templates at load time. During the course of program execution, the program will repeatedly perform the following two steps: first, instantiate a data dictionary, and second, apply the dictionary to the template to produce output.

The template system applies a dictionary to a template by finding all template markers in the template, and replacing them with the appropriate dictionary values. It matches template markers to dictionary keys in the obvious way. For instance, a template marker `{{FOO}}` matches the dictionary key `FOO`. `{{FOO:html_escape}}` matches `FOO` as well. The marker `{{#BAR}}` matches the dictionary key `BAR`, as does the marker `{{/BAR}}`. The marker `{{>BAZ}}` matches the dictionary key `BAZ`. (And of course, the marker `{{! comment}}` doesn't match any dictionary key at all.)

Template-variables can also have modifiers. In that case, the template-system starts by finding the appropriate value for that variable in the dictionary, just like normal. Then it applies each modifier to the variable, left to right. Finally, it emits the modified value to the output. Template-includes can have modifiers in a similar way. In such cases, after the sub-template is expanded, but before its content is injected into the current template, it has the modifiers applied.

If no dictionary key is found for a given template marker, then the template marker is ignored: if a variable, it expands to the empty string; if a section or include-template, the section or include-template is expanded zero times.

All names are case sensitive. Names -- that is, variable keys and, as a result, template markers -- must be made of (7-bit ascii) alphanumeric characters and the underscore. The commment marker, which does not map to dictionary keys, may contain any chararacters whatsoever except `}`, the close-curly brace. It's a syntax error for any template marker to violate this rule.

Outside of the template markers, templates may contain any text whatsoever, including (single) curly braces and NUL characters.

### Modifiers

Recall that variables look like this: `{{VARNAME}}`. We actually allow a more generic form: the variable name may be followed by one or more modifiers. A modifier is a filter that's applied at template-expand time, that munges the value of the variable before it's output. For instance, consider a template that looks like this:

       <html><body>{{NAME:html_escape}}</body></html>

This asks the template system to apply the built-in `html_escape` modifier when expanding `{{NAME}}`. If you set `NAME` in your dictionary to be `Jim & Bob`, what will actually be emitted in the template is `Jim &amp; Bob`.

Modifiers work for variable names and also for template-includes: `{{>SUB_TEMPLATE:html_escape}}` means that when you expand `SUB_TEMPLATE`, html-escape the expanded text before inserting it into the current template.

You can chain modifiers together. This template first html-escapes `NAME`, and then javascript-escapes that result:

       <html><body>{{NAME:html_escape:javascript_escape}}</body></html>

Modifiers typically have a long, descriptive name and also a one-letter abbreviation. So this example is equivalent to the previous one:

       <html><body>{{NAME:h:j}}</body></html>

Here are the modifiers that are built in to the template system:

long name | short name | description
--- | --- | ---
`:html_escape` | `:h` | html-escapes the variable before output (eg `&` -> `&amp;`)
`:pre_escape` | `:p` | pre-escapes the variable before output (same as html_escape but whitespace is preserved; useful for <pre>...</pre>)
`:url_query_escape` | `:u` | performs URL escaping on the variable before output. space is turned into +, and everything other than [0-9a-zA-Z.,_:*/~!()-], is transformed into %-style escapes. Use this when you are building URLs with variables as parameters:
    <a href="http://google.com/search?q={{QUERY:u}}">{{QUERY:h}}</a>
`:javascript_escape` | `:j` | javascript-escapes the variable before output (eg `"` -> `\x27` and `&` -> `\x26`)
`:javascript_escape_with_arg` | `:J` | special purpose javascript escaping. See below for details.
`:cleanse_css` | `:c` | Removes characters not safe for a CSS value. Safe characters are alphanumeric, space, underscore, period, coma, exclamation mark, pound, percent, and dash.
`:json_escape` | `:o` | json-escapes a variable before output as a string in json; HTML characters are escaped using Unicode escape sequences (e.g `&` -> `\u0026`) to comply with RFC 4627.
`:html_escape_with_arg` | `:H` | special purpose html escaping. See below for details
`:url_escape_with_arg` | `:U` | special purpose url escaping. See below for details
`:xml_escape` |  | xml-escapes the variable before output (the five characters `<>&"'` become `&lt&gt;&amp;&quot;&#39;`) suitable for content returned in raw XML. It is not intended for escaping content within CDATA blocks.
`:none` |  | leaves the variable as is

The `html_escape_with_arg` and `url_escape_with_arg` modifiers are a bit different because they require a value to specify the type of escaping to use. For example, this template is equivalent to using the `pre_escape` modifier:

       <html><body><pre>{{BODY:H=pre}}</pre></body></html>

Here are the values that are supported by the `html_escape_with_arg` modifier:

value | description
--- | ---
`=snippet` | like `html_escape`, but allows HTML entities and some tags to pass through unchanged. The allowed tags are `<br>`, `<wbr>`, `<b>`, and `</b>`.
`=pre` | same as `pre_escape`
`=url` | same as `:U=html` below. For backwards compatibility.
`=attribute` | replaces characters not safe for an use in an unquoted attribute with underscore. Safe characters are alphanumeric, underscore, dash, period, and colon.

Here are the values that are supported by the `url_escape_with_arg` modifier:

value | description
--- | ---
`=html` | Ensures that a variable contains a safe URL. Safe means that it is either a http or https URL, or else it has no protocol specified. If the URL is safe it is html-escaped, otherwise it is replaced with `#`.
`=javascript` | Same as `=html`, but using javascript escaping instead of html escaping.
`=css` | Same as `=html` but using CSS escaping instead of html escaping so that the variable can be safely inserted within a CSS `@import` statement or a CSS property. The characters in [\r\n()'"<>*\] are transformed into %-style escapes.
`=query` | Same as `url_query_escape`.

Here are the values that are supported by the `javascript_escape_with_arg` modifier:

value | description
--- | ---
`=number` | Ensures that the variable is a valid number or boolean javascript literal. This includes booleans `true` and `false`, decimal numbers (e.g. `4.10` or `-5.01e+10`) as well as hex numbers (e.g. `0x5FF`). This modifier is intended to ensure the variable not enclosed in quotes cannot contain javascript code that may execute. It does not guarantee that the variable is syntatically well-formed. If the variable is safe, it is returned as-is, otherwise it is replaced with `null`. In the future we may add more logic to support objects and arrays.

NOTE: At the moment, there are no filters for handling XML attributes and text nodes. For HTML snippets, use the html filter; in other situations, it may be appropriate to use CDATA blocks.

In addition to the built-in modifiers, you can write your own modifier. Custom modifiers must have a name starting with "x-", and the name can contain alphanumeric characters plus dashes and underscores. Custom modifiers can also accept values with any character except for `:` and `}`. For example this template could be a valid use of a custom modifier:

    {{VAR:x-my_modifier:value1,value2,value3 has spaces,etc}}

See `<template_modifiers.h>` for details on how to write a modifier and how to register it. Here is an example of the code for a custom modifier:

       class StarEscape : public ctemplate::TemplateModifier {
         void Modify(const char* in, size_t inlen,
                     const ctemplate::PerExpandData* per_expand_data,
                     ExpandEmitter* outbuf, const string& arg) const {
           outbuf->Emit(string("*") + string(in, inlen) + string("*"));
         }
       };

### Special Section Names

Section names may have a special meaning in the template system. Right now, there's one such name.

#### Separator Sections

If you have a section named `SEP`, you can define inside of it a section named `SEP_separator`, and the template system will automatically expand that section every time `SEP` is expanded, except for the last. Thus, the contents of `SEP_separator` can be used to separate repeated values of a section.

Here's an example:

       Here are the meeting attendees:
       {{#ATTENDEES}}
          {{NAME}}
          {{#ATTENDEES_separator}}, {{/ATTENDEES_separator}}
       {{/ATTENDEES}}
       .

Here is a more convoluted example, to show the date:

       {{#DATE}}{{DATE_COMPONENT}}{{#DATE_separator}}{{DATE_SEP}}{{/DATE_separator}}{{/DATE}}

You'd set up a template dictionary to repeat `DATE` three times, with `DATE_COMPONENT` set to the month, day, and year (or day, month, and year, depending on your locale...), and `DATE_SEP` set to `/` or `-` or whatever date-separator is called for.

`SEP_separator` is always evaluated with the current dictionary. Thus, in the date example, if you wanted a different separator each time, you could do so by setting `DATE_SEP` to a different value for each repetition of `DATE`.

While `SEP_separator` is automatically expanded by the template system, it is otherwise a perfectly normal section. You can even instantiate it yourself by calling `AddSectionDictionary("SEP_separator")`. In that case, the section will be expanded both via the automatic expansion as a separator, and as a normal section via the section dictionary you added. This is more confusing than helpful, and you should probably never do it.

There can be at most one "separator" sub-section per section. If there are more, only the last is automatically expanded.

### Auto Escape Mode

The Auto Escape mode helps protect against cross-site scripting (XSS) attacks in web-applications by automatically applying escaping modifiers to your variables. Refer to Guide to using Auto Escape for an overview of Auto Escape as well as its limitations.

Auto Escape is enabled on a template-by-template basis. Simply add the AUTOESCAPE pragma to the desired template. That template will then be automatically escaped, independently of the templates it may include or it may be included from. The AUTOESCAPE pragma must be placed at the top of the template and has the following syntax:

      {{%AUTOESCAPE context="CONTEXT" [state="STATE"]}}

Description of the pragma:

- The context attribute is one of `HTML`, `JAVASCRIPT`, `CSS`, `JSON` or `XML`. It must correspond to the context in which the browser will interpret this template. Warning: Setting the wrong context will result in wrong escaping applied to all variables in the given template. In particular, if the template starts with a `<script>` tag, its context is `HTML` and not `JAVASCRIPT`. Auto-Escape will recognize the `<script>` tag and hence properly Javascript-escape the variables within it. Similarly, if the template starts with a `<style>` tag, its context is `HTML` and not `CSS`.
- The state attribute is commonly omitted. It accepts the value `IN_TAG` in the `HTML` context to indicate that the template only contains (one or more) HTML attribute name and value pairs that are part of an HTML tag formed in a parent template.

#### Examples

The table below shows four small sample templates along with their corresponding AUTOESCAPE pragma. The two most common contexts are `HTML` and `JAVASCRIPT`. We also show a sample template for the `CSS` context as well as a sample template for the `IN_TAG` state of `HTML` (although it is expected to be rarely used).

HTML | JAVASCRIPT | CSS | HTML IN_TAG (uncommon)
--- | --- | --- | ---
        {{%AUTOESCAPE context="HTML"}}

        <body>
          <p>Hello {{USER}}</p>
          <p><a href="{{URL}}">Your Account</a></p>
        </body>
        {{%AUTOESCAPE context="JAVASCRIPT"}}

        function showMessage(user, msg) {
          alert("Hello: " + user + " Message: " + msg);
        }

        var user = '{{USER}}';
        var msg = '{{MSG}}';
        showMessage(user, msg);
        {{%AUTOESCAPE context="CSS"}}

        P.abstract {
          text-align:{{EDGE}};
          font-size:{{FONT_SIZE_PC}};
        }
        .italic {font-style:{{ITALIC}}}
        {{%AUTOESCAPE context="HTML" state="IN_TAG"}}

        class="{{CLASS}}" id="{{ID}}"

### Details on Dictionary Lookup

The dictionary structure is a tree: there's a 'main' dictionary, and then sub-dictionaries for each section or include-template. Even with all this complexity, the lookup rules are mostly straightforward: when looking up a marker -- be it a variable, section, or include-template marker -- the system looks in the currently applicable dictionary. If it's found there, great. If not, and the parent dictionary is not an include-template, it continues the look in the parent dictionary, and possibly the grandparent, etc. That is, lookup has static scoping: you look in your dictionary and any parent dictionary that is associated with the same template-file. As soon as continuing the lookup would require you to jump to a new template-file (which is what include-template would do), we stop the lookup.

For instance, for a template that says `{{#RESULTS}}{{RESULTNUM}}. {{>ONE_RESULT}}{{/RESULTS}}`, `"ONE_RESULT"` is looked for in the "RESULTS" dictionary, and if not found there, is looked for in the main, top-level dictionary. Likewise, the variable `"RESULTNUM"` is looked for first in the "RESULTS" dictionary, then in the main dictionary if necessary. However, "ONE_RESULT" will not do equivalent cascading lookups. In fact, it will have no parent dictionaries at all, because it's a different template file and thus in a different scope.

Because of these scoping rules, it's perfectly reasonable to set all variables that are needed in a given template file, in the top-level dictionary for that template. In fact, the `ShowSection()` function is provided to support just this idiom. To avoid confusion in such a usage mode, it's strongly encouraged that you give unique names to all sections and include-templates in a single template file. (It's no problem, given the template scoping rules, for a single section or include-template name to be repeated across different template files.)

There's a single special case: the global variable dictionary. Every dictionary inherits its initial set of values from the global dictionary. Clients can set variables in the global dictionary just like they can in normal template dictionaries they create.

The system initializes the global dictionary with a few useful values for your convenience. All system variables are prefixed with `BI`, to emphasize they are "built in" variables.

- `BI_SPACE`, which has the value `<space>`. It is used to force a space at the beginning or end of a line in the template, where it would normally be suppressed. (See below.)
- `BI_NEWLINE`, which has the value `<newline>` It is used to force a newline at the end of a line, where it would normally be suppressed. (See below.)

As is usual for inheritance, if a user explicitly assigns a value to these variable-names in its own dictionary, this overrides the inherited value. So, `dict->SetValue("BI_SPACE", "&nbsp;")` causes `BI_SPACE` to have the value `&nbsp;`, rather than `<space>`, when expanding `dict`.

Note that only variables can be inherited from the global dictionary, not section dictionaries or include-file dictionaries.

A couple of small implementation notes: global inheritance is "last chance", so if a section's parent dictionary redefined `BI_SPACE`, say, the section dictionary inherits the parent-dict value, not the global-dict value. Second, variable inheritance happens at expand time, not at dictionary-create time. So if you create a section dictionary, and then afterwards set a variable in its parent dictionary (or in the global dictionary), the section will inherit that variable value, if it doesn't define the value itself.

## Writing Application Code To Use Templates

Most application code concerns filling a template dictionary, but there is also code for loading templates themselves from disk. A final category of code lets you inspect and control the template system.

The code below assumes the default configuration option of putting all template code in namespace `google`.

### Loading A Template

The main routine to load a template is `ctemplate::Template::GetTemplate()`, defined in `template.h`. This is a static, factory method, that loads a template from either disk or from an internal template cache, and returns a pointer to a `Template` object. Besides a filename to load from, this routine takes a 'strip' argument which defines how to expand whitespace found in a template file. It can have one of the following values:

- `ctemplate::DO_NOT_STRIP`: do nothing. This expands the template file verbatim.
- `ctemplate::STRIP_BLANK_LINES`: remove all blank lines when expanding. This ignores any blank lines found in the template file when expanding. When the template is html, this reduces the size of the output text without requiring a sacrifice of readability for the input file.
- `ctemplate::STRIP_WHITESPACE`: remove not only blank lines when expanding, but also whitespace at the beginning and end of each line. It also removes any linefeed (possibly following whitespace) that follows a closing '}}' of any kind of template marker except a template variable. (This means a linefeed may be removed anywhere by simply placing a comment marker as the last element on the line.) When the template is html, this reduces the size of the output html without changing the way it renders (except in a few special cases). When using this flag, the built-in template variables `BI_NEWLINE` and `BI_SPACE` can be useful to force a space or newline in a particular situation.

This factory method returns NULL if the template cannot be found, or if there is a syntax error trying to load it.

Besides loading templates, the application can also ask the template system to reload a template, via `template->ReloadIfChanged()`. (You can also reload all templates at once via `ctemplate::Template::ReloadAllIfChanged()`.) `ReloadIfChanged()` looks on disk, and if it notices the template file has changed since the last load, it will reload the template from disk, replacing the old contents. Actually, the reload is done lazily: `ReloadIfChanged` just sets a bit that causes the template to be reloaded next time `GetTemplate` is called.

### Creating A Template Dictionary

The class `ctemplate::TemplateDictionary` is used for all template dictionary operations. `new ctemplate::TemplateDictionary(name)` is used to create a new top-level dictionary. `dict->AddSectionDictionary(name)` and `dict->AddIncludeDictionary(name)` are used to create sub-dictionaries for sections or include-files. After creating a dictionary, the application should call one or more functions for each marker in the template. As an example, consider the following template:

    <html><body> {{! This page has no head section.}}
    {{#CHANGE_USER}}
    <A HREF="/login">Click here</A> if you are not {{USERNAME}}<br>
    {{/CHANGE_USER}}

    Last five searches:<ol>
    {{#PREV_SEARCHES}
    <li> {{PREV_SEARCH}}
    {{/PREV_SEARCHES}}
    </ol>

    {{>RESULT_TEMPLATE}}

    {{FOOTER}}
    </body></html>

To instantiate the template, the user should call a function to set up `FOOTER`, and a function to say what to do for the sections `CHANGE_USER` and `PREV_SEARCHES`, and for the include-template `RESULT_TEMPLATE`. Quite likely, the application will also want to create a sub-dictionary for `CHANGE_USER`, and in that sub-dictionary call a function to set up `USERNAME`. There will also be sub-dictionaries for `PREV_SEARCHES`, each of which will need to set `PREV_SEARCH`. Only when this is all set up will the application be able to apply the dictionary to the template to get output.

The appropriate function to call for a given template marker depends on its type.

#### Variables

For variables, the only interesting action is to set the variable's value. For most variables, the right method to call is `dict->SetValue(name, value)`. (The name and value can be specified as strings in a variety of ways: C++ strings, char *'s, or char *'s plus length.)

There are two other ways to set a variable's value as well, each with a different scoping rule. You can call `ctemplate::TemplateDictionary::SetGlobalValue(name, value)` -- no `TemplateDictionary` instance needed here -- to set a variable that can be used by all templates in an application. This is quite rare.

You can also call `dict->SetTemplateGlobalValue(name, value)`. This sets a variable that is seen by all child dictionaries of this dictionary: sub-sections you create via `AddSectionDictionary`, and included templates you create via `AddIncludeDictionary` (both described below). This differs from `SetValue()`, because `SetValue()` values are never inherited across template-includes. Almost always, `SetValue` is what you want; `SetTemplateGlobalValue` is intended for variables that are "global" to a particular template but not all templates, such as a color scheme to use, a language code, etc.

To make it easier to use `SetValue()`, there are a few helper routines to help setting values of a few special forms.

- `SetIntValue(name, int)`: takes an int as the value.
- `SetFormattedValue(name, fmt, ...)`: the `fmt` and `...` work just like in `printf`: `SetFormattedValue("HOMEPAGE", "http://%s/", hostname)`.

Example:

       ctemplate::TemplateDictionary* dict = new ctemplate::TemplateDictionary("var example");
       dict->SetValue("FOOTER", "Aren't these great results?");

#### Sections

Sections are used in two ways in templates. One is to expand some text multiple times. This is how `PREV_SEARCHES` is used in the example above. In this case we'll have one small sub-dictionary for each of the five previous searches the user did. To do this, call `AddSectionDictionary(section_name)` to create the sub-dictionary. It returns a `TemplateDictionary*` that you can use to fill the sub-dictionary.

The other use of sections is to conditionally show or hide a block of text at template-expand time. This is how `CHANGE_USER` is used in the example template: if the user is logged in, we show the section with the user's username, otherwise we choose not to show the section.

This second case is a special case of the first, and the "standard" way to show a section is to expand it exactly one time, by calling `AddSectionDictionary()` once, and then setting `USERNAME` in the sub-dictionary.

However, the hide/show idiom is so common there are a few convenience methods to make it simpler. The first takes advantage of the fact sections inherit variables from their parent: you set `USERNAME` in the parent dictionary, rather than a section sub-dictionary, and then call `ShowSection()`, which adds a single, empty dictionary for that section. This causes the section to be shown once, and to inherit all its variable values from its parent.

A second convenience method is written for the particular case we have with `USERNAME`: if the user's username is non-empty, we wish to show the section with `USERNAME` set to the username, otherwise we wish to hide the section and show neither `USERNAME` nor the text around it. The method `SetValueAndShowSection(name, value, section_name)` does exactly that: if value is non-empty, add a single single dictionary to `section_name` and call `section_dict->AddValue(name, value)`.

Example:

       using ctemplate::TemplateDictionary;
       TemplateDictionary* dict = new TemplateDictionary("section example");
       const char* username = GetUsername();   // returns "" for no user
       if (username[0] != '\0') {
          ctemplate::TemplateDictionary* sub_dict = dict->AddSectionDictionary("CHANGE_USER");
          sub_dict->SetValue("USERNAME", username);
       } else {
          // don't need to do anything; we want a hidden section, which is the default
       }

       // Instead of the above 'if' statement, we could have done this:
       if (username[0] != '\0') {
          dict->ShowSection("CHANGE_USER");       // adds a single, empty dictionary
          dict->SetValue("USERNAME", username);   // take advantage of inheritance
       } else {
          // don't need to do anything; we want a hidden section, which is the default
       }

       // Or we could have done this:
       dict->SetValueAndShowSection("USERNAME", username, "CHANGE_USER");

       // Moving on...
       GetPrevSearches(prev_searches, &num_prev_searches);
       if (num_prev_searches > 0) {
          for (int i = 0; i < num_prev_searches; ++i) {
             TemplateDictionary* sub_dict = dict->AddSectionDictionary("PREV_SEARCHES");
             sub_dict->SetValue("PREV_SEARCH", prev_searches[i]);
          }
       }

#### Template-includes

Template-include markers are much like section markers, so `SetIncludeDictionary(name)` acts, not surprisingly, exactly like `SetSectionDictionary(name)`. However, since variable inheritance doesn't work across include boundaries, there is no template-include equivalent to `ShowSection()` or `SetValueAndShowSection()`.

One difference bewteen template-includes and sections is that for a sub-dictionary that you create via `SetIncludeDictionary()`, you must call `subdict->SetFilename()` to indicate the name of the template to include. If you do not set this, the sub-dictionary will be ignored. The filename may be absolute, or relative, in which case it's relative to template_root.

Example:

       using ctemplate::TemplateDictionary;
       TemplateDictionary* dict = new TemplateDictionary("include example");
       GetResults(results, &num_results);
       for (int i = 0; i < num_results; ++i) {
          TemplateDictionary* sub_dict = dict->AddIncludeDictionary("RESULT_TEMPLATE");
          sub_dict->SetFilename("results.tpl");
          FillResultsTemplate(sub_dict, results[i]);
       }

In practice, it's much more likely that `FillResultsTemplate()` will be the one to call `SetFilename()`. Note that it's not an error to call `SetFilename()` on a dictionary even if the dictionary is not being used for a template-include; in that case, the function is a no-op, but is perhaps still useful as self-documenting code.

Another property of template-includes is that they set the indentation level for the included template, that is, every line in the included template is indented by the same amount as the template-includes line itself. For instance, if you have a template `PRINT_STUFF` like this:

    print "Hello!"
    print "You are the 10th caller!"
    print "Congratulations!"

and you include it in the template:

       if ShouldPrintStuff():
         {{>PRINT_STUFF}}
       else:
         pass

then when it is expanded, all three print lines will be indented, not just the first one:

       if ShouldPrintStuff():
         print "Hello!"
         print "You are the 10th caller!"
         print "Congratulations!"
       else:
         pass

Note that this behavior is immaterial when using `STRIP_WHITESPACE`, since in that case all leading whitespace is stripped.

### Expanding a Template

Once you have a template and a template dictionary, it's simplicity itself to expand the template with those dictionary values, putting the output in a string:

       ctemplate::Template* tpl = ctemplate::Template::GetTemplate(<filename>, ctemplate::STRIP_WHITESPACE);
       ctemplate::TemplateDictionary dict("debug-name");
       FillDictionary(&dict, ...);
       string output;
       bool error_free = tpl->Expand(&output, &dict);
       // output now holds the expanded template
       // Expand returns false if the system cannot load any of the template files
       // referenced by the TemplateDictionary.

The expanded template is written to the string `output`. If `output` was not empty before calling `Expand()`, the expanded template is appended to the end of `output`.

#### Per-expand data

There is a "power user" version of `Expand()`, called `ExpandWithData()`, that allows you to pass in per-expand data. It's called like this:

       ctemplate::Template* tpl = ctemplate::Template::GetTemplate(...);
       ctemlpate::TemplateDictionary dict(...);
       ctemplate::PerExpandData per_expand_data;
       string output;
       tpl->ExpandWithData(&output, &dict, &per_expand_data);

Per-expand data is applied to all templates that are seen while expanding: not only the template you called `Expand()` on, but also sub-templates that are brought in via template-includes (`{{>INCLUDE}}`).

There are several types of per-expand data you can set, by calling the appropriate method on a PerExpandData object:

- `SetAnnotateOutput()`: This is a debugging function. When expanding this template, it adds marker-strings to the output to indicate what template-substitutions the system was making. This takes a string argument which can be used to shorten the filenames printed in the annotations: if the filename contains the string you give, everything before that string is elided from the filename before printing. It's confusing, but fear not: it's safe to just always pass in the empty string.
- `Insert()`: This stores an arbitrary key/value pair in the PerExpandData structure. This is used with template modifiers: the `PerExpandData` object you pass to `ExpandWithData` is made available to every template modifier that is called during expand time (including custom modifiers you've defined yourself). The intended use of this functionality is to allow a modifier to work one way when you expand a template with dictionary A, and another way when you expand a template with dictionary B. For instance, you might have a modifier that encrypts part of a webpage using a user's secret-key, and the secret-key is of course different every time you expand the webpage.
- `SetTemplateExpansionModifier()`: This is an advanced feature for those who need a custom hook into template expansion. It will not be used by most programmers. It takes a template-modifier as its argument, but this template modifier is treated specially: instead of applying when an appropriate magic string appears in the text of a template, it applies every time a template is expanded during a call to `ExpandWithData`. Note this means that it's called not only after the top-level template, that `ExpandWithData` is called on, is expanded, but also on every sub-template that is expanded due to a template-include.
  This unusual functionality has a few unusual properties. Since the template expansion modifier is not called in normal fashion, the normal `arg` argument to the template modifier does not make sense. Instead, the `arg` is set to the path of the template which is being expanded. Also, template expansion modifiers can be expensive, since they are applied pretty indiscriminately, so it can be worth implementing the `MightModifiy` predicate for your `TemplateModifier` to avoid unnecessary work.

#### Custom Emitters

By default, `Expand()` and `ExpandWithData` expand the template into a string. If you wish, you can expand the data into another container -- perhaps directly to disk, or to a network socket -- as well. To do this, you must define your own subclass of `ExpandEmitter`, which is defined in `template_emitter.h`. This is a very simple class that just wraps an "append" method. Once you have defined a new ExpandEmitter, you can pass in an ExpandEmitter pointer to `Expand()` or `ExpandWithData()`.

### Getting a Template From a String Rather Than a File

The `Template` class includes a `StringToTemplate()` static method for when you want your template to be built into the executable rather than read from a file. `StringToTemplate()` parses the string you pass in as if it were a template file, and returns a `Template*` object (assuming the parsing was successful). This object is not cached nor is it owned by the template system; you should delete it when you are done.

Another string-to-template method is the static method `StringToTemplateCache`. Unlike `StringToTemplate`, it does not return a `Template*`, instead it inserts the parsed string into the internal template cache, where it can be retrieved via the normal `GetTemplate` call. This is useful when you want to retrieve this template repeatedly in different contexts, and also when you want to include this template inside another template (for instance, via `{{>TEMPLATE_THAT_COMES_FROM_A_STRING}}`). The latter is possible because `StringToTemplateCache` takes a "key", which you can later pass in to `ctemplate::TemplateDictionary::SetFilename()` in order to set up the template-include.

Future calls to `StringToTemplateCache()` with the same key will have no effect. `ReloadIfChanged()` has no effect on string-based templates. The only way to change the string associated with a given filename/key, is to call `ctemplate::Template::ClearCache()`.

Prefer file-based to string-based templates where possible. Updating a file-based template requires merely a data push, rather than pushing the new executable, and it also makes it easier for non-programmers to modify the template. One reason to use string-based templates is if you are in an environment where having data files could be dangerous—for instance, you work on a disk that is usually full, or need the template to work even in the face of disk I/O errors.

This package comes with a script, template-converter, that takes a template file as input and emits a C++ code snippet (an .h file) that defines a string with those template contents. This makes it easy to start by using a normal, file-based template, and then switch to `RegisterStringAsTemplate()` later if you so desire.

### Copying a Template Dictionary

You can use the `MakeCopy()` method on a template dictionary to make a "deep" copy of the template. This can be useful for situations like the following: you want to fill a template several times, each time with 90% of the values the same, but the last 10% different. Computing the values is slow. Here's how you can use `MakeCopy()` to do it:

1. fill dict with 90%
2. `newdict1 = dict->MakeCopy();`
3. fill newdict1 with last 10%
4. `newdict2 = dict->MakeCopy();`
5. fill newdict2 with last 10%
6. etc.

## Security Considerations

Like all web applications, programs that use the Google Template System to create HTML documents can be vulnerable to Cross-Site-Scripting (XSS) attacks unless data inserted into a template is appropriately sanitized and/or escaped. Which specific form of escaping or sanitization is required depends on the context in which the template variable appears within a HTML document (such as, regular "inner text", within a `<script>` tag, or within an `onClick` handler).

If you are concerned with XSS, your are strongly encouraged to leverage the Auto Escape mode developed specifically to better defend your application against XSS. The Auto Escape mode follows the guidelines outlined below. Do note however that regardless of whether you use Auto Escape or not, escaping alone while generally required, is often not enough! You also may need to sanitize or validate the input, as for instance with URL attributes. For further information, refer to additional resources on Cross-Site-Scripting issues.

The remainder of this section provides a brief summary of techniques to prevent XSS vulnerabilities due to template variables in various HTML contexts.

1. Regular text (outside of tags and other special situations).

   Use the `:html_escape` or `:h` modifier to HTML-escape the variable:

           <h1>{{HEADING:h}}</h1>

2. HTML tag attributes.

   Ensure that the attribute is enclosed in double quotes in the template, and use the `:html_escape` or `:h` modifier to escape the variable:

           <form ...
             <input name=q value="{{QUERY:h}}">
           </form>

3. URL attributes (eg., href/src).

   Enclose the URL in quotes in the template and use the `:url_escape_with_arg=html` (or `:U=html`) modifier which validates that the URL has an appropriate scheme (http, https) before HTML-escaping it:

           <img src="{{IMAGE_URL:U=html}}">

   Alternatively, validate that the URL is correct using your own validation logic and apply `:html_escape` instead of `:url_escape_with_arg=html`.

4. Beware of inserting variables containing data from untrusted sources into the context of a `style` tag or attribute.

   Certain CSS style-sheet constructs can result in the invocation of javascript. To prevent XSS, the variable must be carefully validated and sanitized.

5. Populating javascript variables.

   For string literals: Ensure that the literal is enclosed in quotes and apply the `:javascript_escape` or `:j` modifier to escape the variable:

         <script>
           // ...
           var msg_text  = '{{MESSAGE:j}}';
           // ...
         </script>

   Literals of non-string types cannot be quoted and escaped. Instead, ensure that the variable's value is set such that it is guaranteed that the resulting string corresponds to a javascript literal of the expected type. For example, use

           dict->SetValueInt("NUM_ITEMS", num_items);

   to populate an integer javascript variable in the template fragment

         <script>
           // ...
           var num_items = {{NUM_ITEMS}};
           // ...
         </script>

6. Populating javascript variables within event handlers such as `onClick`.

   Tag attributes whose values are evaluated as a javascript expression (such as `on{Click,Load,etc}` handlers) generally require HTML-Escape in addition to Javascript-Escape, since the attribute's value is HTML-unescaped by the browser before it is passed to the javascript interpreter.

   However, the `javascript_escape` implementation provided makes a subsequent HTML-Escape unnecessary, as such you can apply the same escaping for variables within event handlers as you would for javascript variables in string literals:

           <button ...
                        onclick='GotoUrl("{{TARGET_URL:j}}");'>

7. Consider other potential sources of XSS.

   There are a number of scenarios in which XSS can arise that are unrelated to the insertion of values into HTML templates, including,

   - injection into HTTP headers such as `Location`,
   - incorrect browser-side guess of the content-encoding of a HTML document without explicitly specified `charset`,
   - incorrect browser-side guess of a non-HTML document's content-type that overrides the document's specified `Content-Type`,
   - browser-side handling of documents served for download-to-disk (`Content-Disposition: attachment`).

   Please consult additional documentation on Cross-Site-Scripting for more detailed discussion of such issues.

## Working Effectively with Templates

### Registering Template Strings

Both dictionary keys and template filenames are strings. Instead of using raw strings, we encourage you to use a bit of machinery to help protect against various types of errors.

For dictionary keys, you can use the make_tpl_varnames_h tool to create static string variables to use instead of a string constant. This will protect against typos, as the make_tpl_varnames_h documentation describes.

For template filenames that a program uses -- including sub-templates -- we suggest the following idiom:

       #include "example.tpl.varnames.h"   // defines 1 string per dictionary key
       RegisterTemplateFilename(EXAMPLE_FN, "example.tpl");   // defines template
       ...
       ctemplate::Template* tpl = ctemplate::Template::GetTemplate(EXAMPLE_FN, ...);
       ...
       include_dict->SetFilename(EXAMPLE_FN);

By registering the filename, you can query the template system to detect syntax errors, reload-status, and so forth.

### Managing Templates

The following functions affect the global state of the template system.

- `ctemplate::Template::SetTemplateRootDirectory(root)`: when `GetTemplate()` is called with a relative filename, the template system will try to load the template from `root/file`. This defaults to `./`.

There are some administrative tools that can help with tweaking template performance and debugging template problems. The following functions work on registered templates.

- `ctemplate::TemplateNameList::GetMissingList()`: returns a list of all registered templates where the file could not be found on disk.
- `ctemplate::TemplateNameList::AllDoExist()`: true iff the missing-list is empty.
- `ctemplate::TemplateNameList::GetBadSyntaxList()`: returns a list of all registered templates where the template contains a syntax error, and thus cannot be used.
- `ctemplate::TemplateNameList::IsAllSyntaxOkay()`: true iff the bad-syntax list is emtpy.
- `ctemplate::TemplateNameList::GetLastmodTime()`: the latest last-modified time for any registered template-file.

The following functions help with debugging, by allowing you to examine the template dictionaries and expanded templates in more detail.

- `dict->Dump()`: dumps the contents of the dictionary (and any sub-dictionaries) to stderr.
- `dict->DumpToString()`: dumps the contents of the dictionary (and sub-dictionaries) to the given string.

Finally, `ClearCache()` removes all template objects from the cache used by `GetTemplate()`. Typically, this is only used in environments that check for memory leaks: calling this at the end of the program will clean up all memory that the template system uses.

### Template and Threads

All static methods on `Template` and `TemplateDictionary` objects are threadsafe: you can safely call `ctemplate::TemplateDictionary::SetGlobalValue()` without needing to worry about locking.

Non-static methods are not thread-safe. It is not safe for two threads to assign values to the same template-dictionary without doing their own locking. Note that this is expected to be quite rare: usually only one thread will care about a given template-dictionary.

For `Template` objects, the most common idiom is that a template is loaded via `GetTemplate()`, and after that only const methods like `Expand()` are called on the template. With such usage, it's safe to use the same `Template` object in multiple threads without locking. Be careful, however, if you also call functions like `ReloadIfChanged()`.

## Development Tools

This package includes several tools to make it easier to use write and use templates.

### make_tpl_varnames_h: Template Syntax Checker and Header File Generator

`make_tpl_varnames_h` is a "lint" style syntax checker and header file generator. It takes the names of template files as command line arguments and loads each file into a Template object by retrieving the file via the Template factory method. The loading of the file does pure syntax checking and reports such errors as mis-matched section start/end markers, mis-matched open/close double-curly braces, such as `"{{VAR}"`, or invalid characters in template variables/names/comments.

If the template passes the syntax check, by default the utility then creates a header file for use in the executable code that fills the dictionary for the template. If the developer includes this header file, then constants in the header file may be referenced in the dictionary building function, rather than hard-coding strings as variable and section names. By using these constants, the compiler can notify the developer of spelling errors and mismatched names. Here's an example of how this is used, and how it helps prevent errors:

       const char * const kosr_RESULT_NUMBER = "RESULT_NUMBER";  // script output
       dict.SetValue("RESSULT_NUMBER", "4");    // typo is silently missed
       dict.SetValue(kosr_RESSULT_NUMBER, "4");   // compiler catches typo

Each constant is named as follows:

- The initial letter 'k', indicating a defined constant.
- One or more prefix letters which are derived from the template file name. These prefix letters consist of the first letter of the file name, followed by the first letter following each underscore in the name, with the exception of the letter 'p' when it is followed by the letters "ost", as is a recommended convention for template versioning. For example, the prefix letters for the file `one_search_result_post20020815.tpl` are `osr`.
- An underscore.
- The varible or section name itself, same casing.

As an example, the section name "RESULT_NUMBER" in the file one_search_result_post20020815.tpl would be given the constant name `kosr_RESULT_NUMBER` and would appear in the header file as `const char * const kosr_RESULT_NUMBER = "RESULT_NUMBER";` -- as in the example above.

By default, the header file is produced in the current directory. An alternate output directory may be specified by the command line flag `--header_dir`.

The name of the generated header file is the same as the name of the template file with an extension added to the name. By default, that extension is `.varnames.h`. In the above example, the header file containing the constant declarations would be named `one_search_result_post20020815.tpl.varnames.h`. An alternate extension may be provided via the command line flag `--outputfile_suffix`.

Important command line flags:

- `--noheader` -- Indicates that a header file should not be generated; only syntax checking should be done.
- `--header_dir` -- sets the directory where the header is written. Default: "./"
- `--template_dir` -- sets the template root directory. Default: `./` which is the correct specification when it is run from the directory where the templates are located. This is only used if the input template filenames are specified as relative paths rather than absolute paths.
- `--outputfile_suffix` -- the extension added to the name of the template file to create the name of the generated header file. Default: `.varnames.h`.

For a full list of command line flags, run `make_tpl_varnames_h --help`.

### template-converter: convert a template to a C++ string

The `TemplateFromString` class lets you load a template from a string instead of a file. Applications may prefer this option to reduce the dependencies of the executable, or use it in environments where data files are not practical. In such cases, `template-converter` can be used as a template "compiler", letting the developer write a template file as a data file in the normal way, and then "compiling" it to a C++ string to be included in the executable.

Usage is `template-converter <template filename>`. C++ code is output is to stdout; it can be stored in a .h file or included directly into a C++ file. Perl must be installed to use this script.

### diff_tpl_auto_escape: Escaping Modifier Comparaison tool for Templates

Refer to diff_tpl_auto_escape for a description of this tool.

- Auto Escape
- Tips
- Example