Memory footprint has been huge especially for large raw HTML templates, as mentioned in #196 and once improved by #162. Recently I come out with a more elaborate tokenizer which reduces memory footprint dramatically (91.3% improvement for raw HTML and 57.7% for tag intensive template). Following are memory test results by running benchmark/memory.js, I compared the current performance and pmalouin's improvement by flatten the string representation, and the results of the new tokenizer.

No optimization (current master)

3.000 kB lorem-html before GC x 17.744 kB/instance (1024 instances sampled)
3.000 kB lorem-html after GC x 6.276 kB/instance (1024 instances sampled)
2.115 kB todolist before GC x 28.990 kB/instance (1024 instances sampled)
2.115 kB todolist after GC x 15.837 kB/instance (1024 instances sampled)

Flattern optimization (#162)

Forces input string to be re-instantiated in memory using a flat representation
instead of a graph of concatenated strings, implemented by @pmalouin (#162).

3.000 kB lorem-html before GC x 7.068 kB/instance (1024 instances sampled)
3.000 kB lorem-html after GC x 6.246 kB/instance (1024 instances sampled)
2.115 kB todolist before GC x 29.273 kB/instance (1024 instances sampled)
2.115 kB todolist after GC x 15.866 kB/instance (1024 instances sampled)

• Improvements: significant (60%) less memory before GC for raw HTML templates (with less tags and output structures).
• Drawbacks: not supported by all versions of Node.js and browsers.

Eliminate transient strings (the new tokenizer)

Eliminate transient strings by removing str += char operation during template parsing.
It's most usefull in HTML intensive templates thus there could be only ~1 string values
instead of O(n) values.

3.000 kB lorem-html before GC x 1.544 kB/instance (1024 instances sampled)
3.000 kB lorem-html after GC x 0.150 kB/instance (1024 instances sampled)
2.115 kB todolist before GC x 12.261 kB/instance (1024 instances sampled)
2.115 kB todolist after GC x 10.871 kB/instance (1024 instances sampled)

• Improvements: 91.3% less memory before GC for raw HTML templates and 57.7% for tag intensive templates. It just generates less substrings, which leads to less complexity for the actual string representation, thus also reduces CPU time.