Skip to content

Basic Token Types

Vinícius Garcia edited this page May 14, 2017 · 6 revisions

There are a few different token types. In this page we'll only illustrate which tokens are available and how to use and declare them.

There a is a total of 8 basic built-in types and they are given by the following enumeration:

enum tokType { 
  // Internal types: 
  NONE, OP, VAR, 
 
  // Base types: 
  // Note: The mask system accepts at most 29 (32-3) different base types. 
  STR, FUNC, 
 
  // Numerals: 
  NUM = 0x20,   // Everything with the bit 0x20 set is a number. 
  REAL = 0x21,  // == 0x20 + 0x1 => Real numbers. 
  INT = 0x22,   // == 0x20 + 0x2 => Integral numbers. 
  BOOL = 0x23,  // == 0x20 + 0x3 => Boolean Type.
 
  // Complex types: 
  IT = 0x40,      // Everything with the bit 0x40 set are iterators. 
  LIST = 0x41,    // == 0x40 + 0x01 => Lists are iterators. 
  TUPLE = 0x42,   // == 0x40 + 0x02 => Tuples are iterators. 
  STUPLE = 0x43,  // == 0x40 + 0x03 => ArgTuples are iterators. 
  MAP = 0x44,     // == 0x40 + 0x04 => Maps are Iterators 
 
  // References are internal tokens used by the calculator: 
  REF = 0x80, 
 
  // Mask used when defining operations: 
  ANY_TYPE = 0xFF 
};

The types: OP, VAR, NUM, IT, STUPLE and REF are unlikely to be used by normal users. So they will not be explained here. As for the others a short description follows:

NONE type:

packToken none1 = packToken::None;
packToken none2 = calculator::calculate("my_var = None");

The None type is the default return type of any function, and is used to denote "Nothing" or a "Null value".

STR type:

packToken str1 = "normal string";
packToken str2 = calculator::calculate("my_var = 'string'");

FUNC type:

This type is sightly more complex.

To better understand it you should read the tutorial on how to add new functions.

packToken myFunc(TokenMap scope) {
  // my code
}

int main() {
  packToken f1 = CppFunction(&my_func, {"arg_name1", "arg_name2"}, "my_func");
}

REAL type:

packToken real1 = 10.0;
packToken real2 = calculator::calculate("my_var = 10.0");

INT type:

packToken int1 = 10;
packToken int2 = calculator::calculate("my_var = 10");

BOOL type:

packToken bool1 = true;
packToken bool2 = calculator::calculate("my_var = True");

LIST type:

packToken list1 = TokenList();
packToken list2 = calculator::calculate("my_var = list(1,2,3)");
packToken list3 = calculator::calculate("my_var = [1,2,3]");

TUPLE type:

packToken tuple1 = Tuple();
packToken tuple2 = calculator::calculate("my_var = (1,2,3)");

MAP type:

packToken map1 = TokenMap();
packToken map2 = calculator::calculate("my_var = map('a': 1, 'b': 2, 'c': 3)");
packToken map3 = calculator::calculate("my_var = {'a': 1, 'b': 2, 'c': 3}");

Clone this wiki locally