Sometimes, we encounter problems that require us to extend the behavior of a function, but we don’t want to change its implementation.<\/p>\n
Some of those problems could be: logging spent time, caching, validating parameters, etc.<\/p>\n
All these solutions are often needed in more than one function: you often need to log the spent time of every http connection; you often need to cache more than one data base entity; you often need validation in more than one function.<\/p>\n
Today we will solve these 3 mentioned problems with Python decorators:<\/p>\n
[ulp id=’R7QVpFMZmjosABLC’]<\/p>\n
Before we can jump into decorators, we need to understand how functions actually work.<\/p>\n
In essence, functions are procedures that return a value based on some given arguments.<\/p>\n
\r\ndef my_function(my_arg):\r\n return my_arg + 1\r\n<\/pre>\nIn Python, functions are first-class objects.<\/p>\n
This means that functions can be assigned to a variable:<\/p>\n
\r\ndef my_function(foo):\r\n return foo + 1\r\nmy_var = my_function\r\nprint(str(my_var(1))) # prints \"2\"\r\n<\/pre>\nThey can be defined inside another functions:<\/p>\n
\r\ndef my_function(foo):\r\n def my_inner_function():\r\n return 1\r\n return foo + my_inner_function()\r\nprint(str(my_function(1))) # still prints \"2\"\r\n<\/pre>\nThey can be passed as parameters (higher-order functions):<\/p>\n
\r\ndef my_function(foo, my_parameter_function):\r\n return foo + my_parameter_function()\r\ndef parameter_function(): return 1\r\nprint(str(my_function(1, parameter_function))) # still prints \"2\"\r\n<\/pre>\nAnd they can return other functions (also, higher-order functions):<\/p>\n
\r\ndef my_function(constant):\r\n def inner(foo):\r\n return foo + constant\r\n return inner\r\nplus_one = my_function(1)\r\nprint(str(plus_one(1))) # still prints \"2\"\r\n<\/pre>\nAnother thing to notice, is that inner functions have access to the outer scope, that’s why we can use the parameter
constant<\/code> in the inner function of the last example. Also, this access is read-only, we can not modify variables from the outer scope within an inner function.<\/p>\n2. Jumping into decorators<\/h2>\n
Python decorators provide a nice and simple syntax to call higher-order functions. By definition, a decorator takes a function as a parameter, and returns a wrapper of that given function to extend its behavior without actually modifying it.<\/p>\n
Given this definition we can write somthing like:<\/p>\n
\r\ndef decorator(function_to_decorate):\r\n def wrapper(value):\r\n print(\"you are calling {} with '{}' as parameter\".format(function_to_decorate.__name__, value))\r\n return function_to_decorate(value)\r\n return wrapper\r\n\r\n\r\ndef replace_commas_with_spaces(value):\r\n return value.replace(\",\", \" \")\r\n\r\nfunction_to_use = decorator(replace_commas_with_spaces)\r\nprint(function_to_use(\"my,commas,will,be,replaces,with,spaces\"))\r\n<\/pre>\nAnd after execution, the output will look like:<\/p>\n
\r\nyou are calling replace_commas_with_spaces with 'my,commas,will,be,replaces,with,spaces' as parameter\r\nmy commas will be replaces with spaces\r\n<\/pre>\nSo, what is actually happening here?<\/p>\n
We are defining a higher-order function called
decorator<\/code> that receives a function as a parameter and returns a wrapper of that function.<\/p>\nThe wrapper just prints to the console the name of the called function and the given parameters before executing the wrapped function. And the wrapped functions just replaces the commas with spaces.<\/p>\n
Now we have a decorator written here. But it’s kind of annoying to define the decorator, the function and then assigning the wrapper to another variable to finally be able to use it.<\/p>\n
Python provides some sugar syntax to make it easier to write and read, and if we re-write this decorator using it:<\/p>\n
\r\ndef decorator(function_to_decorate):\r\n def wrapper(value):\r\n print(\"you are calling {} with '{}' as parameter\".format(function_to_decorate.__name__, value))\r\n return function_to_decorate(value)\r\n return wrapper\r\n\r\n\r\n@decorator\r\ndef replace_commas_with_spaces(value):\r\n return value.replace(\",\", \" \")\r\n\r\nprint(replace_commas_with_spaces.__name__)\r\nprint(replace_commas_with_spaces.__module__)\r\nprint(replace_commas_with_spaces.__doc__)\r\nprint(replace_commas_with_spaces(\"my,commas,will,be,replaces,with,spaces\"))\r\n<\/pre>\nWe just annotate the function we want to wrap with the decorator function and that’s it. That function will be decorated, and the output will look the same.<\/p>\n
\r\nwrapper\r\n__main__\r\nNone\r\nyou are calling replace_commas_with_spaces with 'my,commas,will,be,replaces,with,spaces' as parameter\r\nmy commas will be replaces with spaces\r\n<\/pre>\nNow, debugging this can be a real pain, as the
replace_commas_with_spaces<\/code> function is overridden with the wrapper, so its__name__<\/code>,__doc__<\/code> and__module__<\/code> will also be overridden (as seen in the output).<\/p>\nTo avoid this behavior we will use
functools.wraps<\/code>, that prevents a wrapper from overriding its inner function properties.<\/p>\n\r\nfrom functools import wraps\r\n\r\n\r\ndef decorator(function_to_decorate):\r\n @wraps(function_to_decorate)\r\n def wrapper(value):\r\n print(\"you are calling {} with '{}' as parameter\".format(function_to_decorate.__name__, value))\r\n return function_to_decorate(value)\r\n return wrapper\r\n\r\n\r\n@decorator\r\ndef replace_commas_with_spaces(value):\r\n return value.replace(\",\", \" \")\r\n\r\nprint(replace_commas_with_spaces.__name__)\r\nprint(replace_commas_with_spaces.__module__)\r\nprint(replace_commas_with_spaces.__doc__)\r\nprint(replace_commas_with_spaces(\"my,commas,will,be,replaces,with,spaces\"))\r\n<\/pre>\nAnd now the output will be:<\/p>\n
\r\nreplace_commas_with_spaces\r\n__main__\r\nNone\r\nyou are calling replace_commas_with_spaces with 'my,commas,will,be,replaces,with,spaces' as parameter\r\nmy commas will be replaces with spaces\r\n<\/pre>\nSo, now we know how decorators work in python. Let’s solve our mentioned problems.<\/p>\n
3. The Practice<\/h2>\n
So, we need to implement cache, spent time logging and validations<\/em>.<\/p>\n
Let’s combine them all by solving a bigger problem: palindromes<\/strong>.<\/p>\n
Let’s make an algorithm that, given a word, will check if it’s a palindrome. If it isn’t, it will convert it to palindrome.<\/p>\n
palindrome.py<\/em><\/span><\/p>\n
\r\ndef is_palindrome(string_value):\r\n char_array = list(string_value)\r\n size = len(char_array)\r\n half_size = int(size \/ 2)\r\n for i in range(0, half_size):\r\n if char_array[i] != char_array[size - i - 1]:\r\n return False\r\n return True\r\n\r\n\r\ndef convert_to_palindrome(v):\r\n def action(string_value, chars):\r\n chars_to_append = list(string_value)[0:chars]\r\n chars_to_append.reverse()\r\n new_value = string_value + \"\".join(chars_to_append)\r\n if not is_palindrome(new_value):\r\n new_value = action(string_value, chars + 1)\r\n return new_value\r\n return action(v, 0)\r\n\r\nuser_input = input(\"string to convert to palindrome (exit to terminate program): \")\r\nwhile user_input != \"exit\":\r\n print(str(convert_to_palindrome(user_input)))\r\n print(\"------------------------------------------------------\")\r\n user_input = input(\"string to check (exit to terminate program): \")\r\n\r\n<\/pre>\nHere, we have a function called
is_palindrome<\/code>, which given an input, returnsTrue<\/code> if its palindrome, orFalse<\/code> otherwise.<\/p>\nThen, there is a function called
convert_to_palindrome<\/code> which, given an input, will add just as many characters (reversed, from the beginning) as necessary to make it palindrome.<\/p>\nAlso, there is a while that reads the user input until he inputs “exit”<\/em>.<\/p>\n
The output looks like:<\/p>\n
\r\nstring to convert to palindrome (exit to terminate program): anita lava la tina\r\nanita lava la tinanit al aval atina\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): anitalavalatina\r\nanitalavalatina\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): menem\r\nmenem\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): mene\r\nmenem\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): casa\r\ncasac\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): casaca\r\ncasacasac\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): exit\r\n<\/pre>\nAs you can see, it works just fine. But we have a couple problems:<\/p>\n
\n
- I don’t know how long does it take it to process the input, or if its related to the length of it. (spent time logging)<\/li>\n
- I don’t want it to process twice the same input, it’s not necessary. (cache)<\/li>\n
- It’s designed to work with words or numbers, so I don’t want spaces around. (validation)<\/li>\n<\/ul>\n
Let’s get dirty here, and start with a spent time logging<\/em> decorator.<\/p>\n
palindrome.py<\/em><\/span><\/p>\n
\r\nimport datetime\r\nfrom functools import wraps\r\n\r\n\r\ndef spent_time_logging_decorator(function):\r\n @wraps(function)\r\n def wrapper(*args):\r\n start = datetime.datetime.now()\r\n result = function(*args)\r\n end = datetime.datetime.now()\r\n spent_time = end - start\r\n print(\"spent {} microseconds in {} with arguments {}. Result was: {}\".format(spent_time.microseconds,\r\n function.__name__, str(args),\r\n result))\r\n return result\r\n\r\n return wrapper\r\n\r\n\r\ndef is_palindrome(string_value):\r\n char_array = list(string_value)\r\n size = len(char_array)\r\n half_size = int(size \/ 2)\r\n for i in range(0, half_size):\r\n if char_array[i] != char_array[size - i - 1]:\r\n return False\r\n return True\r\n\r\n\r\n@spent_time_logging_decorator\r\ndef convert_to_palindrome(v):\r\n def action(string_value, chars):\r\n chars_to_append = list(string_value)[0:chars]\r\n chars_to_append.reverse()\r\n new_value = string_value + \"\".join(chars_to_append)\r\n if not is_palindrome(new_value):\r\n new_value = action(string_value, chars + 1)\r\n return new_value\r\n\r\n return action(v, 0)\r\n\r\n\r\nuser_input = input(\"string to convert to palindrome (exit to terminate program): \")\r\nwhile user_input != \"exit\":\r\n print(str(convert_to_palindrome(user_input)))\r\n print(\"------------------------------------------------------\")\r\n user_input = input(\"string to check (exit to terminate program): \")\r\n\r\n<\/pre>\nIt’s pretty simple, we wrote a decorator which returns a wrapper that gets the time before and after executor, and then calculates the spent time and logs it, with the called function, parameters and result. The output looks like:<\/p>\n
\r\nstring to check (exit to terminate program): anitalavalatina\r\nspent 99 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina\r\nanitalavalatina\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): exit\r\n<\/pre>\nAs you see, there is plenty information in that log line, and our implementation was not changed at all.<\/p>\n
Now, let’s add cache:<\/p>\n
palindrome.py<\/em><\/span><\/p>\n
\r\nimport datetime\r\nfrom functools import wraps\r\n\r\n\r\ndef cache_decorator(function):\r\n cache = {}\r\n\r\n @wraps(function)\r\n def wrapper(*args):\r\n hashed_arguments = hash(str(args))\r\n if hashed_arguments not in cache:\r\n print(\"result for args {} was not found in cache...\".format(str(args)))\r\n cache[hashed_arguments] = function(*args)\r\n return cache[hashed_arguments]\r\n return wrapper\r\n\r\n\r\ndef spent_time_logging_decorator(function):\r\n @wraps(function)\r\n def wrapper(*args):\r\n start = datetime.datetime.now()\r\n result = function(*args)\r\n end = datetime.datetime.now()\r\n spent_time = end - start\r\n print(\"spent {} microseconds in {} with arguments {}. Result was: {}\".format(spent_time.microseconds,\r\n function.__name__, str(args),\r\n result))\r\n return result\r\n\r\n return wrapper\r\n\r\n\r\ndef is_palindrome(string_value):\r\n char_array = list(string_value)\r\n size = len(char_array)\r\n half_size = int(size \/ 2)\r\n for i in range(0, half_size):\r\n if char_array[i] != char_array[size - i - 1]:\r\n return False\r\n return True\r\n\r\n\r\n@spent_time_logging_decorator\r\n@cache_decorator\r\ndef convert_to_palindrome(v):\r\n def action(string_value, chars):\r\n chars_to_append = list(string_value)[0:chars]\r\n chars_to_append.reverse()\r\n new_value = string_value + \"\".join(chars_to_append)\r\n if not is_palindrome(new_value):\r\n new_value = action(string_value, chars + 1)\r\n return new_value\r\n\r\n return action(v, 0)\r\n\r\n\r\nuser_input = input(\"string to convert to palindrome (exit to terminate program): \")\r\nwhile user_input != \"exit\":\r\n print(str(convert_to_palindrome(user_input)))\r\n print(\"------------------------------------------------------\")\r\n user_input = input(\"string to check (exit to terminate program): \")\r\n\r\n<\/pre>\nThis is a very simple implementation of cache. No TTL, no thread safety. It’s just a dictionary which keys are the hash of the arguments. If no value with the given key was found, it creates it, then retrieves it. Output:<\/p>\n
\r\nstring to convert to palindrome (exit to terminate program): anitalavalatina\r\nresult for args ('anitalavalatina',) was not found in cache...\r\nspent 313 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina\r\nanitalavalatina\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): anitalavalatina\r\nspent 99 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina\r\nanitalavalatina\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): exit\r\n<\/pre>\nThere it is. The first execution with “anitalavalatina”<\/em> outputs a line informing us that the result for that input was not found. But when we input it again, that line is gone. Awesome! But we still receive spaces, let’s validate that:<\/p>\n
palindrome.py<\/em><\/span><\/p>\n
\r\nimport datetime\r\nfrom functools import wraps\r\n\r\n\r\ndef validation_decorator(validator, if_invalid=None):\r\n def decorator(function):\r\n @wraps(function)\r\n def wrapper(*args):\r\n if validator(*args):\r\n return function(*args)\r\n else:\r\n return if_invalid\r\n return wrapper\r\n return decorator\r\n\r\n\r\ndef cache_decorator(function):\r\n cache = {}\r\n\r\n @wraps(function)\r\n def wrapper(*args):\r\n hashed_arguments = hash(str(args))\r\n if hashed_arguments not in cache:\r\n print(\"result for args {} was not found in cache...\".format(str(args)))\r\n cache[hashed_arguments] = function(*args)\r\n return cache[hashed_arguments]\r\n return wrapper\r\n\r\n\r\ndef spent_time_logging_decorator(function):\r\n @wraps(function)\r\n def wrapper(*args):\r\n start = datetime.datetime.now()\r\n result = function(*args)\r\n end = datetime.datetime.now()\r\n spent_time = end - start\r\n print(\"spent {} microseconds in {} with arguments {}. Result was: {}\".format(spent_time.microseconds,\r\n function.__name__, str(args),\r\n result))\r\n return result\r\n\r\n return wrapper\r\n\r\n\r\ndef is_palindrome(string_value):\r\n char_array = list(string_value)\r\n size = len(char_array)\r\n half_size = int(size \/ 2)\r\n for i in range(0, half_size):\r\n if char_array[i] != char_array[size - i - 1]:\r\n return False\r\n return True\r\n\r\n\r\ndef should_not_contain_spaces(*args):\r\n return False not in map(lambda x: \" \" not in str(x), args)\r\n\r\n\r\n@spent_time_logging_decorator\r\n@validation_decorator(should_not_contain_spaces, \"input shouldn't contain spaces.\")\r\n@cache_decorator\r\ndef convert_to_palindrome(v):\r\n def action(string_value, chars):\r\n chars_to_append = list(string_value)[0:chars]\r\n chars_to_append.reverse()\r\n new_value = string_value + \"\".join(chars_to_append)\r\n if not is_palindrome(new_value):\r\n new_value = action(string_value, chars + 1)\r\n return new_value\r\n\r\n return action(v, 0)\r\n\r\n\r\nuser_input = input(\"string to convert to palindrome (exit to terminate program): \")\r\nwhile user_input != \"exit\":\r\n print(str(convert_to_palindrome(user_input)))\r\n print(\"------------------------------------------------------\")\r\n user_input = input(\"string to check (exit to terminate program): \")\r\n<\/pre>\nNow, this one is a little tricky. To pass arguments to the decorator we need to wrap it. Yeah, we need a wrapper of the wrapper. Thanks to that, we can pass the validation function and a message if input is invalid. The output looks like:<\/p>\n
\r\nstring to convert to palindrome (exit to terminate program): anita lava la tina\r\nspent 87 microseconds in convert_to_palindrome with arguments ('anita lava la tina',). Result was: input shouldn't contain spaces.\r\ninput shouldn't contain spaces.\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): anitalavalatina\r\nresult for args ('anitalavalatina',) was not found in cache...\r\nspent 265 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina\r\nanitalavalatina\r\n------------------------------------------------------\r\nstring to check (exit to terminate program): exit\r\n<\/pre>\n3. Download the Code Project<\/h2>\n
This was an example of how to write decorators in Python.<\/p>\n