One up to Python expert (1) - Decoratorspython
I have been using Python for almost 4 years. I still remembered how amazed I was by the elegance of code indention, simplicity at the first sight, started from the official tutorial, then dive into it, daily scripts, then couple of side projects, but until today, I still hesitate to claim a Python expert. I would rather share this series of the journey to broaden my vision and deepen my insight on this beautiful language.
Dr. David Mertz has a stunning demonstration in Charming Python column show this language feature, decorater. Here is the sample code to add spam, the favorite food for python, to arbitrary function:
def addspam(fn): def new(*args): "new method" print "spam spam spam" return fn(*args) return new @addspam def add(a,b): "add method" print a**2 + b**2 if __name__ == "__main__": add(3, 4)
The output of is:
spam spam spam 25
According to the Python language reference, the @ operation is defined as:
Decorator expressions are evaluated when the function is defined, in the scope that contains the function definition. The result must be a callable, which is invoked with the function object as the only argument. The returned value is bound to the function name instead of the function object. Multiple decorators are applied in nested fashion.
Therefore, our add is defined as:
add = addspam(add)
new is an anonymous function, the name is arbitrary since we would never call it directly. When add is invoked, addspam is evaluated and returns a callable object, new, which accepts the arguments, then is executed. As a well-behaviored decorator, new eventually calls the decoratee after spreading the word, “spam”.
Before we rush to more sophisticated application, let’s take a look at the flaws of the crystal ball. Yes, it is not crystal transparent, new blocks the signature of add:
>>> print add.__doc__ new method
We can copy the meta data by all means, but is there a smart way to avoid the boilerplate code? Yes, we can use Michele Simionato’s decorator library like this:
from decorator import decorator @decorator def addspam(fn, *args, **kw): "new method" print "spam spam spam", args return fn(*args) @addspam def add(a,b): "add method" print a**2 + b**2 if __name__ == "__main__": print add.__doc__ add(3, 4)
That is quite dizzying, What is the on the earth under the hood?
Under the hood
First, let us inspect the vanilla version, each decorator would block the signature of the decoratee, illustrated by different colors.
Here is the code snippet of decorator.py:
def decorator(caller): def _decorator(func): # the real meat is here infodict = getinfo(func) argnames = infodict['argnames'] assert not ('_call_' in argnames or '_func_' in argnames), ( 'You cannot use _call_ or _func_ as argument names!') src = "lambda %(signature)s: _call_(_func_, %(signature)s)" % infodict # import sys; print >> sys.stderr, src # for debugging purposes dec_func = eval(src, dict(_func_=func, _call_=caller)) return update_wrapper(dec_func, func, infodict) return update_wrapper(_decorator, caller)
The first difference that caught my eye was the argument of
func, instead of
*argv, does this matter?
Yes, that is the trick of the magic. decorator decorates the addspam which decoratesadd, are you still awake? So _add_is the argument for decorator’s anonymous function, i.e __decorator_.
There are two assistants for the magic: getinfo copy the signature of the function; update_wrapper signs the _decorator_with _caller’s signature. When add is invoked, decorator(addspam) is evaluated, which returns _decortor with addspam’s signature, in another word, decorator is transparent to addspam. _decorator is also the decorator of add, so _decorator(add) is executed:
- Sanity check: make sure keyword is not used in the argument names
- Cook the real meat: build addspam(add) in dec_func. Please check 3.6.2 String Formatting Operations for the syntax of mapping dictionary.
- Seal the can with fun(i.e add)’s signature
Here is the dynamic illustration:
I would discuss this topic in detail later, you could take a look at official wiki.
This pattern is well-documented in Wikipedia, and here is an effective but obtrusive implementation in DDJ1, the corresponding python implementation is much more intuitive:
@memoize def fib(n): print "%d is caculated" % n if n < 2 : return 1; else: return fib(n-1) + fib(n-2)
Programming by Contract
Programming by Contract is an approach for software engineering. Microsoft Visual C++ introduced SAL annotations for precondition and postcondition. Decorators helps to separate the contract and logic2:
@precondition("tom > 0") @precondition("jerry > 1") @postcondition(lambda x: x > 1) def foo(tom=1, jerry=2, rose=3, jack=4): print tom return jack
Precondition determines the requirement of the arguments, it is more convenient to use names of arguments for evaluation; postcondition specifies the return value, function object is more appropriate to refer the returned value. Here is the full implementation.
Aspect oriented programming
AOP is quite popular in Java community, there is corresponding Python project, PEAK for enterprise environment. For lightweight AOP developer, the decorator could do some help, such as the canonical fund transfer example:
@precondition("amount > 0") @precondition("fromAccount.amount > amount") def transfer(fromAccount, toAccount, amount): # TODO: add transaction here. fromAccount.withdraw(amount) toAccount.deposit(amount)
Decorator opens a door to override the default behavior of function. The add-on lets the magic shine and hide the mechanism behind the curtain.