Monday, December 21, 2015

Python : Metaclass

Classes are objects


"In Python, everything is an object"

That applies for classes as well which means that classes:

  • can be created at runtime
  • passed as paramters
  • returned from funtions
  • assigned to variables

Builtin type function


The simplest way to create class dynamically is to use type :

def create_klass(name, **kwattrs):
    return type(name, (object,), dict(**kwattrs))

>> my_test_klass = create_klass('MyTestClass', id=0, counter=0)
>> my_test_klass
<class __main__.MyTestClass>
>> mtk = my_test_klass()
>> mtk
<__main__.MyTestClass object at 0x01F8C150>
>> mtk.id, mtk.counter
(1, 0)

my_test_klass is equivalent to

class MyTestClass(object):
    id = 1
    counter = 0

Created at runtime, returned from a function and assigned to a variable.

Class of class


Wait a minute, if everyting is an object, it would mean that my_test_klass is an instance
of a class as well... In Python, you can check the type of a class with __class__ attribute,
let's do some tests :

>> mtk.__class__
<class __main__.MyTestClass>
>> my_test_klass.__class__
>> <type 'type'>

So type is the class of Python classes.

Metaclass


Any class whose instances are themselves classes, is a metaclass.
It means that type is a metaclass : yes, the default one. We will see later how to create our own metaclasses.

When Python parses your script, it runs a routine when detecting the class keyword thats will
collect the attributes and methods into a dictionary. When the class definition is over, python determines the metaclass of the class. Let's call it Meta. At this moment, python executes Meta(name, bases, dct) where :

  • Meta is the metaclass, so this invocation is instantiating it
  • name is the name of the newly created class
  • bases is a tuple of the class's base classes
  • dct maps attribute names to objects, listing all of the class's attributes

A metaclass is defined by setting a __metaclass__ attribute to either a class or one of its bases.

The following class has metaclass definition :

class MyExampleClass(object):
   age = 30

so type is used instead and we can create this class dynamically with the following line of code :

MyExampleClass = type('MyExampleClass', (object,), {'age' : 30})

But if it has been defined like this :

class MyExampleClass(object):
   __metaclass__ = MetaPerson
   age = 30

the creation would have been done with :

MyExampleClass = MetaPerson('MyExampleClass', (object,), {'age' : 30})

__new__ and __init__


To control the creation and initialization of the class in the metaclass, you can implement the metaclass's __new__ (to control the creation of the object) and __init__ (to control the instanciation of the object) methods.

The call to MetaPerson implicitely executes the following operations :

MyExampleClass = MetaPerson.__new__(MetaPerson, 'MyExampleClass', (object,), {'age' : 30})
MetaPerson.__init__(MetaPerson, 'MyExampleClass', (object,), {'age' : 30})

Concretely, with :

class MetaPerson(type):
    def __new__(meta, name, bases, dct):
        print '-----------------------------------'
        print "Allocating memory for class", name
        print meta
        print bases
        print dct
        return super(MetaPerson, meta).__new__(meta, name, bases, dct)
    def __init__(cls, name, bases, dct):
        print '-----------------------------------'
        print "Initializing class", name
        print cls
        print bases
        print dct
        super(MetaPerson, cls).__init__(name, bases, dct)

python will print :

-----------------------------------
Allocating memory for class MyKlass
<class '__main__.MetaPerson'>
(<type 'object'>,)
{'barattr': 2, '__module__': '__main__',
 'foo': <function foo at 0x00B502F0>,
 '__metaclass__': <class '__main__.MetaPerson'>}
-----------------------------------
Initializing class MyKlass
<class '__main__.MyKlass'>
(<type 'object'>,)
{'barattr': 2, '__module__': '__main__',
 'foo': <function foo at 0x00B502F0>,
 '__metaclass__': <class '__main__.MetaPerson'>}

when  parsing the fllowing class definition (done at module import only):

class MyKlass(object):
    __metaclass__ = MetaPerson

    def foo(self, param):
        pass

    barattr = 2


Inherited metaclass


A very common error when dealing with metaclasses is the following error code :

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

It happens with the following scheme when the identification of the metaclass fails.
Check the example below :


>>> class M_A(type):
...     pass
>>> class M_B(type):
...     pass
>>> class A(object):
...     __metaclass__=M_A
>>> class B(object):
...     __metaclass__=M_B
>>> class C(A,B):
...     pass
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Python is lost because it does not know what is the metaclass of C : is it M_A or M_B ?
To get around this issue, the solution is to create a metaclass that inherits from both M_A and M_B. It will ensure C to have a unique and identifiable metaclass :

M_A     M_B
 : \   / :
 :  \ /  :
 A  M_C  B
  \  :  /
   \ : /
     C

which in terms of code is :

>>> class M_C(M_A,M_B): pass
...
>>> class C(A,B):
...     __metaclass__=M_C
>>> C,type(C)
(<class 'C'>, <class 'M_AM_B'>)

Automatic metaclass resolution


If you are planning to code a complex class factory and you're preparing to face metaclass conflict errors, know that out there, there is a generic metaclass creator for your classes :

http://code.activestate.com/recipes/204197-solving-the-metaclass-conflict/

Tuesday, December 15, 2015

Python : filter(), map() and reduce()

filter(), map() and reduce() tools are functional programming methods that can be
applied to iterable objects.

filter(<function>, <object>)

Iterates on object and returns the list of elements from <object> for which <function> returned True.
Usually, lambda functions are used for <function>. A lambda function returns True when the condition is met (example : lambda x: x <= 10).

values = range(100)
print filter(lambda x: x < 50, values)
# prints [0, 1, 2, 3, ..., 49]
print filter(lambda x: x > 50, values)
# prints [51, 52, 53, ..., 100]
print filter(lambda x: x%2 == 0)
# prints [0, 2, 4, ..., 100]



map(<function>, <object>)

Applies a function to every element of a list and returns the modified list.

cities = ['paris', 'london', 'berlin']
print map(lambda n: n.upper(), cities)
# prints ['PARIS', 'LONDON', 'BERLIN']


reduce(<function>, <list>)


Recursive process : at each step, passes the current operation result with the next item from the list.
By default, the first item in the list initializes the first operation result.
Note : reduce takes two parameters

print reduce(lambda x,y: x*y, [1,2,3,4])
# prints 24
# 1st cycle : multiplies 1 (first element) with 2
# 2nd cycle : multiplies 2 (result of previous operation) with 3 (next element)
# 3rd cycle : multiplies 6 (result of previous operation) with 4 (next element)


 
biz.