Python : Metaclass

Classes are objects

"In Python, everything is an object"

That applies for classes as well which means that classes:

• can be created at runtime
• passed as paramters
• returned from funtions
• assigned to variables

Builtin type function

The simplest way to create class dynamically is to use type :

def create_klass(name, **kwattrs):
    return type(name, (object,), dict(**kwattrs))

>> my_test_klass = create_klass('MyTestClass', id=0, counter=0)
>> my_test_klass
<class __main__.MyTestClass>
>> mtk = my_test_klass()
>> mtk
<__main__.MyTestClass object at 0x01F8C150>
>> mtk.id, mtk.counter
(1, 0)

my_test_klass is equivalent to

class MyTestClass(object):
    id = 1
    counter = 0

Created at runtime, returned from a function and assigned to a variable.

Class of class

Wait a minute, if everyting is an object, it would mean that my_test_klass is an instance
of a class as well... In Python, you can check the type of a class with __class__ attribute,
let's do some tests :

>> mtk.__class__
<class __main__.MyTestClass>
>> my_test_klass.__class__
>> <type 'type'>

So type is the class of Python classes.

Metaclass

Any class whose instances are themselves classes, is a metaclass.
It means that type is a metaclass : yes, the default one. We will see later how to create our own metaclasses.

When Python parses your script, it runs a routine when detecting the class keyword thats will
collect the attributes and methods into a dictionary. When the class definition is over, python determines the metaclass of the class. Let's call it Meta. At this moment, python executes Meta(name, bases, dct) where :

• Meta is the metaclass, so this invocation is instantiating it
• name is the name of the newly created class
• bases is a tuple of the class's base classes
• dct maps attribute names to objects, listing all of the class's attributes

A metaclass is defined by setting a __metaclass__ attribute to either a class or one of its bases.

The following class has metaclass definition :

class MyExampleClass(object):
   age = 30

so type is used instead and we can create this class dynamically with the following line of code :

MyExampleClass = type('MyExampleClass', (object,), {'age' : 30})

But if it has been defined like this :

class MyExampleClass(object):
   __metaclass__ = MetaPerson
   age = 30

the creation would have been done with :

MyExampleClass = MetaPerson('MyExampleClass', (object,), {'age' : 30})

__new__ and __init__

To control the creation and initialization of the class in the metaclass, you can implement the metaclass's __new__ (to control the creation of the object) and __init__ (to control the instanciation of the object) methods.

The call to MetaPerson implicitely executes the following operations :

MyExampleClass = MetaPerson.__new__(MetaPerson, 'MyExampleClass', (object,), {'age' : 30})
MetaPerson.__init__(MetaPerson, 'MyExampleClass', (object,), {'age' : 30})

Concretely, with :

class MetaPerson(type):
    def __new__(meta, name, bases, dct):
        print '-----------------------------------'
        print "Allocating memory for class", name
        print meta
        print bases
        print dct
        return super(MetaPerson, meta).__new__(meta, name, bases, dct)
    def __init__(cls, name, bases, dct):
        print '-----------------------------------'
        print "Initializing class", name
        print cls
        print bases
        print dct
        super(MetaPerson, cls).__init__(name, bases, dct)

python will print :

-----------------------------------
Allocating memory for class MyKlass
<class '__main__.MetaPerson'>
(<type 'object'>,)
{'barattr': 2, '__module__': '__main__',
 'foo': <function foo at 0x00B502F0>,
 '__metaclass__': <class '__main__.MetaPerson'>}
-----------------------------------
Initializing class MyKlass
<class '__main__.MyKlass'>
(<type 'object'>,)
{'barattr': 2, '__module__': '__main__',
 'foo': <function foo at 0x00B502F0>,
 '__metaclass__': <class '__main__.MetaPerson'>}

when  parsing the fllowing class definition (done at module import only):

class MyKlass(object):
    __metaclass__ = MetaPerson

    def foo(self, param):
        pass

    barattr = 2

Inherited metaclass

A very common error when dealing with metaclasses is the following error code :

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

It happens with the following scheme when the identification of the metaclass fails.
Check the example below :


>>> class M_A(type):
...     pass
>>> class M_B(type):
...     pass
>>> class A(object):
...     __metaclass__=M_A
>>> class B(object):
...     __metaclass__=M_B
>>> class C(A,B):
...     pass
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Python is lost because it does not know what is the metaclass of C : is it M_A or M_B ?
To get around this issue, the solution is to create a metaclass that inherits from both M_A and M_B. It will ensure C to have a unique and identifiable metaclass :

M_A     M_B
 : \   / :
 :  \ /  :
 A  M_C  B
  \  :  /
   \ : /
     C

which in terms of code is :

>>> class M_C(M_A,M_B): pass
...
>>> class C(A,B):
...     __metaclass__=M_C
>>> C,type(C)
(<class 'C'>, <class 'M_AM_B'>)

Automatic metaclass resolution

If you are planning to code a complex class factory and you're preparing to face metaclass conflict errors, know that out there, there is a generic metaclass creator for your classes :

http://code.activestate.com/recipes/204197-solving-the-metaclass-conflict/

Cem SOYDING

Author & Editor

Senior software engineer with 12 years of experience in both embedded systems and C# .NET