Python : else clause on loop

In python, like in several other languages, break and continue statements exist to control execution flow. But when it comes to else, it is something new :

for item in itemList:
    if item == 'DeveloperLogBook':
        print 'Item found'
        break
else:
    print 'Item not found !'

On the example above, else applies to for loop if it terminates "through exhaustion of the list".

Note: With a while loop, else applies if the condition assertion of while is false.

Read More

Python : django 1.8 + Restful api + ExtJs 6

This article follows the basic steps to quickly create an application server with django and a PostgreSQL database. It will explain how to integrate ExtJs into a django project and will show some examples of templating.

Install django dependencies

sudo pip install django-filter djangorestframework django-rest-auth django-registration

Generate views and routes to handle the following HTTP requests :

• GET /api/deviceproxy (get a list of all device proxies)
• POST /api/deviceproxy (add a new device proxy)
• OPTIONS /api/deviceproxy (get meta-information of the DeviceProxy model)

 Edit poc_supervisor/settings.py  and add the following lines to INSTALLED_APPS :

...
'rest_framework',
'rest_framework.authtoken',
'rest_auth',
...

 and add the Rest configuration :

REST_FRAMEWORK = {
    # Use hyperlinked styles by default.
    # Only used if the `serializer_class` attribute is not set on a view.
    'DEFAULT_MODEL_SERIALIZER_CLASS':
    'rest_framework.serializers.HyperlinkedModelSerializer',
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework.authentication.SessionAuthentication',
    ),

    # Use Django's standard `django.contrib.auth` permissions,
    # or allow read-only access for unauthenticated users.
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.DjangoModelPermissions',
    ],

    'PAGINATE_BY_PARAM': 'limit',
    'ORDERING_PARAM': 'sort',
    'PAGINATE_BY': 10,
    'MAX_PAGINATE_BY': 100,
    'DEFAULT_FILTER_BACKENDS': [
        'rest_framework.filters.DjangoFilterBackend',
        'rest_framework.filters.OrderingFilter',
    ],
}

LOGIN_REDIRECT_URL = '/api/' 

Update your django app

When we followed the django tutorial, we created a basic view in views.py and a url conf in urls.py. Remove both files and add a rest.py file instead with the following code :

from rest_framework import viewsets, routers, serializers
from supervisor.models import DeviceProxy, SystemData, SystemDataAcquisition

#
# This file holds the REST related code
#

#################################################################################
#
#
#  SERIALIZERS : The serializers specify how the records are serialized in the
#                list or detail-views
#
#
#################################################################################
class DeviceProxySerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = DeviceProxy
        fields = ('name', 'description', 'ip_address', 'subnet_mask', 'port',)

class SystemDataSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = SystemData
        fields = ('identifier', 'description', 'resolution', 'unit',)

class SystemDataAcquisitionSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = SystemData
        fields = ('device_proxy', 'identifier', 'timestamp', 'status', 'value',)

#################################################################################
#
#
#  FILTERS : Use filters to retsrict items returned by querysets (RFU)
#
#
#################################################################################


#################################################################################
#
#
#  VIEWSETS : Controllers for incoming requests. They access the model and return
#             the proper output
#
#
#################################################################################
class DeviceProxyViewSet(viewsets.ModelViewSet):
    queryset = DeviceProxy.objects.all()
    serializer_class = DeviceProxySerializer
    # filter_class =

   

class SystemDataViewSet(viewsets.ModelViewSet):
    queryset = SystemData.objects.all()
    serializer_class = SystemDataSerializer
    # filter_class =

class SystemDataAcquisitionViewSet(viewsets.ModelViewSet):
    queryset = SystemDataAcquisition.objects.all()
    serializer_class = SystemDataAcquisitionSerializer
    # filter_class =

#################################################################################
#
# Register viewsets to REST router
#
#################################################################################
def register(restrouter):
    restrouter.register(r'deviceproxy', DeviceProxyViewSet)
    restrouter.register(r'systemdata', SystemDataViewSet)
    restrouter.register(r'SystemDataAcquisition', SystemDataAcquisitionViewSet) 


Update your project url settings (urls.py)

Replace the content with the following one :

from django.conf.urls import patterns, include, url
from django.contrib import admin
from rest_framework import viewsets, routers
from supervisor import rest as supervisorAPI

#
# This url pattern definition is the REST-specific one
#
# admin ui
admin.autodiscover()

# routers for the rest-api
restrouter = routers.DefaultRouter()

# add api-router for the supervisor application
supervisorAPI.register(restrouter)

# root-level url patterns
urlpatterns = patterns('',
    # frontpage
    #url(r'^', include('frontend.urls')),
    # admin-ui
    url(r'^admin/', include(admin.site.urls)),
    # provide a login-link in the browsable api
    url(r'^api/ui-auth/', include('rest_framework.urls', namespace='rest_framework')),
    # login for rest-api 
    #url(r'^api/auth/', include('rest_auth.urls')),
    # login for rest-api
    #url(r'^api/', include('frontend.apiurls')),
    # base for the browsable rest-api
    url(r'^api/', include(restrouter.urls)),
)

Note 1: Do not mind the commented code for the moment, we will see that later
Note 2: Beware of the urlpatterns definition : the function named patterns is now called. Do not forget it or you will have this error : AttributeError: 'str' object has no attribute 'resolve'
Note 3: Do not forget the trailing slash in regex definitions or you will encounter url resolution errors


Finally, migrate your app :

python manage.py migrate

Verify that REST operation are activated

Run the server and go to http://127.0.0.1/api/

Normally, you should see a Django REST framework page. You should be able to log in as well with your predefined admin account.

Add an ExtJS frontend

 ExtJS is a standalone MVC framework which means that, event for the creation of views (no model), we will need to create a new django app to host ExtJS stuff.

python manage.py startapp frontend

Open to frontend/views.py and add this content in it

from django.http import HttpResponse
from django.shortcuts import render
from django.views.decorators.csrf import ensure_csrf_cookie
import json

# Create your views here.

@ensure_csrf_cookie
def index(request):
    return render(request, 'frontend/index.html')

def permissions(request):
    print request.user.user_permissions.all()
    return HttpResponse(json.dumps(
        {
            'is_superuser': request.user.is_superuser,
            'user_permissions': list(map(lambda x:str(x.codename), request.user.user_permissions.all()))
        }

    ), 'applicative/json')

This piece of code is the very basic definition for your index file.
Now open frontend/urls.py and add this piece of code

from django.conf.urls import patterns, url

from frontend import views

urlpatterns = patterns('',
    url(r'^$', views.index, name='index'),
)

Create frontend/apiurls.py file and add the following code in it

from django.conf.urls import patterns, url
from frontend import views

urlpatterns = patterns('',
    url(r'^auth/permissions/$', views.permissions, name='permissions'),
) 

 Ok, we need now to define this app in the projects. Add the name of the app in INSTALLED_APPS in poc_supervisor/settings.py :

...
'supervisor',
'frontend',
...

And uncomment these lines in poc_supervisor/urls.py

...
url(r'^', include('frontend.urls')),
url(r'^api/auth/', include('rest_auth.urls')),
url(r'^api/', include('frontend.apiurls')),
... 

At this moment, you can run your server and try to open http://127.0.0.1/ which will end with a 404 error. Why ? Because the index.html file has not been defined ! This time, instead of creating a basic 'Hello World' html page, we will use django templates to create a default ExtJS model (v6.0.0) that we will inherit in our frontend index page. Huh ?! Yeah yeah, let's go through the following steps :

ExtJs setup

Download ExtJs

https://www.sencha.com/legal/gpl/

Create ExtJs directories

At the root of your project (where manage.py is located) :

mkdir static

mkdir templates

static will contain project-wide static files, like CSS, JavaScript.
templates will store project-wide HTML templates. We will use it to store a generic template with a pre-defined header with extjs inclusion.

Unzip extjs archive to static directory

unzip ext-6.0.0-gpl.zip -d static/

Configure Django

Edit poc_supervisor/settings.py and add the following lines

Add static directory to django project

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'static')
    ]

Add templates directory to django project

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
                os.path.join(BASE_DIR, 'templates')
        ],
    ....

Create a base.html file in poc_supervisor/templates and add the following code :

 <!-- Do NOT put any DOCTYPE here unless you want problems in IEs. -->
<html>
    <head>
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        <!-- Importing Extjs library -->
        <script type="text/javascript" src="/static/ext-6.0.0/build/ext-all-debug.js"></script>
        <script type="text/javascript" src="/static/ext-6.0.0/build/classic/theme-triton/theme-triton-debug.js"></script>

        <!-- Importing the stylesheet triton -->
        <link rel="stylesheet" type="text/css" href="/static/ext-6.0.0/build/classic/theme-triton/resources/theme-triton-all-debug.css">

        {% block head %}{% endblock %}
    </head>

    <body>
        {% block content %}{% endblock %}
    </body>
</html> 

Now let's create an template named index.html in frontend/templates/frontend/ with the following code in it :

{% extends "base.html" %}

{% block title %}Hello ExtJS Supervisor{% endblock %}

{% block head %}

    <script type='text/javascript' src='/static/frontend/app.js'></script>

{% endblock %}

{% block content %}

{% endblock %}

As you can see, index.html will inherit base.html and call a javascript file 'app.js' which will handle the form creation for us. We will first create a simple window to make sure everything works as expected.

Create frontend/static/frontend/app.js file with the following code

Ext.application({
    name: 'Supervisor',
    launch : function() {
        var win = Ext.create("Ext.window.Window", {
        title: 'My first window',
        width: 300,
        height: 200,
        maximizable: true,
        html: 'this is my first window'
        });
    win.show();
    }
});

Finally, edit the index function that we've created before in supervisor/views.py : 

from django.shortcuts import render_to_response

def index(request):
      return render_to_response('supervisor/index.html')

Ok, now run your server and you should see a nice looking window displayed ! Otherwise, double check your ext-js includes.


The next steps are to customize your view with ExtJS !

Credits

https://github.com/MaZderMind/django-vs-extjs

Read More

Python : django 1.8 + PostgreSQL shortened tutorial

This article is a compressed version of the first 2 chapters of django official tutorial. It does not explain anything but simply goes through the basic procedures to create a application server in a short time.

Installation instructions

Django 

sudo pip install django

PostgreSQL 

sudo apt-get install postgresql postgresql-client postgresql-server-dev-9.4 [pgadmin3]

Install database binding for django : 

sudo apt-get install python-dev # necessary to compile the binder
sudo pip install psycopg2

Configuration

Database access

Add a demo user from shell and set its password to 'user' 

sudo adduser spvuser

Log into postgres and create a database for that user 

sudo -u postgres psql
postgres=# CREATE USER spvclient WITH PASSWORD 'user';
postgres=# CREATE DATABASE spvdatabase OWNER spvclient ;
postgres=# \q

Check that the new database is accessible 

psql -d spvdatabase -U spvuser
spvdatabase=> \q # quit

Finally, edit postgres configuration file

sudo nano /etc/postgres/main/9.4/pg_hba.conf

Replace ident or peer with trust 

local all all trust

Develop your own site

Create project 

django-admin startproject poc_supervisor

Run server (default: 127.0.0.1:8000) : 

python manage.py runserver

Ctrl+C to quit

Create an app (let's call it supervisor):

python manage.py startapp supervisor

Edit supervisor/apps.py (app generic configuration information):

from django.apps import AppConfig

class SupervisorConfig(AppConfig):
    name = "supervisor"
   verbose_name = "POC 5.5 Supervision interface"
  

[cd supervisor]

Create a view

- edit views.py

from django.http import HttpResponse


def index(request):
    return HttpResponse("Hello, world. You're at the polls index.")

 - create a file named urls.py with the following code to map view to URL 

from django.conf.urls import url
from . import views

urlpatterns = [
    # ex: /supervisor/
    url(r'^$', views.index, name='index'),
]

[cd ..]

  - edit poc_supervisor/urls.py to point new view to root 

from django.conf.urls import include, url
from django.contrib import admin

urlpatterns = [
    url(r'^supervisor/', include('supervisor.urls')),
    url(r'^admin/', admin.site.urls),
]
 

- Verify

python manage.py runserver

and open 127.0.0.1:8000/supervisor

 

Django Database settings

Edit poc_supervisor/settings.py and replace existing DATABASES descriptor with : 

DATABASES = {

'default' : {
         'ENGINE' :  'django.db.backends.postgresql_psycopg2',
         'NAME' : 'spvdatabase',
         'USER' : 'spvuser',
         'PASSWORD': 'user',
         'HOST' : '',
    }
}

Generate database tables for all INSTALLED_APPS 
 

python manage.py migrate

Verify that tables have been created 
 

psql -d spvdatabase -U spvuser
spvdatabase=> \dt

You should see a list of tables 

public | auth_group | table | spvuser
public | auth_group_persmissions  | table | spvuser

...

public | django_session | table | spvuser

spvdatabase=> \q # quit

Create a database model for your app

For example for this model:


 

Edit supervisor/models.py

    from django.db import models


    # Create your models here.


    ##############################

    ################
    #
    # Custom validators
    #
    ##############################################
    def isStatusValid(value):
      """
      Checks the validity of SystemData status field
      """
      valid_status_list = ['DEFAULT', 'NORMAL', 'NOT_AVAILABLE', 'NO_DEVICE', 'PROTOCOL_ERROR']
      isValid = False
      if value in valid_status_list:
        isValid = True
       
      return isValid

    class DeviceProxy(models.Model):
        name = models.CharField(max_length=64, unique=True)
        description = models.TextField()
        ip_address = models.IPAddressField()
        subnet_mask = models.IPAddressField()
        port = models.IntegerField(default=47808)
       
        def __str__(self):
          return self.name + " (%s)"%self.ip_address

    class SystemData(models.Model):
        identifier = models.IntegerField(primary_key=True)
        description = models.CharField(max_length=64, unique=True)
        resolution = models.DecimalField(max_digits=3, decimal_places=3)
        unit = models.CharField(max_length=10)
       
        def __str__(self):
          return self.identfier + " : %s"%self.description
       
    class SystemDataAcquisition(models.Model):
        # Delete all system data that belongs to deleted device proxy
        device_proxy = models.ForeignKey(DeviceProxy, on_delete=models.CASCADE)
        identifier = models.ForeignKey(SystemData, on_delete=models.CASCADE)
       
        timestamp = models.DateTimeField()
        status = models.CharField(max_length=32, validators=[isStatusValid])
        value = models.DecimalField(max_digits=10, decimal_places=2)
       
        def __str__(self):
          return self.identifier + " = %s %s"%(str(self.value), self.unit) 

Add your app to INSTALLED_APPS in poc_supervisor/settings.py:

INSTALLED_APPS = (
    'supervisor.apps.SupervisorConfig',
    ...

Activate your model

python manage.py makemigrations supervisor
python manage.py migrate

Check this link if you want to play with your model with the django shell.

Add an administration interface 

python manage.py createsuperuser

and enter the user name (e.g. 'admin'), an email address for this user and a password twice. Verify :

python manage.py runserver

and open 127.0.0.1/admin and log yourself.

Add administration option on database entries

Edit supervisor/admin.py 
 

from .models import SystemData, DeviceProxy
 

admin.site.register(SystemData)br />
admin.site.register(DeviceProxy)

Run verification step again.

Read More

Use SQL Compact Server 3.5 with Windows CE 7

Background

I have a .NET CF 3.5 app running on Windows CE 6 that uses SQL Server compact 3.5.
I want it to run on Windows CE 7.

Here is an overview of how I achieved that.

Attempt 1: No modification

Simply build the code with the SDK of the new platform and deploy.
Result: 
- FAILED ! Native exception occurs at runtime (Native.GetKeyInfo ...)

Attempt 2

As explained in this article, double check the native sql dll versions (sqlceca35.dll, sqlcecompact35.dll, sqlceme35.dll, ...) and ensure that they match with the version of the System.Data.SqlServerCe assembly used in your .NET application.
For your information:

•  v. 5386 : SQL Compact server 3.5 RTM
•  v. 5692 : SQL Compact server 3.5 SP1
•  v. 8080 : SQL Compact server 3.5 SP2

You can also take a look at this article for hotfixes list.

Result:
- FAILED ! Did not work either, even with the same versions of dlls. The same exception occured.

Attempt 3 

Uninstall all SQL Server compact versions from your computer, reboot and install SQL Compact server 3.5 SP2.
Update the binaries in your application and try again.

Result:
- FAILED ! New issue !

The application is trying to load native components of version 8154 that are incompatible with ADO.NET provider of version 8080

It's like the OS is loading its own set of native sql dlls instead of the ones in the application directory...

Attempt 4

If you don't already have it, install Windows CE 7 from here (warning: it is a loooong process. Install it only for the required cpu architecture).

In your project, use the native dlls located in C:\WINCE700\others\sqlcompact\<cpu>\retail and the assembly in C:\WINCE700\others\sqlcompact\managed\retail.

Result:
- SUCCEEDED ! The application runs flawlessly now !

Conclusion

After checking the versions of the dlls used in attempt 4, I found out that the binaries have been updgraded to 8154. I did not find any installer of SQL Server Compact 3.5 for Devices with this version (maybe I should have tried harder).
Know that for some people, attempt 2 resolved the issue but the only solution that worked for me was the last one.

Read More

Introduction to LCDs technology

This article gives an overview of the LCD technologies. It is a first step for who might want to design an embedded system with a color (touch) display.

How does it work

An LCD is an array of Liquid Crystal segments.
These segments are polarized and organized randomly when not in an electrical field.
When an electrical field is applied, the crystals align to the field and gate the amount of light
 than can pass through them.

Each LCD segment is aligned with an electric field.
A backlight is required to drive light through aligned crystal.

Types of display

There are passive (STN) and active displays (TFT).
They both consist of a grid of rows and columns electrical signals connected perpendicularly to every segment in the LCD.
An integrated chip controls which row and column are activated to enable/disable the segment at the intersection.

The passive screens apply a small bias current to row/column to generate a field in the intersection.
The active screens have a transistor per segment. This transistor is activated when the row/column of a segment is selected.
Color filters allow the generation of specific color (R, G or B) at a segment which means that a pixel consists of 3 segments.
TFT displays can drive 3 segments (or 1 pixel) per clock with levels of color that depend on the number of data lines (8 bits, 15 bits, 16 bits, 24 bits...).

LCD panel technologies for embedded display

From now on, this article will focus on TFT displays.

Common embedded screen resolutions

Resolution	Pixels	Diagonal
QVGA (Quarter Video Graphics Array)	320x240	3.5”
WQVGA (Wide Quarter VGA)	480x272	4.3”
VGA (Video Graphics Array)	640x480	5.7”
WVGA (Wide VGA)	800x480	7”
SVGA (Super VGA)	800x600	10.4”
XGA (extended graphics array)	1024x768	12.1” or 15”
WXGA (Wide Extended Graphics Array)	1280x800	12.1”

LCD Signals

The TFT LCD displays require these type of timing signals:

VSYNC: Vertical Sync. Used to reset the LCD row pointer to the top of display

HSYNC: Horizontal Sync. Used to reset the LCD column pointer to the edge of the display

D0...Dxx: Data lines. Typically, the encoding of R, G and B color levels.

LCDCLK: LCD Clock. Refresh rate.

The other common signals are:

LCD power signals

LCD backlight power

[Optional] Touchscreen signal (usually 4/5-wires for resistive, I²C link for capacitive)

Frame Buffer

The frame buffer is the memory allocated to store bit mapped image sent to the LCD. This data is used to periodically refresh the display.

The frame buffer can be calculated (Width pixels)*(Height pixels)*(Bytes for colors depth).

Example: For a 24bits SVGA the frame buffer size is 800*600*4 = 1.92 Mb. It gives an idea of the databus speed for a refresh rate of 60 Hz : 1.92*60 = 115.2 Mb/s 

Touchscreen technologies

The following array gives a quick overview of the different types of touch screens.

	5-Wire resistive	Capacitive	Projected Capacitive	Surface Acoustic Wave	Infrared
Transmissivity/Clarity	Good 75-85%	Very Good 90-98%	Very Good 90-98%	Very Good 90-98%	Best 95-100%
Activate with any object	Best Any object	Poor Finger or capacitive stylus	Good Same as capacitive + surgical glove	Good Finger, gloved hand, soft/pliable stylus	Very Good Most objects
High sensitivity (Light touch)	Good	Very Good	Very Good	Very Good	Best
Accuracy	Very Good	Good	Best	Very Good	Very Good

Credits

http://www.nxp.com/wcm_documents/techzones/microcontrollers-techzone/Presentations/graphics.lcd.technologies.pdf

https://elearning.renesas.com/file.php/1/CoursePDFs/DevCon_On-the-Road/DevCon_On-the-Road/HMI/Enhance_Embedded_Systems_with_Low_Cost_TFT_Solutions.pdf

http://www.tru-vumonitors.com/images/Touch_Screen_Basics.Comparisons.pdf

Read More