Tuesday, November 21, 2017

Host a django app in Apache [Windows]

This article explains how to host a django application in Apache server. The following steps are valid for both 32 and 64 bits machine but be sure to install all of the tools for the same architecture.

For the following steps, we imagine that our project is located in C:/tools/SupervisionTool and is structured as shown below:

SupervisionTool/
  Analyzers/
    migrations/
    static/
    templates/
    ...
  SupervisionTool
    settings.py
    urls.py, wsgi.py
  manage.py


Before going further, be sure that you have the Administrator rights on the machine.

1. Install Python 3.6 and ensure that it is added to PATH
2. Install the last version of WAMP
3. Install VC15 redistributable tools
4. Create an environment variable named MOD_WSGI_APACHE_ROOTDIR and set it to apache install directory (c:/<wamp install dir>/bin/apache/apache<version>)
5. If you haven't created a virtual environment for your django application, start with

pip install virtualenvwrapper-win
mkvirtualenv SupervisionTool

<install all the packages necessary for your application (django,pypiwin32,...)>

Then

workon SupervisionTool
pip install mod_wsgi
mod_wsgi-express module-config

The last command will print the lines that will be necessary to configure Apache. You will see something like:


LoadFile "c:/python36/python36.dll"
LoadModule wsgi_module "c:/users/administrateur/envs/supervisiontool/lib/site-packages/mod_wsgi/server/mod_wsgi.cp36-win_amd64.pyd"

Copy these lines and add them to c:/<wamp install dir>/bin/apache/apache<version>/conf/httpd.conf

6. Open c:/<wamp install dir>/bin/apache/apache<version>/conf/extra/httpd-chosts.conf and replace the content with the following lines:

# virtual SupervisionTool
<VirtualHost *:80>
    ServerName localhost
    ServerAlias supervisiontool@lni-swissgas.lni.ads
    ErrorLog "logs/supervisiontool.error.log"
    CustomLog "logs/supervisiontool.access.log" combined
    WSGIScriptAlias /  "C:/tools/SupervisionTool/SupervisionTool/wsgi.py"
    <Directory "C:/tools/SupervisionTool/SupervisionTool">
        <Files wsgi.py>
            Require all granted
        </Files>
    </Directory>

    Alias /Analyzers/static "C:/tools/SupervisionTool/static"
    <Directory "C:/tools/SupervisionTool/static">
        Require all granted
    </Directory>  
</VirtualHost>
# end virtual SupervisionTool


Note: Replace C:/tools/SupervisionTool with your install directory.

7. Run the following command from the project root dir to serve all of the static file (admin included) from a unique directory (defined by STATIC_ROOT):

python manage.py collectstatic

8. Set ALLOWED_HOSTS in SupervisionTool/settings.py to ['*'] (or list the IP adresses or symbols that will give access to the app)
9. Open SupervisionTool/wsgi.py and replace its content with :

activate_this = 'C:/Users/administrateur/Envs/SupervisionTool/Scripts/activate_this.py'
# execfile(activate_this, dict(__file__=activate_this))
exec(open(activate_this).read(),dict(__file__=activate_this))

import os
import sys
import site

# Add the site-packages of the chosen virtualenv to work with
site.addsitedir('C:/Users/administrateur/Envs/SupervisionTool/Lib/site-packages')

# Add the app's directory to the PYTHONPATH
sys.path.append('C:/tools/SupervisionTool')
sys.path.append('C:/tools/SupervisionTool/SupervisionTool')

os.environ['DJANGO_SETTINGS_MODULE'] = 'SupervisionTool.settings'
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "SupervisionTool.settings")

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()


9. Open Services in Windows and fin Apache (here wampapache) and set the start mode to Automatic. Finally, start apache service.

10. Type localhost in your web browser and verify that your app is accessible.


Troubleshooting



  • Compilation error with mod_wsgi-express module-config : The most probable reason is that you mixed 32/64 bits architectures when installing the tools. Try a cleanup and reinstall.
  • Apache fails to start in Services : Potentially a syntax error in configuration files. To get more information about this, right click on Wamp tools in tray icon menu, select Tools->Show Apache loaded Modules
  • Still not working ? Ensure that you configured PYTHONPATH correctly on host (here, it should be set to C:/tools/SupervisionTool/Analyzers)


Read More

Thursday, October 26, 2017

WCE7 : Add chinese characters support

This article explains how chinese characters can be added to WinCE7 OSDesign. Without this feature, the views would display empty square shapes instead of chinese text, meaning that the decoding of input strings failed.

Project properties

Right-click on your OSDesign project and select Locale in the left panel.


On the right side, fill all parameters with languages that you want to include in the image and save.

The NLS Table

The NLS table is a list of locale identifiers that is necessary to .NET Compact framework. The local identifiers of the languages that you want to include in your image must be listed in this table.

To do this, copy the file named nlscfg.inf from %_WINCEROOT%\public\COMMON\oak\files and paste it to %_WINCEROOT%\PLATFORM\<yourBSP>\FILES. Edit this file and overwrite its content with the following:

007f 0403 0406 0407 0807 0c07 1007 1407 0409 0809 0c09 1009 1409 1809 1c09 2009
082C 042C 0423 0402 0845 0445 0405 0465 0408 540A 0425 0429 0464 0447 040D 0439
101A 041A 040E 042B 0411 0437 043F 044B 0412 0457 0440 0427 0426 042F 0450 044E 
0446 0415 0418 0419 044F 041B 0424 041C 1C1A 0C1A 281A 181A 081A 045A 0449 044A 
041E 041F 0444 0422 0420 0843 0443 042A 0804 0C04 1404 1004 0404 2409 2809 2c09 
3009 3409 040a 080a 0c0a 100a 140a 180a 1c0a 200a 240a 280a 2c0a 300a 340a 380a 
3c0a 400a 440a 480a 4c0a 500a 040b 040c 080c 0c0c 100c 140c 180c 040f 0410 0810 
0413 0813 0414 0814 0416 0816 041d 081d 0421 042d 0436 0438 043e 083e 0441 0456
045e 3801 3C01 1401 0C01 0801 2C01 3401 3001 1001 1801 2001 4001 0401 2801 1C01
2401
LOC_INCLUDELOCALES


This table lists all identifiers supported by WCE7. As Erwin pointed out in his blog, some locale identifiers are not supported and will generate exceptions if used in your application.

Pick the right items from Catalog

Look for the following sysgen variables and ensure that they are checked :

SYSGEN_DOTNETV35_SR_CHS
SYSGEN_DOTNETV35_SR_CHT
SYSGEN_UNISCRIBE
SYSGEN_ FONTS_MSYH
SYSGEN_ FONTS_MSYHBD
SYSGEN_ FONTS_MSJH
SYSGEN_ FONTS_MSJHBD
SYSGEN_NLS_ZH
SYSGEN_NLS_ZH_* (all chinese locales)


Now, ensure that the following variables are NOT checked:

SYSGEN_GB18030
SYSGEN_AGFA_FONT

Finally rebuild the entire image. You should see a big difference in the image size (mine went from 37M to 111M).
Read More

Thursday, October 5, 2017

Application logs

Logging is a fundamental mechanism for the application. It brings valuable information to support teams when something goes wrong and saves a lot of time in bug investigation. But this utility can easily turn into a "spam engine" if the developers do not carefully select what/when/how much to log.

What do we expect from logs

A support team will look for relevant events or user actions that can explain a troublesome situation. Here is an example:

09:50:05 [INFO] [AppOperation] Mix configured with CMP=Nitrogen, CST=SynthAir, C=245,8 ppm
09:50:05 [INFO] [Solver] Computed flows for (C=2445,8 pp) : MFC1=1200 mlmin, MFC2=2450 mlmin
09:50:06 [INFO] [AppCalibration] Compensation of MFC1=1200 mlmin to MFC1=1202 mlmin
09:50:06 [INFO] [AppCalibration] Compensation of MFC2=2450 mlmin to MFC1=2449 mlmin
09:50:06 [INFO] [Peripherals] Setting MFC1=1200 mlmin
09:50:06 [INFO] [Peripherals] Setting MFC2=2449 mlmin
09:50:06 [INFO] [Peripherals] Setting Hardware state to Mix2Levels
09:50:06 [INFO] [AppOperation] Mix started
09:52:08 [ERROR] [AppAlarm] (Stability Alarm ON) MFC1 flow is not stable (Measured=1015 mlmin)

In this scenario, the user complains about poor results on his device. We suppose that the Alarm was not signaled on a visual indicator to inform the user. With this bunch of logs, developers can immediately:
  • Point the origin of the issue 
  • Get the conditions of the operation that lead to the issue and try to run the same experiment at his office
Obviously, none of these actions will solve the problem but the information that logs gave to the developer will highlight which functional part went wrong, and help him to follow the right track to resolve the client issue.
 It is essential for developers to have the following information in logs:
  • Timestamp 
  • Log level : developers will filter the errors or warnings first to find an obvious reason for the malfunction 
  • Synthetic information : Long phrases are useless: a log must go the point, there is no litterature trophy for well-written logs. Most of the time, use action verbs and parameters. 
  • Workable data : the parameters listed in our logs specify numerical values with a unit. It seems obvious to say that without the unit, the developer cannot investigate. Plus, the location where the error occured is necessary. In our example, the name of classes are displayed. Either do this or use functional area names logged (ex: Config, Security, Alarms, ...). 

Other things to have in mind:
  • Keep files small : do not go over a few MB for text files (I do not include DB here). Opening and parsing a heavy file is long and inefficient. Use several small rolling files instead. 
  • Keep short history : there is no need to keep traces from last week if nothing happened there. This is closely related to file size for text logs.

What to log

  • Application events: notifications, alarms, state changes, ... 
  • Operation-related information : settings update, user actions, ...

What NOT to log

  • Periodic data : if live values need to be logged regularly, put them in a separate log file and preferably go for a DB that can handle large amount of entries better. Developers can then refer to timestamps for investigations. 
  • GUI input errors : if a user enters invalid value, it is not an error for the application, it is an error for the form. As long as the frontend does not interact with the application, whatever happens between the user and the form does not have an impact on the application.

Where to log

A software architecture is built with modules interacting with each others. The business logic of the application is dispatched between these modules. So basically, each functional area of the application shall manage its own logs. To do things neatly, centralize the logging features in a service shared by all components.
Do not repeat information. Take the following example: A calls B, B fails. This is the type of log that we should expect from it:

10:00:01 [INFO] [A] Action A1 started
10:00:01 [INFO] [B] Setting xxx=1234
10:00:01 [ERROR] [A] Action A1 failed (Code=Invalid Setting Error)

and not something like :

10:00:01 [INFO] [A] Action A1 started
10:00:01 [INFO] [B] Setting xxx=1234
10:00:01 [ERROR] [B] Given value is invalid
10:00:01 [ERROR] [A] Action A1 failed (Code=Invalid Setting Error)

The error occured at a known place, so either log it from there or from the client (recommended).

Debugging information

Take the following example of code:

void ExecuteUserRequest()
{
   env = GetEnvironmentValue();
   if(env < THRESHOLD)
   {
      doA();
   }
   else
   {
      doB();
   }
}
Here, myFunc behaves depending on an external factor. Here, we can chose to log the action (doA or doB) with an INFO level and add debugging information to the user to help him understand why this precise action occured :

void ExecuteUserRequest()
{
   env = GetEnvironmentValue();
   LogDebug("env = %d", env);
   if(env < THRESHOLD)
   {
      LogInfo("Executing A");
      doA();
   }
   else if(env >= THRESHOLD)
   {
      LogInfo("Executing B");
      doB();
   }
}
Generally speaking, Debug level gives execution details and exposes implementation data to the developer.

Memo

Here are some basic rules that I use to write my application logs.

Code type Log level
user input error (GUI) No log.
Frequently accessed functions (ex: sensor value read) Debug or no log if we log in a text file. Use a DB if possible.
Implementation detail (subroutines, intermediate values,...) Debug
User action or application event Info
Alarm or abnormal situation that does not result in an error immediately Warn
Operation failure on user action Warn if the error is normal given the context (the product acts as specified). Error otherwise.
Application error (internal non-critical errors or errors managed by the application) Error
Critical non recoverable error (unmanaged exception, errors that lead to a non-functioning application) Fatal
Read More

Friday, July 21, 2017

Build U-Boot for BeagleCore

BeagleCore is a variant of the BeagleBone Black. It consists of 2 modules : BCM1 and BCS2:
* BCM1 is the System-On-Module that contains most of the BeagleBone Black design
* BCS2 is a mainboard with the form factor of the original BeagleBone Black. The BCM1 is soldered on BCS2 and the pair is an equivalence of the BBB.

Recently, I needed to run U-Boot on a custom board built with a BCM1. I followed some tutorials on the internet and the result was that U-Boot never started. After some investigations, I found out that the EEPROM that contains BBB identification is not present on BCM1, but is on BCS2 instead. The consequence is that U-Boot cannot identify the board and the initialization procedure fails.
In this post, I will detail every step to make U-Boot run on BCM1.

Note: All of these steps were performed on Windows 10 with bash (ubuntu terminal).

Install tools

sudo apt-get install gcc-arm-linux-gnueab

Create an alias to simply following commands


alias armmake='make -j8 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- '

Grab U-Boot

git clone
git://git.denx.de/u-boot.git u-boot
cd u-boot

Edit the code


Usually, we do a patch for these type of things. I'll try to post it later.
We will modify some parts of the code to stub the EEPROM so that U-Boot behaves exactly like it was a BBB board.

Open board_detect.c and add the following right after the #include statements:
static struct ti_common_eeprom stub_eeprom = 
{
    TI_EEPROM_HEADER_MAGIC,
    "A335BNLT",
    "00C0",
    "1716BBBK2450",
    "",
    {{0x11,0x22,0x33,0x44,0x55}, // Dummy Mac addresses
     {0x11,0x22,0x33,0x44,0x55},
     {0x11,0x22,0x33,0x44,0x55}},
    11749353662462671858,
    12656917672993063804
};

Replace the content of the function 'eeprom_am_set' with
{
    return 0;
}

Do the same with 'eeprom_am_get'.

Open board_detect.h.

Replace TI_EEPROM_DATA definition with:
 #define TI_EEPROM_DATA ((struct ti_common_eeprom *)\
                &stub_eeprom)

Config & Compile

armmake  distclean
armmake am335x_boneblack_defconfig
armmake

Copy to an SD Card


Format an SD card to FAT32 (boot flag ON).
Go to your U-Boot directory and copy MLO to SD Card first.
Then copy u-boot.img to SD-Card.

And there you go ! U-Boot shall start normally.
Here is an example of my uEnv.txt :
bootpart=0:1
bootdir=
bootfile=uImage
fdtfile=am335x-boneblack.dtb
fdtaddr=0x80F80000
loadaddr=0x80200000
optargs=quiet
mmcdev=0
mmcroot=/dev/mmcblk0p2 ro
mmcrootfstype=ext4 rootwait
loadbootenv=load mmc ${mmcdev} ${loadaddr} ${bootenv}
loadfdt=load mmc ${bootpart} ${fdtaddr} ${bootdir}/${fdtfile}
uenvcmd=load mmc 0 ${loadaddr} ${bootfile};run loadfdt;setenv bootargs console=${console} ${optargs};bootm ${loadaddr} - ${fdtaddr}


Read More

Tuesday, June 20, 2017

Machine learning basics

There are several types of machine learning but we will focus on the followings in this article:

  • Supervised learning
  • Unsupervised learning
  • Reinforcement learning

Supervised learning


Basically used to make predictions about future data from labeled/categorized training dataset.


A dataset is a table where:

  • each row is a sample
  • each column is a feature
  • each row is labeled with a class label

 Classification


A supervised learning task with discrete class labels is called a classification task.
Classification is a subcategory of supervised learning where the goal is to predict the categorical class labels of new instances based on past observations.

Example:

Dataset with  [data; label]
Data 0 : [ I like sport; Present]
Data 1 : [ I love shopping, Present]
Data 2 : [ I was in Amsterdam, Past]
Data 3 : [ I did something wrong, Past]
Data 4 : [ I do exercise every day, Present]

New data:

I did not know --> Past
I love running --> Present

With 2 possible class labels, the task is a binary classification task.
With more, it is a mutli-class classification task.

Example of multi-class dataset:

[Picture of cat; cat]
[Picture of dog; dog]
[Picture of mouse; mouse]

Here the machine learning system would be able to recognize a dog, a cat or a mouse but wouldn't succeed with any other animal because it is not part of our dataset.

Typical example of two-dimensionnal dataset for a binary classification task:

Data 0 : [  [0;0] ; Orange]
Data 1 : [  [1;1.5] ; Orange]
Data 2 : [  [1;2] ; Orange]
Data 3 : [  [1;2.8] ; Orange]
Data 4 : [  [2;1.5] ; Orange]
Data 5 : [  [2;2.5] ; Orange]
Data 6 : [  [3;0] ; Blue]
Data 7 : [  [3;1.5] ;Blue]
Data 8 : [  [3;2] ; Blue]
Data 9 : [  [4;2.8] ; Blue]
Data 10 : [  [4;1.5] ; Blue]
Data 11 : [  [4;2.5] ; Blue]
Data 12 : [  [5;3] ; Blue]

It is two-dimensionnal because each sample of the dataset has 2 values (usually named x1,x2). If we represent these samples on a 2-dimensionnal graph, we would see this:


The prediction would be based on the distribution of the sample. A point with x1 > 3 would be predicted as Blue and a point with x1 < 2 would potentially be red.

Regression


Regression is also called prediction of continuous outcomes. In regression analysis we give a serie of numbers (x or predictor) and response variables (y or outcome) and we try to find a relationship between them to predict a future outcome.

Ex:

with [x;y]
Data 0 : [ 0 ; 0 ]
Data 1 : [ 1 ; 1.5 ]
Data 2 : [ 1.5 ; 1 ]
Data 3 : [ 2 ; 2 ]
Data 4 : [ 2.5 ; 2.6 ]
Data 5 : [ 3 ; 3.2 ]
Data 5 : [ 4 ; 3.9 ]

Several types of algorithm can be selected to process input data. The following figure illustrates the concept of linear regression:


The computed curve will be used to predict the outcome of new data.

Reinforcement learning


Here the goal is to develop a system (agent) that improves its performances based on interactions with environment. The system will receive a feedback (reward) for every one of its actions. Each reward informs him of the quality of his action.
The agent will learn a series of actions that maximizes this reward via an empirical try-and-error approach.

A typical example is Google's Deepmind which beat the best Go players.

Unsupervised learning


In supervised learning, we include the right answer (labels) into the dataset. Here, we don't know the right answer beforehand. We are dealing with uncategorized data with an unknown structure.
With unsupervised learning, we can explore the structure of our data to extract meaningful information without an outcome or a reward.

Clustering


Clustering is an exploratory data analysis technique which groups data together by similarity (unnsupervised classification).

Dimensionality reduction


Dimensionality reduction is another unsupervised learning field. To prevent against the computation of huge amounts of data which results in performance and storage issues, unsupervised dimensionality reduction preprocesses data to remove noise and retain relevant information.


Read More

Thursday, June 15, 2017

Django + Js : display counting timedelta in view

Suppose you have an entry in DB with a timestamp 'start_date'. You want to display the time elapsed since 'start_date' in your view and you want that delay to grow in real time like a clock. In the following example, the user can select one entry at a time and the counter has to be updated accordingly.

Note: To achieve this it is important to work with UTC timestamps in both DB and your view

Here is what my code in django views.py looks like:

def get_entry_info(request):
    """
    Recover information for the specified entry name
    """
    entry_name = request.POST.get("name", None)
    if request.method == "POST" and entry_name is not None:
        data = {}
        # Get entry object 
        entry = MyEntries.objects.get(name=entry_name)
        data['start_date'] = int(time.mktime(entry.start_date.replace(tzinfo=None).timetuple())) * 1000 
        return JsonResponse(data)
    else:
        return HttpResponse("Invalid entry name")

Here is what my javascript code looks like:
var startDate;

// Loads content into the information panel
// @param data : data to be displayed in the information panel
function reloadInformationPanel(entryname){
    // Get job information from server
    $.ajax({
        headers: { "X-CSRFToken": '{{ csrf_token }}' },
        url: "{% url 'get_entry_info' %}",
        method: 'POST', 
        dataType: 'json',
        data: {
            'name': entryname, // outgoing data
        },
        success: function (data) {        
            startDate = new Date(data.start_date);
            startTime();
        },
        error: function(xhr,errmsg,err) {
            console.log(xhr.status + ": " + xhr.responseText); 
        }
        });
}

// Starts timer of job duration
function startTime() {
   var now = convertDateToUTC(new Date());
   var delay = new Date(now - startDate);
   document.getElementById('entry_duration').innerHTML = delay.getUTCHours() + "h" + delay.getUTCMinutes() + "m" + delay.getUTCSeconds() + "s";
   var t = setTimeout(startTime, 500);
}

// Converts a Date object to UTC
function convertDateToUTC(date) { 
   return new Date(date.getUTCFullYear(), date.getUTCMonth(), date.getUTCDate(), date.getUTCHours(), date.getUTCMinutes(), date.getUTCSeconds()); 
}

Now simply map reloadInformationPanel to the entry selection button ;)

Read More

Thursday, April 27, 2017

What is AJAX ?

AJAX = Asynchronous Javascript And Xml

Ajax is not a tehnology by itself but rather a combination of existing technologies (HTML/CSS/DOM/Javascript/XML/JSON).

What is it intended for ?


Ajax is typically used to refresh data on a web page without having to reload the entire page (e.g. asynchronously). It involves a web browser sending HTTP requests (GET/POST)  to server and processing the response to finally manipulate the page DOM (e.g. HTML tags). The user's view is thereby dynamically updated.


As you can see, Ajax requests are executed by Javascript code and the response is also handled in Javascript. The orange part stands on client's side (the web browser).

What if the format of data ?


Originally, it was XML but nowadays, JSON is preferred (JavaScript Object Notation).


Example of ajax requests


jQuery

<script>
...
$.ajax({
        headers: { "X-CSRFToken": getCookie("csrftoken") },
        url: "myurl",
        method: 'POST', // or another (GET), whatever you need
        data: {
            'mydata': 'value', // outgoing data
        },
        
        success: function (data) {        
            // success callback
            // you can process data returned by server here
        }
        });
...
</script> 

Pure javascript

<script type="text/javascript">
function ajaxFunction()
{
var xmlhttp;
if (window.XMLHttpRequest)
  {
  // code for IE7+, Firefox, Chrome, Opera, Safari
  xmlhttp=new XMLHttpRequest();
  }
else if (window.ActiveXObject)
  {
  // code for IE6, IE5
  xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
  }
else
  {
  alert("Your browser does not support XMLHTTP!");
  }
xmlhttp.onreadystatechange=function()
{
if(xmlhttp.readyState==4)
  {
  document.myForm.time.value=xmlhttp.responseText;
  }
}
xmlhttp.open("GET","time.asp",true);
xmlhttp.send(null);
}
</script>
Read More

Monday, April 10, 2017

What is inside a WCE7 BSP

The BSP (Board Support Package) is the layer that will interface the Operating System with the hardware.

It consists of the following items:
  • [Optional] Bootloader
  • OAL (OEM Abstraction layer)
  • KITL to debug the OS in development phase
  • Configuration files that specifies the board (Ex: config.bib for memory settings)
The BSP image is specific to the OS version and to the target hardware architecture.

 

Bootloader

When powered up, the microprocessor starts executing instructions from a specified memory address located in ROM. This code can contain the entire system for small footprint applications and will run until shutdown.
With WCE7, the system image (nk.bin) needs to be loaded to RAM from a storage disk or network because it can't fit into a ROM memory. In this case, these are the typical steps of a bootloader:
  • Minimal Initialization of the system (memory and com drivers)
  • Loads the OS image to RAM
  • Jumps to OS start address in RAM and calls startup fuction
  • The OAL takes over
Windows CE 7 includes a configurable and reusable bootloader named CE Boot with the following features:
  • Can load files from disks (IDE, PCI)
  • Emulates a console to allow user interraction at boot
  • Network support: DHCP, TFTP, IP

 

XLDR

In some cases, the system can be loaded in two steps : a first bootloader loads a second bootloader to RAM. The second bootloader loads the system image to RAM et jumps to it. This is necessary when:
  • The initial startup program is too small
  • The startup program on the target system is general purpose
  • The CPU has a small amount of RAM to load an image immediately and shall initialize an external RAM to host the system
XLDR is a lightweight loader that can load CE Boot in two stages.


OAL

The OAL allows the kernel to access the hardware layer (interrupts, timers, cache, IOCTL...) through an abstract interface.
When it takes over the boot phase, it accomplishes the following steps:
  • Initialize the CPU state, hardware and kernel's global variables
  • Jumps to the entry point of the kernel
  • The kernel exchanges global pointers with the OAL. From this point on, the kernel has access to all functions and variables defined in OEMGLOBAL and OAL has access to all functions and variables defined in NKGLOBAL
  • On kernel call, the OAL initializes the debug serial port (OEMInitSerialDebug) (like the bootloader did before without the help of the OAL)
  • On kernel call, the OAL initializes the rest of hardware interfaces on the device (OEMInit())
  • [Optional] On kernel call, KITL initialization (OEMKitlInit)
  • The kernel starts executing its first thread

 

Configuration files

File Description
config.bib Memory structure definition file
platform.bib BSP global settings. Selects which files need to included in the image depending on user-selected options/items
platform.dat Not used
platform.db Not used
platform.reg BSP global registry settings. Selects which keys to write in the registry depending on user-selected options/items
sources.cmn Starting point of the BSP build process. On top of source tree. Defines include directories and linked libraries
<MyBSP>.bat Project-specific environment variables to use in the build process. In particular, specifies the devices to include in the build.

To have a deeper look at the BSP's file structure, check the Build process.
Read More

Friday, April 7, 2017

WCE : Build process

Build process


BSP top Folder

BSP Folder Description
CATALOG BSP's catalog file (.pbcxml)
CESYSGEN BSP's makefile
FILES Files to be copied to the final image (nk.bin). Here are the main BSP configuration files. See What is a BSP ?
SRC Files needed to build the BSP

BSP SRC Folder

BSP Folder Description
INC BSP global header files
BOOT Bootloader's build directory
BOOTLOADER Networked bootloader's build directory
COMMON Code that is common to all components of the BSP
DRIVERS Target board's specific drivers
KITL KITL build directory
OAL OAL build directory

Build sequence

The build consists of 4 phases.
Phase Type Description
1 Compile phase Source and resource files are compiled into binaries. You should never need to recompile Windows CE sources.
2 Sysgen phase According to catalog items and dependenciy trees, SYSGEN variables are set. These variables act then as filters in header files to build a specific BSP.
3 Release copy phase The OS run-time files are copied to release directory.
4 Make run time image phase project.bib and project.reg are copied to release directory and according to their settings, the final run-time image is generated. Configuration files are merged: for instance, registry configuration file are merged into reginit.ini before being compiled in the final registry file.
The following diagram gives an overview of the build tools and how they interact with each other.


Behind the scene, Platform Builder relies on a set of batch scripts to compile the code. Build action calls buildemo.bat which in turn calls cebuild.bat, buildrel.bat and finally makeimage.exe.
Read More

WCE7 : Clone existing BSP

Platform Builder offers a wizard to clone an existing BSP. By doing so, we ensure that the original BSP won't be modified and we can tweak the new one for our needs. The menu is accessible from VS2008->Tools->Platform Builder->Clone BSP:

Note: If your source BSP is not in the list, ensure that a folder with its name is located in C:\WINCE700\platform.
Fill the rest of the information and click OK.


On the next window, the catalog items of your BSP will appear. You will be free to Add or Remove items:


Each item is linked to configuration variables that will be used to configure the build of the operating system. If you open the properties of any item, you should see variables like SYSGEN_xxx or BSP_xxx.
Read More

Windows CE 7 Architecture

Windows CE5 was based on a microkernel architecture.



In this type of architecture, the device drivers run in user mode which means that I/O handling routines are embedded into user-mode applications. It also means that the synchronization between the processes that share driver access needs to be done at application level, which is tricky.
NK.exe is the most privileged process of the operating system. It can access kernel memory addresses and make kernel calls.
Since Windows CE6, the architecture moved to a monolithic kernel architecture.


With this architecture, drivers and services have been moved to kernel space as dlls. NK.exe is still the most privileged process but this new architecture, by breaking the inter-process communication lines, enhances performances globally.
Component Description
GWES.dll Graphic, Windowing and Events Subsystem. It loads device drivers and manages their interface while loaded.
DEVMGR.dll Device Manager. It loads stream interface device drivers and manages their interface while loaded.
I/O Resource Manager Part of Device Manager that enumerates the available resources from registry
FILESYS.dll & FSDMGR.dll File system manager and file system driver manager.
Networking Dlls Network functionality (HTTP, TCP/IP, FTP...)
UDEVICE.exe User mode process that runs device drivers in user-mode
OAL OEM Abstraction Layer. Gives to the kernel an access to OEM hardware and maps interrupts with logical IDs
COREDLL.dll OS API (semaphore, thread, mutex, critical section...)
K.COREDLL.dll Kernel version of CoreDll (kernel device drivers links against this one)

Device driver: User mode vs Kernel mode

From performance point of view, it is better to embbed a driver in kernel space.
From security point of view, a bug in the driver might jeopardize the whole system.
For robustness, udevice.exe provides installable device driver feature, thus protecting the kernel.

Interrupt management



Component
Description
Kernel Interrupt Service Handler Can handle up to 64 separate hardware IRQS sources. It determines interrupt priority level and calls an ISR hooked to it and then sets event that is associated with interrupt ID (SYSINTR ID)
Interrupt Service Routines (ISRs) Hooked to IRQ in kernel mode. Acknowledges hardware and determines interrupt ID
Interrupt Service Threads (ISTs) Device-specific servicing for the specified Interrupt ID. Must signal completion of processing to hardware.
1. A device raises a registered interrupt.
2. The kernel catches the interrupt and calls the related Interrupt Service Routine (ISR).
3. ISR handles the interrupt and exits as quick as possible.
4. IST in driver is signaled to process the interrupt.
5. IST completes processing.
Read More

WCE7 detailed boot procedure

Here is a detailed description of how, theoretically, Windows CE boots on a device..

BUILD-TIME:

1. ROMIMAGE fixes up binaries in the OS image
An EXE or a DLL can be fixed up, which means that they are modified to adjust the variables and functions addresses depending on the binary location in virtual memory. At build time, the compiler doesn't know where the binaries will be located, so all of the pre-built addresses are wrong and need to be updated at load time. This is exactly what ROMIMAGE does when inserting the binaries in the final image that reflects the device's memory. To achieve this, ROMIMAGE relies on config.bib.
Note: All EXE and DLL files have a reserved section to be fixed up.
Note 2: All EXE and DLL files have an entry point that can be read in binary.

The image will contain 2 major binaries that will be required to boot:

Binary Description
NK.exe Contains all the functionality that is platform and architecture specific (typically, the OAL)
kernel.dll Contains architecture-neutral functionalities 

2. ROMIMAGE places tge TOC in the image file and fills pTOC variable with the location address
TOC stands for "Table Of Content". When all binaries have been placed in the image by ROMIMAGE, the start address of every binary is listed into a TOC structure. The TOC is then placed in the image as well.
The TOC will be needed at runtime and is accessible in the OAL with a pointer named pTOC. The value of this pointer is -1 at build time because the location of the TOC is unkown at this moment. The value is updated by ROMIMAGE once the TOC has been placed in the image.

Using the TOC, the program can find all other pieces of the operating system image.

BOOT TIME:

3. Bootloader puts the image at the right place in memory
4. Bootloader looks for the TOC in the image
Bootloader does not have access to pTOC variable (it is not part of NK.exe). Instead, it will use a trick done by ROMIMAGE.
When the TOC is placed in the image, it is prepended with a marker : "CECE" (or 0x44424442). The bootloader will find this marker and read the TOC.
5. Bootloader looks for NK.exe in TOC
6. Bootloader jumps to the entry point of NK.exe
7. NK.exe calculates the physical address of OEMAddressTable
At this moment of boot, the memory can only be addressed physically.
OEMAddressTable is a symbol in NK.exe, which means that it has a virtual memory fixed by ROMIMAGE. NK.exe knows its own physical location in memory and its future location in virtual memory.
OEMAddressTable physical address = (Physical address of NK.exe) + ( (OEMAddressTable virtual address) - (Virtual base address of NK.exe) )
8. NK.exe sets up the virtual memory mapping for the MMU
Virtual memory is now enabled and NK.exe moves to virtual memory.
However, RAM has not been initialized yet which means that all read/write variables can have any content whatsoever.
9. NK.exe copies the "copy entries" to RAM (R/W portions or RAM)
The function that does this is KernelReloacte().
The TOC describes also the RAM and where the read/write portions of each module are to be located. Pieces of OS image that need to be copied to RAM are called "copy entries".
Note: Obviously, pTOC is not located in RAM otherwise none of this would work. It is in ROM (const variable).
10. NK.exe sets up the Kernel Data Page and its initial content
Windows CE reserves a few regions of virtual address range for its own private use inside the OS kernel. There are several 4k pages of virtual memory from 0xFFFE0000 upwards. At least one page is reserved to Kernel Data Page.
NK.exe sets up the location and inserts:
  • A copy of TOC
  • The address of OEMAddressTable
  • The address of the function OEMInitGlobals()
11. NK.exe jumps to kernel.dll entry point with the address of the kernel data page as parameter
12. kernel.dll calls OEMInitGlobals() with the address of NKGlobals as parameter and gets the dadress of OEMGlobals as a return value

Variable Description
NKGlobals Static table of functions (ex: SetLastError(), NKwvsprintfW() ) and data in kernel.dll
OEMGlobals Static table of functions pointers and data : PFN_InitDebugSerial(), PFN_Ioctl(),... The functions pointers points to legacy OEM_xxx() functions (like OEMInitDebugSerial())

Once the call to OEMInitGlobals() completes, the kernel.dll has everuthing it needs to architecture-generic and platform-specific processing. It knows how memory is laid out virtually and where are all modules in the image.
NK.exe has also access to a set of kernel functions that it can call.
13. kernel.dll runs architecture-specific setup (ARMSetup() or X86Setup())
During this step, Interlocked API will be copied to Kernel Data Page. This is a critical part of the system that manages the coordination between threads.
14. kernel.dll runs architecture-neutral setup
Initialization of kernel debug output, selection of kernel processor type, check of the presence of KITL.dll.
Note: KITL is the system debugger.
Note 2: KITL can be loaded as a dll or directly included in the OAL. If the dll exists, it is loaded here.
15. kernel.dll runs platform-specific setup (OEMInit())
Platform specific initialization. KITL is started (if included in the image).
16. kernel.dll runs its first thread (SystemStartupFunc())
17. SystemStartupFunc() initializes the system
System loader, paging pool, system logging, system debugger, message queue, watchdog.
Creates a thread for Power Management.
Creates a thread for File System.
18. SystemStartupFunc() creates another thead which calls RunAppAtStartup()
This new thread creates first user processes.
Read More

WCE7 : Types of drivers

The Windows CE device drivers model takes into consideration 3 types of drivers:

Type of driver Description
Native drivers Built-in device drivers. They implement a feature-specific interface (Device Drivers Interface, DDI). They are loaded by GWES. Typical built-in drivers are:
- Display driver
- Touchscreen driver
- Keyboard driver
- Mouse driver
- LEDs drivers
Stream drivers Installable drivers. They implement a standard IO interface used for all types of IO devices (Init, DeInit, Open, Close, Read, Write...). They are configured in the registry and mounted by a device manager. They are very easy to add. Typical stream drivers are:
- Serial drivers
- Bus drivers
- Block drivers (for storage devices)
- Specific stream drivers
Hybrid drivers Stream drivers that implement an additional specific interface to their dedicated feature. Mounted by device manager but used by other processes through their specific interfaces. Example of hybrid drivers:
- Audio drivers
- PCMCIA controller drivers
- USB drivers

Types of architecture

Type of architecture Description
Layered Split into 2 layers : Model Device Driver (MDD) and Platform dependent device (PDD). The MDD implements the functional behaviors of the device that are independent of the device that is managed by the driver. Usually, this layer does not need to be modified.
The PDD implements treatments that are specific to the device being managed.
Windows CE device driver model defines the interface of PDD allowing the development of a PDD without caring about what MDD does. The interface of PDD is standardized but specific for each type of driver (Device Driver Service provider Interface, DDSI)
Monolithic Merge of MDD and PDD into one layer. Poor reusability. This type of drivers are mainly used for simple driver, without functional layer.


Read More

WCE7 : Customize your BSP

We suppose here that all the components required for your OSDesign are already in the catalog of your source platform. This is basically the case when you clone an existing BSP.
To customize your BSP, you will have to check the features you want to include in your OS image. Additionally, you can add registry settings, files or even add a new device driver if required (this is not explained here). This page gives an overview of a BSP customization for the BeagleBone Black platform.

 

Select your features


BSP features

Open Catalog Items View and unfold Third Party->BSP-><your BSP name>.
You have here all platform-specific components that you can reuse on your target platform.
Here is a picture of a possible selection for the BBB:


Core OS

Open Catalog Items View and unfold CoreOS->Windows Embedded Compact.
The core OS features are platform-independent which means that they can be used on any target platform. Some of these options are related to the BSP features (via SYSGEN_xxx variables) and that's why some checkboxes are already checked (with a green square usually).
Below is an overview of the checked features for the BBB. We excluded IE7 support, media players, audio decoders, battery support... :


Add files to your BSP

If you want to add some files to your BSP, they need to be placed in C:\WINCE700\platorm\<your bsp>\Files.
All files located in this directory will be copied to the build release directory during build process.
To manage the files you want to copy to your output directory, you need to edit OSDesign.bib (you can also do this from C:\WINCE700\platorm\<your bsp>\Files\PLATFORM.BIB but it would applied to all OSDesigns).
For instance, if you want to add CoreCon files, you should append the following lines under FILES section:
eDbgTL.dll            $(_FLATRELEASEDIR)\eDbgTL.dll                    NK    S
TcpConnectionA.dll    $(_FLATRELEASEDIR)\TcpConnectionA.dll            NK    S

Reminder: All of these files need to be located in C:\WINCE700\platorm\<your bsp>\Files.
Setting Description
NK NK memory block specified in config.bib file (output NK.bin image)
S System file (will be located as system file in \Windows directory)
H Hidden file
K Kernel file
(for example, use SHK for a hidden system file located in kernel space).

 

What if I put these lines under MODULE section ?

The files put in MODULES will be processed by romimage so that the executable can be executed in eXecute In Place (XIP) from the ROM image. If you are not interested in that, put files in FILES section.

KITL and serial Debug

As explained here
When launching an OS image from the target device’s local storage, built with the above debugging helper components enabled, the OS image may not be able to function as intended and may not be able to complete the boot process, such as:
  • When an OS image is generated with KITL component enabled, it attempts to establish connection with an unavailable KITL connection, which causes the OS not able to complete the boot process.
  • When an OS image is generated with the serial debug component enabled, the serial debug component capture one of the available serial port, making the port unavailable to the device’s application.
So go through the following steps before building the image:
  • Open your project properties and disable KITL. Clik OK to apply

Additional registry settings


FTPD settings

Add these lines to your OSDesign.reg file to allow unauthenticated connections with anonymous credentials.
Set the default directory to root dir to give an access to all files remotely.
[HKEY_LOCAL_MACHINE\COMM\FTPD]
    "IsEnabled"=dword:1
    "UseAuthentication"=dword:0
    "UserList"="@*;" 
    "AllowAnonymous"=dword:1
    "AllowAnonymousUpload"=dword:1
    "AllowAnonymousVroots"=dword:1
    "DefaultDir"="\\" 

.NET CF runtime error bypass

The following error can happen at the execution of a .NET application after boot
.NET CF Initialization Error, The application failed to load required components. If the .NET Compact Framework is installed on a storage card... Support info:
-2147438590 (8000B002)
However, if we deploy an application from a PC to the remote device and start an app again it will work. This strange beahavior is due to a registry setting that is updated when we deploy an application.
To avoid this issue, edit OSDesign.reg and add the following lines
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NetCompactFramework\Managed Debugger]
    "AttachEnabled"=dword:0
And add the following files to your OSDesign (in C:\WINCE700\platform\<your BSP>\FILES):
%COMMONPROGRAMFILES%\Microsoft shared\CoreCon\1.0\Target\wce400\armv4i\eDbgTL.dll
%COMMONPROGRAMFILES%\Microsoft shared\CoreCon\1.0\Target\wce400\armv4i\TcpConnectionA.dll
Read More

WCE7 : How to write a device driver


Quick introduction

An application interacts with a device driver thanks to file system functions (CreateFile, ReadFile, WriteFile,...).
To identify the driver to be opened, the application needs to specify a file name to CreateFile:

* [Usual format] "XXXy:" : with X a capital letter and y a number in [1;9] (0 is authorize to indicate the 10th index)
* [Device mount point] "\\<device_mount_point>\\XXXy:"
* [Bus mount point] "\\<bus_mount_point>\\XXXy:"
The three capital letters form a prefix that will identify the driver. The number indicates the index of the driver if several can be instanciated.
This prefix is also used by Windows to load the device driver methods. In fact, the prefix shall precede the device driver methods in driver dll:
* XXX_CreateFile
* XXX_ReadFile
* ...

All device drivers are listed in registry below HKLM\Drivers\BuiltIn with at least the following subkeys:

Key Description
Prefix Explained above (e.g. "XXX")
Dll Driver dll method to be loaded

At system boot, Device Manager will iterate over these entries and load them. For every loaded driver, an entry will be created in HKLM\Drivers\Active.
The order of load can be specified if the driver entry specifies a key Order followed by a number that indicates when it shall be loaded (with 1 the first to be loaded)
Other optional subkeys can be used for a driver, but they won't be described here.
In the following chapters, we will create the necessary files to add a driver to our BSP. In the scope of this tutorial, we will create a driver for the DS1307 RTC chip.

 

Add a new catalog entry

The first step is to add a catalog entry to our BSP. To do so, either open the *.pbcxml file with Visual Studio and use the graphical interface or edit the *.pbcxml file and add the following lines:
<Bsp>
    ...
    <BspItemId>FT5X06_TOUCH:LNI Swissgass:FT5X06_LniBSP</BspItemId>
    <BspItemId>RTC_DS1307:LNI Swissgass:RTC_DS1307_LniBSP</BspItemId>
    <BspItemId>DVI_Type:LNI Swissgas:DVI_Type_LniBSP</BspItemId>
    ...
</Bsp>
<Item Id="RTC_DS1307:LNI Swissgass:RTC_DS1307_LniBSP">
    <Title>RTC driver</Title>
    <Description>Real Time Clock</Description>
    <Comment>Only valid for DS1307 chip</Comment>
    <Type>BspSpecific</Type>
    <Variable>BSP_RTC_DS1307</Variable>
    <Module>rtc_ds1307.dll</Module>
    <Location ConditionLocation="">Drivers\RTC</Location>
</Item>
These lines add a new driver to the catalog under Driver\RTC. If checked, this driver defines the variable BSP_RTC_DS1307.

 

Hands on code : create the driver component

Create a directory for our driver under <MyBSP>\SRC\Drivers : here we create a folder named RTC. The following steps are to be done in this directory.
The most obvious file to add to our componen is makefile. This file is always the same for all drivers so copy/paste an existing one or create it and add the following lines:
!if 0
Copyright (c) Microsoft Corporation.  All rights reserved.
!endif
!if 0
Use of this source code is subject to the terms of the Microsoft end-user
license agreement (EULA) under which you licensed this SOFTWARE PRODUCT.
If you did not accept the terms of the EULA, you are not authorized to use
this source code. For a copy of the EULA, please see the LICENSE.RTF on your
install media.
!endif
#
# DO NOT EDIT THIS FILE!!!  Edit .\sources. if you want to add a new source
# file to this component.  This file merely indirects to the real make file
# that is shared by all the components of Windows CE.
#
!INCLUDE $(_MAKEENVROOT)\makefile.def
Our driver needs to be declared in registry. Create an rtc_ds1307.reg file and the following lines:
IF BSP_RTC_DS1307
[HKEY_LOCAL_MACHINE\Drivers\BuiltIn\RTC_DS1307]
    "Prefix"="RTC" 
    "Dll"="rtc_ds1307.dll" 
    "Index"=dword:1
    "order"=dword:101                ; Should be loaded after I2C
ENDIF BSP_RTC_DS1307
The file structure of a WEC7 BSP requires a file named sources in every folder that contains files to take into consideration during build. Create one with the following lines:
!if 0
;================================================================================
; RTC Driver DS1307
;================================================================================
!endif

!IF "$(BSP_RTC_DS1307)" == "" 
SKIPBUILD=1
!ENDIF

TARGETNAME=rtc_ds1307
TARGETTYPE=DYNLINK

TARGETLIBS= \
    $(_PLATLIB)\$(_CPUDEPPATH)\ceddk.lib \
    $(_COMMONSDKROOT)\lib\$(_CPUINDPATH)\coredll.lib

SOURCES= \
    rtc_ds1307.c \
Our driver will obviously need to export the stream interface driver methods. This is done with a *.def file as follows:
LIBRARY     RTC_DS1307

EXPORTS RTC_Init
    RTC_Deinit
    RTC_Open
    RTC_Close
    RTC_Read
    RTC_Write
    RTC_IOControl
And now, we can add some code :
//
//  File: rtc_ds1307.c
//
//  This file implements device driver for RTC DS1307. 
//
#include "omap.h" 
#include "ceddkex.h" 
#include "sdk_padcfg.h" 
#include "bsp_padcfg.h" 
#include "bsp.h" 

#include <nkintr.h>
#include <winuserm.h>
#include <pmpolicy.h>

// ==== Debug Zones ================================================================================
// simple handling: debug zones are forced with TRUE or FALSE and need a recompile to be activated
// for sophisticated handling: use macro DEBUGZONE(), DBGPARAM dpCurSettings etc.
// (see MSDN e.g. Win CE 5.0, Debug Zones)
#ifndef SHIP_BUILD

#undef ZONE_ERROR
#undef ZONE_WARN
#undef ZONE_FUNCTION
#undef ZONE_INFO
#undef ZONE_TIPSTATE

#define ZONE_ERROR          DEBUGZONE(0)
#define ZONE_WARN           DEBUGZONE(1)
#define ZONE_FUNCTION       DEBUGZONE(2)
#define ZONE_INFO           DEBUGZONE(3)
#define ZONE_TIPSTATE       DEBUGZONE(4)

static DBGPARAM dpCurSettings = {
    L"RTC_DS1307", {
        L"Errors",      L"Warnings",    L"Function",    L"Info",
            L"IST",         L"Undefined",   L"Undefined",   L"Undefined",
            L"Undefined",   L"Undefined",   L"Undefined",   L"Undefined",
            L"Undefined",   L"Undefined",   L"Undefined",   L"Undefined" 
    },
    // Bitfield controlling the zones.  1 means the zone is enabled, 0 disabled
    // We'll enable the Error, Warning, and Info zones by default here
    1 | 1 << 1 | 1 << 2 | 1 << 3
};

#endif

//------------------------------------------------------------------------------
DWORD RTC_Init(LPCTSTR szContext, LPCVOID pBusContext)
//  Called by device manager to initialize device.
{
    int i;
    DWORD rc = (DWORD)NULL;

    UNREFERENCED_PARAMETER(pBusContext);

    DEBUGMSG(ZONE_FUNCTION, (L"+RTC_Init(%s, 0x%08x)\r\n", szContext, pBusContext));
    RETAILMSG(1, (L"+RTC_Init\r\n"));

cleanUp:
    DEBUGMSG(ZONE_FUNCTION, (L"-RTC_Init(rc = %d\r\n", rc));

    return rc;
}

//------------------------------------------------------------------------------
BOOL RTC_Deinit(DWORD context)
{
    BOOL rc = FALSE;

    DEBUGMSG(ZONE_FUNCTION, (L"+RTC_Deinit(0x%08x)\r\n", context));

    rc = TRUE;

cleanUp:
    DEBUGMSG(ZONE_FUNCTION, (L"-RTC_Deinit(rc = %d)\r\n", rc));
    return rc;
}

//------------------------------------------------------------------------------
DWORD RTC_Open(DWORD context, DWORD accessCode, DWORD shareMode)
{
    UNREFERENCED_PARAMETER(context);
    UNREFERENCED_PARAMETER(accessCode);
    UNREFERENCED_PARAMETER(shareMode);
    return context;
}

//------------------------------------------------------------------------------
BOOL RTC_Close(DWORD context)
{
    UNREFERENCED_PARAMETER(context);
    return TRUE;
}

//------------------------------------------------------------------------------
DWORD RTC_Read(DWORD context, LPVOID pBuffer, DWORD Count)
{
    UNREFERENCED_PARAMETER(context);
    UNREFERENCED_PARAMETER(pBuffer);
    UNREFERENCED_PARAMETER(Count);
    return 0;
}

//------------------------------------------------------------------------------
DWORD RTC_Write(DWORD context, LPCVOID pSourceBytes, DWORD NumberOfBytes)
{
    UNREFERENCED_PARAMETER(context);
    UNREFERENCED_PARAMETER(pSourceBytes);
    UNREFERENCED_PARAMETER(NumberOfBytes);
    return 0;
}

//------------------------------------------------------------------------------
BOOL RTC_IOControl(
    DWORD context, DWORD code, BYTE *pInBuffer, DWORD inSize, BYTE *pOutBuffer,
    DWORD outSize, DWORD *pOutSize
) {
    UNREFERENCED_PARAMETER(context);
    UNREFERENCED_PARAMETER(code);
    UNREFERENCED_PARAMETER(pInBuffer);
    UNREFERENCED_PARAMETER(inSize);
    UNREFERENCED_PARAMETER(pOutBuffer);
    UNREFERENCED_PARAMETER(outSize);
    UNREFERENCED_PARAMETER(pOutSize);
    DEBUGMSG(ZONE_INIT, (L"RTC_IOControl"));
    return FALSE;
}

//------------------------------------------------------------------------------
BOOL __stdcall DllMain(HANDLE hDLL, DWORD reason, VOID *pReserved)
{
    UNREFERENCED_PARAMETER(pReserved);
    switch (reason) {
    case DLL_PROCESS_ATTACH:
        DEBUGREGISTER(hDLL);
        DisableThreadLibraryCalls((HMODULE)hDLL);
        break;
    }
    return TRUE;
}
and a header (optional):
#ifndef __RTC_DS1307_H
#define __RTC_DS1307_H

// Stream interface entry points

// Initialization of controlled hardware, setup of ISR and IST(if any),
// setup of memory access (if any) 
DWORD RTC_Init(LPCTSTR szContext, LPCVOID pBusContext);
// Cleans up what Init did
BOOL RTC_Deinit(DWORD context);
// Implicitely called by CreateFile
// Opens the context for client usage
DWORD RTC_Open(DWORD context, DWORD accessCode, DWORD shareMode);
// Closes the context opened in Open
BOOL RTC_Close(DWORD context);
// Use if required : read data out of driver
DWORD RTC_Read(DWORD context, LPVOID pBuffer, DWORD Count);
// Use if required : write data to driver
DWORD RTC_Write(DWORD context, LPCVOID pSourceBytes, DWORD NumberOfBytes);
// Driver specific functionality
BOOL RTC_IOControl(
    DWORD context, DWORD code, BYTE *pInBuffer, DWORD inSize, BYTE *pOutBuffer,
    DWORD outSize, DWORD *pOutSize
);

#endif // __RTC_DS1307_H

Add driver to project build

The driver component is ready, we need to include it to the project. As said earlier, the file structure of a BSP requires sources files to include files to build but also dirs files to list the directories that will be visited during build. Open <MyBSP>\SRC\Drivers\dirs and add an entry for our new driver.
DIRS=\
        ...
    FT5X06\
    RTC_DS1307\
    WAVEDEV2\
        ...
For rom image creation, we need now to tell the project that it should put the driver's dll into the image.
Open platform.bib and add the following lines:
...
IF BSP_RTC_DS1307
rtc_ds1307.dll    $(_FLATRELEASEDIR)\rtc_ds1307.dll            NK  SHK
ENDIF
...
Finally, add the following lines to platform.reg to integrate the component's registry settings:
...
;===============================================================================
; RTC DS1307 driver
#include "$(PLAT_DRIVERS_DIR)\RTC_DS1307\rtc_ds1307.reg" 
...
You're good to start the development of the driver.
Read More

WCE7 : Develop a touchscreen driver

The Compact 7 touch driver architecture includes a touch driver and a touch proxy driver (WinCE 6 legacy interface).


Component Description
Touch driver Stream interface driver implemented as a layered driver. MDD exports standard stream interface functions (Init, PreDeInit, PreClose, Close, PowerDown, PowerUp, Read and IOContol). Since this is a stream interface driver, it is loaded by Device Manager.
Touch proxy driver Provides the WinCE 6 touch API. GWES loads this driver instead of touch driver.
Touch calibration DLL Not compiled in the driver. Separate DLL provided by Compact 7
The picture above, shows:
  • The initialization mechanism with blue arrows
  • The touch event management with red arrows

Where to start ?

Most of the components introduced in the first part are already included in Compact 7:
  • Touch driver proxy : _WINCEROOT\Public\Common\Oak\Drivers\touch
  • Touch driver MDD : _WINCEROOT\public\common\oak\drivers\touch\tchstreammdd
  • Calibration DLL : _WINCEROOT\public\common\oak\drivers\touch\tchcaldll
The only part that needs to be developed is the touch driver PDD which is the platform-dependent module (MDD is the platform independent module).
The interface of PDD is defined by the DDSI (Device Driver Stream Interface) and includes the following methods:

Function Description
TchPdd_Init Returns the pointers to the DDSI interface and initialize hardware
TchPdd_DeInit Clean up (hardware deinit, memory release...)
TchPdd_PowerUp Called by MDD (implicitely by power manager) at power-up
TchPdd_PowerUp Called by MDD (implicitely by power manager) at power-down
TchPdd_Ioctl Specific-features offered to the MDD

Enable touch screen mechanism

To enable the Compact 7's internal touchscreen, the following variables need to be defined in the OS Design from catalog item view.

Variable Description
SYSGEN_TOUCH If defined, the touch cursor will be set
Read More

WCE7 : How to debug the BSP

Debugging is a key part in driver development. It helps to step into the code at runtime and analyze the memory stacks.

 

How to enable debugging ?

Debugging is handled by KITL (Kernel Independent Transport Layer) in Windows Embedded Compact. KITL makes the connection between host and target device and transports debugging information.
To enable it, here are the main steps:
  • Select the DEBUG configuration in your BSP project
  • Open project properties from Solution Explorer. From the left pane, expand Configuration Properties node and select Build Options.
  • Enable KITL and Enable Profiling.
  • Ensure that ship build is not checked.
  • Disable "CPGMAC Ethernet" driver from catalog (incompatible with Kitl)
  • Save and rebuild the image.
Once built, copy the new NK.bin and EBOOTSD.nb0 to the target's SD card (or wherever you boot from) and boot.
At startup, hit space to enter EBOOT menu when asked to (from the serial debug line UART0):
BOOTLOADER: b# total sectors:1DD77C1
BOOTLOADER: b# sec/fat:      3B9F
BOOTLOADER: b# hidden sec:   1
BOOTLOADER: FileIoInit() initializing data structures.
BOOTLOADER: FileIoReadNextSectors EOF
BOOTLOADER: FileIoReadNextSectors EOF
BLReadBootCfg returns error
WARN: Boot config wasn't found, using defaults
INFO:LCD display configured
INFO:LCD resolution 800x480
INFO:Boot setting: 0x18
INFO:Boot setting: Using SDCard
INFO:PRM_RSTST: 0x00000020
7255c288 d6a2 -> 88 c2 55 72 a2 d6
7255c288 d8a2 -> 88 c2 55 72 a2 d8
Hit space to enter configuration menu [468] 5...
Hit space to enter configuration menu [1468] 4...
Hit space to enter configuration menu [2468] 3...

--------------------------------------------------------------------------------
 Main Menu
--------------------------------------------------------------------------------
 [1] Show Current Settings
 [2] Select Boot Device
 [3] Select KITL (Debug) Device
 [4] Network Settings
 [5] SDCard Settings
 [6] Set Device ID
 [7] Save Settings
 [9] Enable/Disable OAL Retail Messages
 [a] Select Display Resolution
 [b] Select OPP Mode
 [c] Enable clean registry boot
 [0] Exit and Continue

 Selection:
Select menu number 2 and then select "Internal EMAC". Finally select 0 to exit and continue:
--------------------------------------------------------------------------------
 Main Menu
--------------------------------------------------------------------------------
 [1] Show Current Settings
 [2] Select Boot Device
 [3] Select KITL (Debug) Device
 [4] Network Settings
 [5] SDCard Settings
 [6] Set Device ID
 [7] Save Settings
 [9] Enable/Disable OAL Retail Messages
 [a] Select Display Resolution
 [b] Select OPP Mode
 [c] Enable clean registry boot
 [0] Exit and Continue

 Selection: 2

--------------------------------------------------------------------------------
 Select Boot Device
--------------------------------------------------------------------------------
 [1] Internal EMAC
 [2] NK from SDCard FILE
 [3] NK from eMMC FILE
 [0] Exit and Continue

 Selection (actual NK from SDCard FILE ): 1
 Boot device set to Internal EMAC

--------------------------------------------------------------------------------
 Main Menu
--------------------------------------------------------------------------------
 [1] Show Current Settings
 [2] Select Boot Device
 [3] Select KITL (Debug) Device
 [4] Network Settings
 [5] SDCard Settings
 [6] Set Device ID
 [7] Save Settings
 [9] Enable/Disable OAL Retail Messages
 [a] Select Display Resolution
 [b] Select OPP Mode
 [c] Enable clean registry boot
 [0] Exit and Continue

 Selection: 0

You should see the boot process going on and end with several BOOTME message broadcasts:
+Cpsw3gInit(0x4a100000, 0x00000001, 0x8feffedc) v0.3
g_cpsw3g_reg_base = 0xb0500000
Phy_init: Auto negotitation completed
Cpsw3gInit, wait link up on mac port:1.
link up on port 1, speed 100, full duplex
-Cpsw3gInit
INFO: Boot device uses MAC 88:c2:55:72:a2:d6
INFO: *** Device Name LNI33X-41686 ***
InitDHCP():: Calling ProcessDHCP()
ProcessDHCP()::DHCP_INIT
Got Response from DHCP server, IP address: 192.168.200.138

ProcessDHCP()::DHCP IP Address Resolved as 192.168.200.138, netmask: 255.255.255.0
Lease time: 3600 seconds
Got Response from DHCP server, IP address: 192.168.200.138
No ARP response in 2 seconds, assuming ownership of 192.168.200.138
+EbootSendBootmeAndWaitForTftp
Sent BOOTME to 255.255.255.255
!CheckUDP: Not UDP (proto = 0x6)
Sent BOOTME to 255.255.255.255
Sent BOOTME to 255.255.255.255
The target is now looking for a host to start debugging. From Visual Studio, use the "Attach device" button to detect remote targets:

A open shall show up and detect the remote device automatically.
[First time only] Double click on the target device in the popup window.

Visual Studio should now start to download the compiled binary on the target device. Once done, the debugging will start.
Read More

WCE7 : Stamp license key


From Visual Studio

After you building a Release version of NK.Bin, the license key can be "stamped" using Platform Builder :

Here are the steps:
  1. Make sure that you have selected a Release configuration of your OS design to build a run-time image from. (WINCESHIP option shall be set to 1 in project properties. It defines the compile time flag -DSHIP_BUILD)
  2. From the Build menu, verify that Make Run-Time Image After Build and Copy Files to Release Directory After Build are selected.
  3. From the Build menu, select Build and Sysgen to build your run-time image.
  4. From the Tools menu, select Platform Builder for CE 7.0, and then select License Run-Time Image to stamp your .bin file with the license key.
  5. In the License Run-Time Image dialog box, in the Run-time key field, enter the run-time key.
  6. Browse to the location of the run-time image file that you want to license, and then click License.

Automatic

Not tested
Create a text file license.txt with the following content :
<license key> 00000000 00000000 00000000 00000000 00000000
here is an example:
OKJB4-098HN-HFR8F-HBVC8-JF789 00000000 00000000 00000000 00000000 00000000
The trailing 5 DWORDs are OEM-specific.
Run the following command in the release directory:
stampbin -1 -i NK.bin license.txt
Verification:
viewbin -t nk.bin >binaryInfo.txt
Look for ROMHDR and check that PID [0-4] contains the license key:
ROMHDR Extensions -----------------------------
    PID[0] = 0x11111111
    PID[1] = 0x22222222
    PID[2] = 0x33333333
    PID[3] = 0x44444444
    PID[4] = 0x00000000
    PID[5] = 0x00000000
    PID[6] = 0x00000000
    PID[7] = 0x00000000
    PID[8] = 0x00000000
    PID[9] = 0x00000000
Read More