WCE7 : Add chinese characters support

This article explains how chinese characters can be added to WinCE7 OSDesign. Without this feature, the views would display empty square shapes instead of chinese text, meaning that the decoding of input strings failed.

Project properties

Right-click on your OSDesign project and select Locale in the left panel.

On the right side, fill all parameters with languages that you want to include in the image and save.

The NLS Table

The NLS table is a list of locale identifiers that is necessary to .NET Compact framework. The local identifiers of the languages that you want to include in your image must be listed in this table.

To do this, copy the file named nlscfg.inf from %_WINCEROOT%\public\COMMON\oak\files and paste it to %_WINCEROOT%\PLATFORM\<yourBSP>\FILES. Edit this file and overwrite its content with the following:

007f 0403 0406 0407 0807 0c07 1007 1407 0409 0809 0c09 1009 1409 1809 1c09 2009
082C 042C 0423 0402 0845 0445 0405 0465 0408 540A 0425 0429 0464 0447 040D 0439
101A 041A 040E 042B 0411 0437 043F 044B 0412 0457 0440 0427 0426 042F 0450 044E 
0446 0415 0418 0419 044F 041B 0424 041C 1C1A 0C1A 281A 181A 081A 045A 0449 044A 
041E 041F 0444 0422 0420 0843 0443 042A 0804 0C04 1404 1004 0404 2409 2809 2c09 
3009 3409 040a 080a 0c0a 100a 140a 180a 1c0a 200a 240a 280a 2c0a 300a 340a 380a 
3c0a 400a 440a 480a 4c0a 500a 040b 040c 080c 0c0c 100c 140c 180c 040f 0410 0810 
0413 0813 0414 0814 0416 0816 041d 081d 0421 042d 0436 0438 043e 083e 0441 0456
045e 3801 3C01 1401 0C01 0801 2C01 3401 3001 1001 1801 2001 4001 0401 2801 1C01
2401
LOC_INCLUDELOCALES

This table lists all identifiers supported by WCE7. As Erwin pointed out in his blog, some locale identifiers are not supported and will generate exceptions if used in your application.

Pick the right items from Catalog

Look for the following sysgen variables and ensure that they are checked :

SYSGEN_DOTNETV35_SR_CHS
SYSGEN_DOTNETV35_SR_CHT
SYSGEN_UNISCRIBE
SYSGEN_ FONTS_MSYH
SYSGEN_ FONTS_MSYHBD
SYSGEN_ FONTS_MSJH
SYSGEN_ FONTS_MSJHBD
SYSGEN_NLS_ZH
SYSGEN_NLS_ZH_* (all chinese locales)

Now, ensure that the following variables are NOT checked:

SYSGEN_GB18030
SYSGEN_AGFA_FONT

Finally rebuild the entire image. You should see a big difference in the image size (mine went from 37M to 111M).

Read More

Application logs

Logging is a fundamental mechanism for the application. It brings valuable information to support teams when something goes wrong and saves a lot of time in bug investigation. But this utility can easily turn into a "spam engine" if the developers do not carefully select what/when/how much to log.

What do we expect from logs

A support team will look for relevant events or user actions that can explain a troublesome situation. Here is an example:

09:50:05 [INFO] [AppOperation] Mix configured with CMP=Nitrogen, CST=SynthAir, C=245,8 ppm
09:50:05 [INFO] [Solver] Computed flows for (C=2445,8 pp) : MFC1=1200 mlmin, MFC2=2450 mlmin
09:50:06 [INFO] [AppCalibration] Compensation of MFC1=1200 mlmin to MFC1=1202 mlmin
09:50:06 [INFO] [AppCalibration] Compensation of MFC2=2450 mlmin to MFC1=2449 mlmin
09:50:06 [INFO] [Peripherals] Setting MFC1=1200 mlmin
09:50:06 [INFO] [Peripherals] Setting MFC2=2449 mlmin
09:50:06 [INFO] [Peripherals] Setting Hardware state to Mix2Levels
09:50:06 [INFO] [AppOperation] Mix started
09:52:08 [ERROR] [AppAlarm] (Stability Alarm ON) MFC1 flow is not stable (Measured=1015 mlmin)

In this scenario, the user complains about poor results on his device. We suppose that the Alarm was not signaled on a visual indicator to inform the user. With this bunch of logs, developers can immediately:

• Point the origin of the issue 
• Get the conditions of the operation that lead to the issue and try to run the same experiment at his office

Obviously, none of these actions will solve the problem but the information that logs gave to the developer will highlight which functional part went wrong, and help him to follow the right track to resolve the client issue.
 It is essential for developers to have the following information in logs:

• Timestamp 
• Log level : developers will filter the errors or warnings first to find an obvious reason for the malfunction 
• Synthetic information : Long phrases are useless: a log must go the point, there is no litterature trophy for well-written logs. Most of the time, use action verbs and parameters. 
• Workable data : the parameters listed in our logs specify numerical values with a unit. It seems obvious to say that without the unit, the developer cannot investigate. Plus, the location where the error occured is necessary. In our example, the name of classes are displayed. Either do this or use functional area names logged (ex: Config, Security, Alarms, ...). 

Other things to have in mind:

• Keep files small : do not go over a few MB for text files (I do not include DB here). Opening and parsing a heavy file is long and inefficient. Use several small rolling files instead. 
• Keep short history : there is no need to keep traces from last week if nothing happened there. This is closely related to file size for text logs.

What to log

• Application events: notifications, alarms, state changes, ... 
• Operation-related information : settings update, user actions, ...

What NOT to log

• Periodic data : if live values need to be logged regularly, put them in a separate log file and preferably go for a DB that can handle large amount of entries better. Developers can then refer to timestamps for investigations. 
• GUI input errors : if a user enters invalid value, it is not an error for the application, it is an error for the form. As long as the frontend does not interact with the application, whatever happens between the user and the form does not have an impact on the application.

Where to log

A software architecture is built with modules interacting with each others. The business logic of the application is dispatched between these modules. So basically, each functional area of the application shall manage its own logs. To do things neatly, centralize the logging features in a service shared by all components.
Do not repeat information. Take the following example: A calls B, B fails. This is the type of log that we should expect from it:

10:00:01 [INFO] [A] Action A1 started
10:00:01 [INFO] [B] Setting xxx=1234
10:00:01 [ERROR] [A] Action A1 failed (Code=Invalid Setting Error)

and not something like :

10:00:01 [INFO] [A] Action A1 started
10:00:01 [INFO] [B] Setting xxx=1234
10:00:01 [ERROR] [B] Given value is invalid
10:00:01 [ERROR] [A] Action A1 failed (Code=Invalid Setting Error)

The error occured at a known place, so either log it from there or from the client (recommended).

Debugging information

Take the following example of code:

void ExecuteUserRequest()
{
   env = GetEnvironmentValue();
   if(env < THRESHOLD)
   {
      doA();
   }
   else
   {
      doB();
   }
}

Here, myFunc behaves depending on an external factor. Here, we can chose to log the action (doA or doB) with an INFO level and add debugging information to the user to help him understand why this precise action occured :

void ExecuteUserRequest()
{
   env = GetEnvironmentValue();
   LogDebug("env = %d", env);
   if(env < THRESHOLD)
   {
      LogInfo("Executing A");
      doA();
   }
   else if(env >= THRESHOLD)
   {
      LogInfo("Executing B");
      doB();
   }
}

Generally speaking, Debug level gives execution details and exposes implementation data to the developer.

Memo

Here are some basic rules that I use to write my application logs.

Code type	Log level
user input error (GUI)	No log.
Frequently accessed functions (ex: sensor value read)	Debug or no log if we log in a text file. Use a DB if possible.
Implementation detail (subroutines, intermediate values,...)	Debug
User action or application event	Info
Alarm or abnormal situation that does not result in an error immediately	Warn
Operation failure on user action	Warn if the error is normal given the context (the product acts as specified). Error otherwise.
Application error (internal non-critical errors or errors managed by the application)	Error
Critical non recoverable error (unmanaged exception, errors that lead to a non-functioning application)	Fatal

Read More

Build U-Boot for BeagleCore

BeagleCore is a variant of the BeagleBone Black. It consists of 2 modules : BCM1 and BCS2:
* BCM1 is the System-On-Module that contains most of the BeagleBone Black design
* BCS2 is a mainboard with the form factor of the original BeagleBone Black. The BCM1 is soldered on BCS2 and the pair is an equivalence of the BBB.

Recently, I needed to run U-Boot on a custom board built with a BCM1. I followed some tutorials on the internet and the result was that U-Boot never started. After some investigations, I found out that the EEPROM that contains BBB identification is not present on BCM1, but is on BCS2 instead. The consequence is that U-Boot cannot identify the board and the initialization procedure fails.
In this post, I will detail every step to make U-Boot run on BCM1.

Note: All of these steps were performed on Windows 10 with bash (ubuntu terminal).

Install tools

sudo apt-get install gcc-arm-linux-gnueab

Create an alias to simply following commands

alias armmake='make -j8 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- '

Grab U-Boot

git clone
git://git.denx.de/u-boot.git u-boot
cd u-boot

Edit the code

Usually, we do a patch for these type of things. I'll try to post it later.
We will modify some parts of the code to stub the EEPROM so that U-Boot behaves exactly like it was a BBB board.

Open board_detect.c and add the following right after the #include statements:

static struct ti_common_eeprom stub_eeprom = 
{
    TI_EEPROM_HEADER_MAGIC,
    "A335BNLT",
    "00C0",
    "1716BBBK2450",
    "",
    {{0x11,0x22,0x33,0x44,0x55}, // Dummy Mac addresses
     {0x11,0x22,0x33,0x44,0x55},
     {0x11,0x22,0x33,0x44,0x55}},
    11749353662462671858,
    12656917672993063804
};

Replace the content of the function 'eeprom_am_set' with

{
    return 0;
}

Do the same with 'eeprom_am_get'.

Open board_detect.h.

Replace TI_EEPROM_DATA definition with:

 #define TI_EEPROM_DATA ((struct ti_common_eeprom *)\
                &stub_eeprom)

Config & Compile

armmake  distclean
armmake am335x_boneblack_defconfig
armmake

Copy to an SD Card

Format an SD card to FAT32 (boot flag ON).
Go to your U-Boot directory and copy MLO to SD Card first.
Then copy u-boot.img to SD-Card.

And there you go ! U-Boot shall start normally.
Here is an example of my uEnv.txt :

bootpart=0:1
bootdir=
bootfile=uImage
fdtfile=am335x-boneblack.dtb
fdtaddr=0x80F80000
loadaddr=0x80200000
optargs=quiet
mmcdev=0
mmcroot=/dev/mmcblk0p2 ro
mmcrootfstype=ext4 rootwait
loadbootenv=load mmc ${mmcdev} ${loadaddr} ${bootenv}
loadfdt=load mmc ${bootpart} ${fdtaddr} ${bootdir}/${fdtfile}
uenvcmd=load mmc 0 ${loadaddr} ${bootfile};run loadfdt;setenv bootargs console=${console} ${optargs};bootm ${loadaddr} - ${fdtaddr}

Read More

Machine learning basics

There are several types of machine learning but we will focus on the followings in this article:

• Supervised learning
• Unsupervised learning
• Reinforcement learning

Supervised learning

Basically used to make predictions about future data from labeled/categorized training dataset.


A dataset is a table where:

• each row is a sample
• each column is a feature
• each row is labeled with a class label

 Classification

A supervised learning task with discrete class labels is called a classification task.
Classification is a subcategory of supervised learning where the goal is to predict the categorical class labels of new instances based on past observations.

Example:

Dataset with  [data; label]
Data 0 : [ I like sport; Present]
Data 1 : [ I love shopping, Present]
Data 2 : [ I was in Amsterdam, Past]
Data 3 : [ I did something wrong, Past]
Data 4 : [ I do exercise every day, Present]

New data:

I did not know --> Past
I love running --> Present

With 2 possible class labels, the task is a binary classification task.
With more, it is a mutli-class classification task.

Example of multi-class dataset:

[Picture of cat; cat]
[Picture of dog; dog]
[Picture of mouse; mouse]

Here the machine learning system would be able to recognize a dog, a cat or a mouse but wouldn't succeed with any other animal because it is not part of our dataset.

Typical example of two-dimensionnal dataset for a binary classification task:

Data 0 : [  [0;0] ; Orange]
Data 1 : [  [1;1.5] ; Orange]
Data 2 : [  [1;2] ; Orange]
Data 3 : [  [1;2.8] ; Orange]
Data 4 : [  [2;1.5] ; Orange]
Data 5 : [  [2;2.5] ; Orange]
Data 6 : [  [3;0] ; Blue]
Data 7 : [  [3;1.5] ;Blue]
Data 8 : [  [3;2] ; Blue]
Data 9 : [  [4;2.8] ; Blue]
Data 10 : [  [4;1.5] ; Blue]
Data 11 : [  [4;2.5] ; Blue]
Data 12 : [  [5;3] ; Blue]

It is two-dimensionnal because each sample of the dataset has 2 values (usually named x1,x2). If we represent these samples on a 2-dimensionnal graph, we would see this:

The prediction would be based on the distribution of the sample. A point with x1 > 3 would be predicted as Blue and a point with x1 < 2 would potentially be red.

Regression

Regression is also called prediction of continuous outcomes. In regression analysis we give a serie of numbers (x or predictor) and response variables (y or outcome) and we try to find a relationship between them to predict a future outcome.

Ex:

with [x;y]
Data 0 : [ 0 ; 0 ]
Data 1 : [ 1 ; 1.5 ]
Data 2 : [ 1.5 ; 1 ]
Data 3 : [ 2 ; 2 ]
Data 4 : [ 2.5 ; 2.6 ]
Data 5 : [ 3 ; 3.2 ]
Data 5 : [ 4 ; 3.9 ]

Several types of algorithm can be selected to process input data. The following figure illustrates the concept of linear regression:

The computed curve will be used to predict the outcome of new data.

Reinforcement learning

Here the goal is to develop a system (agent) that improves its performances based on interactions with environment. The system will receive a feedback (reward) for every one of its actions. Each reward informs him of the quality of his action.
The agent will learn a series of actions that maximizes this reward via an empirical try-and-error approach.

A typical example is Google's Deepmind which beat the best Go players.

Unsupervised learning

In supervised learning, we include the right answer (labels) into the dataset. Here, we don't know the right answer beforehand. We are dealing with uncategorized data with an unknown structure.
With unsupervised learning, we can explore the structure of our data to extract meaningful information without an outcome or a reward.

Clustering

Clustering is an exploratory data analysis technique which groups data together by similarity (unnsupervised classification).

Dimensionality reduction

Dimensionality reduction is another unsupervised learning field. To prevent against the computation of huge amounts of data which results in performance and storage issues, unsupervised dimensionality reduction preprocesses data to remove noise and retain relevant information.

Read More

Django + Js : display counting timedelta in view

Suppose you have an entry in DB with a timestamp 'start_date'. You want to display the time elapsed since 'start_date' in your view and you want that delay to grow in real time like a clock. In the following example, the user can select one entry at a time and the counter has to be updated accordingly.

Note: To achieve this it is important to work with UTC timestamps in both DB and your view

Here is what my code in django views.py looks like:

def get_entry_info(request):
    """
    Recover information for the specified entry name
    """
    entry_name = request.POST.get("name", None)
    if request.method == "POST" and entry_name is not None:
        data = {}
        # Get entry object 
        entry = MyEntries.objects.get(name=entry_name)
        data['start_date'] = int(time.mktime(entry.start_date.replace(tzinfo=None).timetuple())) * 1000 
        return JsonResponse(data)
    else:
        return HttpResponse("Invalid entry name")

Here is what my javascript code looks like:

var startDate;

// Loads content into the information panel
// @param data : data to be displayed in the information panel
function reloadInformationPanel(entryname){
    // Get job information from server
    $.ajax({
        headers: { "X-CSRFToken": '{{ csrf_token }}' },
        url: "{% url 'get_entry_info' %}",
        method: 'POST', 
        dataType: 'json',
        data: {
            'name': entryname, // outgoing data
        },
        success: function (data) {        
            startDate = new Date(data.start_date);
            startTime();
        },
        error: function(xhr,errmsg,err) {
            console.log(xhr.status + ": " + xhr.responseText); 
        }
        });
}

// Starts timer of job duration
function startTime() {
   var now = convertDateToUTC(new Date());
   var delay = new Date(now - startDate);
   document.getElementById('entry_duration').innerHTML = delay.getUTCHours() + "h" + delay.getUTCMinutes() + "m" + delay.getUTCSeconds() + "s";
   var t = setTimeout(startTime, 500);
}

// Converts a Date object to UTC
function convertDateToUTC(date) { 
   return new Date(date.getUTCFullYear(), date.getUTCMonth(), date.getUTCDate(), date.getUTCHours(), date.getUTCMinutes(), date.getUTCSeconds()); 
}

Now simply map reloadInformationPanel to the entry selection button ;)

Read More

What is AJAX ?

AJAX = Asynchronous Javascript And Xml

Ajax is not a tehnology by itself but rather a combination of existing technologies (HTML/CSS/DOM/Javascript/XML/JSON).

What is it intended for ?

Ajax is typically used to refresh data on a web page without having to reload the entire page (e.g. asynchronously). It involves a web browser sending HTTP requests (GET/POST)  to server and processing the response to finally manipulate the page DOM (e.g. HTML tags). The user's view is thereby dynamically updated.

As you can see, Ajax requests are executed by Javascript code and the response is also handled in Javascript. The orange part stands on client's side (the web browser).

What if the format of data ?

Originally, it was XML but nowadays, JSON is preferred (JavaScript Object Notation).

Example of ajax requests

jQuery

<script>
...
$.ajax({
        headers: { "X-CSRFToken": getCookie("csrftoken") },
        url: "myurl",
        method: 'POST', // or another (GET), whatever you need
        data: {
            'mydata': 'value', // outgoing data
        },
        
        success: function (data) {        
            // success callback
            // you can process data returned by server here
        }
        });
...
</script> 

Pure javascript

<script type="text/javascript">
function ajaxFunction()
{
var xmlhttp;
if (window.XMLHttpRequest)
  {
  // code for IE7+, Firefox, Chrome, Opera, Safari
  xmlhttp=new XMLHttpRequest();
  }
else if (window.ActiveXObject)
  {
  // code for IE6, IE5
  xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
  }
else
  {
  alert("Your browser does not support XMLHTTP!");
  }
xmlhttp.onreadystatechange=function()
{
if(xmlhttp.readyState==4)
  {
  document.myForm.time.value=xmlhttp.responseText;
  }
}
xmlhttp.open("GET","time.asp",true);
xmlhttp.send(null);
}
</script>

Read More

What is inside a WCE7 BSP

The BSP (Board Support Package) is the layer that will interface the Operating System with the hardware.

It consists of the following items:

• [Optional] Bootloader
• OAL (OEM Abstraction layer)
• KITL to debug the OS in development phase
• Configuration files that specifies the board (Ex: config.bib for memory settings)

The BSP image is specific to the OS version and to the target hardware architecture.

 

Bootloader

When powered up, the microprocessor starts executing instructions from a specified memory address located in ROM. This code can contain the entire system for small footprint applications and will run until shutdown.
With WCE7, the system image (nk.bin) needs to be loaded to RAM from a storage disk or network because it can't fit into a ROM memory. In this case, these are the typical steps of a bootloader:

• Minimal Initialization of the system (memory and com drivers)
• Loads the OS image to RAM
• Jumps to OS start address in RAM and calls startup fuction
• The OAL takes over

Windows CE 7 includes a configurable and reusable bootloader named CE Boot with the following features:

• Can load files from disks (IDE, PCI)
• Emulates a console to allow user interraction at boot
• Network support: DHCP, TFTP, IP

 

XLDR

In some cases, the system can be loaded in two steps : a first bootloader loads a second bootloader to RAM. The second bootloader loads the system image to RAM et jumps to it. This is necessary when:

• The initial startup program is too small
• The startup program on the target system is general purpose
• The CPU has a small amount of RAM to load an image immediately and shall initialize an external RAM to host the system

XLDR is a lightweight loader that can load CE Boot in two stages.

OAL

The OAL allows the kernel to access the hardware layer (interrupts, timers, cache, IOCTL...) through an abstract interface.
When it takes over the boot phase, it accomplishes the following steps:

• Initialize the CPU state, hardware and kernel's global variables
• Jumps to the entry point of the kernel
• The kernel exchanges global pointers with the OAL. From this point on, the kernel has access to all functions and variables defined in OEMGLOBAL and OAL has access to all functions and variables defined in NKGLOBAL
• On kernel call, the OAL initializes the debug serial port (OEMInitSerialDebug) (like the bootloader did before without the help of the OAL)
• On kernel call, the OAL initializes the rest of hardware interfaces on the device (OEMInit())
• [Optional] On kernel call, KITL initialization (OEMKitlInit)
• The kernel starts executing its first thread

 

Configuration files

File	Description
config.bib	Memory structure definition file
platform.bib	BSP global settings. Selects which files need to included in the image depending on user-selected options/items
platform.dat	Not used
platform.db	Not used
platform.reg	BSP global registry settings. Selects which keys to write in the registry depending on user-selected options/items
sources.cmn	Starting point of the BSP build process. On top of source tree. Defines include directories and linked libraries
<MyBSP>.bat	Project-specific environment variables to use in the build process. In particular, specifies the devices to include in the build.

To have a deeper look at the BSP's file structure, check the Build process.

Read More