How to move from Jira to Azure DevOps

How to move from Jira to Azure DevOps

This article provides details on how one should organize & administrate Azure DevOps projects with a Jira background. This article won't cover how data needs to be exported from one system to the other, there are plenty of articles covering this part on the web.

Definition of Project

The most striking difference between the two tools is what they call a 'Project'. 

In Jira, it is pretty straightforward as the project has the same meaning as the one defined by the business. A project is a business goal that your team has to attain within a limited timeslot. 

Azure DevOps on the other hand is making thing surprisingly complex. The tool merged the notions of Organizations and Projects and the definitions aren't exactly what you would expect. Creating a Project in Azure equals to creating a division in the company according to Microsoft guidelines.

The question is what's the Microsoft's definition of a business project ?

There are several answers.

Microsoft created a very scalable tool where you can map the structure of your enterprise onto the Organization/Projects notions. In a big company, the natural approach will be to manage divisions as Projects whereas in smaller structure, creating a Project for each business project is still a valid (but not recommended) approach. So how do we have to organize ourselves ?

A backlog filter named Area Path

Microsoft introduced a new notion in backlog management : the Area Path. In a few words, the Areas allows you to categorize the backlog elements. 

This is something you can't do in Jira because each project has its own backlog. In Azure, your division has a main backlog root that is then divided into sub-categories where each of them can be a product, a team or ... a project. This makes it even more confusing because we still don't have a precise definition of the Project. However, this is half of the answer, for each project that your division will manage, create an area.

How do we consolidate the backlogs ?

Making all work items visible from a single place is something that wasn't possible with Jira out-of-the-box. Each project has its own backlog, its own team and its own scrum board. That's one of the rare drawbacks that I experienced when using Jira.

In Azure it becomes possible thanks to the hierarchy of the area paths. In the project settings, make the root area path 'include sub-areas' and boom, the division's backlog now makes all work items visible and displayable in one scrum board. Fantastic ? Hum... not really.

Although it can be helpful for a product owner to have a global overview, you easily find yourself flooded with work items that have no meaning out of their context. Let's assume you have an epic named 'Remote control' in 2 of your area paths. Flattening your backlog will display 2 epics with the same name at the same level which goes against clarity. Even worst, you have absolutely no idea of what the user stories are about when you look at them, mixed with stories related to other projects.

Fortunately, Azure includes backlog filters that are helpful to filter by area path and show relevant information only. Two remarks about this:

• it comes back to having 2 distinct projects
• it is probably a bug but using the filters removes the hierarchichal view of the work items which is a pity

What about teams ?

In Jira, again, the notion of team is straightforward. You add all the stakeholders in the settings, assign the roles and you're good to go.

In Azure, again, a Team is not what you think it is. First of all, keep in mind that creating a team will automatically create:

• An area path (you can uncheck this one)
• A dashborad
• A backlog (and hence a scrum board)
• Velocity/Burndown statistics

Each Team has the possibility of defining their own iteration path (e.g. length and start/stop dates of sprints).

Now, you have 2 choices:

• Either this is something that is interesting for your division's structure where you have several teams working on several projects
• OR you have 1 multidisciplinary team working on several projects and you're not intereseted in having extra dashboards, iteration paths,...

So far, I have been working with one single Team in my division but we are already seeing the limitations:

• Navigation in the backlog is really not easy : the filtering by Area Path is working well but you have to be always careful with creation of new work items and ensure that they are categorized in the right area path
• When filtered, the work items order cannot be modified
• If you're looking for the closed items with the filters on, you won't find them
• The overview shown in Plan is completely pointless and you don't have a clear vision of your project's roadmaps

A compromise would be to create additional Teams without a dedicated Area Path and to map them onto the existing Area Paths of the backlog. I really think that it becomes heavy but it might enhance the overall experience with Azure.

EDIT 20/12/2019: We finally created one team per project and we've been using it for a while now. It really makes the navigation easier which helps a lot for backlog refinements. Another good point is that the 'Plan' overview becomes useful because it shows the roadmap for each 'Team' (a.k.a 'Project'). So to conclude, Team is the second half of the definition of a 'Project' for Microsoft Azure

Mapping of the work items

Jira allows the creation of Epics for groups of features or long-term goals and user stories for the tasks to be executed going forward. Each story can be divided into sub-tasks and the completions of all of them trigs the completion of the user story. 

Azure introduced an additional levels to this scheme.

Microsoft's guidelines to map your tasks into Azure are summarized in the following table:

Azure work item	Completion time
Epic	Months
Features	Weeks
User story	Days
Task	Hours

Still confusing, especially when we work in small iterations without a clear long-term roadmap. The time ranges that Microsoft provides is perfectly valid but it tends developers to think with durations and not efforts. That's probably why they've drawn a separation line between Features and User Stories to let management figure out the roadmap and development team focus on the efforts which is the ideal (but unrealistic) approach.

To frame this a little better, here is a more complete table for the mapping:

Azure work item	Alias	Azure work item	Examples
Area Path	Project	Months/Years	Coffee maker robot
Epic	Goal / Initiative / Final Outcome	Months	Deplacements / Grinding / ...
Features	Step / Module / User-facing function	Weeks	One-click coffee preparation
User story	Action / Testable requirement	Days	As a user, I can stop the robot from the mobile application so that I keep control on the coffee preparation cycles
Task	Sub-Action	Hours	Find a nice-looking flat icon for the button

With that said, I see room for improvement in Azure:

• You can start working on Tasks before moving a story to 'Active'
• Completing the sub-tasks do not trig the completion of the user story
• Same goes to user stories and features

Note : It can be confusing at first but Azure proposes a new type of board to have an overview of the on-going activities within a sprint, the Taskboard. I really like the idea because visualizing the sub-tasks in Jira wasn't convenient

Read More

C# Asynchronous programming

In this article, we'll focus on C# async/await keywords and explain what they're intended for.

When ?

Asynchronous programming has been mainly thought to avoid blocking a thread because of an I/O operation (serial port read, http request, database access ...). It can also be used to handle CPU-bound operations like expensive calculations.

How ?

C# has built-in syntactic sugar keywords (async/await) for easily writing asynchronous code without dealing with callbacks and helps making asynchronous calls on existing synchronous interfaces/APIs (although it is really not the recommended approach). It is known as Task-based Asynchronous Pattern (TAP).

This entire mechanism relies on Task<T> object.

Task vs Thread

A Thread is a worker. It is an OS object which executes a job (e.g. some code) in parallel. 
A Task is a job that needs to be scheduled and executed on available workers, eventually in a ThreadPool. They are a promise for execution.
A ThreadPool is a group of Threads that .NET will handle for you. They are system-shared workers that your application can rely on to execute jobs asynchronously.

With an embedded SW engineering background, it is very tempting for me to instantiate my own Thread objects in my application. It gives me confidence on how my application's jobs are scheduled over time. But generally speaking, it's a mistake.

Threads are expensive objects in terms of memory (1MB / thread in .NET) but also in terms of performances. On a resource-limited system, having several threads per application will exhaust the CPU and slow your system down. .NET will manage the ThreadPool for you, keep threads alive for reuse and take your system's limitations in consideration when doing so. In short, with .NET, prefer using Task.Run for multithreading.

Note: The only situation I came accross that required the creation of my own Thread was when developping a Windows service. When operating system exits, and your background service has pending Task objects in the background, it won't be stopped. This is because the ThreadPool cannot be released.

Definitions

Before showing some examples, we need to understand the meaning of the keywords:

async 

Method declarator which allows the usage of await within method's body.

public async static void RequestDataOverHttp()
    {
        await RequestDataAsync();
    }

Keep in mind that declaring a method as async does not make it asynchronous. If await was removed from the code above, the method would execute synchronously despite async declarator.

await

Start execution of a method and yield.

 public async static Task RequestDataOverHttp()
    {
        await RequestDataAsync();

        // The code below won't be executed until RequestDataAsync returns
        Console.WriteLine("Data received"); 
    }

This keyword creates a state engine in the background to handle job completion. This is what's going to happen:

1. Module A calls RequestDataOverHttp
2. RequestDataOverHttp schedules execution of RequestDataAsync on the same thread. Here, await captures the SynchronizatioContext before awaiting
3. await yields and A continue its processing
4. RequestDataAsync completes and unlocks internal state engine. .NET looks for an available Thread in ThreadPool to resume RequestDataOverHttp. That thread picks up the SynchronizationContext of the original thread.
5. Console finally shows "Data Received"

The most complicated aspect of this mechanism is in understanding how the processing can continue on same thread when hitting an await statement.That is possible thanks to SynchronizationContext and TaskScheduler objects.

SynchronizationContext & TaskScheduler

SynchronizationContext

It is a representation of the environment in which a job is executed. Concretely, this object contains a worker which is usually a thread but can also be a group of threads (ThreadPool), a network instance, a CPU core...

This is what allows a code to be executed on another Thread. For instance, in WPF & Forms, the edition of controls is only possible from UI Thread. By calling control.BeginInvoke from a regular thread, we're placing a delegate to be executed onto the UI Thread.

Under the hood, delegates are queued with Post() or Send() into the context. That's basically what a context does, it's a sort of queue of work for a Thread.

TaskScheduler

We've seen that calling control.BeginInvoke will queue a delegate for UI Thread, which means that it schedules work. This method is part of ISynchronizeInvoke which is part of Control object.

When creating a Task, the scheduling behavior depends on the situation we're in:

On Task creation, the work will first try to be scheduled into the SynchronizationContext of the current thread.
As all threads do not necessarily have a SynchronizationContext, TaskScheduler will schedule the work using the ThreadPool as default choice.
If the Task has been created into another Task, the context of the primary Task will be reused (this is configurable).

Here is a more detailed summary of situations:

Calling thread	Has SynchronizationContext ?	Behavior
Console application	No	Default TaskScheduler used (ThreadPool)
Custom thread	No	Default TaskScheduler used (ThreadPool)
ThreadPool	Yes	All Tasks executed on ThreadPool
UI Thread	Yes	Tasks queued on UI Thread
.NET Core web application	No	All Tasks executed on ThreadPool
ASP.NET web application	Yes	Each request has its own thread. Tasks are scheduled on these threads.
Library code	Unknown	Unexpected behavior, potential deadlock

Task.ConfigureAwait(bool continueOnCapturedContext)

The default behavior of await can be overriden by calling ConfigureAwait(false):

 public async void ReadStringAsync()
    {
    await httpResponse.Content.ReadAsStringAsync().ConfigureAwait(false);
    }

With this call, we indicate that the Task does not have to be executed in caller's context which means that it will be scheduled on the ThreadPool.
When to do that ? If caller is UI Thread and the method does not update the UI elements, doing so is actually better in terms of performances as it will be executed in parallel. Also, it prevents from deadlocks if caller was doing something like ReadStringAsync().Result (see good practices below) which is also why it is a good practice to call ConfigureAwait(false) in library code.

 Usage

Case 1 : I/O bound code

The application awaits an operation which returns a Task<T> inside of an async method.

Synchronous version of an I/O bound method

public string RequestVersion()
    {
        string response = String.Empty;
    
        // Send request
        client.Send(new GetVersionFrame());
        // Wait response
        return client.WaitResponse();
    }

Asynchronous version it

public async Task<string> RequestVersionAsync()
    {
        string response = String.Empty;
    
        // Send request
        await client.SendAsync(new GetVersionFrame());
        // Wait response
        return await client.WaitResponseAsync().ConfigureAwait(false);
    }

Case 2 : CPU bound code

The application awaits an operation which is started on a background thread with the Task.Run method inside an async method.

Synchronous version of a CPU bound method

public List<double> ComputeCoefficients()
    {
        List<double> coefficients = new List<double>();
    
        coefficients.Add(ComputeA());
        coefficients.Add(ComputeB());
        coefficients.Add(ComputeC());
        return coefficients;
    }

Asynchronous version it

public async Task<List<double>> ComputeCoefficientsAsync()
    {
        List<double> coefficients = new List<double>();
    
        coefficients.Add(await Task.Run(() => ComputeA()));
        coefficients.Add(await Task.Run(() => ComputeB()));
        coefficients.Add(await Task.Run(() => ComputeC()));
        return coefficients;
    }

Good practices

Naming

Name asynchronous methods with Async suffix to indicate that the call won't block the caller's thread.

public async void FooAsync()
    {
        await client.DownloadAsync();
    }

Async indicates that the method will offload part of the work to an underlying API (ex: OS networking API).

CPU-bound work

Consider using background threads via Parallel.ForEach or Task.Run for CPU-bound work instead of await unless you're working in a library where you can't do that (see below).

Don't block in async code

1. Bad code

public void Foo()
    {
        client.DownloadAsync().Result;
    }

or 2. Very Bad code

public void Foo()
    {
        Task.Run(() => client.DownloadAsync().Result).Result;
    }

At some point, the async method will be executed/resumed on ThreadPool but if there is no available threads, you'll end with a deadlock. If the example 1 is called from the UI Thread, the task is queued for the UI thread which gets blocked when it reaches Result call --> deadlock.

As asynchronous code relies on execution context, don't block an asynchronous method unless you own the calling thread or if it's the application's main thread. As a general rule : call sync code from sync code and async code from async code, try to not mix them. The application's top layer has control over the context, it can chose whether to use sync or async code.

Note: using Task.Run to delegate some tasks to a ThreadPool while keeping the UI responsive is generally okay

No Task.Run in a library

This rule is related to the previous one. Callers should be the ones to call Task.Run because they have control on the execution context. Functionnally, Task.Run will work but also introduce performance issue because of an additional thread switch.
Additionnally, if a library needs to support both sync and async methods, there should be no relation between them. We can't use async calls in sync code, or we might run into deadlock issues.

Do not use async void

public async void FooAsync()
    {
        await client.DownloadAsync();
    }

As there is no Task object to be returned, exceptions cannot be captured and will be posted in the SynchronizationContext (UI Thread for example).
Also, the caller is unable to know when the execution has finished, it's a "fire and forget" mechanism.

Instead, use

public async Task FooAsync()
    {
        await client.DownloadAsync();
    }

Read More

Leading a software project from A to Z

Although building a project from ground up is a very challenging task that requires solid nerves, patience, attention to details, good communication skills and mostly motivation, it is also the most interesting work that you will do in your career. You will live the full adventure, not just a part of it where you're usually asked to do something with little to no freedom at all. This time, you are setting up the rules, you are picking the tools and the technologies and you are designing YOUR solution.

Here are some rules that you better follow if you don't want to turn your project into a nightmare.

1. Do a pre-study

The initial requirements are usually very general, giving an overview of what we want but not how to make it. Even if I tell you to develop the exact same watch as the Fitbit Ionic, you can't just rush into the development. You will have to understand the technology, analyze the materials, define how the product shall interact with user, computers, etc...

Build a solid pre-sudy document where every aspect of the product is described, eventually listing different potential solutions with pros and cons. This will naturally lead you to your next step : the product architecture specification.

2. Take all the time you need to write good specifications

Spoiler alert: if you were to develop your product alone, this step is where you would spend one third of your time.

Business managers will always push you to provide the product asap, never caring about QUALITY. To be honest, they do care but differently:

• For developers, quality is all about architecture, code metrics, tests

• For business managers, quality is about functional aspects. As long as it works, the quality level is high

• For QA, quality means following the process. Are all the documents ready and signed ? If yes, quality is alright

What about clients ?

Clients will always want the best product for their money. Their quality perception is based on usability, performances, materials, robustness and ... support services !
If a company cannot sustain its own products, it will dig its own grave because of its high operational cost.

How to prevent this ?

It all starts with specifications. Developers complain about documents that we write but never read and there is a little bit of truth but the exercice worth it because it will highlight many aspects that you'd have ignored in your design (error cases, serviceability, deployability, update startegy, ...).

Where do I start ?

My advice is that you first create diagrams to visualize the features. If your project is not 100% software, then create an SADT diagram with interactions between functions (or go for sysML if you have time). Doing this will help you to distribute the functions between hardware and software.

Now, on software side, think about a design concept and try to put it in a document where you'll elaborate the following points:

* Use case view: Think about how the users will access/use your product and put it in a simple UML Use Case diagram

* Development view: define the modules that will compose your software and use a diagram to show how they'll interact with simple arrows. For instance, this is where I usually put my layered architecture overview

* Logical view: Use this part to define your objects/interfaces. Do not go into details ! At this stage, we just want to understand the intent of each module. I usually insert class diagrams with my interfaces in here.

* Process view: Up to here, the reader only had a static overview of your design. Now you tell him how the modules that you described will behave at runtime (processes, threads, message queues, timers, ...). If the product has performance requirement, explain how they'll be met here.

* Components/package view: A package is a deployable unit. How will you organize your modules/components into these packages ?

Once done, you can move forward with the Software Requirements Specifications (usually called SRS). I strongly recommend you to google 'IEEE SRS' and to get inspired by the IEEE template. It gives very good indications on how your specifications should be written.

In a nutshell:

• Each requirement shall be testable: do not use vague words like 'hot', 'cold', 'fast enough' ... Give concrete values that a tester can validate.

• Each requirement shall be uniquely identified: use IDs for all of your requirements. They will be used for traceability in the entire V cycle.

• Split the specifications following the component structure that you defined in the design concept. For each component, define the inputs, the processing and the outputs

• For views, insert mockups and specify the actions of each control

3. Pick up your gears

Some technology choices might already be defined at this stage (they influence your design concept). If not, select the frameworks, development environments, source code manager, issue tracker and all other tools that will be necessary to develop/build/deploy your project.
This information usually goes into a 'quality plan' document. This is also where we usually define the branching model, coding rules, planning overview... DO THIS. Especially if you're a team of developers. You'll notice inconsistencies in code, in source code manager and/or in issue tracker if you don't specify the rules somewhere.

4. Organize the work

The components you defined have natural dependencies with each others. Focus on the ones that have less dependencies first.

Tasks
For each components, create a story/task in your issue tracking tool. This task can be broken down into smaller tasks by the developer when taken care of. Evaluate the difficulty of developments for each task and rate them properly either by duration or with story points (recommended).
If you're lucky to work with tools like Jira, create epics for the features that you'd like to implement and organize your tasks. At the very beginning, you can create an epic for the first prototype.

Milestones
Your business manager expects only one delivery but creating intermediate releases along the way is very helpful for demos, regression investigation, test of deployments/updates...
Define the milestones (v1.0.0, v1.1.0, v1.2.0...) and detail what's going to be included in each of them. Be realistic with delivery dates and do not try to anticipate too much. See, when managing a project, people tend to make promises and set unrealistic release dates. In the end, almost all the projects I've seen have been delayed and the funny fact is that it wasn't even due to the development. Clients can make new requests or change existing ones, business can down-prioritize your project, one of the technology used can be not supported anymore, a key developer can resign ... Unexpected things will happen for sure, be prepared.

Ideally, adopt agile methodology and work in sprints.

5. Plan -> Execute -> Check -> Act -> Plan ...

Note: If you're working agile, you're already covering this.

Always re-assess your software requirements depending on business needs. Ideally, bring the client into the loop and ensure that he is aligned with your plan and satisfied with your latest features. If not, plan for the changes and execute.

Also, even if it seems obvious to me, unit tests are not optional. I really encourage you to track your metrics during this phase and to do the efforts to keep them in the green zone.

If you're coordinating other developers, make sure they always understand what they should do. If necessary, take time to explain them your design over and over again. Don't be afraid of repeating things, it can be annoying but you have to be tolerant with your manpower as much as possible.

6. Release often

If your milestone is supposed to end in 3 months, do not wait 3 months to release.
Releasing means bringing all the features together, versioning and deploying. Do it at least at the end of each sprint (if you're not working agile, plan weekly builds). It will reduce the integration penalty and, ideally, will allow you to anticipate on functional/user acceptance tests.

For the versioning, we usually use 3 digits to do this:

• Major: Only incremented when major changes are made to the code with an impact for the end-user.

• Minor: Only incremented when minor changes are made to the code without really changing the features but rather extending them slightly. "Look and feel" changes are also usually considered to be minor changes if they do not impact the UX (e.g. usability) heavily.

• Micro/Build: Incremented for code rework, minor bug fixes, improvement of test coverage, ...

It's not a rule though, you're free to use your own versioning as long as it is consistent over time and that it helps you to identify your software easily.

7. Validation

Most neglected aspect that nevertheless makes a total difference if executed properly, the validation is in my eyes a must-do. Ideally, try to think about validation when writing functional requirements because each of the requirement will need to be testable. If a requirement is too vague to be tested, it can't be implemented, just as simple as that. As a V&V engineer, I should be able to write my test plan based on the functional requirements without any doubt on what needs to be tested.

To illustrate this, here are some examples:

Req 1.1: As a driver, when I press the brakes pedal, my car should stop.

NOT TESTABLE: I see at least 2 reasons :

• this requirement is not time-bound: if my test proves that the car stops after 30 minutes, the system is valid. 

• 'should' to be replaced by 'shall'. We want no doubt about what the system shall do.

Req 1.1: As a driver, when I press the brakes pedal, my car shall stop within x seconds with x given with the following relation: x = (speed/10)²

TESTABLE

8. Conclusion

This article gives you an overview of what you'll have to do if you drive a project and lots of other aspects have not been detailed such as reporting, interfacing with other stakeholders, CI/CD ... It can be scary and you'll make mistakes (all of us do) but as long as you try to setup a framework for you and your team and keep executing as defined, you'll be right enough. As a developer, I like to have consistency in my code. Well, as I project leader, I like to have consistency in my project & team.

Read More