Create NuGet packages for Managed and Native libraries

Managed libraries

Nuspec file

Before jumping into nuspec file creation, one should know that nuget can generate a package directly from a visual studio project:

nuget pack MyPackage.csproj

If you're willing to enter advanced mode, then you need a nuspec file.

First of all, instead of recreating the wheel, check the official documentation here : https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package

Now, here is a shortcut :
    1. Create a .nuspec file and fill it as follows:

    <?xml version="1.0"?>
    <package >
    <metadata>
        <id>Mypackage</id> <!-- Must be unique to avoid confusion during package resolutions -->
        <version>1.0.0</version> <!-- major.minor.micro[.optional] -->
        <title>My first package</title> <!-- Title of the library as it will appear in nuget manager -->
        <authors>Deadpool</authors> <!-- Author of the component -->
        <owners>Marvel</owners> <!-- Legal owner of the component -->
        <description>
        This is a short description of the component and describes what it is intended for.
        </description>
        <releaseNotes>
        [Optional] If this version has any particularity, it should be listed here
        </releaseNotes>
        <summary>
        Same as description but shorter
        </summary>
        <language>en-US</language>
        <projectUrl>http://mygitserver.com/mypackage</projectUrl>
        <requireLicenseAcceptance>false</requireLicenseAcceptance>
        <licenseUrl>http://opensource.org/licenses/Apache-2.0</licenseUrl> <!-- Or any other -->
        <copyright>Copyright if any</copyright>
        <dependencies> <!-- List any dependency that should be checked out with this package -->
            <group targetFramework="net35"> <!-- check all target frameworks here : https://docs.microsoft.com/en-us/nuget/reference/target-frameworks#supported-frameworks -->
                <dependency id="MyDependency" version="2.0.1" /> <!-- Here we specify to use MyDependency v2.0.1 built for .NET3.5 -->
                <!-- Others listed here -->
            </group>
        </dependencies> 
        <references></references> 
        <tags></tags> <!-- [Optional] any tags that would help your team finding your package here -->
    </metadata>
    <files>
        <!-- This will copy all the files from release build output to a lib subfolder in the package -->
        <!-- All references placed under lib will be automatically added to your project -->
        <!-- lib subfolder is mandatory. Create a folder for each supported frameworks -->
        <file src="bin\Release\**" target="lib\net35" exclude="**.pdb" /> 
        <!-- [Optional] Add documentation to the library -->
        <file src="doc\**" target="doc" />
    </files>
    </package>

Once this is done, creating the package is straightforward:

nuget pack Mypackage.nuspec

Notes that might help :

•  Be careful with dependencies. If your package A depends on a package B 1.0.0.3258 with the latest digit being a build number, ensure that the exact same version of package B is packaged and stored in your nuget server. The build number is re-generated at each build and if it changes, the dependency resolution will fail. Ideally, remove this last digit from version reference.

•  If your managed library relies on a native dll that is platform sepcific, you'll need to add the dlls to your package as follows:

<?xml version="1.0"?>
<package >
  <metadata>
    <id>Phidgets</id>
    <version>2.1.8</version>
    <title>Phidgets</title>
    <authors>Me</authors>
    <owners>Me</owners>
    <description>
      Phidget interfacing library
    </description>
    <releaseNotes></releaseNotes>
    <summary>
      Phidget interfacing library for .NET
    </summary>
    <language>en-US</language>
    <projectUrl>https://mygitserver.com/Phidgets</projectUrl>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <licenseUrl>http://opensource.org/licenses/Apache-2.0</licenseUrl>
    <copyright></copyright>
    <dependencies>
    </dependencies>
    <references></references>
    <tags></tags>
  </metadata>
  <files>
        <file src="lib\net20\**" target="lib\net20" exclude="**.pdb;" />
        <!-- Platform specific libraries are usually placed under a runtime subfolder in the package. -->
        <!-- The second level specifies the platform with an RID identifier (see here : https://docs.microsoft.com/en-us/dotnet/core/rid-catalog) -->
        <!-- Those libraries won't be copied to your output directory automatically. For that, either use a .targets file (see further in this article) or add a build step in your main project -->
        <file src="native\x86\**" target="runtimes\win-x86\native" />
        <file src="native\x64\**" target="runtimes\win-x64\native" />
  </files>
</package>

Native libraries

Native packages can be generated with a third party app called CoApp if you're not running under Windows 10. Unfortunately, this tool is not maintained anymore, and Win10 users will have to do that manually.

The good news is: it's not that hard.

If you understood the steps for a managed package, you've almost understood how to create a native package as well. Here is an example of nuspec file for a native library:

<?xml version="1.0"?>
<package >
  <metadata>
    <id>stringlib</id>
    <version>4.1.0</version>
    <title>formatlib</title>
    <authors>Me</authors>
    <owners>Me</owners>
    <description>
      Provides string formatting functions for C/C++
    </description>
    <releaseNotes></releaseNotes>
    <summary>
      String formatting library for C/C++
    </summary>
    <language>en-US</language>
    <projectUrl>https://mygitserver.com/stringlib</projectUrl>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <licenseUrl>https://opensource.org/licenses/BSD-3-Clause</licenseUrl>
    <copyright></copyright>
    <dependencies>
    </dependencies>
    <references></references>
    <tags>native</tags> <!-- This is important ! -->
  </metadata>
  <files>
      <!-- Note that now, binaries are placed conventionally under a directory named build -->
      <file src="bin\x86\Release\**" target="build\native\libs" exclude="**.pdb"/>
      <file src="include\**" target="build\native\interface" />
      <file src="doc\**" target="doc" />
  </files>
</package>

AFAIK, there is no clear rule on how one should manage the package structure. The structure I use is:

build
|-native
|-|-libs --> binaries go here
|-|-interface --> public headers go here
doc 

If I want to support several platforms, I can do that:

build
|-native
|-|-libs 
|-|-|-win-x86 --> binaries go here
|-|-|-win-x64 --> binaries go here
|-|-|-linux-x86 --> binaries go here
|-|-interface --> public headers go here
doc 

or

build
|-native
|-|-libs 
|-|-|-v100 --> VS2010 binaries go here
|-|-|-v110 --> VS2012 binaries go here
|-|-interface --> public headers go here
doc 

It is up to you to chose how you'd like them stored. Try to be generic (platform->architecture->toolchain).

Once the nuspec file is done, we have to add build rules that will automatically be merged into our solution. To do that, we can create .props file or .targets file or both !
These files will be imported into our project and Visual Studio will execute them during build.

These files need to be placed under 'build' subfolder in your nuget package and shall be named with your package id:

build
|-native
|-|-libs 
stringbuild.props
stringbuild.targets
|-|-interface --> public headers go here
doc 

Copy and paste the following code into your props file :

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" 
    xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemDefinitionGroup>
        <ClCompile>
            <AdditionalIncludeDirectories>$(MSBuildThisFileDirectory)native\interface\;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
        </ClCompile>
    </ItemDefinitionGroup>
    <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
        <ClCompile>
            <RuntimeLibrary>MultiThreadedDebug</RuntimeLibrary>
        </ClCompile>
        <Link>
            <AdditionalLibraryDirectories>$(MSBuildThisFileDirectory)native\libs\;%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
            <!-- Put your lib name here -->
            <AdditionalDependencies>stringlib.lib;%(AdditionalDependencies)</AdditionalDependencies>
        </Link>
    </ItemDefinitionGroup>
    <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
        <ClCompile>
            <RuntimeLibrary>MultiThreaded</RuntimeLibrary>
        </ClCompile>
        <Link>
            <AdditionalLibraryDirectories>$(MSBuildThisFileDirectory)native\libs\;%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
            <!-- Put your lib name here -->
            <AdditionalDependencies>stringlib.lib;%(AdditionalDependencies)</AdditionalDependencies>
        </Link>
    </ItemDefinitionGroup>
</Project>

This file tells Visual studio to add your package directory in include dirs and lib dirs. It also adds a dependency to the project. Say you'd like to automatically copy your dlls to your output directory, in this case, copy and paste the following code into your targets file:

<?xml version="1.0" encoding="utf-8" ?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemGroup>
        <StringLibBinaries Include="$(MSBuildThisFileDirectory)native\libs\*.dll" />
    </ItemGroup>
    <Target Name="StringLibCustomTarget" AfterTargets="Build">
        <Copy SourceFiles="@(StringLibBinaries)" DestinationFolder="$(OutDir)" />
    </Target>
</Project>

We're doing this in a targets file because props files can only add a build step that will overwrite any existing build steps in your project.

Alright, now that you're done, just pack your nuget package

nuget pack stringlib.nuspec

Notes that might help :

•  If you're planning to pack static libraries, you might encounter link issues due to incremental debug level. Unfortunately, the only solution I found here was to pack both Debug and Release libraries and to link the right one against your project depending on your configuration

<AdditionalLibraryDirectories>$(MSBuildThisFileDirectory)native\libs\$(Configuration)\;%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>

•  Same trick can be used if you have packed libraries for different visual studio toolchains

<AdditionalLibraryDirectories>$(MSBuildThisFileDirectory)native\libs\$(PlatformToolset)\;%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>

•  [Still to be investigated] Some packages are not installing properly from NuGet manager. The installation starts and stops without the green check mark that confirms the installation. I noticed that this was related to the name (e.g. ID) of the package. Suprisingly, adding numbers to the ID (and hence to the names of props and targets files) resolved the issue. This has been seen with VS2012, using Nuget v2.8.

Other things to know about Nuget

Check which version of nuget you need according to the version of visual studio you are using to avoid confusion !

List remote packages

nuget list -Source <url or server name>

Delete package on remote

nuget delete mypackage 1.0.0 -Source <url or server name>

Push a package to remote

nuget push mypackage.1.0.0.nupkg -Source <url>[/subfolder]

with subfolder typically being native or managed.

Force visual studio to install packages in a local directory

Nuget configuration can be overriden by adding a nuget.config file at the root of your solution.
To change the install directory, add this to the file:

<?xml version="1.0" encoding="utf-8"?>
<settings>
  <repositoryPath>..\packages</repositoryPath>
</settings>

In this example, packages will be placed in an upper folder.
Other options can be overriden as listed here : https://docs.microsoft.com/en-us/nuget/reference/nuget-config-file#example-config-file

Read More

Migration from SVN to GIT

Step 0 : Define migration strategy

Option 1 [recommended] : If possible, do not bother to migrate the SVN history to git but rather keep the old repositories archived and start over in git from the tip of the trunk.

Option 2 : Migrate trunk and tags only.

Option 3 [can be complex] : Migrate the entire repository.

I picked option 3 because my colleagues wanted to keep all of the history. IMO this is not necessary as long as the old repositories are still accessible. Plus, the migrated history is not perfectly identical to SVN and can be more confusing than helpful.

Step 1 : Create a file with authors

We should convert authors' svn identifiers to git format (Firstname Name <mail>).
To do this, run the following command from your SVN repo

svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors.txt

Now edit the file and fill the missing information to finally have something like that:

MMO = Mickey Mouse <mickey.mouse@disney.com>
DDU = Donald Duck <donald.duck@disney.com>

Step 2 : Migrate dependencies first

If your main project is using externals, those will need to be converted to submodules (other options are possible like subtree or third party git externals but in most cases, submodule will do just fine).

To do that, repeat the following commands for each dependency repository:

git svn clone --trunk=<trunk_dir> --branches=<branches_subdir> --tags=<tags_subdir> --no-metadata --authors-file=authors.txt <svn_repo_url> .

Note: if the svn repository follows the standard 'trunk,branches,tags' structure, replace '--trunk=<trunk_dir> --branches=<branches_subdir> --tags=<tags_subdir>' with --stdlayout

Now export the branch list to a file.

git branch -r > branches.txt

Filter the list of branches in branches.txt and keep only those you want to migrate.
Copy tags to a separate file for now (tags.txt).

Now for each entry in branches.txt, do:

git checkout -b <local_branch_name> <remote_branch_name>

Use the branch name in branches.txt for <remote_branch_name>.

For each tag, do the following:

git checkout -b <local_branch_name> <remote_branch_name>

git tag <local_branch_name>

git checkout master

git branch -D <local_branch_name> 

Tags are exported as branches by git-svn so we are forcing a checkout and we manually add the tag in git repository.

Now run the following command to clean history from empty commits (migration side-effect):

git filter-branch --prune-empty -f -- --all

Delete trunk branch

git branch -D trunk

Configure remote git repository

Here I suppose that you already have created a remote git repository to host your code.

git remote add origin <git_remote_repo_url>

Finally push the repo

git push origin --mirror

Step 3 : Migrate the main project repository

Repeat steps from Step 2 until trunk deletion.
From there, we will need to resolve externals.

We will start with master branch, but these steps need to be repeated for each branch.

git checkout master

Run the following command to export the list of externals to a file.

git svn show-externals --id=origin/trunk > externals.txt

Note: In my case, this command wouldn't work for other branches and it is particularily slow. I managed to achieve the same thing from svn command line in the old repository, for each branch.

svn propget svn:externals -R <externals_dir> > externals.txt

Suppose our svn:externals were in a folder named Externals, here is what should be done for each submodule:

git submodule add --force <dependency_git_remote_repo_url> ./Externals/<submodule_dir>

Then we need to fix the submodule to the commit that matches the version used in svn.

cd ./Externals/<submodule_dir>

For tags:

git checkout <commit_id_or_tag_label>

For branches:

git checkout -b <local_name> <remote_git_branch_name>

cd ../..

Repeat these steps for all submodules. Once done, commit:

git commit -m "Migrated svn:externals properties to .gitmodules"

Repeat these steps for all branches. Before starting, for each branch, do cleanup to avoid confusing error messages:

rm -rf ./Externals

Finally, when all branches have been processed, push the repository to remote server:

git remote add origin <git_remote_repo_url>

git push origin --mirror

Read More