A good introduction to dependency management in large software projects

I have worked on some parts of the build dependency handling in a large software project in my company (monorepo project, > 1800 packages, > 10 million LOC, 100s of external dependencies). I recently stumbled upon this article in Google’s Bazel documentation and really enjoyed reading it:

https://bazel.build/basics/dependencies

As usual, when it comes to developing software at scale, Google has already written about the experiences you make in your journey. I can agree to all of the points about dependency handling that they make.

Invaluable SVN properties for source files

If you add a source file to your repository, think about adding those SVN properties:


svn:eol-style = native
svn:mime-type = text/plain
svn:keywords = Date Revision HeadURL Author

The first line checks a file for consistent line endings before it can be commited.
The last line enables the magic tags like $Date: $ (see previous post).

You can also make TortoiseSVN auto-add properties by setting it up in the [auto-props] section of your SVN config file (%APPDATA%/Subversion/config). Here are some useful lines:


[auto-props]
*.cpp = svn:mime-type=text/plain;svn:eol-style=native
*.hpp = svn:mime-type=text/plain;svn:eol-style=native
*.h = svn:mime-type=text/plain;svn:eol-style=native
*.c = svn:mime-type=text/plain;svn:eol-style=native
*.dbc = svn:mime-type=text/plain;svn:eol-style=CRLF
*.doc = svn:needs-lock=true;svn:mime-type=application/msword

Fixing CANoe remote control via COM

If you need to control replay of a CANoe/CANalyzer logfile from an outside application (e.g. a script), you can do so via the COM interface CANoe/CANalyzer provides. When you create an instance of the CANoe “Application” object, Windows COM service retrieves the CANoe executable path etc. from the registry and starts the application. If that information does not work for you, e.g. because you installed several versions of CANoe on your machine and want the older version to handle the COM requests, you can repair the registry entries by typing:


canoe32 -regserver

You can find the changes in the registry key
[HKEY_CLASSES_ROOT\CLSID\{7F31DEB1-5BCC-11d3-8562-00105A3E017B}].

Not all functions listed in the CANoe COM help have to be available in every version. To be sure, you can inspect the exported object interface withe the Visual Studio helper application “oleview.exe”:

  1. Open a Visual Studio 2005 command prompt and enter “oleview”
  2. In the Tree view part, open “Type Libraries” and select CANoe Type Library
  3. Click on the “View” button view button in the tool bar to open the interface/class viewer for CANoe

Profiling with Python: cProfile and RunSnakeRun

This is my favorite way to analyze the performance of my Python scripts:

  1. Call the script with cProfile:
    python -m cProfile -o profile.dat main.py
  2. Open the profiler data with RunSnakeRun:
    %PYTHON_ROOT%\scripts\runsnake.exe
    A nice GUI will start up, and you can open the profile.dat file in there.
  3. You have to install RunSnakeRun first, of course. You should choose the easy_install procedure I described here for the runsnake.exe to be installed in the scripts directory. You can find the current RunSnakeRun package on pypi.

    RunSnakeRun homepage

Keyword substitution with Subversion

It’s a cumbersome task to keep the revision number, last-changed date and last-modified-by information of a source file up-to-date. That’s what subversion keyword substitution is for.

How it works:
You add special fields to your file comment, like $Author$. This is my default C++ header comment:

/**
*
* \brief Short description of the purpose of this file
*
* @copyright Copyright (c) 2010 me
* @author Original Author: me
* @author Last changed: $Author$
* @date Creation date: 2010-05-28
* @date Last changed: $Date$
* @version $Revision$
*
*/

When you add and commit this file, nothing will happen to the special fields enclosed in dollar signs. First, you have to add them to the subversion property “keywords”. To do so with ToroiseSVN, right click and open the file’s propertys and go to the subversion tab. Select svn:keywords from the drop-down list and enter something like “Date Author Revision” on the edit line. You can also select multiple files and change the property for all of them at once. Your next commit will show the auto-updated fields.

For a description of all possible keywords, look here.

Controlling issue trackers from your version control system

I’ve been looking around for issue trackers and this is a short summary on how to control issues from commit comments.

Trac:
see #Nr – reference an issue (comment shows up in history of the ticket with that number)
fixes #Nr – automatically closes a “bug” ticket with that number
closes #Nr – automatically closes a “feature request” ticket with that number

Redmine:
for referencing issues: refs, references, IssueID
for fixing issues: fixes, closes
example: This commit refs #1, #2 and fixes #3

Mantis:
Has to be configured by a regular expression.
Could be issue, bug #id for referencing and fixes, closes #id etc. for fixing issues.

Some more notes on these systems:
Trac (current version 0.12) requires python with setuptools and Genshi, one DBMS (SQLite, PostgreSQL or MySQL) and the svn repository (one per Trac installation) has to exist on the same server as the Trac.
Redmine (current version 0.95, 1.0 coming soon) is based on Ruby on Rails, and works at least with mysql/pgsql/sqlite as DBMS. SVN binaries have to be installed on the machine running the redmine, but the SVN server can exist somewhere else.
Mantis (current version 1.3) is based on PHP and is proven to work with MySQL/PGsql/MS SQL or DB2, but not SQLite. The SVN server can reside on a different machine.
While Mantis and Trac SVN integration seems to work via a post-commit hook only, Redmine also allows to update SVN information periodically (e.g. cron-job) or on demand.