Development

This chapter will get you started with MSS development.

MSS is written in Python.

Development work is exclusively performed on the develop branch. Bug fixes are backported to the stable branch whenever feasible. If a bug fix cannot be backported, corresponding updates will be made to the develop branch. We release new minor versions on a regular basis. New features should be submitted as pull requests against the develop branch. Changes and documentation are managed through our issue tracker.

When it is ready the developer version becomes the next stable.

The stable version of MSS is tracked on BLACK DUCK Open Hub

Contributing

Please read our contributing guidelines and setup instructions to get started with MSS development.

Using our Issue Tracker on github

How to Report Bugs

Please open a new issue in the appropriate GitHub repository https://github.com/Open-MSS/MSS/issues/new with steps to reproduce the problem you’re experiencing.

Be sure to include as much information including screenshots, text output, and both your expected and actual results.

How to Request Enhancements

First, please refer to the applicable GitHub repository and search the repository’s GitHub issues to make sure your idea has not been (or is not still) considered.

Then, please create a new issue in the GitHub repository describing your enhancement.

Be sure to include as much detail as possible including step-by-step descriptions, specific examples, screenshots or mockups, and reasoning for why the enhancement might be worthwhile.

Forking the Repo

Firstly you have to make your own copy of project. For that you have to fork the repository. You can find the fork button on the top-right side of the browser window.
Kindly wait till it gets forked.
After that copy will look like <your-user-name>/MSS forked from Open-MSS/MSS.

Cloning the Repo

Now you have your own copy of project. Here you have to start your work.
Go to desired location on your computer where you want to set-up the project.
Right click there and click on git bash. A terminal window will pop up
Click The big green button which says “Code”. Copy the URL. Like this
Now Type the command git clone <your-fork-url>.git and hit enter. Also, you could use gh repo clone <repo-owner>/<repo-name>, If you don’t have gh, see this for installation here.
Wait for few seconds till the project gets copied

or simply head over here for cloning a repository

Setting up a git remote

Now you have to set up remote repositories
Type git remote -v in terminal to list remote connections to your repo.

It will show something like this:

origin  git@github.com:<your-user-name>/MSS.git (fetch)
origin  git@github.com:<your-user-name>/MSS.git (push)

Add an upstream remote by the command

git remote add upstream git@github.com:Open-MSS/MSS.git

Again type in command git remote -v to check if remote has been set up correctly

It should show something like this

origin     git@github.com:<your-user-name>/MSS.git (fetch)
origin     git@github.com:<your-user-name>/MSS.git (push)
upstream   git@github.com:Open-MSS/MSS.git (fetch)
upstream   git@github.com:Open-MSS/MSS.git (push)

Update local stable branch

If you don’t have a stable branch, create one first or change to that branch:

git checkout [-b] stable
git pull git@github.com:Open-MSS/MSS.git stable
git push

Setting Up a Local Environment

Requirements

System requirements

Any system with basic configuration.

Operating System : Any (Windows / Linux / Mac).

Software requirement

Pixi

Skill set

Knowledge of git & github

Python

Software environment for development

The dependencies necessary to get a working development environment for MSS are specified in pixi.toml and pixi.lock. This means you can get a shell with all required packages installed using:

pixi shell -e dev

Afterwards, a call to e.g.:

msui

will run the development version of msui.

You can also use pixi’s “run” subcommand to directly run a command in the development environment, like so:

pixi run -e dev msui

Setup MSWMS server

In the MSS package is some demodata included. The default where this is stored is $HOME/mss. Your clone of the MSS repository needs a different folder, e.g. workspace/MSS. Avoid to mix data and source.

demodata is provided by executing:

mswms --seed

To use this data add the mswms_settings.py in your python path:

export PYTHONPATH=~/mss
mswms

Setup MSColab server

The MSColab server is built using the Flask rest framework which communicates with the PyQt5 frontend of MSS. You can view the default configuration of MSColab in the file mslib/mscolab/conf.py. If you want to change any values of the configuration, please take a look at the “Configuring Your MSColab Server” section in Configuring Your MSColab Server

When using for the first time you need to initialise your database. Use the command mscolab start to initialise it. The default database is a sqlite3 database. If you want to play around with example data, you can import it using mscolab db –seed. The content of the example data can be found in the file mslib/mscolab/seed.py.

To start your server use the command mscolab start. This would start the MSColab server on port 8083. Going to http://localhost:8083/status should now show “MSColab server”. This means your server has started successfully. Now you can use the MSS desktop application to connect to it using the MSColab window of the application.

Code Style

We generally follow PEP8, with 120 columns instead of 79.

Output and Logging

When writing logger calls, always use correct log level (debug only for debugging, info for informative messages, warning for warnings, error for errors, critical for critical errors/states).

Building the docs with Sphinx

The documentation (in reStructuredText format, .rst) is in docs/.

Usually building the docs also includes creating the images and pages for the gallery feature. This can be omitted by setting an environment variable

export GALLERY=False

To build the html version of it, you need to have sphinx installed:

cd docs/
make html

Then point a web browser at docs/_build/html/index.html.

For heading hierarchy we use

H1
==

H2
--

H3
..

H4
~~

Run Tests

Considering that the software environment is set up using pixi, you can run the test suite using:

pixi run -e dev pytest -n logical

To avoid getting a lot of opened windows from the test run you can either prepend QT_QPA_PLATFORM=offscreen like so:

QT_QPA_PLATFORM=offscreen pixi run -e dev pytest -n logical

or install xvfb from your distributions package manager and use xvfb-run like so:

xvfb-run pixi run -e dev pytest -n logical

Other options for pytest are possible to use, you can e.g. set a higher verbosity using -v, leave out the -n option to run the tests sequentially instead of in parallel, or select a specific subset of tests to run using the -k option.

Verify Code Style

A flake8 only test is done with flake8 mslib tests.

Coverage

$ pytest --cov mslib tests

This plugin produces a coverage report, example:

----------- coverage: platform linux, python 3.7.3-final-0 -----------
Name                                     Stmts   Miss Branch BrPart  Cover
--------------------------------------------------------------------------
mslib/__init__.py                            2      0      0      0   100%
mslib/msui/__init__.py                      23      0      0      0   100%
mslib/msui/aircraft.py                      52      1      8      1    97%
mslib/msui/constants.py                     12      2      4      2    75%
mslib/msui/flighttrack.py                  383    117    141     16    66%

Profiling

Profiling can be done by e.g.:

$ python -m cProfile  -s time ./mslib/mswms/seed.py --seed > profile.txt

example:

/!\ existing server config: "mswms_settings.py" for demodata not overwritten!


/!\ existing server auth config: "mswms_auth.py" for demodata not overwritten!


To use this setup you need the mswms_settings.py in your python path e.g.
export PYTHONPATH=~/mss
      557395 function calls (543762 primitive calls) in 0.980 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    23    0.177    0.008    0.607    0.026 seed.py:1089(generate_file)
   631    0.113    0.000    0.230    0.000 seed.py:769(_generate_3d_data)
   179    0.077    0.000    0.081    0.000 {method 'createVariable' of 'netCDF4._netCDF4.Dataset' objects}

Writing Tests

Ideally every new feature or bug fix should be accompanied by tests that make sure that the feature works as intended or that the bug is indeed fixed (and won’t turn up again in the future). The best way to find out how to write such tests is by taking a look at the existing tests, maybe finding one that is similar and adapting it to the new test case.

MSS uses pytest as a test runner and therefore their docs are relevant here.

Common resources that a test might need, like e.g. a running MSColab server or a QApplication instance for GUI tests, are collected in tests.fixtures in the form of pytest fixtures that can be requested as needed in tests.

Keyring Features

This document outlines step-by-step instructions for using the keyring features using the command line.

Prerequisites

Confirm the Default Keyring Backend

Use the following command to list available keyring backends and check which one is currently in use:
```
keyring --list-backends
```

Command-Line Commands for Keyring

Set a Password

Store a password for a specific service and user:
```
keyring set SERVICE_NAME USERNAME
```
Example:

Case 1: Standard Service Name

keyring set http://localhost:8083 myname@mydomain

Case 2: Custom Authentication Service

keyring set MSCOLAB_AUTH_http://localhost:8083 mscolab

The command will securely prompt you to input a password (e.g., example_password).

Get a Password

Retrieve the stored password for a service and user:

keyring get SERVICE_NAME USERNAME

Example:

Case 1: Standard Service Name

keyring get http://localhost:8083 myname@mydomain

Case 2: Custom Authentication Service

keyring get MSCOLAB_AUTH_http://localhost:8083 mscolab

Output:

example_password

Delete a Password

Remove the stored password for a service and user:
```
keyring del SERVICE_NAME USERNAME
```
Example:

Case 1: Standard Service Name

keyring del http://localhost:8083 myname@mydomain

Case 2: Custom Authentication Service

keyring del MSCOLAB_AUTH_http://localhost:8083 mscolab

To confirm the deletion, attempt to retrieve the password:

keyring get MSCOLAB_AUTH_http://localhost:8083 mscolab

Changing the database model

Changing the database model requires adding a corresponding migration script to MSS, so that existing databases can be migrated automatically.

To generate such a migration script you can run:

flask --app mslib.mscolab.app db migrate -d mslib/mscolab/migrations -m "To version <next-major-version>"

Depending on the complexity of the changes that were made, the generated migration script might need some tweaking.

If there is already a migration script for the next release, then please incorporate the generated migration script into this existing one, instead of adding a new one. You can still generate a script with the above command first to get a starting point for the changes.

Pushing your changes

Now you have made the changes, tested them and built them. So now it’s time to push them.
Goto your terminal and type git status and hit enter, this will show your changes from the files
Then type in git add and hit enter, this will add all the files to staging area
Commit the changes by git commit -m "<message-describing-your-change>" and hit enter.
Now push your branch to your fork by git push origin <your-branch-name> and hit enter.

Creating a pull request

By this time you can see a message on your github fork as your fork is ahead of Open-MSS:develop by <number> of commits and also you can see a button called Compare and pull request.

Click on Compare and pull request button.

You will see a template.

Fill out the template completely by describing your change, cause of change, issue getting fixed etc.

After filling the template completely click on Pull request

Creating a new minor, patch level release

The stable branch will only receive critical bug fixes, and security patches. These patches must either address a bug that is only present on stable, be a backport of a patch that was already merged into develop, or there must simultaneously be a PR addressing the same issue on develop.

We will review changes for both branches when there is no backport from develop possible.

For a new minor/patch release

make sure all issues for this milestone are closed or moved to the next milestone
update CHANGES.rst, based on git log
check version number of upcoming release in pyproject.toml
verify that pyproject.toml, MANIFEST.in are complete
Update the version in the lock file by pixi lock

tag the release:

git tag -s -m "tagged/signed release X.Y.Z" X.Y.Z
git push origin X.Y.Z

write a release information on https://github.com/Open-MSS/MSS/releases
create a release on anaconda conda-forge
announce on:
- Mailing list
- Mastodon (follow https://fosstodon.org/@MSS)

Creating a new major release

make sure all issues for this milestone are closed or moved to the next milestone
update CHANGES.rst, based on git log
check version number of upcoming release in pyproject.toml
verify that pyproject.toml, MANIFEST.in are complete
Update the version in the lock file by pixi lock
for a new major release rename the existing stable branch to oldstable and recreate it from the develop branch