Skip to content

[NOT READY] Refactor LUA profiles#2032

Closed
emiltin wants to merge 1 commit intomasterfrom
rewrite/profiles
Closed

[NOT READY] Refactor LUA profiles#2032
emiltin wants to merge 1 commit intomasterfrom
rewrite/profiles

Conversation

@emiltin
Copy link
Copy Markdown
Contributor

@emiltin emiltin commented Feb 29, 2016

Rewrites/refactors all LUA profiles. I consider this work in progress, but would like to hear your feedback at this point, especially since the plan is to move profiles to JS.

  • Reorganize code into a class hierachy to enable code reuse between profiles.
  • Split code into managable methods.
  • Make it possible to delegate processing between profiles, eg. to correctly handle foot routing when pushing bikes.
  • Make it possible to run the tag processing from LUA (without the C++ side) to make debugging easier. Processing writes a trail of messages to make it easier to understand what happens.

All tests pass. However, I modified a number of tests because I found them to be incorrect or at least questionable. I found a number of situations that the existing profiles does not seem to handle correctly, or not at all.

The tag processing is actually a bit complex due to the somewhat free-form tagging scheme that OSM uses, and there's a lof of exceptions and interrelations. I've tried to setup a general processing structure in the Base class, which subclasses can reuse. If you look at the car and foot processors (in lib/) you will see that they're not much more than settings.

Subclasses get a copy of the settings from their parent class. They can then merge in changes to easily modify/extend the settings. Sets (used for things like whitelists) are merged, while Sequences (used for things like access tag hierachies) are copied.

The mode code have changed to ensure that all profiles use the same codes for the same modes. I understand you plan to move to string identifiers?

I haven't done any measurements on the performance yet. The debugging trails are currently active even when doing real processing, etc.

@TheMarex
Copy link
Copy Markdown
Member

Given that we eventually want to move way from lua profiles, I would put a hold on bigger lua refactors right now. Some of the changes here conflict with changes we made in the new API branch, namely the mode handling (there is now a predefined list of modes, we might move to setting string identifiers in lua).

That said you seem to incorporate a few relevant fixes and additions (duration parsing for example) that I would like to break out and upstream. Todo from my side:

  • mark the changes that we want to port
  • incorporate after the new API merge

@emiltin
Copy link
Copy Markdown
Contributor Author

emiltin commented Mar 1, 2016

I will leave it for know then. But I hope the PR can inspire a discussion about how to organize the profiles in general. I think the current monolithic profiles are becoming hard to maintain.

@TheMarex
Copy link
Copy Markdown
Member

Please reopen the PR for master, develop is going away.

@TheMarex TheMarex closed this Apr 22, 2016
@daniel-j-h
Copy link
Copy Markdown
Member

What's the plan here? If we're not switching away from Lua in the near future (ref: #1974) we could at least incorporate this work to refactor the profiles.

@emiltin
Copy link
Copy Markdown
Contributor Author

emiltin commented Jun 8, 2016

do you want to do anything further with this? is there a decision on wether or not to move away from lua for the profiles?

@emiltin
Copy link
Copy Markdown
Contributor Author

emiltin commented Sep 30, 2016

Any news on when LUA will be removed, and whether you think this PR is useful?

@emiltin
Copy link
Copy Markdown
Contributor Author

emiltin commented Oct 23, 2016

Refactored on latest master (phew). There are still some tests failing, but I think this is a good time to have a discussion about the general approach.

The goals have been to:

  • Make the LUA code manageable and enable code reuse between profiles.
  • Establish a common framework/flow for how to process tags, ie. highway, oneway, surface, maxspeed, etc. while still enabling full customisation.
  • Make it easier to debug LUA code by making it possible to run the profile code directly from the command line, without the C++ binaries. Ie. provide an input way and see the result.

To achieve this:

All profile code has been reorganised into a class hierarchy. Of course LUA doesn't really have classes, but we can emulate them well enough. Hierarchy:

Base ----- Car
      \----Bicycle
       \---Foot

The Base class established a framework for how tags are processed, and call various methods to handle things like oneway, surface, etc. Sub classes overwrite the methods when needed.

This also means that the monolithic way_function has been split into a number of small more manageable methods.

The move to class hierarchy makes it possible to pull in processing via another mode when necessary. Because you can push your bike, correct bicycle routing really requires full foot handling as well. This was always a problem in the bike profiles. Now the bike profile simply creates a Foot object and let it check the foot routing when required.

The file profiles/debug.lua is an initial demonstration of how the profile code can be run directly form the command line to easily check how a specific set of tags are processed. For this purpose, a log of messages can be written to during processing, to make it easier to understand how the processing reaches a certain result. However, this LUA debugging is currently broken, because there are a few C++ methods that the LUA code relies on, related to guidance.
Make it possible to run the tag processing from LUA (without the C++ side) to make debugging easier. Processing writes a trail of messages to make it easier to understand what happens.

Utility methods have been added for doing common tag processing like looking for forward/backward variants of tags.

Each direction (forward/backward) is processed separately, ie. with the same code, which is passed a direction object . This avoids a lot of code duplication where code handle first forward, then backward. It also makes it easier to make sure forward/backward tags handled in all relevant cases.

Running all tests after doing 'rm -r test/cache' shows no significant difference in processing time between this branch and master.

Proposed next steps:

  • Discuss the overall approach/architecture.
  • Benchmark with real-world data set.
  • Decide whether we want this merged.
  • Iron on remaining test cases.
  • Improve

@emiltin
Copy link
Copy Markdown
Contributor Author

emiltin commented Oct 25, 2016

I can't reopen this PR, wanna reopen it or should I create a new PR? cc @daniel-j-h @TheMarex

@emiltin emiltin changed the base branch from develop to master October 25, 2016 05:41
@daniel-j-h
Copy link
Copy Markdown
Member

Yes I would love to see this refactor going in - can you open a new pull request, Github seems to not allow re-opening this one.

@emiltin emiltin mentioned this pull request Oct 25, 2016
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants