Skip to content

Inconsistent implementation between turn_indexes_write_buffer and turn_weight_penalties/turn_duration_penalties #5862

Closed
#5868
Closed
Inconsistent implementation between turn_indexes_write_buffer and turn_weight_penalties/turn_duration_penalties#5862
#5868
@wangyoucao577

Description

@wangyoucao577
wangyoucao577
opened on Oct 20, 2020

All the index/weight/duration of turns are generated and written into file in void EdgeBasedGraphFactory::GenerateEdgeExpandedEdges. However, their implementation are a little bit different.

Both turn_weight_penalties and turn_duration_penalties are vectors that cross the function, and will be written into file once when the whole processing finished.

osrm-backend/src/extractor/edge_based_graph_factory.cpp

Lines 472 to 474 in 9f80f6d

// Now, renumber all our maneuver overrides to use edge-based-nodes
std::vector<StorageManeuverOverride> storage_maneuver_overrides;
std::vector<NodeID> maneuver_override_sequences;

osrm-backend/src/extractor/edge_based_graph_factory.cpp

Lines 1164 to 1167 in 9f80f6d

// write weight penalties per turn
BOOST_ASSERT(turn_weight_penalties.size() == turn_duration_penalties.size());
files::writeTurnWeightPenalty(turn_weight_penalties_filename, turn_weight_penalties);
files::writeTurnDurationPenalty(turn_duration_penalties_filename, turn_duration_penalties);

By contrast, turn_indexes_write_buffer will be written to file per 1000. It leads to the write to file action dispersedly in a few places.

osrm-backend/src/extractor/edge_based_graph_factory.cpp

Lines 452 to 454 in 9f80f6d

storage::tar::FileWriter turn_penalties_index_file(
turn_penalties_index_filename, storage::tar::FileWriter::GenerateFingerprint);
turn_penalties_index_file.WriteFrom("/extractor/turn_index", (char *)nullptr, 0);

osrm-backend/src/extractor/edge_based_graph_factory.cpp

Lines 490 to 496 in 9f80f6d

// Because we write TurnIndexBlock data as we go, we'll
// buffer them into groups of 1000 to reduce the syscall
// count by 1000x. This doesn't need much memory, but
// greatly reduces the syscall overhead of writing lots
// of small objects
std::vector<lookup::TurnIndexBlock> turn_indexes_write_buffer;
turn_indexes_write_buffer.reserve(TURN_INDEX_WRITE_BUFFER_SIZE);

osrm-backend/src/extractor/edge_based_graph_factory.cpp

Lines 1057 to 1064 in 9f80f6d

// Buffer writes to reduce syscall count
if (turn_indexes_write_buffer.size() >= TURN_INDEX_WRITE_BUFFER_SIZE)
{
turn_penalties_index_file.ContinueFrom("/extractor/turn_index",
turn_indexes_write_buffer.data(),
turn_indexes_write_buffer.size());
turn_indexes_write_buffer.clear();
}

After go through the codes, I think unify the process of turn_indexes same as turn_weight_penalties/turn_duration_penalties should be ok, right? Any concern?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions