Section 13
Section 13
9)
This section explains the use of the program SATME2 in conjunction with the
supplementary program SATPIJA to estimate trip matrices from traffic counts. It
is based on the theoretical procedure generally referred to as ME2, although it
would be better represented as ME2, - Matrix Estimation from Maximum Entropy.
A full list of references is given in Appendix C; a condensed summary follows.
SATME2 essentially tries to improve the fit between modelled and observed flows
by selectively factoring individual cells of the input trip matrix. Thus, with
reference to Figure 13.1 which illustrates the general functions of an assignment
model and which is a slight variation on Figure 2.1, we note in particular that (a)
the trip matrix is one of the two main inputs and that (b) the main output of the
process is modelled flows.
Modelled flows may, in turn, be directly compared with (for the base year, at least)
observed flows obtained from, for example, automatic vehicle detectors.
As also noted in Section 2.1, there are three general sources of errors – trip
matrix, network coding and route finding - of which errors in the trip matrix are
probably the single biggest source of error. What the process known as ME2
attempts to do therefore is, by essentially working backwards through Figure 13.1,
to establish a trip matrix which, when assigned, will give the desired answer, i.e.,
will reproduce the observed flows.
Thus we seek a trip matrix Tij which satisfies, for each of the observed counts, the
following condition:
Equation 13.1
T P ij
ij ija Vaobs
where:
We may think of Equation 13.1 as a “constraint” on the trip matrix arising from the
observed flow on link a.
In addition we would like a trip matrix which is as “near” as possible (or as “least
different” as possible) from any existing estimate of the trip matrix. In more
mathematical terms we seek to minimize some “distance measure” │T – t│
between the updated trip matrix T and the “prior” trip matrix t = tij. The entropy
maximising model differs from other forms of matrix-estimation models in the way
that it defines its measure of distance and, as a consequence, the resultant
equations for T.
Thus the ME2 “distance” definition used to update an existing trip matrix may be
written as:
Equation 13.2
D T , t Tij log Tij / tij Tij tij
ij
And the resulting equation giving the “maximum entropy” solution may be written:
Equation 13.3
Tij tij X aPija
a
Where:
SATME2 uses an iterative procedure to find a set of balancing factors Xa for each
counted link to ensure that the assigned flows match the counts within certain
user-defined limits (See 13.1.8)
The procedure is in fact very similar to that used in the well-established Furness
matrix-balancing procedure with the exception that Furness constrains the flows
into or out of zones as opposed to the flows on selected links. In fact zonal
constraints are just a special case of link constraints and as such may be input
directly to SATME2 (See 13.1.5 and13.3.2).
Equation 13.4
D T , t Tij tij
2
ij
And indeed a large number of alternative matrix estimation models have been
constructed using this definition.
SATME2 may be used in essentially two different ‘modes’. In the ‘update’ mode,
as discussed above, it takes an old or prior trip matrix and current traffic counts to
estimate the most likely trip matrix consistent with the information contained in the
counts and using the old trip matrix as a starting point.
In the second mode no ‘prior’ trip matrix is available and the model therefore
assumes that all trips are equally likely so that, in effect, it starts with a prior trip
matrix in which all elements are equal. (The model in fact chooses an appropriate
average constant value for all tij in Equation 13.4 to start with.)
Note that the second mode is virtually never used in practical studies – nor should
it be! – although it may have some useful applications in setting up purely
synthetic matrices for theoretical studies of small artificial networks.
Users are strongly advised to carefully read Section 13.3.5 regarding the correct
choice of a prior trip matrix.
In order to update a trip matrix in SATURN two linked programs must be run as
illustrated in Figure 13.2: SATPIJA and SATME2.
Thus SATME2 requires a “PIJA” file, each element of which represents the
proportion (P) of trips between a particular origin-destination pair (IJ) which uses
the counted link (A). The PIJA data are obtained through the program SATPIJA
following a SATURN assignment using the SAVEIT option (15.23).
More precisely SATPIJA analyses the output .ufs and .ufc files from a full run of
SATURN in order to produce a ‘.ufp’ file which contains the “PIJA factors”. This
information is then fed into SATME2, along with the prior trip matrix, in order to
produce an updated trip matrix.
Both SATPIJA and SATME2 require control data files as described in Section
13.2 below. The count data may either be included in the SATPIJA file or taken
from the 77777 data segment in the network .dat file originally input to SATNET
(see 13.2.1)
“Counts” may refer to either simulation or buffer links, turning flows within the
simulation network on to entry and exit flows from zones (but not on specific
centroid connectors). All of these are “links” in the sense of the assignment
network (5.5.1). The procedures work equally well on “pure” buffer networks with
no associated simulation network.
Note that, in common with all definitions of “flow” within SATURN (see 15.17), the
counts used in SATME2 must be defined in terms of pcus/hr (or, strictly speaking,
in the same units as all the other flows; see 15.17).
Several complications and/or extensions are introduced into the definition of the
constraint counts in SATME2 due to the level of sophistication inherent in
SATURN assignments.
Firstly, within SATURN link flows are made up of two components - trips assigned
from the trip matrix and a fixed component set up by SATNET representing, for
example, bus routes on the link or PASSQ flows. By default it is assumed that the
‘observed’ count input corresponds to both these components together and that
what the user wishes to generate is the trip matrix which will correctly give a flow
equal to the observed flow less the fixed flow. Hence, by default, the fixed flows
are subtracted from the input counts (unless a parameter SUBFIX is set to
.FALSE.) N.B. this correction is not applied to zonal constraints - see 13.3.2.
(Alternatively only the PASSQ component of the fixed flows may be subtracted by
setting SUBPQ = T and SUBFIX = F; see also 13.3.1 and13.4.8.)
Note that if the fixed flows to be subtracted from the input counts exceed the
counts then the target is artificially set to zero and a non-fatal error is generated
since in this case there is no way that a correct trip matrix can be created. A fatal
error might be more appropriate.
(Technical aside: The total fixed and/or PASSQ flows which are subtracted within
SATME2 are actually read from the network .UFS file(s) within SATPIJA and
passed to SATME2 via the .UFP file. This is done automatically by SATPIJA
whether or not SATME2 will actually use them or not.)
To account for this fact the “observed” counts have to be factored up into “target”
counts corresponding to the full demand for trip making. This conversion is done
automatically within SATPIJA and the demand count flows are then passed to
SATME2 via the .ufp files.
N.B. Table 1 within the SATME2 .LPM file prints both the original (actual flow)
counts plus the target or demand version of the flows.
In the same way that counted flows may be associated with individual links they
may also be associated with individual origins or destinations. Thus SATME2 can
make use of both zonal and link constraints at the same time (see the 22222 and
33333 set of data records in 13,2,2).
Tij Ai B j tij
where the row and/or column balancing factors are chosen such that
Ti
ij Dj
T
j
ij Oi
In the case of SATME2 the row and column sums Oi and Dj are user input as
zonal constraints.
Under Furness the balancing factors Ai and Bj are calculated using a “bi-
proportional” technique and they play a highly similar role to the Xa factors in ME2
(which are calculated by a “multi-proportional” technique). Note, however, that the
zonal balancing factors are not explicitly raised to a power of Pija as in Figure
13.3. (Although, since the flows to or from a zone are 100% to or from that zone
and no other, their effective Pija factors are 1 and we could think of the Ai and Bj
factors as being implicitly raised to a power of 1.)
Post release 10.9 a table (11.c) is included in the output .LPM file from SATME2
listing all origin / destination zonal totals both before and after ME2 is applied,
their changes and (if appropriate) their constrained totals.
(N.B. MX gives slightly different results to SATME2 when applied with O-D
constraints only due to a different convergence algorithm, but it is very difficult to
say whether or not one is “better” than the other.)
SATME2 produces a trip matrix which is in some way a best fit to available count
data while at the same time remaining as near as possible to the original trip
matrix. In normal operation the procedure does not take into account the fact that
some (count) information may be more reliable than others or that the matrix may
be more reliable for certain movements. This may be the case for example if a
recent partial O-D survey at a small number of interview stations has been
merged with an older trip matrix.
Frozen cells may be defined in several ways: (a) in terms of complete zones - in
which case all trips into and out of the nominated zones are frozen - or (b), more
precisely, in terms of a “rectangular area” within the matrix specified by a range of
origin and destination zones. See Records (4) and (5) under 13.2.2.
In addition, post 10.7, the frozen cells may be input via a matrix of frozen cells
whereby a value of 0.0 in a cell of the input matrix implies that that particular cell is
to be considered as “frozen”. See the namelist parameters FRODO and FILICE in
13.2.2 where FRODO = T implies that the .ufm matrix defined as FILICE specifies
the frozen cells. See also 7.5.6 where the same named matrix FILICE fulfils a
similar function within SATALL.
N.B. There are a number of alternative names which may be used in addition to
FILICE: ICEFIL, ME2ICE and ICEME2 work equally well and, indeed, in older
versions of SATME2 (prior to 10.7.8) only ME2ICE and ICEME2 were recognised.
See error 33 in Appendix E.4.
See Table 6 in the output .lpm for a summary of the total number of cells frozen
and Table 2 for the corrections due to frozen cells for each individual input count.
Note that if the flows on a link due to the frozen cells in the matrix exceed the
target count then it is not possible to satisfy the constraint by adjusting the non-
frozen cells. In this case a non-fatal error is generated and the target re-set to
zero – although, arguably, a fatal error might be more appropriate.
Warning! Freezing a large number of cells – or, more correctly speaking, freezing
a large proportion of trips – may be dangerous since, in effect, it puts all the
pressure for change on the unfrozen cells and may result in severe distortion to
the that section of the trip matrix
Traditionally the count information input is taken from observed link flows which
are considered to be absolute constraints on the model output. However there
are circumstances in which one might have information which specifies either an
upper or lower limit on the link flow; for example link capacities may be viewed as
upper limits, current day flows as a lower limit for a future year network.
It is also proposed in 13.3.10 that GT/LT constraints may be used to deal with less
reliable constraints.
Greater than/less than constraints, in which case the factor Xa remains at 1.0
(no effect on the trip matrix) unless the constraint is violated, in which case
the constraint is treated as an equality and balanced exactly.
Note that with “less than” constraints Xa factors can only be less than or equal 1
(in order to reduce the flow through that link), while with “greater than” constraints
they must exceed 1.
All counts input on the .ufp file are assumed to be of the same “constraint type” as
specified by the SATME2 Namelist parameter LEG (which stands for Less
than/Equal/Greater than) but it is possible to have mixed constraints by using the
“change of mind” input records described below.
The same concept of constraint types applies equally to zone origin and/or
destination constraints as input under the 22222/33333 data records in the
SATME2 control file (13.2.2).
There is, however, one major distinction between GT/LT constraints on zones and
on links which is that, with zones, it is possible to have both a LT and GT
constraint on the same zone O or D, in which case the constraints specify a range
of values. In this case at most one of the two constraints will be “active” depending
on whether all other constraints take the zone total above or below the appropriate
limit. Alternatively neither can be active if the total is otherwise within range.
Note that in order to set a zonal O/D range it is necessary to input two records
under 22222/33333: one with the LT constraint and the other with the GT
constraint. Obvious error checks are carried out to determine if the LT value is
indeed greater than the GT value.
Note that the definition of less than etc. constraints is only made on entry to
SATME2; the Pija calculations carried out by SATPIJA are the same no matter
what form of analysis is eventually carried out in SATME2.
This option may help to correct errors and/or inconsistencies introduced by the
assignment. It may also be used to satisfy the total flows measured crossing a
screen line or cordon.
Note that the root of the inconsistency could be either the assignment or the
counts themselves, combining the counts dodges the issue. An alternative
solution would be to remove the counts altogether.
O ---------------A B------------ D
To use this option the individual links which are to be combined into a single
constraint are listed within a single set of 66666 records on the input SATME2
control file; see 13.2.2 (No specific advance action is required in SATPIJA except
that each individual link which is to become a member of a combined constraint
with its count must be defined as an input to SATPIJA.) The count which is to be
satisfied is the sum of all the individual counts while the PIJA factors are assumed
to be the sum of the individual PIJA factors.
Note that once an individual link count has been included within a combined count
it is no longer applied as an individual constraint, only as part of the combined
constraint.
Note that adding the PIJA factors together is correct as long as no od pairs use
two or more of the combined counts, i.e., that there are no double crossings. If
these conditions are not satisfied, i.e., if the summed PIJA factor > 1.0, then it is
The condition is certainly satisfied if a set of turn counts out of the same link are
combined together. Equally it should be satisfied if two adjacent and parallel links
are combined together since a driver is highly unlikely to use one link and then
back-track in order to use the second. On the other hand a screen line which
consists of multiple nearly parallel links, e.g., bridges across a river, may allow
double-crossings in practice if the river is not particularly straight (think of the
Thames in London).
If counts with “double crossings” are combined the end effect is likely to be an
increase in the estimated trip matrix (although it is always difficult to say with
absolute certainty what a complicated system such as ME2 will do). For example,
if two links A,B and B,C in series both have counts of 1,000 and both are
(inadvertently) combined together then their combined screen line count constraint
will be 2,000. However if a particular OD pair has Pija factors of 1.0 on both then
the summed Pija factor of 2.0 will be reduced to 1.0. Since the “correct” solution in
this case should be a Pija factor of 1.0 and a count of 1,000 ME2 will therefore
produce extra trips by trying to create 2,000 trips on A to C, not 1,000.
The “balancing factors” Xa as used in Equation 13.4 are critical in obtaining the
maximum entropy or “best” trip matrix and their interpretation is also important in
understanding what the model is doing. Their role is illustrated in Figure 13.3(a)
and Figure 13.3(b).
Thus these figures illustrate how the Equation 13.4 applies to a particular o-d pair
when there are three counts, represented by W1, W2, and W3. Note that 1 and 3
represent flows on links while 2 represent the flow on the “horizontal” turn at that
junction.
In Figure 13.3 (a) we assume that the assignment for this particular o-d pair is “all
or nothing”, i.e., along a single path that goes through all three counts. Hence Pija
= 1.0 at all three counts and the balancing factors X1, X2, and X3 are all raised to a
power of unity: hence the equation as shown.
By contrast in Figure 13.3 (b) there are two routes used between o and d, 50% on
each. Hence Pija = 0.5 at counts 1 and 2 but 1.0 at count 3 where both paths come
together. Hence the balancing factors at points 1 and 2 are raised to a power 0.5
(i.e., square rooted).
Figure 13.3 – (A) A single o-d path with three count sites
W2 W3
W1
TOD t OD .X1 .X 2 .X 3
(B) A single o-d pair with two o-d paths and 3 count sites
W2 W3
W1
50%
50%
If at count 1 the original modelled flow (assigned using tij) was less than the
observed flow W1 then the balancing factor X1 would (all other things being equal)
be greater than 1 in order to increase all the o-d pairs assigned to that link (in
either case). Similarly if count 2 were over-assigned W2 would be less than 1 in
order to reduce the number of trips through it.
It is easy to see that the balancing factors are not unique. For example in Figure
13.3 (a), if all three counts were under-assigned by 50% and OD was the only trip
pair through all three count sites then setting one of the three W’s to 2.0 and the
other two to 1.0 would correctly double the flow on all three counts. Alternatively
setting all three to the cube root of 2 would have exactly the same effect.
As a general rule one would expect that counts which are under-assigned initially
would have a balancing factor greater than 1 and, conversely, if under-assigned
Xa less than 1. Equally, the greater the difference in the original assigned and
modelled flows, the greater the difference of Xa from 1.0.
However, given the interaction and non-uniqueness of the balancing factors noted
above, such simple rules do not always apply. Thus if X1 >>1.0 in either Figure
13.3 (a) or (b) to correct for a large under-assignment at W1 it may be that, if W2
were only marginally under-assigned, then X2 < 1.0 in order to counter-act the
large increases in path flows caused by X1.
We also note briefly at this point, and discuss it more fully under 13.3.1, that the
balancing factors Xa are restricted to lie in the range (1/XAMAX, XAMAX) where
XAMAX is a user-set parameter. These restrictions are in place in order to
prevent the matrix from being “over-distorted” by ME2 trying to satisfy all counts
when, for many possible reasons, this is not possible. A value of Xa equal to either
the minimum or maximum values is a strong indication that there may be
something funny going on at that particular count site.
X a X a 1.0 X a 1.0
Thus Xa = 1 (no changes to the matrix elements using that counts ite) transforms
to Xa’ = 0, values of Xa > 1 are positive and values of Xa < 1 become negative.
The transformed values are particularly useful when being displayed in P1X (see
11.8.5) in that those values that increase the trip matrix are positive and in one
colour while those that decrease the trip matrix are negative and in a different
colour.
Finally, the PIJA factors described above are not fixed, but are themselves
dependant on the trip matrix in a rather complex fashion. Thus if the trip matrix is
updated according to one set of PIJA factors when it is assigned it will be
assigned to different paths and/or different path proportions from the original
matrix and hence a new set of PIJA factors will result. This feed-back effect
introduces a convergence problem in that we seek to find a trip matrix which is
compatible with a set of PIJA factors such that those factors are themselves
compatible with the trip matrix plus assignment.
The normal (but not necessarily the only) procedure adopted within SATURN,
represented in flow chart form in Figure 13.2 above, follows an ad hoc iterative
algorithm whereby an assignment is used to derive the route choice/PIJA factors
which are in turn used to estimate a revised trip matrix. This is then reassigned
and the process continued until stable values are found. (Six such loops has
become a common empirical practice within the UK but whether convergence is
likely after six loops or whether people simply get fed up after six loops is hard to
judge.)
The iterative procedure commences with a full run of SATURN using an old trip
matrix. (If no such matrix is available then one should either create a purely
artificial trip matrix with ‘roughly’ correct elements in each cell or, preferably,
‘Furness’ a matrix using MX from either known or estimated origin and destination
totals). Subsequent runs of SATURN will use the latest updated trip matrix from
SATME2; considerable computer time may be used by taking advantage of the
re-start facility described in Section 15.4.
Highly congested networks, where route choice is very sensitive to the number of
trips to be assigned, generally present the most severe problems. One possible
solution method here may be to “soften” the changes in the trip matrix by
averaging successive trip matrices, e.g. by using a 1/n rule as in assignment (see
7.2.2) but applied to trip matrices, not flows.
The problem may also be associated with a small number of link counts, possibly
those that are inconsistent between themselves. Identifying and removing them
may improve the situation (see 13.3.4).
limited capacity upstream restricts the flow that can reach that particular link
independent of how much “demand” flow is contained in the trip matrix. In such a
situation convergence can never be achieved. An extreme example of this is a
bottleleneck immediately upstream which strictly limits the flow downstream; see
13.3.7.
The above problem manifests itself by the total number of trips increasing each
time SATME2 is re-run within the loop. In fact increasing the demand matrix may
make it even more difficult to increase the actual flow at the count sites since
increasing flow can decrease total capacity (due, e.g., to gap acceptance),
increase the degree of flow metering and therefore, paradoxically, result in even
lower flows downstream. In such cases the problem may be neither due to errors
in the counts or in the prior trip matrix but to, say, too low saturation flows etc.
which unduly restrict flow.
Careful monitoring of both the trip matrices and network statistics (e.g., average
speed) at each step of such loops is absolutely essential.
It must be strongly emphasised that SATME2 should only be applied after all
other possible forms of validation on the network and original trip matrix have
been carried out. In effect ME2 attributes all differences between observed and
modelled counts to errors in the trip matrix and attempts to reduce these
differences by making changes to the matrix. If the differences are due to network
coding errors the process simply introduces additional “errors” in the trip matrix to
counter-balance those in the network. Only when the network errors have been
minimised as far as possible should ME2 be applied.
With respect to network calibration you should pay very close attention to the
routes being generated by the assignment model with the “unadjusted” prior trip
matrix, particularly those which go through your counted links (or possibly don’t go
through but maybe should). If they’re wrong then so too will be your updated trip
matrix. A very clear indicator of poor routing is having a counted link with zero
modelled flow but a positive count.
Equally the prior trip matrix should be as “accurate” as possible. In particular the
use of the SEED parameter to deal with the problem of zero elements in the prior
matrix should be avoided as far as possible. It is recommended that, with trip
matrices derived directly from observation where a zero cell simply implies no
observations, some form of gravity model be applied in order to introduce suitably
low but non-zero values into the zero or missing cells rather than let ME2
introduce a global flat value via the SEED. The Furness routines within MX may
You are therefore strongly advised to check Table 4a in the .lpm file for gross
discrepancies between the initial estimated modelled flows (i.e., those derived
using the prior trip matrix) and the target (as opposed to actual) counts.
Differences may of course be equally due to errors in the counts as much as
errors in the prior matrix; both need to be checked and corrections made prior to
running SATME2.
Note that flow differences in Table 4a – which are effectively differences at the
level of demand flows, not actual - may also be caused by problems in incorrectly
estimating the degree of flow metering upstream (i.e., incorrect capacities). Thus if
the flows estimated from the initial matrix are lower than the target flows the
explanation may be either that that the trip matrix was too small in the first place
or that the model is reducing the flow reaching the counted link due to capacity
constraints upstream whereas in reality no such flow metering takes place.
You should also check that your prior trip matrix gives total flows across the
counted links which are broadly correct; plus/minus 10% would be a good target
to aim for. See Table 5a in the output .lpm file for relevant statistics. If not, you
should probably think about some form of prior factoring. If the absolute
difference between the prior totals and the counted totals is greater than
10%/20%/50% the .lpm file will contain a Warning/Serious Warning/Extremely
Serious Warning message.
For example, if Table 5a gives a ratio of 2.0 of target counts to input modelled
flows the most obvious course of action is to simply factor your prior matrix by a
factor of 2.0. On the other hand it might be that all your counts are monitoring
flows in a north-south direction, in which case it might be more appropriate to
factor only north-south o-d pairs (hint: use sectors in MX) by 2.0 rather than the
full matrix. (This depends on what you “think” north-south counts are telling you
about east-west movements; it might be the case that previous east-west trips
have now “migrated” to north-south trips and that the east-west cells should
actually be factored down rather than up.).
Equally if you have either a complete or partial set of origin/destination counts you
may wish to consider using a Furness factoring procedure (see 10.7) to balance
the o-d counts prior to running SATME2.
Note that any negative cells in the prior trip matrix (which should not be there in
the first place!) are replaced by 0.0 during the ME2 calculations and are therefore
output as zero’s in the output trip matrix. Non-Fatal errors are generated within
SATME2 when this occurs.
If the prior matrix is a good initial approximation of what the “true” trip matrix
should be it follows that the output trip matrix should not differ substantially from
the input prior matrix. Detailed statistics comparing the pre- and post-ME2
matrices at a cell by cell level as well as at more aggregate levels are given in the
.LPM files (with the most detailed statistics only given if M2 = T under &PARAM;
see 13.2.2).
If the total number of trips in the output trip matrix differs by more than 10% from
that input (refer to Table 15 in .LPM) then a Serious Warning message is
generated; if greater than 5% it is only a Warning.
Using GONZO to factor input trip matrices within SATALL and then running
SATME2 with the unfactored trip matrix as input is extremely dangerous, basically
because SATME2 will ignore the factoring implied by GONZO and attempt to
provide factors to correct the original matrix. This can lead to a form of “double
factoring”.
Warning messages are printed within SATPIJA where GONZO is known from the
input .ufs file but not in SATME2 where the .ufs file is not input.
Equally with multiple user classes (See 13.4) where more than one class may
share a single “level” of the “stacked” trip matrix then the expectation is that the
sum of the individual shares (i.e., the sum of the factors in columns 11-15 in the
8888 network data records, see 6.11) should equal 1, otherwise a form of GONZO
factor has been introduced. Again, warnings are printed within SATPIJA if this
happens.
If you are about ready to start running SATME2 then our final advice is: Wait! Re-
read sections 13.1.11 and 13.1.2, making sure that you have done all you possibly
can in terms of calibrating the network and the trip matrix.
Then, when you are convinced that there is nothing more that you can do there,
run SATME2 and look very, very carefully at the outputs, including the .lpm file.
Check which counts are being balanced with difficulty – maybe some of your
counts should be discarded as unreliable, maybe there are still problems with
some of your network routings, maybe the original trip matrix is wrong, etc. etc.
Make the appropriate changes and re-run.
And never, NEVER accept the outputs from SATME2 without double-
checking. It is easy to apply SATME2 but it is considerably more difficult to
update your matrix correctly.
In order to run SATPIJA to produce a PIJA file and then SATME2 to update the
original trip matrix two ascii DAT files - denoted Control Files A and B in Figure
13.2 - must be prepared.
One record for each turning movement or link for which counts are available
formatted as follows:
N.B. Counts may refer to either simulation or buffer links, but to turns ONLY in the
simulation network. Centroid connectors are not allowed, nor are repeated entries
with the same A-B-C. In addition there is no way to split the counts into separate
sub-sets as with multiple 77777 records in network data files.
NOTES
2) If you have a large number of count sets of which only one or two are to be
used then the simplest method is to first include a namelist definition
‘SET777 = .FALSE.’ which will “turn off” all counts and then explicitly define,
e.g., SET777(29) = .TRUE., etc. if set 29 is to be included. (Post 10.7 only)
3) As per the original input of counts to SATNET from a network .dat files link
counts may be repeated (but only in different count sets); repeated links are
ignored by SATPIJA.
4) If SATME2 will only update positive cells in a trip matrix, i.e., zero cells are
not seeded (SEED = 0.0), there is no point in producing Pija values for cells
where Tij = 0. If ALLIJ = F only positive cells will be examined and, in this
case, an input trip matrix is required and should (indeed MUST) be the same
base (i.e., prior) matrix to be input to SATME2. SATPIJA does not make use
of precise individual cell values, it only checks if they are positive.
b) All user classes based on the same “level” of the input stacked trip
matrix (set by LEVEL)
c) All user classes which share a common “vehicle class” (set by IVC)
In cases (b) and (c) the output PIJA factors are “weighted” averages of
the user classes within that level or vehicle class; see equations (13.5)
and (13.6) respectively. Under (a) the matrix level is effectively defined
by that used by the user class specified by KLASS.
(*) The program terminates if either ITERMX is reached OR all estimated and
observed link flows are either, in relative terms, within EPSILN of one another or
the upper/lower limits on their balancing factor limits are reached.
Additional data input follows the same general principle as used in the network
.dat files input to SATNET; i.e., each section is preceded by a 11111, 22222, etc.
record and terminated by a 99999 record. Data sections must be in increasing
numerical order and not all sections need to be included - indeed no additional
data need be included at all apart from records 7 and 8, the new matrix title.
The set of records must be terminated by a final 99999 record - and note that if no
data is included then a single 99999 record is still required.
These records allow you to “over-ride” either the counts as input on the .ufp file or
else to re-define the type of constraint, e.g., to change an equality into a less than.
RECORD 1.1
Cols. 1-5 11111 start of the change of mind records
RECORD 1.2
One record for each link/turn count to be modified with format:
Cols. 1 - 5 The A node
Cols. 6 – 10 The B node
Cols. 11 – 15 The C node
Cols. 17 The (new) constraint type:
L – Less than
E – Equality
G - Greater than
or
X - Exclude from the calculations
Cols 18-25 The (modified) flow in pcus/hr
The program will search the list of input counts on the .UFP file to find that one
with identical A- B- and C-node values and replace the constraint type and/or flow
with that specified. If the constraint type (col. 17) or the flow is left blank then no
change is made. An unmatched set of nodes results in a non-fatal error.
The X or “exclusion” option is introduced so that if you find that one or more
counts are giving problems then they are removed without the potentially time-
consuming necessity of deriving a new set of PIJA factors.
Note that the definition of A, B and C are identical to that used as input to
SATPIJA (13.2.1 above) but the flow here occupies 8 columns not 5. Note
equally that if DUTCH = T a different format applies and each node occupies 10
columns, not 5; see 15.20).
RECORD 1.3
Cols 1- 5 99999 End of the change of mind records
One record for each link/turn count to be modified with format:
RECORD 2.1
Cols 1- 5 22222 Start of the origin constraints
RECORD(S) 2.2
One record for each origin sum to be constrained with format:
Cols 1- 5 The zone name (N.B. NOT its sequential number)
Cols 7 The constraint type:
L - Less than
E – Equality
G - Greater than
Cols 8 - 15 The desired origin total flow in pcus/hr (as an integer)
If column 7 is left blank the default constraint type as specified by
the parameter LEG is assumed.
Note that if a range of zonal origin values is required (13.1.7) two
records are required with the same zone name in cols. 1-5.
RECORD 2.3
Cols 1- 5 99999 End of the change of origin constraints
RECORD 3.1
Cols 1- 5 33333 start of the destination constraints
RECORD(S) 3.2
One record for each destination sum to be constrained with format:
Cols 1- 5 The zone name (N.B. NOT its sequential number)
Cols 7 The constraint type:
L - Less than
E – Equality
G - Greater than
Cols 8 - 15 The desired destination total flow in pcus/hr (as an integer)
(N.B. Destination flows must be factored up by the user into
“demand” flows as opposed to “observed” flows. See 13.3.2).
If column 7 is left blank the default constraint type as specified by
RECORD 3.3
Cols 1- 5 99999 End of the change of destination constraints
Block definitions; i.e., specification of continuous blocks of zones for which all
associated movements (trips) are to remain unchanged by the estimation process.
Trips to, from and between these zones will be the same in the final and prior
matrices.
Note that it is also possible to define “frozen sectors”, i.e., aggregates of zones
(see 5.1.7), in which case it is necessary to insert an ‘S’ in column 1 under
Records 4.2 and to include the first and last sectors in columns 2-5 and 6-10.
Note that a second S is not necessary and, also, if the second sector is left blank
(or zero) it is assumed equal to the first.
RECORD 4.1
Cols 1- 5 44444 start of the frozen zone records
RECORD(S) 4.2
Cols 1- 5 Start zone name (not sequential)
Cols 6-10 Finish zone of block (greater than or equal to the start zone)
Record type 4.2 is repeated for each ‘frozen’ block of zones.
RECORD 4.3
Cols 1- 5 99999 End of the frozen zone records
Cell list specification - for origin zone sequences a list of destination zones is
given. Trips in the defined cells remain unchanged by the program.
N.B. Both origin and destination zones are specified in terms of their zone names,
not by their sequential numbers.
As with the 44444 frozen zones cells may be defined at the sector level (5.1.7) by
inserting a S in column 1 under 5.2 below, in which case all entries on the line are
assumed to refer to sectors.
RECORD 5.1
Cols 1- 5 55555 start of the frozen cell records
RECORD(S) 5.2
Cols 1- 5 Start origin zone name (/sector if column 1 = S)
Cols 6-10 Finish origin zone name (> = start origin)
Cols 11-15 Number of destination zone pairs to follow
Cols 16-20 Start destination zone name (block 1)
Cols 21-25 Finish destination zone name (block 1)
Cols 26-30 Start destination zone name (block 2)
Cols 31-35 Finish destination zone (block 2)
etc. for further destination zone name pairs
Thus the number of entries to be read following column 15 will be twice the
number that appears in columns 11-15 up to a maximum of 12 per line.
RECORD 5.3
Cols 1- 5 99999 End of the frozen cell records
RECORD 6.1
Cols 1- 5 66666 start of a set of combined constraints (with any “title” following
66666 being displayed within the .LPM file)
RECORD(S) 6.2
Cols 1- 5 The A-node (In columns 1-10 etc. etc. under DUTCH = T)
Cols 6-10 The B-node
Cols 11-15 The C-node (if a turn)
RECORD 6.3
Cols 1- 5 99999 End of a set of combined constraints
24 400
99999
44444
4 5
10 10
99999
55555
14 15 2 4 6 11 13
99999
55555
14
99999
99999
NEW TRIP MATRIX FROM SATME2
First of all it must be emphasised that the ME2 model and programs are still and
always have been to a certain extent experimental in nature rather than
standardised ‘black box’ procedures. It is therefore up to the users to decide upon
their own parameters and procedures in order to get the best results from the
programs. This section offers advice and explanations based on the previous
experience of the authors - suggestions for other points to be included here would
be very welcome.
In addition certain other options which may be invoked within SATME2 are
documented.
The full list of parameters which may be defined in SATME2 is given under 13.2.2
above. Perhaps the best advice is, when in doubt, use the default values.
However the following pointers may help:
ITERMX - The main factor here is run time which is roughly proportional to the
number of iterations. If you are not worried by this choose a large value. If you
are concerned perhaps the best policy is to try a run with a large value of ITERMX
- this should show that the rate of convergence slows rapidly with increasing
iterations, thus helping you to choose a value of ITERMX which represents a
suitable compromise between accuracy and cost for later runs.
SEED - If, equation (13.1), an input value of tij = 0 than the output value Tij must
also equal 0. This is not necessarily what is required. Therefore zero - value cells
may be replaced by a “SEED” value. For example, a SEED value is set when it is
felt that the prior matrix contains an abnormally large number of matrix elements
equal to 0, and that these elements are due more to a restricted survey in which
relatively unlikely O-D movements were not picked up rather than to ‘true’ zeros.
The effect of setting a non-zero SEED is to produce a somewhat ‘smoother’ matrix
although the resulting goodness-of-fit statistics will probably not be very much
improved.
As described in Section 13.1.12 its use is only recommended “in the last resort”
and there are undoubtedly better, although almost certainly more complex ways to
deal with zero cells in a prior trip matrix.
In addition SEED should only be used for the simplest forms of matrix update,
e.g., with a single user class and a square trip matrix. A Serious Warning (which
may well be upgraded soon to a Semi-Fatal Error) results otherwise.
Note that SEED is not used when PRIOR = F since in that case all elements are
effectively seeded with an appropriate average value.
However, whether PRIOR = F or T, seeded values are only set for those IJ pairs
which pass through at least one of the counted links. Hence no adjustment at all
is made to elements in the trip matrix which are, in effect, ‘unobserved’. Equally
seeded values are not applied to matrix cells which are fixed or frozen; see 13.1.5
Note that if SEED = 0.0 and PRIOR = T, (i.e., updating a trip matrix with no
change to zero elements it may be worth setting the parameter ALLIJ in SATPIJA
to .FALSE. so that PIJA’s are not calculated for cells where Tij = 0; this may
considerably reduce the size of the UFP file if there are a large number of such
cells.
XAMAX - In general one would expect the link balancing factors (Xa in equation
(13.1)) in the ‘update’ mode (PRIOR = T) to be somewhere near 1, and a
balancing factor very much different indicates that the corresponding count is very
different from what the original matrix would have given. This may therefore imply
that the count itself is unreliable. By setting maximum and minimum values on the
balancing factors we therefore limit the effect that any one count can have. On
the other hand with PRIOR = F one should probably allow larger values of XAMAX
since in this case we have much less reason to suspect that the initial ‘flat’ matrix
should give reasonable link flows.
It should also be borne in mind, see equation (13.1), that an individual ij pair may
use several counted links so that the maximum change possible in Tij could be
XAMAX to the power n, where n is the number of counted links, each at a factor
XAMAX. There is a case therefore for reducing XAMAX considerably, e.g., to a
value of 2.0 or even less, if you have a large number of counts that could be used
multiple times.
One useful strategy might be to start off with the default value of XAMAX but, if it
seems appropriate, to reduce it for later runs. There is a case for at least seeing
initially what size of Xa factors are required and how many counts require the
maximum/minimum before enforcing a stricter criterion.
SUBFIX - In general this should be TRUE on the assumption that you wish to
update a full trip matrix and that the counts include both assigned and fixed
components. Exceptions to this rule include (a) updating a sub-matrix under MUC
assignment (see 13.4.8 below), in which case it should almost certainly be FALSE
on the assumption that the counts are specifically for one class of vehicles; and
(b) when there are no fixed flows, in which case the value of SUBFIX does not
matter.
SUBPQ – This is a special case of SUBFIX where only the PASSQ element of
the fixed flows is subtracted from the counts. Since the PASSQ flows arise from
queued elements of the trip matrix from the previous time period, not the current
trip matrix which is being updated from counts, they should not be included in the
update procedure. Hence we would recommend, if there are PASSQ flows, to set
SUBPQ = T if SUBFIX is not already set to T. See also 13.4.8.
Note however that some care must be exercised in the definition of the origin or
destination constraints where there are ‘fixed’ flows in the network due, for
example, to bus routes. As mentioned above SATME2 subtracts any bus flows,
etc. from the link constraints since the trip matrix to be estimated only contributes
to the ‘un-fixed’ flows. However no such correction is made by the program for
zonal constraints, in which case users must supply their own corrections prior to
input. For example if a cordon link has an in-bound observed flow of 800 pcu per
hour, of which 60 are due to buses (e.g., 20 buses per hour with a BUSPCU factor
of 3.0) the correct input value as a link constraint to SATPIJA would be 800
whereas it would be 740 as an origin constraint to SATME2.
A second complication is that any constraints for flows into zones in the simulation
network must be converted by the user into “demand” flows to account for the fact
that the flow actually reaching that zone may be reduced by the presence of over-
capacity junctions in the network. Appropriate factors may be determined by
using the “selected zone” option in SATLOOK. These corrections do not apply to
origins, for which there is no restriction due to queuing, or to the buffer zones.
Problems can arise with ‘inconsistent’ observed flows, the simplest case to
visualise being the not infrequent case where the observed flows into a node do
not equal the observed flows out, in violation of “Kirchoff’s Rule” (as originally
applied to electrical circuits). However, more complicated situations may also
arise with two counts (or sets of counts) which are physically separated by
intervening links but where the assignment pattern is such that all flows using the
first count (set) must be routed through the second. A further example of
inconsistencies between counts and assignment was given in 13.1.8. Equally
frozen cells may create problems.
The common thread in all such cases is that it is not mathematically possible to
calculate a trip matrix which will satisfy all constraints.
Similar, but slightly less severe, problems may occur with counts that are, strictly
speaking, consistent in that trip matrices that satisfy equations (13.1) do exist but
which, given the assigned routes, frozen cells, etc., would involve severely
distorting the original matrix.
To a certain extent inconsistent counts as above may simply cancel one another
out. For example, if counts have been taken on the same road above and below a
pedestrian crossing node and they differ – so that Kirchoff’s Rule is violated at the
pedestrian crossing - the higher flow will continually attempt to increase the flow
through it by increasing its Xa factor until it reaches the maximum allowed value
XAMAX while the lower flow will go in the opposite direction until its Xa factor
reaches the minimum 1/XAMAX. The two multipliers will then cancel one another
out so that, in effect, the two counts have been removed. At more complex
intersections the end effect may not be that “simple”.
Secondly, within SATPIJA, if a node has, counts on all its input arms and on all
but one output arm (or vice-versa) then the flow on that output arm is equally
constrained. In these cases SATPIJA compares the “implied” count with the
currently assigned link flow and reports as a Serious Warning those cases where
the differences between the two flows give rise to a GEH value greater than 5.0
and where, presumably, SATME2 would “struggle” to find a consistent solution.
Note that the Kirchoff tests are carried out using the “assignment network
definitions” (see 5.5.1), i.e., with simulation nodes expanded and individual
assignment nodes at either end of simulation roads. This implies that flows to and
from zones are included in the summations of flow-in and flow-out.
Clearly simple factoring up/down Kirchoff violation flows does not help in the
situation where one of the counts is a “rogue” and the others are all genuine.
There are also potential problems of “interactions” whereby a single count might
be adjusted at both its upstream and downstream nodes and the two are
contradictory.
Nor does AVERK deal with problems whereby the inconsistencies arise from a
combination of the counts, the assigned OD routes and the prior matrix.
Ultimately it must be up to the user to detect problems and to adjust the counts to
more correct and consistent values.
1) Basic statistical variability in the counts. This arises if counts have been taken
on different days or even different years (and presumably factored to a
common year), in which case the degree of inconsistency is relatively small
(and corrections using AVERK will probably deal with it).
2) Link transcription errors. It may happen that a link count is attributed to the
wrong link, one example of this being a count that was taken on an on-ramp
leading to a motorway and mistakenly coded as a count on the motorway.
4) Network coding errors. If a junction has, say, two links in and two links out
with flows of 500 on each there is no problem. If, however, one of the entry
links had been left out there will be a Kirchoff violation because we have an
entry flow of 500, an exit flow of 1,000 and no other entry arms to which the
missing flow of 500 can be attributed.
One perhaps useful piece of empirical advice is to try to put the most ‘reliable’ or
‘important’ counts near the bottom of the input list since the last counted links are
balanced last in the iterative procedure.
Similarly, if the network for which trip matrices are being estimated is part of a
PASSQ multiple time period model, then any residual queues at the link stop-line
from the previous time period are also modelled as entry flows at the stop-line
(see 17.3.1). Hence they also create problems in unambiguously defining flows
and counts on links. It is possible, post 10.9 and using the parameter SUBSLQ
(see 13.3.1), to subtract the residual queues from the counts or not depending on
whether the count is assumed to be taken at the stop-line itself or mid-link
(upstream of any residual queues).
However, given these ambiguities there is a strong case for not including such
counts within ME2 on the basis that they are not actually adding information about
O-D travel patterns but simply confusing the issue. In addition, if the link continues
to queue in the current time period (or, strictly, is modelled to queue) then it could
be argued that the count is actually giving you more information about the link
capacity rather than flows and should be used to calibrate the link capacity (e.g.,
by adjusting saturation flows) rather than within SATME2.
N.B. None of the above problems apply to counts on turning movements where it
is assumed that the counts are taken at the junction. However see 13.3.8 for a
discussion as to the advisability of using turn counts in the first place.
From version 10.5 on it is a fatal error to use a prior .ufm matrix which was itself
created by a run of SATME2 – unless the namelist parameter ENCORE is
explicitly set to .TRUE, within the control file (13.2.2). (There is, as yet, no check
that the input prior matrix may have originally come from SATME2 but have been
modified by another program(s) in between.)
It is highly recommended that users make use of the ‘PRIOR’ option within the
SATME2 command line (see 13.5.2) to explicitly define the prior matrix each time
SATME2 is run since otherwise it is possible that, if PRIOR = T, the default is to
use the trip matrix as originally used in SATPIJA which, in turn, may have
defaulted to the trip matrix used in the .ufs file input to SATPIJA. Which is to say,
the last matrix used in an assignment which, as explained above, is not
necessarily what is desired. The “ENCORE” parameter may protect against this
happening but not necessarily.
O-D pairs which do not use any of the counted links in the network can, in effect,
be given totally arbitrary values since these values will not affect the observed
counts. However the following assumptions are made by SATME2: (i) In the
update mode (PRIOR = T) any such elements retain their input value, and (ii) with
PRIOR = F they are set to zero. Hence in the latter case we make the implicit
assumption that an unobserved element is in fact an impossible trip.
inconsistencies in SATME2 if, for example, you have a counted flow of, say, 1500
pcu/hr immediately downstream from a link whose (SATURN - estimated) capacity
is 1000 pcu/hr - either the count or the capacity is wrong. In fact this is an extreme
example of the effect of flow metering on counted flows (see 13.1.10) where, in
this case, the flow metering affects 100% of the traffic and occurs immediately
upstream.
The practical problem that this gives rise to in SATURN is one of potential
exponential growth in the demand trip matrix if one iterates between matrix
estimation and assignment. For example, in the above case imagine the
bottleneck and count are fed by a single OD pair whose initial cell value is 2000 -
of which 1000 will queue at the bottleneck and 1000 will reach the counted link.
Therefore on the first iteration only 50% of the demand reaches the count and
therefore the “demand target” is factored up from 1500 to 3000 - see 13.1.3. To
achieve this demand target the first run of ME2 increases the trip matrix to cell to
3000. However on reassigning the new matrix again only 1000 pcus - of the
total - reach the count, so that the new demand target becomes 4500. Each
succeeding iteration continues to increase the trip matrix without ever satisfying
the count.
The short-term solution to the above problem is clear – either remove the count or
remove the bottleneck. The long-term solution may well be to enable the model to
recognise the situation and take appropriate action.
SATME2 will accept counts which correspond either to a “pure” road link or to
“turning movement” links (within simulation networks). Clearly if you have counts
on all exits from an in-bound link then you also have the total flow on the in-bound
itself. Equally clearly using both sets of count data – link plus turns - as inputs to
SATME2 is “double counting” since if the turning flow constraints are satisfied
then so too is the link constraint. In this case you need to use one or the other,
Turn counts have the advantage of containing extra information which may help
SATME2 in differentiating between different o-d pairs to be factored. On the other
hand the extra level of disaggregation in turn counts means that they are probably
less reliable, statistically speaking, in which case they may do more harm than
good.
Generally speaking, our advice would be, if you have the choice between using
links and/or turn data, to at least start by using the link data and then, if the
changes seem sensible, add the turn data. If the outputs seem “even more
sensible”, keep them in; otherwise go back to the pure link constraints. A warning
message appears in the output .lpj file from SATPIJA to warn you if both
simulation link and turn counts are used.
The SATPIJA control parameters SET777 (see 13.2.1) may provide a useful
mechanism in this context to include/exclude turn counts. Thus, if you put all link
counts within the first 77777 data set and all turn counts in the second, then
setting SET777(1) = T and SET777(2) = F will automatically include only link
counts in SATME2 but still leave you the possibility to use the turn counts for
other validation purposes.
Alternatively you could use the mechanism for combining counts together (13.1.8)
to aggregate turn counts into link counts but, given the extra effort of specifying
the combined constraints within SATME2, it is probably better to do it in other
ways.
As a general rule it is probably best (if possible) to select sites which lie “in
parallel” rather than “in series”, for example, along well defined continuous screen
lines or cordons rather than both entry and exit links at a single junction. Thus a
cordon guarantees that all o-d pairs with an origin on one side of the cordon and a
destination on the other must cross at least one count site and each cordon point
contains a different set of O-d path flows.
By contrast, links which are “in series” give information on the same path flows
which is either repeated or contradictory. For example, if link A-B feeds directly
into B-C with no cross links then the same path flows use each and VA-B must
equal VB-C. If the counted flows on both are the same then there is no new
information in the second count; if they differ then there is clearly no possible
solution that will satisfy both. Having counts on a link and all its exit turns (13.3.8
above) is another example of the same phenomenon. A “warning” is generated in
SATPIJA if any examples of links “in series” are found and a list of the “middle
nodes” is given.
A random selection of sites is not recommended since they are neither likely to
include continuous cordons nor are they likely to exclude links in series
A not uncommon problem with matrix estimation is that of having too many counts
rather than too few, coupled with the problem that certain counts may be
statistically more reliable (and/or useful to the estimation process) than others.
Since there is no mathematical option within SATME2 for weighting different
counts according to their reliability (all constraints as embodied in equation (13.1)
are effectively equal) some method for selecting an optimum sub-set must
therefore be devised.
Thus, it may be best to start with a relatively small number of “good” counts and, if
no problems are encountered, to build up the number of counts as long as they
One obvious strategy, if the available counts have been obtained using different
methods (e.g., automatic counts versus manual classified counts) over varying
time periods such that, a priori, certain counts should be more reliable than others,
is to start with the most reliable counts. However there are also competing
considerations, e.g., that counts should be in parallel rather than in series as
discussed above (13.3.9) and that it is useful to have counts spread over the
whole network rather than, say, concentrated in a city centre where the most
reliable counting methods have been employed.
There is also a reliability issue related to the “order” in which counts are input as
discussed in 13.3.3.6; essentially there is a case for putting the most reliable
counts at the bottom of the input lists so that they are more likely to be exactly
satisfied.
An alternative (and not necessarily highly recommended) strategy for dealing with
less reliable count constraints – or counts which are proving difficult to satisfy – is
to convert them into less than / greater than constraints (13.1.7). For example, if
one has a not very reliable link count of 1,000 which empirically appears to be
difficult to satisfy even with Xa at the maximum value of XAMAX it may make
sense to replace it by a >=900 constraint. However the problem with replacing all
unreliable counts with GT/LT constraints is that: (a) one does not know a priori
which alternative should be selected and (b) the problem with a GT/LT constraint
is that even if ME2 can easily satisfy the exact constraint with a GT/LT constraint it
simply stops when it hits the upper/lower boundary.
Trip length distributions may be compared between the original (prior) and output
trip matrices by defining:
a) a .UFM matrix to define “length” (e.g., cost, distance, time etc.); and
b) a “bandwidth” for the distribution. Both are set under &PARAM inputs in the
control file (13.2.2). The output is the form of a numerical table giving both
the frequency and cumulative distributions plus average trip lengths.
These outputs may inform the user as to whether the ME2 adjustments are
producing more short and/or long distance trips.
A question of interest with ME2 is how much impact each individual count
constraint has on the final output trip matrix or, viewed in reverse, how sensitive
the trip matrix is to each count. Table 10 in the .LPM files, calculated after all the
ME2 calculations have taken place, lists a number of relevant statistics per
constraint and defines three different “sensitivity ratios” which may help to identify
“problem counts”.
LINK Flows IN / OUT / Diff (DF): The first two numbers give the total (demand)
flows assigned to the link using the original trip matrix and the output trip matrix.
Their difference, DF, represents the net impact of running ME2 on this link.
TIJS IN / OUT: This column gives the sum of all Tij which are assigned to the link
in question independent of the proportion assigned per path (i.e., the sum of Tij
where Pija > 0 whether Pija = 0.999 or 0.001). These numbers are always, by
definition, greater than those in the previous column by virtue of the fact that the
former are all reduced by, in effect, the Pija factors. The ratio of, say, Flows(IN) /
TIJS(IN) would represent a weighted average Pija factor for that link.
Their difference, DT, is a measure of how much the trips associated with link A
have been adjusted as a result of running ME2.
However, a conceptual problem with DT, and therefore with DT/DF, is that DT
includes all Tij cells using a link without regard to their level of usage, i.e., Pija; the
next two measures take usage into account.
Cumulative Changes to Tij : the summation of all changes to Tij throughout the
internal ME2 algorithm iterations each time Xa is adjusted.
T = Tij (1 – Xa –Pija )
Note that a negative value of T indicates that (using) the count acts to reduce the
total trip matrix by this amount and, conversely, that a positive value indicates that
the count increases the matrix by this amount.
Arguably this represents the best measure available to asses the effect of using
the count but it does not tell us exactly what the effect of not using the count
would be. For example, in an extreme case, if we have the same count on two
consecutive links separated by a pelican crossing, it is essentially arbitrary which
count has an Xa factor of 1.0 (i.e., is “inactive”) and which has an “active” Xa factor.
If we remove the active count the second would become active and the updated
matrix would be the same.
Equally other counts may be actively factoring the same cells in an opposite
direction, thus negating the effects of this particular count. If we were to re-run
matrix estimation without the count then the factors for these other counts would
be slightly closer to 1.0 since they would not have to “work so hard” to counter the
effect of the count that had been removed.
Since, for Frank-Wolfe assignment at least, the analysis of PIJA factors is based
on the routes as calculated by the “SAVEIT assignment” as opposed to the
“proper” assignment there is a possibility that, due to a lack of proper convergence
between the two, the flows through the counted link identified by the PIJA factors
by SATPIJA will not agree with the actual assigned flows (see Section 15.23.2).
This means that SATME2 will subsequently be in error as it will be trying to match
the “wrong” flows.
Most of the time any errors due to this effect will be small compared to all the
other potential errors but, sometimes, they may prove to be significant.
Post release 10.8.20 the potential for such errors has been assessed in SATPIJA
by taking a GEH difference between the total actual assigned flows and the
SAVEIT assigned flows for each counted link. If GEH exceeds 5 this is identified
as a Serious Warning; if GEH exceeds 1 (but is less than 5) it is a Warning. The
total number of both Warnings and serious Warnings are included within the error
messages summary at the end of the .LPJ file which is also passed to SATME2.
We note, however, that post 10.9 the errors in the SAVEIT assignment may be
reduced to effectively zero by setting UFC109 = T in the network .dat file; see
15.23.3. In certain circumstances this has proved to be extremely beneficial
(although the downside is that it may increase the CPU time required by
SATPIJA).
As noted in 7.3.2.1 there are several ways in which MUC trip matrices may be
specified.
The first and simplest – though least common – situation is to have a single
(square) trip matrix where each user class OD flows are a fixed proportion of the
total flows as specified within the 88888 data records on the network .dat file (see
Section 6.11).
The second situation- at the other extreme and probably the most common – is
where each user class has its own distinct trip matrix.. In this case, each of these
trip matrices are “stacked” into “levels” in a single .UFM file. Further details may
be found in sections 13.4.3 and 13.4.4 respectively.
The third, intermediate situation is a combination of the previous two whereby the
some user classes share the same sub-matrix/level whilst others have their own
exclusive matrix/level. For example, a 5-UC model might consist of three matrix
levels
the first matrix level representing car trips split by fixed proportions into three
purposes (say, Work, Education and Other);
Ideally vehicle classes (see 5.8.2 and 6.11) should be defined for each of the five
user classes to distinguish cars, LGVs and HGVs (see 13.4.4 and 13.4.5).
Alternatively, we could, in the above example, have three separate stacked matrix
levels for car trips instead of the single one, say Work, Education and Other,
which are not simple proportions of a common car trip matrix. However, assuming
that we could only observe total car flows, only the sum of the three matrices may
be updated, not the individual matrices (see 13.4.6).
In order to apply ME2 in the above 5-UC three-matrix example we would need
observed link counts disaggregated by car/LGV/HGV; each separate set of counts
would be used to update the corresponding trip matrix level. But, unless we had
some way of distinguishing three separate purposes for observed car counts, it is
not possible to apply separate matrix estimation by purpose.
Equally, if we had three separate stacked matrices for car trips by Work,
Education and Other which are not simple factors of a common car trip matrix and
we can only observe total car flows, only the sum of the three matrices may be
updated, not the individual matrices (see 13.4.6 below).
Therefore, in terms of MUC ME2, we shall assume that we have a set of one or
more prior trip matrices, each one of which constitutes a separate level within the
assigned stacked trip matrix or is the sum of two or more UC/levels, and a
corresponding set of observed flows which we wish to use to produce improved
estimates of those matrices in order to satisfy equation (13.1).
13.4.2 SATPIJA Options for Defining User Classes: KLASS, LEVEL and IVC
Within SATPIJA the set of one or more user classes for which Pija factors are
required are set using the three parameters KLASS, LEVEL and IVC:
b) All user classes based on the same “level” of the input stacked trip matrix (set
by LEVEL)
c) All user classes which share a common “vehicle class” (set by IVC).
In cases (b) and (c) the output PIJA factors are “weighted” averages of the user
classes within that level; see equations (13.5) and (13.6) respectively. Under (a)
the value of LEVEL is effectively set by the matrix level used by user class KLASS
as defined in the .UFS network file.
Option (a), to analyse a single user class KLASS, is not always appropriate since
it may be difficult to associate counts with a particular user class. For example, if
you had car trips split by purpose into separate sub-matrices, each with its own
level, it might not be possible to similarly disaggregate observed counts of cars by
purpose/level. However, this would not apply if, say, HGVs were specified as a
single user class / level and HGV counts could be easily identified separately.
Note that defining a single value KLASS is only possible if the trip matrix for that
user class is defined in a discrete level; if it “shares” its matrix level with other user
classes it is not possible to do a “partial” update of that level.
In the most common case of a single trip matrix all three parameters LEVEL,
KLASS and IVC are redundant; in effect, all three equal to 1.
Let us first consider the simplest case where there is just a single “square” trip
matrix to be assigned and the various user classes are defined as shared
fractions (weights; see Section 6.11) of that matrix. Equally we have a single set
of observed counts corresponding to the total assigned flows. The problem is
therefore essentially identical to that of a single user class.
Equation 13.5
P ijaW
m m
P ija
m
where Pijam is the normal Pija factor for user class m and Wm is the “fraction” of
the trip matrix associated with user class m as defined within columns 11-15 of the
88888 data records within the original network .dat file
All matrix files shown in Figure 13.2 are simple square trip matrix files (i.e., not
stacked) and SATME2 updates that single trip matrix.
13.4.4 Multiple Level Stacked Trip Matrices: Updating Levels via Separate Matrices
In a more complicated case the trip matrix may contain several different sub-
matrices stacked by level with distinct user classes being defined as universal
factors of a single level (where the factors are input under the 88888 data in a
network .dat file). Most commonly a particular level will correspond to a single
user class, in which case the factor should be 1.0; less frequently a level will
contain multiple user classes, in which case the sum of the factors should be 1.0.
In the former case updating a level is fully equivalent to updating a single user
class.
a) Users may first unstack the trip matrix into individual square trip matrices,
update each sub-matrix/level individually and finally re-stack them; or
b) Users may (post 10.8) update an individual level within the full stacked matrix
while leaving the “other” levels unaltered.
Both cases require separate runs of SATPIJA to calculate PIJA factors and
SATME2 to update the matrix for the designated level; i.e., it is not possible to
produce multiple sets of Pija factors using SATPIJA and/or to update multiple
matrix levels within SATME2.
Case (a) is described in more detail below; case (b) in the following sub-section,
13.4.5.
The level is identified by the value of LEVEL in Control File A as input to SATPIJA
(See note (5), 13.2.1). Note that if a level contains only one user class it is
equally possible to identify the level using the parameter KLASS.
The counts MUST refer to the total flows corresponding to that level or sub-matrix
(and thus SUBFIX should almost certainly be set to FALSE in SATME2 to avoid
subtracting fixed flows; see 13.4.8). For example, if you had stacked a Car and
an HGV matrix into a stacked two-level matrix file you could identify all car trips on
selected links and use that PIJA data plus car-specific counts to update the Car
matrix.
In this case the prior trip matrix input to SATME2 is the single-level un-stacked
car-only prior matrix, the matrix output from SATME2 is an updated car matrix
(also un-stacked) whereas the matrices input to SATURN and/or SATPIJA must
be stacked matrices (since in order to carry out the full assignment both matrices
are required). Thus it is necessary to insert one or two extra steps in the update
process:
Prior to running SATME2 un-stack the full prior trip matrix to produce one or
more square sub-matrices for the level(s) to be updated (N.B. Not necessary
if the stacked prior matrix was itself built from sub-matrices in the first place);
After obtaining a new Car matrix re-stack it with the (current) HGV matrix to
produce a new stacked matrix to use in SATURN.
Having updated the Car matrix to match the counts you could of course then carry
on to update the HGV matrix (with no doubt some extra problems of feedback and
convergence in addition to those described in 13.1.10 since updating the car
matrix may affect the routes taken by both cars and HGVs which will in turn affect
the ME2 HGV calculations!). The main point is that both steps must be carried out
separately.
Post version 10.8 it is possible to run SATME2 using stacked trip matrices for both
the input and output files and to update only one individual level at a time within
the stacked matrix without having to first unstack them as described in 13.4.4.
The end result is identical, the only difference is that the procedures described
below are easier to implement.
As with 13.4.4 first run SATPIJA to produce a .UFP PIJA file for the level of the
trip matrix to be updated and then run SATME2 to update that level.
The value of the level to be updated is set by the parameter LEVEL as input to
SATPIJA and passed to SATME2 via the .UFP file.
The procedure is then repeated once per level (or once for each level which is to
be updated). N.B. As yet there is no automatic procedure to carry out a PIJA
analysis for all levels in SATPIJA and then to update all matrix levels with
SATME2. However it should not be difficult for clever users to set up their own
batch files to run the procedure automatically - further advice is available upon
request.
Note that one very good reason for not putting the complete procedure into a
single “black box” is the user needs to be making decisions after each run of
SATME2, for example whether to repeat the assignment immediately, whether to
repeat the run of SATME2 but with certain counts removed, etc. etc.
For example, if we have a three-level matrix and we first apply ME2 to level 1 we
can create an output matrix which consists of the updated level 1 plus the original
levels 2 and 3 as copied from the prior matrix. However when we apply ME2 to
level 2 we will want the output matrix to contain the updated versions of both
levels 1 and 2. In order to do so we use the first output matrix as an input to the
second ME2 process, copy levels 1 and 3 from that matrix to the output and add
the updated version of level 2.
Hence for each run of SATME2 we need two input trip matrices – the prior matrix
(which should never change) and a “current” trip matrix from which we copy the
levels other than the one being updated. Clearly the output matrix from run n then
becomes the input update matrix for run n+1. (But note, as always, that the prior
matrix never changes.)
For the very first run of SATME2 the prior matrix may be copied into a “current”
trip matrix to be updated but thereafter the prior and update matrices should be
totally different files.
The filename of the copy/update matrix is specified via the Command Line using a
“key word” TIJUP; see 13.5.2. An example of the Command Lines would be:
For updating level 1: SATME2 newod1 counts KR me2kr PRIOR priorod TIJUP newod0
For updating level 2: SATME2 newod2 counts KR me2kr PRIOR priorod TIJUP newod1
For updating level 3: SATME2 newod3 counts KR me2kr PRIOR priorod TIJUP newod2
where newod0.ufm might be a straight copy of priorod.ufm (but, N.B. it must have
a different filename).
13.4.6 Multiple User Classes Aggregated by Vehicle Class: IVC and TURBO
This section describes the situation where, say, you have three separate matrices
of car trips disaggregated by purpose – e.g., Work, Education and Other – each
stored as a separate matrix level for assignment purposes with, possibly, different
values of PPM, PPK etc. and therefore different route choices. In addition you
may have link counts and/or other constraints based on total cars, i.e., not
disaggregated by purpose.
In this case it is possible (post 10.8.13) to use SATME2 to update a prior matrix
which is the sum of all three car trip sub-matrices (and therefore a square matrix)
and to subsequently split the updated matrix back into separate matrices for Work,
Education and Other. (The method of splitting may either be left entirely up to the
Thus, with reference to equation (13.1) Tij is the total car trip matrix, Pija is the
probability that a car trip from origin i to destination j uses link a and Va is the total
car flow on link a. Thus Pija is a weighted sum over all purposes defined by:
Equation 13.6
P ija P ij
m m
P ija
m
where P ijm is the probability (i.e., split) that a car trip from i to j is by purpose (user
class) m.
In order to calculate Pija within SATPIJA it is first necessary to ensure that the all
the sub-matrices (i.e., levels of the stacked matrix) which are to be treated
together have been grouped within the same “vehicle class” using the 88888 data
records of the network .dat file (see 6.11). The namelist parameter IVC identifies
the vehicle class to be used and must be invoked as the only option which
defines multiple levels within SATPIJA
While the matrix updating procedures within SATME2 are always applied to a
square trip matrix (internally) the updated (square) matrix may be either output
directly as a square matrix or proportionately divided between the matrix levels
and output as part of a stacked trip matrix depending on whether a parameter
TURBO wais set to F or T within the SATPIJA &PARAM records. (TURBO =
turboprop – read on!)
Thus, if TURBO = F (its default), SATME2 requires as input a “square” prior matrix
which is the sum of all car trip matrix sub-levels and outputs an updated total car
trip matrix which is also square. Note that neither the prior nor the updated trip
matrices are used directly within the traffic assignment.
It is the responsibility of the user to (a) create the square prior matrix by
aggregating together the relevant sub-levels of the stacked trip matrix and (b)
splitting the output square matrix into sub-matrices by purpose and creating a new
stacked trip matrix prior to re-assignment ( the exact method for which is left to the
user).
On the other hand if TURBO = T then (a) the full stacked matrix is input into
SATME2 and the program automatically aggregates the appropriate sub-levels to
create a square prior matrix and (b), the updated square matrix is automatically
split by purpose/level and output as a stacked trip matrix. The constituent levels of
the output matrix are calculated proportionately via the equation:
Where tij(m = the input prior trips from I to j for matrix level m
N.B. The value of TURBO has no affect within SATPIJA as the weighted Pija
factors are calculated in exactly the same way whether it is T or F; the differences
only occur within SATME2.
“Auxiliary” input matrices such as FILICE used to define the frozen cells and
FILTLD used in the trip length distributions may also be used under MUC.
However, unlike the main trip matrices, they may be either simple square matrices
or stacked. If square it is assumed that it represents the UC/level required, if
stacked the appropriate level is read.
On the other hand all the main trip matrices, e.g., the prior matrix, the update
matrix, etc., must all have the same number of rows and columns and hence the
same number of levels.
Generally speaking, when SATME2 is being used to update a stacked MUC trip
matrix, e.g., one that contains both cars and HGVs, it would be expected that the
input counts would refer either to cars or HGVs on their own and that it would not
be necessary to remove, say, bus flows which had been included in the counts.
Hence the use of the SUBFIX option (13.1.4 and 13.3.1) should generally not be
necessary and SUBFIX should be explicitly set as F (i.e., changed from its default
of T).
On the other hand, if you are using PASSQ, it will be virtually impossible for the
counts to have differentiated between vehicles from this time period and previous
time periods, in which case there are very cogent reasons for making use of the
SUBPQ option in order to subtract the UC-specific PASSQ flows from the target
counts.
There is, however, a technical problem here in that the queued-up flows from the
previous time period as recorded under PASSQ are the total queued flows and
are not disaggregated by, e.g., user class. In order to correct for this problem
when one is updating only one particular level / user class in the stacked trip
matrix SATPIJA calculates the total PASSQ flow per counted links and then
factors it down by the ratio of the user class flow relative to the total flow on the
link in the previous time period. This information is then passed to SATME2 via
the .UFP file. Clearly this is only an approximation but, given the normally
relatively small contribution of PASSQ flows to total flows, it is felt to be justified.
A similar correction is also made to initial residual queues which are subtracted
from the target flow if SUBSLQ = T (see 13.3.1).
N.B. If, for some strange reason, the PASSQ-ed network had a different link
structure from the “current” network then, post 10.9.19, the flows are correctly
“translated” from the old structure into the new. Previously this was not possible:
SUBPQ was set to F in SATME2 and a Serious Warning generated.
To invoke this option it is necessary to define: (a) the number of segments NPART
into which the origin zones are to be divided and (b) which subset IPART a
particular run will process (where 1 <= IPART <= NPART). For example, if a
network has 55 zones and NPART = 4 each segment processes 14 zones
(rounding up) such that the first segment (IPART = 1) contains zones 1 through
14, the second, 15 through 28, etc. etc. Note that the final segment may contain
fewer zones than the rest (13 in the above example). NPART and IPART are set
as Namelist parameters in the control file for SATPIJA or, alternatively, as
command line parameters (see 13.6.2). The process can be controlled from a
single batch file SATPIJA using n#N on the command line (see 13.6.2).
If, however, IPART = 0 but NPART > 0 then SATPIJA operates in an “aggregation
mode” such that all the sub-UFP files created by each zonal segment are read in
and copied into a single .UFP file which contains the PIJA data for all zones as
per normal.
Technically each sub .UFP is identified by a suffix _PART such that (see 13.6.2)
the standard filenames would be counts_1.UFP, counts_2.UFP … whereas the
aggregate .UFP file would be named counts.UFP.
Notes:
(i) PRIOR is only used when the program is run in the “update” mode. Since the
default file SATME20.DAT sets the update mode if KR is absent then PRIOR
must be used in this case
If the parameter PIJAKP is set to T SATPIJA will output an ASCII “card punch” file
containing the same data as in the .ufp file but in a form by which the data may be
read by, e.g. a spreadsheet. The format is as follows.
Each counted link/turn contains a header record plus one or more pija records
where each record is in “comma delimited” format, i.e. each entry separated by a
comma from its neighbours with no spaces.
A-node.
B-node.
Each data records consist of up to 50 pairs of data entries where each pair gives:
where nz is the number of zones. Note the first entry is an integer, the second
real.
Thus if a particular link had 123 Pija “hits” it would occupy 4 records in total, the
header, 2 data records each containing 50 integers and 50 reals, plus a final
record with 23 of each.
Equation 13.7
and Pija is the fraction of Tij trips which pass through link(/turn) a. Figure 13.4
illustrates the order of programs (cf. Figure 13.4).
Once produced the Tija matrix may be printed using MX or perhaps re-assigned
using SATDB. However given the probable “sparsity” of the matrix most users
probably prefer to aggregate the “zonal” trip matrix into, say, a “sector-to-sector”
matrix using MX.
SATU2 may produce more than one matrix in a single run, although the links or
turns need to be defined first via the input to SATPIJA (Figure 13.4).
Note that the same trip matrices as produced by SATU2 may now also be
obtained using the Selected Link Analysis options within SATDB - see Section
11.10.7.5 and 15.19. In fact SATDB offers a better means of analysing selected
trip matrices in that:
It also allows the user to determine the links used by the selected trips before
and after using the selected link.
Currently ME2 files contain one record per input count with the following data per
count:
The (demand) target flow (i.e., with fixed flows etc. removed);
The end of the data file is indicated by a final record containing (as is standard in
SATURN data files) 99999.
Note that it is very likely that the contents and precise format of the above file will
change over time as further needs for post-ME2 analysis are identified. For
example, the current files do not contain any information on zone constraints so
these may be added at a later date.