Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p.

8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

Fuzzy Inference Systems: an integrated modelling

environment for collaboration between expert
knowledge and data using FisPro
Serge Guillaumea,, Brigitte Charnomordicb
a
Irstea, UMR ITAP, BP 5095, 34196 Montpellier, France
b
INRA-SUPAGRO, UMR 729 MISTEA, F-34060 Montpellier, France

Abstract
The present paper aims to demonstrate the interest of fuzzy inference
systems in system modelling when human interaction is important. It dis-
cusses the originality of FIS and their capability to integrate expertise and
rule learning from data into a single framework, analyzing their place rela-
tively to concurrent approaches. An open source software implementation is
presented, with a focus on the useful features for modelling. Two real world
case studies are presented to illustrate the approach and the software utility.
Keywords: cooperation, rule learning, fuzzy logic, knowledge
formalization, real word applications, open source software

1. Introduction
Fuzzy logic, since the pioneer work by [59], has proved to be a powerful
interface between symbolic and numerical spaces, especially by the means of
the linguistic variable concept [60].
Forty years later, fuzzy inference systems (FIS) have become one of the
most famous applications of fuzzy logic. One of the reasons for this success is
the ability of fuzzy systems to incorporate human expert knowledge with its
nuances, as well as to express the behaviour of the system in an interpretable
way for humans.
Historically, the first kind of fuzzy rule based systems focused on the
ability of fuzzy logic to model natural language [42].

Email address: [email protected] (Serge Guillaume)

Published in Expert Systems with Applications February 27, 2015

Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

A second approach, proposed in the mid eighties [54], was based on au-
tomatic learning from data. As discussed by [34], fuzzy methods in machine
learning and data mining have been thoroughly investigated and proved use-
ful.
With these new developments, the age of expert knowledge-based fuzzy
systems was over, and the data driven rule generation methods played the
main role in fuzzy system design, moving away from the philosophy of fuzzy
set theory initially directed to bridge the gap between human reasoning and
machine processing, which can be summarized as Computing with words.
This drift has been clearly stated by [19]: ”Fuzzy controllers, and fuzzy
rule-based modelling which have become the most popular and visible side
of applied fuzzy set theory, are only the emerged part of the fuzzy iceberg,
and as time passes this technology seems to owe less and less to fuzzy set
theory itself, and mainly becomes a tool for approximating functions.”
As a reaction, fuzzy logic researchers developed a new field of research. As
shown in [29], the use of fuzzy formalism is not sufficient to ensure the inter-
pretability of a knowledge base. Three conditions have to be fulfilled. First,
semantic integrity should be respected within the partition. Secondly, the
number of rules should be small. The third condition is specific to complex
systems with a large number of input variables: rules must not systemat-
ically include all input variables, but only the important ones in the rule
context. This kind of rules is usually referred to as incomplete rules. As
the interpretability constraints may conflict with the numerical error mini-
mization objective of automatic learning methods, several works have been
carried out to propose a trade off between interpretability and accuracy [13].
Though these efforts allow to propose FIS as an integrated framework for
system modelling, to get the most out of expert knowledge and data, not
enough attention as deserved has been paid to the potential interest of this
approach. FIS are still mostly used as predictive models, their performance
being evaluated in terms of accuracy, and the behavior of their inference
engine is rarely discussed.
Operational, in a process/system context, denotes a working method or
a philosophy that focuses principally on cause and effect relationships (or
stimulus/response, behavior, etc.) of specific interest to a particular domain
at a particular point in time. In this sense, a rule system is a good way
to model a specific kind of knowledge, the so-called operational knowledge.
Fuzzy concepts, whose content, value, or boundaries of application can vary
according to context, operator and conditions, instead of being fixed once

2
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

and for all, arise naturally in the operational approach of system modelling,
hence the relevance of FIS for that matter.
The work presented here aims to be a contribution to this research trend,
both asserting the need for fuzzy logic in system modelling, and proposing
an advanced software for exploratory design and analysis of FIS, allowing to
understand the system operation and its sensitivity.
Its objectives are i)to discuss the interest of FIS as modelling tools with
imposed interpretability constraints in methods and software implementa-
tion, ii)to compare, using the case studies, the FIS with other models (e.g.
statistical multiple regression) and to analyze two inference mechanisms: im-
plicative rules and conjunctive rules. An open source software implementa-
tion of FIS design and optimization is proposed in FisPro 1 , which corresponds
to ten years of research and software development in the field of learning in-
terpretable FIS from data.
Detailed illustrations will be provided through two case studies of real
world problems in Agriculture and Environmental modelling, a field where
expert knowledge and data often complement each other.
The structure of the paper is as follows: Section 2 recalls FIS principles
and analyses the specific role of fuzzy logic in system modelling. Section 3
explains the ways of cooperation between knowledge and data in a modelling
approach based on FIS. A state of the art of fuzzy software and FisPro main
features are presented in Section 4. The case studies are presented in Section
5. The first one deals with pesticide loss modelling in agricultural spraying,
the second one illustrates an inference mechanism which, though little used,
allows to model logical constraints, and in this way, is closer to classical logic
than the inference mechanism used for fuzzy controllers. Finally a conclusion
is given with some perspectives in Section 6.

2. Is there a need for fuzzy logic in system modelling?

In a recent paper, [61] himself asks the somewhat provoking question Is
there a need for fuzzy logic? He gives general answers by highlighting some
unrecognized features of fuzzy logic.
The current work discusses a special aspect of this same question centered
on system modelling with FIS. A system is taken in the very general sense of
an object including inputs, outputs and a model to map inputs onto outputs.

1
http://www.inra.fr/internet/Departements/MIA/M/fispro/

3
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

In this section we first recall the general FIS structure, then we examine
some concurrent approaches before explaining the originality of FIS.

2.1. FIS structure

This section only focuses on some specific points useful for the proposed
framework. For general details about FIS the reader may refer to [40, 18].
A typical fuzzy inference system is shown in Figure 1.
Fuzzy rule Defuzzification
base crisp
Small Average Large output
µA(x) If...Then y^
µL(x)
x
crisp
input µ output
Fuzzification Inference engine
x

Figure 1: A fuzzy inference system (FIS)

2.1.1. Variable partitioning

The readability of fuzzy partitioning is a pre-requisite condition to build
an interpretable rule base. The necessary conditions for interpretable fuzzy
partitions have been studied by several authors [50, 16, 26]. Let us recall the
main points:
• Distinguishability: Semantic integrity requires that the membership
functions represent a linguistic concept and different from each other.
• A justifiable number of fuzzy sets.
• Coverage: Each data point, x, should belong significantly, µ(x) > , at
least to one fuzzy set.  is called the coverage level.
• Normalization: All the fuzzy sets should be normal.
• Overlapping: All the fuzzy sets should significantly overlap.
These requirements are all fulfilled by the strong fuzzy partitions, illus-
trated in Figure 2, such as:
( P
∀x µf (x) = 1
f =1,2,...,m (1)
∀f ∃x µf (x) = 1

4
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

where m is the number of fuzzy sets in the partition and µf (x) is the mem-
bership degree of x to the f th fuzzy set. Equation 1 means that any point
belongs at most to two fuzzy sets when the fuzzy sets are convex.

1 1 2 3 4 5
µ2 (x)

µ1 (x)
0
x

Figure 2: A five linguistic term strong fuzzy partition

2.1.2. Rule base

There exist different kinds of fuzzy rules [18]. The most common ones
are called conjunctive rules and are used in Mamdani or Sugeno FIS. The
underlying relationship between the input and output spaces is modelled
using a conjunction operator (usually the minimum). They represent joint
sets of possible input and output values.
The other kind of fuzzy rules [58, 22] models the input output relationship
using a fuzzy implication, they are called implicative rules and generalize the
classical logic rules. Their meaning is quite different from the conjunctive
rules one: they encode constraints and each rule defines a fuzzy restriction
on the set of possible values. The difference of nature between conjunctive
and implicative rules impacts rule combination: while several conjunctive
rules are combined disjunctively (as they widen the scope of a single rule),
implicative rules are combined conjunctively, because several constraints lead
to a more restricted feasible set of allowed situations than a single constraint.
Conjunctive rules are suitable to manage positive evidence, to extract
relationships from data or to implement similarity reasoning, while implica-
tive rules can be used to model knowledge, which can be seen as negative
information in the sense that it points out forbidden values. A more detailed
comparison between the two kinds of rules is included in [38].
Implicative rules allow a non trivial generalization of the Modus Ponens
[43].
The classical version, A ∧ (A → O) |= O, becomes: A0 ∧ (A → O) |= O0 .

5
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

2.1.3. Output distribution

The rule aggregation yields a possibility distribution representing the in-
ferred output.
With conjunctive rules the output distribution is usually not normalized.
A defuzzification stage is required and yields a crisp value from the output
possibility distribution.
With implicative rules, the output distribution is normalized, and de-
fuzzification is optional.
Both inference mechanisms are illustrated in the last case study of Section
5.

2.2. Concurrent approaches

As this work focuses on modelling, we will not address the question of
fuzzy control in the present paper. Depending on the kind of model under
design and on the available knowledge, FIS modelling may either compete or
be used in a complementary way with other techniques.
Control systems are based on a deep knowledge of the process to be
modelled. Whenever such a mathematical model exists, there is no need for
approximate techniques to encode well established relationships.
Nevertheless fuzzy models may be useful for managing soft transitions
between two steady-state points, like in industrial process supervision. In
this case they are complementary of control theory.
Discrete event system formalisms (Petri nets, Grafcet, timed automata,
etc.) are intrinsically designed to formalize the time dependence of a dynamic
system, which at any given time has a state given by a multidimensional vec-
tor in an appropriate state space. FIS have no such characteristic, they
provide a static view of a system at a given time, which can be included as
part of a dynamic system.
FIS can also be compared to statistical methods. Both approaches are
able to build models from a data sample and to take into account data impre-
cision or imperfection, as statistics is based on probability theory. Statistics
proved to be efficient in a wide range of situations, and thanks to probabilistic
assumptions, statistical models often yield a useful confidence interval.
Nevertheless, from a concurrent point of view, let us focus on some draw-
backs. Statistical techniques require a large amount of data to produce sig-
nificant results. Most of the techniques are limited to data management, only
a few of them are able to take into account expert knowledge. Bayesian in-
ference, by the means of prior distributions, and decision trees using a priori

6
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

defined partitions, are the most popular ones2 . The statistical models are
often difficult to interpret: for instance it is problematic to give a meaning to
some regression coefficients or to transformed variables. [61] advocates that
most of the theories can be generalized using the fuzzy formalism. This also
applies to statistics. A large effort within the Fuzzy Logic research commu-
nity has been dedicated to this topic with more or less convincing results.
The fuzzy generalization of the well known k-means clustering algorithm,
called fuzzy c-means [9] is a valuable example.
Statistics can also be used in cooperation with other modelling techniques,
including FIS, especially in data pre-processing: univariate or multivariate
analysis, outlier detection, for instance.
Artificial Intelligence provides many knowledge representation tech-
niques (such as graphs or ontologies) or data mining methods (such as as-
sociation rules). How can FIS, used as a modelling framework integrating
knowledge and data, be compared with them?
According to [20]: ”An important part of the concern and research in
fuzzy logic and possibility theory does focus on issues such as knowledge rep-
resentation, approximate reasoning and reasoning under uncertainty, which
are central to artificial intelligence.”
Without going into detail, let us point out that knowledge representation
methods intrinsically handle symbolic knowledge, and data mining tools are
designed for coping with data. A few bridges exist between these two topics
of research [52, 1, 41].

2.3. Originality of FIS

In the entirety of the modelling techniques, FIS are fit to be used for
human machine cooperation as they provide a single framework (member-
ship functions, rules, operators, inference mechanism, . . . ) carrying a given
semantics and allowing the correspondence between numerical and symbolic
elements.
In our opinion, the main reason to employ FIS is the need of semantics.
Many research papers do not tackle this point. They take for granted that
the utilization of fuzzy logic yields an easy to use human interpretable model,
even if they actually design data driven systems as function approximators.

2
Few papers in the literature uses this opportunity, in almost all cases prior distributions
are estimated from data and partition bounds are automatically generated.

7
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

Other methods, particularly statistical models, are more fit to that purpose
than FIS. Generally speaking, semantic is needed when the human being has
to interact with the system. Knowledge formalization or training support
tools require transparent models: the user is willing to easily analyze the
underlying reasoning. Semantic is also needed when the system output is a
symbolic label (instead of a numerical value), such as in quality evaluation
or risk management.
As there is no underlying assumption about the data distribution, FIS
automatic modelling does not yield a confidence interval as regression or
other statistical methods do. Nevertheless, system sensitivity to input values
can be assessed by simulation using noisy data, bootstrap, . . .
FIS can manage expert knowledge and also be used for knowledge discov-
ery. To take advantage of both kinds of knowledge, one should use them in
their respective areas of excellence: the key idea is to keep the expert at the
linguistic level and to use the data to define the numerical characteristics of
the linguistic terms or to refine the rules.

3. Cooperation between knowledge and data

FIS can be entirely built from data, and many research papers address
this question. For an approach to learn interpretable Fuzzy Inference Sys-
tems with FisPro, the reader can refer to [31]. The present paper focuses on
the case when data are not sufficient, and must be complemented with expert
knowledge [32], which has to be formalized. Experts know the main trends
of the most influential variables in the system and are able to describe their
behavior using linguistic rules. Expert rules are based on a large experience
and present a high level of generalization. Dealing with complex systems, ex-
perts face two difficulties: how to define useful variables and how to formalize
variable interactions. Data, the extensive form of induced knowledge, are ba-
sically pieces of positive evidence. The most important data characteristic is
incompleteness: A data sample cannot pretend to cover all the possible sit-
uations, especially in the case of complex systems. Therefore it is a cautious
attitude to consider any sample as part of a whole. The induced rule quality
highly depends on the training set characteristics: the more representative
of the whole the examples, the more general the induced rules.
Cooperation between knowledge and data is possible at various levels, as
well in designing a model as in using it for prediction or decision support.

8
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

• Design: Expert knowledge and data are likely to cooperate in each step
of the FIS design process, especially in input and output partitioning
[32] and rule definition.
Given the input/output partitioning, the rule base may include induced
as well as expert rules. As an example, expert knowledge may serve to
define linguistic terms or rules in areas where no data are available.
• FIS use: Either expert rule and model validation with data, or rule
automatic generation and expert assessment of induced knowledge, is
possible. In the first case, data are used to define the precise meaning,
in the numerical space, of an expert linguistic concept while in the
second one, experts give a linguistic meaning to rules induced from
data, thanks to system interpretability. A dialectic motion between
these two approaches may be beneficial in system modelling.
Figure 3 shows the outline of the proposed approach for modelling with FIS,
allowing to formalize and introduce expert knowledge at all steps: fuzzy par-
tition and rule design, where knowledge can complement automatic design,
by adding MF and rules in areas where no data are available, FIS parameter
optimization and system validation. In this last step, it is proposed not only
to check numerical accuracy, but also to analyze induced knowledge. The sys-
tem generalization ability can be studied through automatic cross-validation
procedures, and also analyzed by studying the areas of low performance and
the links between data items and inference rules.
As the use of the fuzzy formalism increases system complexity (more
parameters, operators . . . ), it has to be justified.
FIS based cooperation between expert knowledge and data is suitable
when there exists prior knowledge. Expertise often expresses trends in purely
linguistic terms, which have to be completed by data to tune the models.
The interpretability constraints restrict this approach to low dimensional
problems. From our experience, as in many cases there is not a unique
system able to model the input-output relationships, several human-machine
iterations are needed.

4. Software environments
Exploratory analysis is essential in a modelling approach associating ex-
pert knowledge and data. Fuzzy software does not always pay a lot of atten-
tion to that point.

9
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

: Domain expert : Data

Design

MF parameters
Validation
# MF (Input data only)
(Graphical tools and indices)
k−means
Partitions Modal points Hfp Check numerical accuracy
Regular
Complement partition and rule base Optimization Analyze induced knowledge
by adding MF or rule in areas MF parameters Study generalization ability:
where there are no available data. Rule conclusion links between items and rules
Manual design Learning poor performance areas
Rules (few variables) Fuzzy Decision Tree cross validation or test
FPA
OLS
Wang & Mendel

Figure 3: Outline of an integrated approach for modelling with FIS

In this section, we present a brief state of the art of available fuzzy soft-
ware, and then we introduce FisPro. We describe its most important fea-
tures, and put them in relation with the modelling approach proposed in the
previous section.

4.1. State of the art regarding fuzzy software

Fuzzy software was first developed for the needs of fuzzy control, which
became popular in the 1990s. Industrial software as well as academic one
became available, and the targeted audience was control engineers, who used
fuzzy software as an alternative in the domain of control system design. For
industrial software, we can cite the TIL Shell [36] by Togai InfraLogic and
Siemens, FIDE by [5] and FuzzyTECH by Inform [27]. Toolboxes were pro-
posed for Matlab, the Fuzzy Logic Toolbox developed by Jang [44], the FMID
written by [6], and Floulib designed by Foulloy, Galichet, and Boukezzoula
[23]. A fuzzy logic add-on is also available for Mathematica [49].
As accuracy was crucial in the domain of control systems, FIS optimiza-
tion and automatic generation soon became available in fuzzy control soft-
ware and provided function approximation capabilities through automatic
learning procedures based on neural networks or genetic algorithms. As a
matter of fact, interpretability was not of prime concern in this field. The
main point for control system users and designers was the interpolation ca-
pability of fuzzy inference systems. Therefore the systems built using auto-

10
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

matic learning were not analyzed in regard to their interpretability, though

that same interpretability was presumed true and often put forward as an
incentive for using fuzzy logic.
Since the end of the 1990s, fuzzy logic expanded to other fields of inter-
est, and more general software was designed by researchers from the fuzzy
logic community, with the objective to attract a new audience through data
analysis and decision support. It includes simple fuzzy inference engines as
FuzzyClips [46] or Sazonov’s specific learning sofware [51], fuzzy decision
trees [37], DataEngine from MIT GmbH [28], or neuro fuzzy classification
[39].
During the special session on software for soft computing that was orga-
nized at the 2007 FuzzIEEE conference [35], some advanced software projects
were presented, as FrlDA [10], a free intelligent data analysis toolbox, or
Xfuzzy [7]. Xfuzzy is a development environment, that integrates a set of
tools to help the user to cover the several stages involved in the design pro-
cess of fuzzy logic-based inference systems. The conference also includes a
comprehensive review of fuzzy software, an interesting discussion of useful
features, and a call for building a fuzzy tool kit that supports the take-up of
fuzzy systems in business applications [45] .
Potential end users may have very different profiles: Fuzzy Logic re-
searcher, adviser in Data Analysis, researcher in Biology or Economics need-
ing system modelling tools, domain expert in the Food Industry looking for
an expert knowledge transmission toolkit, student making a project in Arti-
ficial Intelligence. . .
According to their profile, users have various requirements concerning
fuzzy system structure, available methods and interface. However, one com-
mon denominator is probably the need for automatically generated systems
and for the interpretability of these systems. Indeed, users currently inter-
ested in fuzzy software wish to tackle complex systems, which are extremely
difficult to write from scratch. Thus they typically need to complete avai-
lable expert knowledge with knowledge acquired from data. But that is of
little interest if the software gives results which cannot be interpreted and
discussed with others.
Among fuzzy software products, FisPro stands out because of the inter-
pretability of fuzzy systems automatically learnt from data. Interpretability
is guaranteed in each step of the FIS design with FisPro: variable partition-
ing, rule induction, optimization.

11
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

4.2. Linguistic variable and fuzzy partitioning

Fuzzy partitioning is an important aspect of FIS design.
Even if, in interactive design, other MF shapes are available and fuzzy
partitions can be freely adjusted, FisPro automatic procedures systematically
generate strong fuzzy partitions for inputs, with semi-trapezoidal shapes at
the edges and either triangular or trapezoidal shaped MFs elsewhere, as
shown in Figure 2.
Using strong fuzzy partitions ensures semantic integrity. Moreover, com-
pared to partitions made up of unbounded membership functions, such as
the widely used Gaussian ones, they also lead to more robust systems, where
the number of simultaneously fired rules is limited and kept small [17].

4.3. Rule base

In this part, we cover two aspects of rule bases, rule learning requirements
and inference mechanism characteristics.

Rule learning
When rule learning mechanisms are available, it is important to guarantee
the interpretability of the learnt rule bases. For that purpose, the same
linguistic concepts, therefore the same fuzzy sets, should be shared by all of
the rules whatever the rule induction method.
In FisPro, all of the rule induction methods use predefined linguistic
labels. None of them is allowed to tune the labels nor to add a new concept
(fuzzy set) to the partition.
In order to permit fuzzy set sharing and interpretability, some methods
well known in fuzzy learning such as [56] and Orthogonal least squares (OLS)
[14, 15, 55, 33] have been revisited. Fuzzy decision trees [12, 47, 57] are
available and meet the interpretability criteria.
The implementation of an interpretable fuzzy OLS is presented in [17].
The key idea, which is valid for all these revisited methods, is the use of
predefined strong fuzzy partitions for the rule generation, instead of the data
based Gaussian membership functions used in the original OLS. Besides the
usual learning passes, an extra one is proposed to reduce the number of rule
conclusions with a user defined loss of accuracy.
New methods, such as hierarchical fuzzy partitioning [30], have been im-
plemented with the same preoccupation of fuzzy set readability. They gen-
erate hierarchical embedded partitions and make FIS refinement possible.

12
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

Similarly, FIS optimization (including simplification) is available in Fis-

Pro and allows tuning respectful of linguistic interpretability.
To increase the FIS robustness and generalization capabilities, rule learn-
ing is done based on sampling procedures using learning and test sets, and
guided by evaluation indices.

Inference mechanism
As discussed in Section 2.1.2, there exist different kinds of fuzzy rule
bases: conjunctive ones and implicative ones.
Let us consider an interesting property, called inferential independence.
Let RB be a consistent rule base. In Boolean logic, when a rule premise
is true for a given multidimensional input, the value inferred by RB is that
rule conclusion, independently of the presence and content of the other rules.
Although this is not usually pointed out, this property does not hold for a
fuzzy conjunctive rule base. Given a multidimensional fuzzy input strictly
identical to the fuzzy rule premise, overlapping membership functions trigger
more than one rule and the rule base inference result is computed as the union
of the rule conclusions weighted by the respective matching degrees. With
implicative rules and strong fuzzy partitions, the inferential independence
property holds.
Each kind of rule base: conjunctive and implicative, has its pros and
cons, see [38] for a detailed comparison. In FisPro, rule learning methods
are implemented using conjunctive inference, and implicative rule bases are
available for expert design. Graphical visualization may help to understand
the differences between the inference mechanisms, and specific tools have
been developed.

4.4. A friendly interface

Only the most original and important features of FisPro friendly inter-
face are described in this paper. The user documentation available on line
describes all features in detail.
For an efficient approach in exploratory analysis and system modeling,
special attention has been put on the dynamical behavior of a FIS following
user modifications.
• Each variable or rule can be activated/deactivated within the fuzzy
system. Fuzzy partitions can be edited and the fuzzy system opera-
tors modified. The current data file is displayed in a table, and each

13
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

data row can also be enabled/disabled. All operations are dynamically

handled and all current windows are updated, including the inference
result ones.

• To help the user to assess the rule representativeness, an option that

evaluates the links between rules and examples is available. An acces-
sible detailed cross-summary gives for each rule, the samples that fire
this rule above a given matching degree, and for each sample, the rules
that are fired.

• Inference can be done manually or on the current data file, with eval-
uation criteria which take into account the numerical accuracy as well
as the significance of data items regarding the FIS.
Response surfaces are also available for an exploratory analysis of the
system behaviour.

5. Case study
This section aims to illustrate the potential of fuzzy inference systems to
deal with real world applications involving both data and expert knowledge.
The first study is a supervised learning case, and the second one is essen-
tially based on expertise modelling. First we present the evaluation indices,
that will be used in the case studies.

5.1. FIS evaluation indices

FIS evaluation indices are available in FisPro to assess the prediction
capabilities of a FIS for a given dataset. These indices characterize the
performance and coverage.

5.1.1. Coverage index

Examples are labeled active or inactive for a given rule base. An example
is active if its maximum matching degree over all the rules is greater than a
user defined threshold, inactive otherwise.
Following this definition, a coverage index is calculated by applying the
A
formula CI = where A is the number of active examples, and N the file
N
size. The coverage index is a quality index complementary to the classical
accuracy index.

14
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

5.1.2. Performance indices

Two error indices are automatically computed, the root square of mean
squared errors:
v
u
u1 X A
RM SE = t (ybi − yi )2
A i=1
A
and the maximum error M axError = max |ybi − yi |.
i=1

5.2. Modelling pesticide loss

The modeling objective is to propose a relationship between the propor-
tion of product lost in the atmosphere and some micro-meteorological vari-
ables, during vine spraying. This relationship can be modeled under various
formalisms. The most common one is the multiple linear regression, but we
show how fuzzy rules can also be used. A thorough description of both the
problem statement and the results can be found in [24]. In this section, we
compare the results of the multiple linear regression with the FIS modelling
ones. FIS modelling will allow to incorporate domain knowledge in the fuzzy
partitions, and to learn interpretable rules as an input-output relationship.
The spraying is achieved using air assisted devices to aid the transport
of the droplets toward the target. Two sets of nozzles were used in the
experiments, leading to two kinds of droplets called Fine and Very fine, with
a respective Volume Median Diameter (VMD) of 134 and 65 µm.
The considered explanatory variables are the following ones:

• W : Wind speed (m/s)

• T : Air temperature (o C)

• ∆T : Wet bulb temperature depression (o C)

• z/L : Atmosphere stability parameter

In the present paper, the study is restricted to the Fine case. The sample
size is 32 experiments.

15
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

5.2.1. Multiple linear regression approach

A stepwise multiple linear regression3 is first carried out. This is the
classical approach in this application domain. It yields the following model:

Ybi = 9.719 − 0.229(T ) + 0.109(W ∗ ∆T )

characterized by a determination coefficient of R2 = 0.70.

The plot in Figure 4 shows that the fit is relatively satisfactory.

Figure 4: Linear regression results for pesticide loss data

Nevertheless, such a predictive model is very difficult to analyze (what

is the meaning of the interaction term?) and is not a source of valuable
knowledge.
Regression trees are likely to provide another kind of information as
they discriminate between variables to select the most important ones. The
pruned crisp regression tree, shown in Figure 5 has been obtained with the

3
All the statistical analysis are done using the R environment: http://www.r-
project.org/

16
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

R software rpart implementation of the CART method, commonly used by

statisticians. As expected, the key variable is the wind velocity and the
second one to be introduced is the Air temperature.

Figure 5: The crisp regression tree induced from pesticide loss data

The tree is easy to interpret, but crisp regression trees as this one suffer
from well known drawbacks. The automatic binary variable partitioning for
a given split may or may not be meaningful to the user. The results are
highly sensitive to the split conditions and there is no interpolation between
rules due to the crisp thresholds. Indeed, the possible inferred values are
limited by the number of leaves, three in this case, giving poor convincing
results.
The [47] derived implementations of decision trees allow multiple non
binary splits learning using predefined partitions, but they do not interpolate
either.
These drawbacks are likely to be overcome by fuzzy decision trees (FDT).

5.2.2. FDT design

The FDT design starts with the building of the fuzzy partitions. Then
the tree is generated and pruned. Finally an optimization sequence is run on
the equivalent FIS.
• Partition design

17
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

When available, expert knowledge is introduced in the model through

the fuzzy partition design. As shown in Figure 6, the Wind Speed
variable partitioning is done according to the Beaufort scale, leading
to highly interpretable rules, as each linguistic label corresponds to a
Beaufort degree.

Low Mean High Low High

1 1
(x)

µ(x)
µ

0 0
0.30 1.40 3.00 19.00 25.00
−1 Temperature (°C)
Wind speed (m s )
Low High Unstable Stable
1 1
µ(x)

0 µ(x) 0
5.00 10.00 −10.00 10.00
Wet bulb temperature depression (ΔT) Stability (z/L)−1

Figure 6: Fuzzy input partitions for pesticide loss data

• Tree generation and equivalent FIS

The tree displayed in Figure 7 is a pruned fuzzy regression tree obtained
using FisPro. This kind of system can be used to recommend suited

Figure 7: Fuzzy decision tree for pesticide loss data (fine spray)

spraying periods: avoid windy times but also, even if the wind velocity

18
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

is moderate, prefer times when air temperature is high to minimize

losses.
A fuzzy tree equivalent FIS may automatically be derived from the
fuzzy decision tree, by generating a rule for each path to a leaf. Thanks
to the overlap in the variable partitions, and to the multiple member-
ship, interpolation is allowed giving the output results plotted in Figure
8. The determination coefficient is similar to the one gained using the
linear regression, R2 = 0.70.

Figure 8: Fuzzy inference system results for pesticide loss data

Optimization
FisPro includes an optimization module. The algorithm is based upon
the work by [53] and [25]. It is summarized in Algorithm 1.
As the algorithm memorizes the good directions in the research space,
only a set of connected parameters can be optimized within a given run. The
sequence proposed in FisPro consists in optimizing each of the selected input
variable partitions in turn, then if the output is fuzzy, the output partition,
and finally the rule conclusions.

19
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

input : Initial vector X (0) , noise magnitude N mag

output: The optimized vector, X = X (M ax)
1 Initialization: k = 0, M (0) = 0.
2 while k ≤ Max do
3 Generate a Gaussian vector G(k) ,
4 with mean M (k) and noise magnitude N mag
5 if E(X (k) + G(k) ) < E(X (k) ) then
6 X (k+1) = X (k) + G(k)
7 M (k+1) = 0.2M (k) + 0.4G(k)
8 else if E(X (k) − G(k) ) < E(X (k) ) then
9 X (k+1) = X (k) − G(k)
10 M (k+1) = M (k) − 0.4G(k)
11 else
12 X (k+1) = X (k)
13 M (k+1) = 0.5M (k)
14 end
15 k =k+1
16 end
Algorithm 1: The Solis and Wets algorithm

20
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

This sequence is run to optimize the fuzzy decision tree equivalent FIS.
As the number of data items is small (32), the ten-fold cross validation is
done on a 85/15 % basis, i.e. 27 items for each learning set, and 5 for the
corresponding test set. Only the T and ∆T variables are optimized, as the
z/L one is absent from the rules, and the W (Wind speed) partition is an
expert one, which should be left untouched. The final FIS is computed as the
median FIS: each parameter is replaced by the median of the ten optimized
systems.

∆T T

3 7 11 16 20 25

Figure 9: Loss data FIS: the two modified partitions (in dashed lines)

The two modified partitions are displayed in Figure 9, together with the
initial ones, and the rule base conclusions are printed in Table 1 for the initial
and the final median FIS.

Table 1: Loss FIS: initial (I) and final (F) rule bases - Integers in the rule premise are MF
numbers

Rule premise Rule conclusion

W ∆T T I F
1 1 5.52 5.51
1 1 2 4.64 4.69
1 2 2 4.82 4.79
2 1 6.17 6.33
2 1 2 4.75 4.48
2 2 2 5.33 5.26
3 7.14 7.16

The optimization results are summarized in Table 2. Optim stands for

the optimized FIS MAE, averaged over the ten test samples and Final is the

21
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

final FIS MAE averaged over the same test sets.

Table 2: Loss FIS optimization results: MAE averaged over the test sets

Initial 0.65
Optim 0.62
Gain (%) 4.6
Final 0.54
Gain (%) 14.8

Considering the whole data set, the final R2 corresponding to the median
FIS is 0.79, i.e. a 12% relative gain.
The optimization procedure allows a significant improvement of the FIS
accuracy without modifying the FIS structure. The fuzzy partitions count
with the same number of membership functions, the modifications lie in the
numerical space and do not affect the linguistic interpretation. The same
occurs for the rules: as the premises remain unchanged the rule base analysis
is still valid.
This case study illustrates various ways of cooperation between expert
knowledge and data. First the fuzzy partitions are expert designed to guar-
antee a high level of semantics, especially for the wind speed variable. Then,
the data are used for the rule learning. Thanks to the system interpretabil-
ity, the expert domain is able to analyze the system behavior. Finally, data
is used again to optimize some system parameters while keeping the FIS
structure and its properties. The accuracy is improved and the semantics is
preserved.

5.3. System design using implicative rules

In this case study the main input output relationships are known. The
goal is no longer rule induction and system analysis, like in the previous
section, but expert knowledge modelling taking into account the data im-
precision. The formalism of implicative rules (see Section 2.1.2) seems to be
suited: rules are seen as constraints, each of them restricting the set of pos-
sible output values. Thus rules are combined using a conjunction operator.
This yields a behavior quite different from a conjunctive rule base system,
which is illustrated in the following.

22
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

The problem under study is a predictive diagnosis for a hard-cooked type

cheese-making process. Two parameters are important to determine cheese
firmness: MC (Moisture Content), the cheese moisture content percentage
at the end of the making process and DEE (Dry Extract Evolution), the
loss of water during the first 15 days of the maturation process. The aim is
to predict the cheese firmness at the end of maturation (4 to 10 months or
longer) according to these two parameters. The two measurements (MC and
DEE) come from sensors tainted with significant imprecision.
Input expert partitions are shown in Figure 10. For each partition, a
given input is plotted in dashed lines as a trapezoidal membership function.
A vertical central dotted line corresponds to the precise input value, and the
membership function to the approximate one, representing the measurement
together with its imprecision.

low
high
1

MC
0
54 55 56 57
low high
1

DEE
0
0.2 0.5 0.7 0.8

Figure 10: Input partitions - Precise and imprecise values used for the inference are plotted
in dashed lines

The rule system presented here is a simplified system that does not take
into account the whole process complexity:
• If MC is high and DEE is low then cheese will be soft

• If MC is high and DEE is high then cheese will be normal

• If MC is low and DEE is low then cheese will be normal

• If MC is low and DEE is high then cheese will be hard

23
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

Implicative inference
A screenshot of FisPro interactive inference window, using the precise
data displayed in Figure 10, MC=54.5 and DEE=0.6, is shown in Figure 11.
The aggregation result is plotted in the first row of the Firmness column,
while each of the four following rows illustrate the inference mechanism for
the corresponding rule.
The Resher-Gaines implication, see [38] for details, is used. This operator
yields a crisp interval, which coincides with the core of the output obtained
by all other residuated implications.
The output partition displayed in Figure 11 (top right) and in Figure 12
(bottom) is not a Strong Fuzzy Partition. It is automatically built from a
two term Strong Fuzzy Partition, and it is called a Quasi Strong Partition
[38]. It ensures both the interpretability and the consistency required by the
conjunctive aggregation.

Figure 11: Firmness inference with implicative rules and a precise value

24
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

For crisp data, the FITA4 inference mechanism is used, i.e. the inference
is done rule by rule, and then the aggregation is performed.
The screenshot of FisPro interactive inference window, using the precise
data displayed in Figure 10, MC=54.5 and DEE=0.6,is shown in Figure 11.
Let us comment on this figure. The membership degrees within the rule
premises are combined with the min operator, then the Resher-Gaines impli-
cation operator is applied to the fuzzy set that appears in the rule conclusion,
resulting in a door shape possibility distribution. These intermediate possi-
bility distributions are intersected to give the final one, plotted in the top
right cell of the Firmness column: this is the aggregation step.
Thanks to the conjunctive aggregation, the output width is meaningful,
it contains all the values which satisfy the constraints expressed by the rules.
Note that an empty result would point out a rule base inconsistency [21].
If needed, a precise value can be inferred using a defuzzification step, for
instance, the mean of the output possibility distribution core. This value
(5.75) is displayed just below the Firmness field in Figure 11.
Dealing with imprecise data, the inference cannot be done rule by rule.
Only the FATI5 mechanism is correct, and it is much more difficult to carry
out in practice. In the FisPro implementation, the imprecise data are ap-
proximated by nested doors. Figure 12 shows the Fispro inference window
and the aggregation details, for the imprecise data plotted in Figure 10, and
with a three level nested door decomposition.
The input imprecision is respected in the output distribution.
As the output is a normalized possibility distribution, it can be inter-
preted as such, without the need for defuzzification. Thus the support and
kernel widths increase with the input imprecision. This appears in Figure 12,
where the core and support of the possibility distribution are wider than the
ones resulting from the implicative inference with the precise value, plotted
in Figure 11. This follows expectations, and it is an important point for de-
cision support systems handling imprecise data, as the imprecision remains
interpretable and meaningful. Furthermore, as input imprecision is taken
account of all through the implicative inference process, implicative FIS al-
low to design cascading FIS respectfully of approximate inputs, and thus to
design interpretable hierarchical fuzzy inference systems.

4
First Infer Then Aggregate
5
FATI means ”First Aggregate Then Infer”.

25
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

Figure 12: Firmness inference with implicative rules and an imprecise value: FisPro win-
dow and detail, the output is the union of the three doors.

26
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

Conjunctive inference
To point out the differences with the well known Mamdani FIS, the infer-
ence results of the same rule base, using a conjunctive Mamdani rule base,
with the same data, are now presented.
The screenshot from FisPro is shown on top of Figure 13.

Figure 13: Firmness inference with a conjunctive rule system and a crisp value-FisPro
window and defuzzification detail

The inferred output overlaps the three output fuzzy sets. Consequently,
it is difficult to interpret this result without defuzzification. Centroid-like
defuzzification gives us a firmness equal to 6.3. Note that defuzzification

27
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

is highly sensitive not only to the defuzzification method, but also to the
membership function shapes.
The inference of imprecise data with the conjunctive rule base is similar
to what is done in the case of precise data, as the imprecise data are used
to compute a single matching degree with each of the fuzzy sets within the
partition.
This degree is then used in the same way than the membership degree of
the precise value. The result is not plotted here as, due to defuzzification, it
does not yield any interpretable result when examining the output possibility
distribution versus the input imprecision.
With conjunctive inference the inferred output is a subnormalized pos-
sibility distribution, with no kernel, and it always requires a defuzzification
step to yield a crisp output. Furthermore the support width is not related
to the input imprecision.

Data validation
A set of 103 representative cheese sample has been analyzed by an expert
panel. The final firmness result from the expert score aggregation. This
sample has been used to validate the implicative rule base.
The inference process yields a possibility distribution. For the Resher-
Gaines implication operator, this distribution reduces to an interval, as all
the values have a possibility degree equal to 1 or 0. It is, of course, possible, to
defuzzify this distribution, by computing a real value using this interval, for
instance the mean, minimum or maximum. But, this step is not mandatory
with an implicative rule base. Moreover, it would hide some interesting
properties of the distribution, such as its range.
To take advantage of this inference mechanism, a more complex label is
proposed to characterize the inferred output distribution with respect to the
target value. The label is composed of two distinct integers:
• Precision: this value depends on the output interval width. To avoid
the use of a threshold value, it is defined as the number of distinct MF
in which the membership degrees of any point within the interval are
the highest.
• Accuracy: this value is the prediction error, quantified as a number
of MF. It is computed as the smallest difference between the indexes
of the MF the target value mainly belongs to and the MF previously
defined in the precision label.

28
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

The precision ranges from 1 to f , the number of MF in the partition. As

we use a 3-MF output fuzzy partition, the precision can be precise, imprecise
or very imprecise. The accuracy is between 0 and f − 1, any strictly positive
value is considered as bad.

1111
0000 11111
00000
soft normal hard
I1 I2
1
0000
1111 00000
11111
0000
1111 00000
11111
0000
1111
0000
1111 00000
11111
00000
11111
0
0000
1111
0000
1111 00000
11111
00000
11111
5 6.5 Firmness
x 3.5 y

Figure 14: Precision and accuracy labels

To illustrate the label significance, Figure 14 shows two firmness values,

x and y, and two hypothetical inferred intervals, a narrow one on the left of
the figure filled with a slash pattern, I1 , and a wider one on the right, filled
with a backslash pattern, I2 . The corresponding labels are given in Table 3.

Table 3: Labels according to the target (row) and estimated (column) values

I1 I2
x Correct and precise Incorrect
y Correct Correct but imprecise

We now present the implicative FIS inference results, obtained on the ex-
perimental data set, and labelled in terms of precision and accuracy. Results
are summarized in Table 4.
The use of precision and accuracy labels makes it possible to analyze the
results with regard to these two criteria, contrary to what is usually done
with Mamdani or Sugeno conjunctive FIS, where only numerical accuracy is
discussed, without specifically considering the impact of input data impreci-
sion. First of all, Bad predictions, due to examples that are in contradiction
with the expert rules, show that these rules are not as general as it could be
thought. Nevertheless, though simple, the rule base, made up of only 4 rules,

29
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

Table 4: Implicative rule base validation

Correct and precise 21

Correct but imprecise 49
Incorrect 33

is able to reflect some of the main data trends: 70 % of the predictions are
good. A significant part of these good predictions are still imprecise, and this
imprecision can be reduced by refining the rules with new variables. Indeed,
with an implicative rule base, refinement leads to more constraints applied
onto the data.

6. Conclusion
Cooperation between knowledge and data is still an open challenge in
system modelling. In this paper, we show how fuzzy inference systems can
be used for that purpose, by providing a powerful interface between the
symbolic space used for knowledge representation and the numerical one.
We present the main steps of an integrated approach for modelling with FIS
and discuss its practical application.
Some ways of cooperation are illustrated through two case studies. In the
first one, a study of pesticide loss during air spray, data play the central role in
the modelling process, even though expert knowledge, when available, is also
involved, for instance to base the wind velocity linguistic variable definition
on the Beaufort scale. Valuable operational information can be derived from
the rule base, for making recommendations about pesticide spraying.
The second case study discusses how expert knowledge about a cheese
making process is formalized into an implicative rule base. Thanks to the
conjunctive aggregation of implicative rules, the output distribution width
is meaningful. When the input values are not precise, the imprecision is
respected in the output distribution. From an engineering point of view,
implicative rules allow the incremental design of systems. Adding a new
variable to a given rule makes it more specific. There is no need to modify
the existing rules. It can be noted that additional work is needed to learn
and optimize implicative rule bases.
The approach framework is implemented in a free software, FisPro. Its

30
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

originality stems from rule interpretability and exploratory tools to study the
FIS behavior. Open, it welcomes contributions from scientists or engineers
to increase its functionality and its scope. Some characteristics are given
below.
• FisPro is open source, and to facilitate the inclusion in other programs,
its core is delivered as a C++ library. The interface is written in Java,
and available in several languages.
• Hierarchical FIS are available. This allows the reuse of a FIS output
as input to another FIS.
• An optimization module allows to tune all parts of Mamdani and
Sugeno FIS, regardless of the way they were built, by hand or through
automatic learning.
• Sample generation is available prior to learning, as well as learning
scripts including cross validation procedures.
Fuzzy rule base merging, for instance expert and induced rules, using
the HILK method [4] is implemented in the Generating Understandable and
Accurate fuzzy models in a Java Environment (GUAJE) software [2], which
is based on FisPro.
FisPro has been used for various modelling projects [48, 11, 3, 8], and we
hope that the approach presented in this paper will help in new modelling
tasks.

References
[1] Adomavicius, G., Tuzhilin, A., 2001. Expert-driven validation of rule-
based user models in personalization applications. Data Mining and
Knowledge Discovery 5 (1-2), 33–58.
[2] Alonso, J. M., 2010. Guaje: Generating understand-
able and accurate fuzzy models in a java environment.
http://www.softcomputing.es/guaje.
[3] Alonso, J. M., Magdalena, L., González-Rodrı́guez, G., 2009. Looking
for a good fuzzy system interpretability index: An experimental ap-
proach. International Journal of Approximate Reasoning 51 (1), 115 –
134.

31
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

[4] Alonso, J. M., Magdalena, L., Guillaume, S., 2008. Hilk:a new method-
ology for designing highly interpretable linguistic knowledge bases using
the fuzzy logic formalism. International Journal of Intelligent Systems
23 (7), 761–794.

[5] Aptronics, 2007. Fide power design tool [online].

http://www.aptronix.com.

[6] Babuška, R., 2003. Neuro-fuzzy methods for modeling and identification.
Physica-Verlag GmbH, Heidelberg, Germany.

[7] Baturone, I., Moreno-Velo, F. J., Sánchez-Solano, S., Barrios, A. B.,

Jiménez, P. B., Gersnoviez, A., Brox, M., 2007. Using xfuzzy environ-
ment for the whole design of fuzzy systems. In: [35], pp. 517–522.

[8] Bergasa, L. M., Nuevo, J., Sotelo, M. A., Barea, R., Lopez, M. E., 2006.
Real-time system for monitoring driver vigilance. IEEE Transactions on
Intelligent Transportation Systems 7 (1), 63–77.

[9] Bezdek, J. C., 1981. Pattern Recognition with Fuzzy Objective Func-
tions Algorithms. Plenum Press, New York.

[10] Borgelt, C., Gonzáles-Rodrı́guez, G., 2007. Frida - a free intelligent data
analysis toolbox. In: [35], pp. 1892–1896.

[11] Bossomaier, T., Standish, R. K., Harré, M., 2010. Simulation of trust in
client-wealth management adviser relationships. International Journal
of Simulation and Process Modelling 6 (1), 40–49.

[12] Breiman, L., Friedman, J. H., Olshen, R. A., Stone, C. J., 1984. Classifi-
cation and Regression Trees. Wadsworth International Group, Belmont
CA.

[13] Casillas, J., Cordón, O., Herrera, F., Magdalena, L., 2003. Interpretabil-
ity improvements to find the balance interpretability-accuracy in fuzzy
modeling: an overview. In: Interpretability Issues in Fuzzy Modeling.
Springer, pp. 3–22.

[14] Chen, S., Billings, S. A., Luo, W., 1989. Orthogonal least squares meth-
ods and their application to non-linear system identification. Int. J. Con-
trol 50, 1873–1896.

32
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

[15] Chen, S., Cowan, C. F. N., Grant, P. M., March 1991. Orthogonal
least squares learning algorithm for radial basis function networks. IEEE
Transactions on Neural Networks 2 (No 2), 302–309.

[16] de Oliveira, J. V., 1999. Semantic constraints for membership functions

optimization. IEEE Transactions on Systems, Man and Cybernetics.
Part A 29 (1), 128–138.

[17] Destercke, S., Guillaume, S., Charnomordic, B., 2007. Building an in-
terpretable fuzzy rule base from data using orthogonal least squares-
application to a depollution problem. Fuzzy Sets and Systems 158, 2078–
2094.

[18] Dubois, D., Prade, H., 1996. What are fuzzy rules and how to use them.
Fuzzy Sets and Systems 84(2), 169–186.

[19] Dubois, D., Prade, H., 1997. The three semantics of fuzzy sets. Fuzzy
Sets and Systems 90, 141–150.

[20] Dubois, D., Prade, H., 2003. Fuzzy set and possibility theory-based
methods in artificial intelligence. Artificial Intelligence 148, 1–9.

[21] Dubois, D., Prade, H., Ughetto, L., 1997. Checking the coherence and
redundancy of fuzzy knowledge bases. IEEE Trans. on Fuzzy Systems 5
(5), 398–417.

[22] Dubois, D., Prade, H., Ughetto, L., 2003. A new perspective on rea-
soning with fuzzy rules. International Journal of Intelligent Systems 18,
541–567.

[23] Foulloy, L., Galichet, S., Boukezzoula, R., 2005. Floulib [online].
http://www.polytech.univ-savoie.fr/index.php?id=859.

[24] Gil, Y., Sinfort, C., Guillaume, S., Brunet, Y., Palagos, B., 2008. Influ-
ence of micrometeorological factors on pesticide loss to the air during
vine spraying: Data analysis with statistical and fuzzy inference models.
Biosystems Engineering 100 (2), 184–197.

[25] Glorennec, P.-Y., 1996. Constrained optimization of FIS using an evo-

lutionary method. Genetic Algorithms and Soft Computing. Physica-
Verlag, pp. 349–368.

33
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

[26] Glorennec, P.-Y., 1999. Algorithmes d’apprentissage pour systèmes

d’inférence floue. Editions Hermès, Paris.

[27] GmbH, I., 2007. Fuzzytech [online]. http://www.fuzzytech.com.

[28] GmbH, M., 2001. Dataengine [online]. http://www.dataengine.de.

[29] Guillaume, S., 2001. Designing fuzzy inference systems from data: an
interpretability-oriented review. IEEE Transactions on Fuzzy Systems
9 (3), 426–443.

[30] Guillaume, S., Charnomordic, B., 2004. Generating an interpretable

family of fuzzy partitions. IEEE Transactions on Fuzzy Systems 12 (3),
324–335.

[31] Guillaume, S., Charnomordic, B., 2011. Learning interpretable fuzzy

inference systems with fispro. International Journal of Information Sci-
ences 181, 4409–4427.

[32] Guillaume, S., Magdalena, L., 2006. Expert guided integration of in-
duced knowledge into a fuzzy knowledge base. Soft computing 10 (9),
773 – 784.

[33] Hohensohn, J., Mendel, J. M., 1994. Two pass orthogonal least-squares
algorithm to train and reduce fuzzy logic systems. In: Proc. IEEE Conf.
Fuzzy Syst. Orlando, Florida, pp. 696–700.

[34] Hullermeier, E., 2005. Fuzzy methods in machine learning and data
mining: Status and prospects. Fuzzy sets and Systems 156, 387–406.

[35] IEEE (Ed.), 2007. FUZZ-IEEE 2007, IEEE International Conference

on Fuzzy Systems, Imperial College, London, UK, 23-26 July, 2007,
Proceedings. IEEE.

[36] InfraLogic, T., 1995. The world’s source for fuzzy logic solutions [online].
http://www.ortech-engr.com/fuzzy/togai.html.

[37] Janikow, C. Z., 2005. Fuzzy decision tree/forest [online].

http://www.cs.umsl.edu/ janikow/fid/.

34
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

[38] Jones, H., Charnomordic, B., Dubois, D., Guillaume, S., 2009. Practical
inference with systems of gradual implicative rules. IEEE Transactions
on Fuzzy Systems 17 (1), 61–78.
[39] Kruse, R., 2007. Neural networks and fuzzy systems [online].
http://fuzzy.cs.uni-magdeburg.de/.
[40] Lee, C. C., 1990. Fuzzy logic in control systems: Fuzzy logic controller-
part ii. IEEE Trans. on Systems, Man, and Cybernetics 20 (2), 419–435.
[41] Ling, T., Kang, B. H., Johns, D. P., Walls, J., Bindoff, I., 2008. Expert-
driven knowledge discovery. In: Latifi, S. (Ed.), Proceedings of the fifth
international conference on information technology: new generations.
pp. 174–178.
[42] Mamdani, E. H., Assilian, S., 1975. An experiment in linguistic synthe-
sis with a fuzzy logic controller. International journal of Man-Machine
Studies 7, 1–13.
[43] Martin-Clouaire, R., 1989. Semantic and computation of the generalized
modus ponens: the long paper. International Journal of Approximate
Reasoning 3(2), 195–217.
[44] MathWorks, T., 2007. Fuzzy logic toolbox [online].
http://www.mathworks.com/products/fuzzylogic.
[45] Nauck, D. D., 2007. Gnu fuzzy. In: [35], pp. 1019–1024.
[46] Orchard, R. A., 2004. Fuzzy extension to the
clips expert system shell (fuzzyclips) [online].
http://www.nrc-cnrc.gc.ca/fra/licences/iti/non-commerciale9.html.
[47] Quinlan, J. R., 1986. Induction of decision trees. Machine Learning 1,
81–106.
[48] Rajaram, T., Das, A., 2010. Modeling of interactions among sustain-
ability components of an agro-ecosystem using local knowledge through
cognitive mapping and fuzzy inference system. Expert Systems with Ap-
plications 37 (2), 1734–1744.
[49] Research, W., 2007. Mathematica fuzzy logic [online].
http://www.wolfram.com/products/applications/fuzzylogic.

35
Author-produced version of the article published in Expert Systems with Applications, 2012, N°39, p. 8744- 8755
The original publication is available at http://dx.doi.org/10.1016/j.eswa.2012.01.206
Doi: 10.1016/j.eswa.2012.01.206

[50] Ruspini, E. H., 1982. Recent developments in fuzzy clustering. Pergamon

Press, New York, pp. 133–147.

[51] Sazonov, E., 2002. Open source fuzzy inference engine for java [online].
http://people.clarkson.edu/ esazonov/FuzzyEngine.htm.

[52] Sester, M., 2000. Knowledge acquisition for the automatic interpreta-
tion of spatial data. International Journal of Geographical Information
Science 14 (1), 1–24.

[53] Solis, F. J., Wets, R. J., 1981. Minimization by random search tech-
niques. Mathematics of Operation Research 6, 19–30.

[54] Takagi, T., Sugeno, M., 1985. Fuzzy identification of systems and its
applications to modeling and control. IEEE Transactions on System
Man and Cybernetics 15, 116–132.

[55] Wang, L.-X., Mendel, J. M., 1992. Fuzzy basis functions, universal ap-
proximation, and orthogonal least squares learning. IEEE Transactions
on Neural Networks 3, 807–814.

[56] Wang, L.-X., Mendel, J. M., 1992. Generating fuzzy rules by learning
from examples. IEEE Transactions on Systems, Man and Cybernetics
22 (6), 1414–1427.

[57] Weber, R., 1992. Fuzzy-id3: A class of methods for automatic knowledge
acquisition. In: 2nd International conference on fuzzy logic and neural
networks. pp. 265–268.

[58] Weisbrod, J., 1998. A new approach to fuzzy reasoning. Soft Computing
2, 89–99.

[59] Zadeh, L. A., 1965. Fuzzy sets. Information and Control 8, 338–353.

[60] Zadeh, L. A., 1975. The concept of linguistic variable and its application
to approximate reasoning - parts i, ii and iii. Information Sciences 8-9,
199–249,301–357,43–80.

[61] Zadeh, L. A., 2008. Is there a need for fuzzy logic? Information Sciences
178 (13), 2751–2779.

36