Papers by Fatima Alkhawaldeh
International Journal on Cybernetics & Informatics (IJCI), 2022
Consideration of multiple viewpoints on a contentious issue is critical for avoiding bias and ass... more Consideration of multiple viewpoints on a contentious issue is critical for avoiding bias and assisting in the formulation of rational decisions. We observe that the current model imposes a constraint on diversity. This is because the conventional attention mechanism is biased toward a single semantic aspect of the claim, whereas the claim may contain multiple semantic aspects. Additionally, disregarding common-sense knowledge may result in generating perspectives that violate known facts about the world. The proposed approach is divided into two stages: the first stage considers multiple semantic aspects, which results in more diverse generated perspectives; the second stage improves the quality of generated perspectives by incorporating common-sense knowledge. We train the model on each stage using reinforcement learning and automated metric scores. The experimental results demonstrate the effectiveness of our proposed model in generating a broader range of perspectives on a contentious subject.
International Journal on Natural Language Computing (IJNLC) , 2021
Each argument begins with a conclusion, which is followed by one or more premises supporting the ... more Each argument begins with a conclusion, which is followed by one or more premises supporting the conclusion. The warrant is a critical component of Toulmin's argument model; it explains why the premises support the claim. Despite its critical role in establishing the claim's veracity, it is frequently omitted or left implicit, leaving readers to infer. We consider the problem of producing more diverse and high-quality warrants in response to a claim and evidence. To begin, we employ BART [1] as a conditional sequence tosequence language model to guide the output generation process. On the ARCT dataset [2], we fine-tune the BART model. Second, we propose the Multi-Agent Network for Warrant Generation as a model for producing more diverse and high-quality warrants by combining Reinforcement Learning (RL) and Generative Adversarial Networks (GAN) with the mechanism of mutual awareness of agents. In terms of warrant generation, our model generates a greater variety of warrants than other baseline models. The experimental results validate the effectiveness of our proposed hybrid model for generating warrants.
Lecture Notes in Networks and Systems
Fact-checking is a task to capture the relation between a claim and evidence (premise) to decide ... more Fact-checking is a task to capture the relation between a claim and evidence (premise) to decide this claim’s truth. Detecting the factuality of claim, as in fake news, depending only on news knowledge, e.g., evidence text, is generally inadequate since fake news is intentionally written to mislead readers. Most of the previous models on this task rely on claim and evidence argument as input for their model, where sometimes the systems fail to detect the relation, particularly for ambiguate information. This study aims to improve fact-checking task by incorporating warrant as a bridge between the claim and the evidence, illustrating why this evidence supports this claim, i.e., If the warrant links between the claim and the evidence then the relation is supporting, if not it is either irrelevant or attacking, so warrants are applicable only for supporting the claim. To solve the problem of gap semantic between claim evidence pair, A model that can detect the relation based on existing extracted warrants from structured data is developed. For warrant selection, knowledge-based prediction and style-based prediction models are merged to capture more helpful information to infer which warrant represents the best bridges between claim and evidence. Picking a reasonable warrant can help alleviate the evidence ambiguity problem if the proper relation cannot be detected. Experimental results show that incorporating the best warrant to fact-checking model improves the performance of fact-checking.
7th International Conference on Natural Language Computing (NATL 2021), November 27~28, 2021, London, United Kingdom, 2021
The warrant element of the Toulmin model is critical for fact-checking and assessing the strength... more The warrant element of the Toulmin model is critical for fact-checking and assessing the strength of an argument. As implicit information, warrants justify the arguments and explain why the evidence supports the claim. Despite the critical role warrants play in facilitating argument comprehension, the fact that most works aim to select the best warrant from existing structured data and labelled data is scarce presents a fact-checking challenge, particularly when the evidence is insufficient, or the conclusion is not inferred or generated well based on the evidence. Additionally, deep learning methods for false information detection face a significant bottleneck due to their training requirement of a large amount of labelled data. Manually annotating data, on the other hand, is a time-consuming and laborious process. Thus, we examine the extent to which warrants can be retrieved or reconfigured using unstructured data obtained from their premises.
Hierarchical Reinforcement Learning for Factual Claim Generation, 2020
We propose a novel Hierarchical Reinforcement Learning (HRL) Model to detect the factuality of cl... more We propose a novel Hierarchical Reinforcement Learning (HRL) Model to detect the factuality of claim trying to generate a new correct claim if not. Initially, we segment each sentence into several clauses using sentence-level discourse segmentation then measuring the cosine similarity to decide whether a clause is relevant to a claim or not. All relevant clauses will be sent to the high-level policy where deep communicating agents are implemented for encoding all relevant clauses. Each agent adopts the hierarchical attention mechanism, word-level and clause-level attention networks, to select informative words and clauses relevant to a specific claim. In word-level claim attention network, word encoding layer concatenates claim representation to each word embedding and then summarizes information by bi-directional LSTM. Word attention layer focuses on the terms that are important to the meaning of the clause with respect to the claim, producing clause vectors. In clause-level claim attention, clause encoding layer applies Bi-directional LSTM capture context clause representations. After that, in clause attention Layer, attention mechanism computes the attention weight between each claim-clause representation to produce contextual information conditioned on the claim representation. We will use the message sharing mechanism to help other agents' encoders to generate better contextual information conditioned upon the messages received from other agents. The context and states from the environment are used to create all possible sub-goals (to copy or to generate) which should be achieved by the lower agent policy to select a series of actions (words) and produce a new sequence of words. We will apply a rewarder function to compute the factuality of the new claim using entailment and semantic similarity metrics.
In debatable topics, people use evidence to reason towards a claim. The claim conveys a stance to... more In debatable topics, people use evidence to reason towards a claim. The claim conveys a stance towards a particular aspect in the evidence. Existing studies mainly focus on identifying claim stance; which is determined by its relevant evidence; however, the task to get a factual claim if the claim is non-factual is not considered. We thus study the question to what extent a false claim can be reconstructed from its premises to be true, either by generating a new factual claim from relevant premises or determining the positions for the misleading information in the false claim and modify it concerning to the evidence. To address such issue, we introduce a factual claim-making task, anew task to predict the factuality of the claim that is associated with evidence that supports or refutes the given claim. If the claim is non-factual, we propose two different models to get a factual claim. In the generator model, we generate a factual claim by applying the generation model. In the modifier model, we depend on the sequence operation model to modify the misleading information. The experimental results on Perspectum dataset show the effectiveness of our models. The performances of the proposed system achieved 76.84% and 78.36% of F1 scores for the generator and modifier mode, respectively.
In debatable topics, people use evidence to reason towards a claim. The claim conveys a stance to... more In debatable topics, people use evidence to reason towards a claim. The claim conveys a stance towards a particular aspect in the evidence. Existing studies mainly focus on identifying claim stance; which is determined by its relevant evidence; however, the task to get a factual claim if the claim is non-factual is not considered. We thus study the question to what extent a false claim can be reconstructed from its premises to be true, either by generating a new factual claim from relevant premises or determining the positions for the misleading information in the false claim and modify it concerning to the evidence. To address such issue, we introduce a factual claim-making task, anew task to predict the factuality of the claim that is associated with evidence that supports or refutes the given claim. If the claim is non-factual, we propose two different models to get a factual claim. In the generator model, we generate a factual claim by applying the generation model. In the modifier model, we depend on the sequence operation model to modify the misleading information. The experimental results on Perspectum dataset show the effectiveness of our models. The performances of the proposed system achieved 76.84% and 78.36% of F1 scores for the generator and modifier mode, respectively.
We propose a novel Hierarchical Reinforcement Learning (HRL) Model to detect the factuality of cl... more We propose a novel Hierarchical Reinforcement Learning (HRL) Model to detect the factuality of claim trying to generate a new correct claim if not. Initially, we segment each sentence into several clauses using sentence-level discourse segmentation then measuring the cosine similarity to decide whether a clause is relevant to a claim or not. All relevant clauses will be sent to the high-level policy where deep communicating agents are implemented for encoding all relevant clauses. Each agent adopts the hierarchical attention mechanism, word-level and clause-level attention networks, to select informative words and clauses relevant to a specific claim. In word-level claim attention network, word encoding layer concatenates claim representation to each word embedding and then summarizes information by bi-directional LSTM. Word attention layer focuses on the terms that are important to the meaning of the clause with respect to the claim, producing clause vectors. In clause-level claim attention, clause encoding layer applies Bi-directional LSTM capture context clause representations. After that, in clause attention Layer, attention mechanism computes the attention weight between each claim-clause representation to produce contextual information conditioned on the claim representation. We will use the message sharing mechanism to help other agents' encoders to generate better contextual information conditioned upon the messages received from other agents. The context and states from the environment are used to create all possible sub-goals (to copy or to generate) which should be achieved by the lower agent policy to select a series of actions (words) and produce a new sequence of words. We will apply a rewarder function to compute the factuality of the new claim using entailment and semantic similarity metrics.
Uncertainty detection plays fundamental role to separate new discovering information from those a... more Uncertainty detection plays fundamental role to separate new discovering information from those are factual information. Uncertainty texts have nonfactual information which could be useful for scientific purposes. Speculation and negation words affect the factuality degree. for example, negation reverses the factuality to non-factuality and vice versa and speculation words decrease or increase the degree of factuality. Lately, Deep Neural Networks (DNN) have proven competitive performance in NLP tasks generally and uncertainty detection tasks particularly. Due to the scarcity of labelled data for Arabic texts, most previous works have been examined to the English texts. To our knowledge, there is only one system has been implemented previously to identify the uncertainty statement for biomedical Arabic texts. This research will employ hierarchical attention mechanism to develop Generative Adversarial Networks (GAN)-based uncertainty detection model that able to detect speculated and negated parts in biomedical texts. in this paper, the implemented models are fed by some linguistic features like part of speech, constituency and dependency. The evaluation of this system will be based on the BIOARABIC corpus. Empirical experiments on BIOARABIC corpus confirm that GAN model with hierarchical attention achieve better performance (F-scores of 79) than previously implemented deep learning models (the best F-score 73.55). The proposed Hierarchical Attention Generative Adversarial Networks (HAGAN) shows that it produces richer representation of texts focusing on the shared features rather than specific features by training the generator and discriminator alternatively. Hierarchical attention mechanism proved its effectiveness to demonstrate long range dependencies concentrating on relevant parts of texts during the model training.
INTERNATIONAL JOURNAL OF ADVANCED STUDIES IN COMPUTER SCIENCE AND ENGINEERING IJASCSE, 2020
Fake news is currently seen as a possible risk with a harmful effect on democracy, journalism, an... more Fake news is currently seen as a possible risk with a harmful effect on democracy, journalism, and economies, that comes mainly from social media and online websites. To detect fake news, we propose two models trying to check the factuality of a claim against relevant pieces of evidence. In this paper, the stance of each relevant evidence toward a certain claim is detected, then the result of factuality checking will be decided based on the entire aggregation of all available stances in addition to some salient syntactic and semantic features. In this paper, we propose two models help distinguish fake news from reliable content. The first model is multi-channel LSTM-CNN with attention, where numeric features are merged with syntactic and semantic features as input. Concerning the second model, word-level and clause-level attention networks are implemented to capture the importance degrees of words in each clause and all clauses for each sentence in evidence. Other crucial features will be used in this model to guide the model in stance detection processes such as tree kernel and semantic similarities metrics. In our work, for stance detection evaluation, the PERSPECRUM data set is used for stance detection, while DLEF corpus is used for factuality checking task evaluation. Our empirical results show that merging stance detection with factuality checking helps maximize the utility of verifying the veracity of an argument. The assessment demonstrates that the accuracy improves when more focus is given on each segment (clause) rather than each sentence, so using the proposed word-level and clause-level attention networks demonstrate more effectiveness against multi-channel LSTM-CNN.
JASC: Journal of Applied Science and Computations, 2020
In recent years, the trustworthiness of information on the Internet has arisen as a critical prob... more In recent years, the trustworthiness of information on the Internet has arisen as a critical problem of modern society, and it has incredible impact on the political and social world. Factuality checking is a challenging task to identify the reliability of information sources. Most previously applied models for factuality checking rely on the labelled data to train supervised models. These systems get declined performance when implementing on out of domain test data, as a result of the difference between source and target domain distributions. In this paper, we study domain adaptation for factuality checking task for two related domains: news and twitter, using a novel hybrid model. For that we propose a hybrid model that performs adversarial learning-based domain adaptation with ability to capture domain shared features and reduce the gap between different domains for false information. In this work, we mainly concentrate on the writing style to classify each type of information e.g., hoax, satire, propaganda and trusted News. So that, the detection of false information will be based on linguistic styles of text rather than its the contextual meaning. Axillary linguistic features will be used to enhance the performance of the proposed model. This work incorporates linguistic features with deep learning techniques to distinguish the information veracity. We are applying various deep learning models to capture richer information that guides the model to detect the type of information. The applied models are Bidirectional Long Short-Term Memory (BiLSTM), Bidirectional Long Gated Recurrent Unit (BiGRU) Convolutional Neural Network (CNN), and Capsule network in addition to attention mechanism. Our proposed model outperformed previously related models and get higher accuracy.
JASC: Journal of Applied Science and Computations, 2020
Argument generation aims to produce extra information automatically given a piece of text as inpu... more Argument generation aims to produce extra information automatically given a piece of text as input to provide information helps in decision making. These arguments can reason about the available information, its conflicting or uncertain information and the newly obtained information to construct believes or goals that assert the agreement of a claim with respect to a conclusion justifying this relation (support or refute). In this paper, argumentation-based inference will be developed based on Toulmin's Model of argumentation. The structure of an argument and different related components are discussed in order to help for inferencing and reasoning in factuality checking tasks. Toulmin arguments classified to six types: data, warrant, claim, backing, qualifier, and rebuttal. In this work, different deep learning models with crucial auxiliary information, trained by Reinforcement learning are proposed. In this paper, difference deep learning models with variant auxiliary inputs are employed to create the correct and core features that represent the internal state for a sequence of tokens. At the same time, RL agents are applied to decide on this decoded information to modify or keep the selected words. Experimental results on [1] dataset for perspectives generation task and [2] data set for warrant generation demonstrate the effectiveness of our proposed models for discovering argument reasoning generation. Keywords-Toulmin's Model, Reinforcement Learning and Deep Learning. I. INTRODUCTION There are many Natural Language Processing NLP tasks can be expressed as a sequence to sequence learning that does mapping for a series of words to another set of words based on a specific task like question answering, machine translation and text paraphrasing. Recently, to address these tasks problems, encoder-decoder deep learning models have shown very effective solutions. One of the main problems with factuality checking task is that assessing the truth of a claim differs from reader to reader based on specialist knowledge of the subject matter and linguistic knowledge in addition to the level of experience with another characteristic, so depending only on textual information without considering user and source metaknowledge sometime is not enough. Another limitation is that most of the previous approaches are supervised learning depending on the availability of labelled data sets. Labelled data is often not available in the fake news domain. These systems have an accuracy problem, the obtained accuracy by the current methods, e.g., depends on the different size of the annotated data sets. Most published data sets are either small [3], imbalanced [4], dedicated to either stance detection [4] or fact-checking [5], and no dataset that combines both stance and factuality tasks. In addition to all these limitations, most of previous checking factuality models try to investigate the veracity of a claim posted by user separately without taking other user's claim into account. To address the claim issues, this paper utilizes reinforcement learning to generate Toulmin Arguments that could impact many NLP-Related applications, e.g. factuality checking. To achieve this purpose, we propose novel models to recognize the uncertainty of claims e.g., understanding uncertainty is a crucial stage to express the subjective view of the publisher. We will find to which degree extracting uncertainty values has the role to see the correct factual label and judge the confidence of the news publisher. Since Deep Neural Networks (DNN) have proven better performance to distinguish factual from nonfactual information, in this research, we will develop DNN based models that able to check claims veracity by adapting syntactic and semantic features to adhere fake news problem. In our project, we aim to generate a decoded sequence for Toulmin model arguments by variant deep learning models and employing reinforcement learning agents, e.g., Deep Q-Network (DQN) which help to generate more informative (attention) features and correct the decoded generated output for the generative model by enriching more contextual information to GAN model. Iterative learning for the generative model helps to approximate data distribution and create more realistic data samples and better-generated sentences. The remainder of this paper is organized as follows: Related works are discussed in section II, in section III, RL-GAN network is presented. In section IV, we present RL-GAN Based Toulmin Argument Generation and Analysis with Performance Comparison is illustrated in section V, finally we conclude this paper in section VI.
Uncertainty detection plays fundamental role to separate new discovering information from those a... more Uncertainty detection plays fundamental role to separate new discovering information from those are factual information. Uncertainty texts have nonfactual information which could be useful for scientific purposes. Speculation and negation words affect the factuality degree. for example, negation reverses the factuality to non-factuality and vice versa and speculation words decrease or increase the degree of factuality. Lately, Deep Neural Networks (DNN) have proven competitive performance in NLP tasks generally and uncertainty detection tasks particularly. Due to the scarcity of labelled data for Arabic texts, most previous works have been examined to the English texts. To our knowledge, there is only one system has been implemented previously to identify the uncertainty statement for biomedical Arabic texts. This research will employ hierarchical attention mechanism to develop Generative Adversarial Networks (GAN)-based uncertainty detection model that able to detect speculated and negated parts in biomedical texts. in this paper, the implemented models are fed by some linguistic features like part of speech, constituency and dependency. The evaluation of this system will be based on the BIOARABIC corpus. Empirical experiments on BIOARABIC corpus confirm that GAN model with hierarchical attention achieve better performance (F-scores of 79) than previously implemented deep learning models (the best F-score 73.55). The proposed Hierarchical Attention Generative Adversarial Networks (HAGAN) shows that it produces richer representation of texts focusing on the shared features rather than specific features by training the generator and discriminator alternatively. Hierarchical attention mechanism proved its effectiveness to demonstrate long range dependencies concentrating on relevant parts of texts during the model training.
There are many reasons behind research on speculation and negation: there is a lot of irrelevant ... more There are many reasons behind research on speculation and negation: there is a lot of irrelevant (nonfactual) information, and a huge changing with new discovering information may strengthen or weaken previous knowledge. Speculation and negation values are considered as one of the main factors which play an essential role to predict the factuality of event or sentence. Negation reverses the truth of a statement to give the opposition and speculation increase or decreases the uncertainty of statement. Recently, Deep Neural Networks (DNN) have proven better performance to distinguish factual from nonfactual information. Most previous approaches have been dedicated to the English language. To our knowledge, there is no previous developed research to identify the negative or speculative expression for biomedical texts in the Arabic language. This research will develop DNN-based Speculation and negation detection models that able to check claims (negated or speculated sentences) by considering syntactic paths between speculation or negation cues and the remaining words (candidates) in biomedical texts, using Stanford dependency parser. In this paper, the implemented models are evaluated based on the BIOARABIC corpus. Experiments on BIOARABIC corpus show that DNN models achieve a competitive performance and the Attention based Bidirectional Long Short-Term Memory model achieves the best F-scores of 73.55.
There are many reasons behind research on speculation and negation: there is a lot of irrelevant ... more There are many reasons behind research on speculation and negation: there is a lot of irrelevant (nonfactual) information, and a huge changing with new discovering information may strengthen or weaken previous knowledge. Speculation and negation values are considered as one of the main factors which play an essential role to predict the factuality of event or sentence. Negation reverses the truth of a statement to give the opposition and speculation increase or decreases the uncertainty of statement. Recently, Deep Neural Networks (DNN) have proven better performance to distinguish factual from nonfactual information. Most previous approaches have been dedicated to the English language. To our knowledge, there is no previous developed research to identify the negative or speculative expression for biomedical texts in the Arabic language. This research will develop DNN-based Speculation and negation detection models that able to check claims (negated or speculated sentences) by considering syntactic paths between speculation or negation cues and the remaining words (candidates) in biomedical texts, using Stanford dependency parser. In this paper, the implemented models are evaluated based on the BIOARABIC corpus. Experiments on BIOARABIC corpus show that DNN models achieve a competitive performance and the Attention based Bidirectional Long Short-Term Memory model achieves the best F-scores of 73.55.
Negation and speculation are two common linguistic concepts in natural language processing field,... more Negation and speculation are two common linguistic concepts in natural language processing field, need more semantic understanding of texts. They are used to definite factuality of text. Negation is used to express the opposite of the text and the Speculation is used to determine the degree of certainty. Biomedical text mining is the main natural language processing application concerns with negation and speculation to distinguish between facts and uncertain or negated information in biomedical text. To our knowledge, there is no previous research on annotating Arabic biomedical text to identify the negative or speculative expression and no publicly available standard corpora of suitable size that are practical for evaluating the automatic detection of negation and speculation tools and scope determination. This paper presents produced corpus handling negation and speculative in Arabic biomedical texts with the main annotation (we call this corpus the BioArabic corpus). The goal of building BioArabic corpus is to help biologists and computational linguistics, who develop tools for identifying the negation and speculation, to train and evaluate these tools since in biomedical texts language, assumptions, experimental results and negative results are used extensively. We will report our statistics on corpus size and the consistency of annotations.
Uncertainty detection plays fundamental role to separate new discovering information from those a... more Uncertainty detection plays fundamental role to separate new discovering information from those are factual information. Uncertainty texts have nonfactual information which could be useful for scientific purposes. Speculation and negation words affect the factuality degree. for example, negation reverses the factuality to non-factuality and vice versa and speculation words decrease or increase the degree of factuality. Lately, Deep Neural Networks (DNN) have proven competitive performance in NLP tasks generally and uncertainty detection tasks particularly. Due to the scarcity of labelled data for Arabic texts, most previous works have been examined to the English texts. To our knowledge, there is only one system has been implemented previously to identify the uncertainty statement for biomedical Arabic texts. This research will employ hierarchical attention mechanism to develop Generative Adversarial Networks (GAN) -based uncertainty detection model that able to detect speculated and negated parts in biomedical texts. in this paper, the implemented models are fed by some linguistic features like part of speech, constituency and dependency. The evaluation of this system will be based on the BIOARABIC corpus. Empirical experiments on BIOARABIC corpus confirm that GAN model with hierarchical attention achieve better performance (F-scores of 79) than previously implemented deep learning models (the best F-score 73.55). The proposed Hierarchical Attention Generative Adversarial Networks (HAGAN) shows that it produces richer representation of texts focusing on the shared features rather than specific features by training the generator and discriminator alternatively. Hierarchical attention mechanism proved its effectiveness to demonstrate long range dependencies concentrating on relevant parts of texts during the model training.
Recognizing the entailment relation showed that its influence to extract the semantic inferences ... more Recognizing the entailment relation showed that its influence to extract the semantic inferences in wide-ranging natural language processing domains (text summarization, question answering, etc.) and enhanced the results of their output. For Arabic language, few attempts concerns with Arabic entailment problem. This paper aims to increase the entailment accuracy for Arabic texts by resolving negation of the text-hypothesis pair and determining the polarity of the text-hypothesis pair whether it is Positive, Negative or Neutral. It is noticed that the absence of negation detection feature gives inaccurate results when detecting the entailment relation since the negation revers the truth. The negation words are considered stop words and removed from the text-hypothesis pair which may lead wrong entailment decision. Another case not solved previously, it is impossible that the positive text entails negative text and vice versa. In this paper, in order to classify the text-hypothesis pair polarity, a sentiment analysis tool is used. We show that analyzing the polarity of the text-hypothesis pair increases the entailment accuracy. to evaluate our approach we used a dataset for Arabic textual entailment (ArbTEDS) consisted of 618 text-hypothesis pairs and showed that the Arabic entailment accuracy is increased by resolving negation for entailment relation and analyzing the polarity of the text-hypothesis pair.
Published Articles by Fatima Alkhawaldeh
Each argument begins with a conclusion, which is followed by one or more premises supporting the ... more Each argument begins with a conclusion, which is followed by one or more premises supporting the conclusion. The warrant is a critical component of Toulmin's argument model; it explains why the premises support the claim. Despite its critical role in establishing the claim's veracity, it is frequently omitted or left implicit, leaving readers to infer. We consider the problem of producing more diverse and high-quality warrants in response to a claim and evidence. To begin, we employ BART [1] as a conditional sequence tosequence language model to guide the output generation process. On the ARCT dataset [2], we fine-tune the BART model. Second, we propose the Multi-Agent Network for Warrant Generation as a model for producing more diverse and high-quality warrants by combining Reinforcement Learning (RL) and Generative Adversarial Networks (GAN) with the mechanism of mutual awareness of agents. In terms of warrant generation, our model generates a greater variety of warrants than other baseline models. The experimental results validate the effectiveness of our proposed hybrid model for generating warrants.
Uploads
Papers by Fatima Alkhawaldeh
Published Articles by Fatima Alkhawaldeh