Engineering, Technology & Applied Science Research Vol. 15, No.

3, 2025, 23253-23259 23253

Enhancing Fake News Detection with

Transformer Models and Summarization
Abdelhalim Saadi
Faculty of Technology, Setif 1 University – Ferhat Abbas, Algeria
[email protected] (corresponding author)

Hacene Belhadef
Department of Fundamental Computing and its Applications, Faculty of New Technologies of
Information and Communication, University of Abdelhamid Mehri – Constantine 2, Algeria
[email protected]

Akram Guessas
Department of Fundamental Computing and its Applications, Faculty of New Technologies of
Information and Communication, University of Abdelhamid Mehri – Constantine 2, Algeria
[email protected]

Oussama Hafirassou
Department of Fundamental Computing and its Applications, Faculty of New Technologies of
Information and Communication, University of Abdelhamid Mehri – Constantine 2, Algeria
[email protected]
Received: 22 February 2025 | Revised: 23 March 2025 and 14 April 2025 | Accepted: 19 April 2025
Licensed under a CC-BY 4.0 license | Copyright (c) by the authors | DOI: https://doi.org/10.48084/etasr.10678

ABSTRACT
This study evaluates the performance of transformer-based models such as BERT, RoBERTa, and XLNet
for fake news detection. Using supervised and unsupervised deep learning techniques, we optimized
classification accuracy while reducing computational costs through text summarization. The results show
that RoBERTa, fine-tuned with summarized content, achieves 98.39% accuracy, outperforming the other
models. Additionally, we assessed AI-generated misinformation using GPT-2, confirming that transformer
models effectively distinguish real from synthetic news. We utilized the GPT-2 model instead of more
recent models like GPT-4, as our objective was to generate fake news locally and compare it with
pretrained models from the same time period.
Keywords-fake news detection; NLP; DL; transformers; RoBERTa; GPT-2; text classification

I. INTRODUCTION GPT-2, and T5, which have demonstrated significant

improvements in text classification and information retrieval.
The proliferation of fake news has become a pressing
concern in today's digital information age. With the rapid The objective of this research is to evaluate the
expansion of online platforms and social media, effectiveness of transformer-based DL models in detecting fake
misinformation spreads quickly [1], influencing public opinion, news. By leveraging large-scale datasets, we train and fine-tune
shaping political landscapes, and affecting economic decisions. models to classify news content accurately. Additionally, we
Traditional methods of fact-checking struggle to keep pace explore text summarization techniques to enhance efficiency,
with the volume and speed of misinformation dissemination [2] enabling models to process large amounts of textual data while
[3], necessitating automated and scalable solutions. Deep reducing computational requirements. Furthermore, we
Learning (DL) and Natural Language Processing (NLP) have investigate the role of AI-generated content in spreading
emerged as powerful tools in the fight against fake news. These misinformation by analyzing fake news generated using GPT-
technologies enable the development of automated detection 2.
systems capable of distinguishing between real and fabricated
II. RELATED WORKS
news articles. Among the most effective models for NLP, tasks
are transformer-based architectures, such as BERT, RoBERTa, The increasing spread of fake news has motivated extensive
research on automated detection techniques. Early approaches

www.etasr.com Saadi et al.: Enhancing Fake News Detection with Transformer Models and Summarization
 Engineering, Technology & Applied Science Research Vol. 15, No. 3, 2025, 23253-23259 23254

to fake news classification primarily relied on rule-based selection, preprocessing techniques, model architectures, and
systems and linguistic feature analysis. These methods, training procedures employed in the research.
however, struggled with scalability and adaptability to evolving
misinformation patterns [4]. The rise of Machine Learning A. Dataset Selection
(ML) and DL techniques has significantly improved fake news Two primary datasets were used for training and evaluation:
detection by enabling models to learn complex linguistic and
 Gonzalo/Fake News Dataset: This dataset consists of real
contextual patterns [5].
and fake news articles, providing a balanced corpus for
Several studies have explored traditional ML techniques model training. It includes both headlines and full article
such as Support Vector Machines (SVM), Naïve Bayes (NB), texts, allowing models to learn contextual differences
and Decision Trees (DT) for fake news classification [6]. While between genuine and fabricated news. They contain 40587
these models achieved moderate success, they often require articles [13].
extensive feature engineering and lack the ability to generalize
well across different datasets. DL models, particularly  CC_News Dataset: Is a collection of real news articles used
transformers, has provided more robust solutions for handling to generate synthetic fake news samples using GPT-2. This
large-scale textual data with minimal manual feature extraction. dataset helps in assessing how well models can differentiate
One of the most influential transformer models in NLP is between AI-generated fake news and human-written
BERT (Bidirectional Encoder Representations from articles. It contains 708241 English language news articles
Transformers), which introduced bidirectional contextual published between Jan 2017 and December 2019 [14].
learning [7], allowing models to understand words based on B. Data Preprocessing
both preceding and succeeding context. Its fine-tuned versions, Preprocessing was a crucial step in optimizing the models
such as RoBERTa, have demonstrated superior performance in for classification accuracy. The following preprocessing
text classification tasks, including fake news detection. techniques were applied:
RoBERTa removes the Next Sentence Prediction (NSP) task
from BERT’s training process and incorporates dynamic  Text Cleaning: Removal of special characters, HTML tags,
masking, leading to more efficient and accurate stopwords, and non-alphabetic tokens.
representations.
 Tokenization: Splitting text into individual words or
Another important approach to fake news detection subwords using the BERT tokenizer, ensuring compatibility
involves Generative Pre-trained Transformers (GPT-2 and with transformer-based models.
GPT-3). These models, while primarily designed for text
generation, have also been investigated for their ability to  Summarization: Application of the T5 model to generate
generate and detect misinformation [8]. Research has shown concise summaries of news articles, reducing computational
that AI-generated fake news exhibits linguistic patterns that overhead while preserving key information.
distinguish it from human-written content, enabling models to  Padding and Truncation: Standardization of input sequence
classify news authenticity effectively. Additionally, some lengths to match the transformer model's requirements.
studies have proposed hybrid models, combining BERT-based
feature extraction with Recurrent Neural Networks (RNNs) or  Label Encoding: Assigning numerical values to categorical
Long Short-Term Memory (LSTM) networks for improved labels (1 for real news, 0 for fake news).
classification [9, 10]. By leveraging contextual embeddings
C. Model Selection and Architecture
from BERT and sequential dependencies captured by LSTMs,
these models achieve higher accuracy in fake news The following state-of-the-art transformer models were
classification tasks. evaluated for fake news detection [15]:
Recent advancements have explored multi-modal fake news  BERT: A bidirectional model pre-trained on vast textual
detection [11], integrating textual analysis with image and corpora, fine-tuned for classification tasks.
video processing. Given the rise of misinformation in
 RoBERTa: An optimized variant of BERT with dynamic
multimedia formats, future research directions may involve
masking, enhancing contextual understanding and
incorporating visual and contextual cues alongside textual
classification accuracy.
analysis for more comprehensive fake news detection systems.
Figure 1 in [12] illustrates the Transformer architecture,  XLNet: is a transformer-based language model that
which consists of an encoder-decoder structure leveraging self- improves BERT by using a permutation-based training
attention mechanisms and feed-forward neural networks. This approach. Unlike BERT, which masks words and predicts
architecture forms the basis for advanced NLP models such as them independently, XLNet considers all possible word
BERT, RoBERTa, GPT-2, and T5, which have been orderings during training, allowing it to better capture
extensively used in fake news detection. context and dependencies. It also retains the benefits of
autoregressive models while handling bidirectional context,
III. METHODOLOGY leading to improved performance on various NLP tasks.
The methodology of this study is designed to systematically Figure 1 illustrates the complete workflow of the proposed
evaluate the performance of transformer-based DL models in fake news detection system, starting from data acquisition and
fake news detection. This section describes the dataset preprocessing to model training and evaluation. The pipeline

www.etasr.com Saadi et al.: Enhancing Fake News Detection with Transformer Models and Summarization
 Engineering, Technology & Applied Science Research Vol. 15, No. 3, 2025, 23253-23259 23255

incorporates text summarization using a fine-tuned T5 model,

followed by classification with RoBERTa (and BERT and
XLNet), and a final comparison between two trained models to
determine the best-performing approach.
RoBERTa demonstrated the highest accuracy and
efficiency in classifying fake news, making it the primary
model for final deployment.

RoBERTa

Fig. 2. Architecture of the proposed methodology.

IV. EXPERIMENTS AND RESULTS

This section presents the experimental setup, evaluation
metrics, and performance analysis of the transformer-based
models employed for fake news detection. The objective is to
assess the classification accuracy, computational efficiency,
and robustness of various NLP models in identifying
Fig. 1. Fake news detection workflow using transformer models. misinformation.
D. Model Training and Hyperparameter Tuning A. Experimental Setup
The selected models were fine-tuned using supervised The models were implemented using PyTorch [16] and the
learning on the prepared datasets. Key training configurations Hugging Face Transformers library [17]. Training and
included: evaluation were conducted on a high-performance computing
environment, utilizing an NVIDIA RTX 5000 GPU with 32GB
 Optimizer: AdamW with a learning rate of 1.25e-6. RAM. The Paperspace Gradient cloud service was used for
 Batch Size: 16, chosen to balance memory efficiency and model fine-tuning.
convergence speed. The dataset was split into training (60%), validation (20%),
and testing (20%) subsets to ensure a fair evaluation of model
 Loss Function: Cross-entropy.
performance. The training process was conducted over three
 Early Stopping: Implemented to prevent overfitting, halting epochs, with early stopping enabled to prevent overfitting.
training if validation loss does not improve for three
B. Evaluation Metrics
consecutive epochs.
The following metrics were used to evaluate the model's
 Evaluation Metrics: Accuracy, F1-score, precision, and performance:
recall were used to measure model performance.
 Accuracy: Measures the overall percentage of correct
This methodology ensures a structured and efficient classifications.
approach to fake news detection, leveraging cutting-edge NLP
models and automated text processing. The next section  Precision: Evaluates the proportion of correctly classified
presents the experimental results and performance evaluation fake news instances.
of the proposed models.  Recall: Measures the ability to correctly identify fake news
Figure 2 depicts the various layers of our architecture, cases.
starting with the input layer, which represents the contextual
 F1-score: The harmonic mean of precision and recall,
word embeddings generated by the RoBERTa model. This is
providing a balanced assessment of classification
followed by an LSTM layer (768×256) and a classifier
performance.
consisting of three hidden layers and a binary output layer.

www.etasr.com Saadi et al.: Enhancing Fake News Detection with Transformer Models and Summarization
 Engineering, Technology & Applied Science Research Vol. 15, No. 3, 2025, 23253-23259 23256

These metrics ensure a comprehensive evaluation of the closely following the training accuracy. This confirms the
models' capabilities in detecting fake news. model’s strong generalization capability, ensuring high
reliability in distinguishing real and fake news.
C. Model Performance Comparison
About the training data, we used two datasets extracted D. Impact of Summarization on Model Efficiency
from the GonzaloA/Fake News dataset, Dataset01 that To optimize computational resources, T5-based text
contained the concatenated title and body, and Dataset02 which summarization was applied before classification. The
contained the summarized body (preprocessing). summarized dataset allowed models to process shorter input
sequences, reducing training time while maintaining
Model01 is the RoBERTa -base model fine-tuned with classification accuracy.
Dataset01 and Model02 is the RoBERTa -base model fine-
tuned with Dataset02.  Without summarization, RoBERTa's training time was 55
minutes.
TABLE I. RESULTS OBTAINED FROM THE TWO  With summarization, the training time decreased to 14
RETURNED MODELS TESTING WITH HEADLINES
minutes, demonstrating a 75% reduction in computational
Pretrained Accuracy F1-score Training cost.
Model Dataset
model (%) (%) time (min)
Model01 DataSet01 RoBERTa-base 91.58 92 55
This finding highlights the benefit of summarization in
Model02 DataSet02 RoBERTa-base 97.87 98.2 14 improving processing efficiency without compromising
accuracy.
TABLE II. RESULTS OBTAINED FROM THE TWO E. Analysis of AI-Generated Fake News Detection
RETURNED MODELS TESTING WITH NEWS BODIES
To evaluate the ability of transformer models to detect AI-
Pretrained Accuracy F1-score Training
Model Dataset
model (%) (%) time (min)
generated fake news, a separate experiment was conducted
Model01 DataSet01 RoBERTa-base 98.39 98 55 using BERT and XLNet for unseen data. The results showed
Model02 DataSet02 RoBERTa-base 98.18 98 14 that RoBERTa maintained an accuracy of 97.03%, while
BERT and XLNet showed respectively an accuracy of 84.21%
Table I presents the results of testing the models using and 81.66%. The RoBERTa pre-trained model required a
headlines. Model02 demonstrates a high F1-score of 98%, with longer training time compared to BERT but it achieved better
an overall accuracy of 97.87%. In contrast, Model01 yields the generalization. XLNet, on the other hand, had the highest
lowest performance, with an F1-score of 92% and an overall training time and memory consumption due to its
accuracy of 91.58%. Table II displays the results obtained from autoregressive-like architecture.
testing the models using news bodies. Both models achieve an Figure 4 illustrates the precision, recall, and F1-score of the
identical F1-score of 98%. However, in terms of overall proposed system. The high scores across all metrics indicate
accuracy, Model01 slightly outperforms Model02, attaining strong classification performance, confirming the ability of our
98.39% compared to Model02's 98.18%. model to accurately detect both real and fake news. The high
number of correctly classified instances in both classes
demonstrates the model's effectiveness. The minimal false
positive and false negative values indicate a well-balanced
classification capability.

Fig. 3. Training and validation loss/accuracy of the proposed model.

Figure 3 illustrates the training and validation loss, as well

as the accuracy evolution of our best-performing model during
the training process. The steady decrease in both training and
validation loss indicates efficient learning with minimal Fig. 4. Classification report of the proposed model.
overfitting. Simultaneously, the accuracy curves show a
significant improvement over epochs, with validation accuracy

www.etasr.com Saadi et al.: Enhancing Fake News Detection with Transformer Models and Summarization
 Engineering, Technology & Applied Science Research Vol. 15, No. 3, 2025, 23253-23259 23257

To evaluate the ability of the proposed transformer model A. Key Findings

to detect fake news, another experiment was conducted using
SVM, NB, and DT for fake news classification for unseen data. The experimental results confirm that RoBERTa
The results can be seen Tables III-V. outperforms other models in fake news classification, achieving
the highest accuracy (97.87%) and F1-score (98.2%). The
TABLE III. NB PERFORMANCE superior performance of RoBERTa can be attributed to its
optimized pre-training process, which removes the NSP task
Precision Recall F1-score and implements dynamic masking, allowing it to generalize
0 0.78 0.93 0.85
better for classification tasks. Additionally, the use of text
1 0.91 0.75 0.82
Accuracy 0.84
summarization significantly reduced computational costs
Macro avg 0.85 0.84 0.84 without affecting classification accuracy. This indicates that
Weighted avg 0.85 0.84 0.84 reducing input sequence length through summarization
techniques such as T5-based text compression can enhance
TABLE IV. SVM PERFORMANCE model efficiency while preserving the ability to distinguish
between real and fake news.
Precision Recall F1-score
0 0.93 0.89 0.90 The study also demonstrated that AI-generated fake news,
1 0.89 0.93 0.91 particularly content generated by GPT-2, remains detectable
Accuracy 0.91
using transformer models. RoBERTa maintained 97.03%
Macro avg 0.91 0.91 0.91
Weighted avg 0.91 0.91 0.91
accuracy when distinguishing between real and AI-generated
misinformation. This highlights the potential of deep learning
TABLE V. DT PERFORMANCE models in counteracting AI-driven disinformation.
Precision Recall F1-score We compared our approach with key ML techniques
0 0.87 0.61 0. 72 commonly used in fake news detection, such as SVM, NB, and
1 0.70 0.91 0.79 DT. The results demonstrated that our approach outperformed
Accuracy 0.76 the other methods.
Macro avg 0.79 0.76 0.76
Weighted avg 0.79 0.76 0.76 B. Implications for Fake News Detection
The findings of this research have several practical
The results indicate that transformer-based models implications for fake news detection systems and digital media
outperform the SVM, NB, and DT methods, which achieved regulation:
accuracy values significantly lower than the accuracy achieved
by the proposed RoBERTa model.  Automated Fake News Detection: The deployment of DL-
based detection systems in social media platforms and news
F. Discussion agencies can enhance the identification of misinformation,
reducing its impact on public perception.
The results confirm that transformer-based models
significantly enhance fake news detection accuracy. The  Efficient Computational Approaches: The integration of
findings indicate that: text summarization can optimize processing time, making
real-time detection feasible, especially for large-scale news
 RoBERTa is the most effective model due to its optimized aggregation platforms.
training and dynamic masking strategy.
 Countering AI-Generated Fake News: As AI-generated
 Text summarization reduces training time while misinformation becomes increasingly sophisticated, the
maintaining high classification performance. ability of transformer models to detect these threats
 AI-generated fake news remains detectable, though provides a first line of defense against AI-driven
continuous improvements are necessary to adapt to propaganda.
evolving misinformation tactics. From a practical perspective, these findings can benefit
These insights highlight the potential of DL models to industries such as e-commerce, customer service, and market
address the growing challenge of misinformation detection in research by improving sentiment analysis, automated review
digital media. The next section discusses the implications of classification, and personalized recommendation systems.
these findings and potential improvements for future work. Additionally, the methodology can be applied to other domains,
including healthcare and finance, where opinion mining and
V. DISCUSSION text classification are critical for decision-making.
The results obtained in the previous section demonstrate the C. Limitations
effectiveness of transformer-based models in fake news Despite the promising results, several limitations must be
detection. This section discusses the key findings, their acknowledged:
implications, limitations of the current approach, and potential  Dataset Bias: The models were trained on specific datasets,
directions for future research. which may not fully capture the diversity of fake news
formats across different languages and regions.

www.etasr.com Saadi et al.: Enhancing Fake News Detection with Transformer Models and Summarization
 Engineering, Technology & Applied Science Research Vol. 15, No. 3, 2025, 23253-23259 23258

 Generalization Issues: While RoBERTa performed developing cross-lingual fake news detection systems, real-
exceptionally well on the selected dataset, its effectiveness time deployment solutions, and multimodal deep learning
on real-world, unseen misinformation needs further approaches to enhance the robustness and applicability of
validation. misinformation detection technologies.
 Lack of Multimodal Analysis: This study focused solely on In conclusion, this study contributes to the growing field of
text-based detection. Fake news often includes multimedia AI-driven fake news detection by demonstrating the efficiency
elements (images, videos, deepfake content), which were of RoBERTa and text summarization in classification tasks.
not considered in this work. The results provide a strong foundation for future
advancements in NLP-based misinformation detection, paving
D. Future Research Directions the way for more reliable and scalable solutions in combating
Our study demonstrates that combining advanced word digital disinformation.
embeddings (RoBERTa, BERT, and XLNet) with LSTM
REFERENCES
classifiers significantly improves fake news classification
accuracy and robustness compared to traditional ML methods [1] H. Allcott and M. Gentzkow, "Social Media and Fake News in the 2016
Election," Journal of Economic Perspectives, vol. 31, no. 2, pp. 211–
such as SVM, NB, and DT. This finding reinforces the value of 236, May 2017, https://doi.org/10.1257/jep.31.2.211.
DL-based hybrid models for handling uncertain [2] M. Potthast, J. Kiesel, K. Reinartz, J. Bevendorff, and B. Stein, "A
recommendations and complex textual data. To further enhance Stylometric Inquiry into Hyperpartisan and Fake News." arXiv, Feb. 18,
fake news detection systems, future research should focus on 2017, https://doi.org/10.48550/arXiv.1702.05638.
the following aspects: [3] H. F. Villela, F. Corrêa, J. S. de A. N. Ribeiro, A. Rabelo, and D. B. F.
Carvalho, "Fake news detection: a systematic literature review of
 Multimodal Fake News Detection: Integrating image, machine learning algorithms and datasets," Journal on Interactive
video, and text analysis using multimodal DL techniques to Systems, vol. 14, no. 1, pp. 47–58, Mar. 2023, https://doi.org/
improve detection accuracy. 10.5753/jis.2023.3020.
[4] Y. Dou, K. Shu, C. Xia, P. S. Yu, and L. Sun, "User Preference-aware
 Cross-Language Adaptability: Expanding datasets to Fake News Detection," in Proceedings of the 44th International ACM
multilingual news sources to create robust models capable SIGIR Conference on Research and Development in Information
of detecting misinformation in multiple languages. Retrieval, New York, NY, USA, Apr. 2021, pp. 2051–2055,
https://doi.org/10.1145/3404835.3462990.
 Real-Time Deployment: Implementing lightweight DL [5] C. Raffel et al., "Exploring the Limits of Transfer Learning with a
models for deployment on mobile and web-based Unified Text-to-Text Transformer." arXiv, Sep. 19, 2023,
https://doi.org/10.48550/arXiv.1910.10683.
applications to facilitate instant fake news verification.
[6] N. Rai, D. Kumar, N. Kaushik, C. Raj, and A. Ali, "Fake News
 Our architecture can be used in other fields of natural Classification using transformer based enhanced LSTM and BERT,"
language processing like sentiment analysis, where the International Journal of Cognitive Computing in Engineering, vol. 3, pp.
98–105, Jun. 2022, https://doi.org/10.1016/j.ijcce.2022.03.003.
encouraging results in [18] showed the impact of utilizing
[7] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-
RoBERTa. training of Deep Bidirectional Transformers for Language
Understanding." arXiv, May 24, 2019, https://doi.org/10.48550/arXiv.
VI. CONCLUSION 1810.04805.
The increasing prevalence of fake news poses a significant [8] E. Mustafaraj and P. T. Metaxas, "The Fake News Spreading Plague:
Was it Preventable?" arXiv, Mar. 20, 2017, https://doi.org/
challenge in the digital era, influencing public opinion, political 10.48550/arXiv.1703.06988.
landscapes, and societal trust. This study explored the
[9] J. A. Nasir, O. S. Khan, and I. Varlamis, "Fake news detection: A hybrid
effectiveness of transformer-based deep learning models in CNN-RNN based deep learning approach," International Journal of
detecting misinformation, especially RoBERTa which Information Management Data Insights, vol. 1, no. 1, Apr. 2021, Art.
outperformed the other considered transformer models. The no. 100007, https://doi.org/10.1016/j.jjimei.2020.100007.
integration of text summarization techniques further optimized [10] N. K. Conroy, V. L. Rubin, and Y. Chen, "Automatic deception
computational efficiency without compromising detection detection: Methods for finding fake news," Proceedings of the
accuracy. Association for Information Science and Technology, vol. 52, no. 1, pp.
1–4, 2015, https://doi.org/10.1002/pra2.2015.145052010082.
The results confirm that deep learning models, particularly [11] S. Kumari and M. P. Singh, "A Deep Learning Multimodal Framework
transformers, provide a powerful approach to fake news for Fake News Detection," Engineering, Technology & Applied Science
detection. The ability of RoBERTa to classify both human- Research, vol. 14, no. 5, pp. 16527–16533, Oct. 2024, https://doi.org/
10.48084/etasr.8170.
written and AI-generated fake news highlights its potential as a
[12] A. Vaswani et al., "Attention Is All You Need." arXiv, Aug. 02, 2023,
robust misinformation filtering tool. Additionally, the use of https://doi.org/10.48550/arXiv.1706.03762.
GPT-2 for fake news generation and subsequent detection [13] G0nz4lo-4lvarez-H3rv4s, "G0nz4lo-4lvarez-
demonstrated that AI-driven misinformation is distinguishable H3rv4s/FakeNewsDetection." [Online]. Available: https://github.com/
using advanced NLP techniques. G0nz4lo-4lvarez-H3rv4s/FakeNewsDetection.
[14] S. Nagel, "News Dataset Available," Common Crawl, Oct. 04, 2016.
While the findings are promising, several challenges https://commoncrawl.org/blog/news-dataset-available.
remain, including dataset biases, generalization issues, and the [15] S. Raza, D. Paulen-Patterson, and C. Ding, "Fake News Detection:
need for multimodal analysis incorporating images, videos, and Comparative Evaluation of BERT-like Models and Large Language
social media behaviors. Future research should focus on

www.etasr.com Saadi et al.: Enhancing Fake News Detection with Transformer Models and Summarization
 Engineering, Technology & Applied Science Research Vol. 15, No. 3, 2025, 23253-23259 23259
Models with Generative AI-Annotated Data." arXiv, Dec. 20, 2024,
https://doi.org/10.48550/arXiv.2412.14276.
[16] J. Jouhar, A. Pratap, N. Tijo, and M. Mony, "Fake News Detection using
Python and Machine Learning," Procedia Computer Science, vol. 233,
pp. 763–771, Jan. 2024, https://doi.org/10.1016/j.procs.2024.03.265.
[17] D. Paper, "Introduction to Deep Learning," in TensorFlow 2.x in the
Colaboratory Cloud: An Introduction to Deep Learning on Google’s
Cloud Service, D. Paper, Ed. Berkeley, CA, USA: Apress, 2021, pp. 1–
24.
[18] P. Pookduang, R. Klangbunrueang, W. Chansanam, and T. Lunrasri,
"Advancing Sentiment Analysis: Evaluating RoBERTa against
Traditional and Deep Learning Models," Engineering, Technology &
Applied Science Research, vol. 15, no. 1, pp. 20167–20174, Feb. 2025,
https://doi.org/10.48084/etasr.9703.

www.etasr.com Saadi et al.: Enhancing Fake News Detection with Transformer Models and Summarization