ISSN 1816-6075 (Print), 1818-0523 (Online)

Journal of System and Management Sciences

Vol. 13 (2023) No. xx, pp. xxx-xxx

Sustainability City Assessment Using Crowdsourced Social Media

Data: A Case Study of the Java Island, Indonesia

Usman Ependi 1,2, Adian Fatchur Rochim 3 and Adi Wibowo 4

1
Doctoral Program of Information Systems, Universitas Diponegoro, Semarang, Indonesia
2
Faculty of Science and Technology, Universitas Bina Darma, Palembang, Indonesia
3
Faculty of Engineering, Universitas Diponegoro, Semarang, Indonesia
4
Faculty of Science and Mathematics, Universitas Diponegoro, Semarang, Indonesia

(Received xxx 2023, accepted xxx 2023, will be set by the editor)

Abstract. Journal of System and Management Sciences (JSMS) is a multidisciplinary journal, mainly
publishes original research and applied papers on system and management sciences, including the
application of system science and system engineering, mathematics and statistics, computer and
information, operation research and management, modelling and simulation. Journal of System and
Management Sciences (JSMS) is a multidisciplinary journal, mainly publishes original research and
applied papers on system and management sciences, including the application of system science and
system engineering, mathematics and statistics, computer and information, operation research and
management, modelling and simulation. Journal of System and Management Sciences (JSMS) is a
multidisciplinary journal, mainly publishes original research and applied papers on system and
management sciences, including the application of system science and system engineering, mathematics
and statistics, computer and information, operation research and management, modelling and simulation.
Journal of System and Management Sciences (JSMS) is a multidisciplinary journal, mainly publishes
original research and applied papers on system and management sciences, including the application of
system science and system engineering, mathematics and statistics, computer and information, operation
research and management, modelling and simulation.
Keywords: one, two, three, etc. (Use “keywords” style)

1. Introduction
Over the past decade, the concept of smart cities has gained prominence in urban development, aimed at
improving the quality of life for inhabitants. The rise of smart city concepts can be attributed to the fact
that more than half of the world's population lives in urban areas, and these urban areas consume 75% of
the world's total energy consumption (Kourtzanidis et al., 2021). The focus of smart cities lies in the
innovative use of technology and data to create urban environments that are effective, productive, and
sustainable. Smart cities have multiple objectives, including acting as a catalyst for urban development,
taking an integrated approach to physical, social, and digital planning, predicting and identifying
emerging challenges, and enhancing integrated services and innovation potential within city institutions
(Bayar et al., 2020). The concept of smart cities is rooted in the general system theory, which views a city
as a system that comprises various subsystems, each representing a facet of the smart city (Lom, 2021).
The characteristics of a smart city encompass sustainability, urbanization, quality of life, and smartness,
and serve as the foundation for evaluating a city's level of intelligence through a smart city assessment
(Khan et al., 2022).
The evaluation of smart cities is a crucial method for assessing their development prospects (Patrão et
al., 2020). The primary goal of smart city assessment is to provide direction and feedback for decision-
making, as well as to evaluate whether the implementation is moving in the intended direction
(Ahvenniemi et al., 2017). Frameworks and data sources serve as essential tools for the smart city
assessment process. According to (Sharifi, 2020a), there are various data sources within the smart city
assessment framework, including primary and secondary data. Primary and secondary data are sourced
from city authorities (policy makers), investors, experts (academia), and the public. Currently, primary
data is gathered through methods such as questionnaires, interview, surveys, in-situ analysis, field
observation, and photographic documentation. Meanwhile, secondary data includes census data, city audit
results, annual reports, and historical data records. The smart city assessment process relies on primary or
secondary data sources, but these have their own limitations. Primary data may be costly to collect, while
secondary data may not undergo as intensive an assessment process and may be of poor quality (Sharifi,
2020b). However, the most significant drawback of the smart city assessment framework is its inability to
handle or assess the interrelationships between different subsystems of the urban system (Sharifi, 2019).
According to (Sharifi, 2020a), the primary data sources for smart city assessment consist mostly of
government statistical data, which is static in nature. However, the use of static data has certain
weaknesses, such as the need for time-consuming data updates and the inability to reflect real-time
changes in the city. Furthermore, it is challenging to establish an integrated evaluation standard for
different cities (Wu et al., 2021). Smart city assessment based on statistical data also has limitations.
Firstly, the categories (dimensions) and contents of indicators in the conventional evaluation system
cannot be standardized, since statistical data is determined by different city governments and thus has
varying definitions. Secondly, the use of statistical data in conventional evaluation methods does not
allow for real-time monitoring of smart city development (Wu et al., 2021).
Research suggests that utilizing big data through crowdsourcing can address the weaknesses and
limitations of the smart city assessment data source, as outlined by (Sharifi, 2019) and (Dawodu et al.,
2022). Crowdsourced data offers several advantages as a solution. Firstly, it is a component of big data
and can enhance the capacity for developing more precise models or scenarios in smart city assessment.
Secondly, using big data through crowdsourcing can provide a better understanding of complex urban
dynamics and offer opportunities to consider inter-indicator relationships (Sharifi, 2019). Thirdly,
crowdsourced data can be instantly provided and processed in real-time for smart city assessment, thereby
reducing data collection costs and ensuring the data's quality. Lastly, social media crowdsourced data
presents potential for big data in smart city assessment (Wu et al., 2021), (Zhou et al., 2021).
Social media crowdsourced data has been utilized for evaluating city intelligence by (Wu et al.,
2021). The assessment approach employed for smart city assessment based on social media crowdsourced
data is sentiment analysis (Zhou et al., 2021). However, the possibility of using crowdsourced data for
assessing city sustainability has not been explored yet. On the other hand, social media crowdsourced data
for smart city assessment has tremendous potential, particularly in Indonesia. This potential is evident
from the last five years of data released by the Indonesian Central Bureau of Statistics, which indicates a
rapid growth in the usage of Information and Communication Technology (ICT) in the country. Several
indicators of ICT utilization in Indonesia demonstrate that household internet usage has reached 78.18
percent. As of March 2023, katadata.co.id estimates that there are around 204.7 million internet users in
Indonesia. The country's high number of internet users has the potential to generate social media data
from platforms such as Facebook, Instagram, Twitter, and others.
 In light of the issues, limitations, and focus surrounding smart city assessment, the aim of this study is
to develop a model for smart city assessment that places an emphasis on city sustainability. The
assessment model under development includes indicators for smart city assessment, as well as the
utilization of crowdsourced data from social media for assessment purposes. Furthermore, to evaluate the
efficacy of the assessment model, a assessment will be carried out on Java Island in Indonesia.
Ultimately, this research contributes to the field in a number of ways, including the development of a city
assessment model, the creation of smart city assessment indicators that focus on city sustainability, and
the establishment of a new standard for smart city assessment that employs crowdsourced data,
particularly through social media.

2. Literature Review
2.1. Smart City Assessment
According to (Neirotti et al., 2014), a smart city is an ecosystem consisting of various parts that are
automated. As per the General System Theory (GST), a city can be represented as a system with a tree
structure, comprising subsystems or functions. These functions can be categorized into two domains: hard
and soft. The hard domain includes energy, water, waste, transportation, environment, buildings, and
healthcare infrastructure, whereas the soft domain encompasses education, welfare, social capital, public
administration, employment, civil, and economic activities. Therefore, the key aspect of a smart city is the
interconnection of subsystems within a city system, as noted by (Rochet & Villechenon, 2014).
The assessment of a smart city is based on the application of the General System Theory (GST) to the
development of the smart city concept, as described by (Checkland & Haynes, 1994). GST provides a
language to express and share problems across multiple disciplines, making it applicable to various fields.
Accordingly, problem-solving in smart cities follows the GST system theory, where real-world situations
are addressed within the context of the hard and soft subsystem domains (Noori, Hoppe, et al., 2020).
Based on this problem-solving concept, the evaluation of a smart city is determined by its input-output
(IO) model.
The smart city IO model comprises four characteristics: inputs (resources), processes (throughputs),
outputs (applications), and outcomes (externalities). The inputs relate to human resources, knowledge and
creativity, information and communication technology (ICT) infrastructure, and financial assets. The
processes represent dynamic throughputs, as well as the governance and leadership capabilities to add
value to resources and transform them into outputs. The outputs correspond to applications, while the
externalities reflect the outcomes of the smart city IO processes, which contribute to the sustainability of
the city. Thus, assessing a smart city is crucial as it determines its sustainability, including environmental,
economic, and social sustainability.
The main purpose of evaluating a smart city is to provide feedback and guidance for decision-making,
enabling assessment of whether the implementation is heading towards the desired direction (Ahvenniemi
et al., 2017). The evaluation of a smart city is also a new field that has a lot of potential for future
development (Sharifi, 2019). The evaluation of a smart city can also provide performance monitoring for
evaluating various benefits for various actors and stakeholders, such as city authorities, investors and
funding institutions, researchers, and citizens (Patrão et al., 2020). The following are the benefits obtained
by each stakeholder in the evaluation of a smart city.
The assessment of smart cities has brought many benefits to various parties, such as city authorities,
investors and funding institutions, researchers, and society. City authorities benefit from the identification
of strengths and weaknesses in smart city development planning (Garau & Pavan, 2018). This allows
them to make informed decisions about the direction of development and to improve city transparency
(Fernandez-Anez et al., 2018), among other things. Understanding technical needs in smart cities also
enables authorities to better plan for the future (Debnath et al., 2014).
Investors and funding institutions benefit from their increased ability to determine investment
directions in smart cities. By identifying and utilizing new business opportunities, they can potentially
earn profits from the development of smart cities (Mohan et al., 2020). Scientific evidence in determining
funding allocations also allows them to make informed decisions about where to allocate their resources
(Caird et al., 2016). These benefits can help encourage investment and funding for smart city
development (Giffinger et al., 2010).
Researchers benefit from developing new strategies to improve smart city performance and
simplifying the complexity of smart cities (Caird et al., 2016). This can help them gain new insights into
smart city development and improve the effectiveness of their research (Akande, Cabral, & Casteleyn,
2019). With the growth of smart cities, there is a growing demand for research that can help improve their
performance and address challenges related to their development.
Society benefits from increased awareness related to smart cities, which can help them make
informed decisions about their investments and involvement in smart city development (Mohan et al.,
2020). By communicating their desires to the city authorities, they can help shape the development of
smart cities to better serve their needs (Caird et al., 2016). The involvement of society in smart city
development can also help promote transparency and accountability in the development process (Debnath
et al., 2014). In conclusion, the benefits of smart city development are extensive and touch on a variety of
aspects of society, economy, and governance.

2.2. Smart City Dimension

The dimensions used to evaluate the performance of a smart city are a critical component of the
assessment process. These dimensions are typically categorized into two types: those used to evaluate the
city's smartness and those that assess its sustainability. However, some assessments combine both types
of dimensions to provide a comprehensive evaluation of a smart city. Table 1 below illustrates various
assessment dimensions of a smart city that can be employed. These dimensions are developed by
researchers, smart city developers, or international organizations.
Table 1. Dimension of Smart City Assessment
N
Creator Dimension
o
Cohen (Cohen, 2022), (Pira, Environment, Mobility, Government, Economy,
1
2021a) People, dan Living
The Organization for Economic
Co-operation and Development Social, Environmental, Economic, dan Institutional
2
(OECD) (Hass et al., 2002), (Pira,
2021a)
Socio-cultural, Economic, Environmental, dan
3 Milad Pira (Pira, 2021a)
Governance
Intelligent Governance and Service, Intelligent
4 Zhiqiang Wu (Wu et al., 2021) Economy and Industry, Intelligent Hardware
Construction, dan Residents’ Intelligent Capacity
Sustainable Development Goal UN SDG target 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7,
5
11+ (Patrão et al., 2020) 11a, 11b, 11c, 1.4, 6.3
6 Center of Regional Science (Koca Smart environment, Smart economy, Smart
et al., 2021) governance, Smart people, Smart living, dan Smart
 N
Creator Dimension
o
mobility
Kimiya Rahmani Mokarrari, S.
environment, living, mobility, economy, people, dan
7 Ali Torabi (Kimiya & Torabi,
governance
2021)
Mohammed Agbali (Agbali et al.,
8 Infrastructure, Institution, dan People
2019)
NECE-ITU (Akande, Cabral,
9 Economy, Environment, dan Society
Gomes, et al., 2019)
Center for Globalization and Human capital, social cohesion, Economy,
Strategy dan IESE Business Governance, Environment
10
School’s Department of Strategy Mobility and transportation, Urban planning,
(Berrone & Ricart, 2018) international outreach, dan Technology
Institute for Urban Strategies economy, R&D, cultural interaction, livability,
11
(Takenaka & Ichikawa, 2018) environment, dan accessibility
European Telecommunications
Standards Institute (ESTI)
12 People, Planet, Prosperity, dan Governance
(Huovila et al., 2019), (ETSI,
2017)
economy, education, energy, environment and climate
International Standard change, finance, governance, health, housing,
Organization (ISO) (Huovila et population and social conditions, recreation, safety,
13
al., 2019), (Kristiningrum & solid waste, sport and culture, telecommunication,
Kusumo, 2021) transportation, urban/local agriculture and food
security, urban planning, wastewater, dan water
International Telecomunication
Union
14 Economy, Environment, dan Society & culture
ITU-T Y.4903/L.1603 (ITU-T,
2022), (Huovila et al., 2019)

2.3. Crowdsourced Data

Crowdsourced data is an effort to contribute bottom-up information from society to a particular media,
which is not limited only to geographic information (Crooks et al., 2015). Crowdsourced data has shown
potential in understanding urban activities and patterns underlying them and has been used to solve
complex problems or fill important gaps in data analysis that cannot be covered by traditional data in
urban problem analysis (Long & Liu, 2016). Crowdsourced data has high-frequency characteristics that
provide the latest information and can depict current conditions. In addition, crowdsourced data has a far
more cost-effective advantage than traditional data such as government surveys or censuses. The most
important thing about crowdsourced data is that it is generated voluntarily by the community and has rich
information related to urban activities (Niu & Silva, 2020).
According to (Niu & Silva, 2020), crowdsourced data has three types of sources: (1) social media
platforms including Facebook, Twitter, Instagram, and Sina Weibo; (2) POI data like Open-StreetMap,
business map services such as Google Places and Gaode maps, and check-in data from social networks
like Foursquare or Yelp; and (3) websites such as web services and open street maps. On the other hand,
(Havinga et al., 2020) lists four sources: (1) social media platforms like Flickr, Foursquare, Instagram,
Tencent QQ, Twitter, and Weibo; (2) outdoor activity-sharing platforms such as Condoon, Geocaching,
GPSies, MapMyFitness, Strava, and Wikiloc; (3) community knowledge portals such as eBird and
iNaturalist; and (4) cellular signal data from telecommunications companies.
 It is apparent that social media is a frequently used source of data in crowdsourcing. Consequently,
Figure 1 illustrates how crowdsourced data is obtained from social media platforms. The high frequency
of crowdsourced data is a characteristic of social media, which provides the latest information and a
depiction of current conditions. Moreover, crowdsourced data is far more cost-effective than traditional
data sources like surveys or government censuses. Crowdsourced data is volunteered by individuals from
the community, making it rich in information related to urban activities.
In summary, crowdsourced data provides a valuable tool for understanding urban activities and
patterns and has the potential to solve complex problems or fill important gaps in traditional data analysis
for urban issues. By utilizing social media as a data source, the city authorities, investors, funding
institutions, researchers, and society can reap various benefits such as increased city transparency,
scientific evidence in determining funding allocations, and the ability to communicate their desires to the
city authorities. The abundance of crowdsourced data from social media is a valuable asset that can assist
in developing new strategies and simplifying the complexity of smart cities.

Fig. 1.Crowdsourced Data Media Sosial (Jukka M. Krisp, Mathias Jahnke, 2014)

3. Material and Method

3.1. Material
The present study utilizes two types of materials: case studies and indicators. Case studies are used to
describe the assessment object, while indicators are used to explain the instruments utilized in the
assessment. The following sections provide detailed explanations for each of these materials.

3.1.1 Case Study

The present study utilizes a case study approach to evaluate the capital cities of provinces located on the
island of Java, Indonesia. The selection of these cities is based on their high population density in
comparison to other islands in the country. According to the National Central Bureau of Statistics, the
population of Java Island in 2022 comprises 56.10% or 151.59 million people out of a total population of
270.20 million in Indonesia. As a result, the research focuses on four cities: Jakarta, the capital of DKI
Jakarta Province; Semarang, the capital of Central Java Province; Bandung, the capital of West Java
Province; and Surabaya, the capital of East Java Province. The other two provincial capitals, Serang (the
capital of Banten Province) and Yogyakarta (the capital of DI Yogyakarta Province), were excluded from
the study due to the unavailability of official websites related to smart cities. Furthermore, other cities
besides provincial capitals were not considered to ensure equity in evaluating smart cities. Figure 2
displays the position of the case study in assessing sustainable cities.
Fig. 2.Object of Sustainablelity City Assessment

3.1.2 Assessment Indicators

The development of indicators for assessing sustainable cities is based on the Tree Bottom Line (TBL)
dimensions, which include social, economic, and environmental aspects (Zhou et al., 2021). According to
(Noori, de Jong, et al., 2020), these dimensions are used to evaluate how the impact or outcome of smart
city IOs affects the city's sustainability in the future. To develop the assessment indicators, a literature
review was conducted on smart city assessments (Ependi et al., 2022). The result of this development
produced twelve indicators, consisting of five indicators for social dimensions, four indicators for
economic dimensions, and three indicators for environmental dimensions. The indicators were sourced
from various standards or other assessment models.
The social dimensions were derived from several sources such as the Sustainable Development
Indicators (Pira, 2021b), Lisbon ranking for smart sustainable cities (Akande, Cabral, Gomes, et al.,
2019), IESE Cities in Motion Index (Berrone & Ricart, 2018), ITU-T Y.4903/L.1603 Indicators (ITU-T,
2022), and Sustainability Perspectives Indicators (Benites & Simões, 2021). The economic dimensions
were compiled from sources such as the Smart City Index Master Indicators Survey (Cohen, 2022),
Dimensions of the Smart City Vienna UT (Koca et al., 2021), Sustainability Perspectives Indicators
(Benites & Simões, 2021), Characteristics of Smart City Indicators (Purnomo et al., 2016), Criteria set for
evaluating smart cities (Kimiya & Torabi, 2021), Lisbon ranking for smart sustainable cities (Akande,
Cabral, Gomes, et al., 2019), IESE Cities in Motion Index (Berrone & Ricart, 2018), China smart city
performance (Shen et al., 2018), Juniper analysis of smart city frameworks (Fernandez-anez et al., 2018),
Smart City Dimension (Sharif & Pokharel, 2021), and Smart City Performance Index (Yigitcanlar et al.,
2022).
As for the environmental dimensions, they were sourced from Dimensions of the Smart City Vienna
UT (Koca et al., 2021), Criteria set for evaluating smart cities (Kimiya & Torabi, 2021), Assessing the
Effectiveness of Smart Transport (Gutman & Vorontsova, 2020), China Smart City Performance (Shen et
al., 2018), ITU-T Y.4902/L.1602 Indicator (Patrão et al., 2020), Smart City Dimension (Sharif &
Pokharel, 2021), and City Sustainability Assessment (Zhou et al., 2021). Table 1 shows the indicators
developed for sustainable city assessment in this study.
 Table 2. The Proposed Indicators of Sustainability City Assessment
Dimensions Indicators Crowdsourced Data (Keywords)
Equity (So1) housing, property
Health (So2) health, hospital, nutrition, sanitation, drinking water
Education (So3) education, literacy, schooling
Social (So)
security, unemployment, slavery, crime, criminality,
Security (So4)
peace, violence
Culture and equality (So5) culture, equality, population, female workers
Innovation (Ec1) entrepreneur, company, innovation, technology, industry
Income (Ec2) income, salary, employment, poverty rate, finances
Economy (Ec) Infrastructure (Ec3) infrastructure, cooperation, connections
economic performance, consumption, trade,
Business opportunity (Ec4)
competitiveness, productivity
Air (En1) air, pollution, emissions, defilement, waste
Environment renewable energy, electricity, green industry, solar
Energy (En2)
(En) energy
Public facilities (En3) green space, parks, city parks, vehicles, public transport

Table 2 showcases a comprehensive set of assessment indicators that were rigorously validated by a
panel of five academic experts. The experts, all possessing doctoral degrees and holding the rank of
associate professor or professor, were carefully selected to ensure their expertise in the field of smart city
and sustainability. Through their evaluation, it was confirmed that the indicators were well-developed and
fit for use in the sustainability city assessment. To provide a more detailed understanding of the validation
process, Appendix A presents an overview of the experts involved in the indicator validation. Their
backgrounds and areas of expertise were taken into account to ensure a diverse and well-rounded
perspective. The validation results show unanimous agreement among the experts, further affirming the
suitability of the assessment indicators. The thorough validation process adds credibility to the
assessment, providing a strong foundation for accurate and reliable sustainability assessments in the
future.

3.2. Method
The method employed in this research is focused on utilizing crowdsourced data for assessing city
sustainability, specifically in the context of smart city assessment. The aim of the assessment is to
examine the outcome or impact of smart city implementation based on the smart city IO developed by
each city. It is worth noting that the assessment does not consider how the smart city concept is developed
but rather looks at the impact on citizens and all stakeholders involved through crowdsourced data from
social media.
Figure 3 presents the proposed model used in this research, which has two main focuses: data
generation and visualization. Data generation is the process that creates crowdsourced data used for
assessment, and it can be contributed by citizens, NGOs, private sectors, and city authorities. Data
generation plays a crucial role in the assessment results as it reflects the perspectives of all parties in the
city regarding the developed smart city implementation.
Visualization is the assessment process that begins from data preparation to data analysis. The stages
in visualization can be explained as follows.
 Fig. 3. Proposed model for sustainablelity city assessment

3.2.1 Data Preparation

The process of data preparation commences with data collection, whereby keywords were generated
for each indicator (as presented in Table 2), and social media data were collected by crawling Twitter
using the Rapidminer application tool with an academic account. The search filters were based on the
generated keywords and location, which focused on the provincial capital cities in the Indonesian island
of Java designated as smart cities, consisting of Jakarta, Bandung, Semarang, and Surabaya. The location
filter was restricted to a 20 km radius from each city's longitude and latitude. The crawling process took
place between August 25th and October 25th, 2022, resulting in 12,185 pieces of raw data related to the
social, economic, and environmental dimensions. After filtering and selection, 5,981 relevant pieces of
data were obtained, pertaining to the smart city and its indicators (dimensions), as shown in the
distribution of each indicator in Figure 4(a) and the distribution in each city in Figure 4(b). The NLP
(Natural Language Processing) method was employed to attain high classification accuracy by cleaning
and processing the data.
This processing method is crucial for the computer's understanding of the data (Macrohon et al.,
2022). Several libraries were utilized in the preprocessing stage, including Google Colab, nltk, pandas,
spacy, and the Indonesian Sastrawi library. The Indonesian Sastrawi library was particularly useful for
converting word affixes in the Indonesian language into their basic form (Widiantoro et al., 2021). The
preprocessing phase encompassed several steps that included tag replacement, case folding, stopword
elimination, stemming, normalization, and tokenization. Tag replacement involved removing extraneous
information from the raw data, such as URLs, numbers, special characters, entry attachments, usernames,
question marks, exclamation points, and extra white spaces. Case folding converted all uppercase letters
to lowercase to ensure consistency among the characters in the dataset. Stop-word removal was
performed using the stopwords() library provided by NLTK and the Sastrawi tool to eliminate irrelevant
words. Sastrawi was also used for stemming, which involved transforming words with affixes into their
base form by stripping prefixes and suffixes. Normalization entailed converting non-standard words into a
standard form based on their spelling. This was essential since Twitter often employs slang words such as
"bgt," "dgn," "slalu," "gkmau," "aq," "yuuuk," "sippp," etc. A dictionary of 17,321 Indonesian texts was
utilized to normalize the words. Tokenization involved breaking down sentences into meaningful
expressions, punctuation marks, and words, and the word_tokenize() function from the NLTK library was
utilized for this process.

(a) (b)
Fig. 4. Data distribution: (a) by indicators, (b) by cities

3.2.2 Sentiment Weight

The sentiment weight calculation is performed utilizing the Python programming language in
conjunction with Google Colab tools. Specifically, the sentiment weight is computed by tallying both the
positive and negative word counts for each row of data associated with each indicator. Notably, the
weight calculation procedure accounts for the variability in word count across different rows of data
generated by various individuals, as this can influence the emotional word count. This implies that the
more words present within a given row of data, the greater the likelihood that a higher number of
emotional words will be utilized, and vice versa. Thus, the utilization of positive and negative sentiment
unit character counts enables the representation of the sentiment score linked with each row of data.
Importantly, when negative sentiment dominates positive sentiment in a given indicator, heightened
attention should be placed on that indicator. The following equation outlines the sentiment weight
calculation (Zhou et al., 2021).

The sentiment weight calculation for each dimension, namely social, economic, and environmental,
can be derived from equations (1) and (2). Here, S-m refers to the count of negative words in row data m,
while S+m represents the count of positive words in row data m. Cm denotes the total number of words in
row data m. The terms ∑S-m/Cm and ∑S+m/Cm are indicative of the negative and positive sentiment
representation, respectively. Furthermore, So, Ec, and En are sets of row data pertaining to each indicator
in the social, economic, and environmental dimensions. The equation ∑(S-m/Cm)/∑( S+m/Cm) represents
the rational sentiment score.

3.2.3 Data Analysis

 Papers should be formatted with single columns and should be single-spaced, 11 point Times New
Roman font. Also, do not change the margins as they have been set in the template: Top-1 inch, Right-1
inch, Bottom-1 inch, Left-1 inch. Do not add page numbers to your paper. We will insert these later.
Papers should be formatted with single columns and should be single-spaced, 11 point Times New
Roman font. Also, do not change the margins as they have been set in the template: Top-1 inch, Right-1
inch, Bottom-1 inch, Left-1 inch. Do not add page numbers to your paper. We will insert these later.

4. Result and Discussion

Please acknowledge collaborators or anyone who has helped with the paper at the end of the text.

4.1. Result
You might notice that the first paragraph after a header is not indented. Use the “Body text 1” style for the
first paragraph after a header. Subsequent paragraphs are indented (Use “Body Text” style). The
following is an example of the “Bullet” style, which you may want to use for lists.

4.2. Discussion
You might notice that the first paragraph after a header is not indented. Use the “Body text 1” style for the
first paragraph after a header. Subsequent paragraphs are indented (Use “Body Text” style). The
following is an example of the “Bullet” style, which you may want to use for lists.

5. Conclusion
Please acknowledge collaborators or anyone who has helped with the paper at the end of the text.

6. Acknowledgements
Please acknowledge collaborators or anyone who has helped with the paper at the end of the text.

7. Appendix
Appendix A: Short Profile of Expert as Validator
Validato
No Short Profile
r
He received Doctor of Philosophy degree from RMIT University in Melbourne,
Australia, his research interests encompass various areas of IT, including Green
Expert 1 IT, e-government, Smart City, E-learning, and IT Public Services. Notably, some
1 Associate of his research has focused on Smart City initiatives, such as (1) Building Green
Professor Smart City Capabilities in South Sumatra, Indonesia, (2) E-Government Agility
Concept for Small Towns, and (3) Utilizing Blockchain Technology for
Designing Digital Documents in Public Services.
He received Doctor of Philosophy degree from Curtin University of Technology
in Perth, Australia, his research interests lie in the areas of computer vision,
Expert 2 information systems, and human factors. Currently, he serves on the Board of
2
Professor Governors of the National Agency for Research and Innovation (BRIN) in the
Republic of Indonesia, and is also one of the initiators of the Movement Towards
100 Smart Cities in Indonesia.
3 Expert 3 He received Doctor of Philosophy degree from Delft University of Technology in
Associate Delft, Netherlands, his research interests encompass various areas of IT,
Professor including Computer Networks, Network Security, Digital Forensics
 Validato
No Short Profile
r
Investigation, and Open Government Data. Notably, some of his research has
focused on Smart City initiatives, such as (1) A stakeholders taxonomy for
opening government data decision-making, (2) Open data for evidence-based
decision-making: Data-driven government resulting in uncertainty and
polarization, and (3) A Framework for Analyzing How Governments Open Their
Data: Institution, Technology, and Process Aspects Influencing Decision-
Making.
He received Doctor of Philosophy degree from Delft University of Technology in
Delft, Netherlands, his research interests encompass various areas of IT,
including Open Data Government, Data mining, Information System, and
Expert 4 Technology adoption. Notably, some of his research has focused on Smart City
4 Associate initiatives, such as (1) Implementation of Open Data Interoperability
Professor Management for Citarum using Opendatasoft, (2) The role of trust to enhance the
recommendation system based on social network, and (3) Sentiment Analysis of
social media Twitter with Case of Large-Scale Social Restriction in Jakarta using
Support Vector Machine Algorithm.
He received Doctor of Philosophy degree from Delft University of Technology in
Delft, Netherlands, his research interests encompass various areas of IT,
including Open Data Government, Data mining, Information System, and
Expert 5 Technology adoption. Notably, some of his research has focused on Smart City
5 Associate initiatives, such as (1) Implementation of Open Data Interoperability
Professor Management for Citarum using Opendatasoft, (2) The role of trust to enhance the
recommendation system based on social network, and (3) Sentiment Analysis of
social media Twitter with Case of Large-Scale Social Restriction in Jakarta using
Support Vector Machine Algorithm.

8. References
Agbali, M., Trillo, C., Fernando, T., Oyedele, L., Ibrahim, I. A., & Olatunji, V. O. (2019). Towards a
refined conceptual framework model for a smart and sustainable city assessment. 5th IEEE
International Smart Cities Conference, ISC2 2019, 658–664.
https://doi.org/10.1109/ISC246665.2019.9071697
Ahvenniemi, H., Huovila, A., Pinto-Seppä, I., & Airaksinen, M. (2017). What are the differences between
sustainable and smart cities? Cities, 60, 234–245. https://doi.org/10.1016/j.cities.2016.09.009
Akande, A., Cabral, P., & Casteleyn, S. (2019). Assessing the gap between technology and the
environmental sustainability of European cities. Information Systems Frontiers, 21(3), 581–604.
https://doi.org/https://doi.org/10.1007/s10796-019-09903-3
Akande, A., Cabral, P., Gomes, P., & Casteleyn, S. (2019). The Lisbon ranking for smart sustainable
cities in Europe. Sustainable Cities and Society, 44(August 2018), 475–487.
https://doi.org/10.1016/j.scs.2018.10.009
Bayar, D. Y., Guven, H., Badem, H., & Soylu Sengor, E. (2020). National Smart Cities Strategy and
Action Plan: the Turkey’s Smart Cities Approach. International Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences, XLIV-4/W3-(October), 129–135.
https://doi.org/10.5194/isprs-archives-xliv-4-w3-2020-129-2020
Benites, A. J., & Simões, A. F. (2021). Assessing the urban sustainable development strategy: An
application of a smart city services sustainability taxonomy. Ecological Indicators, 127.
https://doi.org/10.1016/j.ecolind.2021.107734
Berrone, P., & Ricart, J. E. (2018). IESE Cities in Motion Index 2018. In Center for Globalization and
 Strategy and IESE Business School’s Department of Strategy (2018th ed.). IESE Business School’s.
https://doi.org/10.15581/018.ST-471
Caird, S., Hudson, L., & Kortuem, G. (2016). A tale of evaluation and reporting in UK smart cities. The
Open University: Milton Keynes.
Checkland, P. B., & Haynes, M. G. (1994). Varieties of systems thinking: The case of soft systems
methodology. Management Control Theory, 3(January), 151–159.
Cohen, B. (2022). Smart city index master indicators survey. Smart cities council.
http://smartcitiescouncil.com/resources/smart-city-index-master-indicators-survey
Crooks, A., Pfoser, D., Jenkins, A., Croitoru, A., Stefanidis, A., Smith, D., Karagiorgou, S., Efentakis, A.,
& Lamprianidis, G. (2015). Crowdsourcing urban form and function. International Journal of
Geographical Information Science, 29(5), 720–741. https://doi.org/10.1080/13658816.2014.977905
Dawodu, A., Cheshmehzangi, A., Sharifi, A., & Oladejo, J. (2022). Neighborhood sustainability
assessment tools : Research trends and forecast for the built environment. Sustainable Futures,
4(January), 100064. https://doi.org/10.1016/j.sftr.2022.100064
Debnath, A. K., Chin, H. C., Haque, M. M., & Yuen, B. (2014). A methodological framework for
benchmarking smart transport cities. Cities, 37, 47–56.
Ependi, U., Rochim, A. F., & Wibowo, A. (2022). Smart City Assessment for Sustainable City
Development on Smart Governance: A Systematic Literature Review. 2022 International
Conference on Decision Aid Sciences and Applications, DASA 2022, 1088–1097.
https://doi.org/10.1109/DASA54658.2022.9765192
ETSI. (2017). ETSI TS 103 463 key performance indicators for sustainable digital multiservice cities
(Vol. 1).
Fernandez-Anez, V., Fernández-Güell, J. M., & Giffinger, R. (2018). Smart City implementation and
discourses: An integrated conceptual model. The case of Vienna. Cities, 78, 4–16.
Fernandez-anez, V., Velazquez, G., & Perez-prada, F. (2018). Smart City Projects Assessment Matrix:
Connecting Challenges and Actions in the Mediterranean Region. Journal of Urban Technology,
27(4), 1–25. https://doi.org/10.1080/10630732.2018.1498706
Garau, C., & Pavan, V. M. (2018). Evaluating urban quality: Indicators and assessment tools for smart
sustainable cities. Sustainability, 10(3), 575.
Giffinger, R., Haindlmaier, G., & Kramar, H. (2010). The role of rankings in growing city competition.
Urban Research & Practice, 3(3), 299–312.
Gutman, S., & Vorontsova, P. (2020). Issues of Development of Smart Transport Assessment Indicators.
ACM International Conference Proceeding Series. https://doi.org/10.1145/3446434.3446438
Hass, J. L., Brunvoll, F., & Hoie, H. (2002). Overview of Sustainable Development Indicators used by
National and International Agencies. In OECD Statistics Working Papers.
https://doi.org/10.1787/838562874641
Havinga, I., Bogaart, P. W., Hein, L., & Tuia, D. (2020). Defining and spatially modelling cultural
ecosystem services using crowdsourced data. Ecosystem Services, 43(March), 101091.
https://doi.org/10.1016/j.ecoser.2020.101091
Huovila, A., Bosch, P., & Airaksinen, M. (2019). Comparative analysis of standardized indicators for
Smart sustainable cities: What indicators and standards to use and when? Cities, 89(January), 141–
153. https://doi.org/10.1016/j.cities.2019.01.029
ITU-T. (2022). Key performance indicators for smart sustainable cities to assess the achievement of
sustainable development goals. In International Telecomunication Union. https://www.itu.int/rec/T-
REC-Y.4903
Jukka M. Krisp, Mathias Jahnke, H. L. and F. F. (2014). Urban Emotions—Geo-Semantic Emotion
Extraction from Technical Sensors, Human Sensors and Crowdsourced Data. Progress in Location-
Based Services, 33–44. https://doi.org/10.1007/978-3-319-11879-6
Khan, A., Aslam, S., Aurangzeb, K., Alhussein, M., & Javaid, N. (2022). Multiscale modeling in smart
 cities: A survey on applications, current trends, and challenges. Sustainable Cities and Society,
78(November 2021), 103517. https://doi.org/10.1016/j.scs.2021.103517
Kimiya, R. M., & Torabi, S. A. (2021). Ranking cities based on their smartness level using MADM
methods. Sustainable Cities and Society, 72(May), 103030.
https://doi.org/10.1016/j.scs.2021.103030
Koca, G., Egilmez, O., & Akcakaya, O. (2021). Evaluation of the smart city: Applying the dematel
technique. Telematics and Informatics, 62(June 2020), 101625.
https://doi.org/10.1016/j.tele.2021.101625
Kourtzanidis, K., Angelakoglou, K., Apostolopoulos, V., Giourka, P., & Nikolopoulos, N. (2021).
Assessing impact, performance and sustainability potential of smart city projects: Towards a case
agnostic evaluation framework. Sustainability, 13(13). https://doi.org/10.3390/su13137395
Kristiningrum, E., & Kusumo, H. (2021). Indicators of Smart City Using SNI ISO Indicators of Smart
City Using SNI ISO 37122 : 2019. IOP Conference Series: Materials Science and Engineering.
https://doi.org/10.1088/1757-899X/1096/1/012013
Lom, M. (2021). Smart city model based on systems theory. International Journal of Information
Management, 56(February 2019), 102092. https://doi.org/10.1016/j.ijinfomgt.2020.102092
Long, Y., & Liu, L. (2016). Transformations of urban studies and planning in the big/open data era: a
review. International Journal of Image and Data Fusion, 7(4), 295–308.
https://doi.org/10.1080/19479832.2016.1215355
Macrohon, J. J. E., Villavicencio, C. N., Inbaraj, X. A., & Jeng, J. (2022). A Semi-Supervised Approach
to Sentiment Analysis of Tweets during the 2022 Philippine Presidential Election. Information,
13(484), 1–14. https://doi.org/10.3390/info13100484
Mohan, A., Dubey, G., Ahmed, F., & Sidhu, A. (2020). Smart Cities Index: A Tool for Evaluating Cities.
In Indian School of Business: Hyderabad. Indian School of Business-Hyderabad.
Neirotti, P., De Marco, A., Cagliano, A. C., Mangano, G., & Scorrano, F. (2014). Current trends in smart
city initiatives: Some stylised facts. Cities, 38, 25–36. https://doi.org/10.1016/j.cities.2013.12.010
Niu, H., & Silva, E. A. (2020). Crowdsourced Data Mining for Urban Activity: Review of Data Sources,
Applications, and Methods. Journal of Urban Planning and Development, 146(2), 04020007.
https://doi.org/10.1061/(asce)up.1943-5444.0000566
Noori, N., de Jong, M., Janssen, M., Schraven, D., & Hoppe, T. (2020). Input-Output Modeling for Smart
City Development. Journal of Urban Technology, 28(1–2), 71–92.
https://doi.org/10.1080/10630732.2020.1794728
Noori, N., Hoppe, T., & de Jong, M. (2020). Classifying pathways for smart city development:
Comparing design, governance and implementation in Amsterdam, Barcelona, Dubai, and Abu
Dhabi. Sustainability (Switzerland), 12(10). https://doi.org/10.3390/SU12104030
Patrão, C., Moura, P., & Almeida, A. T. de. (2020). Review of Smart City Assessment Tools. Smart
Cities, 3(4), 1117–1132. https://doi.org/10.3390/smartcities3040055
Pira, M. (2021a). A novel taxonomy of smart sustainable city indicators. Humanities and Social Sciences
Communications, 8(1), 1–10. https://doi.org/10.1057/s41599-021-00879-7
Pira, M. (2021b). A novel taxonomy of smart sustainable city indicators. Humanities and Social Sciences
Communications, 8(1). https://doi.org/10.1057/s41599-021-00879-7
Purnomo, F., Meyliana, & Prabowo, H. (2016). Smart city indicators: A systematic literature review.
Journal of Telecommunication, Electronic and Computer Engineering, 8(3), 161–164.
https://jtec.utem.edu.my/jtec/article/view/1023
Rochet, C., & Villechenon, F. P. de. (2014). System Architecture As A Means To Build An Extended
Administration : The Case of Smart Cities (Issue June).
Sharif, R. Al, & Pokharel, S. (2021). Smart City Dimensions and Associated Risks: Review of literature.
Sustainable Cities and Society. https://doi.org/10.1016/j.scs.2021.103542
Sharifi, A. (2019). A critical review of selected smart city assessment tools and indicator sets. Journal of
 Cleaner Production, 233(1 October 2019), 1269–1283. https://doi.org/10.1016/j.jclepro.2019.06.172
Sharifi, A. (2020a). A global dataset on tools, frameworks, and indicator sets for smart city assessment.
Data in Brief, 29. https://doi.org/10.1016/j.dib.2020.105364
Sharifi, A. (2020b). A typology of smart city assessment tools and indicator sets. Sustainable Cities and
Society, 53(May 2019), 1–15. https://doi.org/10.1016/j.scs.2019.101936
Shen, L., Huang, Z., Wai, S., Liao, S., & Lou, Y. (2018). A holistic evaluation of smart city performance
in the context of China. Journal of Cleaner Production, 200(November), 667–679.
https://doi.org/10.1016/j.jclepro.2018.07.281
Takenaka, H., & Ichikawa, H. (2018). Global Power City Index 2018.
https://mori-m-foundation.or.jp/pdf/GPCI2018_summary.pdf
Widiantoro, A. D., Wibowo, A., & Harnadi, B. (2021). User Sentiment Analysis in the Fintech OVO
Review Based on the Lexicon Method. 2021 Sixth International Conference on Informatics and
Computing (ICIC), 1–4. https://doi.org/10.1109/ICIC54025.2021.9632909
Wu, Z., Li, X., Zhou, X., & Yang, T. (2021). City Intelligence Quotient Evaluation System Using
Crowdsourced Social Media Data : A Case Study of the Yangtze River Delta Region , China.
Internationa Journal of Geo-Information, 10(702). https://doi.org/10.3390/ijgi10100702
Yigitcanlar, T., Degirmenci, K., Butler, L., & Desouza, K. C. (2022). What are the key factors affecting
smart city transformation readiness ? Evidence from Australian cities. Cities, 120(August), 103434.
https://doi.org/10.1016/j.cities.2021.103434
Zhou, Y., Yi, P., Li, W., & Gong, C. (2021). Assessment of city sustainability from the perspective of
multi-source data-driven. Sustainable Cities and Society, 70(January), 102918.
https://doi.org/10.1016/j.scs.2021.102918