Node.js Full Stack Web App Design
Node.js Full Stack Web App Design
The aim of this thesis was to design and develop a full stack web application for an
internet service provider, which can be used to manage and control its customers’
internet connections.
The application was designed to be used on different devices, for example on mobile
devices from the worksite, or on computer from the office. Bootstrap CSS framework
which is designed for creating mobile-friendly websites was used for this purpose.
The application’s server was built to a Node.js environment, which allowed for
building a secure and versatile server thanks to its large ecosystem. In addition to the
application’s server, an API was built on to the same Node.js process, which the
application uses to access the database. A separate program reads changes from
the database and executes them on physical devices.
As a result, the desired features were implemented to the application, and it was
launched on the company’s intranet so that employees can use it.
This thesis examined the structure and operation of a full-stack web application, the
requirements of the project and the selected technologies. The thesis discussed the
architecture of the application, as well as the different stages of its development.
Finally, possible future improvements and developments were discussed, such as
new features or turning the user interface to a Progressive Web App.
2.1 Frontend 2
2.2 Backend 2
2.3 Database 3
2.4 Application Programming Interface 3
3 Project requirements 4
3.1 Node.js 5
3.2 MariaDB 6
3.2.1 Data validation 6
3.2.2 REST API 7
3.3 Bootstrap 8
3.4 Information security 10
3.4.1 Client-server communication 10
3.4.2 Client authentication 13
3.4.3 API security 15
4 Application structure 18
5.1 Tools 26
5.2 Roadmap stages 27
6 Conclusion 30
References 35
Appendices
Appendix 1: Roadmap
1
1 Introduction
In the past, websites were content-based sites and they were meant to
distribute information. However, as the internet has evolved, they have become
a delivery platform for applications that used to be desktop applications or so
called “web applications”. [1.]
Because web applications are used through a web browser, they can be used
on different devices, such as computers or smart phones, if the web browser is
compatible. Other benefits of a web application are the following:
• Every user has the same version of the application, meaning there are
no compatibility issues.
• There is no need to install web applications to the device.
• Reduces the company’s costs because the application requires less
maintenance and upkeep.
• Reduces the user’s costs by lowering the devices minimum system
requirements.
[2.]
The goal of this thesis was to design and build a full-stack web application to
Nivos Verkot Oy, so its employees can use the application to administer
customers’ internet connections: change the connection speed, turn the
connection on or off, and change other settings that help maintaining
connections.
2
Web application is a computer program that users can use through their web
browsers. Full-stack applications have at least three main parts: User interface,
called frontend, server, called backend and a database. Users use the frontend
to send commands and requests to backend. Backend takes care of the cyber
security, receives the commands and requests from user and uses the
database. Figure 1 illustrates the structure of a full-stack web application. [3.]
2.1 Frontend
Frontend means the user interface, in other words the part that user sees. A
web application’s frontend is usually programmed using web browser-compliant
programming languages, such as HTML, JavaScript, and CSS.
HTML is a markup language that web browsers can open and show to users.
JavaScript code makes the frontend dynamic, for example, it will make buttons
work. CSS is used to define the frontend’s appearance, such as colors, fonts or
formatting. [1;4.]
2.2 Backend
When a user visits a website, the web browser sends a request to the web
application’s server, also known as backend. Backend performs commands and
3
requests that user sends, such as fetching data from database, processes the
data and sends it back to user. User does not use backend directly but
communicates with it using frontend. [1.] The backend is also responsible for
the cyber security, performance and scalability of the web application.
2.3 Database
more complex code and offer higher-level, simpler syntax instead. [8.] For
example, software applications can fetch data from a database using an API.
3 Project requirements
The goal of this thesis project was to create a full-stack web application that
employees can use to manage customers’ internet connections anywhere, from
a work site via smart phone or from the office using a computer. The connection
information is stored on a database, from which an external software reads
changes and performs necessary operations, such as modifying the download
speed setting of a physical end device.
The application will have multiple features, such as managing the settings of
existing connections, such as download and upload speeds, activation, and
deactivation, creating new connections, making mass changes to multiple
connections and scheduling changes.
Since users must be able to use the web application on smart phones or
computers, the frontend must be responsive. This means that the user sees the
same HTML file at the same URL address regardless of the user’s device, but
the content may look different depending on the screen size. [9.]
3.1 Node.js
Node.js application runs as a single process, so it does not create a new thread
for each request. Node.js can perform a I/O operation, such as reading data from
database, in two ways: blocking or non-blocking. Blocking way stops the Node.js
process and waits for the data to be received. In non-blocking way, the process
does not stop and wait, but the process returns to the operation when the data is
received. This allows a Node.js process to handle thousands of requests and
concurrent connections on a single server. [11.]
Node.js has its own modules, that can, for example, create an HTTP server, read
local files or even create child processes that can access the operating system in
which the Node.js process runs. [14;15;16]
When Node.js is installed, a Node Package Manager (npm) is also installed. It is
a package manager, which developers can use to download modules created by
6
other developers or upload their own modules. [17.] Npm is the largest software
registry in world with more than two million packages. [18.]
However, the modules that Node.js provided, does not provide all necessary
features, additional modules must be added through npm. The selection of these
modules must be based on their popularity, maintenance status and their
publisher. This ensures that the selected modules are safe to use, and their life
cycle is as long as possible, and they are actively maintained.
3.2 MariaDB
The Connection database has been built to a MariaDB -server before this
project was started. The MariaDB -server runs in a company’s Local Area
Network, so the API must work within this environment. The API must integrate
with the database as seamlessly as possible, allowing for any possible changes
in database structure to be easily implemented into the API.
However, this option cannot be used in every case, as the value of the
information may depend on the characteristics of the physical device. For an
example, Nivos Verkot Oy provides internet connections to apartment building
residents, but depending on the cabling of the building, the speed parameters
may vary: If the building does not have CAT-cabling, fastest possible internet
7
speed is 100/100M, but if the buildings with CAT-cabling, fastest possible speed
might be as high as 1000/500M. [20;21.] This is the reason why enumerations
cannot be used as a data format for all data in the database, as the value of the
data may depend on the physical device’s properties, because then the web
application might offer same options for every connection. To overcome this
problem, database stores information about device types and their supported
parameters, such as speeds. The web application must check the suitable
parameters based on the connection technology from the database and offer
them to the user.
Basically, REST API’s functionality is same as surfing in the web: User sends a
request to the REST API’s servers using HTTP protocol, and server returns the
requested data to user in a predefined format. The request must include the
following information:
[22.]
Because REST-API works on the same HTTP-protocol as a web application,
the API can be built to a Node.js -process. MariaDB provides its own npm -
package for Node.js, making it easy and secure to use the database. [23.]
3.3 Bootstrap
Figure 3. Bootstrap’s example how “card” feature is used to build a card. [25.]
9
Bootstrap provides a grid system, that can be used to divide the website into a
grid. In the grid system, webpage is divided to rows and columns, with the
content is within the columns. Each row is divided to 12 parts, where one
column’s width can be at least one part, but at most 12 parts. Each row can
hold at most 12 columns if every column’s width is one part. It is possible to
assign each column its own width, but if it is not assigned, then Bootstrap will
automatically set the width of column. [26.] In figure 4, an example row has
been built using Bootstrap’s grid system. Notice that there is a div element with
class “row” that and inside that element there are three div elements that have
classes “col“. Columns width can be assigned by adding a number from 1 to 12
after the “col” class attribute. The width of the first “col” element has been set to
two parts, the width of the second element is determined by Bootstrap, and the
width of the third “col” element is six, which has also been assigned an sm-
breakpoint.
The example in figure 5 has the same source code as figure 4, but the width of
the screen is 572 pixels instead of 578 pixels. The column that had the
breakpoint assigned, is now 12 parts wide. The column that had a width of two
parts, will remain at the same size. The middle column that had no assigned
width, fills the remaining space of the row. Note that the overall row height
remains the same, so the Bootstrap’s grid system adjusts the column heights as
needed.
Figure 5. Same example code with Figure 4, but the screen size is reduced.
In today’s world, information security plays a crucial role by protecting files and
software from unauthorized and malicious use. [28.]
HTTP protocol is used for communication between a website and a server, and
most of the data transmission on the internet uses this. Downside of this
protocol is that data transmissions can read by anyone who is monitoring the
session, because this protocol is sending plain text data. To prevent this,
HTTPS protocol is used instead of HTTP. HTTPS is almost the same as HTTP,
but it uses TLS/SSL method to encrypt normal, plain text HTTP messages.
TSL/SSL method uses public key cryptography: Two keys are generated, public
11
and private. Public key is shared with the client. When client connects to the
server, both server and client uses both keys to create a new key, session key,
that will be used to encrypt messages. [29.] This protocol must be used both in
the API and in the web application.
When a user wants to browse websites, user must know the address of the
server and a port number. Port is an application-specific software construct,
that works as a communication end point. A server must be configured with a
specific port that it will use to listen or incoming connections.
In 1991, Tim Berners-Lee defined that “if the port number is not specified, 80 is
always assumed for HTTP.” [30.] This is the reason why users don’t have to
know the port number, because browsers can send the data to port 80 by
default. Because the communication between the client and the server needs to
use HTTPS protocol, port 80 cannot be used. HTTPS has a default port
number, 443, which has been defined in year 1994. [31.] To address this issue,
two servers must be created to the Node.js process. One server listens to the
port 443 and second listens to port 80. The purpose of the second port is only to
redirect users to the first server, that will work as the main server for this project.
This way client-server communication uses HTTPS -protocol and the data
transmission is secured.
Figure 6 illustrates the creation of these two servers. On rows 102-104, server
that listens to port 443 is created with parameters of options and app. Options -
parameter includes SSL keys, and app parameter is the express-framework that
the server will use for every request. On rows 106-111, server that listen to port
80 is created. Its only purpose is to redirect requests to the other server. Note
that at row 107, HTTP status code 301 is given, so the client knows to
communicate to the port 443 in the future.
12
Users must be logged in before they can use the web application. To create a
secure way to authenticate users, Passport is used, which is an authentication
middleware that is directly compatible with Express middleware. Passport is a
popular middleware, with a weekly download count of 2 068 408 as of March
22, 2023. [33.]
14
HTTP protocol is stateless, meaning that every request that client sends, can be
treated as a separate message – with no knowledge of last requests or
messages. This presents a problem for web applications because it requires
users to log in between every request. [38.]
To solve this problem, sessions must be created between the client and the
server. When user logs into the application, the server creates a session and an
HTTP cookie, which contains the session data. Then this cookie is sent to the
user, and the user will send it back to the server with every request. The server
can use this cookie to identify the session and the user. This way a stateful
protocol can be created on top of HTTP protocol. [38.] Express middleware has
an additional npm package, express-session, that will also work with Passport.
[38.]
In addition, users are assigned their own roles, which can be used to restrict the
access to resources and features. This can be used to give, for example,
customer service only read access or limited editing rights, such as increasing
speed or disconnecting a connection.
15
Just like the web application, API must be protected from malicious use. The
interface is stateless, as information is only requested from it when needed, so
there is no need for the sessions that are used in web application.
To protect the API, a JSON Web Token (JWT) method is used. JWT is an
independent way to securely transfer information in JSON format, such as
identification data. This information can be verified as it is digitally signed. The
JSON Web Token can also have an expiration time, for example, one hour,
during which time Token will work. After this API stops approving this token.
Header
Header is built from two parts, type of the token, which is JWT in this case, and
a signature algorithm that will be used.
Payload
Payload contains the information that will be transferred, such as user
information and expiration time.
Signature
Signature is used to make sure that the message has not changed. To create a
signature, a string must be created that holds the header, payload, and a secret
key, which will be encrypted with an algorithm that is mentioned in header.
Figure 9 illustrates what information JWT can hold and what it looks like
encrypted:
16
When user logs into the web application and requests data from server, server
authenticates user, uses the JWT that is created for the user, and sends it to
API in a HTTP Authorization Header. Then API checks the JWT and makes
sure that the user has access to the resources they have asked.
Using this method API is not limited only for the web application, but JSON Web
Tokens can be created for other software applications that intended to use the
API.
One of the most common hacking techniques is SQL injection, where SQL
commands are added to the normal user input. This way a hacker can have
sensitive data from database, change it or even remove data. SQL injection is
possible, if API adds user given parameters straight to the SQL command it
intends to execute. [40.]
By default, hackers cannot use API if they do not have a valid JSON Web
Token, as mentioned previously. However, it is still a good practice to add
protection against SQL injection. The MariaDB npm package offers a ready-
17
made solution for this: this package only allows one SQL command to be
executed at a time and parameters can be added as placeholders to the
command. This way, the MariaDB package can verify that the parameters are in
the correct format and are not harmful.
Figure 10. Example from MariaDB on how to add parameters to SQL command.
This is not a danger in this project, as the web application is limited only to
specifically designated employees. However, Helmet npm package provides a
security layer that helps to identify and prevent certain types of attacks, such as
Cross-site scripting and data injections. [32;43.]
18
4 Application structure
To make the source code of the application as easy to maintain as possible, its
structure must be easy to understand. Therefore, the structure of the software
must be carefully planned.
The API and the backend of the web application are built to the same Node.js
process, since the web application is the only software that will use the API and
the API is mainly built for the web application. This simplifies the development
work and software management since this removes the need for doing changes
to multiple different software.
Even though API is in the same process as the web application, web
application’s backend uses API to fetch data. This way information security is
simplified, as it is the only way to access the database, and also makes it easier
to separate API to an independent process, if needed.
The chosen architecture for the web application’s backend follows a practice
similar to the MVC architecture, which consists of the Model, View and
Controller components. [44.] Figure 11 illustrates how the MVC architecture
works.
19
Model
The role of the model is to handle the data processing. Because data is fetched
from API, the model does not use the database but instead uses the Node.js’
Axios module to make HTTP requests to API.
The Axios module is used to create instances for all the data in the database,
such as connections, apartment buildings or fiber optic terminal devices. Each
instance is defined with functions that will send requests to the API. Figure 12
illustrates the creation of the connection client that is created using Axios.
These instances can have their own functions that will send the requests to the
API.
20
View
The role of the view is to handle presentation layer, which means modifying the
page that the user sees as needed.
There are different ways to implement the View’s task, such as client-side
rendering, where user’s device receives the requested data and dynamically
modifies the HTML file to display the data. This method reduces server load and
allows for a more dynamic user interface but can slow down the loading
especially in slow networks and requires additional programming to create
functions that receive and organize data into the HTML file. [46.]
Another method is server-side rendering, where the server adds user requested
data to HTML file before server sends it to the user. This way, only the desired
information can be displayed to the user, without sending any unnecessary
information, for example, creation time of the connection and other metadata,
which is returned by the API. This approach is faster for the initial load and
21
reduces the required performance of the user’s device. However, this method
requires more resources from the server. In addition, creating dynamic pages is
more difficult because the server returns a new HTML file instead of just the
desired information. [46.]
Controller
The purpose of the Controller is to work between the view and the model. It
receives the user requests, asks data from the model, and provides the data to
view. [44] Because model returns all the requested data from API, the controller
can handle this data and select what is sent to the view.
Because the web application has multiple different features, such as modifying
connection, creating new connections, or performing mass changes to
connections, every feature needs a separate .EJS file, which the server can use
to render all necessary data.
Figure 15. Link for a JavaScript file that attaches sidebar to the HTML file in
client side.
link to the HTML file, user’s device can load necessary files and script using the
Content Delivery Network [49.], like shown in Figure 16. This also makes it
possible to always have the latest version of Bootstrap available to the user.
A similar structure to the backend is created for API, with the difference that
there is no Views section. The API’s Controller receives requests and uses
Model that retrieves requested data from database and then returns it to the
Controller. Then Controller sends the retrieved data as a HTTP response to the
sender of the HTTP request.
MariaDB’s npm package is used to read the database, and a custom Model is
built for the API. Using this Model, API can access the database, such as
reading and modifying data, or even get information about the structure of the
database.
To ensure that the API works as seamlessly with the database as possible, the
API does not hard code the names of the database columns in SQL commands.
Instead, the user must input the column names and values to the request.
Before creating a SQL command, API fetches the real column names from the
database and compares these names to the ones that user has input. If column
25
names match, then API adds that parameter to the SQL command. This
ensures that no invalid SQL command is built. As a result, any potential
changes to the database, such as adding or removing a column, will not
necessarily require any modifications to the API software.
One downside of the Node.js process is that if an error occurs, the whole
process shuts down. Therefore, it is important to design the error handling
carefully. Errors could occur for example, faulty code, API returns something
unexpected, the process cannot connect to API, or some file is missing.
One way to handle the errors is to use JavaScript’s throw mechanism, which is
used in a “try..catch” structure. [51.] The use of a throw mechanism generates
an exception, that must handle using try..catch structure, or Node.js process
shuts down immediately. [52.] In Figure 17 is illustrated, how try..catch structure
is built. When the example program is trying to execute the function called in
line 2, an exception occurs and program jumps over to line 4, after which the
program executes lines 5-9.
In both API and web application’s backend, the try..catch structure is built to the
Controller to handle errors that occur somewhere later in the process that user
has started. An error handler -functions are created for both parts, the purpose
of these functions is to additional information, like the HTTP status, to the error
26
The try..catch structure can handle errors created by the software, but also
custom errors defined by the developer. For an example, the API can check the
request from user and if this request does not contain all the necessary
information, or user does not have the rights, then a custom error can be thrown
with custom messages. Then this error can be handled in a same way as the
default errors are handled. [53.]
5.1 Tools
Microsoft Word was used to record in detail the different features what the
project has, and in what stages the features are. Additionally, a journal was
maintained in Word, documenting what was achieved during the day, identifying
any issues that arose, and highlighting what needed to be clarified to solve
these issues.
Regular meetings were not held during the project, but instead meetings were
arranged if there was need to discuss about the features or issues.
For programming, Visual Studio Code was used, which is a free lightweight
source code editor provided by Microsoft. [54.] It has an integrated terminal
[55.], which is a convenient way to initiate, debug, monitor and terminate
27
Another tool available for development is Nodemon, which can be installed from
npm. Nodemon makes development of Node.js processes easier. Using
Nodemon, Node.js process restarts automatically, when file changes in the
directory. [56.] This makes it easier and faster to test Node.js processes.
For saving the codebase to cloud and controlling versions, GitHub was used,
which is a code hosting platform for version control and collaboration. [57.]
GitHub can be used, among other things, to review changes made to the
project.
For testing the API, an application called Postman is used. This application
allows customized HTTP requests to be send to the API, and its functionality to
be checked. This makes it easier and faster to test and develop API. [58.]
The stages of the project have been divided into main headings in roadmap to
keep the overall picture clear. In addition, the biggest stages in roadmap can be
opened in Miro and write more detailed information about what needs to be
done.
In the initial stage of the work, the necessary directories are created for the
application, but the files within the directories are created as the work
progresses. In addition to the directories, a package.json file is created, that
contains metadata about the project, such as project name, version, starting
commands, license and installed modules and packages, which are also
referred to as dependencies.
After these initial stages, the application can be constructed according to the
order planned in the roadmap.
At the beginning of the project, express server is built, which works as a base
for whole application. In this stage, the initial settings and configurations are set
to the express, for example, which directory can be shared to the client and in
what format client will send data. At this stage, Helmet package is also
initialized.
Next step involves creating a login system and establishing a connection to the
database. By creating the login system at this stage, user authorization can be
added to all necessary parts of the project during the development, eliminating
the need to add them later.
The biggest stage is to build the web application and API. This approach allows
for a better understanding of the requirements that web application needs from
the API. For example, to search connections by address, the SQL command
needs to have a “LIKE” clause with a wildcard for searching all connection in the
same building, instead of searching only for the specific apartment in the
building. However, the use of this feature should be limited to address
29
When web application and API is ready, the JWT authentication is created. This
is done only after the API is completed, to speed up the process of building and
testing it, as there is no need to login and create a JWT every time API is
tested.
During the development, the application has been running locally on the
computer, that is used for development. The last stage of the project is to move
the application to a Linux server that is operating within the company’s internal
network, so it can be used by the employees. During the development, the
application has been started with Node.js commands, but in production, a PM2
daemon process manager is utilized. PM2 can launch the application as a
background process, restart it in case the Linux server restart or the application
crashes, and write log information about the process. [61.]
30
6 Conclusion
The goal of this thesis was to design and create a full-stack web application to a
company, so its employees can monitor and manage customers’ internet
connections more easily, using a smart phone or a computer. The application
has multiple features, such as creating a new connection, modifying settings of
an existing connection, or executing mass changes to multiple connections at
the same time. The application was supposed to be as secure as possible, easy
to maintain and require minimal maintenance. Another requirement was that it
can be easily updated in the future.
Web application’s server and API was built to a Node.js process. Web
application uses the API to read and modify data from the database. The
database had been built on a MariaDB server, so it was necessary that the API
can use this database. The MariaDB provided an npm package that was used
by the API.
Both the web application and the API were built within the same Node.js
process. This simplified the development and software management, as
changes did not have to be made separately to multiple software programs.
However, the web application uses the API instead of reading data directly from
the database, even though it would have been possible to do so. This allows
easy separation of the API into its own Node.js process in the future, if
necessary.
framework was used for this purpose, which is designed with mobile-first
approach. Bootstrap provided its own grid system, which made it easy to create
a responsive frontend.
6.2 Results
The result of the project was a functional full-stack web application that met all
the project requirements. The project was developed in the order outlined in the
roadmap, but additional features emerged during development, such as the
ability to add various additional parameters and generate report information.
Despite this, the project was successfully launched on the company’s intranet,
and employees were able to use the application. Figure 18 shows an example
view from the page that users can use to execute changes to a connection.
Most of the parameters can be altered by the users, and they are predefined
and added to a dropdown list, ensuring that the selected value is exact. Users
can also specify time and date when new settings will take effect. On the left-
hand side is the navigation bar that users can use to navigate between different
features.
32
At the time of writing of this thesis, the application was not in full use, since the
external software that executes changes to the physical devices, was not
finished.
6.3 Improvements
After the deployment of the application, a few areas for improvement were
identified. When Node.js process started, the API established a connection to
the database and kept it open. However, MariaDB server terminates idling
connections. This happens after eight hours [62.], so this problem was not
discovered during the development. The issue arose the following day when the
application suddenly stopped working.
There were two choices to fix this: Establish a new connection every time the
API is used or build a mechanism that first checks the connection status and
reopens it if it is closed. However, the connection should remain open
33
throughout the entire user session, as depending on which feature the user is
using in the application, the application reads multiple different pieces of
information from the database. If the connection is disconnected and reopened
after each read operation, it would significantly increase loading times and lower
the user experience. For this reason, the second solution was chosen, where
the connection status is checked and reopened if necessary.
During the development phase, employees came up with many ideas for
additional features that could be added to the application. For example, features
such as checking the status of the connections, or fetching log data from
external log. These would help troubleshooting situations. For these, Node.js
provides an easy solution: Node.js provides child_process module, that can be
used to create separate subprocesses, which can start and execute different
programs and applications outside the node.js process. [16.]
The frontend could be turned into a Progressive Web App (PWA), which is a
web-based application built with technologies such as HTML, CSS and
34
References
27 Bootstrap; Breakpoints
https://getbootstrap.com/docs/5.3/layout/breakpoints/ Accessed 24 March
2023
32 Npmjs; Helmet
https://www.npmjs.com/package/helmet?activeTab=readme Accessed 24
March 2023
38 Passport; Sessions
https://www.passportjs.org/concepts/authentication/sessions/ Accessed 24
March 2023
Roadmap