Research Methodology: Unit 2 Exam

Notes (Detailed Version)

Chapter 1: Research Design Fundamentals
1. What is a Research Design?

A research design is the comprehensive blueprint for a research study. It's not just a work plan
but the strategic framework that connects the research questions to the eventual
conclusions. A good design ensures that the evidence collected is directly relevant to the
problem and that the methods used are sound.
Key Components in Detail:
●​ Research Problem: This is the foundation. A strong research problem identifies a
specific gap in existing knowledge, a practical challenge, or a contradiction that needs
resolving. For example, "Existing sorting algorithms perform poorly on partially sorted
data; this research will develop a hybrid algorithm to improve efficiency in this specific
scenario."
●​ Research Objectives: These break down the research problem into concrete,
achievable steps. Using the SMART criteria, an objective might be: "To reduce the
average sort time for partially sorted arrays of 10,000 integers by 15% within the next 3
months."
●​ Hypotheses: This is a formal, testable prediction.
○​ Null Hypothesis (H₀): States there is no effect or relationship. E.g., "The new UI
design will have no effect on user task completion time."
○​ Alternative Hypothesis (H₁): States there is an effect or relationship. E.g., "The new
UI design will significantly reduce user task completion time."​
Research aims to gather evidence to reject the null hypothesis in favor of the
alternative.
●​ Methods: This section details the how.
○​ Data Collection: Will you run experiments (e.g., benchmark algorithm
performance), conduct surveys (e.g., user satisfaction), or analyze existing data
(e.g., server logs)?
○​ Data Analysis: Which statistical tests will you use? A t-test to compare two
versions? Regression analysis to predict outcomes? The choice depends on your
data and objectives.
●​ Timeline & Resources: A realistic timeline, often visualized with a Gantt chart, prevents
project delays. Resources include budget for software, access to specific hardware (like
GPUs for model training), and necessary personnel.
2. Types of Research Designs
 ●​ Experimental: The "gold standard" for cause-and-effect. A key feature is
randomization, where participants are randomly assigned to either a control group
(receives no treatment) or a treatment group (receives the treatment). This minimizes
bias. Example: An A/B test where users are randomly shown either an old website layout
(control) or a new one (treatment) to see which one leads to more sales.
●​ Quasi-Experimental: Used when true randomization is impossible or unethical. For
instance, you can't randomly assign students to different schools. You might compare
two existing classes, but you must be aware of potential confounding variables (e.g.,
one class might have a more experienced teacher).
●​ Descriptive: Aims to provide a detailed snapshot of a situation. It answers "what"
questions, not "why." Example: A study describing the frequency and type of bugs
reported in a large open-source project over one year.
●​ Correlational: This design explores if there is a statistical relationship between two
variables. Crucially, correlation does not imply causation. Just because two
variables move together doesn't mean one causes the other. Example: Finding a
correlation between hours spent programming and the number of bugs produced.
●​ Exploratory: Conducted when little is known about a topic. It's flexible and often uses
qualitative methods like interviews or focus groups to generate initial ideas or
hypotheses for future, more structured research.
3. Variables in Research

●​ Independent Variable (IV): The "cause" or the input. It's what the researcher
manipulates or changes. In a study on website load time, the IV could be the level of
image compression.
●​ Dependent Variable (DV): The "effect" or the outcome. It's what is measured in
response to changes in the IV. In the same study, the DV would be the page load time in
seconds.
●​ Confounding Variable: A third, unmeasured variable that can influence both the IV and
DV, creating a spurious association. Example: Testing if a new algorithm (IV) is faster
(DV). A confounding variable could be the hardware it's running on. If the new algorithm
is tested on a faster computer, the results will be biased. Controlling for confounders
through randomization or statistical control is essential for valid results.
Control Groups: By comparing the treatment group to a control group that is identical in
every way except for the treatment, researchers can be more confident that any observed
differences are due to the treatment itself and not some other factor.
4. Ethical Considerations in Research

●​ Algorithmic Bias: This often arises because machine learning models are trained on
historical data that contains human biases. If a company's past hiring data shows a bias
against women, an AI trained on that data will learn and perpetuate that bias. Ethical AI
involves auditing data for bias and building models that are fair and explainable.
●​ Data Privacy & Informed Consent: This is more than a checkbox. Participants must be
clearly told what data is being collected, how it will be stored and used, who will see it,
 and what the risks are. They must also be informed of their right to withdraw from the
study at any time. Regulations like GDPR in Europe set strict rules for this.
●​ Ethics Review Boards (IRBs): These independent committees are crucial for
protecting human subjects. They review research plans to ensure the risks to
participants are minimized, consent is properly obtained, and the research is ethically
sound before it can begin.
●​ Academic Integrity:
○​ Plagiarism: Presenting someone else's ideas, code, or words as your own. Using
proper citation is the only way to avoid it.
○​ Self-Plagiarism: Reusing your own work from a previous publication without
citing it. This can be deceptive as it presents old work as new.
○​ Intellectual Property: Copyright automatically protects original work (like
software code), while patents protect novel inventions and require a formal
application process.
○​ Open Access: The practice of making research outputs freely available to all,
accelerating scientific progress. Creative Commons (CC) licenses allow
creators to specify how their work can be reused.
Chapter 2: Data Collection and Measurement
1. Types of Data

●​ Structured Data: Follows a strict model, like a database table. It has clearly defined
fields, data types (e.g., integer, string), and relationships. This makes it easy to query
and analyze with traditional tools like SQL.
●​ Semi-Structured Data: Doesn't fit into a rigid database schema but contains tags or
markers to separate elements. JSON and XML are classic examples. They are flexible
and widely used for APIs and web services.
●​ Unstructured Data: Has no inherent organization. This category includes about 80% of
the world's data (text from emails and documents, images, audio files). It requires
advanced techniques like Natural Language Processing (NLP) and computer vision to
extract meaningful insights.
2. Sources of Data

●​ Primary Data: You collect it. This gives you full control over the data's quality,
relevance, and format, but it is often expensive and time-consuming. Example: Running
a usability study with your own participants to gather feedback on a new app feature.
●​ Secondary Data: Someone else collected it. This is fast and cheap, but the data might
be outdated, incomplete, or not perfectly aligned with your research question. Example:
Using a public dataset from Kaggle to train a machine learning model.
3. Sampling Techniques

The goal of sampling is to select a representative subset of a larger population.

 ●​ Probability Sampling (unbiased): Leads to generalizable results.
○​ Simple Random: Like drawing names from a hat. Every member of the population
has an equal chance of being selected.
○​ Stratified: Ensures representation from key subgroups. If a user base is 70%
male and 30% female, a stratified sample would maintain that ratio.
○​ Systematic: Simple and efficient. You pick a random starting point and then
select every nth member.
○​ Cluster: Divides the population into groups (clusters), randomly selects some
clusters, and then samples everyone within those selected clusters. It's practical
for geographically dispersed populations.
●​ Non-Probability Sampling (biased): Quicker and easier but the results cannot be
reliably generalized to the whole population.
○​ Convenience: The most common method, but also the least reliable. E.g.,
surveying your classmates because they are easily accessible.
○​ Purposive: You use your judgment to select participants who are "experts" or
have specific knowledge you need.
○​ Snowball: Used for hard-to-reach populations. You find one person, and they
refer you to others.
Sample Size and Errors: A larger sample size generally reduces random error (the natural
variation from sampling). However, a large sample size does not fix systematic error (bias
from a flawed sampling method, like only surveying morning users).
4. Validity and Reliability

●​ Validity (Accuracy): Are you measuring the right thing?

○​ Internal Validity: The degree of confidence that the causal relationship being
tested is trustworthy and not influenced by other factors or variables.
○​ External Validity: The extent to which results from a study can be applied
(generalized) to other situations, groups, or events.
○​ Construct Validity: Does your test actually measure the theoretical concept
you're interested in? E.g., Does a "programmer aptitude test" truly measure
aptitude, or just knowledge of a specific language?
●​ Reliability (Consistency): Can you get the same results again?
○​ Test-Retest Reliability: If you give the same test to the same person at two
different times, do they get a similar score?
○​ Inter-Rater Reliability: If two researchers observe the same event (e.g., a user
interacting with an interface), do they come to the same conclusions? This is
crucial for reducing subjectivity in qualitative analysis.
Chapter 3: Data Analysis and Measurement Scales
1. Scales of Measurement

The scale determines which mathematical operations are valid.

 ●​ Nominal: Categorical data. You can count frequencies and find the mode (most
common category), but you can't calculate a mean. Example: Programming languages
(Python, Java, C++).
●​ Ordinal: Ordered categories. You can find the median and percentiles, but the
differences between ranks are not equal. Example: User satisfaction ratings ("Very
Dissatisfied", "Neutral", "Very Satisfied").
●​ Interval: Ordered data with equal intervals. You can calculate mean, median, and
standard deviation. However, there's no "true zero." Example: Temperature in Celsius.
20°C is not "twice as hot" as 10°C.
●​ Ratio: The most informative scale. It has a true zero, meaning the complete absence of
the variable. You can use all statistical methods, including ratios. Example: Server
response time. A response time of 0ms is meaningful, and 200ms is twice as long as
100ms.
2. Choosing Statistical Techniques

●​ Univariate Analysis: Describes a single variable. It's the first step in any analysis, used
to understand distributions (e.g., with a histogram) and find the central tendency (mean,
median, mode).
●​ Bivariate Analysis: Explores the relationship between two variables.
○​ Correlation: A scatter plot is a great way to visualize this. Pearson's r is used for
linear relationships with interval/ratio data, while Spearman's is for ordinal data or
non-linear relationships.
○​ Cross-tabulation: A table that shows the frequency of two categorical variables
together, often analyzed with a Chi-Square test.
●​ Data Analysis Tools:
○​ Excel: Best for small datasets and quick, simple visualizations or calculations.
○​ R & Python: Industry standards for serious data science. R is built for statistics
and has excellent visualization packages. Python is a versatile language with
powerful libraries (like Pandas, Scikit-learn) for everything from data cleaning to
machine learning.
3. Chi-Square Test (χ²)

This test is specifically for categorical data. It compares the frequencies you observed in your
sample to the frequencies you would expect to see if there were no relationship between the
variables (i.e., if the null hypothesis were true).
●​ Purpose: To see if the difference between observed and expected counts is statistically
significant.
●​ Types:
○​ Goodness-of-Fit: Tests if your sample data fits a known distribution. Example:
Does the distribution of bug types in your new software match the historical
distribution?
○​ Test of Independence: Tests if two categorical variables are associated.
Example: Is there an association between the operating system a user has and
 whether they purchase a subscription?
●​ Interpretation: The test produces a p-value.
○​ If p ≤ 0.05, the result is statistically significant. You reject the null hypothesis and
conclude that there is a real association between the variables.
○​ If p > 0.05, the result is not statistically significant. You fail to reject the null
hypothesis, meaning you don't have enough evidence to say there's an
association.