Automated Harvesting by Robotic Arm Fruit

Harvesting Robot
Aiza Arshad Shahroz Ahmad Aneeza Naz Toor
Department of Electrical, Electronics Department of Electrical, Electronics Department of Electrical, Electronics
& Telecommunication Engineering & Telecommunication Engineering & Telecommunication Engineering
The University of Engineering and The University of Engineering and The University of Engineering and
Technology, Lahore Technology, Lahore Technology, Lahore
Lahore, Pakistan Lahore, Pakistan Lahore, Pakistan
[email protected] [email protected] [email protected]

Abstract—This final year project focuses on the creation of an however, most relied on high-end computing hardware and
automated robotic arm system for fruit harvesting, with a lacked real-time adaptability in dynamic farm conditions.
focus on apples and related crops in orchard settings. The need
for automation has grown urgently as Pakistan's agriculture
industry struggles with a workforce deficit, particularly among More recent systems have incorporated vision-based AI,
skilled personnel. Inverse kinematics for accurate robotic arm leveraging convolutional neural networks for fruit detection.
movement, RGB-D sensors for fruit detection and localization, These systems typically require powerful edge processors
and real-time data processing for responsive control are all such as NVIDIA Jetson or Raspberry Pi 4, which increase
included in the suggested system. The robot can navigate the overall cost and energy consumption. While effective,
orchard terrains, avoid obstacles, and gather fruits carefully to such solutions are often unaffordable or impractical for
prevent damage or bruises by utilizing contemporary small-scale farmers in developing countries. Additionally,
technologies. In addition to lowering the need for physical
many robotic harvesters are either stationary or semi-mobile,
labor, this research seeks to increase production, increase
harvesting accuracy, and eventually support more sustainable
limiting their range and deployment flexibility.
agricultural methods.
In contrast, the system proposed in this paper addresses
I. INTRODUCTION these challenges by integrating low-cost components like
This project presents a compact, self-driving robot that the ESP32-CAM, which supports AI inference at the edge,
integrates Mecanum-wheel mobility with a 6-DOF robotic and an Arduino Mega, which manages control operations
arm, enabling omnidirectional navigation and precise object without relying on bulky external hardware. The robot’s
manipulation. Built using an Arduino Mega and cost- mechanical design is kept simple yet adaptable, featuring
effective components, the system is designed for Mecanum wheels for omnidirectional movement and a
affordability, adaptability, and autonomous operation. It pressure-sensitive gripper to prevent fruit bruising. Thus,
responds to real-time sensor input, making it suitable for this work contributes a compact, affordable, and mobile
environments like warehouses, laboratories, and agricultural solution tailored for small to medium-sized orchards,
fields. By combining mobility, perception, and manipulation particularly in developing agricultural regions like Pakistan.
in one platform, the robot demonstrates how modern
robotics can enhance operational efficiency and reduce III. METHODOLOGY
manual labor, offering a scalable solution for diverse real-
A. System Overview
world applications~\cite{author2023}.

II. RELATED WORK The proposed fruit harvesting system consists of a compact
mobile robot equipped with a 6-degree-of-freedom (6-DOF)
Numerous research efforts have explored robotic automation robotic arm, a two-fingered pressure-sensitive gripper, and a
for agricultural harvesting, particularly in fruit picking vision module. The mobility of the system is achieved using
applications. Early developments in this field focused on Mecanum wheels, which allow omnidirectional
robotic arms equipped with simple image processing movement—critical for navigating narrow rows in orchards.
algorithms for detecting and plucking apples, grapes, and Mounted on the mobile platform is the robotic arm, which
citrus fruits under controlled conditions. For instance, Silwal performs fruit-picking operations. The arm receives fruit
et al. developed an apple-picking robot using stereo vision location input from an AI-powered vision system, while an
and a six-DOF arm, which demonstrated success in onboard microcontroller coordinates movement and
structured environments but was limited by cost and actuation. The robot is designed to identify ripe fruits,
processing complexity [1]. Similarly, mobile harvesting approach them gently, pluck them without causing bruises,
robots using wheeled platforms have been studied, offering and place them into a collection basket. All components are
semi-autonomous navigation through orchard rows; optimized to operate efficiently in structured agricultural
environments where fruits grow within a low to medium D. Kinematic Modeling
canopy height.
To enable precise positioning of the end-effector (gripper),
B. Mechanical Design both inverse and forward kinematics were implemented.
Inverse kinematics calculations are used to determine the
required joint angles (θ₃ to θ₆ and vertical offset D₁) based
The robotic arm follows a PRRRRR (Prismatic-Revolute- on the position of the detected fruit. These calculations are
Revolute-Revolute-Revolute-Revolute) configuration, coded in MATLAB, where custom trigonometric
allowing a combination of vertical lift and rotational expressions were derived to handle the PRRRRR
flexibility to approach the fruit from different angles. The configuration. Forward kinematics, using the Denavit–
mechanical parts of the arm and gripper were modeled using Hartenberg (DH) convention, was applied to validate the
SolidWorks, and STL files were generated for 3D printing. joint trajectories and to simulate the robot’s motion. Each
transformation matrix from the base to the end-effector was
Each link and joint were carefully dimensioned to ensure the computed symbolically and multiplied in sequence to derive
arm could extend, rotate, and grip fruits within a reachable the final position. Simulations demonstrated that the end-
workspace of up to 2.5 meters vertically. The gripper, which effector could reach fruit targets with high spatial accuracy.
mimics human fingers, is designed to be soft and adaptive. It Additionally, MATLAB’s plotting functions were used to
includes force-sensitive resistors (FSRs) embedded within visualize the 3D trajectory of the arm across multiple
the gripping pads to ensure fruits are grasped with movement steps, confirming the theoretical reach of the
appropriate pressure, avoiding bruising or detachment of robot in a simulated orchard environment.
unripe produce. The modular design allows easy
maintenance and potential scalability for different fruit sizes
or environments.

C. Electronics and Control

The system’s control logic is implemented on an Arduino

Mega 2560 microcontroller, selected for its wide I/O
availability and compatibility with multiple actuators and
sensors. The arm’s servos and stepper motors are controlled
through pulse-width modulation (PWM) signals generated
by the Arduino, based on position inputs derived from the
vision system. The fruit detection module is powered by an
ESP32-CAM, which uses a lightweight AI model trained on
Edge Impulse to classify and locate fruits. The ESP32 sends
the fruit’s coordinates wirelessly to the Arduino using a
Fig 2: Flow Chart for the System
custom UART-based protocol. To ensure safe interaction
with the fruit, the FSR-equipped gripper provides feedback
to the Arduino to modulate grip force dynamically. This IV. RESULTS AND DISCUSSION
ensures gentle harvesting, particularly important for soft- A. Simulations Results
skinned fruits like guava or citrus. Communication between
components is optimized to minimize latency, and the The forward kinematics model of the 6-DOF robotic arm
control logic supports manual override if needed during was simulated in MATLAB using derived Denavit–
field testing. Hartenberg parameters and symbolic transformation
matrices. Sample joint values, including base offset.

D1=225D_1 = 225D1​ =225 mm and joint angles

θ3=θ4=θ5=θ6=45∘ , were used to evaluate the arm's
reachability. The simulated output returned joint positions
and the end-effector location in 3D space.

The calculated coordinates were stored in the form of XXX,

YYY, and ZZZ arrays, and plotted using MATLAB’s plot3
function. The resulting trajectory plot showed a smooth,
continuous path from the robot base to the fruit position.
This validated the correctness of the inverse kinematics
model and demonstrated the robot’s ability to perform
Fig 1: System Block Diagram
coordinated joint movements to reach specific points in
space.
Fig 3: Arm Movement 1 Fig 6: Fruit Harvesting Robot

C. Discussion

The robotic system proved effective in semi-structured

orchard environments where fruit-bearing branches are
within reachable height. The integration of kinematic
modeling and real-time vision feedback allowed the robot to
detect, approach, and pluck fruits with minimal damage.
The use of Mecanum wheels provided flexibility in
navigating tight tree rows, enhancing the robot’s field
usability.

Fig 4: Arm Movement 2 Despite its success, the system still faces several challenges.
The current height limitation prevents it from accessing
fruits on tall trees, and performance drops under low-light
conditions. Future improvements such as a vertical
extension mechanism and enhanced lighting or thermal
sensing could help overcome these limitations. Nevertheless,
the prototype demonstrates a promising step toward scalable,
cost-efficient robotic fruit harvesting suitable for small and
mid-sized farms.

Fig 5: Arm Movement 3

B. Prototype Testing

The vision system, implemented using an ESP32-CAM and

trained via Edge Impulse, was tested in indoor and semi-
outdoor environments. The AI model demonstrated an
average fruit detection accuracy of approximately 88% for
ripe citrus and guava fruits under normal lighting conditions.
The response time for detection and communication with the Fig 6: Edge Impulse Dataset Overview
Arduino controller was measured at less than 300 ms, which
was acceptable for semi-real-time operation. V. CONCLUSION AND FUTURE SCOPE
A. Success in Automating Part of the Harvesting Task
Mechanical testing of the gripper showed a successful
picking rate of around 75% during trial runs. Most failures
were due to either fruit misclassification or improper grip The development of the automated fruit harvesting robot
alignment. Additionally, while the system performed well demonstrates the potential of low-cost robotics in solving
for fruits located between 0.8 m and 2.0 m, it struggled to real-world agricultural challenges. The designed system
reach higher branches, which currently limits its use in taller successfully integrates mechanical, electrical, and AI
orchards. components into a mobile platform capable of detecting and
plucking fruits from bushes and lower tree branches.
B. Summary of Work’s Impact Differential Drive Mobile Robot in ROS-Gazebo for Greenhouse
Digital Twin Environments," IEEE Access, vol. 12, pp. 19464–19479,
2024, doi: 10.1109/ACCESS.2024.3362145.
Simulation results confirmed that the robot’s arm can follow [2] Y. Yu, K. Zhang, H. Liu, and L. Yang, "Real-Time Visual
accurate trajectories to target coordinates using inverse and Localization of the Picking Points for a Ridge-Planting Strawberry
forward kinematics. Real-world testing showed effective Harvesting Robot," IEEE Access, vol. 8, pp. 174764–174776, 2020,
doi: 10.1109/ACCESS.2020.3025252.
fruit detection using the ESP32-CAM and a reasonable
[3] T. Ilyas, A. Khan, M. Umraiz, and Y. Jeong, "Multi-Scale Context
success rate in harvesting, especially for fruits located below Aggregation for Strawberry Fruit Recognition and Disease
2.5 meters. This confirms that the robot can reduce manual Phenotyping," IEEE Access, vol. 9, pp. 149247–149258, 2021, doi:
labor and improve harvesting efficiency in semi-structured 10.1109/ACCESS.2021.3125786.
orchard environments. [4] R. Oshita, S. Shibata, T. Yamamoto, and S. Mu, "Development of
Harvesting Robot for Strawberry," in Proc. IEEE GCCE, pp. 129–132,
2023, doi: 10.1109/GCCE57775.2023.10304857.
C. Future Work [5] M. F. Ridho and Irwan, "Strawberry Fruit Quality Assessment for
Harvesting Robot using SSD Convolutional Neural Network," in Proc.
Int. Conf. Data Sci. Comput., pp. 195–200, 2021, doi:
However, limitations still exist in the current version, 10.1109/ICDSC54077.2021.9650142.
particularly in reaching higher fruits and operating under [6] S. Muharom, D. A. P. Wardhana, M. Septyan, A. D. Pambudi, et al.,
poor lighting. These challenges open doors for future "Identifying Strawberry Ripeness Level Using Camera for Cartesian
enhancement. Planned improvements include integrating a Robot Application Based on Microcontroller," in Proc. Int. Conf. Voc.
vertical extension mechanism for taller trees, using solar- Educ. Electr. Eng. (ICVEE), pp. 1–6, 2023, doi:
10.1109/ICVEE58025.2023.10355189.
powered modules for sustainable operation, and
[7] A. Tafuro, M. E. Khan, M. Arfini, et al., "Strawberry Picking Point
implementing SLAM-based navigation for full autonomy in Localization, Ripeness and Weight Estimation," in Proc. IEEE Int.
larger orchards. Conf. Robot. Autom. (ICRA), pp. 4524–4530, 2022, doi:
10.1109/ICRA46639.2022.9811654.
[8] J. Haejun, M. HaeJun, Y. Jeong, H. Kwon, et al., "Automated
In conclusion, the project lays a strong foundation for Technology for Strawberry Size Measurement and Weight Prediction
affordable, modular agricultural robots tailored to small- Using AI," IEEE Access, vol. 12, pp. 104627–104639, 2024, doi:
scale farming needs in developing regions. With future 10.1109/ACCESS.2024.3381234.
upgrades, this system can evolve into a versatile and [9] S. Yuan, Y. Cao, and X. Cheng, "Research on Strawberry Quality
intelligent farm assistant. Grading Based on Object Detection and Stacking Fusion Model,"
IEEE Access, vol. 11, pp. 90621–90631, 2023, doi:
10.1109/ACCESS.2023.3297524.
ACKNOWLEDGMENTS [10] H. N. B. Ranasinghe, C. K. Sasika, A. L. Kulasekera, C. V.
Kulasekera, and P. C. Kohona, "Soft Pneumatic Grippers for
We wish to extend our sincere appreciation to Dr. Haris Reducing Fruit Damage During Strawberry Harvesting," in Proc.
Anwar for his exceptional mentorship, continuous support, IEEE MERCon, pp. 267–272, 2022, doi:
and insightful guidance throughout the duration of this 10.1109/MERCon55376.2022.9854065.
research project. His profound knowledge in the field of [11] T. Fujinaga, "Cutting Point Detection for Strawberry Fruit Harvesting
robotics and automation, coupled with his unwavering and Truss Pruning by Agricultural Robot," in Proc. IEEE CASE, pp.
commitment to academic excellence, significantly 100–105, 2023, doi: 10.1109/CASE56687.2023.10260576.
contributed to the successful development and completion of [12] K. Gamage, J. K. Arachchige, S. Siriwardhana, and P. C. Kohona,
"Systems Engineering-Based Approach for the Development of a
this work. Dr. Haris Anwar provided critical feedback at Strawberry Harvesting Robot," in Proc. MERCon, pp. 159–164, 2023,
various stages of the project, helping to refine our research doi: 10.1109/MERCon58055.2023.10159124.
objectives, strengthen the methodology, and ensure the [13] J. Mapes, "Harvesting End-Effector Design with Spiral-Curve Teeth
technical rigor of the proposed system. His thoughtful and Picking Control," IEEE UCF Undergraduate Research Journal,
suggestions and encouragement were instrumental not only pp. 1–7, 2021.
in overcoming technical challenges but also in fostering a [14] A. Smith et al., "Vision-Based Automated Strawberry Grasping with
spirit of innovation and perseverance within our team. We a 6-DoF Delta Robot," in Proc. IEEE Int. Conf. Mechatronics, pp.
223–228, 2020.
are deeply grateful for his valuable time, intellectual
contributions, and professional support, without which this [15] L. Zhang et al., "End-Effector Optimization for Multi-Berry Picking,"
in Proc. IEEE Int. Conf. Ind. Technol., pp. 602–607, 2021..
research would not have reached its present form.
[16] D. Kim et al., "Multi-Modal Sensor Fusion for Outdoor Strawberry
Localization," in Proc. IEEE Sensors Conf., pp. 1214–1219, 2022.
REFERENCES
[17] S. Gupta et al., "Deep Reinforcement Learning for Adaptive
Harvesting Path Planning," in Proc. IEEE Int. Conf. Autom. Sci. Eng.
[1] R. Singh, L. Seneviratne, and I. Hussain, "A Deep Learning-Based (CASE), pp. 771–776, 2023.
Approach to Strawberry Grasping Using a Telescopic-Link