This thesis develops a coordinated multi‑arm viewpoint planning framework for detecting pepper peduncles in densely foliated greenhouse environments, leveraging a three‑armed HortiBot equipped with a wide‑baseline ZED2i stereo camera and two close‑range RealSense D405 sensors. By selecting and sequencing viewpoints either through classical Task & Motion Planning or Reinforcement Learning, the system aims to maximize peduncle visibility while avoiding collisions and minimizing execution time. Comparative experiments against a single‑arm baseline will quantify improvements in detection accuracy and operational efficiency, supporting more reliable robotic harvesting.

1. Background & Motivation

In greenhouse harvesting, accurately detecting the peduncle—the small stem connecting fruit to the plant—is critical for automated picking. However, dense foliage and overlapping leaves often occlude the peduncle, making vision-based detection challenging. A three-armed Horti-Bot equipped with a wide-baseline ZED2i stereo camera and two close-range Intel RealSense D405s can coordinate viewpoints to peer around leaves, balancing global context with high-resolution local scans. By planning multi-arm trajectories that exploit each sensor’s advantages, we aim to robustly reveal and detect peduncles under heavy occlusion.

2. Objectives & Research Questions

Objectives:

• Objective 1: Design a multi-arm viewpoint planning framework for peduncle detection under occlusion, using either TAMP or RL.

• Objective 2: Compare detection performance and efficiency against a single-arm base-line.

• Objective 3: Analyze how sensor coordination strategies (global vs. local views) impact detection accuracy and planning cost.

Research Questions:

1. How effectively can multi-arm coordination overcome leaf occlusions compared to single-arm strategies?

2. What viewpoint selection and sequencing maximize peduncle visibility while minimizing planning and execution time?

3. Within the chosen paradigm (TAMP or RL), which factors most influence detection robustness?

Methodology Overview

1. Simulation Setup:

• NVIDIA Isaac Sim with Hortibot model and BlenderProc pepper plant assets.

• Implement peduncle mask fusion for evaluation.

2. Planner Implementation: Select one planning approach:

• TAMP: Discrete viewpoint selection and continuous motion planning with collision avoidance.

• RL: Define state (visibility map, arm poses), actions (viewpoint targets), and reward (visibility gain minus time/collision penalties).

3. Evaluation Metrics:

• Peduncle detection rate (true positives / total peduncles).

• False positive rate.

• Planning and execution time.

• Collision incidents (should be zero).

4. Experiments:

• Compare multi-arm vs. single-arm under varying leaf densities.

• Ablations: disable global or local sensors to assess contributions.

• Enrolled in computer science or similar MSc program in and around Bonn/Cologne
• Familiarity with motion planning and 3D perception
• Excellent academics and strong background in probability theory, linear algebra, and optimization.
• Programming experience with C++, Python, and ROS (Robot Operating System).
• Experience with reinforcement learning, and computer vision is a plus.
• Enthusiasm for real-world robot deployment and scientific publishing of results
• Ability to work independently as well as collaborate in a team

• Lenz et al. HortiBot: An Adaptive Multi-Arm System for Robotic Horticulture of Sweet Pe
• Song et al. GS-NBV: a Geometry-based, Semantics-aware Viewpoint Planning Algorithm for Avocado Harvesting under Occlusions
• Chen et al. Cooperative Task and Motion Planning for Multi-Arm Assembly Systems