Skip to content

New config with thermal camera: CERBERUS_M100_SENSOR_CONFIG_2#777

Merged
nkoenig merged 6 commits intoadlarkin/add_thermal_camerafrom
cerberus_m100_thermal
Feb 11, 2021
Merged

New config with thermal camera: CERBERUS_M100_SENSOR_CONFIG_2#777
nkoenig merged 6 commits intoadlarkin/add_thermal_camerafrom
cerberus_m100_thermal

Conversation

@iche033
Copy link
Contributor

@iche033 iche033 commented Feb 3, 2021

Created a new config: CERBERUS_M100_SENSOR_CONFIG_2:

This is a copy of CERBERUS_M100_SENSOR_CONFIG_1 with thermal camera sensor added to model.sdf and vehicle_topics.launch updated to bridge the thermal camera topics

Still need clarification on (Edit: done):

  • exact FLIR Tau2 model used (there are a few product variations on the FLIR Tau2 page)
  • location of the sensor on the drone
    • Adjusted camera and thermal camera pose

cc @tiralonghipol @mihirk284

The thermal camera is mounted side by side with the rgb camera:

cerberus_m100_config_2_thermal

to test:

ign launch cave_circuit.ign worldName:=simple_cave_01 circuit:=cave robotName1:=Xa robotConfig1:=CERBERUS_M100_SENSOR_CONFIG_2

Signed-off-by: Ian Chen [email protected]

@iche033 iche033 marked this pull request as draft February 3, 2021 00:47
@osrf-jenkins
Copy link

Build finished. 21 tests run, 0 skipped, 1 failed.

@osrf-jenkins
Copy link

Build finished. 21 tests run, 0 skipped, 1 failed.

@mihirk284
Copy link
Contributor

mihirk284 commented Feb 3, 2021

Hi @iche033 ,
I've made minor modifications to the position of the RGB camera sensor to also accommodate the Thermal camera model.
The updated sdf looks like this (only the pose has been changed) :-

<visual name="camera_visual">
    <pose>0.192 -0.015 -0.023 0 0.174533 0</pose>
    ... same as before ...
</visual>
<sensor name="camera_front" type="camera">
    <pose>0.192 -0.015 -0.023 0 0.174533 0</pose>
    ....same as before
</sensor>

<!-- thermal camera -->
<visual name="thermal_camera_visual">
    <pose>0.192 0.015 -0.023 0 0.174533 0</pose>
    ... same as before ...
</visual>
<sensor name="thermal_camera" type="thermal">
  <pose>0.192 0.015 -0.023 0 0.174533 0</pose>
  ... same as before
</sensor>

Shall I open a PR to this branch?

CC: @tiralonghipol

Thanks,
Mihir Kulkarni

@iche033
Copy link
Contributor Author

iche033 commented Feb 3, 2021

great, thanks for the pose changes. I'll take care of it

Signed-off-by: Ian Chen <[email protected]>
@iche033 iche033 changed the title Add thermal camera to CERBERUS M100 New config with thermal camera: CERBERUS_M100_SENSOR_CONFIG_2 Feb 3, 2021
@iche033 iche033 marked this pull request as ready for review February 3, 2021 19:02
@osrf-jenkins
Copy link

Build finished. 21 tests run, 0 skipped, 1 failed.

Signed-off-by: Ian Chen <[email protected]>
Signed-off-by: Ian Chen <[email protected]>
Copy link
Contributor

@adlarkin adlarkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thermal camera is working for me, but I noticed a line in the camera output in rViz that I did not see when testing other robots with thermal cameras:
thermal_camera_line

@osrf-jenkins
Copy link

Build finished. 21 tests run, 0 skipped, 1 failed.

@iche033
Copy link
Contributor Author

iche033 commented Feb 3, 2021

The thermal camera is working for me, but I noticed a line in the camera output in rViz that I did not see when testing other robots with thermal cameras:

is that the grid in rviz?

@adlarkin
Copy link
Contributor

adlarkin commented Feb 3, 2021

is that the grid in rviz?

Yes, it was, so this actually looks good to me!

@adlarkin adlarkin self-requested a review February 3, 2021 23:21
@mihirk284
Copy link
Contributor

mihirk284 commented Feb 3, 2021

Hi @iche033 and @adlarkin,
I don't know if I should open this as an issue or just ask here on the same thread if there's something different in what I was doing for simulating the robot with the thermal camera.

I noticed that there's a significant difference between the image above, and what I was seeing on my machine.

When I tried to simulate the image, it looked a bit different on my machine.
image (4)

Also, the floor of the tunnel looked a little different to me, with some additional flickering:-

thermal_subt_m100.mp4

Thanks,
Mihir

@adlarkin
Copy link
Contributor

adlarkin commented Feb 3, 2021

I noticed that there's a significant difference between the image above, and what I was seeing on my machine.

Are you using the Ignition binaries, or are you building Ignition from source? We are still in the process of making ign-gazebo and ign-rendering releases that will add the thermal camera functionality. Also, it actually turns out that the image I originally posted is incorrect - there was a bug that was just recently addressed in gazebosim/gz-rendering#230. So, make sure that you pull the latest changes from ign-rendering if you want to try building from source.

@adlarkin
Copy link
Contributor

adlarkin commented Feb 3, 2021

@mihirk284 you should see something like this once inside the tunnel if you're building ign-gazebo and ign-rendering from source:
correct_view_in_tunnel

The command I used here was:

ign launch -v 4 competition.ign worldName:=tunnel_circuit_practice_01 circuit:=tunnel robotName1:=X1 robotConfig1:=CERBERUS_M100_SENSOR_CONFIG_2

@mihirk284
Copy link
Contributor

mihirk284 commented Feb 3, 2021

Thank you @adlarkin,
I think I had not updated the ign dome repos for a while
I'll pull the updated repos and try it out :)

@adlarkin
Copy link
Contributor

adlarkin commented Feb 3, 2021

Thank you @adlarkin,
I think I had not updated the ign dome repos for a while
I'll pull the updated repos and try it out :)

Sounds good! You'll need to build ign-gazebo and ign-rendering from source, but you should be able to use the binaries for all the other ignition libraries.

@nkoenig
Copy link
Contributor

nkoenig commented Feb 11, 2021

The Fuel models have been updated to use:

      <sensor name="thermal_camera" type="thermal">
        <pose>0.435 -.01 0.125 0 0 0</pose>
        <camera name="thermal_camera">
          <horizontal_fov>0.4922</horizontal_fov>
          <lens>
            <intrinsics>
              <!-- fx = fy = width / ( 2 * tan (hfov / 2 ) ) -->
              <fx>764.35630</fx>
              <fy>764.35630</fy>
              <!-- cx = ( width + 1 ) / 2 -->
              <cx>192.5</cx>
              <!-- cy = ( height + 1 ) / 2 -->
              <cy>144.5</cy>
              <s>0</s>
            </intrinsics>
          </lens>
          <image>
            <width>384</width>
            <height>288</height>
            <format>L8</format>
          </image>
          <clip>
            <near>0.1</near>
            <far>100</far>
          </clip>
        </camera>
        <always_on>1</always_on>
        <update_rate>25</update_rate>
        <plugin
          filename="ignition-gazebo-thermal-sensor-system"
          name="ignition::gazebo::systems::ThermalSensor">
          <min_temp>233.15</min_temp>
          <max_temp>833.15</max_temp>
        </plugin>
      </sensor>

@nkoenig nkoenig merged commit 28f094d into adlarkin/add_thermal_camera Feb 11, 2021
nkoenig pushed a commit that referenced this pull request Feb 11, 2021
* starting to add thermal camera to marble_hd2_sensor_config_1

Signed-off-by: Ashton Larkin <[email protected]>

* add optical frame publisher, camera info topic, and more thermal camera sensor parameters

Signed-off-by: Ashton Larkin <[email protected]>

* fix topic mapping (#776)

Signed-off-by: Ian Chen <[email protected]>

* add thermal camera visual, fix location and textures

Signed-off-by: Ashton Larkin <[email protected]>

* update hfov assuming degrees in spec

Signed-off-by: Ashton Larkin <[email protected]>

* break out thermal camera addition to HD2 configs 3 and 4

Signed-off-by: Ashton Larkin <[email protected]>

* set thermal default to 0 for HD2 vehicle_topics.launch

Signed-off-by: Ashton Larkin <[email protected]>

* New config with thermal camera: CERBERUS_M100_SENSOR_CONFIG_2 (#777)

* adding thermal camera to cerberus m100

Signed-off-by: Ian Chen <[email protected]>

* fix bridge

Signed-off-by: Ian Chen <[email protected]>

* update thermal camera spec

Signed-off-by: Ian Chen <[email protected]>

* new config_2

Signed-off-by: Ian Chen <[email protected]>

* remove config_2

Signed-off-by: Ian Chen <[email protected]>

* reuse config_1 files

Signed-off-by: Ian Chen <[email protected]>

Co-authored-by: Ian Chen <[email protected]>
@peci1
Copy link
Collaborator

peci1 commented Mar 15, 2021

The thermocam got lost from Fuel... #821.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants