LIDAR assessment

Good question - I haven’t had time to look. But when I make maps and even when I’ve done SLAM outside of my ‘roboland’ things seem to work fine. So I was surprised by the result. And I looked at the documentation - forward is towards the motors supposedly. Edit - I now see that you had already looked at that as well. Yeah - I’m still confused.

That was my thought as well - the onboard firmware doesn’t seem to capture quickly enough, or something.

I thought about running the stats, but didn’t. If you factor in all of the 0 values the SD may actually be pretty high. For the actual readings, yes, the results seem pretty good.

I was surprised by how many 0’s I got, but based on that I had somewhat assumed that the software was treating the zeros as essentially null values - I hadn’t seen that confirmed anywhere.

ooh - I’ll have to look at that. Maybe that’s my problem.

Those changes were intentional - that was documented in this thread:
https://forum.dexterindustries.com/t/hands-on-ros-chapter-9-more-on-navigation/8390?u=keithw
But I need to look at the driver and the configuration more closely.

/K

2 Likes

Right now I’m not using the IMU at all, so I’m sure nothing is depending on the IMU.

I hadn’t thought about that - I was assuming that ROS would rely more on the accelerometers, but I realize I have no basis for this assumption. @jimrh 's concern about the motor on the lidar throwing things off seems reasonable.

Not at all. I’m sure we’ll have an IMU thread at some point. At the pace you’re going I suspect you’ll start it :slight_smile:

/K

1 Like

The ydlidar node publishes a zero distance for any distance less than the min_value, and if no data is received.

All actual laser ranges are published relative to the center of rotation.

It is impossible for a range from the center of rotation to be 0, since the LIDAR has a 35mm minimum radius.

Thus 0 is a good “invalid” value. (The node does allow configuring NAN/infinity for “invalid” if desired.)

Probably to save message length in queuing and dequeuing time in industrial systems. Rather than publishing a variable size list of angles and a list of ranges, the structure allows for start/stop/increment angle (three values) and the list of ranges (1 to a max of 719 for our 360 by half degree LIDAR). In the never misses a beat case this would save 716 floats.

Since ROS is primarily targeted for industrial robots with costly sensors, perhaps the waste, in my case only 65 good values, of 589 to 654 floats transmitted is of no concern to anyone?

 

sensor_msgs/LaserScan (/scan)

# Single scan from a planar laser range-finder
#
# If you have another ranging device with different behavior (e.g. a sonar
# array), please find or create a different message, since applications
# will make fairly laser-specific assumptions about this data

Header header            # timestamp in the header is the acquisition time of 
                         # the first ray in the scan.
                         #
                         # in frame frame_id, angles are measured around 
                         # the positive Z axis (counterclockwise, if Z is up)
                         # with zero angle being forward along the x axis
                         
float32 angle_min        # start angle of the scan [rad]
float32 angle_max        # end angle of the scan [rad]
float32 angle_increment  # angular distance between measurements [rad]

float32 time_increment   # time between measurements [seconds] - if your scanner
                         # is moving, this will be used in interpolating position
                         # of 3d points
float32 scan_time        # time between scans [seconds]

float32 range_min        # minimum range value [m]
float32 range_max        # maximum range value [m]

float32[] ranges         # range data [m] (Note: values < range_min or > range_max should be discarded)
float32[] intensities    # intensity data [device-specific units].  If your
                         # device does not provide intensities, please leave
                         # the array empty.

Compact Message Definition
std_msgs/Header header
float32 angle_min
float32 angle_max
float32 angle_increment
float32 time_increment
float32 scan_time
float32 range_min
float32 range_max
float32[] ranges
float32[] intensities
1 Like

The parameters are set in the launch file.

    <node name="ydlidar_node" pkg="ydlidar" type="ydlidar_node" output="screen" respawn="false" >
        <param name="port" type="string" value="/dev/ydlidar"/>
        <param name="baudrate" type="int" value="115200"/>
        <param name="frame_id" type="string" value="base_scan"/>
        <param name="low_exposure" type="bool" value="false"/>
        <param name="resolution_fixed" type="bool" value="true"/>
        <param name="auto_reconnect" type="bool" value="true"/>
        <param name="reversion" type="bool" value="false"/>
        <param name="angle_min" type="double" value="-180" />
        <param name="angle_max" type="double" value="180" />
        <param name="range_min" type="double" value="0.1" />
        <param name="range_max" type="double" value="16.0" />
        <param name="ignore_array" type="string" value="" />
        <param name="samp_rate" type="int" value="9"/>
        <param name="frequency" type="double" value="7"/>
    </node>

Looking at this clears up one mystery I think. Note the min angle is -180, and max is 180. it’s not 0 to 360. That would explain why my zeroth distance value is straight behind me I think.

When the robot starts it has a message

[YDLidar]: [YDLIDAR INFO] Current Sampling Rate : 4K
[YDLidar]: [YDLIDAR INFO] Now YDLIDAR is scanning ......

But I think that’s coming from the firmware - when I check the rosparameters the values are the ones set in the launch file. Next thing to check is what happens if I fool with those. But they’re the ones that came from the book’s examples, so I assume they’re not entirely arbitrary.

/K

1 Like

Is your LIDAR the X4 model?

The datasheet doesn’t include a min/max, only “Typical: 128000”, so that is what I am using.

I got my parms from:

browse https://github.com/YDLIDAR/ydlidar_ros2_driver/blob/master/details.md  
  also in src/ydlidar_ros2_ws/src/ydlidar_ros2_driver/details.md  


LIDAR: X4
Model: 6  (device_type?)
Baudrate: 128000
SampleRate(K): 5
Range(m): 0.12~10
Freq(Hz): 5~12(PWM)
Intensity(bit): false
SingleChannel: false
Voltage(V): 4.8~5.2
lidar_type: TYPE_TRIANGLE (1)
Reversion: false (CounterClockWise)
support_motor_dtr: true


 

The ROS(1) version of the table is the same located here.

I don’t know how to see all the values of a scan. I did an echo to the console and it prints the first 128 ranges. I am getting 105 to 109 non-zero ranges in the 128 values displayed for about 84% non-zero. Is that approximately what you are seeing - 84% non-zero out of 719 ranges?

The datasheet for my lidar says frequency 5K. Is yours 7K like your config param?

My time_increment in the header of each scan is 0.00200000000 or 0.000019999999 which corresponds to 5k.

The change you did in the code sets the scan_time to 1/freq or 1/7 in your launch config. When you look at the scan data does the scan_time value change or is it constant at 1/7?

My code is the unchanged version, and my scan_time value is steady at 0.0112199… = 1/8.9285 and varies sometimes slightly 0.0112400 = 1/8.896 is the value for the fourth of four scans I looked at.


header:
  stamp:
    sec: 1630190582
    nanosec: 744315000
  frame_id: laser_frame
angle_min: -3.1415927410125732
angle_max: 3.1415927410125732
angle_increment: 0.011219973675906658
time_increment: 0.00020000000949949026
scan_time: 0.1120000034570694
range_min: 0.11999999731779099
range_max: 10.0
ranges:
- 0.0
- 0.23000000417232513
- 0.2290000021457672
- 0.2280000001192093
- 0.22599999606609344
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.20499999821186066
- 0.20399999618530273
- 0.2029999941587448
- 0.20100000500679016
- 0.20000000298023224
- 0.1992499977350235
- 0.19824999570846558
- 0.19724999368190765
- 0.19625000655651093
- 0.195250004529953
- 0.19425000250339508
- 0.19425000250339508
- 0.19325000047683716
- 0.19224999845027924
- 0.1912499964237213
- 0.1912499964237213
- 0.1902499943971634
- 0.1902499943971634
- 0.18925000727176666
- 0.18825000524520874
- 0.18825000524520874
- 0.18725000321865082
- 0.1862500011920929
- 0.1862500011920929
- 0.1862500011920929
- 0.18524999916553497
- 0.18524999916553497
- 0.18524999916553497
- 0.18424999713897705
- 0.18424999713897705
- 0.18324999511241913
- 0.18324999511241913
- 0.18324999511241913
- 0.18324999511241913
- 0.1822499930858612
- 0.1822499930858612
- 0.1822499930858612
- 0.1822499930858612
- 0.1822499930858612
- 0.1822499930858612
- 0.1822499930858612
- 0.18125000596046448
- 0.18125000596046448
- 0.18125000596046448
- 0.0
- 0.18125000596046448
- 0.18125000596046448
- 0.18025000393390656
- 0.18025000393390656
- 0.18025000393390656
- 0.18025000393390656
- 0.17925000190734863
- 0.1782499998807907
- 0.1770000010728836
- 0.17599999904632568
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.0
- 0.1899999976158142
- 0.1899999976158142
- 0.18925000727176666
- 0.18925000727176666
- 0.18925000727176666
- 0.18925000727176666
- 0.18825000524520874
- 0.18925000727176666
- 0.1902499943971634
- 0.1902499943971634
- 0.1912499964237213
- 0.19224999845027924
- 0.19325000047683716
- 0.19425000250339508
- 0.195250004529953
- 0.19625000655651093
- 0.19724999368190765
- 0.19824999570846558
- 0.1992499977350235
- 0.20024999976158142
- 0.20125000178813934
- 0.0
- 0.20225000381469727
- 0.20424999296665192
- 0.20524999499320984
- 0.20624999701976776
- 0.20724999904632568
- 0.20925000309944153
- 0.21125000715255737
- 0.20999999344348907
- 0.21324999630451202
- 0.21525000035762787
- 0.2172500044107437
- 0.21924999356269836
- 0.2212499976158142
- 0.22325000166893005
- 0.2252500057220459
- 0.22724999487400055
- 0.0
- 0.22824999690055847
- 0.23125000298023224
- 0.23325000703334808
- 0.23524999618530273
- 0.2382500022649765
- 0.24025000631809235
- 0.24300000071525574
- 0.2460000067949295
- 0.24899999797344208
- 0.25200000405311584
- 0.2549999952316284
- '...'
intensities:
- 0.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 0.0
- 0.0
- 0.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 0.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 0.0
- 0.0
- 1008.0
- 1008.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 0.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1008.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 0.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1012.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- 1008.0
- '...'
1 Like

My little script is on my GitHub test branch for now. I’ve attached it for convenience.
laser_logger.py (633 Bytes)

Not that high - I just looked at my spreadsheet (thank god for the COUNTIF function). Looking at the first 50 rows I’m averaging only 366 values (range 357 - 371), so just a hair under 51%.

right now I’ve only captured the ranges. I’ll take a look at the headers to see.

/K

2 Likes

I found a way to generate csv files. I start a bag play in one shell, and then a topic echo in another (must start the play before the echo)

For ROS2:

$ ros2 bag info rosbag2_2021_08_28-18_43_25/

Files:             rosbag2_2021_08_28-18_43_25_0.db3
Bag size:          460.8 KiB
Storage id:        sqlite3
Duration:          10.69s
Start:             Aug 28 2021 18:43:27.995 (1630190607.995)
End:               Aug 28 2021 18:43:38.64 (1630190618.64)
Messages:          91
Topic information: Topic: /scan | Type: sensor_msgs/msg/LaserScan | Count: 91 | Serialization Format: cdr

$ ros2 bag play rosbag2_2021_08_28-18_43_25/
[INFO] [1630252505.294131771] [rosbag2_storage]: Opened database 'rosbag2_2021_08_28-18_43_25//rosbag2_2021_08_28-18_43_25_0.db3' for READ_ONLY.

----- And in second shell ----
$ ros2 topic echo --csv --full-length /scan > scan.csv


That method strips off the field identifiers.  To keep the field identifiers:
$ ros2 topic echo --full-length /scan > scan.txt

In ROS1 as I understand it:

rostopic echo -b bagname.bag -p /topicname > filename.csv

(Have to look at the help for the command to see if there are params comparable to --full-length and --csv)

1 Like

Me: (Totally confused why anyone would put up with all the ROS bull-poop.)

Video:
https://github.com/jsk-ros-pkg/jsk_3rdparty/blob/master/dialogflow_task_executive/img/pr2_demo.gif

Me: Ahh! Now I understand!

Credits:
Clip taken from the posting at

1 Like

Did that work? I’d worry about whether the commas would be in the right places

/K

2 Likes

I think that was one of the first robots to use ROS.
/K

2 Likes

And I think they have/had a much bigger budget than we’ll ever hope to see!
:wink:

BTW, did it go “ZIP!” when it moved, “BOP!” when it stopped, and “Whirrrr!” when it stood still? There wasn’t any audio.

If it didn’t, it’s just a punter wannabe. :stuck_out_tongue_winking_eye:

1 Like

That was in California. I’m pretty sure its repertoire was limited to various forms of “Duuude” :slight_smile:
/K

2 Likes

Looking at the /scan topic repeatedly the header is consistent to the last digit repeatedly (except for the time stamp of course):

header: 
  seq: 1241
  stamp: 
    secs: 1630276516
    nsecs: 385975000
  frame_id: "base_scan"
angle_min: -3.14159274101
angle_max: 3.14159274101
angle_increment: 0.00872664619237
time_increment: 0.000111111112346
scan_time: 0.142857149243
range_min: 0.10000000149
range_max: 16.0

So dead on 1/7

I’m wondering if the discrepancy between the scan time and the rate of spin is contributing to all of the null values? I haven’t had much free time so I haven’t been able to investigate as much as I’d like. But I’ll keep plugging along. I need to think through a structured plan to changing the parameters and seeing what happens.

/K

2 Likes

No idea about the ROS1 version, but both ROS2 commands worked perfectly. The CSV version is the easiest to import.

1 Like

I’ll have to try that. I was afraid everything would be crammed into a few cells
/K

1 Like

Just my opinion, Keith, but this is what I suggest:

  1. Download the current ROS1 ydlidar node or
    current ROS1 ydlidar? <<-- perhaps this one?

  2. Make a new launch file with the parms (or use the manufac’s launch.x4 file) :

    <node name="ydlidar_node" pkg="ydlidar" type="ydlidar_node" output="screen" respawn="false" >
    <param name="port"         type="string" value="/dev/ydlidar"/>  
    <param name="baudrate"         type="int" value="128000"/>  
    <param name="frame_id"     type="string" value="laser_frame"/>
    <param name="resolution_fixed"    type="bool"   value="true"/>
    <param name="auto_reconnect"    type="bool"   value="true"/>
    <param name="reversion"    type="bool"   value="false"/>
    <param name="angle_min"    type="double" value="-180" />
    <param name="angle_max"    type="double" value="180" />
    <param name="range_min"    type="double" value="0.1" />
    <param name="range_max"    type="double" value="12.0" />
    <param name="ignore_array" type="string" value="" />
    <param name="frequency"    type="double" value="8"/>
    <param name="samp_rate"    type="int"    value="5"/>
    </node>

and 3) make a URDF that matches the ydlidar node with frame_id=“laser_frame”:

  <!-- LIDAR -->
  <link name="laser_frame">
    <visual>
      <origin xyz="0 0 0" rpy="0 0 0" />
...
    </visual>
  </link>
  
  <joint name="joint_ydlidar" type="fixed">
    <parent link="base_link"/>
    <child link="laser_frame"/>
    <origin ... /> 
  </joint>

 

  1. Rerun your “rosrun tf2_tools view_frames.py” to be sure everything is linked up correct
  2. Redo your “Rviz with AMCL Nav” to see if the localization is still twitchy
  3. Restart your “preparation for nav tests” now that you have changed:
    • base_link xy is now at the center of rotation between the wheels
    • LIDAR is using the parameters listed by the manufacturer
    • ydlidar node is again the manufacturer’s version
    • URDF uses the same frame_id as the ydlidar node publishes in the joint definition (and link def)

I didn’t look very deep into the ydlidar pull-request - perhaps that is still needed. I just think it is good to let the node set those two values in the header for this round of testing, and then if the original AMCL Nav problem shows up, make the change.

2 Likes

Really good thought. Busy week, so it may be the weekend before I get to try this.
/K

2 Likes

Looking at the two, I see that the last ever commit to ydlidar_ros (the second one) was March 30, 2020, and the initial commit to ydlidar_ros_driver (the first link) was April 7, 2020 (with the last commit a week ago). Some of the commits at least mention the X4. So I think I’ll try the ydlidar_ros_driver.

As I look at this more I realized that the nominal sample rate is 5000 Hz, but mine reports 4K when it starts up. The scan frequency on Finmark is 12 (based on my /scan topic publication rate). This seems to be set by voltage, not the parameter setting (which is set at 7). 5K gives only 417 scans per revolution, 4K gives 333. So there can’t be a measurement at every half degree. Maybe that’s why there are so many zeros in every /scan message???

Doing the math, to get 720 scans per revolution, you’d want the lidar spinning at ~7Hz (so I assume that’s how the book files landed on 7??). But I don’t think I can actually adjust that (there is a comment in one of the commits I looked at to the effect that for the X4 this wasn’t actually controlled by the launch parameter). Ugh - more to look into. And if the navigation stack is just ignoring the zeroes anyway, perhaps it doesn’t matter.

/K

2 Likes

Well, it was worth a try.

Got the ydlidar_ros_driver compiled (had to first download and install the SDK as the GitHub page suggested I might. Didn’t actually have to change my URDF - I just used base_scan instead of laser_frame as the frame_id in the launch file. Figured I’d try that first, and everything worked. Unfortunately there didn’t seem to be any improvement in the performance when I did the 360 rotation test. I think I’ll just stick with the driver I’ve been using, although I did update the baud rate and a couple of other parameter values based on what they had on the ydlidar git hub luanch file.

Next steps for me are to go back to that trouble-shooting guide and work through some of the other suggestions. Also want to get the IMU running to see if that helps at all.

/K

Addendum:
Was curious about what @cyclicalobsessive had pointed out, that I had made changes to my ydlidar driver

I looked at the new driver I had compiled - it doesn’t have those changes. It has what the original driver had before I changed it:

     scan_msg.scan_time = scan.config.scan_time;
     scan_msg.time_increment = scan.config.time_increment;

However I can’t find where the scan.config values actually come from.

2 Likes

I realize that I stated it wrong - the 0 index of the array with all of the values is straight back. Just clarifying here for posterity.
/K

2 Likes