watonomous.github.io

[ Software Division : HERE HD Maps Technical Documentation ]

Created by [ Rowan Dempster], last modified by [ Henry Wang] on Jan 22, 2020

HERE HD Maps Data Spec

This section provides a summary of content from the HDLM Protobuf Data Spec.

Tiling Scheme (pg. 120)

HERE uses a custom tile identifier encoding scheme. It's interesting to see how the tile IDs are derived, but this content is not relevant.

Coordinate Encoding (pg. 127)

[Coordinate values for HERE HD Live Map are relative to the WGS84 GCS, and expressed as decimal degrees with 7 digits of precision after the decimal point. ][To minimize storage requirements in its Protobuf representation, the coordinate values are encoded. ][Rather than using floating point values, latitude and longitudes are represented as 31 and 32-bit signed integers, respectively. We apportion the -180 to 180 degree longitude span over the available 2\^][32 ][range of discrete values. For latitudes in the -90 to 90 degree range, the range is 2\^][31 ][integer values. ]

[This results in a resolution of 360° / 2\^][32 ][= 180° / 2\^][31 ][= 0.00000008381903171539306640625 degrees.]

[The resulting lat/lon integer values ][are then encoded as Morton Codes, interleaving their bits into a single 64-bit signed integer:]

[First bit is always 0.]
[2][nd ][bit is MSB of longitude]
[3][rd ][bit is MSB of latitude]
[Remaining 61 bits alternate between bits from longitude and latitude.]

Coordinate Array Offset Encoding (pg. 128)

[To optimize for Protobuf space, sequences of coordinates (e.g. polylines representing lane and road geometry) are also encoded. ][The incremental offsets of these sequences of values are stored and calculated with a bitwise XOR against a reference value.]

[For sequences of 2D coordinates, the first lat/long value in the sequence is offset-encoded against the parent tile's center point, which is included in the top-level message for the tile. Subsequent values are offset-encoded from the previous value in the sequence. ]

Road Centerline Model (pg. 17)

Link Topology Geometry Model (pg. 17)

Lane Model (pg. 45)

Lane Topology and Geometry Model (pg. 45)

Lane Attributes Model (pg. 52)

Format Specification

The following sections specify the message composition of specific layers, and message definitions.

Link Topology Geometry (pg. 68)

Lane Topology Layer (pg. 101)

[The lane-topology layer contains the lane group, lane group connector, lane and lane connector topology of the lane model. Sadly this layer is not provided for the competition, but it's possible to work around this shortage as lane connection info can be inferred from other layers' data.]

Lane Polyline Geometry Layer (pg. 105)

Lane Attributes Format (pg. 111)

Decoding Protobuf Files

HERE maps are stored in the Protobuf format that can be decoded using protoc. The MCity HERE HD Map is available here on the Shared Drive. The Protobuf files can be decoded using protoc.

protoc -I./ --decode=com.here.pb.hdmap.external.v1.[directory_name].[tile_message_type] com/here/pb/hdmap/external/v1/[directory_name]/[filename.proto] < [input-filename] > [output-filename].txt

For example, to decode Lane Model: Lane Topology Layer:

cd hdlm-dump/topology-geometry
protoc -I./ --decode=com.here.pb.hdmap.external.v1.topology.TopologyLayerTile com/here/pb/hdmap/external/v1/topology/layer-topology-geometry.proto < 321618295 > topology-geometry-321618295.txt

To decode the Lane Model: Lane Polyline Geometry Layer:

cd hdlm-dump/lane-geometry-polyline
protoc -I./ --decode=com.here.pb.hdmap.external.v1.lanes.LaneGeometryPolylineLayerTile com/here/pb/hdmap/external/v1/lanes/layer-lane-geometry-polyline.proto < 321618295 > lane-geometry-polyline-321618295.txt

The decoded data has been uploaded to localization repo. See https://git.uwaterloo.ca/WATonomous/localization/tree/develop/hdmap_processing/data/hdlm-dump-mcity

Documentation From Predecessors Below

Summary notes from HERE HD Map video presentation and Rules FAQs

Below is a summary of some notes from the HERE HD Map presentation linked above. I would still recommend watching it as it is quite informative

In year 2 MCity map, the only thing that would be accurate is the traffic lights and lane lines
Traffic light information is stored in the FAD_Mcity_12-6-18.fad.json document. This document also has the HDLM lane group the light is associated with. The relevant document would be MCity FAD Publication Features and Format V0.1.docx These files can be found in MCity.zip
3d occupancy grid in the roadside abstractions are not given.
- We will not get a pointcloud file https://www.autodrivechallenge.com/cdsweb/rqa/ViewFAQ.aspx?faqid=fcf6c42f-a355-415c-85bd-bc7d433a56da
- For us, the data above is just noise as our GPS is the most accurate source of localization anyone can have
Simon had asked about the GeoJson renderers part of MCity.zip. Looks like they were junk.

lane lines at intersections are "lane boundaries without lane markings" (inferred lane lines) are in the HERE map of MCity

Notes from FAQ

All lane lines in Year 2 (MCity) were in the HERE Map database
October 15 2019: The location of the traffic lights will be published in the map data and will not change. The locations of the stop signs may change between days.
October15 2019: Stop line and cross walk positions will be in the map. Scoring is based on what is in the ground, not on the position of the stop line / cross walk that is in the map
October 15 2019: The map will tell you where all the lane lines should be, but may not be perfect. Spirit of the competition is to steer to be based on vision, not on map data. HD Maps should be safety net, to confirm what lane the vehicle is in, and for navigation when there may not be any lane lines (i.e. intersections, cul-de-sacs)
October 15 2019: The map will never be intentionally wrong (i.e. indicating that a traffic signal / cross walk exists when it does not or vice versa). The maps will have a margin of error on position accuracy (i.e. could be ~1 meter off).

When reading this section, also refer to the parsing the HERE HDLM document and notes from Guy Stoppi and Shigeki on what the files do

We use the lane model. Refer to "lane parse.py" in the "hdmap processing/scripts" folder of the localization repository. The lane file consists of a series of lane geometries, all defined by HERE's method of storing a polyline (i.e. a line defined by an array of points). Each of these lane lines makes up either an actual lane line on the road or an artificial lane line in an intersection that describes the ideal trajectory of the car when "using" that lane.

Essentially, each lat-long coordinate is stored in a single number. To convert from that number X you do the following:

B = to_binary(X)
# every other bit starting with the second bit
lon = to_decimal(B[1::2]) * (360.0 / 2^32)
# every other bit starting with the third bit
lat = to_decimal(B[2::2]) * (180.0 / 2^31)

Now, when storing polylines, HERE map stores every point as an offset from the previous point using the XOR operation. So to decode:

prevB = ... # binary value of previous coordinate
B = XOR(prevB, to_binary(X))
lon = to_decimal(B[1::2]) * (360.0 / 2^32)
lat = to_decimal(B[2::2]) * (180.0 / 2^31)

Using these decode methods, we can generate a series of lane lines which can be used by our nodes running
on the car. Note that these lane lines must be separated into "intersection lines" (artificial lane lines in an
intersection) and "road lines" (actual lane lines) because of lane publisher.

*We can use only data on AutoDrive portal. We don't have login pass for official HERE site, like below image*

Why we developed a visualizer for HDmap

We will need autonomous driving navigation system for 3rd year competition. That's system shows HDmap on screen in Bolty, and if we specify goal point on HDmap, Bolty go to that goal point autonomously. That's system need visualizer for HDmap, also developing visualizer is good for understanding of HDmap.

Road model visualizer

Road model have nodes and links(refer to document p.17 to p.44). All links have information about start node and end node. In road model visualizer, I firstly installed all of nodes on Turtle's screen, next I connected these nodes referring to start node and end node information on link field. road model visualizer is included in test_all.py

Lane model visualizer

Lane model have information about lane boundary, that is consists of point cloud. please refer to document p.44 - p.60. We need lane boundary information because we need width of road(it means distance between left lane and right lane). I visualize these lane boundary point as green in test_all.py.