Lookout AI-Based Collision Avoidance

Boston Harbor is notorious for serving chowder-thick fog when the temperature and dew point align. Such was the case late one night for Brian Moseley and Paul Sullivan, who were enveloped while cruising home aboard Pasithea, their 36-foot Jeanneau NC 1095 Coupe. Fortunately, Pasithea carries a Lookout AI-based collision-avoidance system and an infrared-enabled camera. Instead of staring at the darkened inside of a meteorological pingpong ball, the boaters could use Lookout to see augmented-reality screen views of cans, nuns and nearby shorelines.

Sullivan says the technology allowed them to move along, confident of the location of nearby aids to navigation, islands and the area’s ubiquitous lobster pots. Moseley adds: “I would have felt anxious at the helm without it.”

AI-based watchkeeping and collision-avoidance systems are one of the more exciting pieces of contemporary electronics. While Lookout doesn’t autonomously dodge detected targets or navigational hazards, it does combine AI, a camera, the automatic identification system and cartographic data to provide up to 360-degree situational awareness with collision-avoidance alerts.

Lookout—which has a lower price point than its competition—consists of a marinized black-box processor that connects to a camera, the vessel’s NMEA 2000 data backbone, a 12/24-volt DC power supply and the yacht’s multifunction display.

Owners have a choice of two processors. The Brain ($5,000) is aimed at smaller power yachts and sailboats. It has an Nvidia graphics processing unit that tackles 10 frames per second from a forward-facing high-definition video stream (760 pixels), an augmented-reality processor, an N2K port (for accessing vessel heading and GPS/GNSS information, and for networking with N2K-enabled multifunction displays) and an AIS receiver. It draws 25 watts of continuous DC power with a 60-watt maximum draw.

The Brain Pro ($10,000) is aimed at larger yachts and go-fast rides. It sports similar capabilities, but its Nvidia GPU processes 30 frames per second from multiple full-HD 1080p video streams, providing 360-degree imagery with displayed vessel tracks and buoy annotations. The Brain Pro comes bundled in a larger black box than the Brain and draws 60 watts of continuous power with a 160-watt maximum draw.

Yacht owners can network the Lookout Brain processor with an existing IP-enabled camera—including FLIR thermal-imaging cameras—or spec the Lookout camera ($4,000). While the former can be a great option, the Lookout camera has a long-range HD video feed and includes a near-infrared sensor for nocturnal operations, plus a 360-degree camera for docking (including automotive-style guide lines) and all-around situational awareness.

David Rose, Lookout’s CEO, says that while the system’s hardware is solid, its sorcery resides in its software and implementation of a kind of AI called computer vision. Lookout incorporates three types of computer vision to perceive, identify and track potential threats. This starts with scene segmentation, where the system classifies every pixel in a scene (read: water and not water) and works to stabilize the horizon. A multi-object tracker then follows 100-plus targets within a scene, while a distance-estimation algorithm calculates the range to each target.

Collectively, these AI capabilities allow Lookout to present augmented-reality views on a head-up display on any networked MFD or IP-enabled device, including phones and tablets. It also can present a 3D synthetic view with a bird’s-eye view around the boat. In both cases, chart data is used to create the augmented imagery.

Unlike other AI-based threat-detection systems that consider range using the currency of distance, Rose says, Lookout considers human reaction time and focuses on the potential hazards that lurk in the next 30 seconds. “Radar can detect things miles away, but we focus on extending human perception,” he says. “Thirty seconds at 20 knots is three football fields. We want to be really good at detecting things 30 seconds out.”

Once a target has been detected, Lookout tracks it and uses computer vision to identify it using a deep-learning network that’s been trained and tuned specifically for boating. Rose says Lookout’s algorithm has been trained on hundreds of thousands of images and videos to enable fast, accurate identification. Every Lookout system, he says, captures examples of ambiguous targets and uploads them to Lookout’s cloud, so the system’s model is retrained and pushed out to all Lookout systems.

“We’re using clustering and self-supervised learning,” Rose says, explaining that this largely obviates the need for human labeling.

Lookout systems also use AIS data to detect and identify targets, which the system tracks. Rose says Lookout might someday use MARPA (mini automatic radar plotting aid) data for detecting and tracking distant targets.

On the user-interface side, Lookout provides three alert levels. All detected targets are marked with a white triangle on the augmented-reality view, while targets in the vessel’s path are marked with yellow triangles. Targets that present a collision hazard are marked with a pulsing, automotive-style hazard icon. “Most people like visual alarms more than audible ones,” Rose says.

If a Lookout system “sees” a log or any other navigational danger, it can track the target with a GPS pin and a virtual perimeter. If cloud connectivity exists, it can also share this information with Lookout’s community cloud, which is shared with other connected Lookout users. While Lookout doesn’t require full-time connectivity, cloud access enhances its capabilities and enables over-the-air updates.

In addition to collision avoidance, Lookout can create suggested safe routes using its camera to detect buoys and aids to navigation, and by interrogating the vessel’s vector cartography. Rose says Lookout has also incorporated “Hogwarts-style” graphics, such as hoops over channel entrances and banners that mark a boat owner’s home dock.

While Lookout has been tested aboard vessels ranging from small boats to a 460-foot superyacht, its target market is 30- to 65-footers. Lookout can be added as an aftermarket feature, but Rose says he hopes the technology will eventually be akin to automotive rearview backup cameras, which are standard issue on most new cars. “Tools like this can expand the market and democratize boating,” Rose says.

Lookout’s upsides are quite compelling, but some downsides exist as well. For example, users need to spec out the Brain Pro and the Lookout camera to take advantage of the system’s full capabilities. Also, Lookout can’t use radar-generated MARPA data or autonomously command the vessel’s autopilot to dodge danger.

But given the human tendency to tire during those long watchkeeping duties, there’s little question that Lookout could be a useful helm companion, especially if a chowder-thick fog bank arrives at the wrong time.  

Sensitivity Scale

Lookout allows boaters to adjust the system’s sensitivity level (think radar gain) on a scale of 1 to 5 based on personal preferences. Lower sensitivity equates to greater system confidence that the target exists. Higher sensitivity is likely to catch targets faster, albeit with a greater chance of false positives.

The post Lookout AI-Based Collision Avoidance appeared first on Yachting.