What are the most established tracking methods in Augmented Reality?

At a Glance

At Visometry, everything revolves around industrial Augmented Reality (AR), with tracking technologies being an integral part of it. With VisionLib, we offer our own tracking solution specifically designed for industrial Augmented Reality scenarios. Over the past few years, VisionLib has built a strong reputation. However, when it comes to tracking technologies, there is no universal solution that works equally well for every application. It is therefore worth taking a closer look at which technology is best suited to a particular use case. In this blog article, we compare established tracking solutions and highlight their strengths and weaknesses, particularly for industrial scenarios.

How does tracking work in industrial environments?

In industrial environments, reliable tracking – that is, determining the position and orientation of devices (or objects) in space – is a necessary prerequisite for effectively using Augmented Reality applications. There are many different methods, some of which are particularly interesting. We will take a closer look at the following: marker-based tracking, SLAM, and model tracking, which is also used in our AR solution VisionLib.

In Augmented Reality, digital content is overlaid appropriately onto the real environment, for example via a tablet. This makes it possible, for instance, to compare real and digital components in order to perform visual quality inspections.

During a quality inspection, stable tracking ensures that CAD (Computer-Aided Design) models can be precisely overlaid onto manufactured components without shifting and causing incorrect results. In industrial environments, however, maintaining stable tracking can be challenging due to changing lighting conditions or dynamic surroundings.

Reliable tracking is also required in industrial maintenance because objects must be positioned with pinpoint accuracy so that employees can service products correctly. Even the smallest deviation in an AR overlay can cause digital instructions to be displayed on the wrong components, which can have serious consequences in industrial service. These may include costly operating errors, damaged equipment, or safety risks.

So which tracking methods are best suited for stable tracking in industrial applications?

Today, many mobile AR systems use Visual-Inertial Odometry, which combines camera images with data from inertial sensors such as accelerometers or gyroscopes, to estimate the position and movement of a device in real time. This allows digital objects to remain stably anchored in the physical world without visibly drifting over time.

A central component of this process is computer vision tracking. It is responsible for object recognition and the visual understanding of the environment. Using this technology, images captured by cameras are processed and analyzed. The camera searches for visual features such as corners, edges or other distinctive points, which can be used as references for tracking. During this process, the environment or objects are captured by the camera.

Marker Tracking

So-called ArUco markers were particularly common in the early days of computer vision for AR applications because they were easy to detect and process. These markers are specifically encoded black-and-white patterns that are attached to spaces or objects. Due to their distinctive appearance, optical tracking methods can easily recognize them in the camera image. This, in turn, makes it easy to display digital content. As long as the markers remain fixed in place and are correctly defined, the virtual overlay stays stable and reproducible, which is a necessary requirement for reliable tracking.

One major challenge with fixed markers, however, is that they must be attached to every object that needs to be tracked, which can be very time- and resource-intensive. Also, there is the risk of incorrect marker placement, as this can cause the visual AR content to no longer match the real environment. Another disadvantage is that occlusions of the markers can reduce tracking reliability or interrupt tracking if they are no longer sufficiently visible. In addition, fixed markers are not infinitely scalable, meaning that they cannot be placed everywhere.

What do markers mean in practice?

• Longer preparation times because markers must be attached to the object or the environment.
• Higher error rates if markers on the object are incorrectly aligned with the 3D content.

Markerless tracking methods

In addition to marker-based tracking methods, there are also markerless tracking methods, one of which is SLAM.

The SLAM method

SLAM (Simultaneous Localization and Mapping) has become an established markerless method and is now used in many AR applications. The reason is that SLAM does not require a room or object to be prepared for optical recognition. Instead, feature points and reference points are collected from the environment. SLAM creates a map of the environment while simultaneously determining and tracking its own position within that environment. Virtual content can then be placed at any location.

Advantages of SLAM compared to other methods:

• No special preparation required.
• Virtual content can be displayed at arbitrary locations.
• Fast initialization.
• Does not depend on predefined points and instead relies solely on reference points in the environment.

Disadvantages of SLAM include:

• The system knows the position of the device relative to its starting point or the mapped environment, but it cannot automatically determine its absolute position within a technical coordinate system, such as that of a CAD model.
• The user must ensure that AR content is placed correctly and align it manually.
• Risk of drift, particularly when environmental conditions change.

SLAM is primarily used to track rooms and environments rather than individual objects. As a result, this technology is used less frequently in applications requiring high precision and is less commonly found in industrial applications such as the quality inspection mentioned above. Instead, SLAM is more commonly used in B2C applications intended for a broad audience. For these purposes, approximate object placement and a general understanding of the environment are usually sufficient. Examples include navigation applications, product “holograms,” or context-based visualizations. In these applications, an approximate placement of virtual content is sufficient.

Markerless Object Tracking

In most industrial use cases, the focus is on objects that are measured or inspected, for example. For this reason, our VisionLib solution uses a method called model tracking to track objects reliably and stably.

But how does model tracking work?

In model-based tracking, a CAD or 3D model of a real object is used as the tracking reference to locate and track that object in the camera image. The Augmented Reality visualization is then aligned with the physical object. This method has become popular because it works without markers and does not require manual preparation of the object or environment. It also enables highly accurate AR overlays.

Another advantage of model tracking is the automated registration, which is the process by which the tracking system recognizes the real world or physical objects without manual assistance and automatically and precisely aligns virtual content accordingly. This tracking method therefore provides the foundation for displaying virtual 3D content in real time. Model tracking is particularly beneficial in industrial environments because it remains robust under challenging conditions such as changing lighting or dynamic surroundings.

In general, the following advantages and disadvantages of model-based tracking should be considered:

Advantages of model-based tracking:

• No preparation: no modifications to the environment or physical object are required in order to overlay AR content.
• Simple content creation: the CAD model serves as the tracking reference used to locate and track the physical object.
• Reliable and accurate pose estimation: the object can be located in space using camera images.
• Structural and geometric information about the object being tracked is already known from the 3D model. As a result, the object does not need to be trained using other AI-like methods, which would typically require a significant amount of time.
• Suitable for industrial components with rich structural features, such as steel beams.

However, model-based tracking also has some disadvantages:

• Model data (digital twins) must be available.
• Rotationally, symmetric objects and objects with little structural detail are difficult to detect.

In summary, model-based tracking can be used for simple and reliable content creation, for example for instructions used during quality inspections or maintenance. The method works well for many objects, but not for all shapes and sizes.

Summary of tracking methods

After reviewing the different tracking methods, you now have an overview of their advantages and disadvantages. The most important characteristics of these tracking methods are summarized below:

Overview of tracking methods:

SLAM:

• Requires no special preparation.
• Starts quickly.
• Suitable for B2C applications.
• Not suitable for the precise placement of content.

Marker-Based methods:

• Suitable for precision applications.
• Reliable tracking in controlled environments .
• Attaching markers is very time- and resource-intensive.

Model Tracking:

• No modifications to the environment or physical object are required.
• Suitable for precise object registration and reliable placement of AR content.
• Limitations: rotationally symmetric objects and objects with little structural detail. Not ideal to track spaces or rooms with few distinctive features.

Conclusion

As this blog article shows, there are different tracking methods, each with its own strengths and weaknesses.

Lessons Learned from this blog article:

• Marker-based tracking is reliable but requires considerable preparation.
• SLAM initializes quickly but is often inaccurate, making it less suitable for precision applications.
• Model tracking can provide precise object-centric registration because the CAD or 3D model serves as the geometric tracking reference for the physical object, provided that the required model data is available.

In summary, different tracking methods can be used for a variety of purposes. In a future article, we will explore the impact of different tracking methods on industrial environments and show how they are used in these settings.