Understanding resolution, accuracy and repeatability is critical to specifying a position sensor. This paper defines these terms and discusses how they relate to overall system performance.

What are Resolution, Accuracy & Repeatability

Article from | Genesis Motion Solutions

Understanding resolution, accuracy and repeatability is critical to specifying a position sensor. This paper defines these terms and discusses how they relate to overall system performance. It is important to consider that precision is a system issue which can be dominated by mechanical errors: eccentricity, backlash, straightness and flatness. Eccentricity and gearbox backlash, the most significant contributors to mechanical errors in rotary applications, are reviewed. The important concept of error calibration is also discussed.



Resolution defines the smallest measurable distance and therefore the smallest move increment. The resolution of a rotary encoder expresses the number of units of measure within 360 degrees. Each unit of measure is referred to as a count. Absolute encoder resolution is often expressed in “bits” – a binary expression. A 16-bit encoder has 216 (65,536) counts per revolution (CPR).

Fig. 1 shows an encoder (A) with 360 CPR (1°) resolution and an encoder (B) with 90 CPR (4°) resolution. The numbers in each gray box indicate encoder count readings. The blue dot is the true position to be measured – 1.5°. Encoder A has a count reading of ‘1’ which means the position is between 1° and 2°. Encoder B reads ‘0’ which means the position is between 0° and 4°. Encoder A clearly gives more precise position information. (Note, the range covered by an encoder count is often referred to as a “bin”. 1.5° is in bin 1, the 1° to 2° bin).

TN004 Fig 1

Fig. 1

Now consider a move of one count – the smallest possible position change. The new position targets (2.5° for A and 5.5° for B) are depicted by the green dots in Fig. 2. Clearly it is not possible to make a move of 1° with encoder B as the position change is not measurable.

TN004 Fig 2

The yellow dot indicates the position at which the servo believes the move is complete i.e., when the reading changes by one count. It is clearly possible to get closer to the true target position with the higher resolution encoder. Note that the true position of the yellow dot depends on the latency (position capture, processing and communication delay) involved before the servo sees a position change.



Accuracy represents the deviation of the measured position from a calibrated standard. In Fig.3, Encoder A has perfect accuracy. Each count represents a range of 1° and the bins are identical. Encoder B, however, has different sized bins. Bin 0 produces a count of “0” when the position is between 0° and 1.1°. The blue dot shows the true position. This position is incorrectly reported by Encoder B with an error of 0.1°. If we assume this error is the largest error throughout a revolution, the encoder will have an accuracy rating of ± 0.1°.

TN004 Fig 3

Fig. 3


Accuracy and Resolution

If the true position in Fig. 3 had been 0.5°, then both encoders would have given the same reading. As resolution increases and the bins get smaller, it becomes more likely for the true position to fall into the wrong bin (if we assume the same accuracy of ± 0.1°). In general, high-resolution encoders have higher accuracy but it is important to understand that accuracy and resolution are very distinct parameters.



Repeatability is a measure of the variance in successive position measurements at a specific location. Repeatability is tied to resolution as a measurement can only be as repeatable as the smallest measurable distance. It is typically specified as one least significant bit (LSB) which in terms of angle is the nominal size of one bin. Sometimes repeatability is not specified as it is assumed to be the same as resolution.

Some encoder manufacturers specify repeatability as < 1 LSB. This, however, does not take into account a potential boundary condition as shown in Fig. 4. The blue dot depicts a position which could fall into either bin.

TN004 Fig 4

Fig. 4

It should be noted that a magnetic encoder may give a different reading depending on the direction of approach to the position to be measured. This is due to magnetic hysteresis and repeatability may be 2 LSBs or more.


Repeatability & Accuracy

Accuracy is critical to position control performance but lower accuracy measurements which can be calibrated may yield superior precision. Fig. 5 depicts several accuracy/repeatability scenarios using a target analogy. The high accuracy green shots are ideal but if the high repeatability blue cluster was calibrated effectively the end result would be greater precision than the higher accuracy yellow cluster.

TN004 Fig 4

Fig. 5


Resolution In-Depth

The goal of most encoders is to produce high fidelity sine and cosine signals. In the optical encoder shown in Fig. 6, the scale comprises multiple lines which alternate in reflectivity. The width of a reflective and non-reflective pair of adjacent lines defines the scale pitch. The optics and sensor geometry are designed in such a way that sine and cosine signals are produced as the scale rotates relative to the sensor.

TN004 Fig 6

Fig. 6


The ultimate objective of generating sin/cos signals is to perform interpolation which gives the angle within a scale pitch according to the formula:


Using this formula, it is theoretically possible to have infinite resolution. In practice, resolution is limited by digital to analog converter resolution (signals are digitized for processing) and signal to noise ratio (SNR). Fig. 7 shows two sinewave segments with the same amount of noise. The sinewave on the right can be digitized into more steps (higher resolution) as the signal is larger relative to the noise floor.

TN004 Fig 6

Fig. 7


High resolution is desirable for the highest performance position control. A servo dithers between two counts – the higher the resolution the smaller the dither. High resolution is also important for precise velocity control at low speeds due to “resolution noise”. Resolution noise is a variation in counts in the velocity loop sampling period. In Fig. 8, the velocity is constant (even spacing of counts) but Sample 1 sees three counts and Sample 2 sees four counts – a 33% increase in speed. This effect is significantly diminished at higher resolution.

TN004 Fig 8

Fig. 8

At high speeds, high resolution incremental encoders may generate data rates beyond the tracking capability of the controller or servo drive. Be sure to check the servo drive’s maximum input frequency.


Encoder Jitter

Encoder manufacturers often push the limits of resolution relative to SNR resulting in the least significant bits of position data bouncing around when there is no motion. This is sometimes tolerated because the jitter frequency is high relative to the bandwidth of many positioning systems. It is also possible to extract higher resolution from noisy position data by averaging multiple samples. This technique is often used in metrology but is generally not practical in servo systems as the averaging delay compromises phase margin.


Accuracy In-Depth

As discussed, accuracy represents the deviation of the measured position from a calibrated standard. Assuming a concentrically mounted scale, variation in pitch is the largest source of error. Gain, offset and phase variations between the sin/cos signals contribute to sub divisional error (SDE). There may also be harmonic distortion in the sin/cos signals which further add to SDE, Pitch errors can be calibrated out, but calibration is more difficult with high frequency SDE. Fig. 9 shows high frequency SDE superimposed on pitch error over one revolution.

TN004 Fig 9

Fig. 9


Eccentricity Error

Encoders which employ a readhead at a specific location on the scale generate a significant angular error, known as eccentricity error, if the scale is not centered precisely on the axis of rotation. Bearing runout also a contributes to eccentricity errors. Eccentricty error is repeatable (excluding non-repeatable bearing runout) and can be calibrated out.

TN004 Fig 10

Fig. 10

As discussed, angular error from eccentricity can dominate system accuracy. Angular error magnitude is given by the following formula.


Capacitive and inductive encoders measure position over 360° and are much more tolerant to eccentricity.



Calibration requires a high precision reference, typically an interferential optical encoder. Position data from the reference and the unit to be calibrated are captured over one revolution. Variations from the reference are stored in a lookup table of offsets. The more points captured, the more effective the calibration. Many points are required for high frequency errors.

Some encoders have calibration routines, the offset table being stored in the encoder memory. The calibration function may also be a feature of the drive or controller. In either case, the offset is added to the position reading. Interpolation is performed when the position reading falls between lookup table points. The process of adding offsets does add a delay which may affect servo loop performance.

TN004 Fig 11


Fig. 11

The Gearbox Effect

The use of a gearbox increases effective resolution. For example, the output of a 10:1 gearbox moves 0.1° when the input moves 1°. The smallest possible move is therefore reduced by a factor of ten.

Gearboxes are, however, subject to backlash (mechanical play) in the gear teeth. As shown in Fig.12., when the input gear changes direction the output gear is physically disconnected until the mechanical play is taken up. Having increased resolution in the backlash zone is of no value as the gear output does not move. Backlash compromises accuracy and calibration is complex and only partially effective. System accuracy and repeatability are typically reduced by 1-2 orders of magnitude.

TN004 Fig 12

Fig. 12


Resolution and repeatability are key parameters for high performance position control. It is important to realize that resolution and accuracy are distinct parameters. Encoder accuracy is also critical to system performance but should not be considered in isolation as mechanical errors may dominate. Repeatable errors can be calibrated out. Calibration does increase encoder latency and adds time to system commissioning.


The content & opinions in this article are the author’s and do not necessarily represent the views of RoboticsTomorrow
Genesis Motion Solutions

Genesis Motion Solutions

Genesis LiveDrive® housed and frameless direct drive rotary motors - engineered with patented electromagnetic technology for more torque to mass than competing direct drive motors, resulting in maximum productivity and efficiency for your machinery. Achieve superior torque to length, torque to mass, positioning accuracy and repeatability with power efficiency. Learn how the LDR torque motor, LDD and LDX housed and frameless direct drive motors can bring you speed, accuracy, and performance.

Other Articles

Machine Functional Safety – PL and SIL Ratings
This paper reviews the concept of Functional Safety as it relates to machinery. The design steps for a safe machine are outlined and the methodology for determining appropriate PL/SIL safety ratings discussed.
More about Genesis Motion Solutions

Comments (0)

This post does not have any comments. Be the first to leave a comment below.

Post A Comment

You must be logged in before you can post a comment. Login now.

Featured Product

Stäubli robotic tool changer solutions

Stäubli robotic tool changer solutions

Stäubli is a pioneer in the construction and development of robotic tool changing systems. Our customers benefit from our many years of expertise in all industry sectors, as well as our modular product concept, which offers three efficient solution paths: MPS COMPLETE offers preconfigured robotic tool changers for immediate use. MPS MODULAR allows the user to determine the configuration, while MPS CUSTOMIZED allows the construction of special, application-specific systems.