ADC Resolution vs Accuracy: ENOB, INL and the Reference • The Science 360

Plug an Arduino Uno into a laptop, feed analog pin A0 from a precision 2.500 V reference, and print analogRead(A0) * 5.0 / 1023. The ADC has a step size of 4.9 mV, so it feels as if the answer should be within a few millivolts. On a laptop port that happens to sit at 4.80 V, the code comes back around 533 and the sketch prints 2.605 V. That is 105 mV out, more than twenty steps. Nothing is broken. The converter did exactly what its datasheet says; the reading is wrong because of a number that is not in the ADC section at all.

The pages that rank for "ADC resolution vs accuracy" define the two words correctly and then stop at a ruler analogy. What they rarely do is put numbers on a real board, show which error dominates, or explain why averaging a thousand readings will not fix it. That is what this post does.

Quick answerResolution is the step size:

V_{\mathrm{ref}}/2^N

, 4.9 mV for a 10-bit ADC at 5 V. Accuracy is how close the code is to the truth, set by offset, gain, nonlinearity, noise and the reference.

\text{ENOB} = (\text{SINAD} - 1.76)/6.02

counts the useful bits, and a 5% reference error leaves about 4 bits of absolute accuracy.

Infographic: Is a 10-bit ADC 10 bits accurate? An ADC chip takes its reference from a USB plug and reads a true 2.500 V as 2.632 V. On a 4.75 V USB port the 10-bit converter gives only about 4.2 bits of absolute accuracy: 132 mV, or 27 LSB, of error from a part rated at 2 LSB. A log-scale bar chart of the error budget shows quantization at ±2.4 mV and the ADC's own error at ±9.8 to ±17.1 mV, dwarfed by the USB reference at +132 mV and a sagging cable at +374 mV. Averaging cannot remove this fixed error. — Tap the image to view it full size. View the accessible, editable SVG.

What is the difference between resolution and accuracy?

Resolution is a counting statement. A 10-bit converter has $2^{10} = 1{,}024$ output codes, so it chops its input range into 1,024 slices. With a 5 V reference, one slice, called one LSB (least significant bit), is $5/1{,}024 = 4.88\ \mathrm{mV}$ . Resolution tells you the smallest change the converter could report.

Accuracy is a truth statement: given an input, how far is the reported value from the real one? Figure 1 shows the difference on a toy 3-bit converter. Both staircases have eight codes, so their resolution is identical. The orange one has its transitions in the wrong places, so for an input of 0.68 V_ref it reports 110 where the ideal converter reports 101. Same resolution, different accuracy.

Output code versus input voltage for a 3-bit ADC. A blue ideal staircase steps up every one-eighth of the reference. An orange dashed real staircase has the same eight codes but its transitions are shifted early and unevenly. At an input of 0.68 times the reference the ideal converter outputs 101 and the real one 110. — Figure 1. Resolution is the number of steps. Accuracy is where the steps are. The real converter (orange) has the same 8 codes as the ideal one (blue) but offset, gain error and a bowed nonlinearity move its transitions, so the same input gives a different code.

A perfect converter still has an error, because it has to round. That quantization error is at most ±0.5 LSB and, for a busy signal, is spread evenly across that range. Its rms value is a standard result:

Quantization error is uniform over $\pm q/2$ , so its mean square is $q^2/12$ .

\begin{aligned}\text{rms quantization error} &= q/\sqrt{12} \\ &= 0.289\ \text{LSB}\end{aligned}

For 10 bits at 5 V:

0.289 \times 4.88\ \mathrm{mV} = \mathbf{1.41\ mV\ rms}

worst case ±2.44 mV

Everything else on the list below adds to that floor, and it usually adds a lot.

Which errors make an ADC inaccurate?

Datasheets split the deviation of a real converter into named pieces. The Microchip ATmega328P datasheet defines them in the same order you would remove them in a calibration:

Offset error: the first transition (code 0 to 1) happens at the wrong input. The whole staircase is slid sideways.
Gain error: after removing offset, the last transition is in the wrong place. The staircase is stretched or squeezed.
INL (integral nonlinearity): after removing offset and gain, the largest distance between an actual transition and the straight line. It describes a bowed or wiggly staircase.
DNL (differential nonlinearity): the largest deviation of any single step width from the ideal 1 LSB. A DNL of −1 LSB means a step of zero width, a missing code.
Absolute accuracy or TUE (total unadjusted error): the worst-case combination of everything, including quantization.

Here is what the ATmega328P datasheet (DS40002061B, the commercial part on an Uno) gives, at $V_{\mathrm{ref}} = 4\ \mathrm{V}$ and a 200 kHz ADC clock:

Absolute accuracy (incl. INL, DNL, quantization, gain, offset): 2 LSB typical

INL: 0.5 LSB typical DNL: 0.25 LSB typical

Gain error: 2 LSB typical Offset error: 2 LSB typical

Internal 1.1 V reference: 1.0 min, 1.1 typical, 1.2 max

Notice that the commercial table lists only typical values. The automotive version of the same chip (document 7810D, −40 to +125 °C) does give limits: absolute accuracy 2.2 LSB typical and 3.5 LSB maximum, gain and offset each within ±3.5 LSB, INL 1.5 LSB maximum. Those are a reasonable guess at what the commercial part guarantees, but it is a guess.

Separately, the Arduino core sets the ADC prescaler to 128 on a 16 MHz board, giving a 125 kHz ADC clock, comfortably inside the 50 to 200 kHz window the datasheet asks for full 10-bit performance. At 13 clocks per conversion that is at most $125{,}000/13 = 9{,}615$ conversions per second.

Why the reference voltage dominates

An ADC does not measure volts. It measures the ratio $V_{\mathrm{in}}/V_{\mathrm{ref}}$ and reports that ratio as a fraction of 1,024. If V_ref is 2% high, every reading is 2% low, and no amount of INL or DNL improvement can fix it. That makes the reference the first thing to look at, and on an Arduino Uno the default reference is the one you would least choose.

In the Arduino AVR core, DEFAULT sets the REFS bits to 01, which the datasheet defines as AV_CC, the chip's own 5 V supply. On USB power, that supply is whatever the host port delivers, minus small drops on the board. TI's reference design TIDUF26 summarises the USB 2.0 rule: a 5% tolerance on 5 V, "giving a host voltage range of 4.75 V to 5.25 V", and worst-case cable and connector drops can bring the load down to 4.35 V.

Worked example: reading 2.500 V on a USB-powered Uno. The sketch assumes $V_{\mathrm{ref}} = 5.000\ \mathrm{V}$ . The true reference is somewhere else.

\begin{aligned}\text{code} &= 1024 \times V_{\mathrm{in}}/V_{\mathrm{ref,true}} \\ \text{reported} &= \text{code} \times 5.000/1024 \\ &= 2.500 \times 5.000/V_{\mathrm{ref,true}}\end{aligned}

$V_{\mathrm{ref}} = 4.75\ \mathrm{V}$ :

\begin{aligned}\text{reported} &= 2.500 \times 5/4.75 = 2.632\ \mathrm{V} \\ \text{error} &= \mathbf{+132\ mV\ (+5.3\%)}\end{aligned}

$V_{\mathrm{ref}} = 5.25\ \mathrm{V}$ :

\begin{aligned}\text{reported} &= 2.500 \times 5/5.25 = 2.381\ \mathrm{V} \\ \text{error} &= \mathbf{-119\ mV\ (-4.8\%)}\end{aligned}

$V_{\mathrm{ref}} = 4.35\ \mathrm{V}$ (sagging cable):

\begin{aligned}\text{reported} &= 2.874\ \mathrm{V} \\ \text{error} &= \mathbf{+374\ mV\ (+15\%)}\end{aligned}

In LSB:

132\ \mathrm{mV}/4.88\ \mathrm{mV} = \mathbf{27\ LSB}

against a converter rated at 2 LSB.

Figure 2 stacks these up against the converter's own errors on a log scale. The ADC's datasheet errors are a few millivolts. The reference error is fifty times bigger.

Horizontal bar chart on a log scale of worst-case error in millivolts when an Arduino Uno measures 2.5 volts with its default 5 volt reference. Quantization and INL are 2.4 millivolts each, typical absolute accuracy 9.8 millivolts, automotive maximum absolute accuracy 17.1 millivolts, USB supply tolerance plus 132 or minus 119 millivolts, and a sagging 4.35 volt supply plus 374 millivolts. — Figure 2. Error budget for an absolute voltage reading. The converter's own errors (blue and purple) are dwarfed by the reference (orange). Our arithmetic from the ATmega328P datasheets and the USB 2.0 supply range.

Convert that to "bits of accuracy" and the headline result appears:

\begin{aligned}\text{fractional error} &\approx 132\ \mathrm{mV}/2.5\ \mathrm{V} \\ &= 5.3\% \\ &\ \ \text{about 1 part in } 19 \\ \log_2(19) &= \mathbf{4.2\ bits}\end{aligned}

of absolute accuracy from a 10-bit converter. Typical laptop port at 4.95 V: error 1%,

\log_2(100) = 6.6\ \text{bits}

The internal 1.1 V reference is not a cure on its own. It is far steadier than a USB supply, but the datasheet allows 1.0 to 1.2 V from chip to chip, ±9%, which is worse than the USB tolerance. What the bandgap gives you is stability: measure its actual value once on your board (for example by reading a known voltage) and store a correction factor, and the large fixed error disappears. A dedicated reference IC on AREF is the cleaner fix if you need absolute accuracy.

Source note I did not open the Arduino Uno schematic for this post, so "the 5 V rail is the USB supply minus small drops" is a simplification, and the 4.80 V laptop port in the opening is an illustrative value, not a measurement. The ATmega328P commercial datasheet gives no temperature coefficient for the 1.1 V bandgap, so I cannot say how much a once-calibrated value drifts with temperature.

When reference error cancels: ratiometric measurements

There is one important case where none of this matters. If the sensor is powered from the same rail as the ADC reference, its output scales with that rail. A potentiometer across 5 V, a resistive divider, a thermistor in a divider (see our NTC thermistor calculation) or a load-cell bridge all produce $V_{\mathrm{in}} = k \times V_{\mathrm{supply}}$ :

\begin{aligned}\text{code} &= 1024 \times V_{\mathrm{in}}/V_{\mathrm{ref}} \\ &= 1024 \times k \times V_{\mathrm{supply}}/V_{\mathrm{supply}} \\ &= \mathbf{1024 \times k}\end{aligned}

A pot at 40% reads code 410 whether USB delivers 4.75 V or 5.25 V.

This is why hobby projects with pots and thermistors mostly work fine on the default reference, and why the same project reads nonsense when someone uses it to measure a battery voltage. Ask first whether your measurement is ratiometric or absolute. If it is absolute, the reference is your accuracy. The same logic is behind the bridge excitation in our post on how a load cell works.

What is ENOB, and how do you calculate it?

ENOB, the effective number of bits, answers a different question: if this converter were ideal, how many bits would it need to have to produce the same amount of noise and distortion? It is measured dynamically, by feeding in a clean, near full-scale sine wave and taking an FFT of the output.

The starting point is the signal-to-noise ratio of an ideal converter. TI's application report SLAA013, "Understanding Data Converters", derives it from the $q^2/12$ quantization noise above and a full-scale sine of amplitude $A = 2^{N-1}q$ :

\begin{aligned}\text{signal power} &= A^2/2 = (2^{N-1}q)^2/2 \\ \text{noise power} &= q^2/12 \\ \text{SNR} &= 10\log_{10}(1.5 \times 2^{2N}) \\ &= \mathbf{6.02N + 1.76\ dB}\end{aligned}

\begin{aligned}10\ \text{bits}&:\ 61.96\ \mathrm{dB} \\ 12\ \text{bits}&:\ 74.0\ \mathrm{dB} \\ 16\ \text{bits}&:\ 98.1\ \mathrm{dB}\end{aligned}

Turn it around with the measured SINAD (signal to noise and distortion):

\mathbf{ENOB = (SINAD - 1.76)/6.02}

Example: a 10-bit ADC measuring $\text{SINAD} = 56.0\ \mathrm{dB}$ :

\text{ENOB} = 54.24/6.02 = \mathbf{9.0\ bits}

ENOB is useful because it folds noise, DNL and the harmonic distortion caused by INL into one number. It is not an accuracy figure. A converter with a 5% gain error and a superb SINAD has an excellent ENOB, because gain error only rescales the sine; it does not add noise or distortion. ENOB tells you how finely the converter can see changes. It says nothing about whether the absolute number is right.

Noise alone eats bits in a predictable way. If the input-referred noise is σ LSB rms, it adds to the $q^2/12$ quantization noise:

\text{ENOB} = N - \tfrac12\log_2(1 + 12\sigma^2)

\begin{aligned}\sigma = 0.5\ \text{LSB}&:\ \tfrac12\log_2(1 + 3) \\ &= \mathbf{1.0\ bit\ lost} \\ \sigma = 1\ \text{LSB}&:\ \tfrac12\log_2(13) \\ &= \mathbf{1.85\ bits\ lost} \\ \sigma = 4\ \text{LSB}&:\ \tfrac12\log_2(193) \\ &= 3.8\ \text{bits lost}\end{aligned}

Plot of bits lost versus input-referred rms noise in LSB on a log axis from 0.05 to 8 LSB. The curve is near zero below 0.1 LSB, reaches 0.5 bit at 0.29 LSB, 1 bit at 0.5 LSB, 1.85 bits at 1 LSB, 2.81 bits at 2 LSB and 3.8 bits at 4 LSB, then rises by about one bit per doubling of noise. — Figure 3. ENOB lost to white noise for an otherwise ideal converter (our calculation). Half an LSB of rms noise costs a full bit; above 1 LSB, each doubling of noise costs another bit.

That curve is why a 24-bit delta-sigma part might be specified at 20 effective bits, and why careful datasheets quote "effective resolution" from rms noise. TI's ADS1115 datasheet does exactly that, using $\text{effective resolution} = \log_2(\text{FSR}/V_{\text{rms noise}})$ , and lists all ranges at the full 16 bits for data rates up to 128 SPS, falling to between 13.8 and 15.1 noise-free bits (from peak-to-peak noise) at 860 SPS.

Source note Microchip publishes no ENOB or SINAD figure for the ATmega328P's ADC, and I could not find a measurement of an Arduino Uno with a documented method (clean source, coherent sampling, stated input frequency). Forum figures exist but I am not quoting them. Expect real ENOB to depend heavily on board layout, the 5 V rail's noise and on use of the ADC noise reduction sleep mode.

Is a 16-bit ADC more accurate than a 10-bit one?

The ADS1115 is a good test case, because it is the usual upgrade from an Arduino's built-in ADC. Its LSB on the ±2.048 V range is $4.096\ \mathrm{V}/65{,}536 = 62.5\ \mu\mathrm{V}$ , eighty times finer than the Uno's. The DC accuracy specifications tell a different story:

Gain error (FSR ±2.048 V, 25 °C): 0.01% typical, 0.15% maximum

At 2.000 V input:

\begin{aligned}0.15\% \times 2.000\ \mathrm{V} &= 3.0\ \mathrm{mV} \\ &= 3.0\ \mathrm{mV}/62.5\ \mu\mathrm{V} \\ &= \mathbf{48\ LSB}\end{aligned}

Offset: ±1 LSB typical, ±3 LSB maximum (differential); gain drift up to 40 ppm/°C

Over a 30 °C swing:

30 \times 40\ \mathrm{ppm} = 0.12\%

extra, another 38 LSB.

Absolute accuracy at the limits:

\begin{aligned}\text{accuracy} &\approx 1\ \text{part in } 1/0.0015 \\ &= 667 \\ \log_2(667) &\approx \mathbf{9.4\ bits}\end{aligned}

So at worst-case limits, a 16-bit ADC with an internal reference is a 9-to-10-bit absolute voltmeter before calibration. Typical parts are much better (0.01% typical gain error is about 13 bits), and a two-point calibration against a known voltage removes most of the gain and offset error for that unit at that temperature. What the extra bits really buy you is resolution and low noise: the ability to see a 100 µV change, which is exactly what a bridge sensor needs. That is a real advantage, it just is not the same thing as accuracy.

Does averaging improve ADC accuracy?

Averaging is the most common response to a jumpy reading, and it works, within limits. Microchip's (Atmel's) application note AVR121, "Enhancing ADC resolution by oversampling", gives the standard rules: each extra bit needs 4× as many samples, so n extra bits need $4^n$ samples, and the trick only works if there is noise of at least about 1 LSB to make the last bit toggle. Without noise, every sample returns the same code and the average is just that code.

Random noise of the average:

\sigma/\sqrt{N}

\begin{aligned}N = 16&:\ \text{noise} \div 4 \\ &\ \ \text{two more bits} \to 12\text{-bit result} \\ N = 256&:\ \text{noise} \div 16 \\ &\ \ \text{four more bits} \to 14\text{-bit result}\end{aligned}

Uno at 9,615 conversions/s: 12-bit results at 601/s, 14-bit at 37.6/s (fewer with analogRead overhead)

Figure 4 shows a Monte Carlo simulation of this. With no noise at all (orange), averaging does nothing: the error stays at the 0.29 LSB rms quantization floor no matter how many readings you take. With 0.5 or 1 LSB of noise (blue and green), the error falls on a clean $1/\sqrt{N}$ line, reaching about 0.01 LSB at 4,096 samples. The purple curve is the crucial one: add a 0.5 LSB fixed error, from INL, offset or the reference, and the average converges beautifully to the wrong answer.

Log-log plot of the rms error of an average of N readings of a simulated 10-bit ADC, N from 1 to 4096. With no noise the error is flat at 0.29 LSB. With 0.5 and 1 LSB rms noise the error falls along a one over root N line to about 0.01 LSB. With 1 LSB noise plus a 0.5 LSB fixed error the curve flattens at 0.5 LSB. — Figure 4. Our Monte Carlo simulation (3,000 random inputs per point, ideal 10-bit quantizer). Averaging removes random error at $1/\sqrt{N}$ , needs noise to work at all, and cannot touch a fixed error.

AVR121 says it plainly: oversampling and decimation "will only lower the ADCs quantization error, it does not compensate for the ADCs integral non-linearity." The same is true of offset, gain and reference error, all of which repeat identically in every sample. Averaging buys resolution. Only calibration, or a better reference, buys accuracy.

Two practical notes. First, 1 LSB of noise is often already there on an Arduino, from the supply and the digital core, so averaging usually helps without deliberately added dither. Second, averaging is a low-pass filter. Averaging 256 samples at 9.6 kSPS is a 26.6 ms window, which nulls 37.5 Hz and its multiples; if you want it to reject mains hum, choose a window that is a whole number of mains cycles (20 ms at 50 Hz, 16.7 ms at 60 Hz). And any signal above half the effective sample rate aliases, as explained in our post on the Nyquist theorem and aliasing.

How to get real accuracy out of a microcontroller ADC

Ordered by how much they usually buy you:

Decide if the measurement is ratiometric. If it is, power the sensor from the ADC's reference and most of the problem disappears.
Fix the reference. Use a precision reference IC on AREF, or calibrate the internal bandgap once per board. On USB power, never trust the 5 V rail as a reference for absolute readings, especially through a long cable or a USB-C charger whose 5 V may sit anywhere in its tolerance.
Calibrate offset and gain. Two known inputs, near the bottom and top of the range, let you solve for both and remove them in software.
Keep the input impedance low. The ATmega328P's sample capacitor charges through your source resistance, and the datasheet says the ADC is "optimized for analog signals with an output impedance of approximately 10 k or less". A high-impedance divider (see why a voltage divider is not a power supply) needs a buffer or a capacitor on the pin.
Clean the analog supply. Decoupling on AVCC and AREF (our post on the 100 nF decoupling capacitor covers why) and an RC on the input (see RC low-pass filter cutoff) reduce noise before you need to average it away.
Then average, for resolution and noise.

The short version: resolution is

V_{\mathrm{ref}}/2^N

; accuracy is a separate budget, and the reference usually tops it. A 10-bit Uno on USB has 4.9 mV steps but about ±125 mV of absolute error at 2.5 V.

\text{ENOB} = (\text{SINAD} - 1.76)/6.02

measures noise and distortion, not absolute error. Averaging improves resolution as

\sqrt{N}

if there is noise to dither; it never fixes offset, gain, INL or the reference.

FAQ

What is the difference between ADC resolution and accuracy?

Resolution is how many output codes the converter has, so it sets the smallest step it can report: 5 V divided by 1,024 is 4.9 mV for a 10-bit ADC. Accuracy is how close the reported value is to the true input. It is set by offset, gain error, nonlinearity, noise and above all the reference voltage, and is usually many LSB worse than the resolution.

How do you calculate ENOB?

Measure SINAD, the ratio of a near full-scale sine to everything else (noise plus distortion), in dB, then $\text{ENOB} = (\text{SINAD} - 1.76)/6.02$ . An ideal N-bit converter has $\text{SINAD} = 6.02N + 1.76\ \mathrm{dB}$ , so a 10-bit ADC with 56 dB SINAD has an ENOB of 9.0 bits.

How accurate is the Arduino analogRead?

The ATmega328P converter itself is specified at about 2 LSB typical absolute accuracy, roughly 10 mV with a 5 V reference. But analogRead uses the 5 V supply as its reference by default, and USB allows 4.75 to 5.25 V, so an absolute voltage reading can be off by about 5%, around 125 mV at 2.5 V, unless you calibrate or use a proper reference.

Does oversampling and averaging improve ADC accuracy?

It improves resolution and random noise, not accuracy. Averaging N readings cuts random noise by $\sqrt{N}$ , and $4^n$ samples give n extra bits, but only if there is at least about 1 LSB of noise to dither the input. Offset, gain error, INL and reference error are the same in every sample, so averaging leaves them untouched.

Is a 16-bit ADC more accurate than a 10-bit ADC?

Not automatically. The 16-bit ADS1115 has a maximum gain error of 0.15% at 25 °C, about 48 of its 62.5 µV LSBs at 2 V, which is roughly 9.4 bits of absolute accuracy before calibration. It resolves far finer steps than a 10-bit part, but its absolute accuracy is only modestly better unless you calibrate it.

Sources

Microchip, ATmega48A/PA/88A/PA/168A/PA/328/P datasheet DS40002061B (ADC characteristics table 29-15: absolute accuracy 2 LSB typical, INL 0.5, DNL 0.25, gain and offset 2 LSB; internal reference 1.0 to 1.2 V; 50 to 200 kHz clock; 13 cycles per conversion; error definitions): microchip.com DS40002061B
Microchip (Atmel), ATmega328P automotive datasheet 7810D (absolute accuracy 2.2 typ / 3.5 max LSB, INL 1.5 max, gain and offset ±3.5 LSB): microchip.com 7810D
Atmel, AVR121, "Enhancing ADC resolution by oversampling" ( $4^n$ samples per n bits, at least 1 LSB of noise, "does not compensate for the ADCs integral non-linearity"): microchip.com AVR121 (doc8003)
Texas Instruments, SLAA013, "Understanding Data Converters" (derivation of $\text{SNR} = 6.02n + 1.76\ \mathrm{dB}$ ): ti.com/lit/an/slaa013
Texas Instruments, ADS111x datasheet SBAS444E (62.5 µV LSB at ±2.048 V, gain error 0.01% typ / 0.15% max, gain drift 40 ppm/°C max, noise and effective resolution tables): ti.com/lit/ds/symlink/ads1115.pdf
Texas Instruments, TIDUF26 reference design (USB 2.0 host range 4.75 to 5.25 V, worst case 4.35 V at the load): ti.com TIDUF26
Arduino AVR core source, wiring.c (ADC prescaler 128, 125 kHz at 16 MHz) and Arduino.h / wiring_analog.c (DEFAULT = AVCC reference): github.com/arduino/ArduinoCore-avr

Method note: Figure 1 is an illustrative 3-bit converter with invented errors. Figure 2 and all worked numbers are our arithmetic from the cited datasheet values. Figure 3 uses $\text{ENOB} = N - \tfrac12\log_2(1 + 12\sigma^2)$ , which assumes white Gaussian noise and an otherwise ideal converter. Figure 4 is a numpy Monte Carlo of an ideal rounding quantizer.

Header image: NASA/Ken Ulbrich (image AFRC2017-0021-04), regraded for this article.