Data format, resolution vs. accuracy
The EHS- and VHS- multichannel modules are equipped with 24 bit ADC- and 20 bit DAC-circuits. This allows a very high precision and resolution of set and measured values. Data is transferred using the Floating Point Single Precision Format as specified in IEE 754 (23 bit mantissa and 8 bit exponent). The ADC sampling rate and the digital filter (used by the internal controller for processing measured values) are adjustable within certain limits. This allows the modification of resolution and sampling rate of the measured values. A high sampling rate enables fast data logging but reduces the resolution. The digital filter can be used to improve the resolution again.
The resolution and the accuracy of measured values are different parameters. The resolution is typically much higher than the total accuracy of a measured value. This is illustrated by the following example:
- EHS 82 40p (High Precision 4 kV / 2 mA)
- measurement in range <20 μA, accuracy ±(0.01% Iout + 4 nA), measured value: 5.02045 μA
- the true value of current can be in the range from 5.01595 μA to 5.02495μA
If the current increases by 50 pA, the measured value would change to 5.02050 μA. Thus it is possible to detect this change in current, although the absolute accuracy is much less precise.
A change in the module temperature can lead to a drift of the measured values of < 50 ppm/K. The absolute accuracy is valid only for one year, due to aging effects. Because of that the modules should be recalibrated at regular intervals.