The data we have collected is highly representative of that which can be found in the ICU, and therefore is replete with noise and artifacts due to patient movement, sensor degradation, transmission errors, electromagnetic interference and human error.
We have begun to develop signal quality indices to label useful and noisy sections of the data. Currently we have signal quality indices for the ECG and blood pressure waveforms, and we are close to completion for the pulse oximeter and the respiratory waveforms. A more detailed description of the algorithms for signal quality can be found in section 2.3.5 and in Zong et al. (7), Sun et al. (8) and in Li et al. (10,9).
Signal quality indices for trend data are almost impossible to generate, except by using thresholds on gradients and absolute values that are physiologically impossible. Generally it is better to refer back to the original underlying waveform to derive a signal quality metric.
Data are also missing due to machine or patient disconnections, transmission and recording errors, or human omissions. Some data are also not requested very frequently, and so, although not technically missing, important events may go unobserved.
Although short-term missing data can be mitigated somewhat through interpolation, much of our non-waveform and trend data is sparsely sampled. Moreover, the data is not missing at random, since it can be due to changes in shifts or staff-to-patient ratios, or simply because a clinician or nurse did not think that the data were important. Interpolation, or imputation is therefore impossible, unless a model of how the data are missing can be constructed.
Apart from these problems, there may also be errors in the data matching and alignment, particularly where the data come from different sources and use different clocks. The clocks are not always rigorously synchronized and may drift. The problem is particularly apparent during the beginning/end of daylight saving time. Section 5.4 details these and other known issues with the data.