What does the system do or how does it handle a missing interval of data?

Discussion in 'AVReporter Questions' started by avreporter, Aug 28, 2015.

  1. avreporter

    avreporter Administrator Staff Member

    I do have a technical question about the Cost Allocation module that would be good to have answered. Since this information is based on interval data (typically 15 minute intervals), what does the system do or how does it handle a missing interval of data or an interval with an obvious out of range value? Does it have an algorithm that deals with these instances?
  2. avreporter

    avreporter Administrator Staff Member

    AVReporter has more ways for data quality management:
    1) AVRConnection, when import data from external data sources, it is possible to use data qulity settings (Time jitter, Null value, Zero value correction, etc.)
    upload_2015-8-28_12-35-26.png
  3. avreporter

    avreporter Administrator Staff Member

    Data from the source database may include inaccurate or unusable values you don't want to add to the target database in their current form. AVR Connection Center gives several opportunities to perform the most important operations of correcting unsatisfactory data during transfer.

    - Time jitter correction
    - Null value correction
    - Zero value correction
    - Negative value correction
    - Minimum threshold
    - Maximum threshold

    Time jitter correction fixes inaccurate dates by setting the value of seconds to 0 in every row. Enabling this requires selecting the field containing timestamps.
    After enabling any of the other five features we are expected to specify the numeric column containing values (usually values measured by instruments). All of these options will work on the contents of this field.
    Null value correction is activated if a row of the source table contains an empty (Null) value. It can perform one of the following steps: it either discards the row or replaces the value with a default one. In the latter case the empty value will be overwritten with the number we specify.
    Zero value correction allows replacing zero (0) values with a value of your choice.
    Negative value correction is used to replace values less than zero.
    You can set a Minimum threshold to replace every value below this limit with one supplied beforehand.
    E.g. if you set Minimum value to 10 and Replace values below threshold with to -1 then every value below 10 will be changed to -1.
    Maximum threshold works the same way except that it can be used to modify values which are greater than the limit.
  4. avreporter

    avreporter Administrator Staff Member

    2) AVReporter has a DataManagerService, It is able to use mathematical method for missing data calculation and bad data modification, etc.:

    Treatment conditions (the condition that triggers the treatment):
    1. Missing – matches when a data-sample is not present within the period defined by the current timestamp and the entered Reference Interval in seconds
    2. Null – matches when a NULL sample is found
    3. Equals – matches when the sample is equal to the value entered “Param1” value
    4. Greater than - matches when the sample is greater than the value entered “Param1” value
    5. Less than - matches when the sample is less than the value entered “Param1” value
    6. Greater than or equal - matches when the sample is greater than or is equal to the value entered “Param1” value
    7. Less than or equal - matches when the sample is less than or equal to the value entered “Param1” value
    8. Different - matches when the sample is not equal to the value entered “Param1” value
    9. Between - matches when the sample is between the value of “Param1” and “Param2”
    Treatment actions (the operation being executed once the condition has been met):
    1. Constant – the exact value of “Action constant” is saved
    2. Last – the value of the last of the samples preceding the current
    3. First – the value of the first sample preceding the current (the size of the dataset is defined by the “Sample size” parameter)
    4. Average – the average of the samples preceding the current (the size of the dataset is defined by the “Sample size” parameter)
    5. Maximum – the max value of the samples preceding the current (the size of the dataset is defined by the “Sample size” parameter)
    6. Minimum – the min value of the samples preceding the current (the size of the dataset is defined by the “Sample size” parameter)
    7. Linear interpolation – predicted value calculated from the samples preceding the current (the size of the dataset is defined by the “Sample size” parameter) using linear interpolation
    8. Null – saves NULL value to the database
    9. Nothing – no sample is being saved
    upload_2015-8-28_12-45-54.png
  5. avreporter

    avreporter Administrator Staff Member

    3) Open report & runtime data modification in Business Intelligence report:
    - open the report, we can use filters and mathematical & statistical functions (results are not stored in AVReporter database) - don't suggest this way large amount of data.

    upload_2015-8-28_12-50-0.png
  6. avreporter

    avreporter Administrator Staff Member

Share This Page