OWOX BI uses the at-least-once delivery concept to ensure collecting all hits you send to our endpoint. It means in some cases it may be duplicates of hits with the same 'hitId'. Typically, the share of these hits is less than 0.1%.
Best effort de-duplication
Thus, if you operate directly to hit level data in tables like 'streaming_YYYYMMDD', we recommend adding a specific condition to your SQL query to exclude duplicates.
Example:
WITH uniqueHistTable AS(
SELECT
*
EXCEPT(rowNumber)
FROM
(
SELECT
*,
ROW_NUMBER() OVER (PARTITION BY hitId) rowNumber
FROM
HITS_TABLE
)
WHERE
rowNumber = 1
)
SELECT * FROM uniqueHistTable
Note:While running the session collecting OWOX BI automatically removes all duplicates of hits.
0 Comments