Make sql server extract from historic tables using a single query
When extracting from historic tables in sql server we previously would perform two select statements. The first select would pull from the table, and include rows at the current time, and the second would pull all changes from the historic table. These would then be unioned together in the spark code.
We (in cox automotive) have seen issues where this historic extraction wasn't working correctly: rows would be in the wrong state when we queried in this way. We believe this is related to the two queries being non-transactional: since they are executed outside of a transaction the underlying rows might have changed before we query for the history. This change makes the extraction happen inside of a single sql query, which should provide the transactional isolation we need.
Co-authored-by: James Fielder <firstname.lastname@example.org>