This article provides a conceptual overview of how it works The audience for this article is an IT professional, database engineer, data architect or analyst, who wishes to understand how to import Synctera data into their company’s own data warehouse, data lakes or database.
In brief, we export a table as a CSV, and then encrypt it, using the customer’s public PGP key. Finally, the file is placed into the appropriate folder on the Synctera SFTP server for the customer to download When you, as a customer, receive the file, you’ll need to decrypt it ,using your company’s related private PGP key. What to know
  • Currently, these files are entire table dumps - so you get a complete view of the source table.
  • You should expect to either replace the existing table data in your data warehouse/database or else selectively insert the information
  • We expect to move to “daily deltas” some time in Q2/2023, and will work with you to ease that transition, when the time comes
  • We include an “EOF” file marker for each table export. This file tells you that the entire table has been exported into the corersponding CSV. This is important because…
  • when the exported file size would be > 1 GB, then we split the delivery into a series of ~ 100MB files.
  • Collectively, these files are the same as the source table
  • Each file is well-formed (complete rows)
  • Each file has a row-header
  • The EOF file is written after the table export is completed