The scenario describes a need to track social media data over the past 10 years, capturing all changes (inserts, updates, deletes) while reloading the data four times a day.

To manage this:

ModificationTime: This field is essential for tracking changes over time. It indicates when a record was last modified, allowing the script to determine whether it needs to insert, update, or delete records.

Primary Key Field: A primary key is crucial for uniquely identifying records. It enables the script to match records in the source with those already loaded, facilitating updates and deletions.

Insert and Update Records: The script should handle both inserting new records and updating existing ones based on the ModificationTime.

Remove Records: If records are deleted in the source, they should also be removed in the Qlik Sense data model to maintain consistency.

This approach ensures that all changes in the social media data are accurately captured and reflected in the Qlik Sense application.