Additional date formats incorporated into Attribute ingestion ingestion, including more international formats
Updated the ingestion engine to skip bad data entries instead of halting the entire process, improving overall ingestion success rates while still logging errors and sending notifications for bad data.
Implemented a file size handling solution for S3 ingestion to split large CSVs into smaller, manageable jobs, preventing performance issues and ensuring successful processing of all data.
Updated the ingestion status notification logic to prevent redundant Slack alerts, consolidating notifications for multi-chunk ingestion jobs into a single message to reduce noise.
Added security headers back into Attribute application.
Bug Fixes
Update the Event-Emitter GraphQL mutation to publish to the events topic instead of a hard-coded Index Assets topic, ensuring that recording_cognition_completed events are correctly routed to the RecordingsTopic and fixing the data stream for US Prod NSA.
Investigated and fixed missing duration validation in the ingestion engine, ensuring that all playout data entries are properly validated, and that insufficient or missing duration values are correctly flagged, logged, and reported via email and Slack.
Fixed S3 Sync Mode not detecting new files correctly due to missing "Running" job status, which was causing duplicate ingestion jobs for files that were already in the process of being handled.
