01Explore the recovery policy
- 01SourceAmazon S3
CSV · gzip - 02Validate, filter, aggregate and transformFlower
CSV → Parquet - 03DestinationAzure Blob
Parquet
SourceAmazon S3
DestinationAzure BlobThe sample passes validation and is converted to Parquet. The illustration follows the data through processing, destination checks and completed delivery.
During a temporary destination outage, the flow applies its recovery policy with retries and backoff. Restore the destination to explore how delivery resumes.
The sample contains an invalid record. This scenario places the source file in quarantine with context for review, protecting downstream data quality.
The destination is available again and delivery completes. The illustration shows recovery and source lifecycle managed through configured policies.
02What to verify in your environment
| Scenario | Acceptance check |
|---|---|
| Normal delivery | Check that the Parquet output is readable and matches the agreed schema and expected records after transformation. |
| Temporary outage | Temporarily disconnect the destination to explore policy-driven retries, backoff, recovery and actionable alerts. |
| Invalid file | Submit a file with a record that violates a validation rule. Check that the whole source file is quarantined, with its failure reason, and that invalid data is not delivered downstream. |
| Recovered within the retry policy | Restore the destination within the retry window and verify the completed output. Check source retention or cleanup against its separately configured policy. |
03From illustration to a production flow
For the S3-to-Azure scenario, coordinate endpoint permissions, decompression, CSV schema, validation rules, Parquet mapping and quarantine. Combine destination checks with record counts, schema validation and business totals to document the quality and completion of the transformed delivery.