Run
3 of 4 tasks did not succeed
This run was disrupted and did not finish
3 tasks exhausted its attempts and was dead-lettered. All of it is in the 28 events below.
What happened
The milestones of this run, in the order they were recorded. Each one is a real event; the full sequence is in the timeline below.
- Run started#6 23:31:04.036
RUN_STARTEDA worker picked up the first task in the run.
- Failure injected on purpose validate#11 23:31:04.049
FAULT_INJECTEDReLab degraded the real system here. This is the break, and everything after it is the recovery.
- Retry scheduled validate#13 23:31:04.050
TASK_RETRY_SCHEDULEDThe task will be offered again after a backoff delay.
- Task running validate#16 23:31:04.893
TASK_STARTEDThe handler began executing. Each attempt writes one of these.
- Task gave up validate#25 23:31:06.990
TASK_DEAD_LETTEREDThe task exhausted its attempts and will not be tried again.
- Workflow failed#28 23:31:06.990
RUN_FAILEDA task ran out of attempts and the run could not continue.
What the journal proves
Every number above is a count of events in this run’s journal. Recovery is measured from the first fault, lease expiry, task failure or lost worker to the run completing, the same interval relab test asserts on.
Tasks
| Status | Task | Attempt | Worker | Error |
|---|---|---|---|---|
| DEAD | analyze | 0/3 | · | · |
| SUCCEEDED | import | 1/3 | db3c6da3 | · |
| DEAD | report | 0/3 | · | · |
| DEAD | validate | 3/3 | · | injected http-error fault: upstream returned 503 |
Timeline
The run’s complete recorded history, in sequence order. This is the same journal relab replay reduces. Filtering hides rows from this view; it never changes the sequence numbers, so a gap in them is still a gap.
No worker events in this run. That is a fact about the run, not a missing page: nothing of that kind was recorded.