- What a schedule is actually for
- Choosing a cadence
- Timezone is a correctness question
- The problem that only appears with two servers
- Scoping what runs
- What to watch after enabling one
- Frequently asked questions
A one-off scan tells you what your AWS account looked like on the afternoon you ran it. That is useful exactly once. What makes generated infrastructure code and drift detection worth anything is the second scan, and the twentieth, arriving without anybody remembering to trigger them.
Scheduling sounds like the trivial part of a product. It is where a surprising number of correctness problems live, and most of them only appear once you run more than one copy of the service.
What a schedule is actually for
Three distinct jobs, and it is worth being clear which one you want.
Keeping the code current. Infrastructure changes. A scan from six weeks ago describes an account that no longer exists, and Terraform generated from it will propose changes nobody wants.
Making drift detectable. Drift is a comparison, so it needs two points. Without a regular scan there is no second point and drift detection has nothing to detect against. This is the reason a schedule is not optional if you care about drift.
Making cost analysis meaningful. Utilization-based findings depend on observation windows, and the windows only fill if somebody is looking. A 30-day window is not available on the first day.
Choosing a cadence
Schedules run daily or weekly, at an hour and minute you choose, in a timezone you choose. Minutes are restricted to the quarter hours, which is not an arbitrary limitation: it keeps the scheduler's job cheap and avoids implying a precision that a queue-based system does not have.
| Situation | Cadence | Reasoning |
|---|---|---|
| Production account under active change | Daily | Drift found within a day is still attributable to a change someone remembers making |
| Stable production, change-controlled | Weekly | Changes arrive in known windows; daily scanning mostly confirms nothing happened |
| Development and sandbox accounts | Weekly | Constant intentional change makes daily drift reports noise rather than signal |
| Before a compliance or audit cycle | Daily, temporarily | You want the freshest possible picture and a dense recent history |
The failure mode to avoid is scanning more often than anyone reads the results. A daily drift report nobody opens is worse than a weekly one somebody does, because it teaches the team that the report is noise.
Timezone is a correctness question
Storing a schedule as an hour and a minute without a timezone seems simpler and quietly breaks twice a year. A job set for 02:00 local drifts to 01:00 or 03:00 when daylight saving shifts, and a "weekly on Monday" schedule can land on Sunday for anyone far enough east.
Each schedule therefore carries its own timezone, defaulting to UTC. Storing the zone rather than a fixed offset is the part that matters: an offset is correct on the day you compute it, and a zone stays correct through the transition. Teams spread across regions can also each have a schedule that means what it says locally.
The problem that only appears with two servers
This is the part worth understanding, because it is where naive schedulers fail silently.
The service runs more than one instance. If every instance independently checks "is anything due?" every fifteen minutes, then every due schedule fires once per instance. Two instances means every scheduled scan runs twice, consuming two scan slots and producing two reports that then have to be reconciled.
Locks are the usual answer and they bring their own problems: a lock held by an instance that dies has to expire, and the expiry is a guess.
The approach here uses a conditional update as the claim. Each runner tries to advance the schedule's next run time conditionally on it still holding the value the runner read. The database applies exactly one of those updates. The winner proceeds; the loser finds no matching document and skips, immediately, with no lock to release and nothing to time out.
// Atomic claim — the second runner matches nothing and skips.
ScanSchedule.findOneAndUpdate(
{ _id: schedule._id, nextRunAt: schedule.nextRunAt }, // still unclaimed?
{ $set: { nextRunAt: computeNextRunAt(...), lastRunAt: now } },
)
There is a deliberate tradeoff in it, and it is the kind that should be stated rather than discovered. The next run time advances before the scan is dispatched. So a transient dispatch failure means that occurrence is skipped rather than retried. This is at-most-once rather than at-least-once, chosen because a missed scan is visible in the next run's results, whereas a duplicated scan silently consumes a plan slot and produces a contradictory second report.
Both behaviours are defensible. The point is that the choice was made deliberately and is documented, rather than being whatever the implementation happened to do.
Scoping what runs
A schedule carries its own list of regions and services rather than inheriting whatever the last manual scan used. Two reasons.
Scan cost is real. Enumerating every service in every region takes time and API calls, and most accounts have a handful of regions that matter and a long tail that does not. Scanning what matters daily and everything monthly is usually the right shape.
Drift comparisons need a stable scope. If one scan covers three regions and the next covers five, the difference between them is not drift, it is a change of scope. Pinning scope to the schedule is what makes consecutive runs comparable.
What to watch after enabling one
The first two or three runs are the ones that tell you whether the cadence is right.
- Does anything change between runs? If consecutive scans are identical for a month, the cadence is too fast for this account.
- Is the drift report actionable? Drift on resources that intentionally change, such as autoscaling counts, is noise. Narrow the scope rather than raising the threshold.
- Does the timing collide with your own change windows? A scan mid-deploy captures a half-applied state and reports it as drift. Schedule outside your deployment windows.
Setting up the comparison side is covered in live drift detection, and what counts as drift in the first place in the drift detection guide.
Frequently asked questions
Why do I need scheduled scans rather than running one manually?
Because drift is a comparison and needs two points. Without a regular scan there is no second point, so drift detection has nothing to detect against. Utilization-based cost findings have the same dependency: a 30-day observation window is not available unless something has been observing for 30 days.
How often should I scan an AWS account?
Daily for a production account under active change, so drift is found while someone still remembers making the change. Weekly for stable change-controlled production and for development accounts, where constant intentional change makes daily reports noise. The failure to avoid is scanning more often than anyone reads the results.
Why does a scan schedule need a timezone?
Because an hour and minute without a zone breaks twice a year. A job set for 02:00 local drifts to 01:00 or 03:00 at a daylight saving transition, and a weekly Monday schedule can land on Sunday for users far enough east. Storing the zone rather than a fixed offset is what stays correct through the transition.
How do scheduled jobs avoid running twice when the service has multiple instances?
By making the claim itself atomic. Each runner tries to advance the schedule's next run time conditionally on it still holding the value that runner read, and the database applies exactly one of those updates. The winner proceeds and the loser matches no document and skips, with no lock to release and no expiry to guess at.
What happens if a scheduled scan fails to dispatch?
That occurrence is skipped rather than retried, because the next run time is advanced before dispatch. This is an at-most-once design chosen deliberately: a missed scan shows up in the next run's results, whereas a duplicated scan silently consumes a plan slot and produces a contradictory second report.
Should a schedule scan every region?
Usually not daily. Enumerating every service in every region costs time and API calls, and most accounts have a few regions that matter and a long tail that does not. Scanning what matters frequently and everything occasionally is the common shape. Keep the scope stable between runs, because a scope change between two scans looks like drift but is not.