• Stefy Lanza (nextime / spora )'s avatar
    autoupgrade: a host that is never idle now upgrades at midnight (v0.3.71) · 7a6234ce
    Stefy Lanza (nextime / spora ) authored
    The timer's own comment promised "a restart only happens on a real upgrade, and
    only when the fleet is idle (or after MAX_DEFERRALS hours of never being idle)".
    The second half could not happen:
    
        guard_idle || exit 0          # a busy host exits HERE
        ...upgrade...
        N=$(cat "$DEFERRALS" ...)     # the counter only ever incremented HERE
    
    So on a busy host the script left before the image was even pulled, the counter
    the force depends on never advanced, and MAX_DEFERRALS was unreachable. Measured
    on the Digesta orchestrator 2026-10-14: 24 skips in one day, `deferrals` still
    0, two versions behind, and 0.3.68 landed only because the host happened to be
    idle at 17:01.
    
    Now a busy skip COUNTS, and at MAX_DEFERRALS the script books a forced upgrade
    for the next midnight. The 00:00 tick of the existing hourly timer fires it, so
    there is no second unit to install and nothing stranded by a reboot. Midnight
    because it is the cheapest hour to interrupt a business-hours workload and a
    time an operator can predict without reading the script.
    
    Four things that each had a way to go wrong:
      * booked ONCE, not re-booked on every busy tick -- otherwise the deadline
        walks forward forever and never arrives;
      * a corrupt deadline file is discarded, not obeyed: garbage must not read as
        "the time has passed" and restart the service, nor sit there blocking a
        fresh booking;
      * a forced run does not defer at the restart step either -- deferring there
        would put it back where the booking was made;
      * MAX_DEFERRALS=0 keeps its old meaning (wait for an idle moment, however
        long, never force), so an install that did not opt in is unchanged.
    
    The booking is announced in the config banner of every later run and cancelled
    by deleting one file, because a forced restart that nobody saw coming is worse
    than a late upgrade.
    
    16 tests, and the ones that matter are behavioural rather than source-matching:
    they extract the real arm_midnight_force and force_due from the script, run them
    under bash against a temp state dir, and assert the timestamp really is local
    midnight and that force_due flips at the right moment. A test that restates the
    logic only proves it agrees with itself.
    
    Suite at the 25 pre-existing failures. NOT installed on Aruba: that copy is
    byte-identical to this one and runs hourly, so overwriting it mid-run would be
    editing a running script.
    Co-Authored-By: 's avatarClaude Opus 5 <noreply@anthropic.com>
    Claude-Session: https://claude.ai/code/session_01HXRMe9ZPUQCCCtBai9uqt3
    7a6234ce
Name
Last commit
Last update
..
coderai-autoupgrade Loading commit data...
coderai-entrypoint Loading commit data...
coderai-oci Loading commit data...
coderai-tarball Loading commit data...
coderai-upgrade Loading commit data...
sadtalker Loading commit data...
wav2lip Loading commit data...
with-env Loading commit data...