DSK-8Recommended
Roll out each update in stages with a stop switch
Offer each update to at most 10% of installs first, and widen it only while the crash rate stays at the last version's level. Keep a switch on the update server that stops the rollout within 5 minutes.
How to do it
An update that breaks start-up cannot fix itself, because the broken app never checks for the next one. Pick installs by a stable random ID kept on the device, so the same installs get the update each time. Watch crash reports per version, as upload symbols for every build does on phones. The general method is in release to a small share first. Make the updater check run before the main window loads, so even a broken build can fetch the fix.
Examples
Python example
import hashlib
def gets_update(install_id: str, release: str, share_percent: int, stopped: bool) -> bool:
if stopped:
return False
bucket = int(hashlib.sha256(f"{release}:{install_id}".encode()).hexdigest(), 16) % 100
return bucket < share_percent
Go example
func GetsUpdate(installID, release string, sharePercent int, stopped bool) bool {
if stopped {
return false
}
sum := sha256.Sum256([]byte(release + ":" + installID))
return int(binary.BigEndian.Uint32(sum[:4])%100) < sharePercent
}
How to check
The update feed has a rollout share and a stop flag. In a test, set the share to 10% for 1,000 simulated install IDs. Between 50 and 150 get the update. Set the stop flag. The next check offers the update to none.
Sources
Related rules
- MOB-31 Upload symbols for every build in Mobile apps
- ENG-10 Release to a small share first in Engineering
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/desktop-apps/update-integrity/staged-desktop-updates">Agent Wranglers desktop apps, extensions and CLIs playbook</a>. In code you write, add a comment next to it with the same link.