~/home

Upgrading Node under pm2 without the surprises

2026-11-05T12:00:00Z

Upgrading Node under pm2 without the surprises

2026-11-05 · node, pm2

A security release came out, so I moved a few pm2 apps to a newer Node with nvm. nvm install took a minute. Finding every place the old version was still hiding took longer. Here is the order that worked with one restart per app.

1. Pin the nvm default to a major version

If your default is lts/*, nvm rewrites that alias every time it fetches the remote list. When a new line goes LTS, default points at a version you never installed, and new shells start with no node at all.

nvm install 24 --reinstall-packages-from=22
nvm alias default 24

--reinstall-packages-from reinstalls the same pm2 version you had, not the latest one. That's what you want here.

2. Fork-mode apps remember the old PATH

pm2 saves each app's environment in ~/.pm2/dump.pm2. Fork-mode apps spawn node through the saved PATH, so after a daemon restart they quietly keep running the old binary. Cluster-mode apps follow the daemon's own node.

Back up the dump, then replace the old version string in PATH, NVM_BIN, NVM_INC and _. pm2 restart --update-env also works, but run from a non-interactive SSH command it picks up that shell's thinner PATH.

3. Regenerate the systemd unit

The unit made by pm2 startup hard-codes the node path. Generate it again with the new binary:

NEW=$HOME/.nvm/versions/node/v24.x.y
sudo env PATH=$NEW/bin:$PATH $NEW/bin/node $NEW/lib/node_modules/pm2/bin/pm2 \
  startup systemd -u $USER --hp $HOME
sudo systemctl daemon-reload
sudo systemctl restart pm2-$USER

Avoid pm2 update over SSH for this. It restarts the daemon inside your SSH session's scope instead of under systemd.

4. Verify, then uninstall

for p in $(pgrep -f node); do readlink /proc/$p/exe; done | sort | uniq -c
pm2 save

Only when nothing points at the old version, run nvm uninstall.

Snipping Tool stuck on a busy cursor: a recycled PID

2026-11-03T12:00:00Z

Snipping Tool stuck on a busy cursor: a recycled PID

2026-11-03 · windows, debugging

The screenshot shortcut stopped working. Each press showed a busy cursor for a moment and left one more windowless SnippingTool.exe behind. By the time I looked, 19 of them were sitting in Task Manager.

The fix

taskkill /f /im SnippingTool.exe

All of them have to go. If you kill them one by one, kill the waiting ones before the one holding the lock, or they wake up and each opens an overlay.

What was going on

Wait Chain Traversal showed every new instance waiting on a mutex owned by one old instance. The rest came from reading AppInstance.cpp in the open-source Windows App SDK:

Instances register their PIDs in a shared memory section. Nobody removes their own PID on exit; the next launch prunes the dead ones.
The only liveness check is whether OpenProcess(SYNCHRONIZE) succeeds. If Windows has already given that PID to some unrelated process, the dead entry counts as alive.
For each listed PID, the launcher creates a <app>_<pid>_Mutex with initial ownership, assuming the real owner created it first. For a recycled PID, the launcher itself becomes the owner and keeps it until it exits.
A cancelled snip (press Esc) leaves its process running. If that resident process is the one holding the leaked lock, every later launch waits forever.

In my case a Python process had picked up a PID that Snipping Tool had used an hour earlier. A few hours later one cancelled snip took the lock, and the shortcut died.

Useful tools for this kind of hang

Event 201 in Microsoft-Windows-AppModel-Runtime/Admin logs each launch. Compare with the timestamps in Pictures\Screenshots to see which launches produced a file.
cdb -pv -p <pid> -c "!handle 0 f Mutant; qd" from the Windows SDK lists named mutexes without symbols, and doesn't stop the process.

Windows CPU % under-counts partial load

2026-10-20T12:00:00Z

Windows CPU % under-counts partial load

2026-10-20 · windows, performance

Games never seemed to push my 8-core CPU past 30% in Task Manager, even when they were clearly CPU-bound. So I wrote a small load generator and compared what Windows reports with what the CPU actually did.

The test

Start N threads that spin without ever blocking. For each thread, compare two numbers every second:

the time Windows credits it with (GetThreadTimes, the same source as Task Manager's "Processor Time"), and
the cycles it actually ran (QueryThreadCycleTime), divided by the clock.

What came out

1 busy thread: time accounting credited about 88% of the real run time.
8 to 12 busy threads: it dropped to about 55%.
All 16 hardware threads busy: both said 100%.
The cycle counter said 98 to 99% the whole time, and throughput agreed: 8 threads did 8.2× the work of 1.

With 8 threads pegged, the system-wide "time" number showed 28.5% while about half the machine was really busy. The other counter, "% Processor Utility", went the other way and read 80 to 85%, because it scales by boost clock against the base frequency. It also showed about 13% at idle.

What I use instead

For "am I CPU-bound?" the CPU % is close to useless. GPU usage that sits below its cap, or PresentMon's GPU Busy compared to frame time, answers the question directly. I checked one third-party task manager too: it reads the same time-based counters, so it inherits the same under-count.

Certbot renewals that reload nginx on their own

2026-10-12T12:00:00Z

Certbot renewals that reload nginx on their own

2026-10-12 · tls, nginx

Certbot's systemd timer renews certificates fine. The catch: if you got the certificate with certonly, nothing tells nginx to load the new file, and it keeps serving the old one until the next restart. You find out about 90 days later.

The fix

Drop a script into the deploy hook folder. Certbot runs everything in it after each successful renewal, and only then.

sudo tee /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh <<'SH'
#!/bin/sh
nginx -t -q && systemctl reload nginx
SH
sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh

The nginx -t guard matters: if a broken config is lying around, the reload would fail anyway, and this way the hook doesn't make things worse.

Check it without waiting three months

sudo certbot renew --dry-run --run-deploy-hooks
systemctl list-timers | grep certbot

Certbot skips deploy hooks on dry runs unless you pass --run-deploy-hooks, which is why the flag is there.

An old monitor that forgets to listen

2026-10-10T12:00:00Z

An old monitor that forgets to listen

2026-10-10 · hardware, windows

My second screen is a 22-inch panel with about 35,000 hours on it, fed over DVI. After every boot it stayed black with the backlight on. Windows saw it, the cable was fine, and unplugging it fixed it. That got old fast.

Narrowing it down

The monitor answers DDC/CI, so I could ask it what it thought was going on. When stuck, it reported the right input (DVI) and a locked vertical frequency of 0 (VCP AE), so it simply wasn't listening.

Reproducible: Win+P "PC screen only" for 40 seconds, then Extend. A one-second gap (Win+Ctrl+Shift+B) does not trigger it.
Re-selecting DVI does nothing. Restarting the signal from Windows does nothing.
Switching the input to VGA and back to DVI wakes it up every time.

The workaround

NirSoft's ControlMyMonitor can write VCP codes from the command line. VCP 60 is the input source; on this monitor 1 is VGA and 3 is DVI.

ControlMyMonitor.exe /SetValue "\\.\DISPLAY2\Monitor0" 60 1
timeout /t 2
ControlMyMonitor.exe /SetValue "\\.\DISPLAY2\Monitor0" 60 3

The monitor ignores writes while it's switching, so the real script re-sends until reading 60 back confirms the value, then checks that AE is no longer 0.

It runs from a scheduled task at logon, with a second trigger a minute later as a safety net. 26 boots in a row came up stuck, and the script woke the screen on the first try every time.

PowerShell 5.1 traps that only show up in the admin run

2026-09-23T12:00:00Z

PowerShell 5.1 traps that only show up in the admin run

2026-09-23 · windows, powershell

I wrote a cleanup script and tested it with non-admin dry runs. Every real bug showed up in the elevated run. These are the ones worth remembering.

Keep .ps1 files pure ASCII

Windows PowerShell 5.1 reads a UTF-8 file without a BOM in the system's ANSI code page. One non-ASCII character in a regex turned into three garbage characters and the pattern stopped matching.

A failed cast doesn't stop the script

[xml]$x = $text on bad input writes an error and carries on with $x empty. The error message also includes the whole input, which can be megabytes of red text. Use XmlDocument.LoadXml in try/catch and truncate the message.

$null becomes "" in .NET calls

Passing $null to a .NET string parameter sends an empty string. When the API really needs NULL, pass [NullString]::Value.

The unary comma and the pipeline

A function that ends in return ,$list sends the list as one object. Get-Things | Where-Object { ... } then filters the whole array as a single item. Assign to a variable first, then pipe.

Variables are case-insensitive

A local $folders = @() inside a function hid the script-level $Folders. The loop ran over an empty array and the script reported success. Give config lists names that can't collide.

Guards must fail closed

Start with $blocked = $true and clear it only when the check fully completes. My first version did the opposite, and a parse failure would have let it proceed.

Check native exit codes

pnputil /export-driver fails if the target folder doesn't exist. Piped to Out-Null, nobody noticed, and the next line deleted the package without a backup. Check $LASTEXITCODE and that the file landed.

Restic to a second disk, the boring way

2026-08-30T12:00:00Z

Restic to a second disk, the boring way

2026-08-30 · backups

I tried a few backup tools with web UIs, then went back to restic and a systemd timer. The only backup that works is the one that runs without me.

Service

# /etc/systemd/system/restic-backup.service
[Unit]
Description=restic backup

[Service]
Type=oneshot
EnvironmentFile=/etc/restic/env
ExecStart=/usr/bin/restic backup /etc /home /srv --exclude-caches
ExecStartPost=/usr/bin/restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

/etc/restic/env holds RESTIC_REPOSITORY and RESTIC_PASSWORD_FILE. Make it chmod 600.

Timer

# /etc/systemd/system/restic-backup.timer
[Timer]
OnCalendar=*-*-* 03:30
RandomizedDelaySec=20m
Persistent=true

[Install]
WantedBy=timers.target

Persistent=true runs a missed backup at the next boot, which matters for a machine that is sometimes off at 3:30.

The part people skip

Once a month, restore one random file somewhere else and open it. restic check --read-data-subset=5% is good too, but it doesn't tell you whether you can find the file you actually need.

Bad RAM, one shared DLL, and badmemorylist

2026-08-14T12:00:00Z

Bad RAM, one shared DLL, and badmemorylist

2026-08-14 · windows, hardware

Two months of random blue screens, each with a different stop code: corrupt PTEs, critical process died, list entry corruption, page faults. Plus a few hard resets with no bugcheck at all. No WHEA errors anywhere.

Two wrong assumptions

"No WHEA events, so it isn't RAM." Consumer non-ECC memory has no error detection. A flipped bit never reaches WHEA.
"Crashes come in bursts, not a gradual curve, so it isn't hardware." One bad cell causes a burst the moment a busy page lands on it, and goes quiet when the page moves.

The tell

Three unrelated services crashed with ILLEGAL_INSTRUCTION at the same offset in the same shared DLL, and the DLL on disk hashed clean. Windows maps one physical copy of a DLL's code into every process. If that one frame is bad, every process decodes the same garbage. A reboot moved the page and the crashes stopped.

Confirming it

Windows Memory Diagnostic found errors within the first pass. MemTest86 found 3,278 errors over 4 passes, even at a slow stock speed. Only 12% came from the row-hammer test, so it wasn't a borderline timing issue. 148 bad addresses fell into 16 pages in three clusters, and always the same two bits. That's one chip.

Fencing it off

Windows can refuse to use specific page frame numbers:

bcdedit /set {badmemory} badmemorylist 0x5EF3E0 0x5EF3E1 ...
bcdedit /enum {badmemory}

I padded each cluster to a 16-page block, 48 pages in all, which costs nothing you'd notice. Two things to know:

/set replaces the whole list. To add pages later, repeat all the old ones.
Write down where the clusters are. If a later crash lands outside them, the chip is getting worse and the list isn't enough. Mine did, two months later, so a new stick is on the list.

The editor update that deletes the editor

2026-07-27T12:00:00Z

The editor update that deletes the editor

2026-07-27 · windows, tooling

Twice this month an auto-update on Windows left me with no editor at all. The shortcut pointed at nothing, and the update log claimed success.

What happens

The editor (Cursor, but the updater comes from VS Code) stages the new build in a _ subfolder. The updater then deletes the old install before moving the new one into place. Before it starts, it only checks whether the main .exe is still running.

The editor puts its own bundled node.exe first on PATH when it starts stdio MCP servers. So npx some-mcp-server runs on the editor's node, and that process keeps running after the window closes. The delete hits a locked file, gives up after retries, and leaves a half-deleted install with the new build stranded in _.

Recovering

No reinstall needed. Delete the gutted resources folder, move everything from _ up one level, delete _.

Preventing it

I turned off auto-update ("update.mode": "none") and check for stragglers before updating by hand:

Get-Process | Where-Object { $_.Path -like '*\cursor\resources*' }

Turning the server off in the editor's MCP settings is not enough. One server's child processes survived the toggle. Trust the process list, not the UI. HTTP-based MCP servers don't spawn anything local, so only stdio ones matter.

Folders called "cache" aren't always cache

2026-07-09T12:00:00Z

Folders called "cache" aren't always cache

2026-07-09 · windows, housekeeping

The system SSD was nearly full, so I went through C:\ProgramData looking for big folders that sounded disposable. Logitech G HUB had gigabytes in depots and cache. They sounded like old downloads, so I deleted them. G HUB stopped working.

What `depots` really was

A current.json next to it listed the depot of the installed build with state "PRESENT". The app runs out of that folder and checks it. It held the current version, not leftovers. A reinstall over the top fixed it.

The rule I use now

Inside an app's own data folder, a name like cache, depots or downloads proves nothing. Look for manifests or version files that reference the folder first.
Prefer the app's own cleanup option or its uninstaller.
If you can't tell, file it under "might need a reinstall", not "safe".
Generic caches are still fair game: npm, pip, shader caches, %TEMP%. Those rebuild themselves by design.

Measure your DNS resolvers before picking one

2026-06-21T12:00:00Z

Measure your DNS resolvers before picking one

2026-06-21 · networking

Every "fastest DNS" list was measured from somebody else's network. Anycast routing depends on your ISP's peering, so the resolver that is 5 ms away for them can be 60 ms away for you.

A quick loop

for r in 1.1.1.1 8.8.8.8 9.9.9.9; do
  printf '%-10s' "$r"
  for i in 1 2 3 4 5; do
    dig @"$r" "example$RANDOM.com" +tries=1 +time=2 | awk '/Query time/{printf "%5s ms", $4}'
  done
  echo
done

The random name skips the resolver's cache, so you measure the full lookup, not a cache hit. Run it at a few times of day. Evening numbers are usually the honest ones.

What to look at

Look at the median and the worst result, not the best one. A resolver that holds steady at 8 ms beats one that swings between 3 and 90 ms.

~/home

Upgrading Node under pm2 without the surprises

Upgrading Node under pm2 without the surprises

1. Pin the nvm default to a major version

2. Fork-mode apps remember the old PATH

3. Regenerate the systemd unit

4. Verify, then uninstall

Snipping Tool stuck on a busy cursor: a recycled PID

Snipping Tool stuck on a busy cursor: a recycled PID

The fix

What was going on

Useful tools for this kind of hang

Windows CPU % under-counts partial load

Windows CPU % under-counts partial load

The test

What came out

What I use instead

Certbot renewals that reload nginx on their own

Certbot renewals that reload nginx on their own

The fix

Check it without waiting three months

An old monitor that forgets to listen

An old monitor that forgets to listen

Narrowing it down

The workaround

PowerShell 5.1 traps that only show up in the admin run

PowerShell 5.1 traps that only show up in the admin run

Keep .ps1 files pure ASCII

A failed cast doesn't stop the script

$null becomes "" in .NET calls

The unary comma and the pipeline

Variables are case-insensitive

Guards must fail closed

Check native exit codes

Restic to a second disk, the boring way

Restic to a second disk, the boring way

Service

Timer

The part people skip

Bad RAM, one shared DLL, and badmemorylist

Bad RAM, one shared DLL, and badmemorylist

Two wrong assumptions

The tell

Confirming it

Fencing it off

The editor update that deletes the editor

The editor update that deletes the editor

What happens

Recovering

Preventing it

Folders called "cache" aren't always cache

Folders called "cache" aren't always cache

What depots really was

The rule I use now

Measure your DNS resolvers before picking one

Measure your DNS resolvers before picking one

A quick loop

What to look at

What `depots` really was