Synthesized by Clarity (Claude) from 15 sources · May contain errors — spot one? [email protected] · Methodology →
NadMesh Botnet Scrapes 3,811 AWS Keys from Ollama, ComfyUI
- Sources
- 15
- Words
- 1,057
- Read
- 5min
Topics LLM Inference Agentic AI AI Regulation
◆ The signal
NadMesh scans Shodan for exposed self-hosted AI tooling and already claims 3,811 AWS keys. Shodan finds the box; the operator does the rest. Same day: an OpenSSL HollowByte DoS with a patch already shipped, an unauthenticated WordPress RCE with a live public PoC, and seven RAT-laced npm packages in the Vite chain. Patch the OpenSSL one first. The WordPress PoC is public, so that one won't wait.
◆ INTELLIGENCE MAP
Intelligence map
01 Self-Hosted AI Tooling Is Now an Active Attack Surface
act nowNadMesh, a Go botnet, Shodan-scans exposed ComfyUI, Ollama, and n8n instances and harvests cloud creds — its dashboard claims 3,811 AWS keys. Also live: OpenSSL's HollowByte DoS (11 bytes → 131KB, patch shipped), an unauthenticated WordPress wp2shell RCE with public PoC, and seven RAT-laced npm packages hitting Vite builds via blockchain C2.
- AWS keys stolen
- OpenSSL amplify
- Malicious npm pkgs
- WordPress RCE
02 The 2027 Memory Squeeze Meets an Inference-Cost Inversion
monitorSK Hynix's CEO forecasts the worst-ever memory shortage in 2027, AI demand outrunning supply past 2030 — RAM becomes a first-class capacity variable. Meanwhile Kimi K3 breaks 'open equals cheap': ~half GPT-5.6 Sol's per-token price but 24x DeepSeek V4 Pro. Falling token prices grow total consumption faster than they save.
- RAM shortage year
- K3 vs DeepSeek
- K3 params
- Weights drop
03 Rollback Isn't Recovery: The AWS Billing Failure
monitorAWS's billing computation subsystem broke Thursday night, showing phantom estimates up to ~$2.5B. The engineering signal: rolling back the offending change failed to fix it, because the deploy had recomputed persisted state. Reverting the binary doesn't un-write bad data.
- Phantom bill
- Rollback result
- Expected duration
04 Agent Systems: Cheap Orchestration Wins vs. the Productivity Mirage
backgroundACP folded into A2A, collapsing the stack to two protocols (MCP for tools, A2A for agents). Two orchestration-layer papers land real wins: early-abort classifiers cut compute up to 60% at 90% success; skill-file auto-tuning adds +28.6%. But an NBER study of 25K workers found ~2.8% task time saved vanishing before P&L.
- Protocols now
- Early-abort cut
- Skill-file gain
- Workers studied
- Task time saved that vanishes before P&L2.8
◆ DEEP DIVES
Deep dives
01 Your Ollama Box Is One Shodan Scan From an AWS Takeover
act nowThe mechanism that defeats your usual mitigation
OpenSSL's HollowByte hurts because of where the memory lands, not how much of it. Here's what happens: an 11-byte TLS record makes an unpatched server reserve up to 131KB for a body that never arrives. On glibc that reservation sits in an arena the allocator never returns to the OS until the process restarts. Rate-limiting does nothing. A slow trickle of tiny requests starves the worker, and the free cycles that would recover the arena never come. The only clean interim fix is aggressive worker recycling until the patch is in.
The WordPress wp2shell RCE is a different urgency class. Unauthenticated, CVE-assigned, shipping with a working public PoC. Mass scanning is already running. Redis or Memcached-backed WordPress adds a persistent-object-cache condition, which widens the surface past a single-server install.
Why your self-hosted AI is the sleeper
NadMesh targets how these tools actually deploy. The Go botnet keeps a Shodan queue stocked with exposed ComfyUI, Ollama, and n8n instances, then harvests AWS keys and Kubernetes tokens. The operator dashboard claims 3,811 unique AWS keys. These tools ship without auth by design. That frictionless-local default is fine on localhost and a cloud-account takeover the moment the port faces the internet. The seven RAT-laced npm packages in the Vite ecosystem use blockchain-based C2. A domain seizure won't help. The kill switch is on-chain and immutable.
The shared response
These signals all point at patch economics. AI-accelerated disclosure is outrunning calendar-based cadences. The durable answer is risk-based triage on EPSS + CISA KEV and reachability, not raw CVSS. The floor matters more than the ceiling. Default creds and stale protocols are still the actual entry point in government advisories.
Action items
- Bind every self-hosted AI service (Ollama, ComfyUI, n8n) to localhost/private networks, front with an authenticating reverse proxy, enforce IMDSv2, and rotate any cloud keys reachable from those hosts.
- Patch OpenSSL on all TLS terminators and take/gate every WordPress instance behind a WAF blocking the wp2shell pattern, then hunt for webshells.
- Freeze npm additions in the Vite build chain, enable install-time scanning (Socket/npm audit signatures), pin lockfiles with integrity hashes, and rebase patch triage on EPSS+KEV.
02 The 'Open Means Cheap' Rule Just Broke — and RAM Is About to Get Scarce
monitorThe pricing inversion your routing logic didn't expect
Most model-selection layers encode one rule: Chinese open weight equals cheapest. Kimi K3 violates it. Per Artificial Analysis it runs about half of GPT-5.6 Sol per token but roughly 24x DeepSeek V4 Pro. That is a frontier-class open model priced as a premium, not a floor. It decouples four variables your routing conflates: capability, open-vs-closed weights, self-host cost, and per-token API price. Those were never the same axis. Now you have a decision matrix instead of a default branch.
The harder problem is demand elasticity. Per-token prices fall, total consumption rises faster, and spend flows toward the compute layer, not away from it. Jevons paradox for inference. A 40% price drop with 2x volume growth is a cost increase. Model-shopping stops mattering. Compute efficiency is the durable lever.
Where the hardware bites
SK Hynix's CEO forecasts the industry's worst-ever memory shortage in 2027, with AI demand outrunning supply past 2030. Any cheap-RAM line in your capacity plan breaks there. And efficient-to-train is not cheap-to-serve. K3 is a 2.8T-parameter MoE with only 16 of ~900 experts active per token. As one DeepMind researcher put it, a small lab can compress the compute to build a frontier model but still can't afford to serve one. Open weights hand you the checkpoint. They do not hand you the serving bill.
Caveat: several K3 benchmark scores are self-reported, and one Arena chart used a truncated x-axis to exaggerate the lead — treat the pricing facts as firmer than the capability ranking.
Action items
- Rebuild inference cost forecasts on an elastic-demand assumption — project total token-volume growth at a 30–50% price drop rather than holding volume fixed — this sprint.
- Model a 2x–3x RAM cost scenario against your 2027 capacity plan, right-size your heaviest heaps and caches, and lock reserved commitments before the squeeze.
- Stand up an eval harness on your real tasks with a per-token cost column, benchmark K3 when weights drop July 27, and default to a hosted endpoint — don't reflexively self-host 2.8T params.
03 AWS's Rollback Couldn't Undo a $2.5B Phantom Bill — Here's Why
monitorThe diagnosis behind the dollar figures
AWS's billing computation subsystem broke late Thursday. By Friday customers saw estimates in the millions to billions, one Reddit screenshot showing ~$2.5B. Amazon confirmed the numbers were fake. The engineering signal isn't the figure. It's that a rollback of the offending change did not fix the incident. A failed rollback almost always means the change did more than swap code. It mutated or recomputed persisted state. Reverting the binary doesn't un-write the bad computation. That's when teams learn their rollback plan was never a recovery plan for the data layer.
The two things this changes in your stack
First, the direct exposure for AWS customers is your own automation acting on provider-side garbage. Budget kill-switches, cost-based autoscaling caps, and finance pipelines keyed off the Billing API can all misfire on a phantom estimate. Add anomaly detection that alerts without acting destructively.
Second, classify every P0 deploy as reversible-deploy or irreversible-data-mutation. For the latter, a rollback button is theater. You need a forward-fix runbook written before the incident, not improvised at 2am. Every ledger, billing, and event-sourcing system learns this the expensive way.
If your rollback plan doesn't account for state that got recomputed, that isn't a recovery plan. It's a hope.
Action items
- Audit critical-path deploys (billing, ledger, anything computing money), classify each as reversible-deploy vs. irreversible-data-mutation, and write forward-fix runbooks where rollback won't recover state.
- Add proactive billing anomaly alerting (budget alarms + Cost Anomaly Detection) that notifies but does not trigger destructive automation on provider-side estimate glitches.
04 Two Cheap Agent Wins — and the Study That Says They Won't Reach Your P&L
backgroundOne fewer standard to track
ACP has been merged into A2A. The three-way standards war is now a two-layer stack: MCP for agent-to-tool, A2A for agent-to-agent. A2A absorbed ACP's REST-first patterns. Agent Manifest discovery, direct HTTP, sync-or-SSE responses all carried over, so nothing you built against ACP is gone. If you hedged across three specs, pick one. Commit now.
The wins that live outside the model
Two papers describe techniques that cost no GPU spend. Early failure detection reads a model's first-step internal signals and aborts doomed multi-step tasks. It cuts compute up to 60% while retaining 90% success. That is a lightweight classifier, nothing more, and it is a pure cost play. Skill-file auto-tuning tests skill files against real system-state changes and lifts task success up to +28.6% with zero retraining. That is a pure reliability play. Both sit at the orchestration and prompt layer. Cheap to trial this quarter.
Why the gains may not show up
The counterweight is an NBER study of ~25,000 Danish workers. Self-reported time saved was ~2.8% task time saved. Administrative records showed near-zero change in hours or earnings. Confidence intervals rule out effects above ~2%. Workers redirected the saved time. The value dissipated between the keyboard and the P&L. The fix is architectural. Instrument the downstream KPI with a pre-AI baseline. Close the correction loop. Open-loop automation demos fast and never compounds.
Action items
- Collapse any in-flight agent-comms design onto MCP (tool plane) + A2A (agent plane) and drop standalone ACP implementation plans.
- Prototype an early-abort classifier on one multi-step agent workflow using first-step internal signals to kill doomed runs.
- Before shipping the next AI feature, define the downstream business metric it should move, capture the pre-AI baseline, and verify a closed correction loop exists.
◆ QUICK HITS
Quick hits
Zoom shipped an urgent account-takeover patch it caught internally
Phishing campaign runs Lua malware disguised as .ttf font files, filelessly
93% of teams hit an AI-caused infra incident; only 30% wrote a policy
Kubernetes queue-depth autoscaling: Prometheus exporter wired to HPA
Zero-downtime Postgres credential rotation via connection-pool swap
Anubis-style proof-of-work anti-bot layer now has a published bypass
Redis in production without AOF+RDB is a data-loss incident waiting to happen
◆ Bottom line
The take.
The infrastructure you operate yourself is where both compromise and cost now concentrate — isolate it, instrument it, and write the forward-fix runbook before your own automation outruns your controls.
Frequently asked
- How do I lock down a self-hosted Ollama or ComfyUI box before a scanner finds it?
- Bind every self-hosted AI service to localhost or a private network, front it with an authenticating reverse proxy, enforce IMDSv2, and rotate any cloud keys the host can reach. Tools like Ollama, ComfyUI, and n8n ship without auth by design, so an internet-facing port hands attackers whatever AWS keys or Kubernetes tokens that box can touch.
- Why won't rate-limiting stop the OpenSSL HollowByte DoS?
- Rate-limiting fails because the harm is where the memory lands, not how much arrives. An 11-byte TLS record makes an unpatched server reserve up to 131KB in a glibc arena that is never returned to the OS until the process restarts, so even a slow trickle of tiny requests starves the worker. Until the patch is applied, aggressive worker recycling is the only clean interim fix.
- Why won't a domain takedown stop the seven malicious npm packages?
- The RAT-laced packages in the Vite ecosystem use blockchain-based command-and-control, so the kill switch is on-chain and immutable and seizing a domain does nothing. The durable defense is supply-chain gating: freeze npm additions in that build chain, enable install-time scanning, and pin lockfiles with integrity hashes.
- Why didn't AWS's rollback fix the phantom billing incident?
- The rollback reverted the code but not the persisted state that code had already mutated or recomputed, so the bad computations remained. A failed rollback almost always means the change did more than swap binaries. Classify P0 deploys as reversible-deploy or irreversible-data-mutation, and write a forward-fix runbook for anything that touches persisted state.
- Will switching to a cheaper per-token model actually cut my inference bill?
- Not necessarily, because falling per-token prices tend to push total consumption up faster, so spend shifts toward the compute layer rather than shrinking. A 40% price drop paired with 2x volume growth is a net cost increase, a Jevons-paradox effect. Rebuild cost forecasts on elastic demand and treat compute efficiency, not model-shopping, as the durable lever.
◆ Same day, different angle
Read this day as…
◆ Recent in engineer
Keep reading.
Spot an error? [email protected]