LinkedIn Percentile Bucketing Lifts Embedding Recall 30x

◆ DEEP DIVES

1. 01

   LinkedIn's Percentile Bucketing: Your LLM Embeddings Are Ignoring Every Number You Feed Them

   <h3>The Discovery That Changes Your Feature Engineering</h3><p>LinkedIn disclosed the most detailed public account of replacing a <strong>multi-pipeline recommendation architecture</strong> with a unified LLM-based system at genuine scale: 1.3 billion users, sub-50ms latency. Five separate retrieval systems — chronological, trending, collaborative filtering, industry-specific, and embedding-based — are gone, replaced by a single dual-encoder LLM. But the headline isn't the architecture migration. It's what they found about <strong>how LLMs process numbers</strong>.</p><p>When LinkedIn passed raw engagement counts (e.g., "views:12345") into their LLM encoder prompts, the correlation between item popularity and embedding similarity was <strong>-0.004</strong> — statistically indistinguishable from random. LLMs tokenize "12345" as a sequence of digit characters with <strong>zero magnitude awareness</strong>. The fix was trivially simple: convert raw counts to percentile buckets wrapped in special tokens: <code>&lt;view_percentile&gt;71&lt;/view_percentile&gt;</code>.</p><blockquote>Result: 30x correlation improvement and 15% Recall@10 lift — from a preprocessing step that takes hours to implement.</blockquote><h3>The Training Data Insight Nobody Expected</h3><p>Counter-intuitively, <strong>removing non-engaged posts</strong> from training sequences improved both quality and efficiency. Scrolled-past items added noise and inflated sequence length (quadratic attention cost). Positive-only sequences delivered:</p><ul><li><strong>37% memory reduction</strong> per sequence</li><li><strong>40% more sequences per batch</strong></li><li><strong>2.6x faster training iterations</strong></li></ul><p>For negative examples, LinkedIn used a two-tier strategy: easy negatives (random unshown posts) plus hard negatives (shown but not engaged). Adding just <strong>2 hard negatives per member yielded 3.6% recall improvement</strong>.</p><h3>What Transfers to Your Stack</h3><p><strong>Percentile bucketing</strong> is the single highest-ROI takeaway. If you use any transformer encoder to process structured numerical features — for recommendations, search ranking, or tabular prediction — your embeddings are likely blind to the numbers you're feeding them. The implementation is trivial: compute percentile ranks offline, wrap in special tokens, fine-tune. <em>The -0.004 correlation proves raw numbers are invisible to LLM encoders, and this likely affects every team using transformers for structured data without realizing it.</em></p><p><strong>Positive-only training sequences</strong> challenge a deep assumption. Most recommendation teams include implicit negatives (impressions without clicks). LinkedIn found these hurt quality and ballooned cost. The 2.6x training speedup alone justifies running this ablation even if quality is neutral.</p><p>The <strong>cold-start advantage</strong> is also worth noting: the LLM encoder generates meaningful embeddings from just a profile headline — no engagement history needed — leveraging the model's world knowledge. If cold-start is a pain point in your recommender, an LLM user encoder is the most promising architectural direction.</p><h4>What's Missing</h4><p><em>No online A/B test results were shared.</em> The 15% Recall@10 and 3.6% from hard negatives are offline numbers without confidence intervals or sample sizes. This is a first-party engineering blog — treat the numbers as directionally correct but assume selection bias in what was reported.</p>

   Action items

   • Implement percentile bucketing for all numerical features fed into LLM-based encoders this sprint — wrap discretized values in special tokens and measure correlation vs. raw numeric inputs
   • Run an ablation comparing full-impression training sequences vs. positive-engagement-only sequences on your recommendation models by end of quarter
   • Prototype hard negative mining with 2 hard negatives per positive example for any contrastive learning retrieval model

   Sources:LinkedIn killed 5 retrieval systems with 1 LLM — percentile bucketing trick boosted Recall@10 by 15%

2. 02

   Your Training Data Is Majority Synthetic — And Contaminated with State Propaganda

   <h3>The Contamination Is No Longer Theoretical</h3><p>Four independent signals converge on the same conclusion: <strong>the open web as a training data source is compromised</strong>. A 2025 Ahrefs study found <strong>71.7% of web pages contain AI-generated content</strong>. DFRLab audited Common Crawl and confirmed content from <strong>Pravda, Glassbridge, and RT</strong> — known state propaganda outlets — and demonstrated that LLMs can reproduce this content. Meanwhile, the Greenlandic Wikipedia case provides a <strong>live observation of model collapse</strong>: non-speakers wrote machine-translated articles, AI systems scraped them as ground truth, and the resulting models produce more bad translations that get posted back.</p><blockquote>If your training pipeline ingests web-scraped text, the majority of your input data is now synthetic — and some of it is deliberately planted disinformation.</blockquote><h3>Shannon Entropy: The Metric Your Monitoring Stack Is Missing</h3><p>A separate analysis makes the case that standard data quality checks — <strong>schema validation, row counts, null rates, freshness</strong> — can all pass while your pipeline is semantically broken. Shannon entropy (H = −Σ p(x) log₂ p(x)) measures the information content of a column's distribution. When a bad join collapses a categorical feature from 50 values to 3, row counts don't change, nulls don't spike, schema is intact — but entropy drops from <strong>~5.6 bits to ~1.6 bits</strong>.</p><p>This applies directly to synthetic contamination detection: as AI-generated content homogenizes your corpus, entropy drops. Track it over time and across transformation boundaries.</p><h3>The Propaganda Problem Is Different From the Synthetic Problem</h3><p>Standard perplexity-based filtering won't catch well-written propaganda — it reads like normal news. The DFRLab finding requires <strong>domain-level provenance tracking</strong>: maintaining blocklists of known propaganda outlets and their mirror domains. This is a different detection layer than synthetic content classifiers.</p><table><thead><tr><th>Contamination Type</th><th>Detection Method</th><th>Current Tooling</th></tr></thead><tbody><tr><td>AI-generated text</td><td>Perplexity scoring, stylometric analysis, watermark detection</td><td>Partial — high false positive rates</td></tr><tr><td>State propaganda</td><td>Domain-level blocklists, source provenance tracking</td><td>Manual — no automated pipelines</td></tr><tr><td>Machine-translated garbage</td><td>Cross-lingual quality scoring, native speaker verification</td><td>Minimal — mostly for high-resource languages</td></tr><tr><td>Distribution collapse</td><td>Shannon entropy monitoring, PSI, KL-divergence</td><td>Available but rarely deployed on text corpora</td></tr></tbody></table><h3>Where Sources Disagree</h3><p>The 71.7% figure from Ahrefs is <strong>methodologically opaque</strong> — how was AI content detected, what classifier threshold was used? Even if the true figure is 40-50%, the contamination problem is significant enough to affect fine-tuning quality. <em>The Wikipedia model-collapse case is more concrete but smaller-scale — it's happening visibly in low-resource languages and invisibly in high-resource ones.</em></p>

   Action items

   • Add Shannon entropy monitoring to your top-10 most critical feature columns this sprint — set alerts on drops >15% from a 14-day rolling baseline
   • Audit fine-tuning data and RAG corpora for Common Crawl-sourced content by end of month — implement domain/source filtering for known propaganda outlets
   • Implement a synthetic content detection layer in your data preprocessing pipeline this quarter — start with perplexity filtering, expand to ensemble detection

   Sources:Shannon entropy catches the silent data drift your schema checks miss · Your LLM pipeline just became a single point of failure · 71.7% of web is now AI-generated · Your LLM training data is eating itself

3. 03

   Your LLM Provider Is Silently Degrading — And Your Cloud Provider Is Starving Your GPUs

   <h3>The Opus 4.6 Degradation Signal</h3><p>Analysis of thousands of leaked Claude Code sessions suggests Opus 4.6's reasoning chain depth fell approximately <strong>67%</strong> compared to prior versions. Developers corroborate with reports of "lazier" code edits and are migrating to <strong>OpenAI Codex and GPT 5.4</strong>. Separately, a GitHub issue reports that on <strong>March 6, 2026</strong>, Claude Code's prompt cache TTL silently dropped from <strong>60 minutes to 5 minutes</strong> — a 12x reduction that can increase effective costs 5-12x for agentic loops.</p><p><em>Methodological caveats are substantial.</em> "Thinking depth" isn't a standardized metric. The sample selection is unclear. Anthropic may have adjusted system prompts or output length constraints rather than degrading the underlying model. <strong>But the behavioral signal is real</strong>: developers are building provider-abstraction layers specifically because quality degradation perception matters regardless of root cause.</p><h3>Compute Is Now Zero-Sum</h3><p>Three converging signals confirm that <strong>GPU allocation is a zero-sum game</strong> affecting your inference reliability:</p><ul><li><strong>Microsoft diverted GPUs from Azure to internal products</strong> (M365 Copilot, GitHub Copilot). Amy Hood stated Azure growth would have exceeded 40 if all GPUs had been allocated to Azure customers.</li><li><strong>Anthropic's revenue surged from $9B to $30B annualized</strong> in one quarter, driving multi-provider compute deals at gigawatt scale while the company is simultaneously compute-constrained.</li><li><strong>Lumentum's optical component orders are filled through 2028</strong> from major US tech companies, confirming the data center interconnect bottleneck is durable, not cyclical.</li></ul><blockquote>When your cloud provider's internal AI products compete with your workloads for GPUs, inference reliability becomes a new category of infrastructure risk.</blockquote><h3>The Open-Weight Parity Signal</h3><p>Multiple analyses report small open-weight models <strong>matching Anthropic's Mythos on domain-specific tasks</strong> like cybersecurity vulnerability discovery. AI capability is described as a "jagged frontier" — model performance varies wildly across sub-tasks. The competitive moat is shifting from model access to <strong>expert-led scaffolding systems</strong>. Meanwhile, Stanford's 2026 AI Index confirms Google, Anthropic, and OpenAI have <strong>fully stopped disclosing</strong> dataset sizes, training duration, and training code.</p><h3>Where Sources Converge</h3><p>Every source pointing to provider unreliability converges on the same recommendation: <strong>multi-provider inference routing</strong> with automated quality monitoring. The question isn't whether to build this abstraction layer — it's whether you can afford the technical debt of not having it when the next silent degradation hits.</p>

   Action items

   • Implement continuous eval suites that run daily against your LLM provider — track reasoning chain length, task completion rates, and output distribution statistics against a frozen baseline
   • Build a provider-agnostic inference routing layer supporting Claude, GPT 5.4, and at least one open-weight model with configuration-level switching
   • Run a head-to-head eval of the best open-weight alternative against your top-5 inference cost centers by monthly spend
   • Audit Anthropic API costs for agentic workflows — compare pre- and post-March 6, 2026 metrics to quantify the cache TTL impact

   Sources:Your LLM pipeline just became a single point of failure · Open-weight models match Anthropic Mythos on cybersecurity · Compute is now zero-sum · Your GPU capacity planning just got harder · Your model evaluations just got harder

4. 04

   APT41 Is Hunting Your ML Clusters — Plus Marimo RCE and Compromised LLM Routers

   <h3>APT41's Zero-Detection Backdoor Targets Cloud ML Infrastructure</h3><p>APT41/Winnti deployed a stripped ELF binary with <strong>0/72 VirusTotal detection rate</strong> that specifically harvests cloud IAM and managed-identity credentials. The kill chain is targeted at cloud-native ML infrastructure:</p><ol><li>Queries <strong>cloud metadata APIs</strong> (IMDSv1 on AWS, equivalent on GCP/Azure/Alibaba) for IAM role credentials</li><li><strong>AES-256 encrypts</strong> harvested credentials</li><li>Exfiltrates over <strong>SMTP port 25</strong> to C2 infrastructure</li><li>Achieves lateral movement via <strong>UDP broadcast to 255.255.255.255:6006</strong></li></ol><p>The choice of <strong>port 6006 is TensorBoard's default port</strong>. Whether this is deliberate camouflage targeting ML infrastructure or coincidence, your ML monitoring traffic and APT41 lateral movement would be <strong>indistinguishable</strong> without deep packet inspection.</p><h3>Marimo Notebook: Pre-Auth RCE Under Active Exploitation</h3><p>CVE-2026-39987 is a pre-authentication RCE in the Marimo notebook platform. The WebSocket endpoint exposes an interactive terminal <strong>without authentication</strong> — connect and you have a shell. Sysdig confirmed <strong>active exploitation within 12 hours</strong> of disclosure. Any network-reachable Marimo instance is immediately exploitable. The 12-hour window means traditional weekly patching cadences are inadequate.</p><h3>Your LLM Inference Supply Chain Is Compromised</h3><p>Researchers built <strong>Mine</strong>, a proxy simulating attacks on LLM API routers, and found <strong>9 routers (1 paid, 8 free) actively injecting malicious code</strong> into LLM responses. Attack vectors include payload injection and secret exfiltration. If you route LLM calls through any third-party proxy or router for cost optimization, every inference call is an <strong>unvalidated trust boundary</strong>. Compromised outputs in labeling, extraction, or classification pipelines propagate silently downstream.</p><p>Compounding this: security scanning tools themselves — <strong>Trivy, Xygeni, and KICs</strong> — were all compromised, with shared C2 servers linking the Xygeni compromise to a proxy botnet. If your CI/CD pipeline runs <code>trivy image scan</code>, the scanner may have been the threat.</p><table><thead><tr><th>Threat</th><th>Vector</th><th>ML Impact</th><th>Detection</th></tr></thead><tbody><tr><td>APT41 IAM harvester</td><td>Cloud metadata API (IMDSv1)</td><td>Training data, model weights, secrets</td><td>0/72 AV; monitor UDP 6006</td></tr><tr><td>Marimo RCE</td><td>Unauthenticated WebSocket</td><td>Full shell on notebook server</td><td>Network scan for exposed instances</td></tr><tr><td>LLM router injection</td><td>Third-party API proxies</td><td>Corrupted inference outputs</td><td>Canary comparison vs. direct API</td></tr><tr><td>Scanner compromise</td><td>Trivy/Xygeni packages</td><td>CI/CD pipeline trust chain</td><td>Pinned versions with checksums</td></tr></tbody></table>

   Action items

   • Patch or kill all Marimo notebook instances immediately — version 0.20.4 has pre-auth RCE under active exploitation
   • Enforce IMDSv2 on all AWS instances running ML workloads today, and audit equivalent metadata API protections on GCP/Azure
   • Block or alert on UDP broadcast traffic on port 6006 across all ML infrastructure this week
   • Implement a canary system for LLM inference: send identical prompts through your routed path and directly to the API, compare outputs for divergence
   • Pin CI/CD security scanner versions to content hashes, not version tags, and verify Trivy installations against known-good checksums

   Sources:Your Marimo notebooks have pre-auth RCE · Your Marimo notebooks just got a pre-auth RCE · Your multi-agent pipeline has 3 unsolved failure modes · Your CI/CD pipeline's attack surface just expanded · Anthropic's Mythos model autonomously found & exploited zero-days