Speculative Decoding on Dhruv Patel - Lead Android Engineerhttps://deardhruv.com/tags/speculative-decoding/Recent content in Speculative Decoding on Dhruv Patel - Lead Android EngineerHugoendhruv.time@gmail.com (Dhruv Patel)dhruv.time@gmail.com (Dhruv Patel)<a href="https://creativecommons.org/licenses/by-nc/4.0/" target="_blank" rel="noopener">CC BY-NC 4.0</a>Wed, 21 Oct 2026 18:04:07 +0530Part 6 - Pushing Mobile Hardware to the Edgehttps://deardhruv.com/posts/ai_playground_series/06-pushing-mobile-hardware-to-the-edge/Wed, 21 Oct 2026 15:30:00 +0530dhruv.time@gmail.com (Dhruv Patel)https://deardhruv.com/posts/ai_playground_series/06-pushing-mobile-hardware-to-the-edge/<h1 id="pushing-mobile-hardware-to-the-edge">Pushing Mobile Hardware to the Edge</h1> <h2 id="part-6---dual-model-speculative-decoding-on-demand-gpu-dynamic-delivery-isolated-driver-crashes-and-streaming-fault-tolerance">Part 6 - Dual-model speculative decoding, on-demand GPU dynamic delivery, isolated driver crashes, and streaming fault-tolerance.</h2> <p>When you run open-source large language models on desktop GPUs, latency is rarely the bottleneck that keeps you awake at night. You have 24 GB of VRAM, 300 watts of wall power, and active cooling fans.</p> <p>On a smartphone, you have:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">a shared SoC memory pool </span></span><span class="line"><span class="cl">thermal throttling within minutes </span></span><span class="line"><span class="cl">aggressive OS process killers (Android LMK) </span></span><span class="line"><span class="cl">fragmented GPU drivers </span></span><span class="line"><span class="cl">a strict battery budget </span></span></code></pre></div><p>In <a href="https://deardhruv.com/posts/ai_playground_series/03-the-model-was-fine-the-phone-wasnt/">Part 3</a>, we explored memory admission, quantization tradeoffs, and thermal guardrails. In <a href="https://deardhruv.com/posts/ai_playground_series/05-at-some-point-i-stopped-building-a-chat-app/">Part 5</a>, we established an MVI Clean Architecture to coordinate these moving parts.</p>