Developers are spending $2,400+/year on cloud AI subscriptions when local models can now handle 80% of coding tasks, but there's no easy way to orchestrate the split.
A middleware layer that sits between your IDE/coding agent and model backends. It classifies tasks (spec generation, code review → cloud API; implementation, refactoring, tab completion → local model) and routes them automatically based on complexity and cost thresholds.
Freemium SaaS — free for basic routing rules, $15-29/month for smart routing with cost analytics and quality monitoring
Real but not hair-on-fire. The pain signals are genuine — developers ARE spending $2,400+/year and ARE noticing local models can handle simpler tasks. But most developers currently 'solve' this by just paying the subscription and not thinking about it. The pain is more 'slow bleed awareness' than 'I need this fixed TODAY.' Power users who manually split between local and cloud already exist (as the Reddit comments show), which validates the workflow but also shows the status quo is tolerable. Score would be 8+ if local models were truly production-equivalent and the ONLY barrier was orchestration.
Conservative estimate: 5M+ developers actively paying for AI coding tools globally. Target segment (cost-conscious solo devs and small shops spending $100-200/month) is maybe 500K-1M developers. At $20/month average, addressable market is $120M-240M/year. Not massive VC-scale but very healthy for a bootstrapped/small-team product. The ceiling is real though — enterprises will build this in-house or get it bundled in their existing tools.
This is the critical weakness. The value prop is SAVING money. Asking developers to pay $15-29/month to save on their $100-200/month AI spend means the tool needs to save $45-60+/month minimum to justify itself (3x ROI). That's achievable IF routing works well, but it's a tough psychological sell — paying for a tool that helps you pay less for other tools. The open-source risk is enormous here: LiteLLM, RouteLLM, and Continue.dev already provide free building blocks. A developer savvy enough to care about this optimization is savvy enough to wire up their own solution in a weekend. The free tier needs to be generous enough to prove ROI before conversion.
A solo dev can build a functional MVP in 6-8 weeks, but not 4. Core components: (1) proxy server with OpenAI-compatible API — straightforward with LiteLLM as a base, (2) task classifier — this is the hard part, needs a lightweight model or heuristic system to classify coding tasks by complexity, (3) routing rules engine — moderate complexity, (4) basic cost tracking dashboard — standard web dev. The task classification piece is genuinely hard to get RIGHT — misrouting a complex task to a local model that produces garbage code will destroy user trust instantly. The quality monitoring feedback loop (did the local model actually produce good code?) is a v2 feature but critical for long-term value.
The gap is clear and validated: nobody offers INTELLIGENT, AUTOMATIC routing between local and cloud models specifically for coding tasks with cost optimization. Continue.dev is closest but routing is manual. LiteLLM is a dumb proxy. RouteLLM is a research prototype. OpenRouter is cloud-only. Cursor routes internally but is a closed garden. The specific combination of (automatic task classification) + (local model support) + (cost analytics) + (coding-specific) does not exist as a product today. However, this gap could close quickly — Continue.dev adding smart routing or LiteLLM adding a task classifier would eat into the value prop significantly.
Natural subscription fit — the tool sits in the critical path of daily coding workflow and provides ongoing cost optimization + analytics. Once a developer's routing is configured and saving them money, switching costs are moderate (reconfiguring all their tools). The usage-based angle (track your savings over time) creates good retention hooks. Risk: if a developer sets up routing rules once and they 'just work,' what's the ongoing value? Need continuous optimization, new model support, and analytics to justify monthly payment. Could also explore a percentage-of-savings model (we take 10% of what we save you) which aligns incentives better.
- +Clear, validated market gap — intelligent local/cloud routing for coding tasks doesn't exist as a product despite growing demand
- +Strong timing — local model quality (Qwen 3.5, DeepSeek, Llama 4) just crossed the usability threshold for coding, making hybrid routing genuinely viable for the first time
- +Natural middleware positioning — sits between IDE and models, which means it works with ANY tool, not competing with Cursor/Continue but augmenting them
- +Built-in word-of-mouth — 'I cut my AI coding bill by 60%' is a compelling story developers share
- +Data moat potential — aggregate routing decisions across users to build the best task classifier over time
- !CRITICAL: Willingness-to-pay problem — target users are cost-conscious developers who are also technically capable of building a DIY solution with existing open-source tools (LiteLLM + simple heuristics). The 'build vs buy' calculus works against you
- !CRITICAL: Platform risk — if Cursor, Continue.dev, or any major IDE adds intelligent local model routing natively (likely within 12-18 months), the standalone middleware value proposition collapses overnight
- !Task misclassification is catastrophic — routing a complex architectural task to a 7B local model that produces garbage will make developers distrust the system and revert to cloud-only, destroying your core value prop
- !Local model setup friction — your target user needs Ollama or equivalent already running, with sufficient hardware (16GB+ RAM, ideally GPU). This narrows the addressable market and adds support burden
- !The savings math may not close — if a developer spends $200/month and you save 50% ($100), minus your $25/month fee, net savings is $75/month. That's good but not life-changing. And if cloud AI pricing drops (likely), savings shrink
Open-source IDE extension
Open-source unified proxy that provides an OpenAI-compatible API to 100+ LLM providers. Acts as a translation and load-balancing layer between your app and model backends.
Open-source research framework that trains router models to dynamically route between a strong and weak LLM based on query difficulty. Demonstrated 2x+ cost reduction with minimal quality loss on benchmarks.
Cloud API aggregator that provides unified access to 200+ models from multiple providers with a single API key. Includes basic model comparison and automatic fallback routing.
AI-native code editors that internally use different models for different features
VS Code extension + lightweight local proxy. Extension intercepts AI coding requests, classifies them using a fast heuristic classifier (regex patterns + token count + context analysis — NOT an ML model for v1), routes to either Ollama (local) or configured cloud API. Dashboard shows: requests today, % routed locally, estimated savings, quality scores (thumbs up/down on responses). Support Continue.dev and Cursor API key replacement as day-one integrations. Ship with sensible defaults: tab completion + simple refactoring → local, spec generation + complex debugging + code review → cloud. Let users override any routing decision to build training data for smarter classification later.
Free tier: basic routing with manual rules, up to 500 requests/day, basic cost tracking → Paid ($19/month): automatic smart routing, unlimited requests, detailed cost analytics, quality monitoring dashboard, custom routing rules, multi-model A/B testing → Team ($39/user/month): shared routing policies, team-wide cost analytics, admin controls, SSO → Revenue kicker: negotiate volume discounts with cloud API providers and pass through a margin (become an OpenRouter competitor from below)
8-12 weeks to MVP launch, 14-20 weeks to first paying customer. The long pole is not building the product — it's proving the routing quality is trustworthy enough that developers let it make decisions automatically. Expect a long free-tier-heavy period (2-4 months) where users test routing quality before converting. First $1K MRR likely at month 4-6 post-launch. Faster path: launch as open-source with a hosted/managed premium tier to build community trust first.
- “I could see myself switching finally to a viable hybrid model”
- “I use an API for the SOTA model to generate specs and do reviews and local models for all the work”
- “I've already spent 2000 on Claude”
- “$6800 in subscriptions”