How Baseten achieved 2x faster inference with NVIDIA Dynamo

{
  "signal_id": "f507d183-1a8e-42e6-8207-996357f3bd7b",
  "signal_type": "news",
  "signal_subtype": "partners_with",
  "detected_at": "2025-10-17T00:00:00+00:00",
  "company": {
    "name": "Baseten",
    "domain": "baseten.co"
  },
  "data": {
    "url": "https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo",
    "title": "How Baseten achieved 2x faster inference with NVIDIA Dynamo",
    "excerpt": "How Baseten achieved 2x faster inference with NVIDIA Dynamo.\n\nAt Baseten, BaseTen Labs, Inc. collaborate closely with NVIDIA to push the boundaries of model performance. When NVIDIA releases new tooling, its model performance team immediately starts testing it out, measuring the potential gains against its current stack and battle-hardening new features for production.\n\nOften, NVIDIA releases updates as a result of this work: its engineers submit pull requests to their open-source GitHub repositories, making things more robust and secure for production use cases. This symbiosis is what brought BaseTen Labs, Inc. to quickly adopt NVIDIA Dynamo, NVIDIA's newest open-source inference framework.\n\nHow Baseten uses NVIDIA Dynamo.\n\nNVIDIA Dynamo is built for large-scale LLM serving across distributed GPU clusters with high throughput and low latency. It includes features like disaggregated prefill and decode steps, KV cache-aware routing, KV cache-offload to storage, an SLA-based planner for autoscaling, and dynamic GPU scheduling. Across all models, BaseTen Labs, Inc. has seen huge performance improvements by using Dynamo's KV cache-aware routing - those benefits are what this blog focuses on.\n\nThe KV cache stores a model's previously computed key/value states for past tokens, so it can reuse them instead of recomputing them with each new request. This speeds up inference...",
    "summary": "BaseTen Labs, Inc. partners with NVIDIA Corporation.",
    "category": "partners_with",
    "found_at": "2025-10-17T00:00:00Z",
    "location": "San Francisco, California, 94105, United States",
    "planning": false,
    "image_url": "https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/opengraph-image?acf396358e7a730c",
    "confidence": 0.7375,
    "published_at": "2025-10-17T00:00:00Z",
    "location_data": [
      {
        "city": "San Francisco",
        "state": "California",
        "country": "United States",
        "zip_code": "94105",
        "fuzzy_match": false
      }
    ],
    "article_sentence": "At Baseten, BaseTen Labs, Inc. collaborate closely with NVIDIA to push the boundaries of model performance."
  }
}

curl https://signals.autobound.ai/v1/signals/f507d183-1a8e-42e6-8207-996357f3bd7b \
  -H "X-API-KEY: $AUTOBOUND_API_KEY"

curl -X POST https://signals.autobound.ai/v1/companies/enrich \
  -H "X-API-KEY: $AUTOBOUND_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain":"baseten.co","limit":20}'