Track custom costs

Track and manage costs for LLM operations in Weave

W&B Weave automatically calculates the cost of each LLM call by tracking token usage and applying the pricing for the model used, allowing you to monitor and analyze application spend directly in your traces and evaluations.

You can also track custom costs when you need different pricing, internal cost models, or costs for operations that Weave does not price automatically.

Adding a custom cost

Python

You can add a custom cost by using the add_cost method. The three required fields are llm_id, prompt_token_cost, and completion_token_cost. llm_id is the name of the LLM (e.g. gpt-4o). prompt_token_cost and completion_token_cost are cost per token for the LLM (if the LLM prices were specified inper million tokens, make sure to convert the value). You can also set effective_date to a datetime, to make the cost effective at a specific date, this defaults to the current date.

import weave
from datetime import datetime

client = weave.init("my_custom_cost_model")

client.add_cost(
    llm_id="your_model_name",
    prompt_token_cost=0.01,
    completion_token_cost=0.02
)

client.add_cost(
    llm_id="your_model_name",
    prompt_token_cost=10,
    completion_token_cost=20,
    # If for example I want to raise the price of the model after a certain date
    effective_date=datetime(2025, 4, 22),
)

TypeScript

This feature is not available in TypeScript yet.  Stay tuned!

Querying for costs

Python

You can query for costs by using the query_costs method. There are a few ways to query for costs, you can pass in a singular cost id, or a list of LLM model names.

import weave

client = weave.init("my_custom_cost_model")

costs = client.query_costs(llm_ids=["your_model_name"])

cost = client.query_costs(costs[0].id)

TypeScript

This feature is not available in TypeScript yet.  Stay tuned!

Purging a custom cost

Python

You can purge a custom cost by using the purge_costs method. You pass in a list of cost ids, and the costs with those ids are purged.

import weave

client = weave.init("my_custom_cost_model")

costs = client.query_costs(llm_ids=["your_model_name"])
client.purge_costs([cost.id for cost in costs])

TypeScript

This feature is not available in TypeScript yet.  Stay tuned!

Calculating costs for a Project

Python

You can calculate costs for a project by using our calls_query and adding include_costs=True with a little bit of setup.

import weave

weave.init("project_costs")
@weave.op()
def get_costs_for_project(project_name: str):
    total_cost = 0
    requests = 0

    client = weave.init(project_name)
    # Fetch all the calls in the project
    calls = list(
        client.get_calls(filter={"trace_roots_only": True}, include_costs=True)
    )

    for call in calls:
        # If the call has costs, we add them to the total cost
        if call.summary["weave"] is not None and call.summary["weave"].get("costs", None) is not None:
            for k, cost in call.summary["weave"]["costs"].items():
                requests += cost["requests"]
                total_cost += cost["prompt_tokens_total_cost"]
                total_cost += cost["completion_tokens_total_cost"]

    # We return the total cost, requests, and calls
    return {
        "total_cost": total_cost,
        "requests": requests,
        "calls": len(calls),
    }

# Since we decorated our function with @weave.op(),
# our totals are stored in weave for historic cost total calculations
get_costs_for_project("my_custom_cost_model")

TypeScript

This feature is not available in TypeScript yet.  Stay tuned!

Setting up a custom model with custom costs

Try our cookbook for Setting up costs with a custom model.

Try in Colab