Most providers now have an API for counting tokens, i.e. submit the full request as you otherwise would, but get back a token count rather than an actual response. We should consider adding an interface for this purpose, e.g.
Task<int> CountTokensAsync(IEnumerable<ChatMessage>, ChatOptions, CancellationToken);
It could be implemented by IChatClients that support it and served up from their GetService implementations.
Most providers now have an API for counting tokens, i.e. submit the full request as you otherwise would, but get back a token count rather than an actual response. We should consider adding an interface for this purpose, e.g.
It could be implemented by IChatClients that support it and served up from their GetService implementations.