高性能AI包

genai

The opinionated high performance professional-grade AI package for Go.

genai is intentional. Curious why it was created? See the release announcement at maruel.ca/post/genai-v0.1.0.

Features

- Full functionality: Full access to each backend-specific functionality. Access the raw API if needed with full message schema as Go structs. - Tool calling via reflection: Tell the LLM to call a tool directly, described as a Go struct. No need to manually fiddle with JSON. - Native JSON struct serialization: Pass a struct to tell the LLM what to generate, decode the reply into your struct. No need to manually fiddle with JSON. Supports required fields, enums, descriptions, etc. You can still fiddle if you want to. :) - Streaming: Streams completion reply as the output is being generated, including thinking and tool calling, via go 1.23 iterators. - Multi-modal: Process images, PDFs and videos (!) as input or output. - Web Search: Search the web to answer your question and cite documents passed in. - Smoke testing friendly: record and play back API calls at HTTP level to save 💰 and keep tests fast and reproducible, via the exposed HTTP transport. See example. - Rate limits and usage: Parse the provider-specific HTTP headers and JSON response to get the tokens usage and remaining quota. - Provide access to HTTP headers to enable beta features.

Log probabilities

examples/txt\_to\_txt\_logprobs/main.go: List the alternative tokens that were considered during generation. This helps tune Temperature, TopP or TopK.

Try it locally:

go run github.com/maruel/genai/examples/txt_to_txt_logprobs@latest

When asked Tell a joke, this may print:

Provider huggingface
  Reply:
    Why don't scientists trust atoms?

    Because they make up everything!
  Logprobs:
    *    -0.000082: "Why"
         -9.625082: "Here"
        -11.250082: "What"
        -13.875082: "A"
        -14.500082: "How"
    *    -0.000003: " don"
        -14.125003: " do"
        -14.625003: " did"
        -14.625003: " dont"
        -14.875003: " didn"
    *    -0.000001: "'t"
        -14.000001: "’t"
        -18.062500: "'"
        -18.875000: "'T"
        -19.812500: "'s"
    *    -0.000002: " scientists"
        -14.250002: " Scientists"
        -14.250002: " eggs"
        -15.125002: " skeletons"
        -16.125002: " programmers"
    *    -0.000000: " trust"
        -16.250000: " trusts"
        -16.250000: " Trust"
        -17.250000: " like"
        -18.000000: " trusted"
    *    -0.000006: " atoms"
        -13.250006: "atoms"
        -13.500006: " stairs"
        -14.625006: " their"
        -15.000006: " electrons"
    *    -0.000011: "?\n\n"
        -12.125011: "?\n"
        -12.125011: "?"
        -14.750011: "？\n\n"
        -16.500011: " anymore"
(...)

Features

- [ ] Server-side MCP Client: OpenAI - [x] Anthropic raw API is implemented and smoke tested but there's no abstraction layer yet - [ ] Real-time / Live: Gemini, OpenAI, TogetherAI, ... - [ ] More comprehensive file/cache abstraction - [ ] Tokens counting: Anthropic, Cohere, Gemini, ... - [ ] Embeddings: Anthropic, Cohere, Gemini, OpenAI, TogetherAI, ... - [ ] Image to 3D, e.g. github.com/Tencent-Hunyuan/Hunyuan3D-2