Package com.sap.ai.sdk.orchestration

# Class OrchestrationClient

java.lang.Object
com.sap.ai.sdk.orchestration.OrchestrationClient

public class OrchestrationClient extends Object
Client to execute requests to the orchestration service.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `OrchestrationClient()`
  Default constructor.
  `OrchestrationClient(com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination destination)`
  Constructor with a custom destination, allowing for a custom resource group or otherwise custom destination.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `OrchestrationChatResponse`
  `chatCompletion(OrchestrationPrompt prompt, OrchestrationModuleConfig config, OrchestrationModuleConfig... fallbackConfigs)`
  Generate a completion for the given prompt.
  `OrchestrationChatResponse`
  `chatCompletionUsingReference(OrchestrationConfigReference reference)`
  Generate a completion using a referenced Orchestration config.
  `EmbeddingsPostResponse`
  `embed(EmbeddingsPostRequest request)`
  Generates embeddings using the low-level API request.
  `OrchestrationEmbeddingResponse`
  `embed(OrchestrationEmbeddingRequest request)`
  Generate embeddings for a `OrchestrationEmbeddingRequest` request.
  `CompletionPostResponse`
  `executeRequest(CompletionPostRequest request)`
  Serializes the given request, executes it and deserializes the response.
  `OrchestrationChatResponse`
  `executeRequestFromJsonModuleConfig(OrchestrationPrompt prompt, String moduleConfig)`
  Perform a request to the orchestration service using a module configuration provided as JSON string.
  `Stream<String>`
  `streamChatCompletion(OrchestrationPrompt prompt, OrchestrationModuleConfig config, OrchestrationModuleConfig... fallbackConfigs)`
  Generate a completion for the given prompt.
  `Stream<OrchestrationChatCompletionDelta>`
  `streamChatCompletionDeltas(CompletionRequestConfiguration request)`
  Generate a completion for the given prompt.
  `static CompletionRequestConfiguration`
  `toCompletionPostRequest(OrchestrationPrompt prompt, OrchestrationModuleConfig config, OrchestrationModuleConfig... fallbackConfigs)`
  Convert the given prompt and config into a low-level request data object.
  `OrchestrationClient`
  `withHeader(String key, String value)`
  Create a new orchestration client with a custom header added to every call made with this client
  `OrchestrationClient`
  `withResourceGroup(String resourceGroup, String scenario)`
  Create a new orchestration client for the given resource group and scenario.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### OrchestrationClient

    public OrchestrationClient()
    Default constructor.

  - ### OrchestrationClient

    public OrchestrationClient(@Nonnull com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination destination)
    Constructor with a custom destination, allowing for a custom resource group or otherwise custom destination. The destination needs to be configured with a URL pointing to an orchestration service deployment. Typically, such a destination should be obtained using `AiCoreService.getInferenceDestination(String)`.

    Example:

         new OrchestrationClient(new AiCoreService().getInferenceDestination("custom-rg").forScenario("orchestration"));

    Parameters:
      `destination` - The specific `HttpDestination` to use.
    See Also:
      - `AiCoreService.getInferenceDestination(String)`

- ## Method Details

  - ### toCompletionPostRequest

    @Nonnull public static CompletionRequestConfiguration toCompletionPostRequest(@Nonnull OrchestrationPrompt prompt, @Nonnull OrchestrationModuleConfig config, @Nonnull OrchestrationModuleConfig... fallbackConfigs)
    Convert the given prompt and config into a low-level request data object. The data object allows for further customization before sending the request.
    Parameters:
      `prompt` - The `OrchestrationPrompt` to generate a completion for.
      `config` - The `OrchestrationConfig` configuration to use for the completion.
      `fallbackConfigs` - Fallback configurations to use.
    Returns:
      The low-level request data object to send to orchestration.

  - ### chatCompletion

    @Nonnull public OrchestrationChatResponse chatCompletion(@Nonnull OrchestrationPrompt prompt, @Nonnull OrchestrationModuleConfig config, @Nonnull OrchestrationModuleConfig... fallbackConfigs) throws OrchestrationClientException
    Generate a completion for the given prompt.
    Parameters:
      `prompt` - The `OrchestrationPrompt` to send to orchestration.
      `config` - the configuration to use
      `fallbackConfigs` - fallback configurations
    Returns:
      the completion output
    Throws:
      `OrchestrationClientException` - if the request fails.

  - ### streamChatCompletion

    @Nonnull public Stream<String> streamChatCompletion(@Nonnull OrchestrationPrompt prompt, @Nonnull OrchestrationModuleConfig config, @Nonnull OrchestrationModuleConfig... fallbackConfigs) throws OrchestrationClientException
    Generate a completion for the given prompt.
    Parameters:
      `prompt` - a text message.
      `config` - the configuration to use
      `fallbackConfigs` - fallback configurations
    Returns:
      a stream of message deltas
    Throws:
      `OrchestrationClientException` - if the request fails or if the finish reason is content_filter
    Since:
      1.1.0

  - ### executeRequest

    @Nonnull public CompletionPostResponse executeRequest(@Nonnull CompletionPostRequest request) throws OrchestrationClientException
    Serializes the given request, executes it and deserializes the response.

    Override this method to customize the request execution. For example, to modify the request object before it is sent, use:

         @Override
         protected CompletionPostResponse executeRequest(@Nonnull CompletionPostRequest request) {
           request.setCustomField("myField", "myValue");
           return super.executeRequest(request);
         }

    Alternatively, you can call this method directly with a fully custom request object.

    Parameters:
      `request` - The request data object to send to orchestration.
    Returns:
      The response data object from orchestration.
    Throws:
      `OrchestrationClientException` - If the request fails.

  - ### chatCompletionUsingReference

    @Nonnull public OrchestrationChatResponse chatCompletionUsingReference(@Nonnull OrchestrationConfigReference reference)
    Generate a completion using a referenced Orchestration config.
    Parameters:
      `reference` - A reference to an Orchestration config stored in prompt registry
    Returns:
      The completion output
    Since:
      1.15.0

  - ### executeRequestFromJsonModuleConfig

    @Beta @Nonnull public OrchestrationChatResponse executeRequestFromJsonModuleConfig(@Nonnull OrchestrationPrompt prompt, @Nonnull String moduleConfig) throws OrchestrationClientException
    Perform a request to the orchestration service using a module configuration provided as JSON string. This can be useful when building a configuration in the AI Launchpad UI and exporting it as JSON. Furthermore, this allows for using features that are not yet supported natively by the API.

    NOTE: This method does not support streaming.

    Parameters:
      `prompt` - The input parameters and optionally message history to use for prompt execution.
      `moduleConfig` - The module configuration in JSON format.
    Returns:
      The completion response.
    Throws:
      `OrchestrationClientException` - If the request fails.

  - ### streamChatCompletionDeltas

    @Nonnull public Stream<OrchestrationChatCompletionDelta> streamChatCompletionDeltas(@Nonnull CompletionRequestConfiguration request) throws OrchestrationClientException
    Generate a completion for the given prompt.
    Parameters:
      `request` - the prompt, including messages and other parameters.
    Returns:
      A stream of chat completion delta elements.
    Throws:
      `OrchestrationClientException` - if the request fails
    Since:
      1.1.0

  - ### embed

    @Nonnull public OrchestrationEmbeddingResponse embed(@Nonnull OrchestrationEmbeddingRequest request) throws OrchestrationClientException
    Generate embeddings for a `OrchestrationEmbeddingRequest` request.
    Parameters:
      `request` - the request containing the input text and other parameters.
    Returns:
      the response containing the embeddings.
    Throws:
      `OrchestrationClientException` - if the request fails
    Since:
      1.12.0

  - ### embed

    @Nonnull public EmbeddingsPostResponse embed(@Nonnull EmbeddingsPostRequest request) throws OrchestrationClientException
    Generates embeddings using the low-level API request.

    This method provides direct access to the underlying API for advanced use cases. For most scenarios, prefer `embed(OrchestrationEmbeddingRequest)`.

    Parameters:
      `request` - the low-level API request
    Returns:
      the low level response object
    Throws:
      `OrchestrationClientException` - if the request fails
    Since:
      1.12.0
    See Also:
      - `embed(OrchestrationEmbeddingRequest)`

  - ### withHeader

    @Nonnull public OrchestrationClient withHeader(@Nonnull String key, @Nonnull String value)
    Create a new orchestration client with a custom header added to every call made with this client
    Parameters:
      `key` - the key of the custom header to add
      `value` - the value of the custom header to add
    Returns:
      a new client.
    Since:
      1.11.0

  - ### withResourceGroup

    @Nonnull public OrchestrationClient withResourceGroup(@Nonnull String resourceGroup, @Nonnull String scenario)
    Create a new orchestration client for the given resource group and scenario.
    Parameters:
      `resourceGroup` - the resource group
      `scenario` - the scenario
    Returns:
      a new client configured with the resolved destination.