| Aspect | Ollama (Local) | OpenAI / Cloud API | |----------------------|---------------------------------------------|--------------------------------------------| | | Free (only hardware) | Pay per token; large teams can hit $200k/year | | Latency | 110–300 ms for typical code tasks | 800 ms+ due to network overhead | | Data privacy | Complete – no data leaves your servers | Your prompts are sent to a third party | | Model variety | Llama, Mistral, CodeLlama, DeepSeek, Gemma… | OpenAI’s own models only | | Scaling | Limited by your own hardware | Virtually unlimited via API | | Java integration | REST API / Spring AI / LangChain4j | Also REST API / Spring AI / LangChain4j |
HttpRequest request = HttpRequest.newBuilder() .uri(URI.create("http://localhost:11434/api/generate")) .header("Content-Type", "application/json") .POST(HttpRequest.BodyPublishers.ofString(jsonPayload)) .timeout(Duration.ofSeconds(60)) .build();
: Pass text chunks to Ollama’s embedding API to convert text into vector math arrays. ollamac java work
public AIService(ChatClient.Builder builder) this.chatClient = builder.build();
As of 2026, many local models support function calling. You can use this with Spring AI to allow your model to call Java functions, such as looking up data from a database or checking the weather. | Aspect | Ollama (Local) | OpenAI /
Add the dependency to your pom.xml :
String jsonPayload = """
Spring AI’s ChatModel.stream() returns a Flux<String> that you can directly expose via a WebFlux endpoint. The first token often arrives in less than 300 ms, which is barely perceptible to users.