Setting up Ollama for LLM Development¶

Ollama allows you to run large language models locally, which is useful for development and testing HANI's LLM features without requiring API keys.

Installing Ollama¶

macOS¶

brew install ollama

Or download from ollama.ai

Linux¶

curl -fsSL https://ollama.ai/install.sh | sh

Windows¶

Download the installer from ollama.ai

Starting Ollama¶

Start the Ollama server:

ollama serve

This starts the server at http://localhost:11434.

Pulling Models¶

Download a model to use with HANI:

# Recommended for development (small and fast)
ollama pull llama3.2

# Larger models for better quality
ollama pull llama3.1
ollama pull mistral
ollama pull codellama

Configuring HANI for Ollama¶

Edit ~/negmas/hani/settings/llm_settings.json:

{
    "provider": "ollama",
    "model": "llama3.2",
    "base_url": "http://localhost:11434",
    "temperature": 0.7
}

Configuration Options¶

Setting	Description	Default
`provider`	LLM provider (`ollama`, `openai`, `anthropic`)	`openai`
`model`	Model name	`gpt-4o-mini`
`base_url`	API endpoint URL	Provider default
`temperature`	Response randomness (0.0-2.0)	`0.7`
`api_key`	API key (not needed for Ollama)	-

Testing the Configuration¶

Start Ollama:
```
ollama serve
```
Start HANI:
```
hani
```
Log in and start a negotiation
Use the Response Generator tool to test LLM integration:
Enter an instruction like "Propose a fair offer"
Click Generate Response
The generated text and outcome should appear

Recommended Models¶

Model	Size	Speed	Quality	Best For
`llama3.2`	3B	Fast	Good	Development, testing
`llama3.1`	8B	Medium	Better	General use
`mistral`	7B	Medium	Good	Balanced performance
`llama3.1:70b`	70B	Slow	Excellent	Production quality

Troubleshooting¶

Connection Refused¶

If you see "Connection refused" errors:

Ensure Ollama is running:
```
ollama serve
```
Check the server is accessible:
```
curl http://localhost:11434/api/tags
```

Model Not Found¶

If the model isn't found:

List available models:
```
ollama list
```
Pull the model if needed:
```
ollama pull llama3.2
```

Slow Responses¶

For faster responses:

Use a smaller model (llama3.2 instead of llama3.1:70b)
Reduce the context window in LLM settings
Ensure no other heavy processes are running

Using OpenAI or Anthropic Instead¶

If you prefer cloud-based models:

OpenAI¶

{
    "provider": "openai",
    "model": "gpt-4o-mini",
    "api_key": "sk-..."
}

Or set the environment variable:

export OPENAI_API_KEY="sk-..."

Anthropic¶

{
    "provider": "anthropic",
    "model": "claude-3-haiku-20240307",
    "api_key": "sk-ant-..."
}

Or set the environment variable:

export ANTHROPIC_API_KEY="sk-ant-..."

Admin LLM Settings¶

Administrators can configure LLM settings directly in the HANI interface:

Log in as admin
Open the sidebar
Navigate to LLM Settings
Configure provider, model, and prompts

Changes are saved to llm_settings.json and take effect immediately.