feat: allow setting ollama host by yingjiehe-xyz · Pull Request #874 · block/goose

yingjiehe-xyz · 2025-01-29T03:07:56Z

Allow setting ollama host during configuration

Test step:

Run launchctl setenv OLLAMA_HOST "0.0.0.0:11111" on mac and start run ollama
Check ollama ls: OLLAMA_HOST=0.0.0.0:11111 ollama ls
Verify the ollama host by sending request:

curl http://localhost:11111/api/generate -d '{
 "model": "qwen2.5",
 "prompt":"Why is the sky blue?"
}'
{"model":"qwen2.5","created_at":"2025-01-29T02:53:09.321698Z","response":"The","done":false}
{"model":"qwen2.5","created_at":"2025-01-29T02:53:09.337099Z","response":" sky","done":false}
{"model":"qwen2.5","created_at":"2025-01-29T02:53:09.352554Z","response":" appears","done":false}
{"model":"qwen2.5","created_at":"2025-01-29T02:53:09.368318Z","response":" blue","done":false}

Run the UI can test with host change
Run goose configure CLI:

┌   goose-configure 
│
◇  What would you like to configure?
│  Configure Providers 
│
◇  Which model provider should we use?
│  Ollama 
│
●  OLLAMA_HOST is already configured
│  
◇  Would you like to update this value?
│  Yes 
│
◇  Enter new value for OLLAMA_HOST
│  http://localhost:11111
│
◆  Enter a model from that provider:
│  qwen2.5 (default)
└

Verify the config yaml file:

extensions:
  developer:
    enabled: true
    name: developer
    type: builtin
default_model: claude-3-5-sonnet-latest
GOOSE_MODEL: llama3.2
OLLAMA_HOST: http://localhost:11111
default_provider: anthropic
DATABRICKS_HOST: https://block-lakehouse-production.cloud.databricks.com
GOOSE_PROVIDER: ollama
/Users/yingjiehe/.config/goose/config.yaml (END)

michaelneale · 2025-01-29T03:10:28Z

@yingjiehe-xyz looks good - but that implies that you have to set it each time? can you jsut have it default and press submit?

angiejones · 2025-01-29T03:16:19Z

@acekyd can you update the docs once this is merged?

yingjiehe-xyz · 2025-01-29T04:10:19Z

@yingjiehe-xyz looks good - but that implies that you have to set it each time? can you jsut have it default and press submit?

no, we only need to set it once and it will be updated in the config.yaml

crates/goose-server/src/routes/secrets.rs

salman1993 · 2025-01-29T12:10:11Z

ui/desktop/src/components/settings/providers/ConfigureProvidersGrid.tsx

        body: JSON.stringify({
          key: keyName,
          value: apiKey.trim(),
+          isSecret: isSecretKey(keyName),


@alexhancock @lily-de i think it'd be good to have a secret toggle when setting up the provider - we can probably infer and autofill based on the name (key, token, password, etc)

Yes, we could do this.

Since we're talking here:

On first look, I have mixed feelings about the API design. The /secrets/store endpoint having a boolean param for isSecret is a bit confusing.

What do you both think about renaming the endpoint if we want to store things that are not secret/sensitive sometimes? @yingjiehe-xyz @salman1993

Maybe something like /value/store or /config/store? Those are just the first things that come to mind - likely better options.

yes, I agree on that, how about /config/store? it sounds good. Then we may have one more followup question, do we still want to have the API and handler in secrets.ts file?

I would like to rename the secrets.ts to something like config.ts

yeah we can do /config/store and also rename file to config.ts

synced with @salman1993 offline, let's go with /config/store and rename secrets.ts to something like config.ts

@alexhancock can we do the renaming in a separate refactor PR? i think it'll be easier to review

yeah that's fine

salman1993 · 2025-01-29T12:29:34Z

we can add .with_max_tokens(10) to make provider test faster

goose/crates/goose-cli/src/commands/configure.rs

Line 269 in fc83a8f

let model_config = goose::model::ModelConfig::new(model.clone());

before (took about 10s cause it generated lot of tokens)

much quicker after

salman1993 · 2025-01-29T12:40:52Z

lgtm! tested this out both in the UI and the CLI. added some minor comments. let's also get a review from one of the UI folks

yingjiehe-xyz · 2025-01-29T17:10:03Z

we can add .with_max_tokens(10) to make provider test faster

goose/crates/goose-cli/src/commands/configure.rs

Line 269 in fc83a8f

let model_config = goose::model::ModelConfig::new(model.clone());

before (took about 10s cause it generated lot of tokens)

much quicker after

Thanks, done

lily-de · 2025-01-29T18:41:16Z

documentation/docs/getting-started/providers.md

+◇  Provider Ollama requires OLLAMA_HOST, please enter a value
+│  http://localhost:11434
+│


lowkey just a bit confused because i definitely have ran ollama models without this parameter being set. if the user doesn't set this, can they still use the ollama provider or will the UI lock them out from doing so because of how things currently work?

We have the default value, the config should be empty for the current users, and then the default value will be populated:

goose/crates/goose/src/providers/ollama.rs

Line 40 in 24aaa0c

.unwrap_or_else(|_| OLLAMA_HOST.to_string());

lily-de · 2025-01-29T18:52:46Z

Small things that need to be updated:

The tooltip for the little (!) people see if Ollama is not configured is not technically correct anymore since it's no longer that it just needs to be installed and running, they can also enter a host url -- code here. "Ollama app must be installed on your machine and open."
Users have no way of updating or deleting that host url in the UI due to how we were treating ollama as a special case. If you put in your host url first, you still see the (+) button on the configure screen
Old thing I noticed but applies to Ollama and Databricks -- adding host url gives you this toast: Successfully added API key for Ollama --> should be host
I think this tooltip is now sort of messed up too -- like if you have a key stored for ollama it will say it's "installed an running on your machine" when you hover over the green check

yingjiehe-xyz · 2025-01-29T19:16:44Z

Small things that need to be updated:

The tooltip for the little (!) people see if Ollama is not configured is not technically correct anymore since it's no longer that it just needs to be installed and running, they can also enter a host url -- code here. "Ollama app must be installed on your machine and open."

Users have no way of updating or deleting that host url in the UI due to how we were treating ollama as a special case. If you put in your host url first, you still see the (+) button on the configure screen

Old thing I noticed but applies to Ollama and Databricks -- adding host url gives you this toast: Successfully added API key for Ollama --> should be host

I think this tooltip is now sort of messed up too -- like if you have a key stored for ollama it will say it's "installed an running on your machine" when you hover over the green check

Thanks for the call! Synced offline, let's do it in the following PR

allow setting ollama host

69948d8

yingjiehe-xyz requested review from angiejones and michaelneale January 29, 2025 03:07

yingjiehe-xyz marked this pull request as ready for review January 29, 2025 03:17

fix non secret issue

6b075a0

yingjiehe-xyz added 2 commits January 28, 2025 20:13

update

8112d6f

fix

0dc6f15

sammcj mentioned this pull request Jan 29, 2025

Allow remote Ollama #844

Closed

salman1993 reviewed Jan 29, 2025

View reviewed changes

crates/goose-server/src/routes/secrets.rs Outdated Show resolved Hide resolved

salman1993 reviewed Jan 29, 2025

View reviewed changes

salman1993 approved these changes Jan 29, 2025

View reviewed changes

salman1993 requested review from alexhancock and lily-de January 29, 2025 12:41

resolve comment

24aaa0c

lily-de reviewed Jan 29, 2025

View reviewed changes

small change for toast message

54e23e1

lily-de approved these changes Jan 29, 2025

View reviewed changes

yingjiehe-xyz merged commit 129e4a9 into main Jan 29, 2025

yingjiehe-xyz deleted the yingjiehe/ollama branch January 29, 2025 19:34

yingjiehe-xyz linked an issue Jan 30, 2025 that may be closed by this pull request

Allow custom Ollama #846

Closed

ahau-square pushed a commit that referenced this pull request May 2, 2025

feat: allow setting ollama host (#874)

837f582

cbruyndoncx pushed a commit to cbruyndoncx/goose that referenced this pull request Jul 20, 2025

feat: allow setting ollama host (block#874)

3dfe492

Conversation

yingjiehe-xyz commented Jan 29, 2025 • edited by salman1993 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michaelneale commented Jan 29, 2025

Uh oh!

angiejones commented Jan 29, 2025

Uh oh!

yingjiehe-xyz commented Jan 29, 2025

Uh oh!

Uh oh!

salman1993 Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yingjiehe-xyz Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

salman1993 Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

salman1993 commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

salman1993 commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yingjiehe-xyz commented Jan 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lily-de commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yingjiehe-xyz commented Jan 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

yingjiehe-xyz commented Jan 29, 2025 •

edited by salman1993

Loading

salman1993 Jan 29, 2025 •

edited

Loading

yingjiehe-xyz Jan 29, 2025 •

edited

Loading

salman1993 Jan 29, 2025 •

edited

Loading

salman1993 commented Jan 29, 2025 •

edited

Loading

salman1993 commented Jan 29, 2025 •

edited

Loading

lily-de commented Jan 29, 2025 •

edited

Loading