config.toml ignored ?

[VictorFoxSub]

I've set up up config.toml as specified in tabby's ollama http api reference , I've ollama run the 3 models, emptied the endpoint url in my pycharm setting (has the ui says to do that to use config.toml), but I still get "Cannot connect to Tabby server, please check your settings."

After activating the debug logs I found the following logs showing that it ignores my config.toml (all the ports are 11434) :

{"level":20,"time":1737987490917,"pid":5838,"hostname":"myvHost.local","tag":"TabbyApiClient","msg":"Health check request: GET http://localhost:8080/v1/health. [...]"}

BUT it is not totaly ignored as I activated the log in the config.toml...

What should I do to allow tabby to use the models I specified in config.toml ?

[Kingdivers1]

Dear @VictorFoxSub
I hope this email finds you well. I am writing to seek assistance regarding an issue I am facing while trying to configure and run multiple models on the Tabby server. I have diligently followed the instructions outlined in the Ollama HTTP API reference guide to set up my config.toml file but unfortunately, I am encountering an error message stating, "Cannot connect to Tabby server, please check your settings."

To provide you with a comprehensive overview, I have successfully configured the config.toml file as per the specifications mentioned in Tabby's Ollama HTTP API reference documentation. Subsequently, I have initiated the three models using the Ollama run command. Additionally, I have emptied the endpoint URL field in my PyCharm settings as instructed by the UI to leverage the functionalities of the config.toml file.

However, upon further investigation and enabling the debug logs, I discovered that the Tabby server is seemingly disregarding my config.toml configuration, evidenced by the fact that all ports remain static at 11434. A snippet of the logs indicating the issue is as follows:

{"level":20,"time":1737987490917,"pid":5838,"hostname":"myvHost.local","tag":"TabbyApiClient","msg":"Health check request: GET http://localhost:8080/v1/health. [...]"}

It is intriguing to note that although the config file is being read to some extent (as I have successfully activated logging within the config.toml file), the desired models specified within the configuration file are not being utilized by the Tabby server.

I would greatly appreciate your guidance on resolving this matter and enabling the Tabby server to recognize and deploy the models as detailed in the config.toml file. Your expertise in troubleshooting this issue would be invaluable in allowing me to leverage the full capabilities of the Tabby server effectively.

Thank you for your time and attention to this matter. I eagerly await your insightful suggestions and recommendations to rectify this situation promptly.

Warm regards,

King Divers

[fspv]

same here. Can't get it working, see absolutely nothing in the server logs related to this

[fspv]

update: turned out in my case it was a typo in the volume mount, so config was not found. If you're using docker try docker exec -it tabbycontainer sh and see if the config is actually there.

I'd like so much if tabby server was more verbose and told me that the config is not found. Would've saved me tons of time

[VictorFoxSub]

I think I don't understand how tabby works with ollama. I still need to run tabby serve ?

I tried to run it with or without specifiying --model and --chat-model using the values from the config.toml but without success.