fix(cli): say what to do when the inference server is unreachable by tsayin4 · Pull Request #3129 · roboflow/inference

tsayin4 · 2026-10-06T20:09:09Z

Fixes #314

I hit this myself before I understood what inference infer actually does. With nothing listening on
port 9001 you get a wall of urllib3 output, and nothing in it says that a server is required at all —
which is exactly what the issue describes.

Before:

Command failed. Cause: Error with server connection: HTTPConnectionPool(host='localhost', port=9001):
Max retries exceeded with url: /some-model/1?api_key=... (Caused by NewConnectionError('<urllib3.
connection.HTTPConnection object at 0x10c3f2e50>: Failed to establish a new connection: [Errno 61]
Connection refused'))

After:

Could not reach an inference server at http://localhost:9001.

`inference infer` sends requests to a server; it does not run the model in this
process. Start one locally with `inference server start` (needs Docker), or pass
--host to use a server you already have, for example
--host https://detect.roboflow.com for the hosted API.

The SDK already tells the two cases apart, so the change leans on that rather than inspecting the
message: HTTPCallErrorError means the server answered, so a 404 for a missing model still reports
the way it always did. Only the base HTTPClientError, which the SDK raises when the connection
itself fails, gets the extra text.

I mentioned the hosted API as well as server start, since someone who does not realise a server is
involved probably also does not know that the hosted one is an option.

container_adapter.py already does something similar for the server commands, so this is the same
idea applied to the one path that was missing it.

Tests in tests/inference_cli/unit_tests/test_main_infer_errors.py cover the unreachable case, a
server that answered (should keep the old message), and a custom --host appearing in the output.
226 passed in tests/inference_cli/unit_tests — the three collection errors there are missing
optional deps (torch and friends) and happen on main too.

Happy to adjust the wording if you would rather it were shorter or pointed somewhere else.

Summary by CodeRabbit

Bug Fixes
- Inference connection failures are now distinguished from other HTTP client errors. The CLI reports the connection issue and provides guidance for starting a local server or specifying an existing host, including the configured host.
- Server response and API-key errors continue to use their existing error handling without showing server-reachability guidance. Connection failures exit with a failure status.

With nothing listening on port 9001, `inference infer` printed the raw urllib3 connection error, which never mentions that a server is needed at all. The SDK already separates the cases: HTTPCallErrorError means the server answered, so a 404 for a missing model still reports as before. Only the base HTTPClientError gets the extra text, pointing at `inference server start`, --host, and the hosted API. Fixes roboflow#314

CLAassistant · 2026-10-06T20:09:27Z

All committers have signed the CLA.

coderabbitai · 2026-10-06T20:09:35Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: defaults
Review profile: CHILL
Plan: Advanced
Run ID: 2ca045ab-163f-4aa9-a25a-74dfebfb22f7

📥 Commits

Reviewing files that changed from the base of the PR and between f0467e9 and c6800d8.

📒 Files selected for processing: 5

inference_cli/main.py
inference_sdk/http/client.py
inference_sdk/http/errors.py
tests/inference_cli/unit_tests/test_main_infer_errors.py
tests/inference_sdk/unit_tests/http/test_client.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

The SDK now raises HTTPServerConnectionError for synchronous and asynchronous connection failures. The CLI reports the connection error and server-start guidance, then exits with code 1. Tests cover default and custom hosts, plus response and API-key errors.

Changes: Inference server connection errors

Layer / File(s)	Summary
Classify connection failures `inference_sdk/http/errors.py`, `inference_sdk/http/client.py`, `tests/inference_sdk/unit_tests/http/test_client.py`	The SDK adds `HTTPServerConnectionError` as an `HTTPClientError` subclass. Synchronous and asynchronous connection failures now raise this exception while retaining the existing message and chained cause. Tests expect the new exception.
Report connection guidance `inference_cli/main.py`, `tests/inference_cli/unit_tests/test_main_infer_errors.py`	The CLI prints the connection error and server-start guidance for `HTTPServerConnectionError`, then exits with code 1. Tests check the default and configured hosts, and verify that HTTP response and API-key errors do not produce connection guidance.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Low

Merge Risk: ⚪ Minimal · up to c6800

The CLI provides server-start guidance for connection failures while retaining the existing treatment of server errors. No issue identified here prevents merging after normal checks.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: the CLI now provides guidance when the inference server is unreachable.
Linked Issues check		Issue `#314` requires a clear `inference infer` message when the local inference server is unreachable. `inference_cli/main.py` catches `HTTPServerConnectionError`, reports the configured host, explains…
Out of Scope Changes check		The new SDK exception and its synchronous and asynchronous connection-error mapping support the CLI behavior required by issue `#314`. The CLI tests and SDK tests verify the new classification and prese…

Full details: Linked Issues check

Explanation

Issue #314 requires a clear inference infer message when the local inference server is unreachable. inference_cli/main.py catches HTTPServerConnectionError, reports the configured host, explains that inference infer requires a server, and suggests inference server start with Docker or Podman. It also gives a hosted --host example. inference_sdk/http/client.py raises this error for connection failures, while response errors remain HTTPCallErrorError. Focused tests cover unreachable servers, responding servers, API-key errors, and custom hosts.

Full details: Out of Scope Changes check

Explanation

The new SDK exception and its synchronous and asynchronous connection-error mapping support the CLI behavior required by issue #314. The CLI tests and SDK tests verify the new classification and preserve existing response-error handling. No unrelated changes are evidenced.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

tsayin4 · 2026-10-09T15:25:12Z

CLA is signed now, so this is unblocked - checks are green and it is ready whenever someone has a moment.

Short recap of what it does: when the inference server is unreachable, the CLI surfaced the raw connection exception, which reads as a bug in the user's own code rather than "nothing is listening on that address". It now says what happened and what to do about it. Test covers the unreachable path.

dkosowski87

Hey @tsayin4 thanks for taking this, I left a couple of comments.

dkosowski87 · 2026-10-22T19:21:19Z

+        # The server answered, so the request or the model is the problem, not reachability.
+        typer.echo(f"Command failed. Cause: {error}")
+        raise typer.Exit(code=1)
+    except HTTPClientError:


This is too broad of a catch. If you look at its children, it also covers missing API keys, invalid model IDs, unsupported tasks, and image encoding errors. Would improve this by:

A dedicated SDK exception that would make that distinction reliable.

Preserving the original error message, including for connection failures.

Raise this dedicated exception only the server request ConnectionError.

dkosowski87 · 2026-10-22T19:21:56Z

+            f"Could not reach an inference server at {host}.\n"
+            "\n"
+            "`inference infer` sends requests to a server; it does not run the model in this\n"
+            "process. Start one locally with `inference server start` (needs Docker), or pass\n"


Suggested change

"process. Start one locally with `inference server start` (needs Docker), or pass\n"

"process. Start one locally with `inference server start` (requires Docker or Podman), or pass\n"

dkosowski87 · 2026-10-22T19:28:54Z

+            "`inference infer` sends requests to a server; it does not run the model in this\n"
+            "process. Start one locally with `inference server start` (needs Docker), or pass\n"
+            "--host to use a server you already have, for example\n"
+            "--host https://detect.roboflow.com for the hosted API."


Suggested change

"--host https://detect.roboflow.com for the hosted API."

"--host https://serverless.roboflow.com for Roboflow’s Serverless Cloud API"

dkosowski87 · 2026-10-22T19:30:43Z

+            "\n"
+            "`inference infer` sends requests to a server; it does not run the model in this\n"
+            "process. Start one locally with `inference server start` (needs Docker), or pass\n"
+            "--host to use a server you already have, for example\n"


Suggested change

"--host to use a server you already have, for example\n"

"--host to for an existing server, for example:\n"

Add HTTPServerConnectionError (subclass of HTTPClientError, so existing callers are unaffected) and raise it from wrap_errors/wrap_errors_async when the request to the server cannot connect. The CLI now catches only that error, keeps the original SDK message, and points to Docker/Podman and the Serverless API. Other client errors (missing API key, invalid model id, encoding) fall through to the generic message. Co-Authored-By: Claude Opus 5.5 <[email protected]>

tsayin4 requested review from PawelPeczek-Roboflow, dkosowski87, grzegorz-roboflow, hansent, probicheaux, rafel-roboflow and yeldarby as code owners October 6, 2026 20:09

dkosowski87 reviewed Oct 22, 2026

View reviewed changes

tsayin4 and others added 3 commits October 22, 2026 21:37

Merge branch 'main' into fix/314-clear-error-when-server-unreachable

879c401

Merge branch 'main' into fix/314-clear-error-when-server-unreachable

1dc9763

dkosowski87 enabled auto-merge (squash) October 23, 2026 10:57

dkosowski87 disabled auto-merge October 23, 2026 10:57

Merge branch 'main' into fix/314-clear-error-when-server-unreachable

3dacc24

dkosowski87 approved these changes Oct 23, 2026

View reviewed changes

dkosowski87 merged commit fdf149f into roboflow:main Oct 23, 2026
6 checks passed

dkosowski87 mentioned this pull request Oct 23, 2026

Fix connection error classification after retries and async timeouts #3258

Merged

Navigation Menu

Repository navigation

fix(cli): say what to do when the inference server is unreachable - #3129

fix(cli): say what to do when the inference server is unreachable#3129
dkosowski87 merged 5 commits into
roboflow:mainfrom
tsayin4:fix/314-clear-error-when-server-unreachable

tsayin4 commented Oct 6, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

CLAassistant commented Oct 6, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

tsayin4 commented Oct 9, 2026

Uh oh!

dkosowski87 left a comment

Uh oh!

dkosowski87 Oct 22, 2026

Uh oh!

dkosowski87 Oct 22, 2026

Uh oh!

dkosowski87 Oct 22, 2026

Uh oh!

dkosowski87 Oct 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	"process. Start one locally with `inference server start` (needs Docker), or pass\n"
	"process. Start one locally with `inference server start` (requires Docker or Podman), or pass\n"

	"--host https://detect.roboflow.com for the hosted API."
	"--host https://serverless.roboflow.com for Roboflow’s Serverless Cloud API"

	"--host to use a server you already have, for example\n"
	"--host to for an existing server, for example:\n"

Conversation

tsayin4 commented Oct 6, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

CLAassistant commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tsayin4 commented Oct 9, 2026

Uh oh!

dkosowski87 left a comment

Choose a reason for hiding this comment

Uh oh!

dkosowski87 Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

dkosowski87 Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

dkosowski87 Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

dkosowski87 Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tsayin4 commented Oct 6, 2026 •

edited by coderabbitai Bot

Loading

CLAassistant commented Oct 6, 2026 •

edited

Loading

coderabbitai Bot commented Oct 6, 2026 •

edited

Loading