-
Notifications
You must be signed in to change notification settings - Fork 718
Crashes #396
Description
I installed v0.2.21 and tried TurboQuant. I am getting LLM request failed: network connection error and also random computer crashes on my Mac Studio. The log is:
2026-03-26 17:23:20,559 - omlx.scheduler - DEBUG - [-] - Added request 4931aeb1-58c8-4cea-be82-78a2c1aa2bdf with 48673 prompt tokens
2026-03-26 17:23:26,500 - omlx.scheduler - DEBUG - [-] - Scheduled request 4931aeb1-58c8-4cea-be82-78a2c1aa2bdf (uid=0) with 11809 tokens to process (48673 total), 36864 cached, with cache
2026-03-26 17:23:36,137 - omlx.scheduler - DEBUG - [-] - Captured prefill boundary cache snapshot for 4931aeb1-58c8-4cea-be82-78a2c1aa2bdf at 38912 tokens
2026-03-26 17:23:45,120 - omlx.scheduler - DEBUG - [-] - Captured prefill boundary cache snapshot for 4931aeb1-58c8-4cea-be82-78a2c1aa2bdf at 40960 tokens
2026-03-26 17:24:02,887 - omlx.server - INFO - [-] - CORS origins: ['*']
2026-03-26 17:24:02,887 - omlx.model_settings - INFO - [-] - Loaded settings for 14 models
2026-03-26 17:24:02,888 - omlx.model_discovery - DEBUG - [-] - Skipping DevQuasar: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,888 - omlx.model_discovery - DEBUG - [-] - Skipping Goraint: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,888 - omlx.model_discovery - DEBUG - [-] - Skipping Mungert: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,889 - omlx.model_discovery - DEBUG - [-] - Skipping NexVeridian: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,889 - omlx.model_discovery - DEBUG - [-] - Skipping Qwen: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,890 - omlx.model_discovery - INFO - [-] - Discovered model: Qwen3.5-122B-A10B-oQ4 (type: vlm, engine: vlm, size: 69.02GB)
2026-03-26 17:24:02,890 - omlx.model_discovery - DEBUG - [-] - Skipping TheCluster: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,891 - omlx.model_discovery - DEBUG - [-] - Skipping bartowski: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,891 - omlx.model_discovery - DEBUG - [-] - Skipping bullerwins: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,891 - omlx.model_discovery - DEBUG - [-] - Skipping gpustack: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,891 - omlx.model_discovery - DEBUG - [-] - Skipping huynguyendbs: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,892 - omlx.model_discovery - DEBUG - [-] - Skipping kloseli: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,892 - omlx.model_discovery - DEBUG - [-] - Skipping litmudoc: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,892 - omlx.model_discovery - DEBUG - [-] - Skipping lmstudio-community: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,893 - omlx.model_discovery - INFO - [-] - Discovered model: Qwen3-Embedding-0.6B-4bit-DWQ (type: embedding, engine: embedding, size: 0.33GB)
2026-03-26 17:24:02,894 - omlx.model_discovery - INFO - [-] - Discovered model: Qwen3-Reranker-0.6B-mxfp8 (type: reranker, engine: reranker, size: 0.60GB)
2026-03-26 17:24:02,894 - omlx.model_discovery - INFO - [-] - Discovered model: Qwen3.5-122B-A10B-bf16 (type: vlm, engine: vlm, size: 239.71GB)
2026-03-26 17:24:02,895 - omlx.model_discovery - INFO - [-] - Discovered model: Qwen3.5-4B-MLX-4bit (type: vlm, engine: vlm, size: 2.97GB)
2026-03-26 17:24:02,895 - omlx.model_discovery - DEBUG - [-] - Skipping mradermacher: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,895 - omlx.model_discovery - DEBUG - [-] - Skipping nightmedia: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,896 - omlx.model_discovery - DEBUG - [-] - Skipping nomic-ai: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,896 - omlx.model_discovery - DEBUG - [-] - Skipping saul95: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,896 - omlx.model_discovery - DEBUG - [-] - Skipping second-state: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,896 - omlx.model_discovery - DEBUG - [-] - Skipping stduhpf: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,896 - omlx.model_discovery - DEBUG - [-] - Skipping unsloth: no config.json found (not a model or organization folder)
2026-03-26 17:24:02,896 - omlx.engine_pool - INFO - [-] - Pinned model: Qwen3.5-122B-A10B-oQ4
2026-03-26 17:24:02,896 - omlx.engine_pool - INFO - [-] - Pinned model: Qwen3-Embedding-0.6B-4bit-DWQ
2026-03-26 17:24:02,896 - omlx.engine_pool - INFO - [-] - Pinned model: Qwen3-Reranker-0.6B-mxfp8
2026-03-26 17:24:02,896 - omlx.engine_pool - INFO - [-] - Discovered 5 models, max memory: disabled
2026-03-26 17:24:02,896 - omlx.engine_pool - INFO - [-] - Applied model_type override for Qwen3-Embedding-0.6B-4bit-DWQ: type=embedding, engine=embedding
2026-03-26 17:24:02,897 - omlx.server_metrics - INFO - [-] - Loaded all-time stats from /Users/jxxxx/.omlx/stats.json
2026-03-26 17:24:02,897 - omlx.server - INFO - [-] - Server initialized with 5 models
2026-03-26 17:24:02,897 - omlx.server - INFO - [-] - Default model: Qwen3.5-122B-A10B-oQ4
2026-03-26 17:24:02,897 - omlx.server - INFO - [-] - Max model memory: disabled (no limit)
2026-03-26 17:24:02,897 - omlx.server - INFO - [-] - Default max tokens: 32768
2026-03-26 17:24:02,897 - omlx.server - INFO - [-] - API key authentication: enabled
2026-03-26 17:24:02,898 - omlx.server - INFO - [-] - HF Downloader initialized
2026-03-26 17:24:02,959 - omlx.server - INFO - [-] - ModelScope Downloader initialized
2026-03-26 17:24:02,960 - omlx.server - INFO - [-] - oQ Quantizer initialized
2026-03-26 17:24:02,961 - omlx.server - INFO - [-] - HF Uploader initialized
2026-03-26 17:24:02,962 - asyncio - DEBUG - [-] - Using selector: KqueueSelector
2026-03-26 17:24:02,965 - omlx.engine_pool - INFO - [-] - Preloading pinned model: Qwen3.5-122B-A10B-oQ4
2026-03-26 17:24:02,965 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3.5-122B-A10B-oQ4
2026-03-26 17:24:02,991 - omlx.engine.vlm - DEBUG - [-] - Removed video_processor from MODALITY_TO_AUTOPROCESSOR_MAPPING
2026-03-26 17:24:16,113 - omlx.model_registry - DEBUG - [-] - Engine c16b744e-8565-4db6-94f1-9a1f6ef2df68 acquired model 4631076336
2026-03-26 17:24:16,114 - omlx.scheduler - INFO - [-] - Loaded 1 additional EOS token(s) from generation_config.json: {248044}
2026-03-26 17:24:16,114 - omlx.scheduler - INFO - [-] - Enlarging paged cache block_size=256 to 2048 for ArraysCache hybrid model (reduces boundary snapshot overhead)
2026-03-26 17:24:16,114 - omlx.scheduler - INFO - [-] - paged SSD-only mode: max_blocks=100000, block_size=2048 tokens
2026-03-26 17:24:16,114 - omlx.cache.paged_cache - INFO - [-] - PagedCacheManager initialized: block_size=2048, initial_blocks=256, max_blocks=100000, max_tokens=204800000
2026-03-26 17:24:16,114 - omlx.cache.paged_ssd_cache - INFO - [-] - Scanning SSD cache directory: /Volumes/Cache
2026-03-26 17:24:16,600 - omlx.cache.paged_ssd_cache - INFO - [-] - SSD cache scan complete: scanned=851, indexed=851, errors=0, total_size=160.13 GB
2026-03-26 17:24:16,600 - omlx.cache.paged_ssd_cache - INFO - [-] - PagedSSDCacheManager initialized: dir=/Volumes/Cache, max_size=465.00 GB, existing_files=851, disk_free=305.18 GB, cache_used=160.13 GB
2026-03-26 17:24:16,600 - omlx.cache.paged_cache - INFO - [-] - paged SSD cache manager connected to PagedCacheManager
2026-03-26 17:24:16,600 - omlx.cache.prefix_cache - INFO - [-] - PagedSSDCacheManager connected to BlockAwarePrefixCache
2026-03-26 17:24:16,602 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: cache_dir=/Volumes/Cache, max_size=465.00 GB, block_size=2048 tokens
2026-03-26 17:24:16,602 - omlx.scheduler - INFO - [-] - paged SSD cache enabled: /Volumes/Cache, block_size=2048, max_blocks=100000
2026-03-26 17:24:16,602 - omlx.engine_core - DEBUG - [-] - Engine c16b744e-8565-4db6-94f1-9a1f6ef2df68 initialized
2026-03-26 17:24:16,602 - omlx.engine_core - INFO - [-] - Engine started
2026-03-26 17:24:16,651 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=qwen3_coder
2026-03-26 17:24:16,654 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: /Users/jxxxx/.lmstudio/models/Qwen3.5-122B-A10B-oQ4
2026-03-26 17:24:16,654 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3.5-122B-A10B-oQ4 (estimated: 69.02GB, total: 69.02GB)
2026-03-26 17:24:16,654 - omlx.engine_pool - INFO - [-] - Preloading pinned model: Qwen3-Embedding-0.6B-4bit-DWQ
2026-03-26 17:24:16,654 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3-Embedding-0.6B-4bit-DWQ
2026-03-26 17:24:16,654 - omlx.engine.embedding - INFO - [-] - Starting embedding engine: /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ
2026-03-26 17:24:16,655 - omlx.models.embedding - DEBUG - [-] - Architecture 'Qwen3ForCausalLM' not natively supported for embedding, trying mlx-embeddings
2026-03-26 17:24:16,657 - omlx.models.embedding - INFO - [-] - Loading embedding model via mlx-embeddings: /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ
2026-03-26 17:24:17,266 - omlx.models.embedding - INFO - [-] - mx.compile enabled for /Users/xxxx.lmstudio/models/mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ (primitive embedding path)
2026-03-26 17:24:17,266 - omlx.models.embedding - INFO - [-] - Embedding model loaded successfully: /Users/xxxxlmstudio/models/mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ (hidden_size=1024, compiled=True)
2026-03-26 17:24:17,267 - omlx.engine.embedding - INFO - [-] - Embedding engine started: /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Embedding-0.6B-4bit-DWQ
2026-03-26 17:24:17,267 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3-Embedding-0.6B-4bit-DWQ (estimated: 335.75MB, total: 69.35GB)
2026-03-26 17:24:17,267 - omlx.engine_pool - INFO - [-] - Preloading pinned model: Qwen3-Reranker-0.6B-mxfp8
2026-03-26 17:24:17,267 - omlx.engine_pool - INFO - [-] - Loading model: Qwen3-Reranker-0.6B-mxfp8
2026-03-26 17:24:17,267 - omlx.engine.reranker - INFO - [-] - Starting reranker engine: /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Reranker-0.6B-mxfp8
2026-03-26 17:24:17,267 - omlx.models.reranker - INFO - [-] - Loading reranker model: /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Reranker-0.6B-mxfp8 (arch=Qwen3ForCausalLM)
2026-03-26 17:24:17,902 - omlx.models.reranker - INFO - [-] - CausalLM reranker tokens: yes=9693, no=2152, prefix_len=39, suffix_len=9
2026-03-26 17:24:17,902 - omlx.models.reranker - INFO - [-] - mx.compile skipped for causal-lm reranker /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Reranker-0.6B-mxfp8
2026-03-26 17:24:17,902 - omlx.models.reranker - INFO - [-] - Reranker model loaded successfully: /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Reranker-0.6B-mxfp8 (arch=Qwen3ForCausalLM, num_labels=2, causal_lm=True, compiled=False)
2026-03-26 17:24:17,902 - omlx.engine.reranker - INFO - [-] - Reranker engine started: /Users/xxxx/.lmstudio/models/mlx-community/Qwen3-Reranker-0.6B-mxfp8
2026-03-26 17:24:17,902 - omlx.engine_pool - INFO - [-] - Loaded model: Qwen3-Reranker-0.6B-mxfp8 (estimated: 615.35MB, total: 69.95GB)