Status: [UNTESTED] Endpoints found: http://127.0.0.1:8787/v1/noul http://127.0.0.1:8787/v1/choice http://127.0.0.1:8787/v1/cache/clear Source: https://github.com/ILuce/semif-local # SemIf Local Server Small local HTTP server for [SemIf](https://github.com/TheoLeeCJ/SemIf). The server keeps one SemIf decision model loaded in memory and exposes generation-free semantic decisions over HTTP. It returns option probabilities and logits instead of generating answer text. ## Features - `POST /v1/noul` — binary yes/no decisions - `POST /v1/choice` — decisions between arbitrary options - `POST /v1/shared` — multiple decisions over one shared state - `POST /v1/cache/clear` — clear reusable runtime caches without unloading the model - serial prefix-cache reuse - request latency and probability statistics - lightweight browser playground and dashboard - JSONL request logs - FastAPI/OpenAPI interface - MLX, CUDA, and llama.cpp backends `POST /v1/decision` is kept as a compatibility alias for `POST /v1/choice`. ## Requirements - Python 3.10+ - [uv](https://docs.astral.sh/uv/) - one of the supported runtimes below | Backend | Platforms | Hardware | | --- | --- | --- | | `mlx` | macOS Apple Silicon | Apple GPU / unified memory | | `cuda` | Linux, Windows | one visible NVIDIA CUDA GPU | | `llamacpp` | macOS, Linux, Windows | CPU | The default `config.json` is configured for the MLX backend on Apple Silicon. ## Clone ```bash git clone https://github.com/ILuce/semif-local cd semif-local-server uv python install 3.12 ``` ## Configuration Runtime defaults live in `config.json`: ```json { "backend": "mlx", "model": "models/semif-qwen3.5-4b-mlx-4bit", "model_revision": "local-vinci00-semif-qwen35-4b-mlx4", "max_tokens": 4096, "mlx_cache_mib": 256, "log": "logs/requests.jsonl", "torch_dtype": "bfloat16", "llama_gguf": "models/llamacpp/Qwen_Qwen3.5-4B-Q4_K_M.gguf", "llama_threads": null } ``` Relative file paths are resolved relative to `config.json`. The original environment variables remain available as optional overrides: - `SEMIF_MODEL` - `SEMIF_MODEL_REVISION` - `SEMIF_MAX_TOKENS` - `SEMIF_MLX_CACHE_MIB` - `SEMIF_LOG` A