Instructions to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Sandeep4235/Qwen3-4B-PromptCraft-GGUF")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Sandeep4235/Qwen3-4B-PromptCraft-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Sandeep4235/Qwen3-4B-PromptCraft-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sandeep4235/Qwen3-4B-PromptCraft-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
- SGLang
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Sandeep4235/Qwen3-4B-PromptCraft-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sandeep4235/Qwen3-4B-PromptCraft-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Sandeep4235/Qwen3-4B-PromptCraft-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Sandeep4235/Qwen3-4B-PromptCraft-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with Ollama:
ollama run hf.co/Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with Docker Model Runner:
docker model run hf.co/Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
- Lemonade
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Qwen3-4B-PromptCraft-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Sandeep4235/Qwen3-4B-PromptCraft-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Sandeep4235/Qwen3-4B-PromptCraft-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3-4B-PromptCraft-GGUF
PromptCraft Engine ยท Q4_K_M GGUF ยท Fine-tuned Qwen3-4B ยท Zero-Reasoning System Prompt Engineering
๐ Model Summary
Qwen3-4B-PromptCraft is a 4-billion parameter causal language model fine-tuned (via Unsloth QLoRA) from Qwen/Qwen3-4B to specialize in Zero-Reasoning System Prompt Engineering โ the deterministic transformation of high-level developer requirements into production-grade, constraint-heavy system prompts for downstream LLMs. Unlike general-purpose instruct models, PromptCraft emits structured artifacts (executive summary โ implementation block โ edge-case matrix) without leaking chain-of-thought reasoning tokens into the final output.
Mission. BluePatterns AI built PromptCraft to give every developer a reliable, offline-capable tool for authoring security-aware, architecture-faithful system prompts โ lowering the barrier to deploying LLMs in production without sacrificing safety, reproducibility, or control.
๐ก Model Architecture & Specifications
PromptCraft inherits the dense Transformer architecture of Qwen3-4B and applies a LoRA adapter trained via Unsloth's 4-bit QLoRA pipeline.
| Specification | Detail |
|---|---|
| Developed by | Sandeep Hipparagi (BluePatterns AI) |
| Model type | Causal Language Model (Causal-LM) |
| Base model | Qwen/Qwen3-4B |
| Total parameters | 4.0B (3.6B non-embedding) |
| Architecture | Dense Transformer, decoder-only |
| Attention | Grouped-Query Attention (GQA) โ 32 Q heads, 8 KV heads |
| Layers | 36 |
| Positional encoding | RoPE (base frequency 1,000,000) |
| Normalization | RMSNorm (pre-normalization) + QK-Norm |
| Activation | SwiGLU |
| Tie embedding | Yes (input/output shared) |
| Native context length | 32,768 tokens (up to 131,072 with YaRN) |
| Fine-tuning method | Unsloth QLoRA (4-bit NF4 quantization) |
| Adapter precision | bf16 LoRA ranks |
| Quantization (this repo) | GGUF Q4_K_M |
| Training data | 2,050 synthetic instruction pairs (Roles ร Domains ร Constraints) |
| Optimizer | AdamW 8-bit |
| Learning rate | 2e-4, cosine schedule |
| Epochs | 2 |
| Max sequence length (training) | 4,096 tokens |
| Language(s) | English |
| License | Apache 2.0 |
๐ Quickstart & Usage
Option A โ Python with transformers (recommended for GPU)
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Sandeep4235/Qwen3-4B-PromptCraft-GGUF"
# Load tokenizer and model with automatic device placement
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto", # uses bfloat16 on supported hardware
)
# Build the chat-formatted prompt using Qwen3's ChatML template
messages = [
{
"role": "user",
"content": (
'Transform this input into a production-ready system prompt: '
'"Build a FastAPI validator that rejects malformed JSON '
'payloads with 422 responses."'
),
},
]
# Apply the model's chat template
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
# Generate
output_ids = model.generate(
input_ids,
max_new_tokens=1024,
do_sample=True,
temperature=0.7,
top_p=0.9,
repetition_penalty=1.05,
pad_token_id=tokenizer.eos_token_id,
)
# Decode only the newly generated tokens (exclude the input prompt)
generated = output_ids[0][input_ids.shape[-1]:]
response = tokenizer.decode(generated, skip_special_tokens=True)
print(response)
Option B โ llama.cpp CLI (CPU / edge inference)
# Download the Q4_K_M GGUF file, then run:
./llama-cli \
-m Qwen3-4B-PromptCraft-Q4_K_M.gguf \
-p '<|im_start|>user
Transform this input into a production-ready system prompt: "Build a FastAPI validator that rejects malformed JSON payloads with 422 responses."<|im_end|>
<|im_start|>assistant
' \
-n 1024 \
--temp 0.7 \
--top-p 0.9 \
--repeat-penalty 1.05
Option C โ Python with llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="Qwen3-4B-PromptCraft-Q4_K_M.gguf",
n_ctx=4096,
n_gpu_layers=-1, # offload all layers to GPU if available
)
response = llm.create_chat_completion(
messages=[
{
"role": "user",
"content": (
'Transform this input into a production-ready system '
'prompt: "Build a FastAPI validator that rejects '
'malformed JSON payloads with 422 responses."'
),
}
],
max_tokens=1024,
temperature=0.7,
top_p=0.9,
)
print(response["choices"][0]["message"]["content"])
๐ Evaluation and Benchmarks
Formal quantitative benchmarks for PromptCraft (e.g., structural adherence, constraint compliance, and reasoning-leakage rates against baseline models) have not yet been published. A domain-specific evaluation protocol is planned and will be documented here once results are available.
General Academic Benchmarks (inherited from base)
PromptCraft is fine-tuned from Qwen/Qwen3-4B via QLoRA with a narrow, task-specific scope. The following scores reflect the Qwen3-4B base model as published in the Qwen3 Technical Report and are provided for reference only โ they are not measurements of PromptCraft itself.
| Benchmark | Qwen3-4B (base) | Metric |
|---|---|---|
| MMLU (5-shot) | 72.5 | Pass@1 accuracy |
| HumanEval (0-shot) | 70.1 | Pass@1 |
| GSM8K (8-shot) | 78.4 | Pass@1 accuracy |
| ARC-Challenge (25-shot) | 85.2 | Pass@1 accuracy |
These figures are the base model's published results and do not represent PromptCraft's fine-tuned performance. Independent benchmarking is encouraged.
โ ๏ธ Intended Uses & Limitations
Primary Use Cases
- AI agents & autonomous workflows โ generating deterministic system prompts that constrain agent behavior and tool-usage patterns.
- Security-hardened LLM applications โ embedding OWASP LLM Top 10 mitigations directly into system prompts (injection defense, output schema enforcement, PII redaction directives).
- Domain-specific tutors โ Socratic-method instructional prompts that guide without revealing answers directly.
- Backend logic generation โ producing system prompts for code-generation tasks (FastAPI validators, SQL query builders, regex factories) with strict output contracts.
- Prompt engineering pipelines โ serving as a "prompt-compiler" step that converts human intent into machine-optimized prompts for downstream LLMs.
Out-of-Scope Uses
- General-purpose chatbot or conversational assistant. PromptCraft is optimized for prompt generation, not free-form dialogue.
- Safety-critical automated decision-making. Do not use for medical diagnosis, legal advice, financial trading, or any scenario requiring deterministic, auditable outputs.
- Creative writing, roleplay, or storytelling outside the system-prompt-generation domain.
- Real-time safety filtering. The model is not a content moderation tool; pair it with a dedicated guardrail model (e.g., Llama Guard) for production use.
Limitations & Biases
- Offline prompt generation. The model does not have access to real-time information, APIs, or external tools. Generated prompts reflect the model's training distribution as of the fine-tuning date.
- English-only. Training data is English-centric; prompt quality in other languages is untested and likely degraded.
- Hallucination risk. Like all LLMs, PromptCraft may fabricate constraints, APIs, or library functions that do not exist. Always audit generated prompts for factual correctness before deployment.
- Security is best-effort, not guaranteed. While the model is fine-tuned to include security constraints, it is not a substitute for professional security review. Generated prompts should undergo manual security audit before production use.
- Dataset bias. The 2,050-pair synthetic training dataset reflects the biases and assumptions of its generation pipeline. Underrepresented domains (e.g., embedded systems, kernel development) may produce lower-quality prompts.
- Quantization loss. The Q4_K_M GGUF quantization introduces minor quality degradation compared to the full-precision model. Use Q8_0 for near-lossless quality.
๐ Responsible AI & Safety Alignment
Alignment Techniques
- Supervised Fine-Tuning (SFT). The model was trained on curated instruction pairs that demonstrate safe, structured prompt generation. No RLHF or DPO was applied in this release.
- Zero-Reasoning constraint. The training data was filtered to exclude chain-of-thought reasoning from outputs. The model is trained to suppress reasoning tokens (e.g., ``) in the final prompt artifact, preventing reasoning leakage into downstream application prompts.
- Structured output schema. The three-section contract (Executive Summary โ Implementation Artifact โ Edge Case Matrix) constrains the model to predictable, auditable output.
Deployment Recommendations
Input validation. Sanitize all user requirements before passing them to the model. Reject inputs containing prompt-injection patterns (
Ignore previous instructions,<|im_start|>spoofing, etc.).Output audit. Always review generated system prompts for correctness, safety, and alignment with your application's risk tolerance before production deployment.
Rate limiting. Deploy behind an API gateway with authentication and rate limiting to prevent abuse.
Content filtering. Pair PromptCraft with an input/output content filter (e.g., Llama Guard, NeMo Guardrails) for production use.
System prompt constraints. When deploying, prepend a guardrail system prompt such as:
You are a prompt-generation assistant. Refuse to generate prompts that facilitate harm, exploitation, illegal activity, or deceptive practices. If a request is ambiguous or potentially unsafe, respond with "I cannot generate this prompt. Please clarify your use case."
๐ฆ Quantization Variants
This repository provides GGUF quantizations for CPU and GPU inference via llama.cpp and compatible runtimes. The default and recommended variant is Q4_K_M, which offers the best quality-to-size tradeoff.
| File | Quantization | Approx. Size | Use Case |
|---|---|---|---|
Qwen3-4B-PromptCraft-Q4_K_M.gguf |
Q4_K_M | ~2.4 GB | Recommended โ best balance |
Qwen3-4B-PromptCraft-Q5_K_M.gguf |
Q5_K_M | ~2.9 GB | Higher quality, more VRAM |
Qwen3-4B-PromptCraft-Q8_0.gguf |
Q8_0 | ~4.3 GB | Near-lossless |
๐ค Citation & Acknowledgements
If you use Qwen3-4B-PromptCraft in your research or product, please cite it as follows:
@misc{hipparagi2025promptcraft,
title = {Qwen3-4B-PromptCraft: Zero-Reasoning System Prompt Engineering via QLoRA Fine-tuning},
author = {Sandeep Hipparagi},
organization = {BluePatterns AI},
year = {2025},
url = {https://huggingface.co/Sandeep4235/Qwen3-4B-PromptCraft-GGUF},
note = {Fine-tuned from Qwen/Qwen3-4B under Apache 2.0 license}
}
Acknowledgements
- Qwen Team โ for open-sourcing the Qwen3-4B base model under Apache 2.0.
- Unsloth โ for the efficient QLoRA fine-tuning framework that made this release possible.
- Hugging Face โ for model hosting and the open ML ecosystem.
Built by BluePatterns AI ยท Created using the PromptCraft-v1 Engine ยท Apache 2.0
- Downloads last month
- 37
4-bit
