Local-AI-Recruiter — Ollama + Pydantic agent (CPU friendly) + MCP Integration inside the mcp-agent-master branch
Privacy-first, local recruitment assistant that scores and summarizes candidates using Ollama (local models) and lightweight Pydantic-based tools for structured tool-calls and data validation.
End-to-end modular system for: (1) LinkedIn talent discovery & profile extraction, (2) semantic multi-factor candidate scoring, and (3) an interactive LLM assistant (Ollama) that orchestrates the workflow via validated tool calls.
demo-ai-recruiter-agent.-.mp4
┌────────────────────────┐ ┌────────────────────────┐ ┌────────────────────────┐
│ LinkedIn Search & │ REST │ Candidate Scorer │ REST │ LLM Agent / UI │
│ Extraction Service │ <----> │ (FAISS + Embeddings) │ <----> │ (Ollama + Streamlit) │
│ (FastAPI + Selenium) │ │ /scorer_tool/* │ │ tools: search/score │
└──────────┬─────────────┘ └──────────┬─────────────┘ └──────────┬─────────────┘
│ (Selenium drives Chrome) │ │
│ │ │
▼ ▼ ▼
LinkedIn Web UI JSON Candidate Profiles User queries (JD / intent)
Shared filesystem folder: Full system/tmp_candids_jsons
Job description input: ollama_recruiter/data/jd_input
| Component | Path | Purpose |
|---|---|---|
| LinkedIn API Service | Full system/linkedin_api/server.py |
Provides /search (candidate links) and /extract (profile JSON) using a persistent Selenium Chrome session (profile saved under .chrome-profile-linkedin). |
| Candidate Scorer Service | Full system/candidate_scorer/server.py |
Indexes extracted JSON profiles and scores them against a job description (embeddings + section weighting). Exposes /scorer_tool/* endpoints. |
| Agent / Orchestrator | ollama_recruiter/ (agent_runtime.py, tools.py, streamlit_app.py) |
Uses an Ollama local model to call tools: search -> extract -> score. Presents interactive chat (Streamlit). |
| Temp Candidate JSONs | Full system/tmp_candids_jsons/ |
Intermediate storage of raw extracted candidate data (deleted/archived optionally). |
| Job Description Inputs | ollama_recruiter/data/jd_input/ |
Place a job_description.txt or timestamped file consumed by the agent scoring pipeline. |
| Postman Collection | HR-AI-Agent.postman_collection.json |
Manual testing of REST endpoints for search, extraction, profile loading & scoring. |
- User (or agent) supplies a job description & search intent/query.
- Agent calls LinkedIn service
/search→ returns candidate profile URLs. - For each URL, agent calls
/extract→ stores structured JSON intmp_candids_jsons. - Agent (or manual call) hits scorer
/scorer_tool/load_profiles→ indexes embeddings. - Agent sends JD text to
/scorer_tool/score→ gets ranked candidates with section breakdown. - Streamlit UI / console presents final ranked results.
Environment variables (see below) allow overriding endpoints and Selenium / credentials.
You will now create TWO isolated virtual environments:
- Server environment (
venv_server): hosts BOTH FastAPI apps (LinkedIn search/extraction + candidate scorer) - Agent/UI environment (
venv_agent): runs the Ollama orchestration + Streamlit interface
The previous separation into three environments is deprecated (the two old requirements files remain for reference but are superseded by Full system/requirements_server.txt).
- Python 3.11+ (matching your local dev target)
- Google Chrome (latest stable)
- ChromeDriver (matching your Chrome version)
- Official download: https://chromedriver.chromium.org/downloads
- Put the executable somewhere stable (e.g.
C:/tools/chromedriver/chromedriver.exe).
- (Optional)
Ollamainstalled locally for LLM inference: https://ollama.com - Hardware Note: This system was developed and tested on CPU only (no GPU required). Embedding & model inference paths default to CPU; if you later add GPU support, ensure dependencies (e.g., CUDA-enabled torch) are pinned accordingly.
git clone https://github.com/GhaouiYoussef/Ollama-Local-Recruiter-Agent.git
cd Ollama-Local-Recruiter-AgentRun these in PowerShell from the project root.
python -m venv venv_server
./venv_server/Scripts/Activate.ps1
pip install --upgrade pip
pip install -r "Full system/requirements_server.txt"python -m venv venv_agent
./venv_agent/Scripts/Activate.ps1
pip install --upgrade pip
pip install -r "ollama_recruiter/requirements_agent.txt"Deactivate an environment at any time with:
deactivateCreate a .env file at the project root (or set in your shell) for the LinkedIn service & agent:
LK_USERNAME=your_linkedIn_email
LK_PASSWORD=your_linkedIn_password
CHROMEDRIVER_PATH=C:/tools/chromedriver/chromedriver.exe
CHROME_PROFILE_DIR=.chrome-profile-linkedin
# Optional overrides:
LINKEDIN_SEARCH_URL=http://127.0.0.1:8000/search
LINKEDIN_EXTRACT_URL=http://127.0.0.1:8000/extract
CANDIDATE_SCORER_URL=http://127.0.0.1:8001/scorer_tool
LINKEDIN_LOGIN_WAIT_SECONDS=25
Keep credentials private. For production, consider a secrets manager.
Run (from any activated venv):
& "C:/tools/chromedriver/chromedriver.exe" --versionEnsure the major version matches installed Chrome (open chrome://settings/help).
LinkedIn frequently A/B tests and rotates CSS/class selectors. The extraction utilities in linkedin_api/content_extractor/functions.py or candidate_searcher/functions.py may break suddenly.
Troubleshooting / Updating Selectors:
- Re-run
/searchthen/extractand observe server terminal logs for element lookup failures. - Open the failing profile in a normal Chrome window (or the persisted profile directory) and inspect elements (Right‑click → Inspect) to find updated class names.
- Edit the relevant lookup in the functions file (e.g.,
driver.find_element(By.CLASS_NAME, "newClass")). - Restart the LinkedIn service.
- For iterative debugging, use the notebooks (see
DEBUGGER_NOTEBOOKS/) to manually run the extraction logic and print raw HTML snippets.
Best Practices:
- Avoid heavy rapid scraping (risk of temporary blocks).
- Leverage the persistent Chrome profile to reduce logins & 2FA friction.
- If login loops occur: delete
.chrome-profile-linkedindirectory to force a clean state.
Use the provided Postman collection HR-AI-Agent.postman_collection.json.
Steps:
- Open Postman → Import → Choose the JSON file.
- Ensure the LinkedIn service & scorer service are running (see next section) before invoking endpoints.
- Example sequence:
GET http://127.0.0.1:8000/search?query=python%20backend&num_candidates=5GET http://127.0.0.1:8000/extract?profile_url=<candidate_profile_url>POST http://127.0.0.1:8001/scorer_tool/load_profiles(JSON body:{ "json_folder": "Full system/tmp_candids_jsons", "exp_agg": "sum_norm", "reset": true })POST http://127.0.0.1:8001/scorer_tool/score(JSON body:{ "job_text": "<paste JD text>", "top_k_search": 200 })
- Validate responses:
indexed_profiles,results, and per candidate score breakdown.
Activate venv_server once per terminal you need. You still run two FastAPI processes (they remain separate apps) but they share the same dependencies.
./venv_server/Scripts/Activate.ps1
uvicorn "Full system.linkedin_api.server:app" --host 127.0.0.1 --port 8000 --reload./venv_server/Scripts/Activate.ps1
uvicorn "Full system.candidate_scorer.server:app" --host 127.0.0.1 --port 8001 --reloadHealth check:
curl http://127.0.0.1:8001/scorer_tool/healthIf you attach a text file containing the job description:
- The file will be loaded into ollama_recruiter/data/jd_input/ saved with its name + timestamp, then renamed job_description.txt and used for scoring.
else:
- The agent will only return the cnadidates wuthout scoring them against the job description.
Environment: activate venv_agent.
./venv_agent/Scripts/Activate.ps1
streamlit run ollama_recruiter/streamlit_app.pyEnter a search query (e.g., "senior python data engineer Tunisia") – the agent will chain: search → extract → score. Results and scores appear in the UI.
From inside venv_agent:
python -c "from ollama_recruiter.tools import linkedin_search_tool; print(linkedin_search_tool('python backend engineer', 3))"Set test_mode_extract=True / test_mode_score=True for dry runs:
python -c "from ollama_recruiter.tools import linkedin_search_tool; print(linkedin_search_tool('python backend', 3, test_mode_extract=True))"If you just want to score previously extracted profiles:
# 1. Ensure scorer running
curl -X POST http://127.0.0.1:8001/scorer_tool/load_profiles -H "Content-Type: application/json" `
-d '{"json_folder":"Full system/tmp_candids_jsons","exp_agg":"sum_norm","reset":true}'
# 2. Send job description inline
curl -X POST http://127.0.0.1:8001/scorer_tool/score -H "Content-Type: application/json" `
-d '{"job_text":"We need a Python engineer with FAISS & embeddings experience.","top_k_search":200}'Press Ctrl + C in each terminal. ChromeDriver processes should exit automatically; if orphaned, kill via Task Manager.
If you previously created venv_linkedin and venv_scorer, you can remove them after verifying venv_server works:
Remove-Item -Recurse -Force venv_linkedin
Remove-Item -Recurse -Force venv_scorerThe old requirement files remain for historical reference but are no longer used.
This section explains how the system treats candidate JSON extractions and job description (JD) files—especially when automatic cleanup is enabled.
| Path | Purpose | Created By | Typical Lifetime |
|---|---|---|---|
Full system/tmp_candids_jsons/ |
Raw extracted candidate profile JSON files (one per LinkedIn profile). | LinkedIn extraction tool inside the agent or manual /extract calls. |
Until you manually delete OR automatic cleanup runs. |
ollama_recruiter/data/jd_input/ |
Input job description text files. The system picks the first *.txt. |
User (you) drop files here. | Original file may be archived if cleanup enabled. |
ollama_recruiter/data/jd_input/job_description.txt |
Stable copy the agent reads for scoring. Created if missing. | Agent (_prepare_job_description). |
Persisted (NOT auto-deleted). |
ollama_recruiter/data/jd_history/ |
Archive of past JD files with timestamps. | Agent when cleanup enabled. | Grows over time unless pruned. |
- You place (or reuse) a JD file under
jd_input/(e.g.,20251004_051747_job_desc.txt). - The agent copies the first matching file to a stable name
job_description.txtif it doesn't already exist. - Each candidate extracted is saved as
<linkedin_slug>.jsoninFull system/tmp_candids_jsons/. - Scoring loads all JSONs in that folder and builds an index (reset behaviour is controlled in the scorer request payload).
- Nothing is deleted automatically (default). You can inspect JSON files or rerun scoring without re-extraction.
Set the environment variable before running the agent (recommended) OR modify the flag in code:
Add to your .env or export in PowerShell session:
$env:CLEANUP_AND_ARCHIVE = "1"or place in .env:
CLEANUP_AND_ARCHIVE=1
Edit ollama_recruiter/tools.py and set CLEANUP_AND_ARCHIVE = True (not recommended for production—env var is cleaner).
If CLEANUP_AND_ARCHIVE resolves to true:
- The ORIGINAL JD file that was first discovered (e.g.,
20251004_051747_job_desc.txt) is MOVED intojd_history/asjob_description_<timestamp>.txt.- The stable
job_description.txtcopy remains for reproducibility or audit.
- The stable
- All
*.jsonfiles withinFull system/tmp_candids_jsons/are deleted (count is logged). - Failures (e.g., permission issues) are caught and printed without aborting the overall run.
| Scenario | Recommended Setting | Reason |
|---|---|---|
| Debugging extraction selectors | CLEANUP_AND_ARCHIVE=0 |
Keep JSONs to inspect their structure. |
| Regular iterative sourcing & scoring | CLEANUP_AND_ARCHIVE=0 |
Reuse previous JSONs without re-hitting LinkedIn. |
| Production / clean runs to minimize disk accumulation | CLEANUP_AND_ARCHIVE=1 |
Prevent stale data build-up & archive JD trace. |
- Past JDs: check
jd_history/(timestamp naming). Compare diffs with latestjob_description.txt. - Candidate Data: if cleanup was enabled, raw JSONs are gone—re-run extraction to regenerate. If disabled, you can snapshot the folder or version control selected anonymized examples.
Delete all candidate JSONs manually (PowerShell):
Remove-Item -Force -ErrorAction SilentlyContinue "Full system/tmp_candids_jsons/*.json"List archived JDs:
Get-ChildItem "ollama_recruiter/data/jd_history" -Filter *.txt | Sort-Object LastWriteTime -Descending| Variable | Purpose | Default (if unset) |
|---|---|---|
LK_USERNAME / LK_PASSWORD |
LinkedIn credentials | None (required) |
CHROMEDRIVER_PATH |
Path to ChromeDriver | Hardcoded fallback in code (update recommended) |
CHROME_PROFILE_DIR |
Persisted Chrome profile dir | .chrome-profile-linkedin |
LINKEDIN_SEARCH_URL |
Agent search endpoint | http://127.0.0.1:8000/search |
LINKEDIN_EXTRACT_URL |
Agent extraction endpoint | http://127.0.0.1:8000/extract |
CANDIDATE_SCORER_URL |
Agent scoring endpoint base | http://127.0.0.1:8001/scorer_tool |
LINKEDIN_LOGIN_WAIT_SECONDS |
Max wait after login | 25 |
HEADLESS |
Enable headless (if logic present) | 0 |
| Issue | Cause | Fix |
|---|---|---|
| Selenium fails to locate element | LinkedIn DOM changed | Update selector in relevant function file. |
| Infinite login loop | Stale cookies / checkpoint | Delete .chrome-profile-linkedin dir and retry. |
| Empty scoring results | Profiles not loaded | Call /scorer_tool/load_profiles first. |
chromedriver version mismatch |
Chrome updated | Download matching driver version. |
| Import error after upgrade to combined env | Mixed old venvs lingering | Delete old venv_linkedin / venv_scorer and reinstall venv_server. |
| Cleanup not happening | CLEANUP_AND_ARCHIVE not set correctly |
Ensure env var is 1/true/yes BEFORE starting agent process. |
| JD not archived but JSONs deleted | JD copy logic edge case | Confirm original JD still existed; ensure you didn't start with only job_description.txt. |
| Agent returns only links (no scores) | test_mode_score=True or scoring error |
Check scorer logs & ensure JD file present. |
Logging: watch the terminal running each service for stack traces and debug messages (search/extract scaffolding uses _log) when setting iDEBBUGING=True inside the Full system\linkedin_api\server.py file.
- If your issue isn’t listed here, please raise an issue on this repository with full logs and context.
- Add caching layer for already extracted profiles.
- Replace raw Selenium selectors with resilient XPath + heuristics.
- Add CI test harness mocking LinkedIn HTML snapshots.
- Provide Dockerfiles per service for reproducible deployment.
# 1. Create 2 envs (server + agent) & install deps (see section 2.3)
# 2. Set .env with LinkedIn credentials & CHROMEDRIVER_PATH
# 3. Terminal 1 (LinkedIn service)
./venv_server/Scripts/Activate.ps1; uvicorn "Full system.linkedin_api.server:app" --port 8000 --reload
# 4. Terminal 2 (Scorer service)
./venv_server/Scripts/Activate.ps1; uvicorn "Full system.candidate_scorer.server:app" --port 8001 --reload
# 5. Terminal 3 (Agent UI)
./venv_agent/Scripts/Activate.ps1; streamlit run ollama_recruiter/streamlit_app.py
# 6. Interact via UI; watch tmp_candids_jsons fill with JSON; scores appear.Happy recruiting!