Repository navigation
fix(sudestavenir_fr): parse mixed green-waste schedules - #7079
Conversation
The source doc still described dechets vegetaux as a precomputed date list only. Reflect that the precision text can also carry a bounded weekly rule, and note that bounded rules are not extrapolated into the following year. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
markvp
left a comment
There was a problem hiding this comment.
Nice work on this one. The mixed precision-text parsing is exactly the right shape: keeping precisi authoritative, merging the explicit dates with the bounded rule through a set, and clipping to the request window means bounded ranges never leak into next year.
I verified it live both ways. The committed Creteil test case is byte-for-byte identical to master (278 entries, nothing gained or lost), and the Noiseau address from #7077 picks up 19 Déchets végétaux Wednesdays running to 9 December with nothing else changed. The deterministic regression tests are a welcome addition too.
I made one small tweak: the source doc still described green waste as a fixed date list, so I refreshed that paragraph to cover the mixed case. Nothing needed from you.
* Add Viborg (Revas) to Affaldonline source (#6883) Viborg Kommune's waste operator Revas uses the same Affaldonline platform/response format already supported via the "default" parser. Closes #5681 * Update generated documentation * feat(source): add CEM Ambiente (cem_ambiente_it) (#6884) Adds support for CEM Ambiente's public CEM FACILE API, covering ~77 municipalities in the Milano/Monza e Brianza/Lodi area, closing #6871. * Update generated documentation * Generalize isontinambiente_it source to cover all Isontina Ambiente municipalities (#6885) The source only needs address_id to resolve the correct calendar; the municipality slug in the request URL is cosmetic and does not affect the result. Update TITLE/DESCRIPTION/EXTRA_INFO/TEST_CASES/HOW_TO text to reflect that this single source already covers all 27 municipalities served by Isontina Ambiente, not only Ronchi dei Legionari. Fixes #5543 Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * Add Prince George's County, MD source (#6887) Implements trash, recycling, bulky trash, and yard trim collection lookup via the County's public ArcGIS "Once-A-Week Collection" layer, resolving addresses through the Esri World Geocoder. Closes #5576 * Update generated documentation * feat(source): add City of Yarra, VIC, AU (yarracity_vic_gov_au) (#6888) Co-authored-by: Claude Fable 5 <[email protected]> * Update generated documentation * docs: announce the 3.0.0 architecture pre-release (#6892) Add a short callout to the README and info.md preambles inviting users to test the opt-in 3.0.0-alpha pipeline pre-release via HACS beta versions, focused on the benefits (more reuse, easier addition and management of sources, consistent labels). Forward-ports the announcement shipped in 2.31.1 so it survives into the next release from master. Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> * docs(installation): warn that fetch_time/day_switch_time must be quoted (#6891) PyYAML follows the legacy YAML 1.1 spec, which parses an unquoted H:MM value (hour 10-23) as a sexagesimal integer instead of a string, e.g. fetch_time: 10:00 -> 600, breaking HA's time validator. Document that these values must be quoted (e.g. fetch_time: "10:00") to avoid the pitfall, since this is a parser characteristic outside the integration's control. Refs #5065 * feat(source): add Timrå kommun waste collection (timra_se) (#6894) Adds support for Timrå kommun, Sweden, by querying the municipality's public ArcGIS FeatureServer directly by property address. Resolves the current FeatureServer URL dynamically via the parent Web Map item so the source keeps working when the date-stamped layer is refreshed for a new year. Closes #5541 * Update generated documentation * Add source for Waikato District Council, New Zealand (#5226) (#6898) Queries the council's public rubbish-and-recycling address search API and generates 26 weeks of the combined weekly kerbside collection day. Closes #5226 * Update generated documentation * fix(abfallkalender_prezero_network): remove Willich, no longer served by PreZero (#6897) PreZero's contract for Willich ended (Remondis, then Lankes Entsorgung took over collection in 2026) and abfallkalender.prezero.network/willich now returns 404. Willich is already covered by the existing abfallnavi_de source via Kreis Viersen. * Update generated documentation * Fix sutherlandshire_nsw_gov_au: council replaced lookup form; query public GIS layer (#6890) * Fix sutherlandshire_nsw_gov_au: council replaced lookup form; use public GIS layer The waste-information-booklet page the scraper drove was removed from the council website (now redirects to a map-based page), and the site also started rejecting plain library HTTP clients by TLS fingerprint, so every fetch failed with 403. The council's new 'When is my bin collected?' map is backed by a public ArcGIS layer with one feature per property carrying DAY_OF_WEEK, ZONE_NO and RECYCLE_PICKUP. Query that layer by address instead: - exact situ1/situ2 attribute match, with a prefix fallback for unit numbers and house-number suggestions when the street matches - schedule generation (weekly garbage, fortnightly recycling/garden by zone parity) unchanged from the previous implementation - properties flagged RECYCLE_PICKUP=Weekly (some unit blocks) now get weekly recycling - second TEST_CASE added * Restore COUNTRY constant for sutherlandshire_nsw_gov_au The GIS-layer rewrite dropped the required COUNTRY = "au" constant, which would orphan the source out of the generated README/sources.json. Co-Authored-By: Claude Sonnet 5 <[email protected]> --------- Co-authored-by: Benjamin <[email protected]> Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * Add source for Gloucester City Council, UK (#6899) Closes #2909 * Update generated documentation * feat(portphillip_vic_gov_au): add source for City of Port Phillip, Victoria (#6900) The council's public IntraMaps GIS widget returns a weekly bin-collection day per address; general waste, recycling and FOGO are all collected together on that day. * Update generated documentation * feat(ics): add Gemeinde Neuberg, Germany (#6901) Neuberg publishes a single publicly accessible static ICS calendar covering the whole municipality; add it to the generic ICS provider list per issue #3646. * Update generated documentation * Add Stadt Tengen (Germany) as a new ICS provider (#6903) Closes #3634 * Update generated documentation * Add source for Haugaland Interkommunale Miljøverk (HIM), Norway (#6902) Implements a scraper for the him.as tømmekalender address-search page, covering Haugesund and surrounding municipalities. Closes #3654. * Update generated documentation * Add Gemeinde Hille (Germany) to ICS shared platform (#6904) Hille publishes static per-district ICS collection calendars on its own website; add them as a new entry in doc/ics/yaml so residents can configure the generic ICS source directly. Closes #3526. * Update generated documentation * feat(source): Add SUIBR (Nidwalden, CH) waste collection (#6905) * Update generated documentation * Add source for Aarberg, Switzerland (#3655) (#6906) Aarberg uses the same Weblication CMS as Rapperswil BE, with an additional zone filter covering Aarberg, Grafenmoos, Leimern, Mülital, Spins and Zälgli. * Update generated documentation * Add source: Předměřice nad Labem (predmerice_nad_labem_cz) (#6907) * Update generated documentation * feat(source): add De Afvalapp waste collection schedule (deafvalapp_nl) (#6909) Adds support for De Afvalapp (deafvalapp.nl), covering Maashorst, Helmond, Land van Cuijk and Boekel, requested in issue #5067. * Update generated documentation * Add source: East Suffolk Council (eastsuffolk_gov_uk) (#6910) Closes #5183. Adds bin collection schedule support for East Suffolk Council via their AchieveForms/Bartec-based "Bin collection dates finder", following the same pattern as other AchieveForms sources (e.g. tendring_gov_uk, eastcambs_gov_uk). * Update generated documentation * fix(communitywastedisposal_com): honor call-to-schedule flag and week-of-month ordinals (#6911) HHW (and occasionally Bulk/Yard Waste) streams marked "call to schedule" in CWD's route data were being shown as automatic weekly collections instead of being omitted, and Monthly/BiMonthly streams with an unspecified week-of-month silently defaulted to "every week" instead of being skipped. Also adds a Keller, TX test case (#5072) and switches to the canonical Icons enum. * fix(mijnafvalwijzer_nl): scope date/type lookup to each year's own div (#6912) The fetch loop iterated over each "jaar-XXXX" container but searched dates/types against the whole page instead of the current year's element, duplicating every collection date across all years present on the page and mislabeling half of them with the wrong year (e.g. around a year boundary when both the current and next year's calendars are rendered simultaneously). Fixes #5064 * Add source: GOAP Poznań (goap_poznan_pl) (#6913) Adds support for GOAP (Związek Międzygminny Gospodarka Odpadami Aglomeracji Poznańskiej), the waste-management union covering Murowana Goślina, Czerwonak, Swarzędz and other municipalities in the Poznań agglomeration, closing #5062. * Update generated documentation * feat(fermanaghomagh_gov_uk): add Fermanagh and Omagh District Council source (#6915) Closes #5162 * Update generated documentation * feat(source): add Southend-on-Sea City Council waste collection (southend_gov_uk) (#6916) Closes #4852. Uses the existing Cloud9Client service (already backing arun_gov_uk, northherts_gov_uk, rugby_gov_uk) against the "southend" authority, which powers the council's new Southend Waste app. * Update generated documentation * feat(source): add Surf Coast Shire, VIC, AU (surf_coast_vic_gov_au) (#6917) Adds support for Surf Coast Shire's kerbside bin collection via the same public WhatBinDay API already used by the SCRRApp mobile app, reusing the shared WhatBinDayService (as used by Kingston and Lismore). Closes #4971 * Update generated documentation * Add source: Gemeinde Muttenz (muttenz_ch) (#6918) Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * Add source: City of Tallahassee (talgov_com) (#6919) Implements weekly Garbage/Recycling and biweekly Red/Blue Bulky Items/Yard Waste collection lookup for the City of Tallahassee, FL, using the public JSON endpoints behind talgov.com's address-lookup widget (closes #5496). * Update generated documentation * refactor logan city council to use opencities API (#6920) * Update generated documentation * Add source: Gemeinde Kriftel (kriftel_de) (#6923) Closes #3401 * Update generated documentation * fix: correct 7 wrong council website domains in impactapps_com_au (#6921) Six entries had the wrong Australian state suffix (Brisbane pointed to .nsw instead of .qld; Campbelltown, Canada Bay, Cumberland, Port Stephens, and Wollongong pointed to .vic instead of .nsw) and Lithgow pointed to a lithgow.nsw.gov.au domain that doesn't resolve — the council's real site is council.lithgow.com. The `url` field (used for the actual waste-info.com.au API calls) was unaffected; only `website`, the reference link to each council's own site, was wrong. Found while building an attribution feature in a downstream app that surfaces this field to end users. Verified every corrected domain resolves and is the right council before fixing — cross-referenced against Wikipedia for the ones that didn't respond directly (network/geo restrictions on my end, not the domains themselves). * Update generated documentation * fix: add Le Cotentin (Cherbourg-en-Cotentin) to publidata_fr source (#6928) Closes #5129 * Update generated documentation * fix(ecoharmonogram_pl): disambiguate single/multi-family schedules for multi-id streets (#6924) Zabrze (and possibly other towns) bundle single-family and multi-family housing schedules under one street entry with two comma-separated ids whose `sides` field is identical, so users got both schedules merged into one calendar. Fall back to the housing-type letter in the schedule-name prefix and require additional_sides_matcher to pick one when this ambiguity is detected, while preserving legacy merge behavior for towns where multiple ids are genuinely meant to combine. Fixes #6922 Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * Add source for Stadt Kerpen (via abfall_io) (#6930) * Update generated documentation * feat(omrin_nl): add Omrin source (#6914) * feat(omrin_nl): add Omrin source * Address Omrin review feedback * Address follow-up Omrin review feedback * Update generated documentation * Add source: Perth and Kinross Council (pkc_gov_uk) (#6933) Closes #4529. Implements the AchieveForms-based "Bin collection dates" lookup at my.pkc.gov.uk, returning general/recycling/paper/food-garden waste collection dates by UPRN. * Update generated documentation * feat: add shared service/OpenCities.py client for MyArea/OpenCities councils (#6934) 37 source files independently hand-roll the same "OpenCities"/"MyArea" council-CMS flow (api/v1/myarea/search -> ocapi/Public/myarea/wasteservices) with drifting quality: inconsistent curl_cffi bot-protection, bare Exception raises, and only 1-2 sources implementing geolocation-id caching or ambiguous-address-match suggestions. This is batch 0 of a phased migration (see the execution plan): land the shared OpenCitiesConfig/OpenCitiesClient client and its unit tests first, with no source files touched yet, so the abstraction gets reviewed before 33 follow-on sources depend on its shape. Co-authored-by: Claude Sonnet 5 <[email protected]> * feat: add extra info for cities in RegioEntsorgung source (#6945) * Update generated documentation * refactor: migrate 6 OpenCities sources onto shared service/OpenCities.py (batch 1/5) (#6946) * refactor: migrate 6 OpenCities sources onto the shared service/OpenCities.py client Batch 1 of the phased OpenCities/MyArea consolidation (see PR #6934): kiama_nsw_gov_au, shoalhaven_nsw_gov_au, ballina_nsw_gov_au, cassowarycoast_qld_gov_au, lanecove_nsw_gov_au, willoughby_nsw_gov_au all now delegate to OpenCitiesClient instead of hand-rolling the search/wasteservices flow. Also extends the shared client based on real-world findings while migrating these six: - Some council HTML nests a matching <article> inside a matching div.waste-services-result (confirmed live on ballina and cassowarycoast), which the client's original selector double-counted. Fixed by only keeping the outermost match of a nested pair. - Added OpenCitiesConfig.require_date_precise for Shoalhaven, which mixes precise-date and vague/recurring blocks in the same response and only wants the former (a genuine behavioral difference, not just style). - lanecove and willoughby now 403 under plain requests (confirmed via a raw HTTP replay of their pre-migration code) and need curl_cffi, per the project's existing "403 from a Cloudflare/Akamai site -> curl_cffi" convention; bundled that fix into the migration. kiama's two TEST_CASES geolocation IDs return "no collection services found for this address" from the live API regardless of client implementation (verified against the exact pre-migration HTTP request) -- pre-existing stale test fixture data, not a regression from this change. Co-Authored-By: Claude Sonnet 5 <[email protected]> * change kiama testcases --------- Co-authored-by: Claude Sonnet 5 <[email protected]> * refactor: migrate 6 more OpenCities sources onto the shared client (batch 2/5) (#6951) hobartcity_com_au, moorabool_vic_gov_au, banyule_vic_gov_au, cambridge_wa_gov_au, mildura_vic_gov_au, and logan_qld_gov_au now delegate to OpenCitiesClient. These six were chosen because they collectively exercise every remaining piece of client-side variance not covered by batch 1: curl_cffi, the geolocation_id bypass kwarg, a non-default date_format, and hand-rolled caching/ambiguous-match handling that the shared client's defaults now subsume. Client extensions added while migrating, based on real API responses (not just the plan's sketch): - OpenCitiesConfig.strip_type_suffixes, used by moorabool to strip the "collection" suffix some labels carry. - OpenCitiesConfig.warm_up_before ("search" | "wasteservices"): banyule and cambridge fire their warm-up GET *after* resolving a geolocation id (whether via search or the direct bypass) and *before* the wasteservices call, not before search as the client originally assumed -- the ordering needed to be configurable, not hardcoded. - banyule and cambridge's search endpoints return XML instead of JSON unless an explicit `Accept: application/json` header is sent (confirmed via a raw HTTP replay); added that header to both configs rather than extending search_response_format, since it's an Accept-header quirk, not a deployment that only ever emits XML. Verified via live test_sources.py -l runs that Cambridge's apparent duplicate "General Waste" entry for one test address is genuine upstream duplicate data (confirmed by inspecting the raw wasteservices HTML: the API itself returns two identical blocks), not a client regression -- pre-migration code would have produced the same duplicate. Co-authored-by: Claude Sonnet 5 <[email protected]> * refactor: migrate 7 mechanical-family OpenCities sources (batch 3a/5) (#6953) mansfield_vic_gov_au, maroondah_vic_gov_au, melton_vic_gov_au, moyne_vic_gov_au, mrsc_vic_gov_au, nillumbik_vic_gov_au, and stonnington_vic_gov_au all previously copy-pasted the same warm-up-GET-then-search-then-wasteservices flow with a bare `raise Exception(...)` on no results; they now delegate to OpenCitiesClient. Fixes a real design flaw in the shared client found via live-testing, not an isolated bug: OpenCitiesConfig.strict_address_matching defaulted to True, meaning any multi-result search that didn't textually match the query's AddressSingleLine byte-for-byte (e.g. the server appends a state abbreviation the user's TEST_CASES address omits, as happened for both moyne and maroondah) raised a hard "ambiguous" failure instead of using the search API's own top-ranked (and, in both cases, correct) result. That default punished the common case -- an ordinary top-hit resolution -- to guard against a rare one. Flipped the default to False (trust the top search result, matching how the vast majority of these ~20 councils' original code already behaved) and made strict matching an explicit opt-in, set on hobartcity_com_au and moorabool_vic_gov_au (#6951) since those two are the only sources whose original hand-written code actually implemented this disambiguation themselves. Also confirmed two pre-existing, migration-unrelated outages while live-testing this batch (documented, not fixed here): - mansfield.vic.gov.au has moved off OpenCities entirely onto a Drupal 11 site; /api/v1/myarea/search now 404s regardless of implementation. - melton.vic.gov.au's wasteservices endpoint returns {"success": false} even when replaying the exact pre-migration request sequence (address search itself succeeds and returns a valid geolocation id). Co-authored-by: Claude Sonnet 5 <[email protected]> * refactor: migrate 7 more mechanical-family OpenCities sources (batch 3b/5) (#6955) wangaratta_vic_gov_au, whittlesea_vic_gov_au, yarra_ranges_vic_gov_au, mornpen_vic_gov_au, midcoast_nsw_gov_au, goldcoast_qld_gov_au, and monash_vic_gov_au now delegate to OpenCitiesClient. goldcoast's migration incidentally fixes a pre-existing "Https://" (capital H) typo in its wasteservices URL. Two new client extensions, both found in more than one source in this batch (not one-off, so added to the shared client rather than kept per-source): - OpenCitiesConfig.exclude_types: yarra_ranges and mornpen both mix an unrelated "Burning off" permit article into the wasteservices response and need it dropped. - OpenCitiesClient._parse_date now falls back to resolving "Every <Weekday>" recurring-service text (no concrete next date given at all) to the next occurrence of that weekday, needed by midcoast_nsw_gov_au; midcoast also keeps a local post-filter to the known waste types, matching its original allowlist behaviour (the wasteservices response includes non-waste informational articles too). Confirmed two more pre-existing, migration-unrelated issues while live-testing (documented, not fixed): - mornpen's three TEST_CASES addresses (all in the "<number> <Name> Rd <Suburb> VIC <postcode>" format) return zero search results against the live API today; replaying the exact same request confirms this is independent of implementation. Dropping "VIC <postcode>" and spelling out "Road" both return real matches, so this looks like an abbreviation-sensitive search index change on the council's end, not a broken address. Co-authored-by: Claude Sonnet 5 <[email protected]> * refactor: migrate final 2 mechanical-family OpenCities sources (batch 3c/5) (#6957) lakemac_nsw_gov_au and onkaparingacity_com now delegate to OpenCitiesClient, completing the mechanical old-style family. Both previously paired waste-type <h3> tags with next-service dates via two separate find_all() passes zipped together positionally -- a latent correctness bug (misaligns type/date if any block is skipped, e.g. a non-precise date filtered out of one list but not the other) that the client's per-block extraction doesn't have. Both also had `if addresses == 0: raise SourceArgumentNotFound(...)` guards that could never actually trigger (comparing a JSON dict to int 0), meaning an empty search result would previously crash with an IndexError instead of the intended error -- the client's real "no items" check fixes this too. New client extension: OpenCitiesConfig.exclude_type_prefixes, needed by onkaparingacity to drop "Calendar"-prefixed entries (calendar-download links, not real collections, with a varying zone-number suffix so an exact-match exclude_types wouldn't catch them all). Co-authored-by: Claude Sonnet 5 <[email protected]> * fix(abfall_io): migrate Entsorgungsbetriebe Essen to v3 GraphQL API (#6961) EBE Essen's legacy abfall.io key now returns 401; move the provider to the abfall_io_graphql source with its new v3 key and point the legacy source's error message at the supported replacement. Fixes #4775 * Update generated documentation * feat(source): add Abfallwirtschaft Altenburger Land to AWIDO service map (#6962) Closes #4794 Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * refactor: migrate final 6 OpenCities sources -- 4-field-address/XML family (batch 4/5) (#6960) * refactor: migrate final 6 OpenCities sources -- the 4-field-address/XML family (batch 4/5) blacktown_nsw_gov_au, campbelltown_nsw_gov_au, unley_sa_gov_au, ryde_nsw_gov_au, horowhenua_govt_nz, and hume_vic_gov_au now delegate to OpenCitiesClient. This is the last and trickiest batch: these all take 4 separate address fields (post_code/suburb/street_name/street_number, or post_code/town/street_name/street_number for horowhenua) instead of one address string, concatenated locally before calling OpenCitiesClient.fetch(address=...); campbelltown/unley exercise the JSON/XML search-response variance (json_then_xml and xml respectively). New client capability: OpenCitiesClient.get_waste_services_html(), a public method exposing the raw wasteservices HTML fragment (previously only reachable internally via fetch_by_geolocation_id). hume_vic_gov_au's `predict` feature needs each block's "note" div text (a fortnightly/ weekly hint) to extrapolate future dates, which the shared parser doesn't expose -- hume still delegates address search/resolution to the client, but does its own block-by-block parsing against the raw HTML only for the predict=True path. While preserving that parsing, also fixed a real crash: the original code referenced `weeks` unconditionally after an if/elif that could leave it unset, raising UnboundLocalError whenever a block's "note" text matched neither "fortnight" nor "same day each week"; it now safely falls back to the single resolved date in that case. Two more curl_cffi/Accept-header fixes found via live-testing (same "403 -> curl_cffi" and "no Accept header -> XML not JSON" patterns seen in earlier batches, not new categories): unley_sa_gov_au and horowhenua_govt_nz both now need curl_cffi, and horowhenua also needs an explicit `Accept: application/json` header on its search request. Confirmed one more pre-existing, unrelated outage: blacktown's "Plumpton Marketplace" test address's wasteservices call returns {"success": false} even replaying the exact pre-migration request -- the address search itself succeeds and returns a valid geolocation id, but the council's service-lookup step fails server-side for that specific address today. This completes the OpenCities/MyArea shared-service migration across all 6 batches (0, 1, 2, 3a, 3b, 3c, 4). Co-Authored-By: Claude Sonnet 5 <[email protected]> * remove non working testcase --------- Co-authored-by: Claude Sonnet 5 <[email protected]> * feat(source): add Seuzach (seuzach_ch) waste collection schedule (#6963) Live-parses both the special-collections table and the linked regular-collection detail pages on seuzach.ch/abfalldaten, so the weekly Kehrichtabfuhr/Grünabfuhr weekday and Grünabfuhr season are discovered at runtime rather than hardcoded. Closes #4685 * Update generated documentation * Republic Services: fix holiday schedule adjustments (#6964) * Republic Services: fix holiday schedule adjustments The v3 holiday endpoint returns an empty payload, and the delay lookup never matched the API's description string, so holidays were never applied. Switch to the v2 holiday endpoint and handle One Day Delay / Not Running / Service Moved, matching holidays to pickups by line of business. * Republic Services: run ruff format on holiday-adjustment fix Co-Authored-By: Claude Sonnet 5 <[email protected]> --------- Co-authored-by: carefulcomputer <[email protected]> Co-authored-by: Benjamin <[email protected]> Co-authored-by: Claude Sonnet 5 <[email protected]> * fix: remove rotherham_gov_uk source (backend no longer provides structured data) (#6966) Rotherham council's bin-collection page dropped its per-UPRN table lookup in favour of generic PDF calendars and a mobile app, so the source can no longer fetch personalized schedules. Rotherham is already fully covered by the apps_imactivate_com source, which two users in the issue confirmed works as a drop-in replacement. Fixes #4721 * Update generated documentation * Add Wirtschaftsbetriebe Duisburg (WBD) to abfall.io GraphQL source (#6968) WBD Duisburg's online waste calendar loads the abfall.io v3 widget with service key 80acad6c77fe9342ebafad29a8c58bf6, so it's already covered by the existing abfall_io_graphql source/service map. Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * feat(source): add KELL GmbH (Landkreis Leipzig) to Abfall.IO GraphQL service map (#6969) Also fix Origin/Referer handling so providers whose abfall.io v3 API key enforces an Origin allowlist (like KELL) don't get a 401 on the GraphQL appointments query. * Update generated documentation * feat: add Montri (montri.fr) source (#6965) * feat: add Montri source for French waste collection schedules (#5051) Implements the Montri platform (montri.fr) used by SICOVAD and other French intercommunal waste syndicates, via its anonymous-session JSON API (city lookup, address search, and monthly collection calendar). * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> * fix: add timeout to API requests in Montri source * Update generated documentation * feat: add Clackmannanshire Council (clackmannanshire_gov_uk) source (#6967) * feat: add Clackmannanshire Council (clackmannanshire_gov_uk) source Closes #4997. Fetches bin collection dates via the council's public postcode/address lookup and the per-property .ics calendar feed linked from the property details page. * fix: add timeout to HTTP requests in Clackmannanshire source * Update generated documentation * docs(ics): add Kelkheim (Taunus) to generic ICS source documentation (#6971) Kelkheim's mod_abfallkalender exposes a public, unauthenticated ICS feed already supported by the generic ICS source; add the doc/ics/yaml entry so it is discoverable, closing the gap left after issue #4245's comment. * Update generated documentation * feat(source): add Irado (irado_nl) waste collection source (#6973) * feat: add Irado (irado_nl) source for Vlaardingen and other municipalities Closes #4344 * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <[email protected]> --------- Co-authored-by: Copilot Autofix powered by AI <[email protected]> * feat(source): add Popůvky (popuvky_cz) waste collection schedule (#6972) * feat(source): add Popůvky (popuvky_cz) waste collection schedule (#4441) Adds support for the Obec Popůvky waste collection calendar, covering both collection routes (Popůvky village and the cottage/recreational area), parsed from the site's public AJAX calendar widget. * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <[email protected]> --------- Co-authored-by: Copilot Autofix powered by AI <[email protected]> * Update generated documentation * Add Uddevalla Energi source (#6974) * Update generated documentation * feat: add various agents and skills for waste collection management, including cleanup, new-source, release, issue review, and PR review functionalities (#6975) * Add Älvsbyns Energi source (#6976) * Update generated documentation * Add Peoria (IL) to RecycleCoach EXTRA_INFO (#6989) Peoria, IL already uses the RecycleCoach "My Schedule" widget on its official site; add it to the shared source's EXTRA_INFO list and a TEST_CASES entry using the example address from #5019. * Update generated documentation * fix(publidata_fr): add Sud Sainte Baume (Saint-Cyr-sur-Mer) to Publidata source (#6990) Adds the missing EXTRA_INFO/TEST_CASES entry for instance_id 1483. The "list index out of range" crash originally reported was already fixed by #5896; this closes out the remaining ask (listing the municipality) and adds a label/icon mapping for the "jrm" (paper/magazines) waste type seen in this instance's data. * Update generated documentation * fix(montreal_ca): fall back to Green schedule when Food message has no weekday (#6991) Some Montreal sectors (e.g. PIRO-1, Pierrefonds-Roxboro) report Food waste collection as merely "included in the collection of organic waste" with no weekday of its own, causing Food entries to silently be dropped. Reuse the matching Green (organic) sector's message, which does carry the collection day, and add a regression test case for the reported sector. * Update generated documentation * fix(carmarthenshire_gov_wales): handle UPRNs with no collection data (#6992) The council returns a container without a date paragraph when it has no record for a UPRN, causing an unhandled IndexError ("list index out of range"). Skip such containers and raise a clear SourceArgumentNotFound instead. Fixes #5256 * feat(source): add GLØR (glor_no) waste collection for Innlandet, Norway (#6993) Implements a source for GLØR, covering Lillehammer, Øyer, Ringebu and Gausdal municipalities, using the public proaktiv.glor.offcenit.no search/details JSON API. Closes #4487. * Update generated documentation * Update Lichfield source (#6986) * Update lichfielddc_gov_uk.py Updating Lichfield to accommodate new web layout and weekly food caddy collections. * Update lichfielddc_gov_uk.py Source formatting for test pass * Update lichfielddc_gov_uk.py More test formatting pedantry... * Update lichfielddc_gov_uk.py Spacing update * Update lichfielddc_gov_uk.py * Update lichfielddc_gov_uk.py ruff'd * Update lichfielddc_gov_uk.py spacing * Update lichfielddc_gov_uk.py "Sack" to "Bag" to match source. * Add Gargždų švara source for Klaipėda district, Lithuania (#6994) Closes #4419 * Update generated documentation * Add SIAVED (Valenciennes Métropole) to Publidata FR source (#6995) SIAVED runs on the Publidata platform already supported by publidata_fr.py; add it as a new EXTRA_INFO entry with a live test case, closes #4666. * Update generated documentation * Add Stadtwerke Aschaffenburg to Gipsprojekt ICS platform; remove stale MyMuell listing (#6996) Aschaffenburg's MyMuell (Junker) calendar has returned zero collection dates for every street since 2024 (confirmed live against the MyMuell API), so drop the dead city listing from jumomind_de.py. The city's utility, Stadtwerke Aschaffenburg, now publishes the schedule via the same "GIPS" system already documented as the shared Gipsprojekt ICS platform (used e.g. by Stadtwerke Speyer), so document it there instead of writing a new dedicated source. Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * Add City of Raleigh, NC to ReCollect shared-platform list (#6997) Raleigh's "Raleigh Reuse" waste tool (raleighnc.gov) is powered by the ReCollect platform (area CityofRaleighNC), already supported via the generic ICS source. Closes #4265. * Update generated documentation * fix(abfall_io_graphql): add Landkreis Göttingen to v3 GraphQL service map (#6998) Landkreis Göttingen uses the abfall.io v3 API with a flat idHouseNumber (no kommune/strasse hierarchy), which the legacy abfall_io source cannot drive. Add it to the GraphQL source's SERVICE_MAP so it is discoverable via EXTRA_INFO/config wizard, fixing #4125. * Update generated documentation * Add AWB Birkenfeld (Landkreis Birkenfeld) source (#7001) Fetches the embedded street/collection-group/date JSON from the AWB Birkenfeld waste calendar page to support all municipalities in Landkreis Birkenfeld, Germany. Closes #3762 * Update generated documentation * Add West Kelowna (BC) to recyclecoach_com source (#7003) West Kelowna shares the same RDCO/Recycle Coach project (502) as the existing Kelowna (BC) entry, distinguished only by district_id. Closes #7002 * Update generated documentation * fix(abfall_io): migrate ASG Nordsachsen to v3 GraphQL API (#7005) ASG Nordsachsen's legacy abfall.io key now returns 401; move the provider to the abfall_io_graphql source with its new v3 key found in the live widget embed on asg-nordsachsen.de. Fixes #4335 * Update generated documentation * Add Epsom and Ewell Borough Council to iapp_itouchvision_com source (#4388) (#7006) * Update generated documentation * fix(abfall_io_graphql): add Landkreis Böblingen to v3 GraphQL service map (#7007) Abfallwirtschaft Landkreis Böblingen (awb-bb.de, covering Böblingen and its Dagersheim district) uses the abfall.io v3 GraphQL widget. Add it to the GraphQL source's SERVICE_MAP so it is discoverable via EXTRA_INFO/config wizard, closing #3549. * Update generated documentation * Add RIR (Romsdalshalvøya Interkommunale Renovasjonsselskap) source (#7009) Fetches collection dates from the public address-lookup and per-month calendar HTML endpoints on rir.no, covering Aukra, Hustadvika, Gjemnes, Molde and Rauma, Norway. Closes #4301 * Add Czysty Region source (Opole/Silesia region, Poland) (#7010) Fetches waste collection schedules for the 15 communes served by Związek Międzygminny "Czysty Region" via the cascading gmina/rodzaj/ miejscowość lookup at czystyregion.pl/Harmonogramy, parsing the embedded FullCalendar events JSON from the resulting schedule page. Closes #7004 Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * fix(abfallnavi_de): don't abort fetch when a street name resolves to multiple split segments (#7011) Some AbfallNavi providers (e.g. Solingen) split long streets into several entries with the identical name but disjoint house-number ranges, keyed by city district. get_dates() previously aborted the whole fetch as soon as the first matched segment didn't contain the requested house number, even if a later segment did — causing empty calendars for any address on a split street (#4701). * Add Gemeinde Mils (Tyrol) source (#7012) Mils runs the same RiSKommunal platform already used by Fritzens and other Tyrolean municipalities; reuses the shared RiSKommunalSource base class for address-based (street + house number) lookup. Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * Add Gemeinde Rödinghausen to Abfall Export (vCal) ICS provider (#7013) Closes #3547 * Update generated documentation * Add Gronau (regio iT Abfallkalender) source (#7014) Closes #3585. Implements the kalender-wml regio iT Abfallkalender backend for Gronau (Kreis Borken, NRW): looks up the street's numeric ID from the session-scoped street list, then downloads the ICS feed for the selected waste fractions. * Update generated documentation * Add Gemeinde Kirchlengern (kirchlengern_de) source (#7015) Resolves the street name to an internal ID via the municipality's calendar widget, then fetches and parses the per-year vCal/ICS export. The feed double-encodes non-ASCII waste-type labels; a strict encode/decode round-trip repairs them without corrupting text that is already correct. Closes #3574 * Update generated documentation * Add Senden to MyMuell (Jumomind) source (#7016) Senden (Bavaria) is already served by the MyMuell App backend but was missing from the provider's documented city list. Closes #4162 * Update generated documentation * Add millvalleyrefuse source (#7018) * Add Mill Valley Refuse Service source Implement source for Mill Valley Refuse Service with ICS calendar integration for waste collection schedules. * Create documentation for Mill Valley Refuse Service Added documentation for Mill Valley Refuse Service, including recycling schedules and configuration instructions. * Update generated documentation * Add Schwarzwald-Baar-Kreis (Königsfeld) to Abfall.IO / AbfallPlus (GraphQL) (#7031) Resolves a source request for Königsfeld, Baden-Württemberg, which is served by the Schwarzwald-Baar-Kreis waste-management office via the abfall.io v3 widget already supported by the abfall_io_graphql source. * Update generated documentation * Fix: North Kesteven, UK — use curl_cffi to avoid Cloudflare blocks (#7032) North Kesteven's bin lookup endpoint intermittently enables Cloudflare's bot-check, which blocks plain requests sessions and causes fetch() to silently return no entries. Switch to curl_cffi with browser impersonation (the project's documented mitigation for Cloudflare-protected sites), and make icon lookup match on the leading colour keyword since the council has changed some bin type label wording (e.g. "Orange Lidded Caddy") since the source was first written. * Add City of Stamford, CT source (stamford_ct_us) (#7033) Adds a source for Stamford, CT's garbage/recycling pickup-day lookup (stamfordapps.org), resolving addresses by street name and an optional house number for streets with multiple pickup-day ranges. * Update generated documentation * fix(app_abfallplus_de): stop leaking other municipalities' Giftmobil dates for Landkreis München (#7034) The per-municipality "Sondermüll" skip heuristic in AppAbfallplusDe only matched that literal heading text, so apps whose provider labels the same per-town toggle list "Giftmobil" (e.g. de.k4systems.lkmabfallplus) fell through to a generic fallback that swept up every municipality's toggle id indiscriminately. The per-entry name match was also comparing against the <input>'s (always empty) text instead of the sibling <ion-item> label, so even the "Sondermüll" case never actually selected the searched town - it just got lucky by dropping the whole section instead of leaking. Fixes #7026 Co-authored-by: Claude Sonnet 5 <[email protected]> * Add San Giovanni Teatino to Junker APP source (#3780) (#7035) Junker already serves this municipality generically; list it in SERVICE_PROVIDERS and add live-test coverage for its zone-based collection schedule. * Update generated documentation * Add Nantes Métropole (data.nantesmetropole.fr) source (#7036) Closes #3779 * Update generated documentation * Add Marktgemeinde Pernitz (Austria) source (#7037) Fetches Restmüll (by Rayon 1/2), Biotonne, Gelber Sack, Gelber Container, and Papier collection dates from pernitz.gv.at. Closes #3741 * Update generated documentation * Prevent a stale/renamed source argument from crashing the whole integration (#7038) Source instantiation failures (e.g. after a source's constructor arguments changed, a failed config-entry migration, or a `customize` block accidentally nested under `args`) were raised uncaught out of SourceShell.create(). For YAML setups this aborted the entire component (every configured source went down), and for UI config entries it surfaced as a confusing TypeError instead of a clear "setup failed" reason. Fixes #3580 * Add Kansas City, MO source (#7043) Geocodes the address via the ArcGIS World Geocoder, then queries the city's "Trash Route Boundary" MapServer layer (TRASHDAY field) for the weekly collection day. Trash and recycling are collected on the same day city-wide, so recycling is derived rather than queried separately. Holiday-shifted dates are out of scope since KCMO's GIS exposes no holiday data. Co-authored-by: Claude Sonnet 5 <[email protected]> * Update generated documentation * Add siunet it (#7039) * add siunet_it source * Ignore codespell on italian strings * Update generated documentation * Add sludge (Slam) week-number pickup parsing to edpevent_se (#2355) Fall back to WastePickupFrequency (e.g. "Vecka 27") when NextWastePickup is empty and derive a date from the ISO week number. Keep the existing "v32 Aug 2024" form as an explicit branch and log a warning instead of crashing on unparseable values. Rebased onto current master and removed leftover debug prints per review. Co-authored-by: Benjamin <[email protected]> Co-authored-by: Claude Sonnet 5 <[email protected]> * feat(source): add KVA Thurgau (CH) covering 70 Thurgau communities (#6952) * feat(source): add KVA Thurgau (kvatg_ch) covering 70 Swiss communities Parses the per-community Entsorgungsplan PDFs from kvatg.ch (canton Thurgau plus neighbouring SH/SG communities) using pdfminer word coordinates: - month-grid date extraction incl. per-village/district zone rows, superscript zone markers, glued footnote digits, and year-rollover columns (DEZ | JAN '27) - weekly Kehricht expansion from the stated Abfuhrtag, applying explicitly listed holiday replacement dates - weekly Grüngut season spans ("Von Anfang März bis Ende November" / "Vom 6. März bis 27. November wöchentlich" incl. cancellations) - calendar-week rules (Frauenfeld "KW 2/4/6/8/10, KW 11-52" weekday zones) - inline street-collection dates (Metalle, Häckseldienst, Christbaum) Optional zone parameter filters zoned communities; without it, zoned collections are suffixed with their zone name. Verified against all 70 community PDFs (2026 plans). Closes #3738 Closes #4918 Co-Authored-By: Claude Fable 5 <[email protected]> * docs(kvatg_ch): correct Kehricht holiday-shift note The highlights bullet still said holiday shifts are not applied, which contradicts the parser (explicit replacement dates are now honoured) and the Known limitations section. Reword for consistency. Co-Authored-By: Claude Sonnet 5 <[email protected]> * Ignore codespell false positives in kvatg_ch translations "tre" (Italian "three") and "invalide" (French "invalid") are correct words in the localised PARAM_DESCRIPTIONS, not typos. Add them to .codespellignore so the validate check passes, matching the existing foreign-word entries. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * Revert "Ignore codespell false positives in kvatg_ch translations" This reverts commit e56d17a95aeb8a7b6471dabf70be12332da7999a. * Ignore codespell false positives in kvatg_ch translations "tre" (Italian "three") and "invalide" (French "invalid") are correct words in the localised PARAM_DESCRIPTIONS, not typos. Add them to .codespellignore alongside the existing foreign-word entries. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: Claude Fable 5 <[email protected]> Co-authored-by: Benjamin <[email protected]> Co-authored-by: Mark van Proctor <[email protected]> * Update generated documentation * Test against the supported Home Assistant floor (#6987) * fix: repair master CI (update-docs Python, kvatg_ch translation placeholders) (#7060) Two regressions surfaced on master after recent merges: 1. Update Documentation workflow failed: #6987 raised the requirements to homeassistant>=2024.4.0 (Python >=3.11) but update-docs.yaml still ran Python 3.10, so HA could not be installed. Bump it to 3.12, matching validate.yaml. 2. hassfest failed: kvatg_ch's PARAM_DESCRIPTIONS used markdown links ([label](url)), which the doc generator turned into HA translation placeholders that are not valid identifiers. De-link them to plain label + bare URL (bare URLs, as already used in this source's HOWTO, do not generate placeholders). Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> * Update generated documentation * fix(kvatg_ch): drop URLs from PARAM_DESCRIPTIONS to avoid config-form placeholders (#7061) update_docu_links.py extracts any URL (bare or markdown) from a param description into a {url_...} placeholder. For config-flow data_description those placeholders are never defined at runtime, so the form would show the literal {url_...} text (plus a stray paren). #7060 fixed the invalid-identifier hassfest error but left this cosmetic issue. Reference the sites by bare domain name (kvatg.ch, kreuzlingen.ch) instead; the full URLs remain in HOW_TO_GET_ARGUMENTS_DESCRIPTION, which the generator does not placeholder-ise. Verified extract_urls_from_text now yields no placeholders for any language/field. Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> * Update generated documentation * Test minimum and current Home Assistant runtimes in CI (#6999) * Test against the supported Home Assistant floor * Test minimum and current Home Assistant runtimes * Keep minimum Home Assistant lane importable * Allow retry after failed source fetch (#6983) * fix(config_flow): render the sensor selector as a list, not a dropdown (#6627) The options-flow sensor selector rendered as a dropdown overlay that overflowed the viewport once a config entry had many sensors. That selector has a fixed set of choices (custom_value=False), so SelectSelectorMode.LIST gives a bounded, scrollable list box instead. The custom-value selectors stay on DROPDOWN: Home Assistant's frontend ignores LIST mode whenever custom_value is set, so free-text entry of an unlisted waste type still works. * Adding Carleton Place, Ontario (#7062) * Adding Carleton Place, Ontario * Add Carleton Place TEST_CASE and icon mappings Cover the new CARLETON district with a live TEST_CASE using the Town Hall address so the nightly source tests exercise it, and map the waste types Carleton Place returns to the Icons enum. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Mark van Proctor <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * Update generated documentation * Fix centralbedfordshire new site (#7063) * Fix centralbedfordshire_gov_uk: adapt to new LocalGov Drupal site Central Bedfordshire Council migrated their waste collection page to the LocalGov Drupal Waste Collection module, which uses a different URL flow (postcode search -> UPRN lookup -> direct view by UPRN) and a different page structure than the old scraper expected. - Replace the old multi-step POST/session flow with two plain GETs - Parse dates from the <time datetime> attribute instead of text - Split the combined 'X, Y and Z collections' sentence into individual Collection entries per bin type - Deduplicate bin types per date, since some properties (e.g. those with a separate garden waste subscription) list the same date across two <li> rows * Apply ruff formatting * Fix centralbedfordshire_gov_uk error handling and docs - Raise SourceArgumentNotFound for an unrecognised postcode instead of a page-structure ValueError: the site returns 200 with no address dropdown when the postcode is not found, so the old message misattributed the most common user error. - Trim developer-facing instructions out of the remaining parse error. - Restore the "postcode without space" test coverage the old cases had, on the address that also exercises the garden-waste deduplication. - Update doc/source/centralbedfordshire_gov_uk.md for the new site: the old /info/163/ URL now redirects, and the house_name guidance described the old option format. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Mark van Proctor <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * Added geminfo_app.yaml to the ICS YAML folder to support Thannhausen … (#7040) * Added geminfo_app.yaml to the ICS YAML folder to support Thannhausen in Austria. * Added the geminfo_app.md file. * Corrections made according to the review comments from 23.07.2026. YAML modified. Please check and approve the changes. * Delete .vscode/settings.json Removed from commit * fix(7040): correct geminfo_app ICS provider metadata - add YAML document start marker so the yamlfmt hook passes - use the `at` country code instead of `Austria`, which would have orphaned the provider from README/info/sources.json - add an extra_info entry so Thannhausen is listed by name - drop doc/source/geminfo_app.md; ICS providers generate doc/ics/*.md - restore .vscode/settings.json, deleted unintentionally Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Mark van Proctor <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * Update generated documentation * Fix Esch schedule fetch with identity encoding (#7075) The Esch endpoint's compressed response advertises Transfer-Encoding: chunked twice. Some urllib3 versions reject it as multiple Transfer-Encoding headers, preventing schedule refreshes. Use the canonical trailing-slash endpoint and request identity encoding. This avoids the malformed compressed response without changing user configuration. Add a regression test covering the request and parsed schedule. * Add source provider for S.E.S.A. (issue #6870) (#6982) * sesa_it: add source file boilerplate * sesa_it: fetch pdf from user argument * sesa_it/draft: PDF parsing * sesa_it: working draft of a PDF parser * sesa_it: add source provider * sesaeste_it: fix date parsing, TLS chain, icon lookup and metadata - datetime.date has no strptime; every fetch raised AttributeError - app.sesaeste.it omits the RapidSSL intermediate, so requests and curl_cffi both fail certificate verification; use the same verify=False workaround as awn_de and others - add COUNTRY, PARAM_TRANSLATIONS and HOW_TO_GET_ARGUMENTS_DESCRIPTION - ICON_MAP.get() so an unmapped waste type cannot break the fetch - extend the calendar window into next year so it does not run dry in December - SourceArgumentNotFoundWithSuggestions instead of a bare not-found - add request timeouts, drop noisy info logging, tidy imports - add the missing doc/source/sesaeste_it.md Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Mark van Proctor <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * Update generated documentation * fix(aucklandcouncil_govt_nz): bypass WAF 406 and fail loudly on empty results (#7078) * fix(aucklandcouncil_govt_nz): bypass WAF 406 and fail loudly on empty results The source shipped a hard-coded Chrome/58 User-Agent (2017). Auckland Council's WAF now answers HTTP 406 Not Acceptable to it on the per-address collection page. fetch() never checked the response status, so the 406 error page was handed to BeautifulSoup, no acpl-schedule-card was found, and an empty list was returned - the coordinator logged a successful fetch with no data and nothing appeared in the log. The page markup itself is unchanged; only the request was being rejected. - Use curl_cffi with impersonate="chrome" instead of a hand-written User-Agent, so the client does not go stale again and also matches on TLS fingerprint if the WAF tightens further. - Call raise_for_status() so a future block surfaces as an error. - Raise SourceArgumentNotFound when no collections parse. An unknown area number returns HTTP 200 with the cards rendered but every date field blank, which previously produced the same silent empty schedule. - Extract parsing into a module-level parse_page() and move waste-type normalisation into a COLLECTION_TYPES map. - new.aucklandcouncil.govt.nz now redirects to www.aucklandcouncil.govt.nz; update URL, the fetch URL and the doc links accordingly. Fixes #7070 Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix(aucklandcouncil_govt_nz): add request timeout and capitalise DESCRIPTION Maintainer tweaks on top of the WAF fix: - Pass timeout=30 to the curl_cffi request so a stalled response cannot hang an update cycle, matching toogoodtowaste_co_nz.py. - Capitalise "Auckland Council" in DESCRIPTION, folded in from #7067. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Co-authored-by: Mark van Proctor <[email protected]> * Update generated documentation * Adds garden waste to herefordshire (#7073) * Adds garden waste to herefordshire * fix(herefordshire_gov_uk): correct ICON_MAP access and skip non-date garden entries ICON_MAP is a dict, so ICON_MAP.General raised AttributeError on every fetch and broke both test cases. The garden waste heading is rendered for all properties, but properties without a garden subscription get a calendar link instead of dates, which crashed strptime and took the other two bins down with it. Non-date list items are now skipped. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Mark van Proctor <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * fix(sudestavenir_fr): parse mixed green-waste schedules (#7079) * fix(sudestavenir_fr): parse bounded green-waste schedules * docs(sudestavenir_fr): describe mixed green-waste schedules The source doc still described dechets vegetaux as a precomputed date list only. Reflect that the precision text can also carry a bounded weekly rule, and note that bounded rules are not extrapolated into the following year. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: snowyukitty <[email protected]> Co-authored-by: Mark van Proctor <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * fix(swale_gov_uk): update form field IDs (#7071) Swale Borough Council updated the form field IDs on the Check Your Bin Day page. This updates both POST requests to use the current field names. * Add Coteau-du-Lac ICS waste schedule (#7081) * Add Coteau-du-Lac ICS waste schedule * Fix Coteau-du-Lac ICS provider definition - Correct the malformed test_cases block that failed test_ics_source_has_necessary_parameters - Collapse the two duplicate test cases into one (the feed is French only) - Rewrite the en/fr howto against the real portal instead of brossard.ca - Fix the title capitalisation and drop the trailing slash from url Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Mark van Proctor <[email protected]> Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * Update generated documentation * Release v2.32.0 (#7086) Bump manifest to 2.32.0, add the 2.32.0 changelog section and backfill the 2.31.1 section that was tagged but never merged to master. * test(v3): enforce component reuse in pipeline sources A pipeline source should compose shared components, not define its own. Behaviour written inside a source module is invisible to the next provider on the same platform, so it gets rewritten slightly differently and the two drift. Adds test_pipeline_sources_reuse_shared_components, which fails when a source declares its own Retriever or Parser subclass, with an allowlist for genuinely single-consumer cases. 242 discoverable pipeline sources pass today; regioentsorgung_de is the sole allowlisted entry. Documents the rule in CLAUDE.md and in the anti-patterns section of doc/contributing_source.md, calling out that it binds conversions as much as new sources. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * chore(v3): report declarative migration separately from pipeline debt arch_coverage now headlines sources that are pipeline AND fully declarative, and reports pipeline-with-own-steps as debt with a target of zero. Adds the cassette and step-override gates to the PR template. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix: convert southend_gov_uk to the v3 Cloud9 pipeline components southend_gov_uk arrived from master (#6916) still on the legacy fetch() contract, importing Cloud9Client from service/uk_cloud9_apps. v3 componentised that platform into Cloud9Retriever + Cloud9Parser, so the module failed to import and broke collection of the whole test suite. Re-implement it as a declarative BaseSource, matching arun_gov_uk (the other UPRN-or-postcode Cloud9 council): - retrieve = Cloud9Retriever("southend", ...), parse = Cloud9Parser(), transform = RowTransformer. No source-local step overrides. - PARAMS = alternatives([uprn()], [postcode()]) plus an optional address field, replacing the hand-rolled SourceArgumentRequired check. - ICON_MAP keyword guessing replaced by an explicit type_value_map onto canonical waste types. Southend collects mixed recycling (plastic, glass, cans) in the pink lid bin and paper and card in the blue lid bin, so those map to RECYCLABLES and PAPER respectively. - RAISE_ON_EMPTY for the address lookup path. TEST_CASES and the wire parameter names are unchanged, so existing user configurations keep working. Waste-type labels change, which is the expected breaking change for a pipeline migration. Drop the hand-written doc/source/southend_gov_uk.md: doc_generator.py renders it from the class metadata for pipeline sources. Record offline fixtures for all three test cases. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> (cherry picked from commit 343c6b2028872a3e85bb79fafcdaae546aff2929) * fix: convert yarracity_vic_gov_au to the v3 pipeline The source came across from master (#6888) importing get_next_n_dates and most_recent_weekday from service.ArcGis. Neither survived v3's ArcGIS componentisation, so the module failed to import and broke collection of the whole test suite. Rather than restore those helpers, the source is now declarative on the shared components: - retrieve/parse: ArcGisMultiFeatureRetriever + ArcGisMultiFeatureParser (one geocode, a spatial query against the glass and waste layers, each response tagged with its label) - preprocess: Compose(RecurrenceExpander, HolidayShift) - transform: ICSTransformer onto the canonical waste types The two missing helpers needed no platform work: most_recent_weekday already exists in recurrence, and get_next_n_dates is Schedule(anchor=True) via recurrence.recurring_from_anchor. The private weekday dict is gone in favour of recurrence.weekday(). Adds recurrence.good_friday(), replacing hand-rolled easter-minus-two-days arithmetic; armadale_wa_gov_au and subiaco_wa_gov_au duplicate the same calculation and can adopt it later. Behaviour, TEST_CASES and the codeowner are preserved: 52 weekly Rubbish and FOGO, 26 fortnightly Recycling, 13 Glass on the layer's own cycle, with Good Friday and Christmas Day deferred one day. Both test cases return 143 collections live. Also drops the hand-written doc page (pipeline sources generate it) and records offline fixtures for both test cases. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> (cherry picked from commit ae1acc292f145640098d24e22bad28215e2a03b0) * port(abfall_io): carry master's provider changes onto the v3 pipeline Ports the behaviour and data of three upstream PRs that landed on master while abfall_io was being rewritten as a BaseSource pipeline source. The v3 structure is kept throughout; none of master's legacy code shape is carried across. #6961 (Entsorgungsbetriebe Essen migrated to the v3 GraphQL API): drop the Essen entry from the abfall.io provider registry, which on v3 is the _PROVIDERS list feeding Source.REGIONS rather than the service module's SERVICE_MAP. Update the legacy 401 message to point users at the abfall_io_graphql source; on v3 that message lives in the shared AbfallIoRetriever, so every abfall.io source gets it. #7005 (ASG Nordsachsen migrated to the v3 GraphQL API): drop the ASG Nordsachsen entry from the same registry. #6930 (Stadt Kerpen): convert the source that arrived from master to this branch's abfall.io API. It now subclasses the pipeline Source and declares only metadata, REGIONS, TEST_CASES, PARAMS and __init__, the same shape rh_entsorgung_de uses. No retrieve, parse or transform override: the shared AbfallIoRetriever, AbfallIoParser and ICSTransformer do the work. PARAM_TRANSLATIONS, PARAM_DESCRIPTIONS and HOW_TO_GET_ARGUMENTS_DESCRIPTION give way to field_terms labels on a cascading_select plus HOWTO. This fixes the failing test_params_fields_match_init[stadt_kerpen_de]. Kerpen pins the abfall.io key and kommune, so those never appear in the config flow's selections and the inherited cascade could not resolve the street or house-number dropdowns. Rather than a bespoke get_choices in the source, abfall_io.Source gains a PINNED_PARAMS class attribute that get_choices merges in, so any pinned-provider subclass stays declarative. The matching GraphQL-side additions for Essen and ASG Nordsachsen belong to abfall_io_graphql.py and AbfallIOGraphQL.py and are handled separately. (cherry picked from commit 4c52a4e9c058a68711f5cbdf5a2c5a52b208d0da) * port(abfall_io_graphql): carry master's provider additions and Origin fix into v3 Ports the behaviour and data of the master commits that landed on abfall_io_graphql / AbfallIOGraphQL since the v3 branch point, keeping the v3 pipeline structure. Providers added to the source's _PROVIDERS registry (and so to REGIONS) plus a TEST_CASES entry each: - Entsorgungsbetriebe Essen (#6961) - Wirtschaftsbetriebe Duisburg (WBD) (#6968) - KELL Kommunalentsorgung Landkreis Leipzig GmbH (#6969) - Landkreis Goettingen (#6998) - ASG Nordsachsen (#7005) - Abfallwirtschaft Landkreis Boeblingen (#7007) - Amt fuer Abfallwirtschaft Schwarzwald-Baar-Kreis (#7031) Behavioural fix from #6969: providers that enforce an Origin allowlist on their key answer 401 to a GraphQL query with no matching Origin/Referer. On v3 this lives in the shared AbfallIoGraphQLRetriever, which resolves the provider's website from the source's REGIONS (the same lookup pattern service/AbfallnaviDe.py uses), so every abfall.io GraphQL source gets it and the source module stays declarative. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> (cherry picked from commit 3a86e52cc0c4ff783d18cc9f58d72708c37b3d89) * port(abfallnavi_de): carry master fixes #7011 and #7014 onto the v3 pipeline Ports the behaviour of upstream PR #7011 ("do not abort fetch when a street name resolves to multiple split segments") onto the v3 pipeline path. Master fixed AbfallnaviDe.get_dates(), which no longer exists on v3: the multi-street-id loop now lives in the shared AbfallnaviRetriever component, so the fix goes there and every Abfallnavi/regio iT provider gets it. Some providers split one street name into several entries with disjoint house-number ranges keyed by city district (Solingen "Katternberger Strasse", issue #4701). The retriever previously aborted the whole fetch on the first segment that did not contain the requested house number. It now skips that segment and keeps trying the rest, and only raises once no segment matched, with the combined suggestions from all of them. A single-segment street still propagates the original error unchanged. The raised argument name comes from the retriever's house_number field name rather than a hardcoded string, so it keeps matching the source's __init__ kwarg. Adds the Solingen regression test case from #7011 to TEST_CASES. Upstream PR #7014 (Gronau, regio iT Abfallkalender) needed no port: it added a standalone gronau_de.py against the kalender-wml backend, which is a different product from the AbfallNavi REST app, and the merge carried it across byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> (cherry picked from commit 6e79451c65062fad383d50631b6e4ab85ecf81ab) * port: jumomind_de provider-list changes from master (#7016, #6996) Port the MyMuell provider-list data from master into the v3 pipelin…
Summary
Closes #7077.
Type of change
Validation
python -m pytest tests/test_sudestavenir_fr.py -q— 8 passed (focused file; not selected by the defaultpytest.ini)python -m pytest tests/test_source_components.py -q— 34 passedpython -m pytest tests/ -q— 38 passed (default-discovered suite; does not include the focused file above)python test_sources.py -s sudestavenir_fr— 277 entries for the committed test caseChecklist
python -m pytest tests/test_source_components.py -qpassesruff check --fixandruff formatrun on both changed filesdoc/source/<name>.mdfile required