ActivityPub services are using multi accept headers #1385
Description
Describe the bug
Since v1.7.4 we have Nginx logging improvements, to filter out ActivityPub requests from "regular" requests. This allowed me to find issues within Mbin.
While I do notice that most AP requests are processing correctly by our code base and the Nginx filter for Mbin & Lemmy instances. (both using the same check on the HTTP Accept headers). These ActivityPub calls are indeed ending up in the new mbin_instance.log.
So it seems that some AP services are using multiple HTTP accept headers, we need to be robust for that in our Nginx regex. Now these AP requests will still end up in the normal mbin_access.log log instead of the mbin_instance.log.
Examples are:
3.84.57.0 - - [18/Jan/2025:17:17:12 +0100] "GET /f/object/1e77f22d-23d8-499f-b643-4568b1c7989a HTTP/1.1" 200 12 "-" "http.rb/5.2.0 (Mastodon/3.5.19-qoto; +https://qoto.org/)" "kbin.melroy.org" 0.004
168.119.66.102 - - [18/Jan/2025:17:17:12 +0100] "GET /f/object/1e77f22d-23d8-499f-b643-4568b1c7989a HTTP/1.1" 200 12 "-" "Mastodon/4.3.0-nightly.2024-10-01+glitch (http.rb/5.2.0; +https://tweesecake.social/)" "kbin.melroy.org" 0.004
54.249.151.125 - - [18/Jan/2025:17:17:13 +0100] "GET /f/object/1e77f22d-23d8-499f-b643-4568b1c7989a HTTP/1.1" 200 12 "-" "http.rb/5.1.1 (Mastodon/4.1.15; +https://pawoo.net/)" "kbin.melroy.org" 0.005
95.216.6.56 - - [18/Jan/2025:17:17:15 +0100] "GET /f/object/1e77f22d-23d8-499f-b643-4568b1c7989a HTTP/1.1" 200 12 "-" "Mastodon/4.3.0 (http.rb/5.2.0; +https://lile.cl/)" "kbin.melroy.org" 0.004
51.15.134.186 - - [18/Jan/2025:17:17:16 +0100] "GET /f/object/1e77f22d-23d8-499f-b643-4568b1c7989a HTTP/1.1" 200 12 "-" "Akkoma 3.13.1-0-gc02e343; https://bcn.fedi.cat <[email protected]>" "kbin.melroy.org" 0.005
178.21.23.139 - - [18/Jan/2025:17:17:16 +0100] "GET /f/object/4cb432ee-8c20-40e3-8cc9-7fefcb054cd6 HTTP/1.1" 200 12 "-" "Akkoma 3.13.3-0-g963467f; https://fe.disroot.org <[email protected]>" "kbin.melroy.org" 0.004
5.161.62.139 - - [18/Jan/2025:17:17:17 +0100] "GET /f/object/1e77f22d-23d8-499f-b643-4568b1c7989a HTTP/1.1" 200 12 "-" "Mastodon/4.3.3 (http.rb/5.2.0; +https://social.lol/)" "kbin.melroy.org" 0.004
3.84.57.0 - - [18/Jan/2025:17:19:21 +0100] "GET /f/object/938b5607-e38f-4f26-92ca-bf87d69e6a9c HTTP/1.1" 200 12 "-" "http.rb/5.2.0 (Mastodon/3.5.19-qoto; +https://qoto.org/)" "kbin.melroy.org" 0.005
35.90.255.36 - - [18/Jan/2025:17:26:15 +0100] "GET /f/object/1e77f22d-23d8-499f-b643-4568b1c7989a HTTP/1.1" 200 12 "-" "Mastodon/4.3.3 (http.rb/5.2.0; +https://pnw.zone/)" "kbin.melroy.org" 0.004
35.90.255.36 - - [18/Jan/2025:17:31:01 +0100] "GET /f/object/701388c6-ff33-4e16-8daf-88fb12060612 HTTP/1.1" 200 12 "-" "Mastodon/4.3.3 (http.rb/5.2.0; +https://pnw.zone/)" "kbin.melroy.org" 0.005And:
95.111.230.148 - - [18/Jan/2025:17:15:09 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "Mastodon/4.4.0-alpha.1+glitch (http.rb/5.2.0; +https://mnstdn.monster/)" "kbin.melroy.org" 0.016
95.111.230.148 - - [18/Jan/2025:17:15:09 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "Mastodon/4.4.0-alpha.1+glitch (http.rb/5.2.0; +https://mnstdn.monster/)" "kbin.melroy.org" 0.015
3.84.57.0 - - [18/Jan/2025:17:15:13 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "http.rb/5.2.0 (Mastodon/3.5.19-qoto; +https://qoto.org/)" "kbin.melroy.org" 0.015
3.84.57.0 - - [18/Jan/2025:17:15:14 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "http.rb/5.2.0 (Mastodon/3.5.19-qoto; +https://qoto.org/)" "kbin.melroy.org" 0.016
149.28.78.238 - - [18/Jan/2025:17:15:14 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.1.18; +https://noc.social/)" "kbin.melroy.org" 0.018
149.28.78.238 - - [18/Jan/2025:17:15:15 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.1.18; +https://noc.social/)" "kbin.melroy.org" 0.015
212.85.67.30 - - [18/Jan/2025:17:15:16 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://noclick.se/)" "kbin.melroy.org" 0.016
212.85.67.30 - - [18/Jan/2025:17:15:16 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://noclick.se/)" "kbin.melroy.org" 0.015
212.85.67.30 - - [18/Jan/2025:17:15:36 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://noclick.se/)" "kbin.melroy.org" 0.013
212.85.67.30 - - [18/Jan/2025:17:15:36 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://noclick.se/)" "kbin.melroy.org" 0.014
45.13.104.88 - - [18/Jan/2025:17:15:51 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://mastouille.fr/)" "kbin.melroy.org" 0.015
45.13.104.88 - - [18/Jan/2025:17:16:01 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://mastouille.fr/)" "kbin.melroy.org" 0.015
212.85.67.30 - - [18/Jan/2025:17:16:16 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://noclick.se/)" "kbin.melroy.org" 0.015
212.85.67.30 - - [18/Jan/2025:17:16:16 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://noclick.se/)" "kbin.melroy.org" 0.015
159.69.152.150 - - [18/Jan/2025:17:17:54 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "Mastodon/4.4.0-alpha.2 (http.rb/5.2.0; +https://101010.pl/)" "kbin.melroy.org" 0.014
159.69.152.150 - - [18/Jan/2025:17:17:54 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "Mastodon/4.4.0-alpha.2 (http.rb/5.2.0; +https://101010.pl/)" "kbin.melroy.org" 0.013
80.67.181.194 - - [18/Jan/2025:17:18:03 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "Mastodon/4.3.3 (http.rb/5.2.0; +https://kfem.cat/)" "kbin.melroy.org" 0.015
80.67.181.194 - - [18/Jan/2025:17:18:44 +0100] "GET /u/ElcaineVolta/followers HTTP/1.1" 200 241 "-" "Mastodon/4.3.3 (http.rb/5.2.0; +https://kfem.cat/)" "kbin.melroy.org" 0.016
212.85.67.30 - - [18/Jan/2025:17:20:31 +0100] "GET /u/ElcaineVolta/following HTTP/1.1" 200 241 "-" "http.rb/5.1.1 (Mastodon/4.2.10; +https://noclick.se/)" "kbin.melroy.org" 0.016On which Mbin instance did you find the bug?
kbin.melroy.org
Which Mbin version was running on the instance?
1.7.4
To Reproduce
Steps to reproduce the behavior:
- Use the latest Nginx configs
- Check the different Nginx log files
- Notice that some activitypub calls are ending up in the
mbin_access.log
Expected behavior
Even with multiple HTTP Accept headers, the Nginx regex mapping should work.
Meaning the Nginx $http_accept variable can be an array.
Yes getAcceptableContentTypes call is retrieving the value of the accept header: https://github.com/symfony/symfony/blob/7b0cdc85ee5a3adf372af80d4c790bf9de7a2a78/src/Symfony/Component/HttpFoundation/Request.php#L1697
And also yes the $http_accept in Nginx should reflect the same accept header. But this can be an array of values (separated by commas)
Additional context
Add any other context about the problem here.