logging: Perform filtering on arrays of strings (where possible)#5101
logging: Perform filtering on arrays of strings (where possible)#5101
Conversation
95ad462 to
00a028f
Compare
mholt
left a comment
There was a problem hiding this comment.
Wow, thank you for this fix and for the tests. I haven't tried it myself but I trust it works, given the evidence and who's contributing it :)
LGTM, thanks a bunch!
|
Hi! Thanks for the work you did there that is specifically very useful in my workflow today 😊
Indeed, the query filter is quite useful also in other fields, like the I guess that's not a common usecase. The only workaround I see there is to use regexp, which makes it quite heavy. Thanks a lot! |
|
Ah I didn't think of the Referer header. Good point. If you want to contribute a fix, PRs are welcome. |
|
Alright. I will try to contribute asap! |
|
Thanks for your contribution! |
Thanks to @IndeedNotJames for reporting this problem.
When trying to filter HTTP headers in the logs, many filters don't work at all because they didn't handle fields that are array values, which all headers are because they can have multiple values.
These changes are mainly for HTTP headers, but it'll work in general for any
LoggableStringArraytype being logged.Now, the
regexp,hashandip_maskfilters will correctly iterate over each string in theLoggableStringArrayand apply their filtering on those.Additionally, for
ip_mask, we now split on commas and mask each segment that parses as an IP address. This should be useful forX-Forwarded-Fortype headers which may be a list of comma separated IPs.And finally, I fixed a bug with
renamewhere it would set the zap field type to String, which would totally clobber any array typed fields with an empty string instead. That was just a total oversight when I implemented that filter, because it was basically copy-paste from thereplacefilter.As for the other filters, I didn't touch them for various reasons.
replacedoesn't iterate because it could be a valid usecase to change the type of a field to string... but also maybe not. I dunno about that one, you can probably just useregexpin its place. And fordelete, it's not possible to delete a particular entry in an array; we'd need a new filter to do that, which is totally possible, but I'm not sure the usecase (when is there ever a guarantee that a particular index exists?). Alsocookieandqueryalready do their job; they're very specifically targeted at theCookieheader (and iterates itself just fine), and theurifield.