This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: logging.handlers.DatagramHandler sends bad data to socketserver.DatagramRequestHandler
Type: behavior Stage: resolved
Components: Documentation Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, vinay.sajip, yoonghm
Priority: normal Keywords: patch

Created on 2019-06-18 19:37 by yoonghm, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
attachment.md yoonghm, 2019-06-18 19:37 UDP server receive for syslog message
Pull Requests
URL Status Linked Edit
PR 14234 merged vinay.sajip, 2019-06-19 14:11
PR 14235 merged miss-islington, 2019-06-19 14:30
PR 14236 merged miss-islington, 2019-06-19 14:30
Messages (6)
msg346005 - (view) Author: Yoong Hor Meng (yoonghm) Date: 2019-06-18 19:37
`pysyslog.py`
- A UDP server listens at 0.0.0.0:514 for syslog messages.

`syslogmsg_gen.py`
- Send syslog message to local syslog server via 127.0.0.1:514


Error:

Exception has occurred: UnicodeDecodeError
'utf-8' codec can't decode byte 0xd7 in position 381: invalid continuation byte
  File "C:\pysyslog.py", line 21, in handle
    data = self.request[0].decode('utf-8')
  File "C:\pysyslog.py", line 28, in <module>
    server.serve_forever(poll_interval=0.5)
msg346020 - (view) Author: Yoong Hor Meng (yoonghm) Date: 2019-06-19 03:56
I did some checks, I discovered the following:

logging.handlers.DatagramHandler() sends data in the following format:
1. message length
2. message itself which consists of 
   a. Creation time in ASCII format
   b. Creation time in seconds, ms
   c. Filename, line no
   d. Message level
   e. Message
   f. Logger name
   etc.

Item 1. is packed using struct.pack(">L")
Item 2. is packed using pickle

The message format is not mentioned in any of the standard Python document.  Also it may not be compatible with third-party syslog message library or generator.
msg346048 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-06-19 13:57
It is mentioned here:

https://docs.python.org/3/library/logging.handlers.html?highlight=datagramhandler#logging.handlers.SocketHandler.makePickle

although that doesn't go into the detail of the struct.pack format. I'll update the documentation to rectify this.
msg346050 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-06-19 14:30
New changeset f06b569305cf604f070776ea3f800ed61fdd7d61 by Vinay Sajip in branch 'master':
bpo-37331: Clarify format of socket handler messages in the documentation. (GH-14234)
https://github.com/python/cpython/commit/f06b569305cf604f070776ea3f800ed61fdd7d61
msg346052 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-06-19 14:41
New changeset b64e42e931a3598d6f0605ec78673772f97ecd4c by Vinay Sajip (Miss Islington (bot)) in branch '3.7':
bpo-37331: Clarify format of socket handler messages in the documentation. (GH-14234) (GH-14236)
https://github.com/python/cpython/commit/b64e42e931a3598d6f0605ec78673772f97ecd4c
msg346053 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-06-19 14:42
New changeset d7232f0e4646803f0bbaede6e1fa124156135512 by Vinay Sajip (Miss Islington (bot)) in branch '3.8':
bpo-37331: Clarify format of socket handler messages in the documentation. (GH-14234) (GH-14235)
https://github.com/python/cpython/commit/d7232f0e4646803f0bbaede6e1fa124156135512
History
Date User Action Args
2022-04-11 14:59:16adminsetgithub: 81512
2019-06-19 14:43:45vinay.sajipsetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.8, Python 3.9
2019-06-19 14:42:38vinay.sajipsetmessages: + msg346053
2019-06-19 14:41:57vinay.sajipsetmessages: + msg346052
2019-06-19 14:30:45miss-islingtonsetpull_requests: + pull_request14072
2019-06-19 14:30:39miss-islingtonsetpull_requests: + pull_request14071
2019-06-19 14:30:22vinay.sajipsetmessages: + msg346050
2019-06-19 14:11:41vinay.sajipsetkeywords: + patch
stage: patch review
pull_requests: + pull_request14070
2019-06-19 13:57:38vinay.sajipsetnosy: + docs@python
messages: + msg346048

assignee: docs@python
components: + Documentation, - Library (Lib)
2019-06-19 10:00:25xtreaksetnosy: + vinay.sajip
2019-06-19 03:56:13yoonghmsetmessages: + msg346020
2019-06-18 19:37:02yoonghmcreate