Skip to content

Commit 0e29058

Browse files
authored
Add porting guide for users coming from orjson (msgspec#1007)
- Add docs/porting/ section with index for future guides (Pydantic, attrs) - Add docs/porting/orjson.rst covering API mapping, option equivalents, typed decoding, error handling, and features without direct equivalent - Fix incorrect workarounds for OPT_OMIT_MICROSECONDS, OPT_PASSTHROUGH_DATACLASS, OPT_PASSTHROUGH_SUBCLASS Closes msgspec#642
1 parent 88ce26d commit 0e29058

3 files changed

Lines changed: 346 additions & 0 deletions

File tree

‎docs/index.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,13 @@ few:
202202
inspect.rst
203203
perf-tips.rst
204204

205+
.. toctree::
206+
:hidden:
207+
:maxdepth: 2
208+
:caption: Porting
209+
210+
porting/index.rst
211+
205212
.. toctree::
206213
:hidden:
207214
:maxdepth: 2

‎docs/porting/index.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Porting guides
2+
==============
3+
4+
Guides for migrating to ``msgspec`` from other libraries.
5+
6+
.. toctree::
7+
:maxdepth: 1
8+
9+
orjson

‎docs/porting/orjson.rst‎

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
Porting from orjson
2+
===================
3+
4+
If you're currently using orjson_ and considering migrating to ``msgspec``,
5+
this guide covers the key API differences and shows how to translate common
6+
patterns.
7+
8+
Why migrate?
9+
------------
10+
11+
Both ``orjson`` and ``msgspec`` are high-performance JSON libraries written in
12+
compiled languages.
13+
14+
There are a few reasons you might prefer ``msgspec``:
15+
16+
- **Typed decoding and validation**. ``orjson.loads`` always returns plain
17+
Python builtins (``dict``, ``list``, ...). With ``msgspec`` you can decode
18+
directly into typed objects (``Struct``, ``dataclass``, ``TypedDict``, etc.)
19+
while validating the schema in a single pass. See :doc:`../usage` for details.
20+
21+
- **Schema evolution**. ``msgspec`` has first-class support for evolving
22+
schemas over time without breaking existing clients. See
23+
:doc:`../schema-evolution`.
24+
25+
- **Multiple protocols**. ``msgspec`` also supports MessagePack, YAML, and
26+
TOML with a consistent API, making it easy to switch between formats.
27+
28+
- **JSON Schema generation**. ``msgspec`` can generate JSON Schema definitions
29+
from your types via :func:`msgspec.json.schema`. Useful for OpenAPI
30+
documentation and API tooling.
31+
32+
Basic usage
33+
-----------
34+
35+
For most users, migration is straightforward -- swap function names and you're
36+
done:
37+
38+
.. code-block:: python
39+
40+
# Before (orjson)
41+
import orjson
42+
43+
data = orjson.dumps({"hello": "world"})
44+
obj = orjson.loads(data)
45+
46+
# After (msgspec)
47+
import msgspec
48+
49+
data = msgspec.json.encode({"hello": "world"})
50+
obj = msgspec.json.decode(data)
51+
52+
Both return ``bytes`` from encoding and accept ``bytes`` or ``str`` for
53+
decoding.
54+
55+
Typed decoding
56+
--------------
57+
58+
The main advantage of ``msgspec`` over ``orjson`` is typed decoding. Instead of
59+
getting a plain ``dict`` back, you can decode directly into structured types:
60+
61+
.. code-block:: python
62+
63+
import msgspec
64+
65+
class User(msgspec.Struct):
66+
name: str
67+
age: int
68+
email: str | None = None
69+
70+
data = b'{"name": "Alice", "age": 30}'
71+
user = msgspec.json.decode(data, type=User)
72+
73+
print(user.name) # "Alice"
74+
print(user.age) # 30
75+
print(user.email) # None
76+
77+
This validates the data while decoding -- if a field is missing or has the
78+
wrong type, a ``msgspec.ValidationError`` is raised with a clear error message.
79+
80+
Encoder and Decoder classes
81+
---------------------------
82+
83+
If you're making multiple calls, using the ``Encoder`` and ``Decoder`` classes
84+
is more efficient:
85+
86+
.. code-block:: python
87+
88+
# orjson has no stateful encoder/decoder
89+
90+
# msgspec
91+
encoder = msgspec.json.Encoder()
92+
decoder = msgspec.json.Decoder(User)
93+
94+
data = encoder.encode(user)
95+
user = decoder.decode(data)
96+
97+
Option mapping
98+
--------------
99+
100+
``orjson`` uses bitwise ``OPT_*`` flags. ``msgspec`` uses keyword arguments on
101+
``Encoder``/``Decoder`` or the ``encode``/``decode`` functions.
102+
103+
Sorting keys
104+
^^^^^^^^^^^^
105+
106+
.. code-block:: python
107+
108+
# orjson
109+
orjson.dumps(obj, option=orjson.OPT_SORT_KEYS)
110+
111+
# msgspec
112+
msgspec.json.encode(obj, order="sorted")
113+
114+
# Or with an Encoder
115+
encoder = msgspec.json.Encoder(order="sorted")
116+
117+
``msgspec`` also supports ``order="deterministic"`` which provides a
118+
consistent ordering without the cost of full alphabetical sorting.
119+
120+
Pretty-printing
121+
^^^^^^^^^^^^^^^
122+
123+
.. code-block:: python
124+
125+
# orjson
126+
orjson.dumps(obj, option=orjson.OPT_INDENT_2)
127+
128+
# msgspec -- use format() on already-encoded JSON
129+
raw = msgspec.json.encode(obj)
130+
pretty = msgspec.json.format(raw, indent=2)
131+
132+
Unlike ``orjson``'s single-pass approach, ``msgspec`` uses a two-step process:
133+
encode first, then format. ``msgspec.json.format`` works on any JSON bytes/str,
134+
not just ``msgspec``-encoded data, and supports arbitrary indent levels.
135+
136+
Custom type serialization
137+
^^^^^^^^^^^^^^^^^^^^^^^^^
138+
139+
.. code-block:: python
140+
141+
# orjson
142+
def default(obj):
143+
if isinstance(obj, complex):
144+
return {"real": obj.real, "imag": obj.imag}
145+
raise TypeError
146+
147+
orjson.dumps(obj, default=default)
148+
149+
# msgspec
150+
def enc_hook(obj):
151+
if isinstance(obj, complex):
152+
return {"real": obj.real, "imag": obj.imag}
153+
raise NotImplementedError
154+
155+
msgspec.json.encode(obj, enc_hook=enc_hook)
156+
157+
The key difference: ``orjson``'s ``default`` should raise ``TypeError`` for
158+
unhandled types, while ``msgspec``'s ``enc_hook`` should raise
159+
``NotImplementedError``.
160+
161+
Appending newlines
162+
^^^^^^^^^^^^^^^^^^
163+
164+
.. code-block:: python
165+
166+
# orjson
167+
orjson.dumps(obj, option=orjson.OPT_APPEND_NEWLINE)
168+
169+
# msgspec -- no built-in flag, just append manually
170+
msgspec.json.encode(obj) + b"\n"
171+
172+
Naive datetimes
173+
^^^^^^^^^^^^^^^
174+
175+
By default, both libraries encode naive datetimes identically (without
176+
timezone info):
177+
178+
.. code-block:: python
179+
180+
import datetime
181+
182+
dt = datetime.datetime(2024, 1, 1, 12, 0, 0)
183+
184+
orjson.dumps(dt) # b'"2024-01-01T12:00:00"'
185+
msgspec.json.encode(dt) # b'"2024-01-01T12:00:00"'
186+
187+
``orjson`` has ``OPT_NAIVE_UTC`` to force treating naive datetimes as UTC:
188+
189+
.. code-block:: python
190+
191+
orjson.dumps(dt, option=orjson.OPT_NAIVE_UTC)
192+
# b'"2024-01-01T12:00:00+00:00"'
193+
194+
``msgspec`` has no equivalent flag. Convert before encoding instead:
195+
196+
.. code-block:: python
197+
198+
def ensure_utc(dt):
199+
if dt.tzinfo is None:
200+
return dt.replace(tzinfo=datetime.timezone.utc)
201+
return dt
202+
203+
msgspec.json.encode(ensure_utc(dt))
204+
205+
.. note::
206+
207+
``enc_hook`` is only called for types that ``msgspec`` doesn't natively
208+
support. Since ``datetime`` is natively supported, you need to convert
209+
before passing to ``encode``.
210+
211+
Numpy arrays
212+
^^^^^^^^^^^^
213+
214+
.. code-block:: python
215+
216+
import numpy as np
217+
218+
# orjson -- native support
219+
orjson.dumps(arr, option=orjson.OPT_SERIALIZE_NUMPY)
220+
221+
# msgspec -- use enc_hook
222+
def enc_hook(obj):
223+
if isinstance(obj, np.ndarray):
224+
return obj.tolist()
225+
raise NotImplementedError
226+
227+
msgspec.json.encode(arr, enc_hook=enc_hook)
228+
229+
Pre-serialized JSON fragments
230+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
231+
232+
.. code-block:: python
233+
234+
# orjson
235+
orjson.dumps({"key": orjson.Fragment(b'{"already":"json"}')})
236+
237+
# msgspec
238+
msgspec.json.encode({"key": msgspec.Raw(b'{"already":"json"}')})
239+
240+
Decimal and UUID handling
241+
-------------------------
242+
243+
``msgspec`` has built-in support for both ``Decimal`` and ``UUID``, with
244+
configurable output formats:
245+
246+
.. code-block:: python
247+
248+
from decimal import Decimal
249+
from uuid import uuid4
250+
251+
enc = msgspec.json.Encoder(
252+
decimal_format="number", # or "string" (default)
253+
uuid_format="hex", # or "canonical" (default)
254+
)
255+
256+
``orjson`` requires a ``default`` callback for ``Decimal``.
257+
258+
Line-delimited JSON
259+
-------------------
260+
261+
``msgspec`` has built-in support for encoding and decoding newline-delimited
262+
JSON (NDJSON):
263+
264+
.. code-block:: python
265+
266+
encoder = msgspec.json.Encoder()
267+
decoder = msgspec.json.Decoder(User)
268+
269+
# Encode multiple objects as NDJSON
270+
data = encoder.encode_lines([user1, user2, user3])
271+
272+
# Decode NDJSON back
273+
users = decoder.decode_lines(data)
274+
275+
``orjson`` has no equivalent -- you'd need to encode each object separately and
276+
join with newlines.
277+
278+
Error handling
279+
--------------
280+
281+
.. code-block:: python
282+
283+
# orjson
284+
try:
285+
orjson.loads(bad_data)
286+
except orjson.JSONDecodeError:
287+
...
288+
289+
# msgspec
290+
try:
291+
msgspec.json.decode(bad_data, type=User)
292+
except msgspec.ValidationError:
293+
# Schema validation error (wrong type, missing field, etc.)
294+
...
295+
except msgspec.DecodeError:
296+
# Malformed JSON
297+
...
298+
299+
``msgspec`` distinguishes between malformed JSON (``DecodeError``) and valid
300+
JSON that doesn't match the schema (``ValidationError``, a subclass of
301+
``DecodeError``). Note that ``ValidationError`` must be caught first since it
302+
inherits from ``DecodeError``.
303+
304+
Features with no direct equivalent
305+
-----------------------------------
306+
307+
Some ``orjson`` options have no direct ``msgspec`` flag, but can be achieved
308+
through other means:
309+
310+
.. list-table::
311+
:header-rows: 1
312+
313+
* - orjson
314+
- msgspec workaround
315+
* - ``OPT_NON_STR_KEYS``
316+
- Use ``enc_hook``/``dec_hook`` or ``msgspec.to_builtins(obj, str_keys=True)``
317+
* - ``OPT_OMIT_MICROSECONDS``
318+
- Convert before encoding: ``dt.replace(microsecond=0)``. ``enc_hook`` cannot
319+
be used here because ``datetime`` is a natively supported type.
320+
* - ``OPT_STRICT_INTEGER`` (53-bit)
321+
- Use ``Meta(ge=-(2**53 - 1), le=2**53 - 1)`` constraint
322+
* - ``OPT_PASSTHROUGH_DATACLASS``
323+
- Not directly supported. ``dataclass`` is natively supported, so ``enc_hook``
324+
is not called for it. Convert to a ``dict`` before encoding if you need
325+
custom serialization.
326+
* - ``OPT_PASSTHROUGH_SUBCLASS``
327+
- Not applicable. ``msgspec`` always serializes subclasses of builtins as
328+
their parent type, with no option to redirect them to ``enc_hook``.
329+
330+
.. _orjson: https://github.com/ijl/orjson

0 commit comments

Comments
 (0)