Skip to content

Commit a670c34

Browse files
author
aleksandr.tseluyko
committed
Add porting guide for users coming from orjson
Cover API mapping, option equivalents, typed decoding, error handling, and features without direct equivalent. Closes #642
1 parent b59564d commit a670c34

2 files changed

Lines changed: 323 additions & 0 deletions

File tree

‎docs/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,7 @@ few:
201201
extending.rst
202202
inspect.rst
203203
perf-tips.rst
204+
porting-from-orjson.rst
204205

205206
.. toctree::
206207
:hidden:

‎docs/porting-from-orjson.rst‎

Lines changed: 322 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,322 @@
1+
Porting from orjson
2+
===================
3+
4+
If you're currently using orjson_ and considering migrating to ``msgspec``,
5+
this guide covers the key API differences and shows how to translate common
6+
patterns.
7+
8+
Why migrate?
9+
------------
10+
11+
Both ``orjson`` and ``msgspec`` are high-performance JSON libraries written in
12+
compiled languages. Performance is generally comparable -- for some schemas
13+
``msgspec`` is faster, for others ``orjson`` is. For common message schemas
14+
things are roughly equivalent.
15+
16+
There are a few reasons you might prefer ``msgspec``:
17+
18+
- **Typed decoding and validation**. ``orjson.loads`` always returns plain
19+
Python builtins (``dict``, ``list``, ...). With ``msgspec`` you can decode
20+
directly into typed objects (``Struct``, ``dataclass``, ``TypedDict``, etc.)
21+
while validating the schema in a single pass. See :doc:`usage` for details.
22+
23+
- **Schema evolution**. ``msgspec`` has first-class support for evolving
24+
schemas over time without breaking existing clients. See
25+
:doc:`schema-evolution`.
26+
27+
- **Multiple protocols**. ``msgspec`` also supports MessagePack, YAML, and
28+
TOML with a consistent API, making it easy to switch between formats.
29+
30+
- **JSON Schema generation**. ``msgspec`` can generate JSON Schema definitions
31+
from your types via :func:`msgspec.json.schema`. Useful for OpenAPI
32+
documentation and API tooling.
33+
34+
Basic usage
35+
-----------
36+
37+
For most users, migration is straightforward -- swap function names and you're
38+
done:
39+
40+
.. code-block:: python
41+
42+
# Before (orjson)
43+
import orjson
44+
45+
data = orjson.dumps({"hello": "world"})
46+
obj = orjson.loads(data)
47+
48+
# After (msgspec)
49+
import msgspec
50+
51+
data = msgspec.json.encode({"hello": "world"})
52+
obj = msgspec.json.decode(data)
53+
54+
Both return ``bytes`` from encoding and accept ``bytes`` or ``str`` for
55+
decoding.
56+
57+
Typed decoding
58+
--------------
59+
60+
The main advantage of ``msgspec`` over ``orjson`` is typed decoding. Instead of
61+
getting a plain ``dict`` back, you can decode directly into structured types:
62+
63+
.. code-block:: python
64+
65+
import msgspec
66+
67+
class User(msgspec.Struct):
68+
name: str
69+
age: int
70+
email: str | None = None
71+
72+
data = b'{"name": "Alice", "age": 30}'
73+
user = msgspec.json.decode(data, type=User)
74+
75+
print(user.name) # "Alice"
76+
print(user.age) # 30
77+
print(user.email) # None
78+
79+
This validates the data while decoding -- if a field is missing or has the
80+
wrong type, a ``msgspec.ValidationError`` is raised with a clear error message.
81+
82+
Encoder and Decoder classes
83+
---------------------------
84+
85+
If you're making multiple calls, using the ``Encoder`` and ``Decoder`` classes
86+
is more efficient:
87+
88+
.. code-block:: python
89+
90+
# orjson has no stateful encoder/decoder
91+
92+
# msgspec
93+
encoder = msgspec.json.Encoder()
94+
decoder = msgspec.json.Decoder(User)
95+
96+
data = encoder.encode(user)
97+
user = decoder.decode(data)
98+
99+
Option mapping
100+
--------------
101+
102+
``orjson`` uses bitwise ``OPT_*`` flags. ``msgspec`` uses keyword arguments on
103+
``Encoder``/``Decoder`` or the ``encode``/``decode`` functions.
104+
105+
Sorting keys
106+
^^^^^^^^^^^^
107+
108+
.. code-block:: python
109+
110+
# orjson
111+
orjson.dumps(obj, option=orjson.OPT_SORT_KEYS)
112+
113+
# msgspec
114+
msgspec.json.encode(obj, order="sorted")
115+
116+
# Or with an Encoder
117+
encoder = msgspec.json.Encoder(order="sorted")
118+
119+
``msgspec`` also supports ``order="deterministic"`` which provides a
120+
consistent ordering without the cost of full alphabetical sorting.
121+
122+
Pretty-printing
123+
^^^^^^^^^^^^^^^
124+
125+
.. code-block:: python
126+
127+
# orjson
128+
orjson.dumps(obj, option=orjson.OPT_INDENT_2)
129+
130+
# msgspec -- use format() on already-encoded JSON
131+
raw = msgspec.json.encode(obj)
132+
pretty = msgspec.json.format(raw, indent=2)
133+
134+
Note that ``msgspec.json.format`` works on any JSON bytes/str, not just
135+
``msgspec``-encoded data. It also supports arbitrary indent levels.
136+
137+
Custom type serialization
138+
^^^^^^^^^^^^^^^^^^^^^^^^^
139+
140+
.. code-block:: python
141+
142+
# orjson
143+
def default(obj):
144+
if isinstance(obj, complex):
145+
return {"real": obj.real, "imag": obj.imag}
146+
raise TypeError
147+
148+
orjson.dumps(obj, default=default)
149+
150+
# msgspec
151+
def enc_hook(obj):
152+
if isinstance(obj, complex):
153+
return {"real": obj.real, "imag": obj.imag}
154+
raise NotImplementedError
155+
156+
msgspec.json.encode(obj, enc_hook=enc_hook)
157+
158+
The key difference: ``orjson``'s ``default`` should raise ``TypeError`` for
159+
unhandled types, while ``msgspec``'s ``enc_hook`` should raise
160+
``NotImplementedError``.
161+
162+
Appending newlines
163+
^^^^^^^^^^^^^^^^^^
164+
165+
.. code-block:: python
166+
167+
# orjson
168+
orjson.dumps(obj, option=orjson.OPT_APPEND_NEWLINE)
169+
170+
# msgspec -- no built-in flag, just append manually
171+
msgspec.json.encode(obj) + b"\n"
172+
173+
Naive datetimes
174+
^^^^^^^^^^^^^^^
175+
176+
.. code-block:: python
177+
178+
import datetime
179+
180+
dt = datetime.datetime(2024, 1, 1, 12, 0, 0)
181+
182+
# orjson with OPT_NAIVE_UTC -- treats naive as UTC, adds "+00:00"
183+
orjson.dumps(dt, option=orjson.OPT_NAIVE_UTC)
184+
# b'"2024-01-01T12:00:00+00:00"'
185+
186+
# msgspec -- encodes naive datetimes without timezone
187+
msgspec.json.encode(dt)
188+
# b'"2024-01-01T12:00:00"'
189+
190+
If you need to force UTC, convert before encoding:
191+
192+
.. code-block:: python
193+
194+
def ensure_utc(dt):
195+
if dt.tzinfo is None:
196+
return dt.replace(tzinfo=datetime.timezone.utc)
197+
return dt
198+
199+
msgspec.json.encode(ensure_utc(dt))
200+
201+
.. note::
202+
203+
``enc_hook`` is only called for types that ``msgspec`` doesn't natively
204+
support. Since ``datetime`` is natively supported, you need to convert
205+
before passing to ``encode``.
206+
207+
Numpy arrays
208+
^^^^^^^^^^^^
209+
210+
.. code-block:: python
211+
212+
import numpy as np
213+
214+
# orjson -- native support
215+
orjson.dumps(arr, option=orjson.OPT_SERIALIZE_NUMPY)
216+
217+
# msgspec -- use enc_hook
218+
def enc_hook(obj):
219+
if isinstance(obj, np.ndarray):
220+
return obj.tolist()
221+
raise NotImplementedError
222+
223+
msgspec.json.encode(arr, enc_hook=enc_hook)
224+
225+
Pre-serialized JSON fragments
226+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
227+
228+
.. code-block:: python
229+
230+
# orjson
231+
orjson.dumps({"key": orjson.Fragment(b'{"already":"json"}')})
232+
233+
# msgspec
234+
msgspec.json.encode({"key": msgspec.Raw(b'{"already":"json"}')})
235+
236+
Decimal and UUID handling
237+
-------------------------
238+
239+
``msgspec`` has built-in support for both ``Decimal`` and ``UUID``, with
240+
configurable output formats:
241+
242+
.. code-block:: python
243+
244+
from decimal import Decimal
245+
from uuid import uuid4
246+
247+
enc = msgspec.json.Encoder(
248+
decimal_format="number", # or "string" (default)
249+
uuid_format="hex", # or "canonical" (default)
250+
)
251+
252+
``orjson`` requires a ``default`` callback for ``Decimal``.
253+
254+
Line-delimited JSON
255+
-------------------
256+
257+
``msgspec`` has built-in support for encoding and decoding newline-delimited
258+
JSON (NDJSON):
259+
260+
.. code-block:: python
261+
262+
encoder = msgspec.json.Encoder()
263+
decoder = msgspec.json.Decoder(User)
264+
265+
# Encode multiple objects as NDJSON
266+
data = encoder.encode_lines([user1, user2, user3])
267+
268+
# Decode NDJSON back
269+
users = decoder.decode_lines(data)
270+
271+
``orjson`` has no equivalent -- you'd need to encode each object separately and
272+
join with newlines.
273+
274+
Error handling
275+
--------------
276+
277+
.. code-block:: python
278+
279+
# orjson
280+
try:
281+
orjson.loads(bad_data)
282+
except orjson.JSONDecodeError:
283+
...
284+
285+
# msgspec
286+
try:
287+
msgspec.json.decode(bad_data, type=User)
288+
except msgspec.ValidationError:
289+
# Schema validation error (wrong type, missing field, etc.)
290+
...
291+
except msgspec.DecodeError:
292+
# Malformed JSON
293+
...
294+
295+
``msgspec`` distinguishes between malformed JSON (``DecodeError``) and valid
296+
JSON that doesn't match the schema (``ValidationError``, a subclass of
297+
``DecodeError``). Note that ``ValidationError`` must be caught first since it
298+
inherits from ``DecodeError``.
299+
300+
Features with no direct equivalent
301+
-----------------------------------
302+
303+
Some ``orjson`` options have no direct ``msgspec`` flag, but can be achieved
304+
through other means:
305+
306+
.. list-table::
307+
:header-rows: 1
308+
309+
* - orjson
310+
- msgspec workaround
311+
* - ``OPT_NON_STR_KEYS``
312+
- Use ``enc_hook``/``dec_hook`` or ``msgspec.to_builtins(obj, str_keys=True)``
313+
* - ``OPT_OMIT_MICROSECONDS``
314+
- Handle via ``enc_hook``
315+
* - ``OPT_STRICT_INTEGER`` (53-bit)
316+
- Use ``Meta(ge=-(2**53 - 1), le=2**53 - 1)`` constraint
317+
* - ``OPT_PASSTHROUGH_DATACLASS``
318+
- Use ``enc_hook`` to override default dataclass serialization
319+
* - ``OPT_PASSTHROUGH_SUBCLASS``
320+
- ``msgspec`` does not serialize subclasses of builtins by default
321+
322+
.. _orjson: https://github.com/ijl/orjson

0 commit comments

Comments
 (0)