Skip to content

Commit a43cadd

Browse files
alexey-milovidovalexey-milovidov
authored
Merge branch 'master' into fix-simsimd-sve-msan
2 parents d948394 + 1fc2b35 commit a43cadd

File tree

131 files changed

+3833
-412
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

131 files changed

+3833
-412
lines changed

‎ci/jobs/scripts/fuzzer/query-fuzzer-tweaks-users.xml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,8 +40,6 @@
4040
</constraints>
4141

4242
<!-- Allow all experimental features by default -->
43-
<ast_fuzzer_runs>5</ast_fuzzer_runs>
44-
4543
<allow_create_index_without_type>1</allow_create_index_without_type>
4644
<allow_custom_error_code_in_throwif>1</allow_custom_error_code_in_throwif>
4745
<allow_ddl>1</allow_ddl>

‎contrib/chdig‎

Submodule chdig updated 44 files

‎contrib/replxx‎

‎contrib/rust_vendor‎

Submodule rust_vendor updated 3723 files

‎docs/en/sql-reference/data-types/newjson.md‎

Lines changed: 54 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -322,7 +322,60 @@ SELECT json.^a.b, json.^d.e.f FROM test;
322322
```
323323

324324
:::note
325-
Reading sub-objects as sub-columns may be inefficient, as this may require a near full scan of the JSON data.
325+
When paths are stored in basic (`map`) [shared data](#shared-data-structure), reading sub-object sub-columns may be inefficient as it requires scanning the entire shared data structure. With `map_with_buckets` or `advanced` shared data serialization, reading sub-columns from shared data is highly optimized.
326+
:::
327+
328+
## Reading JSON combined sub-columns {#reading-json-combined-sub-columns}
329+
330+
The `JSON` type supports reading a path as a **combined sub-column** using the special syntax `[email protected]`.
331+
A combined sub-column for a given path returns:
332+
- The literal value stored at that path as `Dynamic`, if the path has a literal value.
333+
- A JSON sub-object at that path as `Dynamic`, if the path has no literal value but has nested sub-paths.
334+
- `NULL`, if neither a literal value nor any sub-paths exist for that path.
335+
336+
This is useful when a path may hold either a scalar value or a nested object across different rows, and is more convenient than separately querying the literal sub-column (`json.a`) and the sub-object sub-column (`json.^a`).
337+
338+
The following example compares all three sub-column types for path `a`:
339+
340+
```sql title="Query"
341+
CREATE TABLE test (json JSON) ENGINE = Memory;
342+
INSERT INTO test VALUES ('{"a" : 42, "b" : {"c" : 1, "d" : "Hello"}}'), ('{"a" : {"x": 1, "y": 2}, "b" : {"c" : 1}}'), ('{"c" : "World"}');
343+
SELECT json FROM test;
344+
```
345+
346+
```text title="Response"
347+
┌─json────────────────────────────┐
348+
│ {"a":42,"b":{"c":1,"d":"Hello"}}│
349+
│ {"a":{"x":1,"y":2},"b":{"c":1}}│
350+
│ {"c":"World"} │
351+
└─────────────────────────────────┘
352+
```
353+
354+
```sql title="Query"
355+
SELECT
356+
json.a,
357+
dynamicType(json.a),
358+
json.^a,
359+
toTypeName(json.^a),
360+
json.@a,
361+
dynamicType(json.@a)
362+
FROM test;
363+
```
364+
365+
```text title="Response"
366+
┌─json.a─┬─dynamicType(json.a)─┬─json.^a───────┬─toTypeName(json.^a)─┬─json.@a───────┬─dynamicType(json.@a)─┐
367+
│ 42 │ Int64 │ {} │ JSON │ 42 │ Int64 │
368+
│ NULL │ None │ {"x":1,"y":2} │ JSON │ {"x":1,"y":2} │ JSON │
369+
│ NULL │ None │ {} │ JSON │ NULL │ None │
370+
└────────┴─────────────────────┴───────────────┴─────────────────────┴───────────────┴──────────────────────┘
371+
```
372+
373+
- Row 1: `a` holds a literal `42`. `json.a` returns it as `Dynamic(Int64)`, `json.^a` returns an empty sub-object `{}` (no nested keys under `a`), and `json.@a` returns the literal `42`.
374+
- Row 2: `a` holds a nested object. `json.a` returns `NULL` (no literal at that path), `json.^a` returns the sub-object as `JSON`, and `json.@a` also returns the sub-object as `Dynamic(JSON)`.
375+
- Row 3: `a` is absent entirely. Both `json.a` and `json.@a` return `NULL`, while `json.^a` returns an empty `{}`.
376+
377+
:::note
378+
When paths are stored in basic (`map`) [shared data](#shared-data-structure), reading combined sub-columns may be inefficient as it requires scanning the entire shared data structure. With `map_with_buckets` or `advanced` shared data serialization, reading sub-columns from shared data is highly optimized.
326379
:::
327380

328381
## Type inference for paths {#type-inference-for-paths}

‎programs/client/Client.cpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -546,7 +546,7 @@ void Client::connect()
546546

547547
if (max_client_network_bandwidth)
548548
{
549-
ThrottlerPtr throttler = std::make_shared<Throttler>("client_network", max_client_network_bandwidth, 0, "");
549+
ThrottlerPtr throttler = std::make_shared<Throttler>(max_client_network_bandwidth, 0, "");
550550
connection->setThrottler(throttler);
551551
}
552552

‎rust/workspace/Cargo.lock‎

Lines changed: 20 additions & 20 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎src/AggregateFunctions/SingleValueData.cpp‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
#if USE_EMBEDDED_COMPILER
1010
# include <DataTypes/Native.h>
1111
# include <llvm/IR/IRBuilder.h>
12+
# include <base/extended_types.h>
1213
#endif
1314

1415
#include <cstring>
@@ -811,7 +812,9 @@ void SingleValueDataFixed<T>::compileMinMax(llvm::IRBuilderBase & builder, llvm:
811812
auto * join_block = llvm::BasicBlock::Create(head->getContext(), "join_block", head->getParent());
812813
auto * if_should_change = llvm::BasicBlock::Create(head->getContext(), "if_should_change", head->getParent());
813814

814-
constexpr auto is_signed = std::numeric_limits<T>::is_signed;
815+
/// Use ClickHouse's is_signed_v which, unlike std::numeric_limits<T>::is_signed, is specialized
816+
/// for Decimal and wide integer types.
817+
constexpr bool is_signed = is_signed_v<T>;
815818

816819
llvm::Value * should_change_after_comparison = nullptr;
817820

@@ -858,7 +861,7 @@ void SingleValueDataFixed<T>::compileMinMaxMerge(
858861
auto * join_block = llvm::BasicBlock::Create(head->getContext(), "join_block", head->getParent());
859862
auto * if_should_change = llvm::BasicBlock::Create(head->getContext(), "if_should_change", head->getParent());
860863

861-
constexpr auto is_signed = std::numeric_limits<T>::is_signed;
864+
constexpr bool is_signed = is_signed_v<T>;
862865

863866
llvm::Value * should_change_after_comparison = nullptr;
864867

‎src/Common/FailPoint.cpp‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,7 @@ static struct InitFiu
160160
REGULAR(rmt_delay_commit_part) \
161161
ONCE(local_object_storage_network_error_during_remove) \
162162
REGULAR(lightweight_show_tables) \
163+
REGULAR(check_database_datalake_negative) \
163164
REGULAR(restart_replica_fail_after_detach) \
164165
REGULAR(database_replicated_force_metadata_digest_check) \
165166
PAUSEABLE(truncate_database_tables_pause) \

‎src/Common/Throttler.cpp‎

Lines changed: 7 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
#include <Common/Exception.h>
44
#include <Common/Stopwatch.h>
55
#include <Common/CurrentThread.h>
6-
#include <Common/logger_useful.h>
76
#include <IO/WriteHelpers.h>
87

98
#include <base/scope_guard.h>
@@ -27,11 +26,10 @@ namespace ErrorCodes
2726
/// Just 10^9.
2827
static constexpr auto NS = 1000000000UL;
2928

30-
Throttler::Throttler(const char * throttler_name_, size_t max_speed_, const ThrottlerPtr & parent_,
29+
Throttler::Throttler(size_t max_speed_, const ThrottlerPtr & parent_,
3130
ProfileEvents::Event event_amount_,
3231
ProfileEvents::Event event_sleep_us_)
33-
: throttler_name(throttler_name_)
34-
, max_speed(max_speed_)
32+
: max_speed(max_speed_)
3533
, max_burst(max_speed_ * default_burst_seconds)
3634
, limit_exceeded_exception_message("")
3735
, tokens(static_cast<double>(max_burst))
@@ -40,15 +38,14 @@ Throttler::Throttler(const char * throttler_name_, size_t max_speed_, const Thro
4038
, event_sleep_us(event_sleep_us_)
4139
{}
4240

43-
Throttler::Throttler(const char * throttler_name_, size_t max_speed_,
41+
Throttler::Throttler(size_t max_speed_,
4442
ProfileEvents::Event event_amount_,
4543
ProfileEvents::Event event_sleep_us_)
46-
: Throttler(throttler_name_, max_speed_, nullptr, event_amount_, event_sleep_us_)
44+
: Throttler(max_speed_, nullptr, event_amount_, event_sleep_us_)
4745
{}
4846

49-
Throttler::Throttler(const char * throttler_name_, size_t max_speed_, size_t limit_, const char * limit_exceeded_exception_message_, const ThrottlerPtr & parent_)
50-
: throttler_name(throttler_name_)
51-
, max_speed(max_speed_)
47+
Throttler::Throttler(size_t max_speed_, size_t limit_, const char * limit_exceeded_exception_message_, const ThrottlerPtr & parent_)
48+
: max_speed(max_speed_)
5249
, max_burst(max_speed_ * default_burst_seconds)
5350
, limit(limit_)
5451
, limit_exceeded_exception_message(limit_exceeded_exception_message_)
@@ -93,15 +90,8 @@ bool Throttler::throttle(size_t amount, size_t max_block_ns)
9390
? std::numeric_limits<UInt64>::max()
9491
: static_cast<UInt64>(block_ns_double);
9592

96-
UInt64 sleep_ns = std::min<UInt64>(max_block_ns, block_ns);
97-
LOG_TRACE(log, "Sleeping: throttler_name={}, amount={}, count={}, tokens={}, max_speed={}, block_ns={}, max_block_ns={}, sleep_ns={}", throttler_name, amount, count_value, tokens_value, max_speed_value, block_ns, max_block_ns, sleep_ns);
98-
9993
// Note that throwing exception from the following blocking call is safe. It is important for query cancellation.
100-
sleepForNanoseconds(sleep_ns);
101-
}
102-
else if (max_speed_value > 0)
103-
{
104-
LOG_TEST(log, "Not sleeping: throttler_name={}, amount={}, count={}, tokens={}, max_speed={}", throttler_name, amount, count_value, tokens_value, max_speed_value);
94+
sleepForNanoseconds(std::min<UInt64>(max_block_ns, block_ns));
10595
}
10696

10797
bool parent_block = false;
@@ -138,8 +128,6 @@ void Throttler::reset()
138128
{
139129
std::lock_guard lock(mutex);
140130

141-
LOG_TRACE(log, "Reset: throttler_name={}, count={}, tokens={}, max_burst={}", throttler_name, count, tokens, max_burst);
142-
143131
count = 0;
144132
tokens = static_cast<double>(max_burst);
145133
prev_ns = 0;
@@ -177,8 +165,6 @@ void Throttler::setMaxSpeed(size_t max_speed_)
177165
{
178166
std::lock_guard lock(mutex);
179167

180-
LOG_TRACE(log, "setMaxSpeed: throttler_name={}, {} -> {}, tokens={}", throttler_name, max_speed, max_speed_, tokens);
181-
182168
max_speed = max_speed_;
183169
max_burst = max_speed_ * default_burst_seconds;
184170
}

0 commit comments

Comments
 (0)