[Feature #22118] Introduce Basic Bit Operations into String by hasumikin · Pull Request #17353 · ruby/ruby

hasumikin · 2026-06-16T04:04:36Z

This patch adds the following methods to String class:

String#bit_get(offset, lsb_first: true) -> 1 | 0 | nil
String#bit_set?(offset, lsb_first: true) -> true | false | nil
String#bit_set(offset, lsb_first: true) -> self
String#bit_clear(offset, lsb_first: true) -> self
String#bit_flip(offset, lsb_first: true) -> self
String#bit_count -> Integer
String#bitwise_not -> String
String#bitwise_not! -> self
String#bitwise_and(other) -> String
String#bitwise_and!(other) -> self
String#bitwise_or(other) -> String
String#bitwise_or!(other) -> self
String#bitwise_xor(other) -> String
String#bitwise_xor!(other) -> self

Other than implementation, tests, specs, and docs are added.

Link: [Feature #22118]

Note

In string.c, I wrote some big macros that create method functions and helper functions for bitwise_* methods:

STR_DEFINE_BINARY_BITWISE_METHOD
STR_DEFINE_UNARY_BITWISE_KERNEL
STR_DEFINE_BINARY_BITWISE_KERNEL

While using macros like this may reduce maintainability, I think it's acceptable because there are no plans to extend the bitwise_* methods beyond this proposal, and the logic is stable.

On the other hand, other methods such as bit_at and bit_count are planned to have argument extensions in the future.

github-actions · 2026-06-16T04:04:52Z

This pull request references the following Redmine tickets:

Feature #22118

kou · 2026-07-27T09:26:50Z

+        return false;
+    }
+    rb_raise(rb_eArgError, "lsb_first must be true or false");
+    UNREACHABLE_RETURN(FALSE);


Suggested change

UNREACHABLE_RETURN(FALSE);

UNREACHABLE_RETURN(false);

Thanks, you're right. Fixed it and force-pushed

kou · 2026-07-27T09:35:03Z

+    // 32 bytes (256 bits) at a time
+    for (; off < unrolled_end; off += 32) {
+        uint64_t w0, w1, w2, w3;
+        memcpy(&w0, ptr + off, 8);
+        memcpy(&w1, ptr + off + 8, 8);
+        memcpy(&w2, ptr + off + 16, 8);
+        memcpy(&w3, ptr + off + 24, 8);
+        count += rb_popcount64(w0);
+        count += rb_popcount64(w1);
+        count += rb_popcount64(w2);
+        count += rb_popcount64(w3);
+    }


Is this faster than 8 bytes loop * 4?

Maybe we need a benchmark. Give me a little time

With Claude Code's help, I benchmarked the current unrolled loop against a plain 8-byte loop (the unrolled loop simply removed).
The unrolled version was consistently faster: about 1.13x on default build flags (where rb_popcount64 becomes a __popcountdi2 libcall), and up to 1.37x when the popcnt instruction is available (-mpopcnt, e.g. x86-64-v2 distros), measured across 256 B - 64 MiB. It was never slower in any measured case, so I would like to keep the unrolled loop.

Full setup, methodology, and raw numbers: https://gist.github.com/hasumikin/911c5871297d7acb3cdde3a02e3a4867

kou · 2026-07-27T09:37:08Z

+        uint64_t word;
+        memcpy(&word, ptr + off, 8);
+        count += rb_popcount64(word);


Can we avoid a copy and is it faster than memcpy()?

Suggested change

uint64_t word;

memcpy(&word, ptr + off, 8);

count += rb_popcount64(word);

count += rb_popcount64(((uint64_t *)(ptr + off))[0]);

There is actually no copy at runtime: since the size is a compile-time constant 8, gcc/clang compile this memcpy() into a single 8-byte mov load. So, no memcpy call remains (see the gist linked in the above comment).

I prefer to keep memcpy(), because the direct uint64_t * cast is undefined behavior: it violates strict aliasing, and ptr + off is not guaranteed to be 8-byte aligned.
For example, with

("\xff" * 100).b[3..].bit_count

the receiver is a shared string whose RSTRING_PTR points into the middle of the parent's buffer (parent + 3), so every load in the loop is misaligned. This will work on x86-64, but it can fault on strict-alignment targets.
The fixed-size memcpy() would be the standard portable idiom for an unaligned load and costs nothing, I think.

This patch adds the following methods to String class: * String#bit_get(offset, lsb_first: true) -> 1 | 0 | nil * String#bit_set?(offset, lsb_first: true) -> true | false | nil * String#bit_set(offset, lsb_first: true) -> self * String#bit_clear(offset, lsb_first: true) -> self * String#bit_flip(offset, lsb_first: true) -> self * String#bit_count -> Integer * String#bitwise_not -> String * String#bitwise_not! -> self * String#bitwise_and(other) -> String * String#bitwise_and!(other) -> self * String#bitwise_or(other) -> String * String#bitwise_or!(other) -> self * String#bitwise_xor(other) -> String * String#bitwise_xor!(other) -> self Other than implementation, tests, specs, and docs are added. Link: [Feature #22118] ## Note In `string.c`, I wrote some big macro that create method functions and helper functions: * STR_DEFINE_BINARY_BITWISE_METHOD * STR_DEFINE_UNARY_BITWISE_KERNEL * STR_DEFINE_BINARY_BITWISE_KERNEL While using macros like this reduces maintainability, I believe it's acceptable because there are no plans to extend the `bitwise_*` methods beyond this proposal, and the logic is stable. On the other hand, other methods such as `bit_get` and `bit_count` are planned to have argument extensions in the future.

kou · 2026-08-06T07:11:17Z

This proposal is approved: https://bugs.ruby-lang.org/issues/22118#note-11

Copilot

Pull request overview

Adds a set of low-level bit access/mutation and byte-wise bitwise operator APIs to String, implemented in string.c, with accompanying Test::Unit coverage, ruby/spec coverage, and RDoc entries.

Changes:

Implement String#bit_get, #bit_set?, #bit_set, #bit_clear, #bit_flip, and #bit_count in string.c.
Implement String#bitwise_not/!, #bitwise_and/!, #bitwise_or/!, #bitwise_xor/! using shared kernel macros and popcount helpers.
Add tests/specs and RDoc pages documenting usage, errors, and encoding behavior.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/ruby/test_string.rb	Adds Test::Unit coverage for new bit and bitwise methods (behavior, errors, encoding).
string.c	Implements new `String` bit APIs, bitwise kernels, and registers methods in `Init_String`.
spec/ruby/core/string/bitwise_xor_spec.rb	Adds ruby/spec coverage for `String#bitwise_xor` and `#bitwise_xor!`.
spec/ruby/core/string/bitwise_or_spec.rb	Adds ruby/spec coverage for `String#bitwise_or` and `#bitwise_or!`.
spec/ruby/core/string/bitwise_not_spec.rb	Adds ruby/spec coverage for `String#bitwise_not` and `#bitwise_not!`.
spec/ruby/core/string/bitwise_and_spec.rb	Adds ruby/spec coverage for `String#bitwise_and` and `#bitwise_and!`.
spec/ruby/core/string/bit_set_spec.rb	Adds ruby/spec coverage for `String#bit_set`.
spec/ruby/core/string/bit_set_p_spec.rb	Adds ruby/spec coverage for `String#bit_set?`.
spec/ruby/core/string/bit_get_spec.rb	Adds ruby/spec coverage for `String#bit_get`.
spec/ruby/core/string/bit_flip_spec.rb	Adds ruby/spec coverage for `String#bit_flip`.
spec/ruby/core/string/bit_count_spec.rb	Adds ruby/spec coverage for `String#bit_count`.
spec/ruby/core/string/bit_clear_spec.rb	Adds ruby/spec coverage for `String#bit_clear`.
doc/string/bitwise_xor.rdoc	Documents `String#bitwise_xor`.
doc/string/bitwise_xor_bang.rdoc	Documents `String#bitwise_xor!`.
doc/string/bitwise_or.rdoc	Documents `String#bitwise_or`.
doc/string/bitwise_or_bang.rdoc	Documents `String#bitwise_or!`.
doc/string/bitwise_not.rdoc	Documents `String#bitwise_not`.
doc/string/bitwise_not_bang.rdoc	Documents `String#bitwise_not!`.
doc/string/bitwise_and.rdoc	Documents `String#bitwise_and`.
doc/string/bitwise_and_bang.rdoc	Documents `String#bitwise_and!`.
doc/string/bit_set.rdoc	Documents `String#bit_set`.
doc/string/bit_set_p.rdoc	Documents `String#bit_set?`.
doc/string/bit_get.rdoc	Documents `String#bit_get`.
doc/string/bit_flip.rdoc	Documents `String#bit_flip`.
doc/string/bit_count.rdoc	Documents `String#bit_count`.
doc/string/bit_clear.rdoc	Documents `String#bit_clear`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

hasumikin · 2026-08-06T08:20:47Z

    return value;
 }

+#define STR_BIT_LEN(byte_len) ((uint64_t)(byte_len) * CHAR_BIT)


Fix in cf2fc0c

+static inline uint64_t
+str_logical_to_physical_bit64(uint64_t logical, bool lsb_first)
+{
+    return lsb_first ? logical : ((logical & ~(uint64_t)7) | (7 - (logical & 7)));
+}
+
+static inline long
+str_logical_to_physical_bit(long logical, bool lsb_first)
+{
+    return lsb_first ? logical : ((logical & ~7L) | (7 - (logical & 7L)));
+}


kou · 2026-08-06T07:16:49Z

+ruby_version_is "4.1" do
+  describe "String#bit_clear" do
+    it "clears a bit in LSB-first order by default and returns self" do
+      str = "\xFF".dup


Could you use +"..." instead of "...".dup?

Fix in f451cc1

kou · 2026-08-06T07:24:24Z

+require_relative '../../spec_helper'
+
+ruby_version_is "4.1" do
+  describe "String#bitwise_and" do


Could you add different length cases? For example, "\xF0".bitwise_and("") and \XF0.bitwise_and("\x00\x00").

Fix in 75adc71

kou

+1

Proposed in https://bugs.ruby-lang.org/issues/22118 Merged in #17353

hasumikin marked this pull request as draft July 11, 2026 05:30

hasumikin force-pushed the feature/string-bits-v1 branch 2 times, most recently from c2ed59d to c342f38 Compare July 13, 2026 05:11

hasumikin marked this pull request as ready for review July 13, 2026 06:22

hasumikin force-pushed the feature/string-bits-v1 branch from c342f38 to 00246a9 Compare July 16, 2026 03:28

kou reviewed Aug 3, 2026

View reviewed changes

hasumikin force-pushed the feature/string-bits-v1 branch from 00246a9 to 61a2522 Compare August 3, 2026 06:35

kou requested a lite review from Copilot August 6, 2026 07:10

Copilot started reviewing on behalf of kou August 6, 2026 07:10 View session

Copilot AI reviewed Aug 6, 2026

View reviewed changes

kou reviewed Aug 6, 2026

View reviewed changes

hasumikin added 3 commits August 6, 2026 17:15

Avoid overflow and wrap

cf2fc0c

String#+@ instead of String#dup

f451cc1

Add cases that other is shorter than self

75adc71

kou approved these changes Aug 6, 2026

View reviewed changes

kou merged commit 2461cae into ruby:master Aug 6, 2026
120 checks passed

hasumikin deleted the feature/string-bits-v1 branch August 6, 2026 12:42

hasumikin mentioned this pull request Aug 6, 2026

[DOC] Add NEWS entry for String bit operations #18232

Merged

shugo pushed a commit that referenced this pull request Aug 7, 2026

[DOC] Add NEWS entry for String bit operations

33752cd

Proposed in https://bugs.ruby-lang.org/issues/22118 Merged in #17353

Uh oh!

Conversation

hasumikin commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Note

Uh oh!

github-actions Bot commented Jun 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kou commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hasumikin commented Jun 16, 2026 •

edited

Loading