Skip to content

Commit a4f44a2

Browse files
wschinebarsoum
authored andcommitted
Add bit-shift operators for supporting hashing (#1931)
* Add bit-shift to source * Update docs * Merge left and right shifts * Add tests and fix bugs * Update test coverage * Fix shape infer tests * Triger build * Revert change
1 parent 0098752 commit a4f44a2

39 files changed

Lines changed: 768 additions & 1 deletion

File tree

‎docs/Changelog.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10283,6 +10283,55 @@ This version of the operator has been available since version 10 of the default
1028310283
This version of the operator has been deprecated since version 10 of the default ONNX operator set.
1028410284

1028510285
## Version 11 of the default ONNX operator set
10286+
### <a name="BitShift-11"></a>**BitShift-11**</a>
10287+
10288+
Bitwise shift operator performs element-wise operation. For each input element, if the
10289+
attribute "direction" is "RIGHT", this operator moves its binary representation toward
10290+
the right side so that the input value is effectively decreased. If the attribute "direction"
10291+
is "LEFT", bits of binary representation moves toward the left side, which results the
10292+
increase of its actual value. The input X is the tensor to be shifted and another input
10293+
Y specifies the amounts of shifting. For example, if "direction" is "Right", X is [1, 4],
10294+
and S is [1, 1], the corresponding output Z would be [0, 2]. If "direction" is "LEFT" with
10295+
X=[1, 2] and S=[1, 2], the corresponding output Y would be [2, 8].
10296+
10297+
Because this operator supports Numpy-style broadcasting, X's and Y's shapes are
10298+
not necessarily identical.
10299+
This operator supports **multidirectional (i.e., Numpy-style) broadcasting**; for more details please check [the doc](Broadcasting.md).
10300+
10301+
#### Version
10302+
10303+
This version of the operator has been available since version 11 of the default ONNX operator set.
10304+
10305+
#### Attributes
10306+
10307+
<dl>
10308+
<dt><tt>direction</tt> : string (required)</dt>
10309+
<dd>Direction of moving bits. It can be either "RIGHT" (for right shift) or "LEFT" (for left shift).</dd>
10310+
</dl>
10311+
10312+
#### Inputs
10313+
10314+
<dl>
10315+
<dt><tt>X</tt> : T</dt>
10316+
<dd>First operand, input to be shifted.</dd>
10317+
<dt><tt>Y</tt> : T</dt>
10318+
<dd>Second operand, amounts of shift.</dd>
10319+
</dl>
10320+
10321+
#### Outputs
10322+
10323+
<dl>
10324+
<dt><tt>Z</tt> : T</dt>
10325+
<dd>Output tensor</dd>
10326+
</dl>
10327+
10328+
#### Type Constraints
10329+
10330+
<dl>
10331+
<dt><tt>T</tt> : tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64)</dt>
10332+
<dd>Constrain input and output types to integer tensors.</dd>
10333+
</dl>
10334+
1028610335
### <a name="Loop-11"></a>**Loop-11**</a>
1028710336

1028810337
Generic Looping construct. This loop has multiple termination conditions:

‎docs/Operators.md‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
* <a href="#Atanh">Atanh</a>
1818
* <a href="#AveragePool">AveragePool</a>
1919
* <a href="#BatchNormalization">BatchNormalization</a>
20+
* <a href="#BitShift">BitShift</a>
2021
* <a href="#Cast">Cast</a>
2122
* <a href="#Ceil">Ceil</a>
2223
* <a href="#Clip">Clip</a>
@@ -1630,6 +1631,226 @@ expect(node, inputs=[x, s, bias, mean, var], outputs=[y],
16301631
</details>
16311632

16321633

1634+
### <a name="BitShift"></a><a name="bitshift">**BitShift**</a>
1635+
1636+
Bitwise shift operator performs element-wise operation. For each input element, if the
1637+
attribute "direction" is "RIGHT", this operator moves its binary representation toward
1638+
the right side so that the input value is effectively decreased. If the attribute "direction"
1639+
is "LEFT", bits of binary representation moves toward the left side, which results the
1640+
increase of its actual value. The input X is the tensor to be shifted and another input
1641+
Y specifies the amounts of shifting. For example, if "direction" is "Right", X is [1, 4],
1642+
and S is [1, 1], the corresponding output Z would be [0, 2]. If "direction" is "LEFT" with
1643+
X=[1, 2] and S=[1, 2], the corresponding output Y would be [2, 8].
1644+
1645+
Because this operator supports Numpy-style broadcasting, X's and Y's shapes are
1646+
not necessarily identical.
1647+
This operator supports **multidirectional (i.e., Numpy-style) broadcasting**; for more details please check [the doc](Broadcasting.md).
1648+
1649+
#### Version
1650+
1651+
This version of the operator has been available since version 11 of the default ONNX operator set.
1652+
1653+
#### Attributes
1654+
1655+
<dl>
1656+
<dt><tt>direction</tt> : string (required)</dt>
1657+
<dd>Direction of moving bits. It can be either "RIGHT" (for right shift) or "LEFT" (for left shift).</dd>
1658+
</dl>
1659+
1660+
#### Inputs
1661+
1662+
<dl>
1663+
<dt><tt>X</tt> : T</dt>
1664+
<dd>First operand, input to be shifted.</dd>
1665+
<dt><tt>Y</tt> : T</dt>
1666+
<dd>Second operand, amounts of shift.</dd>
1667+
</dl>
1668+
1669+
#### Outputs
1670+
1671+
<dl>
1672+
<dt><tt>Z</tt> : T</dt>
1673+
<dd>Output tensor</dd>
1674+
</dl>
1675+
1676+
#### Type Constraints
1677+
1678+
<dl>
1679+
<dt><tt>T</tt> : tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64)</dt>
1680+
<dd>Constrain input and output types to integer tensors.</dd>
1681+
</dl>
1682+
1683+
1684+
#### Examples
1685+
1686+
<details>
1687+
<summary>left_unit16</summary>
1688+
1689+
```python
1690+
node = onnx.helper.make_node(
1691+
'BitShift',
1692+
inputs=['x', 'y'],
1693+
outputs=['z'],
1694+
direction="LEFT"
1695+
)
1696+
1697+
x = np.array([16, 4, 1]).astype(np.uint16)
1698+
y = np.array([1, 2, 3]).astype(np.uint16)
1699+
z = x << y # expected output [32, 16, 8]
1700+
expect(node, inputs=[x, y], outputs=[z],
1701+
name='test_bitshift_left_uint16')
1702+
```
1703+
1704+
</details>
1705+
1706+
1707+
<details>
1708+
<summary>left_unit32</summary>
1709+
1710+
```python
1711+
node = onnx.helper.make_node(
1712+
'BitShift',
1713+
inputs=['x', 'y'],
1714+
outputs=['z'],
1715+
direction="LEFT"
1716+
)
1717+
1718+
x = np.array([16, 4, 1]).astype(np.uint32)
1719+
y = np.array([1, 2, 3]).astype(np.uint32)
1720+
z = x << y # expected output [32, 16, 8]
1721+
expect(node, inputs=[x, y], outputs=[z],
1722+
name='test_bitshift_left_uint32')
1723+
```
1724+
1725+
</details>
1726+
1727+
1728+
<details>
1729+
<summary>left_unit64</summary>
1730+
1731+
```python
1732+
node = onnx.helper.make_node(
1733+
'BitShift',
1734+
inputs=['x', 'y'],
1735+
outputs=['z'],
1736+
direction="LEFT"
1737+
)
1738+
1739+
x = np.array([16, 4, 1]).astype(np.uint64)
1740+
y = np.array([1, 2, 3]).astype(np.uint64)
1741+
z = x << y # expected output [32, 16, 8]
1742+
expect(node, inputs=[x, y], outputs=[z],
1743+
name='test_bitshift_left_uint64')
1744+
```
1745+
1746+
</details>
1747+
1748+
1749+
<details>
1750+
<summary>left_unit8</summary>
1751+
1752+
```python
1753+
node = onnx.helper.make_node(
1754+
'BitShift',
1755+
inputs=['x', 'y'],
1756+
outputs=['z'],
1757+
direction="LEFT"
1758+
)
1759+
1760+
x = np.array([16, 4, 1]).astype(np.uint8)
1761+
y = np.array([1, 2, 3]).astype(np.uint8)
1762+
z = x << y # expected output [32, 16, 8]
1763+
expect(node, inputs=[x, y], outputs=[z],
1764+
name='test_bitshift_left_uint8')
1765+
```
1766+
1767+
</details>
1768+
1769+
1770+
<details>
1771+
<summary>right_unit16</summary>
1772+
1773+
```python
1774+
node = onnx.helper.make_node(
1775+
'BitShift',
1776+
inputs=['x', 'y'],
1777+
outputs=['z'],
1778+
direction="RIGHT"
1779+
)
1780+
1781+
x = np.array([16, 4, 1]).astype(np.uint16)
1782+
y = np.array([1, 2, 3]).astype(np.uint16)
1783+
z = x >> y # expected output [8, 1, 0]
1784+
expect(node, inputs=[x, y], outputs=[z],
1785+
name='test_bitshift_right_uint16')
1786+
```
1787+
1788+
</details>
1789+
1790+
1791+
<details>
1792+
<summary>right_unit32</summary>
1793+
1794+
```python
1795+
node = onnx.helper.make_node(
1796+
'BitShift',
1797+
inputs=['x', 'y'],
1798+
outputs=['z'],
1799+
direction="RIGHT"
1800+
)
1801+
1802+
x = np.array([16, 4, 1]).astype(np.uint32)
1803+
y = np.array([1, 2, 3]).astype(np.uint32)
1804+
z = x >> y # expected output [8, 1, 0]
1805+
expect(node, inputs=[x, y], outputs=[z],
1806+
name='test_bitshift_right_uint32')
1807+
```
1808+
1809+
</details>
1810+
1811+
1812+
<details>
1813+
<summary>right_unit64</summary>
1814+
1815+
```python
1816+
node = onnx.helper.make_node(
1817+
'BitShift',
1818+
inputs=['x', 'y'],
1819+
outputs=['z'],
1820+
direction="RIGHT"
1821+
)
1822+
1823+
x = np.array([16, 4, 1]).astype(np.uint64)
1824+
y = np.array([1, 2, 3]).astype(np.uint64)
1825+
z = x >> y # expected output [8, 1, 0]
1826+
expect(node, inputs=[x, y], outputs=[z],
1827+
name='test_bitshift_right_uint64')
1828+
```
1829+
1830+
</details>
1831+
1832+
1833+
<details>
1834+
<summary>right_unit8</summary>
1835+
1836+
```python
1837+
node = onnx.helper.make_node(
1838+
'BitShift',
1839+
inputs=['x', 'y'],
1840+
outputs=['z'],
1841+
direction="RIGHT"
1842+
)
1843+
1844+
x = np.array([16, 4, 1]).astype(np.uint8)
1845+
y = np.array([1, 2, 3]).astype(np.uint8)
1846+
z = x >> y # expected output [8, 1, 0]
1847+
expect(node, inputs=[x, y], outputs=[z],
1848+
name='test_bitshift_right_uint8')
1849+
```
1850+
1851+
</details>
1852+
1853+
16331854
### <a name="Cast"></a><a name="cast">**Cast**</a>
16341855

16351856
The operator casts the elements of a given input tensor to a data type

0 commit comments

Comments
 (0)