{"@attributes":{"version":"2.0"},"channel":{"title":"Database on Openinx Blog","link":"https:\/\/openinx.github.io\/tags\/database\/","description":"Recent content in Database on Openinx Blog","image":{"title":"Openinx Blog","url":"https:\/\/openinx.github.io\/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E","link":"https:\/\/openinx.github.io\/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E"},"generator":"Hugo -- 0.125.7","language":"en","lastBuildDate":"Fri, 01 Jul 2022 12:00:24 +0800","item":[{"title":"SQL Server's Column Store Update Design","link":"https:\/\/openinx.github.io\/posts\/2022-07-01-realtime-update-in-columar-table\/","pubDate":"Fri, 01 Jul 2022 12:00:24 +0800","guid":"https:\/\/openinx.github.io\/posts\/2022-07-01-realtime-update-in-columar-table\/","description":"I recently read Microsoft’s 2015 VLDB paper, Real-Time Analytical Processing with SQL Server [1]. SQL Server was among the earlier products to ship and productionize an HTAP row\/column update design. Row-store (row-wise index) tradeoffs for OLTP are well understood; efficient millisecond-scale column store updates have several designs in the wild\u2014Kudu [2], Positional Delta Tree [3], and others\u2014but they receive far less discussion than row stores. After a careful review of SQL Server’s approach, I find it worth sharing."},{"title":"PingCAP Hackathon 2019","link":"https:\/\/openinx.github.io\/posts\/2019-10-26-pingcap-hackthon2019\/","pubDate":"Sat, 26 Oct 2019 13:16:43 +0800","guid":"https:\/\/openinx.github.io\/posts\/2019-10-26-pingcap-hackthon2019\/","description":"I joined TiDB’s Hackathon 2019 and finally have time to write this recap.\nA 48-hour hackathon around TiDB: build a demo, present in six minutes. Scoring weights practicality, ease of use, and performance for TiDB (40%), completeness (30%), innovation (20%), and presentation (10%).\nOur team of three: captain Yi Wu from PingCAP’s US office (ex-Facebook RocksDB, now RocksDB at PingCAP), and Bokang Zhang in Beijing (TiKV). All three work on storage, so we aimed at the bottom of the stack."},{"title":"A Journey to Optimize HBase 2.x Write Performance","link":"https:\/\/openinx.github.io\/posts\/2019-09-10-tuning-the-hbase-2-write-performance\/","pubDate":"Tue, 10 Sep 2019 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2019-09-10-tuning-the-hbase-2-write-performance\/","description":"How good is HBase 2.x write performance? Let’s benchmark it.\nTest environment: five-node cluster; each node has twelve 800GB SSDs, 24 CPU cores, 128GB RAM. HBase and HDFS are co-located\u2014RegionServer and DataNode on the same host for better write locality (at least one replica local). Software: HBase 2.1.2, HDFS 2.6.0, OpenJDK 1.8.0_202.\nPer RegionServer we use 50GB heap and 50GB off-heap (100GB total)\u2014heap mainly for memstore (~36GB), off-heap mainly for BucketCache (~36GB)."},{"title":"Nebula Beijing Meetup Summary","link":"https:\/\/openinx.github.io\/posts\/2019-06-29-nebula-meetup\/","pubDate":"Sat, 29 Jun 2019 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2019-06-29-nebula-meetup\/","description":"Nebula held its second Beijing meetup in Jingdong Beichen. CEO Xiaomeng Ye led Geobase, Ant Financial’s graph database. Two technical directors\u2014Heng Chen and Fenglin Hou (dutor)\u2014drive storage (similar to TiKV) and the query engine (similar to TiDB), respectively.\nThe architecture separates storage and compute. The compute layer is SQL-like, with Go syntax for edge hops and pipelines to simplify nested subqueries. Storage is key-value with static hash partitioning by key (no dynamic partitioning yet)."},{"title":"Further GC optimization for HBase3.x: Reading HFileBlock into offheap directly","link":"https:\/\/openinx.github.io\/posts\/2019-06-23-offheap-block-reading\/","pubDate":"Sun, 23 Jun 2019 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2019-06-23-offheap-block-reading\/","description":"In HBASE-21879, we redesigned the offheap read path: read the HFileBlock from HDFS to pooled offheap ByteBuffers directly, while before HBASE-21879 we just read the HFileBlock to heap which would still lead to high GC pressure.\nAfter few months of development and testing, all subtasks have been resovled now except the HBASE-21946 (It depends on HDFS-14483 and our HDFS teams are working on this, we expect the HDFS-14483 to be included in hadoop 2."},{"title":"From HBase Off-Heap to Netty Memory Management","link":"https:\/\/openinx.github.io\/posts\/2019-02-23-netty-memory-management\/","pubDate":"Sat, 23 Feb 2019 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2019-02-23-netty-memory-management\/","description":"HBase Off-Heap Today\nHBase is a widely used distributed NoSQL database. Many workloads\u2014feeds, ads, and similar\u2014demand high throughput and low latency. HBase 2.0 off-heaped the core read and write paths: allocations go to JVM off-heap memory, which is not GC-managed and must be freed explicitly. On the write path, request buffers are allocated off-heap until data is written to the WAL and memstore. The memstore’s ConcurrentSkipListMap holds references to cells, not cell bodies; actual data lives in MSLAB chunks for easier off-heap management."},{"title":"HBaseCon West 2018 Talk - HBase Practice at Xiaomi","link":"https:\/\/openinx.github.io\/posts\/2018-06-18-hbaseconwest2018\/","pubDate":"Mon, 18 Jun 2018 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2018-06-18-hbaseconwest2018\/","description":"HBaseConWest2018 was held on June 18 in San Jose, California, hosted by Hortonworks. Attending HBaseCon West in Silicon Valley each year has become routine for the Xiaomi HBase team\u2014our community presence is well known (seven HBase Committers, two PMC members), and the company is willing to share a year-in-review of internal practice and community contributions.\nIn 2018 we submitted the talk “HBase Practice at Xiaomi,” spent considerable effort preparing it, and rehearsed in English three times internally."},{"title":"TokuDB's Multi-Version Concurrency Control (MVCC)","link":"https:\/\/openinx.github.io\/posts\/2015-12-13-ft-mvcc\/","pubDate":"Sun, 13 Dec 2015 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2015-12-13-ft-mvcc\/","description":"This article covers transaction isolation in TokuDB. The source implementation is complex; for clarity, we focus on the most essential parts and omit minor details.\nBackground In traditional relational databases (Oracle, MySQL, SQL Server, and others), transactions are central to both engineering and discussion. The core properties of a transaction are ACID. A (atomicity) means a transaction’s sub-operations have only two outcomes: all succeed on commit, or all are undone on rollback."},{"title":"TokuDB's Index Structure: Fractal Tree Implementation","link":"https:\/\/openinx.github.io\/posts\/2015-11-25-ft-index-implement\/","pubDate":"Wed, 25 Nov 2015 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2015-11-25-ft-index-implement\/","description":"This article analyzes TokuDB’s index structure\u2014the fractal tree\u2014from an engineering implementation perspective. It describes the on-disk storage layout of ft-index in detail, how ft-index implements point queries, range queries, and insert\/delete\/update operations, and throughout the discussion attempts a detailed comparison with InnoDB’s B+ tree from multiple angles.\nIntroduction to the Fractal Tree The fractal tree is a write-optimized on-disk index data structure. In general, fractal trees offer good write performance (insert\/update\/delete) while still providing read performance close to that of a B+ tree."},{"title":"Vitess Explained","link":"https:\/\/openinx.github.io\/posts\/2015-02-02-vitess-analysis\/","pubDate":"Mon, 02 Feb 2015 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2015-02-02-vitess-analysis\/","description":"Online version: PPT\nThis is a technical talk on Vitess I gave internally at my company. The main topics are:\nVitess feature modules, capabilities, and system architecture Vitess sharding approach SQL syntax supported by Vitess How Vitess resharding works Vitess data backup internals Pros and cons of Vitess compared with traditional relational databases and NoSQL A side note\nTo try building an online PPT with Markdown, I experimented with a few options."},{"title":"How LevelDB Compaction Works","link":"https:\/\/openinx.github.io\/posts\/2014-08-17-leveldb-compaction\/","pubDate":"Tue, 16 Sep 2014 13:27:08 +0800","guid":"https:\/\/openinx.github.io\/posts\/2014-08-17-leveldb-compaction\/","description":"Basic LevelDB constraints With the default options, LevelDB follows these basic constraints:\nLevelDB has 7 levels: 0, 1, 2, 3, 4, 5, and 6. SSTables on level 0 are about 4 MB each. On level i (i > 0), each SSTable is at most 2 MB. Level 0 ideally has 4 SSTables, should stay within 8, and must not exceed 12. The total storage used by all SSTables on level i (i > 0) should stay around 10^i MB."}]}}